Emergence of Human to Robot Transfer in Vision-Language-Action Models

VLA 能支持广泛的开放世界泛化，但需要大规模多样化数据。一个自然问题是：这些数据能否部分来自 human video？然而仅用 human video 训练 VLA 很困难，而且建立 human 与 robot 之间的映射通常需要大量手工工程。作者借鉴大语言模型中的“能力随规模涌现”现象，提出一个问题：对包含 human video 的 VLA，这种现象是否也成立？他们提出一种简单的 co-training recipe，并发现当 VLA 在足够多样的场景、任务和 embodiment 上预训练后，human-to-robot transfer 能力会涌现出来。分析表明，这种能力来自于多样预训练所形成的 embodiment-agnostic 表示。实验进一步显示，在只出现在 human data 中的泛化设置上，加入 human data 几乎可以把性能翻倍。

Vision-language-action (VLA) models can enable broad open world generalization, but require large and diverse datasets. It is appealing to consider whether some of this data can come from human videos, which cover diverse real-world situations and are easy to obtain. However, it is difficult to train VLAs with human videos alone, and establishing a mapping between humans and robots requires manual engineering and presents a major research challenge. Drawing inspiration from advances in large language models, where the ability to learn from diverse supervision emerges with scale, we ask whether a similar phenomenon holds for VLAs that incorporate human video data. We introduce a simple co-training recipe, and find that human-to-robot transfer emerges once the VLA is pre-trained on sufficient scenes, tasks, and embodiments. Our analysis suggests that this emergent capability arises because diverse pretraining produces embodiment-agnostic representations for human and robot data. We validate these findings through a series of experiments probing human to robot skill transfer and find that with sufficiently diverse robot pre-training our method can nearly double the performance on generalization settings seen only in human data.