Physical Words: A Vocabulary for Structuring Latent Reasoning in Robot Manipulation

多模态推理为机器人基础模型提供可靠动作生成所需的预测上下文。语言推理能捕捉任务语义，却难以表达细粒度物理动力学；隐式推理能描述未来状态演化，却缺少把动力学组织成可重用推理基元的结构。作者提出先推理后行动的 PhysWord，以原子动作动词作为连续物理动力学表示的共同语义锚点。模型预测即将发生的原子动作词，并将其上下文隐状态直接对齐对应未来观测的隐表示。在此基础上，模型构建包含视觉及可选触觉动作词变体的时间结构空间，自回归预测多个未来时刻，捕捉执行阶段之间的转换，为动作生成提供多模态预测上下文。PhysWord 在视触觉 UniVTAC 和纯视觉 LIBERO-Long 上分别达到 90.3% 与 99.6%，并在双夹爪与灵巧手真机任务中优于对照方法；进一步评测展示了未见场景泛化和长时程操作能力。

Multimodal reasoning provides robotic foundation models with predictive context needed for reliable action generation. However, language-based reasoning captures task semantics but struggles to express fine-grained physical dynamics, while latent reasoning models future state evolution but offers limited structure for organizing these dynamics into reusable reasoning primitives. To bridge this gap, we introduce PhysWord, a reasoning-before-acting foundation model built on Physical Words, which use atomic action verbs as shared semantic anchors for continuous representations of physical dynamics. Specifically, PhysWord predicts upcoming atomic action words and directly aligns their contextual hidden states with latent targets derived from the corresponding future observations. Building on these primitives, we construct a temporally structured PhysWord space with visual and optional tactile variants of each action word. Autoregressively predicting Physical Words across future timesteps captures transitions between execution stages, providing structured multimodal context for action generation. PhysWord achieves state-of-the-art success rates of 90.3% on visuotactile UniVTAC and 99.6% on vision-based LIBERO-Long, and outperforms competing methods on real-world tasks using bimanual grippers and dexterous hands. Further evaluations demonstrate that our structured latent reasoning framework generalizes to unseen scenarios and enables long-horizon manipulation. Project page: https://nvidia-isaac.github.io/video_to_data/physicalword/