1 MARS Lab, NTU 2 PKU 3 BAAI 4 HKUST(GZ)
* Equal contribution † Project lead ♣ Corresponding authors
Humanoid household tasks often require concurrent loco-manipulation, where the robot must move, adjust posture, maintain balance, and manipulate objects as a single coordinated behavior. Yet existing humanoid policies typically decompose locomotion and manipulation, while recent world-action models remain either arm-centric or video-centered. We present ω-0, a latent predictive whole-body world-action model for real-world humanoid concurrent loco-manipulation. Given a language instruction, current visual observation, and robot proprioceptive state, ω-0 directly predicts controller-compatible whole-body action latents for real-robot execution. Rather than reconstructing future videos, the model learns compact future observation embeddings as a lightweight predictive objective, coupling latent visual foresight with diffusion-based whole-body action generation. It supports egocentric RGB, exocentric RGB, and exocentric depth inputs, and leverages controller-based simulation replay to ground human and public visual-motion priors into robot-executable action latents. We further collect ω-HOME, a 40+ hour real-world household humanoid dataset with synchronized multi-view observations, whole-body SMPL motions, robot states, and action latents. Real-world experiments on 11 household tasks demonstrate that a single ω-0 model produces smooth manipulate-while-moving behaviors and consistently outperforms representative imitation learning, VLA, humanoid, and world-action-model baselines.
ω-0 couples visual foresight with coordinated whole-body control in a single model.
A FAST tokenizer converts continuous humanoid trajectories into discrete whole-body action tokens. A pre-trained Qwen3-VL model then learns to associate language and ego/exo observations with coordinated motion semantics spanning the legs, torso, arms, and hands.
V-JEPA visual features, language, view tokens, and action-aware VLM features condition paired video and motion queries. Cross-attention injects predicted scene dynamics into the motion representation, while an action DiT denoises SONIC-compatible whole-body action chunks.
Public motions are replayed through SONIC to obtain executable robot states and action latents, then the model is fine-tuned on synchronized real-world demonstrations. Real-time chunking carries a clean motion prefix across predictions for continuous receding-horizon deployment.
Language + ego/exo observations + proprioception → future-aware video and motion queries → whole-body action latents → SONIC controller.
Browse real-world deployments by capability in a compact, organized task library.
An introduction to the unified whole-body world action model and its long-horizon household loco-manipulation capabilities.
Browse all real-world deployments with Ego and Exo views.
A large-scale, real-world multimodal household humanoid dataset built specifically for whole-body loco-manipulation.
ω-HOME task suite and multimodal data collection.
An overview of the data collection process, synchronized modalities, and diverse household loco-manipulation tasks.
Use the filters to browse all recorded household tasks.
If you find this work useful, please cite it.
@article{li2026omega0,
title = {{$\omega$-0: A Latent Predictive World Action Model
for Concurrent Humanoid Loco-manipulation}},
author = {Zhe, Li and Zhenzhe, Zhang and Yangyang, Wei and
Wenjie, Zhang and Xichen, Yuan and Peiyuan, Zhi and
Gen, Li and Xinying, Guo and Fengjie, Gao and
Jianfei, Yang and Shanghang, Zhang},
journal = {arXiv preprint},
year = {2026}
}