OpenHelix: An Open-source Dual-System VLA Model for Robotic Manipulation
OpenHelix: An Open-source Dual-System VLA Model for Robotic Manipulation
OpenHelix Team: Can Cui*, Pengxiang Ding*, Wenxuan Song, Shuanghao Bai, Xinyang Tong, Zirui Ge, Runze Suo, and others.
This is our re-implementation of Helix.
We will provide long-term maintenance for this repository.
If you have any questions, please contact us via email!
Create a conda environment with the following commands:
…> cd calvin/dataset
> sh download_data.sh ABC> python data_preprocessing/package_calvin.py --split training
> python data_preprocessing/package_calvin.py --split validation…We provide scripts for encoding language instructions with a CLIP Text Encoder on CALVIN.
Alternatively, you can directly download pre-encoded instructions from here.
# Encode validation instructions
python data_preprocessing/preprocess_calvin_instructions.py \
--output instructions/calvin_task_ABC_D/validation.pkl \
--model_max_length 16 \
--annotation_path ./calvin/dataset/task_ABC_D/validation/lang_annotations/auto_lang_ann.npy
# Encode training instructions
python data_preprocessing/preprocess_calvin_instructions.py \
--output instructions/calvin_task_ABC_D/training.pkl \
--model_max_length 16 \
--annotation_path ./calvin/dataset/task_ABC_D/training/lang_annotations/auto_lang_ann.npyWe uploaded the model weights on Hugging Face.
| MLLM(PT) + Policy(P) | MLLM(PT) + Aux + Policy(P) |
|---|---|
| Weights | Weights |
Notably, you only need to merge the safetensors in the direcctory, e.g. "prompt_tuning_aux/llava_ckpt_safetensors", into a single pytorch_model.bin file. Here is the code:
…This will generate a file named pytorch_model.bin. Copy the path of this file, along with the path to the policy.pth file in the download directory from huggingface, into the "test_trajectory_lcb_pt_act_simple_asy10.sh" script as shown below:
…The --checkpoint argument should be set to the path of policy.pth, and the --llm_ckpt argument should be set to the path of pytorch_model.bin.
The results on CALVIN ABC-D. MLLM (PT) denotes our proposed prompt tuning method for MLLM training. Policy(P) indicates loading from a pretrained policy model. Asy(10) represents inference with a 10-step time delay. AUX denotes the additionally introduced auxiliary tasks.
| Method | 1 | 2 | 3 | 4 | 5 | Avg. Len. ↑ |
|---|---|---|---|---|---|---|
| Only Policy | 92.2 | 78.7 | 63.9 | 51.2 | 41.2 | 3.27 |
| MLLM (PT) + Policy(P) (EP_LEN=60) | 92.2 | 79.2 | 65.0 | 52.9 | 40.9 | 3.30 |
| MLLM (PT) + AUX + Policy(P) + Asy(10) (EP_LEN=60) | 93.3 | 81.8 | 67.9 | 56.6 | 46.0 | 3.45 |
| MLLM (PT) + Policy(P) (EP_LEN=360) | 96.3 | 87.3 | 77.5 | 66.5 | 55.5 | 3.83 |
| MLLM (PT) + AUX + Policy(P) + Asy(10) (EP_LEN=360) | 97.1 | 91.4 | 82.8 | 72.6 | 64.1 | 4.08 |
| Robodual | 94.4 | 82.7 | 72.1 | 62.4 | 54.4 | 3.66 |
| UniVLA | 95.5 | 85.8 | 75.4 | 66.9 | 56.5 | 3.80 |
| Seer | 94.4 | 87.2 | 79.9 | 72.2 | 64.3 | 3.98 |
| GR-MG | 96.8 | 89.3 | 81.5 | 72.7 | 64.4 | 4.04 |
> bash scripts/train_trajectory_lcb_pt_act_simple.shtrain_logs/.> bash scripts/test_trajectory_lcb_pt_act_simple_asy10.shIf you find this code useful for your research, please consider citing our paper.
@article{cui2025openhelix,
title={OpenHelix: A Short Survey, Empirical Analysis, and Open-Source Dual-System VLA Model for Robotic Manipulation},
author={Cui, Can and Ding, Pengxiang and Song, Wenxuan and Bai, Shuanghao and Tong, Xinyang and Ge, Zirui and Suo, Runze and Zhou, Wanqi and Liu, Yang and Jia, Bofang and others},
journal={arXiv preprint arXiv:2505.03912},
year={2025}
}This project is licensed under the MIT License. See the LICENSE file for details.
Parts of this codebase are adapted from:
No open issues yet, or sync has not completed.