百科.dev
全部条目在线工具AI 编程趋势榜开源项目技术资讯提交条目
登录
< 返回工具列表
O

OpenHelix

> 编程语言
开源

OpenHelix: 一种开源的双系统 VLA 模型,用于机器人操作

389 stars0 点赞1 次浏览
访问官网GitHub

工具介绍

OpenHelix: 一种开源的双系统 VLA 模型,用于机器人操作

OpenHelix: An Open-source Dual-System VLA Model for Robotic Manipulation

OpenHelix Team: Can Cui*, Pengxiang Ding*, Wenxuan Song, Shuanghao Bai, Xinyang Tong, Zirui Ge, Runze Suo, and others.

This is our re-implementation of Helix.

We will provide long-term maintenance for this repository.

If you have any questions, please contact us via email!


️ News

  • [2025/04] Initial release of OpenHelix codebase!
  • [2025/05] We released our paper on arXiv.
  • [2025/05] We released the checkpoints of OpenHelix on Hugging Face .
  • [2025/06] We evaluated OpenHelix on CALVIN ABC-D (EP_LEN=360), which is a mainstream setting, and found that OpenHelix achieves SOTA performance among dual-system VLA models. A more powerful version of OpenHelix is on the way! — Stay tuned!

TODO list

  • Release checkpoints for reproduction (Scheduled Release Date: Mid-May, 2025)
  • Update the model until all effects on the robotic arm are satisfied. (Long-term maintenance)
  • Deploying on real robots.
  • Deploying on humanoid robots.
  • Realizing collaboration between humanoid robots.

️ Installation

Create a conda environment with the following commands:

…

Data Preparation

Prepare data on CALVIN

  • Download the play demonstrations from Calvin repo.
> cd calvin/dataset
> sh download_data.sh ABC
  • Package the demonstrations for training
> python data_preprocessing/package_calvin.py --split training
> python data_preprocessing/package_calvin.py --split validation

Expected directory layout

…

️ (Optional) Encode Language Instructions

We provide scripts for encoding language instructions with a CLIP Text Encoder on CALVIN.
Alternatively, you can directly download pre-encoded instructions from here.

bash
# Encode validation instructions
python data_preprocessing/preprocess_calvin_instructions.py \
  --output instructions/calvin_task_ABC_D/validation.pkl \
  --model_max_length 16 \
  --annotation_path ./calvin/dataset/task_ABC_D/validation/lang_annotations/auto_lang_ann.npy

# Encode training instructions
python data_preprocessing/preprocess_calvin_instructions.py \
  --output instructions/calvin_task_ABC_D/training.pkl \
  --model_max_length 16 \
  --annotation_path ./calvin/dataset/task_ABC_D/training/lang_annotations/auto_lang_ann.npy

Checkpoints

We uploaded the model weights on Hugging Face.

MLLM(PT) + Policy(P) MLLM(PT) + Aux + Policy(P)
Weights Weights

Notably, you only need to merge the safetensors in the direcctory, e.g. "prompt_tuning_aux/llava_ckpt_safetensors", into a single pytorch_model.bin file. Here is the code:

…

This will generate a file named pytorch_model.bin. Copy the path of this file, along with the path to the policy.pth file in the download directory from huggingface, into the "test_trajectory_lcb_pt_act_simple_asy10.sh" script as shown below:

…

The --checkpoint argument should be set to the path of policy.pth, and the --llm_ckpt argument should be set to the path of pytorch_model.bin.

The results on CALVIN ABC-D. MLLM (PT) denotes our proposed prompt tuning method for MLLM training. Policy(P) indicates loading from a pretrained policy model. Asy(10) represents inference with a 10-step time delay. AUX denotes the additionally introduced auxiliary tasks.

Method 1 2 3 4 5 Avg. Len. ↑
Only Policy 92.2 78.7 63.9 51.2 41.2 3.27
MLLM (PT) + Policy(P) (EP_LEN=60) 92.2 79.2 65.0 52.9 40.9 3.30
MLLM (PT) + AUX + Policy(P) + Asy(10) (EP_LEN=60) 93.3 81.8 67.9 56.6 46.0 3.45
MLLM (PT) + Policy(P) (EP_LEN=360) 96.3 87.3 77.5 66.5 55.5 3.83
MLLM (PT) + AUX + Policy(P) + Asy(10) (EP_LEN=360) 97.1 91.4 82.8 72.6 64.1 4.08
Robodual 94.4 82.7 72.1 62.4 54.4 3.66
UniVLA 95.5 85.8 75.4 66.9 56.5 3.80
Seer 94.4 87.2 79.9 72.2 64.3 3.98
GR-MG 96.8 89.3 81.5 72.7 64.4 4.04

Getting Started

Train Openhelix on CALVIN:

> bash scripts/train_trajectory_lcb_pt_act_simple.sh

To evaluate pre-trained weights:

  • First, download the weights and place them under train_logs/.
  • Next, you can run the provided evaluation script.
> bash scripts/test_trajectory_lcb_pt_act_simple_asy10.sh

Citation

If you find this code useful for your research, please consider citing our paper.

bibtex
@article{cui2025openhelix,
  title={OpenHelix: A Short Survey, Empirical Analysis, and Open-Source Dual-System VLA Model for Robotic Manipulation},
  author={Cui, Can and Ding, Pengxiang and Song, Wenxuan and Bai, Shuanghao and Tong, Xinyang and Ge, Zirui and Suo, Runze and Zhou, Wanqi and Liu, Yang and Jia, Bofang and others},
  journal={arXiv preprint arXiv:2505.03912},
  year={2025}
}

License

This project is licensed under the MIT License. See the LICENSE file for details.


Acknowledgement

Parts of this codebase are adapted from:

  • 3D Diffuser Actor
  • CALVIN

Issues· 0 开放

查看全部 Issues在 GitHub 打开

暂无开放 Issues,或尚未同步最近议题。

> 标签

Pythondual-systemvision-language-action-model

暂无评论,来聊聊你的看法吧

> 工具信息

发布日期2026年8月1日
最后更新2026年9月17日
分类编程语言
定价开源

> 相关工具

T
TypeScript
JavaScript 的超集,为前端与全栈提供静态类型
P
Python
通用编程语言,广泛用于 Web、数据与 AI
G
Go
Google 推出的简洁高效系统语言