[RSS 2025] 使用高斯点拼接的新型演示生成,实现稳健的一次操作
Novel Demonstration Generation with Gaussian Splatting Enables Robust One-Shot Manipulation
Sizhe Yang*, Wenye Yu*, Jia Zeng, Jun Lv, Kerui Ren, Cewu Lu, Dahua Lin, Jiangmiao Pang
* Equal Contributions
Shanghai AI Laboratory, The Chinese University of Hong Kong, Shanghai Jiao Tong UniversityRobotics: Science and Systems (RSS) 2025
https://github.com/user-attachments/assets/2e719bdd-7f97-4269-aac8-ca0b555f6e6f
RoboSplat is framework that leverages 3D Gaussian Splatting (3DGS) to generate novel demonstrations for RGB-based policy learning.
Starting from a single expert demonstration and multi-view images, our method generates diverse and visually realistic data for policy learning, enabling robust performance across six types of generalization (object poses, object types, camera views, scene appearance, lighting conditions, and embodiments) in the real world.
Compared to previous 2D data augmentation methods, our approach achieves significantly better results across various generalization types.
First, clone this repository.
git clone https://github.com/InternRobotics/RoboSplat.git(Optional) Use conda to manage the python environment.
conda create -n robosplat python=3.10 -y
conda activate robosplatInstall dependencies.
…We provide the reconstructed and preprocessed 3D Gaussians for demonstration generation. Please follow the steps below:
Download data.zip from the Google Drive folder.
Place data.zip in the root directory of the repository. Then, unzip it by running unzip data.zip.
Generate novel demonstrations for the Pick task by running:
python data_aug/generate_demo.py \
--image_size 256 \
--save True \
--save_video True \
--ref_demo_path data/source_demo/real_000000.h5 \
--xy_step_str '[10, 10]' \
--augment_lighting False \
--augment_appearance False \
--augment_camera_pose False \
--output_path data/generated_demo/pick_100Description of Arguments:
--image_size: (int) The size of the images in the generated demonstrations.--save: (bool) Whether to save the generated demonstrations.--save_video: (bool) Whether to save videos for visualizing the generated demonstrations.--ref_demo_path: (str) Path to the reference expert demonstration file.--xy_step_str: (str) A string in the form of '[x, y]', which specifies the density of object placement. Here, x is the number of object positions along the x-axis, and y is the number of positions along the y-axis. The total number of generated demonstrations will be x * y.--augment_lighting: (bool) Whether to apply lighting augmentation.--augment_appearance: (bool) Whether to apply appearance augmentation.--augment_camera_pose: (bool) Whether to apply camera pose augmentation.--output_path: (str) Path to save the generated demonstrations and the videos.The code for preprocessing 3D Gaussians is available on Google Drive.
If you find our work helpful, please cite:
@article{robosplat,
title={Novel Demonstration Generation with Gaussian Splatting Enables Robust One-Shot Manipulation},
author={Yang, Sizhe and Yu, Wenye and Zeng, Jia and Lv, Jun and Ren, Kerui and Lu, Cewu and Lin, Dahua and Pang, Jiangmiao},
journal={arXiv preprint arXiv:2504.13175},
year={2025}
}This repository is released under the Apache 2.0 license.
Our code is built upon 3DGS and diff-gaussian-rasterization. We thank the authors for open-sourcing their code and for their significant contributions to the community.
暂无开放 Issues,或尚未同步最近议题。