"SAR3D: 多尺度 3D VQVAE 实现自回归 3D 对象生成和理解"的官方仓库
Autoregressive Modeling
⚡️ Ultra-fast 3D Generation ( Ubuntu 20.04
Python 3.9.16
PyTorch 2.0.0
CUDA 11.7
NVIDIA A6000
git clone https://github.com/cyw-3d/SAR3D.git
cd SAR3Dconda env create -f environment.ymlThe pretrained models will be automatically downloaded to the checkpoints folder during first run.
You can also manually download them from our model zoo:
| Model | Description | Link |
|---|---|---|
| VQVAE | Base VQVAE model | vqvae-ckpt.pt |
| VQVAE | Flexicubes VQVAE model | vqvae-flexicubes-ckpt.pt |
| Generation | Image-conditioned model | image-condition-ckpt.pth |
| Generation | Text-conditioned model | text-condition-ckpt.pth |
To test the model on your own images:
test_files/test_images folderbash test_image.shTo test the model on your own text prompts:
test_files/test_text.json filebash test_text.shThe dataset is available for download at Hugging Face.
The dataset consists of 8 splits containing preprocessed data based on G-buffer Objaverse, including:
The dataset covers over 170K unique 3D objects, augmented to more than 630K data pairs. A data.json file is provided that maps object IDs to their corresponding categories.
After downloading and unzipping the dataset, you should have the following structure:
/dataset-root/
├── 1/
├── 2/
├── ...
├── 8/
│ └── 0/
│ ├── raw_image.png
│ ├── depth_alpha.jpg
│ ├── c.npy
│ ├── caption_3dtopia.txt
│ ├── normal.png
│ ├── ...
│ └── image_dino_embedding_lrm.npy
└── dataset.jsonThe following scripts allow you to train both image-conditioned and text-conditioned models using the dataset stored in the specified `` location.
For image-conditioned model training:
bash train_image.sh For text-conditioned model training:
bash train_text.sh For VQVAE training
bash train_VQVAE.sh If you find this work useful for your research, please cite our paper:
@inproceedings{chen2024sar3d,
title={SAR3D: Autoregressive 3D Object Generation and Understanding via Multi-scale 3D VQVAE},
author={Chen, Yongwei and Lan, Yushi and Zhou, Shangchen and Wang, Tengfei and Pan, Xingang},
booktitle={CVPR},
year={2025}
}暂无开放 Issues,或尚未同步最近议题。