Baike.dev
All toolsOnline toolsAI codingTrendingOpen sourceNewsSubmit
Log in
< Back to tools
S

SAR3D

> 编程语言
Open source

Official repository for "SAR3D: Autoregressive 3D Object Generation and Understanding via Multi-scale 3D VQVAE"

200 stars0 likes1 views
WebsiteGitHub

About

Official repository for "SAR3D: Autoregressive 3D Object Generation and Understanding via Multi-scale 3D VQVAE"

Features

  • Autoregressive Modeling

  • ⚡️ Ultra-fast 3D Generation ( Ubuntu 20.04

  • Python 3.9.16

  • PyTorch 2.0.0

  • CUDA 11.7

  • NVIDIA A6000

Quick Start

  1. Clone the repository
bash
git clone https://github.com/cyw-3d/SAR3D.git
cd SAR3D
  1. Set up environment
bash
conda env create -f environment.yml
  1. Download pretrained models

The pretrained models will be automatically downloaded to the checkpoints folder during first run.

You can also manually download them from our model zoo:

Model Description Link
VQVAE Base VQVAE model vqvae-ckpt.pt
VQVAE Flexicubes VQVAE model vqvae-flexicubes-ckpt.pt
Generation Image-conditioned model image-condition-ckpt.pth
Generation Text-conditioned model text-condition-ckpt.pth
  1. Run inference

To test the model on your own images:

  1. Place your test images in the test_files/test_images folder
  2. Run the inference script:
bash
bash test_image.sh

To test the model on your own text prompts:

  1. Place your test prompts in the test_files/test_text.json file
  2. Run the inference script:
bash
bash test_text.sh

Training

Dataset

The dataset is available for download at Hugging Face.

The dataset consists of 8 splits containing preprocessed data based on G-buffer Objaverse, including:

  • Rendered images
  • Depth maps
  • Camera poses
  • Text descriptions
  • Normal maps
  • Latent embeddings

The dataset covers over 170K unique 3D objects, augmented to more than 630K data pairs. A data.json file is provided that maps object IDs to their corresponding categories.

After downloading and unzipping the dataset, you should have the following structure:

bash
/dataset-root/
├── 1/
├── 2/
├── ...
├── 8/
│   └── 0/
│       ├── raw_image.png
│       ├── depth_alpha.jpg
│       ├── c.npy
│       ├── caption_3dtopia.txt
│       ├── normal.png
│       ├── ...
│       └── image_dino_embedding_lrm.npy
└── dataset.json

Training Commands

The following scripts allow you to train both image-conditioned and text-conditioned models using the dataset stored in the specified `` location.

For image-conditioned model training:

bash
bash train_image.sh      

For text-conditioned model training:

bash
bash train_text.sh      

For VQVAE training

bash
bash train_VQVAE.sh    

Roadmap

  • Inference and Training Code for Image-conditioned Generation
  • Dataset Release
  • Inference Code for Text-conditioned Generation
  • Training Code for Text-conditioned Generation
  • VQVAE training code
  • Code for Understanding

Citation

If you find this work useful for your research, please cite our paper:

bibtex
@inproceedings{chen2024sar3d,
    title={SAR3D: Autoregressive 3D Object Generation and Understanding via Multi-scale 3D VQVAE},
    author={Chen, Yongwei and Lan, Yushi and Zhou, Shangchen and Wang, Tengfei and Pan, Xingang},
    booktitle={CVPR},
    year={2025}
}

Issues· 11 open

View all issuesOpen on GitHub

No open issues yet, or sync has not completed.

> Tags

Python3d-understandingmultimodal-large-language-modelssingle-image-to-3dtext-to-3d

No comments yet. Be the first to share.

> Details

PublishedAug 1, 2026
UpdatedSep 18, 2026
Category编程语言
PricingOpen source

> Related tools

T
TypeScript
JavaScript 的超集,为前端与全栈提供静态类型
P
Python
通用编程语言,广泛用于 Web、数据与 AI
G
Go
Google 推出的简洁高效系统语言