百科.dev
全部条目在线工具AI 编程趋势榜开源项目技术资讯提交条目
登录
< 返回工具列表
S

SAR3D

> 编程语言
开源

"SAR3D: 多尺度 3D VQVAE 实现自回归 3D 对象生成和理解"的官方仓库

200 stars0 点赞1 次浏览
访问官网GitHub

工具介绍

"SAR3D: 多尺度 3D VQVAE 实现自回归 3D 对象生成和理解"的官方仓库

Features

  • Autoregressive Modeling

  • ⚡️ Ultra-fast 3D Generation ( Ubuntu 20.04

  • Python 3.9.16

  • PyTorch 2.0.0

  • CUDA 11.7

  • NVIDIA A6000

Quick Start

  1. Clone the repository
bash
git clone https://github.com/cyw-3d/SAR3D.git
cd SAR3D
  1. Set up environment
bash
conda env create -f environment.yml
  1. Download pretrained models

The pretrained models will be automatically downloaded to the checkpoints folder during first run.

You can also manually download them from our model zoo:

Model Description Link
VQVAE Base VQVAE model vqvae-ckpt.pt
VQVAE Flexicubes VQVAE model vqvae-flexicubes-ckpt.pt
Generation Image-conditioned model image-condition-ckpt.pth
Generation Text-conditioned model text-condition-ckpt.pth
  1. Run inference

To test the model on your own images:

  1. Place your test images in the test_files/test_images folder
  2. Run the inference script:
bash
bash test_image.sh

To test the model on your own text prompts:

  1. Place your test prompts in the test_files/test_text.json file
  2. Run the inference script:
bash
bash test_text.sh

Training

Dataset

The dataset is available for download at Hugging Face.

The dataset consists of 8 splits containing preprocessed data based on G-buffer Objaverse, including:

  • Rendered images
  • Depth maps
  • Camera poses
  • Text descriptions
  • Normal maps
  • Latent embeddings

The dataset covers over 170K unique 3D objects, augmented to more than 630K data pairs. A data.json file is provided that maps object IDs to their corresponding categories.

After downloading and unzipping the dataset, you should have the following structure:

bash
/dataset-root/
├── 1/
├── 2/
├── ...
├── 8/
│   └── 0/
│       ├── raw_image.png
│       ├── depth_alpha.jpg
│       ├── c.npy
│       ├── caption_3dtopia.txt
│       ├── normal.png
│       ├── ...
│       └── image_dino_embedding_lrm.npy
└── dataset.json

Training Commands

The following scripts allow you to train both image-conditioned and text-conditioned models using the dataset stored in the specified `` location.

For image-conditioned model training:

bash
bash train_image.sh      

For text-conditioned model training:

bash
bash train_text.sh      

For VQVAE training

bash
bash train_VQVAE.sh    

Roadmap

  • Inference and Training Code for Image-conditioned Generation
  • Dataset Release
  • Inference Code for Text-conditioned Generation
  • Training Code for Text-conditioned Generation
  • VQVAE training code
  • Code for Understanding

Citation

If you find this work useful for your research, please cite our paper:

bibtex
@inproceedings{chen2024sar3d,
    title={SAR3D: Autoregressive 3D Object Generation and Understanding via Multi-scale 3D VQVAE},
    author={Chen, Yongwei and Lan, Yushi and Zhou, Shangchen and Wang, Tengfei and Pan, Xingang},
    booktitle={CVPR},
    year={2025}
}

Issues· 11 开放

查看全部 Issues在 GitHub 打开

暂无开放 Issues,或尚未同步最近议题。

> 标签

Python3d-understandingmultimodal-large-language-modelssingle-image-to-3dtext-to-3d

暂无评论,来聊聊你的看法吧

> 工具信息

发布日期2026年8月1日
最后更新2026年9月18日
分类编程语言
定价开源

> 相关工具

T
TypeScript
JavaScript 的超集,为前端与全栈提供静态类型
P
Python
通用编程语言,广泛用于 Web、数据与 AI
G
Go
Google 推出的简洁高效系统语言