百科.dev
全部条目在线工具AI 编程趋势榜开源项目技术资讯提交条目
登录
< 返回工具列表
U

Uni3R

> 编程语言
开源

[CVPR 2026 重点] 通过可通用的高斯点拼接从无定位多视图图像实现统一的 3D 重构和语义理解

200 stars0 点赞1 次浏览
访问官网GitHub

工具介绍

[CVPR 2026 重点] 通过可通用的高斯点拼接从无定位多视图图像实现统一的 3D 重构和语义理解

⭐ Arbitrary view model

  • 2026/04/10: Important updates - Now we release the new model for arbitrary view training and evaluation!

Please check the branch -> arbitrary-view for more details!!!

Get Started

Updating

Both training and inference code are releasing! Including Geometry Loss code.

Installation

  1. Download repo:

    bash
    git clone --recurse-submodules https://github.com/HorizonRobotics/Uni3R
  2. Create and activate conda environment:

    bash
    conda create -n uni3r python=3.10
    conda activate uni3r
  3. Install PyTorch and related packages:

    bash
    conda install pytorch==2.1.0 torchvision==0.16.0 torchaudio==2.1.0 pytorch-cuda=12.1 -c pytorch -c nvidia -y
    conda install pytorch-cluster pytorch-scatter pytorch-sparse -c pyg -y
  4. Install other Python dependencies:

    bash
    pip install -r requirements.txt
    pip install flash-attn --no-build-isolation
  5. Install 3D Gaussian Splatting modules:

    bash
    pip install submodules/diff-gaussian-rasterization
  6. Install OpenAI CLIP:

    bash
    pip install git+https://github.com/openai/CLIP.git
  7. Build croco model:

    bash
    cd submodules/dust3r/croco/models/curope
    python setup.py build_ext --inplace
    cd ../../../../..
  8. Install Pytorch3d:

    bash
    pip install git+https://github.com/facebookresearch/pytorch3d.git
  9. Install gsplat:

    bash
    pip install git+https://github.com/nerfstudio-project/gsplat.git
  10. Download pre-trained models:

The following three model weights need to be downloaded:

bash
# 1. Create directory for checkpoints
mkdir -p checkpoints/pretrained_models

# 2. LSEG demo model weights
gdown 1FTuHY1xPUkM-5gaDtMfgCl3D0gR89WV7 -O checkpoints/pretrained_models/demo_e200.ckpt

Please download the final Uni3R checkpoints from here, including 2, 8 and 16 views. Checkpoints

Data Preparation

  1. For training: The model can be trained on ScanNet and ScanNet++ datasets.

    • Both datasets require signing agreements to access
    • Detailed data preparation instructions are available in data_process/data.md
  2. For testing: Refer to data_process/data.md for details on the test dataset.

Training

After preparing the datasets, you can train the model using the following command:

bash
bash scripts/train.sh

The training results will be saved to SAVE_DIR. By default, it is set to checkpoints/output.

Optional parameters in scripts/train.sh:

bash
# Directory to save training outputs
--output_dir "checkpoints/output"

Evaluation

Please run these scripts to do evaluation on ScanNet Dataset, including 2, 8 and 16 views.

bash
bash scripts/test.sh
bash scripts/test_8views.sh
bash scripts/test_16views.sh

TODO:

  • Release inference code.
  • Release 2, 8 and 16 views checkpoints.
  • Release the training code w/ geometric loss.
  • Verify the multi-views training code.

Acknowledgement

This work is built on many amazing research works and open-source projects, thanks a lot to all the authors for sharing!

  • Gaussian-Splatting and diff-gaussian-rasterization
  • DUSt3R
  • Language-Driven Semantic Segmentation (LSeg)
  • LSM

Citation

If you find our work useful in your research, please consider giving a star :star: and citing the following paper :pencil:.

bibtex
@misc{sun2026uni3runified3dreconstruction,
      title={Uni3R: Unified 3D Reconstruction and Semantic Understanding via Generalizable Gaussian Splatting from Unposed Multi-View Images}, 
      author={Xiangyu Sun and Haoyi Jiang and Liu Liu and Seungtae Nam and Gyeongjin Kang and Xinjie Wang and Wei Sui and Zhizhong Su and Wenyu Liu and Xinggang Wang and Eunbyung Park},
      year={2026},
      eprint={2508.03643},
      archivePrefix={arXiv},
      primaryClass={cs.CV},
      url={https://arxiv.org/abs/2508.03643}, 
}

Issues· 10 开放

查看全部 Issues在 GitHub 打开

暂无开放 Issues,或尚未同步最近议题。

> 标签

Pythonfeedforward-3dgs

暂无评论,来聊聊你的看法吧

> 工具信息

发布日期2026年8月1日
最后更新2026年9月18日
分类编程语言
定价开源

> 相关工具

T
TypeScript
JavaScript 的超集,为前端与全栈提供静态类型
P
Python
通用编程语言,广泛用于 Web、数据与 AI
G
Go
Google 推出的简洁高效系统语言