Baike.dev
All toolsOnline toolsAI codingTrendingOpen sourceNewsSubmit
Log in
< Back to tools
U

Uni3R

> 编程语言
Open source

[CVPR 2026 Highlight] Unified 3D Reconstruction and Semantic Understanding via Generalizable Gaussian Splatting from Unposed Multi-View Images

200 stars0 likes1 views
WebsiteGitHub

About

[CVPR 2026 Highlight] Unified 3D Reconstruction and Semantic Understanding via Generalizable Gaussian Splatting from Unposed Multi-View Images

⭐ Arbitrary view model

  • 2026/04/10: Important updates - Now we release the new model for arbitrary view training and evaluation!

Please check the branch -> arbitrary-view for more details!!!

Get Started

Updating

Both training and inference code are releasing! Including Geometry Loss code.

Installation

  1. Download repo:

    bash
    git clone --recurse-submodules https://github.com/HorizonRobotics/Uni3R
  2. Create and activate conda environment:

    bash
    conda create -n uni3r python=3.10
    conda activate uni3r
  3. Install PyTorch and related packages:

    bash
    conda install pytorch==2.1.0 torchvision==0.16.0 torchaudio==2.1.0 pytorch-cuda=12.1 -c pytorch -c nvidia -y
    conda install pytorch-cluster pytorch-scatter pytorch-sparse -c pyg -y
  4. Install other Python dependencies:

    bash
    pip install -r requirements.txt
    pip install flash-attn --no-build-isolation
  5. Install 3D Gaussian Splatting modules:

    bash
    pip install submodules/diff-gaussian-rasterization
  6. Install OpenAI CLIP:

    bash
    pip install git+https://github.com/openai/CLIP.git
  7. Build croco model:

    bash
    cd submodules/dust3r/croco/models/curope
    python setup.py build_ext --inplace
    cd ../../../../..
  8. Install Pytorch3d:

    bash
    pip install git+https://github.com/facebookresearch/pytorch3d.git
  9. Install gsplat:

    bash
    pip install git+https://github.com/nerfstudio-project/gsplat.git
  10. Download pre-trained models:

The following three model weights need to be downloaded:

bash
# 1. Create directory for checkpoints
mkdir -p checkpoints/pretrained_models

# 2. LSEG demo model weights
gdown 1FTuHY1xPUkM-5gaDtMfgCl3D0gR89WV7 -O checkpoints/pretrained_models/demo_e200.ckpt

Please download the final Uni3R checkpoints from here, including 2, 8 and 16 views. Checkpoints

Data Preparation

  1. For training: The model can be trained on ScanNet and ScanNet++ datasets.

    • Both datasets require signing agreements to access
    • Detailed data preparation instructions are available in data_process/data.md
  2. For testing: Refer to data_process/data.md for details on the test dataset.

Training

After preparing the datasets, you can train the model using the following command:

bash
bash scripts/train.sh

The training results will be saved to SAVE_DIR. By default, it is set to checkpoints/output.

Optional parameters in scripts/train.sh:

bash
# Directory to save training outputs
--output_dir "checkpoints/output"

Evaluation

Please run these scripts to do evaluation on ScanNet Dataset, including 2, 8 and 16 views.

bash
bash scripts/test.sh
bash scripts/test_8views.sh
bash scripts/test_16views.sh

TODO:

  • Release inference code.
  • Release 2, 8 and 16 views checkpoints.
  • Release the training code w/ geometric loss.
  • Verify the multi-views training code.

Acknowledgement

This work is built on many amazing research works and open-source projects, thanks a lot to all the authors for sharing!

  • Gaussian-Splatting and diff-gaussian-rasterization
  • DUSt3R
  • Language-Driven Semantic Segmentation (LSeg)
  • LSM

Citation

If you find our work useful in your research, please consider giving a star :star: and citing the following paper :pencil:.

bibtex
@misc{sun2026uni3runified3dreconstruction,
      title={Uni3R: Unified 3D Reconstruction and Semantic Understanding via Generalizable Gaussian Splatting from Unposed Multi-View Images}, 
      author={Xiangyu Sun and Haoyi Jiang and Liu Liu and Seungtae Nam and Gyeongjin Kang and Xinjie Wang and Wei Sui and Zhizhong Su and Wenyu Liu and Xinggang Wang and Eunbyung Park},
      year={2026},
      eprint={2508.03643},
      archivePrefix={arXiv},
      primaryClass={cs.CV},
      url={https://arxiv.org/abs/2508.03643}, 
}

Issues· 10 open

View all issuesOpen on GitHub

No open issues yet, or sync has not completed.

> Tags

Pythonfeedforward-3dgs

No comments yet. Be the first to share.

> Details

PublishedAug 1, 2026
UpdatedSep 18, 2026
Category编程语言
PricingOpen source

> Related tools

T
TypeScript
JavaScript 的超集,为前端与全栈提供静态类型
P
Python
通用编程语言,广泛用于 Web、数据与 AI
G
Go
Google 推出的简洁高效系统语言