百科.dev
全部条目在线工具AI 编程趋势榜开源项目技术资讯提交条目
登录
< 返回工具列表
L

LiDARCrafter

> 编程语言
开源

[AAAI 2026 口头报告] LiDARCrafter: 从激光雷达序列中动态建模 4D 世界

199 stars0 点赞0 次浏览
访问官网GitHub

工具介绍

[AAAI 2026 口头报告] LiDARCrafter: 从激光雷达序列中动态建模 4D 世界

English | 简体中文

LiDARCrafter: Dynamic 4D World Modeling from LiDAR Sequences




Ao Liang&nbsp;&nbsp;&nbsp;&nbsp;
Youquan Liu&nbsp;&nbsp;&nbsp;&nbsp;
Yu Yang&nbsp;&nbsp;&nbsp;&nbsp;
Dongyue Lu&nbsp;&nbsp;&nbsp;&nbsp;
Linfeng Li

Lingdong Kong&nbsp;&nbsp;&nbsp;&nbsp;
Huaici Zhao&nbsp;&nbsp;&nbsp;&nbsp;
Wei Tsang Ooi





  
&nbsp;

  
&nbsp;

  
&nbsp;

| | :-: |

In this work, we introduce LiDARCrafter, a unified framework for 4D LiDAR generation and editing. We contribute:

  • The first 4D generative world model dedicated to LiDAR data, with superior controllability and spatiotemporal consistency.
  • We introduce a tri-branch 4D layout conditioned pipeline that turns language into an editable 4D layout and uses it to guide temporally stable LiDAR synthesis.
  • We propose a comprehensive evaluation suite for LiDAR sequence generation, encompassing scene-level, object-level, and sequence-level metrics.
  • We demonstrate best single-frame and sequence-level LiDAR point cloud generation performance on nuScenes, with improved foreground quality over existing methods.

:books: Citation

If you find this work helpful for your research, please kindly consider citing our paper:

bibtex
@inproceedings{liang2026lidarcrafter,
    title     = {{LiDARCrafter}: Dynamic {4D} World Modeling from {LiDAR} Sequences},
    author    = {Ao Liang and Youquan Liu and Yu Yang and Dongyue Lu and Linfeng Li and Lingdong Kong and Huaici Zhao and Wei Tsang Ooi},
    booktitle = {Proceedings of the AAAI Conference on Artificial Intelligence (AAAI)},
    volume    = {40},
    year      = {2026},
}

Updates

  • [11.2025] - LiDARCrafter has been accepted to AAAI 2026 for Oral Presentation. :tada:
  • [10.2025] - We will soon start organizing the code. All pretrained weights for evaluation can be found at Hugging Face.
  • [08.2025] - The technical report of LiDARCrafter is available on arXiv.

Outline

  • Updates
  • Outline
  • :gear: Installation
  • :hotsprings: Data Preparation
  • :rocket: Getting Started
    • Evaluation
  • :wrench: Generation Framework
    • Overall Framework
    • 4D Layout Generation
    • Single-Frame Generation
  • :snake: Model Zoo
  • :memo: TODO List
  • License
  • Acknowledgements

:gear: Installation

Please configure your environment according to the version information in environment.yml.

:hotsprings: Data Preparation

  • Create dataset: same as DrivingDiffusion
ln -s ${ROOT_DATA_PATH} ./data/nuscenes

Run bash scripts/create_data.sh for generate:

  • info with track and state

  • Updated pkl with scene graph

  • CLIP feature of scene graph

The file-tree of data is like:

bash
data
├── clips
│   └── nuscenes
│       ├── obj_text_feat.pkl
│       ├── train
│       └── val
├── infos
│   ├── needed_5_framed_token.pkl
│   ├── nuscenes_dbinfos_10sweeps_withvelo.pkl
│   ├── nuscenes_infos_10sweeps_train.pkl
│   ├── nuscenes_infos_10sweeps_val.pkl
│   ├── nuscenes_infos_lidargen_train.pkl
│   ├── nuscenes_infos_lidargen_val.pkl
│   ├── nuscenes_infos_train.pkl
│   ├── nuscenes_infos_val.pkl
│   ├── nuscenes_object_classification_train.pkl
│   └── nuscenes_object_classification_val.pkl
└── nuscenes

:rocket: Getting Started

Evaluation

  • Train classification model
    • python train/train_classification_pointmlp.py
  • Train uncertainty model
    • python train/train_uncertainty_glenet.py

For each generated 1w model

  • Extract foreground samples
    • python evaluation/extract_foreground_samples.py --model ori

:wrench: Generation Framework

Overall Framework

| | :-: |

4D Layout Generation

| | :-: |

Single-Frame Generation

| | :-: |

:snake: Model Zoo

To be updated.

:memo: TODO List

  • Initial release.
  • Release the training code.
  • Release the inference code.
  • Release the evaluation code.
  • Refine the Readme.md

License

This work is under the Apache License Version 2.0, while some specific implementations in this codebase might be under other licenses. Kindly refer to LICENSE.md for a more careful check, if you are using our code for commercial matters.

Acknowledgements

This work is developed based on the MMDetection3D codebase.

MMDetection3D is an open-source toolbox based on PyTorch, towards the next-generation platform for general 3D perception. It is a part of the OpenMMLab project developed by MMLab.

Part of the benchmarked models are from the OpenPCDet and 3DTrans projects.

Related Projects

:sunglasses: Awesome Projects
3D and 4D World Modeling: A Survey
[GitHub Repo] - [Project Page] - [Paper]
WorldLens: Full-Spectrum Evaluations of Driving World Models in Real World
[GitHub Repo] - [Project Page] - [Paper]
3EED: Ground Everything Everywhere in 3D
[GitHub Repo] - [Project Page] - [Paper]
Are VLMs Ready for Autonomous Driving? A Study from Reliability, Data & Metric Perspectives
[GitHub Repo] - [Project Page] - [Paper]
Perspective-Invariant 3D Object Detection
[GitHub Repo] - [Project Page] - [Paper]
DynamicCity: Large-Scale 4D Occupancy Generation from Dynamic Scenes
[GitHub Repo] - [Project Page] - [Paper]

Issues· 4 开放

查看全部 Issues在 GitHub 打开

暂无开放 Issues,或尚未同步最近议题。

> 标签

Python3d-generation3d-object-detection4d-generationaigc

暂无评论,来聊聊你的看法吧

> 工具信息

发布日期2026年8月1日
最后更新2026年9月18日
分类编程语言
定价开源

> 相关工具

T
TypeScript
JavaScript 的超集,为前端与全栈提供静态类型
P
Python
通用编程语言,广泛用于 Web、数据与 AI
G
Go
Google 推出的简洁高效系统语言