Baike.dev
All toolsOnline toolsAI codingTrendingOpen sourceNewsSubmit
Log in
< Back to tools
L

LiDARCrafter

> 编程语言
Open source

[AAAI 2026 Oral] LiDARCrafter: Dynamic 4D World Modeling from LiDAR Sequences

199 stars0 likes0 views
WebsiteGitHub

About

[AAAI 2026 Oral] LiDARCrafter: Dynamic 4D World Modeling from LiDAR Sequences

English | 简体中文

LiDARCrafter: Dynamic 4D World Modeling from LiDAR Sequences




Ao Liang&nbsp;&nbsp;&nbsp;&nbsp;
Youquan Liu&nbsp;&nbsp;&nbsp;&nbsp;
Yu Yang&nbsp;&nbsp;&nbsp;&nbsp;
Dongyue Lu&nbsp;&nbsp;&nbsp;&nbsp;
Linfeng Li

Lingdong Kong&nbsp;&nbsp;&nbsp;&nbsp;
Huaici Zhao&nbsp;&nbsp;&nbsp;&nbsp;
Wei Tsang Ooi





  
&nbsp;

  
&nbsp;

  
&nbsp;

| | :-: |

In this work, we introduce LiDARCrafter, a unified framework for 4D LiDAR generation and editing. We contribute:

  • The first 4D generative world model dedicated to LiDAR data, with superior controllability and spatiotemporal consistency.
  • We introduce a tri-branch 4D layout conditioned pipeline that turns language into an editable 4D layout and uses it to guide temporally stable LiDAR synthesis.
  • We propose a comprehensive evaluation suite for LiDAR sequence generation, encompassing scene-level, object-level, and sequence-level metrics.
  • We demonstrate best single-frame and sequence-level LiDAR point cloud generation performance on nuScenes, with improved foreground quality over existing methods.

:books: Citation

If you find this work helpful for your research, please kindly consider citing our paper:

bibtex
@inproceedings{liang2026lidarcrafter,
    title     = {{LiDARCrafter}: Dynamic {4D} World Modeling from {LiDAR} Sequences},
    author    = {Ao Liang and Youquan Liu and Yu Yang and Dongyue Lu and Linfeng Li and Lingdong Kong and Huaici Zhao and Wei Tsang Ooi},
    booktitle = {Proceedings of the AAAI Conference on Artificial Intelligence (AAAI)},
    volume    = {40},
    year      = {2026},
}

Updates

  • [11.2025] - LiDARCrafter has been accepted to AAAI 2026 for Oral Presentation. :tada:
  • [10.2025] - We will soon start organizing the code. All pretrained weights for evaluation can be found at Hugging Face.
  • [08.2025] - The technical report of LiDARCrafter is available on arXiv.

Outline

  • Updates
  • Outline
  • :gear: Installation
  • :hotsprings: Data Preparation
  • :rocket: Getting Started
    • Evaluation
  • :wrench: Generation Framework
    • Overall Framework
    • 4D Layout Generation
    • Single-Frame Generation
  • :snake: Model Zoo
  • :memo: TODO List
  • License
  • Acknowledgements

:gear: Installation

Please configure your environment according to the version information in environment.yml.

:hotsprings: Data Preparation

  • Create dataset: same as DrivingDiffusion
ln -s ${ROOT_DATA_PATH} ./data/nuscenes

Run bash scripts/create_data.sh for generate:

  • info with track and state

  • Updated pkl with scene graph

  • CLIP feature of scene graph

The file-tree of data is like:

bash
data
├── clips
│   └── nuscenes
│       ├── obj_text_feat.pkl
│       ├── train
│       └── val
├── infos
│   ├── needed_5_framed_token.pkl
│   ├── nuscenes_dbinfos_10sweeps_withvelo.pkl
│   ├── nuscenes_infos_10sweeps_train.pkl
│   ├── nuscenes_infos_10sweeps_val.pkl
│   ├── nuscenes_infos_lidargen_train.pkl
│   ├── nuscenes_infos_lidargen_val.pkl
│   ├── nuscenes_infos_train.pkl
│   ├── nuscenes_infos_val.pkl
│   ├── nuscenes_object_classification_train.pkl
│   └── nuscenes_object_classification_val.pkl
└── nuscenes

:rocket: Getting Started

Evaluation

  • Train classification model
    • python train/train_classification_pointmlp.py
  • Train uncertainty model
    • python train/train_uncertainty_glenet.py

For each generated 1w model

  • Extract foreground samples
    • python evaluation/extract_foreground_samples.py --model ori

:wrench: Generation Framework

Overall Framework

| | :-: |

4D Layout Generation

| | :-: |

Single-Frame Generation

| | :-: |

:snake: Model Zoo

To be updated.

:memo: TODO List

  • Initial release.
  • Release the training code.
  • Release the inference code.
  • Release the evaluation code.
  • Refine the Readme.md

License

This work is under the Apache License Version 2.0, while some specific implementations in this codebase might be under other licenses. Kindly refer to LICENSE.md for a more careful check, if you are using our code for commercial matters.

Acknowledgements

This work is developed based on the MMDetection3D codebase.

MMDetection3D is an open-source toolbox based on PyTorch, towards the next-generation platform for general 3D perception. It is a part of the OpenMMLab project developed by MMLab.

Part of the benchmarked models are from the OpenPCDet and 3DTrans projects.

Related Projects

:sunglasses: Awesome Projects
3D and 4D World Modeling: A Survey
[GitHub Repo] - [Project Page] - [Paper]
WorldLens: Full-Spectrum Evaluations of Driving World Models in Real World
[GitHub Repo] - [Project Page] - [Paper]
3EED: Ground Everything Everywhere in 3D
[GitHub Repo] - [Project Page] - [Paper]
Are VLMs Ready for Autonomous Driving? A Study from Reliability, Data & Metric Perspectives
[GitHub Repo] - [Project Page] - [Paper]
Perspective-Invariant 3D Object Detection
[GitHub Repo] - [Project Page] - [Paper]
DynamicCity: Large-Scale 4D Occupancy Generation from Dynamic Scenes
[GitHub Repo] - [Project Page] - [Paper]

Issues· 4 open

View all issuesOpen on GitHub

No open issues yet, or sync has not completed.

> Tags

Python3d-generation3d-object-detection4d-generationaigc

No comments yet. Be the first to share.

> Details

PublishedAug 1, 2026
UpdatedSep 18, 2026
Category编程语言
PricingOpen source

> Related tools

T
TypeScript
JavaScript 的超集,为前端与全栈提供静态类型
P
Python
通用编程语言,广泛用于 Web、数据与 AI
G
Go
Google 推出的简洁高效系统语言