mmdetection3d

[Documentation](https://mmdetection3d.readthedocs.io/en/latest/) | [️Installation](https://mmdetection3d.readthedocs.io/en/latest/get_started.html) | [Model Zoo](https://mmdetection3d.readthedocs.io/en/latest/model_zoo.html) | [Update News](https://mmdetection3d.readthedocs.io/en/latest/notes/changelog.html) | [Ongoing Projects](https://github.com/open-mmlab/mmdetection3d/projects) | [Reporting Issues](https://github.com/open-mmlab/mmdetection3d/issues/new/choose) ## Introduction MMDetection3D is an open source object detection toolbox based on PyTorch, towards the next-generation platform for general 3D detection. It is a part of the [OpenMMLab](https://openmmlab.com/) project. The main branch works with **PyTorch 1.8+**. Major features - **Support multi-modality/single-modality detectors out of box** It directly supports multi-modality/single-modality detectors including MVXNet, VoteNet, PointPillars, etc. - **Support indoor/outdoor 3D detection out of box** It directly supports popular indoor and outdoor 3D detection datasets, including ScanNet, SUNRGB-D, Waymo, nuScenes, Lyft, and KITTI. For nuScenes dataset, we also support [nuImages dataset](https://github.com/open-mmlab/mmdetection3d/tree/main/configs/nuimages). - **Natural integration with 2D detection** All the about **300+ models, methods of 40+ papers**, and modules supported in [MMDetection](https://github.com/open-mmlab/mmdetection/blob/3.x/docs/en/model_zoo.md) can be trained or used in this codebase. - **High efficiency** It trains faster than other codebases. The main results are as below. Details can be found in [benchmark.md](./docs/en/notes/benchmarks.md). We compare the number of samples trained per second (the higher, the better). The models that are not supported by other codebases are marked by `✗`. | Methods | MMDetection3D | [OpenPCDet](https://github.com/open-mmlab/OpenPCDet) | [votenet](https://github.com/facebookresearch/votenet) | [Det3D](https://github.com/poodarchu/Det3D) | | :-----------------: | :-----------: | :--------------------------------------------------: | :----------------------------------------------------: | :-----------------------------------------: | | VoteNet | 358 | ✗ | 77 | ✗ | | PointPillars-car | 141 | ✗ | ✗ | 140 | | PointPillars-3class | 107 | 44 | ✗ | ✗ | | SECOND | 40 | 30 | ✗ | ✗ | | Part-A2 | 17 | 14 | ✗ | ✗ | Like [MMDetection](https://github.com/open-mmlab/mmdetection) and [MMCV](https://github.com/open-mmlab/mmcv), MMDetection3D can also be used as a library to support different projects on top of it. ## What's New ### Highlight In version 1.4, MMDetecion3D refactors the Waymo dataset and accelerates the preprocessing, training/testing setup, and evaluation of Waymo dataset. We also extends the support for camera-based, such as Monocular and BEV, 3D object detection models on Waymo. A detailed description of the Waymo data information is provided [here](https://mmdetection3d.readthedocs.io/en/latest/advanced_guides/datasets/waymo.html). Besides, in version 1.4, MMDetection3D provides [Waymo-mini](https://download.openmmlab.com/mmdetection3d/data/waymo_mmdet3d_after_1x4/waymo_mini.tar.gz) to help community users get started with Waymo and use it for quick iterative development. **v1.4.0** was released in 8/1/2024： - Support the training of [DSVT](<(https://arxiv.org/abs/2301.06051)>) in `projects` - Support [Nerf-Det](https://arxiv.org/abs/2307.14620) in `projects` - Refactor Waymo dataset **v1.3.0** was released in 18/10/2023: - Support [CENet](https://arxiv.org/abs/2207.12691) in `projects` - Enhance demos with new 3D inferencers **v1.2.0** was released in 4/7/2023 - Support [New Config Type](https://mmengine.readthedocs.io/en/latest/advanced_tutorials/config.html#a-pure-python-style-configuration-file-beta) in `mmdet3d/configs` - Support the inference of [DSVT](<(https://arxiv.org/abs/2301.06051)>) in `projects` - Support downloading datasets from [OpenDataLab](https://opendatalab.com/) using `mim` **v1.1.1** was released in 30/5/2023: - Support [TPVFormer](https://arxiv.org/pdf/2302.07817.pdf) in `projects` - Support the training of BEVFusion in `projects` - Support lidar-based 3D semantic segmentation benchmark ## Installation Please refer to [Installation](https://mmdetection3d.readthedocs.io/en/latest/get_started.html) for installation instructions. ## Getting Started For detailed user guides and advanced guides, please refer to our [documentation](https://mmdetection3d.readthedocs.io/en/latest/): User Guides - [Train & Test](https://mmdetection3d.readthedocs.io/en/latest/user_guides/index.html#train-test) - [Learn about Configs](https://mmdetection3d.readthedocs.io/en/latest/user_guides/config.html) - [Coordinate System](https://mmdetection3d.readthedocs.io/en/latest/user_guides/coord_sys_tutorial.html) - [Dataset Preparation](https://mmdetection3d.readthedocs.io/en/latest/user_guides/dataset_prepare.html) - [Customize Data Pipelines](https://mmdetection3d.readthedocs.io/en/latest/user_guides/data_pipeline.html) - [Test and Train on Standard Datasets](https://mmdetection3d.readthedocs.io/en/latest/user_guides/train_test.html) - [Inference](https://mmdetection3d.readthedocs.io/en/latest/user_guides/inference.html) - [Train with Customized Datasets](https://mmdetection3d.readthedocs.io/en/latest/user_guides/new_data_model.html) - [Useful Tools](https://mmdetection3d.readthedocs.io/en/latest/user_guides/index.html#useful-tools) Advanced Guides - [Datasets](https://mmdetection3d.readthedocs.io/en/latest/advanced_guides/index.html#datasets) - [KITTI Dataset](https://mmdetection3d.readthedocs.io/en/latest/advanced_guides/datasets/kitti.html) - [NuScenes Dataset](https://mmdetection3d.readthedocs.io/en/latest/advanced_guides/datasets/nuscenes.html) - [Lyft Dataset](https://mmdetection3d.readthedocs.io/en/latest/advanced_guides/datasets/lyft.html) - [Waymo Dataset](https://mmdetection3d.readthedocs.io/en/latest/advanced_guides/datasets/waymo.html) - [SUN RGB-D Dataset](https://mmdetection3d.readthedocs.io/en/latest/advanced_guides/datasets/sunrgbd.html) - [ScanNet Dataset](https://mmdetection3d.readthedocs.io/en/latest/advanced_guides/datasets/scannet.html) - [S3DIS Dataset](https://mmdetection3d.readthedocs.io/en/latest/advanced_guides/datasets/s3dis.html) - [SemanticKITTI Dataset](https://mmdetection3d.readthedocs.io/en/latest/advanced_guides/datasets/semantickitti.html) - [Supported Tasks](https://mmdetection3d.readthedocs.io/en/latest/advanced_guides/index.html#supported-tasks) - [LiDAR-Based 3D Detection](https://mmdetection3d.readthedocs.io/en/latest/advanced_guides/supported_tasks/lidar_det3d.html) - [Vision-Based 3D Detection](https://mmdetection3d.readthedocs.io/en/latest/advanced_guides/supported_tasks/vision_det3d.html) - [LiDAR-Based 3D Semantic Segmentation](https://mmdetection3d.readthedocs.io/en/latest/advanced_guides/supported_tasks/lidar_sem_seg3d.html) - [Customization](https://mmdetection3d.readthedocs.io/en/latest/advanced_guides/index.html#customization) - [Customize Datasets](https://mmdetection3d.readthedocs.io/en/latest/advanced_guides/customize_dataset.html) - [Customize Models](https://mmdetection3d.readthedocs.io/en/latest/advanced_guides/customize_models.html) - [Customize Runtime Settings](https://mmdetection3d.readthedocs.io/en/latest/advanced_guides/customize_runtime.html) ## Overview of Benchmark and Model Zoo Results and models are available in the [model zoo](docs/en/model_zoo.md).

Backbones	Heads	Features
PointNet (CVPR'2017) PointNet++ (NeurIPS'2017) RegNet (CVPR'2020) DGCNN (TOG'2019) DLA (CVPR'2018) MinkResNet (CVPR'2019) MinkUNet (CVPR'2019) Cylinder3D (CVPR'2021)	FreeAnchor (NeurIPS'2019)	Dynamic Voxelization (CoRL'2019)

LiDAR-based 3D Object Detection	Camera-based 3D Object Detection	Multi-modal 3D Object Detection	3D Semantic Segmentation
Outdoor SECOND (Sensor'2018) PointPillars (CVPR'2019) SSN (ECCV'2020) 3DSSD (CVPR'2020) SA-SSD (CVPR'2020) PointRCNN (CVPR'2019) Part-A2 (TPAMI'2020) CenterPoint (CVPR'2021) PV-RCNN (CVPR'2020) CenterFormer (ECCV'2022) Indoor VoteNet (ICCV'2019) H3DNet (ECCV'2020) Group-Free-3D (ICCV'2021) FCAF3D (ECCV'2022) TR3D (ArXiv'2023)	Outdoor ImVoxelNet (WACV'2022) SMOKE (CVPRW'2020) FCOS3D (ICCVW'2021) PGD (CoRL'2021) MonoFlex (CVPR'2021) DETR3D (CoRL'2021) PETR (ECCV'2022) Indoor ImVoxelNet (WACV'2022)	Outdoor