Complex-YOLOv4-Pytorch

Complex YOLOv4

The PyTorch Implementation based on YOLOv4 of the paper: Complex-YOLO: Real-time 3D Object Detection on Point Clouds

Features

• Realtime 3D object detection based on YOLOv4

• Support distributed data parallel training

• Tensorboard

• Mosaic/Cutout augmentation for training

• Use GIoU loss of rotated boxes for optimization.

• Update 2020.08.26: Super Fast and Accurate 3D Object Detection based on 3D LiDAR Point Clouds

  • Faster training, faster inference

  • An Anchor-free approach

  • No need for Non-Max-Suppression

  • Demonstration (on a GTX 1080Ti) Youtube link

2. Getting Started

2.1. Requirement

pip install -U -r requirements.txt

For mayavi and shapely libraries, please refer to the installation instructions from their official websites.

2.2. Data Preparation

Download the 3D KITTI detection dataset from here.

The downloaded data includes:

• Velodyne point clouds (29 GB): input data to the Complex-YOLO model
• Training labels of object data set (5 MB): input label to the Complex-YOLO model
• Camera calibration matrices of object data set (16 MB): for visualization of predictions
• Left color images of object data set (12 GB): for visualization of predictions

Please make sure that you construct the source code & dataset directories structure as below.

For 3D point cloud preprocessing, please refer to the previous works:

• VoxelNet-Pytorch
• Complex-YOLOv2
• Complex-YOLOv3

2.3. Complex-YOLO architecture

This work has been based on the paper YOLOv4: Optimal Speed and Accuracy of Object Detection.

Please refer to several implementations of YOLOv4 using PyTorch DL framework:

• Tianxiaomo/pytorch-YOLOv4
• Ultralytics/yolov3_and_v4
• WongKinYiu/PyTorch_YOLOv4
• VCasecnikovs/Yet-Another-YOLOv4-Pytorch

2.4. How to run

2.4.1. Visualize the dataset (both BEV images from LiDAR and camera images)

cd src/data_process

• To visualize BEV maps and camera images (with 3D boxes), let's execute (the output-width param can be changed to show the images in a bigger/smaller window):

python kitti_dataloader.py --output-width 608

• To visualize mosaics that are composed from 4 BEV maps (Using during training only), let's execute:

python kitti_dataloader.py --show-train-data --mosaic --output-width 608 

By default, there is no padding for the output mosaics, the feature could be activated by executing:

python kitti_dataloader.py --show-train-data --mosaic --random-padding --output-width 608 

• To visualize cutout augmentation, let's execute:

python kitti_dataloader.py --show-train-data --cutout_prob 1. --cutout_nholes 1 --cutout_fill_value 1. --cutout_ratio 0.3 --output-width 608

2.4.2. Inference

Download the trained model from here, then put it to ${ROOT}/checkpoints/ and execute:

python test.py --gpu_idx 0 --pretrained_path ../checkpoints/complex_yolov4/complex_yolov4_mse_loss.pth --cfgfile ./config/cfg/complex_yolov4.cfg --show_image

2.4.3. Evaluation

python evaluate.py --gpu_idx 0 --pretrained_path  --cfgfile  --img_size  --conf-thresh  --nms-thresh  --iou-thresh 

(The conf-thresh, nms-thresh, and iou-thresh params can be adjusted. By default, these params have been set to 0.5)

2.4.4. Training

2.4.4.1. Single machine, single gpu

python train.py --gpu_idx 0 --batch_size  --num_workers ...

2.4.4.2. Multi-processing Distributed Data Parallel Training

We should always use the nccl backend for multi-processing distributed training since it currently provides the best distributed training performance.

• Single machine (node), multiple GPUs

python train.py --dist-url 'tcp://127.0.0.1:29500' --dist-backend 'nccl' --multiprocessing-distributed --world-size 1 --rank 0

• Two machines (two nodes), multiple GPUs

First machine

python train.py --dist-url 'tcp://IP_OF_NODE1:FREEPORT' --dist-backend 'nccl' --multiprocessing-distributed --world-size 2 --rank 0

Second machine

python train.py --dist-url 'tcp://IP_OF_NODE2:FREEPORT' --dist-backend 'nccl' --multiprocessing-distributed --world-size 2 --rank 1

To reproduce the results, you can run the bash shell script

./train.sh

Tensorboard

• To track the training progress, go to the logs/ folder and

cd logs//tensorboard/
tensorboard --logdir=./

• Then go to http://localhost:6006/:

2.5. List of usage for Bag of Freebies (BoF) & Bag of Specials (BoS) in this implementation

Contact

If you think this work is useful, please give me a star!

If you find any errors or have any suggestions, please contact me (Email: [email protected]).

Thank you!

Citation

@article{Complex-YOLO,
  author = {Martin Simon, Stefan Milz, Karl Amende, Horst-Michael Gross},
  title = {Complex-YOLO: Real-time 3D Object Detection on Point Clouds},
  year = {2018},
  journal = {arXiv},
}

@article{YOLOv4,
  author = {Alexey Bochkovskiy, Chien-Yao Wang, Hong-Yuan Mark Liao},
  title = {YOLOv4: Optimal Speed and Accuracy of Object Detection},
  year = {2020},
  journal = {arXiv},
}

Folder structure

…

Usage

…