您只需一次查看即可实现实时和通用多任务 此仓库(Yolov8 多任务)是 PyTorch 对 YOLOv8 多任务模型的官方实现,可在 PyTorch 中进行部署。
您只需一次查看即可实现实时和通用多任务 此仓库(Yolov8 多任务)是 PyTorch 对 YOLOv8 多任务模型的官方实现,可在 PyTorch 中进行部署。
This repository(Yolov8 multi-task) is the official PyTorch implementation of the paper "You Only Look at Once for Real-time and Generic Multi-Task".
You Only Look at Once for Real-time and Generic Multi-Task
by Jiayuan Wang, Q. M. Jonathan Wu :email: and Ning Zhang
(:email:) corresponding author.
| Model | Parameters | FPS (bs=1) | FPS (bs=32) |
|---|---|---|---|
| YOLOP | 7.9M | 26.0 | 134.8 |
| HybridNet | 12.83M | 11.7 | 26.9 |
| YOLOv8n(det) | 3.16M | 102 | 802.9 |
| YOLOv8n(seg) | 3.26M | 82.55 | 610.49 |
| A-YOLOM(n) | 4.43M | 39.9 | 172.2 |
| A-YOLOM(s) | 13.61M | 39.7 | 96.2 |
| Model | Recall (%) | mAP50 (%) |
|---|---|---|
| MultiNet | 81.3 | 60.2 |
| DLT-Net | 89.4 | 68.4 |
| Faster R-CNN | 81.2 | 64.9 |
| YOLOv5s | 86.8 | 77.2 |
| YOLOv8n(det) | 82.2 | 75.1 |
| YOLOP | 88.6 | 76.5 |
| A-YOLOM(n) | 85.3 | 78.0 |
| A-YOLOM(s) | 86.9 | 81.1 |
| Model | mIoU (%) |
|---|---|
| MultiNet | 71.6 |
| DLT-Net | 72.1 |
| PSPNet | 89.6 |
| YOLOv8n(seg) | 78.1 |
| YOLOP | 91.6 |
| A-YOLOM(n) | 90.5 |
| A-YOLOM(s) | 91.0 |
| Model | Accuracy (%) | IoU (%) |
|---|---|---|
| Enet | N/A | 14.64 |
| SCNN | N/A | 15.84 |
| ENet-SAD | N/A | 16.02 |
| YOLOv8n(seg) | 80.5 | 22.9 |
| YOLOP | 84.8 | 26.5 |
| A-YOLOM(n) | 81.3 | 28.2 |
| A-YOLOM(s) | 84.9 | 28.8 |
| Training method | Recall (%) | mAP50 (%) | mIoU (%) | Accuracy (%) | IoU (%) |
|---|---|---|---|---|---|
| YOLOM(n) | 85.2 | 77.7 | 90.6 | 80.8 | 26.7 |
| A-YOLOM(n) | 85.3 | 78 | 90.5 | 81.3 | 28.2 |
| YOLOM(s) | 86.9 | 81.1 | 90.9 | 83.9 | 28.2 |
| A-YOLOM(s) | 86.9 | 81.1 | 91 | 84.9 | 28.8 |
| Model | Parameters | mIoU (%) | Accuracy (%) | IoU (%) |
|---|---|---|---|---|
| YOLOv8(segda) | 1004275 | 78.1 | - | - |
| YOLOv8(segll) | 1004275 | - | 80.5 | 22.9 |
| YOLOv8(multi) | 2008550 | 84.2 | 81.7 | 24.3 |
| YOLOM(n) | 15880 | 90.6 | 80.8 | 26.7 |
YOLOv8(multi) and YOLOM(n) only display two segmentation head parameters in total. They indeed have three heads, we ignore the detection head parameters because this is an ablation study for segmentation structure.
Notes:
Multinet (paper,code),DLT-Net (paper),Faster R-CNN (paper,code),YOLOv5s(code) ,PSPNet(paper,code) ,ENet(paper,code) SCNN(paper,code) SAD-ENet(paper,code), YOLOP(paper,code), HybridNets(paper,code), YOLOv8(code). Thanks for their wonderful works.This codebase has been developed with Python==3.7.16 with PyTorch==1.13.1.
You can use a 1080Ti GPU with 16 batch sizes. That will be fine. Only need more time to train. We recommend using a 4090 or more powerful GPU, which will be fast.
We strongly recommend you create a pure environment and follow our instructions to build yours. Otherwise, you may encounter some issues because the YOLOv8 has many mechanisms to detect your environment package automatically. Then it will change some variable values to further affect the code running.
cd YOLOv8-multi-task
pip install -e .Download the images from images.
Pre-trained model: A-YOLOM # which include two version, scale "n" and "s".
Download the annotations of detection from detection-object.
Download the annotations of drivable area segmentation from seg-drivable-10.
Download the annotations of lane line segmentation from seg-lane-11.
We recommend the dataset directory structure to be the following:
# The id represent the correspondence relation
├─dataset root
│ ├─images
│ │ ├─train2017
│ │ ├─val2017
│ ├─detection-object
│ │ ├─labels
│ │ │ ├─train2017
│ │ │ ├─val2017
│ ├─seg-drivable-10
│ │ ├─labels
│ │ │ ├─train2017
│ │ │ ├─val2017
│ ├─seg-lane-11
│ │ ├─labels
│ │ │ ├─train2017
│ │ │ ├─val2017Update the your dataset path in the ./ultralytics/datasets/bdd-multi.yaml.
You can set the training configuration in the ./ultralytics/yolo/cfg/default.yaml.
python train.pyYou can change the setting in train.py
…data: Please change the "data" path to yours. You can find it under "./ultralytics/datasets"
device: If you have multi-GPUs, please list your GPU numbers, such as [0,1,2,3,4,5,6,7,8]
name: Your project name, the result and trained model will save under "./ultralytics/runs/multi/Your Project Name"
task: If you want to use the Multi-task model, please keep "multi" here
classes: You can change this to control which classfication in training, 10 and 11 means drivable area and lane line segmentation. You can create or change dataset map under "./ultralytics/datasets/bdd-multi.yaml"
combine_class: means the model will combine "classes" into one class, such as our project combining the "car", "bus", "truck", and "train" into "vehicle".
single_cls: This will combine whole detection classes into one class, for example, you have 7 classes in your dataset, and when you use "single_cls", it will automatically combine them into one class. When you set single_cls=False or delete the single_cls from model.train(). Please follow the below Note to change the "tnc" in both dataset.yaml and model.yaml, "nc_list" in dataset.yaml, the output of the detection head as well.
You can set the evaluation configuration in the ./ultralytics/yolo/cfg/default.yaml
python val.pyYou can change the setting in val.py
…HybridNets(code)python predict.pyYou can change the setting in predict.py
# setting
sys.path.insert(0, "/home/jiayuan/ultralytics-main/ultralytics")
number = 3 #input how many tasks in your work, if you have 1 detection and 3 segmentation tasks, here should be 4.
model = YOLO('/home/jiayuan/ultralytics-main/ultralytics/runs/best.pt')
model.predict(source='/data/jiayuan/dash_camara_dataset/daytime', imgsz=(384,672), device=[3],name='v4_daytime', save=True, conf=0.25, iou=0.45, show_labels=False)
# The predict results will save under "runs" folderPS: If you want to use our provided pre-trained model, please make sure that your input images are (720,1280) size and keep "imgsz=(384,672)" to achieve the best performance, you can change the "imgsz" value, but the results maybe different because he is different from the training size.
暂无开放 Issues,或尚未同步最近议题。