YOLOv9

Implementation of paper - YOLOv9: Learning What You Want to Learn Using Programmable Gradient Information

Performance

MS COCO

Model	Test Size	AP^val	AP₅₀^val	AP₇₅^val	Param.	FLOPs
YOLOv9-T	640	38.3%	53.1%	41.3%	2.0M	7.7G
YOLOv9-S	640	46.8%	63.4%	50.7%	7.1M	26.4G
YOLOv9-M	640	51.4%	68.1%	56.1%	20.0M	76.3G
YOLOv9-C	640	53.0%	70.2%	57.8%	25.3M	102.1G
YOLOv9-E	640	55.6%	72.8%	60.6%	57.3M	189.0G

Useful Links

Expand

Custom training: #30 (comment)

ONNX export: #2 (comment) #40 (comment) #130 (comment)

ONNX export for segmentation: #260 (comment)

TensorRT inference: #143 (comment) #34 (comment) #79 (comment) #143 (comment)

QAT TensorRT: #327 (comment) #253 (comment)

TensorRT inference for segmentation: #446

TFLite: #374 (comment)

OpenVINO: #164 (comment)

C# ONNX inference: #95 (comment)

C# OpenVINO inference: #95 (comment)

OpenCV: #113 (comment)

Hugging Face demo: #45 (comment)

CoLab demo: #18

ONNXSlim export: #37

YOLOv9 ROS: #144 (comment)

YOLOv9 ROS TensorRT: #145 (comment)

YOLOv9 Julia: #141 (comment)

YOLOv9 MLX: #258 (comment)

YOLOv9 StrongSORT with OSNet: #299 (comment)

YOLOv9 ByteTrack: #78 (comment)

YOLOv9 DeepSORT: #98 (comment)

YOLOv9 counting: #84 (comment)

YOLOv9 speed estimation: #456

YOLOv9 face detection: #121 (comment)

YOLOv9 segmentation onnxruntime: #151 (comment)

Comet logging: #110

MLflow logging: #87

AnyLabeling tool: #48 (comment)

AX650N deploy: #96 (comment)

Conda environment: #93

AutoDL docker environment: #112 (comment)

Installation

Docker environment (recommended)

Expand

# create the docker container, you can change the share memory size if you have more.
nvidia-docker run --name yolov9 -it -v your_coco_path/:/coco/ -v your_code_path/:/yolov9 --shm-size=64g nvcr.io/nvidia/pytorch:21.11-py3

# apt install required packages
apt update
apt install -y zip htop screen libgl1-mesa-glx

# pip install required packages
pip install seaborn thop

# go to code folder
cd /yolov9

Evaluation

yolov9-s-converted.pt yolov9-m-converted.pt yolov9-c-converted.pt yolov9-e-converted.pt yolov9-s.pt yolov9-m.pt yolov9-c.pt yolov9-e.pt gelan-s.pt gelan-m.pt gelan-c.pt gelan-e.pt

# evaluate converted yolov9 models
python val.py --data data/coco.yaml --img 640 --batch 32 --conf 0.001 --iou 0.7 --device 0 --weights './yolov9-c-converted.pt' --save-json --name yolov9_c_c_640_val

# evaluate yolov9 models
# python val_dual.py --data data/coco.yaml --img 640 --batch 32 --conf 0.001 --iou 0.7 --device 0 --weights './yolov9-c.pt' --save-json --name yolov9_c_640_val

# evaluate gelan models
# python val.py --data data/coco.yaml --img 640 --batch 32 --conf 0.001 --iou 0.7 --device 0 --weights './gelan-c.pt' --save-json --name gelan_c_640_val

You will get the results:

 Average Precision  (AP) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.530
 Average Precision  (AP) @[ IoU=0.50      | area=   all | maxDets=100 ] = 0.702
 Average Precision  (AP) @[ IoU=0.75      | area=   all | maxDets=100 ] = 0.578
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.362
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.585
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.693
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=  1 ] = 0.392
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets= 10 ] = 0.652
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.702
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.541
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.760
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.844

Training

Data preparation

bash scripts/get_coco.sh

Download MS COCO dataset images (train, val, test) and labels. If you have previously used a different version of YOLO, we strongly recommend that you delete train2017.cache and val2017.cache files, and redownload labels

Single GPU training

# train yolov9 models
python train_dual.py --workers 8 --device 0 --batch 16 --data data/coco.yaml --img 640 --cfg models/detect/yolov9-c.yaml --weights '' --name yolov9-c --hyp hyp.scratch-high.yaml --min-items 0 --epochs 500 --close-mosaic 15

# train gelan models
# python train.py --workers 8 --device 0 --batch 32 --data data/coco.yaml --img 640 --cfg models/detect/gelan-c.yaml --weights '' --name gelan-c --hyp hyp.scratch-high.yaml --min-items 0 --epochs 500 --close-mosaic 15

Multiple GPU training

# train yolov9 models
python -m torch.distributed.launch --nproc_per_node 8 --master_port 9527 train_dual.py --workers 8 --device 0,1,2,3,4,5,6,7 --sync-bn --batch 128 --data data/coco.yaml --img 640 --cfg models/detect/yolov9-c.yaml --weights '' --name yolov9-c --hyp hyp.scratch-high.yaml --min-items 0 --epochs 500 --close-mosaic 15

# train gelan models
# python -m torch.distributed.launch --nproc_per_node 4 --master_port 9527 train.py --workers 8 --device 0,1,2,3 --sync-bn --batch 128 --data data/coco.yaml --img 640 --cfg models/detect/gelan-c.yaml --weights '' --name gelan-c --hyp hyp.scratch-high.yaml --min-items 0 --epochs 500 --close-mosaic 15

Re-parameterization

See reparameterization.ipynb.

Inference

# inference converted yolov9 models
python detect.py --source './data/images/horses.jpg' --img 640 --device 0 --weights './yolov9-c-converted.pt' --name yolov9_c_c_640_detect

# inference yolov9 models
# python detect_dual.py --source './data/images/horses.jpg' --img 640 --device 0 --weights './yolov9-c.pt' --name yolov9_c_640_detect

# inference gelan models
# python detect.py --source './data/images/horses.jpg' --img 640 --device 0 --weights './gelan-c.pt' --name gelan_c_c_640_detect

Citation

@article{wang2024yolov9,
  title={{YOLOv9}: Learning What You Want to Learn Using Programmable Gradient Information},
  author={Wang, Chien-Yao  and Liao, Hong-Yuan Mark},
  booktitle={arXiv preprint arXiv:2402.13616},
  year={2024}
}

@article{chang2023yolor,
  title={{YOLOR}-Based Multi-Task Learning},
  author={Chang, Hung-Shuo and Wang, Chien-Yao and Wang, Richard Robert and Chou, Gene and Liao, Hong-Yuan Mark},
  journal={arXiv preprint arXiv:2309.16921},
  year={2023}
}

Teaser

Parts of code of YOLOR-Based Multi-Task Learning are released in the repository.

Object Detection

gelan-c-det.pt

object detection

# coco/labels/{split}/*.txt
# bbox or polygon (1 instance 1 line)
python train.py --workers 8 --device 0 --batch 32 --data data/coco.yaml --img 640 --cfg models/detect/gelan-c.yaml --weights '' --name gelan-c-det --hyp hyp.scratch-high.yaml --min-items 0 --epochs 300 --close-mosaic 10

Model	Test Size	Param.	FLOPs	AP^box
GELAN-C-DET	640	25.3M	102.1G	52.3%
YOLOv9-C-DET	640	25.3M	102.1G	53.0%

Instance Segmentation

gelan-c-seg.pt

object detection instance segmentation

# coco/labels/{split}/*.txt
# polygon (1 instance 1 line)
python segment/train.py --workers 8 --device 0 --batch 32  --data coco.yaml --img 640 --cfg models/segment/gelan-c-seg.yaml --weights '' --name gelan-c-seg --hyp hyp.scratch-high.yaml --no-overlap --epochs 300 --close-mosaic 10

Model	Test Size	Param.	FLOPs	AP^box	AP^mask
GELAN-C-SEG	640	27.4M	144.6G	52.3%	42.4%
YOLOv9-C-SEG	640	27.4M	145.5G	53.3%	43.5%

Panoptic Segmentation

gelan-c-pan.pt

object detection instance segmentation semantic segmentation stuff segmentation panoptic segmentation

# coco/labels/{split}/*.txt
# polygon (1 instance 1 line)
# coco/stuff/{split}/*.txt
# polygon (1 semantic 1 line)
python panoptic/train.py --workers 8 --device 0 --batch 32  --data coco.yaml --img 640 --cfg models/panoptic/gelan-c-pan.yaml --weights '' --name gelan-c-pan --hyp hyp.scratch-high.yaml --no-overlap --epochs 300 --close-mosaic 10

Model	Test Size	Param.	FLOPs	AP^box	AP^mask	mIoU_164k/10k^semantic	mIoU^stuff	PQ^panoptic
GELAN-C-PAN	640	27.6M	146.7G	52.6%	42.5%	39.0%/48.3%	52.7%	39.4%
YOLOv9-C-PAN	640	28.8M	187.0G	52.7%	43.0%	39.8%/-	52.2%	40.5%

Image Captioning (not yet released)

object detection instance segmentation semantic segmentation stuff segmentation panoptic segmentation image captioning

# coco/labels/{split}/*.txt
# polygon (1 instance 1 line)
# coco/stuff/{split}/*.txt
# polygon (1 semantic 1 line)
# coco/annotations/*.json
# json (1 split 1 file)
python caption/train.py --workers 8 --device 0 --batch 32  --data coco.yaml --img 640 --cfg models/caption/gelan-c-cap.yaml --weights '' --name gelan-c-cap --hyp hyp.scratch-high.yaml --no-overlap --epochs 300 --close-mosaic 10

Model	Test Size	Param.	FLOPs	AP^box	AP^mask	mIoU_164k/10k^semantic	mIoU^stuff	PQ^panoptic	BLEU@4^caption	CIDEr^caption
GELAN-C-CAP	640	47.5M	-	51.9%	42.6%	42.5%/-	56.5%	41.7%	38.8	122.3
YOLOv9-C-CAP	640	47.5M	-	52.1%	42.6%	43.0%/-	56.4%	42.1%	39.1	122.0

Acknowledgements

Expand

Name		Name	Last commit message	Last commit date
Latest commit History 113 Commits
classify		classify
data		data
figure		figure
models		models
panoptic		panoptic
scripts		scripts
segment		segment
tools		tools
utils		utils
LICENSE.md		LICENSE.md
README.md		README.md
benchmarks.py		benchmarks.py
detect.py		detect.py
detect_dual.py		detect_dual.py
export.py		export.py
hubconf.py		hubconf.py
requirements.txt		requirements.txt
train.py		train.py
train_dual.py		train_dual.py
train_triple.py		train_triple.py
val.py		val.py
val_dual.py		val_dual.py
val_triple.py		val_triple.py

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Repository files navigation

YOLOv9

Performance

Useful Links

Installation

Evaluation

Training

Re-parameterization

Inference

Citation

Teaser

Object Detection

Instance Segmentation

Panoptic Segmentation

Image Captioning (not yet released)

Acknowledgements

About

Releases 1

Packages

Languages

License

WongKinYiu/yolov9

Folders and files

Latest commit

History

Repository files navigation

YOLOv9

Performance

Useful Links

Installation

Evaluation

Training

Re-parameterization

Inference

Citation

Teaser

Object Detection

Instance Segmentation

Panoptic Segmentation

Image Captioning (not yet released)

Acknowledgements

About

Topics

Resources

License

Stars

Watchers

Forks

Releases 1

Packages 0

Languages

Packages