『深度应用』YoloV5 RTX2080Ti TensorRT与PyTorch速度对比

1.概述

此次实验是为了探究YoloV5在RTX2080Ti平台上使用TensorRT对于模型推理的加速效果，同时也比对一下RTX2080Ti平台上GPU对于i7-8700 CPU的加速。

照例先提出来实验硬件环境：

系统：Ubuntu 18.04.3 LTS
CPU：Intel® Core™ i7-8700 CPU @ 3.20GHz × 12
GPU：GeForce RTX 2080Ti
Cuda：10.1
Pytorch：1.5.0
TensorRT：7.1.0

2.实验

实验参考代码如下：

PyTorch模型训练与推理代码：

https://github.com/ultralytics/yolov5

TensorRT7模型转换与推理代码：

https://github.com/wang-xinyu/tensorrtx/tree/master/yolov5

1.i7-8700 CPU&PyTorch推理实验

输入尺寸：576x960

CUDA_VISIBLE_DEVICES=-1 python detect.py --weights  runs/hm960_945/weights/best.pt  --img 960 --conf 0.15 --source  data/hels/testimgs/

输出平均耗时：420ms ；GPU占用：0MB

image 1/8 /home/song/code/yolov5/data/hels/testimgs/lADPD26eMep_iEjNBDjNB4A_1920_1080.jpg: 576x960 1 hs, Done. (0.511s)
image 2/8 /home/song/code/yolov5/data/hels/testimgs/lADPD26eMep_iEnNBDjNB4A_1920_1080.jpg: 576x960 1 ns, 1 hs, Done. (0.481s)
image 3/8 /home/song/code/yolov5/data/hels/testimgs/lADPD2eDNKD3Ng7NBDjNB4A_1920_1080.jpg: 576x960 2 ns, 2 hs, Done. (0.416s)
image 4/8 /home/song/code/yolov5/data/hels/testimgs/lADPD2sQs0W5CEfNBDjNB4A_1920_1080.jpg: 576x960 2 ns, Done. (0.422s)
image 5/8 /home/song/code/yolov5/data/hels/testimgs/lADPD3lGrdjbTqnNBDjNB4A_1920_1080.jpg: 576x960 1 hs, Done. (0.441s)
image 6/8 /home/song/code/yolov5/data/hels/testimgs/lADPD3zULH2hzqjNBDjNB4A_1920_1080.jpg: 576x960 1 ns, 2 hs, Done. (0.963s)
image 7/8 /home/song/code/yolov5/data/hels/testimgs/lADPD4PvKccrNg_NBDjNB4A_1920_1080.jpg: 576x960 1 ns, Done. (0.448s)
image 8/8 /home/song/code/yolov5/data/hels/testimgs/lADPD4d8qGv1TqvNBDjNB4A_1920_1080.jpg: 576x960 2 ns, 3 hs, Done. (0.417s)

2.RTX 2080Ti GPU&PyTorch推理实验

输入尺寸：576x960

CUDA_VISIBLE_DEVICES=0 python detect.py --weights  runs/hm960_945/weights/best.pt  --img 960 --conf 0.15 --source  data/hels/testimgs/

输出平均耗时：12ms GPU占用：1000MB

image 1/8 /home/song/code/yolov5/data/hels/testimgs/lADPD26eMep_iEjNBDjNB4A_1920_1080.jpg: 576x960 1 hs, Done. (0.012s)
image 2/8 /home/song/code/yolov5/data/hels/testimgs/lADPD26eMep_iEnNBDjNB4A_1920_1080.jpg: 576x960 1 ns, 1 hs, Done. (0.014s)
image 3/8 /home/song/code/yolov5/data/hels/testimgs/lADPD2eDNKD3Ng7NBDjNB4A_1920_1080.jpg: 576x960 2 ns, 2 hs, Done. (0.015s)
image 4/8 /home/song/code/yolov5/data/hels/testimgs/lADPD2sQs0W5CEfNBDjNB4A_1920_1080.jpg: 576x960 2 ns, Done. (0.011s)
image 5/8 /home/song/code/yolov5/data/hels/testimgs/lADPD3lGrdjbTqnNBDjNB4A_1920_1080.jpg: 576x960 1 hs, Done. (0.011s)
image 6/8 /home/song/code/yolov5/data/hels/testimgs/lADPD3zULH2hzqjNBDjNB4A_1920_1080.jpg: 576x960 1 ns, 2 hs, Done. (0.012s)
image 7/8 /home/song/code/yolov5/data/hels/testimgs/lADPD4PvKccrNg_NBDjNB4A_1920_1080.jpg: 576x960 1 ns, Done. (0.014s)
image 8/8 /home/song/code/yolov5/data/hels/testimgs/lADPD4d8qGv1TqvNBDjNB4A_1920_1080.jpg: 576x960 2 ns, 3 hs, Done. (0.011s)

3.RTX 2080Ti GPU&TensorRT7推理实验

输入尺寸：576x960

./yolov5m -d ../testimgs/

输出平均耗时：6ms GPU占用：700MB

503ms
6ms
6ms
5ms
6ms
6ms
6ms
5ms

3.总结

通过这个对比实验可以发现：

1.RTX2080Ti相较于i7-8700 CPU速度提升非常明显（420ms-->12ms），速度提升了35倍

2.TensorRT7相较于PyTorch在相同环境下也有不错的速度提升（12ms-->6ms），速度提升了1倍的同时，GPU显存占用（1000MB-->700MB）也降低了30%。也能保证推理的精度基本不变（1%的波动，在可接受范围）。

-1.参考

实验参考开源：

YoloV5：https://github.com/ultralytics/yolov5

TensorRTX：https://github.com/wang-xinyu/tensorrtx