NVIDIA Tesla Deep Learning Product Performance

For HPC performance, please go here.

Deep Learning Training

NVIDIA’s complete solution stack, from GPUs to libraries, and containers on NVIDIA GPU Cloud (NGC), allows data scientists to quickly get up and running with deep learning. NVIDIA® Tesla® V100 Tensor Core GPUs leverage mixed precision to accelerate deep learning training throughputs across every framework and every type of neural network. NVIDIA captured all the top spots on 6 benchmarks submitted to MLPerf, the AI’s first industry-wide benchmark, a testament to our GPU-accelerated platform approach.

NVIDIA Performance on MLPerf AI Benchmarks

ResNet-50 Time to Solution on V100

Training Image Classification on CNNs

ResNet-50 V1.5 Throughput on NVIDIA Tesla V100

DGX-2: 8x Tesla V100 32GB, Platinum 8168 2.7 GHz | Batch Size = 256 for MXNet, 512 for all others | 19.03-py3 | Precision: Mixed | Dataset: ImageNet2012

ResNet-50 V1.5 Throughput on NVIDIA Tesla T4

Supermicro SYS-4029GP-TRT T4: 8x Tesla T4, Gold 6140 2.3 GHz | Batch Size = 208 for MXnet, 256 for all others | 19.03-py3 | Precision: Mixed | Dataset: ImageNet2012

Training Performance

NVIDIA Performance on MLPerf AI Benchmarks

FrameWork	Network	Network Type	Time to Solution	GPU	Server	Container	Precision	BatchSize	DataSet	GPU Version
MXNet	ResNet-50 v1.5	CNN	135 minutes	8x V100	DGX-1V	18.11-py3	Mixed	208	ImageNet2012	V100-SXM2-16GB
	ResNet-50 v1.5	CNN	73.9 minutes	16x V100	DGX-2	18.11-py3	Mixed	256	ImageNet2012	V100-SXM3-32GB
	ResNet-50 v1.5	CNN	70 minutes	16x V100	DGX-2H	18.11-py3	Mixed	256	ImageNet2012	V100-SXM3-32GB
	ResNet-50 v1.5	CNN	6.3 minutes	640x V100	DGX-1V Saturn	18.11-py3	Mixed	26	ImageNet2012	V100-SXM2-16GB
	ResNet-50 v1.5	CNN	7.4 minutes	512x V100	DGX-2H Pod	18.11-py3	Mixed	32	ImageNet2012	V100-SXM3-32GB

PyTorch	SSD	CNN	27 minutes	8x V100	DGX-1V	18.11-py3	Mixed	152	COCO2017	V100-SXM2-16GB
	SSD	CNN	15.9 minutes	16x V100	DGX-2	18.11-py3	Mixed	128	COCO2017	V100-SXM3-32GB
	SSD	CNN	14.1 minutes	16x V100	DGX-2H	18.11-py3	Mixed	128	COCO2017	V100-SXM3-32GB
	SSD	CNN	6.5 minutes	64x V100	DGX-1V Saturn	18.11-py3	Mixed	32	COCO2017	V100-SXM2-16GB
	SSD	CNN	5.6 minutes	64x V100	DGX-2H Pod	18.11-py3	Mixed	32	COCO2017	V100-SXM3-32GB
	Mask R-CNN	CNN	323 minutes	8x V100	DGX-1V	18.11-py3	Mixed	4	COCO2014	V100-SXM2-16GB
	Mask R-CNN	CNN	176.3 minutes	16x V100	DGX-2	18.11-py3	Mixed	4	COCO2014	V100-SXM3-32GB
	Mask R-CNN	CNN	166.9 minutes	16x V100	DGX-2H	18.11-py3	Mixed	4	COCO2014	V100-SXM3-32GB
	Mask R-CNN	CNN	81 minutes	64x V100	DGX-1V Saturn	18.11-py3	Mixed	2	COCO2014	V100-SXM2-16GB
	Mask R-CNN	CNN	72.1 minutes	64x V100	DGX-2H Pod	18.11-py3	Mixed	2	COCO2014	V100-SXM3-32GB
	NCF	CNN	0.47 minutes	8x V100	DGX-1V	18.11-py3	Mixed	131072	MovieLens 20 Million	V100-SXM2-16GB
	NCF	CNN	0.4 minutes	16x V100	DGX-2	18.11-py3	Mixed	65536	MovieLens 20 Million	V100-SXM3-32GB
	NCF	CNN	0.4 minutes	16x V100	DGX-2H	18.11-py3	Mixed	65536	MovieLens 20 Million	V100-SXM3-32GB

PyTorch	GNMT	RNN	18 minutes	8x V100	DGX-1V	18.11-py3	Mixed	128	WMT16 English-German	V100-SXM2-16GB
	GNMT	RNN	10.5 minutes	16x V100	DGX-2	18.11-py3	Mixed	64	WMT16 English-German	V100-SXM3-32GB
	GNMT	RNN	9.8 minutes	16x V100	DGX-2H	18.11-py3	Mixed	64	WMT16 English-German	V100-SXM3-32GB
	GNMT	RNN	2.8 minutes	256x V100	DGX-1V Saturn	18.11-py3	Mixed	32	WMT16 English-German	V100-SXM2-16GB
	GNMT	RNN	2.7 minutes	256x V100	DGX-2H Pod	18.11-py3	Mixed	32	WMT16 English-German	V100-SXM3-32GB

PyTorch	Transformer	Attention	33 minutes	8x V100	DGX-1V	18.11-py3	Mixed	5120	WMT17 English-German	V100-SXM2-16GB
	Transformer	Attention	21.2 minutes	16x V100	DGX-2	18.11-py3	Mixed	10240	WMT17 English-German	V100-SXM3-32GB
	Transformer	Attention	19.2 minutes	16x V100	DGX-2H	18.11-py3	Mixed	10240	WMT17 English-German	V100-SXM3-32GB
	Transformer	Attention	6.2 minutes	192x V100	DGX-1V Saturn	18.11-py3	Mixed	2560	WMT17 English-German	V100-SXM2-16GB

V100 Training Performance

FrameWork	Network	Network Type	Throughput	GPU	Server	Container	Precision	BatchSize	DataSet	GPU Version
MXNet	Inception V3	CNN	544 images/sec	1x V100	DGX-2	19.03-py3	Mixed	512	ImageNet2012	V100-SXM3-32GB
		CNN	4234 images/sec	8x V100	DGX-2	19.03-py3	Mixed	512	ImageNet2012	V100-SXM3-32GB
	ResNet 50	CNN	1442 images/sec	1x V100	DGX-2	19.02-py3	Mixed	256	ImageNet2012	V100-SXM3-32GB
		CNN	10530 images/sec	8x V100	DGX-2	19.02-py3	Mixed	256	ImageNet2012	V100-SXM3-32GB
	ResNet-50 V1.5	CNN	1456 images/sec	1x V100	DGX-2	19.03-py3	Mixed	256	ImageNet2012	V100-SXM3-32GB
		CNN	10925 images/sec	8x V100	DGX-2	19.03-py3	Mixed	256	ImageNet2012	V100-SXM3-32GB

PyTorch	Inception V3	CNN	572 images/sec	1x V100	DGX-2	19.03-py3	Mixed	512	ImageNet2012	V100-SXM3-32GB
		CNN	4156 images/sec	8x V100	DGX-1	19.03-py3	Mixed	512	ImageNet2012	V100-SXM2-32GB
	NCF	CNN	20982955 samples/sec	1x V100	DGX-2	19.03-py3	Mixed	1048576	ImageNet2012	V100-SXM3-32GB
		CNN	97202281 samples/sec	8x V100	DGX-2	19.03-py3	Mixed	1048576	ImageNet2012	V100-SXM3-32GB
	ResNet 50	CNN	849 images/sec	1x V100	DGX-1	18.10-py3	Mixed	256	ImageNet2012	V100-SXM2-32GB
		CNN	6675 images/sec	8x V100	DGX-1	18.10-py3	Mixed	256	ImageNet2012	V100-SXM2-32GB
	ResNet-50 V1.5	CNN	837 images/sec	1x V100	DGX-2	19.03-py3	Mixed	512	ImageNet2012	V100-SXM3-32GB
		CNN	6175 images/sec	8x V100	DGX-2	19.03-py3	Mixed	512	ImageNet2012	V100-SXM3-32GB
	SSD	CNN	231 images/sec	1x V100	DGX-2	19.03-py3	Mixed	64	ImageNet2012	V100-SXM3-32GB
		CNN	1642 images/sec	8x V100	DGX-2	19.03-py3	Mixed	64	ImageNet2012	V100-SXM3-32GB

TensorFlow	GoogleNet	CNN	1856 images/sec	1x V100	DGX-2	19.03-py3	Mixed	1024	ImageNet2012	V100-SXM3-32GB
		CNN	13076 images/sec	8x V100	DGX-2	19.03-py3	Mixed	1536	ImageNet2012	V100-SXM3-32GB
	Inception V3	CNN	574 images/sec	1x V100	DGX-2	19.03-py3	Mixed	384	ImageNet2012	V100-SXM3-32GB
		CNN	4012 images/sec	8x V100	DGX-2	19.03-py3	Mixed	384	ImageNet2012	V100-SXM3-32GB
	NCF	CNN	14302190 samples/sec	1x V100	DGX-2	19.03-py3	Mixed	1048576	ImageNet2012	V100-SXM3-32GB
		CNN	59415774 samples/sec	8x V100	DGX-2	19.03-py3	Mixed	1048576	ImageNet2012	V100-SXM3-32GB
	ResNet 50	CNN	921 images/sec	1x V100	DGX-2	19.02-py3	Mixed	512	ImageNet2012	V100-SXM3-32GB
		CNN	6505 images/sec	8x V100	DGX-2	19.02-py3	Mixed	512	ImageNet2012	V100-SXM3-32GB
	ResNet-50 V1.5	CNN	848 images/sec	1x V100	DGX-2	19.03-py3	Mixed	512	ImageNet2012	V100-SXM3-32GB
		CNN	6015 images/sec	8x V100	DGX-2	19.3-py3	Mixed	512	ImageNet2012	V100-SXM3-32GB

PyTorch	GNMT V2	RNN	84737 total tokens/sec	1x V100	DGX-2	19.03-py3	Mixed	512	WMT16 English-German	V100-SXM3-32GB
		RNN	598008 total tokens/sec	8x V100	DGX-2	19.03-py3	Mixed	512	WMT16 English-German	V100-SXM3-32GB

T4 Training Performance

FrameWork	Network	Network Type	Throughput	GPU	Server	Container	Precision	BatchSize	DataSet	GPU Version
MXNet	Inception V3	CNN	173 images/sec	1x T4	Supermicro SYS-4029GP-TRT T4	19.03-py3	Mixed	208	ImageNet2012	Tesla T4
		CNN	1372 images/sec	8x T4	Supermicro SYS-4029GP-TRT T4	19.03-py3	Mixed	208	ImageNet2012	Tesla T4
	ResNet 50	CNN	425 images/sec	1x T4	Supermicro SYS-4029GP-TRT T4	19.02-py3	Mixed	208	ImageNet2012	Tesla T4
		CNN	3322 images/sec	8x T4	Supermicro SYS-4029GP-TRT T4	19.02-py3	Mixed	208	ImageNet2012	Tesla T4
	ResNet-50 V1.5	CNN	455 images/sec	1x V100	Supermicro SYS-4029GP-TRT T4	19.03-py3	Mixed	208	ImageNet2012	Tesla T4
		CNN	3590 images/sec	8x V100	Supermicro SYS-4029GP-TRT T4	19.03-py3	Mixed	208	ImageNet2012	Tesla T4

PyTorch	Inception V3	CNN	179 images/sec	1x T4	Supermicro SYS-4029GP-TRT T4	19.03-py3	Mixed	256	ImageNet2012	Tesla T4
		CNN	1405 images/sec	8x T4	Supermicro SYS-4029GP-TRT T4	19.03-py3	Mixed	256	ImageNet2012	Tesla T4
	NCF	CNN	6622674 samples/sec	1x V100	Supermicro SYS-4029GP-TRT T4	19.03-py3	Mixed	1048576	ImageNet2012	Tesla T4
		CNN	25115319 samples/sec	8x V100	Supermicro SYS-4029GP-TRT T4	19.03-py3	Mixed	1048576	ImageNet2012	Tesla T4
	ResNet 50	CNN	250 images/sec	1x T4	Supermicro SYS-4029GP-TRT T4	19.01-py3	Mixed	256	ImageNet2012	Tesla T4
		CNN	2054 images/sec	8x T4	Supermicro SYS-4029GP-TRT T4	19.01-py3	Mixed	256	ImageNet2012	Tesla T4
	ResNet-50 V1.5	CNN	262 images/sec	1x V100	Supermicro SYS-4029GP-TRT T4	19.03-py3	Mixed	256	ImageNet2012	Tesla T4
		CNN	2082 images/sec	8x V100	Supermicro SYS-4029GP-TRT T4	19.03-py3	Mixed	256	ImageNet2012	Tesla T4
	SSD	CNN	73 images/sec	1x V100	Supermicro SYS-4029GP-TRT T4	19.03-py3	Mixed	64	ImageNet2012	Tesla T4
		CNN	587 images/sec	8x V100	Supermicro SYS-4029GP-TRT T4	19.03-py3	Mixed	64	ImageNet2012	Tesla T4

TensorFlow	GoogleNet	CNN	577 images/sec	1x T4	Supermicro SYS-4029GP-TRT T4	19.03-py3	Mixed	512	ImageNet2012	Tesla T4
		CNN	4482 images/sec	8x T4	Supermicro SYS-4029GP-TRT T4	19.03-py3	Mixed	512	ImageNet2012	Tesla T4
	Inception V3	CNN	181 images/sec	1x T4	Supermicro SYS-4029GP-TRT T4	19.03-py3	Mixed	192	ImageNet2012	Tesla T4
		CNN	1368 images/sec	8x T4	Supermicro SYS-4029GP-TRT T4	19.03-py3	Mixed	192	ImageNet2012	Tesla T4
	ResNet 50	CNN	294 images/sec	1x T4	Supermicro SYS-4029GP-TRT T4	19.02-py3	Mixed	256	ImageNet2012	Tesla T4
		CNN	2262 images/sec	8x T4	Supermicro SYS-4029GP-TRT T4	19.02-py3	Mixed	256	ImageNet2012	Tesla T4
	ResNet-50 V1.5	CNN	274 images/sec	1x V100	Supermicro SYS-4029GP-TRT T4	19.03-py3	Mixed	256	ImageNet2012	Tesla T4
		CNN	2170 images/sec	8x V100	Supermicro SYS-4029GP-TRT T4	19.03-py3	Mixed	256	ImageNet2012	Tesla T4
PyTorch	GNMT V2	RNN	25718 total tokens/sec	1x T4	Supermicro SYS-4029GP-TRT T4	19.03-py3	Mixed	256	WMT16 English-German	Tesla T4
		RNN	183955 total tokens/sec	8x T4	Supermicro SYS-4029GP-TRT T4	19.03-py3	Mixed	256	WMT16 English-German	Tesla T4

NVIDIA Deep Learning Inference Performance

NVIDIA® TensorRT™ running on NVIDIA GPUs enable the most efficient deep learning inference performance across multiple application areas and models. This versatility provides wide latitude to data scientists to create the optimal low-latency solution. Visit NVIDIA GPU Cloud (NGC) to download any of these containers.

NVIDIA Tesla® V100 Tensor Cores GPUs leverage mixed-precision to combine high throughput with low latencies across every type of neural network. Tesla P4 is an inference GPU, designed for optimal power consumption and latency, for ultra-efficient scale-out servers. Read the inference whitepaper to learn more about NVIDIA’s inference platform.

Measuring the inference performance involves balancing a lot of variables. PLASTER is an acronym that describes the key elements for measuring deep learning performance. Each letter identifies a factor (Programmability, Latency, Accuracy, Size of Model, Throughput, Energy Efficiency, Rate of Learning) that must be considered to arrive at the right set of tradeoffs and to produce a successful deep learning implementation. Refer to NVIDIA’s PLASTER whitepaper for more details.

Inference Image Classification on CNNs with TensorRT

ResNet-50 Throughput

ResNet-50 Latency

ResNet-50 Power Efficiency

Inference Performance

V100 Inference Performance

Network	Network Type	Batch Size	Throughput	Efficiency	Latency	GPU	Server	Container	Precision	Dataset	GPU Version
GoogleNet	CNN	1	1719 images/sec	13 images/sec/watt	0.58	1x V100	DGX-2	19.03-py3	INT8	Synthetic	V100-SXM3-32GB
	CNN	2	2214 images/sec	16 images/sec/watt	0.9	1x V100	DGX-2	19.03-py3	INT8	Synthetic	V100-SXM3-32GB
	CNN	4	3210 images/sec	22 images/sec/watt	1.3	1x V100	DGX-2	19.03-py3	INT8	Synthetic	V100-SXM3-32GB
	CNN	8	5335 images/sec	30 images/sec/watt	1.5	1x V100	DGX-2	19.03-py3	INT8	Synthetic	V100-SXM3-32GB
	CNN	64	11807 images/sec	37 images/sec/watt	5.4	1x V100	DGX-2	19.03-py3	INT8	Synthetic	V100-SXM3-32GB
	CNN	84	12143 images/sec	37 images/sec/watt	6.9	1x V100	DGX-2	19.03-py3	INT8	Synthetic	V100-SXM3-32GB
	CNN	128	12774 images/sec	38 images/sec/watt	10	1x V100	DGX-2	19.03-py3	INT8	Synthetic	V100-SXM3-32GB

MobileNet V2	CNN	1	1737 images/sec	18 images/sec/watt	0.58	1x V100	DGX-1	-	INT8	Synthetic	V100-SXM2-16GB
	CNN	2	2886 images/sec	30 images/sec/watt	0.69	1x V100	DGX-1	-	INT8	Synthetic	V100-SXM2-16GB
	CNN	4	5432 images/sec	48 images/sec/watt	0.74	1x V100	DGX-1	-	INT8	Synthetic	V100-SXM2-16GB
	CNN	8	8848 images/sec	63 images/sec/watt	0.9	1x V100	DGX-1	-	INT8	Synthetic	V100-SXM2-16GB
	CNN	32	17649 images/sec	77 images/sec/watt	1.8	1x V100	DGX-1	-	INT8	Synthetic	V100-SXM2-16GB
	CNN	64	21167 images/sec	81 images/sec/watt	3	1x V100	DGX-1	-	INT8	Synthetic	V100-SXM2-16GB
	CNN	128	23262 images/sec	81 images/sec/watt	5.5	1x V100	DGX-1	-	INT8	Synthetic	V100-SXM2-16GB

ResNet-50	CNN	1	1165 images/sec	7.2 images/sec/watt	0.86	1x V100	DGX-2	19.03-py3	INT8	Synthetic	V100-SXM3-32GB
	CNN	2	1608 images/sec	9.2 images/sec/watt	1.2	1x V100	DGX-2	19.03-py3	INT8	Synthetic	V100-SXM3-32GB
	CNN	4	2028 images/sec	14.0 images/sec/watt	2	1x V100	DGX-2	19.03-py3	Mixed	Synthetic	V100-SXM3-32GB
	CNN	8	3459 images/sec	18.0 images/sec/watt	2.3	1x V100	DGX-2	19.03-py3	Mixed	Synthetic	V100-SXM3-32GB
	CNN	39	6043 images/sec	18.0 images/sec/watt	6.5	1x V100	DGX-2	19.03-py3	INT8	Synthetic	V100-SXM3-32GB
	CNN	64	7064 images/sec	22.0 images/sec/watt	9.1	1x V100	DGX-2	19.03-py3	Mixed	Synthetic	V100-SXM3-32GB
	CNN	128	7844 images/sec	23.0 images/sec/watt	16	1x V100	DGX-2	19.03-py3	Mixed	Synthetic	V100-SXM3-32GB

VGG16	CNN	1	743 images/sec	3.1 images/sec/watt	1.4	1x V100	DGX-2	19.02-py3	Mixed	Synthetic	V100-SXM3-32GB
	CNN	2	1201 images/sec	4.5 images/sec/watt	1.7	1x V100	DGX-2	19.02-py3	Mixed	Synthetic	V100-SXM3-32GB
	CNN	4	1662 images/sec	5.9 images/sec/watt	2.4	1x V100	DGX-2	19.02-py3	Mixed	Synthetic	V100-SXM3-32GB
	CNN	8	2138 images/sec	6.4 images/sec/watt	3.7	1x V100	DGX-2	19.02-py3	Mixed	Synthetic	V100-SXM3-32GB
	CNN	32	2680 images/sec	7.9 images/sec/watt	12	1x V100	DGX-2	19.02-py3	Mixed	Synthetic	V100-SXM3-32GB
	CNN	64	2748 images/sec	8.2 images/sec/watt	23	1x V100	DGX-2	19.02-py3	Mixed	Synthetic	V100-SXM3-32GB
	CNN	128	2795 images/sec	8.2 images/sec/watt	46	1x V100	DGX-2	19.02-py3	Mixed	Synthetic	V100-SXM3-32GB

TensorRT 5.1, except TensorRT 5.0 for MobileNet V2 and VGG16

T4 Inference Performance

Network	Network Type	Batch Size	Throughput	Efficiency	Latency	GPU	Server	Container	Precision	Dataset	GPU Version
GoogleNet	CNN	1	1735 images/sec	27 images/sec/watt	0.58	1x T4	Supermicro SYS-4029GP-TRT T4	19.03-py3	INT8	Synthetic	Tesla T4
	CNN	2	2402 images/sec	37 images/sec/watt	0.83	1x T4	Supermicro SYS-4029GP-TRT T4	19.03-py3	INT8	Synthetic	Tesla T4
	CNN	4	4049 images/sec	62 images/sec/watt	0.99	1x T4	Supermicro SYS-4029GP-TRT T4	19.03-py3	INT8	Synthetic	Tesla T4
	CNN	8	5684 images/sec	82 images/sec/watt	1.4	1x T4	Supermicro SYS-4029GP-TRT T4	19.03-py3	INT8	Synthetic	Tesla T4
	CNN	32	7302 images/sec	105 images/sec/watt	4.4	1x T4	Supermicro SYS-4029GP-TRT T4	19.03-py3	INT8	Synthetic	Tesla T4
	CNN	64	7435 images/sec	107 images/sec/watt	8.6	1x T4	Supermicro SYS-4029GP-TRT T4	19.03-py3	INT8	Synthetic	Tesla T4
	CNN	128	7546 images/sec	108 images/sec/watt	17	1x T4	Supermicro SYS-4029GP-TRT T4	19.03-py3	INT8	Synthetic	Tesla T4

MobileNet V2	CNN	1	1766 images/sec	34 images/sec/watt	0.57	1x T4	Supermicro SYS-4029GP-TRT T4	-	INT8	Synthetic	Tesla T4
	CNN	2	3235 images/sec	55 images/sec/watt	0.62	1x T4	Supermicro SYS-4029GP-TRT T4	-	INT8	Synthetic	Tesla T4
	CNN	4	5262 images/sec	79 images/sec/watt	0.76	1x T4	Supermicro SYS-4029GP-TRT T4	-	INT8	Synthetic	Tesla T4
	CNN	8	7251 images/sec	106 images/sec/watt	1.1	1x T4	Supermicro SYS-4029GP-TRT T4	-	INT8	Synthetic	Tesla T4
	CNN	32	8869 images/sec	129 images/sec/watt	3.6	1x T4	Supermicro SYS-4029GP-TRT T4	-	INT8	Synthetic	Tesla T4
	CNN	64	9003 images/sec	131 images/sec/watt	7.1	1x T4	Supermicro SYS-4029GP-TRT T4	-	INT8	Synthetic	Tesla T4
	CNN	128	9059 images/sec	131 images/sec/watt	14	1x T4	Supermicro SYS-4029GP-TRT T4	-	INT8	Synthetic	Tesla T4

ResNet-50	CNN	1	1087 images/sec	16 images/sec/watt	0.9	1x T4	Supermicro SYS-4029GP-TRT T4	19.03-py3	INT8	Synthetic	Tesla T4
	CNN	2	1735 images/sec	26 images/sec/watt	1.2	1x T4	Supermicro SYS-4029GP-TRT T4	19.03-py3	INT8	Synthetic	Tesla T4
	CNN	4	2887 images/sec	42 images/sec/watt	1.4	1x T4	Supermicro SYS-4029GP-TRT T4	19.03-py3	INT8	Synthetic	Tesla T4
	CNN	8	3766 images/sec	54 images/sec/watt	2.1	1x T4	Supermicro SYS-4029GP-TRT T4	19.03-py3	INT8	Synthetic	Tesla T4
	CNN	32	4677 images/sec	67 images/sec/watt	6.8	1x T4	Supermicro SYS-4029GP-TRT T4	19.03-py3	INT8	Synthetic	Tesla T4
	CNN	64	4815 images/sec	69 images/sec/watt	13	1x T4	Supermicro SYS-4029GP-TRT T4	19.03-py3	INT8	Synthetic	Tesla T4
	CNN	128	4944 images/sec	71 images/sec/watt	26	1x T4	Supermicro SYS-4029GP-TRT T4	19.03-py3	INT8	Synthetic	Tesla T4

VGG16	CNN	1	287 images/sec	4.2 images/sec/watt	3.5	1x T4	Supermicro SYS-4029GP-TRT T4	19.02-py3	INT8	Synthetic	Tesla T4
	CNN	2	487 images/sec	7.0 images/sec/watt	4.1	1x T4	Supermicro SYS-4029GP-TRT T4	19.02-py3	INT8	Synthetic	Tesla T4
	CNN	4	812 images/sec	12.0 images/sec/watt	4.9	1x T4	Supermicro SYS-4029GP-TRT T4	19.02-py3	INT8	Synthetic	Tesla T4
	CNN	8	1236 images/sec	18.0 images/sec/watt	6.5	1x T4	Supermicro SYS-4029GP-TRT T4	19.02-py3	INT8	Synthetic	Tesla T4
	CNN	32	1598 images/sec	23.0 images/sec/watt	20	1x T4	Supermicro SYS-4029GP-TRT T4	19.02-py3	INT8	Synthetic	Tesla T4
	CNN	64	1594 images/sec	23.0 images/sec/watt	40	1x T4	Supermicro SYS-4029GP-TRT T4	19.02-py3	INT8	Synthetic	Tesla T4
	CNN	128	1716 images/sec	25.0 images/sec/watt	75	1x T4	Supermicro SYS-4029GP-TRT T4	19.02-py3	INT8	Synthetic	Tesla T4

TensorRT 5.1, except TensorRT 5.0 for MobileNet V2 and VGG16

Last updated: April 9th, 2019

Search form

NVIDIA Tesla Deep Learning Product Performance

Deep Learning Training

NVIDIA Performance on MLPerf AI Benchmarks

Training Image Classification on CNNs

Training Performance

NVIDIA Performance on MLPerf AI Benchmarks

V100 Training Performance

T4 Training Performance

NVIDIA Deep Learning Inference Performance

Inference Image Classification on CNNs with TensorRT

Inference Performance

V100 Inference Performance

T4 Inference Performance