Jetson Benchmarks

Jetson is used to deploy a wide range of popular DNN models and ML frameworks to the edge with high performance inferencing, for tasks like real-time classification and object detection, pose estimation, semantic segmentation, and natural language processing (NLP).

MLPerf Inference Benchmarks

The tables below show inferencing benchmarks from the NVIDIA Jetson submissions to the MLPerf Inference Edge category.

Jetson AGX Orin Jetson Orin NX MLPerf v3.0 Results

Model NVIDIA Jetson AGX Orin (TensorRT) NVIDIA Orin MaxQ (TensorRT) NVIDIA Jetson Orin NX
Single Stream (Samples/s) Offline (Samples/s) Multi Stream (Samples/s) Offline (Samples/s) System Power(W) Offline (Samples/s)
Image Classification
ResNet-50
1538 6438.10 3686 3525.91 23.06 2517.99
Object Detection
Retinanet
51.57 92.40 60.00 34.6 22.4 36.14
Medical Imaging
3D-Unet
.26 .51 N/A 3.28 28.64 .19
Speech-to-text
RNN-T
9.822 1170.23 N/A 14472 25.64 405.27
Natural Language Processing
BERT
144.36 544.24 N/A 3685.36 25.91 163.57
  • Steps to reproduce these results can be found at v3.0 Results | MLCommons
  • These results were achieved with the NVIDIA Jetson AGX Orin Developer Kit running a preview of TensorRT 8.5.0, and CUDA 11.4
  • Note different configurations were used for single stream, offline and multistream. Reference the MLCommons page for more details


Jetson AGX Xavier and Jetson Xavier NX MLPerf v1.1 Results

Model Jetson Xavier NX (TensorRT) Jetson AGX Xavier 32GB (TensorRT)
Image Classification
ResNet-50
1245.10 2039.11
Object Detection
SSD-small
1786.91 2833.59
Object Detection
SSD-Large
36.97 55.16
Speech to Text
RNN-T
259.67 416.13
Natural Language
Processing
BERT-Large
61.34 96.73


NVIDIA Pretrained Model Benchmarks

NVIDIA pretrained models from NGC start you off with highly accurate and optimized models and model architectures for various use cases. Pretrained models are production-ready. You can further customize these models by training with your own real or synthetic data, using the NVIDIA TAO (Train-Adapt-Optimize) workflow to quickly build an accurate and ready to deploy model.The table below shows inferencing benchmarks for some of our pretrained models running on Jetson modules.

Jetson Pretrained Model Benchmarks


Jetson Orin Results

Model Jetson Orin Nano 4GB Jetson Orin Nano 8GB Jetson Orin NX 8GB Jetson Orin NX 16GB Jetson AGX Orin 32GB Jetson AGX Orin 64GB
PeopleNet (v2.5 unpruned) 57 117 192 240 409 685
Action Recognition 2D 220 372 440 483 1158 1517
Action Recognition 3D 13 26 32 39 71 108
LPR Net 552 974 1314 1427 2800 4213
Dashcam Net 200 400 689 877 1482 2139
Bodypose Net 69 137 169 203 360 563
Model Jetson Nano Jetson TX2 NX Jetson Xavier NX Jetson AGX Xavier
PeopleNet (v2.5 unpruned) 2 5 120 195
Action Recognition 2D 32 88 245 472
Action Recognition 3D 1 3 21 32
LPR Net 47 86 714 1236
Dashcam Net 11 26 424 667
Bodypose Net 3 7 104 172

  • Jetson Orin & Jetson Xavier Benchmarks were run using Jetpack 5.1.1
  • Each Jetson module was run with maximum performance (MAXN for JAO64, JAO32, ONX16, ONX8; and 15W mode for JON8, and 10W mode for JON4)
  • For Jetson Nano and Jetson TX2 NX, these benchmarks were run using Jetpack 4.6.1
  • Each Jetson module was run with maximum performance (MAXN)
  • Reproduce these results by downloading these models from our NGC catalog

Jetson Family Benchmarks

Model Jetson Orin Nano 4GB Jetson Orin Nano 8GB Jetson Orin NX 8GB Jetson Orin NX 16GB Jetson AGX Orin 32GB Jetson AGX Orin 64GB
Inveption_V4 182 361 593 769 1337.8 1702.6
VGG19 174 361 442 532 937 1471
Super_resolution 102 203 280 386 610 882
UNET-sgmentation 76 148 183 217 387 584
Pose Estimation 280 546 665 800 1424 2048
Yolov3-tiny 371 731 1156 1440 2611 3179
Resnet50 621 1158 1725 2183 3717 4834
SSD-Mobilnet 1094 2156 2893 3457 6415 7671
SSD_Resnet34_1200x1200 18 34 52 72 120 163
Yolov5m 69 131 162 193 342 519
Yolov5s 158 301 379 449 785 1135
  • These Benchmarks were run using Jetpack 5.1.1
  • Each Jetson module was run with maximum performance (Max Frequencies in MAXN for JAO64, JAO32, ONX16, ONX8; and 15W mode for JON8, and 10W mode for JON4)
  • Steps to reproduce these results can be found here