Jetson Benchmarks
Jetson is used to deploy a wide range of popular DNN models and ML frameworks to the edge with high performance inferencing, for tasks like real-time classification and object detection, pose estimation, semantic segmentation, and natural language processing (NLP). The tables below show inferencing benchmarks from the NVIDIA Jetson submissions to the MLPerf Inference Edge category.
| Model | Jetson Xavier NX (TensorRT) | NVIDIA Orin (TensorRT) |
|---|---|---|
| Image Classification ResNet-50 | 1243.14 | 6138.84 |
| Object Detection SSD-small | 1782.53 | 6883.49 |
| Object Detection SSD-Large | 36.69 | 207.66 |
| Speech to Text RNN-T | 260.52 | 1110.23 |
| Natural Language Processing BERT-Large | 61.40 | 476.34 |
- NVIDIA Orin can be found in Jetson AGX Orin
- These are preview results, and can be found at: v2.0 Results | MLCommons
- These results were achieved with the NVIDIA Jetson AGX Orin Developer Kit running a preview of TensorRT 8.4.0, and CUDA 11.4
- ResNet-50, SSD-small, and SSD-Large were run on the GPU and both DLAs
- Learn more about these results in our blog: NVIDIA Orin Leaps Ahead in Edge AI in MLPerf Tests | NVIDIA Blog
| Model | Jetson Xavier NX (TensorRT) | Jetson AGX Xavier 32GB (TensorRT) |
|---|---|---|
| Image Classification ResNet-50 | 1245.10 | 2039.11 |
| Object Detection SSD-small | 1786.91 | 2833.59 |
| Object Detection SSD-Large | 36.97 | 55.16 |
| Speech to Text RNN-T | 259.67 | 416.13 |
| Natural Language Processing BERT-Large | 61.34 | 96.73 |
- Full Results can be found at v1.1 Results | MLCommons
- These results were achieved on Jetson AGX Xavier Developer Kit and Jetson Xavier NX Developer kit running Jetpack 4.6, TensorRT 8.0.1, CUDA 10.2
- ResNet-50, SSD-small, and SSD-Large were run on the GPU and both DLAs
- These MLPerf Results can be reproduced with the code in the following link: https://github.com/mlcommons/inference_results_v2.0/tree/master/closed/NVIDIA
Jetson Pretrained Model Benchmarks
NVIDIA pretrained models from NGC start you off with highly accurate and optimized models and model architectures for various use cases. Pretrained models are production-ready. You can further customize these models by training with your own real or synthetic data, using the NVIDIA TAO (Train-Adapt-Optimize) workflow to quickly build an accurate and ready to deploy model.The table below shows inferencing benchmarks for some of our pretrained models running on Jetson modules.
| Jetson Xavier NX | Jetson AGX Xavier | Jetson AGX Orin | |
|---|---|---|---|
| PeopleNet | 124 | 196 | 536 |
| Action Recognition 2D | 245 | 471 | 1577 |
| Action Recognition 3D | 21 | 32 | 105 |
| LPR Net | 706 | 1190 | 4118 |
| Dashcam Net | 425 | 671 | 1908 |
| Bodypose Net | 105 | 172 | 559 |
| ASR: Citrinet 1024 | 27 | 34 | 113 |
| NLP: BERT-base | 58 | 94 | 287 |
| TTS: Fastpitch-HifiGAN | 7 | 9 | 42 |
- These Benchmarks were run using Jetpack 5.0
- Each Jetson module was run with maximum performance (MAX-N mode for Jetson AGX Xavier (32 GB) and Jetson AGX Orin, and 20W mode for Jetson Xavier NX)
- Reproduce these results by downloading these models from our NGC catalog