AI Training
Deploying AI in real-world applications requires training networks to convergence at a specified accuracy. This is the best methodology to test whether AI systems are ready to be deployed in the field to deliver meaningful results.
Click here to view other performance data.
NVIDIA Performance on MLPerf 5.0 Training Benchmarks
NVIDIA Performance on MLPerf 5.0’s AI Benchmarks: Single Node, Closed Division
| Framework | Network | Time to Train (mins) |
MLPerf Quality Target | GPU | Server | MLPerf-ID | Precision | Dataset | GPU Version |
|---|---|---|---|---|---|---|---|---|---|
| Nemo | Llama2-70B-lora | 11 | 0.925 Eval loss | 8x GB200 | BM.GPU.GB200.4 | 5.0-0020 | Mixed | SCROLLS GovReport | NVIDIA Blackwell GPU (GB200) |
| 11.2 | 0.925 Eval loss | 8x B200 | SYS-422GA-NBRT-LCC | 5.0-0089 | Mixed | SCROLLS GovReport | NVIDIA Blackwell GPU (B200-SXM-180GB) | ||
| Stable Diffusion | 12.9 | FID⇐90 and and CLIP>=0.15 | 8x GB200 | Tyche (1x NVIDIA GB200 NVL72) | 5.0-0071 | Mixed | LAION-400M-filtered | NVIDIA Blackwell GPU (GB200) | |
| 13 | FID⇐90 and and CLIP>=0.15 | 8x B200 | SYS-422GA-NBRT-LCC | 5.0-0089 | Mixed | LAION-400M-filtered | NVIDIA Blackwell GPU (B200-SXM-180GB) | ||
| PyTorch | BERT | 3.4 | 0.72 Mask-LM accuracy | 8x GB200 | Tyche (1x NVIDIA GB200 NVL72) | 5.0-0072 | Mixed | Wikipedia 2020/01/01 | NVIDIA Blackwell GPU (GB200) |
| 3.5 | 0.72 Mask-LM accuracy | 8x B200 | 1xXE9680Lx8B200-SXM-180GB | 5.0-0033 | Mixed | Wikipedia 2020/01/01 | NVIDIA Blackwell GPU (B200-SXM-180GB) | ||
| RetinaNet | 22.3 | 34.0% mAP | 8x GB200 | Tyche (1x NVIDIA GB200 NVL72) | 5.0-0072 | Mixed | A subset of OpenImages | NVIDIA Blackwell GPU (GB200) | |
| 21.8 | 34.0% mAP | 8x B200 | AS-A126GS-TNBR | 5.0-0085 | Mixed | A subset of OpenImages | NVIDIA Blackwell GPU (B200-SXM-180GB) | ||
| DGL | R-GAT | 5 | 72.0 % classification | 8x GB200 | Tyche (1x NVIDIA GB200 NVL72) | 5.0-0069 | Mixed | IGBH-Full | NVIDIA Blackwell GPU (GB200) |
| 5.1 | 72.0 % classification | 8x B200 | G893-SD1 | 5.0-0046 | Mixed | IGBH-Full | NVIDIA Blackwell GPU (B200-SXM-180GB) | ||
| NVIDIA Merlin HugeCTR | DLRM-dcnv2 | 2.2 | 0.80275 AUC | 8x GB200 | Tyche (1x NVIDIA GB200 NVL72) | 5.0-0070 | Mixed | Criteo 3.5TB Click Logs (multi-hot variant) | NVIDIA Blackwell GPU (GB200) |
| 2.3 | 0.80275 AUC | 8x B200 | Nyx (1x NVIDIA DGX B200) | 5.0-0061 | Mixed | Criteo 3.5TB Click Logs (multi-hot variant) | NVIDIA Blackwell GPU (B200-SXM-180GB) |
NVIDIA Performance on MLPerf 5.0’s AI Benchmarks: Multi Node, Closed Division
| Framework | Network | Time to Train (mins) |
MLPerf Quality Target | GPU | Server | MLPerf-ID | Precision | Dataset | GPU Version |
|---|---|---|---|---|---|---|---|---|---|
| NVIDIA NeMo | Llama 3.1 405B | 240.3 | 5.6 log perplexity | 256x GB200 | Tyche (4x NVIDIA GB200 NVL72) | 5.0-0075 | Mixed | c4/en/3.0.1 | NVIDIA Blackwell GPU (GB200) |
| 121.1 | 5.6 log perplexity | 512x GB200 | Carina (8x NVIDIA GB200 NVL72) | 5.0-0005 | Mixed | c4/en/3.0.1 | NVIDIA Blackwell GPU (GB200) | ||
| 62.1 | 5.6 log perplexity | 1,024x GB200 | Carina (16x NVIDIA GB200 NVL72) | 5.0-0001 | Mixed | c4/en/3.0.1 | NVIDIA Blackwell GPU (GB200) | ||
| 27.3 | 5.6 log perplexity | 2,496x GB200 | Carina (39x NVIDIA GB200 NVL72) | 5.0-0004 | Mixed | c4/en/3.0.1 | NVIDIA Blackwell GPU (GB200) | ||
| 20.8 | 5.6 log perplexity | 8,192x H100 | Eos-dfw (1024x NVIDIA HGX H100) | 5.0-0010 | Mixed | c4/en/3.0.1 | NVIDIA H100-SXM5-80GB | ||
| Llama2-70B-lora | 1.9 | 0.925 Eval loss | 64x GB200 | 16xXE9712x4GB200 | 5.0-0031 | Mixed | SCROLLS GovReport | NVIDIA Blackwell GPU (GB200) | |
| 1.1 | 0.925 Eval loss | 144x GB200 | Tyche (2x NVIDIA GB200 NVL72) | 5.0-0073 | Mixed | SCROLLS GovReport | NVIDIA Blackwell GPU (GB200) | ||
| 0.6 | 0.925 Eval loss | 512x GB200 | Tyche (8x NVIDIA GB200 NVL72) | 5.0-0076 | Mixed | SCROLLS GovReport | NVIDIA Blackwell GPU (GB200) | ||
| 6.1 | 0.925 Eval loss | 16x B200 | AS-4126GS-NBR-LCC_N2 | 5.0-0083 | Mixed | SCROLLS GovReport | NVIDIA Blackwell GPU (B200-SXM-180GB) | ||
| 2 | 0.925 Eval loss | 64x B200 | BM.GPU.B200.8 | 5.0-0018 | Mixed | SCROLLS GovReport | NVIDIA Blackwell GPU (B200-SXM-180GB) | ||
| Stable Diffusion | 7.6 | FID⇐90 and and CLIP>=0.15 | 16x GB200 | 4xXE9712x4GB200 | 5.0-0040 | Mixed | LAION-400M-filtered | NVIDIA Blackwell GPU (GB200) | |
| 4.3 | FID⇐90 and and CLIP>=0.15 | 32x GB200 | 8xXE9712x4GB200 | 5.0-0041 | Mixed | LAION-400M-filtered | NVIDIA Blackwell GPU (GB200) | ||
| 2.7 | FID⇐90 and and CLIP>=0.15 | 64x GB200 | 16xXE9712x4GB200 | 5.0-0031 | Mixed | LAION-400M-filtered | NVIDIA Blackwell GPU (GB200) | ||
| 1 | FID⇐90 and and CLIP>=0.15 | 512x GB200 | Tyche (8x NVIDIA GB200 NVL72) | 5.0-0076 | Mixed | LAION-400M-filtered | NVIDIA Blackwell GPU (GB200) | ||
| 2.8 | FID⇐90 and and CLIP>=0.15 | 64x B200 | BM.GPU.B200.8 | 5.0-0018 | Mixed | LAION-400M-filtered | NVIDIA Blackwell GPU (B200-SXM-180GB) | ||
| PyTorch | BERT | 2.1 | 0.72 Mask-LM accuracy | 16x GB200 | 4xXE9712x4GB200 | 5.0-0040 | Mixed | Wikipedia 2020/01/01 | NVIDIA Blackwell GPU (GB200) |
| 1.5 | 0.72 Mask-LM accuracy | 32x GB200 | 8xXE9712x4GB200 | 5.0-0041 | Mixed | Wikipedia 2020/01/01 | NVIDIA Blackwell GPU (GB200) | ||
| 0.7 | 0.72 Mask-LM accuracy | 64x GB200 | Tyche (1x NVIDIA GB200 NVL72) | 5.0-0065 | Mixed | Wikipedia 2020/01/01 | NVIDIA Blackwell GPU (GB200) | ||
| 0.3 | 0.72 Mask-LM accuracy | 512x GB200 | Tyche (8x NVIDIA GB200 NVL72) | 5.0-0077 | Mixed | Wikipedia 2020/01/01 | NVIDIA Blackwell GPU (GB200) | ||
| 2.3 | 0.72 Mask-LM accuracy | 16x B200 | 2xXE9680Lx8B200-SXM-180GB | 5.0-0037 | Mixed | Wikipedia 2020/01/01 | NVIDIA Blackwell GPU (B200-SXM-180GB) | ||
| RetinaNet | 12.3 | 34.0% mAP | 16x GB200 | 4xXE9712x4GB200 | 5.0-0040 | Mixed | A subset of OpenImages | NVIDIA Blackwell GPU (GB200) | |
| 9 | 34.0% mAP | 32x GB200 | 8xXE9712x4GB200 | 5.0-0041 | Mixed | A subset of OpenImages | NVIDIA Blackwell GPU (GB200) | ||
| 4.3 | 34.0% mAP | 64x GB200 | 16xXE9712x4GB200 | 5.0-0031 | Mixed | A subset of OpenImages | NVIDIA Blackwell GPU (GB200) | ||
| 1.4 | 34.0% mAP | 512x GB200 | Tyche (8x NVIDIA GB200 NVL72) | 5.0-0077 | Mixed | A subset of OpenImages | NVIDIA Blackwell GPU (GB200) | ||
| 14 | 34.0% mAP | 16x B200 | 2xXE9680Lx8B200-SXM-180GB | 5.0-0037 | Mixed | A subset of OpenImages | NVIDIA Blackwell GPU (B200-SXM-180GB) | ||
| 4.4 | 34.0% mAP | 64x B200 | BM.GPU.B200.8 | 5.0-0018 | Mixed | A subset of OpenImages | NVIDIA Blackwell GPU (B200-SXM-180GB) | ||
| DGL | R-GAT | 1.1 | 72.0 % classification | 72x GB200 | Tyche (1x NVIDIA GB200 NVL72) | 5.0-0066 | Mixed | IGBH-Full | NVIDIA Blackwell GPU (GB200) |
| 0.8 | 72.0 % classification | 256x GB200 | Tyche (4x NVIDIA GB200 NVL72) | 5.0-0074 | Mixed | IGBH-Full | NVIDIA Blackwell GPU (GB200) | ||
| NVIDIA Merlin HugeCTR | DLRM-dcnv2 | 0.7 | 0.80275 AUC | 64x GB200 | SRS-GB200-NVL72-M1 (16x ARS-121GL-NBO) | 5.0-0087 | Mixed | Criteo 3.5TB Click Logs (multi-hot variant) | NVIDIA Blackwell GPU (GB200) |
MLPerf™ v5.0 Training Closed: 5.0-0001, 5.0-0004, 5.0-0005, 5.0-0010, 5.0-0018, 5.0-0020, 5.0-0031, 5.0-0033, 5.0-0037, 5.0-0040, 5.0-0041, 5.0-0046, 5.0-0061, 5.0-0065, 5.0-0066, 5.0-0068, 5.0-0069, 5.0-0070, 5.0-0071, 5.0-0072, 5.0-0073, 5.0-0074, 5.0-0075, 5.0-0076, 5.0-0077, 5.0-0083, 5.0-0085, 5.0-0087, 5.0-0089 | MLPerf name and logo are trademarks. See https://mlcommons.org/ for more information.
For Training rules and guidelines, click here
NVIDIA Performance on MLPerf 3.0’s Training HPC Benchmarks: Closed Division
| Framework | Network | Time to Train (mins) |
MLPerf Quality Target | GPU | Server | MLPerf-ID | Precision | Dataset | GPU Version |
|---|---|---|---|---|---|---|---|---|---|
| PyTorch | CosmoFlow | 2.1 | Mean average error 0.124 | 512x H100 | eos | 3.0-8006 | Mixed | CosmoFlow N-body cosmological simulation data with 4 cosmological parameter targets | H100-SXM5-80GB |
| DeepCAM | 0.8 | IOU 0.82 | 2,048x H100 | eos | 3.0-8007 | Mixed | CAM5+TECA climate simulation with 3 target classes (atmospheric river, tropical cyclone, background) | H100-SXM5-80GB | |
| OpenCatalyst | 10.7 | Forces mean absolute error 0.036 | 640x H100 | eos | 3.0-8008 | Mixed | Open Catalyst 2020 (OC20) S2EF 2M training split, ID validation set | H100-SXM5-80GB | |
| OpenFold | 7.5 | Local Distance Difference Test (lDDT-Cα) >= 0.8 | 2,080x H100 | eos | 3.0-8009 | Mixed | OpenProteinSet and Protein Data Bank | H100-SXM5-80GB |
MLPerf™ v3.0 Training HPC Closed: 3.0-8006, 3.0-8007, 3.0-8008, 3.0-8009 | MLPerf name and logo are trademarks. See https://mlcommons.org/ for more information.
For MLPerf™ v3.0 Training HPC rules and guidelines, click here
LLM Training Performance on NVIDIA Data Center Products
B200 Training Performance
| Framework | Model | Time to Train (days) | Throughput per GPU | GPU | Server | Container Version | Sequence Length | TP | PP | CP | Precision | Global Batch Size | GPU Version |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Nemo | Llama3 8B | 0.4 | 30,131 tokens/sec | 8x B200 | DGX B200 | nemo:25.07 | 8192 | 1 | 1 | 1 | FP8 | 128 | NVIDIA B200 |
| Llama3 70B | 3.1 | 3,690 tokens/sec | 64x B200 | DGX B200 | nemo:25.07 | 8192 | 1 | 1 | 1 | FP8 | 128 | NVIDIA B200 | |
| Llama3 405B | 16.8 | 674 tokens/sec | 128x B200 | DGX B200 | nemo:25.07 | 8192 | 4 | 8 | 2 | FP8 | 64 | NVIDIA B200 | |
| Llama4 Scout | 1 | 11,260 tokens/sec | 64x B200 | DGX B200 | nemo:25.07 | 8192 | 1 | 2 | 1 | FP8 | 1024 | NVIDIA B200 | |
| Llama4 Maverick | 1.2 | 9,811 tokens/sec | 128x B200 | DGX B200 | nemo:25.07 | 8192 | 1 | 2 | 1 | FP8 | 1024 | NVIDIA B200 | |
| Mixtral 8x7B | 0.7 | 16,384 tokens/sec | 64x B200 | DGX B200 | nemo:25.07 | 4096 | 1 | 1 | 1 | FP8 | 256 | NVIDIA B200 | |
| Mixtral 8x22B | 4.4 | 2,548 tokens/sec | 256x B200 | DGX B200 | nemo:25.07 | 65536 | 2 | 4 | 8 | FP8 | 64 | NVIDIA B200 | |
| Nemotron5-H56B | 2.7 | 4,123 tokens/sec | 64x B200 | DGX B200 | nemo:25.07 | 8192 | 4 | 1 | 1 | FP8 | 192 | NVIDIA B200 | |
| DeepSeek v3 | 6.9 | 1,640 tokens/sec | 256x B200 | DGX B200 | nemo:25.07 | 4096 | 2 | 16 | 1 | FP8 | 2048 | NVIDIA B200 |
TP: Tensor Parallelism
PP: Pipeline Parallelism
CP: Context Parallelism
Time to Train is estimated time to train on 1T tokens with 1K GPUs
View More Performance Data
AI Inference
Real-world inferencing demands high throughput and low latencies with maximum efficiency across use cases. An industry-leading solution lets customers quickly deploy AI models into real-world production with the highest performance from data center to edge.
Learn MoreAI Pipeline
NVIDIA Riva is an application framework for multimodal conversational AI services that deliver real-performance on GPUs.
Learn More