LLM Benchmarking

Sep 23, 2026

How SWE-Serve Exposes the Gap Between Local Tests and Live Serving

An AI coding agent’s patch can pass tests yet fail when the server loads a real model and handles requests. Evaluating changes to inference-serving software...
7 MIN READ
Sep 21, 2026

How to Evaluate AI Agents From Tool Calls to Task Completion

When you ship an AI agent, the key question is whether it can execute a chain of work across dozens of sequential tool calls against a live environment, and...
11 MIN READ
Sep 16, 2026

TensorRT Edge-LLM Completes the MLPerf Edge Agentic Benchmark 6.4x Faster on Jetson AGX Thor

AI agents are moving from cloud data centers to vehicles, robots, and other edge devices. Unlike a chatbot that answers a single prompt, an agent works through...
7 MIN READ
Jun 16, 2026

NVIDIA Blackwell Tops MLPerf Training 6.0 with Industry-Leading Scale and Performance

NVIDIA delivered a clean sweep in MLPerf Training v6.0, the latest edition of industry-standard AI training benchmarks developed by the MLCommons consortium....
11 MIN READ
Jun 16, 2026

How to Optimize Transformer-Based Models for Low-Precision Training

Transformer architectures are the backbone of many modern large language and generative AI models. As these models grow in size, training runs consume more GPU...
9 MIN READ
Mar 11, 2026

Introducing Nemotron 3 Super: An Open Hybrid Mamba-Transformer MoE for Agentic Reasoning

Agentic AI systems need models with the specialized depth to solve dense technical problems autonomously. They must excel at reasoning, coding, and...
12 MIN READ
Oct 24, 2025

How NVIDIA DGX Spark's Performance Enables Intensive AI Tasks

Today’s demanding AI developer workloads often need more memory than desktop systems provide or require access to software that laptops or PCs lack. This...
5 MIN READ
Sep 11, 2025

How Quantization Aware Training Enables Low-Precision Accuracy Recovery

After training AI models, a variety of compression techniques can be used to optimize them for deployment. The most common is post-training quantization (PTQ),...
10 MIN READ
Aug 29, 2025

Fine-Tuning gpt-oss for Accuracy and Performance with Quantization Aware Training

Major open-source foundational model releases are an exciting time for the AI community, bringing unique architectural innovations and capabilities. As the...
7 MIN READ
Jul 07, 2025

LLM Inference Benchmarking: Performance Tuning with TensorRT-LLM

This is the third post in the large language model latency-throughput benchmarking series, which aims to instruct developers on how to benchmark LLM inference...
11 MIN READ
Decorative image.
Jun 18, 2025

LLM Inference Benchmarking: How Much Does Your LLM Inference Cost?

Learn how to calculate LLM inference costs using NVIDIA GenAI-Perf benchmarking tools and TCO formulas. This guide covers performance metrics (TTFT,...
11 MIN READ
Jun 04, 2025

Reproducing NVIDIA MLPerf v5.0 Training Scores for LLM Benchmarks

The previous post, NVIDIA Blackwell Delivers up to 2.6x Higher Performance in MLPerf Training v5.0, explains how the NVIDIA platform delivered the fastest time...
11 MIN READ
Jun 04, 2025

NVIDIA Blackwell Delivers up to 2.6x Higher Performance in MLPerf Training v5.0

The journey to create a state-of-the-art large language model (LLM) begins with a process called pretraining. Pretraining a state-of-the-art model is...
12 MIN READ
May 18, 2025

Announcing NVIDIA Exemplar Clouds for Benchmarking AI Cloud Infrastructure

Developers and enterprises training large language models (LLMs) and deploying AI workloads in the cloud have long faced a fundamental challenge: it’s nearly...
4 MIN READ
Decorative image of a datacenter with floating icons overlaid.
May 06, 2025

LLM Inference Benchmarking Guide: NVIDIA GenAI-Perf and NIM

This is the second post in the LLM Benchmarking series, which shows how to use GenAI-Perf to benchmark the Meta Llama 3 model when deployed with NVIDIA NIM. ...
11 MIN READ
Apr 24, 2025

Benchmarking Agentic LLM and VLM Reasoning for Gaming with NVIDIA NIM

This is the first post in the LLM Benchmarking series, which shows how to use GenAI-Perf to benchmark the Meta Llama 3 model when deployed with NVIDIA NIM. ...
7 MIN READ