Inference Performance

Sep 21, 2026

Simplifying Model Serving Across Multiple GPUs with NVIDIA TensorRT Multi-Device Integration in NVIDIA Dynamo-Triton

The compute and memory demands of generative AI increasingly exceed what a single GPU can provide. NVIDIA TensorRT multi-device inference is a new capability...
7 MIN READ
Sep 18, 2026

Benchmarking LLM Inference at Scale with AIPerf

You’re deploying a model on a system. It starts up, prompts are getting responses. Now the hard question: Is this fast? Your instincts might lead you to send...
11 MIN READ
Sep 15, 2026

How NVIDIA Groq 3 LPX Deterministic Execution Drives Power-Efficient High-Interactivity Inference on NVIDIA Vera Rubin

Power is a defining constraint for AI factories. As AI workloads demand a full compute platform to serve them, each component of that platform must maximize...
10 MIN READ
Sep 10, 2026

How Full-Stack NIM Optimizations Deliver 2.5x More Users on Nemotron 3 Ultra

Deploying a large language model is only the first step toward production-ready serving. Production teams also need to serve as many concurrent users as...
6 MIN READ
Sep 02, 2026

Co-Designing AI Models Using Speculative Decoding for Faster LLM Inference

This post is the third in a series on AI model co-design. It explores how to accelerate LLM inference while maintaining accuracy using speculative decoding and...
15 MIN READ
Sep 01, 2026

How to Size GPUs for AI Inference and TCO Without Overspending

The surge in AI adoption is transforming everything from chatbots to content generation. Still, a common pain point remains: How can organizations confidently...
13 MIN READ
Aug 24, 2026

How NVIDIA Groq 3 LPX Unlocks Ultrafast Interactivity at Long Context on NVIDIA Vera Rubin

NVIDIA Groq 3 LPX is the interactive AI inference accelerator for the NVIDIA Vera Rubin platform. At the core of the platform is NVIDIA Vera Rubin NVL72, the...
13 MIN READ
Jul 31, 2026

Co-Designing AI Model Attention for Fast, Interactive Long-Context Inference

As agentic and long-context workloads become common, the context lengths increase and attention consumes a larger share of inference time (Figure 1). Because...
14 MIN READ
Jul 24, 2026

ModelExpress: Distributing Model Artifacts at the Speed of Light

Every byte moved has a cost. As model checkpoints grow to hundreds of gigabytes or even a terabyte, that cost adds up quickly. To make things even worse,...
12 MIN READ
Jul 10, 2026

AI Model Co-Design: Hardware-Friendly LLM Design

AI performance comes down to three dimensions:  Accuracy: How well the model reasons and produces outputs Throughput: How many tokens per second a...
17 MIN READ
Decorative image.
Jul 02, 2026

Hardware-Rooted AI Security That Won't Slow You Down

AI has transformed how organizations operate, driving unprecedented levels of productivity and innovation. However, AI adoption can be impeded by concerns...
6 MIN READ
Decorative image.
Jun 25, 2026

Scaling AI Inference Across Multiple GPUs Using NVIDIA TensorRT with Multi-Device Inference Support

Generative AI workloads are rapidly outgrowing the memory and compute budget of single GPUs. For inference developers building media generation pipelines, the...
11 MIN READ
Jun 23, 2026

Maximize AI Factory Energy Efficiency Through Full-Stack Inference and Training Optimizations

Power can account for 40% of the operating expenses (OpEx) to run an AI factory. Each watt can be spent on overhead, data ingestion, training, or generating...
10 MIN READ
Jun 23, 2026

Boost Inference Performance up to 15x on NVIDIA Blackwell Using DFlash Speculative Decoding

As AI systems move from single-turn interactions to coordinated multiagent workflows, low-latency inference becomes increasingly important. Autoregressive LLMs...
7 MIN READ
Jun 12, 2026

NVIDIA Achieves Leading Agentic Coding Performance on First Agentic AI Benchmark

AI agents have fundamentally changed the complexity of inference workloads. Until now, the industry has struggled to define a standard for measuring how...
6 MIN READ
Decorative image.
Jun 09, 2026

Model Quantization: Turn FP8 Checkpoints into High-Performance Inference Engines with NVIDIA TensorRT

This post is the third of a three-part series. See also Model Quantization: Concepts, Methods, and Why It Matters and Model Quantization: Post-Training...
10 MIN READ