AI Inference

Oct 23, 2026

Scaling NVIDIA TensorRT Performance Optimization Beyond a Single Build Route with the Global Performance Tuner, Part 2

The Global Performance Tuner is a feature for automatic performance tuning released in NVIDIA TensorRT 11.1. It provides automatic, accuracy-aware and...
12 MIN READ
Oct 23, 2026

Scaling NVIDIA TensorRT Performance Optimization Beyond a Single Build Route with the Global Performance Tuner, Part 1

Deploying a trained neural network involves more than translating each operator into a GPU kernel. It is a compilation problem with many legitimate answers....
14 MIN READ
Oct 22, 2026

Running Frontier-Class AI Models Across 8 NVIDIA DGX Spark Nodes

AI developers aren’t just running a single model anymore—they’re orchestrating multiple models in parallel, chaining inference calls, and experimenting with...
9 MIN READ
Oct 16, 2026

Code, Test, and Deploy on One Machine with NVIDIA RTX Spark

AI agents are changing how software gets built. Developers have gone from writing every line of code by hand to directing agents that write most of the code....
7 MIN READ
A cube assembled from smaller dark blocks, with glowing cyan, pink, yellow, and green sections against a black background.
Oct 12, 2026

ReplaySSM: Rethinking State Storage for Faster Hybrid-Model Decoding

Hybrid models interleave state-space model (SSM) layers with a few attention layers. NVIDIA Nemotron 3, Qwen3.5 through Qwen3.8, Kimi-K3, and GLM-5.3-Flash are...
14 MIN READ
Sep 30, 2026

Deploying an HSTU Generative Recommender with NVIDIA Dynamo-Triton

Generative recommender (GR) systems are emerging as a powerful new approach for large-scale personalization. Instead of treating recommendation as a set of...
11 MIN READ
Sep 22, 2026

Enabling Private High-Performance Production AI Inference with NVIDIA Confidential Computing

As large language model (LLM) inference increasingly processes sensitive information and proprietary model context across personal, enterprise, and regulated...
6 MIN READ
Sep 21, 2026

Simplifying Model Serving Across Multiple GPUs with NVIDIA TensorRT Multi-Device Integration in NVIDIA Dynamo-Triton

The compute and memory demands of generative AI increasingly exceed what a single GPU can provide. NVIDIA TensorRT multi-device inference is a new capability...
7 MIN READ
Sep 18, 2026

Benchmarking LLM Inference at Scale with AIPerf

You’re deploying a model on a system. It starts up, prompts are getting responses. Now the hard question: Is this fast? Your instincts might lead you to send...
11 MIN READ
Sep 16, 2026

TensorRT Edge-LLM Completes the MLPerf Edge Agentic Benchmark 6.4x Faster on Jetson AGX Thor

AI agents are moving from cloud data centers to vehicles, robots, and other edge devices. Unlike a chatbot that answers a single prompt, an agent works through...
7 MIN READ
Sep 15, 2026

Dense vs. MoE Models: Active Parameters, Throughput, and When to Choose Each

How can a 30B-parameter model activate only 3B parameters per token, and still use the capacity of the larger model? Nemotron 3.5 Lightning illustrates the...
9 MIN READ
Sep 15, 2026

How NVIDIA Groq 3 LPX Deterministic Execution Drives Power-Efficient High-Interactivity Inference on NVIDIA Vera Rubin

Power is a defining constraint for AI factories. As AI workloads demand a full compute platform to serve them, each component of that platform must maximize...
10 MIN READ
Sep 09, 2026

When to Use Encode-Prefill-Decode Disaggregation to Accelerate Multimodal Model Serving

Encode-prefill-decode (EPD) disaggregation is an inference optimization technique for multimodal models that separates the vision encoder stage from the...
14 MIN READ
Sep 03, 2026

NVIDIA PAIR Virtual Inference Router Expands Available Compute on Your Local Network

AI agents are learning to do more by working together. A lead agent can break a complex task into smaller jobs and assign those jobs to specialized subagents....
11 MIN READ
Sep 02, 2026

Co-Designing AI Models Using Speculative Decoding for Faster LLM Inference

This post is the third in a series on AI model co-design. It explores how to accelerate LLM inference while maintaining accuracy using speculative decoding and...
15 MIN READ
Sep 01, 2026

How to Size GPUs for AI Inference and TCO Without Overspending

The surge in AI adoption is transforming everything from chatbots to content generation. Still, a common pain point remains: How can organizations confidently...
13 MIN READ