AI Inference

Sep 30, 2026

Deploying an HSTU Generative Recommender with NVIDIA Dynamo-Triton

Generative recommender (GR) systems are emerging as a powerful new approach for large-scale personalization. Instead of treating recommendation as a set of...
11 MIN READ
Sep 22, 2026

Enabling Private High-Performance Production AI Inference with NVIDIA Confidential Computing

As large language model (LLM) inference increasingly processes sensitive information and proprietary model context across personal, enterprise, and regulated...
6 MIN READ
Sep 21, 2026

Simplifying Model Serving Across Multiple GPUs with NVIDIA TensorRT Multi-Device Integration in NVIDIA Dynamo-Triton

The compute and memory demands of generative AI increasingly exceed what a single GPU can provide. NVIDIA TensorRT multi-device inference is a new capability...
7 MIN READ
Sep 18, 2026

Benchmarking LLM Inference at Scale with AIPerf

You’re deploying a model on a system. It starts up, prompts are getting responses. Now the hard question: Is this fast? Your instincts might lead you to send...
11 MIN READ
Sep 16, 2026

TensorRT Edge-LLM Completes the MLPerf Edge Agentic Benchmark 6.4x Faster on Jetson AGX Thor

AI agents are moving from cloud data centers to vehicles, robots, and other edge devices. Unlike a chatbot that answers a single prompt, an agent works through...
7 MIN READ
Sep 15, 2026

Dense vs. MoE Models: Active Parameters, Throughput, and When to Choose Each

How can a 30B-parameter model activate only 3B parameters per token, and still use the capacity of the larger model? Nemotron 3.5 Lightning illustrates the...
9 MIN READ
Sep 15, 2026

How NVIDIA Groq 3 LPX Deterministic Execution Drives Power-Efficient High-Interactivity Inference on NVIDIA Vera Rubin

Power is a defining constraint for AI factories. As AI workloads demand a full compute platform to serve them, each component of that platform must maximize...
10 MIN READ
Sep 09, 2026

When to Use Encode-Prefill-Decode Disaggregation to Accelerate Multimodal Model Serving

Encode-prefill-decode (EPD) disaggregation is an inference optimization technique for multimodal models that separates the vision encoder stage from the...
14 MIN READ
Sep 03, 2026

NVIDIA PAIR Virtual Inference Router Expands Available Compute on Your Local Network

AI agents are learning to do more by working together. A lead agent can break a complex task into smaller jobs and assign those jobs to specialized subagents....
11 MIN READ
Sep 02, 2026

Co-Designing AI Models Using Speculative Decoding for Faster LLM Inference

This post is the third in a series on AI model co-design. It explores how to accelerate LLM inference while maintaining accuracy using speculative decoding and...
15 MIN READ
Sep 01, 2026

How to Size GPUs for AI Inference and TCO Without Overspending

The surge in AI adoption is transforming everything from chatbots to content generation. Still, a common pain point remains: How can organizations confidently...
13 MIN READ
Aug 28, 2026

Deploy an Open Model from Checkpoint to Inference in Two Commands with NVIDIA TensorRT Model Connect

Open AI models are evolving faster than ever, but bringing them into native applications can still require model-specific conversion, preprocessing,...
6 MIN READ
Aug 24, 2026

How NVIDIA Groq 3 LPX Unlocks Ultrafast Interactivity at Long Context on NVIDIA Vera Rubin

NVIDIA Groq 3 LPX is the interactive AI inference accelerator for the NVIDIA Vera Rubin platform. At the core of the platform is NVIDIA Vera Rubin NVL72, the...
13 MIN READ
Aug 24, 2026

Maximizing AI Factory Performance per Watt with NVIDIA DSX MaxLPS

AI factories are power-constrained industrial systems. The question is no longer how many GPUs fit in a data center, but how much AI output each available...
13 MIN READ
Aug 20, 2026

How Generative Recommenders Are Redefining RecSys at Scale

Recommender systems (RecSys) are one of the most ubiquitous machine learning problems in the consumer internet industry yet notoriously difficult to train and...
11 MIN READ
Aug 11, 2026

NVIDIA JetPack 7.2.1 Adds Agentic Video Skills and T3000 Emulation

Video is a core data path across NVIDIA Jetson applications, from robotics and intelligent video analytics to industrial automation, healthcare, media...
7 MIN READ