TensorRT

Oct 15, 2026

Scale Agentic Coding Workflows with Meta’s Muse Spark Open on NVIDIA GB300 NVL72  

Meta launches Muse Spark Open, a coding-optimized 816B mixture-of-experts (MoE) model with 42B active parameters and a 1M token context window. Trained on...
6 MIN READ
A cube assembled from smaller dark blocks, with glowing cyan, pink, yellow, and green sections against a black background.
Oct 12, 2026

ReplaySSM: Rethinking State Storage for Faster Hybrid-Model Decoding

Hybrid models interleave state-space model (SSM) layers with a few attention layers. NVIDIA Nemotron 3, Qwen3.5 through Qwen3.8, Kimi-K3, and GLM-5.3-Flash are...
14 MIN READ
Laptop surrounded by application icons and a compact computing device, illustrating local AI applications built with C++ and NVIDIA TensorRT RTX.
Oct 01, 2026

Build Local AI Apps with C++ and NVIDIA TensorRT RTX Samples

Adding AI models to local applications requires a portable model format, a reliable runtime, and acceleration that works across target systems. Do Inference...
3 MIN READ
Sep 29, 2026

AI Native by Design: Lessons Learned from Building NVIDIA TensorRT Model Connect

Parallel work, model-family isolation, reversible changes, and GPU-backed validation shaped an open source project designed around coding agents NVIDIA...
10 MIN READ
Server rack connected to a power gauge and control nodes, illustrating AI factory power management with NVIDIA DSX MaxLPS.
Sep 27, 2026

How NVIDIA DSX MaxLPS Maximizes AI Factory Throughput and Efficiency

Every unused watt is capacity left on the table. AI factories are typically provisioned for the unlikely moment when every GPU reaches peak power, creating a...
9 MIN READ
Sep 22, 2026

Enabling Private High-Performance Production AI Inference with NVIDIA Confidential Computing

As large language model (LLM) inference increasingly processes sensitive information and proprietary model context across personal, enterprise, and regulated...
6 MIN READ
Sep 21, 2026

Simplifying Model Serving Across Multiple GPUs with NVIDIA TensorRT Multi-Device Integration in NVIDIA Dynamo-Triton

The compute and memory demands of generative AI increasingly exceed what a single GPU can provide. NVIDIA TensorRT multi-device inference is a new capability...
7 MIN READ
Sep 18, 2026

Benchmarking LLM Inference at Scale with AIPerf

You’re deploying a model on a system. It starts up, prompts are getting responses. Now the hard question: Is this fast? Your instincts might lead you to send...
11 MIN READ
Sep 16, 2026

TensorRT Edge-LLM Completes the MLPerf Edge Agentic Benchmark 6.4x Faster on Jetson AGX Thor

AI agents are moving from cloud data centers to vehicles, robots, and other edge devices. Unlike a chatbot that answers a single prompt, an agent works through...
7 MIN READ
Sep 10, 2026

High-Throughput Structure Prediction with BioNeMo Inference Runtime

Biomolecular structure prediction is now often run at proteome scale, where the goal is to move an entire worklist through the pipeline efficiently. NVIDIA...
11 MIN READ
Sep 01, 2026

How to Size GPUs for AI Inference and TCO Without Overspending

The surge in AI adoption is transforming everything from chatbots to content generation. Still, a common pain point remains: How can organizations confidently...
13 MIN READ
Aug 28, 2026

Deploy an Open Model from Checkpoint to Inference in Two Commands with NVIDIA TensorRT Model Connect

Open AI models are evolving faster than ever, but bringing them into native applications can still require model-specific conversion, preprocessing,...
6 MIN READ
Aug 24, 2026

NVIDIA Vera Rubin and Blackwell Set a New Standard for Agentic AI Performance per Watt 

AI agents have expanded inference from single-turn interactions into multi-step workflows that reason, invoke tools, coordinate subagents, and carry growing...
9 MIN READ
Aug 20, 2026

How Generative Recommenders Are Redefining RecSys at Scale

Recommender systems (RecSys) are one of the most ubiquitous machine learning problems in the consumer internet industry yet notoriously difficult to train and...
11 MIN READ
Jul 31, 2026

Co-Designing AI Model Attention for Fast, Interactive Long-Context Inference

As agentic and long-context workloads become common, the context lengths increase and attention consumes a larger share of inference time (Figure 1). Because...
14 MIN READ
Decorative image.
Jul 22, 2026

Make Long-Running NVIDIA TensorRT Engine Builds Observable and Cancelable in Python or C++

A TensorRT engine build can take seconds to many minutes. Large strongly typed models, deep tactic search, and a cold timing cache on a brand-new GPU SKU can...
11 MIN READ