TensorRT

Jan 08, 2026

Delivering Massive Performance Leaps for Mixture of Experts Inference on NVIDIA Blackwell

As AI models continue to get smarter, people can rely on them for an expanding set of tasks. This leads users—from consumers to enterprises—to interact with...
6 MIN READ
Jan 08, 2026

Accelerating LLM and VLM Inference for Automotive and Robotics with NVIDIA TensorRT Edge-LLM

Large language models (LLMs) and multimodal reasoning systems are rapidly expanding beyond the data center. Automotive and robotics developers increasingly want...
6 MIN READ
Four-image grid illustrating AI agents, robotics, data center infrastructure, and simulated environments.
Dec 31, 2025

AI Factories, Physical AI, and Advances in Models, Agents, and Infrastructure That Shaped 2025

2025 was another milestone year for developers and researchers working with NVIDIA technologies. Progress in data center power and compute design, AI...
4 MIN READ
Dec 16, 2025

Accelerating Long-Context Inference with Skip Softmax in NVIDIA TensorRT-LLM

For machine learning engineers deploying LLMs at scale, the equation is familiar and unforgiving: as context length increases, attention computation costs...
6 MIN READ
Dec 09, 2025

Top 5 AI Model Optimization Techniques for Faster, Smarter Inference

As AI models get larger and architectures more complex, researchers and engineers are continuously finding new techniques to optimize the performance and...
6 MIN READ
Dec 08, 2025

Optimizing Inference for Long Context and Large Batch Sizes with NVFP4 KV Cache

Quantization is one of the strongest levers for large-scale inference. By reducing the precision of weights, activations, and KV cache, we can reduce the memory...
10 MIN READ
Dec 02, 2025

NVIDIA-Accelerated Mistral 3 Open Models Deliver Efficiency, Accuracy at Any Scale 

The new Mistral 3 open model family delivers industry-leading accuracy, efficiency, and customization capabilities for developers and enterprises. Optimized...
6 MIN READ
Decorative image.
Nov 24, 2025

Model Quantization: Concepts, Methods, and Why It Matters

AI models are becoming increasingly complex, often exceeding the capabilities of available hardware. Quantization has emerged as a crucial technique to address...
12 MIN READ
Decorative math image.
Nov 10, 2025

How to Achieve 4x Faster Inference for Math Problem Solving

Large language models can solve challenging math problems. However, making them work efficiently at scale requires more than a strong checkpoint. You need the...
7 MIN READ
Nov 04, 2025

How to Predict Biomolecular Structures Using the OpenFold3 NIM

​​For decades, one of biology’s deepest mysteries was how a string of amino acids folds itself into the intricate architecture of life. Researchers built...
5 MIN READ
Oct 20, 2025

Scaling Large MoE Models with Wide Expert Parallelism on NVL72 Rack Scale Systems

Modern AI workloads have moved well beyond single-GPU inference serving. Model parallelism, which efficiently splits computation across many GPUs, is now the...
11 MIN READ
Oct 13, 2025

NVIDIA Blackwell Leads on SemiAnalysis InferenceMAX v1 Benchmarks

SemiAnalysis recently launched InferenceMAX v1, a new open source initiative that provides a comprehensive methodology to evaluate inference hardware...
11 MIN READ
Oct 07, 2025

Pruning and Distilling LLMs Using NVIDIA TensorRT Model Optimizer

Large language models (LLMs) have set a high bar in natural language processing (NLP) tasks such as coding, reasoning, and math. However, their deployment...
11 MIN READ
Sep 23, 2025

Deploy High-Performance AI Models in Windows Applications on NVIDIA RTX AI PCs

Today, Microsoft is making Windows ML available to developers. Windows ML enables C#, C++ and Python developers to optimally run AI models locally across PC...
8 MIN READ
Sep 17, 2025

An Introduction to Speculative Decoding for Reducing Latency in AI Inference

Generating text with large language models (LLMs) often involves running into a fundamental bottleneck. GPUs offer massive compute, yet much of that power sits...
11 MIN READ
Sep 11, 2025

How Quantization Aware Training Enables Low-Precision Accuracy Recovery

After training AI models, a variety of compression techniques can be used to optimize them for deployment. The most common is post-training quantization (PTQ),...
10 MIN READ