TensorRT

Oct 09, 2025

NVIDIA Blackwell Leads on SemiAnalysis InferenceMAX v1 Benchmarks

SemiAnalysis recently launched InferenceMAX v1, a new open source initiative that provides a comprehensive methodology to evaluate inference hardware...

11 MIN READ

Oct 07, 2025

Pruning and Distilling LLMs Using NVIDIA TensorRT Model Optimizer

Large language models (LLMs) have set a high bar in natural language processing (NLP) tasks such as coding, reasoning, and math. However, their deployment...

11 MIN READ

Sep 23, 2025

Deploy High-Performance AI Models in Windows Applications on NVIDIA RTX AI PCs

Today, Microsoft is making Windows ML available to developers. Windows ML enables C#, C++ and Python developers to optimally run AI models locally across PC...

8 MIN READ

Sep 17, 2025

An Introduction to Speculative Decoding for Reducing Latency in AI Inference

Generating text with large language models (LLMs) often involves running into a fundamental bottleneck. GPUs offer massive compute, yet much of that power sits...

11 MIN READ

Sep 11, 2025

How Quantization Aware Training Enables Low-Precision Accuracy Recovery

After training AI models, a variety of compression techniques can be used to optimize them for deployment. The most common is post-training quantization (PTQ),...

10 MIN READ

Sep 10, 2025

Accelerate Protein Structure Inference Over 100x with NVIDIA RTX PRO 6000 Blackwell Server Edition

The race to understand protein structures has never been more critical. From accelerating drug discovery to preparing for future pandemics, the ability to...

6 MIN READ

Sep 03, 2025

Accelerate Autonomous Vehicle Development with the NVIDIA DRIVE AGX Thor Developer Kit

Autonomous vehicle (AV) technology is rapidly evolving, fueled by ever-larger and more complex AI models deployed at the edge. Modern vehicles now require not...

8 MIN READ

Aug 29, 2025

Fine-Tuning gpt-oss for Accuracy and Performance with Quantization Aware Training

Major open-source foundational model releases are an exciting time for the AI community, bringing unique architectural innovations and capabilities. As the...

7 MIN READ

Aug 22, 2025

Inside NVIDIA Blackwell Ultra: The Chip Powering the AI Factory Era

As the latest member of the NVIDIA Blackwell architecture family, the NVIDIA Blackwell Ultra GPU builds on core innovations to accelerate training and AI...

14 MIN READ

Aug 22, 2025

NVIDIA Hardware Innovations and Open Source Contributions Are Shaping AI

Open source AI models such as Cosmos, DeepSeek, Gemma, GPT-OSS, Llama, Nemotron, Phi, Qwen, and many more are the foundation of AI innovation. These models are...

8 MIN READ

Aug 01, 2025

Optimizing LLMs for Performance and Accuracy with Post-Training Quantization

Quantization is a core tool for developers aiming to improve inference performance with minimal overhead. It delivers significant gains in latency, throughput,...

14 MIN READ

Jul 24, 2025

Double PyTorch Inference Speed for Diffusion Models Using Torch-TensorRT

NVIDIA TensorRT is an AI inference library built to optimize machine learning models for deployment on NVIDIA GPUs. TensorRT targets dedicated hardware in...

8 MIN READ

Jul 07, 2025

LLM Inference Benchmarking: Performance Tuning with TensorRT-LLM

This is the third post in the large language model latency-throughput benchmarking series, which aims to instruct developers on how to benchmark LLM inference...

11 MIN READ

Jul 02, 2025

Optimizing FLUX.1 Kontext for Image Editing with Low-Precision Quantization

FLUX.1 Kontext, the recently released model from Black Forest Labs, is a fascinating addition to the repertoire of community image generation models. The open...

10 MIN READ

Jun 24, 2025

Introducing NVFP4 for Efficient and Accurate Low-Precision Inference

To get the most out of AI, optimizations are critical. When developers think about optimizing AI models for inference, model compression techniques—such as...

11 MIN READ

Jun 12, 2025

Run High-Performance AI Applications with NVIDIA TensorRT for RTX

NVIDIA TensorRT for RTX is now available for download as an SDK that can be integrated into C++ and Python applications for both Windows and Linux. At...

7 MIN READ