Benchmark

Nov 19, 2024

Llama 3.2 Full-Stack Optimizations Unlock High Performance on NVIDIA GPUs

Meta recently released its Llama 3.2 series of vision language models (VLMs), which come in 11B parameter and 90B parameter variants. These models are...

6 MIN READ

Nov 15, 2024

Streamlining AI Inference Performance and Deployment with NVIDIA TensorRT-LLM Chunked Prefill

In this blog post, we take a closer look at chunked prefill, a feature of NVIDIA TensorRT-LLM that increases GPU utilization and simplifies the deployment...

4 MIN READ

Nov 13, 2024

NVIDIA Blackwell Doubles LLM Training Performance in MLPerf Training v4.1

As models grow larger and are trained on more data, they become more capable, making them more useful. To train these models quickly, more performance,...

8 MIN READ

Nov 08, 2024

5x Faster Time to First Token with NVIDIA TensorRT-LLM KV Cache Early Reuse

In our previous blog post, we demonstrated how reusing the key-value (KV) cache by offloading it to CPU memory can accelerate time to first token (TTFT) by up...

5 MIN READ

Nov 06, 2024

State-of-the-Art Multimodal Generative AI Model Development with NVIDIA NeMo

Generative AI has rapidly evolved from text-based models to multimodal capabilities. These models perform tasks like image captioning and visual question...

6 MIN READ

Workflow diagram for disaggregating data into clusters and running in batches and merging.

Oct 31, 2024

Even Faster and More Scalable UMAP on the GPU with RAPIDS cuML

UMAP is a popular dimension reduction algorithm used in fields like bioinformatics, NLP topic modeling, and ML preprocessing. It works by creating a k-nearest...

12 MIN READ

Oct 28, 2024

NVIDIA GH200 Superchip Accelerates Inference by 2x in Multiturn Interactions with Llama Models

Deploying large language models (LLMs) in production environments often requires making hard trade-offs between enhancing user interactivity and increasing...

7 MIN READ

Oct 08, 2024

Mistral-NeMo-Minitron 8B Model Delivers Unparalleled Accuracy

This post was originally published August 21, 2024 but has been revised with current data. Recently, NVIDIA and Mistral AI unveiled Mistral NeMo 12B, a leading...

7 MIN READ

Nemotron icon in front of multiple tiles with icons and three sliders each, in colors of green, purple, and grey.

Oct 03, 2024

New Reward Model Helps Improve LLM Alignment with Human Preferences

Reinforcement learning from human feedback (RLHF) is essential for developing AI systems that are aligned with human values and preferences. RLHF enables the...

4 MIN READ

Sep 11, 2024

Scaling Up to One Billion Rows of Data in pandas using RAPIDS cuDF

The One Billion Row Challenge is a fun benchmark to showcase basic data processing operations. It was originally launched as a pure-Java competition, and has...

11 MIN READ

Aug 28, 2024

NVIDIA Triton Inference Server Achieves Outstanding Performance in MLPerf Inference 4.1 Benchmarks

Six years ago, we embarked on a journey to develop an AI inference serving solution specifically designed for high-throughput and time-sensitive production use...

8 MIN READ

Aug 28, 2024

NVIDIA Blackwell Platform Sets New LLM Inference Records in MLPerf Inference v4.1

Large language model (LLM) inference is a full-stack challenge. Powerful GPUs, high-bandwidth GPU-to-GPU interconnects, efficient acceleration libraries, and a...

13 MIN READ

Aug 12, 2024

NVIDIA NVLink and NVIDIA NVSwitch Supercharge Large Language Model Inference

Large language models (LLM) are getting larger, increasing the amount of compute required to process inference requests. To meet real-time latency requirements...

8 MIN READ

Jul 17, 2024

NVIDIA NeMo Accelerates LLM Innovation with Hybrid State Space Model Support

Today’s large language models (LLMs) are based on the transformer model architecture introduced in 2017. Since then, rapid advances in AI compute performance...

7 MIN READ

Jun 12, 2024

NVIDIA Sets New Generative AI Performance and Scale Records in MLPerf Training v4.0

Generative AI models have a variety of uses, such as helping write computer code, crafting stories, composing music, generating images, producing videos, and...

11 MIN READ

An illustration representing an embedding model.

Jun 10, 2024

NVIDIA Text Embedding Model Tops MTEB Leaderboard

The latest embedding model from NVIDIA—NV-Embed—set a new record for embedding accuracy with a score of 69.32 on the Massive Text Embedding Benchmark...

6 MIN READ