Training AI Models

Image shows cloud-based GPU clusters dedicated to AI training.
Mar 10, 2025

Ensuring Reliable Model Training on NVIDIA DGX Cloud

Training AI models on massive GPU clusters presents significant challenges for model builders. Because manual intervention becomes impractical as job scale...
8 MIN READ
Stack diagram for LLM Megatron Core.
Feb 05, 2025

OpenAI Triton on NVIDIA Blackwell Boosts AI Performance and Programmability

Matrix multiplication and attention mechanisms are the computational backbone of modern AI workloads. While libraries like NVIDIA cuDNN provide highly optimized...
5 MIN READ
Nov 22, 2024

Hymba Hybrid-Head Architecture Boosts Small Language Model Performance

Transformers, with their attention-based architecture, have become the dominant choice for language models (LMs) due to their strong performance,...
12 MIN READ
Nov 13, 2024

NVIDIA Blackwell Doubles LLM Training Performance in MLPerf Training v4.1

As models grow larger and are trained on more data, they become more capable, making them more useful. To train these models quickly, more performance,...
8 MIN READ
Aug 15, 2024

NVIDIA TensorRT Model Optimizer v0.15 Boosts Inference Performance and Expands Model Support

NVIDIA has announced the latest v0.15 release of NVIDIA TensorRT Model Optimizer, a state-of-the-art quantization toolkit of model optimization techniques...
5 MIN READ
Illustration showing models and NeMo.
Jul 17, 2024

NVIDIA NeMo Accelerates LLM Innovation with Hybrid State Space Model Support

Today’s large language models (LLMs) are based on the transformer model architecture introduced in 2017. Since then, rapid advances in AI compute performance...
7 MIN READ
A GIF showing the creation of a building image with diffusion models.
Jul 10, 2024

Understanding Diffusion Models: An Essential Guide for AEC Professionals

Generative AI, the ability of algorithms to process various types of inputs—such as text, images, audio, video, and code—and generate new content, is...
13 MIN READ
Decorative image of rows of GPUs.
Jun 12, 2024

NVIDIA Sets New Generative AI Performance and Scale Records in MLPerf Training v4.0

Generative AI models have a variety of uses, such as helping write computer code, crafting stories, composing music, generating images, producing videos, and...
11 MIN READ
Decorative image of VILA and Jetson Orin workflow.
May 03, 2024

Visual Language Intelligence and Edge AI 2.0 with NVIDIA Cosmos Nemotron

Note: As of January 6, 2025, VILA is now part of the Cosmos Nemotron VLM family. NVIDIA is proud to announce the release of NVIDIA Cosmos Nemotron, a family of...
8 MIN READ
Decorative image of different workflows against a grey background.
Apr 23, 2024

Democratizing AI Workflows with Union.ai and NVIDIA DGX Cloud

GPUs were initially specialized for rendering 3D graphics in video games, primarily to accelerate linear algebra calculations. Today, GPUs have become one of...
7 MIN READ
Mar 21, 2024

Rethinking How to Train Diffusion Models

After exploring the fundamentals of diffusion model sampling, parameterization, and training as explained in Generative AI Research Spotlight: Demystifying...
15 MIN READ
Dec 14, 2023

Generative AI Research Spotlight: Demystifying Diffusion-Based Models

With Internet-scale data, the computational demands of AI-generated content have grown significantly, with data centers running full steam for weeks or months...
26 MIN READ
Nov 28, 2023

One Giant Superchip for LLMs, Recommenders, and GNNs: Introducing NVIDIA GH200 NVL32

At AWS re:Invent 2023, AWS and NVIDIA announced that AWS will be the first cloud provider to offer NVIDIA GH200 Grace Hopper Superchips interconnected with...
9 MIN READ
Nov 16, 2023

Mastering LLM Techniques: Training 

Large language models (LLMs) are a class of generative AI models built using transformer networks that can recognize, summarize, translate, predict, and...
15 MIN READ