Run DiffusionGemma on NVIDIA for Developer-Ready, High-Throughput Text Generation

Developers building real-time AI—such as chat assistants, copilots, and agentic workflows—are often constrained by token-by-token generation speed. This limits responsiveness, increases serving costs, and makes fluid, interactive experiences difficult to achieve.  

DiffusionGemma, created by Google DeepMind and optimized to run efficiently across NVIDIA platforms, introduces a new approach to text generation, producing tokens in parallel rather than one at a time, enabling faster, higher-throughput AI applications. The model uses diffusion-based denoising to generate 256 tokens in parallel per step, delivering up to 1,000 tokens/sec on a single NVIDIA H100 Tensor Core GPU, up to 150 tokens/sec on NVIDIA DGX Spark, and the fastest local performance on NVIDIA DGX Station. 

For enterprise developers, this speed translates into lower serving costs, higher concurrency, and more responsive user experiences without sacrificing model quality. DiffusionGemma is built on the Gemma 4 26B A4B MoE architecture and optimized for low-latency, memory-bound inference. 

In addition to NVIDIA data center GPUs, developers can enjoy optimal performance on a variety of client GPUs and systems. 

Build and prototype on NVIDIA 

Access DiffusionGemma through Hugging Face Transformers for initial testing and prototyping on NVIDIA GeForce RTX 5090 or DGX Spark. For higher throughput or concurrent multi-user serving on DGX Spark, DGX Station, and RTX PRO, use vLLM by following our playbooks in Table 2.  

With Day 0 support across NVIDIA hardware and software—from local prototyping to production deployment—developers can quickly move from experimentation to real-world applications. 
 
NVIDIA GPU-accelerated endpoints 

Start building with DiffusionGemma with free access for prototyping to GPU-accelerated endpoints on build.nvidia.com as part of the NVIDIA Developer Program. The browser experience can also be connected to custom data sources.

BF16 and NVFP4

The model is available today on Hugging Face with BF16 checkpoints, and an NVFP4 quantized checkpoint for DiffusionGemma is also available using NVIDIA Model Optimizer.  

Enterprise deployments with NVIDIA NIM 

NVIDIA NIM makes it simple to deploy DiffusionGemma from development into production. NIM packages the model as an optimized, containerized inference microservice — with performance tuning, standardized APIs, and the flexibility to run on-premises, in the cloud, or across hybrid environments. NIM exposes a standard OpenAI-compatible API for sending inference requests to the server. 

1. Download the container. 
2. Start the NIM server. 

$ export NIM_IMAGE_PATH = “nvcr.io/nim/google/diffusiongemma-26b-a4b-it:latest”
$ docker run --gpus=all \ 
  -e NGC_API_KEY=$NGC_API_KEY \ 
  -v "$LOCAL_NIM_CACHE:/opt/nim/.cache" \ 
  -p 8000:8000 \ 
 ${NIM_IMAGE_PATH}

3. Make a test request and read the full NIM documentation. 

from openai import OpenAI 
client = OpenAI( 
    base_url="http://localhost:8000/v1", 
    api_key="not-required" 
) 
response = client.chat.completions.create( 
    model="google/diffusiongemma-26b-a4b-it”,
    messages=[ 
        {"role": "user", "content": "Write a poem about text diffusion"} 
    ], 
    max_tokens=256 
) 
print(response.choices[0].message.content) 

Adapt to specific use cases

Fine-tuning is available through the  NVIDIA NeMo Framework for developers looking to adapt the model to specific tasks or domains. 

NVIDIA is an active contributor to the open-source ecosystem and has released several hundred projects under open-source licenses. NVIDIA is committed to open models such as DiffusionGemma that promote AI transparency and enable users to share their work in AI safety and resilience.  

Check out DiffusionGemma on Hugging Face or test for free using NVIDIA APIs at build.nvidia.com.