Popular Local AI Quickstart Guides
How to Build Local AI
Build an AI Agent
Build agents with agentic harnesses, MCP tool connections, along with a local AI backend.
Explore Agent Tools
Use Optimized Models
Run NVIDIA-optimized open-weight models locally with day-0 support, NVFP4 quantization, and tuned Tensor Core performance.
Browse Models
Run a Model Efficiently
Run open-weight models with accelerated runtimes such as Ollama, llama.cpp, TensorRT, SGLang, vLLM, WindowsML, or PyTorch with CUDA.
See Frameworks
Add AI to Applications
Add 3D, generative AI, and video features with NVIDIA-accelerated SDKs.
Explore SDKs
Optimize and Deploy Your Application
Use 400+ CUDA-X™ libraries to get the most from your GPU and NVIDIA Nsight tools to build, profile, and optimize GPU-accelerated applications.
Explore CUDA-XGet Local NVIDIA Hardware
Run AI locally, whether it's from your primary system, companion machine, or deskside supercomputer.
See Hardware
Benefits
Why Build and Run Local AI With NVIDIA GPUs?
Broadest Ecosystem
Every layer of the AI stack is accelerated on NVIDIA - CUDA libraries, open-source frameworks, SDKs, and models - with the broadest community and ISV support.
Easy-to-Adopt Tools
NVIDIA and the open-source community partner closely to optimize popular AI tools like ComfyUI, Hermes, llama.cpp, Ollama, ONNX, OpenClaw, PyTorch, vLLM and more. These tools can be installed easily and run on the same CUDA stack across NVIDIA local and cloud GPUs.
Day 0 Support for Latest Models
From LLMs to image generators, new open source models run fast on NVIDIA GPUs from day 0 due to close collaboration between NVIDIA and model labs.
Largest Installed Base and Unified Development Stack
Over 100M local NVIDIA GPU installed base. Develop once and deploy across NVIDIA GPUs in PC and cloud.
Performance
Dedicated Tensor Cores accelerate AI workloads alongside continuously optimized frameworks and NVFP4 checkpoints.
Build and Run Agents Locally
Build agents that interact with applications, generate content, automate personal workflows and manage multi-step reasoning tasks, all while running privately on device either locally or hybrid across both cloud and local configurations.
NVIDIA OpenShell
NVIDIA OpenShell is a safe runtime for AI agents. It provides sandboxed execution environments and policy enforcement to govern agent behavior.
NVIDIA NemoClaw
NVIDIA NemoClaw™ is a collection of open blueprints for building autonomous agents: domain-specialized, always-on AI systems that reason, plan, and act across real-world workflows
Hermes Agent
Hermes Agent is a self-improving agent built by Nous Research. It runs as a terminal or app interface, has persistent skills and memory that help it improve over time, works well with local models and can connect to channels like Telegram and Slack.
OpenCode
Use an open-source coding agent with a supported local model provider for code-aware development workflows.
OpenClaw
Run OpenClaw autonomous agents with NVIDIA NemoClaw to move from prototype to safer, governed deployment. NemoClaw adds OpenShell policy controls, lifecycle management, and sandboxing.
Optimized Models and Checkpoints for Local AI
NVIDIA and the community optimize every open-source model for local NVIDIA GPUs.
Choose the Best Model for Your Local AI Use-Case
1. Determine the target VRAM and performance requirements
2. Shortlist:
Filter candidates based on public benchmarks (ex. Artificial Analysis Index)
If you are using llama.cpp: prefer Q4_K_M quantized checkpoints for best performance while retaining quality
If you are using vLLM or PyTorch: Prefer NVFP4 quantized checkpoints for best performance while retaining quality
3. Evaluate:
Build a Custom Evaluation Dataset for Your Use-Case
Grade the model with Human Evaluation
For LLMs, grade the model using LLM-as-a-judge to scale
(Click diagram to enlarge)
(Click diagram to enlarge)
DIY Model Checkpoint Optimization
NVIDIA TensorRT Model Optimizer is a unified library of SOTA model optimization techniques like quantization, pruning, distillation, and speculative decoding. It compresses deep learning models for downstream deployment frameworks, optimizing inference speed.
Get Model OptimizerInference Backends for Local AI
Choose an inference backend based on your operating system, model format, GPU architecture and memory, API requirements, and throughput target.
NVIDIA AI SDKs
Add cutting-edge AI features to your RTX PC application with NVIDIA SDKs across 3D graphics, generative AI, and video and broadcast.
NVIDIA DLSS
Neural graphics to multiply performance using AI. Generate new frames, reconstruct images, and improve ray-traced content.
Learn More About DLSS
NVIDIA Omniverse
Accelerated libraries, microservices, and skills for physical AI simulation and agentic workflows.
Learn More About Omniverse
NVIDIA OptiX
Application framework for optimal ray-tracing performance on the GPU.
Learn More About OptiX
NVIDIA AI for Media
AI-enhanced audio, video, and AR effects for video conferencing and telepresence.
Learn More About AI for Media
RTX Video
AI-enhanced Super Resolution and HDR effects for creative and media playback apps.
Learn More About RTX Video
Video Codec SDK
Comprehensive APIs, tools, and samples for hardware-accelerated video encode and decode on NVIDIA GPUs across popular codecs.
Learn More About Video Codec SDKDeveloper Tools to Build Accelerated AI Applications
Use CUDA-X libraries and NVIDIA Nsight tools to build, profile, debug, and optimize GPU-accelerated AI applications.
CUDA-X Libraries
NVIDIA CUDA-X is a powerful suite of libraries designed to deliver industry-leading GPU acceleration across AI and high-performance computing use cases. They provide highly optimized implementations of complex algorithms that far outperform CPU-only alternatives. Also, now available as portable instruction sets and skills that extend the capabilities of AI Agents.
Learn More About CUDA-X LibrariesNVIDIA Nsight Tools
NVIDIA Nsight™ tools are a powerful set of libraries, SDKs, and developer tools to build, debug, profile, and develop software that utilizes the latest accelerated computing hardware.
Get Started With Nsight Developer Tools
(Click diagram to enlarge)
Which NVIDIA GPU Should I Use for Local AI?
Choose hardware based on operating system, available GPU or unified memory, model size, and workflow. Prototype on NVIDIA DGX Spark™, run AI and agent workflows on GeForce RTX™, NVIDIA RTX Spark™ or NVIDIA RTX PRO™, and scale to NVIDIA DGX Station™ for larger local models and long-running agents.
Best For |
System Role |
Memory |
OS |
Form Factor |
Model Capacity |
|
|---|---|---|---|---|---|---|
Prototype & Test Large AI Models and Agents |
Single Lightweight System |
Up to 128GB Unified |
Windows |
Laptop / Compact Desktop |
Up to 200 B |
|
Prototype & Test Large AI Models and Agents |
Companion System |
Up to 128GB Unified |
Linux |
Small Desktop |
Up to 200 B per DGX Spark |
|
Develop and Test Small AI Models |
Single Primary System |
6 - 32GB VRAM |
Linux / Windows |
Laptop / Desktop |
Up to 60 B |
|
Develop and Test Large AI Models |
Single Primary System |
16 - 96 GB VRAM |
Linux / Windows |
Laptop / Desktop |
Up to 150 B |
|
AI Development at Max Perf & Memory for Multi-User, Long-Running Agents |
On-Prem Deskside System |
748 GB Unified Coherent |
Linux / Windows |
Deskside System |
Up to 1 T |
RTX Spark
RTX Spark lets creators and AI developers render ultralarge 90GB+ 3D scenes, edit 12K 4:2:2 video, generate 4K AI videos, run 120B-parameter LLMs with up to 1 million tokens context using agents locally, and play AAA games at 1440p and over 100 frames per second.
DGX Spark
Personal AI supercomputer powered by the GB10 Grace Blackwell Superchip. Up to 1 petaFLOP at FP4, 128GB unified memory, and DGX OS preinstalled—fine-tune up to 70B parameters and run inference up to 200B parameters from your desk.
DGX Station
Powered by the NVIDIA GB300 Grace Blackwell Ultra Desktop Superchip with up to 748 GB of coherent memory and 20 petaFLOPS of FP4 AI compute, NVIDIA DGX Station delivers data-center-level performance, supporting large AI models up to 1 trillion parameters, locally.
GeForce RTX
Upgrade to advanced AI with RTX AI PCs and accelerate gaming, creating, productivity, and development.
RTX PRO
Accelerate professional AI, graphics, rendering, and compute workloads with NVIDIA RTX PRO Blackwell workstations.
Latest Local AI News From NVIDIA
June 10, 2026
NVIDIA Accelerates Google DeepMind’s DiffusionGemma for Local AI
Today, Google DeepMind released DiffusionGemma — an experimental open model built for exceptionally fast text generation. NVIDIA has optimized DiffusionGemma to run even faster across NVIDIA GeForce RTX GPUs, the NVIDIA RTX PRO platform and NVIDIA DGX Spark systems, from local PCs to the cloud.
May 31, 2026
NVIDIA Levels Up Local AI Agents Across RTX PCs and DGX Spark
Personal agents are exploding in popularity, with open source projects like OpenClaw and Hermes seeing rapid adoption by AI developer communities on GitHub. Built to adapt to individual preferences and workflows, these agents can interact with applications, generate content, automate repetitive processes and manage multi-step tasks — all while running locally on device.
May 13, 2026
Hermes Unlocks Self-Improving AI Agents, Powered by NVIDIA RTX PCs and DGX Spark
Agentic AI is changing the way users get work done. Following the success of OpenClaw, the community is embracing new open source agentic frameworks. The latest is Hermes Agent, which crossed 140,000 GitHub stars in under three months and, as of last week, is the most used agent in the world according to OpenRouter.
Stay up to date on the latest NVIDIA developer news.