1. [Topics](https://developer.nvidia.com/topics/) 2. [Artificial Intelligence](https://developer.nvidia.com/topics/ai/) 3. Local AI # Local AI Build and run AI locally on NVIDIA GPUs—from RTX laptops, desktops, and workstations to DGX Spark and Station AI supercomputers—for fast, private, and unlimited AI. Powered by CUDA, NVIDIA GPUs accelerate all the top AI frameworks, open-weight models, and agents so developers can experiment, build, and deploy AI faster than ever. ## How to Build Local AI ![Decorative icon](https://developer.download.nvidia.com/icons/nvidia-generative-agentic-ai.svg) ### Build an AI Agent Build agents with agentic harnesses, MCP tool connections, along with a local AI backend. [Explore Agent Tools](#run-agents-locally) ![Decorative icon](https://developer.download.nvidia.com/icons/neural-network.svg) ### Use Optimized Models Run NVIDIA-optimized open-weight models locally with day-0 support, NVFP4 quantization, and tuned Tensor Core performance. [Browse Models](#optimized-models) ![Decorative icon](https://developer.download.nvidia.com/icons/model-customization.svg) ### Run a Model Efficiently Run open-weight models with accelerated runtimes such as Ollama, llama.cpp, TensorRT, SGLang, vLLM, WindowsML, or PyTorch with CUDA. [See Frameworks](#inference-backends) ![Decorative icon](https://developer.download.nvidia.com/icons/building-agentic-ai-application-with-large-language-model.svg) ### Add AI to Applications Add 3D, generative AI, and video features with NVIDIA-accelerated SDKs. [Explore SDKs](#nvidia-ai-sdks) ![Decorative icon](https://developer.download.nvidia.com/icons/m48-accellerated-computing-with-cuda-python-256px-blk.png) ### Optimize and Deploy Your Application Use 400+ CUDA-X™ libraries to get the most from your GPU and NVIDIA Nsight tools to build, profile, and optimize GPU-accelerated applications. [Explore CUDA-X](#build-ai-applications) ### Get Local NVIDIA Hardware Run AI locally, whether it's from your primary system, companion machine, or deskside supercomputer. [See Hardware](#nvidia-gpu) ## Popular Local AI Quickstart Guides ![NVIDIA DGX Spark workstation and RTX system setup for local AI agent development.](https://developer.download.nvidia.com/images/local-ai/nvidia-claw-dgx-spark-ari.jpg) ### Build Local Agents - [NemoClaw Guide](https://docs.nvidia.com/nemoclaw/user-guide/openclaw/home) - [Hermes Agent Guide](https://build.nvidia.com/spark/hermes-agent/overview) ![Tiny Jensen with claws.](https://developer.download.nvidia.com/images/local-ai/llm-performance-ari.jpeg) ### Run LLMs Locally - [Llama.cpp Guide](https://build.nvidia.com/spark/llama-cpp) - [Nemotron Guide](https://build.nvidia.com/spark/nemotron) - [vLLM Recipes](https://recipes.vllm.ai/nvidia) ![Animation demonstrating the iterative image generation process using local diffusion models.](https://developer.download.nvidia.com/images/local-ai/diffusion-model-building.gif) ### Run Creative AI Models - [ComfyUI Guide](https://build.nvidia.com/spark/comfy-ui/overview) ![Diagram showing model quantization workflow reducing precision from FP16 to NVFP4 using NVIDIA TensorRT Model Optimizer for local GPU deployment](https://developer.download.nvidia.com/images/local-ai/nvidia-model-quantization-ari.jpg) ### Model Quantization - [Concepts, Methods, and Why It Matters](https://developer.nvidia.com/blog/model-quantization-concepts-methods-and-why-it-matters/) - [Post-Training Quantization](https://developer.nvidia.com/blog/model-quantization-post-training-quantization-using-nvidia-model-optimizer/) - [Turn FP8 Checkpoints into High-Performance Inference Engines](https://developer.nvidia.com/blog/model-quantization-turn-fp8-checkpoints-into-high-performance-inference-engines-with-nvidia-tensorrt/) ![NVIDIA DGX Spark workstation running optimized model fine-tuning with Unsloth on a local open-weight language model](https://developer.download.nvidia.com/images/local-ai/nvidia-llm-visual-finetuning-ari.jpg) ### Fine-Tuning a Model Locally - [Optimized Fine-Tuning with Unsloth on DGX Spark](https://build.nvidia.com/spark/unsloth) ## Benefits ## Why Build and Run Local AI With NVIDIA GPUs? ![Decorative image](https://developer.download.nvidia.com/icons/m48-neural-network-1.svg) #### Broadest Ecosystem Every layer of the AI stack is accelerated on NVIDIA - CUDA libraries, open-source frameworks, SDKs, and models - with the broadest community and ISV support. ![Decorative image](https://developer.download.nvidia.com/icons/m48-monitor-performance.svg) #### Easy-to-Adopt Tools NVIDIA and the open-source community partner closely to optimize popular AI tools like ComfyUI, Hermes, llama.cpp, Ollama, ONNX, OpenClaw, PyTorch, vLLM and more. These tools can be installed easily and run on the same CUDA stack across NVIDIA local and cloud GPUs. ![Decorative image](https://developer.download.nvidia.com/icons/m48-microservices-2.svg) #### Day 0 Support for Latest Models From LLMs to image generators, new open source models run fast on NVIDIA GPUs from day 0 due to close collaboration between NVIDIA and model labs. ![Decorative image](https://developer.download.nvidia.com/icons/m48-people-growth.svg) #### Largest Installed Base and Unified Development Stack Over 100M local NVIDIA GPU installed base. Develop once and deploy across NVIDIA GPUs in PC and cloud. ![Decorative image](https://developer.download.nvidia.com/icons/m48-speed.svg) #### Performance Dedicated Tensor Cores accelerate AI workloads alongside continuously optimized frameworks and NVFP4 checkpoints. ## Build and Run Agents Locally Build agents that interact with applications, generate content, automate personal workflows and manage multi-step reasoning tasks, all while running privately on device either locally or hybrid across both cloud and local configurations. ![NVIDIA RTX Spark compact AI workstation for creators and developers, supporting 90 GB+ 3D scenes, 12K video editing, and 120B-parameter local LLM infer](https://developer.download.nvidia.com/images/local-ai/nemotron-labs-openclaw-1920x1080-1-1280x720.jpg) ### NVIDIA OpenShell NVIDIA OpenShell is a safe runtime for AI agents. It provides sandboxed execution environments and policy enforcement to govern agent behavior. [Learn About OpenShell](https://build.nvidia.com/openshell) ![NVIDIA DGX Spark personal AI supercomputer showing front, rear, and side panel views of the GB10 Grace Blackwell Superchip compact deskside system](https://developer.download.nvidia.com/images/local-ai/NemoClaw-sfg.jpg) ### NVIDIA NemoClaw NVIDIA NemoClaw™ is a collection of open blueprints for building autonomous agents: domain-specialized, always-on AI systems that reason, plan, and act across real-world workflows [Learn About NemoClaw](https://www.nvidia.com/en-us/ai/nemoclaw/) ### Hermes Agent Hermes Agent is a self-improving agent built by Nous Research. It runs as a terminal or app interface, has persistent skills and memory that help it improve over time, works well with local models and can connect to channels like Telegram and Slack. [Learn About Hermes](https://build.nvidia.com/spark/hermes-agent) ### OpenCode Use an open-source coding agent with a supported local model provider for code-aware development workflows. [Learn About OpenCode](https://opencode.ai/download) ### OpenClaw Run OpenClaw autonomous agents with NVIDIA NemoClaw to move from prototype to safer, governed deployment. NemoClaw adds OpenShell policy controls, lifecycle management, and sandboxing. [Learn About OpenClaw](https://docs.openclaw.ai/providers/nvidia) * * * ## Optimized Models and Checkpoints for Local AI NVIDIA and the community optimize every open-source model for local NVIDIA GPUs. #### Choose the Best Model for Your Local AI Use-Case 1. Determine the target VRAM and performance requirements​ 2. Shortlist:​ - Filter candidates based on public benchmarks (ex. Artificial Analysis Index)​ - If you are using llama.cpp: prefer Q4\_K\_M quantized checkpoints for best performance​ while retaining quality - If you are using vLLM or PyTorch: Prefer NVFP4 quantized checkpoints for best performance​ while retaining quality 3. Evaluate: - Build a Custom Evaluation Dataset for Your Use-Case​ - Grade the model with Human Evaluation​ - For LLMs, grade the model using LLM-as-a-judge to scale​ [![Decision flowchart for selecting a local AI model: VRAM requirements, NVFP4 vs. Q4_K_M quantization selection, benchmark filtering, and LLM-as-judge evaluation steps](https://developer.download.nvidia.com/images/local-ai/nvidia-recommended-models-table-ari.jpg)](https://developer.download.nvidia.com/images/local-ai/nvidia-recommended-models-table-ari.jpg) (Click diagram to enlarge) [![VRAM comparison for NVFP4 vs. BF16 over FLUX.1, FLUX.2, and Qwen Image. Lower VRAM is better.](https://developer.download.nvidia.com/images/local-ai/nvidia-diy-model-checkpoint-optimization-chart.jpg)](https://developer.download.nvidia.com/images/local-ai/nvidia-diy-model-checkpoint-optimization-chart.jpg) (Click diagram to enlarge) #### DIY Model Checkpoint Optimization NVIDIA TensorRT Model Optimizer is a unified library of SOTA model optimization techniques like quantization, pruning, distillation, and speculative decoding. It compresses deep learning models for downstream deployment frameworks, optimizing inference speed. [Get Model Optimizer](https://github.com/NVIDIA/TensorRT-Model-Optimizer) * * * ## Inference Backends for Local AI Choose an inference backend based on your operating system, model format, GPU architecture and memory, API requirements, and throughput target. [Access Inference Backends](/ai-apps-for-rtx-pcs/inference-backends) ![Comparison table of local AI inference backends — PyTorch, Ollama, llama.cpp, TensorRT-LLM, SGLang, vLLM, and WindowsML — by OS, GPU architecture, API type, and throughput target](https://developer.download.nvidia.com/images/local-ai/Table.png) * * * ## NVIDIA AI SDKs Add cutting-edge AI features to your RTX PC application with NVIDIA SDKs across 3D graphics, generative AI, and video and broadcast. 3D and Graphic Design Generative AI for Games Video and Broadcast ![Multiply performance with NVIDIA DLSS](https://developer.download.nvidia.com/images/local-ai/nvidia-dlss4-ari.jpg) ### NVIDIA DLSS Neural graphics to multiply performance using AI. Generate new frames, reconstruct images, and improve ray-traced content. [Learn More About DLSS](/rtx/dlss) ![Physical AI simulation with NVIDIA Omniverse](https://developer.download.nvidia.com/images/local-ai/nvidia-omniverse-ari.jpg) ### NVIDIA Omniverse Accelerated libraries, microservices, and skills for physical AI simulation and agentic workflows. [Learn More About Omniverse](/omniverse) ![Optimal ray-tracing with NVIDIA Optix](https://developer.download.nvidia.com/images/local-ai/optix-8-ari.jpg) ### NVIDIA OptiX Application framework for optimal ray-tracing performance on the GPU. [Learn More About OptiX](/rtx/ray-tracing/optix) ![Neural rendering with NVIDIA RTX Kit](https://developer.download.nvidia.com/images/local-ai/nvidia-rtx-kit-zorah-ari.jpg) ### NVIDIA RTX Kit Suite of neural rendering technologies.. [Learn More About RTX Kit](/rtx-kit) ![NVIDIA ACE digital human technologies suite showing AI-driven character animation](https://developer.download.nvidia.com/images/local-ai/nvidia-ace-for-games-hiroki1-ari.jpg) ### NVIDIA ACE Suite of digital human technologies for games. [Learn More About ACE](/ace) ![NVIDIA AI4M interface showing AI-enhanced audio, video, and AR effects for video conferencing and telepresence applications on RTX hardware](https://developer.download.nvidia.com/images/local-ai/nvidia-ai-for-media-and-entertainment-ari.jpg) ### NVIDIA AI for Media AI-enhanced audio, video, and AR effects for video conferencing and telepresence. [Learn More About AI for Media](/maxine) ![NVIDIA RTX Video AI Super Resolution and HDR enhancement interface showing upscaled video playback on an RTX-equipped display](https://developer.download.nvidia.com/images/local-ai/nvidia-rtx-video-sdk-clarity-ari.jpg) ### RTX Video AI-enhanced Super Resolution and HDR effects for creative and media playback apps. [Learn More About RTX Video](/rtx-video-sdk) ![NVIDIA Video Codec SDK architecture diagram showing hardware-accelerated encode and decode APIs for NVIDIA GPU-based video processing pipelines](https://developer.download.nvidia.com/images/local-ai/nvidia-video-codec-sdk-ari.jpg) ### Video Codec SDK Comprehensive APIs, tools, and samples for hardware-accelerated video encode and decode on NVIDIA GPUs across popular codecs. [Learn More About Video Codec SDK](/video-codec-sdk) * * * ## Developer Tools to Build Accelerated AI Applications Use CUDA-X libraries and NVIDIA Nsight tools to build, profile, debug, and optimize GPU-accelerated AI applications. ![NVIDIA CUDA-X library ecosystem showing 400+ GPU-accelerated computing libraries for AI inference and high-performance computing](https://developer.download.nvidia.com/images/local-ai/nvidia-cuda-x-libraries-ari.jpg) #### CUDA-X Libraries NVIDIA CUDA-X is a powerful suite of libraries designed to deliver industry-leading GPU acceleration across AI and high-performance computing use cases. They provide highly optimized implementations of complex algorithms that far outperform CPU-only alternatives. Also, now available as [portable instruction sets and skills](https://github.com/nvidia/skills) that extend the capabilities of AI Agents. [Learn More About CUDA-X Libraries](/cuda/cuda-x-libraries) #### NVIDIA Nsight Tools NVIDIA Nsight™ tools are a powerful set of libraries, SDKs, and developer tools to build, debug, profile, and develop software that utilizes the latest accelerated computing hardware. [Get Started With Nsight Developer Tools](/tools-overview) [![NVIDIA Nsight developer tools interface displaying GPU performance profiling data, kernel timelines, and memory utilization for a local AI application](https://developer.download.nvidia.com/images/local-ai/Nsight-Tools.png)](https://developer.download.nvidia.com/images/local-ai/Nsight-Tools.png) (Click diagram to enlarge) * * * ## Which NVIDIA GPU Should I Use for Local AI? Choose hardware based on operating system, available GPU or unified memory, model size, and workflow. Prototype on NVIDIA DGX Spark™, run AI and agent workflows on GeForce RTX™, NVIDIA RTX Spark™ or NVIDIA RTX PRO™, and scale to NVIDIA DGX Station™ for larger local models and long-running agents. | | Best For | System Role | Memory | OS | Form Factor | Model Capacity | | --- | --- | --- | --- | --- | --- | --- | | [NVIDIA RTX Spark](https://www.nvidia.com/en-us/products/rtx-spark/) | Prototype & Test Large AI Models and Agents | Single Lightweight System | Up to 128GB Unified | Windows | Laptop / Compact Desktop | Up to 200 B | | [NVIDIA DGX Spark](https://www.nvidia.com/en-us/products/workstations/dgx-spark/) | Prototype & Test Large AI Models and Agents | Companion System | Up to 128GB Unified | Linux | Small Desktop | Up to 200 B per DGX Spark | | [NVIDIA GeForce RTX](https://www.nvidia.com/en-us/ai-on-rtx/) | Develop and Test Small AI Models | Single Primary System | 6 - 32GB VRAM | Linux / Windows | Laptop / Desktop | Up to 60 B | | [NVIDIA RTX PRO](https://www.nvidia.com/en-us/products/workstations/) | Develop and Test Large AI Models | Single Primary System | 16 - 96 GB VRAM | Linux / Windows | Laptop / Desktop | Up to 150 B | | NVIDIA DGX Station ([Windows](https://www.nvidia.com/en-us/products/workstations/dgx-station-for-windows/) | [Linux](https://www.nvidia.com/en-us/products/workstations/dgx-station/)) | AI Development at Max Perf & Memory for Multi-User, Long-Running Agents | On-Prem Deskside System | 748 GB Unified Coherent | Linux / Windows | Deskside System | Up to 1 T | ![NVIDIA RTX Spark compact AI workstation for creators and developers, supporting 90 GB+ 3D scenes, 12K video editing, and 120B-parameter local LLM infer](https://developer.download.nvidia.com/images/local-ai/nvidia-rtx-spark-ari.jpg) ### RTX Spark RTX Spark lets creators and AI developers render ultralarge 90GB+ 3D scenes, edit 12K 4:2:2 video, generate 4K AI videos, run 120B-parameter LLMs with up to 1 million tokens context using agents locally, and play AAA games at 1440p and over 100 frames per second. [Learn About RTX Spark](https://www.nvidia.com/en-us/products/rtx-spark/) ![NVIDIA DGX Spark personal AI supercomputer showing front, rear, and side panel views of the GB10 Grace Blackwell Superchip compact deskside system](https://developer.download.nvidia.com/images/local-ai/nvidia-dgx-spark-panels-ari.jpg) ### DGX Spark Personal AI supercomputer powered by the GB10 Grace Blackwell Superchip. Up to 1 petaFLOP at FP4, 128GB unified memory, and DGX OS preinstalled—fine-tune up to 70B parameters and run inference up to 200B parameters from your desk. [Explore DGX Spark](https://www.nvidia.com/en-us/products/workstations/dgx-spark/) ![NVIDIA DGX Station GB300 Grace Blackwell Ultra desktop supercomputer supporting AI models up to 1 trillion parameters with 748 GB coherent memory and 20 petaFLOPS FP4 compute](https://developer.download.nvidia.com/images/local-ai/workstation-dgx-station-1920x1080.jpg) ### DGX Station Powered by the NVIDIA GB300 Grace Blackwell Ultra Desktop Superchip with up to 748 GB of coherent memory and 20 petaFLOPS of FP4 AI compute, NVIDIA DGX Station delivers data-center-level performance, supporting large AI models up to 1 trillion parameters, locally. [Explore DGX Station for Windows](https://www.nvidia.com/en-us/products/workstations/dgx-station-for-windows/)[Explore DGX Station (Linux)](https://www.nvidia.com/en-us/products/workstations/dgx-station/) ![NVIDIA GeForce RTX 5090 graphics card for AI-accelerated gaming, creative workloads, and local AI development on a desktop PC](https://developer.download.nvidia.com/images/local-ai/geforce-rtx-5090-learn-more-nv-sfg-thumbnail-1920x1080.jpg) ### GeForce RTX Upgrade to advanced AI with RTX AI PCs and accelerate gaming, creating, productivity, and development. [Explore GeForce RTX](https://www.nvidia.com/en-us/ai-on-rtx/) ![NVIDIA RTX PRO Blackwell professional GPU workstation for AI inference, graphics rendering, and high-performance compute workloads](https://developer.download.nvidia.com/images/local-ai/rtx-professional-workstations-ari.jpg) ### RTX PRO Accelerate professional AI, graphics, rendering, and compute workloads with NVIDIA RTX PRO Blackwell workstations. [Explore RTX PRO](https://www.nvidia.com/en-us/products/workstations/) * * * ## Latest Local AI News From NVIDIA ![NVIDIA Accelerates Google DeepMind’s DiffusionGemma for Local AI](https://developer.download.nvidia.com/images/local-ai/diffusiongemma-nv-blog-1280x680-1.jpg) June 10, 2026 ### NVIDIA Accelerates Google DeepMind’s DiffusionGemma for Local AI Today, Google DeepMind released DiffusionGemma — an experimental open model built for exceptionally fast text generation. NVIDIA has optimized DiffusionGemma to run even faster across NVIDIA GeForce RTX GPUs, the NVIDIA RTX PRO platform and NVIDIA DGX Spark systems, from local PCs to the cloud. [Read the Blog](https://blogs.nvidia.com/blog/rtx-ai-garage-local-gemma-diffusion/) ![NVIDIA Levels Up Local AI Agents Across RTX PCs and DGX Spark](https://developer.download.nvidia.com/images/local-ai/computex-2026-nv-blog-1280x680-1.jpg) May 31, 2026 ### NVIDIA Levels Up Local AI Agents Across RTX PCs and DGX Spark Personal agents are exploding in popularity, with open source projects like OpenClaw and Hermes seeing rapid adoption by AI developer communities on GitHub. Built to adapt to individual preferences and workflows, these agents can interact with applications, generate content, automate repetitive processes and manage multi-step tasks — all while running locally on device. [Read the Blog](https://blogs.nvidia.com/blog/rtx-ai-garage-computex-spark-local-agents/) ![Hermes Unlocks Self-Improving AI Agents, Powered by NVIDIA RTX PCs and DGX Spark](https://developer.download.nvidia.com/images/local-ai/rtx-ai-garage-hermes.jpeg) May 13, 2026 ### Hermes Unlocks Self-Improving AI Agents, Powered by NVIDIA RTX PCs and DGX Spark Agentic AI is changing the way users get work done. Following the success of OpenClaw, the community is embracing new open source agentic frameworks. The latest is Hermes Agent, which crossed 140,000 GitHub stars in under three months and, as of last week, is the most used agent in the world according to OpenRouter. [Read the Blog](https://blogs.nvidia.com/blog/rtx-ai-garage-hermes-agent-dgx-spark/) [See AI on RTX Blogs](https://www.nvidia.com/en-us/ai-on-rtx/blog/) * * * ## More Local AI Developer Resources ![decorative icon](https://developer.download.nvidia.com/icons/m48-people-group.svg) ### Developer Forums ![decorative icon](https://developer.download.nvidia.com/icons/m48-document-support-guide-2.svg) ### Guides and Playbooks ![decorative icon](https://developer.download.nvidia.com/icons/discord-icon-logo.svg) ### Join our Developer Discord Server **Stay up to date on the latest NVIDIA developer news.** [Subscribe](/email-signup)