NVIDIA NeMo Agent Toolkit
NVIDIA NeMo™ Agent Toolkit is an open-source AI library that adds intelligence to AI agents across any framework—enhancing speed, accuracy, and decision-making through enterprise-grade instrumentation, observability, and continuous learning. By exposing hidden bottlenecks and costs and optimizing the workflow, it helps enterprises scale agentic systems efficiently while maintaining reliability.
NeMo Agent Toolkit is part of the broader NVIDIA Agent Toolkit, a collection of tools, models and runtimes for building, evaluating and optimizing safe, long-running autonomous agents.
See NeMo Agent Toolkit in Action
How NeMo Agent Toolkit Works
NVIDIA NeMo Agent Toolkit provides unified monitoring and optimization for AI agent systems, working across LangChain, Google ADK, CrewAI, and custom frameworks. It captures granular metrics on cross-agent coordination, tool usage efficiency, and computational costs, enabling data-driven optimizations through NVIDIA Accelerated Computing. It can be used to parallelize slow workflows, cache expensive operations, and maintain and evaluate system accuracy quickly. Compatible with OpenTelemetry and major agent frameworks, the toolkit reduces cost and enhances performance while providing insights to scale from single agents to enterprise-grade digital workforces.
Simplify Development
Experiment and prototype new agentic AI applications quickly and easily with the toolkit’s YAML configuration builder. With universal descriptors for agents, tools, and workflows, you can flexibly choose and connect agent frameworks best suited to each task in a workflow. Access a reusable collection of tools, pipelines, and agentic workflows to ease the development of agentic AI systems.
Increase Reliability
Put evaluation at the center of the development workflow with built-in commands that test agents against datasets, score outputs with customizable metrics, and generate detailed reports—so you can iterate faster and ship more reliable AI agents. Define your workflow, run baseline evals, apply fixes, measure impact, and repeat until your agent hits production-grade accuracy.
Streamline Agent Optimization
Automatically tune agents with the Agent Hyperparameter Optimizer, which selects optimal large language model types, temperature, max_tokens, and prompts optimizing for accuracy, latency, cost, or custom metrics. Accelerate runtime performance through intelligent request routing using telemetry hints with NVIDIA Dynamo, and fine-tune models using collected trajectories with reinforcement learning to reduce costs while maintaining quality.
Complete Execution Visibility
Provides plugin-based observability through an event-driven architecture that traces every step of your agent workflows and exports telemetry to platforms like Phoenix, Langfuse, Weave, or any OpenTelemetry-compatible service—so you can debug failures, optimize performance, and track costs in both development and production. Configure multiple exporters simultaneously in your workflow config to gain complete visibility into agent behaviors, token usage, and execution paths.
Safety and Security
Use NeMo Agent Toolkit safety and security middleware features to Red Team agentic workflows and find points of exploitability and vulnerabilities like prompt injection, jail break, tool poisoning, and other custom attacks. Visualize the results on a dashboard and analyze risks. Apply pluggable defense layers and models to reduce risks and make agentic workflows safer.
Production Scaling Blog
Scale the AI-Q research agent to hundreds of concurrent users using NeMo Agent Toolkit profiling, load testing, and observability.
Introductory Video
Watch a video walk-through to see how you can get started with NeMo Agent Toolkit.
Tutorial Blog
Take a technical deep dive to learn how to extend the toolkit by adding integration with an additional agentic framework, such as Agno.
Notebooks
Through this series of notebooks, we demonstrate how you can use NeMo Agent Toolkit to build, connect, evaluate, profile, and deploy an agentic system.
Get Started With NeMo Agent Toolkit
Quick Install With Pip (Recommended)
pip install nvidia-nat # Verify the library installation: nat --help nat --version
Local Setup for Examples
# Clone the repo: git clone -b main git@github.com:NVIDIA/NeMo-Agent-Toolkit.git nemo-agent-toolkit cd nemo-agent-toolkit # Initialize the Git repository: git submodule update --init --recursive # Download the datasets: git lfs install git lfs fetch git lfs pull # Create a Python environment: uv sync --all-groups --all-extras uv venv --python 3.12 --seed .venv source .venv/bin/activate uv sync --all-groups --all-extras # Verify the library installation: nat --help nat --version
Note: For the instructions above, you must have uv already installed. If you do not, to install uv, get started here.
Starter Kits
Start developing agentic AI applications with NeMo Agent Toolkit with tutorials, best practices, and documentation. The AI-Q NVIDIA Blueprint showcases examples for building agentic workflows that use the toolkit.
Getting Started With NeMo Agent Toolkit
Access the toolkit documentation, and start building, connecting, and evaluating agentic AI systems.
NeMo Agent Toolkit Learning Library
More Resources
Ethical AI
NVIDIA believes trustworthy AI is a shared responsibility, and we have established policies and practices to enable development for a wide array of AI applications. When downloaded or used in accordance with our terms of service, developers should work with their supporting model team to ensure their model meets the requirements for the relevant industry and use case and addresses unforeseen product misuse.
For more detailed information on ethical considerations for this model, please see the Model Card++ Explainability, Bias, Safety and Security, and Privacy Subcards. Please report security vulnerabilities or NVIDIA AI Concerns here.