NVIDIA NeMo Agent Toolkit
NVIDIA NeMo™ Agent Toolkit is an open-source AI framework for building, profiling, and optimizing agents and tools from any framework, enabling unified, cross-framework integration across connected AI agent systems. By exposing hidden bottlenecks and costs and optimizing the workflow, it helps enterprises scale agentic systems efficiently while maintaining reliability.
NeMo Agent Toolkit is part of the NVIDIA NeMo software suite for managing the AI agent lifecycle, providing telemetry, orchestration, and observability tools that accelerate development, uncover bottlenecks, and streamline performance across multi-agent systems.
See NeMo Agent Toolkit in Action
18:31 | ||||||||||||||||||||||
11:04 | ||||||||||||||||||||||
23:06 | ||||||||||||||||||||||
05:29 |
How NeMo Agent Toolkit Works
NVIDIA NeMo Agent Toolkit provides unified monitoring and optimization for AI agent systems, working across LangChain, CrewAI, and custom frameworks. It captures granular metrics on cross-agent coordination, tool usage efficiency, and computational costs, enabling data-driven optimizations through NVIDIA Accelerated Computing. It can be used to parallelize slow workflows, cache expensive operations, and maintain and evaluate system accuracy quickly. Compatible with OpenTelemetry and major agent frameworks, the toolkit reduces cloud spend and enhances performance while providing insights to scale from single agents to enterprise-grade digital workforces.
NeMo Agent Toolkit supports the Model Context Protocol (MCP), enabling developers to use the toolkit to access tools served by remote MCP servers, or as a server to make their own tools available to others via MCP. This means agents built with the toolkit can easily use any tool registered in an MCP registry.
Simplify Development
Experiment and prototype new agentic AI applications quickly and easily with the toolkit’s YAML configuration builder. With universal descriptors for agents, tools, and workflows, you can flexibly choose and connect agent frameworks best suited to each task in a workflow. Access a reusable collection of tools, pipelines, and agentic workflows to ease the development of agentic AI systems.
Accelerate Development and Improve Reliability
Build agentic systems with ease and repeatability. In the tool registry, access the best retrieval-augmented generation (RAG) architectures, workflows, and search tools available across your organization, or leverage the AI-Q NVIDIA Blueprint, built with NVIDIA NIM™ and NeMo. With the AI-Q blueprint, developers have an example to build highly accurate, scalable multimodal ingestion and RAG pipelines that connect AI agents to enterprise data and reasoning for various use cases, including AI agents for research and reporting.
Streamline Agent Optimization
Use the Agent Hyperparameter Optimizer to automatically select the best hyperparameters, such as the type of large language model (LLM), its temperature, and max_token, by optimizing the agentic workflow on metrics including accuracy, groundedness, latency, token counts, or custom. The toolkit also supports prompt optimization to further refine agent performance. This allows developers to quickly identify optimal settings for agents, tools, and workflows, reducing trial and error while accelerating innovation across projects.
Accelerate Agent Responses
Use fine-grained telemetry to enhance agentic AI workflows. This profiling data can be used by NVIDIA NIM and NVIDIA Dynamo to optimize the performance of agentic systems. These forecasted metrics—which can include details about an inference call to an LLM for a particular agent, such as what prompt is in memory, where it might reside, and which other agents are likely to call it—can be used to drive a more efficient workflow, enabling better business outcomes without requiring an upgrade to underlying infrastructure.
Increase Accuracy
Evaluate an agentic system’s accuracy using metrics collected with NeMo Agent Toolkit, and connect them with your preferred observability and orchestration tools. Understand and debug inputs and outputs for each component in an agentic workflow, and identify areas for improvement. Swap out tools or models and use the toolkit’s YAML config to quickly reevaluate the pipeline to understand its impact.
Introductory Blog
Learn how to leverage AI code generation with NeMo Agent Toolkit to build a test-driven coding agent.
Introductory Video
Watch a video walk-through to see how you can get started with NeMo Agent Toolkit.
Tutorial Blog
Take a technical deep dive to learn how to extend the toolkit by adding integration with an additional agentic framework, such as Agno.
Notebooks
Through this series of notebooks, we demonstrate how you can use NeMo Agent Toolkit to build, connect, evaluate, profile, and deploy an agentic system.
Get Started With NeMo Agent Toolkit
Quick Install With Pip (Recommended)
pip install nvidia-nat # Verify the library installation: nat --help nat --version
Local Installer Instructions (For running examples)
# Clone the repo: git clone -b main git@github.com:NVIDIA/NeMo-Agent-Toolkit.git nemo-agent-toolkit cd nemo-agent-toolkit # Initialize the Git repository: git submodule update --init --recursive # Download the datasets: git lfs install git lfs fetch git lfs pull # Create a Python environment: uv sync --all-groups --all-extras uv venv --python 3.12 --seed .venv source .venv/bin/activate uv sync --all-groups --all-extras # Verify the library installation: nat --help nat --version
Note: For the instructions above, you must have uv already installed. If you do not, to install uv, get started here.
Starter Kits
Start developing agentic AI applications with NeMo Agent Toolkit with tutorials, best practices, and documentation. The AI-Q NVIDIA Blueprint showcases examples for building agentic workflows that use the toolkit.
Getting Started With NeMo Agent Toolkit
Access the toolkit documentation, and start building, connecting, and evaluating agentic AI systems.
NeMo Agent Toolkit Learning Library
More Resources
Ethical AI
NVIDIA believes trustworthy AI is a shared responsibility, and we have established policies and practices to enable development for a wide array of AI applications. When downloaded or used in accordance with our terms of service, developers should work with their supporting model team to ensure their model meets the requirements for the relevant industry and use case and addresses unforeseen product misuse.
For more detailed information on ethical considerations for this model, please see the Model Card++ Explainability, Bias, Safety and Security, and Privacy Subcards. Please report security vulnerabilities or NVIDIA AI Concerns here.