NVIDIA TAO Toolkit

Looking for a faster, easier way to create highly accurate, customized, and enterprise-ready AI models to power your vision AI applications? The open-source TAO toolkit for AI training and optimization delivers everything you need, putting the power of the world’s best Vision Transformers (ViTs) in the hands of every developer and service provider. You can now create state-of-the-art computer vision models and deploy them on any device—GPUs, CPUs, and MCUs—whether at the edge or in the cloud.



Download TAO     Get Started

What Is the NVIDIA TAO Toolkit?

Eliminate the need for mountains of data and an army of data scientists as you create AI/machine learning models and speed up the development process with transfer learning. This powerful technique instantly transfers learned features from an existing neural network model to a new customized one.

The open-source NVIDIA TAO Toolkit, built on TensorFlow and PyTorch, uses the power of transfer learning while simultaneously simplifying the model training process and optimizing the model for inference throughput on practically any platform. The result is an ultra-streamlined workflow. Take your own models or pre-trained models, adapt them to your own real or synthetic data, then optimize for inference throughput. All without needing AI expertise or large training datasets.



What is TAO Toolkit and how does it fit into AI model development workflow?

Key Benefits

TAO toolkit lets you train models with Jupyter notebooks easily.

Train Models Efficiently

Use TAO Toolkit’s AutoML capability to eliminate the need for manual tuning and get to your solutions faster.

TAO toolkit helps you build highly accurate AI models for your use-case.

Build Highly Accurate AI

Use SOTA vision transformer and NVIDIA pretrained models to create highly accurate and custom AI models for your use case.

TAO toolkit allows you to optimize the model for inference.

Optimize for Inference

Go beyond customization and achieve up to 4X performance by optimizing the model for inference.

TAO toolkit helps you deploy optimized models with ease.

Deploy On Any Device

Deploy optimized models on GPUs, CPUs, MCUs, and more.

Enterprise-Grade Solution for Your Mission-Critical AI

NVIDIA TAO is available as a part of NVIDIA AI Enterprise, an enterprise-ready AI software platform with security, stability, manageability, and support to speed time to value while mitigating the potential risks of open-source software. Three exclusive foundation models trained on commercially viable datasets are included with NVIDIA AI Enterprise:


  • NV-DINOv2 is the only commercially viable visual foundational model trained using self-supervised learning on over 100M images. This model can be quickly fine-tuned for various vision AI tasks with only a handful of training data.
  • PCB classification, built on NV-DINOv2, delivers high accuracy for detecting missing components on a PCB.
  • Retail recognition, built on NV-DINOv2, can be used to identify large number of retail SKUs.

NV-DINOv2 based foundational models can be fine-tuned for custom vision AI tasks starting with TAO 5.1.


Apply for a free, 90-day evaluation license for NVIDIA AI Enterprise Experience TAO Toolkit and NVIDIA AI Enterprise on NVIDIA LaunchPad
TAO Toolkit is part of NVIDIA AI Enterprise to help deploy AI anywhere.

Why It Matters to Your AI Development

Bring customized Generative AI to your Application

Generative AI is a transformative force that will change many industries. Driving this are foundation models that've been trained on a large corpus of text, image, sensor and other data. Now with TAO, you can fine-tune and customize these foundation models and create domain specific generative AI applications. TAO enables fine-tuning of multi-modal models such as NV-DINOv2, NV-CLIP, Grounding-DINO and more.


Watch GTC Talk
Models trained with the NVIDIA TAO Toolkit can be deployed on any platform.
Models trained with the NVIDIA TAO Toolkit can be deployed on any platform.

Deploy Models on Any Platforms

NVIDIA TAO Toolkit can help power AI across billions of devices. The new NVIDIA TAO Toolkit 5.0 supports model export in ONNX, an open format for better interoperability. This makes it possible to deploy a model trained with the NVIDIA TAO Toolkit on any computing platform.


Learn More About the Integration With STMicroelectronics Learn More About the Integration with ARM Ethos-U NPUs Learn More About the Integration With Edge Impulse Learn More About the Integration With Nota LaunchX Learn More About Integration With Arm CPU and NPU

AI-Assisted Data Annotations

New AI-assisted annotation capabilities give you a faster and less expensive way to label segmentation masks. You can use the weakly supervised transformer-based segmentation architecture, Mask Auto Labeler (MAL), to assist in segmentation annotation and in fixing and tightening bounding boxes for object detection.


Learn More
Workflow for running TAO Toolkit as-a-Service with REST APIs.

Integrate TAO Toolkit in Your Application With Rest APIs

You now have an easier way to deploy TAO Toolkit in a modern cloud-native infrastructure using Kubernetes and integrate it in your application with REST APIs. Build a new AI service or integrate the TAO Toolkit into your existing service and enable automation between disparate tools.


Learn More

Make AI Easier With AutoML

Training and optimizing AI is a time-consuming process, requiring intimate knowledge of what model to choose and what hyperparameters to tune. Now, you can easily train high-quality models with AutoML without the hassle of manually fine-tuning hundreds of parameters.


Learn More Watch the Video
 Hyperparameters metrics to show TAO makes it easy to train models with AutoML.
 A visual diagram to show what cloud services are integrated with TAO

Run on Your Favorite Cloud

The cloud-native technology in TAO provides the agility, scalability, and portability you need to more effectively manage and deploy AI applications. TAO services can be deployed on VMs from any leading cloud provider, as well as with managed Kubernetes services like Amazon EKS, Google GKE or Azure AKS. It can also be used with cloud machine learning services such as Google Colab, Google Vertex AI, and Azure Machine Learning to simplify infrastructure management and scaling.

TAO also enables integrations with several cloud and third-party MLOPs services to provide developers and enterprises with an optimized AI workflow. Developers can now track and manage their TAO experiment and manage models using the W&B or ClearML platform.


Learn More About Running TAO in Cloud Learn More About Integrating TAO with MLOPs services

Inference Performance

Unlock peak inference performance with NVIDIA pretrained models across platforms from the edge with NVIDIA Jetson™ solutions to the cloud featuring NVIDIA Ampere architecture GPUs. For more details on batch size and other models, check the detailed performance datasheet.



Model Arch
Resolution
Accuracy
Jetson Orin Nano
Jetson Orin Nx
Jetson Orin 64GB
A2
T4
L4
L40
H100
PeopleSemSegFormer
SegFormer
512x512
91% mIoU
6.6
9.7
24.2
23
40 
83
210
454
Retail Object Detection
DINO - FAN-B
960x544
97%
2.3
3.4
8.1
8.8
15.4 
34
89
167
DINO COCO
DINO - FAN-S
960x544
72% mAP50
3.1
4.4
11.2
11.7
20
44
120
213
GC-ViT ImageNet
GC-ViT-Tiny
224x224
84% Top1 Accuracy
75
110
293
336
517
1266
3118
6381
OCRNet
ResNet50 - Bi-LSTM
32x100
93%
935
1373
3876
2094
3649 
8036
18970
55720
OCDNet
DCN-ResNet18
640x640
81% Hmean
31
45
120
93
155
333
940
1468
Optical Inspection
Siamese CNN
2x512x128
100%/<1% FP
399
482
1538
1391
2314
2821
10390
24110

Customer Stories

OneCup AI Customer Story

OneCup AI

OneCup AI’s computer vision system tracks and classifies animal activity using NVIDIA pretrained models and TAO Toolkit, significantly reducing their development time from months to weeks.


Learn More
KoiReader Customer Story

KoiReader

KoiReader developed an AI-powered machine vision solution using NVIDIA developer tools including TAO to help PepsiCo achieve precision and efficiency in dynamic distribution environments.


Read the Blog
Trifork Customer Story

Trifork

Trifork jumpstarted their AI model development with NVIDIA pretrained models and TAO Toolkit to develop their AI-based baggage tracking solution for airports.


Learn More

Leading Adopters

arruga ai
booz allen
Kion group
Inex tech
Lexmark ventures
nota ai
One cup AI
Rocketboots
SmartCow
two i
appen
cvedia
hasty
lexset
lightly
Rendered AI
Sky Engine
Yuva AI
roboflow

General FAQ

Transfer learning is the process of transferring learned features from one application to another. It’s a commonly used training technique where a model trained on one task is re-trained for use on a different task. You can apply transfer learning on vision, speech, and language-understanding models.
Yes, with the standard ONNX output, TAO model can be deployed on any device that supports ONNX-RT or has a compiler to convert ONNX to hardware runtime.
With TAO 5.0, we’re open sourcing the toolkit on GitHub .
Yes. For exact licensing terms, refer to model EULA. However, unencrypted models are only available with NVIDIA AI Enterprise licenses.
The TAO Toolkit supports 100+ permutations of NVIDIA-optimized model architectures and backbones. These include State-of-the-art Vision Transformers like FAN, DINO, and GC-ViT, along with tons of efficient CNNs such as EfficientDet, YOLOs, UNET, and many more.

You can find the full matrix of supported model architectures here.
Under the hood, TAO Toolkit uses TensorFlow and PyTorch frameworks but those are completely abstracted away from the user. Users operate TAO Toolkit through documented spec files and no prior knowledge of deep learning framework is required.
NVIDIA AI Enterprise is an end-to-end, secure, cloud-native AI software platform optimized to accelerate enterprises to the leading edge of AI. Benefits of using TAO Toolkit with NVIDIA AI Enterprise:
  • Access to exclusive foundation models for vision AI
  • Validation and integration for NVIDIA AI open-source software
  • Access to AI solution workflows to speed time to production
  • Certifications to deploy AI everywhere
  • Enterprise-grade support, security, manageability, and API stability to mitigate potential risks of open source software
You can download the sample Jupyter notebooks from the NGC catalog.
  • Vision models can be deployed through DeepStream or NVIDIA Triton™.
  • You can also deploy the models in ONNX format on any platform.

Refer to the documentation section for deployment details.

Yes, TAO can be deployed at the infrastructure level using VMs from the cloud or can be deployed in various cloud services like Amazon EKS, Azure AKS, Google GKE, Google Vertex AI, Azure Machine Learning, or Google Colab. Please refer to the TAO Toolkit documentation to learn more about running the TAO Toolkit on AWS, Azure, or GCP.
You can only train with TAO toolkit on an x86 system. You can, however, deploy the optimized models on a Jetson solution.
NVIDIA Metropolis is an application framework, set of developer tools, and partner ecosystem that brings visual data and AI together to improve operational efficiency and safety across a broad range of industries. Learn More here.

Resources

New Blog - TAO 5.0

The NVIDIA TAO Toolkit version 5.0 supercharges vision AI development for practically any developer, in any service and on any device.


Read the Blog

New Blog - Vision Transformers

Learn how to improve accuracy and robustness of vision AI apps with Vision Transformers (ViTs) and NVIDIA TAO

 


Read the Blog

New Blog -Character Detection and Recognition

Learn how to train and deploy a custom optical character detection and recognition model using NVIDIA TAO and NVIDIA Triton.


Read the Blog - Part 1   |   Part 2


Simplify and speed up AI training with TAO Toolkit.

Get started