NVIDIA TAO Toolkit

Create highly accurate, customized, and production-ready AI models to power your speech and computer vision AI applications.



Download TAO     Get Started


What Is the NVIDIA TAO Toolkit?

Eliminate your need for mountains of data and an army of data scientists as you create AI/machine learning models, and speed up the development process with transfer learning - a powerful technique that instantly transfers learned features from an existing neural network model to a new customized one

The NVIDIA TAO Toolkit, built on TensorFlow and PyTorch, uses the power of transfer learning while simultaneously simplifying the model training process and optimizing the model for inference throughput on the target platform. The result is an ultra-streamlined workflow. Take your own models or pre-trained models, adapt them to your own real or synthetic data, then optimize for inference throughput. All without needing AI expertise or large training datasets.



What is TAO Toolkit and how does it fit into AI model development workflow?

Announcing TAO 5.0

The next version of TAO Toolkit brings the power of the world’s best vision Transformers in the hands of every developer and every service provider to create state-of-the-art computer vision models and deploy them on any device - GPUs, CPUs, MCUs - whether at the edge or in the cloud.

The new TAO 5.0 will be available in June.


Read Blog

Key Benefits

TAO toolkit lets you train models with Jupyter notebooks easily.

Train Models Efficiently

Use TAO Toolkit’s AutoML capability to eliminate the need for manual tuning and get to your solutions faster.

TAO toolkit helps you build highly accurate AI models for your use-case.

Build Highly Accurate AI

Use SOTA vision Transformer and NVIDIA pretrained models to create highly accurate and custom AI models for your use-case.

TAO toolkit allows you to optimize the model for inference.

Optimize for Inference

Go beyond customization and achieve up to 4X performance by optimizing the model for inference.

TAO toolkit helps you deploy optimized models with ease.

Deploy With Ease

Deploy optimized models using NVIDIA DeepStream for vision AI, Riva for speech AI, and Triton Inference Server™ on any device - GPUS, CPUs, MCUs and more.

Get Enterprise-Ready with NVIDIA AI Enterprise

Access Complete Source Code and Model Weights

NVIDIA AI Enterprise includes access to TAO source code and the model weights for pretrained models without encryption for a wide range of use cases. Now developers can view the weights and biases of the model which can help in model explainability and understand model bias. In addition, unencrypted models are easier to debug and integrate into custom AI apps.

Enterprise support is included with NVIDIA AI Enterprise to ensure business continuity and AI projects stay on track.

Experience TAO Toolkit and NVIDIA AI Enterprise on NVIDIA LaunchPad

Have an NVIDIA H100? Activate NVIDIA AI Enterprise software included in your NVIDIA H100 integrated mainstream servers.

TAO Toolkit is part of NVIDIA AI Enterprise to help deploy AI anywhere.

What Are Pretrained AI Models?

Pretrained AI Models have been trained on representative datasets and fine-tuned with weights and biases. You can quickly and easily customize these models with only a fraction of the real-world or synthetic data needed to train from scratch.



Explore Pretrained Models For Vision AI

Create custom deep learning models for computer vision tasks like image processing and classification, object detection, and semantic segmentation using 100+ NVIDIA-optimized model architectures.

You can also use task-based models to recognize human actions and poses, detect people in crowded spaces, detect and classify vehicles and license plates, and much more.

Check out our new transformer-based vision AI models to develop robust AI-enabled video analytics applications.


Download Jupyter Notebooks From NGC View Performance Chart
NVIDIA Pre-trained Models for Speech AI includes speech recognition.

Explore Pretrained Models for Conversational AI

Design personalized real-time call center experiences, smart kiosks, and other conversation AI services by fine-tuning Automatic Speech Recognition (ASR) , Natural Language Processing (NLP), and Text-to-Speech (TTS)-based modes.


Download Jupyter Notebooks From NGC

Key Features

Integrate TAO Toolkit in Your Application with Rest APIs

You now have an easier way to deploy TAO Toolkit in a modern cloud-native infrastructure on Kubernetes and integrate it in your application with REST APIs. Build a new AI service or integrate the TAO Toolkit into your existing service and enable automation between disparate tools.


Learn More
Workflow for running TAO Toolkit as-a-Service with REST APIs

Make AI easier with AutoML

Training and optimizing AI is a time-consuming process, requiring intimate knowledge of what model to choose and what hyperparameters to tune. Now, you can easily train high-quality models with AutoML without the hassle of manually fine-tuning hundreds of parameters.


Learn More Watch the Webinar

Run on Your Favorite Cloud

The cloud-native technology in TAO provides the agility, scalability, and portability you need to more effectively manage and deploy AI applications. TAO services can be deployed on VMs from any leading cloud provider, as well as with Kubernetes services like Amazon EKS or Azure AKS . It can also be used with cloud machine learning services such as Google Colab, Google Vertex AI, and Azure Machine Learning to simplify infrastructure management and scaling.


Learn More
Generalize model with Data Augmentation features.
A workflow to show how TAO is integrated with MLOps services.

Enhance AI Workflow with MLOPs Integrations

Building an AI model is an iterative process, from dataset curation, model development, experiment tracking, and model management to model deployment. TAO enables integrations with several cloud and third-party MLOPs services to provide developers and enterprises with an optimized AI workflow. Developers can now track and manage their TAO experiment and manage models using the W&B or ClearML platform.


Learn More

Inference Performance

Unlock peak inference performance with NVIDIA pretrained models across platforms from the edge with NVIDIA Jetson™ solutions to the cloud featuring NVIDIA Ampere architecture GPUs. For more details on batch size and other models, check the detailed performance datasheet.



Xavier™ NX
AGX Xavier
AGX Orin™
T4
A2
A30
A100
Model Arch
Inference Resolution
Precision
Model Accuracy
GPU + 2*DLA(FPS)
GPU + 2*DLA(FPS)
GPU + 2*DLA(FPS)
GPU(FPS)
GPU (FPS)
GPU (FPS)
GPU (FPS)
PeopleNet
960x544x3
INT8
80%mAP
263
418
1294
1064 
581
3160
6245
3D Pose Estimation
256x192x3
FP16
8 pixel error
147
235
711
713 
471
2242
4179
Pose Action Classification
3x300x34x1
FP16
90%
87
150
262
376
211
1122
2145
DashCamNet
960x544x3
INT8
84% mAP
423
670
1895
1676
865
4900
9500
License Plate Recognition
96x48x3
FP16
98% mAP
706
1190
4118
3959 
2180
12400
27200
Action Recognition 2D
224x224x96
FP16
83%
245
471
1577
1897
1044
6000
12600
People Semantic Segmentation
960x544x3
FP16
87% mIoU
199
356
673
1027
631
2862
5745

Customer Stories

OneCup AI Customer Story

OneCup AI

OneCup AI’s computer vision system tracks and classifies animal activity using NVIDIA pretrained models and TAO Toolkit, significantly reducing their development time from months to weeks.


Learn More
KoiReader Customer Story

KoiReader

KoiReader developed an AI-powered machine vision solution using NVIDIA developer tools including TAO to help PepsiCo achieve precision and efficiency in dynamic distribution environments.


Read the Blog
Trifork Customer Story

Trifork

Trifork jumpstarted their AI model development with NVIDIA pretrained models and TAO Toolkit to develop their AI-based baggage tracking solution for airports.


Learn More

Leading Adopters

arruga ai
booz allen
Kion group
Inex tech
Lexmark ventures
nota ai
One cup AI
Rocketboots
SmartCow
two i
appen
cvedia
hasty
lexset
lightly
Rendered AI
Sky Engine
Yuva AI
roboflow

General FAQ

NVIDIA TAO is a framework for training, adapting and optimizing computer vision and conversational AI models with your custom data, in a fraction of time without large training datasets or AI expertise.

There are two versions of TAO.

  1. TAO Toolkit, which is the low-code version of TAO
  2. A no-code, GUI-based solution currently in development. Apply for early access here.
Transfer learning is the process of transferring learned features from one application to another. It’s a commonly used training technique where a model trained on one task is re-trained for use on a different task. You can apply transfer learning on vision, speech, and language-understanding models.
Yes. For specific licensing terms, refer to model EULA.
The TAO Toolkit supports 100+ permutations of NVIDIA-optimized model architectures and backbones such as EfficientNet, YOLOv3/v4, RetinaNet, FasterRCNN, UNET, and many more.

The full matrix of supported model architectures can be found here.
TAO Toolkit uses TensorFlow and PyTorch framework completely abstracted away from the user. Users operate TAO Toolkit through documented spec files and do not have to learn about DL framework.
NVIDIA AI Enterprise is an end-to-end, secure, cloud-native suite of AI software with key benefits including validation and integration for NVIDIA AI open-source software, access to AI solution workflows to speed time to production, certifications to deploy AI everywhere, and enterprise-grade support, security & API stability.

Use TAO Toolkit with NVIDIA AI Enterprise to get full access to the source code of TAO Toolkit and model weights for the pretrained models, as well as enterprise support that provides guaranteed response times, priority security notifications, and access to AI experts from NVIDIA.
  • Vision models can be deployed through DeepStream
  • Speech Models can deployed through Riva
  • You can also deploy the models through Triton Inference Server.

Refer to the documentation section for deployment details.

With the TAO Toolkit, you can use a model that detects people, and add a new “helmet class” to that model. Fine-tune it with the dataset that contains classes for both people and helmet. Read more about it in the white paper.
Yes, you can deploy the TAO Toolkit in the cloud. Please refer to the TAO Toolkit documentation to Learn More about running the TAO Toolkit on AWS, Azure, or GCP.
You can only train with TAO toolkit on an x86 system. You can, however, deploy the optimized models on a Jetson solution.
NVIDIA Metropolis is an application framework, set of developer tools, and partner ecosystem that brings visual data and AI together to improve operational efficiency and safety across a broad range of industries. Learn More here.

Resources

New Developer Blog

The next version of TAO Toolkit supercharges vision AI development for practically any developer, in any service and on any device.


Read the Blog

New AutoML Webinar

Learn how to accelerate AI Model creation using AutoML in NVIDIA TAO 4.0.

 


Watch the Webinar

GTC Webinar

Learn how to create and deploy custom, production-ready vision AI models without any expertise in AI.


Watch the On-Demand Webinar

Simplify and speed-up AI training with TAO Toolkit

Get started