NVIDIA JARVIS

NVIDIA Jarvis is an application framework for multimodal conversational AI services that delivers real-time performance on GPUs.

Introductory Webinar  Apply Now for Early Access




Jarvis is a fully accelerated application framework for building multimodal conversational AI services that use an end-to-end deep learning pipeline. Developers at enterprises can easily fine-tune state-of-art-models on their data to achieve a deeper understanding of their specific context and optimize for inference to offer real-time services that run in 150 milliseconds (ms), versus the 25 seconds required on CPU-only platforms.

The Jarvis framework includes pre-trained conversational AI models, tools in the NVIDIA AI Toolkit, and optimized end-to-end services for speech, vision, and natural language understanding (NLU) tasks.

Fusing vision, audio, and other sensor inputs simultaneously provides capabilities such as multi-user, multi-context conversations in applications such as virtual assistants, multi-user diarization, and call center assistants.

Jarvis-based applications have been optimized to maximize performance on the NVIDIA EGX™ platform in the cloud, in the data center, and at the edge.



Real-Time Performance

Run deep learning-based conversational AI applications in under 300 ms, the latency threshold for real-time performance.

Multimodal

Fuse speech and vision to offer accurate and natural interactions in virtual assistants, chatbots, and other conversational AI applications.

Automated Deployment

Use one command to deploy conversational AI services in the cloud or at the edge.



Create State-of-the-Art Deep Learning Models


Figure 1: Pre-trained models

Use state-of-the-art deep learning models trained for more than 100,000 hours on NVIDIA DGX™ systems for speech, language understanding, and vision tasks. Pre-trained models and scripts used in Jarvis are freely available in NGC™.

You can fine-tune these models for your domain with your data using NVIDIA NeMo and then use tools in the NVIDIA AI Toolkit to easily deploy them as services.



Develop New Multimodal Skills


Build multimodal skills such as multi-speaker transcription, chatbots, gesture recognition, and look-to-talk for your conversational AI applications.

Jarvis includes multi-skill samples that you can customize for your use case. With Jarvis, you can use speech, language understanding, and vision pipelines along with a dialog manager that supports multi-user and multi-context to author new skills.

Figure 2: Multimodal application with multiple users and contexts


Optimized Task-Specific Services


Figure 3: Jarvis AI services

Access high-performance services for tasks such as speech recognition, intent recognition, text-to-speech, pose estimation, gaze detection, and facial landmark detection through a simple API.

Pipelines for each skill can be fused to build new skills. Each pipeline is performance-tuned to deliver the highest performance possible and can be customized for your specific use case.



Build and Deploy Services Easily



Automates the steps that go from pre-trained models to optimized services deployed in the cloud, in the data center, and at the edge. Under the hood, Jarvis applies powerful NVIDIA® TensorRT™ optimizations to models, configures the NVIDIA Triton™ Inference Server, and exposes the models as a service through a standard API.

To deploy, you can use a single command to download, set up, and run the entire Jarvis application or individual services through Helm charts on Kubernetes clusters. The Helm charts can be customized for your use case and setup.

Figure 4: Helm command to deploy models to production



Resources




Apply for exclusive news, updates, and early access to NVIDIA Jarvis.

Apply Now