NVIDIA DeepStream SDK
Develop vision AI applications and services faster with DeepStream’s multi-platform support. Deploy on-premises, on the edge, and in the cloud with the click of a button.
Get started Try on LaunchPadWhat is NVIDIA DeepStream?
There are billions of cameras and sensors worldwide, capturing an abundance of data that can be used to generate business insights, unlock process efficiencies, and improve revenue streams. Whether it’s at a traffic intersection to reduce vehicle congestion, health and safety monitoring at hospitals, surveying retail aisles for better customer satisfaction, sports analytics, or at a manufacturing facility to detect component defects, every application demands reliable, real-time Intelligent Video Analytics (IVA). NVIDIA’s DeepStream SDK is a complete streaming analytics toolkit based on GStreamer for AI-based multi-sensor processing, video, audio, and image understanding. It’s ideal for vision AI developers, software partners, startups, and OEMs building IVA apps and services. Developers can now create stream processing pipelines that incorporate neural networks and other complex processing tasks such as tracking, video encoding/decoding, and video rendering. DeepStream pipelines enable real-time analytics on video, image, and sensor data.
DeepStream key benefits
Powerful and Flexible SDK
DeepStream SDK is suitable for a multitude of use-cases across a broad set of industries.
Multiple Programming Options
Create powerful vision AI applications using C/C++, Python, or Graph Composer’s simple and intuitive UI.
Real-Time Insights
Understand rich and multi-modal real-time sensor data at the edge.
Managed AI Services
Deploy AI services in cloud native containers and orchestrate them using Kubernetes.
Reduced TCO
Increase stream density by training, adapting, and optimizing models with TAO toolkit and deploying models with DeepStream.
Explore programming options
C/C++
Create applications in C/C++, interact directly with GStreamer and DeepStream plug-ins, and use reference applications and templates.
Learn more about C/C++
Python
DeepStream pipelines can be constructed using Gst Python, the GStreamer framework's Python bindings. The source code for the binding and Python sample applications are available on GitHub.
Learn more about Python
Graph Composer
Graph Composer is a low-code development tool that enhances the DeepStream user experience. Using a simple, intuitive UI, processing pipelines are constructed with drag-and-drop operations.
Learn More About Graph Composer
Improve accuracy and real-time performance
DeepStream offers exceptional throughput for a wide variety of object detection, image processing, and instance segmentation AI models. The following table shows the end-to-end application performance from data ingestion, decoding, and image processing to inference. It takes multiple 1080p/30fps streams as input. Note that running on the DLAs for Jetson devices frees up the GPU for other tasks. For performance best practices, watch this video tutorial.
(Version 2.5) | ||||||||||||||||||
(Version 2.5) | ||||||||||||||||||
LPDNet LPRNet | 640x480 96x48 | INT8 FP16 | ||||||||||||||||
192x256 | FP16 | |||||||||||||||||
N/A* = Numbers are not available in JetPack 5.0.2
The DeepStream SDK lets you apply AI to streaming video and simultaneously optimize video decode/encode, image scaling, and conversion and edge-to-cloud connectivity for complete end-to-end performance optimization.
To learn more about the performance using DeepStream, check the documentation.
Explore key features
Enjoy seamless development
Developers can build seamless streaming pipelines for AI-based video, audio, and image analytics using DeepStream. It brings development flexibility by giving developers the option to develop in C/C++,Python, or use Graph Composer for low-code development. DeepStream ships with various hardware accelerated plug-ins and extensions.
DeepStream is built for both developers and enterprises and offers extensive AI model support for popular object detection and segmentation models such as state of the art SSD, YOLO, FasterRCNN, and MaskRCNN. You can also integrate custom functions and libraries.
Get flexibility, from rapid prototyping to full production level solutions, and choose your inference path. With native integration to NVIDIA Triton™ Inference Server, you can deploy models in native frameworks such as PyTorch and TensorFlow for inference. Using NVIDIA TensorRT™ for high-throughput inference with options for multi-GPU, multi-stream and batching support helps you achieve the best possible performance.
In addition to supporting native inference, DeepStream applications can communicate with independent/remote instances of Triton Inference Server using gRPC, allowing the implementation of distributed inference solutions.
Learn more
Get cloud-native
The use of cloud-native technologies offers the flexibility and agility that are necessary for rapid product development and continuous product improvement over time. With the cloud-native approach, organizations have the ability to build applications that are resilient and manageable, thereby enabling faster deployments of applications.
Developers can use the DeepStream Container Builder tool to build high-performance, cloud-native AI applications with NVIDIA NGC containers. The generated containers are easily deployed at scale and managed with Kubernetes and Helm Charts.
Learn more
Build end-to-end AI solutions
Speed up overall development efforts and unlock greater real-time performance by building an end-to-end vision AI system with NVIDIA Metropolis. Start with production-quality vision AI models, adapt and optimize them with TAO Toolkit, and deploy using DeepStream.
DeepStream offers turnkey integration of several detection and segmentation models including SSD, MaskRCNN, YOLOv4, RetinaNet and more. Developers can use their own custom model by leveraging Triton server and DeepStream’s custom pre- and post- processing plugins.
Learn more
Access reference applications
DeepStream SDK is bundled with 30+ sample applications designed to help users kick-start their development efforts. Most samples are available in C/C++, Python, and Graph Composer versions and run on both NVIDIA Jetson™ and dGPU platforms. Reference applications can be used to learn about the features of the DeepStream plug-ins or as templates and starting points for developing custom vision AI applications.
Learn more
Work with Graph Composer
With Graph Composer, DeepStream developers now have a powerful, low-code development option. A simple and intuitive interface makes it easy to create complex processing pipelines and quickly deploy them using Container Builder.
Graph Composer abstracts much of the underlying DeepStream, GStreamer, and platform programming knowledge required to create the latest real-time, multi-stream vision AI applications.
Instead of writing code, users interact with an extensive library of components, configuring and connecting them using the drag-and-drop interface. Developers can use NVIDIA’s repository of optimized extensions for different hardware platforms or create their own.
Learn more
Read customer stories
.jpg)
OneCup AI
OneCup AI’s computer vision system tracks and classifies animal activity using NVIDIA pretrained models, TAO Toolkit, and DeepStream SDK, significantly reducing their development time from months to weeks.
Learn more about OneCup AI
Tapway
Using NVIDIA GPUs, DeepStream SDK, and other NVIDIA software tools, Tapway trained and ran AI models that could read a vehicle’s license plate and detect its class, make, and color in just 50 milliseconds—about one tenth of one eye blink—even if it’s traveling at up to 40 kilometers/hour.
Learn more about Tapway
Two-i
Two-i’s technology—which combines computer vision, data science and deep learning—is helping to prevent deadly accidents in the oil and gas industry, one of the world’s most dangerous sectors.
Two-i’s flexibility stems from its reliance on using the NVIDIA Metropolis platform for AI-enabled video analytics applications, leveraging advanced tools and adopting a full-stack approach.
Learn more about Two-i
General FAQ
DeepStream is a closed-source SDK. Note that sources for all reference applications and several plugins are available.
The DeepStream SDK can be used to build end-to-end AI-powered applications to analyze video and sensor data. Some popular use cases are retail analytics, parking management, managing logistics, optical inspection, robotics, and sports analytics.
Yes, that’s now possible with the integration of the Triton Inference server. Also with DeepStream 6.1.1, applications can communicate with independent/remote instances of Triton Inference Server using gPRC.
DeepStream supports several popular networks out of the box. For instance, DeepStream supports MaskRCNN. Also, DeepStream ships with an example to run the popular YOLO models, FasterRCNN, SSD and RetinaNet.
Yes, DS 6.0 or later supports the Ampere architecture.
Yes, audio is supported with DeepStream SDK 6.1.1. To get started, download the software and review the reference audio and Automatic Speech Recognition (ASR) applications. Learn more by reading the ASR DeepStream Plugin.