Analyze Data From Sensors and IoT Gateways in Real-Time

With more than a billion cameras and sensors continuously generating video streams and data, getting real-time actionable insights is more challenging than ever. NVIDIA’s DeepStream SDK delivers a complete streaming analytics toolkit for situational awareness through computer vision, intelligent video analytics (IVA) and multi-sensor processing. The DeepStream application framework features hardware-accelerated building blocks that bring deep neural networks and other complex processing tasks into a stream processing pipeline. Focus on building core deep learning networks and IP rather than designing end-to-end solutions from scratch. DeepStream is an integral part of NVIDIA Metropolis, the platform for building end-to-end solutions with computer vision, IVA and multi-sensor processing.

Rocket Fuel

Edge-to-Cloud Deployment

DeepStream SDK supports a diversity of use cases, using AI to perceive pixels and sensors and analyze metadata. It also offers the flexibility to deploy from NVIDIA Jetson on the edge to NVIDIA Tesla® in the cloud. The SDK lets you integrate the edge to the cloud with standard message brokers like Kafka for large-scale, wide-area deployments. This is ideal for applications like retail analytics, intelligent traffic control, automated optical inspection, freight and goods tracking, web content filtering, target ad injection, and more.

Seamlessly Develop Complex Stream Processing Pipelines


The DeepStream SDK uses the open source GStreamer to deliver high throughput with a low-latency streaming framework. The runtime system is pipelined to enable deep learning capabilities, as well as image and sensor processing and fusion algorithms in a streaming application.

DeepStream uses:

  • NVIDIA® TensorRT™ and NVIDIA CUDA® for AI and other GPU computing tasks
  • Video CODEC SDK and multimedia APIs for accelerated encoding and decoding
  • Imaging APIs for capture and processing
  • A graph-based architecture and modular plug-ins to create configurable processing pipelines

To solve the most complex problems, you can easily build applications by using the NVIDIA-provided hardware-accelerated plug-ins. DeepStream also offers the ability to build custom plugins for user-created libraries and functions. One of many open-source plug-ins can also be easily adapted for use with the DeepStream framework.

Plug-ins available with DeepStream SDK

  • H.264 and H.265 video decoding
  • Stream aggregation and batching
  • TensorRT-based inferencing for detection and classification
  • Object tracking reference implementation
  • On-screen display API for highlighting objects and text overlay
  • Frame rendering from multi-source into a 2D grid array
  • Accelerated X11/EGL-based rendering
  • Scaling, format conversion, and rotation
  • Dewarping for 360-degree camera input
  • Metadata generation and encoding
  • Messaging to cloud
  • On-screen display API for highlighting objects and text overlay
  • Frame rendering from multi-source into a 2D grid array
  • Accelerated X11/EGL-based rendering
  • Scaling, format conversion, and rotation
  • Dewarping for 360-degree camera input
  • Metadata generation and encoding
  • Messaging to cloud

DeepStream’s modular architecture also lets you run heterogeneous concurrent neural networks that are critical to understanding video that’s rich and multi-modal. The SDK offers complete reference applications and pre-trained neural networks to jumpstart development.

Start in Seconds, Scale Instantly with Docker

Applications built with the DeepStream SDK can now be deployed using a Docker container on NVIDIA Tesla platforms, enabling flexible system architectures and straightforward upgrades that greatly improve system manageability. The Docker containers with full reference applications are available on NVIDIA GPU Cloud (NGC).

End-to-End AI Workflow for IVA

The Transfer Learning Toolkit, along with DeepStream SDK 3.0, offers an end-to-end deep learning solution for IVA on Tesla GPUs. You can now accelerate development of efficient deep learning networks and prune networks to tightly pack complex applications, delivering high throughput and stream density.


Resources

FAQ

Blogs

Forums

Webinars

GitHub Repositories

NVIDIA Deep Learning Institute