NVIDIA Optical Flow SDK

The NVIDIA Optical Flow SDK taps in to the latest hardware capabilities of NVIDIA Turing™, Ampere, and Ada architecture GPUs dedicated to computing the relative motion of pixels between images. The hardware uses sophisticated algorithms to yield highly accurate flow vectors, ideal for handling frame-to-frame intensity variations and tracking true object motion.

Get Started

Turing hardware generated optical flow map sample Turing hardware-generated optical flow map sample — source footage

Optical Flow Benefits

Accurate Video Analytics

Accurately detect and track objects in successive video frames while significantly reducing the computational complexity requirements.

Real-Time Performance

Interpolate or extrapolate video frames in real time, improving smoothness of video playback or reducing latency in VR experiences.

GPU Acceleration

Get performance optimized for Turing, Ampere, and future generations of NVIDIA GPU architectures that includes high-speed computation of accurate flow vectors with little impact on the CPU or GPU.


Performance

A chart showing performance of frame interpolation time

Frame Interpolation Time



A graph showing optical flow quality metrics

Optical Flow Quality metrics


Optical Flow Engine-Assisted Frame Rate Up-Conversion Library


Interpolated frames are generated in between the original frames to create a smoother image

NvOFFRUC interpolates new frames using optical flow vectors to double the effective frame rate of a video. The result is improved smoothness of video playback and perceived visual quality.

Learn More About Frame Rate Up-Conversion




Object Tracking for Intelligent Video Analytics


Optical Flow SDK 2.0 introduced an object tracker library based on optical flow, along with source code and ready-to-use API. In our experiments, the optical flow-based object tracker has been shown to reduce the GPU utilization by up to 80%, compared to some of the most popular algorithms without compromising the accuracy of tracking. Optical Flow SDK 3.0 introduces a DirectX12 Interface, forward and backward flow and a global flow vector.

Learn More About Optical Flow SDK
A flowchart showing object tracking for Intelligent Video Analytics




Video Frame Interpolation and Extrapolation


Optical flow can also be used very effectively for interpolating or extrapolating video frames in real time. This can be useful in improving the smoothness of video playback, generating slow-motion videos, or reducing the apparent latency in VR experience. Optical Flow functionality in Turing and Ampere GPUs accelerates these use cases by offloading the intensive flow vector computation to a dedicated hardware engine on the GPU silicon, thereby freeing up GPU and CPU cycles for other tasks. This functionality in hardware is independent of CUDA cores.

Learn More About Video Frame Interpolation and Extrapolation

Videos & Webinars

Get started developing with the Optical Flow SDK.

Download Now