NVIDIA Optical Flow SDK

The NVIDIA® Optical Flow SDK exposes the latest hardware capability of NVIDIA Turing, Ampere, and Ada architecture GPUs dedicated to computing the relative motion of pixels between images. The hardware uses sophisticated algorithms to yield highly accurate flow vectors, which are robust to frame-to-frame intensity variations and track the true object motion.

Get started
Turing hardware generated optical flow map sample
Turing hardware generated optical flow map sample --- source footage

Accurate Video Analytics

Detect and track objects in successive video frames accurately while heavily reducing the computational complexity requirements.

Real-Time Performance

Interpolate or extrapolate video frames in real-time, improving smoothness of video playback or reducing latency in VR experiences

GPU Accelerated

Optimized for Turing, Ampere and future generations of NVIDIA GPU architectures. High speed computation of accurate flow vectors with little impact on the CPU or GPU.

Optical Flow Engine-Assisted Frame Rate Up Conversion Library

NVIDIA Optical Flow assisted frame rate up conversion (NvOFFRUC) interpolates new frames using optical flow vectors to double the effective frame rate of a video. The result is improved smoothness of video playback and perceived visual quality.

Learn More

Object Tracking for Intelligent Video Analytics

Optical Flow SDK 2.0 introduced an object tracker library based on optical flow, along with source code and ready-to-use API. In our experiments, the optical flow-based object tracker has shown to reduce the GPU utilization by up to 80%, compared to some of the most popular algorithms without compromising the accuracy of tracking. Optical Flow SDK 3.0 introduces a DirectX12 Interface, forward and backward flow and a global flow vector.

Learn More

Video Frame Interpolation and Extrapolation

Optical flow can also be used very effectively for interpolating or extrapolating the video frames in real-time. This can be useful in improving the smoothness of video playback, generating slow-motion videos or reducing the apparent latency in VR experience, as used by Oculus (details). Optical Flow functionality in Turing and Ampere GPUs accelerates these use-cases by offloading the intensive flow vector computation to a dedicated hardware engine on the GPU silicon, thereby freeing up GPU and CPU cycles for other tasks. This functionality in hardware is independent of CUDA cores..

Read News About Optical Flow

Optical Flow on Vulkan

Accelerated Motion Processing Brought to Vulkan with the NVIDIA Optical Flow SDK

We are excited to announce the availability of Optical Flow SDK 5.0, which adds support for generating optical flow in Vulkan applications.



Learn More
NvOFFRUC Explained

Harnessing the NVIDIA Ada Architecture for Frame-Rate Up-Conversion in the NVIDIA Optical Flow SDK

The NVIDIA Optical Flow SDK 4.0 is now available, enabling you to fully harness the new NVIDIA Optical Flow Accelerator on the NVIDIA Ada architecture with NvOFFRUC.



Learn More
Optical flow sdk 4.0

Technical Blog: AV1 Encoding and NvOFFRUC: Video Performance Boosts and Higher Fidelity on the NVIDIA Ada Architecture

Developers can effectively double video frame rates with the new NVIDIA Optical Flow assisted frame rate up conversion (NvOFFRUC) library, included in the latest Optical Flow SDK.



Learn More
GeForce RTX 30 Series GPUs: Ushering In A New Era of Video Content With AV1 Decode

NVIDIA Video Technologies in Ada

NVIDIA GPUs contain dedicated hardware for video encoding, decoding, JPEG sill image decoding and optical flow computation. This talk covers latest features supported by Ada GPUs as well as software updates such as new SDK features, use-cases etc.

Watch

Get started developing with Optical Flow SDK.

Download the SDK