NVIDIA Optical Flow SDK
Optical Flow SDK exposes the latest hardware capability of Turing GPUs dedicated to computing the relative motion of pixels between images. The hardware uses sophisticated algorithms to yield highly accurate flow vectors, which are robust to frame-to-frame intensity variations, and track the true object motion. Computation of these flow vectors is faster than most other available methods at comparable accuracy, with very little load on CPU or GPU, as the vectors are computed on dedicated hardware which is independent of the GPU’s CUDA cores.
NVIDIA Optical Flow SDK exposes a new set of APIs for this hardware functionality:
- C-API – Windows and Linux
- Windows – CUDA and DirectX
- Linux – CUDA
- Granularity: 4x4 vectors at ¼ pixel resolution
- Raw vectors – directly from hardware
- Pre-/post-processed vectors via algorithms to improve accuracy
- Accuracy: low average EPE (End-Point-Error) and outliers
- Performance: Up to 150 fps at 4K resolution*
- Robust to intensity changes
- OpenCV integration (GitHub)
By clicking the "Agree & Download" button, you are confirming that you have read and agree to be bound by the SOFTWARE DEVELOPER KITS, SAMPLES AND TOOLS LICENSE AGREEMENT for use of the SDK package. The download will begin immediately after clicking on the "Agree & Download" button.
Until a few years ago, tasks such as recognizing and tracking an object or classifying an action in video streams were out of reach for computers due to complexity involved. With the advent of DNNs and massive acceleration made possible by GPUs, all these tasks can now be automated. One of the most important applications of optical flow is to track objects within video frames.
The following diagram illustrates a network which uses optical flow for improving the accuracy of video action recognition:
Optical Flow also benefits many other use cases including: Stereo depth estimation, video frame interpolation and extrapolation. For example, Oculus uses NVIDIA optical flow for improving the VR experience (details)
Optical Flow functionality in Turing GPUs accelerates these use-cases by offloading the intensive flow vector computation to a dedicated hardware engine on the GPU silicon, thereby freeing up GPU and CPU cycles for other tasks. This functionality in hardware is independent of CUDA cores.
Our forum community is where Developers can ask questions, share experiences and participate in discussions with NVIDIA and other experts in the field.
Check out the forums here.
- NVIDIA Video Codec SDK
- DeepStream SDK for Intelligent Video Analytics
- Deep Learning Software
- GitHub - NVIDIA Optical flow in OpenCV
- GTC 2020 Session - NVIDIA Video Technologies: Video Codec and Optical Flow SDK [S21337]
- GTC 2020 Session - Using Video Codec SDK and Optical Flow SDK on NVIDIA GPUs Effectively [CWE21120]
- NVIDIA DevBlog - December 5, 2019 - Accelerate OpenCV: Optical Flow Algorithms with NVIDIA Turing GPUs
- NVIDIA Developer News - August 23, 2019 - Researchers at VideoGorillas Use AI to Remaster Archived Content to 4K Resolution and Above
- Oculus Developer blog - May 23, 2019 - ASW and Passthrough+ with NVIDIA Optical Flow
- NVIDIA DevBlog - February 13, 2019 - An Introduction to the NVIDIA Optical Flow SDK