NVIDIA Broadcast Engine
NVIDIA Broadcast Engine
AI-powered features for broadcasting, content creation and video conferencing
NVIDIA Broadcast Engine is a suite of SDKs that aim to provide AI-powered features for broadcasting, content creation and video conferencing. Developers can provide their customers with advanced AI effects such as noise removal, virtual background, or face tracking. These features are possible in real-time - without special cameras, microphones or other hardware - thanks to the power of NVIDIA RTX GPUs and their dedicated Tensor Cores.
This page contains information for developers. If you are an end user, you can check out the NVIDIA Broadcast app or the Broadcast Engine Redistributables for applications that have integrated these SDKs.
AUDIO EFFECTS SDK
Noise-free audio is crucial when hosting a live stream or podcast, or for video or audio conferencing. NVIDIA’s Audio Effects SDK uses AI to remove distracting background noise from your incoming and/or outgoing audio feeds, making conversation easier to understand.
VIDEO EFFECTS SDK
Apply incredible AI effects over video, without requiring special cameras or other hardware.
Virtual Background delivers real-time background removal of a webcam feed, so only your face and body show up on your livestream. It uses AI to separate the talent from the background without the hassle of a physical greenscreen.
Video Super Resolution
Video Super Resolution enhances video quality with AI. It can upscale content up to 4K in real time and remove encoding artifacts. The SDK offers a high performance and a high quality mode.
Enables real-time 3D tracking of a person’s face using a standard web camera. Developers can create fun, engaging augmented reality effects, such as overlaying 3D content on your face, allowing you to control 3D characters, or adding immersive effects that make you look like you are in the game. The SDK features include:
AI-based technology that detects human faces in images and video feeds. It determines the location of the face and specifies the size using a bounding box.
Face Landmark Tracking
AI-based technology that identifies the facial features and facial contour of a person, including face shape, lips, eyes, and eyelids, using 126 key points. Accelerated on tensor cores, it tracks 3 degrees of freedom head pose and facial deformation due to head movement and expressions.
GPU-accelerated optimization using a 3D morphable model to create a 3D mesh representation of a human face with up to 3000 vertices and 6 degrees of freedom head pose in real-time.
VIDEO CODEC SDK
Unlock the full potential of streaming apps with dedicated hardware encoding and decoding, reducing the FPS impact of streaming by up to 50%. With RTX GPUs, NVENC delivers up to 15% better efficiency for H.264 encoding, and 25% for HEVC encoding compared to prior generations.
"NVIDIA Broadcast Engine allows developers to tap into NVIDIA's RTX-powered AI effects and the XSplit team has been working closely with NVIDIA to get these enhancements into the hands of our users. In particular, we've integrated their Noise Removal technology from their Audio Effects SDK, enabling broadcasters to consistently produce a clean and noise-free stream for their viewers. We are committed to continuously bring value to our customers and look forward to future collaborations with NVIDIA in their innovations.” - Miguel Malvar Molina - Director of Developer Relations, XSplit.
"Our mission is to make IP video easy to use and fun for anyone to make a great show," said Dr. Andrew Cross, head of Research and Development for the Vizrt Group and NDI®. "Offering full NVIDIA GPU accelerated support for NDI allows gamers to use applications like OBS Studio to share gaming experiences with higher quality, performance, and frame-rates than ever previously possible."
“The speed and quality of the new (and easy-to-integrate!) AR SDK face tracker is incredible. Through the use of dedicated Tensor Cores, Notch users have access to sub-frame accurate face tracking with high-resolution cameras in a live environment. We're excited to see what this, as well as other NVIDIA Broadcast features, will bring to the live show and virtual production environment.” - Matt Swoboda, Founder and Director at Notch.