Technology Debuts in StreamFX, a Popular OBS Plugin
The NVIDIA AR SDK opens up creative new options for live streaming, video conferencing, and gameplay. The SDK lets developers detect a face or the face landmarks, and obtain a 3D mesh of the face through a regular webcam.
Now available as an open beta, it enables several use cases:
- Tracking faces on the camera to identify one or several subjects on an image,
- Overlaying assets on a person to put costumes or effects on people,
- Controlling an animated or game character with your face and head movements.
The NVIDIA AR SDK utilizes the Tensor Cores on NVIDIA RTX GPUs to enable fast, high quality tracking through an AI network. This network can deliver accurate face tracking; face landmark tracking of face shape, lips, eyes, eyebrows and nose with 3 degrees of freedom; and an advanced face mesh (a 3D reconstruction of a human face and head pose) with 6 degrees of freedom.
StreamFX, a popular plugin for OBS, is releasing a new version that uses the SDK to create a smart camera that automatically crops and zooms in on the user, keeping them centered in the frame whenever they move.
“The new Face Tracking filter in StreamFX required complex math and machine learning that would have taken months if not years to tune to run on many machines. However thanks to NVIDIA’s AR SDK it was done in no time at all, and runs on any NVIDIA RTX hardware which is becoming more and more common thanks to the NVENC H.264 encoder on it. This has drastically reduced the necessary development time from years to just two weeks, and at no extra cost too,” said Michael Fabian ‘Xaymar’ Dirks, StreamFX Author.
Learn more or download the AR SDK now.
