NVIDIA Maxine SDK - Early Access Program

The Maxine Early Access Program is best suited for application developers from the following segments:

• Providers of video conferencing, unified communications services or communications platforms
• Providers of video streaming platforms or content delivery platforms
• Providers of online video editing services
• Application developers or content creators who would like to use SDKs to integrate with client side applications
• Or generally, if you'd like to integrate Maxine features into your backend infrastructure or client side software applications

Please apply to become a member of the program by checking the license agreement box below, then clicking the “Join now” button and completing the form. Note that you must be a registered NVIDIA developer in order to join the program.

Audio Effects SDK

Audio Super-resolution perf improvements [UPDATED]

• Model performance is improved, max streams increased by 50% for 16kHz to 48kHz models, max streams increased by 70% for 8kHz to 16kHz models

Support for ADA L4/L40 [NEW]

• Performance for L4 and L40 greatly increased as compared to GPUs from previous families (On an average, L4 has 1.75x perf of T4, on an average, L40 has 1.4x perf of A40)

Voice Font [NEW] [Early Access Only]

• This effect provides a zero shot voice conversion effect, converting the input voice to match the ref speaker’s voice while keeping linguistic information and prosody unchanged
• Feature can convert input audio to any speaker's voice using just 30 seconds of reference audio

Video Effects SDK

Triton support for AIGS [NEW]

• Enables dynamic batching, batched inference, and concurrent execution leading to higher throughput
• Multi-GPU support via Triton server
• Average 30.8% throughput gain, comparing Triton based vs native SDK usage, across T4, A10, A40, L4 and L40

Augmented Reality Effects SDK

Live Portrait [NEW]

• Animates a person's portrait photo using a driving video by matching the head movement and facial expressions in it
• Support both quality mode for higher visual fidelity, and performance mode for quicker run-time

Eye Contact [UPDATED]

• Enhanced naturalness of gaze by introducing micro eye movement and randomized “look away” including more robust occlusion handling and quality improvement - increased gaze stability
• 62% throughput gain on T4 and 83% throughput gain on L4, comparing Triton based vs native SDK usage
• Lower Latency in ADA GPUs: 36% lower in L4 v/s T4, 25% lower in L40 vs A40

3D Body Pose Estimation [UPDATED]

• Pose Estimation when only the upper body is in camera view, joint angle postprocessing. And multi-object tracking is supported on Linux.

Facial Expression Estimation [UPDATED]

• Head pose accuracy improvements - Reduced Mean Absolute error across Pitch, Yaw and Roll from 6.16 to 5.11, as measured on the BIWI dataset
• Head pose stability improvements - Reduced Std dev across Pitch, Yaw and Roll from 1.71 to 0.88, as measured on an internal test video dataset

Triton support for Eye Contact, Landmark Detection and Face Detection [NEW]

• Enables dynamic batching, batched inference, and concurrent execution leading to higher throughput
• Multi-GPU support via Triton server
• 29% throughput gain for Landmark Detection and 68.6% throughput gain for Face Detection comparing Triton based vs native SDK usage, across T4, A10, A40, L4 and L40