At GDC 2024, NVIDIA announced that leading AI application developers such as Inworld AI are using NVIDIA digital human technologies to accelerate the deployment of generative AI-powered game characters alongside updated NVIDIA RTX SDKs that simplify the creation of beautiful worlds.
NVIDIA digital human technologies power next-generation game characters
You can incorporate the full suite of NVIDIA digital human technologies or individual microservices into your product offerings:
- NVIDIA Avatar Cloud Engine (ACE): Technologies that help bring digital humans to life, with facial animation powered by NVIDIA Audio2Face (A2F) and speech powered by NVIDIA Riva automatic speech recognition (ASR) and text-to-speech (TTS).
- NVIDIA NeMo: An end-to-end framework for developing custom generative AI. NeMo enables you to deliver enterprise-ready models with precise data curation, cutting-edge customization, retrieval-augmented generation, and accelerated performance.
- NVIDIA RTX: A collection of rendering technologies such as RTX Global Illumination (RTXGI) and DLSS 3.5 that enable real-time path tracing in games and applications.
These innovations can be seen in the Covert Protocol demo, created by Inworld AI in partnership with NVIDIA, where players take on the role of a skilled private detective and push the boundary of what NPC interactions could look like in games.
The demo uses OpenAI’s ChatGPT 3.5, ElevenLabs Text to Speech (TTS), Riva ASR, and A2F microservices to augment Inworld’s language, speech, and animation pipelines. Inworld’s AI engine takes a multimodal approach to NPC performance, bringing together cognition, perception, and behavior systems together while leveraging their Unreal Engine plugin to make integrating MetaHumans fast and simple.
You can start your journey on NVIDIA ACE by applying for the early access program to get in-development AI models. For more information about available models, see the NVIDIA API Catalog. On that site, you’ll find NVIDIA NIM, a set of easy-to-use microservices designed to accelerate the deployment of generative AI, for Riva ASR (the model name is Parakeet) and A2F. Experience and evaluate these microservices directly from a browser or through API endpoints running a fully accelerated stack. If you have an NVIDIA AI Enterprise license, download NIM and deploy on DGX Cloud or any CSP and private cloud.
For game developers looking for an end-to-end digital human solution, reach out to NVIDIA middleware partners that have integrated ACE including Convai, Inworld, Data Monsters, Quantiphi, and UneeQ.
For questions or feedback regarding digital human technologies, including NVIDIA ACE, see the Digital Human forum.
Improved lighting and easier rendering integrations powered by AI
NVIDIA has launched the RTX Global Illumination (RTXGI) 2.0 SDK, which enables ray-traced, indirect lighting with AI. RTXGI joins the ever-growing list of neural rendering technologies available today:
The latest algorithm, Neural Radiance Cache (NRC), is an AI-powered solution that trains the radiance cache about a scene in real-time, handling fully dynamic scenes with reactive lighting and without the need to bake static lighting for geometry and materials beforehand. It uses AI acceleration to train the radiance cache in real time to offer improved performance and more accurate global illumination in highly demanding scenes.
To offer scalability, NVIDIA introduced Spatial Hash Radiance Cache (SHaRC), a version that offers similar benefits as NRC without using a neural network. It is compatible with any DirectX or Vulkan ray tracing–capable GPU. It is a radiance cache that is built on a spatial hash data structure designed for a fast and scalable global illumination technique for path tracing.
Download the NVIDIA RTXGI SDK with all the available algorithms. For more information about all NVIDIA technologies for game developers, see Game Development and be sure to catch the full list of NVIDIA activities at GDC this week.
