Neural Reconstruction and 3D Gaussian Splatting with NVIDIA Omniverse NuRec
NVIDIA Omniverse™ NuRec is a set of accelerated 3D Gaussian splatting libraries that ingest real sensor data to reconstruct and render interactive simulation in OpenUSD. NuRec libraries are integrated with NVIDIA Isaac Sim™ for robotics testing, and NVIDIA AlpaSim and CARLA open source simulator for autonomous driving development.
How Omniverse NuRec Works: Real-Time 3D Reconstruction and Rendering
Omniverse NuRec reconstructs camera or lidar data of a given environment and packages the result as a USD scene, including metadata such as trajectories. The open NCore data standard provides a consistent way to represent data inputs to reconstruction workflows. Using open-source gsplat Gaussian splatting rendering libraries, NuRec then renders the scene interactively inside simulation frameworks like Isaac Sim or AlpaSim.
Rendering is usually done by connecting the simulator/runtime to the NuRec service via a gRPC API. Developers can then further enhance the reconstruction using generative models such as Fixer, which fills in artifacts or adjusts blurs, or Asset Harvester, which removes assets for use in other environments.
Enhance Neural Reconstructions with Fixer
Address artifacts and adjust blurs in reconstructed scenes using Fixer, built on NVIDIA Cosmos™.
Build Asset Libraries with Harvester for USD Scenes
Easily remove objects in a reconstruction for use in other scenes with Asset Harvester.
Reconstruct a 3D Scene Using Your Smartphone
Learn how to quickly turn smartphone captures into an interactive simulation environment in Isaac Sim using Omniverse NuRec.
Simulate Real-World Sensor Data in CARLA
Follow this step-by-step tutorial to turn real-world scenes into autonomous vehicle simulation in CARLA.
Get Started With Omniverse NuRec for Accelerated 3D Reconstruction
Omniverse NuRec on NGC for Reconstruction Pipelnes
Access containers, libraries, and models to start your reconstruction pipelines.
Download Omniverse NuRec
Explore NCore Data Standard for Sensor Data Reconstruction
Reconstruct sensor data using consistent data representations.
Access NCore Data Standard
Physical AI Open Datasets for Reconstructed Data
Kick-start your rendering pipelines with reconstructed data, openly available on the NVIDIA Physical AI Open Datasets collection.
Explore Open DatasetsStarter Kits
Neural Reconstruction and Rendering Starter Kit
Turn real-world sensor data into interactive simulation using 3D Gaussian splatting-based rendering for enhanced efficiency and accuracy.
Realistic Physics Simulation for Physical AI
Model the physical behavior of objects and systems foundational to physical AI.
Isaac Sim can accurately simulate rigid body and vehicle dynamics, multi-joint articulation, SDF colliders, and more.
Scalable Synthetic Data Generation for AI Model Training
Bootstrap AI model training with synthetic data.
Generate training data by randomizing attributes like lighting, reflection, color, and position of scene and assets.
More Resources
Ethical AI
NVIDIA believes Trustworthy AI is a shared responsibility and we have established policies and practices to enable development for a wide array of AI applications. When downloaded or used in accordance with our terms of service, developers should work with their supporting model team to ensure this model meets requirements for the relevant industry and use case and addresses unforeseen product misuse.
For more detailed information on ethical considerations for this model, please see the Model Card++ Explainability, Bias, Safety & Security, and Privacy Subcards. Please report security vulnerabilities or NVIDIA AI Concerns here.
Download NVIDIA Omniverse for NuRec for 3D Reconstruction