1. [Home ](/) [Omniverse](/omniverse) NuRec # Neural Reconstruction and 3D Gaussian Splatting with NVIDIA Omniverse NuRec NVIDIA Omniverse™ NuRec is a set of accelerated 3D Gaussian splatting libraries that ingest real sensor data to reconstruct and render interactive simulation in OpenUSD. NuRec libraries are integrated with [NVIDIA Isaac Sim](https://developer.nvidia.com/isaac/sim)™ for robotics testing, and [NVIDIA AlpaSim](https://github.com/NVlabs/alpasim) and [CARLA](https://carla.org/) open source simulator for autonomous driving development. [Download Omniverse NuRec](https://catalog.ngc.nvidia.com/orgs/nvidia/teams/nre/containers/nre-ga?version=latest "Download")[View Documentation](https://docs.nvidia.com/nurec/index.html "Documentation")[NuRec Forum](https://forums.developer.nvidia.com/c/omniverse/platform/nurec/752 "Forum") * * * ## How Omniverse NuRec Works: Real-Time 3D Reconstruction and Rendering Omniverse NuRec reconstructs camera or lidar data of a given environment and packages the result as a USD scene, including metadata such as trajectories. The [open NCore data standard](https://github.com/nvidia/ncore) provides a consistent way to represent data inputs to reconstruction workflows. Using open-source [gsplat](https://github.com/nerfstudio-project/gsplat) Gaussian splatting rendering libraries, NuRec then renders the scene interactively inside simulation frameworks like Isaac Sim or AlpaSim. Rendering is usually done by connecting the simulator/runtime to the NuRec service via a gRPC API. Developers can then further enhance the reconstruction using generative models such as [Fixer](https://huggingface.co/nvidia/Fixer), which fills in artifacts or adjusts blurs, or [Asset Harvester](https://huggingface.co/nvidia/asset-harvester), which removes assets for use in other environments. ![Flow diagram illustrating how Omniverse NuRec uses real sensor data for 3D reconstruction and interactive simulation rendering via Gaussian Splatting libraries.](https://developer.download.nvidia.com/images/nurec/public-sector-news-lilly-ari-1920x1280.png) ### Enhance Neural Reconstructions with Fixer Address artifacts and adjust blurs in reconstructed scenes using Fixer, built on NVIDIA Cosmos™. [Access Fixer Model](https://huggingface.co/nvidia/Fixer) ### Build Asset Libraries with Harvester for USD Scenes Easily remove objects in a reconstruction for use in other scenes with Asset Harvester. [Access Harvester](https://huggingface.co/nvidia/asset-harvester) ### Reconstruct a 3D Scene Using Your Smartphone Learn how to quickly turn smartphone captures into an interactive simulation environment in Isaac Sim using Omniverse NuRec. [Read Blog: Reconstruct 3D](https://developer.nvidia.com/blog/reconstruct-a-scene-in-nvidia-isaac-sim-using-only-a-smartphone/) ### Simulate Real-World Sensor Data in CARLA Follow this step-by-step tutorial to turn real-world scenes into autonomous vehicle simulation in CARLA. [Watch Tutorial (00:45)](https://www.youtube.com/watch?v=TRcFBSA-Lnk) * * * ## Get Started With Omniverse NuRec for Accelerated 3D Reconstruction ![decorative icon](https://developer.download.nvidia.com/icons/m48-workflow-complex.svg) ### Omniverse NuRec on NGC for Reconstruction Pipelnes Access containers, libraries, and models to start your reconstruction pipelines. [Download Omniverse NuRec](https://catalog.ngc.nvidia.com/orgs/nvidia/teams/nre/containers/nre-ga?version=latest) ![decorative icon](https://developer.download.nvidia.com/icons/m48-demo-topics.svg) ### Explore NCore Data Standard for Sensor Data Reconstruction Reconstruct sensor data using consistent data representations. [Access NCore Data Standard ](https://docs.nvidia.com/nurec/basics/get-data.html) ![decorative icon](https://developer.download.nvidia.com/icons/m48-neural-network-3.svg) ### Physical AI Open Datasets for Reconstructed Data Kick-start your rendering pipelines with reconstructed data, openly available on the NVIDIA Physical AI Open Datasets collection. [Explore Open Datasets](https://huggingface.co/collections/nvidia/physical-ai) * * * ## Starter Kits ### Neural Reconstruction and Rendering Starter Kit Turn real-world sensor data into interactive simulation using 3D Gaussian splatting-based rendering for enhanced efficiency and accuracy. - [Getting Started With Neural Rendering](https://docs.nvidia.com/nurec/index.html) - [How to Instantly Render Real-World Scenes in Interactive Simulation](https://developer.nvidia.com/blog/how-to-instantly-render-real-world-scenes-in-interactive-simulation/) - [3DGUT: Enabling Distorted Cameras and Secondary Rays in Gaussian Splatting](https://research.nvidia.com/labs/toronto-ai/3DGUT/) - [Reconstruct a Scene in NVIDIA Isaac Sim Using Only a Smartphone](https://developer.nvidia.com/blog/reconstruct-a-scene-in-nvidia-isaac-sim-using-only-a-smartphone/) - [Simulate Robotic Environments Faster With NVIDIA Isaac Sim and World Labs Marble](https://developer.nvidia.com/blog/simulate-robotic-environments-faster-with-nvidia-isaac-sim-and-world-labs-marble) ### Realistic Physics Simulation for Physical AI Model the physical behavior of objects and systems foundational to physical AI. Isaac Sim can accurately simulate rigid body and vehicle dynamics, multi-joint articulation, SDF colliders, and more. - [Physics Simulation Fundamentals](https://docs.omniverse.nvidia.com/isaacsim/latest/features/sensors_simulation/index.html) - [Getting Started Guides for Sensor Simulation](https://docs.omniverse.nvidia.com/isaacsim/latest/features/sensors_simulation/index.html) - [NVIDIA® PhysX®](/physx-sdk) - [Newton](/newton-physics) ### Scalable Synthetic Data Generation for AI Model Training Bootstrap AI model training with synthetic data. Generate training data by randomizing attributes like lighting, reflection, color, and position of scene and assets. - [Synthetic Data Generation Use Cases](https://docs.nvidia.com/tao/tao-toolkit/text/qat_and_amp_for_training.html#knowledge-distillation) - [Omniverse Replicator Getting Started Guide](https://docs.nvidia.com/tao/tao-toolkit/text/qat_and_amp_for_training.html#knowledge-distillation) * * * ## More Resources ![Decorative image representing Community](https://developer.download.nvidia.com/icons/m48-people-group.svg) ### Join the Community ![Decorative image representing Developer Newsletter](https://developer.download.nvidia.com/icons/m48-email-settings.svg) ### Sign Up for the Developer Newsletter ![Decorative image representing Developer Program](https://developer.download.nvidia.com/icons/m48-developer-1.svg) ### Join the NVIDIA Developer Program * * * ## Ethical AI NVIDIA believes Trustworthy AI is a shared responsibility and we have established policies and practices to enable development for a wide array of AI applications. When downloaded or used in accordance with our terms of service, developers should work with their supporting model team to ensure this model meets requirements for the relevant industry and use case and addresses unforeseen product misuse. For more detailed information on ethical considerations for this model, please see the Model Card++ Explainability, Bias, Safety & Security, and Privacy Subcards. Please report security vulnerabilities or NVIDIA AI Concerns [here](https://www.nvidia.com/en-us/support/submit-security-vulnerability/). Download NVIDIA Omniverse for NuRec for 3D Reconstruction [Download Omniverse NuRec](https://catalog.ngc.nvidia.com/orgs/nvidia/teams/nre/containers/nre-ga?version=latest)