The increasing demand for robotics is driving the need for physics-accurate simulation at an unprecedented scale. Universal Scene Description (OpenUSD) is key to this transformation, offering a powerful, open standard for building virtual worlds where robots learn.
This guide showcases three practical ways to supercharge your robotics development workflows using OpenUSD. We explore the following:
- Data ingestion: How to use OpenUSD data ingestion to unify fragmented CAD, Unified Robot Description Format (URDF), and sensor data into simulation-ready pipelines.
- Data aggregation: How OpenUSD’s composition enables massive virtual worlds that scale to hundreds of thousands of objects for unlimited training scenarios.
- SimReady: How the SimReady approach unifies your robotics pipeline with plug-and-play assets that work across the entire NVIDIA physical AI stack.
Whether you’re a seasoned robotics engineer or just starting, you’ll learn how this foundational technology can significantly reduce deployment time and improve robot simulation and training performance.
1. Data ingestion: expanding the robotics ecosystem
Data ingestion converts various data formats into OpenUSD, serving as a gateway to NVIDIA Isaac Sim and the NVIDIA robotics ecosystem.
Modern robotics projects are built on a complex foundation of disparate data sources, including CAD files, URDF descriptions for robot kinematics, live sensors, and IoT data. OpenUSD acts as the universal aggregator, unifying these sources into a single, cohesive format.
This unification:
- Enables advanced workflows like synthetic data generation, software in-the-loop testing of robotics algorithms, and reinforcement learning on frameworks such as Isaac Sim and NVIDIA Isaac Lab.
- Accelerates development by creating a common USD representation that streamlines your entire pipeline from design to AI training.
Apply it now to your workflows:
Several converters are useful for robotics workflows, including:
- Wandlebots OpenUSD library: Wandelbots NOVA includes an extensive library of annotated OpenUSD robot models from manufacturers such as FANUC, Yaskawa, Universal Robots, ABB, and KUKA.
- SICK virtual sensor models: Access certified digital twins of industrial LiDAR sensors, safety laser scanners, and vision sensors in OpenUSD format, ready for training simulations in Isaac Sim.
- Newton’s MuJoCo-USD Converter: Transform MuJoCo (MJCF) files into OpenUSD with physics, geometry, and material support.
Create a data pipeline to automate your MJCF file conversions to OpenUSD:
pip install mujoco-usd-converter
mujoco_usd_converter /path/to/robot.xml /path/to/usd_robot
2. Data aggregation: scale to massive virtual worlds
Data aggregation uses OpenUSD’s layer-based composition to combine modular, reusable assets from disparate sources into organized, scalable, and performant virtual worlds.
OpenUSD manages hundreds of thousands of objects within single environments, enabling large-scale robotic simulations where fleets of robots train, test, and optimize in realistic scenarios. By building modular, reusable assets—like warehouse shelves or factory robots—you create endless environment configurations. This accelerates AI model training, enhances synthetic data generation diversity, and produces more robust, versatile robot performance in real-world deployments.
Apply it now to your workflows:
- The Physical AI Warehouse OpenUSD Dataset on Hugging Face offers developers a head start with nearly 1,000 OpenUSD assets for warehouse robotics simulation and training.
- USD Search can help you manage large amounts of assets and locate what you need faster (even if the 3D data is unstructured and untagged) using AI-powered natural language or image queries.
Start aggregating assets into countless large virtual environments for training.
You can automate your scene construction by non-destructively referencing assets from the dataset in Python:
from pathlib import Path
from pxr import Usd
def ref_all_dataset_assets(root_dir: Path, stage: Usd.Stage):
for usd_file in root_path.rglob('*.usd'):
dir_name = usd_file.parent.name
file_name = usd_file.stem
if file_name == dir_name:
print(f"Found asset entry point: {usd_file}")
# Define a typeless prim
prim_path = f"/{dir_name}"
prim = stage.DefinePrim(prim_path)
# Add reference to the layer
prim.GetReferences().AddReference(str(usd_file))
Learn more about the USD Search API.
3. SimReady: unify your robotics pipeline with the broader ecosystem
SimReady assets are high-fidelity OpenUSD objects that incorporate physically accurate properties—materials, kinematics, and behaviors—making them immediately usable for realistic simulation, robotics, AI training, and digital twins.
Using a SimReady asset catalog streamlines your robotics pipeline by avoiding fragmentation and compatibility issues that plague ad-hoc 3D models. This standardization enables asset interoperability, reuse, and integration across simulation runtimes. SimReady assets are immediately usable in OpenUSD-powered frameworks like Isaac Sim, eliminating time-consuming asset preparation and conversion. This enables developers to focus on core value-add activities—training and simulation—while using the right tool for every pipeline stage.
Apply it to your workflows:
Lightwheel provides an extensive library of SimReady assets optimized for robot learning, imitation learning, and video-language-action (VLA) training methods with compatibility for research benchmarks. Powered by USD Search, developers can easily search SimReady assets based on color, kinematics, and physics data.
Get started by downloading assets from Lightwheel’s library or the NVIDIA open-source physical AI dataset, and try them within NVIDIA Isaac Sim.
Getting started
OpenUSD represents a paradigm shift in robotics development, moving from fragmented, tool-specific workflows to a unified, scalable, and interoperable ecosystem. By mastering data ingestion, using massive aggregated datasets, and embracing SimReady standards, robotics teams can accelerate their development cycles while building more robust, transferable AI systems ready for the real world.
NVIDIA provides a comprehensive collection of OpenUSD resources to accelerate your learning journey. Start with the self-paced Learn OpenUSD, Digital Twins, and Robotics training curricula that build the foundational skills covered in this guide.
For professionals ready to take the next steps in their robotics career, the OpenUSD Development certification offers a professional-level exam that validates your expertise in building, maintaining, and optimizing 3D content pipelines using OpenUSD. Headed to NVIDIA GTC Washington D.C.? Maximize your experience by taking the certification, in person, offered at no additional charge to conference attendees.
Tune in to upcoming OpenUSD Insiders livestreams and connect with the NVIDIA Developer Community. Stay up to date by following NVIDIA Omniverse on Instagram, LinkedIn, X, Threads, and YouTube.
Learn more about the research being showcased at CoRL and Humanoids, happening September 27-October 2 in Seoul, Korea. Also, don’t miss the keynote by NVIDIA CEO Jensen Huang at NVIDIA GTC Washington, D.C., on how breakthroughs in physical AI are powering the era of general robotics for every industry.
