Training robot policies from real-world demonstrations is costly, slow, and prone to overfitting, limiting generalization across tasks and environments. A sim-first approach streamlines development, lowers risk and cost, and enables safer, more adaptable deployment.
The latest version of Isaac Lab 2.3, in early developer preview, improves humanoid robot capabilities with advanced whole-body control, enhanced imitation learning, and better locomotion. The update also expands teleoperation for data collection by supporting more devices, like Meta Quest VR and Manus gloves, to accelerate the creation of demonstration datasets. Additionally, it includes a motion planner-based workflow for generating data in manipulation tasks.
New reinforcement and imitation learning samples and examples
Isaac Lab 2.3 offers new features that support dexterous manipulation tasks, including dictionary observation space for perception and proprioception, and Automatic Domain Randomization (ADR) and Population Based Training (PBT) techniques to enable better scaling for RL training. These new features extend on environments implemented in DexPBT: Scaling up Dexterous Manipulation for Hand-Arm Systems with Population Based Training and Visuomotor Policies to Grasp Anything with Dexterous Hands.
To launch training for the dexterous environment, use the following script:
./isaaclab.sh -p -m torch.distributed.run --nnodes=1 --nproc_per_node=4
scripts/reinforcement_learning/rsl_rl/train.py --task Isaac-Dexsuite-Kuka-Allegro-Reorient-v0
--num_envs 40960 --headless --distributed
Expanding on prior releases, Isaac Lab 2.3 introduces new benchmarking environments with suction grippers, enabling manipulation across both suction and traditional gripper setups. The previous version included a surface gripper sample in the direct workflow. This update adds CPU-based surface gripper support to the manager-based workflow for imitation learning.
To record demonstrations with this sample, use the following command:
./isaaclab.sh -p scripts/tools/record_demos.py --task Isaac-Stack-Cube-UR10-Long-Suction-IK-Rel-v0
--teleop_device keyboard --device cpu
For more details, see the tutorial on interacting with a surface gripper.
Improved teleoperation for dextrous manipulation
Teleoperation in robotics is the remote control of a real or simulated robot by a human operator with an input device over a communication link, enabling remote manipulation and locomotion control.
Isaac Lab 2.3 includes teleoperation support for the Unitree G1 robot, with dexterous retargeting for both the Unitree three-finger hand and the Inspire five-finger hand.
Dexterous retargeting is the process of translating human hand configurations to robot hand joint positions for manipulation tasks. This allows efficient, human‑to‑robot skill transfer, improves performance on contact‑rich in‑hand tasks, and yields rich demonstrations to train robust manipulation policies.
The dextrous retargeting workflow takes advantage of the retargeter teleoperation framework built into Isaac Lab which enables per-task teleoperation device configuration.
Additional improvements have also been made to upper body control across all bimanual robots, like the Fourier GR1T2 and the Unitree G1. This has been done by improving the Pink IK (Inverse Kinematics) controller to keep bimanual robot arms in a more natural posture, reducing unnecessary elbow flare. New environments that allow for the robot to rotate its torso have been included in this release, to increase robots’ reachable space. Additional tuning has been done to improve speed and reduce errors in the end effector and end effector goal.
The Isaac Lab 2.3 release additionally includes UI enhancements for more intuitive usage. UI elements have been added to alert teleoperators of inverse kinematic (IK) controller errors, like at-limit joints and no-solve states. A pop-up has also been added to inform teleoperators when demonstration collection has concluded.
Introducing collision-free motion planning for manipulation data generation
SkillGen is a workflow for generating adaptive, collision-free manipulation demonstrations. It combines human-provided subtask segments with GPU-accelerated motion planning to enable learning real-world contact-rich manipulation tasks from a handful of human demonstrations.
Developers can use SkillGen within Isaac Lab Mimic to generate demonstrations in this latest version of Isaac Lab. SkillGen enables multiphase planning (approach, contact, retreat), supports dynamic object attachment and detachment with appropriate collision sphere management, and synchronizes the world state to respect kinematics and obstacles during skill stitching. Manual subtask “start” and “end” annotations separate contact-rich skills from motion planning segments, ensuring consistent trajectory synthesis for downstream users and reproducible results.
In previous releases, Isaac Lab Mimic used the MimicGen implementation for data generation. SkillGen has improved on limitations in MimicGen, and the Isaac Lab 2.3 release now enables you to use either SkillGen or MimicGen inside Isaac Lab Mimic.
To run the pipeline using a pre-annotated dataset for two stacking tasks, use the following commands. You can also download the dataset.
Use the following command for launching the vanilla cube stacking task:
./isaaclab.sh -p scripts/imitation_learning/isaaclab_mimic/generate_dataset.py \
--device cpu \
--num_envs 1 \
--generation_num_trials 10 \
--input_file ./datasets/annotated_dataset_skillgen.hdf5 \
--output_file ./datasets/generated_dataset_small_skillgen_cube_stack.hdf5 \
--task Isaac-Stack-Cube-Franka-IK-Rel-Skillgen-v0 \
--use_skillgen
Use the following command for launching the cube stacking in a bin task:
./isaaclab.sh -p scripts/imitation_learning/isaaclab_mimic/generate_dataset.py \
--device cpu \
--num_envs 1 \
--generation_num_trials 10 \
--input_file ./datasets/annotated_dataset_skillgen.hdf5 \
--output_file ./datasets/generated_dataset_small_skillgen_bin_cube_stack.hdf5 \
--task Isaac-Stack-Cube-Bin-Franka-IK-Rel-Mimic-v0 \
--use_skillgen
For information about prerequisites and installation, see SkillGen for Automated Demonstration Generation. For policy training and inference, refer to the Imitation Learning workflow in Isaac Lab. For details about commands, see the SkillGen documentation.
End-to-end navigation for mobile robots
Beyond manipulation, humanoids and mobile robots must navigate complex and dynamic spaces safely. Developers can now use the mobility workflow in Isaac Lab to post-train NVIDIA COMPASS, a vision-based mobility pipeline enabling navigation across robot types and environments. The workflow involves synthetic data generation (SDG) in Isaac Sim, mobility model training, and deployment with NVIDIA Jetson Orin or NVIDIA Thor. Cosmos Transfer improves synthetic data to reduce the sim-to-real gap.
By combining NVIDIA Isaac CUDA-accelerated libraries, a robot can know its location using cuVSLAM, learn how to build a map using cuVGL and understand the scene to generate action with COMPASS, enabling it to navigate changing environments and obstacles in real time. COMPASS also provides developers means to generate synthetic data for training advanced Vision Language Action (VLA) foundation models like GR00T N. ADATA, UCR and Foxlink are integrating COMPASS into their workflows.
Loco-manipulation synthetic data generation for humanoids
Loco-manipulation is the coordinated execution of locomotion and manipulation—robots move their bodies (walk or roll) while simultaneously acting on objects (grasping, pushing, pulling), treated as one coupled whole-body task.
This workflow synthesizes robot task demonstrations that couple manipulation and locomotion by integrating navigation with a whole-body controller (WBC). This enables robots to execute complex sequences, such as picking up an object from a table, traversing a space, and placing the object elsewhere.
The system augments demonstrations by randomizing tabletop pick and place locations, destinations, and ground obstacles. The process restructures data collection into pick and place segments separated by locomotion, enabling large-scale loco-manipulation datasets from manipulation-only human demonstrations to train humanoid robots for combined tasks.
An example of how to run this augmentation is shown below. Download the sample input dataset.
./isaaclab.sh -p \\
scripts/imitation_learning/disjoint_navigation/generate_navigation.py \\
--device cpu \\
--kit_args="--enable isaacsim.replicator.mobility_gen" \\
--task="Isaac-G1-Disjoint-Navigation" \\
--dataset ./datasets/generated_dataset_g1_locomanip.hdf5 \\
--num_runs 1 \\
--lift_step 70 \\
--navigate_step 120 \\
--enable_pinocchio \\
--output_file ./datasets/generated_dataset_g1_navigation.hdf5
The interface is flexible for users to switch to different embodiments, such as humanoids and mobile manipulators with the controller users choose.
Policy evaluation framework
Evaluating learned robot skills—such as manipulating objects or traversing a space—does not scale when limited to real hardware. Simulation offers a scalable way to evaluate these skills against a multitude of scenarios, tasks and environments.
However, from sampling simulation-ready assets, to setting up and diversifying environments, to orchestrating and analyzing large-scale evaluations, users need to hand-curate several components on top of Isaac Lab to achieve desired results. This leads to fragmented setups with limited scalability, high overhead, and a significant entry barrier.
To address this problem, NVIDIA and Lightwheel are co-developing NVIDIA Isaac Lab – Arena, an open source policy evaluation framework for scalable simulation-based experimentation. Using the framework APIs, developers can streamline and execute complex, large-scale evaluations without system-building. This means they can focus on policy iteration while contributing evaluation methods to the community, accelerating robotics research and development.
This framework provides simplified, customizable task definitions and extensible libraries for metrics, evaluation and diversification. It features parallelized, GPU-accelerated evaluations using Isaac Lab and interoperates with data generation, training, and deployment frameworks for a seamless workflow.
Built on this foundation is a library of sample tasks for manipulation, locomotion and loco-manipulation. NVIDIA is also collaborating with policy developers and benchmark authors, as well as simulation solution providers like Lightwheel, to enable their evaluations on this framework, while contributing evaluation methods back to the community.
For large‑scale evaluation, workloads can be orchestrated with NVIDIA OSMO, a cloud‑native platform that schedules and scales robotics and autonomous‑machine pipelines across on‑prem and cloud compute. Isaac Lab – Arena will be available soon.
Infrastructure support
Isaac Lab 2.3 is supported on NVIDIA RTX PRO Blackwell Servers, and on NVIDIA DGX Spark, powered by the NVIDIA GB10 Grace Blackwell Superchip. Both RTX PRO and DGX Spark provide an excellent platform for researchers to experiment, prototype, and run every robot development workload across training, SDG, robot learning, and simulation.
Note that teleoperation with XR/AVP and Imitation Learning in Isaac Lab Mimic are not supported in Isaac Lab 2.3 on DGX Spark. Developers are expected to have precollected data for humanoid environments, while Franka environments support standard devices like the keyboard and spacemouse.
Ecosystem adoption
Leading robotics developers Agility Robotics, Boston Dynamics, Booster Robotics, Dexmate, Figure AI, Hexagon, Lightwheel, General Robotics, maxon, and Skild AI are tapping NVIDIA libraries and open models to advance robot development.
Get started with Isaac Lab 2.3
Isaac Lab 2.3 accelerates robot learning by enhancing humanoid control, expanding teleoperation for easier data collection, and automating the generation of complex manipulation and locomotion data.
To get started with the early developer release of Isaac Lab 2.3, visit the GitHub repo and documentation.
To learn more about how Isaac Lab extends GPU-native robotics simulation into large-scale multimodal learning to drive the next wave of breakthroughs in robotics research, see Isaac Lab: A GPU-Accelerated Simulation Framework For Multi-Modal Robot Learning.
Learn more about the research being showcased at CoRL and Humanoids, happening September 27–October 2 in Seoul, Korea.
Also, join the 2025 BEHAVIOR Challenge, a robotics benchmark for testing reasoning, locomotion, and manipulation, featuring 50 household tasks and 10,000 tele-operated demonstrations.
Stay up to date by subscribing to our newsletter and following NVIDIA Robotics on LinkedIn, Instagram, X, and Facebook. Explore NVIDIA documentation and YouTube channels, and join the NVIDIA Developer Robotics forum. To start your robotics journey, enroll in free NVIDIA Robotics Fundamentals courses.
Get started with NVIDIA Isaac libraries and AI models for developing physical AI systems.
