Understanding Model Development
Model development for autonomous vehicles starts with data collected from fleet vehicles, curated and annotated with NVIDIA Cosmos™ Curator, and augmented with synthetic scenarios from NVIDIA Cosmos world foundation models. NVIDIA Cosmos Dataset Search retrieves rare and long-tail scenarios instantly, accelerating post-training data pipelines. NVIDIA Alpamayo VLA reasoning model is then fine-tuned on those datasets, with NVIDIA AlpaGym running closed-loop reinforcement learning post-training.
Tools for Data Processing and Searching AV Training Data
Cosmos Curator
NVIDIA Cosmos Curator is a framework for quickly filtering, annotating, and deduplicating large volumes of sensor data for physical AI development. It uses NVIDIA Cosmos Reason VLM to annotate and critique data, shortening processing pipelines from months to days.
Cosmos Dataset Search
The NVIDIA Cosmos Dataset Search vector search workflow retrieves relevant driving scenarios from massive datasets instantly. Powered by NVIDIA Cosmos Embed NIM™ for semantic search across billions of clips, it cuts rare scenario discovery from weeks to seconds.
CoC Autolabeling Pipeline
The NVIDIA CoC Auto-Labeling Pipeline automatically generates Chain of Causation (CoC) reasoning labels for driving clips. It identifies decision-making keyframes, labels driving decisions from a closed action set, and organizes cause-and-effect relationships into structured CoC traces.
Physical AI Dataset for Autonomous Vehicles
The NVIDIA Physical AI Dataset for Autonomous Vehicles is an open, large-scale multimodal driving dataset spanning over a thousand hours across dozens of countries. It includes CoC reasoning labels that provide causally grounded decision traces for training reasoning-based AV models.
Generate Photorealistic Synthetic Training Data
Cosmos Transfer 2.5
NVIDIA Cosmos Transfer 2.5 uses world foundation models to generate faster, better photorealistic world variations from text prompts and spatial controls. This expands real‑world driving data into rich scenario families.
Omniverse NuRec
NVIDIA Omniverse™ NuRec is a set of Gaussian-based libraries that reconstruct real-world driving scenes into interactive 3D environments. This enables new sensor views and scenario variations for synthetic training data generation.
Cosmos Evaluator
NVIDIA Cosmos Evaluator is an open framework that automatically scores and validates synthetic video generated by NVIDIA Cosmos, measuring physical accuracy and object fidelity to improve data quality for AV model training.
Cosmos 3
NVIDIA Cosmos™ 3 is the frontier open world foundation model for physical AI and the first full omnimodel to understand and generate text, image, video, ambient sound, and action. It unifies vision reasoning, world and action generation to accelerate vision analytics, synthetic data, simulation, and policy improvement.
Model Training and Optimization
Alpamayo 2 Super
NVIDIA Alpamayo 2 Super is an open 34-billion-parameter reasoning VLA model that reasons, plans, and acts across the full driving stack for safer, scalable Level 4 development.
Alpamayo 1.5 Nano
NVIDIA Alpamayo 1.5 Nano is an open 10B reasoning VLA model, RL post-trained for improved reasoning quality. It supports flexible multi-camera configurations, natural-language navigation guidance, and SFT and RL fine-tuning.
AlpaGym
NVIDIA AlpaGym is an open-source modular closed-loop reinforcement learning framework for training AV policy models at GPU scale. It helps expose compounding errors that open-loop, log-replay training leaves uncorrected.
TensorRT
The NVIDIA TensorRT™ ecosystem of tools for high-performance deep learning inference includes compilers, runtimes, and model optimizations that deliver low latency and high throughput for production applications.
TensorRT-LLM
The NVIDIA TensorRT-LLM open-source library optimizes large language and vision-language model inference on NVIDIA GPUs, with support for FP8 and NVFP4 quantization, in-flight batching, and paged attention. It’s the inference backend for Alpamayo 1.5.
Dynamo-Triton
NVIDIA Dynamo-Triton™ is an open-source model serving framework that deploys AI models across NVIDIA TensorRT, PyTorch, ONNX, and other major frameworks. It uses dynamic batching, concurrent model execution, and ensemble pipelines to run multiple AI models, including preprocessing and postprocessing on GPU.
Learn About NVIDIA Alpamayo
NVIDIA Alpamayo is an open family of vision-language-action models, simulation frameworks, and physical AI datasets for reasoning-based autonomous vehicle development.
- Reason through complex driving scenarios with Alpamayo 2 Super.
- Auto-generate causally grounded reasoning labels with CoC Auto-Labeling Pipeline.
- Simulate and evaluate AV policies in closed-loop with AlpaSim.
- Post-train AV policies with GPU-scale reinforcement learning with AlpaGym.
- Access the Physical AI Open Dataset, an open-source large, diverse multimodal driving dataset.
Model Development Learning Resources
Other DRIVE Resources
Simulation and Validation
Test and validate AV models in high-fidelity closed-loop simulation.
DRIVE AGX
Get developer kits and SDKs for in-vehicle compute development.
Downloads Center
Download DRIVE SDKs, open models, and simulation tools.
Support
Ask questions and get answers in our developer forum.