NVIDIA Cosmos for Developers
NVIDIA Cosmos™ is a platform of state-of-the-art generative world foundation models (WFMs), guardrails, and an accelerated data processing and curation pipeline for autonomous vehicles (AVs) and robotics developers.
Build, evaluate, deploy, and simulate physical AI models faster while minimizing testing and validation risks in the real world.
How It Works
Cosmos WFMs accelerate physical AI development, helping developers augment datasets and post-train downstream world models for robots and autonomous vehicles.
Cosmos Predict generates next frames based on input to build datasets predicting various edge cases and serves as the foundation for all world models.
Cosmos Reason acts as a critic, using chain-of-thought reasoning to evaluate synthetic visuals and reward outcomes. It can also generate captions to speed up data curation.
Cosmos Transfer amplifies structured video across various environments and lighting conditions.
Developers can use the available PyTorch inference and post-training scripts along with model checkpoints. Cosmos NIM microservices are in development—Cosmos Predict NIM microservices are available here.
NVIDIA Cosmos World Foundation Models
A family of pretrained models for world generation as videos and world understanding for accelerating physical AI development. Available openly to developers on NGC, Hugging Face, and GitHub.
Cosmos Predict
For future world state generation or as a base for custom world models.
Input: Text or Image
Output: Video
- Framerate: 10FPS, 16FPS (default)
- Resolution: 720P (default), 480P
Cosmos Transfer
Multi-control net for fast, photorealistic video data augmentation.
Input: Segmentation maps, depth signals, HD maps or CG simulation videos.
Pair with: NVIDIA Omniverse
Output: Photorealistic world scenes
Cosmos Reason
World reasoning for synthetic data curation, robot decision-making, and runtime video analytics for AI agents.
Input: Video
Output: Chain-of-thoughts reasoning and text
Cosmos AV Samples
Post-trained Cosmos WFMs for autonomous vehicles for multiview generation and camera control.
Cosmos Curator
Filter, annotate, and deduplicate large datasets for physical AI development using advanced AI models and distributed computing.
Cosmos Guardrails
A set of guardrails, including a pre-guard to block harmful inputs and a post-guard to ensure safety and consistency in generations.
Cosmos Prompt Upsampler
Transform original input prompts into more detailed and enriched versions for higher-quality outputs from Cosmos WFMs.
Introductory Resources
Develop Custom Physical AI Foundation Models With NVIDIA Cosmos Predict-2
Cosmos Predict-2 is a suite of improved physical AI foundation models designed to generate realistic, physics-aware simulation data for training robots and AVs.
Reconstruct Real-World Scenes With NVIDIA Omniverse NuRec
Omniverse NuRec neural reconstruction libraries turn real world sensor data into 3D interactive simulation, accelerating robotics and AV development on platforms like Isaac Sim and CARLA.
Physical AI Common Sense and Real-World Understanding With Cosmos Reason
Cosmos Reason is a reasoning VLM for physical AI—built not just to see, but to reason. See how Cosmos Reason is developed, where it’s used, and how you can use openly available model checkpoints and scripts to run the model for physical AI tasks.
Starter Kits
Start solving physical AI challenges by developing custom world models with Cosmos or using Cosmos WFMs for downstream use cases. Explore implementation scripts, explainer blogs, and more how-to documentation for various stages of physical AI development.
Post-Training Cosmos WFMs
Cosmos WFMs are purpose-built for post-training. Use domain-specific datasets to build world models or post-train for different types of output, such as action generation for policy models.
Synthetic Data Generation
Build and deploy world models for infinite domain-specific synthetic data. Use NVIDIA Omniverse for physics-based conditioning.
Cosmos Learning Library
More Resources
Ethical Considerations
NVIDIA believes Trustworthy AI is a shared responsibility, and we have established policies and practices to enable development for a wide array of AI applications. When downloading or using this model in accordance with our terms of service, developers should work with their internal model team to ensure this model meets requirements for the relevant industry and use case and addresses unforeseen product misuse.
NVIDIA has collaborated with Google Deepmind to watermark generated videos from the NVIDIA API catalog.
For more detailed information on ethical considerations for this model, please see the System Card, Model Card++ Explainability, Bias, Safety & Security, and Privacy Subcards. Please report security vulnerabilities or NVIDIA AI concerns here.
Get Started With NVIDIA Cosmos Today