1. [Topics](/topics) [AI](/topics/ai) [AI Inference](https://deve/topics/ai/ai-inference) Workload Map # NVIDIA Workload Map NVIDIA Workload Map, an open source framework within [NVIDIA DSX OS](https://www.nvidia.com/en-us/data-center/products/dsx/os/)™, gives Kubernetes tools a standard way to understand the structure of AI workloads. It enables schedulers, controllers, and platforms to inspect and manage distributed training jobs, inference services, and custom resource types through a shared, declarative definition. Designed for developers building across diverse workload frameworks, Workload Map describes resource hierarchies, pod specifications, scaling parameters, and status in a reusable format. Its Go library lets tools read and update workloads through a common API, making it easier to support new workload types without maintaining separate integration code for each one. Workload Map is already used by KAI Scheduler, NVCF, NVIDIA Recovery (Heimdall), and Elotl Nova. [Get Started](https://github.com/dsx-ai-factory/workload-map "Get Started")[Documentation](https://github.com/dsx-ai-factory/workload-map/blob/main/docs/Technical%20Guide.md "Documentation") * * * ## How NVIDIA Workload Map Works Kubernetes workload types share common building blocks but organize them differently. Workload Map uses declarative definitions and JQ expressions to map those fields into a consistent structure that tools can interpret across frameworks. ![DynamoGraphDeployment as one workload with frontend, prefill, and decode services, each with its own pod spec and replica range, exposed through Workload Map as structure, replicas, status, and a gang group per service](https://developer.download.nvidia.com/images/workload-map/ai-for-dev-diagram-karta-dz-page-workload-map.svg) - **Workload Structure:** Describe root resources, child components, and pod templates so tools can understand how a workload’s roles and resources fit together. - **Scaling Information:** Map replica counts and scaling fields so tools can read each component’s intended scale through the same API, regardless of workload type. - **Workload Status:** Map workload-specific conditions to common states such as running, completed, and failed so tools can interpret progress consistently across workload types. - **Gang Scheduling:** Tell schedulers which pods belong together, so a job is admitted as one unit instead of landing halfway and waiting. This works for frameworks the scheduler has never seen, once they have a definition. **Example: A DynamoGraphDeployment** A DynamoGraphDeployment describes a multi-service inference graph as one workload. This example has a frontend, a prefill worker service, and a decode worker service, each with its own pod spec and autoscaling range. The green header represents the complete deployment, the light gray section groups its services, and the dark gray blocks represent individual services. Workload Map exposes this structure through the matching definition: per-service pod specs; replicas with min and max; a status of initializing, running, or failed; and a gang-scheduling group per service. ![DynamoGraphDeployment diagram](https://developer.download.nvidia.com/images/workload-map/ai-for-dev-diagram-karta-dz-page-dynamograph.svg) ![img-alt-text](https://developer.download.nvidia.com/images/workload-map/ai-for-dev-diagram-karta-dz-page-jobset.svg) ### Quick-Start Guide Run your first Workload Map example and explore how a common API reads and updates different Kubernetes workload types. [Get Started](https://github.com/dsx-ai-factory/workload-map#quick-start) ### Why Workload Map: The Workload Integration Problem Learn how declarative workload definitions reduce the repeated integration work that comes with supporting new Kubernetes frameworks. [Read the Blog](https://developer.nvidia.com/blog/support-new-ai-frameworks-faster-using-nvidia-workload-map) ### Discover More About Workload Map Explore the definition schema, Go API, and workload catalog to build integrations for your controllers, schedulers, and platforms. [Read Docs](https://github.com/dsx-ai-factory/workload-map/blob/main/docs/Technical%20Guide.md) * * * ## Get Started With NVIDIA Workload Map Explore an existing workload definition and bring a common workload API to your Kubernetes tooling. ![decorative icon](https://brand-assets.cne.ngc.nvidia.com/assets/marketing-icons/2.4.2/multi-gpus.svg) ### Get Workload Map Running Install the CLI to inspect workload structure, or add the Go library to your application. For integrations that store definitions in Kubernetes, install the CRD and apply one of the pre-built definitions (JobSet, PyTorchJob, RayCluster, KServe, Dynamo, and more) or write your own. [Install the CLI ](https://github.com/dsx-ai-factory/workload-map#install-the-cli) [Use the Go Library](https://github.com/dsx-ai-factory/workload-map#use-the-go-library) ![decorative icon](https://developer.download.nvidia.com/icons/m48-demo-topics.svg) ### Take a Deep Dive Into NVIDIA Workload Map Learn how Workload Map describes complex resource hierarchies, exposes pod templates and status, and enables reusable integrations across AI workload frameworks. [Explore the Guide](https://github.com/dsx-ai-factory/workload-map/blob/main/docs/Technical%20Guide.md) * * * ## NVIDIA Workload Map Starter Kits Explore custom workload definitions, examine real integration examples, and understand distributed AI workloads. Introductory setup resources stay in the page’s upper sections. ### Define Custom Workload Types Compare how declarative definitions describe different Kubernetes workload structures. Use the catalog and concrete JobSet and RayCluster mappings as references for your own workload type. - [Browse the Workload Definition Catalog](https://github.com/dsx-ai-factory/workload-map/tree/main/docs/catalog) - [Explore a JobSet Definition](https://github.com/dsx-ai-factory/workload-map/blob/main/docs/catalog/jobset-x-k8s-io-jobset-v1alpha2.yaml) - [Explore a RayCluster Definition](https://github.com/dsx-ai-factory/workload-map/blob/main/docs/catalog/ray-io-raycluster-v1.yaml) ### Build a Workload-Aware Tool Move beyond setup with a runnable controller example, inspect KAI’s workload-grouping integration, and explore how other platforms use Workload Map. - [Try the Controller-Runtime Example](https://github.com/dsx-ai-factory/workload-map/tree/main/docs/examples/controller-runtime) - [Inspect KAI’s Workload Map Integration](https://github.com/kai-scheduler/KAI-Scheduler/pull/1877) - [Explore Workload Map Adoption Examples](https://github.com/dsx-ai-factory/workload-map/blob/main/ADOPTERS.md) ### Understand Distributed AI Workloads Explore coordinated scheduling in Ray and component roles in Grove, then examine how a Workload Map definition describes a Dynamo inference graph. - [Gang Scheduling and Prioritization in Ray](https://developer.nvidia.com/blog/enable-gang-scheduling-and-workload-prioritization-in-ray-with-nvidia-kai-scheduler/) - [Streamline Complex Inference With Grove](https://developer.nvidia.com/blog/streamline-complex-ai-inference-on-kubernetes-with-nvidia-grove/) - [Explore the Workload Map Dynamo Definition](https://github.com/dsx-ai-factory/workload-map/blob/main/docs/catalog/nvidia-com-dynamographdeployment-v1alpha1.yaml) * * * ## Learning Library Tech Blog NVIDIA Run:ai Is Now Open Source as Part of NVIDIA DSX AI Factory Platform **NVIDIA Run:ai / Workload Map** Discover Workload Map’s role in open-source Run:ai: a common contract for understanding and managing Kubernetes AI workloads across frameworks. Tech Blog Advanced AI Workload Scheduling With KAI Scheduler **KAI Scheduler** Learn how KAI groups and schedules AI workloads. Explore scheduling concepts behind a documented Workload Map adopter. Video Introduction to NVIDIA Dynamo **NVIDIA Dynamo** Explore Dynamo’s distributed inference architecture and the component relationships that Workload Map helps Kubernetes tools understand. * * * ## More Resources ![Decorative image representing Community](https://developer.download.nvidia.com/icons/m48-people-group.svg) ### Join the DSX AI Factory Community | GitHub Discussions ![Decorative image representing Developer Newsletter](https://developer.download.nvidia.com/icons/m48-certification-ribbon-2.svg) ### Get Training and Certification ![Decorative image representing Developer Program](https://developer.download.nvidia.com/icons/m48-developer-1.svg) ### Join the NVIDIA Developer Program * * * ## Ethical AI NVIDIA believes Trustworthy AI is a shared responsibility and we have established policies and practices to enable development for a wide array of AI applications. When downloaded or used in accordance with our terms of service, developers should work with their supporting model team to ensure this model meets requirements for the relevant industry and use case and addresses unforeseen product misuse. For more detailed information on ethical considerations for this model, please see the Model Card++ Explainability, Bias, Safety and Security, and Privacy Subcards. Please report security vulnerabilities or NVIDIA AI Concerns [here](https://www.nvidia.com/en-us/support/submit-security-vulnerability/). Get started with NVIDIA Workload Map today. [Get Started](https://github.com/dsx-ai-factory/workload-map "Get Started")