NVIDIA CloudXR™ SDK
NVIDIA CloudXR™ SDK
CloudXR is NVIDIA's solution for streaming virtual reality (VR), augmented reality (AR), and mixed reality (MR) content from any OpenVR XR application on a remote server—cloud, data center, or edge. The CloudXR streaming solution includes NVIDIA RTX™ hardware, NVIDIA RTX Virtual Workstation (vWS) drivers, and the CloudXR software development kit (SDK).Apply for access
Untethered XR Device Experience
High-fidelity, immersive VR applications require the use of a restrictive tethered HMD. CloudXR allows VR users to move to wireless HMDs without any noticeable degradation in the VR experience.
Large Wireless Ecosystem
Beyond the multitude of cellular phones and tablets, there are several OEMs working on 5G- and Wi-Fi-enabled VR All-in-One HMDs.
Streaming of Any OpenVR Application
Whether rendering complex graphics, stitching together 360-degree video, or rendering volumetric scenes, the CloudXR streaming stack lets developers leverage a flexible high-performance cloud instance as needed.
The CloudXR server application supports AR content by including an alpha channel in the left eye stream. This alpha channel is used to blend the rendered content with the live camera stream.
HOW IT WORKS
With CloudXR, VR/AR content can be streamed over wireless or wired networks to any end device, whether it’s untethered or tethered thin clients. CloudXR thin clients are supported on Windows and Android platforms.
The SDK provides a way to stream graphics-intensive XR content by accessing a high-powered graphics server and streaming over a radio signal (5G or Wi-Fi, for instance) to a relatively low-powered client, such as PC-connected head-mounted displays (HMDs) like Vive Pro or other Windows- or Android-based HMDs. The SDK also enables streaming of OpenVR applications to a number of 5G-connected Android devices, including 5G-enabled phones. This enables more mobile access to high-powered servers, as well as access to graphics-intensive applications on relatively low-powered graphics hardware.
- CloudXR Server Driver: Server-side binaries and libraries (Windows 10 or higher)
- CloudXR Client App: OS-specific binaries and libraries
- CloudXR Client SDK: Operating system (OS)-specific sample application (with source code sample)
The CloudXR Server provides a virtual HMD driver allowing the application to see the HMD as being locally connected thus not requiring any changes at the application level. The CloudXR server driver receives frames from the OpenVR application and subsequently encodes and transports the frames to the CloudXR Client Application.
On the client, the Cloud XR Client Application receives and decodes the frames and audio, and presents them to the end device runtime. This application also relays controller and HMD tracking data and inputs back to the server.
NATIVELY SUPPORTED HMDs
Head Mounted Displays
- OpenVR-capable HMDs tethered to NVIDIA Pascal™ or NVIDIA Turing™ based GPU (VIVE, VIVE Pro, VIVE Pro Eye,Valve Index)
- Untethered Android-based device (e.g., VIVE Focus Plus, Oculus Quest and Oculus Quest 2)
- Untethered Windows MR device (e.g. HoloLens 2)
- HMD partners wishing to support CloudXR can use the CloudXR Client SDK to create a device-specific application package (APK)
NVIDIA CloudXR 3.0 Delivers Support for Bidirectional Audio to Enhance Immersive Collaboration
The NVIDIA CloudXR 3.0 release provides bidirectional audio support to enhance collaboration in immersive environments ...Read More
NVIDIA and Google Cloud Deliver High-Quality XR Streaming
Discover how Masterpiece Studio Pro running on Google Cloud with CloudXR accelerates character creation in VR...Read More
Watch: Making Masterpieces in the Cloud With Virtual Reality
See how Masterpiece Studio Pro running on Google Cloud, and streaming with NVIDIA CloudXR, helps users enhance character creation in an immersive...Read More
Rich Immersive Experiences Across the Enterprise with NVIDIA CloudXR and ESI Immersive Cloud Technology [S21432]
We explore the challenges in designing user-centric, collaborative enterprise extended-reality experiences, and the architectural considerations in deploying a scalable virtualized infrastructure to deliver...Watch Video (requires login)
World's First Boundless XR over 5G Retail Experience
ZeroLight today announced one of the world's first commercial, boundless XR over 5G virtual reality (VR) experiences, created in collaboration with Qualcomm Technologies and NVIDIA. Built on state-of-the-art technologies — including NVIDIA CloudXR™...Read More
Power up your XR displays and 5G devices because NVIDIA is taking streaming to the next level. With the announcement today of the NVIDIA CloudXR 1.0 software development kit, we’re bringing major advancements to streaming augmented reality...Read More
Enterprises can now deliver virtual and augmented reality experiences across 5G networks to any device with the introduction of NVIDIA CloudXR. Built on NVIDIA GPU technology, the new NVIDIA CloudXR software development kit helps businesses create...Read More
Learn about the plans of market leaders in streaming VR and AR content from the cloud in this panel discussion. From enterprise use cases to streaming VR to the 5G edge, panelists will describe the state-of-the-art and challenges to making XR truly mobile.Learn More
A: All VR-Ready GPUs from the Pascal™ and later architectures are supported. If a laptop is used as a server, ensure that both SteamVR and the applications are running on the NVIDIA GPU. Streaming from an iGPU is not supported.
A: All GPUs that are based on the NVIDIA Pascal™ and later architectures are supported. On a laptop client, ensure that both SteamVR and the CloudXR Client application are running on the NVIDIA GPU. Streaming to an iGPU is not supported.
A: Yes, assuming that the vGPU driver is associated with a Windows 10 virtual machine.
A: This condition is still in testing.
A: The NVIDIA CloudXR server and client support all Microsoft Windows 10 and later releases, including the Windows 10 Server. The client additionally supports Windows 7.
A: For the server, the overlay window is rendered on the iGPU, so a MSHybrid server is not supported. For the client, the decode must occur on the dGPU, and in this case, a dual GPU system is supported.
A: For the NVIDIA CloudXR server driver to be activated, before you start SteamVR, ensure that the VR device, such as Vive, is disconnected from the system .
A: CloudXR has been tested with HoloLens2, HTC Vive, HTC Vive Pro, HTC Focus Plus, Oculus Quest, Oculus Quest 2 for Android client, and Valve Index for Windows client.
A: NVIDIA CloudXR supports any OpenVR-compatible application.
A: If the application queries the device parameters via OpenVR, and does not hard-code HMD parameters, it will work without modifications. CloudXR will forward the client device specifications, such as a resolution, a field of view, and the refresh rate.
A: Currently, NVIDIA CloudXR does not support protected content.
A: For details about licensing, see the EULA that is packaged with the SDK.
A: This is not currently supported.
A: The relationship between client and server is one-to-one. One client can only connect to one server at a time and vice versa.
A: Yes, for wireless networks with limited download speeds, and many potential sources of interference, the quality of a remote VR session might be impaired. We recommend 5GHz routers with beam forming for a more reliable connection.
A: For a Wi-Fi connection, the client requires download speeds of at least 50 Mbps, ideally at least 60 Mbps. Similarly, the server requires upload speeds of at least 50 Mbps, ideally at least 60 Mbps.
The CloudXR installer will open the ports that are associated with CloudXR in the Windows Firewall. If the connection cannot be established, turn Windows Firewall off temporarily and try again.
A: CloudXR use cases are AR and VR workflows that drive design review, virtual production, location-based entertainment, and more.
Ready to get started with CloudXR?Apply for access