NVIDIA CloudXR for Apple Platforms
NVIDIA and Apple have built a high-performance "bridge" that natively enables NVIDIA CloudXR 6.0 experiences on visionOS, iOS, and iPadOS. This solution enables developers and users to stream the world’s most graphically demanding games, simulations, and professional 3D applications—powered by NVIDIA RTX™—directly to Apple devices from a local workstation or the cloud. For Apple Vision Pro, this integration leverages privacy-preserving dynamic foveated streaming— prioritizing the transmission of pixels based on the approximate region where the user is looking—to deliver uncompromised 4K resolution and ultra-low latency while ensuring that sensitive gaze data remains strictly protected at the streaming level.
How CloudXR for Apple Platforms Works
NVIDIA CloudXR™ for Apple Platforms is NVIDIA’s Swift framework for building native client apps on Apple devices that stream spatial experiences from OpenXR-compatible server applications (built with CloudXR Runtime). The framework uses a low-latency stream protocol and provides a modern SwiftUI-based API for integrating spatial streaming into your visionOS, iOS, and iPadOS applications.
Two frameworks enable this capability: (1) Standalone CloudXRKit SDK and (2a) FoveatedStreaming or (2b) StreamingSession framework
- CloudXRKit: NVIDIA's standalone SDK for streaming spatial content. The framework is bundled directly with your application. You ship CloudXRKit as part of your app binary, giving you full control over the framework version and features.
FoveatedStreaming (visionOS): Apple's session-based API for establishing connections from Apple Vision Pro to local and cloud streaming endpoints that wraps CloudXRKit technology within visionOS. It enables dynamic foveated streaming to Apple Vision Pro in a secure, privacy-preserving manner, allowing users to perceive a high-fidelity spatial experience while reducing GPU and network costs. See Apple Developer website for more detailed information.
- StreamingSession.xcframework (iOS, iPadOS): Apple’s framework to stream an OpenXR experience to an iPhone or iPad. StreamingSession.xcframework has an API similar to that of FoveatedStreaming.framework on visionOS, making it easy to build cross-platform streaming applications.
NVIDIA CloudXR for Apple Platforms delivers real-time, server-rendered spatial content to Apple Vision Pro, iPhone, or iPad. An OpenXR application on the server uses the NVIDIA CloudXR Runtime to render, capture, encode, and transmit compressed video, depth, alpha data, and spatial audio. The client application, using the CloudXR Framework, receives, hardware-decodes, and displays the stream. Concurrently, the client captures and sends back head pose and controller input—and, for visionOS specifically, additionally hand tracking and the approximate focus area of the user—completing a low-latency feedback loop for the OpenXR application to render the subsequent frame correctly.
.svg)
When leveraging FoveatedStreaming on visionOS, developers and users benefit from the same core streaming, with dynamic foveated streaming support using a secure, layered architecture. The OpenXR application uses Stream Manager for orchestration and coordinates with the NVIDIA CloudXR Runtime for frame capture, encoding, and transmission. On the client, the application uses Apple's Swift FoveatedStreaming framework, which interfaces with a visionOS system service for the streaming and, in turn, communicates with the embedded NVIDIA Client Framework.
This architecture is intentionally designed to protect user privacy. To enable high-quality streaming, your Vision Pro also shares with the host the approximate location inside Vision Pro where your eyes are looking. This ensures that the content you see is rendered at a higher resolution where you’re looking, while the peripheral content is shown with less detail. Information about where you’re looking isn’t shared with the visionOS app you’re using or the application on the host. This data is intended to be used by the CloudXR Runtime only for the duration required to send the content. See the Apple Developer website for more details.
.svg)
Which Framework Should You Choose?
Capability | Streaming Session or CloudXRKit (Standalone) | FoveatedStreaming |
|---|---|---|
Underlying Technology | CloudXRKit | CloudXRKit (system-embedded) |
Recommended Apple Platform | iOS, iPadOS, visionOS | visionOS |
Framework Location | Bundled in iOS / iPadOS / visionOS app | Embedded in the operating system (visionOS) |
Dynamic Foveated Streaming | ❌ No | ❎ Yes, with user consent |
Minimum OS | OS/iPadOS 18.0+, visionOS 2.4+ | visionOS 26.4+ |
visionOS Support | ⚠️ Yes, limited functionality | ❎ Yes |
App Binary Size | Larger (includes framework) | Smaller (uses system framework) |
Framework Updates | Developer-controlled | Automatic with OS updates |
NVIDIA Cloud Hosting (GDN) | ❎ Yes | ❌ No |
Public Cloud Hosting | ❎ Yes | ❎ Yes |
Local Network Streaming | ❎ Yes | ❎ Yes |
Local Authentication | ❎ Custom | ❎ QR-code pairing |
Data Channels | ❎ Yes | ❎ Yes |
Troubleshoot Statistics | ❎ Yes (Raw metrics and HUD) | ❎ Yes (per-frame real-time diagram in Xcode) |
Choose FoveatedStreaming to benefit from:
Dynamic, eye-tracking optimized foveated streaming
Smaller app binary (no bundled framework)
Automatic framework updates via visionOS
Simplified device pairing (built-in QR codes)
Apple's officially supported API
Choose StreamingSession or CloudXRKit (standalone) if you require:
NVIDIA GDN cloud streaming (as opposed to NVCF).
Support for visionOS versions before 26.4.
Control over the framework version included with your app.
Detailed network statistics and quality metrics.
Customized authentication.
Still can’t decide?
CloudXRKit and FoveatedStreaming can be used together in the same Xcode project via conditional compilation.
Both frameworks are built on CloudXR, sharing similar APIs for session management, connections, and data channels, making migration or dual support straightforward via API translation.
How to Get Started with CloudXR for Apple Platforms (CloudXR Framework)
CloudXR is an end-to-end streaming system that requires both components to function. A server application using CloudXR Runtime must be paired with a compatible client app; neither can operate independently. Both your server and client must be properly configured to establish a working streaming session.
Download the CloudXRKit from GitHub:
Step 1: Run OpenXR Server Application
git clone https://github.com/NVIDIA/cloudxr-lovr-sample # run and build server ./run.bat
Step 2: Download SDK and Client Sample
git clone https://github.com/NVIDIA/cloudxr-apple-generic-viewer Add CloudXR Framework (https://github.com/NVIDIA/cloudxr-framework) as a package dependency
Step 3: Build and Run an Example
Build and run the sample to launch it in the simulator. Enter your server’s IP address and connect.
CloudXR Runtime
CloudXR Runtime is required on the server and is distributed through NVIDIA NGC (NVIDIA GPU Cloud).
To download:
Go to the CloudXR Runtime page on NGC.
Sign in with your NVIDIA account.
Download the SDK for your target platform:
To get started: Refer to CloudXR Runtime for detailed instructions, or follow the LÖVR CloudXR sample for a quick start.
Foveated Streaming Framework
The Foveated Streaming framework provides a session-based API for establishing connections from Apple Vision Pro to local and cloud streaming endpoints. Foveated Streaming Framework is an Apple-built system framework exclusively for visionOS.
The Multi-platform Foveated Streaming App template is available within Xcode. It is recommended to leverage this template, which will select the optimal CloudXR reference based on the platform you are targeting.
Alternatively, you can manually add the Foveated Streaming Framework to your Xcode Project: in Xcode, import FoveatedStreaming in the project Swift file.
To get started: Refer to FoveatedStreaming Framework for detailed instructions, or follow the FoveatedStreaming tutorial for a quick start.
For more information, see the Apple FoveatedStreaming documentation.
CloudXR Framework
CloudXR Framework (standalone CloudXRKit) is distributed as an open-source Swift package on GitHub.
To add CloudXR Framework to your Xcode Project:
In Xcode, go to File → Add Package Dependencies.
Enter the repository URL: https://github.com/NVIDIA/cloudxr-framework.
Select the latest version.
Add the package to your project targets.
To get started: Refer to CloudXR Framework for detailed instructions, or follow the CloudXR Framework tutorial for a quick start.
Get Started with Spatial Streaming for Apple Platform today