GPU Memory Carveout Configuration and Tuning (CUDA and Graphics)#

Overview#

GPU workloads, including CUDA and graphics (e.g. EGL/OpenGL/Vulkan) clients, rely on a reserved GPU memory carveout. The carveout must be sized and reserved appropriately for the target use case: models, batch sizes, concurrency, graphics + compute mix, resolutions, number of surfaces, and effects/shaders.

If the carveout is undersized, GPU allocations can fail even when the system has free RAM.

Typical symptoms include:

  • For CUDA: cudaMalloc() failing with out-of-memory, i.e. returning cudaErrorMemoryAllocation.

  • For graphics: failures in GPU memory allocations for render targets, textures, or buffers (for example, errors from vkAllocateMemory, eglCreateWindowSurface, or driver-level OOM messages in logs).

Because CUDA and graphics share the carveout, pressure from either side (or both together) can exhaust the region.

When to Adjust the Carveout#

Increase (or re-tune) the carveout when you see one or more of the following:

  • Workload changes

    • Larger or additional CUDA models, higher batch size, more CUDA streams, or more CUDA clients.

    • Higher graphics load: increased resolution (for example, 1080p to 4K), higher frame rate, more layers/windows, more complex shaders, or additional GPU-using applications.

    • Enabling features that increase GPU memory usage (for example, more buffering, extra intermediate surfaces, multi-view rendering).

  • Concurrency changes

    • Increased overlap of CUDA and graphics work (for example, graphics UI running concurrently with heavy DNN inference).

    • Additional GPU clients (compute or graphics) running at the same time.

  • Fragmentation sensitivity

    • Workloads that do many allocations/frees on CUDA or graphics paths may need extra headroom to tolerate fragmentation and allocator behavior.

  • CUDA OOM at runtime

    cudaMalloc() returns cudaErrorMemoryAllocation under your target workload.

  • Graphics OOM or instability

    • Failures creating render targets, swapchain images, textures, or buffers.

    • Frame drops, compositor failures, or window-system errors that correlate with GPU memory usage.

How to Adjust the Carveout#

The procedure for configuring the GPU carveout depends on the operating system.

Linux#

On Linux, the GPU carveout is available only on Thor. Unlike the QNX configuration, this carveout is not defined as a dedicated device-tree reserved region. Instead, it is backed by hugetlbfs huge pages.

Each huge page provides 2 MB of physically contiguous memory, which can be used for GPU/GFX allocations. This carveout is used for GPU/GFX workloads that require or benefit from physically contiguous buffers, including performance-sensitive compression use cases.

The initial carveout size is established during boot by the nv_hugetlbfs_init.sh systemd script.

To adjust the carveout size, write the required number of 2 MB huge pages to the following sysfs node:

/sys/devices/system/node/node0/hugepages/hugepages-2048kB/nr_hugepages

For example, to configure a carveout of 2048 MB, set the huge-page count to 1024:

echo 1024 > /sys/devices/system/node/node0/hugepages/hugepages-2048kB/nr_hugepages

Note

nr_hugepages specifies a page count, not a size in bytes. The total carveout size is:

nr_hugepages x 2 MB

Note

As with QNX, the carveout is shared by CUDA and graphics clients. When sizing the carveout, account for the combined peak demand across all concurrent GPU workloads.

Optimization Guidance (Right-Sizing)#

  1. Start from measured peak usage.

    • Measure peak GPU memory usage under your real CUDA + graphics workload, including expected concurrency.

    • Size the carveout to cover this peak demand, including both:

      • CUDA allocations (model weights, activations, intermediate buffers, and similar allocations)

      • Graphics allocations (swapchain images, render targets, textures, intermediate surfaces, and similar allocations)

  2. Add headroom.

    • Add margin for:

      • Allocator behavior and fragmentation (both CUDA and graphics allocators)

      • Occasional spikes in demand (scene changes, temporary CUDA buffers, extra render passes)

      • Future growth in models, resolutions, effects, and new features

  3. Validate after changes.

    • Rerun the worst-case CUDA + graphics workload.

      • Confirm cudaMalloc() no longer returns cudaErrorMemoryAllocation.

      • Confirm graphics no longer shows OOM errors and that rendering remains stable (for example, no dropped surfaces, compositor failures, or allocation errors).

    • Include realistic concurrency (for example, UI + multiple applications + CUDA inference) in validation.

Carveout Default Sizing#

Keeping the default sizing to a minimum is motivated by the following factors:

  • DriveOS does not consume this carveout for its own internal usage. Instead, it prepares a contiguous buffer for application use and provides the required mechanisms and APIs. As a result, the carveout should ideally be kept as small as practical so that the visible post-boot memory footprint more closely matches the memory available to applications.

  • Many DriveOS customers fully customize DriveOS before deployment and prefer to maximize the amount of memory available to user applications by default. In principle, this suggests minimizing the carveout size. (There is a known issue in some cases when attempting to reduce the carveout size to zero.)

  • DriveOS also has performance and functional requirements that depend on the carveout, such as CB feature support, CUDA physical memory addressing range, and GFX performance benchmarks. These requirements establish a practical minimum carveout size for default testing and validation. The default sizing is therefore based on typical or average memory needs. Exceptional workloads and benchmarks with unusually high carveout requirements should be validated separately after configuring the carveout specifically for those use cases, rather than driving the default sizing.

Carveout Size Limits#

The platform defines minimum and maximum supported GPU carveout sizes. These are constrained by overall memory layout, alignment, and platform-specific requirements.

  • Minimum GPU carveout size: 6 GB

  • Maximum GPU carveout size: DRAM size - 8 GB (for DriveOS) - 4 GB for system cache and related usage

When configuring the carveout:

  • Do not reduce the carveout below the documented minimum.

  • Do not exceed the documented maximum, even if there appears to be free RAM in the system memory map.

  • Follow platform release notes and memory layout guidelines, and update this section with official values if they change.