Platform Software Stacks

This section describes the platform component software stacks.

Use the NVIDIA DRIVE® OS software stack to build your autonomous vehicle applications.

Foundation Services Stack

The NVIDIA DRIVE AGX Orin platform Foundation services runtime software stack provides the infrastructure for all the components of the platform. With this infrastructure, guest OS systems can run on the hardware, with the Hypervisor managing use of hardware resources.


Foundation Components

Component Description

Trusted Software server that separates the system into partitions. Each partition can contain an operating system or a bare-metal application. The Hypervisor manages:

  • Guest OS partitions and the isolation between them.

    Note: Multiple Guest OSs are not supported. In addition, you can run QNX or Linux, but not both.
  • Partitions' virtual views of the CPU and memory resources.

  • Hardware interactions

  • Run-lists

  • Channel recovery

Hypervisor is optimized to run on the ARMv8.2 Architecture.

guest OS

Allocates peripherals that Guest OS needs to control.


Services for DRIVE Update.


Firmware that runs during boot to load firmware components, such as boot images, partition images, and other firmware.

Orin SoC

System on a Chip hardware resources.

Virtualized Servers

The virtualized configurable servers are as follows.

Component Description
Boot and Power Manager Processor (BPMP) Server

Facilitates communication between Guest Virtual Machines (VM) and BPMP firmware.

VM Server

host1X virtualizes NvHost.

Security Engine (SE) Server

Para-virtualizes and allows multiple Guest Virtual Machines access to the security engine cryptographic hardware accelerator.

Trusted Applications (TA)

Through Trusted Applications, Trusted OS exposes a set of core services that use managed security assets in cryptographic operations without exposing them to non-secure guest software.


Runs on top of the virtualization core and handles sharing of GPU between multiple client Guest Virtual Machines. The GPU virtualization also includes the para-virtualized GPU client driver running inside of each Guest Virtual Machine.

Storage Server

Para-virtualizes storage access to enable sharing physical storage devices among multiple Guest Virtual Machines.

Debug Server Provides support for kernel-level debugging of Guest Virtual Machines (VM).

NvMedia Architecture

NvMedia provides powerful processing of multimedia data for true hardware acceleration across NVIDIA DRIVE Orin devices. With the NvMedia and Orin firmware components, multimedia applications support multiple simultaneous camera feeds for simultaneous processing. The NvMedia features include:

  • Robust image processing and sensor control blocks for use by applications.
  • Hardware surface to handle all types of image formats such as RGB, YUV and RAW.
  • Functional components for image capture, processing, acceleration, encoding, and interoperating with other libraries.

Applications leverage the NvMedia Application Programming Interface (API) to process image and video data. Additionally, NvMedia can route image data to and from other components, such as OpenGL ES and NVIDIA CUDA®.

NvMedia Stack

The NvMedia software stack supports these types of interactions:

  • Applications call the NvMedia Framework components to string together a sequence of processing steps for images.
  • The NvMedia Framework calls low-level hardware drivers to interact with the SoC components on the SoC chip.

Consult building and running NvMedia sample applications to build an NvMedia sample application.


Each component functionality is as follows. The drivers expose a subset of the functionality available.

Tegra Hardware


Video Input (VI)

Receives CSI data from the camera.

Image Signal Processor


Produces a processed image from image data captured from an image sensor. For example, it can make pixel-level changes such as inverting pixel bits, auto exposure, and white balance correction.


Converts raw image data into one of the supported image formats.


Converts encoded image data into raw image data.

Video Interlace Compositor (VIC)

Converts video data for deinterlacing, composition, and format conversion.

Optical Flow Accelerator (OFA)

Accelerate optical flow and stereo disparity computation between the frames.

NvMedia Components

The NvMedia components are as follows:

Image Component



The SIPL framework processes incoming image data with AE, AWB controls and includes the ISP processing capability. It supports camera tuning tools.


Image 2D provides the ability to perform manipulation of image data; such as cropping, scaling, copying, and converting the format.


Image Encode Processing provides the ability to encode processed YCRCb surface inputs to H.264 and H.265.


Image JPEG Encoding provides the ability to encode YUV surfaces to JPEG format.


Image JPEG Decoding provides the ability to decode images compressed in JPEG format to raw YUV surfaces.

LDC Image LDC provides the ability to perform lens distortion correction and temporal noise reduction on image data.

NvMedia APIs and Thread Safety

NvMedia APIs are not designed to be thread safe. It is the responsibility of the application to maintain thread safety. To ensure thread safety:

  • NvMedia components can be created and used in any thread, but the APIs cannot be used from different threads concurrently.
  • Different instances of the same component can be used in parallel from different threads.
  • Encoders are designed to be fed from one thread and receive the encoded bitstream from another thread.