Camera Stack#
Overview#
The Camera software stack provides comprehensive functionality for capturing and processing images from camera sensors connected to NVIDIA® DRIVE AGX platforms. The Camera software architecture is decomposed into multiple units and sub-elements that work together to provide a complete camera solution for automotive applications.
The Camera software stack provides C++ APIs to camera clients, enabling them to complete the following tasks:
Capture output from image sensors connected to NVIDIA DRIVE AGX platforms
Process captured images using NVIDIA Tegra ISP hardware
Control camera device drivers with access-controlled I2C and GPIO services
Utilize ISP settings and auto-control functionalities for optimized image quality
Access platform cryptographic resources for secure camera operations
All components of the Camera software target:
ISO 21434 (Road Vehicles - Cybersecurity Engineering) for Automotive Security Certification
ASPICE capability level - 1 (CL-1) for the Automotive Software Development Process
Architectural Decomposition#
The Camera software is decomposed into the following units and sub-elements:
The architecture diagram above illustrates the relationships and interactions between all Camera software components, including data flow paths, control interfaces, and hardware abstraction layers.
Common Unit#
The Camera SW - Common provides utility functions to the sub-elements and units of Camera software. It serves as a foundational layer that supports the other components with common functionality and shared resources.
Core Unit#
The Camera SW - Core is the public API provider of the Camera software. It provides the C++ APIs to the Camera Client to capture the output of the image sensor(s) connected to NVIDIA DRIVE AGX platforms and then process the captured images.
The Core unit serves as the primary interface between camera clients and the underlying camera hardware and processing capabilities.
Control Unit#
The Camera SW - Control provides ISP settings and auto control functionalities to the Camera software to optimize the quality of the processed images. This unit is responsible for the following tasks:
ISP (Image Signal Processor) configuration and tuning
Auto control algorithms for adaptive image processing
Image quality optimization based on environmental conditions
Device Block Unit#
The Camera SW - Device Block provides the functionalities to complete the following tasks:
Initialize and control camera device drivers
Provide access-controlled I2C and GPIO services
Manage services for platform cryptographic resources allocation and sharing
This unit acts as the hardware abstraction layer, providing controlled access to platform resources required by camera operations.
CamCap Sub-Element#
CamCap is a sub-element of Camera software that provides the functionalities to complete the following tasks:
Capture images from CSI (Camera Serial Interface) bus
Process the images using the NVIDIA Tegra ISP hardware
CamCap handles the low-level image capture and initial processing pipeline, interfacing directly with the camera hardware interfaces.
Note
Refer to CamCap SWAD for detailed architectural decomposition of CamCap.
CamPG Sub-Element#
CamPG is a sub-element of Camera software that provides the functionalities to complete the following tasks:
Generate ISP programming configurations
Decode ISP statistics
Initialize calibration data
CamPG is responsible for the intelligent programming of the ISP pipeline and extraction of image statistics for feedback to control algorithms.
Note
Refer to CamPG SWAD for detailed architectural decomposition of CamPG.
CDD-NVIDIA Sub-Element#
Camera Device Drivers (CDD) provide access to Camera hardware including the following components:
Camera modules (image sensors)
Deserializers
To support camera hardware from different vendors, the Camera Device Drivers are implemented as plugins and are loaded by the Camera software during initialization.
CDD-NVIDIA is a sub-element of Camera software that provides the Camera Device Driver implementation for the NVIDIA reference platform.
Note
Refer to CDD-NVIDIA SWAD for detailed architectural decomposition of CDD-NVIDIA.
Static Constraints#
Operating System and Platform#
The Camera software is targeted for Linux Standard on a single or two guest Virtual Machine(s).
Note
Even on systems with multiple virtual machines, all Camera devices are controlled by one virtual machine.
Plugins#
The Camera software supports the following components to be developed by System Integrators and plugged in:
Camera Device Drivers
Auto Control Algorithms
This plugin architecture allows for customization and integration of vendor-specific camera hardware and algorithms while maintaining a consistent interface.
GPIO Pins#
GPIO pins are shared across camera components, as the reference Tegra board does not have enough dedicated GPIO pins for all Sensors, Serializers, and Deserializers. Proper GPIO resource management is required to avoid conflicts.
Crypto Accelerator Resources#
The Camera software provides access to system crypto resources which can be used by camera drivers for cybersecurity operations. However, system crypto resources have the following constraints:
Limited number of hardware cryptographic engines physically available on the platform
Maximum total bandwidth of crypto operations which crypto engines can sustain
Limit to the number of hardware keyslots which can be active at the same time
Note
Refer to Security Engine Internal Architecture Specification for HW crypto engine performance specifications.
Data Flow#
The Camera software components interact as follows:
Camera Inputs: Image data enters the system through CamFSYNC, NvSIPL, and Auto Control interfaces
Core Processing: The Core unit receives requests from camera clients and coordinates the capture and processing pipeline
Control Loop: The Control unit processes auto-control inputs and generates ISP settings, working with CamPG for ISP programming
Device Management: Device Block units interface with hardware through FuSa UMD (Functional Safety - User Mode Driver)
Capture and Processing: CamCap captures images from the CSI bus and processes them through the Tegra ISP
Calibration: Load calibration data through the Tuna interface to CamPG for accurate image processing
Security: Crypto operations are performed through the Device Block for secure camera operations
Output: Processed images are delivered through CDI (Camera Data Interface) and DDI (Display Data Interface) outputs