NVIDIA DriveWorks SDK

DriveWorks 5 for the DRIVE Orin™ SoC comes with new major features. With the Compute Graph Framework (CGF) and System Task Manager (STM) Scheduler, DriveWorks becomes a versatile automotive middleware.

CGF enables developers to express their application as nodes of a directed acyclic graph (DAG). Nodes can be re-used; the graph structure provides an intuitive way to organize complex applications. Compute resource consumption of nodes is defined in a fine-grained fashion:

For scheduling, the graph is handed over to the STM scheduler. STM is a static, non-preemptive scheduler: STM’s compiler runs offline to derive an optimal schedule that execution will follow at runtime. It is designed to be a non-preemptive scheduler - where execution is not interrupted - enabling safety-critical applications and highly reliable deterministic behavior.

The Sensor Abstraction Layer (SAL) provides:

All DriveWorks modules are compatible with the SAL, reducing the need for specialized sensor data processing. The SAL supports a diverse set of Hyperion compatible sensors out of the box. Support for additional sensors may be added using the robust sensor plugin architecture.

DriveWorks provides a wide array of optimized low-level image and point cloud processing modules for incoming sensor data to use in higher level perception, mapping, and planning algorithms. Selected modules can be seamlessly run and accelerated on different DRIVE AGX hardware engines (such as Programmable Vision Accelerator or GPU), giving the developer options and control over their application. Image processing capabilities include feature detection and tracking, structure from motion, and rectification. Point cloud processing capabilities include lidar packet accumulation, registration, and planar segmentation (and more).

The VehicleIO module supports multiple production drive-by-wire backends to send commands to and receive status from the vehicle. In the event your drive-by-wire device is not supported out-of-the-box, the VehicleIO Plugin framework enables easy integration with custom interfaces.

The DriveWorks Deep Neural Network (DNN) Framework can be used for loading and inferring TensorRT models that have been independently trained. If DNNs are composed of layers that are not supported by TensorRT, plugins can be used to introduce custom layers and leverage the acceleration provided by the inference engine. The DNN Framework also enables inference acceleration using the DRIVE AGX’s integrated GPU.

Sensor Data Recording and Post-Recording Tools are a suite of tools to record, synchronize, and playback the data captured from multiple sensors interfaced to the NVIDIA DRIVE™ AGX platform. Recorded data can be used as a high quality synchronized source for training and other development purposes.

Calibration provides a reliable foundation for building AV solutions with high fidelity, consistent, up-to-date sensor data. It supports alignment of the vehicle’s camera, lidar, radar, and Inertial Measurement Unit (IMU) sensors that are compatible with the DriveWorks Sensor Abstraction Layer.

The DriveWorks Egomotion module uses a motion model to track and predict a vehicle’s pose. DriveWorks uses two types of motion models: an odometry-only model and, if an IMU is available, a model based on IMU and odometry.