The NVIDIA® DriveWorks SDK is the foundation for all autonomous vehicle (AV) software development. It provides an extensive set of fundamental AV capabilities, including processing modules, tools, and frameworks that are required for advanced AV development.
With the DriveWorks SDK, developers can begin innovating their own AV solution, instead of spending time developing basic low-level functionality. DriveWorks is modular, open, and readily customizable. Developers can use a single module within their own software stack to achieve a specific function, or use multiple modules to accomplish a higher-level objective.
DriveWorks is suited for the following objectives:
- Integrate automotive sensors within your software.
- Accelerate image and Lidar data processing for AV algorithms.
- Interface with a vehicle’s ECUs and receive its state.
- Accelerate neural network inference for AV perception.
- Capture and post-process data from multiple sensors.
- Calibrate multiple sensors with precision.
- Track and predict a vehicle’s pose.
Sensor Abstraction Layer
The Sensor Abstraction Layer (SAL) provides: List of supported sensors
- A unified interface to the vehicle’s sensors.
- Synchronized timestamping of sensor data.
- Ability to serialize the sensor data for recording.
- Ability to replay recorded data.
- Image signal processing acceleration via Xavier SoC ISP hardware engine.
All DriveWorks modules are compatible with the SAL, reducing the need for specialized sensor data processing. The SAL supports a diverse set of automotive sensors out of the box. Additional support can be added with the robust Sensor Plugin architecture.
These Sensor Plugins provide flexible options for interfacing new AV sensors with the SAL. Features including sensor lifecycle management, timestamp synchronization, and sensor data replay can be realized with minimal development effort
- Custom Camera Feature: DriveWorks enables external developers to add DriveWorks support for any image sensor using the GMSL interface on DRIVE AGX Developer Kit.
- Comprehensive Sensor Plugin Framework: This enables developers to bring new Lidar, Radar, IMU, GPS, and CAN-based sensors into the DriveWorks SAL that are not natively supported by DriveWorks.
Watch Webinar Recording
Image and Point Cloud Processing
DriveWorks provides a wide array of optimized low-level image and point cloud processing modules for incoming sensor data to use in higher level perception, mapping, and planning algorithms. Selected modules can be seamlessly run and accelerated on different DRIVE AGX hardware engines (such as PVA or GPU), giving the developer options and control over their application. Image processing capabilities include feature detection and tracking, structure from motion, and rectification. Point cloud processing capabilities include lidar packet accumulation, registration and planar segmentation (and more). New and improved modules are delivered with each release.
Watch Webinar Recording
The VehicleIO module supports multiple production drive-by-wire backends to send commands to and receive status from the vehicle. In the event your drive-by-wire device is not supported out of the box, the VehicleIO Plugin framework enables easy integration with custom interfaces.
The DriveWorks Deep Neural Network (DNN) Framework can be used for loading and inferring TensorRT models that have either been provided in the DRIVE Networks library or have been independently trained. The DNN Plugins module enables DNN models that are composed of layers that are not supported by TensorRT to benefit from the efficiency of TensorRT. The DNN Framework enables inference acceleration using integrated GPU (in Xavier SoC), discrete GPU (in DRIVE AGX Pegasus), or integrated DLA (Deep Learning Accelerator in Xavier SoC).
In order to improve developer productivity, DriveWorks provides an extensive set of tools, reference applications, and documentation including:
- Sensor Data Recording and Post-Recording Tools: A suite of tools to record, synchronize and playback the data captured from multiple sensors interfaced to the NVIDIA DRIVE™ AGX platform. Recorded data can be used as a high quality synchronized source for training and other development purposes.
If a sensor’s position or inherent properties deviate from nominal/assumed parameters, then any downstream processing may be faulty, whether it’s for self-driving or data collection. Calibration gives you a reliable foundation for building AV solutions with the assurance that they’ll achieve high fidelity, consistent, up-to-date sensor data. Calibration supports calibration of the vehicle’s camera, LIDAR, radar, and Inertial Measurement Unit (IMU) sensors that are compatible with the DriveWorks Sensor Abstraction Layer.
Static Calibration Tools: Measures manufacturing variation for multiple AV sensors to a high degree of accuracy. Camera Calibration includes both extrinsic and intrinsic calibration, while the IMU Calibration Tool calibrates vehicle orientation with respect to the coordinate system.
Self-Calibration: Provides real-time compensation for environmental changes or mechanical stress on sensors caused by events such as changes in road gradient, tire pressure, vehicle passenger loading, and other minor changes. It corrects nominal calibration parameters (captured using the Static Calibration Tools) based on current sensor measurements in real-time, meaning that the algorithms are performant, safety-compliant, and optimized for the platform.
The DriveWorks Egomotion module uses a motion model to track and predict a vehicle’s pose. DriveWorks uses two types of motion models: an odometry-only model and, if an IMU is available, a model based on IMU and odometry.
Developing with DriveWorks
How to set up
You will need:
- Host Development PC
- DRIVE AGX Developer Kit
- DRIVE Software available through our NVIDIA DRIVE™ Early Access Developer Program
- Optional: DRIVE Hyperion Kit
- Install DriveWorks using the SDK Manager.
- Try out the DriveWorks "Hello World" getting started tutorial.
- Refer to the DRIVE Software Documentation to experiment with DriveWorks samples. The DriveWorks Samples are demonstrations of key DriveWorks capabilities and are intended to be used as a starting point for developing and optimizing code.
How to develop
For the below development tasks please refer to the “samples” folder of DriveWorks SDK Reference guide contained in the DRIVE Software Documentation.
|Development Tasks||Getting Started|
|Integrate your sensors into the DriveWorks SAL||
To ensure that your sensors are fully integrated with the SAL and/or for more detail and examples of how to perform the SAL integration, please refer to the following sensor samples:
|Record and playback using DriveWorks Tools||
|Accelerate image processing using DriveWorks Image Processing modules||
For more detail and examples on how to use the DriveWorks image processing modules, please refer to the following samples:
|Accelerate lidar processing algorithms using DriveWorks Point Cloud Processing modules||
For more detail and examples on how to use the DriveWorks point cloud processing modules, please refer to the following sample:
|Use the DriveWorks DNN Framework to load and infer neural networks||
For more detail and examples on how to use the DriveWorks DNN Framework modules, please refer to the following samples:
|Interface your software solution with the vehicle using the DriveWorks VehicleIO module||
For more detail and examples on how to use the DriveWorks VehicleIO modules, please refer to the following samples:
|Use the DriveWorks Static Calibration Tools for one-time calibration of sensors mounted on a vehicle i.e. measure and record nominal calibration parameters||
Follow the “Camera Calibration Tutorial” in the “Tutorials” section of the DriveWorks SDK Reference Guide contained in the DRIVE Software Documentation.
For IMU Calibration and additional information, refer to "Calibration Tools" under the "Tools" section of the DriveWorks SDK Reference Guide contained in the DRIVE Software Documentation.
|Develope dynamic sensor calibration capabilitities using the DriveWorks Self-Calibration modules||
For more detail and examples on how to use the DriveWorks self-calibration modules, please refer to the following samples:
|Use the DriveWorks Egomotion module to model and predict the vehicle’s motion||
For more details and examples on how to use the DriveWorks Egomotion module, please refer to the following sample:
Additional Development Resources:
- All NVIDIA DRIVE documentation can be found here: /drive/documentation