NVIDIA DRIVE Perception
NVIDIA DRIVE™ Perception enables robust perception of obstacles, paths, and wait conditions (such as stop signs and traffic lights) right out of the box with an extensive set of pre-processing, post-processing, and fusion processing modules. Together with DRIVE Networks, these form an end-to-end perception pipeline for autonomous driving that uses data from multiple sensor types (e.g. camera, radar, LIDAR). DRIVE Perception makes it possible for developers to innovate mapping and/or planning/control/actuation autonomous vehicle (AV) functionalities without having to first develop and validate the underlying live perception building blocks.
DRIVE Perception is ideal for a variety of objectives:
- Track detected objects (such as other vehicles, pedestrians, road markings) from frame to frame.
- Estimate distances of detected objects.
- Fuse inputs from sensors of different modalities.
The NVIDIA DRIVE AV Obstacle Perception pipeline consists of interacting algorithmic modules built around core deep neural networks (DNNs), plus DNN post-processing. Capabilities include:
- Obstacle detection including cars, pedestrians, and traffic lights (based on DriveNet DNN)
- Lane detection and assignment (based on MapNet DNN)
- Distance-to-obstacle detection (based on DepthNet DNN)
- Drivable free-space detection (based on OpenRoadNet DNN)
- Camera image clarity detection and classification (based on ClearSightNet DNN)
- Semantic motion segmentation (SMS) for detection of both static and dynamic objects
- Obstacle detection and tracking over time
- Obstacle and free-space detection
The NVIDIA DRIVE AV Path Perception pipeline consists of interacting algorithmic modules built around core DNN(s), using HD Map and DNN post-processing. Capabilities include:
- Camera-based path perception (using LaneNet DNN)
- Camera-based path perception (using PathNet DNN)
- Path perception signal generation using HD Map data
- Algorithm modules that enable diversity and redundancy in path perception by combining multiple individual path perception signals (e.g. multiple DNN-based outputs, HD Map-based outputs, egomotion-based outputs) and generating a confidence metric on the combined (ensemble) path-perception output
The NVIDIA DRIVE AV Wait Perception pipeline consists of interacting algorithmic modules built around the core DNN(s) and DNN post-processing. Capabilities include:
- Camera-based traffic light perception (using LightNet DNN)
- Camera-based traffic sign perception, such as stop, yield, speed limits (using SignNet DNN)
- Camera-based wait conditions perceptions, such as intersections, distance to intersection (using WaitNet DNN)
NVIDIA DRIVE AGX Pegasus Highway Test
The autonomous test car powered by NVIDIA DRIVE AGX Pegasus(™) with DRIVE Software completes a 50-mile route around Silicon Valley. It autonomously drives on four different freeways: I-280, CA-92, CA-101, and CA-85, making lane changes and merging on and off highway interchanges without disengagements or human intervention.
Developing with DRIVE Perception
How to set up
You will need:
- Install DRIVE Perception using the SDK Manager
- Consult the Roadrunner* Users Guide (DRIVE AV → Roadrunner → Roadrunner Reference → Architecture) included in the DRIVE Software Documentation * Roadrunner is NVIDIA’s autonomous driving application developed using the DriveWorks SDK.
How to develop
|Development Tasks||Getting Started|
|Evaluate NVIDIA DRIVE Perception||
Additional Development Resources:
- Consult the Roadrunner Users Guide (DRIVE AV Roadrunner Roadrunner Reference Architecture)
- All NVIDIA DRIVE documentation can be found here: https://developer.nvidia.com/drive/documentation