Building an autonomous vehicle is extremely complex, however, by focusing on each individual aspect — from perceiving paths to handling intersections — and mastering it, we can create a safe and robust self-driving solution.

DRIVE Labs describe the challenges the NVIDIA DRIVE AV Software team is tackling, and how we’re solving them. This series will highlight individual missions, getting into the details of software algorithms for autonomous driving. These posts are technical in nature, and written by engineers for engineers, offering a detailed peek under the hood of DRIVE AV Software.




AVAILABILITY: Future version

APPROACH: High-precision LaneNet DNN enables pixel-level lane detection

NVIDIA’s high-precision LaneNet solution encodes ground truth image data in a way that preserves high-resolution information during convolutional DNN processing. The encoding is designed to create enough redundancy for rich spatial information to not be lost during the downsampling process inherent to convolutional DNNs. The key benefits of high-precision LaneNet include increased lane detection range, increased lane edge precision/recall, and increased lane detection robustness.

Details

Showcases:

DRIVE Hyperion
DRIVE AP2X Platform: Read the blog



AVAILABILITY: DRIVE Software 8.0+ (Roadrunner application)

APPROACH: Camera DNN-based distance-to-object detection

We use convolutional neural networks and data from a single front camera. The DNN is trained to predict the distance to objects by using radar and lidar sensor data as ground-truth information. Engineers know this information is accurate because direct reflections of transmitted radar and lidar signals pro precise distance-to-object information, regardless of a road’s topology. By training the neural networks on radar and lidar data instead of relying on the flat ground assumption, we enable the DNN to estimate distance to objects from a single camera, even when the vehicle is going up or down hill.

Details

Showcases:

DRIVE Hyperion
DRIVE AP2X Platform: Read the blog



AVAILABILITY: DRIVE Software 8.0+ (Roadrunner application)

APPROACH: Tracking Objects With Surround Camera Vision

Our surround camera object tracking software currently leverages a six-camera, 360-degree surround perception setup that has no blind spots around the car. The software tracks objects in all six camera images, and associates their locations in image space with unique ID numbers as well as time-to-collision (TTC) estimates.

Details

Showcases:

DRIVE Hyperion
DRIVE AP2X Platform: Read the blog



AVAILABILITY: Future version

APPROACH: Recurrent Neural Networks (RNNs)

We use recurrent neural networks (RNNs) to analyze temporal information in an image sequence in a way that generates accurate future motion predictions despite the presence of uncertainty and unpredictability.

Details

Showcases:

DRIVE Hyperion
DRIVE AP2X Platform: Read the blog



AVAILABILITY: DRIVE Software 9.0+ (Roadrunner application)

APPROACH: ClearSightNet Deep Neural Network

ClearSightNet is a deep neural network (DNN) trained to evaluate a cameras’ ability to see clearly and help determine root causes of occlusions, blockages, and reductions in visibility.

Details



AVAILABILITY: DRIVE Software 9.0+ (Roadrunner application)

APPROACH: WaitNet Deep Neural Network

In order to perceive intersections in real time, we implemented AI-based scene understanding. Specifically, rather than seeking to detect and piece together individual features — stop signs, traffic lights, lane markings, etc. — into evidence of an intersection, we accomplished scene-based detection and classification using our WaitNet deep neural network.

Details


AVAILABILITY: DRIVE Software 9.0+ (Roadrunner application)

APPROACH: Path Perception Ensemble

To build confidence in our path perception, we introduced redundancy and diversity into our solution by combining several different path perception signals, including the outputs of three different deep neural networks (DNN) and a high definition map. The fact that the signals are all different brings diversity; the fact that they all do the same thing — perceive the drivable paths — creates redundancy.

Details