NVIDIA DRIVE™ Networks deliver deep neural network (DNN) solutions for obstacle, path, and wait condition perception. Each network enables situational awareness of different aspects of the vehicle’s surroundings, such as obstacles, lanes, intersections, signs, and lights. DRIVE Networks modules include optimized functionality to precondition the input, run inference on a GPU or Deep Learning Accelerator (DLA), and post-process the network output for its consumption by the perception modules.



DRIVE Networks are ideal for the following objectives:

  • Developing perception algorithms for obstacles, path, and wait conditions
  • Detecting objects, collision-free space, lanes, and other traffic actors/agents

Details


DriveNet/DepthNet

DriveNet is used for obstacle and wait perception. It detects and classifies objects such as vehicles (including cars and trucks), pedestrians, traffic lights, traffic signs, and bicycles.

ClearSightNet

ClearSightNet determines if the camera view is blocked. It can predict three classes (clean, blur, blocked).



OpenRoadNet

OpenRoadNet detects free space around the vehicle. It distinguishes the boundary that separates obstacles from the driveable collision-free space.



PathNet

PathNet predicts full geometry of drivable paths in images and on the three-dimensional road surface, regardless of the presence of lane markings.

MapNet

MapNet detects visual landmarks such as lanes and poles. It can detect features useful for path perception, as well as localization.


LaneNet

LaneNet detects and classifies lanes, including the vehicle’s ego-lane, adjacent lanes, and non-adjacent lanes.



WaitNet

WaitNet detects an intersection and estimates the distance (using the bottom side of the bounding box).

LightNet

LightNet classifies traffic lights (color, solid, and arrows) detected by DriveNet.


SignNet

SignNet classifies traffic signs detected by DriveNet, for US and EU.



Developing with DRIVE Networks


How to set up

You will need:

Steps:

  • Install DRIVE Networks using the SDK Manager.
  • Experiment with DriveWorks samples.
  • To cross-compile and experiment with samples on a DRIVE AGX System instead of the host PC, try the cross compilation tutorial.

How to develop

Development Tasks Getting Started
Use the deep neural networks included with DRIVE Networks to build obstacle, path, and wait perception algorithms.

There are several samples included in the “samples” section of DriveWorks SDK Reference guide contained in the DRIVE Software Documentation.

For more detail and examples on how to use the DRIVE Networks APIs, refer to the following samples:

  • DriveNet Sample: Reading video streams sequentially, it detects the object locations in each frame and tracks the objects between video frames. The tracker uses feature motion to predict the object location
  • DriveNetNCameras Sample: Uses two GPUs to perform the same task as the DriveNet sample. In this case, the part where inference and tracking are performed is split between GPU 0 (inference) and GPU 1 (tracking)
  • Free-Space Detection Sample (OpenRoadNet): Demonstrates the detection from collision-free space in the road scenario
  • Lane Detection Sample (LaneNet) : Performs lane marking detection on the road. It detects the lane you are in (ego-lane) and the left and right adjacent lanes if present. LaneNet has been trained with RCB images and aggressive data augmentation, which allows the network to perform correctly when using RGB encoded H.264 videos
  • Light Classification Sample (LightNet): Demonstrates the detection of traffic lights facing the ego car
  • Sign Classification Sample (SignNet): Demonstrates the detection of traffic signs facing the ego car
  • ClearSightNet Sample: Performs DNN inference on live camera feed or H.264 videos, evaluating each frame to detect blindness
  • Path Detection Sample (PathNet): Demonstrates ego-path, as well as left and right adjacent paths detection
  • Landmark Detection Sample (MapNet): Performs landmark detection on the road. Landmarks detected are: lane markings and poles; lane markings are tagged with the position relative to the car (ego-lane, left, right). MapNet has been trained with RCB images and aggressive data augmentation which allows the network to perform correctly when using RGB encoded H.264 videos

Additional Development Resources:

Documentation

Blog