Computer Vision / Video Analytics

Generating AI-Based Potential Accident Scenarios for Autonomous Vehicles

A gif showing cars on a street with potential collision

Autonomous vehicles (AVs) must be able to safely handle any type of traffic scenario that could be encountered in the real world. This includes hazardous near-accidents, where an unexpected maneuver by other road users in traffic could lead to collision.

However, developing and testing AVs in these types of scenarios is challenging. Real-world collision data is sparse, and recreating such situations in the real world is dangerous and difficult to scale. 

To learn about recent research from NVIDIA that uses AI to automatically generate accident scenarios in simulation for safe and scalable AV testing, watch the NVIDIA DRIVE Labs video below.

NVIDIA DRIVE Labs series takes an engineering-focused look at individual autonomous vehicle challenges and how the NVIDIA DRIVE team is mastering them. Catch up on more NVIDIA DRIVE posts.

Stress testing the AV planner with STRIVE

In the AV stack, the planner module is responsible for deciding how the ego vehicle should drive. Thoroughly testing the planner requires simulating a realistic and diverse dataset of potential-accident traffic scenarios. 

We developed a method called STRIVE (Stress-Test Drive) to create a rich set of accident scenarios for an AV planner (Figure 1). The planner’s behavior can be improved by analyzing where it fails in simulation, and correcting it before deploying on the real road.

A flow chart with boxes labeled Real-World Data, Adversarial Optimization, and Solution Optimization. These surround a Traffic Prior at the center.
Figure 1. STRIVE generates traffic scenarios where an autonomous vehicle is involved in an accident

Generating potential accidents through optimization

STRIVE starts from a real-world scenario where the AV planner performs as expected. It uses an adversarial optimization to modify the 2D trajectories of vehicles in the scene to cause a collision with the AV. The adversary vehicle that collides with the AV can be specified by the user or chosen automatically during optimization.

A second solution optimization finds a maneuver for the AV to avoid the accident to improve the capabilities of the planner. STRIVE creates realistic and diverse accidents with useful solutions to improve the AV (Figure 2).

Several pairs of 2D traffic trajectories labeled Adversarial and Solution. In each example, Adversarial contains an accident and Solution does not.
Figure 2. Example potential accidents and solutions generated by STRIVE

Data-driven traffic model for realism

Simulated accidents must be realistic, accurately reflecting real-world traffic situations and physics. STRIVE uses an AI traffic model that learns what realistic vehicle trajectories look like by training on a large real-world dataset. This model is shown as Traffic Prior in Figure 1. During optimization, STRIVE checks with this trained AI model to ensure that the scenario being generated is still realistic, even though it contains an accident.

From 2D trajectories to 3D simulation

STRIVE is used within NVIDIA DRIVE Sim to create several variations of accidents at an intersection. Starting from regular traffic reconstructed from a real-world drive, STRIVE generates multiple plausible accidents with different vehicles (Figure 3).

Figure 3. Accidents generated in NVIDIA DRIVE Sim, each starting from the same real-world scenario
Figure 3. Potential accidents generated in NVIDIA DRIVE Sim, each starting from the same real-world scenario


STRIVE creates rare and dangerous potential accident scenarios automatically in simulation. With this tool, AV developers can comprehensively train and evaluate self-driving systems to make sure they behave safely. Generating possible collisions among vehicles is just one example of how to control traffic in simulation with AI. We are working to incorporate data-driven models into all types of traffic simulation, including pedestrians, cyclists, and scenario editing. 

Additional resources

To learn more, check out these related resources:

Register for NVIDIA GTC 2023 for free and join us March 20–23 for these related sessions:

Discuss (0)