Generating Adversarial Self-Driving Scenarios in High-Fidelity Simulators
How can we evaluate the performance of self-driving cars in terms of safety in the face of rare events? One approach is to measure miles per intervention from real driving data on the road. According to recent estimates, the number of miles that would have to be driven on the road in order to demonstrate the safety of a car in a statistically significant sense is in the range of hundreds of millions of miles, which corresponds to decades of driving, assuming current projections about the sizes of fleets of self-driving vehicles. Another, complementary, approach is to design rich simulation scenarios that will provide a preliminary assessment of the self-driving software stack. The question then becomes: what is the best way to design these driving scenarios in order to minimize the chances of accidents on public roads?