ACAV

Abstract

The rapid progress of autonomous vehicles (AVs) has brought the prospect of a driverless future closer than ever. Recent fatalities, however, have emphasized the importance of safety validation through large-scale testing. Multiple approaches achieve this fully automatically using high-fidelity simulators, i.e., by generating diverse driving scenarios and evaluating autonomous driving systems (ADSs) against different test oracles. While effective at finding violations, these approaches do not identify the decisions and actions that caused them — information that is critical for improving the safety of ADSs.

To address this challenge, we propose ACAV, an automated framework designed to conduct causality analysis for AV accident recordings in two stages. First, we apply feature extraction schemas based on the messages exchanged between ADS modules, and use a weighted voting method to discard frames of the recording unrelated to the accident. Second, we use safety specifications to identify safety-critical frames and deduce causal events by applying CAT — our causal analysis tool — to a station-time graph. We evaluate ACAV on the Apollo ADS, finding that it can identify five distinct types of causal events in 93.64% of 110 accident recordings generated by an AV testing engine. We further evaluated ACAV on 1206 accident recordings collected from versions of Apollo injected with specific faults, finding that it can correctly identify causal events in 96.44% of the accidents triggered by prediction errors, and 85.73% of the accidents triggered by planning errors.

Video Demos

Videos of six accidents involving the Apollo ADS, along with the complete accident reports generated by ACAV are all available in the Video Demos sections.
View details »

Research Paper

[ICSE 2024] ACAV: A Framework for Automatic Causality Analysis in Autonomous Vehicle Accident Recordings
Huijia Sun, Christopher M. Poskitt, Yang Sun, Jun Sun, Yuqi Chen
[Preprint] [Paper] [Slides] [Code]
BibTex for citation:
@inproceedings{sun2024acav, author = {Sun, Huijia and Poskitt, Christopher M. and Sun, Yang and Sun, Jun and Chen, Yuqi}, title = {ACAV: A Framework for Automatic Causality Analysis in Autonomous Vehicle Accident Recordings}, year = {2024}, isbn = {9798400702174}, publisher = {Association for Computing Machinery}, address = {New York, NY, USA}, url = {https://doi.org/10.1145/3597503.3639175}, doi = {10.1145/3597503.3639175}, booktitle = {Proceedings of the IEEE/ACM 46th International Conference on Software Engineering}, articleno = {102}, numpages = {13}, series = {ICSE '24} }