On the night of March 18, 2018, an Uber Advanced Technologies Group test vehicle - a Volvo XC90 running Uber’s automated driving system with a single safety operator behind the wheel - struck and killed Elaine Herzberg as she walked a bicycle across N. Mill Avenue in Tempe, Arizona. It was the first known case of a pedestrian killed by a self-driving car, and it triggered a federal investigation by the National Transportation Safety Board.
The system’s failures were specific and technical. The NTSB found that the vehicle’s sensors detected Herzberg about six seconds before impact, but the perception software kept reclassifying her - first as an unknown object, then as a vehicle, then as a bicycle - and each reclassification reset its prediction of where she was going. The system was not designed to expect pedestrians outside crosswalks, and Uber had disabled the Volvo’s factory automatic emergency braking to avoid erratic behavior, while also suppressing the car’s own emergency braking to prevent false alarms. The result was a car that recognized something in its path and did not brake.
At its November 19, 2019 board meeting, the NTSB adopted report HAR-19/03 and determined the probable cause to be the operator’s failure to monitor the road “because she was visually distracted throughout the trip by her personal cell phone.” But the board placed heavy emphasis on the organization behind her: it cited Uber ATG’s inadequate safety risk assessment, ineffective oversight of operators, and lack of any mechanism to address automation complacency, calling these “a consequence of its inadequate safety culture.” Arizona’s loose oversight of testing was also named as a contributing factor.
Why business readers should care: the lethal failure here was not only a software bug but a management one. A capable team disabled safeguards, leaned on a single distracted human as the last line of defense, and shipped a system into public roads without a formal safety plan. When automation can cause physical harm, the safety case lives in process and culture as much as in code.