nuScenes is an autonomous-driving dataset and benchmark published in 2019 by nuTonomy (an Aptiv company, later Motional), introduced in the paper “nuScenes: A multimodal dataset for autonomous driving.” Its distinguishing feature is that it carries the full production sensor suite of a self-driving car: 6 cameras, 5 radars, and 1 lidar, “all with full 360 degree field of view.” That made it one of the first public datasets to include radar alongside lidar and vision, the combination real vehicles actually use.
The dataset comprises 1,000 driving scenes, each 20 seconds long, recorded in the dense, difficult traffic of Boston and Singapore, and “fully annotated with 3D bounding boxes for 23 classes and 8 attributes.” The authors emphasize the jump in scale over what came before: nuScenes has “7x as many annotations and 100x as many images as the pioneering KITTI dataset.” It also defined new metrics for 3D detection and tracking and shipped baselines for both lidar- and image-based methods.
nuScenes became a workhorse benchmark for the next wave of perception research, including the move toward fusing camera and lidar and toward bird’s-eye-view representations, precisely because it provided a realistic, multi-sensor, multi-city test set.
For a general reader, nuScenes shows how the bar for “serious” autonomy research rose over the 2010s: a credible benchmark now had to span multiple sensors, multiple cities, and tens of thousands of carefully labeled scenes, not a few hours of clear-weather footage.