COCO, short for Common Objects in Context, was introduced in the 2014 paper “Microsoft COCO: Common Objects in Context” by Tsung-Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollar, and C. Lawrence Zitnick. The dataset contains roughly 328,000 images with about 2.5 million labeled object instances across 91 object categories, with everyday objects shown in natural, cluttered scenes rather than centered and isolated.
What set COCO apart from ImageNet was the richness of its annotations. Rather than a single label per image, COCO provides precise per-instance segmentation masks - outlining each object pixel by pixel - along with multiple human-written captions per image. That made one dataset usable for several tasks at once: object detection, instance segmentation, keypoint detection, and image captioning. The annual COCO challenges became the proving ground where successive detection and segmentation architectures were compared.
COCO’s design - many objects per image, in context, with dense labels - pushed computer vision past whole-image classification toward the localized, multi-object understanding that real applications need, and its caption annotations helped seed the multimodal image-and-text work that followed. It remains one of the most cited and most used benchmarks in the field. For business readers, COCO illustrates how the shape of a benchmark steers research: by rewarding fine-grained, in-context understanding, it pulled the field toward capabilities that transfer to real-world vision systems.