The iNaturalist Species Classification and Detection Dataset, released at CVPR 2018, contained 859,000 images representing over 5,000 species of plants and animals, every image verified by multiple citizen scientists on the iNaturalist platform.
What made it valuable was its difficulty. Unlike benchmarks with balanced categories, the dataset preserved the long-tailed imbalance of the real world: a few common species appear thousands of times while rare ones appear only a handful. Combined with visually near-identical species and wildly varying photo quality, this drove the best non-ensemble models of the day to only about 67% top-one accuracy - far below the near-perfect scores models posted on cleaner datasets - with the worst performance on the rarest species.