To address a reproducibility problem in machine-learning research, McGill professor Joelle Pineau created a Machine Learning Reproducibility Checklist that authors are asked to satisfy. The checklist, hosted at McGill, requires that authors document, for every experiment, details such as network architecture, “training procedures, hyperparameters, and stopping criteria,” the computational resources used, code availability, dataset descriptions, baseline comparisons, ablation studies, and the variability of results across multiple runs.
The checklist was prominently adopted at NeurIPS, one of the field’s largest conferences, where it became part of the submission process to encourage standardized reporting and make published results easier for others to reproduce. It is widely associated with the broader “reproducibility crisis” conversation in machine learning, the recognition that many headline results were hard to replicate because key implementation details went unreported.
The reproducibility checklist is one concrete institutional response to that problem: rather than relying on authors to volunteer enough detail, the conference made a structured disclosure a condition of submission.