“Hidden Technical Debt in Machine Learning Systems” was presented at NeurIPS in 2015 by D. Sculley, Gary Holt, Daniel Golovin, and colleagues at Google. It applied the software-engineering concept of technical debt, the long-term cost incurred by quick or expedient choices, to machine-learning systems, and it became one of the most-cited cautionary papers in the field.
Its core observation is that the machine-learning model itself is only a small fraction of a real production system; the much larger surrounding mass of data collection, feature extraction, configuration, monitoring, and serving infrastructure is where maintenance cost accumulates. The paper names specific ML-only forms of debt: entanglement, where changing anything changes everything (summarized as CACE, Changing Anything Changes Everything); hidden feedback loops, where a model’s predictions influence the data it later trains on; undeclared consumers who quietly depend on a model’s outputs; data dependencies that are harder to track than code dependencies; and pipeline jungles and glue code that accumulate over time. It argues these costs are often paid silently and recommends treating ML systems with the same engineering discipline as other software.
The paper reframed practical machine learning as a systems and maintenance problem, not just a modeling problem, and it is widely regarded as a founding text of what became MLOps.
For a business reader, it is the clearest early warning that the real expense of an AI system shows up after the model works, in keeping it correct and maintainable over time.