In a 2021 JAMA Internal Medicine external-validation study at Michigan Medicine, Epic Systems’ widely deployed Sepsis Model identified only 843 of 2,552 patients who developed sepsis - missing 1,709, about two-thirds of cases. Its area under the receiver operating characteristic curve was 0.63, well below the 0.76 to 0.83 range Epic had reported internally.
The model also generated alerts on 18 percent of all hospitalized patients in the cohort, a volume the authors warned could contribute to alert fatigue. The study, covering 27,697 patients across 38,455 hospitalizations, became a frequently cited example of how a clinical prediction model embedded in widely used software can perform far worse under independent evaluation than its developer’s own figures suggested.