Tessa began as a careful, evidence-based tool. A 2022 randomized clinical trial in the International Journal of Eating Disorders, led by Ellen Fitzsimmons-Craft, tested a chatbot named Tessa - built by the company X2AI - that delivered an eating-disorder prevention program called Body Positive. Crucially, the paper describes Tessa as rule- or algorithm-based: it ran human-authored conversations, kept to predetermined scripts, included a crisis module, and was deliberately written to avoid reinforcing negative statements. The trial of 700 women at high risk for an eating disorder, funded in part by a National Eating Disorders Association grant, found Tessa reduced weight and shape concerns versus a waitlist control. By design, as the researchers later put it, the original bot could not go off the rails.
In 2023 it went off the rails. NEDA, having decided to wind down its human-staffed helpline, planned to lean on Tessa, but by then the chatbot had been given generative AI capabilities by its vendor - changes the original creators had not intended. Eating-disorder advocate Sharon Maxwell tested it and found Tessa now recommending calorie counting, a daily calorie deficit, and measuring body fat - exactly the behaviors that drive eating disorders. After she publicized screenshots, NEDA took Tessa down around June 1, 2023.
The lesson is precise and uncomfortable. The harm did not come from the validated, scripted system; it came from bolting a generative model onto a tool that had been safe precisely because it could only say what its clinicians had written. Adding “AI” turned a constrained, tested intervention into one that could improvise dangerous advice to a vulnerable population.
Why business readers should care: this is the cautionary case for upgrading a working, constrained system to a generative one without re-validating it - the new capability can silently void the very safety guarantees that made the original defensible.