In a personal-injury case against the airline Avianca, the plaintiff’s lawyers filed a brief citing several court decisions to support their argument. The problem was that the cases did not exist. The lawyers had used ChatGPT to do legal research, and the tool had fabricated the opinions - complete with fake quotes and fake internal citations - and the lawyers filed them without checking whether they were real.
On June 22, 2023, U.S. District Judge P. Kevin Castel issued an Opinion and Order on Sanctions. He wrote that “there is nothing inherently improper about using a reliable artificial intelligence tool for assistance,” but that the lawyers and their firm “abandoned their responsibilities when they submitted non-existent judicial opinions with fake quotes and citations created by the artificial intelligence tool ChatGPT, then continued to stand by the fake opinions after judicial orders called their existence into question.” The order named six fabricated decisions: Varghese, Shaboon, Petersen, Martinez, Durden, and Miller.
The court found the lawyers had acted in “subjective bad faith” based on “acts of conscious avoidance and false and misleading statements to the Court,” and imposed “a penalty of $5,000 … jointly and severally” under Federal Rule of Civil Procedure 11. The judge also required the respondents to notify their client and each real judge whose name had been falsely attached to a fake opinion.
The one-line lesson: a language model will invent confident, well-formatted citations that do not exist, and the professional using it - not the tool - is accountable for verifying every fact before it goes out the door.