In the mid-1960s the MIT computer scientist Joseph Weizenbaum wrote ELIZA, a program that imitated a Rogerian psychotherapist by doing little more than rephrasing the user’s own words back as questions. He described it in a 1966 paper in the Communications of the ACM. The whole point was to demonstrate how superficial the trick was - there was no understanding inside it, only pattern-matching and reflection.
Then people reacted to it. In his 1976 book “Computer Power and Human Reason,” Weizenbaum recounts being startled by how readily users attributed genuine understanding and feeling to the program. The most famous example is his own secretary, who had watched him build the thing and knew exactly what it was - and who, after a few exchanges with it, asked him to leave the room so she could talk to it privately. He wrote that he had not realized how short an exposure to a simple program could “induce powerful delusional thinking in quite normal people.”
That reaction unsettled Weizenbaum enough to change the course of his career. He spent much of the rest of his life warning against confusing the appearance of understanding with the real thing, and against handing human judgment over to machines. The man who built one of the first chatbots became one of the field’s sharpest skeptics.
The phenomenon now carries his program’s name - the “ELIZA effect,” the human tendency to read mind and meaning into a machine’s output. It is the same pull that would later lead people to believe a 2022 chatbot was sentient. The striking part of the original story is that the susceptibility did not require ignorance: the secretary knew it was a program, and confided in it anyway.