In June 2024 Wired published an investigation, bluntly titled “Perplexity Is a Bullshit Machine,” accusing the AI search startup Perplexity of two related problems. First, Wired’s analysis indicated that Perplexity accessed website content by apparently ignoring the Robots Exclusion Protocol, the widely respected robots.txt standard, to scrape pages that operators had asked bots not to crawl, despite the company’s claims that it would not. Wired traced a machine almost certainly tied to Perplexity, running on an Amazon server, doing this on wired.com and other Conde Nast sites.
Second, Wired found that Perplexity’s answer engine reproduced the substance of original reporting, including paywalled scoops, while providing only minimal attribution, behavior that publishers including Forbes also alleged that summer. The story noted, pointedly, that Perplexity had even closely paraphrased the very Wired article criticizing it. Facing mounting accusations, Perplexity later announced a revenue-sharing program for publishers.
The Perplexity dispute sharpened a central tension of AI search: products that answer questions by drawing on others’ journalism can divert the traffic and revenue that funds that journalism, sometimes while circumventing the technical signals meant to govern crawling. It stands in contrast to the licensing path taken by OpenAI’s publisher deals. For a general reader, it is a vivid example of how AI answer engines can extract value from the open web faster than norms, contracts, or law can adapt.