In March 2024, The Markup reporter Colin Lecher published an investigation into MyCity, a chatbot New York City launched in October 2023 to help business owners navigate government rules. Mayor Eric Adams had promoted it as a way to deliver city services more effectively. Built on Microsoft’s Azure AI and presented as official NYC business information, the bot turned out to be confidently wrong about the law it was supposed to explain.
The Markup documented specific, dangerous errors. Asked whether a business could take a cut of workers’ tips, the bot said yes, contradicting New York labor law. It told landlords they did not have to accept tenants using housing vouchers, contradicting source-of-income anti-discrimination rules. It claimed there were no rules requiring businesses to accept cash, ignoring a 2020 city law. A small business following this guidance could have broken the law and exposed itself to liability.
After the reporting, the city added disclaimers calling the tool a beta product that might be inaccurate, but kept it running, and follow-up reporting found it still gave bad advice. The chatbot was eventually taken down, with the city noting that its beta test had ended.
Why a business reader should care: a chatbot that speaks with an official voice creates real reliance, and a confident wrong answer about regulated activity is worse than no answer at all. Deploying a general-purpose language model as an authority on law or policy, without tight grounding and review, invites exactly this failure.