The city’s Microsoft-powered AI chatbot is giving users false and — in some cases — illegal business advice, the nonprofit investigative outlet The Markup finds.
In one example, The Markup asked the chatbot if “landlords have to accept tenants on rental assistance” to which the chatbot responded, “No, landlords are not required to accept tenants on rental assistance.” Well, that’s not the case: the city’s own website says that discriminating “based on lawful source of income,” which includes assistance, has been an illegal practice since 2008 with some exceptions. (When Quartz asked the chatbot the same question, the chatbot responded “Yes, landlords are required to accept tenants on rental assistance.”)
When asked if a boss can take a cut of their worker’s tips, the chatbot responded to The Markup saying, “Yes, you can take a cut of your worker’s tips,” and cited NYC’s Payroll and Tip Reporting information. That’s also incorrect, andNew York’s Department of Labor says employers are prohibited from taking any part of an employee’s tip. The chatbot gave a similar response to Quartz.