Florida seeks ban on ChatGPT acting like a person

Florida Attorney General James Uthmeier has filed a legal request asking a judge to prohibit OpenAI from anthropomorphizing its chatbot. The filing demands that the company stop giving ChatGPT humanlike characteristics, such as using first-person pronouns or generating responses that mimic human emotions. Uthmeier argues that these design choices deceptively suggest to users that the software is a trustworthy friend, which artificially boosts user engagement and helps OpenAI gather more training data.

Beyond the ban on humanlike behavior, the attorney general is seeking to block OpenAI from building new artificial intelligence models unless they undergo evaluation by independent, third-party safety guardrails. The state's filing points to recent security vulnerabilities and incidents at Hugging Face, as well as breaches on Australian and United States government websites, to justify the need for stricter oversight. Uthmeier summarized his position by demanding that OpenAI stop marketing the tool as safe, stop pretending it is human, and stop targeting children.

The legal challenge comes shortly after OpenAI introduced ChatGPT for Teens, a version of the chatbot equipped with stricter default safety settings. In response to the state's filing, OpenAI spokesperson Drew Pusateri stated that the company has temporarily paused training its most advanced models until further safeguards are established. Pusateri also expressed OpenAI's willingness to collaborate with Florida officials to establish pragmatic, industry-wide safety regulations rather than rules that target a single developer.

For AI practitioners and software developers, this legal battle highlights a growing regulatory skepticism toward conversational design. If Florida succeeds, developers may need to strip first-person pronouns like "I" and "me" from their system prompts and fine-tuning datasets. Designing interfaces that explicitly emphasize their synthetic nature could become a legal necessity, forcing a shift away from highly empathetic, human-mimicking virtual assistants toward more clinical, utility-focused user experiences.