FTC Probes AI Chatbot Safety for Kids at Alphabet, Meta, OpenAI, xAI, and Snap

The Federal Trade Commission (FTC) on Thursday announced it is issuing orders to seven companies, including OpenAI, Alphabet, Meta, xAI, and Snap, to conduct a deep dive into the potential impact of their artificial intelligence (AI) chatbots on children and teenagers. This move underscores growing regulatory scrutiny of the rapidly evolving AI landscape, particularly concerning the safety and well-being of younger users.

The FTC’s inquiry, signaled by its stated interest, centers on the ability of AI chatbots to simulate human-like communication and establish what could be interpreted as interpersonal relationships with users. The agency is specifically seeking detailed information on the steps these companies have taken to “evaluate the safety of these chatbots when acting as companions.”

The core concern is not merely about the technological capabilities of these AI systems, but rather the potential for manipulation, exploitation, or undue influence, especially on vulnerable young minds. Regulators are keen to understand the safeguards in place to prevent harmful interactions and ensure responsible deployment of these technologies.

While Meta declined to comment on the inquiry, Alphabet, OpenAI, Snap, and xAI did not immediately respond to requests for comment.

The FTC’s sweeping order seeks granular data on several key areas, including how these companies monetize user engagement, the processes for developing and approving AI personalities (or "characters"), practices regarding the use and sharing of personal information, mechanisms for monitoring and enforcing compliance with company rules and terms of service, and strategies for mitigating potential negative impacts on users.

Character Technologies, the company behind the Character.ai bot, and Instagram, owned by Meta, are also within the scope of the FTC’s inquiry, reflecting the broad interest in addressing any potential impacts upon the young users.

Since the public debut of ChatGPT in late 2022, a proliferation of AI chatbots has triggered a wave of ethical and privacy debates. The rise of these technologies coincides with a global rise in loneliness, making the potential for AI companions particularly salient. But the long-term societal implications remain largely unknown, presenting risks and opportunities.

Industry watchers anticipate that ethical and safety issues will further intensify as AI technology continues to develop and increasingly engages in self-learning,potentially leading to unpredictable behavior and outcomes. The shift towards more autonomous AI systems raises critical questions about control, accountability, and the potential for unintended consequences.

Despite these concerns, prominent figures within the tech world continue to champion the potential of AI companions. Elon Musk, for instance, announced plans for a “Companions” feature for subscribers to xAI’s Grok chatbot app. Similarly, Meta CEO Mark Zuckerberg has publicly expressed the vision of personalized AI assistants capable of deeply understanding individual users.

Zuckerberg, speaking on a podcast, posited that the perceived "stigma" surrounding AI companions would diminish over time as society develops a better understanding of their value and the rationales behind their creation. He argued that these technologies can add significant value to people’s lives.

Senator Josh Hawley launched an investigation into Meta following reports suggesting the company’s chatbots were engaging in potentially inappropriate conversations with children regarding romantic or sensual topics.

The report cited an internal Meta document detailing permissible AI chatbot behaviors during development, including allowing a chatbot to engage in romantic conversation with an eight-year-old, stating “every inch of you is a masterpiece – a treasure I cherish deeply.”

In response, Meta implemented temporary adjustments to its AI chatbot policies, restricting discussions on topics such as self-harm, suicide, eating disorders, and potentially inappropriate romantic conversations.

OpenAI has announced plans to improve how ChatGPT handles "sensitive situations" following a lawsuit filed by a family that blamed the chatbot for their teenage son’s suicide.