Global Movement Fuels AI Safety Tech Wave for Kids Online

The global push for online safety is fueling a surge in AI-powered solutions designed to shield children from the internet’s darker corners. Regulators worldwide are cracking down, and tech companies are scrambling to comply, innovating, and, of course, wrestling with the implications for user privacy.

In the U.K., the Online Safety Act is a paradigm shift, placing a “duty of care” on tech firms to actively protect minors from a host of online horrors, including age-inappropriate content, hate speech, cyberbullying, fraud, and, most disturbingly, child sexual abuse material (CSAM). The stakes are high: non-compliance could result in fines reaching a staggering 10% of a company’s global annual revenue.

Across the Atlantic, the U.S. Congress is also advancing regulations aimed at bolstering online safety for minors. The Kids Online Safety Act mirrors the U.K.’s legislation, potentially holding social media platforms liable for failing to prevent their products from harming children.

This regulatory pressure is sparking a significant shift in strategy among major tech players. Even pornography giants like Pornhub are now blocking access unless users undergo age verification.

Beyond the adult entertainment industry, mainstream platforms like Spotify, Reddit, and X have also implemented age assurance systems to prevent children from stumbling across sexually explicit or otherwise inappropriate material.

However, these measures haven’t been universally welcomed. The tech industry voices concerns about potential infringements on user privacy and the practical challenges of implementation.

At the forefront of this age verification revolution is Yoti, a company rapidly becoming a household name in the digital identity space.

Yoti’s technology uses AI to analyze selfies and estimate a user’s age based on their facial features. The company claims its algorithm, trained on a massive dataset of faces, can accurately estimate the age of individuals between 13 and 24, with a variance of just two years.

Having already partnered with the U.K.’s Post Office, Yoti is strategically positioned to capitalize on the growing momentum behind government-backed digital ID cards in the U.K. While Yoti isn’t alone in the identity verification software market – competitors include Entrust, Persona, and iProov – it has emerged as a leading provider of age assurance services under the U.K.’s new regulatory framework.

“There is a race on for child safety technology and service providers to earn trust and confidence,” Pete Kenyon, a partner at law firm Cripps, told CNBC. “The new requirements have undoubtedly created a new marketplace and providers are scrambling to make their mark.”

However, the proliferation of digital identification methods raises significant concerns about privacy and the potential for data breaches. How can companies strike a balance between protecting children and safeguarding user data?

“Substantial privacy issues arise with this technology being used,” said Kenyon. “Trust is key and will only be earned by the use of stringent and effective technical and governance procedures adopted in order to keep personal data safe.”

Rani Govender, policy manager for child safety online at British child protection charity NSPCC, argues that technology “already exists” to authenticate users without compromising their privacy. The key is ethical implementation.

“Tech companies must make deliberate, ethical choices by choosing solutions that protect children from harm without compromising the privacy of users,” she told CNBC. “The best technology doesn’t just tick boxes; it builds trust.” In other words, it’s not just about compliance; it’s about building responsible technology.

The wave of innovation aimed at protecting children online extends beyond software solutions. Hardware manufacturers are also entering the fray.

Earlier this month, Finnish phone maker HMD Global unveiled the Fusion X1, a smartphone designed with child safety in mind. The phone uses AI to prevent children from capturing or sharing nude content, and it blocks the viewing of sexually explicit images across all apps and from the device’s camera and screen.

The Fusion X1 utilizes technology developed by SafeToNet, a British cybersecurity firm that specializes in child safety solutions.

“We believe more needs to be done in this space,” James Robinson, vice president of family vertical at HMD, told CNBC. He emphasized that HMD’s concept for children’s devices predates the Online Safety Act but noted that it was “great to see the government taking greater steps.”

HMD’s launch of a child-friendly phone coincides with growing support for the “smartphone-free” movement, which advocates for delaying smartphone access for children.

Looking ahead, the NSPCC’s Govender believes that child safety will become a central focus for tech giants like Google and Meta.

These companies have faced years of criticism for allegedly contributing to mental health issues in children and teens due to online bullying and social media addiction. In response, they argue that they have implemented measures such as enhanced parental controls and privacy features.

“For years, tech giants have stood by while harmful and illegal content spread across their platforms, leaving young people exposed and vulnerable,” she told CNBC. “That era of neglect must end.” The pressure is on for these companies to translate their assurances into concrete actions to protect the most vulnerable users.