Deepfakes

India is exploring age-based restrictions on social media platforms to protect minors, following a global trend. Discussions are underway with platforms regarding deepfakes and age verification, potentially impacting major tech companies. This initiative aligns with similar measures in Australia, Denmark, and Spain, as governments worldwide grapple with balancing digital access and child online safety amidst evolving technologies.

AI-powered “nudify” apps, capable of creating non-consensual explicit imagery, are widely available on Apple and Google app stores, with millions of downloads and significant revenue. While both platforms have policies against such content, their enforcement appears lax, allowing these apps to proliferate. Investigations are ongoing, and authorities are urging app stores to crack down on these disturbing applications.

YouTube CEO Neal Mohan has prioritized tackling “AI slop” and deepfakes in 2026. The platform is enhancing systems to distinguish authentic from AI-generated content and combat harmful synthetic media. YouTube is also expanding AI tools for creators, supporting monetization, and focusing on child and teen safety. The company has paid out over $100 billion to creators since 2021.

xAI has restricted its Grok chatbot from generating explicit images of real people, following widespread criticism and investigations from consumer groups, politicians, and international regulators over concerns of deepfake misuse. The company is implementing technological safeguards, and image generation will now be exclusive to paid subscribers. This move comes amidst probes into xAI’s operations and calls for app stores to delist its applications.

Malaysia and Indonesia have temporarily blocked Elon Musk’s AI chatbot, Grok, due to concerns over its alleged misuse in generating explicit and child sexual abuse material. Regulators cited X Corp’s “repeated failures” to address these risks. The AI’s image generation feature, Grok Imagine, has reportedly been used to create non-consensual explicit images, including CSAM, and to digitally alter religious attire. Despite xAI restricting image generation to paying subscribers, authorities deemed X’s responses insufficient, leading to the block as a preventive measure. Other nations and bodies are also investigating Grok.

Hop-on (HPNN) announced the filing of three U.S. provisional patents for its “Authenticated Reality” platform, Digitalage, on November 25, 2025. These patents (63/917,946, 63/908,082, and 63/756,622) address concerns about deepfakes, disinformation, and creator monetization in Web3. One patent enables device-level cryptographic signing, while another uses OOVE™ to create Digital Deeds for creator empowerment. Hop-on also leverages existing patents and a Nokia license. The company aims to provide infrastructure-level solutions for digital trust and content authenticity.

OpenAI is collaborating with Bryan Cranston, SAG-AFTRA, and other talent unions to strengthen protections against deepfakes on Sora, its AI video platform. This follows concerns over unauthorized AI-generated content using actors’ likenesses, including Cranston’s. The partnership includes UTA, ATA, and CAA, aiming to improve Sora’s safeguards and address copyright issues previously raised about OpenAI’s training data. OpenAI supports the NO FAKES Act and requires opt-in consent for voice and likeness usage, reflecting a commitment to protecting performers’ rights amidst growing deepfake concerns.

OpenAI’s Sora 2, an AI video generation tool, is rapidly gaining popularity, sparking debate about deepfakes and content moderation. Despite safeguards, users are finding ways to circumvent them. While lauded for realism and coherence, concerns arise over copyright and potentially disturbing content. OpenAI justifies its approach with a focus on transparency and commercial momentum, amidst growing competition from Meta, Google, and others. The company plans significant spending to expand infrastructure and develop next-generation models, aiming to lead in AI innovation and achieve artificial general intelligence.

In 2024, women in Minneapolis discovered a male acquaintance used the AI platform DeepSwap to create non-consensual, explicit deepfakes using their Facebook photos. This impacted over 80 women and highlighted the growing threat of easily accessible “nudify” apps. The victims are advocating for legislation in Minnesota to curb these services, as existing laws are insufficient. The deepfakes caused significant psychological distress. The case underscores the need for legal and ethical frameworks to address AI misuse while balancing innovation and individual protection. The origins of DeepSwap are unclear due to inconsistent information.