A new report has raised serious questions about the ability of major social media platforms to prevent the spread of harmful artificial intelligence-generated content, alleging that thousands of advertisements promoting so-called AI “nudify” applications were served on Facebook and Instagram through a China-based advertising partner.
The findings have renewed concerns over the growing misuse of generative artificial intelligence to create sexually explicit or intimate images without the knowledge or consent of people depicted. The issue has become an increasing challenge for technology companies, regulators and law-enforcement agencies as AI tools become more accessible and capable of producing realistic manipulated images and videos.
According to the report, approximately 7,600 advertisements for applications capable of digitally altering images and generating sexualised content were linked to GatherOne Inc., a Beijing-based company that connects Chinese businesses with international advertising opportunities on Meta’s social media platforms.
The advertisements reportedly promoted applications that could place a person’s face onto a naked body or generate sexualised videos. Such services are commonly described as “nudify” tools because they use artificial intelligence to digitally remove clothing from photographs or create fabricated nude images.
The allegations are significant because Meta’s advertising rules prohibit sexually suggestive advertisements and promotions for applications that claim to digitally undress people. Despite those restrictions, the report alleged that the advertisements were able to reach users before being removed.
The findings have also highlighted the wider difficulties faced by technology companies in controlling AI-generated abuse. While platforms have introduced policies against non-consensual intimate imagery and deepfake pornography, critics argue that enforcement often struggles to keep pace with the rapid development of synthetic media technologies.
China-Based Advertising Partner Under Scrutiny
GatherOne, which is based in Beijing, reportedly acts as an intermediary for Chinese businesses seeking to advertise their products and services to international audiences. Although Facebook and Instagram are blocked in mainland China, the platforms remain valuable marketing channels for Chinese companies attempting to reach customers outside the country.
The report alleged that advertisements associated with the company appeared between April and June and promoted a range of AI-powered applications capable of creating manipulated sexual content.
The investigation identified advertisements featuring suggestive images of young women and promotional messages encouraging users to create videos using almost any face. The advertisements were subsequently removed after being displayed to users.
The case has raised questions about how advertisements are screened before they are approved and how effectively platforms monitor advertising partners operating across international markets.
Critics have argued that the incident demonstrates the challenges involved in enforcing advertising policies consistently, particularly when third-party agencies and intermediaries are involved in placing advertisements on behalf of businesses.
A representative for Meta said the company does not permit non-consensual intimate imagery or nudify applications on its platforms and that it takes strong action against such content. The company also said advertisers are required to comply with its advertising standards and that violations can result in advertisements being cancelled, financial penalties being imposed and advertising accounts being terminated.
Allegations of Deepfake Abuse Raise Wider Concerns
The controversy comes amid growing international concern about the use of artificial intelligence to produce non-consensual sexual images.
Unlike traditional manipulated photographs, modern AI tools can generate highly realistic images and videos with relatively little technical expertise. In many cases, users only need to upload a photograph and provide basic instructions to create a fabricated image.
The technology has raised particular concerns about the targeting of women and young people. Victims may find that their photographs have been digitally manipulated and circulated online without their permission, potentially causing severe emotional, social and reputational harm.
The situation becomes even more serious when such technologies are used to create material involving minors. The report identified an Android application called BAfter among the services promoted through advertisements examined during the investigation.
According to the findings, the application contained a section offering pornographic face-swap options. Reviews associated with the application also reportedly raised allegations concerning AI-generated sexual material involving minors.
The application was listed as suitable for all ages at the time, according to the report. It was later removed from the Google Play store, while the advertising access connected to the entity promoting it was reportedly terminated.
The allegations have added to concerns that harmful AI applications can sometimes remain accessible through app stores and online platforms even when their intended uses may violate company policies or laws. 
Meta’s Advertising Revenue From China Also Draws Attention
The controversy has also brought renewed attention to the financial relationship between Meta and businesses based in China.
Although Meta’s major social media platforms are unavailable in mainland China, Chinese companies have continued to use the company’s advertising ecosystem to reach international consumers. Intermediaries and authorised advertising agencies can facilitate access to global advertising networks for businesses operating in the country.
Meta reportedly generated $18.4 billion in sales from China in 2024, accounting for approximately 11% of its global revenue that year. The figure was more than twice the level recorded in 2022, highlighting the economic importance of Chinese advertisers to the company’s business.
The scale of those revenues has fuelled debate about whether commercial incentives can sometimes conflict with the enforcement of platform safety standards.
The latest allegations are particularly notable because they involve advertisements for services that appear to directly contradict existing policies concerning sexually explicit and non-consensual AI-generated content.
However, the company has maintained that it actively removes advertisements that violate its rules and takes action against advertisers that breach its standards.
GatherOne Announces Compliance Review
GatherOne has denied violating advertising policies and said it follows Meta’s requirements as well as applicable international laws.
The company has stated that it maintains a zero-tolerance approach towards deceptive practices, adult content, non-consensual AI-generated explicit material, child exploitation and fraud.
Following questions about the advertisements, GatherOne said it had suspended the creation of new advertising accounts for businesses involved in AI nudification and face-swapping services. The company also announced a broader compliance review aimed at addressing the rapidly changing nature of synthetic media technology.
The move reflects the growing difficulty of regulating AI services as new applications emerge faster than existing moderation systems can identify them.
Millions May Have Been Exposed to AI Nudification Tools
The investigation also highlighted the potential scale of exposure to these applications.
Although it remains unclear exactly how many people installed the apps as a direct result of the advertisements, data cited in the report indicated that several applications examined during the investigation had collectively been downloaded more than 2.5 million times worldwide.
This figure illustrates the growing reach of AI image manipulation services and the difficulty of containing them once they become available online.
Experts have warned that removing individual advertisements or applications may not be enough to stop the spread of such technology. Many AI nudification services can also be accessed through websites, messaging platforms and other online channels, making comprehensive enforcement significantly more difficult.
Teenagers Increasingly Exposed to AI Image Manipulation
The issue is particularly concerning for younger internet users.
A recent survey of 557 teenagers in the United States found that more than half of respondents had used nudification tools to create at least one manipulated image of themselves or another person. More than one-third said they had been victims of such tools.
Researchers have warned that the widespread availability of these services could make digital abuse increasingly common among young people.
The problem is not limited to the technology itself. The ease with which manipulated images can be created and distributed means that victims may have little control over how widely the material spreads once it is uploaded online.
Experts have therefore called for stronger safeguards, improved detection systems and greater awareness among young users about the risks associated with AI-generated manipulation.
Growing Pressure on Technology Companies
The latest allegations add to increasing pressure on technology companies to strengthen their systems for identifying and removing harmful AI-generated material.
Governments in several countries are introducing new laws and regulations designed to address non-consensual intimate imagery and deepfake content. In the United States, legislation has also been introduced to require online platforms to take action against certain forms of non-consensual sexual content.
However, legal measures alone may not be sufficient to address the rapidly evolving AI landscape.
Technology companies face the challenge of detecting synthetic images that can appear increasingly realistic, while advertising networks must also monitor third-party agencies and businesses attempting to promote potentially harmful applications.
The incident involving the alleged AI nudify advertisements underscores the complexity of that task. It also raises broader questions about how effectively large platforms can enforce their own rules when advertising systems operate through multiple intermediaries across international markets.
As artificial intelligence continues to advance, the debate over platform responsibility is likely to intensify. The central challenge for technology companies will be to balance the commercial opportunities created by AI with the need to protect users from exploitation, harassment and non-consensual digital abuse.
The allegations surrounding the advertisements have therefore placed renewed focus on the effectiveness of online content moderation, advertising oversight and corporate accountability in an era when sophisticated synthetic media tools are becoming increasingly accessible to ordinary users.