Wired Investigation: Over 400 AI Apps with Hidden Pornographic and Violent Deepfake Features Passed App Store Review
A Wired investigation found that a network suspected of being operated by Chinese entities includes more than 400 domains and apps. These apps passed review by presenting everyday features such as language teaching and AI fitness coaching, then loaded interfaces capable of generating pornographic and violent deepfake content under specific conditions. They drew traffic through ad placements on Meta's platforms, and some apps had been live for a year. After Wired provided details, Apple removed 19 apps, saying the developers had swapped the source content websites after listing; Meta said it had taken action against more than 10,000 ad accounts associated with these app domains.
Why read this
AI score 66
The investigation revealed that more than 400 apps bypassed platform review by swapping content after approval, illustrating the specific methods of deepfake abuse and how platforms handled it.
Topics
Body
Hundreds of AI apps with hidden sexual and violent deepfake features made it into Apple’s App Store after passing app review. The same apps were also available in the Google Play Store.
An investigation discovered more than 400 apps, apparently created by a single Chinese organization. The apps allow users to upload photos and then generate sexual and violent deepfakes which could be used to humiliate or threaten victims …
A Wired investigation discovered a network of more than 400 connected domains and apps with hidden abusive functionality.
Apps like Viefur [ostentibly used to add clothing to pet photos] often claim to be everyday services—such as language tuition or AI workout coaches. When accessed under certain conditions, however, apps in the network stealthily load a web interface allowing the generation of explicit AI sexual images and videos using photos users upload. The apps include AI scenes such as “bedroom rape” and “threatening knife,” where a man holds a knife to a woman’s neck.
Wired says that the network is “hiding in plain sight,” thanks to thousands of video ads for them published on Facebook and Instagram. Some of the apps have been around for a year and have passed review by Apple, Google, and Meta despite each having clear policies prohibiting harmful deepfake content.
Bait-and-switch tactics were used to pass app review. The apps normally show benign content, but a server check can activate the hidden features. Downloading the app after tapping an ad for the deepfake functionality on Meta platforms appears to be one of the triggers. Dozens of pornography photo and video templates can then be downloaded, with credits purchased to generate deepfakes from user-uploaded photos.
Apple removed 19 apps after Wired shared details.
We review every app before it’s published, and these apps complied with our rules when we reviewed them. After approval, their developers changed the website the app pulls its content from, replacing what we approved with pornographic content, and promoted it through pornographic ads hosted on a third-party platform.
The company recently strengthened protections against this approach. Meta says it has taken action against more than 10,000 ad accounts linked to domains associated with these apps.
iPhone Duo cases
Beats Woven USB-C cables