Generated $170 Million Worldwide... 'Nude Deepfake' AI Apps Rampant on App Stores

"Apple and Google Failing to Properly Regulate Sexual Content"

It has been revealed that artificial intelligence (AI) apps capable of generating nude images through photo synthesis are openly available on smartphone app stores.

The nonprofit watchdog group, the Technology Transparency Project (TTP), stated in a report released on the 27th (local time) that it found 47 such apps on the Apple App Store and 55 on the Google Play Store. Although Apple and Google prohibit apps containing sexual content or those that demean or objectify others, the report pointed out that their management and blocking functions have not been effectively enforced in practice.

The problematic apps have been downloaded more than 705 million times worldwide, with estimated revenues reaching 117 million dollars (approximately 170 billion won). Since Apple and Google take up to 30% of app developers' revenue, TTP criticized both companies for directly profiting by allowing these apps to remain on their platforms.

These apps use AI technology to transform people in photos into nude or bikini-clad figures, or to synthesize faces onto other sexual images using "deepfake" (AI-manipulated video) features. While these apps were promoted as entertainment image synthesis tools or "AI fitting room" services, they were in fact capable of generating sexual content.

Some apps included terms of service that prohibit the creation of obscene or potentially offensive content, but in actual use, they did little to restrict the generation of sexual images. Other apps even provided templates for explicit videos, such as "clothes ripping."

TTP also pointed out that for China-based apps, user data is stored on servers within China, raising concerns about privacy and security. In response, an Apple spokesperson told CNBC that the company had removed 28 of the apps identified by TTP and warned developers that they could be banned for policy violations. A Google spokesperson also stated that the apps mentioned in the report had been suspended for policy violations, but did not disclose the number of apps affected.

Meanwhile, controversy surrounding generative AI continues to grow, as "Grok," an AI developed by Elon Musk's company xAI, has come under investigation by governments in various countries for generating sexual images such as those featuring people in bikinis.