Generated $170 Million Worldwide... 'Nude Deepfake' AI Apps Rampant on App Stores

"Apple and Google Failing to Properly Regulate Sexual Content"

It has been revealed that artificial intelligence (AI) apps capable of generating nude images through photo synthesis are openly available on smartphone app stores.

The nonprofit watchdog group, Tech Transparency Project (TTP), reported on January 27 (local time) that it had identified 47 such apps on the Apple App Store and 55 on the Google Play Store. Although Apple and Google prohibit apps containing sexual content or those that demean or objectify others, the report pointed out that their monitoring and blocking mechanisms have not been functioning properly in practice.

The problematic apps have been downloaded more than 705 million times worldwide, and their estimated revenue reaches $117 million (approximately 170 billion won). Since Apple and Google take up to 30% of app developer revenue, TTP criticized both companies for directly profiting by allowing these apps to remain available.

These apps use AI technology to transform people in photos into nude or bikini-clad figures, or to superimpose faces onto other sexual images, providing so-called "deepfake" (AI-manipulated video) functionalities. While these apps were promoted as entertainment image synthesis tools or "AI fitting room" services, in reality, they enabled the creation of sexual content.

Some apps included terms of service that prohibit the creation of obscene or offensive content involving others, yet in actual use, they did little to prevent the generation of sexual images. Other apps even offered templates for sexually suggestive videos, such as "clothes ripping" effects.

TTP also pointed out that, in the case of China-based apps, user data is stored on servers within China, raising concerns about privacy and security. In response, an Apple spokesperson told CNBC, "We have removed 28 of the apps cited by TTP and warned developers that they could be removed from the platform for policy violations." A Google spokesperson also stated that the apps mentioned in the report had been suspended for policy violations, but did not disclose the number of apps affected.

Meanwhile, controversy surrounding generative AI is spreading, as "Grok," developed by Elon Musk's AI company xAI, has come under investigation by governments in various countries for generating sexual images such as those featuring people in bikinis.