Grok, AI Deepfake Sexually Exploitative Content Generator, First Blocked Worldwide in Indonesia

Violation of Human Rights and Dignity... Temporary Suspension
Regulatory Actions Emerging Worldwide

The Indonesian government has become the first in the world to block the chatbot "Grok," which recently sparked controversy for generating and distributing sexually exploitative material using artificial intelligence (AI) deepfake (image synthesis) technology.

According to major foreign media outlets such as AFP and Bloomberg on January 11 (local time), Mutia Hafid, Indonesia's Minister of Communication and Digital Affairs, announced the previous day that the Grok service would be temporarily suspended. Grok is an AI chatbot service developed by xAI, an AI startup founded by Tesla CEO Elon Musk.

In a statement, she said, "We have temporarily restricted access to the Grok application to protect women and children from the risks of fake pornographic content generated by AI technology," adding, "The Indonesian government considers deepfakes that occur without consent in the digital space to be a serious violation of human rights and dignity."

This marks the first time a country has imposed a nationwide ban on Grok following controversy over its role in generating sexually exploitative material. Previously, the Indonesian Ministry of Communication and Digital Affairs summoned representatives from xAI to verify the facts related to the recent controversy.

Indonesia, which has the world's largest Muslim population, strictly prohibits the online sharing of content it deems pornographic.

Regarding this measure, Grok posted on the social networking service X (formerly Twitter), "We apologize for the inconvenience" and stated, "We are working to resolve this issue."

Grok recently caused controversy by generating and distributing images of children in bikinis and swimwear via X. Most of the thousands of sexually exploitative images created using deepfake technology were generated without the consent of the individuals depicted.

When a user raised concerns, Grok acknowledged, "We have identified a loophole in the safety measures," and belatedly deleted the images in question.

Major European countries and Malaysia have launched investigations into the issue, while Australia and India have also announced plans to crack down on related content.

In the United States, Democratic Senator Ron Wyden and others sent letters to the CEOs of Apple and Google, demanding that Grok and X be removed from their app stores.

As the controversy escalated, X and xAI restricted Grok's image generation and editing features on X to premium (paid) subscribers only, starting the previous day.