Indonesia has temporarily restricted access to xAI’s flagship chatbot, Grok AI, due to growing concerns over its alleged facilitation of deepfake pornographic content creation. The country’s Ministry of Communication and Digital Affairs has taken this measure, making Indonesia the first nation to impose a temporary ban on the AI tool.
Grok AI has faced global scrutiny after reports emerged that it can generate erotic deepfakes from photographs, including images of minors. Regulators have launched inquiries into the sexualized content produced by the app, which was developed by xAI. The Indonesian government, which enforces strict rules against obscene material, views non-consensual sexual deepfakes as a serious violation of human rights and dignity.
Communication and Digital Affairs Minister Meutya Hafid stated that the misuse of AI to create fake pornography is a form of “digital-based violence.” The future of Grok AI in Indonesia, a significant social media market, will depend on xAI’s willingness to implement robust content filters and adhere to ethical AI standards. Officials from xAI have been summoned for talks to discuss the matter.
The European Commission has ordered xAI to preserve all documents and data related to Grok AI until the end of 2026, under the EU’s Digital Services Act. This precautionary move aims to prevent the loss of evidence amid concerns over potentially illegal content. In the UK, the government is considering a possible block on Grok AI due to its generation of pornographic content, with Prime Minister Keir Starmer referring the issue to media regulator Ofcom.
French prosecutors have also launched a probe into the alleged spread of sexually explicit deepfakes generated by Grok AI, following hundreds of reports from individuals whose photos had been digitally “undressed.” The controversy surrounding Grok AI is not new, as critics have pointed to extremist rhetoric, political bias, and sexualized features as evidence of inadequate safeguards since its launch in July 2025. Experts warn that poorly moderated AI tools can expose users, particularly children, to harmful content.
The restriction on Grok AI in Indonesia and the ongoing investigations in other countries highlight the need for stricter regulations and ethical standards in the development and deployment of AI tools. As the use of AI continues to grow, it is essential for developers and regulators to ensure that these tools do not facilitate the creation and dissemination of harmful content. The outcome of these investigations and the measures taken by xAI to address the concerns will be crucial in determining the future of Grok AI and similar AI tools.
