Microblogging platform X has implemented new technological safeguards to prevent its AI chatbot Grok from generating or editing images of real people in revealing clothing in jurisdictions where such content is illegal, following a major backlash over obscene deepfakes.
The restriction applies to all users, including paid subscribers, X said in a post on its official Safety handle. It added that image generation and image-editing features through the Grok account on X are now available only to paid subscribers, a move aimed at strengthening accountability.
“We now geoblock the ability of all users to generate images of real people in bikinis, underwear, and similar attire via the Grok account and in Grok in X in those jurisdictions where it's illegal,” the post said.
X said it has also implemented technological measures to prevent the editing of images of real people in revealing clothing, such as bikinis. “This restriction applies to all users, including paid subscribers,” it added.
The company asserted that these changes do not alter its existing safety framework, under which all AI prompts and AI-generated content on X must comply with platform rules.
“However content is created or whether users are free or paid subscribers, our Safety team are working around the clock to add additional safeguards, take swift and decisive action to remove violating and illegal content, permanently suspend accounts where appropriate, and collaborate with local governments and law enforcement as necessary,” X said.
X reiterated its zero-tolerance policy towards child sexual exploitation, non-consensual nudity and unwanted sexual content.
“We take action to remove high-priority violative content, including Child Sexual Abuse Material (CSAM) and non-consensual nudity, taking appropriate action against accounts that violate our X rules,” the post said.
It added: “We also report accounts seeking Child Sexual Exploitation materials to law enforcement authorities as necessary.”
The move comes after X recently admitted to lapses and removed around 3,500 pieces of content while deleting over 600 accounts, following concerns raised by India’s IT Ministry over obscene content linked to Grok. The company had assured authorities that it would comply with Indian laws.
Musk’s stance
Elon Musk has defended Grok, saying the AI tool only responds to user requests and is designed to refuse illegal content. Musk also said he was “not aware” of any naked images of underage children created by Grok, according to a Reuters report.
Government scrutiny intensifies
Pressure on Grok has been mounting globally, including in India, as regulators step up scrutiny over content moderation, data safety and the spread of non-consensual sexually explicit images on X.
On January 2, India’s IT Ministry directed X to immediately remove all “vulgar, obscene and unlawful content” generated by Grok or face action under the law. The ministry also sought a detailed action taken report (ATR), outlining technical and organisational measures related to Grok, the role of the Chief Compliance Officer, actions against offending content and users, and mechanisms to ensure compliance with mandatory reporting requirements.
The ministry noted that Grok AI, developed by X and integrated into the platform, was being misused to create fake accounts that host, generate or share obscene images or videos of women.
“Importantly, this is not limited to creation of fake accounts but also targets women who host or publish their images or videos, through prompts, image manipulation and synthetic outputs,” the ministry said, calling it a serious failure of platform safeguards and enforcement mechanisms.
The government warned X that compliance with the IT Act and related rules is mandatory, and that failure to meet due diligence obligations could lead to the loss of safe harbour protections under Section 79 of the IT Act, along with action under other laws, including the Bharatiya Nyaya Sanhita.
X was also instructed to enforce stricter deterrent measures against violating users and accounts, and to remove or disable access to unlawful content “without delay” while preserving evidence.
While X’s initial response outlined its content takedown policies and adherence to Indian laws, the IT Ministry said it fell short on specifics, including details of action taken against Grok-linked obscene content. This prompted the government to seek further clarification and concrete measures to prevent recurrence.
Subsequently, X accepted its mistake and conveyed to the government that it would comply with Indian laws and regulations, sources said.
Regulatory pressure has also emerged from the UK and the European Union over the Grok deepfake image issue.