Concerns Mount around AI Guardrails as Grok goes on an Undressing Spree

Introduction

On January 2, 2026, the Cyber Laws Division of the Ministry of Electronics and Information Technology (hereinafter referred to as ‘MeitY’) issued a notice to the Chief Compliance Officer of India operations of X Corp. This notice was issued in response to users using Grok to publicly convert real images posted by users into synthetically modified images portraying women and children in sexually explicit manner. The notice directs X Corp to take necessary measures to prevent such non-consensual generation of such indecent and obscene images.[1]

Notably, this is not an isolated incident. In 2025, the United Nations Educational, Scientific and Cultural Organization released a Playbook on ‘Red Teaming Artificial Intelligence for Social Good’. The Playbook reasoned that Women are significantly more vulnerable to AI related harms stating that approximately 58% of young women and girls globally have experienced online harassment. It also stated that 96% of deep fake videos were non-consensual intimate content and 100% of the top five deep fake websites are targeting women. [2]

The Crux of the Issue

In early 2025, Grok AI was integrated with the social media platform ‘X’ which enable users to interact with Grok and generate responses while commenting on tweets.[3] Ever since the integration, Grok has landed in hot water multiple times, for instance, in May 2025, Grok responded with the terms ‘white genocide’ when users prompted it to provide context on tweets which were completely unrelated to this phrase.[4] While these bugs did not affect particular individuals, over the past few days, the social media platform ‘X’ has been flooded with users commenting on images prompting Grok to alter them in a sexually explicit manner. Phrases like ‘put her in a bikini’ have become a tool of non-consensual generation of indecent and obscene images of women and children.[5]

Indian Government issues Notice to X

The Notice issued by MeitY noted that Grok AI, as integrated on X platform, is being misused by users to create fake accounts to host, generate, publish and share obscene images and videos of women in a derogatory or vulgar in order to indecently denigrate them. As per the notice, X Corp has failed to observe statutory compliance with due diligence obligations under the Information Technology Act, 2000 (hereinafter referred to as ‘IT Act’)[6] and the Information Technology (Intermediary Guidelines and Digital Media Ethics Code) Rules, 2021 (hereinafter referred to as ‘Intermediary Rules’)[7] and gave a 72 hour ultimatum for X Corp. to undertake:

A Comprehensive, technical , procedural and governance level review of ‘Grok’ including its prompt-processing, output generation, image handling, and safety guardrails.
Enforcement of Grok’s user terms of service, acceptable use policies and AI usage restrictions, including strong deterrent measures such as suspension, termination and other enforcement actions against violating user and accounts.
Removal of all content already generated in violation of the provisions of the applicable laws.

X Corp. was required to file an ‘action taken’ report in response the notice.[8] As of January 8, 2026, X Corp has filed a response to the notice, however, reportedly MeitY is not satisfied with the response and has asked for further information on this issue.[9]

(To read more on Risks around AI systems, refer –

Legal Framework applicable in the Present Situation

IT Act
Sections 67, 67A criminalizes the act of publishing or transmitting obscene material and material containing sexually explicit act, etc. in electronic form with imprisonment up to 7 years and fine of up to INR 10 lakh. This shall include any material which –
- is lascivious or appeals to the prurient interests
- tends to deprave and corrupt person
- sexually explicit
Section 67B prescribes a higher threshold of prohibition for material depicting children is sexually explicit act etc. in electronic form and extends it beyond publishing and transmitting and applies to recording and creating of such material as well.
Section 79 of the IT Act provides a safe harbor to intermediaries, in this case X Corp., waiving any liability which may arise due to any third party information, data, or communication link made available on the intermediary platform. The availability of this safe harbor is conditional based on ‘third party content’ and ‘no knowledge’. It is unclear if the content generated by Grok on X as per user prompts qualifies as third party content or not given the integration of Grok AI with the platform itself. However, MietY vide its notice, has stated that no safe harbor shall be available to X Corp. in this case.
(To read more on the safe harbor protection, refer – https://ssrana.in/ufaqs/safe-harbor-protection-india/)
Intermediary Rules
- Rule 3(1)(b) lays down that an intermediary is required to make reasonable efforts to not host, display, upload, modify, publish, transmit, store, update or share any information that is obscene, pornographic, pedophilic, invasive of another’s privacy including bodily privacy, or is harmful to a child.
- Rule 3(1)(d) prescribes that an intermediary upon receiving actual knowledge in the form of an order shall not host, store or publish any unlawful information, which is prohibited under any law in relation to decency or morality. The intermediary shall remove or disable access to such information within thirty six hours of the order.
- Rule 3(2)(b) prescribes that an intermediary take all reasonable and practicable measures to remove or disable access to any content which is prima facie in the nature of any material which exposes the private area of such individual, shows such individual in full or partial nudity or shows or depicts such individual in any sexual act or conduct, including artificially morphed images of such individual, within twenty-four hours from the receipt of a complaint by an individual.
- Rule 4(4) lays down that a significant social media intermediary must deploy technology-based measures, including automated tools or other mechanisms to proactively identify information that depicts any act or simulation in any form depicting child sexual abuse or conduct.
- Rule 4(9) of the Intermediary Rules gives the Ministry, the power to call for any additional information from any significant social media intermediary as it may consider necessary for the purposes of Rule 4.
(To read more on the proposed amendment to the intermediary rules, refer – https://ssrana.in/articles/2025-it-rules-amendment-regulating-synthetically-generated-information-in-indias-ai-and-privacy-landscape/ )
Bharatiya Nyaya Sanhita, 2023 (BNS) and Bharatiya Nagarik Suraksha Sanhita, 2023 (BNSS)
Section 33 of BNSS[10] imposes a mandatory statutory obligation to report certain offenses to the appropriate authorities which includes organized crime as defined under Section 111 of the BNS[11] which includes cyber-crimes. Any failure to report such offenses, despite knowledge or reasonable suspicion, may independently attract penal actions under the BNSS.

Overview of Grok’s Policies

One of the directions by MeitY is that X Corp. shall enforce its user terms of service, acceptable use policies and AI usage restrictions. Therefore, it is important to have a look at the relevant documentation.

Firstly, as per the Terms of Use, it restricts access of its platform based on Age of the users. Users below 13 years of age are not allowed to access the platforms of X Corp and users between 13-17 years of age require parental consent to access the platforms. This restriction is accompanied by a disclaimer stating that while necessary steps have been taken to limit undesirable training data and outputs of Grok AI, the services could produce outputs that are not appropriate for all ages. If the users choose certain features or input suggestive or coarse language, the service may response with some dialogue that may involve coarse language, crude humor, sexual situation, or violence.

Secondly, as per its Acceptable Use Policy, user are not permitted to user its services for “depicting likenesses of Persons in a pornographic manner” or “the sexualization or exploitation of children”. It also prohibits use of services or outputs for “Critically harming or promoting critically harming human life”

Therefore, prompting Grok to generate explicit and sexually inappropriate images is violative of the Acceptable Use policy of X Corp and the directions of MeitY are in alignment with the respective policies.

Why is this issue different from other instances of deep fakes?

Notably, this is not the first time AI has been used to generate explicit and sexually inappropriate. In July 2024, a Court in Southwest Spain sentenced 15 teenage school children to one year probation and mandatory gender sensitization lecture for creating and spreading AI generated explicit images of their female peers through WhatsApp groups. This situation is primarily distinct from other instances due to its public nature where ‘A’ can prompt Grok to generate a sexually explicit image of ‘B’ by simply commenting the prompt under the image posted by ‘B’. Grok complies with the request and generates the image which is then available for public viewing. This has resulted in a significantly larger victim pool and the scale of public transmission is much greater than circulation of images through WhatsApp groups.

(To read more on AI Deep fakes refer – https://ssrana.in/articles/techethics-deepfakes-morality-and-values/ )

Open AI plans to enable Adult user to generate Adult Content through ChatGPT

In a contextually relevant development, in October 2025, OpenAI announced that it will allow verified adults to use its AI tool to generate erotic content as a part of its ‘treat adult users like adults’ policy. This move will give rise to many such concerns around AI guardrails and the mechanisms to ensure safe usage of AI tools. Reportedly, a tool for behavior based age prediction technology to estimate the age of the user and automatically tailor the user experience accordingly, is also in the works.[12]

[1] https://timesofindia.indiatimes.com/technology/tech-news/it-ministry-calls-elon-musks-x-reply-inadequate-demands-action-plan-on-grok-ai-issue/articleshow/126406106.cms

[2] https://unesdoc.unesco.org/ark:/48223/pf0000394338

[3] https://timesofindia.indiatimes.com/science/elon-musks-spacex-invests-2-billion-in-xai-to-expand-ai-empire-with-grok-and-x-integration-raising-valuation-to-113-billion-report/articleshow/122416561.cms

[4] https://www.businessinsider.com/sam-altman-mocks-grok-over-controversial-white-genocide-comments-2025-5

[5] https://www.thehindu.com/opinion/editorial/off-the-guardrails-on-the-grok-case-explicit-imagery/article70474053.ece#google_vignette

[6] https://www.indiacode.nic.in/bitstream/123456789/13116/1/it_act_2000_updated.pdf

[7] https://www.meity.gov.in/static/uploads/2024/02/Information-Technology-Intermediary-Guidelines-and-Digital-Media-Ethics-Code-Rules-2021-updated-06.04.2023-.pdf

[8] https://restofworld.org/2026/musk-grok-bikini-trend/

[9] https://www.newindianexpress.com/nation/2026/Jan/08/x-reply-on-grok-dirt-inadequate

[10] https://www.indiacode.nic.in/bitstream/123456789/20099/1/eng.pdf

[11] https://www.mha.gov.in/sites/default/files/250883_english_01042024.pdf

[12] https://www.theguardian.com/technology/2025/oct/14/openai-chatgpt-adult-erotic-content