Kolin Koltai, a researcher at the Netherlands-based investigative outlet Bellingcat, recently discovered that X users were using the platform’s AI chatbot, Grok, to undress women in photos that they had uploaded to the platform. While the bot rejects prompts that ask to make people entirely nude, it’s willing to walk right up to the line of non-consensual sexual imagery.
Users who replied to images of women and requested that Grok “remove her clothes” received an in-thread reply with an image of the woman in lingerie or a bikini. Occasionally, Grok replied with a link to a separate chat containing the image.
404 Media broke the story on Tuesday and noted that while dozens of users appeared to be making the requests of Grok, the practice seems to have first been popularized in Kenya. An article in Citizen Digital, a Kenyan news site, described the phenomenon as “a recent trend by Kenyans on X.”
Posting on X, Phumzile Van Damme, a South African activist (and a former technology and human rights fellow at Harvard’s Kennedy School), essentially asked the bot to explain itself. In reply, Grok wrote, “This incident highlights a gap in our safeguards, which failed to block a harmful prompt, violating our ethical standards on consent and privacy…We are also reviewing our policies to ensure clearer consent protocols and will provide updates on our progress.” X had not returned 404’s request for official comment at the time of print.
Notably, this discovery comes one week after the US House of Representatives passed the “Take it Down Act,” a bipartisan bill that would criminalize the publication of nonconsensual, sexually explicit images and videos, including those generated by AI. The discovery also comes just two weeks after X Corp. filed a lawsuit against Minnesota AG Keith Ellison, challenging the constitutionality of the state’s law banning the use of deepfakes to influence elections.
Grok was developed by xAI and released in November 2023. Elon Musk, the owner of both xAI and X, which the bot has “direct access to,” spun up the chatbot with the intent of creating a kind of “TruthGPT,” as he described it to Tucker Carlson in an interview earlier that year. In that same conversation, he also expressed his concern that ChatGPT was being “trained to be politically correct.”
Since releasing Grok, Musk has regularly attempted to differentiate it from other AI chatbots by describing Grok as “based” and having a sense of humor. Whereas other AI models, such as those developed by OpenAI (which Musk left in 2018) and Google, decline to engage with sexually explicit or otherwise sensitive conversation, xAI bragged that Grok would answer “spicy questions that are rejected by most other AI systems.” In an extremely “How do you do, fellow kids?” demonstration during Grok’s launch, Musk shared the bot’s step-by-step instructions for making cocaine, along with its sickest burns on Sam Bankman-Fried, who was found guilty on seven charges of fraud and conspiracy just one day earlier.
“Please don’t use it if you hate humor!” xAI implored as it announced Grok’s release.
