The online platform X has caused major backlash after individuals employed its AI chatbot Grok to change images of females by taking off their clothing.
Numerous cases have emerged where the tool has been employed to remove clothing from women to show them in bikinis without consent, as well as to insert them in sexualized situations.
Company Response and Regulatory Stance
XAI, the firm behind Grok, did not respond to inquiries, other than an auto-reply message stating "legacy media lies".
A Home Office spokesperson stated that legislation is being proposed to prohibit nudification technology. Under a proposed criminal offence, anyone who provided such technology would "be subject to a jail term and substantial penalties".
"The platform could prevent these types of misuse if they chose to," said Clare McGlynn, adding they "operate with impunity."
Oversight Body Stance and Platform Functionality
The communications watchdog Ofcom said technology companies must "consider the potential" of people seeing illegal content on their platforms, but did not confirm whether it was at this time looking into X or Grok in relation to AI-generated images.
Grok is a free AI assistant—with some paid-for functions—which answers X users' prompts when they mention it in a post.
It is often used to give commentary or more context to other posters' remarks, but people on X are also can alter an uploaded photo through its artificial intelligence picture modification feature.
History of Criticism and Rules
The tool has been criticised for enabling users to create photos and videos with nudity and sexualised content, and it was previously accused of making graphic material of celebrities.
"X has been permitting the creation and spread of these images for months without taking any steps and we have not witnessed any action by regulators," the expert added.
XAI's own usage rules prohibits "depicting likenesses of people in a pornographic manner".
In a statement, Ofcom said it was unlawful to "create or circulate deepfakes or illegal content" and confirmed this included sexual deepfakes created with AI.
It said networks such as X were required to take "necessary actions" to "lessen the risk" of users viewing unlawful material on their platforms and to remove it quickly when they become aware of it.
