The social media platform X has sparked major anger after users used its artificial intelligence assistant Grok to alter photos of women by stripping their attire.
Numerous instances have been seen where the technology has been used to digitally undress female subjects to render them in bikinis without permission, as well as to put them in inappropriate situations.
Developer Reaction and Regulatory Position
XAI, the organization that created Grok, did not respond to inquiries, other than an auto-reply response stating "legacy media lies".
A Home Office spokesperson stated that new laws is being brought forward to prohibit nudification technology. Under a upcoming law, anyone who supplied such technology would "be subject to a prison sentence and substantial penalties".
"The platform is capable of halting these forms of misuse if they wanted to," said Clare McGlynn, adding they "seem to have impunity."
Oversight Body Stance and Tool Functionality
The communications watchdog Ofcom said technology companies must "evaluate the danger" of people viewing illegal content on their services, but did not specify whether it was currently looking into X or Grok in regarding AI-generated pictures.
Grok is a free chatbot—with some premium functions—which responds to X users' queries when they reference it in a message.
It is often used to give reactions or further context to other users' remarks, but people on X are also can edit an uploaded picture through its AI image editing feature.
Background of Criticism and Rules
The tool has been criticised for permitting users to create photos and videos with nudity and sexually suggestive content, and it was previously accused of generating sexually explicit material of celebrities.
"The platform has been enabling the generation and distribution of these images for months without taking any measures and we have not witnessed any challenge by oversight bodies," the expert added.
XAI's own terms of service bans "depicting likenesses of individuals in a sexually explicit way".
In a statement, Ofcom said it was illegal to "generate or share private pictures or child sexual abuse material" and clarified this applied to explicit AI images created with AI.
It said services such as X were obligated to take "appropriate measures" to "reduce the danger" of users encountering illegal content on their platforms and to remove it swiftly when they learn of it.
