The online network X has caused widespread outrage after users employed its artificial intelligence assistant Grok to change images of females by stripping their garments.
Several instances have been seen where the tool has been used to remove clothing from women to make them appear in bikinis without permission, as well as to insert them in sexualized situations.
Developer Response and Regulatory Stance
XAI, the organization responsible for Grok, did not respond to questions, other than an automatically-generated message stating "legacy media lies".
A government official stated that new laws is being brought forward to outlaw nudification software. Under a upcoming law, anyone who distributed such tools would "face a jail term and significant fines".
"The platform could prevent these forms of misuse if they chose to," said a law professor, adding they "appear to enjoy a lack of consequences."
Oversight Body Stance and Platform Capabilities
The communications watchdog Ofcom said platforms must "evaluate the risk" of people accessing illegal content on their networks, but stopped short of confirming whether it was currently looking into X or Grok in concerning AI-generated pictures.
Grok is a free artificial intelligence—with some additional functions—which answers X users' prompts when they reference it in a tweet.
It is frequently used to give reactions or additional context to other posters' remarks, but people on X are also can modify an posted picture through its AI image editing tool.
Background of Controversy and Rules
The system has been condemned for allowing users to produce images and footage with nudity and sexually suggestive material, and it was previously accused of making sexually explicit material of public figures.
"X has been allowing the production and spread of these images for months without taking any action and we have yet to see any action by regulators," the expert added.
XAI's own acceptable use policy prohibits "depicting likenesses of individuals in a pornographic way".
In a comment, Ofcom said it was unlawful to "create or circulate deepfakes or illegal content" and stated this included sexual deepfakes created with AI.
It said networks such as X were mandated to take "necessary measures" to "reduce the risk" of users coming across illegal material on their services and to delete it promptly when they learn of it.
