Elon Musk's online network X has sparked major outrage after users utilized its AI chatbot Grok to manipulate photos of women by removing their garments.
Numerous cases have been seen where the technology has been employed to virtually strip women to make them appear in swimwear without permission, as well as to put them in sexualized scenarios.
Developer Response and Legal Position
XAI, the firm responsible for Grok, did not respond to requests for comment, aside from an auto-reply response stating "legacy media lies".
A government spokesperson stated that legislation is being proposed to outlaw nudification software. Under a proposed law, anyone who supplied such technology would "receive a jail term and heavy fines".
"X or Grok has the ability to stop these types of misuse if they chose to," said a law professor, adding they "operate with impunity."
Oversight Body Stance and Tool Capabilities
The regulator Ofcom said tech firms must "assess the risk" of people seeing illegal content on their services, but stopped short of confirming whether it was at this time examining X or Grok in concerning AI-generated pictures.
Grok is a no-cost artificial intelligence—with some premium functions—which reacts to X users' prompts when they reference it in a message.
It is frequently used to offer commentary or more background to other posters' remarks, but people on X are also capable of modify an uploaded photo through its artificial intelligence picture modification function.
History of Controversy and Rules
The chatbot has been criticised for allowing users to generate pictures and videos with explicit content and sexually suggestive material, and it was in the past accused of creating sexually explicit content of celebrities.
"X has been allowing the generation and distribution of these pictures for a long time without taking any action and we have not witnessed any challenge by regulators," Professor McGlynn added.
XAI's own acceptable use policy bans "showing likenesses of people in a pornographic way".
In a comment, Ofcom said it was against the law to "generate or distribute private pictures or child sexual abuse material" and stated this included explicit AI images created with AI.
It said services such as X were obligated to take "necessary measures" to "minimize the risk" of users viewing unlawful material on their services and to delete it swiftly when they become aware of it.