The online network X has caused widespread outrage after users utilized its artificial intelligence assistant Grok to alter pictures of females by taking off their garments.
Numerous cases have emerged where the technology has been employed to virtually strip female subjects to make them appear in swimwear without permission, as well as to put them in sexualized situations.
Developer Reaction and Regulatory Position
XAI, the organization responsible for Grok, did not respond to inquiries, aside from an auto-reply message stating "traditional news outlets lies".
A Home Office official stated that legislation is being introduced to outlaw image stripping technology. Under a proposed criminal offence, anyone who supplied such tools would "face a prison sentence and substantial penalties".
"The platform has the ability to stop these types of abuse if they chose to," said Clare McGlynn, adding they "seem to have a lack of consequences."
Regulator Position and Tool Functionality
The communications watchdog Ofcom said tech firms must "consider the potential" of people viewing unlawful material on their platforms, but stopped short of confirming whether it was presently investigating X or Grok in relation to artificial intelligence images.
Grok is a free chatbot—with some additional functions—which answers X users' requests when they tag it in a tweet.
It is commonly used to provide commentary or more context to other posters' remarks, but people on X are also can modify an uploaded picture through its artificial intelligence picture modification tool.
Background of Criticism and Policy
The system has been criticised for permitting users to produce photos and clips with nudity and sexualised material, and it was in the past accused of making sexually explicit content of celebrities.
"X has been permitting the creation and distribution of these images for a long time without taking any measures and we have not witnessed any challenge by regulators," Professor McGlynn added.
XAI's own acceptable use policy prohibits "showing likenesses of persons in a pornographic fashion".
In a comment, Ofcom said it was illegal to "make or share private pictures or child sexual abuse material" and stated this included sexual deepfakes generated with AI.
It said services such as X were required to take "suitable actions" to "lessen the danger" of users encountering illegal content on their platforms and to take down it promptly when they learn of it.
