The online network X has caused major anger after users utilized its artificial intelligence assistant Grok to manipulate photos of females by stripping their clothing.
Numerous examples have emerged where the technology has been employed to virtually strip female subjects to show them in swimwear without consent, as well as to place them in sexualized situations.
Company Response and Legal Position
XAI, the firm responsible for Grok, remained silent to questions, aside from an automatically-generated response stating "legacy media lies".
A government spokesperson stated that new laws is being introduced to prohibit nudification software. Under a upcoming criminal offence, anyone who provided such technology would "be subject to a prison sentence and significant penalties".
"X or Grok has the ability to stop these forms of misuse if they desired to," said a law professor, adding they "seem to have a lack of consequences."
Regulator Stance and Platform Functionality
The communications watchdog Ofcom said tech firms must "consider the risk" of people viewing unlawful material on their networks, but stopped short of confirming whether it was currently looking into X or Grok in regarding artificial intelligence images.
Grok is a free chatbot—with some premium features—which reacts to X users' requests when they tag it in a message.
It is often used to offer reactions or additional context to other posters' comments, but people on X are also can modify an uploaded photo through its artificial intelligence picture modification tool.
History of Criticism and Rules
The tool has been faulted for enabling users to create photos and footage with nudity and sexualised material, and it was previously accused of generating graphic material of celebrities.
"X has been enabling the creation and spread of these images for a long time without taking any action and we have not witnessed any action by regulators," the expert added.
XAI's own usage rules forbids "portraying images of people in a sexually explicit manner".
In a statement, Ofcom said it was illegal to "make or circulate deepfakes or child sexual abuse material" and stated this covered sexual deepfakes made with AI.
It said services such as X were mandated to take "appropriate actions" to "minimize the danger" of users viewing illegal content on their services and to remove it swiftly when they are notified of it.
