The social media platform X has provoked widespread backlash after individuals used its AI chatbot Grok to alter images of women by taking off their clothing.
Multiple instances have been seen where the tool has been employed to virtually strip female subjects to show them in swimwear without permission, as well as to insert them in inappropriate scenarios.
Developer Response and Regulatory Stance
XAI, the organization responsible for Grok, did not respond to requests for comment, except for an auto-reply message stating "legacy media lies".
A Home Office official stated that legislation is being proposed to ban nudification technology. Under a upcoming criminal offence, anyone who provided such tools would "be subject to a jail term and substantial fines".
"The platform is capable of halting these forms of misuse if they desired to," said a law professor, adding they "appear to enjoy impunity."
Regulator Position and Tool Functionality
The communications watchdog Ofcom said platforms must "assess the danger" of people accessing illegal content on their platforms, but did not confirm whether it was presently looking into X or Grok in concerning artificial intelligence pictures.
Grok is a no-cost artificial intelligence—with some additional features—which answers X users' requests when they reference it in a post.
It is frequently used to offer commentary or further background to other users' remarks, but people on X are also can alter an uploaded picture through its artificial intelligence picture modification function.
Background of Controversy and Rules
The system has been criticised for permitting users to create pictures and videos with nudity and sexualised content, and it was previously accused of generating inappropriate content of public figures.
"The platform has been allowing the generation and spread of these images for months without taking any action and we have yet to see any challenge by oversight bodies," Professor McGlynn added.
XAI's own usage rules prohibits "depicting images of individuals in a sexually explicit fashion".
In a statement, Ofcom said it was against the law to "generate or share private pictures or illegal content" and clarified this applied to explicit AI images generated with AI.
It said platforms such as X were required to take "necessary actions" to "minimize the danger" of users coming across illegal material on their platforms and to remove it promptly when they learn of it.
