UK Regulator Ofcom Investigates X Over Grok AI Deepfakes

The UK's media regulator, Ofcom, has initiated a formal investigation into X under the Online Safety Act. The probe follows reports that the Grok AI chatbot was utilized to generate and distribute intimate images of individuals and sexualized content involving children.

Investigators will determine if X has fulfilled its legal obligations to shield UK users from illegal content. The scope includes assessing measures to prevent access to priority illegal content, the speed of content removal, and the execution of risk assessments prior to platform changes. Ofcom has demanded urgent clarification from xAI regarding user protection strategies. Penalties for non-compliance could reach £18 million or 10% of global revenue. Concurrently, Malaysia and Indonesia have suspended Grok access, citing a lack of controls against harmful deepfakes.

The UK regulator Ofcom has officially opened a formal investigation into the platform X regarding compliance with the Online Safety Act. The decision follows concerning reports regarding the Grok AI chatbot account on X. According to the regulator, there have been instances where the chatbot was used to create and share undressed images of people, potentially constituting intimate image abuse or pornography. Furthermore, the reports indicated the generation of sexualized images of children, which may amount to child sexual abuse material (CSAM).

The investigation will specifically focus on whether X has complied with its duties to protect people in the UK from content that is illegal in the UK. The regulator outlined several key areas of inquiry:

• Whether X takes appropriate measures to prevent UK users from seeing "priority" illegal content, such as CSAM and non-consensual intimate images.
• If the platform removes illegal content quickly after becoming aware of it.
• Whether X carried out an updated risk assessment before making "any significant changes" to the platform.

Additionally, the probe will consider if X assessed the risks its platform poses to UK children and if it possesses "highly effective age assurance to protect UK children from seeing pornography." Ofcom noted that it contacted X on January 5 and received a response by the January 9 deadline. The regulator is conducting an "expedited assessment of available evidence as a matter of urgency" and has asked xAI for "urgent clarification" on the steps the company is taking to protect UK users.

If Ofcom determines that a company has broken the law, it possesses significant enforcement powers. The regulator can "require platforms to take specific steps to come into compliance or to remedy harm caused by the breach." These measures are designed to ensure immediate rectification of safety failures.

Financial penalties are a major component of the regulatory framework. Ofcom can impose fines of up to £18 million ($24.3 million) or 10 percent of "qualifying" worldwide revenue, whichever figure is higher. Beyond monetary fines, the regulator can seek court orders to stop payment providers or advertisers from working with a platform. It can also require internet service providers to block access to the site within the UK.

The UK government has indicated it would back any action Ofcom takes against X. Reports suggest the UK has held discussions with allies regarding a coordinated response to Grok-generated deepfakes. Regulators in other jurisdictions, including India and the European Union, are also investigating X.

Malaysia and Indonesia have become the first countries to block access to Grok. Both nations claim that X’s chatbot lacks sufficient safeguards to prevent the creation and dissemination of explicit AI-generated deepfakes of women and children. Indonesia temporarily blocked Grok on a Saturday, followed by Malaysia on Sunday.

Meutya Hafid, Indonesia’s Communication and Digital Affairs Minister, issued a statement regarding the ban. "The government sees non-consensual sexual deepfakes as a serious violation of human rights, dignity and the safety of citizens in the digital space," Hafid said. Indonesian officials reported that initial findings showed Grok lacks effective controls to prevent users from creating and sharing sexually explicit deepfakes based on photos of Indonesian residents.

Alexander Sabar, the country's director general of digital space supervision, elaborated on the harms. He stated that generating deepfakes can violate individuals' image and privacy rights when photos are shared or manipulated without consent. Sabar added that these actions can lead to reputational, social, and psychological harm. The Malaysian Communications and Multimedia Commission cited "repeated misuse" of Grok to generate explicit and non-consensual deepfakes. The regulator stated that Grok will remain blocked until X Corp and parent xAI establish strong enough safeguards.

Despite the regulatory pressure, access to image generation tools on X has been inconsistent. Last week, the Grok account on X began informing users that its image generation and editing tools were being limited to paying subscribers. However, as of Monday, it remained possible for non-paying users to generate images through the Grok tab on the X website and app. This discrepancy suggests ongoing adjustments to the platform's accessibility features while the investigation proceeds.