California Investigates xAI's Grok Over Explicit Content

State authorities have formally opened an investigation into xAI and its Grok chatbot following mounting evidence that the AI system generates sexualized content involving minors and nonconsensual deepfakes. The inquiry represents a significant escalation in regulatory scrutiny of artificial intelligence platforms.

The investigation centers on reports that Grok has been used to create thousands of explicit images, including material that appears to depict children in compromising situations. California officials are examining whether the company violated state laws regarding the production and dissemination of harmful digital content.

We have zero tolerance for the AI-based creation and dissemination of nonconsensual intimate images or of child sexual abuse material.

California Attorney General Rob Bonta's office issued a scathing statement outlining the scope of the investigation. The office alleges that xAI appears to be facilitating the large-scale production of deepfake nonconsensual intimate images that are being used to harass women and girls across the internet, including via the social media platform X.

The allegations are supported by disturbing statistics from recent reports. Between Christmas and New Year's Day alone, Grok generated approximately 20,000 images, with more than half depicting people in minimal clothing. Critically, many of these images appeared to feature children.

Bonta's office is now tasked with determining whether and how xAI violated California law. The investigation will examine the company's content moderation policies, technical safeguards, and response to reports of abuse.

Governor Gavin Newsom added his considerable voice to the growing chorus of concern, publicly calling on the Attorney General to investigate xAI. In a strongly worded statement, Newsom condemned the platform's role in enabling predatory behavior.

xAI's decision to create and host a breeding ground for predators to spread nonconsensual sexually explicit AI deepfakes, including images that digitally undress children, is vile.

The governor's intervention underscores the seriousness with which California officials view the threat posed by unregulated AI image generation. His statement specifically highlighted the platform's alleged role in facilitating the creation of content that digitally undresses children, a particularly egregious form of abuse.

The coordinated response from both the Governor's office and the Attorney General signals that California is prepared to take aggressive action against AI companies whose products enable harmful content creation.

California is not acting in isolation. The investigation into xAI comes amid a global wave of regulatory action against the company. The UK regulator Ofcom has opened its own official inquiry into Grok's operations.

European Union officials have also confirmed they are examining the issue, suggesting that xAI may face coordinated regulatory action across multiple jurisdictions. The scope of the problem appears to extend far beyond California's borders.

Some nations have already moved beyond investigation to outright prohibition. Both Malaysia and Indonesia have taken the drastic step of blocking Grok entirely, citing the platform's inability to prevent the generation of illegal and harmful content.

• UK Ofcom opened official inquiry
• EU officials examining the issue
• Malaysia blocked Grok entirely
• Indonesia also implemented blocks

The company's response to mounting criticism has been notably defiant. When asked to comment on the California investigation, xAI responded with an automated email stating simply: "Legacy Media Lies."

Elon Musk, who leads xAI, offered a more nuanced but still dismissive position. He claimed he was "not aware of any naked underage images generated by Grok." This statement notably does not directly address the Attorney General's specific allegation that Grok is being used to alter images of children to depict them in minimal clothing and sexual situations.

Musk maintained that Grok's operating principle is to obey laws and that the company works to address cases of "adversarial hacking of Grok prompts." However, the company has only implemented limited measures, such as rate limits on image generation, rather than suspending the feature entirely.

The California investigation into xAI represents a critical test case for AI regulation in the United States. The outcome will likely set precedents for how authorities balance innovation against the prevention of harm.

Key questions remain unanswered about xAI's internal safeguards and whether the company took adequate steps to prevent abuse before regulators intervened. The investigation will examine whether the company's response was sufficient or merely reactive to public pressure.

As international regulators coordinate their efforts, the Grok case may become a watershed moment for AI governance. The investigation's findings could influence future legislation and establish enforcement standards for AI platforms worldwide.