The European Commission announced today that it has launched formal proceedings under the Digital Services Act to investigate whether X properly assessed risks before deploying its Grok artificial intelligence tool, following its use to generate sexually explicit images.

The commission noted that these potential risks “seem to have materialised,” seeing that the AI-powered tool was used to create “manipulated sexually explicit images, including content that may amount to child sexual abuse material.”

“Sexual deepfakes of women and children are a violent, unacceptable form of degradation,” said EU tech commissioner Henna Virkkunen. “With this investigation, we will determine whether X has met its legal obligations under the DSA, or whether it treated rights of European citizens – including those of women and children – as collateral damage of its service.”

UK authorities are also scrutinizing the platform after the Grok AI chatbot account on X was used to generate undressed images of X users and child sexual abuse material (CSAM).

Following the widespread reports raising concerns about Grok-generated content, the Information Commissioner’s Office contacted X and xAI on January 7 for more details on the measures taken to comply with data protection law, while Ofcom (the UK’s independent online safety watchdog) launched its own investigation on January 12.

Days later, after California Attorney General Rob Bonta opened another investigation into the nonconsensual sexually explicit material generated using Grok, X announced that it would limit Grok’s image generation and editing capabilities to paid subscribers.

X’s move was later described by a UK Prime Minister’s spokesperson as “insulting to victims of misogyny and sexual violence” because it turned the Grok “AI feature that allows the creation of unlawful images into a premium service.”

Since it was designated a very large online platform under EU law (following its announcement that it had reached over 45 million monthly active users in the European Union), X must now mitigate all potential systemic risks as defined in the EU’s Digital Services Act (DSA), including the spread of illegal content and threats to fundamental rights.

The European Commission also fined X €120 million ($140 million) in December for violations of transparency obligations and transparency requirements under the Digital Services Act (DSA).