A Dutch court has ordered xAI to stop its Grok chatbot from generating non-consensual sexual images, escalating regulatory pressure on AI content systems. The ruling, issued by the District Court of Amsterdam, prohibits the creation and distribution of explicit imagery involving individuals without consent.
The injunction applies to both adults and minors and extends to the platform X, where Grok is integrated. The court imposed penalties of €100,000 per day for non-compliance, with a maximum fine of €10 million. The case was brought by Dutch nonprofit Offlimits, which focuses on combating online abuse.
The decision follows evidence that Grok had generated large volumes of sexualized content, including images involving minors. Despite earlier safeguards introduced by xAI, the court found that protections were insufficient and could be bypassed by users.
The ruling represents one of the first major legal actions in Europe targeting generative AI tools for harmful content. It comes amid broader investigations by regulators, including the European Commission under the Digital Services Act, as well as probes by authorities in the United Kingdom.
xAI also faces legal challenges in the United States, where lawsuits allege consumer protection violations and harm caused by generated content. The case highlights increasing global scrutiny of AI platforms and their responsibility to prevent misuse.
