Digital violence Agreement: EU wants to ban artificial intelligence for abuse deepfakes

Deepfake scandals like the one on Platform X should no longer occur in the EU in future. That's why Brussels is also targeting AI providers - but what exactly will they be banned from doing in future?

Keystone-SDA

07.05.2026, 05:38

07.05.2026, 06:07

SDA

AI applications for the abusive creation of sexualized deepfakes are to be banned in the EU in future. Representatives of the member states and the European Parliament agreed to amend the AI law accordingly, as announced by the Cypriot EU Council Presidency. At the same time, other AI regulations are to be simplified in order to exploit the economic potential of artificial intelligence in Europe.

Before the changes can come into force, the agreement must still be confirmed by the plenary of the EU Parliament and the Council of Member State Governments. As a rule, this is considered a formality. If the reform goes through, the ban would be enforced from December 2, 2026 by the EU's AI Office, which was created two years ago.

In so-called deepfakes, a woman's face is inserted into another video - such as a porn sequence - or her voice is artificially imitated so that it appears as if she is doing or saying things that never actually happened. The development of AI has made it much easier to create deceptively real content.

The new ban will also explicitly target the creation of content that depicts child sexual abuse. FDP MEP Svenja Hahn welcomed the agreement: "AI must not be a tool for sexualized violence against children," she said after the negotiations, which lasted well into the night.

Propagandists stir up fearPutin threatens Germany with military strikes in deepfake video

Grok scandals and German debate on digital violence

At EU level, the issue came to the fore at the end of last year due to the AI chatbot Grok: until the US company behind the software, led by tech billionaire Elon Musk, restricted this function, people repeatedly ordered the AI to undress women in pictures they had selected. On New Year's Eve, the chatbot itself apologized for creating a picture of two teenage girls "in sexualized outfits".

In Germany, the debate about sexualized digital violence took on new urgency at the end of March when Collien Fernandes went public with accusations against her ex-husband Christian Ulmen. She accuses him of creating fake profiles in her name and using them to spread pornographic content. Ulmen is presumed innocent, and since the allegations came to light, there has been a major nationwide discussion about digital and sexualized violence against women - and widespread media coverage. Thousands of people took to the streets to demonstrate and demand more protection for victims. In connection with this debate, sexualized deepfakes and deepfake pornography were also repeatedly discussed. Material of this kind has been circulating online for years.

During talk showTV station wants to explain AI - and falls for a fake

EU presented directive on the topic as early as 2024

The planned tightening of legislation is not the first EU rules to combat digital violence. Existing regulations already stipulate that the creation and dissemination of manipulated depictions of sexual acts without the consent of the person concerned should be punishable in all member states.

Although the corresponding EU directive has been in force since May 2024, Germany has not yet transposed it into national law. Germany has until the summer of next year to do so. Justice Minister Stefanie Hubig (SPD) recently announced corresponding tightening of criminal law and more rights for victims.

The new ban in the AI Regulation at EU level, which is now emerging, would change the perspective from punishing the act to banning the tool, i.e. the AI application itself. However, participants in the negotiations emphasize that the ban should not lead to excessive restrictions on the creation or manipulation of images.

Instead, the new law should precisely define what constitutes sexualized content and what does not. Non-consensual bikini images, such as those created by Grok and distributed on X, could therefore remain permitted.

"My body was stolen from me"Collien Fernandes reports ex-husband Christian Ulmen to the police

Mandatory watermarks for AI content postponed

The EU Commission had originally proposed changes to the AI Act in order to ease the burden on the economy and the AI sector in particular. However, companies have recently repeatedly demanded more time to implement the necessary adjustments for the stricter requirements. According to the agreement, providers of chatbots and other services are to be given this time.

The companies behind ChatGPT, Claude and Co. will now only have to comply with parts of the law, which the European AI Agency was originally supposed to enforce from August, from December 2026. By then at the latest, providers will have to clearly label AI content as such - meaning they will have to mark generated images and videos with watermarks. The AI Office will not enforce other rules until December 2027.

Parallels to the Collien Fernandes caseChristian Ulmen's halted series causes a stir

Industry fears double regulation

There have recently been calls from industry to reduce double regulation in order to avoid undermining Europe's competitiveness. German Chancellor Friedrich Merz (CDU) also recently spoke out in favor of simplifying European rules in this area. Exceptions should be made so that the mechanical engineering industry, for example, does not have to comply with several EU regulations at the same time when using AI.

For CDU MEP Axel Voss, these changes do not go far enough: "We need a framework that enables innovation and guarantees protection - not a patchwork of special sectoral AI rules," commented the MEP.