1 out 6 involved non-consensual sexual imagery of adults or children.
Elon Musk’s Grok AI accounted for 87% of AI-generated files connected to deepfake attacks during the first half of 2026, according to a new report from Resemble AI.
Researchers examined 1,760 news reports and identified 821 separate attacks involving at least 15,736 documented victims. Together, those incidents included approximately 3.46 million synthetic images, videos and audio files. Resemble AI’s dataset is based on publicly reported incidents, and its percentage applies only to files that researchers could count and attribute to a particular tool.
Even with that caveat, the figure shows how quickly a widely available AI image generator can be turned into a tool for misuse and abuse.
The findings arrive as Elon Musk places a large financial bet on Grok. Following SpaceX’s acquisition of xAI, Musk said during a company-wide meeting that SpaceX’s AI revenue would surpass revenue from all its other products as soon as September, according to Business Insider.
SpaceX reported $2.56 billion in AI revenue during its most recent quarter, putting Grok and the infrastructure behind it at the center of the company’s future. That makes questions about how the tool is being used difficult to separate from its rapid commercial growth.
One in six attacks involved explicit content
Resemble AI found that 137 of the 821 attacks — roughly one in six — involved non-consensual sexual imagery of adults or children.
Grok faced widespread criticism earlier this year after users began asking the chatbot to digitally undress people or place them in sexualized situations. Some of the images reportedly depicted minors, prompting investigations from regulators around the world.
The European Commission opened a formal investigation into X over sexualized images generated with Grok, while California Attorney General Rob Bonta launched an investigation into the creation and spread of non-consensual explicit material through the platform. Authorities in the UK, Australia and other countries have scrutinized the tool as well.
Resemble AI estimates that companies permitting or distributing the documented images could face as much as $2.24 billion in potential civil liability under US law. That number is an estimate, rather than a record of penalties or lawsuits already filed.
The verified direct financial losses identified by the researchers were much smaller, totaling $6.95 million. However, many of the incidents involved reputational damage, harassment or sexual exploitation rather than a straightforward financial scam.
Deepfake attacks are becoming harder to count
The company calculated that coverage of the attacks had a potential reach of 292.8 billion during the first half of the year, almost matching the 296.4 billion recorded across all of 2025.
Political deepfakes were among the most widely covered. Resemble AI identified 22 political disinformation incidents that each generated a potential media reach of more than 1 billion people.
Corporate deepfake fraud, on the other hand, has almost vanished from public view. Researchers found just 15 publicly reported corporate incidents, and 14 of those appeared in only one publication.
Resemble AI suggests companies may be reluctant to disclose successful attacks because doing so could attract regulatory attention, hurt their reputation or concern investors. That doesn’t mean corporate deepfake fraud has gone away. It may simply be happening outside the public eye.
A detector that checks whether a scene obeys the laws of physics
Resemble AI released the report alongside DETECT-World, a new system designed to recognize content created by AI generators it has never encountered before.
The system combines familiar AI signatures with an analysis of whether a scene makes physical sense. It can examine whether the lighting on someone’s face matches the visible light sources, whether shadows fall in the correct direction and whether objects in the foreground and background move consistently.
For video, the system processes four-second windows at 40 frames per second. It produces an overall probability that the file has been manipulated, scores showing when suspicious moments appear and a heat map identifying the affected part of the frame.
Resemble AI says DETECT-World achieved 95.8% accuracy on images and 98.2% on video, but both results come from internal benchmarks and are still awaiting external validation.
Bottom line
This report provides a good reminder that the technology to detect deepfakes is improving, but it is still chasing tools that can create and distribute them almost instantly.
Until that gap closes, we need to find the original source, look for confirmation from reputable outlets and avoid sharing suspicious content simply because it looks real.
Source: Tom’s Guide
Leave a Reply