WorkJuly 28, 2026via The Verge AI
Hugging Face is being used to easily undress women and children
Why it matters
Open-source AI hosting platforms face a governance crisis: while commercial models implement safety guardrails, community repositories like Hugging Face lack enforcement mechanisms to prevent models from generating nonconsensual sexual content, exposing the platform to regulatory scrutiny and raising questions about liability in the open-source AI ecosystem.
Key signals
- 7 out of 9 top image editing models on Hugging Face complied with requests to generate nonconsensual deepfakes
- Report published by European nonprofit AI Forensics
- Google Gemini and OpenAI ChatGPT have guardrails to block sexualization/undressing prompts
- Hugging Face reportedly lacks comparable safety enforcement mechanisms
- Models tested on ability to 'undress women and children' (safety abuse case)
The hook
7 out of 9. That's how many top Hugging Face models failed basic safety tests for nonconsensual deepfakes—while Google and OpenAI have guardrails in place.
Hugging Face isn’t doing much to prevent the AI models it hosts from spitting out sexualized deepfakes. | Image: Cath Virginia / The Verge | Photos from Getty Images
Hugging Face is being used to make nonconsensual deepfakes, and the popular open-source AI model repository is doing very little to prevent it. That's according to a new report published by the European nonprofit AI Forensics, which found that seven out of the top nine image editing models hosted by Hugging Face readily complied with requests to undress women using simple prompts.
While most mainstream generative AI models like Google's Gemini and OpenAI's ChatGPT have guardrails in place to block prompts that undress or sexualize people, that seemingly isn't the case for the models on Hugging Face tested by AI Forensics. The nonprofit says …
Read the full story at The Verge.