
OpenAI is rolling out a new image classifier that detects child sexual abuse material (CSAM), including images made with AI, and it will offer the tool free to vetted organizations. The OpenAI CSAM classifier arrives alongside new ChatGPT warnings that pause access for 24 hours when someone tries to find abusive content. The company is also publishing guidance for developers and launching tools for child-safety investigators.
How the OpenAI CSAM Classifier Works
The new classifier scans images for visual signs of abuse, so it can catch new material that has never been seen or catalogued before. It returns a score for each image, which helps human reviewers decide which ones to look at first.
OpenAI says this is the first broadly available CSAM classifier that includes a specific signal for whether the material may be AI-generated. That is a growing problem, since image tools can now produce realistic content that older hash-matching systems were never built to catch.
The company trained and tested the model with help from the National Center for Missing and Exploited Children (NCMEC), which runs the CyberTipline used to report child exploitation in the US. Access goes through the OpenAI API at no cost, and the Tech Coalition and ROOST are helping share it across the industry.
Early test results
OpenAI compared the new classifier with its existing protections on user-uploaded images, and early partners ran their own tests. Here is what the company reported.
| Test | Reported result |
|---|---|
| OpenAI internal comparison | Flagged 48% fewer images while finding roughly the same number of abusive ones |
| OpenAI internal comparison | Flagged images were about twice as likely to contain abuse |
| fal production images | Reviewers confirmed 72% of high-confidence flags as new CSAM |
| invideo AI testing | About one in four flagged cases led to a ban |
| SafetyKit drawings and animations | Detected 90% of cases |
These numbers come from OpenAI and its partners, not an independent audit. Still, fewer false alarms matters a lot for the people who review this material, because it cuts down on how much disturbing content they must look at.
New ChatGPT Warnings and 24-Hour Pauses
ChatGPT already blocks CSAM and refuses requests for it. Now, when someone tries to find abusive material, ChatGPT will show a warning that explains the harm and the consequences, and it will point the person toward specialist help to stop the behavior.
Alongside the warning, OpenAI will remove the flagged conversation and pause the account’s access to ChatGPT for 24 hours. Repeat violations can lead to longer restrictions, account closure or reports to authorities.
OpenAI built the approach with groups that study how to prevent abuse, including Protect Children and the Lucy Faithfull Foundation. A researcher at the University of Tasmania independently evaluated an earlier trial of these deterrence messages.
CSAM Classifier Guidance for Developers and Investigators
The OpenAI child safety update also reaches beyond its own apps. The company published CSAM guidance for developers, created with input from the Tech Coalition, Thorn, NCMEC and the Internet Watch Foundation.
- Clear rules and user reporting options that apps built on OpenAI tools can adopt.
- Advice on pairing the new classifier with the Moderation API and other detection services.
- Steps for reviewing suspected abuse and reporting it to NCMEC.
- Tips on supporting staff who have to review sensitive content.
For law enforcement, OpenAI is offering plugins, investigative tools and data analysis features, with access managed by program administrators. The first tool helps investigators review long chat logs and legal records, spot possible connections and build timelines linked to source material.
Why the OpenAI Child Safety Push Matters Now
AI companies are under heavy pressure over child safety. OpenAI faces lawsuits over harms to young users, including a recent case where states sued OpenAI over ChatGPT’s protections for kids.
Lawmakers are also stepping in, as seen when New York signed a chatbot safety law for minors. Federal regulators have looked closely at the same issues through an FTC safety probe of leading AI labs.
What This Means for You
For most people using ChatGPT, nothing changes in daily use. The warnings and pauses only trigger on attempts to find abusive material.
- Parents get a bit more reassurance that major AI tools are tightening protections for children.
- App developers who allow image uploads can request free access to a strong detection tool.
- Smaller platforms without big safety teams gain options that used to be hard to afford.
Sources
Primary source: OpenAI’s child safety announcement. Additional reporting from TechCrunch.
Conclusion
OpenAI is pairing a free, AI-aware CSAM classifier with firmer ChatGPT warnings, developer guidance and investigator tools. The early test numbers suggest the classifier can help reviewers find abuse with fewer false alarms, though they come from OpenAI and its partners. Sharing the tool for free could help smaller platforms that lack big safety teams. The real measure will be how widely the industry adopts it and whether it helps protect more children.


