OpenAI releases CSAM classifier and new tools to strengthen child safety

OpenAI releases CSAM classifier and new tools to strengthen child safety
News

OpenAI has announced a package of tools and partnerships aimed at preventing and responding to online child sexual exploitation and abuse. The central release is an image classifier that can identify potential child sexual abuse material, including material generated with AI. OpenAI says the system looks for visual indicators and returns scores so safety teams can decide which cases need attention first. It is being offered at no cost through the OpenAI API to vetted organizations.

The company developed and evaluated the classifier with help from the National Center for Missing and Exploited Children, or NCMEC. OpenAI says early tests found that the new system could reduce the number of images requiring review by 48 percent while finding roughly the same number of abusive images as its existing protection. Images flagged by the classifier were about twice as likely to contain abuse. Those figures come from OpenAI's evaluation and should be treated as reported results, not an independent guarantee of performance.

OpenAI is also introducing product interventions in ChatGPT. When the system detects attempts to find abusive material, a warning will explain the harm and point users toward help. The flagged conversation is removed and access is paused for 24 hours; repeated violations can lead to longer restrictions, account closure or reports to authorities. The approach was developed with child-safety organizations and academic advisers who study deterrence.

For developers, OpenAI has published practical guidance covering prevention, reporting, review processes, safety identifiers and support for people who handle sensitive material. The company is also joining the Tech Coalition's Lantern program and is supporting specialist law-enforcement teams with training and tools for reviewing long chat logs, linking information and building timelines.

The significance is broader than one model. Child safety increasingly depends on shared signals, trained reviewers, reporting channels and cooperation between platforms, nonprofits and investigators. For companies building image or chat products, the announcement raises the practical expectation that detection, user intervention and escalation procedures should be designed together. It does not remove the need for human review or independent testing, but it gives vetted organizations another tool for addressing abuse at scale.

Source openai.com