Can generative AI models be built in a way that prevents creation of Child Sexual Abuse Materials (CSAM)?
- Thorn identified how even generative AI models created by well-intentioned Builders, such as Stable Diffusion 1.5, can contain CSAM in their training data or be fine-tuned by bad actors to create CSAM.
- They also highlight how the use of generative AI to create CSAM furthers harm beyond the creation of the content itself: it can impede victim identification, increase revictimization, and reduce barriers to harm.
- Builders and hosting sites of generative AI models can help mitigate the risk of their tools creating CSAM by removing models trained or capable of creating CSAM from their platforms, and evaluating training data to ensure abuse material is not included.
This is Thorn’s case submission as a supporter of PAI’s Synthetic Media Framework. Explore the other case studies