Content Type: Case Study

How Meta updated its approach to direct disclosure based on user feedback

 

How can labels help audiences better understand AI-edited media?

  • In May 2024, Meta began using a direct disclosure label (“Made with AI”) for synthetic content that was posted across its platforms.
  • Meta discovered that even content with minor AI edits was being flagged as “Made with AI,” which surprised many content creators.
  • In order to provide more context about the nature of the synthetic media being disclosed, Meta updated its label to “AI info,” accounting for content made or edited with AI tools.

This is Meta’s case submission as a supporter of PAI’s Synthetic Media Framework. Explore the other case studies

Download this case study

Direct disclosure has limited impact on AI-generated Child Sexual Abuse Material

An analysis by researchers at Stanford HAI

How can disclosure support harm mitigation methods for AI-generated Child Sexual Abuse Material?

  • Child Sexual Abuse Material (CSAM) poses a unique challenge when it comes to mitigating harm from generative AI models – the harm is done as soon as the content is created, unlike other synthetic content categories which cause harm only when shared.
  • However, both direct and indirect disclosure can still be helpful to a number of non-user audiences that seek to mitigate harm from this content such as Trust and Safety teams, researchers, and law enforcement.
  • Although bad actors have little incentive to disclose AI-generated CSAM, direct and indirect disclosure should still be incorporated by Builders into their models in order to mitigate harm from such content.

This is a case submission by researchers Riana Pfefferkorn and Caroline Meinhardt of Stanford HAI as a supporter of PAI’s Synthetic Media Framework. Explore the other case studies

Download this case study

How Microsoft and LinkedIn gave users detailed context about media on the professional networking platform

 

What details about media can help audiences understand its origin and history?

  • Microsoft and LinkedIn utilize C2PA metadata to disclose media characteristics to users.
  • Engineers had to consider what information from C2PA technical details was most helpful for audiences and how subtle language changes about the details can impact user interpretation.
  • Media literacy is an important component of Microsoft’s overall strategy to ensure societal resilience to AI harms.

This is Microsoft’s case submission as a supporter of PAI’s Synthetic Media Framework. Explore the other case studies

Download this case study

Mitigating the risk of generative AI models creating Child Sexual Abuse Materials

An analysis by child safety nonprofit Thorn

 

Can generative AI models be built in a way that prevents creation of Child Sexual Abuse Materials (CSAM)?

  • Thorn identified how even generative AI models created by well-intentioned Builders, such as Stable Diffusion 1.5, can contain CSAM in their training data or be fine-tuned by bad actors to create CSAM.
  • They also highlight how the use of generative AI to create CSAM furthers harm beyond the creation of the content itself: it can impede victim identification, increase revictimization, and reduce barriers to harm.
  • Builders and hosting sites of generative AI models can help mitigate the risk of their tools creating CSAM by removing models trained or capable of creating CSAM from their platforms, and evaluating training data to ensure abuse material is not included.

This is Thorn’s case submission as a supporter of PAI’s Synthetic Media Framework. Explore the other case studies

Download this case study

How Truepic used disclosures to help authenticate cultural heritage imagery in conflict zones

 

How do indirect disclosures support user-facing direct disclosures for cultural heritage content?

  • Truepic utilizes its indirect disclosure tools to help platforms identify where content comes from and then provide direct disclosure to users.
  • Truepic highlights the importance of not only authenticating and disclosing synthetic content, but also non-synthetic content, in an effort to promote transparency across all digital media.
  • Truepic discusses Project Providence, a collaborative effort with Microsoft to leverage its authentication technology to document over 500 attacks in Ukraine and utilize direct and indirect disclosure outputs to support prosecutors in accountability cases.

This is Truepic’s case submission as a supporter of PAI’s Synthetic Media Framework. Explore the other case studies

Download this case study

How Adobe designed its Firefly generative AI model with transparency and disclosure

 

Can companies include disclosure in the design of generative AI models?

  • In building Firefly, Adobe’s family of creative generative AI models, Adobe wanted to be sure the product would be commercially safe, provide transparency to consumers, and respect the rights of artists and creators.
  • Adobe had to consider technical, legal, policy, and ethical standards in building Firefly, including how to insulate creator content from model development, if requested, and attach disclosures to content.
  • The Framework provided Adobe with guidance on how to “take steps to provide disclosure mechanisms for those creating and distributing synthetic media.” They did this by developing Firefly with both direct and indirect disclosure built in.

This is Adobe’s case submission as a supporter of PAI’s Synthetic Media Framework. Explore the other case studies

Download this case study

How Bumble is preventing malicious AI-generated dating profiles

 

How can dating apps authenticate user profiles in the era of generative AI?

  • Bumble’s Photo Verification process requires users to provide unique photos to Bumble in order to validate the authenticity of their profiles. However, advances in generative AI technology have made using photos for validation increasingly challenging.
  • Bumble sought to balance two potentially conflicting goals: they sought to identify synthetic media when used to create fake or malicious profiles while simultaneously allowing individuals’ creative use of generative AI within their authentic profiles.
  • PAI’s Framework provided Bumble with a reference from which to identify the harm it sought to address, setting the stage for Bumble to roll out new policies on the use of synthetic media within the app.

This is Bumble’s case submission as a supporter of PAI’s Synthetic Media Framework. Explore the other case studies

Download this case study

How the BBC used face swapping to anonymize interviewees

 

Is it possible to disclose AI use in a documentary without negatively affecting storytelling?

  • The BBC wanted to leverage AI tools to “face swap” interviewees in a documentary (instead of face blurring or pixelization) in order to more clearly tell a story.
  • The BBC knew it was necessary to be transparent about the use of AI and considered: how could they disclose to audiences that they were seeing synthetic faces?
  • By applying the Framework, the BBC was able to implement transparent direct disclosures that enabled documentary audiences to view the subjects without the bias that is typically inherent with traditional anonymization techniques.

This is the BBC’s case submission as a supporter of PAI’s Synthetic Media Framework. Explore the other case studies

Download this case study