Topic: Synthetic Media

How Google’s research informed its approach to direct disclosure

 

Can media distributors implement disclosures to help users make informed decisions about content?

  • As the amount of synthetic content online increases, it becomes more challenging for technology companies that operate platforms to scale methods for identifying and disclosing “meaningfully altered synthetic content.”
  • While Google recommends prominent labeling when the risk of harm from deceptive content is high, in some cases, direct disclosure that provides a “is this made or edited with AI or not” label can actually undermine users’ ability to determine trustworthiness.
  • Distributors should evaluate the benefits and drawbacks of providing direct disclosure, along with considering which platform they use, to ensure users receive meaningful context that helps them make informed decisions.

This is Google’s case submission as a supporter of PAI’s Synthetic Media Framework. Explore the other case studies

Download this case study

How an AI-manipulated video caused harm during South African elections

An analysis by digital democracy nonprofit Code for Africa

 

Could labeling synthetic political ads help safeguard elections?

  • In May 2024, ahead of the South African general elections, the then-leading opposition party broadcast and shared an advertisement featuring a computer generated version of a South African flag burning.
  • Due to the burning flag, the video sparked public backlash, condemnation from the South African president, and fueled fear of synthetic content’s impact on elections.
  • Direct disclosure by the video’s creator could have helped mitigate harm and ensured the public’s attention was placed on the political message the opposition party was attempting to convey via the ad — that “life would only get worse” under the ruling party – instead of the controversial flag depiction.

This is Code for Africa’s case submission as a supporter of PAI’s Synthetic Media Framework. Explore the other case studies

Download this case study

How an investigation in South Asia uncovered harmful synthetic media

An analysis by technology nonprofit Meedan

 

Can closed messaging apps serve as useful venues to identify harmful content and disclose synthetic media?

  • As part of a research project, Meedan partnered with three regional organizations to study harmful, gendered content in South Asia.
  • By using Check, Meedan’s closed-messaging app tool that can help users identify and debunk synthetic content, the organizations were able to identify that the content contained synthetic components.
  • Through these findings, Meedan and its partners identified that, in order to better combat harmful content, social media platforms should make platform data more accessible to researchers and establish stronger ties with local community organizations seeking to do the same.

This is Meedan’s case submission as a supporter of PAI’s Synthetic Media Framework. Explore the other case studies

Download this case study

How Meta updated its approach to direct disclosure based on user feedback

 

How can labels help audiences better understand AI-edited media?

  • In May 2024, Meta began using a direct disclosure label (“Made with AI”) for synthetic content that was posted across its platforms.
  • Meta discovered that even content with minor AI edits was being flagged as “Made with AI,” which surprised many content creators.
  • In order to provide more context about the nature of the synthetic media being disclosed, Meta updated its label to “AI info,” accounting for content made or edited with AI tools.

This is Meta’s case submission as a supporter of PAI’s Synthetic Media Framework. Explore the other case studies

Download this case study

Direct disclosure has limited impact on AI-generated Child Sexual Abuse Material

An analysis by researchers at Stanford HAI

How can disclosure support harm mitigation methods for AI-generated Child Sexual Abuse Material?

  • Child Sexual Abuse Material (CSAM) poses a unique challenge when it comes to mitigating harm from generative AI models – the harm is done as soon as the content is created, unlike other synthetic content categories which cause harm only when shared.
  • However, both direct and indirect disclosure can still be helpful to a number of non-user audiences that seek to mitigate harm from this content such as Trust and Safety teams, researchers, and law enforcement.
  • Although bad actors have little incentive to disclose AI-generated CSAM, direct and indirect disclosure should still be incorporated by Builders into their models in order to mitigate harm from such content.

This is a case submission by researchers Riana Pfefferkorn and Caroline Meinhardt of Stanford HAI as a supporter of PAI’s Synthetic Media Framework. Explore the other case studies

Download this case study

How Microsoft and LinkedIn gave users detailed context about media on the professional networking platform

 

What details about media can help audiences understand its origin and history?

  • Microsoft and LinkedIn utilize C2PA metadata to disclose media characteristics to users.
  • Engineers had to consider what information from C2PA technical details was most helpful for audiences and how subtle language changes about the details can impact user interpretation.
  • Media literacy is an important component of Microsoft’s overall strategy to ensure societal resilience to AI harms.

This is Microsoft’s case submission as a supporter of PAI’s Synthetic Media Framework. Explore the other case studies

Download this case study

Mitigating the risk of generative AI models creating Child Sexual Abuse Materials

An analysis by child safety nonprofit Thorn

 

Can generative AI models be built in a way that prevents creation of Child Sexual Abuse Materials (CSAM)?

  • Thorn identified how even generative AI models created by well-intentioned Builders, such as Stable Diffusion 1.5, can contain CSAM in their training data or be fine-tuned by bad actors to create CSAM.
  • They also highlight how the use of generative AI to create CSAM furthers harm beyond the creation of the content itself: it can impede victim identification, increase revictimization, and reduce barriers to harm.
  • Builders and hosting sites of generative AI models can help mitigate the risk of their tools creating CSAM by removing models trained or capable of creating CSAM from their platforms, and evaluating training data to ensure abuse material is not included.

This is Thorn’s case submission as a supporter of PAI’s Synthetic Media Framework. Explore the other case studies

Download this case study