Topic: Transparency

How Google’s research informed its approach to direct disclosure

 

Can media distributors implement disclosures to help users make informed decisions about content?

  • As the amount of synthetic content online increases, it becomes more challenging for technology companies that operate platforms to scale methods for identifying and disclosing “meaningfully altered synthetic content.”
  • While Google recommends prominent labeling when the risk of harm from deceptive content is high, in some cases, direct disclosure that provides a “is this made or edited with AI or not” label can actually undermine users’ ability to determine trustworthiness.
  • Distributors should evaluate the benefits and drawbacks of providing direct disclosure, along with considering which platform they use, to ensure users receive meaningful context that helps them make informed decisions.

This is Google’s case submission as a supporter of PAI’s Synthetic Media Framework. Explore the other case studies

Download this case study

How an AI-manipulated video caused harm during South African elections

An analysis by digital democracy nonprofit Code for Africa

 

Could labeling synthetic political ads help safeguard elections?

  • In May 2024, ahead of the South African general elections, the then-leading opposition party broadcast and shared an advertisement featuring a computer generated version of a South African flag burning.
  • Due to the burning flag, the video sparked public backlash, condemnation from the South African president, and fueled fear of synthetic content’s impact on elections.
  • Direct disclosure by the video’s creator could have helped mitigate harm and ensured the public’s attention was placed on the political message the opposition party was attempting to convey via the ad — that “life would only get worse” under the ruling party – instead of the controversial flag depiction.

This is Code for Africa’s case submission as a supporter of PAI’s Synthetic Media Framework. Explore the other case studies

Download this case study

How an investigation in South Asia uncovered harmful synthetic media

An analysis by technology nonprofit Meedan

 

Can closed messaging apps serve as useful venues to identify harmful content and disclose synthetic media?

  • As part of a research project, Meedan partnered with three regional organizations to study harmful, gendered content in South Asia.
  • By using Check, Meedan’s closed-messaging app tool that can help users identify and debunk synthetic content, the organizations were able to identify that the content contained synthetic components.
  • Through these findings, Meedan and its partners identified that, in order to better combat harmful content, social media platforms should make platform data more accessible to researchers and establish stronger ties with local community organizations seeking to do the same.

This is Meedan’s case submission as a supporter of PAI’s Synthetic Media Framework. Explore the other case studies

Download this case study

Documenting the Impacts of Foundation Models

A Progress Report on Post-Deployment Governance Practices

Albert Tanjaya, Jacob Pratt
Download the Report

Web pages related to ChatGPT were viewed over 3.7 billion times in January 2025, making it the 13th most viewed domain on the internet. Meanwhile, billions of people use products powered by foundation models, such as Gemini in Google Search and Meta AI in Meta products. However, we are just beginning to understand the use and societal impact of these models after they have been deployed.

What are people using these systems for? How are they helping people do things better? What are the most common or severe harms they cause?

This report on the impacts of foundation models explores the progress of the field in answering these questions, and is PAI’s inaugural Progress Report. It provides key insights for different audiences, such as:

Policymakers
  • analysis of the current adoption of four documentation practices
  • recommended actions to take over the next 6 – 24 months
Model providers, model deployers, or actors in the value chain
  • benefits of adopting four documentation practices
  • current challenges to adopting these practices
  • recommended actions to take to improve foundation model governance
  • over 70 examples of how organizations collect and document information on the impacts of foundation models
Academic or civil society researchers or interested users of AI models and systems
  • open questions that merit further exploration
  • information on benefits and challenges to support future research
  • over 70 examples of how organizations collect and document information on the impacts of foundation models

Key Findings

Developed with the input of over 30 different stakeholders across academia, civil society, and industry, the report builds on PAI’s Guidance for Safe Foundation Model Deployment and Policy Alignment on AI Transparency work, and produced the following key findings.

Practices

The report highlights four key practices for documenting the impacts of foundation models.

Practice 1

Share usage information

E.g.

  • Activity data (input data; output data)
  • Usage by geography, sector, and use case
  • Total chat time usage
  • Information on downstream applications.
Practice 2

Enable and share research on post-deployment societal impact indicators

E.g.

  • Labor impact indicators
  • Environmental impact indicators
  • Synthetic content impact indicators
Practice 3

Report incidents and disclose policy violations

E.g.

  • Safety incidents
  • Violations of terms of use and policies
  • Mitigation and remediation actions
Practice 4

Share user feedback

E.g.

  • How users submit feedback
  • Feedback received and type
  • How provider utilizes feedback

Challenges & Recommendations

The main challenges for collecting and sharing information on post-deployment impacts can be grouped into five themes. To overcome these challenges and ensure the benefits of effective post-deployment governance, we recommend that stakeholders take the following actions to move the field forward.

Challenge 1

Lack of standardization and established norms

Recommendation 1

Define norms for the documentation of post-deployment impacts through multistakeholder processes, which may be formalized through technical standards

Challenge 2

Data sharing and coordination barriers

Recommendation 2

Explore mechanisms for responsible data sharing

Challenge 3

Misaligned incentives

Recommendation 3

Policymakers should explore where guidance and rules on documenting post-deployment impacts are needed

Challenge 4

Limited data sharing infrastructure

Recommendation 4

Policymakers should develop blueprints for national post-deployment monitoring functions

Challenge 5

Decentralized nature of open model deployment

Recommendation 5

Conduct research into methods for collecting information on open model impacts

Looking forward

PAI will continue to improve collective understanding of the field and drive accountability through the development of future progress reports. If you would like to know more, please contact policy@partnershiponai.org.

Acknowledgments

This report reflects the collaborative efforts of our multistakeholder community. We are grateful to our working group members who contributed their expertise and dedicated their time to shape this work. We also thank the experts who provided critical insights through our initial questionnaire and Policy Forum workshop. This work was developed with guidance from the Policy Steering Committee. For a complete list of contributors, please see the full report.

How Meta updated its approach to direct disclosure based on user feedback

 

How can labels help audiences better understand AI-edited media?

  • In May 2024, Meta began using a direct disclosure label (“Made with AI”) for synthetic content that was posted across its platforms.
  • Meta discovered that even content with minor AI edits was being flagged as “Made with AI,” which surprised many content creators.
  • In order to provide more context about the nature of the synthetic media being disclosed, Meta updated its label to “AI info,” accounting for content made or edited with AI tools.

This is Meta’s case submission as a supporter of PAI’s Synthetic Media Framework. Explore the other case studies

Download this case study

Direct disclosure has limited impact on AI-generated Child Sexual Abuse Material

An analysis by researchers at Stanford HAI

How can disclosure support harm mitigation methods for AI-generated Child Sexual Abuse Material?

  • Child Sexual Abuse Material (CSAM) poses a unique challenge when it comes to mitigating harm from generative AI models – the harm is done as soon as the content is created, unlike other synthetic content categories which cause harm only when shared.
  • However, both direct and indirect disclosure can still be helpful to a number of non-user audiences that seek to mitigate harm from this content such as Trust and Safety teams, researchers, and law enforcement.
  • Although bad actors have little incentive to disclose AI-generated CSAM, direct and indirect disclosure should still be incorporated by Builders into their models in order to mitigate harm from such content.

This is a case submission by researchers Riana Pfefferkorn and Caroline Meinhardt of Stanford HAI as a supporter of PAI’s Synthetic Media Framework. Explore the other case studies

Download this case study