Documenting the Impacts of Foundation Models

Web pages related to ChatGPT were viewed over 3.7 billion times in January 2025, making it the 13th most viewed domain on the internet. Meanwhile, billions of people use products powered by foundation models, such as Gemini in Google Search and Meta AI in Meta products. However, we are just beginning to understand the use and societal impact of these models after they have been deployed.

What are people using these systems for? How are they helping people do things better? What are the most common or severe harms they cause?

This report on the impacts of foundation models explores the progress of the field in answering these questions, and is PAI’s inaugural Progress Report. It provides key insights for different audiences, such as:

Policymakers

• analysis of the current adoption of four documentation practices
• recommended actions to take over the next 6 – 24 months

Model providers, model deployers, or actors in the value chain

• benefits of adopting four documentation practices
• current challenges to adopting these practices
• recommended actions to take to improve foundation model governance
• over 70 examples of how organizations collect and document information on the impacts of foundation models

Academic or civil society researchers or interested users of AI models and systems

• open questions that merit further exploration
• information on benefits and challenges to support future research
• over 70 examples of how organizations collect and document information on the impacts of foundation models

Key Findings

Developed with the input of over 30 different stakeholders across academia, civil society, and industry, the report builds on PAI’s Guidance for Safe Foundation Model Deployment and Policy Alignment on AI Transparency work, and produced the following key findings.

Practices

The report highlights four key practices for documenting the impacts of foundation models.

Practice 1

Share usage information

E.g.

• Activity data (input data; output data)
• Usage by geography, sector, and use case
• Total chat time usage
• Information on downstream applications.

Practice 2

Enable and share research on post-deployment societal impact indicators

E.g.

• Labor impact indicators
• Environmental impact indicators
• Synthetic content impact indicators

Practice 3

Report incidents and disclose policy violations

E.g.

• Safety incidents
• Violations of terms of use and policies
• Mitigation and remediation actions

Practice 4

Share user feedback

E.g.

• How users submit feedback
• Feedback received and type
• How provider utilizes feedback

Challenges & Recommendations

The main challenges for collecting and sharing information on post-deployment impacts can be grouped into five themes. To overcome these challenges and ensure the benefits of effective post-deployment governance, we recommend that stakeholders take the following actions to move the field forward.

Challenge 1

Lack of standardization and established norms

Recommendation 1

Define norms for the documentation of post-deployment impacts through multistakeholder processes, which may be formalized through technical standards

Challenge 2

Data sharing and coordination barriers

Recommendation 2

Explore mechanisms for responsible data sharing

Challenge 3

Misaligned incentives

Recommendation 3

Policymakers should explore where guidance and rules on documenting post-deployment impacts are needed

Challenge 4

Limited data sharing infrastructure

Recommendation 4

Policymakers should develop blueprints for national post-deployment monitoring functions

Challenge 5

Decentralized nature of open model deployment

Recommendation 5

Conduct research into methods for collecting information on open model impacts

Looking forward

PAI will continue to improve collective understanding of the field and drive accountability through the development of future progress reports. If you would like to know more, please contact policy@partnershiponai.org.

Acknowledgments

This report reflects the collaborative efforts of our multistakeholder community. We are grateful to our working group members who contributed their expertise and dedicated their time to shape this work. We also thank the experts who provided critical insights through our initial questionnaire and Policy Forum workshop. This work was developed with guidance from the Policy Steering Committee. For a complete list of contributors, please see the full report.