Content Type: Report

Documenting the Impacts of Foundation Models

A Progress Report on Post-Deployment Governance Practices

Albert Tanjaya, Jacob Pratt
Download the Report

Web pages related to ChatGPT were viewed over 3.7 billion times in January 2025, making it the 13th most viewed domain on the internet. Meanwhile, billions of people use products powered by foundation models, such as Gemini in Google Search and Meta AI in Meta products. However, we are just beginning to understand the use and societal impact of these models after they have been deployed.

What are people using these systems for? How are they helping people do things better? What are the most common or severe harms they cause?

This report on the impacts of foundation models explores the progress of the field in answering these questions, and is PAI’s inaugural Progress Report. It provides key insights for different audiences, such as:

Policymakers
  • analysis of the current adoption of four documentation practices
  • recommended actions to take over the next 6 – 24 months
Model providers, model deployers, or actors in the value chain
  • benefits of adopting four documentation practices
  • current challenges to adopting these practices
  • recommended actions to take to improve foundation model governance
  • over 70 examples of how organizations collect and document information on the impacts of foundation models
Academic or civil society researchers or interested users of AI models and systems
  • open questions that merit further exploration
  • information on benefits and challenges to support future research
  • over 70 examples of how organizations collect and document information on the impacts of foundation models

Key Findings

Developed with the input of over 30 different stakeholders across academia, civil society, and industry, the report builds on PAI’s Guidance for Safe Foundation Model Deployment and Policy Alignment on AI Transparency work, and produced the following key findings.

Practices

The report highlights four key practices for documenting the impacts of foundation models.

Practice 1

Share usage information

E.g.

  • Activity data (input data; output data)
  • Usage by geography, sector, and use case
  • Total chat time usage
  • Information on downstream applications.
Practice 2

Enable and share research on post-deployment societal impact indicators

E.g.

  • Labor impact indicators
  • Environmental impact indicators
  • Synthetic content impact indicators
Practice 3

Report incidents and disclose policy violations

E.g.

  • Safety incidents
  • Violations of terms of use and policies
  • Mitigation and remediation actions
Practice 4

Share user feedback

E.g.

  • How users submit feedback
  • Feedback received and type
  • How provider utilizes feedback

Challenges & Recommendations

The main challenges for collecting and sharing information on post-deployment impacts can be grouped into five themes. To overcome these challenges and ensure the benefits of effective post-deployment governance, we recommend that stakeholders take the following actions to move the field forward.

Challenge 1

Lack of standardization and established norms

Recommendation 1

Define norms for the documentation of post-deployment impacts through multistakeholder processes, which may be formalized through technical standards

Challenge 2

Data sharing and coordination barriers

Recommendation 2

Explore mechanisms for responsible data sharing

Challenge 3

Misaligned incentives

Recommendation 3

Policymakers should explore where guidance and rules on documenting post-deployment impacts are needed

Challenge 4

Limited data sharing infrastructure

Recommendation 4

Policymakers should develop blueprints for national post-deployment monitoring functions

Challenge 5

Decentralized nature of open model deployment

Recommendation 5

Conduct research into methods for collecting information on open model impacts

Looking forward

PAI will continue to improve collective understanding of the field and drive accountability through the development of future progress reports. If you would like to know more, please contact policy@partnershiponai.org.

Acknowledgments

This report reflects the collaborative efforts of our multistakeholder community. We are grateful to our working group members who contributed their expertise and dedicated their time to shape this work. We also thank the experts who provided critical insights through our initial questionnaire and Policy Forum workshop. This work was developed with guidance from the Policy Steering Committee. For a complete list of contributors, please see the full report.

Policy Alignment on AI Transparency

Analyzing Interoperability of Documentation Requirements across Eight Frameworks

DOWNLOAD THE REPORT

As governments and organizations worldwide race to develop policy frameworks for foundation models, we face a juncture that demands both swift action and careful coordination. However, without coordination, we risk creating an inconsistent patchwork of frameworks and divergent understandings of best practices.

Ensuring these frameworks work together is critical.

Partnership on AI’s Policy Alignment on AI Transparency conducts a comparative analysis of eight leading policy frameworks for foundation models, with a particular focus on documentation requirements, which are a critical lever for achieving transparency and safety.

In this report, we analyze current and potential near-term interoperability challenges between the documentation requirements in leading policy frameworks, and offer recommendations that aim to promote interoperability as well as establish a common baseline for best practices for accountability and transparency. However, we recognize that this is only the beginning of a much larger conversation. Achieving global interoperability will require ongoing efforts and substantial input, particularly from the Global Majority, to reflect diverse perspectives and priorities.

The full report provides a comprehensive exploration of interoperability challenges, including the nuances of our methodology, key findings, and detailed recommendations. This summary serves to highlight our most salient conclusions, aiming to inform and guide ongoing policy discussions and decision-making in this rapidly evolving field.

As we navigate the complex landscape of AI governance, the need for coordinated, interoperable policy frameworks becomes increasingly clear. By working together across borders and sectors, we can create a more coherent, effective approach to managing the risks and harnessing the potential of foundation models, ensuring accountability, transparency and fostering innovation in the global AI ecosystem.

⇩ Jump to Summary of Recommendations ⇩

Read the Report

Download the complete Policy Alignment on AI Transparency report (41 pages) as a PDF.
Download the full report

Without coordination, we risk creating an inconsistent patchwork of frameworks and divergent understandings of best practices.

Frameworks Reviewed

Documentation and transparency play a key role in managing risk for foundation models, and are a common feature of policy frameworks.

To explore how documentation guidance is being incorporated in current policy frameworks, we compared key frameworks from the US, EU, UK, and select multilateral initiatives (see Table 1). The table outlines each framework’s provisions for foundation model documentation, ranging from high-level transparency guidelines to specific artifact requirements. It also indicates ongoing development processes for each framework.

Table 1. Frameworks reviewed in this report

A B C D E
Multilateral
OECD AI Principles •
Seoul Frontier AI Safety Commitments • • •
Hiroshima AI Process Code of Conduct • • • •
Council of Europe AI Convention •
Regional
EU AI Act • • •
National
US AI Executive Order 14110 • • •
NIST AI RMF (with Gen-AI Profile) • • •
UK AI White Paper and followup • •
A: Contains high-level commitments to transparency
B: Requires/recommends documentation practices
C: Requires documentation artifacts
D: Further/more detailed provisions proposed or in development
E: Specifically addresses foundation models

Mapping Documentation Requirements Across the Frameworks

Documentation guidance and requirements under the reviewed frameworks are summarized in Table 2A below. Key findings include:

  • Documentation is a common feature of the frameworks, though this is couched in various terms. Several of the high-level frameworks recommend providing certain kinds of information to various actors; some require recording and/or reporting of information, and some require the preparation of specific documentation artifacts.
  • The most commonly referenced artifacts are technical documentation, instructions for use, information about datasets, and incident reports. However, there is little detail in most of the frameworks about what should be included in each of these documents and the form each document should take.
  • This analysis suggests that there is an opportunity to develop standardized requirements for some of the key documentation artifacts required across frameworks – provided that agreement can be reached about what the content of these artifacts should be.

Tables 2A, 2B and 2C contain a comparison of documentation requirements across in-scope frameworks. Specific documentation artifacts are shown in red. The principal documentation guidelines from PAI’s Model Deployment Guidance are included as a comparator.

Table 2A. Comparison of documentation requirements across in-scope frameworks

Stage in AI Lifecycle Framework
PAI Model Deployment Guidance EU AI Act AI Executive Order NIST RMF and Generative AI Companion Hiroshima Code of Conduct Seoul Frontier AI Commit­ments COE Convention OECD AI Principles UK AI White Paper, AI Principles, Response
R&D • • •
Pre-deployment/ on deployment • • • • • • • •
Post-deployment • • • • • •
Across lifecycle • • • •
Unspecified • • • •
• Documentation requirements for specific stage in the AI lifecycle
• Specific documentation artifacts
• General documentation requirements
Table 2B. Comparison of documentation requirements across in-scope frameworks
Specific documentation artifacts are shown in red. The principal documentation guidelines from PAI’s Model Deployment Guidance are included as a comparator.

Stage in AI Lifecycle Framework
PAI Model Deployment Guidance EU AI Act AI Executive Order NIST RMF and Generative AI Companion Hiroshima Code of Conduct
R&D Pre-system card: Planned testing, evaluation, and risk management procedures for foundation/ frontier models prior to development. Including:

  • Intended training data approach
  • Responsible AI practices
  • Development Team
  • Written “safety case”
 Notify EU Commission of models with systemic risk Report dual-use models to Department of Commerce; report cybersecurity protections N/A N/A
Pre-deployment/ on deployment

Publicly report model impacts

“Key ingredient list”: including details of evaluations, limitations, risks, compute, parameters, architecture, training data approach, model documentation

Disclose performance benchmarks, intended use, risks and mitigations, testing and evaluation methodologies, environmental and labor impacts

Downstream use documentation: including appropriate uses, limitations, mitigations, safe development practices

Share red-teaming findings

Technical documentation: including information about training, testing, and evaluations

Documentation for downstream developers: including information about capabilities, limitations, and to aid downstream compliance

Public summary of training data

 Report red-teaming results to Department of Commerce Multiple guidelines for documentation, including of:

  • Risks and potential impacts
  • Knowledge limits
  • TEVV considerations & tools
  • Measures of trustworthiness
  • Residual risks after mitigations
  • Model details
  • Data curation policies
  • Environmental impacts

Technical documentation

Transparency reports: with “meaningful information”

Instructions for Use

Technical Documentation

Documentation to include details of evaluations, capabilities/ limitations re: domains of use; risks to safety and society; red-teaming results
Post-deployment

Incident reporting

Transparency reporting (frontier model usage and policy violations)

 Serious incident reports N/A

Incident and performance reporting

Transparency reports with steps taken to update generative AI systems

 Maintain “appropriate documentation” of reported incidents
Across lifecycle

Iteration of model development

Provide documentation to government as required

Environmental Impacts

Severe labor market risks

Human rights impact assessments

 N/A N/A Multiple guidelines to document processes and management systems “Work towards” information sharing and incident reporting, including on:

  • Evaluation reports
  • Safety & security risks
  • “ensuring appropriate and relevant documentation and transparency across the AI lifecycle”

Document datasets, processes and decisions during development

Regularly update Technical Documentation
Table 2C. Comparison of more general documentation and transparency requirements, at unspecified stages of the AI lifecycle

Framework
Seoul Frontier AI Commit­ments COE Convention OECD AI Principles UK AI White Paper, AI Principles, Response

Publicly report model or system capabilities, limitations, and domains of appropriate and inappropriate use

Provide public transparency on implementation of commitments, including on:

  • Risk assessments, effectiveness of mitigations, evaluation results
  • Risk thresholds
  • Approach to mitigations
  • Processes to follow if risk thresholds are met/ exceeded
 Countries ratifying the convention must have frameworks (such as national laws) that:

  • Contain documentation requirements that will allow people to seek remedies for human rights violations
  • Require developers to adopt measures to identify, prevent, and mitigate risk. These measures are to include documentation of risks and mitigations
 Principles include:

Transparency and Explainability:

  • “Provide meaningful information” to “foster understanding of AI Systems”
  • “Provide plain and easy-to-understand information on the sources of data/input, factors, processes and/or logic”
  • “Provide information [to] enable those adversely affected by an AI system to challenge its output.”

Accountability:

  • “Ensure traceability, including in relation to datasets, processes and decisions made during the AI system lifecycle”
 Provide transparency and accountability, including “documentation on key decisions throughout the AI system life cycle”

Other Features of the Frameworks Relevant to Interoperability

In reviewing the in-scope frameworks, a number of additional factors emerged including their binding nature, enforcement mechanisms, scope of applicable models, overseeing institutions, development processes, and emphasis on collaboration, as detailed in Table 3.

Table 3. In-scope frameworks
Normative status, coverage/thresholds, reference to international standardization processes and collaboration/interoperability

Framework PAI Model Deployment Guidance EU AI Act AI Executive Order NIST RMF and Generative AI Companion Hiroshima Code of Conduct UK AI White Paper, Consultation Response OECD Seoul frontier AI Commitments
Binding or Voluntary? Voluntary Binding Partly binding Voluntary Voluntary Voluntary (guidance for sectoral regulators) Voluntary Voluntary
Coverage Foundation models (with guidance tailored according to three capability bands and four release strategies). The most stringent guidance applies to “paradigm-shifting or frontier” models

General- purpose AI models (baseline requirements)

“General- purpose AI models with systemic risk”

 “Dual-use foundation models” AI systems (NIST AI RMF)

Generative foundation models (Gen-AI Profile)

 “The most advanced AI systems, including the most advanced foundation models and generative AI systems”

AI systems; generally a sectoral approach

Initial focus of UK AISI on advanced systems

Planned laws for “the most powerful AI systems”

 AI Systems “Frontier AI” – “highly capable general- purpose AI models or systems that can perform a wide variety of tasks and match or exceed the capabilities present in the most advanced models”
Initial Threshold N/A

None (baseline requirements)

10^25 FLOPs (models “with systemic risk”)

 10^26 FLOPs (10^23 FLOPs for models trained on biological sequence data) N/A N/A N/A N/A N/A
Institutions/ Oversight N/A AI Office Dept of Commerce (for reporting requirements) N/A OECD (monitoring mechanism under development) AISI N/A N/A
Next Steps N/A

Codes of Practice for GPAI due August 2025

Templates for training data (AI Office)

Harmonized standards

Delegated acts – thresholds for GPAI with systemic risk; documentation requirements

Various, including:

OMB materials for federal procurement

Copyright guidance

Dept of Commerce can change threshold for dual-use model reporting

 NIST/the NIST AISI have a broad work plan including developing tools, evaluations, metrics

COC to be iterated by G7 HAIP

OECD developing monitoring mechanism

 Intention to legislate announced re advanced models, and to place AISI on statutory footing OECD developing Due Diligence Guidance (DDG) for AI under OECD Responsible Business Conduct (RBC) guidelines

AI Action Summit February 2025 (France)

AI Safety Science Report to be published at AI Action Summit
Commitment to cooperation/ collaboration? Collaborate with cross-sector Al stakeholders re risk identification, methodologies, best practices, standardization Mandates creation of AI Board, Advisory Forum; multistakeholder participation in development of Codes of Practice and harmonized standards Under EO, NIST released plan for global engagement on AI standards; Secretary of State is developing Global Development Playbook; EO contained several consultation requirements

Several references to collaboration e.g. with external researchers, industry experts, and community representatives about best risk measurement and management practices

NIST is committed to collaboration/ cooperation, e.g. through AISI Consortium and pending Network of AISIs

 Across sectors, including on research to assess/ adopt risk mitigations, document incidents, and share information with the public to promote safety Focus on collaboration across government, stakeholder groups, and internationally OECD convenes the Network of Experts Information sharing, collaboration on safety research (Seoul AI Principles)
Commitment to standards Development and adoption of standards EU harmonized Standards—though EU committed to adopting international standards where possible

NIST is required to develop standards

Under EO, NIST has released plan for global engagement on promoting and developing AI standards

Contains references to considering relevance of standards (including NIST frameworks)

NIST will continue to align AI RMF with international standards

 Advance development and adoption of standards Support for work on assurance techniques and technical standards Governments should promote standards development Contribute to/ take account of international standards

Summary of Findings

  • Interoperability and collaboration are explicitly included as policy goals in a number of the international frameworks, though there is no current agreement on how those goals will be achieved.
  • The frameworks emphasize the importance of documentation for foundation models, but remain light on detail about the form and content it should take.
  • There are a number of steps that could be taken to advance interoperability now and in the future, leveraging existing and proposed forums, mechanisms and processes.
  • An early focus should be on agreeing upon thresholds for regulation, to provide international consistency about which foundation models are captured by regulatory and policy frameworks.
  • While there are some challenges to relying on international standardization processes to align AI policy frameworks, standards remain an important plank in that effort.
  • Harmonizing key documentation requirements across national, regional, and international foundation model policy frameworks – and in particular, harmonizing the form and content of documentation artifacts – should be made a priority.
  • The lack of consensus on the best approaches to manage AI risks is a significant challenge to developing interoperable frameworks, including for documentation.
  • The existing and newly announced AI Safety Institutes can establish a foundation for AI safety consensus through research, evaluations, scientific advancement, and collaborative development of safety standards and documentation practices.
  • Participation by civil society and the global community is needed in all major foundation model policy initiatives if we are to ensure that they lead to alignment around best practices, and that the agenda for global interoperability is not set by a comparatively small group of nations from the Global North.

Participation by civil society and the global community is needed in all major foundation model policy initiatives.

Summary of Recommendations

  • National governments and the EU should prioritize cooperation in setting thresholds for identifying which foundation models require additional governance measures, including through supporting the OECD’s work on this issue. The AI Summit Series could also be used to take this forward. Agreeing on a common definition, and thresholds, for the models covered by policy frameworks, should flow through to greater alignment between the frameworks, including in relation to documentation requirements.
  • The G7 Presidency should continue developing the Hiroshima Code of Conduct into a more detailed framework to provide more detail about thresholds, relevant risks, and the form and content of documentation artifacts. This work should be a focus of Canada’s G7 Presidency in 2025, including aligning closely with the EU Code of Practice development timeline. In doing this, it should seek input from foundation model providers, civil society, academia and other stakeholder groups equally.
  • When creating and approving initial Codes of Practice for the EU AI Act, all involved parties should prioritize compatibility with other major AI governance frameworks where possible. The involvement of non-EU model providers, experts and civil society organizations will help advance this objective.
  • To support the development of standardized documentation artifacts, Standards Development Organizations should ensure that their processes are informed by socio-technical expertise and diverse perspectives as well as required resources. To that end, SDOs, industry, governments, and other bodies should invest in capacity building for civil society and academic stakeholders to engage in standards-making processes, including to ensure participation from the Global South.
  • The development of standardized documentation artifacts for foundation models should be a priority in AI standardization efforts. This would promote internationally comparable documentation requirements for foundation models – promoting interoperability and establishing a baseline for best practice internationally.
  • International collaboration and research initiatives should prioritize research that will support the development of standards for foundation model documentation artifacts. Documentation is a key feature of foundation model policy requirements, and common requirements for artifacts will directly improve interoperability. It will also make comparisons between models from different countries easier, promoting accountability and innovation.
  • National governments should continue to prioritize both international dialogue and collaboration on the science of AI Safety, however with more specificity and tracking of progress on commitments that will foster good practice. This work will inform a common understanding of what should be included in documentation artifacts to promote accountability and address foundation model risks.
  • National governments should support the creation/development of AI Safety Institutes (or equivalent bodies), and ensure they have the resources, functions, and powers necessary to fulfill their core tasks. Efforts should be made to align the functions of these bodies with those common among existing AISIs. This will promote efforts to develop trusted mechanisms to evaluate advanced foundation models, and may, at a later stage, lead to the potential to work towards “institutional interoperability.”
  • The Network of AISIs (and bodies with equivalent or overlapping functions such as the EU AI Office) should be supported and efforts should be made to expand its membership. Consideration should be given to how the Network could support broader AI Safety research initiatives.

Background and Methodology

The work plan leading to this report was developed with guidance from PAI’s Policy Steering Committee. This report has been informed through desk research and consultations with experts from industry, civil society, academia, and non-profit organizations, drawn from PAI’s partner and wider stakeholder networks. We tested our initial thinking in a virtual multistakeholder workshop in August 2024. The views and recommendations in this report remain those of PAI.

For a comprehensive exploration of interoperability challenges, including our methodology, key findings, detailed recommendations, and more, please download the full report. To stay in touch with our latest policy work, sign up here.

DOWNLOAD THE FULL REPORT

The development of standardized documentation artifacts for foundation models should be a priority in AI standardization efforts.

Eyes Off My Data

Exploring Differentially Private Federated Statistics To Support Algorithmic Bias Assessments Across Demographic Groups

PAI Staff
Download the Paper

Executive Summary

Executive Summary

Designing and deploying algorithmic systems that work as expected every time for all people and situations remains a challenge and a priority. Rigorous pre- and post-deployment fairness assessments are necessary to surface any potential bias in algorithmic systems. As they often involve collecting new user data, including sensitive demographic data, post-deployment fairness assessments to observe whether the algorithm is operating in ways that disadvantage any specific group of people can pose additional challenges to organizations. The collection and use of demographic data is difficult for organizations because it is entwined with highly contested social, regulatory, privacy, and economic considerations. Over the past several years, Partnership on AI (PAI) has investigated key risks and harms individuals and communities face when companies collect and use demographic data. In addition to well-known data privacy and security risks, such harms can stem from having one’s social identity being miscategorized or data being used beyond data subjects’ expectations, which PAI has explored through our demographic data workstream. These risks and harms are particularly acute for socially marginalized groups, such as people of color, women, and LGBTQIA+ people.

Given these risks and concerns, organizations developing digital technology are invested in the responsible collection and use of demographic data to identify and address algorithmic bias. For example, in an effort to deploy algorithmically driven features responsibly, Apple introduced IDs in Apple Wallet with mechanisms in place to help Apple and their partner issuing state authorities (e.g., departments of motor vehicles) identify any potential biases users may experience when adding their IDs to their iPhones.＊IDs in Wallet, in partnership with state identification-issuing authorities (e.g., departments of motor vehicles), were only available in select US states at the time of the writing of this report.

In addition to pre-deployment algorithmic fairness testing, Apple followed a post-deployment assessment strategy as well. As part of IDs in Wallet, Apple applied differentially private federated statistics as a way to protect users’ data, including their demographic data. The main benefit of using differentially private federated statistics is the preservation of data privacy by combining the features of differential privacy (e.g., adding statistical noise to data to prevent re-identification) and federated statistics (e.g., analyzing user data on individual devices, rather than on a central server, to avoid the creation and transfer of datasets that can be hacked or otherwise misused). What is less clear is whether differentially private federated statistics can attend to some of the other risks and harms associated with the collection and analysis of demographic data. To understand this, a sociotechnical lens is necessary to understand the potential social impact of the application of a technical approach.

This report is the result of two expert convenings independently organized and hosted by PAI. As a partner organization of PAI, Apple shared details about the use of differentially private federated statistics as part of their post-deployment algorithmic bias assessment for the release of this new feature.

During the convenings, responsible AI, algorithmic fairness, and social inequality experts discussed how algorithmic fairness assessments can be strengthened, challenged, or otherwise unaffected by the use of differentially private federated statistics. While the IDs in Wallet use case is limited to the US context, the participants expanded the scope of their discussion to consider differential private federated statistics in different contexts. Recognizing that data privacy and security are not the only concerns people have regarding the collection and use of their demographic data, participants were directed to consider whether differentially private federated statistics could also be leveraged to attend to some of the other social risks that can arise, particularly for marginalized demographic groups.

The multi-disciplinary participant group repeatedly emphasized the importance of having both pre- and post-deployment algorithmic fairness assessments throughout the development and deployment of an AI-driven system or product/feature. Post-deployment assessments are especially important as they enable organizations to monitor algorithmic systems once deployed in real-life social, political, and economic contexts. They also recognized the importance of thoughtfully collecting key demographic data in order to help identify group-level algorithmic harms.

The expert participants, however, clearly stated that a secure and privacy-preserving way of collecting and analyzing sensitive user data is, on its own, insufficient to deal with the risks and harms of algorithmic bias. In fact, they expressed that such a technique is not entirely sufficient for dealing with the risks and harms of collecting demographic data. Instead, the convening participants identified key choice points facing AI-developing organizations to ensure the use of differentially private federated statistics contributes to overall alignment with responsible AI principles and ethical demographic data collection and use.

This report provides an overview of differentially private federated statistics and the different choice points facing AI-developing organizations in applying differentially private federated statistics in their overall algorithmic fairness assessment strategies. Recommendations for best practices are organized into two parts:

  1. General considerations that any AI-developing organization should factor into their post-deployment algorithmic fairness assessment
  2. Design choices specifically related to the use of differentially private federated statistics within a post-deployment algorithmic fairness strategy

The choice points identified by the expert participants emphasize the importance of carefully applying differentially private federated statistics in the context of algorithmic bias assessment. For example, several features of the technique can be determined in such a way that reduces the efficacy of the privacy-preserving and security-enhancing aspects of differentially private federated statistics. Apple’s approach to using differentially private federated statistics aligned with some of the practices suggested during the expert convenings: the decision to limit the data retention period (90 days), allowing users to actively opt-into data sharing (rather than creating an opt-out model), clearly and simply sharing what data the user will be providing for the assessment, and maintaining organizational oversight of the query process and parameters.

The second set of recommendations surfaced by the expert participants primarily focus on the resources (e.g., financial, time allocation, and staffing) necessary to achieve a level of alignment and clarity on the nature of “fairness” and “equity” AI-developing organizations are seeking for their AI-driven tools and products/features. While these considerations may seem tangential, expert participants emphasized the importance of establishing a robust foundation on which differentially private federated statistics could be effectively utilized. Differentially private federated statistics, in and of itself, does not mitigate all the potential risks and harms related to collecting and analyzing sensitive demographic data. It can, however, strengthen overall algorithmic fairness assessment strategies by supporting better data privacy and security throughout the assessment process.

Eyes Off My Data

Executive Summary

Introduction

The Challenges of Algorithmic Fairness Assessments

Prioritization of Data Privacy: An Incomplete Approach for Demographic Data Collection?

Premise of the Project

A Sociotechnical Framework for Assessing Demographic Data Collection

Differentially Private Federated Statistics

Differential Privacy

Federated Statistics

Differentially Private Federated Statistics

A Sociotechnical Examination of Differentially Private Federated Statistics as an Algorithmic Fairness Technique

General Considerations for Algorithmic Fairness Assessment Strategies

Design Considerations for Differentially Private Federated Statistics

Conclusion

Acknowledgments

Funding Disclosure

Appendices

Appendix 1: Fairness, Transparency and Accountability Program Area at Partnership on AI

Appendix 2: Case Study Details

Appendix 3: Multistakeholder Convenings

Appendix 4: Glossary

Appendix 5: Detailed Summary of Challenges and Risks Associated with Demographic Data Collection and Analysis

Table of Contents
1
2
3
4
5
6
7
8
9
10