Creating Equitable AI Systems with the Participatory & Inclusive Demographic Data Guidelines

AI’s integration into everyday life has become inevitable. AI has permeated nearly every aspect of life, from employment and healthcare to social media and transportation. The rapid adoption of AI and the race to innovate these advanced systems has left users vulnerable to the risks of algorithmic discrimination — the systematic distortion in data or a system’s development that causes unjust outcomes or harms to marginalized groups of people. Algorithmic discrimination disproportionately affects marginalized communities such as Black, Indigenous, and other people of color, LGBTQIA+ communities, and women. To address this issue and create more equitable AI systems, developers often collect users’ demographic data to assess the AI system’s impacts across various identity groups. However, collecting sensitive demographic data from users can cause additional harm, particularly to people already impacted by algorithmic discrimination.

To address these challenges, Partnership on AI developed the Participatory & Inclusive Demographic Data Guidelines. The guidelines provide AI developers, teams within technology companies, and other data practitioners with guidance on how to collect and use demographic data for fairness assessments to advance the needs of data subjects and communities, particularly those most at risk of harm from algorithmic systems. Organized around the demographic data lifecycle, the guidelines identify in each stage the key risks faced by data subjects and communities (especially marginalized groups), baseline requirements and recommended practices that organizations should undertake to prevent these risks, and guiding questions that organizations can use to achieve the recommended practices. An accompanying Implementation Workbook and Case Study provide further guidance for developers.

Read the Guidelines

The Guidelines were developed in collaboration with our multistakeholder community. A working group of 16 experts, representing perspectives from the technology industry, academia, civil society, and government offices across six countries (US, UK, Canada, South Africa, the Netherlands, and Australia) convened monthly to draft each component of the Guidelines. Feedback was gathered from attendees at workshops, and through a public comment period which was held last year from May to December. Seven equity experts, who specialize in topics such as data justice, AI ethics in the Majority World, racial justice, LGBTQ+ justice, and disability rights were commissioned to advise on the development of the resources.

These resources were first released for public comment in Spring 2024. During the public comment period, we received useful feedback that led to the following changes:

• Additional emphasis on intersectionality throughout the resource.
• Inclusion of collective rights in our definition of data justice.
• Strengthened definition of accessible consent to emphasize the need to account for various disabilities.
• Expanded definition of demographic data to include age.
• Additional context on target audience for this resource.

We would like to thank the people who submitted comments during our public comment period, the working groups that met to develop these resources, and the expert reviewers who contributed valuable time. We would like to give a special thanks to Research ICT Africa and Google Equitable AI Research Roundtable whose comments were instrumental in improving the guidelines.

The Participatory and Inclusive Demographic Data Guidelines are part of a larger multi-year workstream on Demographic Data and Algorithmic Fairness. While this initiative has drawn to a close, the multistakeholder insights and ethical considerations surfaced through the Demographic Data and Algorithmic Fairness workstream will serve as important groundwork used to shape our research and programming across PAI, especially in our data supply chain, AI safety, and participatory and inclusive AI initiatives. We look forward to building upon these learnings and ensuring AI development benefits all. To stay up to date on our progress in this space sign up for our newsletter.