Why is it so challenging for tech companies to incorporate inclusive methods into their AI/ML development, and where do traditional participatory design methods fall short?

The Inclusion Illusion: How to Approach Impacted Communities When Building AI/ML Systems

Participatory design approaches used in technology development have been around since the 1970s, relying on different stakeholder engagement practices like interviews, focus groups, user surveys, and system evaluations. User Experience research, today, is an important way for companies to understand how someone uses, interacts with, and generally experiences a product or service.

Whether beta testing a new product with a set of trusted users or conducting focus groups with potential customers to get a better idea of what users want and need in the latest iteration of a product, this collaboration between developer and user is key to producing and launching a successful AI/ML product (or any product).

However, it is important to acknowledge that participation is not the same thing as inclusion when it comes to technology development.

Participation is broadly concerned with getting others involved; it is non-specific about who is involved, for how long, and to what end. In other words, participation itself is not tied to any particular value commitments, other than the belief that more input will result in a better outcome (in this case, product or service). For this reason, metrics of participation often focus on how many are involved in the development process.

Inclusion, on the other hand, is participation that is specifically oriented towards achieving a sense of integration within a group or institution. Within the framework of “diversity and inclusion,” inclusion means creating an environment in which people of many different backgrounds, experiences, and expertise, are involved and empowered to make decisions within the group or organization.

Achieving a sense of community consultation and empowerment within deployed AI/ML systems requires more than simply soliciting feedback from stakeholders.

It requires mapping out pathways through which stakeholders can actually directly engage in the decision-making process, ideally becoming one of many decision-makers and directly influencing the creation and/or deployment of the technology. This means expanding participation beyond whom we normally consider “experts” on AI/ML technology, and identifying how those who hold non-technical knowledge and experiences can become necessary contributors to the development of successful AI/ML systems. It also means cultivating trustworthy relationships between everyone involved so different insights and opinions, including ones that may run counter to existing assumptions, can be freely shared and incorporated into the broader pool of knowledge used to inform the development of a new AI/ML system.

Inclusion also requires acknowledging that exclusion exists – that not everyone who should participate in the technology development process is allowed or able to participate to the same degree. Exclusion can occur for many reasons, ranging from a lack of awareness around who else could be included and historic practices rooted in biases and prejudices to institutional policies that explicitly seek to keep certain people out of technology development.

This attention to the power dynamics that privilege the needs and opinions of some groups over others is an important distinction between simple participation and full-fledged inclusion.

In short, participation often assumes that anyone who wants to get involved simply can. Inclusion recognizes this is not always the case and that special accommodations must be made to make sure that no one is left out of the process.

The Benefits and Limitations of Participatory Design: Lessons from the Tech Policy Lab’s Diverse Voices Method

Since PAI’s earliest days, we received a lot of feedback across the Partnership that teams wanted to get more input from marginalized communities and stakeholders who are not always consulted. Part of the promise of PAI for partner organizations is our ability to increase the diversity of input into the field, more broadly, and their projects, more specifically. For example, the researchers behind foundational ABOUT ML papers pushed PAI to think beyond the existing model of bringing the “usual suspects” to the table and insisted we think more about – and learn directly from – the communities who are impacted by the use of various AI/ML technology.

PAI’s first attempt to address this gap was through working with the Tech Policy Lab at the University of Washington, a PAI Partner with extensive expertise in applying value-sensitive design approaches to technology policy. In 2019, the ABOUT ML Team at PAI worked with the Tech Policy Lab to implement their Diverse Voices methodology within PAI’s ABOUT ML project. The aim was to explicitly solicit views and feedback from communities who are often the least likely to be consulted in the formation of machine learning system documentation practices that may impact them. The Diverse Voices consultants we worked with coordinated three experiential expert panels in Seattle, WA to review and comment on the first draft of the ABOUT ML report.

Through that experience of applying the Diverse Voices method to the ABOUT ML V0 document, we learned a lot from the careful way the Tech Policy Lab team applies their methodology. Specifically, it underscored for us two crucial benefits of participatory methods in technology development:

• It generates important and meaningful insights for tech policy documents, highlighting potential harm or unusability, as well as other uses that were not previously considered.
• It leverages the expertise of groups that are historically excluded from the development and deployment of technology in mitigating future harm from the use of that technology.

However, we also realized that our broader Partner community could benefit from an even larger toolkit of participatory methods. As currently conceived, the Diverse Voices methodology can be both resource and time-intensive and is not always applicable across the research, design, and development stages of a given project.

While Diverse Voices is respectfully attentive to the work generated by non-technical stakeholders – making sure their time and experiential expertise is appropriately compensated – the inherent power dynamic between the project and the team implementing Diverse Voices cannot guarantee (not for lack of trying from the Diverse Voices’ team) that a final technology product will reflect the input given by non-technical stakeholders.

Introducing Methods for Inclusion: Expanding the Possibilities of Participatory Design

The desire to engage the public (and non-specialists) in projects – be it policy-making, product development, or research more generally – is not new, nor is it limited to those in the AI and technology sectors.

Meaningfully including impacted communities enables AI/ML developers to provide an array of products and services that can better meet the needs of ever-diversifying populations around the world, without further deepening existing social inequalities or generating harm.

In order to further research these topics, we created the Methods for Inclusion Fellowship to commit time and resources to research and create materials to capture the nuances of participation and inclusion. Building from the lessons we’ve learned from the Diverse Voices team and the work of other responsible AI advocates, PAI is launching Methods for Inclusion, a new research project that aims to enable AI researchers and developers to more effectively and ethically engage with a broad base of constituents and stakeholders in the development of their AI/ML projects.

Working with communities to develop products and services early and throughout the development process helps AI/ML developers mitigate potential harm or negative impact. A truly inclusive approach can help developers build long-lasting, trusting relationships with the people they want to ultimately serve through their technology.

With Methods for Inclusion, we are broadening the aperture in recognition of the other scholarship that exists on the topic of inclusion in various domains. The project builds on the existing work of scholars, practitioners, and most importantly, advocates, who have, for years, tried to open up AI/ML development to people outside of the close circle of engineers and developers. This project is multidisciplinary in nature, learning from fields outside of computer science and technology that have grappled with questions of participation and inclusion for many decades. This includes fields like civic governance, education, planning and policy, public health/healthcare, and the social sciences. It also takes insights and guidance from community organizing, which can cover many topics and disciplines.

Over the next year, the Methods for Inclusion project will work to:

1. identify a range of participatory practices from different fields that could be adapted for use by AI researchers, designers, and developers;
2. better understand the challenges of incorporating inclusive methods into AI development, with a specific eye towards the different barriers and incentives facing AI developers, on the one hand, and members of impacted communities on the other; and
3. create real-life case study resources that outline attempted participatory methods, the challenges faced by each, and the improvements experienced by companies as a result.

Ultimately, we hope to place AI developers and community members invested in avoiding potential harm resulting from AI/ML systems into conversation with one another.

Drawing from Method for Inclusion Research Fellow Tina Park’s research background in sociology and urban planning, we are excited to launch this multidisciplinary project at the Partnership on AI, work that will build upon PAI’s mission to advance responsible AI while uplifting the voices and perspectives of those traditionally excluded from the process.

PAI exists to serve not just our Partner organizations, but also for the individual people who are passionately dedicating their careers and time to changing how technology companies operate. Methods for Inclusion aims to serve those individual champions by providing them with a thoroughly researched resource that they can use in their day to day work.

Calling All AI Researchers, Developers, and Community Advocates: Get Involved in Making AI/ML Development More Meaningfully Inclusive

We are currently recruiting participants for PAI’s research study on Methods for Inclusion and we would love to hear from you.

We are specifically interested in learning from:

• responsible AI researchers and AI/ML developers who have incorporated some type of participatory practice in their work, and
• advocates who are not directly involved in AI/ML research or development and work with communities who are subject to the negative consequences of AI/ML use.

If you are interested in becoming a participant, complete this survey (and share with others!). Please visit the project FAQ for more details about the project’s privacy policy, data management plan, and more.