Guidance for Inclusive AI

While bringing greater transparency into the development process is a part of ethical development practices, opening up the process in and of itself does not resolve the many issues that might concern members of the public. In fact, public participation can, and historically has, harmed marginalized communities (e.g., through exploitation of intellectual labor from socially marginalized participants, as well as the use of public collaborations to “ethics-wash.” The engagement process can also be unintentionally coercive, whereby participants may feel compelled to support any suggestions brought forward by the company or not to share critical feedback candidly. The degree to which a public participation engagement is harmful or empowering will depend on its engagement practices, such as how people are treated, how power relations between the organizer and the participants are maintained or disrupted, and what is produced through the engagement (output or outcome).

• Engaging with members of the public will automatically result in better relationships with users or clients, improve the product, or minimize harm.
• Interacting with a tech company does not impact the lives of the participants (negatively or positively).
• Participants will feel comfortable or empowered to provide critical feedback.

• Before recruiting for and launching a public engagement strategy, have explicit internal (within the team and company) discussions about intent of the engagement and delimit the scope of what findings and other data can be used for.
• Identify and address ways in which participants may experience coercive pressure (e.g., people may feel compelled to provide only positive or supportive feedback, or not share any critical feedback or concerns) throughout the engagement process; consult with experts and other community/group leaders, including experts on participatory engagement practices.
• Conduct an impact assessment of the public engagement strategy (with an expert internal team or with the support of external intermediaries) to identify likely risks or harms to participants that need to be addressed and/or mitigated in advance.

While most AI developers do not intend their products to be harmful, it is possible to build AI-driven tools that harm others or make it significantly easier to create harm for others. Such damage can arise through direct bodily or psychological injury (e.g., AI-driven tools used in warfare to kill), harm to someone’s reputation or public standing (e.g., tools used to create pornographic or other types of “deepfakes”), or threats to autonomy (e.g., the threat of constant surveillance through tools enabled by facial recognition), among many others. It is antithetical to include people in the design of products that are actively harmful to their well-being; their insights should not be used to further enhance the means through which they and members of their communities will be harmed.

• Significant harm is unlikely to arise simply because the intentions behind the project are not based on ill will.

• Have clear processes for determining follow-up steps — including withdrawing from a project or reevaluating the project’s aims — if participants identify clear and direct harms to the public resulting from successful development and deployment of the product or service.

More often than not, when processes are opened up to people outside the company, those who already have some close ties to the company are more likely to get involved, such as other technical professionals in the sector. Homogenous participant groups are also likelier if the team conducting the public engagement process is homogenous, because the professional and personal networks available when conducting outreach may be similar and even overlap.

• There is no such thing as a marginalized user or market because technology works for everyone.
• Representative or generalizable population samples are the gold standard for all forms of public engagement.
• Everyone will immediately feel welcome and inclined to participate and therefore does not require explicit recruitment.

• Design from the margins by identifying those who are most likely not to be invited into the conversation due to their social, political, and/or economic standing. Be especially attentive to recruiting those who are structurally excluded from participating, or are actively criminalized and targeted; they are the people we know the least about regarding the social impact of emerging technologies.
• Work with intermediaries, individuals, and organizations that have strong expertise and relationships with socially marginalized communities in order to strategize how to effectively engage different participants. They can also help identify people or groups you may not immediately recognize as important to consult or collaborate with.
• Review your recruitment process and the engagement approach to determine who will want to join your process and who might be discouraged, including such elements as:
  • outreach methods (e.g., who has access to those networks?)
  • physical accessibility (e.g., can someone who uses a wheelchair get in and out of the physical space used for the session; is the meeting venue require multiple modes of public transit to reach?)
  • digital resourcing needed (e.g., do participants have stable internet access, do their data plans allow for unlimited internet access, and are there language/communication barriers, etc.?)

“Public” is a broad term meant to capture the full array of customers, users, and people directly and indirectly impacted by AI-driven technologies. All of these people (and organizations) have their own sets of priorities, limitations, and expectations and must be treated accordingly. However, they can also be understood as existing in smaller groups, each organized with different social characteristics and identities shaped by shared experiences and treatment by others. Communities have a wide variety of ways of thinking about representation, authority, and sovereignty, and doing public engagement obligates us to pay attention to how this works in each context. Each group will have its own variety (and disagreements), but this does not necessarily diminish its cohesiveness as a community or invalidate members’ insights.

• A community is cohesive and will behave in a monolithic, consistent manner (including having one common, shared opinion).
• A single participant is fully representative of that entire group or has the authority to speak on behalf of that group simply because they identify as a member of that group or community.
• Your personal experience provides sufficient expertise regarding the complex histories and experiences of a whole community or group of people, so further research prior to engagement is unnecessary.

• Conduct research (e.g, desk research, consultations with internal and external experts) prior to developing the strategy to identify which specific communities or sets of people can help explore the topics you seek to explore.
• Follow the lead provided by communities (and their leadership) in terms of how they describe themselves and their constituents/members. Communities best understand their own community groups’ identities, histories, interests, needs, and priorities.
• Treat moments of disagreement and inconsistencies that may arise among your participants (especially those who see themselves as part of the same group or community) as opportunities to deepen your understanding and ask more questions.
• Work with trusted intermediaries (e.g., participatory engagement experts or community/domain leaders) to support recruitment, as well as to support analysis of data collected from public engagements in order to best situate the findings.

Public engagement practices need to be aligned with organizational structures of decision-making and power. Forms of public engagement, where the intention is to shift the paradigm by fully empowering socially marginalized participants (even at the expense of the technology developer or deployer) may not be an option. Such an approach requires more resources and full organizational commitment toward making the structural changes needed to cede decision-making authority to those participants. And depending on the set of participants you are working with, those individuals may not have the time or desire to commit to long-term engagement that requires attending weekly or monthly sessions with company organizers.

• Public engagement strategies can be developed without some baseline understanding of who is being engaged (members of the public, socially marginalized communities).
• Participants have the same ability, availability, and desire to contribute to a project, even if the topic and issue interest them.

• Evaluate what scope of changes participants are permitted to make and what level of decision-making they realistically have —as supported by your organization and its leadership — to ensure that the approaches you take are within the realm of possibility for participants to have impact.
• Plan to work with several groups, rather than just one, in order to leverage the many different communities and varied expertise that can help solve different issues/problems arising at various stages of AI development.
• Align the issue or question you are trying to answer with specific sets of people who can offer insights to help you resolve that issue or question, and conduct research in advance, such as reviewing existing literature and working with other experts.
• When working with community-based organizations (e.g., nonprofits, labor unions), or other, more formalized bodies of participants, invest time to learn about their work and communication habits and organizational culture to ensure alignment between your public engagement approach and their best methods of contributing.

Participants have many legitimate reasons to not directly engage with a private-sector business, as well as barriers to getting involved. The private sector has logged notable examples of using public engagements to obscure and legitimize harmful business practices, leading some members of the public to mistrust it. Specific communities or community-based organizations may have had direct experience with ill treatment by tech companies. Moreover, even participants enthusiastic about engaging on technology topics may experience limitations in how involved they can be. From jobs to family and community responsibilities, many struggle to meet such basic needs as housing, food, and health care and still have time to dedicate to other important issues. These external factors, as well as any circumstances that create coercive pressure, need to be addressed if you are to work thoughtfully with people.

• Participants can conform to existing expectations, requirements, and deadlines, even when these things are not communicated in advance, and can perform consistently throughout the engagement.
• All participants are able-bodied, speak English, are technologically confident, and have access to digital technology infrastructure (e.g., stable internet access).
• Participants will feel comfortable advocating for their own needs in order to join.

• Conduct your own (desk) research on:
  • prior interactions between your company (or similar companies) and the participant communities you wish to work with
  • structural discrimination and bias experienced by members of the communities you with to work with
  • histories of exclusion, criminalization, and other forms of harm experienced by members of the communities you wish to work with
• Properly scope what is possible and not possible to do or change as part of the public engagement process so that the nature and boundaries of the relationship between the organizer and the participants are explicitly defined.
• Identify sites for negotiation or opportunities to expand the typical scope to cede more decision-making to the participants.
• Accommodate deadlines and project timelines based on feedback you get from your participants, especially if this helps expand their capacity to participate.
• Adapt how you solicit feedback to align with how your participants feel most confident and capable of contributing (e.g., conducting the session in-person, rather than online, because they have limited internet access; and conducting tutorials on how to use online discussion platforms before virtual sessions).
• Allocate appropriate budgets to host sessions, with the materials, resources, and staffing needed (e.g., pay for time at a community center that is centrally located for the participants, have an American Sign Language interpreter on a video call for participants, develop materials in multiple languages).
• Explicitly create environments that welcome and support participants in sharing critical and negative feedback or concerns. Do this in consultation with participatory engagement experts and others who are deeply familiar with how coercion might arise when working with that specific set of participants.

Public participation is an important mechanism for learning about how new technologies, products, or services may help or harm different people. However, it cannot be relied upon as the only mechanism for ensuring the safe and responsible development and deployment of AI-driven tools. In fact, relying on specific groups outside the company to regularly provide these insights can result in “participation fatigue.” This refers to the drain on time, energy, and resources that members of the public experience when engaging on such matters outside their usual sphere of responsibilities as jobs and familial, community, or social responsibilities. This is especially true for those who struggle to receive or maintain the basic material resources they need to live, such as shelter, food, and health care. Fatigue may also arise from having to frequently retell stories about the same traumatic or otherwise-difficult experiences (e.g., having to reiterate one’s experiences with poverty or assault).

• Other people are responsible for telling you what harms might arise from the development or deployment of the AI system.
• The same small groups of people or organizations can be relied upon to uphold the rights and well-being of broader communities or the general public.

• Conduct literature reviews and work with external experts to comprehend already well-understood harms and risks for different communities and groups to avoid reinventing the wheel with participants by having them repeatedly share insights about the same set or sets of harms. Address those issues in advance.
• While building long-term relationships with community members and organizations, be sure to seek new potential participants as well in order to expand the network of individuals supporting your work.
• Implement social protections and benefits for participants before engaging with socially marginalized communities to ensure their safety and financial, social, and general well-being throughout the participation process.
• Minimize the effort for a participant to join. For example, provide as much administrative support as possible to support interaction with your company, such as the paperwork needed to be paid a stipend or getting onboarded onto platforms needed for participation.

While building trust between companies and the public is an important motivating factor for conducting public engagement, it is not necessarily a requisite for meaningful and insightful discussion. Trust is built over time, in part through consistent and transparent communication. The extent to which the individual participant or the overall body of users and the general public trust the product, team, or sponsoring company can fluctuate through the engagement process. The mistrust, so long as everyone remains engaged, can generate productive and civically healthy interactions and yield important discussions and insights.

• Participants’ trust is earned inherently through their involvement.
• Participants trust you or your company because you have good intentions.
• Even if communication with participants is limited or restricted, trust can be maintained.

• Work and learn with those who may be most critical of your work.
• Explicitly establish communication pathways and cadences with participants (e.g., will this be a one-time engagement?; what are the ways that participants can reach back out to the engagement organizers to give additional feedback?).
• Communicate any changes to the engagement process or the overall development of the product/service/feature quickly and with as much explanatory detail as possible.
• Reiterate what you learned and how you will use this learning (if at all) with participants throughout the engagement process, including what you will not be incorporating or addressing and how else their participation might be used by other teams in the company.

Participatory public engagement strategies are effective when people across different roles in AI-developing organizations directly engage in its implementation. However, doing so successfully requires additional training and skills to know how to effectively and thoughtfully navigate complex social and power dynamics between sponsoring organizations and participating individuals. From cultural competency and awareness of historic (and current) harms and inequality facing socially marginalized communities, to attending to individual and community needs for full participation and designing discussion spaces and activities to encourage people to draw on their varied expertise and experiences, additional training is needed to do public engagement well.

• A public engagement strategy can be developed or implemented without experience, expertise, or advanced research.
• Anyone who identifies themselves as an expert has direct experience working with a given community, is capable of speaking for it, or has aligned their own interests with the overall well-being of that community.
• Information is easily accessible to everyone (including others working in your organization), even if it is not intentionally organized and explicitly shared.

• Conduct the necessary desk research and expert consultations to have a working baseline knowledge of participatory engagement practices and the communities you hope you work with.
• Prioritize existing literature and research (scholarly and community-based reporting) conducted by members and advocates for the community themselves (as opposed to outsiders with no connection to those communities other than to report on them).
• Train yourself and your team on participatory research and engagement methods and frameworks prior to developing the public engagement strategy, including connecting with other staff members inside the organization who specialize in subjects like user experience research, as well as external experts.
• Identify your (and your team’s) biases and limitations, including the forms of expertise that are valued and the communities or groups that are most accessible for public engagement initiatives.
• Recognize that while all people can offer interesting and important insights and knowledge, there are different forms of expertise at play; apply the appropriate forms of expertise to the issues you intend to explore.
• Maintain good records of communication, including interactions with participants, data that is collected, and insights that are synthesized as a matter of organizational records and institutional memory. Make documentation of the process and project available to participants in a clear, organized fashion.