Custom Guidance for Developers and Deployers Experienced with Public Engagement
A necessary part of developing a participatory public engagement strategy is tailoring the approaches and tactics to different circumstances.
The first step is to determine the specific purpose for the public engagement: What issue do you face, and how can others help you resolve or navigate it effectively? Defining the specific purpose allows you to determine whom to connect with and how best to design engagement to build trust and interact meaningfully in order to elicit key insights.
While you and your team are best positioned to articulate the specific purpose, the tool below offers a resource to help you better determine the “how” aspect. Reflecting common challenges, insights, and recommended practices collected across the AI industry and other sectors and domains, this tool will help you build public engagement strategies that deepen trust while mitigating potential harm to the communities you work with.
1 Choose your AI model type
Specialized Narrow-Purpose Model
Description
Models designed for narrowly defined tasks or purposes with limited general capabilities for which there is a lower potential for harm across contexts.
The key difference from Advanced Narrow- and General-Purpose models is that these have tightly constrained capabilities in terms of input, domain, output complexity, and potential generalizability.
Key Considerations
Do any of the following apply to the model, even if it does not satisfy all criteria?
- Is the model designed for a narrow, well-defined domain or task?
- Is the model built and trained on data explicitly collected for a well-defined domain or task?
- Are its capabilities less applicable across contexts?
- Does the model present lower risk of misuse across contexts?
Examples
- Recommendation engines
- Image and speech recognition models
- Predictive maintenance and analytic models
- Fraud detection
Advanced Narrow- and General-Purpose Model
Description
Encompasses general-purpose models capable across diverse contexts, such as chatbots/LLMs and multimodal models.
Key Considerations
Do any of the following apply to the model, even if it does not satisfy all criteria?
- Can the model generate synthetic content difficult for people to distinguish from reality (text, audio, video), even if narrow-purpose?
- Could the model facilitate impersonation, disinformation or other societal, chemical, biological, or cyber attacks if misused, even if focused on a specific task?
- Or is the model more generally applicable across contexts, rather than narrowly focused?
- Does the model involve multiple modalities such as text, image, audio, and video?
Examples
- Text to speech
- Voice impersonation
- Text to image/video
- Code generation (e.g., GitHub Copilot)
- Scientific models
- Multimodal models (e.g., GPT-4)
- Chatbots (e.g., Claude, ChatGPT, Bard, Llama)
Customized Pretrained Model
Description
Pretrained (advanced-purpose or general-purpose) models fine-tuned for narrower defined tasks or purposes by a third-party organization or by a different team (if being adapted within the organization that developed the base model).
The key difference from Narrow-Purpose models is that these are developed on existing, pretrained advanced narrow- and general-purpose models, rather than being fully developed from scratch.
Key Considerations
Do any of the following apply to the model, even if it does not satisfy all criteria?
- Is the model based on an existing advanced-purpose or general-purpose model and then fine-tuned for a narrow, well-defined domain or task?
- Is the model fine-tuned without the direct involvement of the team or organization that developed the base advanced-purpose or general-purpose model?
- Does the organization collect its own proprietary data (i.e., from its employees, clients, or other users) to support the fine-tuning of models?
Examples
- Business optimization tools
- Document-review tools for specific domains
- Customer service chatbots
My model type is:
2 Choose your engagement objective
Choose the intended purpose for working with communities and people outside the product team. In cases where public engagement is sought for multiple purposes, select the most timely stage and revisit this guidance as development progresses.
Market Study
Purpose
To better understand an existing or potential market of users, customers, or people otherwise impacted by a product or service. A concept may not yet be in development, and the aim may be to develop a foundational understanding of what a set of potential users may or may not want in an AI-driven product or service.
Data Collection & Enrichment
Purpose
To create or refine datasets for training or fine-tuning purposes. Participants may be engaged both to serve as data providers (i.e., people knowingly volunteering their personal and user data to help diversify datasets) and to improve data annotation or enrichment (i.e., people from many different contexts employed to provide annotation services).
Risk & Failure Assessments
Purpose
To adversarially engage with a prototype or product/feature in development in order to identify likely risks and potential failures. Individuals unfamiliar with the specific project — including people from other teams in the same organization; academics; or other subject-matter experts, users, advocates, and other general members of the public — may be invited.
User Experience & Testing
Purpose
To work with existing or likely users to better understand how they engage with a product or service, what they like and dislike about it, and what adjustments can be made to make it easier to use or more likely to be used. User experience research and additional testing can be applied before and after a product is released.
Oversight
Purpose
To engage individuals outside the product team and/or organization to provide ongoing oversight of a deployed AI product or service. This may be done to help the organization adjudicate user-reported harms, provide a mechanism for ongoing accountability, or monitor changes.
My engagement objective is:
3 See your guidance
Note: this tool is meant to provide an initial set of recommendations on challenges that commonly arise in developing public engagement strategies or activities. It should be treated as a brainstorming tool and not as a definitive guide.
Market Study for Specialized Narrow-Purpose Models
This custom Guidance for Inclusive AI, for use by commercial sector AI developers and deployers, focuses on an initial set of recommendations for creating public engagement strategies. Additional guidance and cautions are available here. This should be treated as a brainstorming tool and not as a definitive checklist guide.
In general, market studies can be challenging because the scope of the engagement might be loosely defined, and it may be unclear how participant insights will be used (or not used) for product development. It is easier to specify participants or the expertise needed to engage with models that have narrowly defined tasks or purposes.
Understand Incentives & Risks
Learn more about understanding the incentives & risks for engagement
Common Challenges
- Incentives for participatory public engagement are unclear.
- The sponsoring organization has a poor reputation among the participant community.
Recommended Practices
Identify, address, and communicate biases and exclusions
- Host internal meeting(s) to discuss specific team biases.
- Conduct pre-engagement research to better understand audiences that are frequently not engaged.
Align organizationally on purpose and capacity
- Host meeting(s) with organizational and team leaders and key staff to establish a shared understanding of the value and importance of participatory public engagement.
- Specify concrete key-performance indicators or other organizational metrics, or recognize and award employees implementing public engagement strategies.
Address past harms
- Research and directly acknowledge any past harms committed by your organization with participants.
- Research and directly acknowledge any past harms related to the technology being discussed with participants.
Conduct internal training
- Host training sessions for the engagement team (and the broader organization) on such topics as:
- purpose and value of participatory public engagement
- considerations and sensitivities when working with a specific socially marginalized community
- specific skills development for engagement facilitation and moderation
- data analysis skills, such as social scientific methods for qualitative data analysis
- Consult and hire external experts and intermediaries to lead training sessions specific to the needs of your team and the participants you will engage.
Identify Key Actors
Learn more about identifying key actors
Common Challenges
- It is unclear who to target as participants.
- It is difficult to recruit participants in new or unfamiliar markets.
- Participants consider the topic outside their expertise or usual scope of work.
Recommended Practices
Conduct research on communities and known issues prior to engagement
- Consult intermediaries and other experts to develop baseline knowledge of the social impact issues affecting socially marginalized communities.
- Map known experts’ and intermediaries’ relevant expertise, and collaborate with them to identify gaps in your team’s knowledge or networks.
- Prioritize existing literature and research led by members of the communities, including work and people that are critical of your organization or the technology.
Design from the margins
- Explicitly prioritize people who might be outlier users facing greater likelihood of direct harm.
- Include recruitment of people who are likely to be impacted by the deployment of the product/service but are not direct users.
- Include participants who are explicitly critical of your organization or of AI technology.
Expand participant recruitment networks
- Work with employee resource groups (ERGs) or other internal audiences to identify additional external networks for participant recruitment.
- Reach out to customers or users who do not typically engage with the company to learn ways they could be encouraged to participate.
- Dedicate time to connect with and build relationships with many different civil society or community groups, rather than rely on a couple of strong existing relationships.
Maintain transparent recruitment processes
- Publish participant selection criteria and allow people to provide input on any exclusions that might harm the project.
- Host recruitment sessions wherein interested participants can pose questions to the organizers.
Collaborate with intermediaries
- Consult with intermediaries to identify and address team and organizational biases and limitations.
- Solicit external input on designing the participatory public engagement strategy and activities.
- Work with intermediaries or community leaders who have direct experience building relationships with the specific socially marginalized community(ies) you seek to work with.
Determine Organizational Capacity
Learn more about determining organizational capacity
Common Challenges
- Organizational leadership is unfamiliar with participatory public engagement.
- It is unclear how to incorporate participant feedback.
Recommended Practices
Dedicate time for research and development
- Conduct existing literature and research review to establish baseline understanding of social impacts and harms of the technology.
- Conduct existing literature and research review to establish baseline understanding of historic and current issues affecting socially marginalized communities.
- Research past public engagements hosted by your organization, including the impact those engagements had on different participant groups.
Establish internal processes and criteria for decision-making
- Internally discuss and set parameters on what participants can and cannot influence or change.
- Confirm that the engagement team has clear direction from organizational leaders on what can and cannot be shared with participants (or has a process for getting this guidance).
Consult with internal experts or audiences and intermediaries
- Seek out and work with any public engagement or community experts on staff, even those outside the specific product team.
- Seek out and work with other internal audiences to revise, validate, and/or pilot engagement strategies.
- Work with intermediaries or community leaders to translate insights into actionable changes for the product.
Budget for the full participatory public engagement strategy
- Allocate funds to support staff time for all stages of the engagement, including pre-engagement research and post-engagement follow-up, as well as time spent on administrative and logistics tasks.
- Allocate funds for expenses related to participant support, including:
- participant compensation
- participant access costs like travel vouchers, internet data credit, etc.
- support activities such as translation or live captioning services, on-site childcare, etc.
- support services such as mental health counselors, legal support
Define the Relationship
Learn more about defining the relationship
Common Challenges
- Participants are concerned they might not see their feedback reflected in the process or final product.
- Participants have concerns about potential exploitation or “participant washing.”
Recommended Practices
Establish participant communication expectations and processes
- Share details about how you plan to communicate with participants during and after the engagement.
- Establish communication mechanisms for participants to reach out to you after the engagement.
- Maintain multiple points of contact for participants to reach out to you during and after the engagement.
Address specific participant needs
- Dedicate time to learn about and develop strategies to address differences in cultural and communication style between the engagement team and the participants.
- Ask for input on what needs to change to invite participants to be more engaged and share honest feedback.
- Adapt data collection or input solicitation methods to make them culturally appropriate and accessible to participants, such as storytelling methods, oral histories, and use of drawings or physical models.
Create opportunities for “out of scope” discussions
- Encourage participants to provide feedback beyond the scoped topic to identify any major failures or risks they project.
- Incorporate time for participants to ask more general questions about the AI development process or the impact of the technology.
Practice active listening
- Repeat what you’ve learned from participants and outline how their insights will be used.
- Share any publications with participants prior to release and confirm their consent to having their insights or participation referenced or used.
Address participant needs in high-stakes situations
- Dedicate time to learn about and develop strategies to address specific high-stakes situations participants might face, such as criminalization or imprisonment.
- Work with intermediaries to initially understand specific ways some groups might experience greater scrutiny or harm.
- Align internal resources, such as mental health professionals or access to legal support, for participants.
Assess the Overall Process
Learn more about assessing the overall process
Common Challenges
- Different sets of participants provide opposing feedback.
- Participant feedback feels highly specific to certain individuals or communities.
Recommended Practices
Limit generalizations of learning
- Recognize the limits of participatory public engagement, and avoid generalizing the experiences from individuals to broad populations without validating them against other studies or engagements.
- Continuously refine and specify distinctions within socially marginalized groups to better understand how members of a given social identity group may have different experiences or needs.
Optimize for comprehensive feedback from participants
- Balance inquiries with limited-choice options or highly specified questions, such as surveys, with more open-ended discussions.
Conduct a participant impact and risk assessment
- Work with internal and external experts and intermediaries to understand potential harms and risks to participants that can arise from working with your organization.
- Address potential confirmation bias in your engagement (e.g., leading questions, prompts that force agreement by the participant) by designing engagement activities, including the questions asked, to generate more open-ended discussions with participants.
Is this guidance missing something important? Suggest an update
Explore additional resources
Summary Guidance
Emerging Technology
Project Background
Resource LibraryPAI’s Guidance for Inclusive AI
Data Collection & Enrichment for Specialized Narrow-Purpose Models
This custom Guidance for Inclusive AI, for use by commercial sector AI developers and deployers, focuses on an initial set of recommendations for creating public engagement strategies. Additional guidance and cautions are available here. This should be treated as a brainstorming tool and not as a definitive checklist guide.
In general, data collection and enrichment can be challenging because good datasets are a key input for AI models, and possession of these datasets is a financial asset for companies. These financial gains are not often shared with the people who contribute, or annotate, the data unless structured into properly paid jobs. It is easier to specify participants or the expertise needed to engage with models that have narrowly defined tasks or purposes.
Understand Incentives & Risks
Learn more about understanding the incentives & risks for engagement
Common Challenges
- Organizational leadership is unfamiliar with participatory public engagement.
- Participant incentives for participatory public engagement are unclear.
- Engagement may slow down product launches.
Recommended Practices
Conduct internal training
- Host training sessions for the engagement team (and the broader organization) on such topics as:
- purpose and value of participatory public engagement
- existing gaps in available datasets that might result in poorer-quality AI products
Align organizationally on purpose and capacity
- Host meeting(s) with organizational and team leaders and key staff to establish a shared understanding of the value and importance of participatory public engagement.
- Specify concrete key-performance indicators or other organizational metrics, or recognize and award employees implementing public engagement strategies.
Consult intermediaries and participants to validate the engagement design
- Work with intermediaries and participants to identify any additional needs participants may have that could impact their ability to participate.
Establish internal processes and criteria for decision-making
- Coordinate with product development engineers and others in the organization to ensure participant input supports better development.
- Determine the gaps in existing datasets to optimize data collection from participants.
Identify Key Actors
Learn more about identifying key actors
Common Challenges
- Expanding the participant recruitment network is difficult.
Recommended Practices
Conduct research on communities and known issues prior to engagement
- Conduct social, scientific, and sociotechnical research to understand current and historical issues impacting socially marginalized communities.
- Map known experts’ and intermediaries’ relevant expertise, and collaborate with them to identify gaps in your team’s knowledge or networks.
- Prioritize existing literature and research led by members of the communities, including work and people that are critical of your organization or the technology.
Design from the margins
- Explicitly prioritize people who might be outlier users facing greater likelihood of direct harm.
- Include recruitment of people who are likely to be impacted by the deployment of the product/service but are not direct users.
- Include participants who are explicitly critical of your organization or of AI technology.
Expand participant recruitment networks
- Work with employee resource groups (ERGs) or other internal audiences to identify additional external networks for participant recruitment.
- Reach out to customers or users who do not typically engage with the company to learn ways they could be encouraged to participate.
- Dedicate time to connect with and build relationships with many different civil society or community groups, rather than rely on a couple of strong existing relationships.
Determine Organizational Capacity
Learn more about determining organizational capacity
Common Challenges
- Participants lack resources necessary to join engagement activity.
Recommended Practices
Prioritize participant well-being
- Negotiate internal policies and processes such as nondisclosure agreement requirements to better meet participant needs.
- Identify areas in which the organization can share decision-making or benefits to participants, such as establishing data dividends for contributing to the dataset.
- Reduce any administrative barriers for participants by simplifying onboarding or compensation processes.
- Request feedback from participants on the design of the engagement activity to align with participants’ working and communication styles.
- Provide post-engagement support to address any emotional, physical, or psychological harms experienced by participants due to the engagement.
Determine specific participant needs
- Dedicate time to learn about and develop strategies to address differences in cultural and communication style between the engagement team and the participants.
- Discuss timelines and deadlines with participants.
- Ask for input on what needs to change to invite participants to be more engaged and share honest feedback.
Determine participant needs in high-stakes situations
- Dedicate time to learn about and develop strategies to address specific high-stakes situations participants might face, such as criminalization or imprisonment.
- Work with intermediaries to initially understand specific ways some groups might experience greater scrutiny or harm.
- Align internal resources, such as mental health professionals or access to legal support, for participants.
- Increase participants’ data privacy and security.
Budget for the full participatory public engagement strategy
- Allocate funds to support staff time for all stages of the engagement, including pre-engagement research and post-engagement follow-up, as well as time spent on administrative and logistics tasks.
Define the Relationship
Learn more about defining the relationship
Common Challenges
- Participants assume they lack familiarity with the topic to engage.
- The sponsoring organization has a poor reputation among the participant community.
Recommended Practices
Establish a participant onboarding process
- Communicate the specific role, opportunities for impact, and limitations on what participants can influence before starting the engagement process.
- Clearly outline expectations of time required for participation.
- Allow participants to “off-board” easily if they disagree with the proposed engagement process.
- Host pre-engagement sessions to help participants get online or become familiar with the engagement platform.
- Establish clear work processes and expectations for participants contributing or annotating data
Address past harms
- Research and directly acknowledge any past harms committed by your organization with participants.
- Research and directly acknowledge any past harms related to the technology being discussed with participants.
Assess the Overall Process
Learn more about assessing the overall process
Common Challenges
- The overall impact of collecting or processing data on participants is unclear.
- Participants have concerns about potential exploitation or “participant washing.”
Recommended Practices
Expand the impact of participants
- Identify topics or issues that can be negotiated to permit greater influence by participants.
- Allow product teams to revisit prior development stages to address issues identified by participants.
- Encourage participants to identify issues beyond the scoped topic for the engagement.
Compensate and credit participants
- Prior to, and throughout the engagement, host discussions explicitly about forms of compensation (financial, public acknowledgement, in-kind donations, etc.) that participants want.
- Provide financial compensation for participation.
- Provide financial support for additional costs related to participating, including travel vouchers, on-site childcare, meals.
Conduct a participant impact and risk assessment
- Work with internal and external experts and intermediaries to understand potential harms and risks to participants that can arise from working with your organization.
- Assess the engagement strategy and specific activities to address any coercive pressure participants may experience to discourage criticism of your organization or the product.
Limit how participation and participant feedback is used
- Restrict how other teams in the organization can use the data collected from participants, unless participants are fully aware of and consent to the use of their insights or data for these additional uses.
- Provide participants with highly specified release forms (dictating the use of their participation details) rather than general release forms that permit the organization to freely use any information collected during the engagement.
Is this guidance missing something important? Suggest an update
Explore additional resources
PAI’s Guidance for Inclusive AI
Risk & Failure Assessments for Specialized Narrow-Purpose Models
This custom Guidance for Inclusive AI, for use by commercial sector AI developers and deployers, focuses on an initial set of recommendations for creating public engagement strategies. Additional guidance and cautions are available here. This should be treated as a brainstorming tool and not as a definitive checklist guide.
In general, risk and failure assessment can be challenging because participants need access to early versions of the AI models to test them, leaving sponsors vulnerable. These financial gains are not often shared with the people who contribute, or annotate, the data unless structured into properly paid jobs. Assessing for risks and failure might also expose participants to harmful content or output. It is easier to specify participants or the expertise needed to engage with models that have narrowly defined tasks or purposes.
Understand Incentives & Risks
Learn more about understanding the incentives & risks for engagement
Common Challenges
- Organizational concerns about public criticism or negative PR.
- Engagement may cause friction between teams within an organization.
- Participants are worried about their credibility if they work with tech companies.
Recommended Practices
Conduct research on communities and known issues prior to engagement
- Conduct social, scientific, and sociotechnical research to understand current and historical issues impacting socially marginalized communities.
- Prioritize existing literature and research led by members of the communities, including work and people that are critical of your organization or the technology.
- Research past public engagements hosted by your organization, including the impact those engagements had on different participant groups.
- Consult intermediaries and other experts to develop baseline knowledge of the social impact issues affecting socially marginalized communities.
Align organizationally on purpose and capacity
- Host meeting(s) with organizational and team leaders and key staff to establish a shared understanding of the value and importance of participatory public engagement.
- Specify concrete key-performance indicators or other organizational metrics, or recognize and award employees implementing public engagement strategies.
Establish internal processes and criteria for decision-making
- Internally discuss and set parameters on what participants can and cannot influence or change.
- Confirm that the engagement team has clear direction from organizational leaders on what can and cannot be shared with participants (or has a process for getting this guidance).
Address past harms
- Research and directly acknowledge any past harms committed by your organization with participants.
- Research and directly acknowledge any past harms related to the technology being discussed with participants.
Identify Key Actors
Learn more about identifying key actors
Common Challenges
- The same set of external experts are recruited.
- Participants consider the topic outside their expertise or usual scope of work.
Recommended Practices
Design from the margins
- Explicitly prioritize people who might be outlier users facing greater likelihood of direct harm.
- Include recruitment of people who are likely to be impacted by the deployment of the product/service but are not direct users.
- Include participants who are explicitly critical of your organization or of AI technology.
- Engage with, and explore, conflicting data points or outliers to determine whether they are true outliers or indicators of key edge cases.
Specify the types of expertise needed
- Align the different forms of formal and informal expertise that can help address your team’s participatory public engagement objectives and/or your team’s specific gaps in awareness.
- Conduct additional research on available formal and informal experts for a topic, rather than rely on such common proxies for expertise as elite university educational affiliations.
Expand participant recruitment networks
- Work with employee resource groups (ERGs) or other internal audiences to identify additional external networks for participant recruitment.
- Reach out to customers or users who do not typically engage with the company to learn ways they could be encouraged to participate.
- Dedicate time to connect with and build relationships with many different civil society or community groups, rather than rely on a couple of strong existing relationships.
- Encourage past participants or others in your network to recommend new participants.
Collaborate with intermediaries
- Work with intermediaries or community leaders who have direct experience building relationships with the specific socially marginalized community(ies) you seek to work with.
- Map known experts’ and intermediaries’ relevant expertise, and collaborate with them to identify gaps in your team’s knowledge or networks.
Maintain transparent recruitment processes
- Publish participant selection criteria and allow people to provide input on any exclusions that might harm the project.
- Include information about what participants will do and how participants’ feedback will be used in the recruitment materials.
- Host recruitment sessions wherein interested participants can pose questions to the organizers.
Determine Organizational Capacity
Learn more about determining organizational capacity
Common Challenges
- Participants lack resources necessary to join engagement activity.
- It is unclear how to incorporate participant feedback, particularly non-technical feedback.
Recommended Practices
Align resources to address specific participant needs
- Dedicate time to learn about and develop strategies to address differences in cultural and communication style between the engagement team and the participants.
- Discuss timelines and deadlines with participants.
- Provide participants with multiple methods for providing feedback, such as written comments or voice recordings.
- Prior to the engagement, survey participants to identify additional needs they may have that could impact their ability to participate.
- Adapt data collection or input solicitation methods to make them culturally appropriate and accessible to participants, such as storytelling methods, oral histories, and use of drawings or physical models.
Assess for Accessibility of Engagement Venue or Platform
- Work with experts, intermediaries, and participants to assess the engagement venue (in-person sessions) or platform (virtual sessions) for accessibility, including:
- physical accessibility
- language/content accessibility
- financial accessibility
- technological accessibility
- Provide easy mechanisms for participants to confidentially share their accessibility needs.
- Prioritize scheduling and time zones of participants to better ensure their full engagement and attentiveness during the engagement activity.
Conduct internal training
- Host training sessions for the engagement team (and the broader organization) on such topics as:
- purpose and value of participatory public engagement
- considerations and sensitivities when working with a specific socially marginalized community
- specific skills development for engagement facilitation and moderation
- data analysis skills, such as social scientific methods for qualitative data analysis
Consult with internal experts or audiences and intermediaries
- Seek out and work with any public engagement or community experts on staff, even those outside the specific product team.
- Seek out and work with other internal audiences to revise, validate, and/or pilot engagement strategies.
- Work with intermediaries or community leaders to translate insights into actionable changes for the product.
Consult intermediaries and participants to validate the engagement design
- Work with intermediaries and participants to identify any additional needs participants may have that could impact their ability to participate.
- Brainstorm and define risk and harm categories with experts, intermediaries, and participants to revise engagement design.
Define the Relationship
Learn more about defining the relationship
Common Challenges
- Participants are worried about providing critical or negative feedback.
- Participants have concerns about potential exploitation or “participant washing.”
Recommended Practices
Create opportunities for “out of scope” discussions
- Encourage participants to provide feedback beyond the scoped topic to identify any major failures or risks they project.
- Incorporate time for participants to ask more general questions about the AI development process or the impact of the technology.
Conduct a participant impact and risk assessment
- Work with internal and external experts and intermediaries to understand potential harms and risks to participants that can arise from working with your organization.
- Assess the engagement strategy and specific activities to address any coercive pressure participants may experience to discourage criticism of your organization or the product.
Prioritize participant well-being
- Negotiate internal policies and processes such as nondisclosure agreement requirements to better meet participant needs.
- Work with intermediaries if participants need greater protection or separation from the organization sponsoring the engagement.
- Provide post-engagement support to address any emotional, physical, or psychological harms experienced by participants due to the engagement.
- Provide explicit mechanisms for participants to opt-out of the engagement at any time.
Address participant needs in high-stakes situations
- Dedicate time to learn about and develop strategies to address specific high-stakes situations participants might face, such as criminalization or imprisonment.
- Work with intermediaries to initially understand specific ways some groups might experience greater scrutiny or harm.
- Adopt whistleblower or other legal protections for participants to protect their ability to provide honest and critical feedback.
- Provide advocates or other support personnel for participants to contact with questions or concerns about their participation, particularly if they experience harm.
- Align internal resources, such as mental health professionals or access to legal support, for participants.
- Increase participants’ data privacy and security.
Compensate and credit participants
- Prior to, and throughout the engagement, host discussions explicitly about forms of compensation (financial, public acknowledgement, in-kind donations, etc.) that participants want.
- Provide financial compensation for participation.
- Provide financial support for additional costs related to participating, including travel vouchers, on-site childcare, meals.
Welcome opt outs
- Create clear pathways for participants to withdraw or opt out during and after engagement.
Assess the Overall Process
Learn more about assessing the overall process
Common Challenges
- Different sets of participants provide opposing feedback.
- It is unclear how to incorporate participant feedback.
Recommended Practices
Expand the impact of participants
- Identify topics or issues that can be negotiated to permit greater influence by participants.
- Allow product teams to revisit prior development stages to address issues identified by participants.
- Encourage participants to identify issues beyond the scoped topic for the engagement.
Conduct internal training
- Host training sessions for the engagement team (and the broader organization) on such topics as:
- purpose and value of participatory public engagement
- considerations and sensitivities when working with a specific socially marginalized community
- specific skills development for engagement facilitation and moderation
- data analysis skills, such as social scientific methods for qualitative data analysis
Collaborate with intermediaries
- Work with intermediaries or community leaders who have direct experience building relationships with the specific socially marginalized community(ies) you seek to work with.
- Work with intermediaries or community leaders to translate insights into actionable changes for the product.
Is this guidance missing something important? Suggest an update
Explore additional resources
PAI’s Guidance for Inclusive AI
User Experience & Testing for Specialized Narrow-Purpose Models
This custom Guidance for Inclusive AI, for use by commercial sector AI developers and deployers, focuses on an initial set of recommendations for creating public engagement strategies. Additional guidance and cautions are available here. This should be treated as a brainstorming tool and not as a definitive checklist guide.
In general, user experience and testing is a well-established stage of model development, increasing the likelihood that organizational leaders will readily support this public engagement activity. It is easier to specify participants or the expertise needed to engage with models that have narrowly defined tasks or purposes.
Understand Incentives & Risks
Learn more about understanding the incentives & risks for engagement
Common Challenges
- The sponsoring organization has a poor reputation among the participant community.
- Engagement may slow down product launches.
- There are concerns that participants will leak IP or publicly criticize the project.
Recommended Practices
Dedicate time for research and development
- Conduct existing literature and research review to establish baseline understanding of social impacts and harms of the technology.
- Conduct existing literature and research review to establish baseline understanding of historic and current issues affecting socially marginalized communities.
- Prioritize existing literature and research led by members of the communities, including work and people that are critical of your organization or the technology.
- Research past public engagements hosted by your organization, including the impact those engagements had on different participant groups.
- Consult intermediaries and other experts to develop baseline knowledge of the social impact issues affecting socially marginalized communities.
Consult with internal experts or audiences and intermediaries
- Inventory internal expertise and skills available across organizational teams.
- Seek out and work with any public engagement or community experts on staff, even those outside the specific product team.
Align organizationally on purpose and capacity
- Host meeting(s) with organizational and team leaders and key staff to establish a shared understanding of the value and importance of participatory public engagement.
- Specify concrete key-performance indicators or other organizational metrics, or recognize and award employees implementing public engagement strategies.
Establish internal processes and criteria for decision-making
- Internally discuss and set parameters on what participants can and cannot influence or change.
- Confirm that the engagement team has clear direction from organizational leaders on what can and cannot be shared with participants (or has a process for getting this guidance).
- Communicate organizational processes and limitations to participants as part of the participant onboarding process.
Identify Key Actors
Learn more about identifying key actors
Common Challenges
- It is difficult to recruit participants in new or unfamiliar markets.
- The same set of participants are recruited.
Recommended Practices
Collaborate with intermediaries
- Work with intermediaries or community leaders who have direct experience building relationships with the specific socially marginalized community(ies) you seek to work with.
- Map known experts’ and intermediaries’ relevant expertise, and collaborate with them to identify gaps in your team’s knowledge or networks.
Design from the margins
- Explicitly prioritize people who might be outlier users facing greater likelihood of direct harm.
- Include recruitment of people who are likely to be impacted by the deployment of the product/service but are not direct users.
- Include participants who are explicitly critical of your organization or of AI technology.
- Engage with, and explore, conflicting data points or outliers to determine whether they are true outliers or indicators of key edge cases.
Expand participant recruitment networks
- Work with employee resource groups (ERGs) or other internal audiences to identify additional external networks for participant recruitment.
- Reach out to customers or users who do not typically engage with the company to learn ways they could be encouraged to participate.
- Dedicate time to connect with and build relationships with many different civil society or community groups, rather than rely on a couple of strong existing relationships.
- Encourage past participants or others in your network to recommend new participants.
Determine Organizational Capacity
Learn more about determining organizational capacity
Common Challenges
- Engagement may cause friction between teams within an organization.
Recommended Practices
Conduct internal training
- Host training sessions for the engagement team (and the broader organization) on such topics as:
- purpose and value of participatory public engagement
- considerations and sensitivities when working with a specific socially marginalized community
- specific skills development for engagement facilitation and moderation
- data analysis skills, such as social scientific methods for qualitative data analysis
- Host retrospectives to discuss and learn from previous public engagement activities undertaken by your organization.
Coordinate application of participant insights to product development
- Coordinate with product development engineers and others in the organization to ensure participant input supports better development.
Budget for the full participatory public engagement strategy
- Allocate funds to support staff time for all stages of the engagement, including pre-engagement research and post-engagement follow-up, as well as time spent on administrative and logistics tasks.
- Allocate funds for expenses related to participant support, including:
- participant compensation
- participant access costs like travel vouchers, internet data credit, etc.
- support activities such as translation or live captioning services, on-site childcare, etc.
- support services such as mental health counselors, legal support
Define the Relationship
Learn more about defining the relationship
Common Challenges
- Participants have concerns about potential exploitation or “participant washing.”
- Participants consider the topic outside their expertise or usual scope of work.
- Participants are worried about providing critical or negative feedback.
Recommended Practices
Expand the impact of participants
- Identify topics or issues that can be negotiated to permit greater influence by participants.
- Allow product teams to revisit prior development stages to address issues identified by participants.
- Encourage participants to identify issues beyond the scoped topic for the engagement.
Compensate and credit participants
- Prior to, and throughout the engagement, host discussions explicitly about forms of compensation (financial, public acknowledgement, in-kind donations, etc.) that participants want.
- Provide financial compensation for participation.
- Provide financial support for additional costs related to participating, including travel vouchers, on-site childcare, meals.
Prioritize participant well-being, including in high-stakes situations
- Negotiate internal policies and processes such as nondisclosure agreement requirements to better meet participant needs.
- Identify areas in which the organization can share decision-making or benefits to participants, such as establishing data dividends for contributing to the dataset.
- Request feedback from participants on the design of the engagement activity to align with participants’ working and communication styles.
- Work with intermediaries to initially understand specific ways some groups might experience greater scrutiny or harm.
- Adopt whistleblower or other legal protections for participants to protect their ability to provide honest and critical feedback.
Conduct a participant impact and risk assessment
- Work with internal and external experts and intermediaries to understand potential harms and risks to participants that can arise from working with your organization.
- Assess the engagement strategy and specific activities to address any coercive pressure participants may experience to discourage criticism of your organization or the product.
Limit how participation and participant feedback is used
- Restrict how other teams in the organization can use the data collected from participants, unless participants are fully aware of and consent to the use of their insights or data for these additional uses.
- Provide participants with highly specified release forms (dictating the use of their participation details) rather than general release forms that permit the organization to freely use any information collected during the engagement.
- Obtain explicit permission from participants to use their involvement to promote the public image of the organization.
Assess the Overall Process
Learn more about assessing the overall process
Common Challenges
- Participants are concerned they might not see their feedback reflected in the process or final product.
- Participant feedback feels highly specific to certain individuals or communities.
Recommended Practices
Optimize for comprehensive feedback from participants
- Balance inquiries with limited-choice options or highly specified questions, such as surveys, with more open-ended discussions.
Establish participant communication expectations and processes
- Share details about how you plan to communicate with participants during and after the engagement.
- Establish communication mechanisms for participants to reach out to you after the engagement.
- Maintain multiple points of contact for participants to reach out to you during and after the engagement.
Create opportunities for “out of scope” discussions
- Encourage participants to provide feedback beyond the scoped topic to identify any major failures or risks they project.
- Incorporate time for participants to ask more general questions about the AI development process or the impact of the technology.
Practice active listening
- Repeat what you’ve learned from participants and outline how their insights will be used.
- Share any publications with participants prior to release and confirm their consent to having their insights or participation referenced or used.
Limit generalizations of learning
- Recognize the limits of participatory public engagement, and avoid generalizing the experiences from individuals to broad populations without validating them against other studies or engagements.
- Continuously refine and specify distinctions within socially marginalized groups to better understand how members of a given social identity group may have different experiences or needs.
Is this guidance missing something important? Suggest an update
Explore additional resources
PAI’s Guidance for Inclusive AI
Oversight for Specialized Narrow-Purpose Models
This custom Guidance for Inclusive AI, for use by commercial sector AI developers and deployers, focuses on an initial set of recommendations for creating public engagement strategies. Additional guidance and cautions are available here. This should be treated as a brainstorming tool and not as a definitive checklist guide.
In general, oversight is challenging because the scope of work for participants might be more open-ended and require longer-term engagements working with the sponsors across the full development life cycle. It is easier to specify participants or the expertise needed to engage with models that have narrowly defined tasks or purposes.
Understand Incentives & Risks
Learn more about understanding the incentives & risks for engagement
Common Challenges
- Participatory public engagement is not recognized as an organizational priority.
- Organizational concerns about public criticism or negative PR.
Recommended Practices
Align organizationally on purpose and capacity
- Host meeting(s) with organizational and team leaders and key staff to establish a shared understanding of the value and importance of participatory public engagement.
- Specify concrete key-performance indicators or other organizational metrics, or recognize and award employees implementing public engagement strategies.
Conduct internal training
- Host training sessions for the engagement team (and the broader organization) on such topics as:
- purpose and value of participatory public engagement
- considerations and sensitivities when working with a specific socially marginalized community
- specific skills development for engagement facilitation and moderation
- data analysis skills, such as social scientific methods for qualitative data analysis
Dedicate time for research and development prior to recruitment and engagement
- Conduct existing literature and research review to establish baseline understanding of social impacts and harms of the technology.
- Conduct existing literature and research review to establish baseline understanding of historic and current issues affecting socially marginalized communities.
- Research past public engagements hosted by your organization, including the impact those engagements had on different participant groups.
Collaborate with intermediaries
- Consult with intermediaries to identify and address team and organizational biases and limitations, particularly where the organization might face public scrutiny.
- Work with intermediaries or community leaders who have direct experience building relationships with the specific socially marginalized community(ies) you seek to work with.
Address past harms
- Research and directly acknowledge any past harms committed by your organization with participants.
- Research and directly acknowledge any past harms related to the technology being discussed with participants.
Identify Key Actors
Learn more about identifying key actors
Common Challenges
- The same set of external experts are recruited.
- It is difficult to determine participants’ (including experts’ and intermediaries’) credibility.
- Participants might have limited time to commit to an extended engagement.
Recommended Practices
Design from the margins
- Explicitly prioritize people who might be outlier users facing greater likelihood of direct harm.
- Include recruitment of people who are likely to be impacted by the deployment of the product/service but are not direct users.
- Include participants who are explicitly critical of your organization or of AI technology.
- Engage with, and explore, conflicting data points or outliers to determine whether they are true outliers or indicators of key edge cases.
- Map known experts’ and intermediaries’ relevant expertise, and collaborate with them to identify gaps in your team’s knowledge or networks.
Expand participant recruitment networks
- Work with employee resource groups (ERGs) or other internal audiences to identify additional external networks for participant recruitment.
- Reach out to customers or users who do not typically engage with the company to learn ways they could be encouraged to participate.
- Dedicate time to connect with and build relationships with many different civil society or community groups to validate expertise and credibility.
- Encourage past participants or others in your network to recommend new participants.
Specify the types of expertise needed
- Align the different forms of formal and informal expertise that can help address your team’s participatory public engagement objectives and/or your team’s specific gaps in awareness.
- Conduct additional research on available formal and informal experts for a topic, rather than rely on such common proxies for expertise as elite university educational affiliations.
- Prior to the engagement, survey participants to identify any additional needs they may have that can impact their ability to participate.
Collaborate with intermediaries
- Consult with intermediaries to identify and address team and organizational biases and limitations.
- Work with intermediaries or community leaders who have direct experience building relationships with the specific socially marginalized community(ies) you seek to work with.
Establish a participant onboarding process
- Communicate the specific role, opportunities for impact, and limitations on what participants can influence before starting the engagement process.
- Clearly outline expectations of time required for participation.
- Allow participants to “off-board” easily if they disagree with the proposed engagement process.
- Create an easy process for withdrawing from participation.
- Host pre-engagement sessions to help participants get online or become familiar with the engagement platform.
Maintain transparent recruitment processes
- Publish participant selection criteria and allow people to provide input on any exclusions that might harm the project.
- Include information about what participants will do and how participants’ feedback will be used in the recruitment materials.
- Clearly outline expectations of time required for participation.
Determine Organizational Capacity
Learn more about determining organizational capacity
Common Challenges
- Engagement projects pull product teams away from core duties.
- Participants lack resources necessary to join engagement activity.
- Engagement may slow down product launches.
Recommended Practices
Budget for the full participatory public engagement strategy
- Allocate funds to support staff time for all stages of the engagement, including pre-engagement research and post-engagement follow-up, as well as time spent on administrative and logistics tasks.
- Allocate funds for expenses related to participant support, including:
- participant compensation
- participant access costs like travel vouchers, internet data credit, etc.
- support activities such as translation or live captioning services, on-site childcare, etc.
- support services such as mental health counselors, legal support
Consult with internal experts or audiences and intermediaries
- Inventory internal expertise and skills available across organizational teams.
- Seek out and work with any public engagement or community experts on staff, even those outside the specific product team.
Establish internal processes and criteria for decision-making
- Internally discuss and set parameters on what participants can and cannot influence or change.
- Confirm that the engagement team has clear direction from organizational leaders on what can and cannot be shared with participants (or has a process for getting this guidance).
- Coordinate with product development engineers and others in the organization to ensure participant input supports better development.
Define the Relationship
Learn more about defining the relationship
Common Challenges
- Participants are worried about providing critical or negative feedback.
- Participants are worried about their credibility if they work with tech companies.
- Participants have concerns about potential exploitation or “participant washing.”
- Participants are concerned they might not see their feedback reflected in the process or final product.
Recommended Practices
Conduct a participant impact and risk assessment
- Work with internal and external experts and intermediaries to understand potential harms and risks to participants that can arise from working with your organization.
- Assess the engagement strategy and specific activities to address any coercive pressure participants may experience to discourage criticism of your organization or the product.
Address past harms
- Research and directly acknowledge any past harms committed by your organization with participants.
- Research and directly acknowledge any past harms related to the technology being discussed with participants.
Expand the impact of participants
- Identify topics or issues that can be negotiated to permit greater influence by participants.
- Allow product teams to revisit prior development stages to address issues identified by participants.
- Encourage participants to identify issues beyond the scoped topic for the engagement.
Create opportunities for “out of scope” discussions
- Encourage participants to provide feedback beyond the scoped topic to identify any major failures or risks they project.
- Incorporate time for participants to ask more general questions about the AI development process or the impact of the technology.
Prioritize participant well-being
- Prior to the engagement, survey participants to identify additional needs they may have that could impact their ability to participate.
- Dedicate time to learn about and develop strategies to address specific high-stakes situations participants might face, such as criminalization or imprisonment.
- Work with intermediaries to initially understand specific ways some groups might experience greater scrutiny or harm.
- Adopt whistleblower or other legal protections for participants to protect their ability to provide honest and critical feedback.
- Provide advocates or other support personnel for participants to contact with questions or concerns about their participation, particularly if they experience harm.
Compensate and credit participants
- Prior to, and throughout the engagement, host discussions explicitly about forms of compensation (financial, public acknowledgement, in-kind donations, etc.) that participants want.
- Provide financial compensation for participation.
- Provide financial support for additional costs related to participating, including travel vouchers, on-site childcare, meals.
Welcome opt outs
- Create clear pathways for participants to withdraw or opt out during and after engagement.
Limit how participation and participant feedback is used
- Restrict how other teams in the organization can use the data collected from participants, unless participants are fully aware of and consent to the use of their insights or data for these additional uses.
- Provide participants with highly specified release forms (dictating the use of their participation details) rather than general release forms that permit the organization to freely use any information collected during the engagement.
- Obtain explicit permission from participants to use their involvement to promote the public image of the organization.
Assess the Overall Process
Learn more about assessing the overall process
Common Challenges
- Participants consider the engagement work outside their expertise or usual scope of work.
- Participants lack resources necessary to join engagement activity.
- Different sets of participants provide opposing feedback.
- Participants and sponsoring teams might change in composition over the course of the entire engagement.
Recommended Practices
Address specific participant needs
- Dedicate time to learn about and develop strategies to address differences in cultural and communication style between the engagement team and the participants.
- Provide participants with multiple methods for providing feedback, such as written comments or voice recordings.
- Prior to the engagement, survey participants to identify additional needs they may have that could impact their ability to participate.
- Adapt data collection or input solicitation methods to be culturally appropriate and accessible to participants, such as story-telling methods, oral histories, use of drawings or physical models.
Establish a participant onboarding process
- Communicate the specific role, opportunities for impact, and limitations on what participants can influence before starting the engagement process.
- Clearly outline expectations of time required for participation.
- Allow participants to “off-board” easily if they disagree with the proposed engagement process.
- Create an easy process for withdrawing from participation.
- Host pre-engagement sessions to help participants get online or become familiar with the engagement platform.
Optimize for comprehensive feedback from participants
- Balance inquiries with limited-choice options or highly specified questions, such as surveys, with more open-ended discussions.
- Host engagement activities across the AI development life cycle that address different issues and concerns.
- Prioritize the schedules of participants to enable their most active and alert engagement.
Maintain thorough and secure documentation
- Explicitly assign team members to manage notes, documentation, and other materials from public engagement to ensure everything is organized and archived.
- Summarize and share the learning after engagement activities with participants.
- Securely store raw data and other engagement materials with identifying information or other personal details for the security of participants.
Enable communication and feedback mechanisms across phases
- Plan for follow-up communication with participants after the end of an engagement activity to share learning and offer further opportunities to provide feedback.
- Create mechanisms for participants to get regular updates on the project or changes to the product/service.
- Allow participants to opt in (rather than be forced to opt out) of post-engagement communications from the organization.
- Proactively communicate and provide evidence of how participant input influenced the development and deployment of the final product/feature.
Is this guidance missing something important? Suggest an update
Explore additional resources
PAI’s Guidance for Inclusive AI
Market Study for Advanced Narrow- and General-Purpose Models
This custom Guidance for Inclusive AI, for use by commercial sector AI developers and deployers, focuses on an initial set of recommendations for creating public engagement strategies. Additional guidance and cautions are available here. This should be treated as a brainstorming tool and not as a definitive checklist guide.
In general, market studies can be challenging because the scope of the engagement might be loosely defined, and it may be unclear how participant insights will be used (or not used) for product development. It is challenging to specify participants, or the expertise needed to engage with models, where AI use cases are broad and generalized.
Understand Incentives & Risks
Learn more about understanding the incentives & risks for engagement
Common Challenges
- Incentives for participatory public engagement are unclear.
- Participatory public engagement is not recognized as an organizational priority.
- Engagement projects pull product teams away from core duties.
Recommended Practices
Align organizationally on purpose and capacity
- Host meeting(s) with organizational and team leaders and key staff to establish a shared understanding of the value and importance of participatory public engagement.
- Specify concrete key-performance indicators or other organizational metrics, or recognize and award employees implementing public engagement strategies.
Conduct internal training
- Host training sessions for the engagement team (and the broader organization) on such topics as:
- purpose and value of participatory public engagement
- considerations and sensitivities when working with a specific socially marginalized community
- specific skills development for engagement facilitation and moderation
- data analysis skills, such as social scientific methods for qualitative data analysis
- Consult and hire external experts and intermediaries to lead training sessions specific to the needs of your team and the participants you will engage.
Identify, address, and communicate biases and exclusions
- Host internal meeting(s) to discuss specific team biases.
- Conduct pre-engagement research to better understand audiences that are frequently not engaged.
Address past harms
- Research and directly acknowledge any past harms committed by your organization with participants.
- Research and directly acknowledge any past harms related to the technology being discussed with participants.
Identify Key Actors
Learn more about identifying key actors
Common Challenges
- It is unclear who to target as participants.
- It is difficult to recruit participants in new or unfamiliar markets.
- Participants consider the topic outside their expertise or usual scope of work.
Recommended Practices
Conduct research on communities and known issues prior to engagement
- Consult intermediaries and other experts to develop baseline knowledge of the social impact issues affecting socially marginalized communities.
- Map known experts’ and intermediaries’ relevant expertise, and collaborate with them to identify gaps in your team’s knowledge or networks.
- Prioritize existing literature and research led by members of the communities, including work and people that are critical of your organization or the technology.
Design from the margins
- Explicitly prioritize people who might be outlier users facing greater likelihood of direct harm.
- Include recruitment of people who are likely to be impacted by the deployment of the product/service but are not direct users.
- Include participants who are explicitly critical of your organization or of AI technology.
Expand participant recruitment networks
- Work with employee resource groups (ERGs) or other internal audiences to identify additional external networks for participant recruitment.
- Reach out to customers or users who do not typically engage with the company to learn ways they could be encouraged to participate.
- Dedicate time to connect with and build relationships with many different civil society or community groups, rather than rely on a couple of strong existing relationships.
Collaborate with intermediaries
- Consult with intermediaries to identify and address team and organizational biases and limitations.
- Solicit external input on designing the participatory public engagement strategy and activities.
- Work with intermediaries or community leaders who have direct experience building relationships with the specific socially marginalized community(ies) you seek to work with.
Determine Organizational Capacity
Learn more about determining organizational capacity
Common Challenges
- It is unclear how to incorporate participant feedback.
- Engagement may yield no useful insights.
Recommended Practices
Dedicate time for research and development
- Conduct existing literature and research review to establish baseline understanding of social impacts and harms of the technology.
- Conduct existing literature and research review to establish baseline understanding of historic and current issues affecting socially marginalized communities.
- Research past public engagements hosted by your organization, including the impact those engagements had on different participant groups.
Establish internal processes and criteria for decision-making
- Internally discuss and set parameters on what participants can and cannot influence or change.
- Confirm that the engagement team has clear direction from organizational leaders on what can and cannot be shared with participants (or has a process for getting this guidance).
Consult with internal experts or audiences and intermediaries
- Seek out and work with any public engagement or community experts on staff, even those outside the specific product team.
- Seek out and work with other internal audiences to revise, validate, and/or pilot engagement strategies.
- Work with intermediaries or community leaders to translate insights into actionable changes for the product.
Budget for the full participatory public engagement strategy
- Allocate funds to support staff time for all stages of the engagement, including pre-engagement research and post-engagement follow-up, as well as time spent on administrative and logistics tasks.
- Allocate funds for expenses related to participant support, including:
- participant compensation
- participant access costs like travel vouchers, internet data credit, etc.
- support activities such as translation or live captioning services, on-site childcare, etc.
- support services such as mental health counselors, legal support
Consult with internal experts or audiences and intermediaries
- Inventory internal expertise and skills available across organizational teams.
- Seek out and work with any public engagement or community experts on staff, even those outside the specific product team.
Define the Relationship
Learn more about defining the relationship
Common Challenges
- Participants are concerned they might not see their feedback reflected in the process or final product.
- Participants have concerns about potential exploitation or “participant washing.”
- Participant incentives for participatory public engagement are unclear.
Recommended Practices
Establish participant communication expectations and processes
- Share details about how you plan to communicate with participants during and after the engagement.
- Establish communication mechanisms for participants to reach out to you after the engagement.
- Maintain multiple points of contact for participants to reach out to you during and after the engagement.
Address specific participant needs, including for those in high-stakes situations
- Dedicate time to learn about and develop strategies to address differences in cultural and communication style between the engagement team and the participants.
- Work with intermediaries to initially understand specific ways some groups might experience greater scrutiny or harm.
- Ask for input on what needs to change to invite participants to be more engaged and share honest feedback.
- Adapt data collection or input solicitation methods to make them culturally appropriate and accessible to participants, such as storytelling methods, oral histories, and use of drawings or physical models.
Expand the impact of participants
- Identify topics or issues that can be negotiated to permit greater influence by participants.
- Allow product teams to revisit prior development stages to address issues identified by participants.
- Encourage participants to identify issues beyond the scoped topic for the engagement.
Create opportunities for “out of scope” discussions
- Encourage participants to provide feedback beyond the scoped topic to identify any major failures or risks they project.
- Incorporate time for participants to ask more general questions about the AI development process or the impact of the technology.
Enable communication and feedback mechanisms across phases
- Plan for follow-up communication with participants after the end of an engagement activity to share learning and offer further opportunities to provide feedback.
- Create mechanisms for participants to get regular updates on the project or changes to the product/service.
- Allow participants to opt in (rather than be forced to opt out) of post-engagement communications from the organization.
- Proactively communicate and provide evidence of how participant input influenced the development and deployment of the final product/feature.
Welcome opt outs
- Create clear pathways for participants to withdraw or opt out during and after engagement.
Assess the Overall Process
Learn more about assessing the overall process
Common Challenges
- Different sets of participants provide opposing feedback.
- Participant feedback feels highly specific to certain individuals or communities.
- Participants lack resources necessary to join engagement activity.
Recommended Practices
Limit generalizations of learning
- Recognize the limits of participatory public engagement, and avoid generalizing the experiences from individuals to broad populations without validating them against other studies or engagements.
- Continuously refine and specify distinctions within socially marginalized groups to better understand how members of a given social identity group may have different experiences or needs.
Limit how participation and participant feedback is used
- Restrict how other teams in the organization can use the data collected from participants, unless participants are fully aware of and consent to the use of their insights or data for these additional uses.
- Provide participants with highly specified release forms (dictating the use of their participation details) rather than general release forms that permit the organization to freely use any information collected during the engagement.
- Obtain explicit permission from participants to use their involvement to promote the public image of the organization.
Optimize for comprehensive feedback from participants
- Balance inquiries with limited-choice options or highly specified questions, such as surveys, with more open-ended discussions.
Conduct a participant impact and risk assessment
- Work with internal and external experts and intermediaries to understand potential harms and risks to participants that can arise from working with your organization.
- Address potential confirmation bias in your engagement (e.g., leading questions, prompts that force agreement by the participant) by designing engagement activities, including the questions asked, to generate more open-ended discussions with participants.
Address specific participant needs
- Dedicate time to learn about and develop strategies to address differences in cultural and communication style between the engagement team and the participants.
- Discuss timelines and deadlines with participants.
- Provide participants with multiple methods for providing feedback, such as written comments or voice recordings.
- Prior to the engagement, survey participants to identify additional needs they may have that could impact their ability to participate.
- Adapt data collection or input solicitation methods to make them culturally appropriate and accessible to participants, such as storytelling methods, oral histories, and use of drawings or physical models.
Assess for accessibility of engagement venue or platform
- Work with experts, intermediaries, and participants to assess the engagement venue (in-person sessions) or platform (virtual sessions) for accessibility, including:
- physical accessibility
- language/content accessibility
- financial accessibility
- technological accessibility
- Prioritize scheduling and time zones of participants to better ensure their full engagement and attentiveness during the engagement activity.
- Provide easy mechanisms for participants to confidentially share their accessibility needs.
Is this guidance missing something important? Suggest an update
Explore additional resources
PAI’s Guidance for Inclusive AI
User Data Collection & Enrichment for Advanced Narrow- & General-Purpose Models
This custom Guidance for Inclusive AI, for use by commercial sector AI developers and deployers, focuses on an initial set of recommendations for creating public engagement strategies. Additional guidance and cautions are available here. This should be treated as a brainstorming tool and not as a definitive checklist guide.
In general, data collection and enrichment can be challenging because good datasets are a key input for AI models, and possession of these datasets is a financial asset for companies. These financial gains are not often shared with the people who contribute, or annotate, the data unless structured into properly paid jobs. It is challenging to specify participants, or the expertise needed to engage with models, where AI use cases are broad and generalized.
Understand Incentives & Risks
Learn more about understanding the incentives & risks for engagement
Common Challenges
- Organizational leadership is unfamiliar with participatory public engagement.
- Participant incentives for participatory public engagement are unclear.
- Engagement may slow down product launches.
Recommended Practices
Conduct internal training
- Host training sessions for the engagement team (and the broader organization) on such topics as:
- purpose and value of participatory public engagement
- existing gaps in available datasets that might result in poorer-quality AI products
Align organizationally on purpose and capacity
- Host meeting(s) with organizational and team leaders and key staff to establish a shared understanding of the value and importance of participatory public engagement.
- Specify concrete key-performance indicators or other organizational metrics, or recognize and award employees implementing public engagement strategies.
Consult intermediaries and participants to validate the engagement design
- Work with intermediaries and participants to identify any additional needs participants may have that could impact their ability to participate.
Establish internal processes and criteria for decision-making
- Coordinate with product development engineers and others in the organization to ensure participant input supports better development.
- Determine the gaps in existing datasets to optimize data collection from participants.
Identify Key Actors
Learn more about identifying key actors
Common Challenges
- Expanding the participant recruitment network is difficult.
Recommended Practices
Conduct research on communities and known issues prior to engagement
- Conduct social, scientific, and sociotechnical research to understand current and historical issues impacting socially marginalized communities.
- Map known experts’ and intermediaries’ relevant expertise, and collaborate with them to identify gaps in your team’s knowledge or networks.
- Prioritize existing literature and research led by members of the communities, including work and people that are critical of your organization or the technology.
Design from the margins
- Explicitly prioritize people who might be outlier users facing greater likelihood of direct harm.
- Include recruitment of people who are likely to be impacted by the deployment of the product/service but are not direct users.
- Include participants who are explicitly critical of your organization or of AI technology.
Expand participant recruitment networks
- Work with employee resource groups (ERGs) or other internal audiences to identify additional external networks for participant recruitment.
- Reach out to customers or users who do not typically engage with the company to learn ways they could be encouraged to participate.
- Dedicate time to connect with and build relationships with many different civil society or community groups, rather than rely on a couple of strong existing relationships.
Determine Organizational Capacity
Learn more about determining organizational capacity
Common Challenges
- Participants lack resources necessary to join engagement activity.
Recommended Practices
Prioritize participant well-being
- Negotiate internal policies and processes such as nondisclosure agreement requirements to better meet participant needs.
- Identify areas in which the organization can share decision-making or benefits to participants, such as establishing data dividends for contributing to the dataset.
- Reduce any administrative barriers for participants by simplifying onboarding or compensation processes.
- Request feedback from participants on the design of the engagement activity to align with participants’ working and communication styles.
- Provide post-engagement support to address any emotional, physical, or psychological harms experienced by participants due to the engagement.
Determine specific participant needs
- Dedicate time to learn about and develop strategies to address differences in cultural and communication style between the engagement team and the participants.
- Discuss timelines and deadlines with participants.
- Ask for input on what needs to change to invite participants to be more engaged and share honest feedback.
Determine participant needs in high-stakes situations
- Dedicate time to learn about and develop strategies to address specific high-stakes situations participants might face, such as criminalization or imprisonment.
- Work with intermediaries to initially understand specific ways some groups might experience greater scrutiny or harm.
- Align internal resources, such as mental health professionals or access to legal support, for participants.
- Increase participants’ data privacy and security.
Budget for the full participatory public engagement strategy
- Allocate funds to support staff time for all stages of the engagement, including pre-engagement research and post-engagement follow-up, as well as time spent on administrative and logistics tasks.
Define the Relationship
Learn more about defining the relationship
Common Challenges
- Participants assume they lack familiarity with the topic to engage.
- The sponsoring organization has a poor reputation among the participant community.
Recommended Practices
Establish a participant onboarding process
- Communicate the specific role, opportunities for impact, and limitations on what participants can influence before starting the engagement process.
- Clearly outline expectations of time required for participation.
- Allow participants to “off-board” easily if they disagree with the proposed engagement process.
- Host pre-engagement sessions to help participants get online or become familiar with the engagement platform.
- Establish clear work processes and expectations for participants contributing or annotating data
Address past harms
- Research and directly acknowledge any past harms committed by your organization with participants.
- Research and directly acknowledge any past harms related to the technology being discussed with participants.
Assess the Overall Process
Learn more about assessing the overall process
Common Challenges
- The overall impact of collecting or processing data on participants is unclear.
- Participants have concerns about potential exploitation or “participant washing.”
Recommended Practices
Expand the impact of participants
- Identify topics or issues that can be negotiated to permit greater influence by participants.
- Allow product teams to revisit prior development stages to address issues identified by participants.
- Encourage participants to identify issues beyond the scoped topic for the engagement.
Compensate and credit participants
- Prior to, and throughout the engagement, host discussions explicitly about forms of compensation (financial, public acknowledgement, in-kind donations, etc.) that participants want.
- Provide financial compensation for participation.
- Provide financial support for additional costs related to participating, including travel vouchers, on-site childcare, meals.
Conduct a participant impact and risk assessment
- Work with internal and external experts and intermediaries to understand potential harms and risks to participants that can arise from working with your organization.
- Assess the engagement strategy and specific activities to address any coercive pressure participants may experience to discourage criticism of your organization or the product.
Limit how participation and participant feedback is used
- Restrict how other teams in the organization can use the data collected from participants, unless participants are fully aware of and consent to the use of their insights or data for these additional uses.
- Provide participants with highly specified release forms (dictating the use of their participation details) rather than general release forms that permit the organization to freely use any information collected during the engagement.
Is this guidance missing something important? Suggest an update
Explore additional resources
PAI’s Guidance for Inclusive AI
Risk & Failure Assessments for Advanced Narrow- & General-Purpose Models
This custom Guidance for Inclusive AI, for use by commercial sector AI developers and deployers, focuses on an initial set of recommendations for creating public engagement strategies. Additional guidance and cautions are available here. This should be treated as a brainstorming tool and not as a definitive checklist guide.
In general, risk and failure assessment can be challenging because participants need access to early versions of the AI models to test them, leaving sponsors vulnerable. These financial gains are not often shared with the people who contribute, or annotate, the data unless structured into properly paid jobs. Assessing for risks and failure might also expose participants to harmful content or output. It is challenging to specify participants, or the expertise needed to engage with models, where AI use cases are broad and generalized.
Understand Incentives & Risks
Learn more about understanding the incentives & risks for engagement
Common Challenges
- Organizational concerns about public criticism or negative PR.
- Engagement may cause friction between teams within an organization.
- Participants are worried about their credibility if they work with tech companies.
Recommended Practices
Conduct research on communities and known issues prior to engagement
- Conduct social, scientific, and sociotechnical research to understand current and historical issues impacting socially marginalized communities.
- Prioritize existing literature and research led by members of the communities, including work and people that are critical of your organization or the technology.
- Research past public engagements hosted by your organization, including the impact those engagements had on different participant groups.
- Consult intermediaries and other experts to develop baseline knowledge of the social impact issues affecting socially marginalized communities.
Align organizationally on purpose and capacity
- Host meeting(s) with organizational and team leaders and key staff to establish a shared understanding of the value and importance of participatory public engagement.
- Specify concrete key-performance indicators or other organizational metrics, or recognize and award employees implementing public engagement strategies.
Establish internal processes and criteria for decision-making
- Internally discuss and set parameters on what participants can and cannot influence or change.
- Confirm that the engagement team has clear direction from organizational leaders on what can and cannot be shared with participants (or has a process for getting this guidance).
Address past harms
- Research and directly acknowledge any past harms committed by your organization with participants.
- Research and directly acknowledge any past harms related to the technology being discussed with participants.
Identify Key Actors
Learn more about identifying key actors
Common Challenges
- The same set of external experts are recruited.
- Participants consider the topic outside their expertise or usual scope of work.
Recommended Practices
Design from the margins
- Explicitly prioritize people who might be outlier users facing greater likelihood of direct harm.
- Include recruitment of people who are likely to be impacted by the deployment of the product/service but are not direct users.
- Include participants who are explicitly critical of your organization or of AI technology.
- Engage with, and explore, conflicting data points or outliers to determine whether they are true outliers or indicators of key edge cases.
Specify the types of expertise needed
- Align the different forms of formal and informal expertise that can help address your team’s participatory public engagement objectives and/or your team’s specific gaps in awareness.
- Conduct additional research on available formal and informal experts for a topic, rather than rely on such common proxies for expertise as elite university educational affiliations.
Expand participant recruitment networks
- Work with employee resource groups (ERGs) or other internal audiences to identify additional external networks for participant recruitment.
- Reach out to customers or users who do not typically engage with the company to learn ways they could be encouraged to participate.
- Dedicate time to connect with and build relationships with many different civil society or community groups, rather than rely on a couple of strong existing relationships.
- Encourage past participants or others in your network to recommend new participants.
Collaborate with intermediaries
- Work with intermediaries or community leaders who have direct experience building relationships with the specific socially marginalized community(ies) you seek to work with.
- Map known experts’ and intermediaries’ relevant expertise, and collaborate with them to identify gaps in your team’s knowledge or networks.
Maintain transparent recruitment processes
- Publish participant selection criteria and allow people to provide input on any exclusions that might harm the project.
- Include information about what participants will do and how participants’ feedback will be used in the recruitment materials.
- Host recruitment sessions wherein interested participants can pose questions to the organizers.
Determine Organizational Capacity
Learn more about determining organizational capacity
Common Challenges
- Participants lack resources necessary to join engagement activity.
- It is unclear how to incorporate participant feedback, particularly non-technical feedback.
Recommended Practices
Align resources to address specific participant needs
- Dedicate time to learn about and develop strategies to address differences in cultural and communication style between the engagement team and the participants.
- Discuss timelines and deadlines with participants.
- Provide participants with multiple methods for providing feedback, such as written comments or voice recordings.
- Prior to the engagement, survey participants to identify additional needs they may have that could impact their ability to participate.
- Adapt data collection or input solicitation methods to make them culturally appropriate and accessible to participants, such as storytelling methods, oral histories, and use of drawings or physical models.
Assess for accessibility of engagement venue or platform
- Work with experts, intermediaries, and participants to assess the engagement venue (in-person sessions) or platform (virtual sessions) for accessibility, including:
- physical accessibility
- language/content accessibility
- financial accessibility
- technological accessibility
- Provide easy mechanisms for participants to confidentially share their accessibility needs.
- Prioritize scheduling and time zones of participants to better ensure their full engagement and attentiveness during the engagement activity.
Conduct internal training
- Host training sessions for the engagement team (and the broader organization) on such topics as:
- purpose and value of participatory public engagement
- considerations and sensitivities when working with a specific socially marginalized community
- specific skills development for engagement facilitation and moderation
- data analysis skills, such as social scientific methods for qualitative data analysis
Consult with internal experts or audiences and intermediaries
- Seek out and work with any public engagement or community experts on staff, even those outside the specific product team.
- Seek out and work with other internal audiences to revise, validate, and/or pilot engagement strategies.
- Work with intermediaries or community leaders to translate insights into actionable changes for the product.
Consult intermediaries and participants to validate the engagement design
- Work with intermediaries and participants to identify any additional needs participants may have that could impact their ability to participate.
- Brainstorm and define risk and harm categories with experts, intermediaries, and participants to revise engagement design.
Define the Relationship
Learn more about defining the relationship
Common Challenges
- Participants are worried about providing critical or negative feedback.
- Participants have concerns about potential exploitation or “participant washing.”
Recommended Practices
Create opportunities for “out of scope” discussions
- Encourage participants to provide feedback beyond the scoped topic to identify any major failures or risks they project.
- Incorporate time for participants to ask more general questions about the AI development process or the impact of the technology.
Conduct a participant impact and risk assessment
- Work with internal and external experts and intermediaries to understand potential harms and risks to participants that can arise from working with your organization.
- Assess the engagement strategy and specific activities to address any coercive pressure participants may experience to discourage criticism of your organization or the product.
Prioritize participant well-being
- Negotiate internal policies and processes such as nondisclosure agreement requirements to better meet participant needs.
- Work with intermediaries if participants need greater protection or separation from the organization sponsoring the engagement.
- Provide post-engagement support to address any emotional, physical, or psychological harms experienced by participants due to the engagement.
- Provide explicit mechanisms for participants to opt-out of the engagement at any time.
Address participant needs in high-stakes situations
- Dedicate time to learn about and develop strategies to address specific high-stakes situations participants might face, such as criminalization or imprisonment.
- Work with intermediaries to initially understand specific ways some groups might experience greater scrutiny or harm.
- Adopt whistleblower or other legal protections for participants to protect their ability to provide honest and critical feedback.
- Provide advocates or other support personnel for participants to contact with questions or concerns about their participation, particularly if they experience harm.
- Align internal resources, such as mental health professionals or access to legal support, for participants.
- Increase participants’ data privacy and security.
Compensate and credit participants
- Prior to, and throughout the engagement, host discussions explicitly about forms of compensation (financial, public acknowledgement, in-kind donations, etc.) that participants want.
- Provide financial compensation for participation.
- Provide financial support for additional costs related to participating, including travel vouchers, on-site childcare, meals.
Welcome opt outs
- Create clear pathways for participants to withdraw or opt out during and after engagement.
Assess the Overall Process
Learn more about assessing the overall process
Common Challenges
- Different sets of participants provide opposing feedback.
- It is unclear how to incorporate participant feedback.
Recommended Practices
Expand the impact of participants
- Identify topics or issues that can be negotiated to permit greater influence by participants.
- Allow product teams to revisit prior development stages to address issues identified by participants.
- Encourage participants to identify issues beyond the scoped topic for the engagement.
Conduct internal training
- Host training sessions for the engagement team (and the broader organization) on such topics as:
- purpose and value of participatory public engagement
- considerations and sensitivities when working with a specific socially marginalized community
- specific skills development for engagement facilitation and moderation
- data analysis skills, such as social scientific methods for qualitative data analysis
Collaborate with intermediaries
- Work with intermediaries or community leaders who have direct experience building relationships with the specific socially marginalized community(ies) you seek to work with.
- Work with intermediaries or community leaders to translate insights into actionable changes for the product.
Is this guidance missing something important? Suggest an update
Explore additional resources
PAI’s Guidance for Inclusive AI
User Experience & Testing for Advanced Narrow- & General-Purpose Models
This custom Guidance for Inclusive AI, for use by commercial sector AI developers and deployers, focuses on an initial set of recommendations for creating public engagement strategies. Additional guidance and cautions are available here. This should be treated as a brainstorming tool and not as a definitive checklist guide.
In general, user experience and testing is a well-established stage of model development, increasing the likelihood that organizational leaders will readily support this public engagement activity. It is challenging to specify participants, or the expertise needed to engage with models, where AI use cases are broad and generalized.
Understand Incentives & Risks
Learn more about understanding the incentives & risks for engagement
Common Challenges
- The sponsoring organization has a poor reputation among the participant community.
- Engagement may slow down product launches.
- There are concerns that participants will leak IP or publicly criticize the project.
Recommended Practices
Dedicate time for research and development
- Conduct existing literature and research review to establish baseline understanding of social impacts and harms of the technology.
- Conduct existing literature and research review to establish baseline understanding of historic and current issues affecting socially marginalized communities.
- Prioritize existing literature and research led by members of the communities, including work and people that are critical of your organization or the technology.
- Research past public engagements hosted by your organization, including the impact those engagements had on different participant groups.
- Consult intermediaries and other experts to develop baseline knowledge of the social impact issues affecting socially marginalized communities.
Consult with internal experts or audiences and intermediaries
- Inventory internal expertise and skills available across organizational teams.
- Seek out and work with any public engagement or community experts on staff, even those outside the specific product team.
Align organizationally on purpose and capacity
- Host meeting(s) with organizational and team leaders and key staff to establish a shared understanding of the value and importance of participatory public engagement.
- Specify concrete key-performance indicators or other organizational metrics, or recognize and award employees implementing public engagement strategies.
Establish internal processes and criteria for decision-making
- Internally discuss and set parameters on what participants can and cannot influence or change.
- Confirm that the engagement team has clear direction from organizational leaders on what can and cannot be shared with participants (or has a process for getting this guidance).
- Communicate organizational processes and limitations to participants as part of the participant onboarding process.
Identify Key Actors
Learn more about identifying key actors
Common Challenges
- It is unclear who to target as participants.
- It is difficult to recruit participants in new or unfamiliar markets.
- The same set of participants are recruited.
Recommended Practices
Collaborate with intermediaries
- Work with intermediaries or community leaders who have direct experience building relationships with the specific socially marginalized community(ies) you seek to work with.
- Map known experts’ and intermediaries’ relevant expertise, and collaborate with them to identify gaps in your team’s knowledge or networks.
Design from the margins
- Explicitly prioritize people who might be outlier users facing greater likelihood of direct harm.
- Include recruitment of people who are likely to be impacted by the deployment of the product/service but are not direct users.
- Include participants who are explicitly critical of your organization or of AI technology.
- Engage with, and explore, conflicting data points or outliers to determine whether they are true outliers or indicators of key edge cases.
Expand participant recruitment networks
- Work with employee resource groups (ERGs) or other internal audiences to identify additional external networks for participant recruitment.
- Reach out to customers or users who do not typically engage with the company to learn ways they could be encouraged to participate.
- Dedicate time to connect with and build relationships with many different civil society or community groups, rather than rely on a couple of strong existing relationships.
- Encourage past participants or others in your network to recommend new participants.
Determine Organizational Capacity
Learn more about determining organizational capacity
Common Challenges
- Engagement may cause friction between teams within an organization.
Recommended Practices
Conduct internal training
- Host training sessions for the engagement team (and the broader organization) on such topics as:
- purpose and value of participatory public engagement
- considerations and sensitivities when working with a specific socially marginalized community
- specific skills development for engagement facilitation and moderation
- data analysis skills, such as social scientific methods for qualitative data analysis
- Host retrospectives to discuss and learn from previous public engagement activities undertaken by your organization.
Coordinate application of participant insights to product development
- Coordinate with product development engineers and others in the organization to ensure participant input supports better development.
Budget for the full participatory public engagement strategy
- Allocate funds to support staff time for all stages of the engagement, including pre-engagement research and post-engagement follow-up, as well as time spent on administrative and logistics tasks.
- Allocate funds for expenses related to participant support, including:
- participant compensation
- participant access costs like travel vouchers, internet data credit, etc.
- support activities such as translation or live captioning services, on-site childcare, etc.
- support services such as mental health counselors, legal support
Define the Relationship
Learn more about defining the relationship
Common Challenges
- Participants have concerns about potential exploitation or “participant washing.”
- Participants consider the topic outside their expertise or usual scope of work.
- Participants are worried about providing critical or negative feedback.
Recommended Practices
Expand the impact of participants
- Identify topics or issues that can be negotiated to permit greater influence by participants.
- Allow product teams to revisit prior development stages to address issues identified by participants.
- Encourage participants to identify issues beyond the scoped topic for the engagement.
Compensate and credit participants
- Prior to, and throughout the engagement, host discussions explicitly about forms of compensation (financial, public acknowledgement, in-kind donations, etc.) that participants want.
- Provide financial compensation for participation.
- Provide financial support for additional costs related to participating, including travel vouchers, on-site childcare, meals.
Prioritize participant well-being, including in high-stakes situations
- Negotiate internal policies and processes such as nondisclosure agreement requirements to better meet participant needs.
- Identify areas in which the organization can share decision-making or benefits to participants, such as establishing data dividends for contributing to the dataset.
- Request feedback from participants on the design of the engagement activity to align with participants’ working and communication styles.
- Work with intermediaries to initially understand specific ways some groups might experience greater scrutiny or harm.
- Adopt whistleblower or other legal protections for participants to protect their ability to provide honest and critical feedback.
Conduct a participant impact and risk assessment
- Work with internal and external experts and intermediaries to understand potential harms and risks to participants that can arise from working with your organization.
- Assess the engagement strategy and specific activities to address any coercive pressure participants may experience to discourage criticism of your organization or the product.
Limit how participation and participant feedback is used
- Restrict how other teams in the organization can use the data collected from participants, unless participants are fully aware of and consent to the use of their insights or data for these additional uses.
- Provide participants with highly specified release forms (dictating the use of their participation details) rather than general release forms that permit the organization to freely use any information collected during the engagement.
- Obtain explicit permission from participants to use their involvement to promote the public image of the organization.
Assess the Overall Process
Learn more about assessing the overall process
Common Challenges
- There are concerns that engagement might yield no useful insights.
- Participants are concerned they might not see their feedback reflected in the process or final product.
- Participant feedback feels highly specific to certain individuals or communities.
Recommended Practices
Optimize for comprehensive feedback from participants
- Balance inquiries with limited-choice options or highly specified questions, such as surveys, with more open-ended discussions.
Establish participant communication expectations and processes
- Share details about how you plan to communicate with participants during and after the engagement.
- Establish communication mechanisms for participants to reach out to you after the engagement.
- Maintain multiple points of contact for participants to reach out to you during and after the engagement.
Create opportunities for “out of scope” discussions
- Encourage participants to provide feedback beyond the scoped topic to identify any major failures or risks they project.
- Incorporate time for participants to ask more general questions about the AI development process or the impact of the technology.
Practice active listening
- Repeat what you’ve learned from participants and outline how their insights will be used.
- Share any publications with participants prior to release and confirm their consent to having their insights or participation referenced or used.
Limit generalizations of learning
- Recognize the limits of participatory public engagement, and avoid generalizing the experiences from individuals to broad populations without validating them against other studies or engagements.
- Continuously refine and specify distinctions within socially marginalized groups to better understand how members of a given social identity group may have different experiences or needs.
Is this guidance missing something important? Suggest an update
Explore additional resources
PAI’s Guidance for Inclusive AI
Oversight for Advanced Narrow- & General-Purpose Models
This custom Guidance for Inclusive AI, for use by commercial sector AI developers and deployers, focuses on an initial set of recommendations for creating public engagement strategies. Additional guidance and cautions are available here. This should be treated as a brainstorming tool and not as a definitive checklist guide.
In general, oversight is challenging because the scope of work for participants might be more open-ended and require longer-term engagements working with the sponsors across the full development life cycle. It is challenging to specify participants, or the expertise needed to engage with models, where AI use cases are broad and generalized.
Understand Incentives & Risks
Learn more about understanding the incentives & risks for engagement
Common Challenges
- Participatory public engagement is not recognized as an organizational priority.
- Organizational concerns about public criticism or negative PR.
Recommended Practices
Align organizationally on purpose and capacity
- Host meeting(s) with organizational and team leaders and key staff to establish a shared understanding of the value and importance of participatory public engagement.
- Specify concrete key-performance indicators or other organizational metrics, or recognize and award employees implementing public engagement strategies.
Conduct internal training
- Host training sessions for the engagement team (and the broader organization) on such topics as:
- purpose and value of participatory public engagement
- considerations and sensitivities when working with a specific socially marginalized community
- specific skills development for engagement facilitation and moderation
- data analysis skills, such as social scientific methods for qualitative data analysis
Dedicate time for research and development prior to recruitment and engagement
- Conduct existing literature and research review to establish baseline understanding of social impacts and harms of the technology.
- Conduct existing literature and research review to establish baseline understanding of historic and current issues affecting socially marginalized communities.
- Research past public engagements hosted by your organization, including the impact those engagements had on different participant groups.
Collaborate with intermediaries
- Consult with intermediaries to identify and address team and organizational biases and limitations, particularly where the organization might face public scrutiny.
- Work with intermediaries or community leaders who have direct experience building relationships with the specific socially marginalized community(ies) you seek to work with.
Address past harms
- Research and directly acknowledge any past harms committed by your organization with participants.
- Research and directly acknowledge any past harms related to the technology being discussed with participants.
Identify Key Actors
Learn more about identifying key actors
Common Challenges
- The same set of external experts are recruited.
- It is difficult to determine participants’ (including experts’ and intermediaries’) credibility.
- Participants might have limited time to commit to an extended engagement.
Recommended Practices
Design from the margins
- Explicitly prioritize people who might be outlier users facing greater likelihood of direct harm.
- Include recruitment of people who are likely to be impacted by the deployment of the product/service but are not direct users.
- Include participants who are explicitly critical of your organization or of AI technology.
- Engage with, and explore, conflicting data points or outliers to determine whether they are true outliers or indicators of key edge cases.
- Map known experts’ and intermediaries’ relevant expertise, and collaborate with them to identify gaps in your team’s knowledge or networks.
Expand participant recruitment networks
- Work with employee resource groups (ERGs) or other internal audiences to identify additional external networks for participant recruitment.
- Reach out to customers or users who do not typically engage with the company to learn ways they could be encouraged to participate.
- Dedicate time to connect with and build relationships with many different civil society or community groups to validate expertise and credibility.
- Encourage past participants or others in your network to recommend new participants.
Specify the types of expertise needed
- Align the different forms of formal and informal expertise that can help address your team’s participatory public engagement objectives and/or your team’s specific gaps in awareness.
- Conduct additional research on available formal and informal experts for a topic, rather than rely on such common proxies for expertise as elite university educational affiliations.
- Prior to the engagement, survey participants to identify any additional needs they may have that can impact their ability to participate.
Collaborate with intermediaries
- Consult with intermediaries to identify and address team and organizational biases and limitations.
- Work with intermediaries or community leaders who have direct experience building relationships with the specific socially marginalized community(ies) you seek to work with.
Establish a participant onboarding process
- Communicate the specific role, opportunities for impact, and limitations on what participants can influence before starting the engagement process.
- Clearly outline expectations of time required for participation.
- Allow participants to “off-board” easily if they disagree with the proposed engagement process.
- Create an easy process for withdrawing from participation.
- Host pre-engagement sessions to help participants get online or become familiar with the engagement platform.
Maintain transparent recruitment processes
- Publish participant selection criteria and allow people to provide input on any exclusions that might harm the project.
- Include information about what participants will do and how participants’ feedback will be used in the recruitment materials.
- Clearly outline expectations of time required for participation.
Determine Organizational Capacity
Learn more about determining organizational capacity
Common Challenges
- Engagement projects pull product teams away from core duties.
- Participants lack resources necessary to join engagement activity.
- Engagement may slow down product launches.
Recommended Practices
Budget for the full participatory public engagement strategy
- Allocate funds to support staff time for all stages of the engagement, including pre-engagement research and post-engagement follow-up, as well as time spent on administrative and logistics tasks.
- Allocate funds for expenses related to participant support, including:
- participant compensation
- participant access costs like travel vouchers, internet data credit, etc.
- support activities such as translation or live captioning services, on-site childcare, etc.
- support services such as mental health counselors, legal support
Consult with internal experts or audiences and intermediaries
- Inventory internal expertise and skills available across organizational teams.
- Seek out and work with any public engagement or community experts on staff, even those outside the specific product team.
Establish internal processes and criteria for decision-making
- Internally discuss and set parameters on what participants can and cannot influence or change.
- Confirm that the engagement team has clear direction from organizational leaders on what can and cannot be shared with participants (or has a process for getting this guidance).
- Coordinate with product development engineers and others in the organization to ensure participant input supports better development.
Define the Relationship
Learn more about defining the relationship
Common Challenges
- Participants are worried about their credibility if they work with tech companies.
- Participants have concerns about potential exploitation or “participant washing.”
- Participants are concerned they might not see their feedback reflected in the process or final product.
Recommended Practices
Conduct a participant impact and risk assessment
- Work with internal and external experts and intermediaries to understand potential harms and risks to participants that can arise from working with your organization.
- Assess the engagement strategy and specific activities to address any coercive pressure participants may experience to discourage criticism of your organization or the product.
Address past harms
- Research and directly acknowledge any past harms committed by your organization with participants.
- Research and directly acknowledge any past harms related to the technology being discussed with participants.
Expand the impact of participants
- Identify topics or issues that can be negotiated to permit greater influence by participants.
- Allow product teams to revisit prior development stages to address issues identified by participants.
- Encourage participants to identify issues beyond the scoped topic for the engagement.
Create opportunities for “out of scope” discussions
- Encourage participants to provide feedback beyond the scoped topic to identify any major failures or risks they project.
- Incorporate time for participants to ask more general questions about the AI development process or the impact of the technology.
Prioritize participant well-being
- Prior to the engagement, survey participants to identify additional needs they may have that could impact their ability to participate
- Dedicate time to learn about and develop strategies to address specific high-stakes situations participants might face, such as criminalization or imprisonment.
- Work with intermediaries to initially understand specific ways some groups might experience greater scrutiny or harm.
- Adopt whistleblower or other legal protections for participants to protect their ability to provide honest and critical feedback.
- Provide advocates or other support personnel for participants to contact with questions or concerns about their participation, particularly if they experience harm.
Compensate and credit participants
- Prior to, and throughout the engagement, host discussions explicitly about forms of compensation (financial, public acknowledgement, in-kind donations, etc.) that participants want.
- Provide financial compensation for participation.
- Provide financial support for additional costs related to participating, including travel vouchers, on-site childcare, meals.
Welcome opt outs
- Create clear pathways for participants to withdraw or opt out during and after engagement.
Limit how participation and participant feedback is used
- Restrict how other teams in the organization can use the data collected from participants, unless participants are fully aware of and consent to the use of their insights or data for these additional uses.
- Provide participants with highly specified release forms (dictating the use of their participation details) rather than general release forms that permit the organization to freely use any information collected during the engagement.
- Obtain explicit permission from participants to use their involvement to promote the public image of the organization.
Assess the Overall Process
Learn more about assessing the overall process
Common Challenges
- Participants consider the engagement work outside their expertise or usual scope of work.
- Participants lack resources necessary to join engagement activity.
- Different sets of participants provide opposing feedback.
- Participants and sponsoring teams might change in composition over the course of the entire engagement.
Recommended Practices
Address specific participant needs
- Dedicate time to learn about and develop strategies to address differences in cultural and communication style between the engagement team and the participants.
- Provide participants with multiple methods for providing feedback, such as written comments or voice recordings.
- Prior to the engagement, survey participants to identify additional needs they may have that could impact their ability to participate.
- Adapt data collection or input solicitation methods to be culturally appropriate and accessible to participants, such as story-telling methods, oral histories, use of drawings or physical models.
Establish a participant onboarding process
- Communicate the specific role, opportunities for impact, and limitations on what participants can influence before starting the engagement process.
- Clearly outline expectations of time required for participation.
- Allow participants to “off-board” easily if they disagree with the proposed engagement process.
- Create an easy process for withdrawing from participation.
- Host pre-engagement sessions to help participants get online or become familiar with the engagement platform.
Optimize for comprehensive feedback from participants
- Balance inquiries with limited-choice options or highly specified questions, such as surveys, with more open-ended discussions.
- Host engagement activities across the AI development life cycle that address different issues and concerns.
- Prioritize the schedules of participants to enable their most active and alert engagement.
Maintain thorough and secure documentation
- Explicitly assign team members to manage notes, documentation, and other materials from public engagement to ensure everything is organized and archived.
- Summarize and share the learning after engagement activities with participants.
- Securely store raw data and other engagement materials with identifying information or other personal details for the security of participants.
Enable communication and feedback mechanisms across phases
- Plan for follow-up communication with participants after the end of an engagement activity to share learning and offer further opportunities to provide feedback.
- Create mechanisms for participants to get regular updates on the project or changes to the product/service.
- Allow participants to opt in (rather than be forced to opt out) of post-engagement communications from the organization.
- Proactively communicate and provide evidence of how participant input influenced the development and deployment of the final product/feature.
Is this guidance missing something important? Suggest an update
Explore additional resources
PAI’s Guidance for Inclusive AI
Market Study for Customized Pretrained Models
This custom Guidance for Inclusive AI, for use by commercial sector AI developers and deployers, focuses on an initial set of recommendations for creating public engagement strategies. Additional guidance and cautions are available here. This should be treated as a brainstorming tool and not as a definitive checklist guide.
In general, market studies can be challenging because the scope of the engagement might be loosely defined, and it may be unclear how participant insights will be used (or not used) for product development. It might be easier to specify participants for customized, pretrained models, because the models might be built for users who are already well-researched or easily accessible. Some issues identified might require the foundation model developer to address.
Understand Incentives & Risks
Learn more about understanding the incentives & risks for engagement
Common Challenges
- Incentives for participatory public engagement are unclear.
- Organizational leadership is unfamiliar with participatory public engagement.
- Participatory public engagement is not recognized as an organizational priority.
Recommended Practices
Align organizationally on purpose and capacity
- Host meeting(s) with organizational and team leaders and key staff to establish a shared understanding of the value and importance of public engagement.
- Specify concrete key-performance indicators or other organizational metrics, or recognize and award employees implementing public engagement strategies.
Conduct internal training
- Host training sessions for the engagement team (and the broader organization) on such topics as:
- purpose and value of public engagement
- considerations and sensitivities when working with a specific socially marginalized community
- specific skills development for engagement facilitation and moderation
- data analysis skills, such as social scientific methods for qualitative data analysis
- Consult and hire external experts and intermediaries to lead training sessions specific to the needs of your team and the participants you will engage.
Identify, address, and communicate biases and exclusions
- Host internal meeting(s) to discuss specific team biases.
- Conduct pre-engagement research to better understand audiences that are frequently not engaged.
Identify Key Actors
Learn more about identifying key actors
Common Challenges
- It is unclear who to target beyond existing users or customers, especially those who impacted downstream.
- It is difficult to recruit participants in new or unfamiliar markets.
- The same set of participants are recruited.
Recommended Practices
Conduct research on communities and known issues prior to engagement
- Consult intermediaries and other experts to develop baseline knowledge of the social impact issues affecting socially marginalized communities.
- Map known experts’ and intermediaries’ relevant expertise, and collaborate with them to identify gaps in your team’s knowledge or networks.
- Prioritize existing literature and research led by members of the communities, including work and people that are critical of your organization or the technology.
Design from the margins
- Explicitly prioritize people who might be outlier users facing greater likelihood of direct harm.
- Include recruitment of people who are likely to be impacted by the deployment of the product/service but are not direct users.
- Include participants who are explicitly critical of your organization or of AI technology.
Expand participant recruitment networks
- Work with employee resource groups (ERGs) or other internal audiences to identify additional external networks for participant recruitment.
- Reach out to customers or users who do not typically engage with the company to learn ways they could be encouraged to participate.
- Dedicate time to connect with and build relationships with many different civil society or community groups, rather than rely on a couple of strong existing relationships.
Maintain transparent recruitment processes
- Publish participant selection criteria and allow people to provide input on any exclusions that might harm the project.
- Host recruitment sessions wherein interested participants can pose questions to the organizers.
Collaborate with intermediaries
- Consult with intermediaries to identify and address team and organizational biases and limitations.
- Solicit external input on designing the participatory public engagement strategy, recruitment and activities.
- Work with intermediaries or community leaders who have direct experience building relationships with the specific socially marginalized community(ies) you seek to work with.
Determine Organizational Capacity
Learn more about determining organizational capacity
Common Challenges
- Organizational leadership is unfamiliar with participatory public engagement.
- It is unclear how to incorporate participant feedback.
- Engagement may cause friction between teams within an organization.
Recommended Practices
Dedicate time for research and development
- Conduct existing literature and research review to establish baseline understanding of social impacts and harms of the technology.
- Conduct existing literature and research review to establish baseline understanding of historic and current issues affecting socially marginalized communities.
- Research past public engagements hosted by your organization, including the impact those engagements had on different participant groups.
Establish internal processes and criteria for decision-making
- Internally discuss and set parameters on what participants can and cannot influence or change.
- Confirm that the engagement team has clear direction from organizational leaders on what can and cannot be shared with participants (or has a process for getting this guidance).
- Coordinate with product development engineers and others in the organization to ensure participant input supports better development.
Consult with internal experts or audiences and intermediaries
- Seek out and work with any public engagement or community experts on staff, even those outside the specific product team.
- Seek out and work with other internal audiences to revise, validate, and/or pilot engagement strategies.
- Work with intermediaries or community leaders to translate insights into actionable changes for the product.
Budget for the full public engagement strategy
- Allocate funds to support staff time for all stages of the engagement, including pre-engagement research and post-engagement follow-up, as well as time spent on administrative and logistics tasks.
- Allocate funds for expenses related to participant support, including:
- participant compensation
- participant access costs like travel vouchers, internet data credit, etc.
- support activities such as translation or live captioning services, on-site childcare, etc.
- support services such as mental health counselors, legal support
Define the Relationship
Learn more about defining the relationship
Common Challenges
- Participants are concerned they might not see their feedback reflected in the process or final product.
- Participants have concerns about potential exploitation or “participant washing.”
- Participants are worried about providing critical or negative feedback.
- Organizational concerns about public criticism or negative PR.
Recommended Practices
Establish participant communication expectations and processes
- Share details about how you plan to communicate with participants during and after the engagement.
- Communicate organizational decision-making processes and limitations to participants as part of the participant onboarding process.
- Establish communication mechanisms for participants to reach out to you after the engagement.
- Maintain multiple points of contact for participants to reach out to you during and after the engagement.
Address specific participant needs
- Dedicate time to learn about and develop strategies to address differences in cultural and communication style between the engagement team and the participants.
- Ask for input on what needs to change to invite participants to be more engaged and share honest feedback.
- Adapt data collection or input solicitation methods to make them culturally appropriate and accessible to participants, such as storytelling methods, oral histories, and use of drawings or physical models.
Create opportunities for “out of scope” discussions
- Encourage participants to provide feedback beyond the scoped topic to identify any major failures or risks they project.
- Incorporate time for participants to ask more general questions about the AI development process or the impact of the technology.
Practice active listening
- Repeat what you’ve learned from participants and outline how their insights will be used.
- Share any publications with participants prior to release and confirm their consent to having their insights or participation referenced or used.
Limit how participation and participant feedback is used
- Restrict how other teams in the organization can use the data collected from participants, unless participants are fully aware of and consent to the use of their insights or data for these additional uses.
- Provide participants with highly specified release forms (dictating the use of their participation details) rather than general release forms that permit the organization to freely use any information collected during the engagement.
- Limit how participation and participant feedback is used
- Restrict how other teams in the organization can use the data collected from participants, unless participants are fully aware of and consent to the use of their insights or data for these additional uses.
- Provide participants with highly specified release forms (dictating the use of their participation details) rather than general release forms that permit the organization to freely use any information collected during the engagement.
Assess the Overall Process
Learn more about assessing the overall process
Common Challenges
- Different sets of participants provide opposing feedback.
- Participant feedback feels highly specific to certain individuals or communities.
Recommended Practices
Limit generalizations of learning
- Recognize the limits of public engagement, and avoid generalizing the experiences from individuals to broad populations without validating them against other studies or engagements.
- Continuously refine and specify distinctions within socially marginalized groups to better understand how members of a given social identity group may have different experiences or needs.
Optimize for comprehensive feedback from participants
- Balance inquiries with limited-choice options or highly specified questions, such as surveys, with more open-ended discussions.
Conduct a participant impact and risk assessment
- Work with internal and external experts and intermediaries to understand potential harms and risks to participants that can arise from working with your organization.
- Address potential confirmation bias in your engagement (e.g., leading questions, prompts that force agreement by the participant) by designing engagement activities, including the questions asked, to generate more open-ended discussions with participants.
Is this guidance missing something important? Suggest an update
Explore additional resources
PAI’s Guidance for Inclusive AI
User Data Collection & Enrichment for Customized Pretrained Models
This custom Guidance for Inclusive AI, for use by commercial sector AI developers and deployers, focuses on an initial set of recommendations for creating public engagement strategies. Additional guidance and cautions are available here. This should be treated as a brainstorming tool and not as a definitive checklist guide.
In general, data collection and enrichment can be challenging because good datasets are a key input for AI models, and possession of these datasets is a financial asset for companies. These financial gains are not often shared with the people who contribute, or annotate, the data unless structured into properly paid jobs. It might be easier to specify participants for customized, pretrained models, because the models might be built for users who are already well-researched or easily accessible. Some issues identified might require the foundation model developer to address.
Understand Incentives & Risks
Learn more about understanding the incentives & risks for engagement
Common Challenges
- Organizational leadership is unfamiliar with participatory public engagement.
- Participant incentives for participatory public engagement are unclear.
- Engagement may slow down product launches.
Recommended Practices
Conduct internal training
- Host training sessions for the engagement team (and the broader organization) on such topics as:
- purpose and value of participatory public engagement
- existing gaps in available datasets that might result in poorer-quality AI products
Align organizationally on purpose and capacity
- Host meeting(s) with organizational and team leaders and key staff to establish a shared understanding of the value and importance of participatory public engagement.
- Specify concrete key-performance indicators or other organizational metrics, or recognize and award employees implementing public engagement strategies.
Consult intermediaries and participants to validate the engagement design
- Work with intermediaries and participants to identify any additional needs participants may have that could impact their ability to participate.
Establish internal processes and criteria for decision-making
- Coordinate with product development engineers and others in the organization to ensure participant input supports better development.
- Determine the gaps in existing datasets to optimize data collection from participants.
Identify Key Actors
Learn more about identifying key actors
Common Challenges
- Expanding the participant recruitment network is difficult.
Recommended Practices
Conduct research on communities and known issues prior to engagement
- Conduct social, scientific, and sociotechnical research to understand current and historical issues impacting socially marginalized communities.
- Map known experts’ and intermediaries’ relevant expertise, and collaborate with them to identify gaps in your team’s knowledge or networks.
- Prioritize existing literature and research led by members of the communities, including work and people that are critical of your organization or the technology.
Design from the margins
- Explicitly prioritize people who might be outlier users facing greater likelihood of direct harm.
- Include recruitment of people who are likely to be impacted by the deployment of the product/service but are not direct users.
- Include participants who are explicitly critical of your organization or of AI technology.
Expand participant recruitment networks
- Work with employee resource groups (ERGs) or other internal audiences to identify additional external networks for participant recruitment.
- Reach out to customers or users who do not typically engage with the company to learn ways they could be encouraged to participate.
- Dedicate time to connect with and build relationships with many different civil society or community groups, rather than rely on a couple of strong existing relationships.
Determine Organizational Capacity
Learn more about determining organizational capacity
Common Challenges
- Participants lack resources necessary to join engagement activity.
Recommended Practices
Prioritize participant well-being
- Negotiate internal policies and processes such as nondisclosure agreement requirements to better meet participant needs.
- Identify areas in which the organization can share decision-making or benefits to participants, such as establishing data dividends for contributing to the dataset.
- Reduce any administrative barriers for participants by simplifying onboarding or compensation processes.
- Request feedback from participants on the design of the engagement activity to align with participants’ working and communication styles.
- Provide post-engagement support to address any emotional, physical, or psychological harms experienced by participants due to the engagement.
Determine specific participant needs
- Dedicate time to learn about and develop strategies to address differences in cultural and communication style between the engagement team and the participants.
- Discuss timelines and deadlines with participants.
- Ask for input on what needs to change to invite participants to be more engaged and share honest feedback.
Determine participant needs in high-stakes situations
- Dedicate time to learn about and develop strategies to address specific high-stakes situations participants might face, such as criminalization or imprisonment.
- Work with intermediaries to initially understand specific ways some groups might experience greater scrutiny or harm.
- Align internal resources, such as mental health professionals or access to legal support, for participants.
- Increase participants’ data privacy and security.
Budget for the full participatory public engagement strategy
- Allocate funds to support staff time for all stages of the engagement, including pre-engagement research and post-engagement follow-up, as well as time spent on administrative and logistics tasks.
Define the Relationship
Learn more about defining the relationship
Common Challenges
- Participants assume they lack familiarity with the topic to engage.
- The sponsoring organization has a poor reputation among the participant community.
Recommended Practices
Establish a participant onboarding process
- Communicate the specific role, opportunities for impact, and limitations on what participants can influence before starting the engagement process.
- Clearly outline expectations of time required for participation.
- Allow participants to “off-board” easily if they disagree with the proposed engagement process.
- Host pre-engagement sessions to help participants get online or become familiar with the engagement platform.
- Establish clear work processes and expectations for participants contributing or annotating data
Address past harms
- Research and directly acknowledge any past harms committed by your organization with participants.
- Research and directly acknowledge any past harms related to the technology being discussed with participants.
Assess the Overall Process
Learn more about assessing the overall process
Common Challenges
- The overall impact of collecting or processing data on participants is unclear.
- Participants have concerns about potential exploitation or “participant washing.”
Recommended Practices
Expand the impact of participants
- Identify topics or issues that can be negotiated to permit greater influence by participants.
- Allow product teams to revisit prior development stages to address issues identified by participants.
- Encourage participants to identify issues beyond the scoped topic for the engagement.
Compensate and credit participants
- Prior to, and throughout the engagement, host discussions explicitly about forms of compensation (financial, public acknowledgement, in-kind donations, etc.) that participants want.
- Provide financial compensation for participation.
- Provide financial support for additional costs related to participating, including travel vouchers, on-site childcare, meals.
Conduct a participant impact and risk assessment
- Work with internal and external experts and intermediaries to understand potential harms and risks to participants that can arise from working with your organization.
- Assess the engagement strategy and specific activities to address any coercive pressure participants may experience to discourage criticism of your organization or the product.
Limit how participation and participant feedback is used
- Restrict how other teams in the organization can use the data collected from participants, unless participants are fully aware of and consent to the use of their insights or data for these additional uses.
- Provide participants with highly specified release forms (dictating the use of their participation details) rather than general release forms that permit the organization to freely use any information collected during the engagement.
Is this guidance missing something important? Suggest an update
Explore additional resources
PAI’s Guidance for Inclusive AI
Risk & Failure Assessments for Customized Pretrained Models
This custom Guidance for Inclusive AI, for use by commercial sector AI developers and deployers, focuses on an initial set of recommendations for creating public engagement strategies. Additional guidance and cautions are available here. This should be treated as a brainstorming tool and not as a definitive checklist guide.
In general, risk and failure assessment can be challenging because participants need access to early versions of the AI models to test them, leaving sponsors vulnerable. These financial gains are not often shared with the people who contribute, or annotate, the data unless structured into properly paid jobs. Assessing for risks and failure might also expose participants to harmful content or output. It might be easier to specify participants for customized, pretrained models, because the models might be built for users who are already well-researched or easily accessible. Some issues identified might require the foundation model developer to address.
Understand Incentives & Risks
Learn more about understanding the incentives & risks for engagement
Common Challenges
- Organizational leadership is unfamiliar with participatory public engagement.
- Organizational concerns about public criticism or negative PR.
- Engagement may cause friction between teams within an organization.
Recommended Practices
Conduct research on communities and known issues prior to engagement
- Conduct social, scientific, and sociotechnical research to understand current and historical issues impacting socially marginalized communities.
- Prioritize existing literature and research led by members of the communities, including work and people that are critical of your organization or the technology.
- Research past public engagements hosted by your organization, including the impact those engagements had on different participant groups.
- Consult intermediaries and other experts to develop baseline knowledge of the social impact issues affecting socially marginalized communities.
Align organizationally on purpose and capacity
- Host meeting(s) with organizational and team leaders and key staff to establish a shared understanding of the value and importance of participatory public engagement.
- Specify concrete key-performance indicators or other organizational metrics, or recognize and award employees implementing public engagement strategies.
Establish internal processes and criteria for decision-making
- Internally discuss and set parameters on what participants can and cannot influence or change.
- Confirm that the engagement team has clear direction from organizational leaders on what can and cannot be shared with participants (or has a process for getting this guidance).
Conduct internal training
- Host training sessions for the engagement team (and the broader organization) on such topics as:
- purpose and value of public engagement
- considerations and sensitivities when working with a specific socially marginalized community
- specific skills development for engagement facilitation and moderation
- data analysis skills, such as social scientific methods for qualitative data analysis
- Consult and hire external experts and intermediaries to lead training sessions specific to the needs of your team and the participants you will engage.
Identify, address, and communicate biases and exclusions
- Host internal meeting(s) to discuss specific team biases.
- Conduct pre-engagement research to better understand audiences that are frequently not engaged.
Identify Key Actors
Learn more about identifying key actors
Common Challenges
- The same set of external experts are recruited.
- Participants consider the topic outside their expertise or usual scope of work.
Recommended Practices
Design from the margins
- Explicitly prioritize people who might be outlier users facing greater likelihood of direct harm.
- Include recruitment of people who are likely to be impacted by the deployment of the product/service but are not direct users.
- Include participants who are explicitly critical of your organization or of AI technology.
- Engage with, and explore, conflicting data points or outliers to determine whether they are true outliers or indicators of key edge cases.
Specify the types of expertise needed
- Align the different forms of formal and informal expertise that can help address your team’s participatory public engagement objectives and/or your team’s specific gaps in awareness.
- Conduct additional research on available formal and informal experts for a topic, rather than rely on such common proxies for expertise as elite university educational affiliations.
Expand participant recruitment networks
- Work with employee resource groups (ERGs) or other internal audiences to identify additional external networks for participant recruitment.
- Reach out to customers or users who do not typically engage with the company to learn ways they could be encouraged to participate.
- Dedicate time to connect with and build relationships with many different civil society or community groups, rather than rely on a couple of strong existing relationships.
- Encourage past participants or others in your network to recommend new participants.
Collaborate with intermediaries
- Work with intermediaries or community leaders who have direct experience building relationships with the specific socially marginalized community(ies) you seek to work with.
- Map known experts’ and intermediaries’ relevant expertise, and collaborate with them to identify gaps in your team’s knowledge or networks.
Maintain transparent recruitment processes
- Publish participant selection criteria and allow people to provide input on any exclusions that might harm the project.
- Include information about what participants will do and how participants’ feedback will be used in the recruitment materials.
- Host recruitment sessions wherein interested participants can pose questions to the organizers.
Determine Organizational Capacity
Learn more about determining organizational capacity
Common Challenges
- Participants lack resources necessary to join engagement activity.
- It is unclear how to incorporate participant feedback, particularly non-technical feedback.
Recommended Practices
Align resources to address specific participant needs
- Dedicate time to learn about and develop strategies to address differences in cultural and communication style between the engagement team and the participants.
- Discuss timelines and deadlines with participants.
- Provide participants with multiple methods for providing feedback, such as written comments or voice recordings.
- Prior to the engagement, survey participants to identify additional needs they may have that could impact their ability to participate.
- Adapt data collection or input solicitation methods to make them culturally appropriate and accessible to participants, such as storytelling methods, oral histories, and use of drawings or physical models.
Assess for Accessibility of Engagement Venue or Platform
- Work with experts, intermediaries, and participants to assess the engagement venue (in-person sessions) or platform (virtual sessions) for accessibility, including:
- physical accessibility
- language/content accessibility
- financial accessibility
- technological accessibility
- Provide easy mechanisms for participants to confidentially share their accessibility needs.
- Prioritize scheduling and time zones of participants to better ensure their full engagement and attentiveness during the engagement activity.
Conduct internal training
- Host training sessions for the engagement team (and the broader organization) on such topics as:
- purpose and value of participatory public engagement
- considerations and sensitivities when working with a specific socially marginalized community
- specific skills development for engagement facilitation and moderation
- data analysis skills, such as social scientific methods for qualitative data analysis
Consult with internal experts or audiences and intermediaries
- Seek out and work with any public engagement or community experts on staff, even those outside the specific product team.
- Seek out and work with other internal audiences to revise, validate, and/or pilot engagement strategies.
- Work with intermediaries or community leaders to translate insights into actionable changes for the product.
Consult intermediaries and participants to validate the engagement design
- Work with intermediaries and participants to identify any additional needs participants may have that could impact their ability to participate.
- Brainstorm and define risk and harm categories with experts, intermediaries, and participants to revise engagement design.
Define the Relationship
Learn more about defining the relationship
Common Challenges
- Participants are worried about providing critical or negative feedback.
- Participants have concerns about potential exploitation or “participant washing.”
Recommended Practices
Create opportunities for “out of scope” discussions
- Encourage participants to provide feedback beyond the scoped topic to identify any major failures or risks they project.
- Incorporate time for participants to ask more general questions about the AI development process or the impact of the technology.
Conduct a participant impact and risk assessment
- Work with internal and external experts and intermediaries to understand potential harms and risks to participants that can arise from working with your organization.
- Assess the engagement strategy and specific activities to address any coercive pressure participants may experience to discourage criticism of your organization or the product.
Prioritize participant well-being
- Negotiate internal policies and processes such as nondisclosure agreement requirements to better meet participant needs.
- Work with intermediaries if participants need greater protection or separation from the organization sponsoring the engagement.
- Provide post-engagement support to address any emotional, physical, or psychological harms experienced by participants due to the engagement.
- Provide explicit mechanisms for participants to opt-out of the engagement at any time.
Address participant needs in high-stakes situations
- Dedicate time to learn about and develop strategies to address specific high-stakes situations participants might face, such as criminalization or imprisonment.
- Work with intermediaries to initially understand specific ways some groups might experience greater scrutiny or harm.
- Adopt whistleblower or other legal protections for participants to protect their ability to provide honest and critical feedback.
- Provide advocates or other support personnel for participants to contact with questions or concerns about their participation, particularly if they experience harm.
- Align internal resources, such as mental health professionals or access to legal support, for participants.
- Increase participants’ data privacy and security.
Compensate and credit participants
- Prior to, and throughout the engagement, host discussions explicitly about forms of compensation (financial, public acknowledgement, in-kind donations, etc.) that participants want.
- Provide financial compensation for participation.
- Provide financial support for additional costs related to participating, including travel vouchers, on-site childcare, meals.
Welcome opt outs
- Create clear pathways for participants to withdraw or opt out during and after engagement.
Assess the Overall Process
Learn more about assessing the overall process
Common Challenges
- Different sets of participants provide opposing feedback.
- It is unclear how to incorporate participant feedback.
- Some issues might be due to the pretrained model, outside the purview of the sponsoring organization.
Recommended Practices
Expand the impact of participants
- Identify topics or issues that can be negotiated to permit greater influence by participants.
- Allow product teams to revisit prior development stages to address issues identified by participants.
- Encourage participants to identify issues beyond the scoped topic for the engagement.
Conduct internal training
- Host training sessions for the engagement team (and the broader organization) on such topics as:
- purpose and value of participatory public engagement
- considerations and sensitivities when working with a specific socially marginalized community
- specific skills development for engagement facilitation and moderation
- data analysis skills, such as social scientific methods for qualitative data analysis
Collaborate with intermediaries
- Work with intermediaries or community leaders who have direct experience building relationships with the specific socially marginalized community(ies) you seek to work with.
- Work with intermediaries or community leaders to translate insights into actionable changes for the product.
Establish internal processes and criteria for decision-making
- Internally discuss and set parameters on what participants can and cannot influence or change.
- Confirm that the engagement team has clear direction from organizational leaders on what can and cannot be shared with participants (or has a process for getting this guidance).
- Communicate organizational processes and limitations to participants as part of the participant onboarding process.
Coordinate efforts to influence pretrained model developers
- Work with other developers customizing from the same pretrained model to address issues with the underlying model
Is this guidance missing something important? Suggest an update
Explore additional resources
PAI’s Guidance for Inclusive AI
User Experience & Testing for Customized Pretrained Models
This custom Guidance for Inclusive AI, for use by commercial sector AI developers and deployers, focuses on an initial set of recommendations for creating public engagement strategies. Additional guidance and cautions are available here. This should be treated as a brainstorming tool and not as a definitive checklist guide.
In general, user experience and testing is a well-established stage of model development, increasing the likelihood that organizational leaders will readily support this public engagement activity. It might be easier to specify participants for customized, pretrained models, because the models might be built for users who are already well-researched or easily accessible. Some issues identified might require the foundation model developer to address.
Understand Incentives & Risks
Learn more about understanding the incentives & risks for engagement
Common Challenges
- The sponsoring organization has a poor reputation among the participant community.
- Engagement may slow down product launches.
- There are concerns that participants will leak IP or publicly criticize the project.
Recommended Practices
Dedicate time for research and development
- Conduct existing literature and research review to establish baseline understanding of social impacts and harms of the technology.
- Conduct existing literature and research review to establish baseline understanding of historic and current issues affecting socially marginalized communities.
- Prioritize existing literature and research led by members of the communities, including work and people that are critical of your organization or the technology.
- Research past public engagements hosted by your organization, including the impact those engagements had on different participant groups.
- Consult intermediaries and other experts to develop baseline knowledge of the social impact issues affecting socially marginalized communities.
Consult with internal experts or audiences and intermediaries
- Inventory internal expertise and skills available across organizational teams.
- Seek out and work with any public engagement or community experts on staff, even those outside the specific product team.
Align organizationally on purpose and capacity
- Host meeting(s) with organizational and team leaders and key staff to establish a shared understanding of the value and importance of participatory public engagement.
- Specify concrete key-performance indicators or other organizational metrics, or recognize and award employees implementing public engagement strategies.
Establish internal processes and criteria for decision-making
- Internally discuss and set parameters on what participants can and cannot influence or change.
- Confirm that the engagement team has clear direction from organizational leaders on what can and cannot be shared with participants (or has a process for getting this guidance).
- Communicate organizational processes and limitations to participants as part of the participant onboarding process.
Identify Key Actors
Learn more about identifying key actors
Common Challenges
- It is unclear who to target beyond existing users or customers, especially those who impacted downstream.
- It is difficult to recruit participants in new or unfamiliar markets.
- The same set of participants are recruited.
Recommended Practices
Collaborate with intermediaries
- Work with intermediaries or community leaders who have direct experience building relationships with the specific socially marginalized community(ies) you seek to work with.
- Map known experts’ and intermediaries’ relevant expertise, and collaborate with them to identify gaps in your team’s knowledge or networks.
Design from the margins
- Explicitly prioritize people who might be outlier users facing greater likelihood of direct harm.
- Include recruitment of people who are likely to be impacted by the deployment of the product/service but are not direct users.
- Include participants who are explicitly critical of your organization or of AI technology.
- Engage with, and explore, conflicting data points or outliers to determine whether they are true outliers or indicators of key edge cases.
Expand participant recruitment networks
- Work with employee resource groups (ERGs) or other internal audiences to identify additional external networks for participant recruitment.
- Reach out to customers or users who do not typically engage with the company to learn ways they could be encouraged to participate.
- Dedicate time to connect with and build relationships with many different civil society or community groups, rather than rely on a couple of strong existing relationships.
- Encourage past participants or others in your network to recommend new participants.
Determine Organizational Capacity
Learn more about determining organizational capacity
Common Challenges
- Engagement may cause friction between teams within an organization.
- Some issues might be due to the pretrained model, outside the purview of the sponsoring organization.
Recommended Practices
Conduct internal training
- Host training sessions for the engagement team (and the broader organization) on such topics as:
- purpose and value of participatory public engagement
- considerations and sensitivities when working with a specific socially marginalized community
- specific skills development for engagement facilitation and moderation
- data analysis skills, such as social scientific methods for qualitative data analysis
- Host retrospectives to discuss and learn from previous public engagement activities undertaken by your organization.
Coordinate application of participant insights to product development
- Coordinate with product development engineers and others in the organization to ensure participant input supports better development.
Budget for the full participatory public engagement strategy
- Allocate funds to support staff time for all stages of the engagement, including pre-engagement research and post-engagement follow-up, as well as time spent on administrative and logistics tasks.
- Allocate funds for expenses related to participant support, including:
- participant compensation
- participant access costs like travel vouchers, internet data credit, etc.
- support activities such as translation or live captioning services, on-site childcare, etc.
- support services such as mental health counselors, legal support
Coordinate efforts to influence pretrained model developers
- Work with other developers customizing from the same pretrained model to address issues with the underlying model
Define the Relationship
Learn more about defining the relationship
Common Challenges
- Participants have concerns about potential exploitation or “participant washing.”
- Participants consider the topic outside their expertise or usual scope of work.
- Participants are worried about providing critical or negative feedback.
Recommended Practices
Expand the impact of participants
- Identify topics or issues that can be negotiated to permit greater influence by participants.
- Allow product teams to revisit prior development stages to address issues identified by participants.
- Encourage participants to identify issues beyond the scoped topic for the engagement.
Compensate and credit participants
- Prior to, and throughout the engagement, host discussions explicitly about forms of compensation (financial, public acknowledgement, in-kind donations, etc.) that participants want.
- Provide financial compensation for participation.
- Provide financial support for additional costs related to participating, including travel vouchers, on-site childcare, meals.
Prioritize participant well-being, including in high-stakes situations
- Negotiate internal policies and processes such as nondisclosure agreement requirements to better meet participant needs.
- Identify areas in which the organization can share decision-making or benefits to participants, such as establishing data dividends for contributing to the dataset.
- Request feedback from participants on the design of the engagement activity to align with participants’ working and communication styles.
- Work with intermediaries to initially understand specific ways some groups might experience greater scrutiny or harm.
- Adopt whistleblower or other legal protections for participants to protect their ability to provide honest and critical feedback.
Conduct a participant impact and risk assessment
- Work with internal and external experts and intermediaries to understand potential harms and risks to participants that can arise from working with your organization.
- Assess the engagement strategy and specific activities to address any coercive pressure participants may experience to discourage criticism of your organization or the product.
Limit how participation and participant feedback is used
- Restrict how other teams in the organization can use the data collected from participants, unless participants are fully aware of and consent to the use of their insights or data for these additional uses.
- Provide participants with highly specified release forms (dictating the use of their participation details) rather than general release forms that permit the organization to freely use any information collected during the engagement.
- Obtain explicit permission from participants to use their involvement to promote the public image of the organization.
Assess the Overall Process
Learn more about assessing the overall process
Common Challenges
- Participants are concerned they might not see their feedback reflected in the process or final product.
- Participant feedback feels highly specific to certain individuals or communities.
Recommended Practices
Optimize for comprehensive feedback from participants
- Balance inquiries with limited-choice options or highly specified questions, such as surveys, with more open-ended discussions.
Establish participant communication expectations and processes
- Share details about how you plan to communicate with participants during and after the engagement.
- Establish communication mechanisms for participants to reach out to you after the engagement.
- Maintain multiple points of contact for participants to reach out to you during and after the engagement.
Create opportunities for “out of scope” discussions
- Encourage participants to provide feedback beyond the scoped topic to identify any major failures or risks they project.
- Incorporate time for participants to ask more general questions about the AI development process or the impact of the technology.
Practice active listening
- Repeat what you’ve learned from participants and outline how their insights will be used.
- Share any publications with participants prior to release and confirm their consent to having their insights or participation referenced or used.
Limit generalizations of learning
- Recognize the limits of participatory public engagement, and avoid generalizing the experiences from individuals to broad populations without validating them against other studies or engagements.
- Continuously refine and specify distinctions within socially marginalized groups to better understand how members of a given social identity group may have different experiences or needs.
Is this guidance missing something important? Suggest an update
Explore additional resources
PAI’s Guidance for Inclusive AI
Oversight for Customized Pretrained Models
This custom Guidance for Inclusive AI, for use by commercial sector AI developers and deployers, focuses on an initial set of recommendations for creating public engagement strategies. Additional guidance and cautions are available here. This should be treated as a brainstorming tool and not as a definitive checklist guide.
In general, oversight is challenging because the scope of work for participants might be more open-ended and require longer-term engagements working with the sponsors across the full development life cycle. It might be easier to specify participants for customized, pretrained models, because the models might be built for users who are already well-researched or easily accessible. Some issues identified might require the foundation model developer to address.
Understand Incentives & Risks
Learn more about understanding the incentives & risks for engagement
Common Challenges
- Organizational leadership is unfamiliar with participatory public engagement.
- Participatory public engagement is not recognized as an organizational priority.
- Organizational concerns about public criticism or negative PR.
Recommended Practices
Align organizationally on purpose and capacity
- Host meeting(s) with organizational and team leaders and key staff to establish a shared understanding of the value and importance of participatory public engagement.
- Specify concrete key-performance indicators or other organizational metrics, or recognize and award employees implementing public engagement strategies.
Conduct internal training
- Host training sessions for the engagement team (and the broader organization) on such topics as:
- purpose and value of participatory public engagement
- considerations and sensitivities when working with a specific socially marginalized community
- specific skills development for engagement facilitation and moderation
- data analysis skills, such as social scientific methods for qualitative data analysis
Dedicate time for research and development prior to recruitment and engagement
- Conduct existing literature and research review to establish baseline understanding of social impacts and harms of the technology.
- Conduct existing literature and research review to establish baseline understanding of historic and current issues affecting socially marginalized communities.
- Research past public engagements hosted by your organization, including the impact those engagements had on different participant groups.
Collaborate with intermediaries
- Consult with intermediaries to identify and address team and organizational biases and limitations, particularly where the organization might face public scrutiny.
- Work with intermediaries or community leaders who have direct experience building relationships with the specific socially marginalized community(ies) you seek to work with.
Address past harms
- Research and directly acknowledge any past harms committed by your organization with participants.
- Research and directly acknowledge any past harms related to the technology being discussed with participants.
Identify Key Actors
Learn more about identifying key actors
Common Challenges
- The same set of external experts are recruited.
- It is difficult to determine participants’ (including experts’ and intermediaries’) credibility.
- Participants might have limited time to commit to an extended engagement.
Recommended Practices
Design from the margins
- Explicitly prioritize people who might be outlier users facing greater likelihood of direct harm.
- Include recruitment of people who are likely to be impacted by the deployment of the product/service but are not direct users.
- Include participants who are explicitly critical of your organization or of AI technology.
- Engage with, and explore, conflicting data points or outliers to determine whether they are true outliers or indicators of key edge cases.
- Map known experts’ and intermediaries’ relevant expertise, and collaborate with them to identify gaps in your team’s knowledge or networks.
Expand participant recruitment networks
- Work with employee resource groups (ERGs) or other internal audiences to identify additional external networks for participant recruitment.
- Reach out to customers or users who do not typically engage with the company to learn ways they could be encouraged to participate.
- Dedicate time to connect with and build relationships with many different civil society or community groups to validate expertise and credibility.
- Encourage past participants or others in your network to recommend new participants.
Specify the types of expertise needed
- Align the different forms of formal and informal expertise that can help address your team’s participatory public engagement objectives and/or your team’s specific gaps in awareness.
- Conduct additional research on available formal and informal experts for a topic, rather than rely on such common proxies for expertise as elite university educational affiliations.
- Prior to the engagement, survey participants to identify any additional needs they may have that can impact their ability to participate.
Collaborate with intermediaries
- Consult with intermediaries to identify and address team and organizational biases and limitations.
- Work with intermediaries or community leaders who have direct experience building relationships with the specific socially marginalized community(ies) you seek to work with.
Establish a participant onboarding process
- Communicate the specific role, opportunities for impact, and limitations on what participants can influence before starting the engagement process.
- Clearly outline expectations of time required for participation.
- Allow participants to “off-board” easily if they disagree with the proposed engagement process.
- Create an easy process for withdrawing from participation.
- Host pre-engagement sessions to help participants get online or become familiar with the engagement platform.
Maintain transparent recruitment processes
- Publish participant selection criteria and allow people to provide input on any exclusions that might harm the project.
- Include information about what participants will do and how participants’ feedback will be used in the recruitment materials.
- Clearly outline expectations of time required for participation.
Determine Organizational Capacity
Learn more about determining organizational capacity
Common Challenges
- Engagement projects pull product teams away from core duties.
- Participants lack resources necessary to join engagement activity.
- Engagement may slow down product launches.
Recommended Practices
Budget for the full participatory public engagement strategy
- Allocate funds to support staff time for all stages of the engagement, including pre-engagement research and post-engagement follow-up, as well as time spent on administrative and logistics tasks.
- Allocate funds for expenses related to participant support, including:
- participant compensation
- participant access costs like travel vouchers, internet data credit, etc.
- support activities such as translation or live captioning services, on-site childcare, etc.
- support services such as mental health counselors, legal support
Consult with internal experts or audiences and intermediaries
- Inventory internal expertise and skills available across organizational teams.
- Seek out and work with any public engagement or community experts on staff, even those outside the specific product team.
Establish internal processes and criteria for decision-making
- Internally discuss and set parameters on what participants can and cannot influence or change.
- Confirm that the engagement team has clear direction from organizational leaders on what can and cannot be shared with participants (or has a process for getting this guidance).
- Coordinate with product development engineers and others in the organization to ensure participant input supports better development.
Define the Relationship
Learn more about defining the relationship
Common Challenges
- Participants are worried about providing critical or negative feedback.
- Participants have concerns about potential exploitation or “participant washing.”
- Participants are concerned they might not see their feedback reflected in the process or final product.
Recommended Practices
Conduct a participant impact and risk assessment
- Work with internal and external experts and intermediaries to understand potential harms and risks to participants that can arise from working with your organization.
- Assess the engagement strategy and specific activities to address any coercive pressure participants may experience to discourage criticism of your organization or the product.
Expand the impact of participants
- Identify topics or issues that can be negotiated to permit greater influence by participants.
- Allow product teams to revisit prior development stages to address issues identified by participants.
- Encourage participants to identify issues beyond the scoped topic for the engagement.
Create opportunities for “out of scope” discussions
- Encourage participants to provide feedback beyond the scoped topic to identify any major failures or risks they project.
- Incorporate time for participants to ask more general questions about the AI development process or the impact of the technology.
Prioritize participant well-being
- Prior to the engagement, survey participants to identify additional needs they may have that could impact their ability to participate
- Dedicate time to learn about and develop strategies to address specific high-stakes situations participants might face, such as criminalization or imprisonment.
- Work with intermediaries to initially understand specific ways some groups might experience greater scrutiny or harm.
- Adopt whistleblower or other legal protections for participants to protect their ability to provide honest and critical feedback.
- Provide advocates or other support personnel for participants to contact with questions or concerns about their participation, particularly if they experience harm.
Compensate and credit participants
- Prior to, and throughout the engagement, host discussions explicitly about forms of compensation (financial, public acknowledgement, in-kind donations, etc.) that participants want.
- Provide financial compensation for participation.
- Provide financial support for additional costs related to participating, including travel vouchers, on-site childcare, meals.
Welcome opt outs
- Create clear pathways for participants to withdraw or opt out during and after engagement.
Limit how participation and participant feedback is used
- Restrict how other teams in the organization can use the data collected from participants, unless participants are fully aware of and consent to the use of their insights or data for these additional uses.
- Provide participants with highly specified release forms (dictating the use of their participation details) rather than general release forms that permit the organization to freely use any information collected during the engagement.
- Obtain explicit permission from participants to use their involvement to promote the public image of the organization.
Assess the Overall Process
Learn more about assessing the overall process
Common Challenges
- Participants consider the engagement work outside their expertise or usual scope of work.
- Participants lack resources necessary to join engagement activity.
- Different sets of participants provide opposing feedback.
- Participants and sponsoring teams might change in composition over the course of the entire engagement.
- Some issues might be due to the pretrained model, outside the purview of the sponsoring organization.
Recommended Practices
Address specific participant needs
- Dedicate time to learn about and develop strategies to address differences in cultural and communication style between the engagement team and the participants.
- Provide participants with multiple methods for providing feedback, such as written comments or voice recordings.
- Prior to the engagement, survey participants to identify additional needs they may have that could impact their ability to participate.
- Adapt data collection or input solicitation methods to be culturally appropriate and accessible to participants, such as story-telling methods, oral histories, use of drawings or physical models.
Establish a participant onboarding process
- Communicate the specific role, opportunities for impact, and limitations on what participants can influence before starting the engagement process.
- Clearly outline expectations of time required for participation.
- Allow participants to “off-board” easily if they disagree with the proposed engagement process.
- Create an easy process for withdrawing from participation.
- Host pre-engagement sessions to help participants get online or become familiar with the engagement platform.
Optimize for comprehensive feedback from participants
- Balance inquiries with limited-choice options or highly specified questions, such as surveys, with more open-ended discussions.
- Host engagement activities across the AI development life cycle that address different issues and concerns.
- Prioritize the schedules of participants to enable their most active and alert engagement.
Maintain thorough and secure documentation
- Explicitly assign team members to manage notes, documentation, and other materials from public engagement to ensure everything is organized and archived.
- Summarize and share the learning after engagement activities with participants.
- Securely store raw data and other engagement materials with identifying information or other personal details for the security of participants.
Enable communication and feedback mechanisms across phases
- Plan for follow-up communication with participants after the end of an engagement activity to share learning and offer further opportunities to provide feedback.
- Create mechanisms for participants to get regular updates on the project or changes to the product/service.
- Allow participants to opt in (rather than be forced to opt out) of post-engagement communications from the organization.
- Proactively communicate and provide evidence of how participant input influenced the development and deployment of the final product/feature.
Coordinate efforts to influence pretrained model developers
- Work with other developers customizing from the same pretrained model to address issues with the underlying model.
Is this guidance missing something important? Suggest an update
Explore additional resources
PAI’s Guidance for Inclusive AI