Appendices

Appendix 1: Recruitment Document

Diversity Equity and Inclusion in AI

Interview Study

Overview

You are invited to take part in a study looking at the experiences of diverse professionals working in the AI space. The purpose of the study is to gain insight into the various factors that contribute to an inclusive climate in tech more broadly, and AI teams more specifically. This work is crucial to understanding more about attrition in the field of AI.

Researchers

The study is being conducted by Jeff Brown, Diversity and Inclusion Research Fellow at the Partnership on AI (PAI), and supervised by Christine Custis, Head of Fairness, Accountability, Transparency and Safety at PAI. Equity and social justice are at the core of PAI’s mission, which uses a multistakeholder model to advance these efforts as best practices in the AI field.

How Will the Results of the Study Be Used?

The data will be analyzed for themes present, and used to write a research paper, executive summary, and will inform products which aim to increase the inclusivity of teams working in AI. The recommendations made from this study will be disseminated to PAI’s partner organizations, who represent a wide array of tech companies and civil society organizations. You can find out more about PAI’s mission and Partner organizations here. The researchers will keep the participant informed of progress in the research study.

Eligibility

If you have worked in a tech organization in any of the below roles either currently or in the past 5 years. The study is looking specifically for people who fit any of the following criteria:

• Worked in Diversity, Equity, and Inclusion (or Diversity & Inclusion)
• Managed teams working on projects involving AI
• Worked on a team working on AI projects, and identify as female, non-binary, LGBTQ+, Black, Indigenous, LatinX, Asian, person of color, person with a disability, and/or belonging to minoritized/marginalized identities.

If you are unsure of your eligibility, feel free to email Jeff at jeff@partnershiponai.org.

Study Procedure

The study consists of a 45-60 minute interview over Zoom. You will be asked several questions about your experiences working in tech and AI teams, including questions about the climate of teams you have worked on and the organization as a whole. Any personally identifying information gathered from the interview will be redacted from the study.

Privacy and Confidentiality

Data gathered from the screening questionnaire or interviews will not be shared outside the team at PAI. The interviews will be recorded and transcribed. Once the transcribed, recorded video files will be deleted. These files will be password protected, stored on the researcher’s PAI issued laptop, and inaccessible to those outside the FTA research team at PAI. Each participant will be sent a copy of their transcript to add to, clarify, or redact any additional information from the interview. Any personally identifiable information, including company names, garnered from interviews will be redacted from transcripts. The final research products will not contain any personally identifiable information, but will rather discuss general themes, and de-identified illustrative examples and quotes. The researcher will contact the participant for any permission to use a quote. Due to the interviewer’s status as a mandated reporter, he has a legal duty to report to the appropriate authorities any plans harm to self, others, or instances of child abuse occurring in the last 3 years.

Compensation

Participants will be compensated $75 for their participation in the interview.

Risks and Benefits

There are no direct risks to participating in this study. However, the interviewer will be asking questions about your experiences working in the tech industry, and may involve topics that are difficult to discuss. Participation is completely voluntary, and you may avoid discussing any topics that you would rather not. There are no direct benefits to participating in this study. However, the aims of the project are to give some insight into the experiences of underrepresented groups in tech and the AI space more specifically.

How to Sign Up

Please fill out this brief questionnaire. You will be contacted by someone at PAI to schedule an interview if you meet the study criteria.

Appendix 2: Privacy Document

Data Privacy Plan
Diversity, Equity, and Inclusion Study

Researchers
Primary: Jeff Brown, Diversity & Inclusion Research Fellow
Supervisor: Christine Custis, Head of AboutML and FTA
Email: jeff@partnershiponai.org
Phone: [REDACTED]

Communication with Researchers

The researchers and support staff at PAI will communicate with participants primarily through email. You may contact either of the researchers at any time.

Signup, Screening Tool, and Consent Form

Participants will sign up for this study via Qualtrics. First, you will fill out a screening tool to ensure your eligibility for this study. If you are eligible to participate, you will be contacted by a staff member at PAI who will send you a link to a consent form and time to sign up for the study. The screening form will ask for your name, demographic information, and an email address. Read more about Qualtrics data privacy practices here. You may enter a name or pseudonym and email address of your choosing. Any files containing your name or contact information will be stored on the researchers’ PAI issued laptops and protected with a password. This information will not be shared with anyone other than the researchers.

Interview

The interview will be conducted on Zoom. Only the participant and researcher(s) will be present. The room will be password protected.

Recording and Transcription

With the consent of the participant, the interview will be recorded so that it can be transcribed using services from temi.com. The recording will be stored on the primary researcher’s PAI issued laptop in a password protected folder. Once transcribed, the recording will be deleted. Any personally identifying information on the transcript will be redacted. The redacted transcripts will not be shared beyond the PAI research team.

Research Products

No personally identifiable information, such as individual or company names, will appear on any product stemming from this research, including but not limited to conference papers, journal articles, press releases, executive summaries, or training materials.

Compensation

Compensation will be set up using Bill.com. You may read more about Bill.com’s privacy practices here.

Appendix 3: Research Protocol

Overview

• Welcome participant and introduce self
• Review purpose of the interview/study
• Review how data will be used and safeguards for privacy
• Encourage participant to follow up if they have any questions

Day before the interview

• Send reminder email to participant with day/time and zoom link

Protocol

After participant signs on:
Greet participants, and say:

Again, I want to thank you for agreeing to participate in this study. My name is Jeff Brown, a Diversity and Inclusion Research fellow at PAI. The purpose of this study is to look at some of the issues impacting diverse populations in the field of AI. I’ll be taking some notes during the interview. Any names or other personally identifying information will be redacted. The transcript won’t be shared beyond the research team. This will be kept private and confidential, however, because I’m a mandated reporter, I am legally and ethically obligated to report any instances of potential harm to self or others, or child abuse within the past 3 years. Additionally I’ll make sure to give you a copy of the transcript of the interview in case you’d like to clarify any additional details. The video file will be deleted after the transcription is made. It should take about 45 minutes but may be shorter or longer depending on the number of follow up questions that I ask.

I also know that we might be speaking about difficult themes or personal stories, and so you can take a break at any time.

Are there any questions before we begin?

Do I have your permission to record? This will be deleted after it is transcribed.

If yes, start recording. If no, do not begin recording.

Do you consent to taking part in this study?

If yes, begin recording. If no, thank the participant for their time and invite them to follow up by email if they would like to take part in this study or a questionnaire.

Okay, great. Today is [Date/time/location]

Then I’ll jump right into the first question:

 

Participant	Interview Questions
DEI Leader	What is your position and where? How long have you been working in this position? What are some of your main responsibilities? From your perspective as [position], what are some of the factors that have led to workers of marginalized identities to leave your organization? Query: do you have any examples of this? Have you worked with people on AI teams specifically? If so, are there any issues that are prevalent in those teams? What has your organization tried to mitigate this attrition? [if they do not know what you mean, query]: How has the leadership in your organization supported or failed to support your efforts? What were the results of your intervention? What are some barriers to implementing efforts to reduce attrition? Is there anything else that you would like to add?
Manager	Tell me about your position. How many employees do you manage? How long? Do you work on projects involving AI? How would you describe your teams in terms of the race/gender/sexual orientation/ or any other known identities? What are some common challenges in managing a team of people? What is your experience with mentorship? Have you received any training specifically in working with or managing people who are of different cultural identities than you? Has the organization made any steps to make the workplace more inclusive? Have you taken any steps to make your team more inclusive? What role has DEI initiatives organization wide played with this? How effective have these DEI initiatives been? Have you been treated differently because of any of your identities? Are there any ways in which you have been made to feel uncomfortable by others at your workplace? By your manager? Other team members? Can you think of a time when this has happened? Has any of your reports discussed this with you? Is there anything else that you would like to add?
Folks	Tell me about your position? How long have you been in this position? How do you identify? You may mention race, sexual orientation, gender, or any other identities you find salient How would you describe the culture or climate on your team? How has the manager influenced this? Has there been anyone in your organization that has helped you to navigate your current role or the organization in general? Are there any ways in which you have been made to feel uncomfortable by others at your workplace/in AI more generally? By your manager? Other team members? Can you think of a time when this has happened? Have you considered leaving your team your team/company/AI? Why? What made you want to stay longer or delay your departure? Have you experienced microaggressions? Other instances of discrimination? Has the wider organization done anything to mitigate these? Is there anything else that you would like to add?

Stop recording.

Okay I have stopped the recording. Thank you again for taking the time to take part in this interview. I will send an email following up with instructions for how to get the compensation for this, as well as an email in the next few days with a transcript of your interview, so you can clarify anything that you would like to. Please let me know via email me if you have any more questions or concerns.

After the interview

Send the participant with an email with instructions on how to obtain the incentive for participating in the study.

Transcribe interview, going over the script to look for any typos etc. Send the transcript to the participant, inviting them to clarify any parts of the transcript or ask additional questions. [create secure system to do this]

Appendix 4: Important Terms

The following section contains a clarification of terms as used in this report.

Minoritized Workers

We define minoritized workers as those employees who belong to a minoritized identity. This report used minoritized, as conceptualized by Yasnim Guanaratnam Gunaratnam, Y. (2003). Researching’race’and ethnicity: Methods, knowledge and power. Sage. instead of the more commonly used “minority” to broadly refer to groups that do not fall under members of the dominant group for a given identity axis. This emphasizes the institutional power dynamics of the dominant or “majority” group regardless of actual proportional representation.

Artificial Intelligence Teams

The study broadly construed AI teams to include work focused on a broad range of techniques under the umbrella of AI, whether it is focused on engineering, research, policy, ethics, or other aspects of AI. These roles may thus focus on specific AI techniques like machine learning or natural language processing, or may focus on broader implications of AI. The study focuses on AI organizations outside of university settings, although these include both for-profit and non-profit organizations.

Race

Race commonly refers to the “[physical characteristics or] differences that groups determine to be significant.” Race and Ethnicity. American Sociological Association. (2022). Retrieved 29 January 2022, archived at https://web.archive.org/web/20190821170406/https://www.asanet.org/topics/race-and-ethnicity This has been commonly conceived as related to phenotypical skin color, and commonly linked with ethnicity. Racial categories are thus defined by different societies in different ways, and thus someone’s racial category may change depending on their broader social context. This report defines racial categories similar to how they are defined in the US census, which is necessarily a limited framework. To begin to account for this, the study asked open-ended questions for participants to self-identify, so that they could define their own racial categories.

Racism

The study treats racism as the systemic and institutionalized, race based form of oppression perpetrated by dominant racial groups to minoritized racial groups. The report acknowledges however that the common definition of racism includes or emphasizes interpersonal racial discrimination regardless of power dynamics. Because racialized institutions and racial dynamics differ between and sometimes even within societies, some racial groups may benefit from or be subject to racism depending on their wider social context. However, racial instiutions such as White supremacy sometimes span different societies due to a more globalized world.

Ethnicity

This report uses the definition of ethnicity as “shared social, cultural, and historical experiences, stemming from common national or regional backgrounds.” University of Minnesota Libraries (2022). 10.2 The Meaning of Race and Ethnicity. Open.lib.umn.edu. Retrieved 29 January 2022, from https://open.lib.umn.edu/sociology/chapter/10-2-the-meaning-of-race-and-ethnicity/. This is sometimes but not always synonymous with race (e.g. Asian-American). This study asked participants to self-identify, and some chose to write their ethnicity, while others did not.

Microaggression

This report uses the definition of microaggression as “brief and commonplace daily verbal, behavioral, or environmental indignities, whether intentional or unintentional, that communicate hostile, derogatory, or negative racial slights and insults.” This has also been broadened to include slights or insults based on other axes of identity.

DEI-Focused Work

This is work that prioritizes principles of diversity, equity, and inclusion within an organization, including but not limited to how socially marginalized groups can access full participation and benefits traditionally privileged to members of the dominant group. For instance, DEI work has focused on reducing gender and race based discrimination, affording equal opportunity to minoritized groups, and increasing the representation of minoritized groups since the status qup privileges the dominant groups.

Ability

The report uses the definition of disabled as stated within the Americans with Disabilities Act. That is “A physical or mental impairment that substantially limits one or more major life activities, a record of such an impairment, or being regarded as having such an impairment.” https://adata.org/glossary-terms#D As with other identity categories, participants were asked to self-identify ability status.