Current Study and Methodology
Current Study and Methodology
For further information on qualitative methods in social science research, see:
The purpose of the current study was to use qualitative research to investigate why folks from minoritized backgrounds leave AI teams or organizations, the role of organizational culture or climate in this, and what can be done to stem attrition and make AI teams more inclusive for diverse individuals. It is important to note that recruitment took place primarily in the fall of 2020 and the spring of 2021 during the COVID-19 pandemic.
The research team created a recruitment document that we then distributed to those in the Partnership on AI’s (PAI) professional network through our partners and other individual collaborators. This recruitment document contained information about the study such as its purpose, format, compensation ($75) and brief information about data privacy and protections for participants Robinson, O. C. (2014). Sampling in interview-based qualitative research: A theoretical and practical guide. Qualitative research in psychology, 11(1), 25-41.Yancey, A. K., Ortega, A. N., & Kumanyika, S. K. (2006). Effective recruitment and retention of minority research participants. Annu. Rev. Public Health, 27, 1-28.. The full recruitment document can be viewed in Appendix 1. We chose this targeted sampling method because of the specialized nature of the participants.
We defined minoritized identities to be members from the non-dominant group in their country or within a global context. These minoritized
identities were along the lines of race, ethnicity, gender, sexual orientation, and ability.
The participants in this study fell into 3 broad categories:
- People who worked on AI teams and who identified with one or more minoritized identities.
- People who managed AI teams, regardless of how they identified.
- People who worked with DEI in tech organizations.
We defined “AI team” broadly, including both people working in “technical” roles such as engineers and data scientists, but also those working in non-technical roles or on AI teams that were non-technical or interdisciplinary such as AI policy or AI ethics teams. Interested participants filled out a screening form, and we contacted those eligible for the study via email with a link to the consent form and a time to schedule the interview via Zoom video teleconferencing. The full privacy document can be viewed in Appendix 2.
The lead researcher used a semi-structured interview protocol developed through the following process:
- A review of literature pertaining to attrition and DEI in tech and more specific literature focused on AI
- Scoping calls with workers in DEI in tech
- Discussions among the research team at PAI
- A series of 3 pilot interviews
- Further feedback from the research team
The final research protocol (Appendix 3) consisted of 8 questions that focused on 3 research questions, each focusing on one domain:
- Why do diverse folks leave research teams? (Domain: attrition)
- What is the culture like on AI teams and organizations? (Domain: culture)
- What is being done to make these teams and organizations more inclusive? (Domain: efforts for inclusion)
- After signing the consent form, each participant met with the lead researcher via Zoom video teleconferencing.
- The lead researcher greeted each participant and explained the purpose of the interview and the parameters of confidentiality and privacy, as well as reminding them of the voluntary nature of the study.
- The lead researcher then asked for verbal consent and permission to record the interview.
- The lead researcher proceeded with the questions in the interview protocol, following up on some answers to questions for clarification, asking for additional information or examples, or additional questions based on the responses given.
- At the end of the interview, the lead researcher thanked the participant and discussed follow-up procedures after the interview. Most interviews lasted between 45 minutes and an hour.
- After the interview, the lead researcher converted the saved audio file to a transcript and then redacted private or identifiable details from the transcript.
- Along with payment information, the lead researcher sent this redacted transcript to the participant via encrypted email and asked if they would like to redact or clarify additional information.
The researchers used a variation of the consensual qualitative research (CQR) analysis Hill, C. E., Knox, S., Thompson, B. J., Williams, E. N., Hess, S. A., & Ladany, N. (2005). Consensual qualitative research: An update. Journal of counseling psychology, 52(2), 196. procedure to analyze the interview transcripts. The lead researcher read through each transcript to redact private, identifying, or sensitive information and to gain basic familiarity with the data. The analysis consisted of the following steps:
- The lead researcher articulated the 3 research questions to 3 other researchers.
- The lead researcher articulated the 3 domains under research questions.
- The lead researcher scanned each participant’s transcript and extracted quotes corresponding to each domain.
- The lead researcher and a second researcher both listed core ideas for each participant’s quotes, summarizing this in one paragraph.
- The third and fourth researchers scanned the summarized core ideas and alerted the lead researcher as to any inconsistencies, while suggesting ways for each core idea to be reconciled.
- The lead and second researcher grouped these core ideas into broad categories and more specific themes.
- The lead and second researcher came to a consensus as to themes with which they disagreed.
- The lead researcher created the final grouping of themes.