Appendix 2: Research Methods

Appendix 2: Research Methods

The themes and recommendations from this report are grounded in a literature review of related subjects, as well as primary research with workers at each site, virtually fielded from November 2021 until March 2022. All research was conducted in English, unless noted below. Where the literature revealed opportunities to better understand AI’s impact on job quality and worker well-being, we incorporated these areas into guides for diary studies to be completed by participating workers. These diary studies then served as a foundation for a series of semi-structured interviews with diary study participants at each site.

Participants shared their reflections on the diary study questions and prompts through a combination of text, voice, and video answers, as well as image responses to a limited number of questions. To better understand themes emerging from the diary studies, a subset of the diary study participants were each invited to participate in a 60-minute semi-structured interview via Zoom where they were asked a combination of universal questions about emerging themes as well as specific questions about their diary study and interview responses. All research was conducted in English except where noted below. The participants had high degrees of competency in English due to living in countries where English is frequently spoken, using it as the main language in their jobs, or both. For the participants based in India, these interviews were conducted one-on-one in English with the primary researcher. A translator fluent in Hindi was also on the call to offer translation support if requested by participants. US interviewees participated in one-on-one Zoom calls with the primary researcher.

Unlike the other sites, participants in the sub-Saharan Africa site were colleagues. To prevent the possible appearance of choosing favorites by selecting only a subset of participants for interviews, all participants at this site who completed the diary study were invited to participate in interviews. Due to strong interest from the participants who completed the diary study and comparatively limited research team time, these interviews were structured as group interviews with three to four participants per group. Due to last-minute schedule changes from participants, the actual number of interviewees per group ranged from one to four. The research team explored possible upsides and downsides of this approach with leads from the sub-Saharan African group, who offered guidance that this approach should work well; employees were used to participating in focus groups and other similar discussions with each other. This participant site included several individuals with team/project management, mentorship, or coaching responsibilities. Each of these individuals was interviewed one-on-one to ensure that they could speak freely about aspects of their managerial, supervision, or mentorship duties they might prefer not to discuss in front of potential reports. Similarly, this design ensured that workers could speak about experiences they might prefer not to share or discuss with managers or supervisors.

Participant recruitment

For this work, the research team sought to include a wide range of possible workers who frequently use or interact with AI in their workplaces with a focus on workers who were structurally likely to be more vulnerable to any harms from these technologies. We sought to learn from:

• Vulnerable workers in a high income country (e.g., workers with fewer formal education credentials and thus fewer opportunities for higher paying jobs)
• Middle class workers in a LMIC (e.g., workers who were less vulnerable in the context of their own country, but could be substantively affected by global market changes driven by decisions abroad or in high income countries)
• Working class or working poor workers in a LMIC (e.g., workers who could be both individually and collectively affected by the forces described for the prior two sites, and thus at highest risk of harm).

The team considered three possible approaches for recruitment:

Working with an independent research recruiter

Advantages	Disadvantages & Mitigations
Workers could participate independently of their employer	Participants would need their anonymity protected to ensure their employers would not punish their participation Lack of employer permission might make participants feel less comfortable speaking frankly about their experiences

Collaborating with a supportive employer

Advantages	Disadvantages & Mitigations
Workers participate with their employer’s full permission	Participants’ individual and collective ability to participate voluntarily and speak freely would need to be protected through confidentiality and anonymity provisions

Collaborating with a worker organization or union

Advantages

Disadvantages & Mitigations

Workers could participate independently of their employer Workers would likely have higher familiarity with how technologies affect working conditions and more

Employers often scrutinize organizers and active union participants more heavily, increasing their possible risk of participation and the importance of protecting their anonymity High levels of familiarity with technology’s impacts built through organizing work might not be representative of the broader group of workers Worker organizations/unions are rare or non-existent in the LMIC occupations that work closely with AI technologies

Due to the nature of the research, there was no ideal set of sites to conduct this work. Each approach would require: (a) setting up provisions to protect participants’ ability to voluntarily participate and speak freely, and (b) attention by the research team to ensure the representativeness of the participant pools and the perspectives they shared. The team ultimately decided against recruiting through worker organizations and unions, due to the difficulty of this approach in the LMIC site, and concerns about representativeness in the high-income country.

For the participants based in India and the United States, we worked with a professional research recruiter to identify and invite qualified participants for the study. Participants were recruited through advertisements and targeted outreach.

For the participants based in sub-Saharan Africa, we collaborated with a company which was developing a series of ML tools to assist their data annotators in completing their work. This company agreed to identify a group of employees who were experienced in using these tools and to forward them an introductory note from the research team which outlined the research and invited participation. Participation was entirely voluntary — not required or encouraged by the company, as was made clear in the introductory note. If participants were interested, they were asked to sign up and participate on their own time, using their personal phones or devices (rather than during their work hours, using company devices). The note was forwarded to the group’s personal email addresses (not their work addresses) to further underscore the independence of the research and voluntary nature of participation.
All participants at all sites were compensated for their time, with interview participants receiving an additional amount for their additional time. Compensation amounts were set to be generous relative to participants’ normal hourly wages, without being so high as to create undue pressure on participants to join the study (and thus potentially undermine the voluntariness of their participation).

Ethics and informed consent

In line with best practices in qualitative research, each participant in the study was informed of the goals, content and format of the study, the benefits and risks of their participation, and the organization and individuals responsible for the study. They were additionally informed that any stories or quotes shared in public research outputs would be anonymized to protect their identity and mitigate any risks of their participation. All participants digitally signed consent forms confirming that they received and understood the information about the study and that their participation was entirely voluntary and could be withdrawn by them for any reason at any time. They were also reminded of this at the outset of each interview, where they were also informed that they could choose to decline to answer any questions posed to them and choose to end the interview at any time.

Since the introduction to the participants at the sub-Saharan Africa site was performed by participants’ employer, additional information was included in the call for participants and the consent form for that site to clarify that the research was being conducted and managed by an external group. The call for participation also included the confidentiality protections participants could expect, namely that their individual responses would not be shared with their employer. The participating company additionally signed a memorandum of understanding in advance of embarking on the study confirming that: (a) no individual or attributable data from participants would be shared with them, and (b) any information shared with them from the research would be synthesized and shared in the form of high-level themes.

For each site, the research team at PAI was the only group with access to participants’ diary study and interview responses. In the case of group interviews, participants agreed at the start of interviews to keep any comments shared by others in the group confidential and to protect the privacy of other participants. In line with standard qualitative research practices, the employers at all sites are not named, to assist in protecting the anonymity of participants.

Confidentiality and data storage

All participants’ identities have been anonymized in the research output. After completion of the diary studies and interviews, participant responses have been stored separately from identifying information about the participants who provided them.

Data analysis

Information from this study has been analyzed using two approaches. To determine a framework of workplace AI product types from workers’ perspectives and to categorize the technologies mentioned in this study according to the framework, a deductive approach was used. To identify themes from the participant responses, an iterative inductive approach was used both to identify initial and emerging themes and to synthesize and cluster those initial themes into the major themes in this report.