Background

Background

Problems Due to Lack of Diversity of AI Teams

Problems Due to Lack of Diversity of AI Teams

AI algorithms have caused well-documented problems due to a lack of consideration about the nuance and sensitivity of human diversity in race, gender, sexual orientation, ability, religion, and other identity categories. Categories within these identities and the intersections thereof are subject to power imbalances based on historical marginalization and oppression which contribute to contemporary systemic inequality. One striking example is differences in gender representation as employees move up the corporate ladder.

A 2021 report by McKinsey found that the proportion of White women, women of color, and men of color went from 30%, 17%, and 17% at the entry level to 20%, 4%, and 13%, respectively, in the C-suite. Women in the Workplace (2021). 2021. Retrieved 23 November 2021, from https://www.mckinsey.com/featured-insights/diversity-and-inclusion/women-in-the-workplace In contrast, White men comprise about 30% of entry level positions and 62% of C-Suite positions. A survey of entrepreneurs by Silicon Valley Bank suggested similar numbers among tech startups, with less than half of respondents reporting even one female or ethnic minority C-suite executive on their teams. Silicon Valley Bank. (2021). 2020 Global Startup Outlook: Key insights from the Silicon Valley Bank startup outlook survey (Ebook). Retrieved 23 November 2021, from https://www.svb.com/globalassets/library/uploadedfiles/content/trends_and_insights/reports/startup_outlook_report/suo_global_report_2020-final.pdf Tech teams thus lack representation by the people affected by their products, although a tokenistic representation is not enough to undo the systemic harms caused by these products.

There are several examples of AI that have harmed or had the potential to harm diverse populations all over the world. Some examples include:

• An algorithm used to determine credit lines being biased against female applicants Firth, N. (2021). Apple Card is being investigated over claims it gives women lower credit limits. MIT Technology Review. Retrieved 23 November 2021, from https://www.technologyreview.com/2019/11/11/131983/apple-card-is-being-investigated-over-claims-it-gives-women-lower-credit-limits/.
• The exclusion or erasure of trans people in health tech apps and other medical tech where outdated binary conceptions of gender are used Tomasev, N., McKee, K.R., Kay, J., & Mohamed, S. (2021). Fairness for Unobserved Characteristics: Insights from technological impacts on queer communities. In Proceedings of the 2021 AAAI/ACM Conference on AI, Ethics, and Society (AIES ’21), Retrieved October 1, 2021 from https://doi.org/10.1145/3461702.3462540
• Discrimination against applicants for housing loans, where Black and Latinx applicants were charged mortgage rates several times higher than other groups Martinez, E., & Kirchner, L. (2021). The secret bias hidden in mortgage-approval algorithms | AP News. AP News. Retrieved 24 November 2021, from https://apnews.com/article/lifestyle-technology-business-race-and-ethnicity-mortgages-2d3d40d5751f933a88c1e17063657586

AI ethics researchers have recommended several steps to mitigate algorithmic bias, including training them on data from diverse individuals or ensuring that humans oversee the deployment of these algorithms and overrule biased decisions if necessary. Turner Lee, N., Resnick, P., & Barton, G. (2021). Algorithmic bias detection and mitigation: Best practices and policies to reduce consumer harms. Brookings. Retrieved 24 November 2021, from https://www.brookings.edu/research/algorithmic-bias-detection-and-mitigation-best-practices-and-policies-to-reduce-consumer-harms/. However, this may not be possible without a team of workers who have a deep and comprehensive understanding of how these algorithms can negatively impact affected populations, deeper than a superficially created team that is diverse but lacks understanding of systemic prejudice.

More Diverse Teams Yield Better Outcomes

More Diverse Teams Yield Better Outcomes

Research has repeatedly shown that diversity on a team serves a greater purpose than good optics. Several studies suggest that diverse teams make more accurate decisions Rock, D., & Grant, H. (2016). Why diverse teams are smarter. Harvard Business Review, 4(4), 2-5. and help to drive innovation. Wang, J., Cheng, G. H. L., Chen, T., & Leung, K. (2019). Team creativity/innovation in culturally diverse teams: A meta‐analysis. Journal of Organizational Behavior, 40(6), 693-708. Organizational diversity is associated with higher profits. Lorenzo, R., Voigt, N., Tsusaka, M., Krentz, M., & Abouzahr, K. (2018). How Diverse Leadership Teams Boost Innovation. BCG Global. Retrieved 24 November 2021, from https://www.bcg.com/publications/2018/how-diverse-leadership-teams-boost-innovation Internationally, companies with higher numbers of women in executive positions were more successful. Hoobler, J. M., Masterson, C. R., Nkomo, S. M., & Michel, E. J. (2018). The business case for women leaders: Meta-analysis, research critique, and path forward. Journal of Management, 44(6), 2473-2499. Many experts agree with the general conclusions of studies on corporate diversity more broadly and how they could apply to tech teams. Chakravorti, B. (2020). To Increase Diversity, U.S. Tech Companies Need to Follow the Talent. Harvard Business Review. Retrieved 24 November 2021, from https://hbr.org/2020/12/to-increase-diversity-u-s-tech-companies-need-to-follow-the-talent.

A joint report from Accenture, the American Association of People with Disabilities, and Disability:IN emphasized the business case for hiring employees with disabilities, reporting a 28% advantage in revenue and twice the net income among other benefits, such as increased returns for shareholders, among companies rated highly for their disability programs and initiatives. Accenture. (2018). Getting to Equal 2018: The Disability Inclusion Advantage. Retrieved from https://www.accenture.com/_acnmedia/pdf-89/accenture-disability-inclusion-research-report.pdf This is crucial for AI teams specifically because of the inherent interdisciplinary nature of the issues that these teams tackle. For instance, AI systems must take into account disability status as an axis of diversity in order to avoid the marginalization of these populations. Whittaker, M., Alper, M., Bennett, C. L., Hendren, S., Kaziunas, L., Mills, M., … & West, S. M. (2019). Disability, bias, and AI. AI Now Institute.

Current Level of Diversity in Tech

Current Level of Diversity in Tech

After the murder of George Floyd, the summer of 2020 brought the appearance of a reckoning around anti-Black racism across the US and the world. Heater, B. (2020). Tech companies respond to George Floyd’s death, ensuing protests and systemic racism. Techcrunch.com. Retrieved 24 November 2021, from https://techcrunch.com/2020/06/01/tech-co-protests/. Despite several pledges to increase diversity of staff and build more inclusive work environments, the tech industry continues to have problems with both. The available data reveal a pattern of increasing diversity within the past few years, but still far from having a workforce that is diverse and representative of the population as a whole. Numbers are most readily available for workers in the US, but some of the same patterns apply to the tech workforce in European countries. This report avoids a direct comparison between the racial demographics in the workforce of other countries since there is a difference in how these places construct race. Here are some examples from recent diversity reports:

Google’s 2021 Diversity report Google (2021). 2021 Diversity Annual Report. Retrieved 24 November 2021, from https://static.googleusercontent.com/media/diversity.google/en//annual-report/static/pdfs/google_2021_diversity_annual_report.pdf?cachebust=2e13d07. showed that two groups of women of color left the company at particularly high rates. The company’s “attrition index,” anchored at around 100, went from 110 to 146 for Black women and from 123 to 148 for Native American women. Overall, the attrition indices rose for all racially minoritized groups, despite recruiting more racially minoritized individuals in the US in 2020, and more women globally in 2021 than in 2020.

Between 2020 and 2021, the number of Black people at Facebook rose from 3.9% to 4.4%. Facebook. (2021). Facebook Diversity Update: Increasing Representation in Our Workforce and Supporting Minority-Owned Businesses | Meta. Meta. Retrieved 24 November 2021, from https://about.fb.com/news/2021/07/facebook-diversity-report-2021/.

Amazon’s corporate workforce showed increases in Black and Latinx workers from 2019 to 2020. The percentage of women decreased, however, from 37% to 36.7% that year. Amazon Staff. (2020). Our workforce data. US About Amazon. Retrieved 24 November 2021, from https://www.aboutamazon.com/news/workplace/our-workforce-data

Similar numbers were reported by other companies, such as Adobe, which reported an increase in “underrepresented minorities” from 10.2% to 10.8% between 2019 and 2020. Adobe. (2021). Adobe Diversity By the Numbers. Adobe. Retrieved 24 November 2021, from https://www.adobe.com/diversity/data.html

Although they account for 47% of the workforce in the US, the National Center for Women in Tech reports that only about 26% of tech jobs belong to women. Only about 2% of the tech workforce were Hispanic women, 6% Asian women, and 3% Black women. National Center for Women in Tech. (2020). NCWIT Scorecard: The Status of Women in Computing (2020 Update). Retrieved https://ncwit.org/resource/scorecard/ This is compared with the wider US workforce, where about 6% of workers are Hispanic women, 2% are Asian women, and 6% are Black women. Center for American Progress (2012). The State of diversity in Today’s workforce. Retrieved from https://www.americanprogress.org/article/the-state-of-diversity-in-todays-workforce/ In 2020, just 22% of CIO positions at Fortune 500 companies were reportedly held by women. Gillenwater, S. (2020). Meet the CIOs of the Fortune 500 — 2021 edition. Boardroom Insiders. Retrieved from https://www.boardroominsiders.com/blog/meet-the-cios-of-the-fortune-500-2021-edition

Data on tech workers with disabilities is difficult to come by. However, of nearly 65,000 responses to Stack Overflow’s worldwide survey Stack Overflow. (2020). 2020 Developer Survey. Retrieved from https://insights.stackoverflow.com/survey/2020#developer-profile-disability-status-mental-health-and-differences of developers, 9,532 responses identified as having a mental difference (such as an anxiety, mood, concentration, or autism spectrum disorder) and 1,284 responses identified as having a physical difference (such as difficulty hearing, seeing, standing or typing).

Diversity in AI

Diversity in AI

Numbers concerning diversity in AI are generally more difficult to pin down because people working in AI work at both corporations and non-profit organizations across a variety of different product teams as well as in non-technical roles. Among the previous diversity reports mentioned, there are no data specific to their AI teams. Stanford’s Human-Centered Artificial Intelligence’s (HAI) 2021 AI Index Stanford HAI. (2021). The AI Index Report: Measuring Trends in Artificial intelligence (Ebook). Retrieved 24 November 2021, from https://aiindex.stanford.edu/wp-content/uploads/2021/03/2021-AI-Index-Report-_Chapter-6.pdf. report gave some clues as to the percentage of minoritized folks working in AI. For instance, very few CS PhDs and tenure-track faculty members are Black (3% and .6% respectively), but the organization Black in AI has continued to grow. According to the report, fewer than 23% of AI PhD graduates are women. The few specific studies available also suggest that the AI field has much farther to go to reflect the diversity of the members of the population that it affects. About 25% of Queer in AI members describe themselves as “junior industry” and 81% of members characterize the “lack of role models” in the industry as a barrier.

What Has Been Done

What Has Been Done

Researchers and activists have been pushing for more representation in the field to tackle AI ethics issues for which solutions require diverse perspectives. Some researchers have observed that AI ethics practices were divorced from broader DEI work. Chi, N., Lurie, E., & Mulligan, D. K. (2021). Reconfiguring Diversity and Inclusion for AI Ethics. arXiv preprint arXiv:2105.02407. Organizations have often kept their DEI teams separate from their AI team areas, with little interaction between the two. This represents a fairly siloed approach to AI ethics, in which content area experts do not weigh in on issues like algorithmic bias and whether technology should be used to automate everything from loan applications to screening applicants for job interviews. To tackle this problem, organizations must move beyond a more tokenistic approach to diversity, doing more than just increasing the pipeline of diverse candidates and increasing the number of minoritized folks interviewed for positions. Instead they must fundamentally audit internal systems to make them more supportive for everyone.

What Has Been Done

Looking at the Pipeline

Tech companies have long blamed their lack of racial and gender diversity on a lack of diverse candidates from which to recruit. Selyukh, A. (2016). Why Some Diversity Thinkers Aren’t Buying The Tech Industry’s Excuses. NPR. Retrieved 24 November 2021, from https://www.npr.org/sections/alltechconsidered/2016/07/19/486511816/why-some-diversity-thinkers-arent-buying-the-tech-industrys-excuses. Available data in the US show a lack of diversity at all levels between K-12 and college, but do not account for the continued lack of diversity in corporate-level jobs:

The National Association for Educational Progress’ 2019 Nation’s Report Card National Association for Educational Progress. (2020). NAEP Report Card: Mathematics. Retrieved from https://www.nationsreportcard.gov/mathematics/nation/achievement/?grade=4 found racial disparities in the percentage of children in the 4th grade who received proficiency in math. 69% of Asian children, for instance, achieved proficiency, compared to just 20% of Black children. These numbers showed a slight increase from previous years. While this is only a proxy to predict future engagement in computer science and tech, available data suggest similar patterns.

Only 7.6% of high school students who took computer science courses had disabilities, as compared with 12.9% of high school students who had disabilities. Ladner, R. (2021). Expanding the pipeline: The status of persons with disabilities in the Computer Science Pipeline. Retrieved February 1, 2022, from https://cra.org/cra-wp/expanding-the-pipeline-the-status-of-persons-with-disabilities-in-the-computer-science-pipeline/ Accurate figures are difficult to come by for undergraduate and graduate students in computer science with disabilities, but the Data Buddies Survey reported that about 7.6% of undergraduate CS students and 11.4% of graduate CS students had disabilities. Center for Evaluating the Research Pipeline (2021). “Data Buddies Survey 2019 Annual Report”. Computing Research Association, Washington, D.C.

A report from Code.org reported that only 22% of students taking AP computer science were female and about 13% identified as some racially minoritized identity. Code.org. (2021). Code.org’s Approach to Diversity & Equity in Computer Science. Code.org. Retrieved 24 November 2021, from https://code.org/diversity The report further stated that the higher percentage of minoritized students that comprised a high school, the less likely the school was to offer computer science.

The most recent Taulbee survey from the Computing Research Association (CRA) found that about 23% of new PhD enrollments in computer science identified as female and about 14% belonged to some minoritized identity and were residents of the US. ✳These percentages belie the true number of minoritized students because they leave out race/ethnicity data for foreign students in the US. However, this recent growth in diversity cannot be seen in the recent numbers in diversity reports from tech companies.Zweben, S., & Bizot, B. (2021). 2020 Taulbee Survey: Bachelor’s and Doctoral Degree Production Growth Continues but New Student Enrollment Shows Declines (Ebook). Computing Research Association. Retrieved 24 November 2021, from https://cra.org/wp-content/uploads/2021/05/2020-CRA-Taulbee-Survey.pdf The CRA’s Generation CS report found growth in the number of enrollments in computer science courses at the undergraduate and graduate level between 2006 and 2015. Computing Research Association (2017). Generation CS: Computer Science Undergraduate Enrollments Surge Since 2006

Figures in the UK mirror those in the US, with female students making up 18% of those enrolled in computer science at the undergraduate level. The Higher Education Statistics Agency (2021). Higher Education Student Statistics. Retrieved from https://www.hesa.ac.uk/news/16-01-2020/sb255-higher-education-student-statistics/subjects Data from both BCS BCS. (2014). Women in IT Survey (Ebook). BCS: The Chartered Institute for IT. Retrieved 24 November 2021, from https://www.bcs.org/media/4446/women-it-survey.pdf and Inclusive Boards Inclusive Boards. (2018). Inclusive Tech Alliance Report 2018 (Ebook). Retrieved 24 November 2021, from https://www.inclusivetechalliance.co.uk/wp-content/uploads/2019/07/Inclusive-Tech-Alliance-Report.pdf. indicate about 17% of the broader tech workforce in the UK is female. This is similar to figures of women in tech in Europe overall. No accurate data exist for Europe overall, but estimates point to a similar lack of diversity as that in the US. Atomico. (2020). The State of European Tech 2020. 2020.stateofeuropeantech.com. Retrieved 24 November 2021, from https://2020.stateofeuropeantech.com/chapter/diversity-inclusion/article/diversity-inclusion/.

These data suggest that a lack of diversity in the tech pipeline accounts for some lack of diversity in the tech workplace, but there is little evidence that the lack of growth in diversity in big tech companies is mostly due to continued pipeline problems.

What Has Been Done

What Leads to Attrition?

While no large-scale studies have been done on general reasons that workers leave tech organizations, there are some potential clues in studies of general workplace attrition. These studies point to overall job satisfaction, which is linked to autonomy, psychological well-being, and fair compensation. Chung-Yan, G. A. (2010). The nonlinear effects of job complexity and autonomy on job satisfaction, turnover, and psychological well-being. Journal of occupational health psychology, 15(3), 237. Similar results have been found in the IT sector. McKnight, D. H., Phillips, B., & Hardgrave, B. C. (2009). Which reduces IT turnover intention the most: Workplace characteristics or job characteristics?. Information & Management, 46(3), 167-174. Employees also look to characteristics of the workplace such as a fair and transparent systems for promotion, and effective managers. Vaamonde, J. D., Omar, A., & Salessi, S. (2018). From organizational justice perceptions to turnover intentions: The mediating effects of burnout and job satisfaction. Europe’s journal of psychology, 14(3), 554. Several recent studies collected data relevant to this:

Instructure (2019). How to get today’s employees to stay and engage? Develop their careers. PR Newswire. Retrieved from https://www.prnewswire.com/news-releases/how-to-get-todays-employees-to-stay-and-engage-develop-their-careers-300860067.htmlMcCarty, E. (2021). Integral and The Harris Poll Find Employees are giving Employers a Performance Review – Integral. Integral. Retrieved 24 November 2021, from https://www.teamintegral.com/2021/news-release-integral-employee-activation-index/McCarty, E. (2021). Integral and The Harris Poll Find Employees are giving Employers a Performance Review – Integral. Integral. Retrieved 24 November 2021, from https://www.teamintegral.com/2021/news-release-integral-employee-activation-index/

The employment landscape in the US has rapidly changed, with some media outlets even dubbing the events of 2021 as “the great resignation.” The US Bureau of Labor statistics reported that 4.3 million Americans, or 2.9% of the workforce, left their jobs in August 2021. Bureau of Labor Statistics. (2021). News Release – The Employment Situation – October 2021 (Ebook). Retrieved 24 November 2021, from https://www.bls.gov/news.release/pdf/empsit.pdf

These findings cannot readily generalize to tech, AI, or other workplaces outside the US, but they might give some clues as to how to approach attrition in the AI workplace.

Attrition in Tech

Attrition in Tech

The Kapor Center’s 2017 tech leaver’s study gives the most comprehensive account of attrition focused on tech in the US. Scott, A., Kapor Klein, F., & Onovakpuri, U. (2017). Tech Leavers Study (Ebook). Retrieved 24 November 2021, from https://www.kaporcenter.org/wp-content/uploads/2017/08/TechLeavers2017.pdf. The study found that:

• Unfairness was the main driver of workers who voluntarily left tech jobs, with 37% of respondents citing this as the main reason they left their previous employers.
• About 78% of respondents reported that they experienced unfair treatment in their organization.
• Different gender and racial/ethnic groups experienced this unfairness differently. For instance, 30% of Black, Latinx, and Indigenous women (categorized in the report as underrepresented women of color) reported being passed over for a promotion, a difference that was significantly higher than underrepresented men of color and White and Asian women.
• Almost 25% of underrepresented people of color experienced stereotyping in their previous positions and 1 in 10 women reported receiving unwanted sexual attention in their previous jobs.
• At 20%, LGBTQ+ employees were also most likely to report bullying.

Findings from the Women in Data Science and AI project by the Turing Institute provide further support for these findings. The report found that there were gender differences in the types of AI jobs occupied by men and women. Young, E., Wajcman, J. and Sprejer, L. (2021). Where are the Women? Mapping the Gender Job Gap in AI. Policy Briefing: Full Report. The Alan Turing Institute. For instance, women were more likely to occupy jobs that were lower in status and pay and less technical than those occupied by men. It also reported that women left their jobs at higher rates than men left their jobs. Several high-profile cases over the last year, including the firing of Dr. Timnit Gebru and Dr. Meg Mitchell at Google AI, Metz, C. (2021). A second Google A.I. researcher says the company fired her.. Nytimes.com. Retrieved 24 November 2021, from https://www.nytimes.com/2021/02/19/technology/google-ethical-artificial-intelligence-team.html and Ifeoma Ozoma and Aerica Shimizu Banks at Pinterest, Myrow, R. (2021). Pinterest Sounds A More Contrite Tone After Black Former Employees Speak Out. Npr.org. Retrieved 24 November 2021, from https://www.npr.org/2020/06/23/881624553/pinterest-sounds-a-more-contrite-tone-after-black-former-employees-speak-out have demonstrated many of the hallmarks of these research findings. While there is no comprehensive study about the large-scale attrition of people with disabilities in tech, there is still much progress to be made for hiring and increasing belonging at tech companies for tech workers with disabilities. Scheer, S. (2021). The Tech Sector’s Big Disability Inclusion Problem. ERE. Retrieved from https://www.ere.net/the-tech-sectors-big-disability-inclusion-problem/

For AI, a field that aims to attract talent that draws from diverse perspectives and backgrounds, it is especially important to intentionally build a positive climate that is welcoming to all identities, since the status quo is likely to prove unwelcome or even hostile for workers belonging to minoritized identities.