Content Type: Research and Publications

Guidelines for AI and Shared Prosperity



Home

Our economic future is too important to leave to chance.

AI has the potential to radically disrupt people’s economic lives in both positive and negative ways. It remains to be determined which of these we’ll see more of. In the best scenario, AI could widely enrich humanity, equitably equipping people with the time, resources, and tools to pursue the goals that matter most to them.

Our current moment serves as a profound opportunity — one that we will miss if we don’t act now. To achieve a better future with AI, we must put in the work today.

In medicine and other fields, new innovations are put through rigorous testing to ensure they are fit for purpose. The AI community, however, has no established practice for assessing the impact of AI systems on inequality or job quality. Without one, it remains difficult to ensure AI deployments are bringing us closer to the economic future we want to live in.

You can help guide AI’s impact on jobs

AI developers, AI users, policymakers, labor organizations, and workers can all help steer AI so its economic benefits are shared by all. Using Partnership on AI’s (PAI) Shared Prosperity Guidelines, these stakeholders can minimize the chance that individual AI systems worsen shared prosperity-relevant outcomes.

The Shared Prosperity Guidelines can be used by following a guided, three-step process.

 

 


AI-Creating Organizations ➜


AI-Using Organizations ➜


Policymakers ➜


Labor Organizations and Workers ➜

Download the Guidelines

Get Involved

Partnership on AI needs your help to refine, test, and drive adoption of the Guidelines for AI and Shared Prosperity.

Fill out the form below to share your feedback on the Guidelines, ask about collaboration opportunities, and receive updates about events and other future work by the AI and Shared Prosperity Initiative.

Get in Touch

Guidelines for AI and Shared Prosperity

Home

Step 1: Learn About the Guidelines

The Need for the Guidelines

The Origin of the Guidelines

Design of the Guidelines

Key Principles for Using the Guidelines

Step 2: Apply the Job Impact Assessment Tool

Instructions for Performing a Job Impact Assessment

Signals of Opportunity to Advance Shared Prosperity

Signals of Risk to Shared Prosperity

STEP 3: Stakeholder-Specific Recommendations

For AI-Creating Organizations

For AI-Using Organizations

For Policymakers

For Labor Organizations and Workers

Get Involved

Endorsements

Acknowledgments

AI and Shared Prosperity Initiative’s Steering Committee

Sources Cited

  1. ​​Acemoglu, D. (Ed.). (2021). Redesigning AI: Work, democracy, and justice in the age of automation. Boston Review.
  2. Korinek, A., and Stiglitz, J.E. (2020, April). Steering technological progress. In NBER Conference on the Economics of AI.
  3. Acemoglu, D., and Johnson, S. (2023). Power and Progress: Our Thousand-Year Struggle Over Technology and Prosperity. Public Affairs, New York.
  4. International Labour Organization. (n.d.). Decent work. https://tinyurl.com/yur776yd
  5. US Department of Commerce and US Department of Labor. (n.d.). Department of Commerce and Department of Labor Good Jobs Principles, DOL. https://tinyurl.com/mtbpemkn
  6. Institute for the Future of Work. (n.d.). The Good Work Charter. https://tinyurl.com/ycxtaax4
  7. Klinova, K., and Korinek, A. (2021). AI and shared prosperity. In Proceedings of the 2021 AAAI/ACM Conference on AI, Ethics, and Society (pp. 645-651).
  8. Bell, S. A. (2022). AI and Job Quality: Insights from Frontline Workers. https://papers.ssrn.com/sol3/papers.cfm?abstract_id=4337611
  9. Partnership on AI, 2021. Redesigning AI for Shared Prosperity: an Agenda. https://partnershiponai.org/paper/redesigning-ai-agenda/
  10. Negrón, W. (2021). Little Tech is Coming for Workers. Coworker.org. https://home.coworker.org/wp-content/uploads/2021/11/Little-Tech-Is-Coming-for-Workers.pdf.
  11. Korinek, A., 2022. How innovation affects labor markets: An impact assessment.
  12. Brynjolfsson, E., Collis, A., Diewert, W.E., Eggers, F., and Fox, K.J. (2019). GDP-B: Accounting for the value of new and free goods in the digital economy (No. w25695). National Bureau of Economic Research.
  13. Acemoglu, D., and Restrepo, P. (2019). Automation and new tasks: How technology displaces and reinstates labor. Journal of Economic Perspectives, 33(2), 3-30.
  14. Bell, S. A. (2022). AI and Job Quality: Insights from Frontline Workers. https://papers.ssrn.com/sol3/papers.cfm?abstract_id=4337611
  15. Valentine, M., and Hinds, R. (2022). How Algorithms Change Occupational Expertise by Prompting Explicit Articulation and Testing of Experts’ Theories. https://tinyurl.com/pxyr8ev3
  16. Autor, D. (2022). The labor market impacts of technological change: From unbridled enthusiasm to qualified optimism to vast uncertainty (No. w30074). National Bureau of Economic Research.
  17. Mateescu, A., and Elish, M. (2019). AI in context: the labor of integrating new technologies.
  18. Elish, M. C. (2019). Moral crumple zones: Cautionary tales in human-robot interaction (pre-print). Engaging Science, Technology, and Society (pre-print).
  19. World Bank. (2017). World development report 2018: Learning to realize education's promise. The World Bank.
  20. Korinek, A., and Stiglitz, J.E. (2021). Artificial intelligence, globalization, and strategies for economic development (No. w28453). National Bureau of Economic Research.
  21. Diao, X., Ellis, M., McMillan, M. S., and Rodrik, D. (2021). Africa's manufacturing puzzle: Evidence from Tanzanian and Ethiopian firms (No. w28344). National Bureau of Economic Research.
  22. Rodrik, D. (2022). 4 Prospects for global economic convergence under new technologies. An inclusive future? Technology, new dynamics, and policy challenges, 65.
  23. O'Keefe, C., Cihon, P., Garfinkel, B., Flynn, C., Leung, J., and Dafoe, A. (2020, February). The windfall clause: Distributing the benefits of AI for the common good. In Proceedings of the AAAI/ACM Conference on AI, Ethics, and Society (pp. 327-331).
  24. Bell, S. A. (2022). AI and Job Quality: Insights from Frontline Workers. https://papers.ssrn.com/sol3/papers.cfm?abstract_id=4337611
  25. Scherer, M., and Brown, L. X. (2021). Warning: Bossware May Be Hazardous to Your Health. Center for Democracy and Technology. https://cdt.org/wp-content/uploads/2021/07/2021-07-29-Warning-Bossware-May-Be-Hazardous-To-Your-Health-Final.pdf
  26. Bell, S. A. (2022). AI and Job Quality: Insights from Frontline Workers. https://papers.ssrn.com/sol3/papers.cfm?abstract_id=4337611
  27. Acemoglu, D., and Restrepo, P. (2022). Tasks, automation, and the rise in US wage inequality. Econometrica, 90(5), 1973-2016.
  28. Valentine, M., and Hinds, R. (2022). How Algorithms Change Occupational Expertise by Prompting Explicit Articulation and Testing of Experts’ Theories. https://tinyurl.com/pxyr8ev3
  29. Nurski, L., and Hoffmann, M. (2022). The Impact of Artificial Intelligence on the Nature and Quality of Jobs. Working Paper. Bruegel. https://tinyurl.com/jxayzdcz
  30. Pritchett, L. (2020). The future of jobs is facing one, maybe two, of the biggest price distortions ever. Middle East Development Journal, 12(1), 131-156.
  31. Eloundou, T., Manning, S., Mishkin, P., and Rock, D. (2023). GPTs are GPTs: An early look at the labor market impact potential of large language models. arXiv preprint arXiv:2303.10130.
  32. Noy, S., and Zhang, W. (2023). Experimental evidence on the productivity effects of generative artificial intelligence. https://papers.ssrn.com/sol3/papers.cfm?abstract_id=4375283
  33. Korinek, A. (2023). Language models and cognitive automation for economic research (No. w30957). National Bureau of Economic Research.
  34. Case, A., and Deaton, A. (2020). Deaths of Despair and the Future of Capitalism. Princeton University Press.
  35. Gihleb, R., Giuntella, O., Stella, L., and Wang, T. (2022). Industrial robots, workers’ safety, and health. Labour Economics, 78, 102205.
  36. Pritchett, L. (2020). The future of jobs is facing one, maybe two, of the biggest price distortions ever. Middle East Development Journal, 12(1), 131-156.
  37. Pritchett, L. (2023). Choose People. LaMP Forum. https://lampforum.org/2023/03/02/choose-people/
  38. Gray, M. L., and Suri, S. (2019). Ghost work: How to stop Silicon Valley from building a new global underclass. Eamon Dolan Books.
  39. Dubal, V. (2023). On Algorithmic Wage Discrimination. https://papers.ssrn.com/sol3/papers.cfm?abstract_id=4331080
  40. Bell, S. A. (2022). AI and Job Quality: Insights from Frontline Workers. https://papers.ssrn.com/sol3/papers.cfm?abstract_id=4337611
  41. Schneider, D., and Harknett, K. (2017, April). Schedule Instability and Unpredictability and Worker and Family Health and Well-being. In PAA 2017 Annual Meeting. PAA.
  42. Williams, J. et al. (2022). Stable scheduling study: Health outcomes report. https://ssrn.com/abstract=4019693
  43. Bell, S. A. (2022). AI and Job Quality: Insights from Frontline Workers. https://papers.ssrn.com/sol3/papers.cfm?abstract_id=4337611
  44. Dzieza, J. (2020). Robots aren’t taking our jobs — They’re becoming our bosses. The Verge. https://tinyurl.com/5a9mxeuz
  45. Levy, K. (2022). Data Driven: truckers, technology, and the new workplace surveillance. Princeton University Press.
  46. Moore, P.V. (2017). The quantified self in precarity: Work, technology and what counts. Routledge.
  47. Scherer, M., and Brown, L. X. (2021). Warning: Bossware May Be Hazardous to Your Health. Center for Democracy and Technology. https://cdt.org/wp-content/uploads/2021/07/2021-07-29-Warning-Bossware-May-Be-Hazardous-To-Your-Health-Final.pdf.
  48. Brand, J., Dencik, L. and Murphy, S. (2023). The Datafied Workplace and Trade Unions in the UK. Data Justice Lab. https://datajusticeproject.net/wp-content/uploads/sites/30/2023/04/Unions-Report_final.pdf.
  49. Nurski, L., and Hoffmann, M. (2022). The Impact of Artificial Intelligence on the Nature and Quality of Jobs. Working Paper. Bruegel. https://tinyurl.com/2a943p8f
  50. Nanavaty, R. (2023). Interview with Reema Nanavaty, Self-Employed Women’s Association.
  51. Beane, M. (2022). Today's Robotic Surgery Turns Surgical Trainees into Spectators: Medical Training in the Robotics Age Leaves Tomorrow's Surgeons Short on Skills. IEEE Spectrum, 59(8), 32-37. https://tinyurl.com/wyhxukhk
  52. Gray, M. L., and Suri, S. (2019). Ghost work: How to stop Silicon Valley from building a new global underclass. Eamon Dolan Books.
  53. Center for Democracy and Technology et al. 2022
  54. Buolamwini, J., and Gebru, T. (2018, January). Gender shades: Intersectional accuracy disparities in commercial gender classification. In Conference on fairness, accountability and transparency (pp. 77-91). PMLR.
  55. Benjamin, R. (2019). Race After Technology: Abolitionist Tools for the New Jim Code. John Wiley and Sons.
  56. Keyes, O. (2018). The misgendering machines: Trans/HCI implications of automatic gender recognition. Proceedings of the ACM on human-computer interaction, 2(CSCW), 1-22.
  57. Rosales, A., and Fernández-Ardèvol, M. (2019). Structural ageism in big data approaches. Nordicom Review, 40(s1), 51-64.
  58. Klinova, K. (2022) Governing AI to Advance Shared Prosperity. In Justin B. Bullock et al. (Eds.), The Oxford Handbook of AI Governance. Oxford Handbooks.
  59. Park, H., Ahn, D., Hosanagar, K., and Lee, J. (2021, May). Human-AI interaction in human resource management: Understanding why employees resist algorithmic evaluation at workplaces and how to mitigate burdens. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems (pp. 1-15).
  60. Bernhardt, A., Suleiman, R., and Kresge, L. (2021). Data and algorithms at work: the case for worker technology rights. https://laborcenter.berkeley.edu/wp-content/uploads/2021/11/Data-and-Algorithms-at-Work.pdf.
  61. Colclough, C.J. (2022). Righting the Wrong: Putting Workers’ Data Rights Firmly on the Table. https://tinyurl.com/26ycnpv2
  62. Pasquale, F. (2020). New Laws of Robotics. Harvard University Press.
  63. Rodrik, D. (2022). 4 Prospects for global economic convergence under new technologies. An inclusive future? Technology, new dynamics, and policy challenges, 65.
  64. Anderson, E. (2019). Private Government: How Employers Rule Our Lives (and Why We Don’t Talk about it). Princeton University Press.
  65. Korinek, A. (2022). How innovation affects labor markets: An impact assessment.
  66. Institute for the Future of Work. (2023). Good Work Algorithmic Impact Assessment Version 1: An approach for worker involvement. https://tinyurl.com/mr4yn5yt
  67. Bernhardt, A., Suleiman, R., and Kresge, L. (2021). Data and algorithms at work: the case for worker technology rights. https://laborcenter.berkeley.edu/wp-content/uploads/2021/11/Data-and-Algorithms-at-Work.pdf.
  68. Colclough, C.J. (2022). Righting the Wrong: Putting Workers’ Data Rights Firmly on the Table. https://tinyurl.com/26ycnpv2
  69. Brand, J., Dencik, L. and Murphy, S. (2023). The Datafied Workplace and Trade Unions in the UK. Data Justice Lab. https://datajusticeproject.net/wp-content/uploads/sites/30/2023/04/Unions-Report_final.pdf.
  70. Park, H., Ahn, D., Hosanagar, K., and Lee, J. (2021, May). Human-AI interaction in human resource management: Understanding why employees resist algorithmic evaluation at workplaces and how to mitigate burdens. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems (pp. 1-15).
  71. Mateescu, A., and Elish, M. (2019). AI in context: the labor of integrating new technologies.
  72. Elish, M. C. (2019). Moral crumple zones: Cautionary tales in human-robot interaction (pre-print). Engaging Science, Technology, and Society (pre-print).
Table of Contents

Implementing Responsible Data Enrichment Practices at an AI Developer: The Example of DeepMind

Sonam Jindal
Download the Case Study

Executive Summary

Executive Summary

As demand for AI services grows, so, too, does the need for the enriched data used to train and validate machine learning (ML) models. While these datasets can only be prepared by humans, the data enrichment workers who do so (performing tasks like data annotation, data cleaning, and human review of algorithmic outputs) are an often-overlooked part of the development lifecycle, frequently working in poor conditions continents away from AI-developing companies and their customers.

Workers

For the purposes of this white paper we refer to individuals completing data enrichment as “workers.” In doing so, we recognize the variety of employment statuses that can exist in the data enrichment industry, including independent contractors on self-service crowdsourcing platforms, subcontractors of data enrichment providers, and full-time employees.

Last year, the Partnership on AI (PAI) published “
Responsible Sourcing of Data Enrichment Services,” a white paper exploring how the choices made by AI practitioners could improve the working conditions of these data enrichment professionals. This case study documents an effort to put that paper’s recommendations into practice at one AI developer: DeepMind, a PAI Partner.

In addition to creating guidance for responsible AI development and deployment, PAI’s Theory of Change includes collaborating with Partners and others to implement our recommendations in practice. From these collaborations, PAI collects findings which help us further develop our curriculum of responsible AI resources. This case study serves as one such resource, offering a detailed account of DeepMind’s process and learnings for other organizations interested in improving their data enrichment sourcing practices.

Sourcing enriched data
Sourcing data enrichment work is a process that requires a number of steps including, but not limited to, defining the enrichment goal, choosing the enrichment provider, defining the enrichment tools, defining the technical requirements, writing instructions, ensuring that instructions make sense, setting worker hours, determining time spent on a particular task, communicating with enrichment workers, rejecting or accepting work, defining a project budget, determining workers’ payment, checking work quality, and providing performance feedback.

After assessing DeepMind’s existing practices and identifying what was needed to consistently source enriched data responsibly, PAI and DeepMind worked together to prototype the necessary policies and resources. The Responsible Data Enrichment Implementation Team (which consisted of PAI and members of DeepMind’s Responsible Development and Innovation team, which we will refer to as “the implementation team” in this case study) then collected multiple rounds of feedback, testing the following outputs and changes with smaller teams before they were rolled out organization-wide:

A two-page document offering fundamental guidelines for responsible data enrichment sourcing
An updated ethics review process
A checklist detailing what constitutes “good instructions” for data enrichment workers
A table to easily compare the salient features of various data enrichment platforms and vendors
A spreadsheet listing the living wages in areas where data enrichment workers commonly live

Versions of these resources have been added to PAI’s responsible data enrichment sourcing library and are now available for any organization that wishes to improve its data enrichment sourcing practices.

Ultimately, DeepMind’s multidisciplinary teams developing AI research, including applied AI researchers (or “researchers” for the purposes of this case study, though this term might be defined differently elsewhere) said that these new processes felt efficient and helped them think more deeply about the impact of their work on data enrichment workers. They also expressed gratitude for centralized guidance that had been developed through a rigorous process, removing the burden for them to individually figure out how to set up data enrichment projects.

Data Enrichment

Data enrichment is curation of data for the purposes of machine learning model development that requires human judgment and intelligence. This can include data preparation, cleaning, labeling, and human review of algorithmic outputs, sometimes performed in real time.

Examples of data enrichment work:

Data preparation, annotation, cleaning, and validation:
Intent recognition, Sentiment tagging, Image labeling

Human review (sometimes referred to as “human in the loop”):
Content moderation, Validating low confidence algorithmic predictions, Speech-to-text error correction

While organizations hoping to adopt these resources may want to similarly engage with their teams to make sure their unique use cases are accounted for, we hope these tested resources will provide a better starting point to incorporate responsible data enrichment practices into their own workflows. Furthermore, to identify where the implemented changes fall short of ideal, we plan to continue developing this work through engagement and convenings. To stay informed, sign up for updates on PAI’s Responsible Sourcing Across the Data Supply Line Workstream page.

This case study details the process by which DeepMind adopted responsible data enrichment sourcing recommendations as organization-wide practice, how challenges that arose during this process were addressed, and the impact on the organization of adopting these recommendations. By sharing this account of how DeepMind did it and why they chose to invest time to do so, we intend to inspire other organizations developing AI to undertake similar efforts. It is our hope that this case study and these resources will empower champions within AI organizations to create positive change.

Implementing Responsible Data Enrichment Practices at an AI Developer: The Example of DeepMind

Executive Summary

Background

Importance of Data Enrichment Workers and Pathways to Improve Working Conditions

Case Study as a Method of Increasing Transparency and Sharing Actionable Guidance

Background on DeepMind’s Motivations

Process and Outcomes of the DeepMind and PAI Collaboration

Changes and Resources Introduced to Support Adoption of Recommendations

Two-Page Data Enrichment Sourcing Guidelines Document

Adapted Review Process

Good Instructions Checklist

Vendor and Platform Feature Comparison Table

Living Wages Spreadsheet

Addressing Practical Complexities That Arose While Finalizing Changes

Assessing Clarity of Guidelines and Rolling Out Changes Organization-Wide

Reactions, Impact, and Next Steps

Response from Research and Development Teams

Key Stakeholders/Leadership Reflections and Motivations

Continued Work for DeepMind

Limitations of Case Study Applicability

Conclusion

Acknowledgements

Appendix A: Initial Discovery Process and Getting Reactions to PAI Responsible Sourcing Recommendations

Sources Cited

  1. Geiger, R. Stuart, et al. “Garbage in, garbage out? Do machine learning application papers in social computing report where human-labeled training data comes from?.” Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency. 2020
  2. Denton, Emily, et al. “On the genealogy of machine learning datasets: A critical history of ImageNet.” Big Data u0026amp; Society 8.2 (2021): 20539517211035955.
  3. Hutchinson, Ben, et al. “Towards accountability for machine learning datasets: Practices from software engineering and infrastructure.” Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency. 2021
  4. Gray, Mary L., and Siddharth Suri. Ghost work: How to stop Silicon Valley from building a new global underclass. Eamon Dolan Books, 2019
  5. Geiger, R. Stuart, et al. “Garbage in, garbage out? Do machine learning application papers in social computing report where human-labeled training data comes from?.” Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency. 2020
Table of Contents
1
2
3
4
5
6
7
8

AI and Job Quality

Insights from Frontline Workers

PAI Staff
Download the Report

Executive Summary

Based on an international study of on-the-job experiences with AI, this report draws from workers’ insights to point the way toward a better future for workplace AI. In addition to identifying common themes among workers’ stories, it provides guidance for key stakeholders who want to make a positive impact. These opportunities for impact can be downloaded individually as audience-specific summaries below.

Opportunities for impact for:

Across industries and around the world, AI is changing work. In the coming years, this rapidly advancing technology has the potential to fundamentally reshape humanity’s relationship with labor. As highlighted by previous Partnership on AI (PAI) research, however, the development and deployment of workplace AI often lacks input from an essential group of experts: the people who directly interact with these systems in their jobs.

Bringing the perspectives of workers into this conversation is both a moral and pragmatic imperative. Despite the direct impact of workplace AI on them, workers rarely have direct influence in AI’s creation or decisions about its implementation. This neglect raises clear concerns about unforeseen or overlooked negative impacts on workers. It also undermines the optimal use of AI from a corporate perspective.

This PAI report, based on an international study of on-the-job experiences with AI, seeks to address this gap. Through journals and interviews, workers in India, sub-Saharan Africa, and the United States shared their stories about workplace AI. From their reflections, PAI identified five common themes:

  1. Executive and managerial decisions shape AI’s impacts on workers, for better and worse. This starts with decisions about business models and operating models, continues through technology acquisitions and implementations, and finally manifests in direct impacts to workers.
  2. Workers have a genuine appreciation for some aspects of AI in their work and how it helps them in their jobs. Their spotlights here point the way to more mutually beneficial approaches to workplace AI.
  3. Workplace AI’s harms are not new or novel — they are repetitions or extensions of harms from earlier technologies and, as such, should be possible to anticipate, mitigate, and eliminate.
  4. Current implementations of AI often serve to reduce workers’ ability to exercise their human skills and talents. Skills like judgment, empathy, and creativity are heavily constrained in these implementations. To the extent that the future of AI is intended to increase humans’ ability to use these talents, the present of AI is sending many workers in the opposite direction.
  5. Empowering workers early in AI development and implementation increases the opportunities to attain the aforementioned benefits and avoid the harms. Workers’ deep experience in their own roles means they should be treated as subject-matter experts throughout the design and implementation process.

In addition, PAI drew from these themes to offer opportunities for impact for the major stakeholders in this space:

  1. AI-implementing companies, who can commit to AI deployments that do not decrease employee job quality.
  2. AI-creating companies, who can center worker well-being and participation in their values, practices, and product designs.
  3. Workers, unions, and worker organizers, who can work to influence and participate in decisions about technology purchases and implementations.
  4. Policymakers, who can shape the environments in which AI products are developed, sold, and implemented.
  5. Investors, who can account for the downside risks posed by practices harmful to workers and the potential value created by worker-friendly technologies.

The actions of each of these groups have the potential to both increase the prosperity enabled by AI technologies and share it more broadly. Together, we can steer AI in a direction that ensures it will benefit workers and society as a whole.

AI and Job Quality

Executive Summary

Introduction

The need for workers’ perspectives on workplace AI

The contributions of this report

Our Approach

Key research questions

Research methods

Site selection

Who we learned from

Participant recruitment

Major Themes and Findings

Theme 1: Executive and managerial decisions shape AI’s impacts on workers, for better and worse

Theme 2: Workers appreciate how some uses of AI have positively changed their jobs

Theme 3: Workplace AI harms repeat, continue, or intensify known possible harms from earlier technologies

Theme 4: Current implementations of AI in work are reducing workers’ opportunities for autonomy, judgment, empathy, and creativity

Theme 5: Empowering workers early in AI development and implementation increases opportunities to implement AI that benefits workers as well as their employers

Opportunities for Impact

Stakeholder 1: AI-implementing companies

Stakeholder Group 2: AI-creating companies

Stakeholder Group 3: Workers, unions, and worker organizers

Stakeholder Group 4: Policymakers

Stakeholder Group 5: Investors

Conclusion

Acknowledgements

Appendix 1: Detailed Site and Technology Descriptions

Appendix 2: Research Methods

Sources Cited

  1. Daniel Zhang et al., “The AI Index 2022 Annual Report” (AI Index Steering Committee, Stanford Institute for Human-Centered AI, Stanford University, March 2022), https://aiindex.stanford.edu/wp-content/uploads/2022/03/2022-AI-Index-Report_Master.pdf.
  2. Michael Chui et al., “Global AI Survey 2021,” Survey (McKinsey u0026amp; Company, December 8, 2021), https://ceros.mckinsey.com/global-ai-survey-2020-a-desktop-3-1/p/1
  3. Jacques Bughin et al., “Artificial Intelligence: The Next Digital Frontier?,” Discussion Paper (McKinsey Global Institute, June 2017), https://www.mckinsey.com/~/media/mckinsey/industries/advanced%20electronics/our%20insights/how%20artificial%20intelligence%20can%20deliver%20real%20value%20to%20companies/mgi-artificial-intelligence-discussion-paper.ashx
  4. Partnership on AI, “Redesigning AI for Shared Prosperity: An Agenda” (Partnership on AI, May 2021), https://partnershiponai.org/paper/redesigning-ai-agenda/
  5. David Autor, David A. Mindell, and Elisabeth B. Reynolds, The Work of the Future: Building Better Jobs in an Age of Intelligent Machines (The MIT Press, 2022), https://doi.org/10.7551/mitpress/14109.001.0001
  6. Daniel Zhang et al., “The AI Index 2022 Annual Report” (AI Index Steering Committee, Stanford Institute for Human-Centered AI, Stanford University, March 2022), https://aiindex.stanford.edu/wp-content/uploads/2022/03/2022-AI-Index-Report_Master.pdf
  7. Lant Pritchett, “The Future of Jobs Is Facing One, Maybe Two, of the Biggest Price Distortions Ever,” Middle East Development Journal 12, no. 1 (January 2, 2020): 131–56, https://doi.org/10.1080/17938120.2020.1714347
  8. James K. Harter, Frank L. Schmidt, and Theodore L. Hayes, “Business-Unit-Level Relationship between Employee Satisfaction, Employee Engagement, and Business Outcomes: A Meta-Analysis,” Journal of Applied Psychology 87, no. 2 (2002): 268–79, https://doi.org/10.1037/0021-9010.87.2.268
  9. Kaoru Ishikawa, What Is Total Quality Control? The Japanese Way, trans. David John Lu (Englewood Cliffs, N.J.: Prentice-Hall, 1985)
  10. Gary P. Pisano, The Development Factory: Unlocking the Potential of Process Innovation (Harvard Business Press, 1997)
  11. Terje Slåtten and Mehmet Mehmetoglu, “Antecedents and Effects of Engaged Frontline Employees: A Study from the Hospitality Industry,” in New Perspectives in Employee Engagement in Human Resources (Emerald Group Publishing, 2015)
  12. Kayhan Tajeddini, Emma Martin, and Levent Altinay, “The Importance of Human-Related Factors on Service Innovation and Performance,” International Journal of Hospitality Management 85 (February 1, 2020): 102431, https://doi.org/10.1016/j.ijhm.2019.102431
  13. Sergio Fernandez and David W. Pitts, “Understanding Employee Motivation to Innovate: Evidence from Front Line Employees in United States Federal Agencies,” Australian Journal of Public Administration 70, no. 2 (2011): 202–22, https://doi.org/10.1111/j.1467-8500.2011.00726.x
  14. Edward P. Lazear, “Compensation and Incentives in the Workplace,” Journal of Economic Perspectives 32, no. 3 (August 2018): 195–214, https://doi.org/10.1257/jep.32.3.195
  15. Joan Robinson, The Economics of Imperfect Competition (Springer, 1969)
  16. José Azar, Ioana Marinescu, and Marshall I. Steinbaum, “Labor Market Concentration,” Working Paper, Working Paper Series (National Bureau of Economic Research, December 2017), https://doi.org/10.3386/w24147
  17. Alan Manning, Monopsony in Motion: Imperfect Competition in Labor Markets, Monopsony in Motion (Princeton University Press, 2013), https://doi.org/10.1515/9781400850679
  18. Caitlin Lustig et al., “Algorithmic Authority: The Ethics, Politics, and Economics of Algorithms That Interpret, Decide, and Manage,” in Proceedings of the 2016 CHI Conference Extended Abstracts on Human Factors in Computing Systems, CHI EA ’16 (New York, NY, USA: Association for Computing Machinery, 2016), 1057–62, https://doi.org/10.1145/2851581.2886426
  19. Aiha Nguyen, “The Constant Boss: Work Under Digital Surveillance” (Data and Society, May 2021), https://datasociety.net/library/the-constant-boss/
  20. Matt Scherer, “Warning: Bossware May Be Hazardous to Your Health” (Center for Democracy u0026amp; Technology, July 2021), https://cdt.org/wp-content/uploads/2021/07/2021-07-29-Warning-Bossware-May-Be-Hazardous-To-Your-Health-Final.pdf
  21. Mary L. Gray and Siddharth Suri, Ghost Work: How to Stop Silicon Valley from Building a New Global Underclass (Houghton Mifflin Harcourt, 2019)
  22. Alexandra Mateescu and Aiha Nguyen, “Algorithmic Management in the Workplace,” Explainer (Data and Society, February 2019), https://datasociety.net/wp-content/uploads/2019/02/DS_Algorithmic_Management_Explainer.pdf
  23. Daniel Schneider and Kristen Harknett, “Schedule Instability and Unpredictability and Worker and Family Health and Wellbeing,” Working Paper (Washington Center for Equitable Growth, September 2016), http://cdn.equitablegrowth.org/wp-content/uploads/2016/09/12135618/091216-WP-Schedule-instability-and-unpredictability.pdf
  24. V.B. Dubal. “Wage Slave or Entrepreneur?: Contesting the Dualism of Legal Worker Identities.” California Law Review 105, no. 1 (2017): 65–123, https://www.jstor.org/stable/24915689
  25. Ramiro Albrieu, ed., Cracking the Future of Work: Automation and Labor Platforms in the Global South, 2021, https://fowigs.net/wp-content/uploads/2021/10/Cracking-the-future-of-work.-Automation-and-labor-platforms-in-the-Global-South-FOWIGS.pdf
  26. Phoebe V. Moore, “OSH and the Future of Work: Benefits and Risks of Artificial Intelligence Tools in Workplaces,” Discussion Paper (European Agency for Safety and Health at Work, 2019), https://osha.europa.eu/en/publications/osh-and-future-work-benefits-and-risks-artificial-intelligence-tools-workplaces
  27. Frank Pasquale, The Black Box Society: The Secret Algorithms That Control Money and Information (Harvard University Press, 2015)
  28. Ifeoma Ajunwa, “The ‘Black Box’ at Work,” Big Data u0026amp; Society 7, no. 2 (July 1, 2020): 2053951720966181, https://doi.org/10.1177/2053951720938093
  29. Isabel Ebert, Isabelle Wildhaber, and Jeremias Adams-Prassl, “Big Data in the Workplace: Privacy Due Diligence as a Human Rights-Based Approach to Employee Privacy Protection,” Big Data u0026amp; Society 8, no. 1 (January 1, 2021): 20539517211013052, https://doi.org/10.1177/20539517211013051
  30. Andrea Dehlendorf and Ryan Gerety, “The Punitive Potential of AI,” in Redesigning AI, Boston Review (MIT Press, 2021), https://bostonreview.net/forum_response/the-punitive-potential-of-ai/
  31. Partnership on AI, “Framework for Promoting Workforce Well-Being in the AI-Integrated Workplace” (Partnership on AI, August 2020), https://partnershiponai.org/paper/workforce-wellbeing/
  32. Karen Hao, “Artificial Intelligence Is Creating a New Colonial World Order,” MIT Technology Review, accessed July 24, 2022, https://www.technologyreview.com/2022/04/19/1049592/artificial-intelligence-colonialism/
  33. Shakir Mohamed, Marie-Therese Png, and William Isaac, “Decolonial AI: Decolonial Theory as Sociotechnical Foresight in Artificial Intelligence,” Philosophy u0026amp; Technology 33 (December 1, 2020), https://doi.org/10.1007/s13347-020-00405-8
  34. Aarathi Krishnan et al., “Decolonial AI Manyfesto,” https://manyfesto.ai/
  35. OECD.AI (2021), powered by EC/OECD (2021). “Database of National AI Policies.” https://oecd.ai/en/dashboards
  36. Kofi Yeboah, “Artificial Intelligence in Sub-Saharan Africa: Ensuring Inclusivity.” (Paradigm Initiative, December 2021), https://paradigmhq.org/report/artificial-intelligence-in-sub-saharan-africa-ensuring-inclusivity/
  37. Adapted from Qualtrics’ employee lifecycle model, “Employee Lifecycle: The 7 Stages Every Employer Must Understand and Improve,” Qualtrics, https://www.qualtrics.com/experience-management/employee/employee-lifecycle/
  38. Mayank Kumar Golpelwar, Global Call Center Employees in India: Work and Life between Globalization and Tradition (Springer, 2015)
  39. Hye Jin Rho, Shawn Fremstad, and Hayley Brown, “A Basic Demographic Profile of Workers in Frontline Industries” (Center for Economic and Policy Research, April 2020), https://cepr.net/wp-content/uploads/2020/04/2020-04-Frontline-Workers.pdf
  40. U.S. Bureau of Labor Statistics. “All Employees, Warehousing and Storage.” FRED, Federal Reserve Bank of St. Louis. FRED, Federal Reserve Bank of St. Louis, July 2022. https://fred.stlouisfed.org/series/CES4349300001
  41. Lee Rainie et al., “AI and Human Enhancement: Americans’ Openness Is Tempered by a Range of Concerns” (Pew Research Center, March 2022), https://www.pewresearch.org/internet/wp-content/uploads/sites/9/2022/03/PS_2022.03.17_AI-HE_REPORT.pdf
  42. James Manyika et al., “Jobs Lost, Jobs Gained: What the Future of Work Will Mean for Jobs, Skills, and Wages” (McKinsey Global Institute, November 28, 2017), https://www.mckinsey.com/featured-insights/future-of-work/jobs-lost-jobs-gained-what-the-future-of-work-will-mean-for-jobs-skills-and-wages
  43. Mary L. Gray and Siddharth Suri, Ghost Work: How to Stop Silicon Valley from Building a New Global Underclass (Houghton Mifflin Harcourt, 2019)
  44. International Labour Office. “Women and Men in the Informal Economy: A Statistical Picture (Third Edition).” International Labour Office, 2018. http://www.ilo.org/wcmsp5/groups/public/u002du002d-dgreports/u002du002d-dcomm/documents/publication/wcms_626831.pdf
  45. International Labour Office. “Women and Men in the Informal Economy: A Statistical Picture (Third Edition).” International Labour Office, 2018. http://www.ilo.org/wcmsp5/groups/public/u002du002d-dgreports/u002du002d-dcomm/documents/publication/wcms_626831.pdf
  46. OECD, and International Labour Organization. “Tackling Vulnerability in the Informal Economy,” 2019. https://www.oecd-ilibrary.org/content/publication/939b7bcd-en
  47. James C. Scott, Seeing like a State: How Certain Schemes to Improve the Human Condition Have Failed, Yale Agrarian Studies (New Haven, Conn.: Yale Univ. Press, 2008)
  48. Reema Nanavaty, Expert interview with Reema Nanavaty, Director of Self Employed Women’s Association (SEWA), July 11, 2022
  49. Paul E. Spector, “Perceived Control by Employees: A Meta-Analysis of Studies Concerning Autonomy and Participation at Work,” Human Relations 39, no. 11 (November 1, 1986): 1005–16, https://doi.org/10.1177/001872678603901104
  50. Henry Ongori, “A Review of the Literature on Employee Turnover,” African Journal of Business Management 1, no. 3 (June 30, 2007): 049–054, https://academicjournals.org/article/article1380537420_Ongori.pdf
  51. See Virginia Doellgast and Sean O’Brady, “Making Call Center Jobs Better: The Relationship between Management Practices and Worker Stress,” June 1, 2020, https://ecommons.cornell.edu/handle/1813/74307 for additional detail and impacts of punitive managerial uses of monitoring technology in call centers, including increased worker stress
  52. Aiha Nguyen, “The Constant Boss: Work Under Digital Surveillance” (Data and Society, May 2021), https://datasociety.net/library/the-constant-boss/
  53. Matt Scherer, “Warning: Bossware May Be Hazardous to Your Health” (Center for Democracy u0026amp; Technology, July 2021), https://cdt.org/wp-content/uploads/2021/07/2021-07-29-Warning-Bossware-May-Be-Hazardous-To-Your-Health-Final.pdf
  54. Mary L. Gray and Siddharth Suri, Ghost Work: How to Stop Silicon Valley from Building a New Global Underclass (Houghton Mifflin Harcourt, 2019)
  55. Alexandra Mateescu and Aiha Nguyen, “Algorithmic Management in the Workplace,” Explainer (Data and Society, February 2019), https://datasociety.net/wp-content/uploads/2019/02/DS_Algorithmic_Management_Explainer.pdf
  56. Andrea Dehlendorf and Ryan Gerety, “The Punitive Potential of AI,” in Redesigning AI, Boston Review (MIT Press, 2021), https://bostonreview.net/forum_response/the-punitive-potential-of-ai/
  57. Human Impact Partners and Warehouse Worker Resource Center, “The Public Health Crisis Hidden in Amazon Warehouses,” January 2021, https://humanimpact.org/wp-content/uploads/2021/01/The-Public-Health-Crisis-Hidden-In-Amazon-Warehouses-HIP-WWRC-01-21.pdf
  58. V.B. Dubal. “Wage Slave or Entrepreneur?: Contesting the Dualism of Legal Worker Identities.” California Law Review 105, no. 1 (2017): 65–123, https://www.jstor.org/stable/24915689
  59. Ramiro Albrieu, ed., Cracking the Future of Work: Automation and Labor Platforms in the Global South, 2021, https://fowigs.net/wp-content/uploads/2021/10/Cracking-the-future-of-work.-Automation-and-labor-platforms-in-the-Global-South-FOWIGS.pdf
  60. Daniel Schneider and Kristen Harknett, “Schedule Instability and Unpredictability and Worker and Family Health and Wellbeing,” Working Paper (Washington Center for Equitable Growth, September 2016), http://cdn.equitablegrowth.org/wp-content/uploads/2016/09/12135618/091216-WP-Schedule-instability-and-unpredictability.pdf
  61. Arvind Narayanan, “How to Recognize AI Snake Oil,” https://www.cs.princeton.edu/~arvindn/talks/MIT-STS-AI-snakeoil.pdf
  62. Frederike Kaltheuner, ed., Fake AI (Meatspace Press, 2021), https://fakeaibook.com
  63. Aiha Nguyen, “The Constant Boss: Work Under Digital Surveillance” (Data and Society, May 2021), https://datasociety.net/library/the-constant-boss/
  64. Strategic Organizing Center, “Primed for Pain,” May 2021, https://thesoc.org/wp-content/uploads/2021/02/PrimedForPain.pdf
  65. Alessandro Delfanti and Bronwyn Frey, “Humanly Extended Automation or the Future of Work Seen through Amazon Patents,” Science, Technology, u0026amp; Human Values 46, no. 3 (May 1, 2021): 655–82, https://doi.org/10.1177/0162243920943665
  66. Phoebe V. Moore, “OSH and the Future of Work: Benefits and Risks of Artificial Intelligence Tools in Workplaces,” Discussion Paper (European Agency for Safety and Health at Work, 2019), https://osha.europa.eu/en/publications/osh-and-future-work-benefits-and-risks-artificial-intelligence-tools-workplaces
  67. Strategic Organizing Center, “Primed for Pain,” May 2021, https://thesoc.org/wp-content/uploads/2021/02/PrimedForPain.pdf
  68. Annette Bernhardt, Lisa Kresge, and Reem Suleiman, “Data and Algorithms at Work: The Case for Worker” (UC Berkeley Labor Center, November 2021), https://laborcenter.berkeley.edu/wp-content/uploads/2021/11/Data-and-Algorithms-at-Work.pdf
  69. Andrea Dehlendorf and Ryan Gerety, “The Punitive Potential of AI,” in Redesigning AI, Boston Review (MIT Press, 2021), https://bostonreview.net/forum_response/the-punitive-potential-of-ai/
  70. Carl Benedikt Frey and Michael A. Osborne, “The Future of Employment: How Susceptible Are Jobs to Computerisation?,” Technological Forecasting and Social Change 114 (January 1, 2017): 254–80, https://doi.org/10.1016/j.techfore.2016.08.019
  71. “These Are the Top 10 Job Skills of Tomorrow – and How Long It Takes to Learn Them,” World Economic Forum, https://www.weforum.org/agenda/2020/10/top-10-work-skills-of-tomorrow-how-long-it-takes-to-learn-them/
  72. Daniel Susskind, “Technological Unemployment,” in The Oxford Handbook of AI Governance, ed. Justin Bullock et al. (Oxford University Press), https://doi.org/10.1093/oxfordhb/9780197579329.013.42
  73. Christopher Mims, “Self-Driving Cars Could Be Decades Away, No Matter What Elon Musk Said,” WSJ, https://www.wsj.com/articles/self-driving-cars-could-be-decades-away-no-matter-what-elon-musk-said-11622865615
  74. Mary L. Gray and Siddharth Suri, Ghost Work: How to Stop Silicon Valley from Building a New Global Underclass (Houghton Mifflin Harcourt, 2019)
  75. Erik Brynjolfsson, “The Turing Trap: The Promise u0026amp; Peril of Human-Like Artificial Intelligence,” January 11, 2022, https://doi.org/10.48550/arXiv.2201.04200
  76. World Economic Forum. “Positive AI Economic Futures.” Insight Report. World Economic Forum, November 2021. https://www.weforum.org/reports/positive-ai-economic-futures/
  77. Nithya Sambasivan and Rajesh Veeraraghavan, “The Deskilling of Domain Expertise in AI Development,” in Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems, CHI ’22 (New York, NY, USA: Association for Computing Machinery, 2022), 1–14, https://doi.org/10.1145/3491102.3517578
  78. Sabrina Genz, Lutz Bellmann, and Britta Matthes, “Do German Works Councils Counter or Foster the Implementation of Digital Technologies?,” Jahrbücher Für Nationalökonomie Und Statistik 239, no. 3 (June 1, 2019): 523–64, https://doi.org/10.1515/jbnst-2017-0160
  79. Alan G. Robinson and Dean M. Schroeder, “The Role of Front-Line Ideas in Lean Performance Improvement,” Quality Management Journal 16, no. 4 (January 1, 2009): 27–40, https://doi.org/10.1080/10686967.2009.11918248
  80. Jeffrey K. Liker, The Toyota Way: 14 Management Principles From the World’s Greatest Manufacturer (McGraw Hill Professional, 2003)
  81. Taiichi Ohno, Toyota Production System: Beyond Large-Scale Production (CRC Press, 1988)
  82. Kayhan Tajeddini, Emma Martin, and Levent Altinay, “The Importance of Human-Related Factors on Service Innovation and Performance,” International Journal of Hospitality Management 85 (February 1, 2020): 102431, https://doi.org/10.1016/j.ijhm.2019.102431
  83. Katherine C. Kellogg, Mark Sendak, and Suresh Balu, “AI on the Front Lines,” MIT Sloan Management Review, May 4, 2022, https://sloanreview.mit.edu/article/ai-on-the-front-lines/
  84. Zeynep Ton, “The Good Jobs Solution,” Harvard Business Review, 2017, 32. https://goodjobsinstitute.org/wp-content/uploads/2018/03/Good-Jobs-Solution-Full-Report.pdf
  85. Abigail Gilbert et al., “Case for Importance: Understanding the Impacts of Technology Adoption on ‘Good Work’” (Institute for the Future of Work, May 2022), https://uploads-ssl.webflow.com/5f57d40eb1c2ef22d8a8ca7e/62a72d3439edd66ed6f79654_IFOW_Case%20for%20Importance.pdf
  86. Daniel Zhang et al., “The AI Index 2022 Annual Report” (AI Index Steering Committee, Stanford Institute for Human-Centered AI, Stanford University, March 2022), https://aiindex.stanford.edu/wp-content/uploads/2022/03/2022-AI-Index-Report_Master.pdf
  87. Julian Posada, “The Future of Work Is Here: Toward a Comprehensive Approach to Artificial Intelligence and Labour,” Ethics of AI in Context, 2020, http://arxiv.org/abs/2007.05843
  88. Jeffrey Brown, “The Role of Attrition in AI’s ‘Diversity Problem’” (Partnership on AI, April 2021), https://partnershiponai.org//wp-content/uploads/dlm_uploads/2022/04/PAI_researchpaper_aftertheoffer.pdf
  89. Tina M Park, “Making AI Inclusive: 4 Guiding Principles for Ethical Engagement” (Partnership on AI, July 2022), https://partnershiponai.org//wp-content/uploads/dlm_uploads/2022/07/PAI_whitepaper_making-ai-inclusive.pdf
  90. Fabio Urbina et al., “Dual Use of Artificial-Intelligence-Powered Drug Discovery,” Nature Machine Intelligence 4, no. 3 (March 2022): 189–91, https://doi.org/10.1038/s42256-022-00465-9
  91. Aarathi Krishnan et al., “Decolonial AI Manyfesto,” accessed July 24, 2022, https://manyfesto.ai/
  92. Lama Nachman, “Beyond the Automation-Only Approach,” in Redesigning AI, Boston Review (MIT Press, 2021), https://bostonreview.net/forum_response/beyond-the-automation-only-approach/
  93. Christina Colclough, “Righting the Wrong: Putting Workers’ Data Rights Firmly on the Table,” in Digital Work in the Planetary Market, –International Development Research Centre Series (MIT Press, 2022), https://idl-bnc-idrc.dspacedirect.org/bitstream/handle/10625/61034/IDL-61034.pdf
  94. Christina Colclough, “When Algorithms Hire and Fire,” International Union Rights 25, no. 3 (2018): 6–7. https://muse.jhu.edu/article/838277/summary
  95. Brishen Rogers, “The Law and Political Economy of Workplace Technological Change,” Harvard Civil Rights-Civil Liberties Law Review 55 (2020): 531
  96. Wilneida Negrón, “Little Tech Is Coming for Workers” (Coworker.org, 2021), https://home.coworker.org/wp-content/uploads/2021/11/Little-Tech-Is-Coming-for-Workers.pdf
  97. Jeremias Adams-Prassl, “What If Your Boss Was an Algorithm? Economic Incentives, Legal Challenges, and the Rise of Artificial Intelligence at Work,” Comparative Labor Law u0026amp; Policy Journal 41 (2021 2019): 123
  98. Daniel Zhang et al., “The AI Index 2022 Annual Report” (AI Index Steering Committee, Stanford Institute for Human-Centered AI, Stanford University, March 2022), https://aiindex.stanford.edu/wp-content/uploads/2022/03/2022-AI-Index-Report_Master.pdf
  99. Kofi Yeboah, “Artificial Intelligence in Sub-Saharan Africa: Ensuring Inclusivity.” (Paradigm Initiative, December 2021), https://paradigmhq.org/report/artificial-intelligence-in-sub-saharan-africa-ensuring-inclusivity/
  100. Fekitamoeloa ‘Utoikamanu, “Closing the Technology Gap in Least Developed Countries,” United Nations (United Nations), accessed July 25, 2022, https://www.un.org/en/chronicle/article/closing-technology-gap-least-developed-countries
  101. Annette Bernhardt, Lisa Kresge, and Reem Suleiman, “Data and Algorithms at Work: The Case for Worker” (UC Berkeley Labor Center, November 2021), https://laborcenter.berkeley.edu/wp-content/uploads/2021/11/Data-and-Algorithms-at-Work.pdf
  102. Allison Levitsky, “California Might Require Employers to Disclose Workplace Surveillance,” Protocol, April 21, 2022, https://www.protocol.com/bulletins/ab-1651-california-workplace-surveillance
  103. “The EU Artificial Intelligence Act,” The AI Act, September 7, 2021, https://artificialintelligenceact.eu/
  104. Daron Acemoglu, Andrea Manera, and Pascual Restrepo, “Does the US Tax Code Favor Automation?,” Working Paper, Working Paper Series (National Bureau of Economic Research, April 2020), https://doi.org/10.3386/w27052
  105. Emmanuel Moss et al., “Assembling Accountability: Algorithmic Impact Assessment for the Public Interest” (Data and Society, June 2021), https://datasociety.net/wp-content/uploads/2021/06/Assembling-Accountability.pdf
  106. Kofi Yeboah, “Artificial Intelligence in Sub-Saharan Africa: Ensuring Inclusivity.” (Paradigm Initiative, December 2021), https://paradigmhq.org/report/artificial-intelligence-in-sub-saharan-africa-ensuring-inclusivity/
  107. Daniel Zhang et al., “The AI Index 2022 Annual Report” (AI Index Steering Committee, Stanford Institute for Human-Centered AI, Stanford University, March 2022), https://aiindex.stanford.edu/wp-content/uploads/2022/03/2022-AI-Index-Report_Master.pdf
  108. Business Roundtable, “Statement on the Purpose of a Corporation,” July 2021, https://s3.amazonaws.com/brt.org/BRT-StatementonthePurposeofaCorporationJuly2021.pdf
  109. Larry Fink, “Larry Fink’s Annual 2022 Letter to CEOs,” accessed May 27, 2022, https://www.blackrock.com/corporate/investor-relations/larry-fink-ceo-letter
  110. Katanga Johnson, “U.S. SEC Chair Provides More Detail on New Disclosure Rules, Treasury Market Reform | Reuters,” https://www.reuters.com/business/sustainable-business/sec-considers-disclosure-mandate-range-climate-metrics-2021-06-23/
  111. “Your Guide to Amazon’s 2022 Shareholder Event,” United for Respect, accessed May 27, 2022, https://united4respect.org/amazon-shareholders/
Table of Contents
1
2
3
4
5
6
7
8
9
10

Making AI Inclusive: 4 Guiding Principles for Ethical Engagement

Tina Park
Download the Paper
Download Recommendation Summary

Introduction

While the concept of “human-centered design” is hardly new to the technology sector, recent years have seen growing efforts to build inclusive artificial intelligence (AI) and machine learning (ML) products. Broadly, inclusive AI/ML refers to algorithmic systems which are created with the active engagement of and input from people who are not on AI/ML development teams. This includes both end users of the systems and non-users who are impacted by the systems.✳“Impacted non-user” refers to people who are impacted by the deployment of an AI/ML system, but are not the direct user or customer of that system. For example, in the case of students in the United Kingdom in 2020 whose A-level grades were determined by an algorithm, the “user” of the algorithmic system is Ofqual, the official exam regulator in England, and the students are “impacted non-users.” To collect this input, practitioners are increasingly turning to engagement practices like user experience (UX) research and participatory design.

Amid rising awareness of structural inequalities in our society, embracing inclusive research and design principles helps signal a commitment to equitable practices. As many proponents have pointed out, it also makes for good business: Understanding the needs of a more diverse set of people expands the market for a given product or service. Once engaged, these people can then further improve an AI/ML product, identifying issues like bias in algorithmic systems.

Despite these benefits, however, there remain significant challenges to greater adoption of inclusive development in the AI/ML field. There are also important opportunities. For AI practitioners, AI ethics researchers, and others interested in learning more about responsible AI, this Partnership on AI (PAI) white paper provides guidance to help better understand and overcome the challenges related to engaging stakeholders in AI/ML development.

Ambiguities around the meaning and goals of “inclusion” present one of the central challenges to AI/ML inclusion efforts. To make the changes needed for a more inclusive AI that centers equity, the field must first find agreement on foundational premises regarding inclusion. Recognizing this, this white paper provides four guiding principles for ethical engagement grounded in best practices:

  1. All participation is a form of labor that should be recognized
  2. Stakeholder engagement must address inherent power asymmetries
  3. Inclusion and participation can be integrated across all stages of the development lifecycle
  4. Inclusion and participation must be integrated to the application of other responsible AI principles

To realize ethical participatory engagement in practice, this white paper also offers three recommendations aligned with these principles for building inclusive AI:

  1. Allocate time and resources to promote inclusive development
  2. Adopt inclusive strategies before development begins
  3. Train towards an integrated understanding of ethics

This white paper’s insights are derived from the research study “Towards An Inclusive AI: Challenges and Opportunities for Public Engagement in AI Development.” That study drew upon discussions with industry experts, a multidisciplinary review of existing research on stakeholder and public engagement, and nearly 70 interviews with AI practitioners and researchers, as well as data scientists, UX researchers, and technologists working on AI and ML projects, over a third of whom were based in areas outside of the US, EU, UK, or Canada. Supplemental interviews with social equity and Diversity, Equity, and Inclusion (DEI) advocates contributed to the development of recommendations for individual practitioners, business team leaders, and the field of AI and ML more broadly.

This white paper does not provide a step-by-step guide for implementing specific participatory practices. It is intended to renew discussions on how to integrate a wider range of insights and experiences into AI/ML technologies, including those of both users and the people impacted (either directly or indirectly) by these technologies. Such conversations — between individuals, inside teams, and within organizations — must be had to spur the changes needed to develop truly inclusive AI.

Making AI Inclusive: 4 Guiding Principles for Ethical Engagement

Introduction

Guiding Principles for Ethical Participatory Engagement

Principle 1: All Participation Is a Form of Labor That Should Be Recognized

Principle 2: Stakeholder Engagement Must Address Inherent Power Asymmetries

Principle 3: Inclusion and Participation Can Be Integrated Across All Stages of the Development Lifecycle

Principle 4: Inclusion and Participation Must Be Integrated to the Application of Other Responsible AI Principles

Recommendations for Ethical Engagement in Practice

Recommendation 1: Allocate Time and Resources to Promote Inclusive Development

Recommendation 2: Adopt Inclusive Development Strategies Before Development Begins

Recommendation 3: Train Towards an Integrated Understanding of Ethics

Conclusion

Acknowledgements

Sources Cited

  1. Jean-Baptiste, A. (2020). Building for Everyone: Expand Your Market with Design Practices from Google’s Product Inclusion Team. John Wiley and Sons, Inc.
  2. Romao, M. (2019, June 27). “A vision for AI: Innovative, Trusted and Inclusive.” Policy@Intel. https://community.intel.com/t5/Blogs/Intel/Policy-Intel/A-vision-for-AI-Innovative-Trusted-and-Inclusive/post/1333103
  3. Zhou, A., Madras, D., Raji, D., Milli, S., Kulynych, B. and Zemel, R. (2020, July 17). “Participatory Approaches to Machine Learning.” (Workshop). International Conference on Machine Learning 2020.
  4. Lewis, J. E., Abdilla, A., Arista, N., Baker, K., Benesiinaabandan, S., Brown, M., ... and Whaanga, H. (2020). Indigenous protocol and artificial intelligence position paper. Indigenous AI. https://www.indigenous-ai.net/position-paper
  5. Costanza-Chock, S. (2020). Design justice: Community-led practices to build the worlds we need. The MIT Press.
  6. Hamraie, A., and Fritsch, K. (2019). “Crip technoscience manifesto.” Catalyst: Feminism, Theory, Technoscience, 5(1), 1-33. https://catalystjournal.org/index.php/catalyst/article/view/29607
  7. Taylor, L. (2017). “What is data justice? The case for connecting digital rights and freedoms globally.” Big Data and Society, 4(2), 2053951717736335. https://doi.org/10.1177/2053951717736335
  8. Benjamin, Ruha. (2019). Race After Technology: Abolitionist Tools for the New Jim Code. Polity. https://www.ruhabenjamin.com/race-after-technology
  9. Hanna, A., Denton, E., Smart, A., and Smith-Loud, J. (2020). “Towards a critical race methodology in algorithmic fairness.” In Proceedings of the 2020 conference on fairness, accountability, and transparency (pp. 501-512). https://arxiv.org/abs/1912.03593
  10. Sloane, M., Moss, E., Awomolo, O. and Forlano, L. (2020). ''Participation is not a design fix for machine learning.'' arXiv. https://arxiv.org/abs/2007.02423
  11. Cifor, M., Garcia, P., Cowan, T.L., Rault, J., Sutherland, T., Chan, A., Rode, J., Hoffmann, A.L., Salehi, N. and Nakamura, L. (2019). “Feminist Data Manifest-No.” Feminist Data Manifest-No. Retrieved October 1, 2020 from https://www.manifestno.com/home
  12. Harrington, C., Erete, S. and Piper, A.M. (2019). “Deconstructing Community-Based Collaborative Design: Towards More Equitable Participatory Design Engagements.” In Proceedings of the ACM on Human-Computer Interaction 3(CSCW):1–25. https://doi.org/10.1145/3359318
  13. Freimuth V.S., Quinn, S.C., Thomas, S.B., Cole, G., Zook, E and Duncan, T. (2001). “African Americans’ Views on Research and the Tuskegee Syphilis Study.” Social Science and Medicine 52(5):797–808. https://doi.org/10.1016/S0277-9536(00)00178-7
  14. George, S., Duran, N. and Norris, K. (2014). “A Systematic Review of Barriers and Facilitators to Minority Research Participation Among African Americans, Latinos, Asian Americans, and Pacific Islanders.” American Journal of Public Health 104(2):e16–31. https://doi.org/10.2105/AJPH.2013.301706
  15. Barabas, C., Doyle, C., Rubinovitz, J.B., and Dinakar, K. (2020). “Studying Up: Reorienting the Study of Algorithmic Fairness around Issues of Power.” In Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency (pp. 167-176).
  16. Harrington, C., Erete, S. and Piper, A.M.. (2019). “Deconstructing Community-Based Collaborative Design: Towards More Equitable Participatory Design Engagements.” In Proceedings of the ACM on Human-Computer Interaction 3(CSCW):1–25. https://dl.acm.org/doi/10.1145/3359318.
  17. Chan, A., Okolo, C. T., Terner, Z., and Wang, A. (2021). “The Limits of Global Inclusion in AI Development.” arXiv. https://arxiv.org/abs/2102.01265
  18. Sanders, E. B. N. (2002). “From user-centered to participatory design approaches.” In Design and the social sciences (pp. 18-25). CRC Press. https://www.taylorfrancis.com/chapters/edit/10.1201/9780203301302-8/user-centered-participatory-design-approaches-elizabeth-sanders
  19. Leslie, D., Katell, M., Aitken, M., Singh, J., Briggs, M., Powell, R., ... and Burr, C. (2022). “Data Justice in Practice: A Guide for Developers.” arXiv. https://arxiv.org/ftp/arxiv/papers/2205/2205.01037.pdf
  20. Zdanowska, S., and Taylor, A. S. (2022). “A study of UX practitioners roles in designing real-world, enterprise ML systems.” In CHI Conference on Human Factors in Computing Systems (pp. 1-15). https://dl.acm.org/doi/abs/10.1145/3491102.3517607
  21. Leslie, D., Katell, M., Aitken, M., Singh, J., Briggs, M., Powell, R., ... and Burr, C. (2022). “Data Justice in Practice: A Guide for Developers.” arXiv. https://arxiv.org/ftp/arxiv/papers/2205/2205.01037.pdf
  22. Saulnier, L., Karamcheti, S., Laurençon, H., Tronchon, L., Wang, T., Sanh, V., Singh, A., Pistilli, G., Luccioni, S., Jernite, Y., Mitchell, M. and Kiela, D. (2022). “Putting Ethical Principles at the Core of the Research Lifecycle.” Hugging Face Blog. Retrieved from https://huggingface.co/blog/ethical-charter-multimodal
  23. Ada Lovelace Institute. (2021). “Participatory data stewardship: A framework for involving people in the use of data.” Ada Lovelace Institute. https://www.adalovelaceinstitute.org/report/participatory-data-stewardship/
  24. Delgado, F., Yang, S., Madaio, M., and Yang, Q. (2021). “Stakeholder Participation in AI: Beyond ‘Add Diverse Stakeholders and Stir.’” arXiv. https://arxiv.org/pdf/2111.01122.pdf
  25. Sloane, M., Moss, E., Awomolo, O. and Forlano, L. (2020). ''Participation is not a design fix for machine learning.'' arXiv. https://arxiv.org/abs/2007.02423
Table of Contents
1
2
3
4
5
6