Conclusion and Acknowledgements
Balancing the risks of use and non-use of demographic data when it comes to fair algorithmic decision-making is ultimately a choice between risk trade-offs. In this paper we sought to provide an overview of some of the most pressing risks, but this is just the start of a much larger conversation. Each of these risks presents a whole suite of research questions that can only be tackled by individuals representing a diverse set of disciplines and industries. During 2022, Partnership on AI (PAI) will consult with partners to help develop a guidebook on how to responsibly collect and use demographic data to inform fair algorithmic decision-making. This guidebook will consider contexts in which it is appropriate to collect demographic data, assess what types of data are necessary, and provide recommendations on how organizations should collect and utilize sensitive information (including considerations around meaningful consent and compensation).
Additional research will seek to explore alternative data governance strategies, namely data cooperatives and data trusts. Open questions guiding our preliminary exploration into this area include: What factors should be considered for the establishment of a data collective? What type of third-party organization would be suitable for establishing and managing a data collective for sensitive data used to train machine learning systems?
If you have any feedback on this white paper or if you would like to receive updates about future demographic data research, please reach out to Sarah Villeneuve (firstname.lastname@example.org) and McKane Andrus (email@example.com).
We are grateful to the diverse set of stakeholders who engaged with us over the last year through one-on-one calls as well as the PAI-hosted FAccT CRAFT workshop and RightsCon session. We are especially grateful to danah boyd (Microsoft Research, Data & Society), Nick Couldry (London School of Economics and Political Science), Emily Denton (Google Research), Ulises A. Mejias (SUNY Oswego), and Nithya Sambasivan (Google Research) for presenting at our CRAFT workshop on a number of the issues detailed in this report and helping us to start a wave of productive conversations.
Many PAI staff members contributed directly and indirectly to this work. In particular, McKane Andrus and Sarah Villeneuve, who led the project and the writing of the white paper, as well as Christine Custis, Tina Park, Hudson Hongo, Neil Uhl, and Penelope Bremner, who provided valuable ideas, advice, and assistance.
While this document reflects the input of individuals representing many PAI Partner organizations, it should not be read as representing the views of any particular organization or individual or any specific PAI Partner.