ABOUT ML (Annotation and Benchmarking on Understanding and Transparency of Machine learning Lifecycles) is a multi-year, multi-stakeholder initiative led by PAI. This initiative aims to bring together a diverse range of perspectives to develop, test, and implement machine learning system documentation practices at scale.
The initiative is an ongoing, iterative process designed to co-evolve with the rapidly advancing field of AI development and deployment. In recognition that documentation is both an artifact and a process, ABOUT ML is structured into an artifact workstream and a process workstream.
Read the ABOUT ML Reference Document here.
Contribute to 2021 work
Our goal for 2021 is to design testable pilots in a multi stakeholder manner. To make this process more tractable, we’ve broken this into two different workstreams. Each button below leads to a key subtask in this process, and we invite you to share your thoughts, comments, and feedback on any that you are interested in.
Help us solve the challenge of how documentation can be created at scale within an organization.
Join the debate on what information stakeholders deserve to know about ML systems, and how that information should be presented.
Deployed Examples of ML Documentation
See real-world deployed examples of ML documentation which can focus on datasets, models, and ML systems. Provide your feedback on these examples as part of ABOUT ML’s public feedback comment process.
To guide ABOUT ML, let the steering committee know what you think of these examples. Which questions are useful? What questions are these examples missing? Is there anything about the format of one of these examples that is effective? Leave a comment with the comment button on the left.
Microsoft’s Face API Transparency Note
Facial recognition is an important and useful technology that can improve efficiency, security, and customer experiences.
This site provides an overview of the FactSheet project, a research effort to foster trust in AI by increasing transparency and enabling governance.
Google, Model Cards
Whether it’s knowing the nutritional content in our food, the conditions of our roads, or a medication’s interaction warnings, we rely on information to make responsible decisions.
Inspired by Model Cards for Model Reporting (Mitchell et al.), we’re providing some accompanying information about the GPT-2 family of models we’re releasing.
IBM, Image Caption Generator
This document is a FactSheet accompanying the Image Caption Generator model on IBM Developer Model Asset eXchange. FactSheets aim at increasing trust in AI services through supplier’s declarations of conformity and this FactSheet documents the process of training the Image Caption Generator model as well as its expected results and…
The ABOUT ML Steering Committee is comprised of around 30 experts, researchers and practitioners recruited from a diverse set of PAI Partner organizations. The Steering Committee guides the process of updating ABOUT ML drafts based on the public comments submitted and new developments in research and practice. They vote to approve new releases by “rough consensus” commonly used by other multi-stakeholder working groups. They convene 1-3 times a year, depending on the volume of proposed changes and velocity of change of research and practice.
To allow for as many diverse perspectives as possible, PAI limits participation in the Steering Committee to up to 2 people per organization, with 1 vote per organization. As needed, PAI will periodically reopen applications to the Steering Committee and recruit more members.
AT&T Research Labs
Privacy & Public Policy Manager
General Manager, Innovation & Growth
Principal Researcher & Manager
Human-AI Collaboration IBM
Leverhulme Centre for The Future of Intelligence
Distinguished Data Scientist
Future of Humanity Institute
Tech Projects Director
Director of AI Research
Senior Counsel, AI & Ethics
Future of Privacy Forum
Corporate VP/Head of Lab
Senior Counsel & Director of Strategy
Future of Privacy Forum
Graduate Research Associate
Tech Policy Lab/University of Washington
AI/ML Engagement Lead
Director, Internet Ethics Program
Markkula Center for Applied Ethics
Data & Society
Research Staff Member
Open Technology Institute
Senior Programme Manager
Centre for Internet & Society
Senior Research Scientist
AI Collaboration Office Sony
Associate Director of Research Partnerships
ADA Lovelace Institute
Senior Principal Researcher
Senior Research Fellow
Leverhulme Center for the Future of Intelligence
Author & Filmmaker
Vision & Image Processing(VIP) Research Group(University of Waterloo)
Jennifer Wortman Vaughan
Senior Principal Researcher
Senior Developer Advocate
As you explore ABOUT ML, we invite you to learn more about this work’s origins and the amazing researchers who helped from the very beginning.
Notably, Hanna Wallach, Meg Mitchell, Jenn Wortman Vaughan, Timnit Gebru, Lassana Magassa, and Jingying Yang were instrumental in the foundations of the work and we thank them for their significant contributions. Francesa Rossi and Kush Varshney, both from IBM, and Eric Horvitz at Microsoft were also key contributors in making this work possible. Please read more about the origins of ABOUT ML and contributors to the project below.