ABOUT ML Resources Library

Overview


ABOUT ML (Annotation and Benchmarking on Understanding and Transparency of Machine learning Lifecycles) is a multi-year, multi-stakeholder initiative aimed at building transparency into the AI development process, industry-wide, through full lifecycle documentation. On this page, you will find the collected outputs of ABOUT ML, a library of resources designed to help organizations and individuals begin implementing transparency at scale. To further increase the usability of these resources, recommended reading plans for different readers are provided below.

Learn more about the origins of ABOUT ML and contributors to the project here.

Recommended Reading Plans

At the foundation of these resources lies the newly revised ABOUT ML Reference Document, which both identifies transparency goals and offers suggestions on how they might be achieved. Using principles provided by the Reference Document and insights about implementation gathered through our research, PAI plans to release additional ML documentation guides, templates, recommendations, and other artifacts. These future artifacts will also be available on this page.

Read the full ABOUT ML Reference Document

 

Recommended Reading Plans for…


ML System Developers/Deployers

ML system developers/deployers are encouraged to do a deep dive exploration of Section 3: Preliminary Synthesized Documentation Suggestions and use it to highlight gaps in their current understanding of both data- and model-related documentation and planning needs. This group will most benefit from further participation in the ABOUT ML effort by engaging with the community in the forthcoming online forum and by testing the efficacy and applicability of templates and specifications to be published in the PLAYBOOK and PILOTS, which will be developed based on use cases as an opportunity to implement ML documentation processes within an organization.


ML System Procurers

ML system procurers might explore Section 2.2: Documentation to Operationalize AI Ethics Goals to get ideas about what concepts to include as requirements for models and data in future requests for proposals relevant to ML systems. Additionally, they could use Section 2.3: Research Themes on Documentation for Transparency to shape conversations with the business owners and requirements writers to further elicit detailed key performance indicators and measures for success for any procured ML systems.


Users of ML System APIs and/or Experienced End Users of ML Systems

Users of ML system APIs and/or experienced end users of ML systems might skim the document and review all of the coral Quick Guides to get a better understanding of how ML concepts are relevant to many of the tools they regularly use. A review of Section 2.1: Demand for Transparency and AI Ethics in ML Systems will provide insight into conditions where it is appropriate to use ML systems. This section also explains how transparency is a foundation for both internal accountability among the developers, deployers, and API users of an ML system and external accountability to customers, impacted non-users, civil society organizations, and policymakers.


Internal Compliance Teams

Internal compliance teams are encouraged to explore Section 4: Current Challenges of Implementing Documentation and use it to shape conversations with developer/deployment teams to find ways to measure compliance throughout the Machine Learning Lifecycle (MLLC).


External Auditors

External auditors could skim Appendix: Compiled List of Documentation Questions and familiarize themselves with high-level concepts as well as tactically operationalized tenets to look for in their determination of whether or not an ML System is well-documented.


Lay Users of ML Systems and/or Members of Low-Income Communities

Lay users of ML systems and/or members of low-income communities might skim the document and review all of the blue “How We Define” boxes in order to get an overarching understanding of the text’s contents. These users are encouraged to continue learning ABOUT ML systems by exploring how they might impact their everyday lives. Additional insights can be gathered from the Glossary section of the ABOUT ML Reference Document.