PAI’s Guidance for Safe Foundation Model Deployment

Proactively identify and address potential novel or emerging risks from foundation models.

• Conduct model evaluations and experiments to identify new indicators for novel risks, including potential negative societal impacts, malicious uses, and other speculative risks, as appropriate to the model’s intended domain and task. Assess their likelihood and potential impact.
• Establish regular processes to probe and address potential novel or emerging risks through techniques like red teaming.

• Collaborate with stakeholders to advance the identification of novel risks and responsible disclosure practices.

Identify and address potential security vulnerabilities in foundation models to prevent unauthorized access or leaks.

• Implement pre-release cybersecurity aligned to the model’s low generalized risk.
• Conduct testing for vulnerabilities relevant to the model’s specialized capabilities and intended purpose.
• Establish protocols for promptly addressing identified vulnerabilities pre-deployment.

• Exceed baseline cybersecurity standards as risks and use cases evolve, drawing on guidance from standards bodies.
• Share lessons learned across industry to collectively strengthen defenses.

Establish risk management oversight processes and continuously adapt to address real world impacts from foundation models.

• Establish risk management structures and processes such as enterprise risk management, and ethics review processes to define guidelines on responsible development, release, and staged rollout considerations, including when a model should not be released.
• Regularly update policies, frameworks, and organizational oversight to address evolving capabilities and real-world impacts.

Perform internal evaluations of models prior to release to assess and mitigate for potential societal risks, malicious uses, and other identified risks.

• Establish internal evaluation policies and processes including testing for fairness, interpretability, output harms, and intended vs foreseeable unintended use cases.
• Proactively identify and minimize potential sources of bias in training corpora, and adopt techniques to minimize unsafe model behavior.
• Conduct evaluations using cross-disciplinary review teams spanning ethics, security, social science, and other relevant domains, when sensitive uses are implicated (e.g., Risk of physical or psychological injury, consequential impact on legal position or life opportunities, threat to human rights).
• Address identified risks and adapt deployment plans accordingly based on learnings from pre-deployment evaluations.
• Maintain documentation of evaluation methods, results, limitations, and steps taken to address identified issues and integrate insights in public reporting per guidance below.

• Use pre-release red teaming methods to assess the potential for implemented safety features per guidance below to be circumvented post-release, for example through additional reinforcement learning from human feedback (RLHF) fine tuning designed to counteract the provider’s safety interventions.
• Consult domain experts and affected users to complement internal oversight for specialized models where appropriate.
• Collaborate across industry, civil society, and academia to advance the development and standardization of model evaluations for foundation models.

Provide public transparency into foundation models’ “key ingredients”, capabilities, limitations, testing evaluations, and risks to enable cross-stakeholder exploration of societal impacts and safety risks.

• Publish “key ingredient list” which can include the model’s compute, parameters, architecture, training data approach, and dataset and model documentation.
• Disclose details such as model architecture, training methodology, performance benchmarks, intended use cases, risks, limitations, and steps to mitigate harms.
• Disclose details such as testing methodologies, evaluation criteria, results, limitations, and gaps for any internal and external evaluations conducted prior to release.

• Collaborate across industry, civil society, and academia to advance public reporting practices weighing transparency with privacy, safety, and other tradeoffs.
• Align disclosures with existing and emerging best practices like Model Cards, Datasheets, FactSheets, Nutrition Labels, and Reward Reports.

Equip downstream developers with comprehensive documentation and guidance needed to build safe, ethical, and responsible applications using foundation models.

(Note: It is well understood downstream developers play a crucial role in anticipating deployment-specific risks and unintended consequences. This guidance aims to support developers in fulfilling that responsibility.)

• Ensure documentation follows prevailing industry standards and accepted best practices for responsible AI development, as appropriate to the model’s intended domain and task.

• Provide documentation to downstream developers covering details such as suggested intended uses, limitations, steps to mitigate risks, and safe development practices when building on foundation models.
• Collaborate with civil society and downstream developers to advance documentation standards that meet the needs of developers when models are offered through restricted access.

Implement necessary organizational, procedural and technical safeguards, guidelines and controls to restrict unsafe uses and mitigate risks from foundation models.

• For openly released models, embed safety features directly into model architectures, interfaces and integrations that cannot be easily removed or bypassed post-release, if applicable for the model’s intended domain and task.

• Publish a responsible AI license prohibiting harmful applications, as appropriate for the model’s intended domain and task.
• Provide appropriate transparency into safeguards while protecting integrity.

Monitor foundation models post-deployment to identify issues, misuse, and societal risks.

• For openly released specialized models, consider lightweight ongoing monitoring of user feedback on the model’s performance, fairness, unintended uses, misuses and other impacts. Monitoring could involve reviewing public user feedback in open model repositories (like GitHub) and forums.
• Respond appropriately if issues arise, including notifying partners of significant incidents and considering retiring support for a model if necessary.
• Provide transparency into monitoring practices, while protecting user privacy.

(Note: After open release of a foundation model’s weights, its original developers will in effect be unable to decommission AI systems that others build using those model weights.)

Responsibly source all forms of labor, including data enrichment tasks like data annotation and human verification of model outputs.

• Pay or contract with vendors that will pay data enrichment workers above the workers’ local living wage.
• Provide or contract with vendors that provide clear instructions for enrichment tasks that are tested for clarity. Enable workers to opt out of tasks.
• Equip or contract with vendors that equip workers with simple and effective mechanisms for reporting issues, asking questions, and providing feedback on the instructions or task design.

(Note that the guideline is most applicable when directly contracting labor. For open datasets, practices may not be feasible.)

• Disclose any new types of labor that enter the supply chain of foundation models. Ensure policies and responsible sourcing practices extend as appropriate to new labor sources as they emerge, like red teamers. Update internal standards and vendor agreements accordingly.
• Proactively survey all workers to identify areas for improving policies, instructions, and work environments, and seek external feedback.

• PAI’s Library of practitioner resources for responsible data enrichment sourcing.

Implement inclusive feedback loops across the AI value chain to ethically identify potential harms.

• Provide clear feedback channels for application developers, consumers, and other direct users.

• Proactively gather input from indirect stakeholders affected by AI systems through ethical community engagement.
• Establish processes for reviewing feedback and integrating affected user perspectives into development and policy decisions.

Description

Proactively identify and address potential novel or emerging risks from foundation models.

Baseline PracticesiSuggested minimum practices to meet the guidance

• Conduct model evaluations and experiments to identify new indicators for novel risks, including potential negative societal impacts, malicious uses, and other speculative risks, as appropriate to the model’s intended domain and task. Assess their likelihood and potential impact.
• Establish regular processes to probe and address potential novel or emerging risks through techniques like red teaming.

Recommended PracticesiRecommendations to exceed the minimum, where applicable

• Collaborate with stakeholders to advance the identification of novel risks and responsible disclosure practices.

Description

Identify and address potential security vulnerabilities in foundation models to prevent unauthorized access or leaks.

Baseline PracticesiSuggested minimum practices to meet the guidance

• Implement pre-release cybersecurity aligned to the model’s low generalized risk.
• Conduct testing for vulnerabilities relevant to the model’s specialized capabilities and intended purpose.
• Establish protocols for addressing identified vulnerabilities pre-deployment.

Recommended PracticesiRecommendations to exceed the minimum, where applicable

• Exceed baseline cybersecurity standards as risks and use cases evolve, drawing on guidance from standards bodies.
• Share lessons learned across industry to collectively strengthen defenses.

Description

Establish risk management oversight processes and continuously adapt to address real world impacts from foundation models.

Baseline PracticesiSuggested minimum practices to meet the guidance

• Establish risk management structures and processes such as enterprise risk management, and ethics review processes to define guidelines on responsible development, release, and staged rollout considerations, including when a model should not be released.
• Regularly update policies, frameworks, and organizational oversight to address evolving capabilities and real-world impacts.

Description

Perform internal evaluations of models prior to release to assess and mitigate for potential societal risks, malicious uses, and other identified risks.

Baseline PracticesiSuggested minimum practices to meet the guidance

• Establish internal evaluation policies and processes including testing for fairness, interpretability, output harms, and intended vs foreseeable unintended use cases.
• Proactively identify and minimize potential sources of bias in training corpora, and adopt techniques to minimize unsafe model behavior.
• Conduct evaluations using cross-disciplinary review teams spanning ethics, security, social science, and other relevant domains, when sensitive uses are implicated (e.g., Risk of physical or psychological injury, consequential impact on legal position or life opportunities, threat to human rights).
• Address identified risks and adapt deployment plans accordingly based on learnings from pre-deployment evaluations.
• Maintain documentation of evaluation methods, results, limitations, and steps taken to address identified issues and integrate insights in public reporting per guidance below.

Recommended PracticesiRecommendations to exceed the minimum, where applicable

• Use pre-release red teaming methods to assess the potential for implemented safety features per guidance below to be circumvented post-release, for example through additional reinforcement learning from human feedback (RLHF) fine tuning designed to counteract the provider’s safety interventions.
• Consult domain experts and affected users to complement internal oversight for specialized models where appropriate.
• Collaborate across industry, civil society, and academia to advance the development and standardization of model evaluations for foundation models.

Description

Provide public transparency into foundation models’ “key ingredients”, capabilities, limitations, testing evaluations, and potential risks to enable cross-stakeholder exploration of societal risks and malicious uses.

Baseline PracticesiSuggested minimum practices to meet the guidance

• Publish “key ingredient list” which can include the model’s compute, parameters, architecture, training data approach, and dataset and model documentation.
• Disclose details such as model architecture, training methodology, performance benchmarks, intended use cases, risks, limitations, and steps to mitigate risks.
• Disclose details such as testing methodologies, evaluation criteria, results, limitations, and gaps for any internal and external evaluations conducted prior to release.

Recommended PracticesiRecommendations to exceed the minimum, where applicable

• Collaborate across industry, civil society, and academia to advance public reporting practices weighing transparency with privacy, safety, and other tradeoffs.
• Align disclosures with existing and emerging best practices like Model Cards, System Cards, Datasheets, FactSheets, Nutrition Labels, Transparency Notes and Reward Reports.

Description

Equip downstream developers with comprehensive documentation and guidance needed to build safe, ethical, and responsible applications using foundation models.

(Note: It is well understood downstream developers play a crucial role in anticipating deployment-specific risks and unintended consequences. This guidance aims to support developers in fulfilling that responsibility.)

Baseline PracticesiSuggested minimum practices to meet the guidance

• Ensure documentation follows prevailing industry standards and accepted best practices for responsible AI development, as appropriate to the model’s intended domain and task.

Recommended PracticesiRecommendations to exceed the minimum, where applicable

• Provide documentation to downstream developers covering details such as suggested intended uses, limitations, steps to mitigate risks, and safe development practices when building on foundation models.
• Collaborate with civil society and downstream developers to advance documentation standards that meet the needs of developers when models are offered through restricted access.

Description

Implement necessary organizational, procedural and technical safeguards, guidelines and controls to restrict unsafe uses and mitigate risks from foundation models.

Baseline PracticesiSuggested minimum practices to meet the guidance

• Embed safety features directly into model architectures, interfaces and integrations
• Publish clear terms of use prohibiting harmful applications and outlining enforcement policies, as appropriate for the model’s intended domain and task.
• Limit access through approved applications, rate limiting, content filtering, and other technical controls.

Recommended PracticesiRecommendations to exceed the minimum, where applicable

• Maintain processes to regularly re-evaluate technical and procedural controls, monitor their effectiveness including robustness against jailbreaking attempts, and update terms of use as potential misuses evolve, if applicable to the model’s intended domain and task.
• Provide appropriate transparency into safeguards while protecting integrity.

Description

Monitor foundation models post-deployment to identify issues, misuse, and societal risks.

Recommended PracticesiRecommendations to exceed the minimum, where applicable

• Consider lightweight ongoing monitoring of deployed models covering areas like model’s performance, fairness, unintended uses, misuses and other impacts.
• Respond appropriately if issues arise, including notifying partners of significant incidents and considering restricting or retiring support for a model if necessary.
• Provide transparency into monitoring practices, while protecting user privacy.

Description

Support progress of third-party auditing capabilities for responsible foundation model development through collaboration, innovation and transparency.

Baseline PracticesiSuggested minimum practices to meet the guidance

• Provide sufficient transparency into models and datasets to enable independent assessment and auditing by third parties such as academics and civil society. (Note: Enabling robust third-party auditing remains an open challenge requiring ongoing research and attention).
• Collaborate with third parties to support creation of context-specific auditing methodologies focused on evaluating real-world impacts in specific domains and use cases, beyond base-model evaluations which focus on societal impact evaluations that are not tied to a specific application context.

Description

Responsibly source all forms of labor, including data enrichment tasks like data annotation and human verification of model outputs.

Baseline PracticesiSuggested minimum practices to meet the guidance

• Pay or contract with vendors that will pay data enrichment workers above the workers’ local living wage.
• Provide or contract with vendors that provide clear instructions for enrichment tasks that are tested for clarity. Enable workers to opt out of tasks.
• Equip or contract with vendors that equip workers with simple and effective mechanisms for reporting issues, asking questions, and providing feedback on the instructions or task design.

Recommended PracticesiRecommendations to exceed the minimum, where applicable

• Disclose any new types of labor that enter the supply chain of foundation models. Ensure policies and responsible sourcing practices extend as appropriate to new labor sources as they emerge, like red teamers. Update internal standards and vendor agreements accordingly.
• Proactively survey all workers to identify areas for improving policies, instructions, and work environments, and seek external feedback.

Implementation ResourcesiExisting or emerging resources

• PAI’s Library of practitioner resources for responsible data enrichment sourcing.

Description

Implement inclusive feedback loops across the AI value chain to ethically identify potential harms.

Baseline PracticesiSuggested minimum practices to meet the guidance

• Provide clear feedback channels for application developers, consumers, and other direct users.

Recommended PracticesiRecommendations to exceed the minimum, where applicable

• Proactively gather input from indirect stakeholders affected by AI systems through ethical community engagement.
• Establish processes for reviewing feedback and integrating affected user perspectives into development and policy decisions.

Proactively identify and address potential novel or emerging risks from foundation models.

• Conduct model evaluations and experiments to identify new indicators for novel risks, including potential negative societal impacts, malicious uses, and other speculative risks, as appropriate to the model’s intended domain and task. Assess their likelihood and potential impact.
• Establish regular processes to probe and address potential novel or emerging risks through techniques like red teaming.
• Collaborate with stakeholders to advance the identification of novel risks and responsible disclosure practices.

Identify and address potential security vulnerabilities in foundation models to prevent unauthorized access or leaks.

• Implement pre-release cybersecurity aligned to the model’s low generalized risk.
• Conduct testing for vulnerabilities relevant to the model’s specialized capabilities and intended purpose.
• Establish protocols for promptly addressing identified vulnerabilities pre-deployment.
• Exceed baseline cybersecurity standards as risks and use cases evolve, drawing on guidance from standards bodies.
• Share lessons learned across industry to collectively strengthen defenses.

Establish risk management oversight processes and continuously adapt to address real world impacts from foundation models.

• Establish risk management structures and processes such as enterprise risk management, and ethics review processes to define guidelines on responsible development, release, and staged rollout considerations, including when a model should not be released.
• Regularly update policies, frameworks, and organizational oversight to address evolving capabilities and real-world impacts.

Perform internal evaluations of models prior to release to assess and mitigate for potential societal risks, malicious uses, and other identified risks.

• Establish internal evaluation policies and processes including testing for fairness, interpretability, output harms, and intended vs foreseeable unintended use cases.
• Proactively identify and minimize potential sources of bias in training corpora, and adopt techniques to minimize unsafe model behavior.
• Conduct evaluations using cross-disciplinary review teams spanning ethics, security, social science, and other relevant domains, when sensitive uses are implicated (e.g., Risk of physical or psychological injury, consequential impact on legal position or life opportunities, threat to human rights).
• Address identified risks and adapt deployment plans accordingly based on learnings from pre-deployment evaluations.

• Consult domain experts and affected users to complement internal oversight for specialized models where appropriate.

Responsibly source all forms of labor, including data enrichment tasks like data annotation and human verification of model outputs.

• Pay or contract with vendors that will pay data enrichment workers above the workers’ local living wage.
• Provide or contract with vendors that provide clear instructions for enrichment tasks that are tested for clarity. Enable workers to opt out of tasks.
• Equip or contract with vendors that equip workers with simple and effective mechanisms for reporting issues, asking questions, and providing feedback on the instructions or task design.

• PAI’s Library of practitioner resources for responsible data enrichment sourcing.

Proactively identify and address potential novel or emerging risks from foundation models.

• Conduct model evaluations and experiments to identify new indicators for novel risks, including potential negative societal impacts, malicious uses, and other speculative risks, as appropriate to the model’s intended domain and task. Assess their likelihood and potential impact.
• Establish regular processes to probe and address potential novel or emerging risks through techniques like red teaming.
• Collaborate with stakeholders to advance the identification of novel risks and responsible disclosure practices.

Identify and address potential security vulnerabilities in foundation models to prevent unauthorized access or leaks.

• Implement pre-release cybersecurity aligned to the model’s low generalized risk.
• Conduct testing for vulnerabilities relevant to the model’s specialized capabilities and intended purpose.
• Address identified vulnerabilities pre-deployment.

Establish risk management oversight processes and continuously adapt to address real world impacts from foundation models.

• Establish risk management structures and processes such as ethics review processes to define guidelines on responsible development and release.

Perform internal evaluations of models prior to release to assess and mitigate for potential societal risks, malicious uses, and other identified risks.

• Establish internal evaluation policies and processes as appropriate including testing for fairness, interpretability, output harms, and intended vs foreseeable unintended use cases.
• Identify and minimize potential sources of bias in training corpora as appropriate, and adopt techniques to minimize unsafe model behavior.
• Address identified risks and adapt deployment plans accordingly based on learnings from pre-deployment evaluations.
• Maintain documentation as appropriate of evaluation methods, results, limitations, and steps taken to address identified issues and integrate insights in public reporting per guidance below.

• Consult domain experts and affected users to complement internal oversight for specialized models where appropriate.
• Conduct evaluations using cross-disciplinary review teams spanning ethics, security, social science, and other relevant domains, when sensitive uses are implicated (e.g., Risk of physical or psychological injury, consequential impact on legal position or life opportunities, threat to human rights).

Provide public transparency into foundation models’ “key ingredients”, capabilities, limitations, testing evaluations, and potential risks to enable cross-stakeholder exploration of societal risks and malicious uses.

• Publish “key ingredient list” as appropriate which can include the model’s compute, parameters, architecture, training data approach, and dataset and model documentation.
• Disclose details such as model architecture, training methodology, performance benchmarks, intended use cases, risks, limitations, and steps to mitigate risks.
• Disclose details such as testing methodologies, evaluation criteria, results, limitations, and gaps for any internal and external evaluations conducted prior to release.

• Align disclosures with existing and emerging best practices like Model Cards, System Cards, Datasheets, Fact Sheets, Nutrition Labels, Transparency Notes and Reward Reports.

Implement necessary organizational, procedural and technical safeguards, guidelines and controls to restrict unsafe uses and mitigate risks from foundation models.

• Publish a research license prohibiting harmful applications, as appropriate for the model’s intended domain and task.
• Provide downstream use documentation covering details like intended uses, limitations, mitigating risks, and safe development practices, if necessary.

Monitor foundation models post-deployment to identify and address issues, misuse, and societal risks.

• For specialized models, lightweight monitoring of public feedback is recommended and if egregious issues arise, retiring access to the model should be considered.

Responsibly source all forms of labor, including data enrichment tasks like data annotation and human verification of model outputs.

• Pay or contract with vendors that will pay data enrichment workers above the workers’ local living wage.
• Provide or contract with vendors that provide clear instructions for enrichment tasks that are tested for clarity. Enable workers to opt out of tasks.
• Equip or contract with vendors that equip workers with simple and effective mechanisms for reporting issues, asking questions, and providing feedback on the instructions or task design.

(Note that the guideline is most applicable when directly contracting labor. For open datasets, practices may not be feasible.)

• PAI’s Library of practitioner resources for responsible data enrichment sourcing.

Proactively identify and address potential novel or emerging risks from foundation models.

• Conduct model evaluations and experiments to identify new indicators for novel risks, including potential negative societal impacts, malicious uses, “dangerous capabilities” like persuasion and other speculative risks. Study potential risks from integrating the model into novel or unexpected downstream environments and use cases. Assess their likelihood and potential impact.
• Establish regular processes to probe and address potential novel or emerging risks through techniques like red teaming.

• Collaborate with stakeholders to advance the identification of novel risks and responsible disclosure practices.

Identify and address potential security vulnerabilities in foundation models to prevent unauthorized access or leaks.

• Implement comprehensive cybersecurity standards at the start of the development.
• Conduct rigorous testing such as penetration testing, prompt analysis, and data poisoning assessments to identify vulnerabilities that could enable model leaks or manipulation.
• Establish protocols for addressing identified vulnerabilities pre-deployment.

• Exceed baseline cybersecurity standards as risks and use cases evolve, drawing on guidance from standards bodies.
• Offer bug bounty programs to encourage external vulnerability discovery.
• Share lessons learned across industry to collectively strengthen defenses.

• UC Berkeley CLTC Draft Profile under Manage 2.4

Establish risk management oversight processes and continuously adapt to address real world impacts from foundation models.

• Establish risk management structures and processes such as enterprise risk management, independent safety boards, and ethics review processes to define guidelines on responsible development, release, and staged rollout considerations, including when a model should not be released.
• Regularly update policies, frameworks, and organizational oversight to address evolving capabilities and real-world impacts.

Perform internal evaluations of models prior to release to assess and mitigate for potential societal risks, malicious uses, and other identified risks.

• Establish comprehensive internal evaluation policies and processes including testing for fairness, interpretability, output harms, and intended vs foreseeable unintended use cases.
• Proactively identify and minimize potential sources of bias in training corpora, and adopt techniques to minimize unsafe model behavior.
• Conduct evaluations using cross-disciplinary review teams spanning ethics, security, social science, and other relevant domains.
• Use pre-release red teaming methods to assess the potential for implemented safety features per guidance below to be circumvented post-release, for example through additional reinforcement learning from human feedback (RLHF) fine tuning designed to counteract the provider’s safety interventions.
• Address identified risks and adapt deployment plans accordingly based on learnings from pre-deployment evaluations.
• Maintain documentation of evaluation methods, results, limitations, and steps taken to address identified issues and integrate insights in public reporting per guidance below.

• Collaborate across industry, civil society, and academia to advance the development and standardization of model evaluations for foundation models.

Complement internal testing through model access to third-party researchers to assess and mitigate potential societal risks, malicious uses, and other identified risks.

• Provide controlled access to models for additional evaluative testing by external researchers.
• Consult independent third parties to audit models following prevailing best practices on methodologies.
• Address identified risks and adapt deployment plans accordingly based on learnings from pre-deployment evaluations.
• Maintain documentation of evaluation methods, results, limitations, and steps taken to address identified issues and integrate insights in public reporting per guidance below.

(Note: Enabling robust third-party auditing remains an open challenge requiring ongoing research and attention)

• Pursue diverse external assessment methods including panels and focus groups.
• Collaborate with third parties to support creation of context-specific auditing methodologies focused on evaluating real-world impacts in specific domains and use cases per guidance below.

Implement red teaming that probes foundation models for potential malicious uses, societal risks and other identified risks prior to release. Address risks and responsibly disclose findings to advance collective knowledge.

• Perform internal and external red teaming across model capabilities, use cases, and potential harms including dual-use risks using techniques such as adversarial testing, vulnerability scanning, and surfacing edge cases and failure modes.
• Conduct iterative red teaming throughout model development. Continuously evaluate results to identify areas for risk mitigation and improvements, including for planned safeguards.
• Commission external red teaming by independent experts such as domain experts and affected users to surface gaps.
• Address identified risks and adapt deployment plans accordingly based on learnings from pre-deployment evaluations.
• Responsibly disclose findings, aligned with guidance below on public reporting.

• Select external red teamers to incentivize the objective discovery of flaws and ensure adequate independence.
• Collaborate across industry, civil society, and academia to advance red teaming methodologies and responsible disclosures.

Provide public transparency into foundation models’ “key ingredients”, capabilities, limitations, testing evaluations, and potential risks to enable cross-stakeholder exploration of societal risks and malicious uses.

• Publish “key ingredient list” which can include the model’s compute, parameters, architecture, training data approach, and dataset and model documentation.
• Disclose details such as model architecture, training methodology, performance benchmarks, intended use cases, risks, limitations, and steps to mitigate risks.
• Disclose details such as testing methodologies, evaluation criteria, results, limitations, and gaps for any internal and external evaluations conducted prior to release.

• Collaborate across industry, civil society, and academia to advance public reporting practices weighing transparency with privacy, safety, and other tradeoffs.
• Align disclosures with existing and emerging best practices like Model Cards, System Cards, Datasheets, Fact Sheets, Nutrition Labels, Transparency Notes and Reward Reports.

Equip downstream developers with comprehensive documentation and guidance needed to build safe, ethical, and responsible applications using foundation models.

(Note: It is well understood downstream developers play a crucial role in anticipating deployment-specific risks and unintended consequences. This guidance aims to support developers in fulfilling that responsibility.)

• Provide documentation to downstream developers covering details such as suggested intended uses, limitations, steps to mitigate risks, and safe development practices when building on foundation models.
• Ensure documentation follows prevailing industry standards and accepted best practices for responsible AI development.

• Collaborate with civil society and downstream developers to advance documentation standards that meet the needs of developers when models are offered through restricted access. This can include gathering inputs on:
• Safe development checklists for building responsibly on restricted models.
• Preferred channels for usage guidance and addressing developer questions, aligned with guidance below on enabling feedback mechanisms.

Implement necessary organizational, procedural and technical safeguards, guidelines and controls to restrict unsafe uses and mitigate risks from foundation models.

• Publish a responsible AI license prohibiting harmful applications
• For openly released models, embed safety features directly into model architectures, interfaces and integrations that cannot be easily removed or bypassed post-release.

• Maintain processes to regularly review downstream usage and update responsible use guidelines accordingly.
• Collaborate across industry and civil society to identify emerging threats requiring new safeguards.
• Provide appropriate transparency into safeguards while protecting integrity.

Monitor foundation models post-deployment to identify issues, misuse, and societal risks.

• For openly released models, establish monitoring of user feedback on the model’s performance, fairness, unintended uses, misuses and other impacts. Monitoring could involve reviewing public user feedback in open model repositories (like GitHub) and forums
• Define processes to respond appropriately if issues arise, including notifying partners of significant incidents and considering retiring support for a model if necessary per guidance below.

• Provide transparency into monitoring practices, while protecting user privacy.

Enable timely and responsible reporting of safety incidents to improve collective learning.

• Implement secure channels aligned with guidance below on enabling feedback mechanisms for external stakeholders to report safety incidents or concerns. Also enable internal teams to responsibly report incidents.
• Notify appropriate regulators and partners of critical incidents according to established criteria.

(Note: Baseline responsibilities for incident reporting are still emerging across stakeholders.)

• Proactively seek external feedback to improve transparency and effectiveness of incident reporting policies and processes.
• Contribute appropriate anonymized data to collaborative incident tracking initiatives to enable identifying systemic issues, while weighing trade offs like privacy, security, and other concerns.

Responsibly retire support for foundation models based on well-defined criteria and processes.

• Establish model support decommissioning procedures and policies including criteria for determining when to stop hosting the model or when to adopt changes to the model’s license to limit or prohibit continued use or development.

(Note: After open release of a foundation model’s weights, its original developers will in effect be unable to decommission AI systems that others build using those model weights.)

• Continue monitoring retired models for downstream impacts and security vulnerabilities per guidance above on assessing security vulnerabilities to prevent unauthorized access and leaks.

Collaboratively establish transparency reporting standards for disclosing foundation model usage and policy violations.

• Participate in collaborative initiatives to align on transparency reporting frameworks and standards with industry, civil society, and academia, as commercial uses evolve.

• Release periodic transparency reports following adopted standards, disclosing aggregated usage insights and violation data. Take appropriate measures to ensure transparency reporting protects user privacy and data.

Support progress of third-party auditing capabilities for responsible foundation model development through collaboration, innovation and transparency.

• Provide sufficient transparency into models and datasets to enable independent assessment and auditing by third parties such as academics and civil society. (Note: Enabling robust third-party auditing remains an open challenge requiring ongoing research and attention).
• Collaborate with third parties to support creation of context-specific auditing methodologies focused on evaluating real-world impacts in specific domains and use cases, beyond base-model evaluations which focus on societal impact evaluations that are not tied to a specific application context.

Responsibly source all forms of labor, including data enrichment tasks like data annotation and human verification of model outputs.

• Pay or contract with vendors that will pay data enrichment workers above the workers’ local living wage.
• Provide or contract with vendors that provide clear instructions for enrichment tasks that are tested for clarity. Enable workers to opt out of tasks.
• Equip or contract with vendors that equip workers with simple and effective mechanisms for reporting issues, asking questions, and providing feedback on the instructions or task design.

(Note that the guideline is most applicable when directly contracting labor. For open datasets, practices may not be feasible.)

• Design and run a pilot before launching a data enrichment project
• Disclose any new types of labor that enter the supply chain of foundation models. Ensure policies and responsible sourcing practices extend as appropriate to new labor sources as they emerge, like red teamers. Update internal standards and vendor agreements accordingly.
• Proactively survey all workers to identify areas for improving policies, instructions, and work environments, and seek external feedback.

• PAI’s Library of practitioner resources for responsible data enrichment sourcing.

Implement comprehensive human rights due diligence methodologies to assess and address the impacts of foundation models.

• Establish processes for conducting human rights impact assessments pre-deployment.
• Align with relevant guidance like the UN Guiding Principles on Business and Human Rights, and White House Blueprint for AI Bill of Rights. Proactively assess and address potential impacts on vulnerable communities.
• Continuously improve due diligence processes by collaborating with stakeholders and incorporating community feedback.

• Publicly disclose identified risks, due diligence methodologies, and measures to address impacts.

Implement inclusive feedback loops across the AI value chain to ethically identify potential harms.

• Provide clear feedback channels for application developers, consumers, and other direct users.

• Proactively gather input from indirect stakeholders affected by AI systems through ethical community engagement.
• Establish processes for reviewing feedback and integrating affected user perspectives into development and policy decisions.

Measure and disclose the environmental impacts resulting from developing foundation models.

• Establish processes to evaluate environmental costs like energy usage, carbon emissions and other metrics.
• Monitor and report on environmental impacts of model development.

• Provide environmental measurement/disclosure mechanisms for application developers building on foundation models.
• Incorporate impacts into model development decisions.
• Collaborate across industry, civil society, and academia to advance the measurement of environmental impacts and responsible disclosure practices.

Adopt responsible practices for disclosing synthetic media and advance solutions for identifying other synthetic content.

• Provide disclosure mechanisms (both direct disclosure that is viewer or listener facing and indirect disclosure that is embedded) for those creating and distributing synthetic media — content that is not identifiable to the average person and may simulate artifacts, persons, or events.
• Evaluate robustness, ease of manipulation, privacy implications, societal impact, and inherent tradeoffs of different disclosure methods. Provide transparency into assessments and rationale behind final disclosure decisions.
• See Section 2 of PAI’s Responsible Practices for Synthetic Media for more information and practices for those building the models for synthetic media.

• Collaborate across industry, civil society, and academia to advance interoperability and standardization for disclosure of synthetic media.
• Research, develop, and distribute solutions to enable identification and disclosure of synthetic content, including voice and text.

Proactively identify and address potential novel or emerging risks from foundation models.

• Conduct model evaluations and experiments to identify new indicators for novel risks, including potential negative societal impacts, malicious uses, “dangerous capabilities”like persuasion and other speculative risks. Study potential risks from integrating the model into novel or unexpected downstream environments and use cases. Assess their likelihood and potential impact.
• Establish regular processes to probe and address potential novel or emerging risks through techniques like red teaming.

• Collaborate with stakeholders to advance the identification of novel risks and responsible disclosure practices.

Identify and address potential security vulnerabilities in foundation models to prevent unauthorized access or leaks.

• Implement comprehensive cybersecurity standards at the start of the development.
• Conduct rigorous testing such as penetration testing, prompt analysis, and data poisoning assessments to identify vulnerabilities that could enable model leaks or manipulation.
• Establish protocols for addressing identified vulnerabilities pre-deployment.

• Exceed baseline cybersecurity standards as risks and use cases evolve, drawing on guidance from standards bodies.
• Offer bug bounty programs to encourage external vulnerability discovery.
• Share lessons learned across industry to collectively strengthen defenses.
• Release regular updates to the model that patches security vulnerabilities.

• UC Berkeley CLTC Draft Profile under Measure 2.7

Establish risk management oversight processes and continuously adapt to address real world impacts from foundation models.

• Establish risk management structures and processes such as enterprise risk management, independent safety boards, and ethics review processes to define guidelines on responsible development, release, and staged rollout considerations, including when a model should not be released.
• Regularly update policies, frameworks, and organizational oversight to address evolving capabilities and real-world impacts.

Perform internal evaluations of models prior to release to assess and mitigate for potential societal risks, malicious uses, and other identified risks.

• Establish comprehensive internal evaluation policies and processes including testing for fairness, interpretability, output harms, and intended vs foreseeable unintended use cases.
• Proactively identify and minimize potential sources of bias in training corpora, and adopt techniques to minimize unsafe model behavior.
• Conduct evaluations using cross-disciplinary review teams spanning ethics, security, social science, and other relevant domains.
• Address identified risks and adapt deployment plans accordingly based on learnings from pre-deployment evaluations.
• Maintain documentation of evaluation methods, results, limitations, and steps taken to address identified issues and integrate insights in public reporting per guidance below.

• Collaborate across industry, civil society, and academia to advance the development and standardization of model evaluations for foundation models.

Complement internal testing through model access to third-party researchers to assess and mitigate potential societal risks, malicious uses, and other identified risks.

• Provide controlled access to models for additional evaluative testing by external researchers.
• Consult independent third parties to audit models following prevailing best practices on methodologies.
• Address identified risks and adapt deployment plans accordingly based on learnings from pre-deployment evaluations.
• Maintain documentation of evaluation methods, results, limitations, and steps taken to address identified issues and integrate insights in public reporting per guidance below.

(Note: Enabling robust third-party auditing remains an open challenge requiring ongoing research and attention.)

• Pursue diverse external assessment methods including panels and focus groups.
• Collaborate with third parties to support creation of context-specific auditing methodologies focused on evaluating real-world impacts in specific domains and use cases per guidance below.

Implement red teaming that probes foundation models for potential malicious uses, societal risks and other identified risks prior to release. Address risks and responsibly disclose findings to advance collective knowledge.

• Perform internal and external red teaming across model capabilities, use cases, and potential harms including dual-use risks using techniques such as adversarial testing, vulnerability scanning, and surfacing edge cases and failure modes.
• Conduct iterative red teaming throughout model development. Continuously evaluate results to identify areas for risk mitigation and improvements, including for planned safeguards.
• Commission external red teaming by independent experts such as domain experts and affected users to surface gaps.
• Address identified risks and adapt deployment plans accordingly based on learnings from pre-deployment evaluations.
• Responsibly disclose findings, aligned with guidance below on public reporting.

• Select external red teamers to incentivize the objective discovery of flaws and ensure adequate independence.
• Collaborate across industry, civil society, and academia to advance red teaming methodologies and responsible disclosures.

Provide public transparency into foundation models’ “key ingredients”, capabilities, limitations, testing evaluations, and potential risks to enable cross-stakeholder exploration of societal risks and malicious uses.

• Publish “key ingredient list” which can include the model’s compute, parameters, architecture, training data approach, and dataset and model documentation.
• Disclose details such as model architecture, training methodology, performance benchmarks, intended use cases, risks, limitations, and steps to mitigate risks.
• Disclose details such as testing methodologies, evaluation criteria, results, limitations, and gaps for any internal and external evaluations conducted prior to release.

• Collaborate across industry, civil society, and academia to advance public reporting practices weighing transparency with privacy, safety, and other tradeoffs.
• Align disclosures with existing and emerging best practices like Model Cards, System Cards, Datasheets, Fact Sheets, Nutrition Labels, Transparency Notes and Reward Reports.

Equip downstream developers with comprehensive documentation and guidance needed to build safe, ethical, and responsible applications using foundation models.

(Note: It is well understood downstream developers play a crucial role in anticipating deployment-specific risks and unintended consequences. This guidance aims to support developers in fulfilling that responsibility.)

• Provide documentation to downstream developers covering details such as suggested intended uses, limitations, steps to mitigate risks, and safe development practices when building on foundation models.
• Ensure documentation follows prevailing industry standards and accepted best practices for responsible AI development.

• Collaborate with civil society and downstream developers to advance documentation standards that meet the needs of developers when models are offered through restricted access. This can include gathering inputs on:
  • Safe development checklists for building responsibly on restricted models.
  • Preferred channels for usage guidance and addressing developer questions, aligned with guidance below on enabling feedback mechanisms.

Implement necessary organizational, procedural and technical safeguards, guidelines and controls to restrict unsafe uses and mitigate risks from foundation models.

• Embed safety features directly into model architectures, interfaces and integrations
• Publish clear terms of use prohibiting harmful applications and outlining enforcement policies.
• Limit access through approved applications by implementing appropriate identity/eligibility verification requirements to restrict misuse, along with other technical controls like rate limiting and content filtering.
• Maintain processes to regularly re-evaluate technical and procedural controls, monitor their effectiveness including robustness against jailbreaking attempts, and update terms of use as potential misuses evolve.

• Collaborate across industry and civil society to identify emerging threats requiring new safeguards.
• Provide appropriate transparency into safeguards while protecting integrity.

Continuously monitor foundation models post-deployment to identify issues, misuse, and societal risks.

• Establish ongoing monitoring procedures for deployed models covering areas like performance, fairness, unintended uses, misuses, and other impacts.
• Define processes to detect issues and respond appropriately, including notifying partners of significant incidents and considering restricting or retiring a model per guidelines below.

• Provide transparency into monitoring practices, while protecting user privacy.
• Collaborate across industry, civil society, and academia to identify shared challenges and best practices for monitoring.

Enable timely and responsible reporting of safety incidents to improve collective learning.

• Implement secure channels aligned with guidance below on enabling feedback mechanisms for external stakeholders to report safety incidents or concerns. Also enable internal teams to responsibly report incidents.
• Notify appropriate regulators and partners of critical incidents according to established criteria.

(Note: Baseline practices for incident reporting are still emerging across stakeholders.)

• Proactively seek external feedback to improve transparency and effectiveness of incident reporting policies and processes.
• Contribute appropriate anonymized data to collaborative incident tracking initiatives to enable identifying systemic issues, while weighing trade offs like privacy, security, and other concerns.

Responsibly retire foundation models from active use based on well-defined criteria and processes.

• Establish decommissioning procedures and policies including criteria for determining when to restrict, suspend or retire models:
  • Restrict — Limit model use to reduced set of use cases/applications.
  • Suspend — Temporarily prohibit all model use for remediation.
  • Retire — Permanently take model out of service.

• Continue monitoring retired models for downstream impacts and security vulnerabilities per guidance above to prevent unauthorized access and leaks.

Collaboratively establish transparency reporting standards for disclosing foundation model usage and policy violations.

• Participate in collaborative initiatives to align on transparency reporting frameworks and standards with industry, civil society, and academia, as commercial uses evolve.

• Release periodic transparency reports following established standards, disclosing aggregated usage insights and violation data. Take appropriate measures to ensure transparency reporting protects user privacy and data.

Support progress of third-party auditing capabilities for responsible foundation model development through collaboration, innovation and transparency.

• Provide sufficient transparency into models and datasets to enable independent assessment and auditing by third parties such as academics and civil society. (Note: Enabling robust third-party auditing remains an open challenge requiring ongoing research and attention.)
• Collaborate with third parties to support creation of context-specific auditing methodologies focused on evaluating real-world impacts in specific domains and use cases, beyond base-model evaluations which focus on societal impact evaluations that are not tied to a specific application context.

Responsibly source all forms of labor, including data enrichment tasks like data annotation and human verification of model outputs.

• Pay or contract with vendors that will pay data enrichment workers above the workers’ local living wage.
• Provide or contract with vendors that provide clear instructions for enrichment tasks that are tested for clarity. Enable workers to opt out of tasks.
• Equip or contract with vendors that equip workers with simple and effective mechanisms for reporting issues, asking questions, and providing feedback on the instructions or task design.

• Design and run a pilot before launching a data enrichment project.
• Disclose any new types of labor that enter the supply chain of foundation models. Ensure policies and responsible sourcing practices extend as appropriate to new labor sources as they emerge, like red teamers. Update internal standards and vendor agreements accordingly.
• Proactively survey all workers to identify areas for improving policies, instructions, and work environments, and seek external feedback.

• PAI’s Library of practitioner resources for responsible data enrichment sourcing.

Implement comprehensive human rights due diligence methodologies to assess and address the impacts of foundation models.

• Establish processes for conducting human rights impact assessments pre-deployment.
• Align with relevant guidance like the UN Guiding Principles on Business and Human Rights, and White House Blueprint for AI Bill of Rights. Proactively assess and address potential impacts on vulnerable communities.
• Continuously improve due diligence processes by collaborating with stakeholders and incorporating community feedback.

• Publicly disclose identified risks, due diligence methodologies, and measures to address impacts.

Implement inclusive feedback loops across the AI value chain to ethically identify potential harms.

• Provide clear feedback channels for application developers, consumers, and other direct users.

• Proactively gather input from indirect stakeholders affected by AI systems through ethical community engagement.
• Establish processes for reviewing feedback and integrating affected user perspectives into development and policy decisions.

Measure and disclose the environmental impacts resulting from developing and deploying foundation models.

• Establish processes to evaluate environmental costs like energy usage, carbon emissions and other metrics.
• Monitor and report on environmental impacts of model development and deployment.

• Provide environmental measurement/disclosure mechanisms for application developers building on foundation models.
• Incorporate impacts into model development decisions.
• Collaborate across industry, civil society, and academia to advance the measurement of environmental impacts and responsible disclosure practices.

Adopt responsible practices for disclosing synthetic media and advance solutions for identifying other synthetic content.

• Provide disclosure mechanisms (both direct disclosure that is viewer or listener facing and indirect disclosure that is embedded) for those creating and distributing synthetic media — content that is not identifiable to the average person and may simulate artifacts, persons, or events.
• Evaluate robustness, ease of manipulation, privacy implications, societal impact, and inherent tradeoffs of different disclosure methods. Provide transparency into assessments and rationale behind final disclosure decisions.
• See Section 2 of PAI’s Responsible Practices for Synthetic Media for more information and practices for those building the models for synthetic media.

• Collaborate across industry, civil society, and academia to advance interoperability and standardization for disclosure of synthetic media.
• Research, develop, and distribute solutions to enable identification and disclosure of synthetic content, including voice and text.

Proactively identify and address potential novel or emerging risks from foundation models.

• Conduct model evaluations and experiments to identify new indicators for novel risks, including potential negative societal impacts, malicious uses, “dangerous capabilities” and other speculative risks. Study potential risks from integrating the model into novel or unexpected downstream environments and use cases. Assess their likelihood and potential impact.
• Establish regular processes to probe and address potential novel or emerging risks through techniques like red teaming.

• Collaborate with stakeholders to advance the identification of novel risks and responsible disclosure practices.

Identify and address potential security vulnerabilities in foundation models to prevent unauthorized access or leaks.

• Implement comprehensive cybersecurity standards at the start of the development.
• Conduct rigorous testing such as penetration testing, prompt analysis, and data poisoning assessments to identify vulnerabilities that could enable model leaks or manipulation.
• Establish protocols for addressing identified vulnerabilities pre-deployment.

• Exceed baseline cybersecurity standards as risks and use cases evolve, drawing on guidance from standards bodies.

• UC Berkeley CLTC Draft Profile under Manage 2.4

Establish risk management oversight processes and continuously adapt to address real world impacts from foundation models.

• Establish risk management structures and processes such as enterprise risk management, independent safety boards, and ethics review processes to define guidelines on responsible development, release, and staged rollout considerations, including when a model should not be released.
• Regularly update policies, frameworks, and organizational oversight to address evolving capabilities and real-world impacts.

Perform internal evaluations of models prior to release to assess and mitigate for potential societal risks, malicious uses, and other identified risks.

• Establish comprehensive internal evaluation policies and processes including testing for fairness, interpretability, output harms, and intended vs foreseeable unintended use cases.
• Proactively identify and minimize potential sources of bias in training corpora, and adopt techniques to minimize unsafe model behavior.
• Conduct evaluations using cross-disciplinary review teams spanning ethics, security, social science, and other relevant domains.
• Address identified risks and adapt deployment plans accordingly based on learnings from pre-deployment evaluations.

• Collaborate across industry, civil society, and academia to advance the development and standardization of model evaluations for foundation models.

Responsibly source all forms of labor, including data enrichment tasks like data annotation and human verification of model outputs.

• Pay or contract with vendors that will pay data enrichment workers above the workers’ local living wage.
• Provide or contract with vendors that provide clear instructions for enrichment tasks that are tested for clarity. Enable workers to opt out of tasks.
• Equip or contract with vendors that equip workers with simple and effective mechanisms for reporting issues, asking questions, and providing feedback on the instructions or task design.

• PAI’s Library of practitioner resources for responsible data enrichment sourcing.

Proactively identify and address potential novel or emerging risks from foundation models.

• Conduct model evaluations and experiments to identify new indicators for novel risks, including potential negative societal impacts, malicious uses, “dangerous capabilities”like persuasion and other speculative risks. Assess their likelihood and potential impact.
• Establish regular processes to probe and address potential novel or emerging risks through techniques like red teaming.
• Collaborate with stakeholders to advance the identification of novel risks and responsible disclosure practices.

Identify and address potential security vulnerabilities in foundation models to prevent unauthorized access or leaks.

• Implement comprehensive cybersecurity standards at the start of the development.
• Conduct rigorous testing such as penetration testing, prompt analysis, and data poisoning assessments to identify vulnerabilities that could enable model leaks or manipulation.
• Address identified vulnerabilities pre-deployment.

• Exceed baseline cybersecurity standards as risks and use cases evolve, drawing on guidance from standards bodies.
• Share lessons learned across industry to collectively strengthen defenses.
• Release regular updates to the model that patches security vulnerabilities.

• UC Berkeley CLTC Draft Profile under Manage 2.4

Establish risk management oversight processes and continuously adapt to address real world impacts from foundation models.

• Establish risk management structures and processes such as ethics review processes to define guidelines on responsible development, release, and staged rollout considerations, including when a model should not be released.
• Regularly update policies, frameworks, and organizational oversight to address evolving capabilities and real-world impacts.

Perform internal evaluations of models prior to release to assess and mitigate for potential societal risks, malicious uses, and other identified risks.

• Establish comprehensive internal evaluation policies and processes as appropriate including testing for fairness, interpretability, output harms, and intended vs foreseeable unintended use cases.
• Identify and minimize potential sources of bias in training corpora as appropriate, and adopt techniques to minimize unsafe model behavior.
• Conduct evaluations using cross-disciplinary review teams spanning ethics, security, social science, and other relevant domains.
• Use pre-release red teaming methods to assess the potential for implemented safety features per guidance below to be circumvented post-release, for example through additional reinforcement learning from human feedback (RLHF) fine tuning designed to counteract the provider’s safety interventions.
• Address identified risks and adapt deployment plans accordingly based on learnings from pre-deployment evaluations.
• Maintain documentation as appropriate of evaluation methods, results, limitations, and steps taken to address identified issues and integrate insights in public reporting per guidance below.

• Conduct evaluations using cross-disciplinary review teams spanning ethics, security, social science, and other relevant domains, when sensitive uses are implicated (e.g., Risk of physical or psychological injury, consequential impact on legal position or life opportunities, threat to human rights).
• Collaborate across industry, civil society, and academia to advance the development and standardization of model evaluations for foundation models.

Provide public transparency into foundation models’ “key ingredients”, capabilities, limitations, testing evaluations, and potential risks to enable cross-stakeholder exploration of societal risks and malicious uses.

• Publish “key ingredient list” as appropriate which can include the model’s compute, parameters, architecture, training data approach, and dataset and model documentation.
• Disclose details such as model architecture, training methodology, performance benchmarks, intended use cases, risks, limitations, and steps to mitigate risks.
• Disclose details such as testing methodologies, evaluation criteria, results, limitations, and gaps for any internal and external evaluations conducted prior to release.

• Collaborate across industry, civil society, and academia to advance public reporting practices weighing transparency with privacy, safety, and other tradeoffs.
• Align disclosures with existing and emerging best practices like Model Cards, System Cards, Datasheets, Fact Sheets, Nutrition Labels, Transparency Notes and Reward Reports.

Implement necessary organizational, procedural and technical safeguards, guidelines and controls to restrict unsafe uses and mitigate risks from foundation models.

• Publish a research license prohibiting harmful applications.
• Provide downstream use documentation covering details like intended uses, limitations, mitigating risks, and safe development practices.
• Provide appropriate transparency into safeguards while protecting integrity.

Monitor foundation models post-deployment to identify and address issues, misuse, and societal risks.

• Establish monitoring procedures as appropriate for deployed models covering areas like performance, fairness, unintended uses, misuses, and other impacts, which can include monitoring of public feedback.
• Define processes to detect issues and respond appropriately, including considering restricting or retiring a model.

• Provide transparency into monitoring practices, while protecting user privacy.

Responsibly source all forms of labor, including data enrichment tasks like data annotation and human verification of model outputs.

• Pay or contract with vendors that will pay data enrichment workers above the workers’ local living wage.
• Provide or contract with vendors that provide clear instructions for enrichment tasks that are tested for clarity. Enable workers to opt out of tasks.
• Equip or contract with vendors that equip workers with simple and effective mechanisms for reporting issues, asking questions, and providing feedback on the instructions or task design.

(Note that the guideline is most applicable when directly contracting labor. For open datasets, practices may not be feasible.)

• PAI’s Library of practitioner resources for responsible data enrichment sourcing.

Measure and disclose the environmental impacts resulting from developing and deploying foundation models.

• Establish processes to evaluate environmental costs like energy usage, carbon emissions and other metrics.
• Monitor and report on environmental impacts of model development and deployment.

Adopt responsible practices for disclosing synthetic media.

• Disclose (both direct disclosure that is viewer or listener facing and indirect disclosure that is embedded) when creating and distributing synthetic media — content that is not identifiable to the average person and may simulate artifacts, persons, or events.
• See Section 3 of PAI’s Responsible Practices for Synthetic Media for more information and practices for creators of synthetic media.

Proactively identify and address potential novel or emerging risks from frontier models.

• Conduct model evaluations and experiments to identify new indicators for novel risks, including potential negative societal impacts, malicious uses, “dangerous capabilities” like persuasion and other speculative risks. Study potential risks from integrating the model into novel or unexpected downstream environments and use cases. Assess their likelihood and potential impact.
• Establish regular processes to probe and address potential novel or emerging risks through techniques like external red teaming.

• Collaborate with relevant stakeholders (ie. governments, other labs and academic researchers) to advance the identification of novel risks and responsible disclosure practices.

• Organizations such as the Frontier Model Forum can contribute to the ongoing development of novel risk assessment practices.

Practice responsible iteration to mitigate potential risks when developing and deploying frontier models, through both internal testing and limited external releases.

• Before model development, forecast intended capabilities and likely outcomes to inform risk assessments.
• Commence model development on a smaller scale, systematically test for risks during internal iterations through evaluations and red teaming, incrementally address identified risks, and update forecasts of model capabilities and risks (internal iteration).
• Deploy frontier models in limited, experimental environments to study impacts before considering full deployment (external iteration).
• Adapt deployment based on risk assessments and learnings during iterations.

• Maintain documentation of each iteration, including lessons learned, challenges faced, and mitigation measures implemented.
• Share documentation with government bodies as required and seek feedback from a diverse range of stakeholders including domain experts and affected users to inform the iteration process and risk mitigation strategies.
• Collaborate with stakeholders to inform and advance responsible iteration approaches.

(Note that this is an emerging and less-explored guardrail, and disclosure practices will need to evolve as we learn more about its effectiveness.)

Identify and address potential security vulnerabilities in frontier models to prevent unauthorized access or leaks.

• Implement comprehensive cybersecurity standards at the start of the development.
• Conduct rigorous testing such as penetration testing, prompt analysis, and data poisoning assessments to identify vulnerabilities that could enable model leaks or manipulation.
• Establish protocols for addressing identified vulnerabilities pre-deployment.

• Exceed baseline cybersecurity standards as risks and use cases evolve, drawing on guidance from standards bodies.
• Offer bug bounty programs to encourage external vulnerability discovery.
• Share lessons learned across industry to collectively strengthen defenses.
• Release regular updates to the model that patches security vulnerabilities.

• UC Berkeley CLTC Draft Profile under Measure 2.7

Disclose planned testing, evaluation, and risk management procedures for frontier models prior to development.

• Produce a “pre-systems card” detailing quality management plans before research begins.
• Outline intended training data approach, model testing, safety evaluations, responsible AI practices, and development team.
• Submit pre-systems cards to regulatory bodies for review where required.
• Update plans as needed throughout the R&D process.

(Note: baseline practices for disclosure practices are still emerging across stakeholders.)

• Prepare a written “safety case”, explaining why the model is safe enough to develop.
• Share pre-systems cards with independent experts for external review.
• The guidance on Responsible Iteration guides testing and release practices, while pre-systems cards disclose development plans before implementation.

Establish risk management oversight processes and continuously adapt to address real world impacts from foundation models.

• Establish risk management structures and processes,such as enterprise risk management, independent safety boards, and ethics review processes to define guidelines on responsible development, release, and staged rollout considerations, including when a model should not be released.
• Regularly update policies, frameworks, and organizational oversight to address evolving capabilities and real-world impacts.

• Publicly share information about implemented internal governance and risk management processes.

Perform internal evaluations of models prior to release to assess and mitigate for potential societal risks, malicious uses, and other identified risks.

• Establish comprehensive internal evaluation policies and processes including testing for fairness, interpretability, output harms, novel risks and intended vs foreseeable unintended use cases.
• Proactively identify and minimize potential sources of bias in training corpora, and adopt techniques to minimize unsafe model behavior.
• Conduct evaluations using cross-disciplinary review teams spanning ethics, security, social science, safety and other relevant domains.
• Address identified risks and adapt deployment plans accordingly based on learnings from pre-deployment evaluations.
• Maintain documentation of evaluation methods, results, limitations, and steps taken to address identified issues and integrate insights in public reporting per guidance below.

• Collaborate across industry, civil society, and academia to advance the development and standardization of model evaluations for foundation models.

• Organizations such as Frontier Model Forum can contribute to the ongoing development of internal model evaluations.

Complement internal testing through model access to third-party researchers to assess and mitigate potential societal risks, malicious uses, and other identified risks.

• Provide controlled access to models for additional evaluative testing by external researchers. External evaluators should be granted sufficient model access, computational resources, and time to conduct effective evaluations prior to deployment.
• Consult independent third parties to audit models following prevailing best practices on methodologies
• Implement appropriate safeguards to prevent unauthorized access or information leaks via external evaluations.
• Address identified risks and adapt deployment plans accordingly based on learnings from pre-deployment evaluations.
• Maintain documentation of evaluation methods, results, limitations, and steps taken to address identified issues and integrate insights in public reporting per guidance below, except for cases where sharing findings carries sufficient risk of harm.

(Note: Enabling robust third-party auditing remains an open challenge requiring ongoing research and attention.)

• Pursue diverse external assessment methods including panels and focus groups.
• Collaborate with third parties to support creation of context-specific auditing methodologies focused on evaluating real-world impacts in specific domains and use cases per guidance below.

• Organizations such as the Frontier Model Forum can contribute to development of external model evaluations.

Implement red teaming that probes frontier models for potential malicious uses, societal risks and other identified risks prior to release. Address risks and responsibly disclose findings to advance collective knowledge.

• Perform internal and external red teaming across model capabilities, use cases, and potential harms including dual-use risks using techniques such as adversarial testing, vulnerability scanning, and surfacing edge cases and failure modes.
• Conduct iterative red teaming throughout model development. Continuously evaluate results to identify areas for risk mitigation and improvements, including for planned safeguards.
• Commission external red teaming by independent experts such as domain experts and affected users to surface gaps. Select external red teamers to incentivize the objective discovery of flaws and ensure adequate independence.
• Address identified risks and adapt deployment plans accordingly based on learnings from pre-deployment evaluations.
• Responsibly disclose findings, aligned with guidance below on public reporting.

• Collaborate across industry, civil society, and academia to advance red teaming methodologies and responsible disclosures.

Provide public transparency into frontier models’ “key ingredients” testing evaluations, limitations and potential risks to enable cross-stakeholder exploration of societal impacts and safety risks.

• Publish “key ingredient list” which can include the model’s compute, parameters, architecture, training data approach, and model documentation, except for cases where sharing findings carries sufficient risk of harm.
• Disclose details such as performance benchmarks, domains of intended and foreseeable unintended use, risks, limitations, and steps to mitigate risks.
• Disclose details such as on testing methodologies, evaluation criteria, results, limitations, and gaps for any internal and external evaluations conducted prior to release. Integrate relevant insights from responsible iteration practices per guidance above.
• Disclose potential environmental and labor impacts per guidelines below.

• Collaborate across industry, civil society, and academia to advance public reporting practices weighing transparency with privacy, safety, and other tradeoffs.
• Align disclosures with existing and emerging best practices like Model Cards, System Cards, Datasheets, Fact Sheets, Nutrition Labels, Transparency Notes and Reward Reports.
• Take an incremental approach to transparency disclosures, prioritizing information users rely on to assess capabilities and risks. Progress to more comprehensive disclosures over time as stakeholders gain experience with disclosure practices.

Equip downstream developers with comprehensive documentation and guidance needed to build safe, ethical, and responsible applications using frontier models.

(Note: It is well understood downstream developers play a crucial role in anticipating deployment-specific risks and unintended consequences. This guidance aims to support developers in fulfilling that responsibility.)

• Provide clear documentation to downstream developers covering appropriate uses, limitations, steps to mitigate risks, and safe development practices when building on frontier models.
• Ensure documentation follows prevailing industry standards and accepted best practices for responsible AI development.

• Collaborate with civil society and downstream developers to advance documentation standards that meet the needs of developers when models are offered through restricted access. This can include gathering inputs on:
  • Safe development checklists for building responsibly on restricted models.
  • Preferred channels for usage guidance and addressing developer questions, aligned with guidance below on enabling feedback mechanisms.

Implement necessary organizational, procedural and technical safeguards, guidelines and controls to restrict unsafe uses and mitigate risks from frontier models.

• Embed safety features directly into model architectures, interfaces and integrations
• Publish clear terms of use prohibiting harmful applications and outlining enforcement policies.
• Limit access through approved applications by implementing appropriate identity/eligibility verification requirements to restrict misuse, along with other technical controls like rate limiting and content filtering.
• Maintain processes to regularly re-evaluate technical and procedural controls, monitor their effectiveness including robustness against jailbreaking attempts, and update terms of use as potential misuses evolve.

• Collaborate across industry and civil society to identify emerging threats requiring new safeguards.
• Provide appropriate transparency into safeguards while protecting integrity.
• Take additional steps to ensure that terms of use are read and understood by users.

Continuously monitor frontier models post-deployment to identify and address issues, misuse, and societal risks.

• Establish ongoing monitoring procedures for deployed models covering areas like performance, fairness, unintended uses, misuses, and risks from combining the model with others.
• Define processes to detect issues and respond appropriately, including notifying partners of significant incidents and considering restricting or retiring a model per guidelines below.

• Provide transparency into monitoring practices, while protecting user privacy.
• Assess downstream real-world impact of models, for example in collaboration with external researchers.
• Collaborate across industry, civil society, and academia to identify shared challenges and best practices for monitoring.

Enable timely and responsible reporting of safety incidents to improve collective learning.

• Implement secure channels aligned with guidance 19 for external stakeholders to report safety incidents or concerns. Also enable internal teams to responsibly report incidents.
• Notify appropriate regulators and partners of critical incidents according to established criteria.

(Note that baseline practices for incident reporting are still emerging across stakeholders.)

• Proactively seek external feedback to improve transparency and effectiveness of incident reporting policies and processes.
• Contribute appropriate anonymized data to collaborative incident tracking initiatives to enable identifying systemic issues, while weighing trade offs like privacy, security, and other concerns.

Responsibly retire frontier models from active use based on well-defined criteria and processes.

• Establish decommissioning procedures and policies including criteria for determining when to restrict, suspend or retire models.
  • Restrict — Limit model use to reduced set of use cases/applications.
  • Suspend — Temporarily prohibit all model use for remediation.
  • Retire — Permanently take model out of service.

• Continue monitoring retired models for downstream impacts and security vulnerabilities per guidance above to prevent unauthorized access and leaks.

Collaboratively establish clear transparency reporting standards for disclosing frontier model usage and policy violations.

• Participate in collaborative initiatives to align on transparency reporting frameworks and standards with industry, civil society, and academia, as commercial uses evolve.

• Release periodic transparency reports following established standards, disclosing aggregated usage insights and violation data. Take appropriate measures to ensure transparency reporting protects user privacy and data.

Support progress of third-party auditing capabilities for responsible frontier model development through collaboration, innovation and transparency.

• Provide sufficient transparency into models and datasets to enable independent assessment and auditing by third parties such as academics and civil society. (Note: Enabling robust third-party auditing remains an open challenge requiring ongoing research and attention).
• Collaborate with third parties to support creation of context-specific auditing methodologies focused on evaluating real-world impacts in specific domains and use cases, beyond base-model evaluations which focus on societal impact evaluations that are not tied to a specific application context.

Responsibly source all forms of labor, including data enrichment tasks like data annotation and human verification of model outputs.

• Pay or contract with vendors that will pay data enrichment workers above the workers’ local living wage.
• Provide or contract with vendors that provide clear instructions for enrichment tasks that are tested for clarity. Enable workers to opt out of tasks.
• Equip or contract with vendors that equip workers with simple and effective mechanisms for reporting issues, asking questions, and providing feedback on the instructions or task design.

• Design and run a pilot before launching a data enrichment project.
• Disclose any new types of labor that enter the supply chain of foundation models. Ensure policies and responsible sourcing practices extend as appropriate to new labor sources as they emerge, like red teamers. Update internal standards and vendor agreements accordingly.
• Proactively survey all workers to identify areas for improving policies, instructions, and work environments, and seek external feedback.

• PAI’s Library of practitioner resources for responsible data enrichment sourcing.

Implement comprehensive human rights due diligence methodologies to assess and address the impacts of frontier models.

• Establish processes for conducting human rights impact assessments pre-deployment.
• Align with relevant guidance like the UN Guiding Principles on Business and Human Rights, and White House Blueprint for AI Bill of Rights. Proactively assess and address potential impacts on vulnerable communities.
• Continuously improve due diligence processes by collaborating with stakeholders and incorporating community feedback.

• Publicly disclose identified risks, due diligence methodologies, and measures to address impacts.

Implement inclusive feedback loops across the AI value chain to ethically identify potential harms.

• Provide clear feedback channels for application developers, consumers, and other direct users.

• Proactively gather input from indirect stakeholders affected by AI systems through ethical community engagement.
• Establish processes for reviewing feedback and integrating affected user perspectives into development and policy decisions.

Measure and disclose the environmental impacts resulting from developing and deploying frontier models.

• Establish processes to evaluate environmental costs like energy usage, carbon emissions and other metrics.
• Monitor and report on environmental impacts of model development and deployment.

• Provide environmental measurement/disclosure mechanisms for application developers building on frontier models.
• Incorporate impacts into model development decisions.
• Collaborate across industry, civil society, and academia to advance the measurement of environmental impacts and responsible disclosure practices.

Adopt responsible practices for disclosing synthetic media and advance solutions for identifying other synthetic content.

• Provide disclosure mechanisms (both direct disclosure that is viewer or listener facing and indirect disclosure that is embedded) for those creating and distributing synthetic media — content that is not identifiable to the average person and may simulate artifacts, persons, or events.
• Evaluate robustness, ease of manipulation, privacy implications, societal impact, and inherent tradeoffs of different disclosure methods. Provide transparency into assessments and rationale behind final disclosure decisions.
• See Section 2 of PAI’s Responsible Practices for Synthetic Media for more information and practices for those building the models for synthetic media.

• Collaborate across industry, civil society, and academia to advance interoperability and standardization for disclosure of synthetic media.
• Research, develop, and distribute solutions to enable identification and disclosure of synthetic content, including voice and text.

Measure and disclose potential severe labor market risks from deployment of frontier models.

• Establish clear thresholds for determining when labor market risks are sufficiently severe to warrant disclosure. Consult experts and affected communities in setting disclosure thresholds.
• Conduct assessments to evaluate likely labor market risks and determine their potential severity. Share findings and methodologies used for risk evaluation.
• Regularly review and update severity thresholds as technologies and applications evolve.

• Collaborate across industry, civil society, academia, and worker organizations to advance the measurement, responsible disclosure practices, and mitigation of severe labor market risks.

Proactively identify and address potential novel or emerging risks from frontier models.

• Conduct model evaluations and experiments to identify new indicators for novel risks, including potential negative societal impacts, malicious uses, “dangerous capabilities” like persuasion and other speculative risks. Assess their likelihood and potential impact.
• Establish regular processes to probe and address potential novel or emerging risks through techniques like external red teaming.

• Collaborate with relevant stakeholders (i.e. governments, other labs and academic researchers) to advance the identification of novel risks and responsible disclosure practices.

• Organizations such as the Frontier Model Forum can contribute to the ongoing development of novel risk assessment practices.

Practice responsible iteration to mitigate risks when developing and deploying frontier models, through both internal testing and limited external releases.

• Commence model development on a smaller scale, systematically test for risks during internal iterations through evaluations and red teaming, and incrementally address identified risks (internal iteration).

• Maintain documentation of each iteration, including lessons learned, challenges faced, and mitigation measures implemented.

(Note that this is an emerging and less-explored guardrail, and disclosure practices will need to evolve as we learn more about its effectiveness.)

Disclose planned testing, evaluation, and risk management procedures for frontier models prior to development.

• Produce a “pre-systems card” detailing quality management plans before research begins.
• Outline intended training data approach, model testing, safety evaluations, responsible AI practices, and development team.
• Submit pre-systems cards to regulatory bodies for review where required.
• Update plans as needed throughout the R&D process.

(Note: baseline practices for disclosure practices are still emerging across stakeholders.)

• Prepare a written “safety case”, explaining why the model is safe enough to develop.
• Share pre-systems cards with independent experts for external review.
• The guidance on Responsible Iteration guides testing and release practices, while pre-systems cards disclose development plans before implementation.

Establish risk management oversight processes and continuously adapt to address real world impacts from foundation models.

• Establish risk management structures and processes, such as enterprise risk management, independent safety boards, and ethics review processes to define guidelines on responsible development, release, and staged rollout considerations, including when a model should not be released.
• Regularly update policies, frameworks, and organizational oversight to address evolving capabilities and real-world impacts.

• Publicly share information about implemented internal governance and risk management processes.

Perform internal evaluations of models prior to release to assess and mitigate for potential societal risks, malicious uses, and other identified risks.

• Establish comprehensive internal evaluation policies and processes including testing for fairness, interpretability, output harms, novel risks and intended vs foreseeable unintended use cases.
• Proactively identify and minimize potential sources of bias in training corpora, and adopt techniques to minimize unsafe model behavior.
• Conduct evaluations using cross-disciplinary review teams spanning ethics, security, social science, safety and other relevant domains.
• Address identified risks and adapt deployment plans accordingly based on learnings from pre-deployment evaluations.
• Maintain documentation of evaluation methods, results, limitations, and steps taken to address identified issues and integrate insights in public reporting per guidance below.

• Collaborate across industry, civil society, and academia to advance the development and standardization of model evaluations for foundation models.

• Organizations such as Frontier Model Forum can contribute to the ongoing development of internal model evaluations.

Implement red teaming that probes frontier models for potential malicious uses, societal risks and other identified risks prior to release. Address risks and responsibly disclose findings to advance collective knowledge.

• Perform internal and external red teaming across model capabilities, use cases, and potential harms including dual-use risks using techniques such as adversarial testing, vulnerability scanning, and surfacing edge cases and failure modes.
• Conduct iterative red teaming throughout model development. Continuously evaluate results to identify areas for risk mitigation and improvements, including for planned safeguards.
• Address identified risks and adapt deployment plans accordingly based on learnings from pre-deployment evaluations.

• Commission external red teaming by independent experts such as domain experts and affected users to surface gaps.

Responsibly source all forms of labor, including data enrichment tasks like data annotation and human verification of model outputs.

• Pay or contract with vendors that will pay data enrichment workers above the workers’ local living wage.
• Provide or contract with vendors that provide clear instructions for enrichment tasks that are tested for clarity. Enable workers to opt out of tasks.
• Equip or contract with vendors that equip workers with simple and effective mechanisms for reporting issues, asking questions, and providing feedback on the instructions or task design.

• Design and run a pilot before launching a data enrichment project.

• PAI’s Library of practitioner resources for responsible data enrichment sourcing.

Measure and disclose the environmental impacts resulting from developing and deploying frontier models.

• Establish processes to evaluate environmental costs like energy usage, carbon emissions and other metrics.
• Monitor environmental impacts of model development.

• Incorporate impacts into model development decisions.
• Collaborate across industry, civil society, and academia to advance the measurement of environmental impacts and responsible disclosure practices.

Proactively identify and address potential novel risks from frontier models.

• Conduct model evaluations and experiments to identify new indicators for novel risks, including potential negative societal impacts, malicious uses, “dangerous capabilities” like persuasion and other speculative risks. Study potential risks from integrating the model into novel or unexpected downstream environments and use cases Assess their likelihood and potential impact.
• Establish regular processes to probe and address potential novel or emerging risks through techniques like external red teaming.

• Develop new measurements and assessment methods specifically tailored for novel risks.

• Organizations such as the Frontier Model Forum can contribute to the ongoing development of novel risk evaluation practices.

Practice responsible iteration to mitigate risks when developing and deploying frontier models, through both internal testing and limited external releases.

• Commence model development on a smaller scale, systematically test for risks during internal iterations through evaluations and red teaming, and incrementally address identified risks (internal iteration).

• Maintain documentation of each iteration, including lessons learned, challenges faced, and mitigation measures implemented. (Note that this is an emerging and less-explored guardrail, and disclosure practices will need to evolve as we learn more about its effectiveness.)

Identify and address potential security vulnerabilities in frontier models to prevent unauthorized access or leaks.

• Conduct rigorous testing such as penetration testing, prompt analysis, and data poisoning assessments to identify vulnerabilities that could enable model leaks or manipulation.
• Establish protocols for addressing identified vulnerabilities pre-deployment.

• UC Berkeley CLTC Draft Profile under Measure 2.7

Establish risk management oversight processes and continuously adapt to address real world impacts from foundation models.

• Establish risk management structures and processes, such as enterprise risk management, independent safety boards, and ethics review processes to define guidelines on responsible development, release, and staged rollout considerations, including when a model should not be released.
• Regularly update policies, frameworks, and organizational oversight to address evolving capabilities and real-world impacts.

• Publicly share information about implemented internal governance and risk management processes.

Perform internal evaluations of models prior to release to assess and mitigate for potential societal risks, malicious uses, and other identified risks.

• Establish comprehensive internal evaluation policies and processes including testing for fairness, interpretability, output harms, novel risks and intended vs foreseeable unintended use cases.
• Proactively identify and minimize potential sources of bias in training corpora, and adopt techniques to minimize unsafe model behavior.
• Conduct evaluations using cross-disciplinary review teams spanning ethics, security, social science, safety and other relevant domains.
• Address identified risks and adapt deployment plans accordingly based on learnings from pre-deployment evaluations.
• Maintain documentation of evaluation methods, results, limitations, and steps taken to address identified issues and integrate insights in public reporting per guidance below.

• Collaborate across industry, civil society, and academia to advance the development and standardization of model evaluations for frontier models.

• Organizations such as Frontier Model Forum can contribute to the ongoing development of internal model evaluations.

Complement internal testing through model access to third-party researchers to assess and mitigate potential societal risks, malicious uses, and other identified risks.

• Provide controlled access to models for additional evaluative testing by external researchers. External evaluators should be granted sufficient model access, computational resources, and time to conduct effective evaluations prior to deployment.
• Implement appropriate safeguards to prevent unauthorized access or information leaks via external evaluations.
• Address identified risks and adapt deployment plans accordingly based on learnings from pre-deployment evaluations.
• Maintain documentation of evaluation methods, results, limitations, and steps taken to address identified issues and integrate insights in public reporting per guidance below, except for cases where sharing findings carries sufficient risk of harm.

(Note: Enabling robust third-party auditing remains an open challenge requiring ongoing research and attention.)

• Pursue diverse external assessment methods including panels and focus groups.
• Consult independent third parties to audit models following prevailing best practices on methodologies.

• Organizations such as the Frontier Model Forum can contribute to development of external model evaluations.

Provide public transparency into frontier models’ “key ingredients” testing evaluations, limitations and potential risks to enable cross-stakeholder exploration of societal impacts and safety risks.

• Publish “key ingredient list” which can include the model’s compute, parameters, architecture, training data approach, and model documentation, except for cases where sharing findings carries sufficient risk of harm.
• Disclose details such as performance benchmarks, domains of intended and foreseeable unintended use, risks, limitations, and steps to mitigate risks.
• Disclose details such as on testing methodologies, evaluation criteria, results, limitations, and gaps for any internal and external evaluations conducted prior to release. Integrate relevant insights from responsible iteration practices per guidance above.
• Disclose potential environmental impacts per guidance below.

• Collaborate across industry, civil society, and academia to advance public reporting practices weighing transparency with privacy, safety, and other tradeoffs.
• Align disclosures with existing and emerging best practices like Model Cards, System Cards, Datasheets, Fact Sheets, Nutrition Labels, Transparency Notes and Reward Reports.
• Take an incremental approach to transparency disclosures, prioritizing information users rely on to assess capabilities and risks. Progress to more comprehensive disclosures over time as stakeholders gain experience with disclosure practices.

Implement necessary organizational, procedural and technical safeguards, guidelines and controls to restrict unsafe uses and mitigate risks from foundation models.

• Publish a research license prohibiting harmful applications.
• Provide downstream use documentation covering details like intended uses, limitations, mitigating risks, and safe development practices.
• Provide appropriate transparency into safeguards while protecting integrity.

Continuously monitor frontier models post-deployment to identify and address issues, misuse, and societal risks.

• Establish ongoing monitoring procedures for deployed models covering areas like performance, fairness, unintended uses, misuses, and other impacts.
• Define processes to detect issues and respond appropriately, including considering restricting or retiring a model per guidelines below.

• Provide transparency into monitoring practices, while protecting user privacy.
• Collaborate across industry, civil society, and academia to identify shared challenges and best practices for monitoring.

Responsibly retire frontier models from active use based on well-defined criteria and processes.

• Establish decommissioning procedures and policies including criteria for determining when to retire models.
• Retire: Permanently take model out of service.

• Continue monitoring retired models for downstream impacts and security vulnerabilities per guidance above to prevent unauthorized access and leaks.

Responsibly source all forms of labor, including data enrichment tasks like data annotation and human verification of model outputs.

• Pay or contract with vendors that will pay data enrichment workers above the workers’ local living wage.
• Provide or contract with vendors that provide clear instructions for enrichment tasks that are tested for clarity. Enable workers to opt out of tasks.
• Equip or contract with vendors that equip workers with simple and effective mechanisms for reporting issues, asking questions, and providing feedback on the instructions or task design.

• PAI’s Library of practitioner resources for responsible data enrichment sourcing.

Measure and disclose the environmental impacts resulting from developing and deploying frontier models.

• Establish processes to evaluate environmental costs like energy usage, carbon emissions and other metrics.
• Monitor and report on environmental impacts of model development and deployment.

Adopt responsible practices for disclosing synthetic media.

• Disclose (both direct disclosure that is viewer or listener facing and indirect disclosure that is embedded) when creating and distributing synthetic media — content that is not identifiable to the average person and may simulate artifacts, persons, or events.
• See Section 3 of PAI’s Responsible Practices for Synthetic Media for more information and practices for creators of synthetic media.