According to an Axios poll, a third of Americans say AI will reduce their trust in the results of the 2024 U.S. election. For Partnership on AI (PAI) and our community of organizations building, creating, and distributing synthetic media, this raises important questions: How can everyday citizens know if the media they see and hear has been created or modified by AI and, more importantly, is misleading? And what can organizations do to improve transparency?
There is no perfect answer. However, those building, creating, and distributing synthetic media can act to support better explanations of how content has been created and edited. They have a responsibility, when appropriate, to make clear to audiences that content is AI-generated or AI-modified (and any uncertainty about such judgments).
While many disparate efforts have emerged to help audiences navigate an increasingly synthetic information environment, largely by providing context about content, the community has not aligned on which combination of tactics to implement, when to share insights with audiences, and how they can evaluate their efficacy in supporting trustworthy content.
The need for alignment is urgent. Policymakers—including the White House in its recent Executive Order—have begun to explore different techniques for technology, government, civil society, and media organizations, to provide audiences with greater insight into what they see and hear.
To improve alignment and understanding of synthetic media transparency, PAI has developed a series of resources. Part 1 of the series, focused on indirect disclosure, includes:
- A community-driven glossary defining the common technical methods that can provide insight into whether media is synthetic or not—what we describe as indirect disclosure methods;
- Proposed questions for evaluating indirect disclosure methods; and
- Initial guidance on how policymakers and those building, creating, and distributing synthetic media should implement these techniques.
The glossary below builds off of a seven-part workshop series with 18 organizations that are collaborators on PAI’s Responsible Practices for Synthetic Media. While the information in this document is provided by PAI and is not intended to reflect the view of any other organization, much of it has been informed through collaboration.
Different organizations use different terms to refer to the class of methods described below. However, we encourage the use of the term indirect disclosure to center these methods in how they contribute to the goal of broader AI transparency. While the disclosure taking place will not always translate to transparency for audiences, it does reveal details about content’s AI-generation or AI-modification to entities involved in content development, creation, and distribution. Such information can be used to support and underscore broader societal transparency goals. We will explore direct disclosure in future blog posts.
We include methods in the glossary based on a combination of factors, including whether or not they are in use by those developing, creating, and distributing synthetic media, as well as input from our 18 Framework collaborators. Inclusion in the glossary does not equate to recommended use—rather, the Initial Recommendation section below offers initial guidance on the optimal combination of indirect disclosure methods, which we will further develop in future blogs. We also include key terms in the glossary to help support understanding of the methods in the graphic above.
Visual, auditory, or multimodal content that has been generated or modified (commonly via artificial intelligence). Such outputs are often highly realistic, would not be identifiable as synthetic to the average person, and may simulate artifacts, persons, or events.
The umbrella term used to describe signals for conveying whether a piece of media is AI-generated or AI-modified. Such signals can either be indirect (not user facing) or direct (user facing). Indirect disclosure signals can support understanding of whether content has been AI-generated or AI-modified and, when appropriate, guide development of direct disclosures to audiences and end-users.
A signal for conveying whether a piece of media is AI-generated or AI-modified, based on information about a piece of content’s origin and/or its evolution; is not user facing. The “disclosure” that takes place is typically to entities involved in content development, creation, and distribution—but it can be used to inform direct, or audience/user-facing, disclosure.
- At generation
Synthetic media transparency methods that rely upon an actor purposefully applying a signal that can then be identified by a third-party who is able to detect or interpret the signal. Notably, in an adversarial setting, bad actors will not leverage such techniques and may attempt to alter existing signals.
Can be classified further into two categories:
Signal is applied automatically by a media generation model at the moment of creation.
Signal is applied after creation.
A signal for conveying to users whether a piece of media is AI-generated or AI-modified; often informed by indirect disclosures.
Labels, content overlays
The proactive (at, or post, generation) insertion of modifications into a piece of content that can help support interpretations of how the content was generated and/or edited. Can come in two forms:
Modifications made to a piece of synthetic content that are detectable to the human eye or ear and do not require the use of a detector to interpret them.
- Cryptographic hashing
- Perceptual hashing
The proactive (at, or post, generation) process by which a hash is generated for a piece of content for the purpose of identifying that content at a later date. Such hashes must be stored in a database in order to verify future content against the original. Unlike watermarking, this hash is not embedded in the content file itself. Also known as “hashing and matching” or “hashing and logging.” Can come in two forms:
An exact-match form of hashing where the hash for a piece of synthetic content will not match if the content has been modified in any way.
A probabilistic-match form of hashing where the hash for a piece of synthetic content is resilient to minor perturbations (i.e., will still match with minor changes).
YouTube’s Content ID
Information about the origin, structure, and/or editing history of a piece of content that is proactively attached to the content itself.
Information that is proactively attached to the content itself and stored using secure encryption; a trusted/validated signer certificate is added post generation. State of the art methods leverage cryptographic signatures.
Information that is proactively attached to the content itself at generation but is not stored with secure encryption or validated with a trusted signer certificate, and potentially can be changed imperceptibly (weakening robustness).
In policy and public discourse, metadata has sometimes been described as “media provenance” despite the fact that provenance can be a broader term describing all the methods under the umbrella of indirect disclosure. For example, several PAI Partners refer to indirect disclosure methods as provenance methods. It is our hope that the terminology expressed in this glossary helps align the field on nomenclature.
Methods that rely on detecting unintentionally added patterns/forensic cues differentiating synthetic media from non-synthetic media to determine the likelihood that a piece of content was AI-generated or AI-modified; such methods do not rely on theproactive addition of artifacts such as watermarks in content. Synthetic media detection is a derived transparency method.
Only available to a select number of organizations, minimizing the risk of adversarial exploitation at the expense of accessibility.
Widely available, maximizing accessibility at the expense of increased risk of adversarial exploitation.
Detector access does not necessarily have to be a binary choice between open and closed. PAI has conducted much work on the tradeoffs between open and closed access; see here, here, and here, all incorporating recommendations for “goldilocks” exposure between open and closed.
Questions for Evaluating Indirect Disclosure Methods
The following questions for evaluating indirect disclosure methods relate to the methods’ impact on media transparency. They emerged from this this MIT Tech Review piece and discussion with PAI’s AI & Media Integrity Steering Committee.
- How resilient is [disclosure mechanism] to manipulation or forgery? How easily can it be removed by a bad actor?
- How accessible is [disclosure mechanism] to diverse audiences?
- How difficult would it be for organizations to adopt [disclosure mechanism] at scale?
- Is the [disclosure mechanism] associated with a piece of content maintained separately or is it embedded within the content’s pixels or metadata?
- What organizations involved in the lifecycle of a piece of synthetic media (Builders, Creators, Distributors) need to opt-in in order for [disclosure mechanism] to be successful and can it still be viable if it is adopted partially?
- How can [disclosure mechanism] signals be interpreted by those that interact with them through the lifecycle of a piece of synthetic media content, i.e. end users, internal decision makers, distributors, etc.?
- How resilient are associated detection tools to manipulation or adversarial attack?
- Can [disclosure mechanism] complement any other mechanisms to provide more robust disclosure?
- Does [disclosure mechanism] clash with any other existing disclosure mechanisms, i.e. are there other disclosure mechanisms that may render [disclosure mechanism] ineffective?
PAI’s Initial Recommendation on Indirect Disclosure
PAI’s years of work on indirect disclosure make clear that only a multi-faceted approach, one that involves permutations of metadata, watermarking, and fingerprinting (together or separately, depending on the use case) can respond to the challenge of synthetic media identification and transparency. This should inspire those building, creating, and distributing synthetic media, as well as policymakers, to align on implementation, recognizing that such techniques are meaningful, though imperfect solutions, as synthetic media becomes more ubiquitous.
None of the methods, or combinations of methods, above is a silver bullet solving the challenges synthetic media poses.
Indirect disclosure methods will not always be able to provide those interpreting them with a binary “yes, this is AI-generated or modified” or “no, it is not.” Determining whether content is misleading and deceptive, which is often more important than whether content is AI-generated or modified generally, is beyond the scope of these methods.
Synthetic media detection, while in use at many institutions and a signal of content type that does not rely on good actors’ proactivity, has proven limited in its applicability to real-world content. There is much work to be done about how institutions can make sense of such signals and, at times, best communicate the limits of such approaches to audiences without discouraging their implementation, which we will expand upon in our upcoming blogs.
In Part 2 of this series, we will evaluate the pros and cons of the different indirect disclosure methods defined above, and add more detail to this recommendation. Later, we will evaluate different scenarios and methods for direct disclosure.
Future Work on Disclosure (Indirect and Direct)
Analyzing the impact of technical transparency measures also requires us to consider the broader challenges that come to light when a synthetic media transparency method is employed amidst the complexity of the open world. Evaluating the social impact of such methods requires answering questions related to user trust and belief, accessibility, epistemic certainty, real-world applicability, and more. While this comprehensive analysis is beyond the scope of this document, we will attempt to answer some of these questions in future work.
Follow-on blogs from PAI will:
- Evaluate the pros and cons of different indirect disclosure methods, responding to the questions posed above
- Explain our initial recommendation to implement a multi-faceted approach including different methods
- Share a community-driven glossary of direct disclosure, or user facing, methods for providing insight into content’s origin and edits
- Provide insight into the sociotechnical challenges of content disclosures
- Suggest best practices for implementing these direct disclosure methods in policy and practice.