Building a Glossary for Synthetic Media Transparency Methods
Part 1: Indirect Disclosure
![$hero_image['alt']](https://partnershiponai.org/wp-content/uploads/2024/11/fingerprints_green.png)
According to an Axios poll, a third of Americans say AI will reduce their trust in the results of the 2024 U.S. election. For Partnership on AI (PAI) and our community of organizations building, creating, and distributing synthetic media, this raises important questions: How can everyday citizens know if the media they see and hear has been created or modified by AI and, more importantly, is misleading? And what can organizations do to improve transparency?
There is no perfect answer. However, those building, creating, and distributing synthetic media can act to support better explanations of how content has been created and edited. They have a responsibility, when appropriate, to make clear to audiences that content is AI-generated or AI-modified (and any uncertainty about such judgments).
While many disparate efforts have emerged to help audiences navigate an increasingly synthetic information environment, largely by providing context about content, the community has not aligned on which combination of tactics to implement, when to share insights with audiences, and how they can evaluate their efficacy in supporting trustworthy content.
The need for alignment is urgent. Policymakers—including the White House in its recent Executive Order—have begun to explore different techniques for technology, government, civil society, and media organizations, to provide audiences with greater insight into what they see and hear.
To improve alignment and understanding of synthetic media transparency, PAI has developed a series of resources. Part 1 of the series, focused on indirect disclosure, includes:
- A community-driven glossary defining the common technical methods that can provide insight into whether media is synthetic or not—what we describe as indirect disclosure methods;
- Proposed questions for evaluating indirect disclosure methods; and
- Initial guidance on how policymakers and those building, creating, and distributing synthetic media should implement these techniques.
The glossary below builds off of a seven-part workshop series with 18 organizations that are collaborators on PAI’s Responsible Practices for Synthetic Media. While the information in this document is provided by PAI and is not intended to reflect the view of any other organization, much of it has been informed through collaboration.
Different organizations use different terms to refer to the class of methods described below. However, we encourage the use of the term indirect disclosure to center these methods in how they contribute to the goal of broader AI transparency. While the disclosure taking place will not always translate to transparency for audiences, it does reveal details about content’s AI-generation or AI-modification to entities involved in content development, creation, and distribution. Such information can be used to support and underscore broader societal transparency goals. We will explore direct disclosure in future blog posts.
Glossary
We include methods in the glossary based on a combination of factors, including whether or not they are in use by those developing, creating, and distributing synthetic media, as well as input from our 18 Framework collaborators. Inclusion in the glossary does not equate to recommended use—rather, the Initial Recommendation section below offers initial guidance on the optimal combination of indirect disclosure methods, which we will further develop in future blogs. We also include key terms in the glossary to help support understanding of the methods in the graphic above.
Visual, auditory, or multimodal content that has been generated or modified (commonly via artificial intelligence). Such outputs are often highly realistic, would not be identifiable as synthetic to the average person, and may simulate artifacts, persons, or events.
The umbrella term used to describe signals for conveying whether a piece of media is AI-generated or AI-modified. Such signals can either be indirect (not user facing) or direct (user facing). Indirect disclosure signals can support understanding of whether content has been AI-generated or AI-modified and, when appropriate, guide development of direct disclosures to audiences and end-users.
Indirect disclosures, direct disclosures, synthetic media detection
A signal for conveying whether a piece of media is AI-generated or AI-modified, based on information about a piece of content’s origin and/or its evolution; is not user facing. The “disclosure” that takes place is typically to entities involved in content development, creation, and distribution—but it can be used to inform direct, or audience/user-facing, disclosure.
Proactive Methods
- At generation
- Post-generation
Synthetic media transparency methods that rely upon an actor purposefully applying a signal that can then be identified by a third-party who is able to detect or interpret the signal. Notably, in an adversarial setting, bad actors will not leverage such techniques and may attempt to alter existing signals.
Can be classified further into two categories:
Signal is applied automatically by a media generation model at the moment of creation.
Signal is applied after creation.
Synthetic media transparency methods that determine the origin or evolution of media based on signals that do not rely upon a disclosure signal being applied by an actor.
A signal for conveying to users whether a piece of media is AI-generated or AI-modified; often informed by indirect disclosures.
Labels, content overlays
Watermarking
- Invisible
- Visible
The proactive (at, or post, generation) insertion of modifications into a piece of content that can help support interpretations of how the content was generated and/or edited. Can come in two forms:
Modifications made to a piece of content that are imperceptible to the human eye or ear. Can only be identified by a watermark detector (distinct from Synthetic Media Detection).
Modifications made to a piece of synthetic content that are detectable to the human eye or ear and do not require the use of a detector to interpret them.
Fingerprinting
- Cryptographic hashing
- Perceptual hashing
The proactive (at, or post, generation) process by which a hash is generated for a piece of content for the purpose of identifying that content at a later date. Such hashes must be stored in a database in order to verify future content against the original. Unlike watermarking, this hash is not embedded in the content file itself. Also known as “hashing and matching” or “hashing and logging.” Can come in two forms:
An exact-match form of hashing where the hash for a piece of synthetic content will not match if the content has been modified in any way.
A probabilistic-match form of hashing where the hash for a piece of synthetic content is resilient to minor perturbations (i.e., will still match with minor changes).
YouTube’s Content ID
Metadata
- Signed
- Unsigned
Information about the origin, structure, and/or editing history of a piece of content that is proactively attached to the content itself.
Information that is proactively attached to the content itself and stored using secure encryption; a trusted/validated signer certificate is added post generation. State of the art methods leverage cryptographic signatures.
Information that is proactively attached to the content itself at generation but is not stored with secure encryption or validated with a trusted signer certificate, and potentially can be changed imperceptibly (weakening robustness).
In policy and public discourse, metadata has sometimes been described as “media provenance” despite the fact that provenance can be a broader term describing all the methods under the umbrella of indirect disclosure. For example, several PAI Partners refer to indirect disclosure methods as provenance methods. It is our hope that the terminology expressed in this glossary helps align the field on nomenclature.
C2PA Standard (signed), IPTC Standard (unsigned)
Methods that rely on detecting unintentionally added patterns/forensic cues differentiating synthetic media from non-synthetic media to determine the likelihood that a piece of content was AI-generated or AI-modified; such methods do not rely on theproactive addition of artifacts such as watermarks in content. Synthetic media detection is a derived transparency method.
Intel’s FakeCatcher, Google Jigsaw’s Assembler (no longer active)
Detector Access
- Open
- Closed
Whether for watermark detection or synthetic media detection, systems identifying synthetic media can either be shared broadly or restricted in their access, or somewhere in between.
Only available to a select number of organizations, minimizing the risk of adversarial exploitation at the expense of accessibility.
Widely available, maximizing accessibility at the expense of increased risk of adversarial exploitation.
Detector access does not necessarily have to be a binary choice between open and closed. PAI has conducted much work on the tradeoffs between open and closed access; see here, here, and here, all incorporating recommendations for “goldilocks” exposure between open and closed.