How AI video company D-ID received consent to digitally resurrect victims of domestic violence

A contextual introduction to the case study.

D-ID is a technology company that operates on a global basis with offices in North America, the UK, Singapore, and with an HQ in Tel-Aviv. We serve individuals and businesses in almost every country around the world. Our clientele encompasses creators, leading e-learning platforms, Fortune 500 companies, financial service entities, the automotive industry, technology, retail, entertainment sectors, as well as marketing agencies, production companies, and social media platforms.

Our generative AI technology allows users to generate photorealistic, presenter-led video from either text or audio – significantly reducing the expenses and complexities associated with large-scale video production. As we can generate video at up to 100fps (3-4x real time), our technology also allows users to create and interact with AI-driven digital humans.

This video provides a good introduction of all the services we offer.

Founded in 2017, D-ID is venture-backed and has a team of 60 employees. The company’s technology has facilitated the creation of over 150 million videos. Our solutions are accessible through a self-service studio, an API, and various plug-ins, playing a pivotal role in animating AI assistants.

D-ID was initially founded as a company aimed at protecting people’s privacy. By generating very small alterations to images of faces uploaded to the internet, our technology was able to help individuals protect themselves against online facial recognition engines which regularly scrape the internet. (D-ID originally stood for ‘De-Identification’).

Given this background, privacy and ethical considerations are deeply imprinted in our DNA. To assert our focus on the ethical use of technology, we made a clear pledge about our ethical commitments, which is available here. D-ID considers itself a Builder using the definitions provided in Partnership on AI’s (PAI) Framework.

Elaborate on the challenge being addressed in the case study, i.e. the issue to which your organization is applying the Framework.

In 2021, we collaborated with social tech entrepreneur, Ms. Shiran Mlamdovsky Somech, to raise awareness about domestic violence in Israel.

She was involved in an initiative to raise awareness of domestic abuse by highlighting the cases of five diverse women who had all lost their lives to domestic violence. Of course, this is a sensitive, albeit important topic.

Working with Shiran and the team, we realized that we could “bring to life” the images of the victims by animating their photos with our technology and syncing them with new voices. Doing so, we hoped, would draw much needed attention to the issue of domestic violence. By using the vocal talent of acclaimed Israeli actors, we hoped this would attract attention and interest to this topic.

The project itself can be viewed on the Listen To My Voice website. An example of one of the videos can be found here.

Right from the project’s inception, we identified a tension concerning the responsible AI practices outlined in the Framework, which D-ID is committed to.

The challenge primarily centered around the concept of “informed consent from the subject(s).” It is imperative to obtain consent from individuals portrayed in content generated by the model. Yet, realistically, we couldn’t secure their approval since they were, tragically, no longer alive.

The pressing question became evident: How can individuals who are no longer alive provide their consent?

This dilemma was not just theoretical; at its most critical interpretation, the project could be misconstrued as disseminating disinformation since the women depicted never uttered the scripted words and could not actively agree to represent the campaign. Others might perceive it as contributing to the re-traumatization of the bereaved.

Consequently, this case indeed touches upon potential harms outlined in
Appendix B of PAI’s Framework, as it raises complex issues surrounding consent and representation.

Describe what your organization is attempting to accomplish by addressing this challenge and/or furthering the opportunities.

When enacting PAI’s Framework, we also worked to uphold the complementary and overlapping ethical standards and principles D-ID had already set forth in our own pledge (which can be viewed here).

One of our core tenets, ensuring the consent of visible or audible actors, overlapped with the Framework’s stipulation on being transparent about the informed consent from content subjects.

A pivotal element in this process was our active collaboration with the affected families. We aimed to get explicit consent from the closest relatives of the victim – i.e. parents, siblings, or children. In cases where we could not get full agreement (e.g., where some family members were in agreement and some not), we did not include those women in the project. We talked to nuclear family members when possible, and if not, the extended family.

Important questions arise in cases where consent can’t be given – for example, if the subject is dead, missing, or even kidnapped.

Beyond just approving the idea of the project, the families were also integral in co-creating the content and scripts, and we further ensured the authenticity and sensitivity of the narrative by involving domestic violence experts in the process. Doing so helped ensure that the videos were created in a way that was unlikely to re-traumatize the bereaved and would most closely reflect the wishes of the deceased. At no point did we change our goal as the process ran its course for the simple reason that the families we were in contact with really believed in the project and actually encouraged us to keep going.

To bolster our commitment to ethical and legal soundness, we worked closely with a law firm. The two main issues they raised were: consent (as discussed above) and ensuring we could use the images of the women – that we weren’t infringing anyone’s intellectual property.

Their guidance was invaluable in ensuring our efforts did not transgress legal boundaries nor inadvertently cause harm – suggesting the importance of laws alongside documents like the Framework for advising technology companies on how to develop projects.

The primary goal when embarking on this project was to remain consistent with both our pledge and the Framework’s directives. We prioritized the welfare of the affected families and the memory of the victims, alongside supporting increasing awareness about abusive relationships in an effective and impactful manner.

The end goal remained steadfast throughout, which was to respect and prioritize informed consent and genuine representation. Even though the Framework didn’t fundamentally alter our existing procedures or policies, it undeniably prompted us to grapple earnestly with the complexities of consent, reinforcing our commitment to ethical considerations.

Identify which Framework principle was used to help address the challenge/opportunity, how it was chosen and implemented, and describe how it was applied.

The primary principle from the Framework that we utilized to navigate this challenge was consent. We closely engaged with the families of the deceased women to ensure their approval on several elements including the message, script, voice talent, and distribution method. This collaborative process was further bolstered by a partnership with a law firm to ensure alignment with the factual details as outlined in legal documents.

As mentioned above, some families were more reluctant than others both about the project in general and the use of AI, which at the time was not a well-known concept. Generally, however, attitudes were positive. As a result, we were encouraged to complete the project as intended.

We were also careful to follow the Framework’s exhortation to “be transparent to users about tools and technologies” – we were always clear that the content we produced was created with synthetic media AI tools. For example, in the script, the women all say, “I was [name]” making it clear that they are no longer with us and implying that they were synthetic. We also had disclaimers on the website and all social posts. And, of course, the PR message was all based around the fact that we had used AI to create the messages.

Elaborate on any internal or external obstacles intrinsic to the Framework that were overcome.

While we did our best to navigate the challenge of consent and felt confident in our collaborative approach with the families, the challenge of true informed consent, as traditionally defined, was not entirely overcome due to the complex nature of the project. This underscores the complexity of applying the Framework’s guidelines to unique scenarios where direct consent is unattainable.

From a purely legal point of view, we understood that human rights law does not apply after death. So we felt like the very high bar we set ourselves of getting unanimous approval from the closest family members was more than enough – especially in conjunction with the level of transparency we brought to the project.

Implementing the consent principle in such a unique situation, although it offered a structured approach, also required us to operate in gray areas and make judgment calls as the Framework’s guidelines couldn’t anticipate every unique scenario.

But given the importance of the cause we were championing, which had the power to literally save lives, we were reassured that the trade-offs we were making were more than justified.

Identify the opportunities created for your organization by utilizing the Framework to address the challenge.

Utilizing the Framework offered our organization a structured approach to navigate the ethical intricacies of the project, especially around the challenging concept of consent. By working within the Framework’s parameters, we achieved the following:

• Process Improvement: The need to address the nuances of the consent principle forced us to establish a more collaborative approach with stakeholders, notably the victims’ families. This has fostered a deeper sense of empathy and responsibility in our work, which we believe will enhance future projects.
• Policy Enhancement: The project and PAI’s Framework guidance together led to a rigorous reevaluation of our internal policies, especially around sensitive subjects. This has led to a more comprehensive and respectful approach to content creation, which aligns with both our ethics pledge and broader industry standards.
• Secondary Benefits: A notable secondary effect of Framework implementation was the increased awareness of our team to potential ethical pitfalls. This awareness will likely lead to proactive measures in future projects, enhancing our organization’s reputation and trustworthiness in the industry.

A description of how implementing the Framework ended for your organization, including any lessons learned.

It is at the heart of D-ID’s values to be a leader of ethics in this space. We believe that we have a duty to our stakeholders and society at large to do the right thing. And we see the moral and commercial to be intertwined, not separate – “doing good by doing right.” On that basis, this project exceeded our expectations, achieving not only massive viewership but also tangible, societal impact. With 10 million views within three days, it heightened public awareness and possibly played a role in saving lives, evident by the surge in calls to related helplines. The campaign’s recognition at global institutions like the UN, NATO, and various parliaments underscores its significance.

Furthermore, from D-ID’s standpoint, anticipated concerns over potential backlash were mitigated by our collaborative approach with the victims’ families, in which we sought as high a degree of consent as was feasible, and in communicating the synthetic nature of the videos in a highly transparent way.

We do, nevertheless, recognize that the context of the information environment can affect the attitude that institutions/families might have to consent. For example, in more political settings, the use of AI at all may be frowned upon entirely due to more general concerns about being misconstrued as misinformation or “deepfakes”.

As well as contributing to this project in itself, we feel this project made a broader contribution to societal understanding of synthetic media. It was a high-profile project which showcased generative AI in a positive and moving way, showcasing the power of this new technology to do good.

Key Lessons Learned:

Importance of Collaboration: Engaging closely with stakeholders, especially affected families, was pivotal in navigating the delicate nuances of consent, ensuring both ethical and public acceptance of the project.

Gaps in the Framework: PAI’s Framework could delve deeper into situations involving deceased individuals. As our project revealed, traditional notions of consent and human rights become ambiguous when dealing with posthumous representations or even situations such as missing people or people who have been kidnapped where it is normal for families, rather than the individual concerned, to give consent.

Open Questions: A lingering concern is the ethical implications and best practices surrounding the use of AI for representing the deceased, particularly in contexts that aim to drive societal change. Further guidance on this within the Framework would be invaluable.

Suggested Updates: Considering the evolving nature of generative AI and its diverse applications, the Framework might benefit from periodic updates, specifically addressing scenarios involving posthumous representations. From our experience, the Framework was most helpful in advocating transparency and disclosure, though it might require more clarity on unique situations like ours.