ABOUT THE PROJECT
In 2015, I launched Reckonings — a narrative podcast that explores how people shift their political worldviews, transcend extremism, and make other kinds of transformative change. Stories have included that of a deeply conservative Congressman who made a spiritual conversion on climate change, a white supremacist who transcended a life of hate, and the architect of Facebook's business model who's since devoted his life to tackling technology addiction. In Reckonings' early days, I started keeping a wish list of moonshot guests — high-profile figures like Pope Francis, Charles Koch, and Mark Zuckerberg whose personal reckonings I thought would be most socially transformative. I fantasized about producing a fictional film about how Charles Koch's transformation in turn transformed our climate trajectory.
Then in 2017, I discovered the phenomenon of deepfakes. BOOM.
In early 2019, I released an imaginary reckoning with Pope Francis on Reckonings, as an audio prototype of what I hoped to do with synthetic video. I wrote the script, had a voice actor perform it, and made explicit that it was fiction. This was the first time I'd ever aired fiction on the show, and I didn’t know how listeners would respond. To my surprise, listeners loved it. I even heard from survivors of clergy sex abuse who, although they knew it was fiction, found it remarkably helpful to hear the imaginary Pope say the kinds of things they wished to hear the real Pope say. Deep Reckonings is now the culmination of a years-long exploration of how people change, the beauty of critical self-reflection, and the power of purposeful fiction.
Deep Reckonings is a series of explicitly-marked deepfake videos that imagine the most morally courageous versions of our most controversial public figures. It is an exercise in steel-manning where we often straw-man. The public figures include: Brett Kavanaugh wrestling with the way he responded to the sexual allegations against him; Alex Jones grappling with his spread of deceitful conspiracy theories; and Mark Zuckerberg confronting his techno-utopianism. These figures don't admit to actions that aren't publicly known, but instead take responsibility for actions that are. Deep Reckonings exists in dialogue with the broader conversation about the ethical implications of synthetic media, and how artificial intelligence impacts our understanding of truth. The project seeks not to deceive nor demean, but to imagine and inspire. The videos make their fakery explicit not only to prevent misinformation, but also to leverage a superpower of synthetic media — that we can know they're fake and they still affect us. In this spirit, Deep Reckonings explores the question: how might we use our synthetic selves to elicit our better angels?
Then in 2017, I discovered the phenomenon of deepfakes. BOOM.
In early 2019, I released an imaginary reckoning with Pope Francis on Reckonings, as an audio prototype of what I hoped to do with synthetic video. I wrote the script, had a voice actor perform it, and made explicit that it was fiction. This was the first time I'd ever aired fiction on the show, and I didn’t know how listeners would respond. To my surprise, listeners loved it. I even heard from survivors of clergy sex abuse who, although they knew it was fiction, found it remarkably helpful to hear the imaginary Pope say the kinds of things they wished to hear the real Pope say. Deep Reckonings is now the culmination of a years-long exploration of how people change, the beauty of critical self-reflection, and the power of purposeful fiction.
Deep Reckonings is a series of explicitly-marked deepfake videos that imagine the most morally courageous versions of our most controversial public figures. It is an exercise in steel-manning where we often straw-man. The public figures include: Brett Kavanaugh wrestling with the way he responded to the sexual allegations against him; Alex Jones grappling with his spread of deceitful conspiracy theories; and Mark Zuckerberg confronting his techno-utopianism. These figures don't admit to actions that aren't publicly known, but instead take responsibility for actions that are. Deep Reckonings exists in dialogue with the broader conversation about the ethical implications of synthetic media, and how artificial intelligence impacts our understanding of truth. The project seeks not to deceive nor demean, but to imagine and inspire. The videos make their fakery explicit not only to prevent misinformation, but also to leverage a superpower of synthetic media — that we can know they're fake and they still affect us. In this spirit, Deep Reckonings explores the question: how might we use our synthetic selves to elicit our better angels?
|
In other words, how might we deepfake it ‘til we make it?
Deep Reckonings uses synthetic media to take Reckonings in a fictional direction. Hence the double-double meaning of "Deep Reckonings" — as the project is both deep and a deepfake, and speaks to both Reckonings-the-podcast and reckoning-the-phenomenon. Deep Reckonings won two Webby Awards in 2021. |
Deep Reckonings explores the question: how might we use our synthetic selves to elicit our better angels? |
WHY I MADE IT
The intention of Deep Reckonings is twofold: first, to make more room for growth in public, and second, to expand the prosocial use of synthetic media.
In Mark Zuckerberg’s now-infamous words, "to think [Russia] influenced the election in any way is a pretty crazy idea." In a similar vein, Brett Kavanaugh declared at his Congressional hearing, "this whole two-week effort has been a calculated and orchestrated political hit!" We're painfully accustomed to the deny-and-deflect playbook. But when accusations of wrongdoing are credible, this playbook is dishonest, polarizing, and misses an opportunity for growth. We need an alternative playbook.
My job as an artist is to make an alternative playbook that's more beautiful and powerful than the original. My job is to make a playbook that would move Zuckerberg, Kavanaugh, and Jones, along with Louis CK, Bill O'Reilly, and others to say: "that is the me I want to be." The purpose of this new playbook is to make critical self-reflection look stunning — so that we are moved to do it, and our public figures are moved to do it, and we make room for each other to do it. Especially because we live so much of our lives in public, and our public sphere can be so unforgiving, we need more room to be wrong, learn, change, redeem ourselves, and ultimately grow….in public.
In this respect, my first intention with Deep Reckonings is to make more room for ourselves to grow in public. Not only is it impossible to never transgress, but our transgressions can be vital sources of growth. What makes Deep Reckonings subversive is that it makes room for everyone to grow: it allows for Zuckerberg, Kavanaugh, and other public figures to be the kinds of people who would publicly take responsibility for their actions, and for the rest of us to be the kinds of people who would enable them to do that. And the next step of this work is to make it real.
In Mark Zuckerberg’s now-infamous words, "to think [Russia] influenced the election in any way is a pretty crazy idea." In a similar vein, Brett Kavanaugh declared at his Congressional hearing, "this whole two-week effort has been a calculated and orchestrated political hit!" We're painfully accustomed to the deny-and-deflect playbook. But when accusations of wrongdoing are credible, this playbook is dishonest, polarizing, and misses an opportunity for growth. We need an alternative playbook.
My job as an artist is to make an alternative playbook that's more beautiful and powerful than the original. My job is to make a playbook that would move Zuckerberg, Kavanaugh, and Jones, along with Louis CK, Bill O'Reilly, and others to say: "that is the me I want to be." The purpose of this new playbook is to make critical self-reflection look stunning — so that we are moved to do it, and our public figures are moved to do it, and we make room for each other to do it. Especially because we live so much of our lives in public, and our public sphere can be so unforgiving, we need more room to be wrong, learn, change, redeem ourselves, and ultimately grow….in public.
In this respect, my first intention with Deep Reckonings is to make more room for ourselves to grow in public. Not only is it impossible to never transgress, but our transgressions can be vital sources of growth. What makes Deep Reckonings subversive is that it makes room for everyone to grow: it allows for Zuckerberg, Kavanaugh, and other public figures to be the kinds of people who would publicly take responsibility for their actions, and for the rest of us to be the kinds of people who would enable them to do that. And the next step of this work is to make it real.
Especially because we live so much of our lives in public, we need more room to be wrong, learn, change, redeem ourselves, and ultimately grow….in public. |
My second intention for Deep Reckonings is to expand the use of synthetic media for prosocial purposes. The overwhelming majority of synthetic media online are nefarious in nature — mostly involuntary pornography with a dose of general disinformation and mean-spirited parody. Amid this sea of nefarious deepfakery, there’s a small but growing canon of synthetic media for good. Project Revoice creates personalized synthetic voices for ALS sufferers who’ve lost their ability to speak. In Event Of Moon Disaster features a fake Richard Nixon delivering the contingency speech he would have given if Apollo 11 had failed, warning viewers that synthetic media can be used not only to rewrite the present but also the past. Welcome to Chechnya, a documentary about the persecution of LGBTQ-identifying people in Chechnya, uses synthetic video to replace the faces and therefore protect the identities of the film’s protagonists. How else might synthetic media be used in prosocial ways?
|
Deep Reckonings invites the synthetic media community to leverage the extraordinary power of the medium for prosocial purposes. If this invitation moves you, check out the artist statement on prosocial synthetic media and let's join forces.
HOW IT WAS MADE
Deep Reckonings combines synthetic video of a public figure with real audio of a voice actor. (It would have been possible to produce a 'complete deepfake' by creating synthetic audio of each public figure, but for budgetary and legal reasons, the project opted for voice actors.)
In terms of the step-by-step process: first, a "target" video of a protagonist was chosen, e.g. of the press conference for Kavanaugh's acceptance speech. Then, the protagonist's script was written with the target video in mind, e.g. also of a press conference, but where Kavanaugh takes responsibility for the way he responded to the allegations against him. Next, a voice actor recorded a video of himself reading the script. Then, clips from the target video were 'matched up' with the video of the voice actor to align their facial movements, e.g. so that Kavanaugh was speaking when the voice actor was speaking, and pausing when the voice actor was pausing. Then came the deepfakery: the use of artificial intelligence to do "video dialogue replacement," i.e. manipulate Kavanaugh's mouth so that it looks like he’s saying the words of the voice actor. (Most deepfakes online involve face-swapping, which swaps someone's face onto someone else's body; video dialogue replacement involves keeping the face but replacing the dialogue, which is more technically complex.) The resulting synthetic video looks exactly like clips from the original target video, but with the dialogue replaced. The final video was then marked as fake in multiple ways: via intro credits, outro credits, a visual watermark, metadata, and in the scripts themselves, e.g. with the imaginary Kavanaugh saying 'I didn't write these words and I've never said them.' The Kavanaugh video also temporarily morphs his face into that of other high-profile men who've been credibly accused of sexual misconduct — Joe Biden, Louis C.K., and Bill O’Reilly — as an artful way of making its fakery explicit.
It's worth noting: this process is complicated! While the capacity to synthesize people's faces and voices is becoming more accessible, realistic-looking and sounding synthetic media is still a high-barrier endeavor.
In terms of the step-by-step process: first, a "target" video of a protagonist was chosen, e.g. of the press conference for Kavanaugh's acceptance speech. Then, the protagonist's script was written with the target video in mind, e.g. also of a press conference, but where Kavanaugh takes responsibility for the way he responded to the allegations against him. Next, a voice actor recorded a video of himself reading the script. Then, clips from the target video were 'matched up' with the video of the voice actor to align their facial movements, e.g. so that Kavanaugh was speaking when the voice actor was speaking, and pausing when the voice actor was pausing. Then came the deepfakery: the use of artificial intelligence to do "video dialogue replacement," i.e. manipulate Kavanaugh's mouth so that it looks like he’s saying the words of the voice actor. (Most deepfakes online involve face-swapping, which swaps someone's face onto someone else's body; video dialogue replacement involves keeping the face but replacing the dialogue, which is more technically complex.) The resulting synthetic video looks exactly like clips from the original target video, but with the dialogue replaced. The final video was then marked as fake in multiple ways: via intro credits, outro credits, a visual watermark, metadata, and in the scripts themselves, e.g. with the imaginary Kavanaugh saying 'I didn't write these words and I've never said them.' The Kavanaugh video also temporarily morphs his face into that of other high-profile men who've been credibly accused of sexual misconduct — Joe Biden, Louis C.K., and Bill O’Reilly — as an artful way of making its fakery explicit.
It's worth noting: this process is complicated! While the capacity to synthesize people's faces and voices is becoming more accessible, realistic-looking and sounding synthetic media is still a high-barrier endeavor.
BIO
Stephanie Lepp is a producer whose work strives to hold up a mirror — inviting us to grow from what we see. She’s the Executive Director at the Institute for Cultural Evolution, a think tank devoted to evolving consciousness and culture in America. Before that, she was the Executive Producer at the Center for Humane Technology, the organization at the heart of the Netflix documentary The Social Dilemma. Stephanie won two Webbys for Deep Reckonings, her series of explicitly-marked deepfake videos that imagine morally courageous versions of our public figures.
ACKNOWLEDGEMENTS |
Deep Reckonings was developed with the generous support of the Mozilla Foundation, Topic, and the Firehouse Fund. Editorial and ethical guidance were graciously contributed by Henry Ajder, Aviv Ovadya, Sam Gregory, Halsey Burgund, Shamir Allibhai, Jenn Beard, Brett Gaylor, Paul Johnson, Kevin Zawacki, Gena Konstantinakos, Ryan Chanatry, Jeff Seelbach, Drew Westen, Andrew Marantz, Aden van Noppen, Schuyler Brown, David Sauvage, Trix + Phil, Ryan Nakade, Layman Pascal, and Mitch Lewis. The contents of Deep Reckonings do not reflect the views of the Mozilla Foundation or the contributors, and do not speak for or on behalf of the Mozilla Foundation or the contributors.
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.