For many decades, filmmakers have sought to change or enhance human biology with makeup, prosthetics, and body doubles to make their scenes feel more authentic. But until modern visual effect (VFX) technology, the results were a mixed bag. From films like The Irishman and Captain Marvel to Avengers: Endgame, VFX allows performers to overcome physical impossibilities. Sometimes, Hollywood VFX allows filmmakers to merge the face of actors with the bodies of other performers for roles they’re not ready for or capable of performing.
But as you can imagine, modern Hollywood VFX technology is expensive, delicate, and detailed work. Clearly, not everyone has a budget of a few hundred million USD to hire a VFX studio. Here is where deepfake technology comes into the picture.
What is a deepfake?
A “deepfake” refers to recreated media of a person’s appearance and/or voice by a type of artificial intelligence called deep learning (hence the name, deepfake). A Reddit user who shared deepfakes on the site coined the term in 2017.
Deepfakes are typically fake images, videos, or audio recordings. You may have seen popular videos of celebrities or politicians saying something they are unlikely to say in real life. These are common examples of deepfakes.
How do deepfakes work?
Instead of pure computer-generating imagery (CGI), deepfake uses a subset of artificial intelligence (AI) called deep learning to make synthetic media such as fake images, videos, or audio. There are multiple ways to make deepfakes. The most common method employs deep neural networks, encoder algorithms, a base video where you want to insert the face on someone else, and a collection of your target's videos. The deep learning AI studies the data in various conditions and finds common features between both subjects before mapping the target’s face on the person in the base video.
Generative Adversarial Networks (GANs) is another way to make deepfakes. GANs employ two machine learning (ML) algorithms with dual roles; the first algorithm creates forgeries, and the second detects them. The process completes when the second ML model can't find inconsistencies.
The accuracy of GANs depends on the data volume. That's why you see so many deep fakes of politicians, celebrities, and adult film stars, as there is often a lot of media of those people available to train the machine learning algorithm.
How much does it cost to make a deepfake?
Compared to Hollywood CGI, deepfakes require a small budget. You just need a good computer with a powerful graphics card. The time it takes to create one depends on the processing power and the project's scope — it can take weeks or hours. For much faster results, you can use a deepfake app that employs technology in the cloud.
Why doesn’t Hollywood use deepfakes?
The answer to “why doesn’t Hollywood use deepfake?" is that Hollywood does use AI and ML to enhance VFX. However, pure deepfake technology is still not good enough to meet the standards of cinema. Although deepfake images may look believable on your mobile phone, their flaws are far more evident on a bigger screen.
But deepfake technology is catching up. You may have seen deepfake technology improve videos in Star Wars films or The Irishman by Youtuber Shamook. Lucasfilm finally hired the video editor after he tweaked Mark Hamill’s appearance in the second season finale of The Mandalorian. But it's important to note that the deepfake edits on YouTube build upon the expensive work already done by traditional VFX.
Examples of deepfakes
Well-known examples of deepfakes include a viral video of a fake Barack Obama saying: “President Trump is a complete and total dipshit.” There’s also a deepfake video of Mark Zuckerberg bragging about controlling the data of billions of people. But one of the most amusing deepfakes is by Collider, featuring a roundtable of deepfaked celebrities like Tom Cruise, George Lucas, Robert Downey Jr., Jeff Goldblum, and Ewan McGregor.
What is deepfake audio?
Creators can also use deepfake technology to make fake audio that sounds remarkably similar to the original voice. An example of this is Roadrunner: A Film About Anthony Bourdain (2021), where the director used deepfake to recreate the late celebrity chef's voice digitally. While the AI algorithm narrated only a few lines, it divided opinions on the ethics of the technology.
What is a deepfake on TikTok?
Face swapping is taking off on TikTok thanks to viral Tom Cruise deepfakes on the social media platform. But the quality of the Tom Cruise TikTok deepfakes is a cut above because the creator, Belgium VFX specialist Chris Ume, took weeks to make them. He also had the services of Miles Fisher, a Tom Cruise impersonator.
Can deepfake be detected?
For some time, there was a widely held belief that deepfakes were going to change everything in terms of political manipulation, but that didn’t happen. There were also fears of deepfake ransomware attacks in the future, but that hasn’t materialized so far either. Deepfake fakes aren’t advanced enough to fool everyone, especially nowadays, when a lot of people hold a natural distrust for media. Here are some signs of a deepfaked video:
- Poor lip-synching
- Mismatched skin tones
- Flickering at the edges
- Unnatural expressions
- Unnatural eye movement
- Lack of emotion leading to the uncanny valley effect
- Strange looking hair
- Fake looking teeth
- Misaligned faces
- Videos look unnatural when zoomed in or slowed down
Although there are multiple ways to recognize deepfakes, the technology is improving. For example, when American researchers declared that people in deepfakes don't blink normally in 2018, deepfakes with realistic blinking began hitting the Internet almost instantly.
Some researchers from places like Facebook and Michigan State University are fighting deepfake AI with an AI technology that identifies deepfakes. Facebook’s AI uses reverse engineering to track a deepfake’s origin, even when it’s the only data an investigator has for analysis. Meanwhile, Sony is also creating technology that detects deepfaked videos. Such tools may come in handy and stop tricksters like the Zoom deepfaker, who fooled politicians twice.