“Deep Fake” Video Technology Is Advancing Faster Than Our Policies Can Keep Up
This article is part of the magazine, "The Future of Science In America: The Election Issue," co-published by LeapsMag, the Aspen Institute Science & Society Program, and GOOD.
Alethea.ai sports a grid of faces smiling, blinking and looking about. Some are beautiful, some are oddly familiar, but all share one thing in common—they are fake.
Alethea creates "synthetic media"— including digital faces customers can license saying anything they choose with any voice they choose. Companies can hire these photorealistic avatars to appear in explainer videos, advertisements, multimedia projects or any other applications they might dream up without running auditions or paying talent agents or actor fees. Licenses begin at a mere $99. Companies may also license digital avatars of real celebrities or hire mashups created from real celebrities including "Don Exotic" (a mashup of Donald Trump and Joe Exotic) or "Baby Obama" (a large-eared toddler that looks remarkably similar to a former U.S. President).
Naturally, in the midst of the COVID pandemic, the appeal is understandable. Rather than flying to a remote location to film a beer commercial, an actor can simply license their avatar to do the work for them. The question is—where and when this tech will cross the line between legitimately licensed and authorized synthetic media to deep fakes—synthetic videos designed to deceive the public for financial and political gain.
Deep fakes are not new. From written quotes that are manipulated and taken out of context to audio quotes that are spliced together to mean something other than originally intended, misrepresentation has been around for centuries. What is new is the technology that allows this sort of seamless and sophisticated deception to be brought to the world of video.
"At one point, video content was considered more reliable, and had a higher threshold of trust," said Alethea CEO and co-founder, Arif Khan. "We think video is harder to fake and we aren't yet as sensitive to detecting those fakes. But the technology is definitely there."
"In the future, each of us will only trust about 15 people and that's it," said Phil Lelyveld, who serves as Immersive Media Program Lead at the Entertainment Technology Center at the University of Southern California. "It's already very difficult to tell true footage from fake. In the future, I expect this will only become more difficult."
How do we know what's true in a world where original videos created with avatars of celebrities and politicians can be manipulated to say virtually anything?
As the U.S. 2020 Presidential Election nears, the potential moral and ethical implications of this technology are startling. A number of cases of truth tampering have recently been widely publicized. On August 5, President Donald Trump's campaign released an ad featuring several photos of Joe Biden that were altered to make it seem like was hiding all alone in his basement. In one photo, at least ten people who had been sitting with Biden in the original shot were cut out. In other photos, Biden's image was removed from a nature preserve and praying in church to make it appear Biden was in that same basement. Recently several videos of Speaker of the House Nancy Pelosi were slowed down by 75 percent to make her sound as if her speech was slurred.
During a campaign event in Florida on September 15 of this year, former Vice President Joe Biden was introduced by Puerto Rican singer-songwriter Luis Fonsi. After he was introduced, Biden paid tribute to the singer-songwriter—he held up his cell phone and played the hit song "Despecito". Shortly afterward, a doctored version of this video appeared on self-described parody site the United Spot replacing the Despicito with N.W.A.'s "F—- Tha Police". By September 16, Donald Trump retweeted the video, twice—first with the line "What is this all about" and second with the line "China is drooling. They can't believe this!" Twitter was quick to mark the video in these tweets as manipulated media.
Twitter had previously addressed several of Donald Trump's tweets—flagging a video shared in June as manipulated media and removing altogether a video shared by Trump in July showing a group promoting the hydroxychloroquine as an effective cure for COVID-19. Many of these manipulated videos are ultimately flagged or taken down, but not before they are seen and shared by millions of online viewers.
These faked videos were exposed rather quickly, as they could be compared with the original, publicly available source material. But what happens when there is no original source material? How do we know what's true in a world where original videos created with avatars of celebrities and politicians can be manipulated to say virtually anything?
"This type of fake media is a profound threat to our democracy," said Reid Blackman, the CEO of VIRTUE--an ethics consultancy for AI leaders. "Democracy depends on well-informed citizens. When citizens can't or won't discern between real and fake news, the implications are huge."
In light of the importance of reliable information in the political system, there's a clear and present need to verify that the images and news we consume is authentic. So how can anyone ever know that the content they are viewing is real?
"This will not be a simple technological solution," said Blackman. "There is no 'truth' button to push to verify authenticity. There's plenty of blame and condemnation to go around. Purveyors of information have a responsibility to vet the reliability of their sources. And consumers also have a responsibility to vet their sources."
Yet the process of verifying sources has never been more challenging. More and more citizens are choosing to live in a "media bubble"—gathering and sharing news only from and with people who share their political leanings and opinions. At one time, United States broadcasters were bound by the Fairness Doctrine—requiring them to present controversial issues important to the public in a way that the FCC deemed honest, equitable and balanced. The repeal of this doctrine in 1987 paved the way for new forms of cable news channels such as Fox News and MSNBC that appealed to viewers with a particular point of view. The Internet has only exacerbated these tendencies. Social media algorithms are designed to keep people clicking within their comfort zones by presenting members with only the thoughts and opinions they want to hear.
"I sometimes laugh when I hear people tell me they can back a particular opinion they hold with research," said Blackman. "Having conducted a fair bit of true scientific research, I am aware that clicking on one article on the Internet hardly qualifies. But a surprising number of people believe that finding any source online that states the fact they choose to believe is the same as proving it true."
Back to the fundamental challenge: How do we as a society root out what's false online? Lelyveld suggests that it will begin by verifying things that are known to be true rather than trying to call out everything that is fake. "The EU called me in to talk about how to deal with fake news coming out of Russia," said Lelyveld. "I told them Hollywood has spent 100 years developing special effects technology to make things that are wholly fictional indistinguishable from the truth. I told them that you'll never chase down every source of fake news. You're better off focusing on what can be proved true."
Arif Khan agrees. "There are probably 100 accounts attributed to Elon Musk on Twitter, but only one has the blue checkmark," said Khan. "That means Twitter has verified that an account of public interest is real. That's what we're trying to do with our platform. Allow celebrities to verify that specific videos were licensed and authorized directly by them."
Alethea will use another key technology called blockchain to mark all authentic authorized videos with celebrity avatars. Blockchain uses a distributed ledger technology to make sure that no undetected changes have been made to the content. Think of the difference between editing a document in a traditional word processing program and editing in a distributed online editing system like Google Docs. In a traditional word processing program, you can edit and copy a document without revealing any changes. In a shared editing system like Google Docs, every person who shares the document can see a record of every edit, addition and copy made of any portion of the document. In a similar way, blockchain helps Alethea ensure that approved videos have not been copied or altered inappropriately.
While AI companies like Alethea are moving to ensure that avatars based on real individuals aren't wrongly identified, the situation becomes a bit murkier when it comes to the question of representing groups, races, creeds, and other forms of identity. Alethea is rightly proud that the completely artificial avatars visually represent a variety of ages, races and sexes. However, companies could conceivably license an avatar to represent a marginalized group without actually hiring a person within that group to decide what the avatar will do or say.
"I don't know if I would call this tokenism, as that is difficult to identify without understanding the hiring company's intent," said Blackman. "Where this becomes deeply troubling is when avatars are used to represent a marginalized group without clearly pointing out the actor is an avatar. It's one thing for an African American woman avatar to say, 'I like ice cream.' It's entirely different thing for an African American woman avatar to say she supports a particular political candidate. In the second case, the avatar is being used as social proof that real people of a certain type back a certain political idea. And there the deception is far more problematic."
"It always comes down to unintended consequences of technology," said Lelyveld. "Technology is neutral—it's only the implementation that has the power to be good or bad. Without a thoughtful approach to the cultural, moral and political implications of technology, it often drifts towards the bad. We need to make a conscious decision as we release new technology to ensure it moves towards the good."
When presented with the idea that his avatars might be used to misrepresent marginalized groups, Khan was thoughtful. "Yes, I can see that is an unintended consequence of our technology. We would like to encourage people to license the avatars of real people, who would have final approval over what their avatars say or do. As to what people do with our completely artificial avatars, we will have to consider that moving forward."
Lelyveld frankly sees the ability for advertisers to create avatars that are our assistants or even our friends as a greater moral concern. "Once our digital assistant or avatar becomes an integral part of our life—even a friend as it were, what's to stop marketers from having those digital friends make suggestions about what drink we buy, which shirt we wear or even which candidate we elect? The possibilities for bad actors to reach us through our digital circle is mind-boggling."
Ultimately, Blackman suggests, we as a society will need to make decisions about what matters to us. "We will need to build policies and write laws—tackling the biggest problems like political deep fakes first. And then we have to figure out how to make the penalties stiff enough to matter. Fining a multibillion-dollar company a few million for a major offense isn't likely to move the needle. The punishment will need to fit the crime."
Until then, media consumers will need to do their own due diligence—to do the difficult work of uncovering the often messy and deeply uncomfortable news that's the truth.
[Editor's Note: To read other articles in this special magazine issue, visit the beautifully designed e-reader version.]
Martin Taylor was only 32 when he was diagnosed with Parkinson's, a disease that causes tremors, stiff muscles and slow physical movement - symptoms that steadily get worse as time goes on.
“It's horrible having Parkinson's,” says Taylor, a data analyst, now 41. “It limits my ability to be the dad and husband that I want to be in many cruel and debilitating ways.”
Today, more than 10 million people worldwide live with Parkinson's. Most are diagnosed when they're considerably older than Taylor, after age 60. Although recent research has called into question certain aspects of the disease’s origins, Parkinson’s eventually kills the nerve cells in the brain that produce dopamine, a signaling chemical that carries messages around the body to control movement. Many patients have lost 60 to 80 percent of these cells by the time they are diagnosed.
For years, there's been little improvement in the standard treatment. Patients are typically given the drug levodopa, a chemical that's absorbed by the brain’s nerve cells, or neurons, and converted into dopamine. This drug addresses the symptoms but has no impact on the course of the disease as patients continue to lose dopamine producing neurons. Eventually, the treatment stops working effectively.
BlueRock Therapeutics, a cell therapy company based in Massachusetts, is taking a different approach by focusing on the use of stem cells, which can divide into and generate new specialized cells. The company makes the dopamine-producing cells that patients have lost and inserts these cells into patients' brains. “We have a disease with a high unmet need,” says Ahmed Enayetallah, the senior vice president and head of development at BlueRock. “We know [which] cells…are lost to the disease, and we can make them. So it really came together to use stem cells in Parkinson's.”
In a phase 1 research trial announced late last month, patients reported that their symptoms had improved after a year of treatment. Brain scans also showed an increased number of neurons generating dopamine in patients’ brains.
Increases in dopamine signals
The recent phase 1 trial focused on deploying BlueRock’s cell therapy, called bemdaneprocel, to treat 12 patients suffering from Parkinson’s. The team developed the new nerve cells and implanted them into specific locations on each side of the patient's brain through two small holes in the skull made by a neurosurgeon. “We implant cells into the places in the brain where we think they have the potential to reform the neural networks that are lost to Parkinson's disease,” Enayetallah says. The goal is to restore motor function to patients over the long-term.
Five patients were given a relatively low dose of cells while seven got higher doses. Specialized brain scans showed evidence that the transplanted cells had survived, increasing the overall number of dopamine producing cells. The team compared the baseline number of these cells before surgery to the levels one year later. “The scans tell us there is evidence of increased dopamine signals in the part of the brain affected by Parkinson's,” Enayetallah says. “Normally you’d expect the signal to go down in untreated Parkinson’s patients.”
"I think it has a real chance to reverse motor symptoms, essentially replacing a missing part," says Tilo Kunath, a professor of regenerative neurobiology at the University of Edinburgh.
The team also asked patients to use a specific type of home diary to log the times when symptoms are well controlled and when they prevent normal activity. After a year of treatment, patients taking the higher dose reported symptoms were under control for an average of 2.16 hours per day above their baselines. At the smaller dose, these improvements were significantly lower, 0.72 hours per day. The higher-dose patients reported a corresponding decrease in the amount of time when symptoms were uncontrolled, by an average of 1.91 hours, compared to 0.75 hours for the lower dose. The trial was safe, and patients tolerated the year of immunosuppression needed to make sure their bodies could handle the foreign cells.
Claire Bale, the associate director of research at Parkinson's U.K., sees the promise of BlueRock's approach, while noting the need for more research on a possible placebo effect. The trial participants knew they were getting the active treatment, and placebo effects are known to be a potential factor in Parkinson’s research. Even so, “The results indicate that this therapy produces improvements in symptoms for Parkinson's, which is very encouraging,” Bale says.
Tilo Kunath, a professor of regenerative neurobiology at the University of Edinburgh, also finds the results intriguing. “I think it's excellent,” he says. “I think it has a real chance to reverse motor symptoms, essentially replacing a missing part.” However, it could take time for this therapy to become widely available, Kunath says, and patients in the late stages of the disease may not benefit as much. “Data from cell transplantation with fetal tissue in the 1980s and 90s show that cells did not survive well and release dopamine in these [late-stage] patients.”
Searching for the right approach
There's a long history of using cell therapy as a treatment for Parkinson's. About four decades ago, scientists at the University of Lund in Sweden developed a method in which they transferred parts of fetal brain tissue to patients with Parkinson's so that their nerve cells would produce dopamine. Many benefited, and some were able to stop their medication. However, the use of fetal tissue was highly controversial at that time, and the tissues were difficult to obtain. Later trials in the U.S. showed that people benefited only if a significant amount of the tissue was used, and several patients experienced side effects. Eventually, the work lost momentum.
“Like many in the community, I'm aware of the long history of cell therapy,” says Taylor, the patient living with Parkinson's. “They've long had that cure over the horizon.”
In 2000, Lorenz Studer led a team at the Memorial Sloan Kettering Centre, in New York, to find the chemical signals needed to get stem cells to differentiate into cells that release dopamine. Back then, the team managed to make cells that produced some dopamine, but they led to only limited improvements in animals. About a decade later, in 2011, Studer and his team found the specific signals needed to guide embryonic cells to become the right kind of dopamine producing cells. Their experiments in mice, rats and monkeys showed that their implanted cells had a significant impact, restoring lost movement.
Studer then co-founded BlueRock Therapeutics in 2016. Forming the most effective stem cells has been one of the biggest challenges, says Enayetallah, the BlueRock VP. “It's taken a lot of effort and investment to manufacture and make the cells at the right scale under the right conditions.” The team is now using cells that were first isolated in 1998 at the University of Wisconsin, a major advantage because they’re available in a virtually unlimited supply.
Other efforts underway
In the past several years, University of Lund researchers have begun to collaborate with the University of Cambridge on a project to use embryonic stem cells, similar to BlueRock’s approach. They began clinical trials this year. A company in Japan, Sumitomo, is using a different strategy; instead of stem cells from embryos, they’re inducing pluripotent stem cells made from adults’ blood or skin and then reprogramming them into dopamine producing neurons. Although Sumitomo started clinical trials earlier than BlueRock, they haven’t yet revealed any results.
“It's a rapidly evolving field,” says Emma Lane, a pharmacologist at the University of Cardiff who researches clinical interventions for Parkinson’s. “But BlueRock’s trial is the first full phase 1 trial to report such positive findings with stem cell based therapies.” The company’s upcoming phase 2 research will be critical to show how effectively the therapy can improve disease symptoms, she added.
The cure over the horizon
BlueRock will continue to look at data from patients in the phase 1 trial to monitor the treatment’s effects over a two-year period. Meanwhile, the team is planning the phase 2 trial with more participants, including a placebo group.
For patients with Parkinson’s like Martin Taylor, the therapy offers some hope, though Taylor recognizes that more research is needed.
“Like many in the community, I'm aware of the long history of cell therapy,” he says. “They've long had that cure over the horizon.” His expectations are somewhat guarded but, he says, “it's certainly positive to see…movement in the field again.”
"If we can demonstrate what we’re seeing today in a more robust study, that would be great,” Enayetallah says. “At the end of the day, we want to address that unmet need in a field that's been waiting for a long time.”
Story by Freethink
Try burning an iron metal ingot and you’ll have to wait a long time — but grind it into a powder and it will readily burst into flames. That’s how sparklers work: metal dust burning in a beautiful display of light and heat. But could we burn iron for more than fun? Could this simple material become a cheap, clean, carbon-free fuel?
In new experiments — conducted on rockets, in microgravity — Canadian and Dutch researchers are looking at ways of boosting the efficiency of burning iron, with a view to turning this abundant material — the fourth most common in the Earth’s crust, about about 5% of its mass — into an alternative energy source.
Iron as a fuel
Iron is abundantly available and cheap. More importantly, the byproduct of burning iron is rust (iron oxide), a solid material that is easy to collect and recycle. Neither burning iron nor converting its oxide back produces any carbon in the process.
Iron oxide is potentially renewable by reacting with electricity or hydrogen to become iron again.
Iron has a high energy density: it requires almost the same volume as gasoline to produce the same amount of energy. However, iron has poor specific energy: it’s a lot heavier than gas to produce the same amount of energy. (Think of picking up a jug of gasoline, and then imagine trying to pick up a similar sized chunk of iron.) Therefore, its weight is prohibitive for many applications. Burning iron to run a car isn’t very practical if the iron fuel weighs as much as the car itself.
In its powdered form, however, iron offers more promise as a high-density energy carrier or storage system. Iron-burning furnaces could provide direct heat for industry, home heating, or to generate electricity.
Plus, iron oxide is potentially renewable by reacting with electricity or hydrogen to become iron again (as long as you’ve got a source of clean electricity or green hydrogen). When there’s excess electricity available from renewables like solar and wind, for example, rust could be converted back into iron powder, and then burned on demand to release that energy again.
However, these methods of recycling rust are very energy intensive and inefficient, currently, so improvements to the efficiency of burning iron itself may be crucial to making such a circular system viable.
The science of discrete burning
Powdered particles have a high surface area to volume ratio, which means it is easier to ignite them. This is true for metals as well.
Under the right circumstances, powdered iron can burn in a manner known as discrete burning. In its most ideal form, the flame completely consumes one particle before the heat radiating from it combusts other particles in its vicinity. By studying this process, researchers can better understand and model how iron combusts, allowing them to design better iron-burning furnaces.
Discrete burning is difficult to achieve on Earth. Perfect discrete burning requires a specific particle density and oxygen concentration. When the particles are too close and compacted, the fire jumps to neighboring particles before fully consuming a particle, resulting in a more chaotic and less controlled burn.
Presently, the rate at which powdered iron particles burn or how they release heat in different conditions is poorly understood. This hinders the development of technologies to efficiently utilize iron as a large-scale fuel.
Burning metal in microgravity
In April, the European Space Agency (ESA) launched a suborbital “sounding” rocket, carrying three experimental setups. As the rocket traced its parabolic trajectory through the atmosphere, the experiments got a few minutes in free fall, simulating microgravity.
One of the experiments on this mission studied how iron powder burns in the absence of gravity.
In microgravity, particles float in a more uniformly distributed cloud. This allows researchers to model the flow of iron particles and how a flame propagates through a cloud of iron particles in different oxygen concentrations.
Existing fossil fuel power plants could potentially be retrofitted to run on iron fuel.
Insights into how flames propagate through iron powder under different conditions could help design much more efficient iron-burning furnaces.
Clean and carbon-free energy on Earth
Various businesses are looking at ways to incorporate iron fuels into their processes. In particular, it could serve as a cleaner way to supply industrial heat by burning iron to heat water.
For example, Dutch brewery Swinkels Family Brewers, in collaboration with the Eindhoven University of Technology, switched to iron fuel as the heat source to power its brewing process, accounting for 15 million glasses of beer annually. Dutch startup RIFT is running proof-of-concept iron fuel power plants in Helmond and Arnhem.
As researchers continue to improve the efficiency of burning iron, its applicability will extend to other use cases as well. But is the infrastructure in place for this transition?
Often, the transition to new energy sources is slowed by the need to create new infrastructure to utilize them. Fortunately, this isn’t the case with switching from fossil fuels to iron. Since the ideal temperature to burn iron is similar to that for hydrocarbons, existing fossil fuel power plants could potentially be retrofitted to run on iron fuel.