- Deepfakes are made using AI to swap the likeness of one person in an image with another.
- QTCinderella was among Twitch streamers who had deepfake porn of them shared online recently.
- Experts told Insider deepfake porn can be traumatic for victims, leaving them scarred.
Twitch gamer Atrioc paying for deepfake pornography of women streamers has reignited the debate around their potential harm.
Atrioc, real name Brandon Ewing, was live streaming on January 30 when viewers saw an open tab on his browser for a deepfake website, featuring images created using artificial intelligence to make it appear like people were engaging in sexual acts, according to entertainment news website Dexerto.
One user screenshoted the stream and shared it on Reddit. The widely viewed post featured the address of the deepfake site, meaning that the images and names of women deepfaked were seen by many. The thread has since been locked my moderators.
One of the victims of the deepfake website visited by Ewing was fellow gamer, QTCinderella.
QTCinderella tweeted the day after the screenshot was shared: "The amount of body dysmorphia I've experienced since seeing those photos has ruined me.
"It's not as simple as 'just' being violated. It's so much more than that."
—QTCinderella (@qtcinderella) January 31, 2023
Ewing later apologized for viewing the content, saying in a livestream that he clicked an advert on Pornhub for the deepfake website due to "morbid curiosity."
He later announced he would stop making content, and said that QTCinderella told him his actions had created a "wildfire" of deepfake pornography.
He said he wanted to "combat the damage," and was engaging with law firms in order to remove the content from the internet.
Most deepfakes are non-consensual porn featuring women
Since deepfakes started appearing online in 2015, they have become more convincing year on year, growing in quality and quantity, Matthew B. Kugler, a professor of law at Northwestern who researches them, told Insider.
Experts fear that deepfakes can be used to spread misinformation, for example for political gain, but evidence suggests that women in particular have been victimized.
Research by Sensity AI, a company that monitors deepfakes, found in 2018 that 90% to 95% of deepfakes were non-consensual porn, which can be traumatic for the victim, said Kuglar. 90% of that was non-consensual porn of women.
A 2019 report from DeepTrace, a technology research company, found that 99% of pornographic deepfakes featured female celebrities.
In the report, Danielle Citron, professor of law at Boston University, said: "Deepfake sex videos say to individuals that their bodies are not their own and can make it difficult to stay online, get or keep a job, and feel safe."
Kugler chimed that pornographic deepfakes threaten the victim's sense of self-control, self-presentation, and self-ownership. The results can be disorienting for anyone having AI content created about them, even consensually. This includes users generating images on Lensa AI, an app that creates avatars from a photo of a person, he said.
"With deepfakes, you could make me do anything. You could make me do things that are illegal, you can make me do things that are depraved. I might actually prefer to be featured in the real stuff," he said.
Victims of deepfake porn can be left feeling scarred and betrayed
Psychotherapist Lisa Sanfilippo, whose expertise includes sexual trauma, told Insider that creating false pornographic images "is a major violation."
She said that for the victim "seeing images of yourself — or images that are falsified to look like you, in acts that you might find reprehensible, scary or that would be only for your personal life can be very destabilizing — even traumatizing. There is no ability to give consent there."
"It's abuse when someone takes something from another person that has not been freely given to them," she said.
The effect on the victim depends on the person, Sanfilippo, but it can leave them feeling scared, scarred, and betrayed.
There is also a threat to safety for victims of deepfake porn. Journalist Rana Ayyub wrote in HuffPost that in 2018 she fell victim to deepfake porn while investigating corruption within the Indian government.
The deepfakes went viral, which led to harassment and threats. In the article she said: "From the day the video was published, I have not been the same person."
Kugler is now advocating for a change in law when it comes to non-consensual deepfakes, with deepfakes not technically being illegal in the US.
In the UK, an amendment to the Online Safety Bill will mean that creating, viewing or sending deepfake porn will be illegal and could be punishable by two years in prison. The bill is set to come into effect this year.
Back in the US, Kugler said: "Will laws be passed prohibiting any of this? I tend to think yes."