The AI Arms Race: Can Deepfakes Be Defeated by Creating More Deepfakes?

In an era where manipulated media has become a significant threat, the concept of deepfakes - highly realistic and convincing audio-visual content created using artificial intelligence (AI) - has taken center stage. With the advent of consumer-grade AI tools, the creation of fake audio, video, and images has become remarkably easy, giving rise to a cottage industry of deepfake detection startups that use machine learning to identify manipulated media.

Reality Defender, one such startup, recently conducted an experiment with me, where we created a deepfake voice designed to mimic my own. The goal was to test the effectiveness of their AI-powered deepfake detection technology. What emerged was a fascinating insight into the challenges and limitations of detecting deepfakes.

The experiment revealed that even with significant training data, creating a convincing AI agent capable of carrying on a conversation remains a daunting task. The voice we created sounded human but lacked the nuances and familiarity that family members would recognize. This highlights the difficulty in defeating deepfakes - you need to be able to create them yourself.

Deepfakes have been used for nefarious purposes, including fraud, harassment, and memes. Scammers have cloned people’s voices, called their relatives, and had the voice say they’re being held for ransom. The proliferation of nonconsensual sexual deepfakes has led to a significant ethical concern, with child sexual abuse material being particularly disturbing.

The deepfake detection industry primarily exists to address one of these problems: corporate fraud. According to a study, deepfake fraud is now “industrial,” and companies like Reality Defender are using AI to combat AI. Their approach involves training AI models on large datasets to identify patterns that distinguish real from fake content.

Reality Defender’s CTO, Alex Lisle, explained their “inference-based model” which uses a student/teacher paradigm to train the AI. The company is effectively training AI to combat AI by creating an AI that can detect other AIs. This approach has its limitations, as we discovered during our experiment. Even with fine-tuning and significant data, the voice was not convincing enough for family members.

This raises questions about the effectiveness of deepfake detection tools in real-world scenarios. If a deepfake is designed to mimic someone’s voice or appearance, how can you detect it? The answer lies in speed and quality. Generative AI is slow, but Reality Defender’s model sacrificed quality for speed to generate a convincing voice.

The proliferation of deepfakes has led to concerns about personal identity protection. Chief product officer at Pindrop, Nicholas Holland, highlighted the challenge of protecting one’s personal identity in an era where AI-generated content can be used to deceive institutions and individuals alike. The question remains: how do we protect our personal identity?

In conclusion, defeating deepfakes requires creating more deepfakes - but with limitations. The effectiveness of deepfake detection tools is directly linked to the quality and speed of their AI-powered models. As the battle between AI-generated content and deepfake detection continues, it becomes increasingly important for institutions and individuals to implement robust security measures to protect against these manipulated media threats.

Source: https://www.theverge.com/report/913445/deepfake-detection-reality-defender-pindrop-ai