Deepfakes and Synthetic Media Defense

Kaila Mathis: Welcome. Thank you everyone for taking your Monday night to hang out with us. We know it is not easy to decide which New York security event to go to each night, so we appreciate you joining us for this one.

Today our panel is bringing together the founders of Dune Security and Reality Defender to announce a new partnership focused on one of the fastest growing threats in cybersecurity: synthetic media. Together we are tackling the challenge from both sides. Reality Defender is detecting deepfakes and manipulated content in real time. Dune Security is helping organizations build a workforce that can recognize and resist these attacks before they cause harm to the company.

Before we get started with questions, we wanted to play a clip from two of our CEOs.

[Deepfake video clip plays]

David DellaPelle (deepfake): You look a bit different today.

Ben Colman (deepfake): Actually, funny you mention that. Wait — are you — am I?

David DellaPelle (deepfake): Yep. We are both deepfakes right now.

Ben Colman (deepfake): That is actually brilliant and deeply unsettling. So I was thinking we should partner on an event about synthetic media threats.

David DellaPelle (deepfake): Are we about to authorize a $25 million wire transfer?

Ben Colman (deepfake): Well, as David's representative, I can authorize just about anything.

[Laughter]

Kaila Mathis: Pretty good, right? Do you remember saying that? Shout out to the content team for putting that together. But the truth is, anyone can do that now with today's tools.

I want to get started with introductions. I will hand it off to you first, Ben, to introduce yourself and what you are doing at Reality Defender.

Ben Colman: Sure. My name is Ben Colman. I am the co-founder and CEO of Reality Defender. We are a 48-person team based just south of Canal Street. We have spent the last five years building the leading real-time solution for deepfake detection — detecting AI-generated audio, video, images, and text. We are provenance-free and context-free, meaning we do not touch any personal data. We have brought to market a leading solution for regulated industries, scanning communications, phone calls, and video conferencing.

Our biggest challenge has been educating the world on how serious the problem is, which is why it was so exciting to meet a member of Dune on a flight back from Las Vegas and kick off this whole partnership. The education Dune brings really complements the detection inference that we provide. I will pass it to our CTO, Alex. I need to step out for a family matter — not an emergency — but I will leave you in very good hands.

Alex Lisle: Thanks, Ben. My name is Alex Lisle. I am the CTO here at Reality Defender. As Ben mentioned, we do context-free, provenance-free, large-scale real-time deepfake detection and AI manipulation detection.

Kaila Mathis: Amazing. Thank you. My name is Kaila. I am here with Dune Security as director of growth, and I am pleased to introduce our two founders, David and Michael.

David DellaPelle: Thanks so much. Really excited about this partnership. It is a perfect marriage of complementary technologies. When Zach first approached me about Reality Defender, I had heard of the company but did not realize they were in New York — five blocks from our office on Broadway just south of Canal. Just an amazing company and great to work with them.

My name is David. I am the co-founder and CEO of Dune. What we do is quantify employee risk and then individually remediate that risk for each employee in large companies. The reason we do that is because 90 percent of breaches start with employee error, and the attacks are getting much, much worse.

As we just saw on that screen, deepfake quality has skyrocketed. And let us be honest — we are somewhat the coastal elites here who have been watching this technology develop. We work in technology. A lot of people are completely unaware that it has come this far. It is an exciting time and also a very scary time, and technologies like Dune and Reality Defender are well positioned to help.

Michael Waite: I am Michael Waite, co-founder and CTO of Dune Security. We are very excited about this partnership. The entire paradigm we have been operating in within the cybersecurity space is evolving at a pace I do not think we have ever seen before. It is going to be fun to dive deep into that tonight.

Kaila Mathis: The first topic we wanted to discuss is the evolution of synthetic media. How is it changing, and why does it matter? Alex, this question is most relevant to Reality Defender: what has changed to make deepfakes such a pressing enterprise risk?

Alex Lisle: There are a few things. Like most attacks, deepfakes started as a science experiment — something research-based and quite far out there. As these attacks became more prevalent, they became commoditized. Tools were built. The hardware barrier to entry dropped. Then two other key factors fundamentally changed the landscape.

First, cybercriminals and hackers will see what works and copy that playbook. Once they realized there were vulnerabilities to exploit through synthetic media, they absolutely started leveraging them. Second, the most successful avenue for attack has always been social engineering. The most famous hacker in history, Kevin Mitnick, was a social engineer. That playbook has existed for a long time, and deepfakes have essentially poured gasoline on it.

The commoditization of hardware and tools, combined with an established and proven attack paradigm dialed up to full power, is really when you start to see a tremendous influx of this type of attack. This is likely only really happened this year. This is the year when you started seeing these attacks go from the exception to the norm, and you started seeing traditional attack vectors like data exfiltration and lateral movement begin to leverage deepfakes as part of the chain.

Kaila Mathis: That makes a lot of sense. Thinking about how we prepare users for this, a lot of companies are testing for the basics like phishing. What are they potentially missing by not incorporating testing and training for all attack vectors across different types of synthetic media?

David DellaPelle: When we started Dune a few years ago, the user risk space was completely overlooked and underinvested in. Things have really changed with the advent of generative AI and how social engineering attacks can evolve.

To answer the question: it is critical to not just deal with phishing. The email inbox is only one channel through which attackers can gain access. It is absolutely critical to look at encrypted channels that may not even be on the corporate network, SMS-based smishing, and deepfake voice calls. Vishing attacks have surged by a preposterous percentage — something like 5,000 percent increase. We are seeing it happen everywhere. If you are a CISO trying to reduce risk for a major enterprise, you have to quantify risk based on all available data, and multi-channel attack data is critical to that.

Kaila Mathis: Michael, who are the big groups now leading the charge on these attacks, and what does that look like for enterprises?

Michael Waite: Historically, as David mentioned, we just saw phishing messages trying to get through the secure email gateway and land in the inbox. That paradigm has evolved enormously. As cybersecurity infrastructure in organizations has gotten more secure and the inbox has become harder to reach, we now see large threat actors like Scattered Spider simply ditching the inbox entirely.

What is very frightening about that shift is that enterprises typically have no visibility into off-channel, encrypted attacks. They cannot even see the threats being launched against their organization.

And when you look at the amount of personal data publicly available on the dark web about everyone in this room — if you have ever been a T-Mobile or AT&T customer, or shopped at Home Depot, your full name, address, and social security number are likely available. That is exactly the information needed to call an IT help desk at a large enterprise and have credentials reset. That is what we saw with the MGM attack.

The threat actors behind this are not just a few people in Russia or China. They are well-architected, distributed groups with people in the US and a large global footprint, operating with more powerful tools than ever before.

Alex Lisle: I would just add that the attack has scaled dramatically. Before, social engineering was one person targeting one person. Now you can go to the dark web, pull a thousand identities, spin up an AI text-to-speech tool, plug it into an LLM, and bombard a large number of targets simultaneously. You do not need it to succeed every time. You only need it to succeed one or two percent of the time. There is a force multiplier at work that did not exist before.

You are also seeing attacks leverage channels that have been neglected for a long time, like VoIP, voicemail systems, and internal PBX infrastructure. The email phishing playbook worked because you could attack every employee at a company at once. Now you can do that at far greater scale with an LLM and a text-to-speech model. That is a very different landscape than what we had before.

Kaila Mathis: Are there specific functions or departments that you have found are most vulnerable to these types of attacks?

Alex Lisle: I think one of the worst things that can happen is when something that worked invisibly and was trusted suddenly stops working. Seeing was believing. Hearing was trusting. The judicial system records testimony because there is an inherent trust in direct perception. That has fundamentally shifted.

When that happens, systems that were built on that trust are now broken. We are seeing attacks appear in very unexpected places. We spoke to a large security company that said if someone failed their biometric check, they would get them on a Zoom meeting with their manager and ask: is this the person who works for you? That is now an attack vector. We have seen it used.

Hiring fraud is another major one. I used to think those cases were the exception. We spoke to a customer who conducted 10,000 job interviews last year. More than 6,000 of them were deepfakes. The majority. That is a staggeringly large volume, and this was a cybersecurity company with reason to attract nefarious actors.

There is also the issue of repudiation. We have customers in banking who say: a client asked us to do something, we did it, and then the client says that was a deepfake and they never gave that instruction. There is no way to repudiate it. These are aspects of the problem that do not appear in a normal cybersecurity playbook.

Kaila Mathis: You make a great point that it is not just business risk. It is also personal risk for the individuals involved. Someone with zero malicious intent could have their career negatively impacted because of these attacks. That brings me to you, Michael and David: how do you actually get people to relate to the risk? How do you make it relevant to the individual user?

Michael Waite: One of the best ways to help people understand this is to let them experience it hands-on. This was a real unlock for us when we were building out our platform. We want users to understand not just legacy risks, but the bleeding-edge attacks we are starting to see more and more.

You can give someone a video that explains how easy it is for a hacker to clone their voice with just five seconds of audio, but it lands so much more when you allow the user to actually do it themselves. When we were building voice cloning models in the office, the developers were having so much fun with it that we realized: why are we not putting this directly in the platform? We have the GPU infrastructure. Let us let users record a short snippet, create a high-fidelity clone of their voice, and hear it say something they typed. That is when they get it. When someone hears a convincing clone of their own voice and realizes it only took a few seconds of audio, it becomes real to them in a way that no video or training module can match. Showing users threats that we are seeing in the wild, in a way that actually lands, is critical.

David DellaPelle: I want to go back to one of the earlier points because I think it is important. The attacks are coming from every single angle. Imagine storming a castle — they are coming from the land, the sea, and the sky. Alex, I like what you said about how trusted verification methods have now become attack vectors. Bring your manager onto a Zoom call to authenticate — that is now something that can be deepfaked and exploited. That is a perfect example.

It is also important to recognize who these people are and what they want. And unfortunately it is all of the above. We have nation-state hackers sponsored by countries like North Korea. We are hearing from CISOs every day about a massive influx of fake job applicants. We now have a step in our own hiring process where we require candidates to come in person for a final interview, because that is currently the most reliable way to address it.

The commoditization of the technology also means that very junior actors — script kiddies, to use the term — can now break into very sophisticated organizations. This space is going to explode, and companies like Reality Defender and Dune are at the bleeding edge of it.

Alex Lisle: The other shift worth noting is that deepfakes used to be the attack vector and the payload at the same time — the classic "I'm your boss, send me $25 million" scenario. Now deepfakes are becoming a component of the broader attack chain. They can serve the reconnaissance phase, the access phase, or the social engineering phase. It is part of the kill chain in a much more sophisticated and productized way. The tools around this are becoming to deepfakes what Metasploit was to traditional exploitation — a packaged, accessible capability that lowers the bar for everyone.

[Deepfake detection game — audience guesses whether images are real or AI-generated]

Kaila Mathis: I think that exercise really drives home the point. When you are focused on it and it is top of mind, you get about half of them right. Now think about what happens when you pick up a phone call at 2 in the afternoon and someone you know asks you an innocuous favor. You would not stop to think: is this really the person I know?

And the latest models do not just sound like the person. They use the same turns of phrase. They talk the way that person talks. Think about how well you performed when it was the only thing on your mind. Then think about it when it is the last thing on your mind — when you would not even remember at the end of the day whether someone called and asked you to reset a password. That is the paradigm shift.

Alex Lisle: I want to add to that. Two years ago, one of the first widely circulated AI-generated videos was Will Smith eating spaghetti. It was laughably horrifying — an obvious abomination. And now, just two years later, the quality is photorealistic and there is often no way to tell.

The pace of innovation has been insane. You used to need a GPU cluster that was out of reach for most people. Now many of these models run on consumer or prosumer hardware. Not only are they getting dramatically better, but they can be run on machines that are affordable for ordinary people, let alone well-funded threat actors.

Kaila Mathis: Before we get into Q&A, I want to give each of you a chance to speak to what enterprises should actually do to prevent these types of attacks.

Alex Lisle: Training is always valuable. There are two aspects to think about. Training gives you the pause — it stops the click. But then you need something else: verification. The premise we operate from at Reality Defender is that the trust boundary is gone. Any information that flows into an organization should be verified. That is the whole point of what we do. Dune can provide the training to give you the pause; we provide the verification layer. You have to understand that we are living in a post-trust reality, and you need both the pause and the verification to have a complete response.

David DellaPelle: We are building workflows that address what happens once you have quantified risk. For roughly 95 percent of users, you can automatically reduce risk through adaptive training that gives the pause. The question is what you do with the remaining 5 percent — because it only takes one.

What we are building is a set of adaptive workflows that represent the last mile of Dune on the outbound side, where you can start to individually lock down high-risk users through performance management processes, identity security, email security, and so on.

Michael Waite: The other thing I would add is that enterprises need to take a critical look at how they have done human risk management historically. They need to review their policies. If someone is on a Zoom call, there needs to be a mechanism to verify who they are talking to, and people need to be trained on that process.

This needs to exit the GRC model of just ticking a box. A 10-minute annual video is not enough anymore. Enterprises need a holistic approach that looks at the individual, the nature of their role, the specific risks introduced by that role, and then trains them accordingly while adapting security controls around them. For large enterprises where change management is slower, this is especially important. We are still seeing just the tip of the iceberg with this next generation of attacks. It is going to get worse before it gets better, and we need to be training people on the attacks that are happening today, not the ones from 10 years ago.

Kaila Mathis: Let us open it up for Q&A. Any questions on synthetic media, detection, or response.

Audience member: Michael, you mentioned change management. This threat clearly applies beyond enterprise — it is also a national security issue, with a lot coming out of North Korea and China as a threat to western governments. How do you think new startups and the government are going to interact around this, given that it is a ubiquitous threat to everyone?

Michael Waite: You make a great point. Change management is slow in large enterprises, and it is even slower in government. But attacks are not just targeting companies anymore. Nation-state actors are going after critical infrastructure — telecommunications, water supplies, power grids — and they may already be inside those systems. Building relationships with both large enterprises and governments is absolutely essential, because those attacks are not just targeting a company's bottom line. They are putting health and safety at risk.

Audience member: What is your view on the promise of using real-time threat data to automatically inform and adjust compliance requirements and security controls, essentially making that a closed-loop, data-driven process?

David DellaPelle: I think we need federal guidelines, and probably legislation, to set the rules around AI. We saw this summer that the GENIUS Act gave good clarity to the stablecoin industry. We really need something similar for AI. Cybersecurity is going to be one of the most impacted areas, and we cannot have a fragmented state-by-state approach that creates friction and confusion. Federal clarity is essential.

Alex Lisle: I am going to say something slightly more contrarian. Forget about technology for a moment. AI is a tool. The government needs to think about two questions: what can I do now with generative AI that I could not do before, and what problems has it created? I do not think regulating AI itself will work, because all you are doing is giving actors in other parts of the world an unfair advantage. What you really need to think about is what decisions are allowed and what decisions are not allowed when generative AI is involved — controlling decisions around data, around the consequences of using AI, and around the biases inherent in these models.

We are living in a world where the line between online and physical reality is merging. The government needs to think about it in those broader terms, not just in terms of what generative AI can and cannot do. And critically: none of this works without data. Who has access to what data, what data is used to make decisions, and what information about individuals is allowed to drive automated decisions — those are the questions that will define the world we live in. With a typical LLM, there is no way of knowing what biases it uses in your context window to reach a decision. That is going to become a very serious problem very quickly.

Audience member: With the EU AI Act coming into full effect next year requiring synthetic media to be labeled, but with attackers obviously not going to label their content — how far has detection technology actually come, and can it be a reliable verification method?

Alex Lisle: A couple of things on that. First, on labeling: I used to work in computer games. Every major gaming platform has a private key used to sign software — a billion-dollar key that every company has tried to protect and every company has eventually lost. Building a trust boundary that you do not control does not work. That tells you something about how reliable a labeling regime will be when the people you are trying to stop are not going to comply.

On detection: when building any generative AI model, the objective is to produce output that a human believes is realistic, not to actually create a perfectly realistic physical simulation of the world. That distinction is important. A computer-generated image optimized to fool a human eye will still contain artifacts and statistical patterns that a machine can detect — an inherent messiness that comes from the real world and that AI struggles to fully replicate.

That is where our detection comes from: looking for those differentials. And because all generative AI models work on similar underlying algorithms, the signatures we look for are inherent across them. For the next five to six years, for asynchronous content, we are in a strong position. Real-time synchronous detection is harder purely because of the compute required, but we are in a good position for the foreseeable future.

Will someone eventually build a model designed from the ground up to defeat detection? Probably. But it is going to cost an enormous amount of money. As for provenance-based solutions like C2PA, they may eventually help, but we are probably 20 years away from that being a practical, widely deployed reality. In the meantime, inference-based detection is the right approach.

Kaila Mathis: Let us take one more question.

Audience member: I am the founder of an early-stage startup. I am using third-party services for payroll, bookkeeping, and international payments. No matter what I do to secure my own company, how do I know that those third parties have done their part? Are there certifications or standards I should be looking at?

Michael Waite: The best baseline you can look for is SOC 2 Type II compliance, and alignment with frameworks like ISO 27001 or the NIST CSF 2.0. I do hesitate to lean too hard on that advice though, because compliance does not equal security. It is correlated with good security outcomes but is not a guarantee. There is also a quality-of-auditor problem, even with SOC 2 Type II.

Third-party risk management is a genuinely hard problem. What I do personally is grill the leaders of the companies we are considering working with, and try to use industry-standard, larger vendors where possible. Some AI tooling can also help you analyze a vendor's security documentation quickly. Vanta also has a solid third-party risk management tool worth looking at.

Alex Lisle: The honest answer is that when you are a startup you have to move fast, and that means accepting some level of risk. Think about the worst-case scenario if a third party you depend on is breached, and make that tradeoff consciously. Some of those third-party service providers actually take security very seriously — probably more seriously than you can at your current stage. The first step of any security program is understanding your own risk, knowing what you are relying on, and making an informed decision about it. Compliance is nice, but it is not security. If I had a foolproof way to quantify third-party risk, I would be telling you from my private island.

Kaila Mathis: Thank you all. Our teams from Dune and Reality Defender are here tonight and happy to talk more. We are very excited to be building the future of synthetic media preparation, detection, and response, and happy that you are all here to do it with us. We have food, drinks, and great people for you to connect with. Thank you everyone.