DtSR Episode 722 - Vulnerability Math Ain't Mathing

TL;DR: Robert "RSnake" Hansen & Jeremiah Grossman join the pod to talk about the "Vuln-pocalypse", or rather, the lack thereof. We do some math, discuss whether it's worth patching vulnerabilities attackers don't exploit, and discuss a potential alternative to the madness of vuln management and patching we're all living with today.

Guests

• Robert "RSnake" Hansen
• Jeremiah Grossman

Description

The “vulnerability apocalypse” makes great headlines, but our day-to-day reality is stranger: more scanners, more CVEs, more urgency, and still the same breaches. We sit down with Jeremiah Grossman and Robert Hansen from Root Evidence to ask a blunt question most security teams avoid saying out loud: if the overwhelming majority of known vulnerabilities are never exploited, why are we treating every patch like a debt we must repay at any cost?

We dig into where vulnerability management went off the rails. CVSS scores and severity tiers often turn into a kind of black magic, especially at enterprise scale where “patch everything” can be operationally impossible and sometimes actively dangerous. We share real-world patching fallout, talk about why prioritizing millions of findings is like turning the Titanic with a teaspoon, and outline what “reasonable” can look like when you stop optimizing for perfect dashboards and start optimizing for outcomes.

The turning point is data. Cyber insurance carriers and DFIR teams perform root cause analysis on real claims, and that actuarial view shows which remotely exploitable vulnerabilities actually drive financial loss. We talk about focusing on known exploited vulnerabilities, why a relatively small list of CVEs can matter more than thousands of “critical” alerts, and how controls like MDR, canary tokens, and segmentation help prevent exploitation from becoming a business-ending event.

We also pressure-test the AI panic. According to multiple carriers, AI-attributed losses are effectively zero when you exclude phishing, raising an uncomfortable thought: are we funding fear instead of risk reduction? If you want a more evidence-based approach to patch management, cyber risk, and cybersecurity ROI, hit play, subscribe, and share the episode, then leave us a review with the one change you’d make to how your team prioritizes vulnerabilities.

YouTube Video: https://youtube.com/live/wC1v8Vg7qr4

Transcript

Copy Transcript Copied

Welcome And The Vuln Apocalypse Claim

SPEAKER_02 0:51

Alright, here we go. Good morning, good afternoon, and good evening. Welcome down the security rabbit hole to another edition of your favorite cybersecurity podcast. Welcome down the rabbit hole. Welcome to the Down the Security Rabbit Hole podcast. I've got, well, returning guests, luckily, and this is uh slightly coincidental because I didn't know that the thing that we're talking about is gonna happen. But like I just got off a really cool webinar with Robin Jeremiah here. You don't say so. What's that? You don't say listening to those guys talk about the Volm apocalypse. And we had a podcast scheduled on this topic. So I figured let's let's poke at some of those interesting points because the I I agree the math don't math. It's not mathing. Uh the the the the apocalypse I was promised has not shown up, and wait longer doesn't seem like a good answer. But first, quick intros. Oh, Jeremiah, you go first.

SPEAKER_06 1:53

Raps, good to see you. So as you know, I've been in cybersecurity not quite as long as Robert, but still almost three decades at this point. I've been doing it longer than I have it. Founder of Whitehead Security. I was an early employee over at Sentinel One, founded and exited BitDiscovery, and now CEO of Root Evidence. So cyber is what I do.

SPEAKER_02 2:13

So so you know a little bit about the cyber thing and you know a little bit about warranties. We'll come back to that.

SPEAKER_00 2:18

Sure.

SPEAKER_02 2:19

Mr.

SPEAKER_03 2:20

Snake. Yeah, I'm Robert Hansen. Thanks for having me, Raf. I have been in computer security for 31 years and the CTO of Root Evidence. But today I think we're here to talk about Jarr and his crazy experience because a good chunk of what we talked about was actually all his brainchild this time. So it was refreshing because I normally I'm I'm like front and center on that data stuff, but this was all his.

SPEAKER_02 2:45

So well, you guys have both been putting out a bunch of interesting content on analyses, and and Rob, we've had you on twice to talk about the tweets, about the the the whole supposition and and the position that we've we've effectively been doing vulnerability management and patching wrong. I tend to agree. I I you know having been in the field not quite as long as you guys, but long enough to know. I remember when we called it computer security, infosec, and enterprise security, cybersecurity, you know, all the things. Or just the weird kid that you know spent his nights banging on the keyboard trying to break things. Um maybe I'll start with this. The the background, uh, background for this is just you know, we we've had years and years and years of vulnerabilities piling up. Uh, what amounts to like what is it, a dozen now different scoring, half dozen scoring systems? And is it exploitable? Is it not exploitable? Is it currently being exploited? Does somebody know about it? This is somewhere between a crystal ball and and and magic,

Tech Debt Versus Real Risk

SPEAKER_02 3:55

and I don't I don't mean that in a good way. And now, of course, or recently, the oh I by the way, I just read the open AI. You know, our latest model can now find vulnerabilities nobody's ever seen before. Like, okay, cool, cool, well. Um can't wait for that one to break out of consolitary confinement and hack another.

SPEAKER_06 4:19

That's that's what we needed more volumes that bad guys don't care about.

SPEAKER_02 4:23

Right. So we've had three decades, probably more, of legacy tech debt and volume debt that's been piling up. And and the question I asked on a recent show of ours is if you've got like remember Jeremiah asked me, uh gosh, was it it was in National Harbor, Maryland, one time. We were talking about vulnerabilities and web apps. He said if you have 10 uh SQL injection vulnerabilities, you saw nine of them. Can you tell me how much risk you've decreased? And the answer is like, eh. Somewhere between here and there, but maybe somewhere like this. But so you know, we we've piled up decades of tech debt. We've piled up decades of vulnerabilities we haven't patched. We've played whack-a-mole with all the uh scoring systems, and now AI is gonna cause the apocalypse because they're gonna find vulnerabilities and everything, everything will be vulnerable, everything will get hacked. But wait, it hasn't quite turned out that way.

SPEAKER_06 5:24

Can we talk about that tech debt concept? Because when we talk about debt, at least in a financial context, it's we've borrowed and then there's interest and we have to pay it back one day. When it comes to patching security vulnerabilities, does that scenario really apply? Meaning we have a backlog of patches we haven't applied for security problems. But if the adversary never, ever, ever exploits them, why do we ever, ever, ever have to patch them? Is it really debt that accumulates with interest in the same way as we imagine the term? Is that really true? Okay. 98.5% of all known vulnerabilities have never been exploited. Ever. That we know of.

SPEAKER_03 6:10

That we know of. We can only go by what we know about CVEs. If you actually increase it to all the things that haven't made a CVE that should be a C VE, it's it's substantially way worse than that.

SPEAKER_06 6:25

Well, the percentage is lower. It gets into the tenths.

SPEAKER_03 6:29

Yeah, tenths of a percent.

SPEAKER_06 6:33

What okay, let's let's let's try it this way. And let's be really honest about this because we're asking people to spend time and money organizations patching that stuff is not free. I mean, we we have to really think of if most vones will never be exploited, what is the value of finding let alone patching these this can I can I give you can I give a potentially dangerous real-world example?

SPEAKER_02 6:59

Because I I don't I wonder if if anybody else is having a difficult time.

SPEAKER_06 7:02

We're down the rabbit hole, Raf. Lay it on us.

SPEAKER_02 7:05

So you you you go to your you go to a mechanic shop to get your oil change, and they tell you that you've got seven other things that need to be done immediately because they could cause your car to explode on the highway. You choose not to fix any of those. 300,000 miles later, nothing's happened. Is where we are.

Why 98.5% Never Get Exploited

unknown 7:25

Yeah.

SPEAKER_02 7:27

Okay.

SPEAKER_06 7:27

Um, and it's not just like from the last year. I mean, go back as far as you want. This is the number. It's like, you know, the I just can't count CVE vulnerabilities, but it's about 1.5% now. It was about that number the year before, the year before that. A decade back, it was the same number. The adversaries don't need that many vulnerabilities. We're we're wasting a lot of time and energy and money fixing things that don't have to be.

SPEAKER_03 7:52

But to your to your analogy there, Raf, I mean, you you're saying like a qualified mechanic finds things wrong with your car. Any one of them could cause massive catastrophe. And and uh, and according to that mechanic, whether we can trust them or not, some amount of time those things end in catastrophe. You you might even say, well, that's such a slow, that's such a low number, it's only eight things. I might as well get it fixed. But what if it's not eight things? What if it's hundreds of thousands of things or millions of things, or tens or hundreds of millions of things, like some of our customers we're talking to? Well, and then some of them are like, Well, this thing could poke you if you rubbed your eyeball on it, or whatever. It's like, I mean, what? Like, no one's doing that. Like, these are just coming to reasonable.

SPEAKER_02 8:34

I I keep coming back to Spectre and Meltdown. Like the world had a literal meltdown over this, and it was such a nothing burger for 99.9998% of the world. I I I don't, yeah, you're right. And by the way, I've been having worked in large enterprise, I have been in the position where you get a vet, you get an industry score that says this is a 10 out of 10, this must be patched immediately. You go to bat, you win the argument, the mid-range Unix team applies the patch, you blow up an entire card processing system for 36 hours, and the and I'm willing to stake my career on the fact that that was never going to be exploited anywhere in that environment.

SPEAKER_06 9:23

That's that's the deal. Now, here's the thing before, and I'll qualify before, these arbitrary scoring systems, patching these issues the way we have been, it's the best we could do with the information that we had at the time. It's only someone recently do we know what vulnerabilities are exploited. It's only someone recently that we gotten data from the insurance carriers on what leads to financial loss. What it's told us is that all our guessing previously, unfortunately, was just wrong. It was just wrong. So the issue wasn't that we were doing it wrong before. The issue now is are we going to continue doing it wrong now? Because that's the way it's always been done.

SPEAKER_02 9:59

But what we're what you're asking for is an entire industry of multi-billion dollar, maybe more industry that's built around scanning, classifying, and forcing and tracking and prioritizing and God, whatever else is in there of vulnerabilities that effectively you're saying don't freaking matter.

SPEAKER_06 10:23

We know statistically they don't matter. And maybe right there, all those characteristics is exactly the problem. Why aren't they focused on reducing financial loss rather than focus on fixing as many vulnerabilities as humanly possible? That's that's contradiction. I mean, what what benefit is to the business if you're gonna spend the same amount of money patching as you would of getting hacked? I mean, what's what's the issue?

SPEAKER_03 10:46

Back back to your car analogy. Let's say you have 100,000 volumes to go fix or something. You're gonna spend more money in the car's worth. Why are we doing this again?

SPEAKER_02 10:54

I've had that conversation as well. Like, why would we why would we do this? And and I I know we gosh, it's been a long time, but I had this, we've had this conversation on this podcast. I don't know if it was you, Rob, I think somebody else before. But the the conversation of like at some point you just go, uh I I'm not gonna fix all this. I'm gonna let my insurance carrier like just like I'm gonna do my best, and then the rest is gonna be insurance.

SPEAKER_06 11:20

And then right there, what is our best? What is what is reasonable? Right. But like is our is our best using everything we got and all the patch capacity that we have, is that our best? Or is it best like this is the conversation we have with CSOs now? What if you just uh look for, scan for, and fix all the known exploited vulnerabilities? That way, when and if you get breached, you'll be the first one that would ever get breached on that issue. The first one. I mean, who can expect any more than that? You were the first. There's nothing you really could more you could have done. How would you how could anybody possibly know what the next vulnerability is? You know, that there are ways to do this. Like patch management can only go so far. At some point, fast detection response, like we talked about earlier, MDR, canaries, and things like this. Just because you get exploited doesn't mean there's a loss. There's a way to minimize damage, other than patch everything at all times. That's absurd.

SPEAKER_02 12:21

So this this is this is part of, I think, a broader problem in the way that our our niche of IT operates. It's very silo focused, right? So the Vuln management, the scanning and patching people will tell you this is the most important thing, do this above all else. The detection and response people will tell you that's

When Patching Causes The Outage

SPEAKER_02 12:45

the most important thing, do this above all else. The like everybody else has their the most, this is the most important thing. And we really have been terrible. And I mean, 30 years in, we've been terrible at saying this is what reasonable looks like across the entire spectrum of things you're supposed to do. I think maybe because that's hard. I haven't seen a lot of effort there.

SPEAKER_06 13:10

You know, it might be cultural. Let me just get let me just lay a different analogy for you or a different concept. Going back 25, 30, 25 or so years when voe management was really getting going, you really could scan for everything, and you really could you could scan for everything, and you really could patch for everything. Yeah. You you really could. Over the years, the environments got so big, the number of bones got so big, where now you really just can't. But I think that culture stuck that scan and patch everything. I mean, before then you know, we when we figured out earlier that we couldn't patch everything, then we prioritized. You have to patch everything but in this order.

SPEAKER_02 13:52

And right now, well, but we we drew this again, it's it's criticals, highs, mediums, lows, and then everything else. And and rather than taking it as a I think rather than looking at it from a rational, like what is going to cause the damage perspective, we and Rob, we talked about this, like the whole the fact that you know scoring is an absurd black magic that is it's also wrong, for instance.

SPEAKER_03 14:21

We I think there's a many vulnerabilities that have no score at all that are currently being used by the financial adversaries. None of no score.

SPEAKER_01 14:29

Oops.

SPEAKER_02 14:30

Um so uh the the the the ins I think the incentive structure is completely broken too. Like we you Jemai, you're you're right. In that went back a long, long time ago, when you could when somebody asked, hey, where are all the computers in this company? You could go, there they are, yeah, and there's my T1, and there's my firewall at the end of it, uh, in between that the internet and us, and I can I can put my you know, I can go touch all the stuff. But that that that ability disappeared real fast. Long ago, long ago. Right, and nothing's and we have we haven't like the the the the practices of invulnerabil vulnerability management remediation hasn't evolved at all. Safe to say in the like prioritization space, but like prioritize prior trying to prioritize 10 million things feels like trying to turn to Titanic with a teaspoon, and it's just it like it is categorically failed. Like there's nothing that's gonna convince me that we've done a reasonable job anywhere.

SPEAKER_06 15:35

Well, historically, we we didn't know who the bad guys were, we didn't know we're gonna show up, we didn't know their level of sophistication, we didn't know what they're what they're gonna hit, they didn't know, we didn't know what exploits they were gonna use against us, we didn't know a lot. So, all of cybersecurity, all of our standards, practices, best practices is all against all those unknowns. Protect everything against everything at all times, especially the super hacker. It was a large guess because we were blind to all the actuarial data. That was before. Now we have actuarial data. We know who the adversaries are, we know their general level of sophistication, we know what phones they use, we know the types of systems they target, we know the industries that they target, we know the lost data, we know exactly what to do. The only question is will we do it a more evidence-based approach?

SPEAKER_03 16:25

That is what we can actually hold on. So Jared, this is probably a good time. So we we've we've been talking about this internally. At some point, let's say the Vulhampocalypse continues and we continue to see this rapid increase in new CDEs being created, not utilized, just created, right? I think everyone can agree that's actually happening. So at what point does the cost of goods sold for the average, you know, very large vulnerability management company start going up to the moon as well? And at what point do they get so expensive that people literally can't afford them, like except for maybe a handful of very specific machines? And therefore they become all but useless until they realize, oh shit, we really need to prioritize and only look for the things that matter. Like eventually everyone has to come to where we are. We're just already there.

SPEAKER_02 17:26

So you're right. So I I think it's an interesting point you make that you didn't you didn't say this out loud, but I think it bears putting this out there. I I think there may be a misconception that that's been following us for a long time that all the scanners, vulnerability scanners out there scan for like all the known bones, and they don't. Right? They scan for some small subset that they think is important and that they can re reliably and reasonably find in you know in networks. So you're still not like no matter what, you're not getting the whole thing. You're getting somebody's version of what they believe is important against a scoring system that some person believes is important

Patch The Known Exploited Set

SPEAKER_02 18:10

that's created some math to prove it.

SPEAKER_03 18:12

Well, but even that even that limited list is still far too long.

SPEAKER_06 18:16

It's it's two it's two guesses. I'm going to guess at which ones I have to scan for, and I'm gonna guess the relative severity of them. There's no hard data behind any of this, it's all just guesses. What we're saying is like we just wrote a book about it, it's called The End of Guessing. How about we don't guess anymore? We know what the bad guys are using. And they're not using new stuff every day. Just fix the ones that they're using. There's 619 of them. We know the exact number. Fix those, force a shift, force an expense onto the adversary. It's the exact thing we haven't done. Imagine for a moment that we have just talked with tons of cyber insurance carriers. They are suffering 30 to 40 percent of all their claims are remotely exploitable vulnerabilities, and it reduces down to 619 total CVEs. We're fixing the wrong ones in the wrong order. That's clear.

SPEAKER_02 19:12

Yeah. So this is this is a a long-standing thing that you guys have you guys have both said in different, probably going back 30 years, but the idea here is to make ratchet up the expense for the adversary. We have been doing a terrible job at that because you know it's it's widely been known, and I think it's now accepted that just because some exotic new vulnerability got announced yesterday doesn't mean anybody's gonna go try to do that. And adversaries very rarely go spend their own money to go buy a zero day that nobody else knows, likely because the target they're trying to exploit has so many much easier targets, right? That they can just simply hit that don't cost them anything.

SPEAKER_06 19:59

It might even be simpler than that. What they're doing is working, they don't even pay attention to the other stuff. When it stops working, they'll buy another tool. That's it. There's there's no more calculus than that. Yeah.

SPEAKER_02 20:12

So if we ratch up, so okay, we we we we you said 617. Is that the number? Is that the number you use 619? 619, fine, six nineteen. I as a as a as a company fix all 619. All my peers start to get towards that. Does the then the adversary then shift to another?

SPEAKER_03 20:32

Does that now do we keep playing the um we don't we don't know? I mean they they could shift to other vulnerabilities, uh, they could shift to entirely new forms of attack that we've never seen before. They could you start utilizing AI, which they are actually not doing right now. They could switch to other areas like uh Web AppSec, for instance, which they've kind of ignored for a while, um, or entirely new areas or double down on places that are very successful already, like phishing or drive-by downloads, that kind of thing. So there's a lot of options. But we haven't made them shift from CVEs at all. I mean, they're they're still running rampant there, so why would they shift?

SPEAKER_02 21:16

Yeah, why would anybody shift if if what they're doing is cheap and working?

SPEAKER_03 21:21

Right and and isn't getting them caught, right? They're not go, they're not getting busted. Very small half-oR, but most of them are not.

SPEAKER_06 21:29

So, in our view, in the case of voe management, scan everything for at least the bones that matter every single day. Force the shift. And since we're only talking about 619, use your excess patch capacity and make some guesses. Uh, vulnerabilities are exploited do cluster around a few characteristics by particular vendor, by class of attack. They do uh concentrate. You can make a guess with the patch capacity that you have. After that point, then you're gonna get a lot more bang from the buck after with canaries and networking. segmentation and MDR and things like that. But back to patching everything immediately. No. No, no, no, no. Go talk to it in a retailer around Christmas about that. Like, yeah, right. Yeah. Yeah.

SPEAKER_02 22:15

Anytime, anytime between the week before Thanksgiving to two weeks, two weeks after New Year's.

SPEAKER_06 22:20

And it it boggles the brain because this is this is a our reputation on the line here. When cybersecurity is telling the head of IT to patch everything at all times, that's a reputational hit. Everybody knows you can't do that. It's not worth the risk. You will risk the hack rather than risk a self-inflicted harm. Which one do you want to go to the board with? Yeah.

SPEAKER_02 22:42

Yeah. Well there there's two there's two possibilities here when you when you say that we must do this or bad things will happen. You do it and bad things still happen or you don't do it and nothing happens. Both of those are equally terrible for us from a reputational perspective.

SPEAKER_06 22:58

Well you know the in the case the business doesn't really care about so much the reputation of the cybersecurity professional. They care about downtime. They care about financial loss. And let's can we do can we if we're going to talk about Vaughns and patchy and whatever else we're going to spend on let's talk about it in terms of financial loss. Like you know if we want to cover a little bit different topic here, this is the the thought exercise I run everybody through these days. If you go to RSA and Black Hat, you stand on the show floor, you look at every single product and service out there, what percentage of them if they work perfectly as advertised lower financial loss? The highest answer, you know, the average answer that I get amongst security pros is 20%.

SPEAKER_03 23:43

Yeah I'm degenerate similar answers.

SPEAKER_06 23:46

And and then if you ask them when products when you when you invest money in developing a cybersecurity product does most of your dollars go towards feature parity with the competition or disrupting the bad guy? Everybody says feature parity. So all of a sudden we have an entire industry of malinvestment because we haven't focused on the one thing that matters financial loss.

SPEAKER_02 24:14

I tell you guys like this is this is something that has bothered me for a long time in that the entire industry as it has exploded over the last 15 years like it it feels like every year something new comes out and there's 15x of it more than there should be they are all playing catch up with each other. So whoever is the standard bearer whoever gets to market first with something they can convince an analyst is important some new thing everybody tries to copy them and catch and do a feature parody or a one up on them like I have yet to see a true analysis by an analyst firm that would tell me not which of these is better but which of these is relevant

Why CVSS And Scanners Mislead

SPEAKER_02 25:10

this goes to what's called the what I call the lack of negative feedback loop. What analyst would tell the enterprises that buying this entire category let alone a particular product is a complete waste of money well that's part of the like the terrible incentive structure this whole industry is built on the we're we're pharmaceuticals again right like forget curing you buy more vitamins and and and pain meds like you'll feel better it it'll make you make you feel better. Never mind that you're dying.

SPEAKER_06 25:43

And and that's the issue right there our industry for the moment for the moment lacks a negative feedback loop for for wasteful spending if you if you spend money you didn't need to spend and nothing happens you could say the product worked or it could be that the product wasn't going to do anything anyway.

SPEAKER_02 26:01

Well I I this is also a particular soapbox of mine but like the measurements behind it suck.

SPEAKER_03 26:08

How do you tell if any of this stuff works you can the feedback loop from the DFIR guys and the reinsurers and insurers gives us a really good sense of what does and doesn't work with a couple of caveats.

SPEAKER_06 26:22

But I don't hear that out there in the industry with it because there's they have no interest in telling the market when they're wasting money.

SPEAKER_03 26:30

They have the data but they why should they carry or tell their customers when they're wasting money I mean I think why would the analyst jeopardize future revenue to say that this product definitely doesn't solve any problems.

SPEAKER_06 26:44

We believe there's a way around this to give me a way around this chair.

SPEAKER_02 26:50

Well warranties come up yeah okay so talk to me about warranties because warranties in cyber are always interesting to me because you're saying you're basically saying I'm I think I know the solution and I'm willing to put real dollars behind it.

SPEAKER_06 27:07

So let's say you go to the business and you say because if I don't have to invest money developing a product for feature parity that has no relevance to the adversary whatsoever, my focus is purely on disrupting the bad guy, purely on lowering your financial loss. I can make a very laser focused product that just does one thing incredibly well protecting you from the adversary. It'll cost less it'll be faster low impact and should anything happen I will cover the first $5 million on your breach costs. We've built that we've done that. And it's not like the insurance carriers are agreeing with us. We're agreeing with them. We're just scanning for the things that they already know. That's how we're going to get there. If the business still wants to spend double triple or whatever on something else cause themselves a lot of pain with no financial downside protection that's on them. But we have something different so how do you backstop that because that's like if you're wrong that's that's a lot of million times five it's this that's it's for the very same reason that you can get every company is buried in volunteers we know this right and every company can get cyber insurance today. How does this work? Like so what we're finding is that these are the vones that matter we'll cover the financial loss because we know if you fix those you're not going to have a loss statistically unprobable. And our liability is transferred to the insurance carriers we're not holding the bag bag here we we're we're not betting the farm on here. This is the same model we've been using for the last decade or more they know we're right because it's their data. It's their data.

SPEAKER_02 28:49

Oh that's interesting. Okay.

SPEAKER_06 28:51

I hear you because here here's here's the challenge with the cyber insurance carriers and they'll tell you straight out they can't tell their customers what to do because what they call is a soft market. If the their carrier tells you what to do you'll go use another carrier because that carrier will write a policy that's not the same for us.

SPEAKER_02 29:12

That makes sense. Okay so this is this is a little bit different than where what I was thinking that would that was going to be interesting. So where do you get the data on like real working data on what's actually being exploited are the carriers tracking this yeah they they have to track it it's in their financial interest to track it.

SPEAKER_06 29:39

So a claim a breach happens a claim comes in their DFIR teams internally externally get deployed they do a root cause analysis in the cases that say remotely exploitable vulnerability was the initial point here's the here's the CVE number and they send it to us okay okay is this is this a consortium of carriers is this a universally accepted practice among the carriers to share this data no it's a it's it's all brand new yeah we're we're definitely breaking breaking ground on this one um okay we we've been working on this for two or three years collecting all this data from them we were the first ones to ever ask you know you can't get the right answer unless you ask this is interesting because you're now giving focus to something that has been chaotic best effort chaotic before up until I mean still is right up until now largely focusing chaotic what's the best possible outcome for you guys that you from an industry perspective we've we've gone through this a many times we were we you know we did whitehead security together and at some point we pushed AppSec so high that the adversary shifted and they went to ransomware you know they they shifted tactics I did Sentinel one and helped them and helped them out. I didn't found the company obviously but I helped them out with ransomware before ransomware was a thing. And if you notice those companies they still they have issues like no product is perfect but the adversary from there got pushed to CDEs where we land now we do have the ability with the right solutions to chase the bad guy from attack type to attack type and we believe very strongly we know how to go after the right vulnerabilities that forces a shift forces the their cost to go up.

SPEAKER_03 31:35

We're actually optimists believe it or not uh we have enough data now we know

Make Cyber About Financial Loss

SPEAKER_03 31:40

what we're doing.

SPEAKER_06 31:40

Yeah like once we figured out you don't have to patch everything at all times you're like wow this is this is doable that feels like that that feels like to me there are going to be some very hurt feelings in in in in and around this space because this is the this is the I don't want to say snake oil but I'll say snake oil that they've been selling their customers for 15 to 20 years and suddenly somebody new shows up and goes that's bullshit only one percent of this I've been having this conversation with Jar on and off for the last two years like we're going to make a lot of enemies well they didn't know at the time they they did their best guess they didn't know I don't know now we know now we know more like the carriers didn't have the data five years ago they didn't now they do like we yeah the industry made its best guess it just unfortunately was wrong and now it's the actuario era let's let's let's chase it I'm looking forward to we're not we're not we're not we're not we're not exclusive over this data. Okay actually everybody else can go to go do it.

SPEAKER_03 32:45

Well I think this dovetails into the to the webinar we did earlier but there are things our industry are doing that is absolutely their fault like calling O days other things other than O days I mean that there's some real big things out there saying that the adversaries are speeding up with no data to back any of those statements up.

SPEAKER_02 33:04

Yeah well you've got to the hype train has to listen this is this is the classic problem right it's it's it's the same in politics as it is in cyber as it is in in the news you have to keep ratchet up the the the the rhetoric the the you know it today it's you know it's bullets tomorrow it's nuclear weapons and after that it's bio thermo nuclear warfare because that's what's coming to sell you the next thing I have to scare you with something bigger and worse than yesterday but the problem continues to be the stuff that worked 20 years ago actually saw a so total sidebar but actually very recently saw evidence at a past employer where we had access to this of a vulnerability being scanned for that was older than my kids yeah I believe it yeah there are vones there are bones that are being utilized that are 20 years old now by the which like are you serious? And the question was like we all sat around table sort of looked at the data and we're like are they kidding me like and somebody asked the question like well are they wrong and the and the answer I think quite quite honestly is no because why would somebody spend their time scanning for something that didn't work like somewhere out there there are enough systems that that are 20 years old that still have a missed that have a patch that's not applied that makes this attacker's time to go scan for them worthwhile and that is a brain breaking thing right because whether you're talking about like all the all the municipal water systems that people are kind of losing their minds over right now right that's like I've seen NT4 in those environments because it works and nobody has the time or or money there's no you know there's no profit in in running a municipal water treatment facility.

SPEAKER_06 35:04

Well to your earlier point there will you know the data that we're bringing up the concepts that we're voicing we're we're used to it we will hurt feelings but we're we're in it to protect our customers that's why we're we're in this and here's the thing what's wrong with cybersecurity vendors sharing financial risk with their customers who is really going to speak against that let's find out what's what's the problem with this you're asking them to pay a bunch of money and then spend a bunch more money doing the stuff you believe you don't know you believe is wrong with no evidence to back it up.

SPEAKER_02 35:43

This is wrong we have to do better it's amazing what kind of behaviors can be driven from empirical evidence that stands on its own, from data that doesn't need interpretation right and I think that's been the hard part is we've had we'll say selective data that's been largely open to interpretation but I think as this as we mature this is a this is a this is I think long time coming as we mature as a as a practice right as cybersecurity because we are the youngest part of IT I think at this point right like we we as a practice you know cyber has is is not I think AI might might be a bit younger well okay fine but that's that's a separate thing. Okay fine AI but cyber is relatively new in the in the expanse of technology right and we were inevitably going to have to go through that like awkward teenage phase where we don't quite fit in like we look at the you know where we've come from we go wow glad I'm not that anymore and we look at where we're going to I feel like this is going to be a a relatively difficult transition because there's going to be security programs in the enterprise that are fundamentally going to have to change and if you're not there as a CISO or as a director or whoever runs this program if you're not able to articulate correctly why the change is happening it is going to be scary and it's going to feel like you're going to go tell your boss that you've been screwing around for the last 20 years and now it's a you know oh I I need to go do something different. Well why I think Jerry you said it best right the and the data is now available it wasn't available earlier. But now is the time to make that change because if you don't then this starts to sound like that net word that rhymes with negligence.

SPEAKER_06 37:43

I mean yeah I mean it's just there's a big opportunity here to where we can actually make an impact as an industry. We can figure out what works we can figure out what doesn't work the data is there we can share risk with our customers this whole market is disruptible there's 200 billion spent every year and it's up for grabs now because all of it was based upon a guess and we know the guesses were wrong. So for for anybody out there for startups or the or the big uh the big security vendors the game is on now we we can do this I mean we're we're supposed to be reducing financial loss here if you read the metrics in the industry if they're to be believed are we spending

Warranties Backed By Insurer Data

SPEAKER_06 38:20

200 billion a year to lose a trillion is that is that what we're doing here and are we you know we we just said a moment ago that you know everybody thinks mo about 20% of our products work. That means if you're pitching your product to your peers four out of five of them think you're it's worthless I mean come on we we could do a lot better than this I I hope we do I I really hope we we we get to a point where that what we do on a daily basis whether it's vulnerabilities ai whatever is is much more data driven than it's been in the past i i i'm glad you guys are doing this i i want to come back in in any you know six months a year and figure out like all right how are you getting on because uh adoption is going to I think adoption is going to be interesting we're we're we're we're a month we're a month we're we've spent a year building the product we raised money a year ago our one year anniversary or our GA was a month ago and we are already crushing it people want this it would seem a little insane of somebody to say I know what's going to you know I I know the safety features that I should be using because there's data for it but you know what YOLO I want to keep doing the thing that I've been doing like this that that seems like the definition of insan insanity that Einstein said. Well just real quick on that we we in Vole management we found two different religions and there's they they kind of are religions there's the ones that really want to there's the security departments that want to tell IT to fix all volans at all times. There's really they want that to happen. And then there's the other ones that know that all these volunteers there have they're not patched they haven't been patched but it's on their dashboards and they have to report to the board why 100 million vulnerabilities aren't fixed. They want a valid reason why these don't have to be fixed a way to say those can wait forever if necessary.

SPEAKER_03 40:20

Well actually well there's one kind of negative incentive that's worth calling out here if you're a CISO and you're in a position where you have to make that decision, are you playing with your chips or are you playing company chips? Because if you're playing with your chips you're going to go, hmm, I really do not want to get fired, but I also have a limited budget. I'm gonna work within the constraints I'm working for the betterment of the company and the upside of the company et cetera, et cetera. If you're playing with the company's chips, you'll say fix everything because any risk at all I'm not going to expose myself and I don't care how much the company loses to have to get there. And so I think I think that there isn't a negative feedback loop unfortunately to to weed out that behavior uh which is why you're saying spend in the billions of dollars literally for things like mythos but I think over time eventually the CFO is going to say what do we get for spending two billion a year on this like I think that I think that that's our I think that that's start that's been happening for a while.

SPEAKER_02 41:22

I think we've the the hand wavy smoking mirrors don't look over here look at how bad this is stuff right the jazz hands has has worked and then has been losing its luster and fade over time. I think the biggest risk if I'm totally honest right now to a CISO is that they've gone and created this must patch like you said Jared must patch everything all the time they have they now have this mandate that they've been driving and to go back to those people and say I was wrong the the the shift there's a shift now we're gonna go do this small sub section I think a lot of that has it feels like it feel it isn't but it feels like I was wrong I have to accept defeat it's not we're pivoting based on available data and like in new intelligence but there's this is this is going to be a shift I think for a lot of people and it could be very difficult because again you're telling your board you know you've been fighting that that battle of like here we got to fix these 500 and they've got this scorecard that says there's 5,000 there's 4000 there's three thousand there's nine thousand there's two thousand and now you're like 12 well like wait a minute.

SPEAKER_06 42:34

At some point the conversation's gonna be had because the CFO will ask the CISO why isn't our insurance carrier making us fix all this stuff? They're giving us a policy anyway. I mean that's gonna be a really uncomfortable conversation if you don't have a good answer. Like you know that these policies are getting into multi-billions and they're cheap. It's like 5% premium for the liability limit. I mean it's it's cheap. So why why do you want me to spend another hundred million dollars patching when our downside is protected explain this to me or or a billion yeah like seriously we're talking to companies who are spending a billion or more a year on SAST.

SPEAKER_03 43:16

Like what are you doing?

SPEAKER_02 43:18

Are you are we removing a billion dollars of liability because otherwise you're wasting company money well did you make a billion and more sales because of this this strategy I don't think so when you're when you're playing with house money and and what's happening in security is so far disconnected what's happening with the company's bottom line we've become sort of numb to this that it's acceptable that these things are out of whack. Now what you're saying is I now have evidence for what parody like what we should start spending and and working on like this is gonna hurt this is going to hurt people and they're gonna be mad that you're you know you're giving them an alternative and why why would they like this is going to shift some some we're we're excited about

SPEAKER_06 44:00

By it really, because we get to spend more efficiently. We have a really interesting story to tell, a really interesting perspective with data behind it that we can tell it's different from everybody else out there.

SPEAKER_03 44:12

Yeah, that that's the key there. We have data, we have receipts for all this. So it's not like we're just we have an opinion. This is like what this is what we're getting from people who actually have to pay out.

SPEAKER_02 44:23

Yeah, that that's the that's the catch here. This isn't just another company with an idea to make money, although it is, but you but this is based you guys are basing this on like real this is the problem according to insurance companies. This is the problem we're going to solve, and those circles overlap. Not a little bit, not like we think they overlap, but you could actually show how they overlap. I'm excited. All right, cool. You guys are always doing some cool shit. That's that's really and thank you.

SPEAKER_06 44:53

And we're also not in the position, we're not telling you you don't you can if you want to keep spending money over there, you can. We're saying right here, do this.

SPEAKER_02 45:05

Yeah, you you can keep doing the stuff, but this is the part that matters, yeah. Right?

AI Hype Reality Check And Wrap

SPEAKER_02 45:10

So, all right, well, cool guys. I I'm out of time, but this has been fun. I appreciate the uh the insight. This is so let me ask, do you guys do you guys actually believe that that uh in like 30 seconds or less that as these crazy new what seems to be overhyped and overmarketed AI models keep getting released by vendor by these two big vendors that try to keep one-upping each other? Are we gonna drive are we ever actually gonna get it get to a point where we're gonna drive the vulnerability the cost of identifying new vulnerabilities that'll actually blow some of this model that you guys are having up away?

SPEAKER_06 45:48

There's no data to indicate that. Tools are already in the hundreds of dollars. Make what is making it cheaper matter. Yeah, fair enough. You go into the block, you go on the dark web right now and buy a few hacking tools for a few hundred couple of grand. Drive it down to what? Yeah, yeah. We're not seeing anything. In fact, uh Robert, tell them what you've asked, been asking the carriers.

SPEAKER_03 46:12

I asked eight different carriers how many losses they saw that have been at all attributed in any way to AI. And if you exclude phishing, because that's that's a that's a real thing, yeah, the answer is zero.

SPEAKER_02 46:26

All eight of them is a lot of things. Oh, that's a that's an entire podcast entirely on its own, because I've got some thoughts on that.

SPEAKER_03 46:33

I mean, there's holes in the data, but but still, like they're not seeing anything. No, no increase to anything. Not not one. In fact, the the closest thing that anyone could come up with was well, sometimes the frontier models get sued because they're scraping too much. I'm like, that doesn't, that's not what we're talking about here.

SPEAKER_02 46:56

The interesting thing, the follow-up to that is, and we can just keep going with this, but I think we'll stop here, is like, how would we even know at this point?

SPEAKER_03 47:04

There's there's signals. There's signals. But you're but you're but you're you're right to be skeptical and there's holes in the data because DFIR can't do as good a job as we'd all like them to do.

SPEAKER_06 47:15

Part of the next conversation, Robert has a great theory, and I'm on board with it, is that the use of AI for the adversaries, unless they're very careful during the attack, it's an evolutionary dead end. Using AI will get you caught, which is all of what we've been seeing.

SPEAKER_02 47:30

That's an interesting thought to end on. Gentlemen, thanks for your time. I appreciate you. This has been an interesting conversation. All right, tell me, tell me the uh web uh company people to go what website to go check out.

SPEAKER_06 47:40

Rootevidence.com.

SPEAKER_02 47:41

Roote evidence. Not dot AI.

SPEAKER_06 47:44

Negative.

SPEAKER_02 47:48

I love it. All right, guys. Rob German, thank you so much for joining us. This has been a lot of fun and educational. We'll definitely have you guys back because I want to I want to hear as this data evolves. Let's let's uh let's let's wait six, nine months and see what happens. I want to I want to see where this goes. Happy to. Gentlemen, thank you so much. Have a good one. Thanks, folks. Thanks for listening. This has been yet another episode of the Downtown Security Rabbit Hole Podcast. I hope you've enjoyed it. I hope you've learned something. Give these guys a listen and a follow, and I obviously go check out what they're doing. Evidence should lead the way, and we'll catch you guys another time, another place on another Downtown Security Rabbit Hole Podcast.

SPEAKER_04 48:29

This is Bella. Thanks for listening. Don't forget to leave my dad a review and share this with your friends. Bye.