The Rise of Data-Driven Security

The phrase "you're doing it wrong" is a common refrain in the security community these days as people wander around in various states of disillusionment with the technology and processes that have led to what many perceive as a systemic failure. But that refrain usually is not followed by any useful discussion of what's going wrong or what can be done about it. To researcher Claudio Guarnieri, one of the major problems is obvious: we're completely backward in the way we prioritize protection.

On any given day, the headlines are full of dire warnings about new zero-days, another bug discovered in Android or a new flaw in a major database. Inside enterprise IT departments, those bugs are simply added to the already massive pile they'll eventually get around to patching when they have time. And often, that patching plan will be based upon one or another of the myriad vulnerability scoring systems that have emerged in the last 10 years or so.

Therein lies the problem, according to Guarnieri. Which bugs to fix first and how quickly to patch them should not be based on a CVSS score or criticality rating, but rather on how likely it is that an attacker is going to try and exploit any given vulnerability.

"We tend to be too flat and don't take into account whether vulnerabilities are actually being exploited in the wild," Guarnieri, a researcher at Rapid7, said in a recent interview. "It's not efficient because there's no context. We need to understand how bugs are being used by the bad guys. There needs to be a connection between bugs, attacks and threats. People need to understand that this kind of vulnerability is being used by this kind of attacker for this kind of attack. So then I can walk it up the chain as a high priority."

 

There are thousands and thousands of vulnerabilities discovered each year now, but the vast majority of those don't end up being used in attacks. They're the bench players, the guys who are kept around to fill out the roster and take a beating from the big boys in practice. They just sort of hang out, like Rudy waiting for the coach to call his name, hoping that one day they'll get in the game. But, unless it's one of the stars--say a nice ASLR and DEP bypass bug in Internet Explorer 10--then it's probably going to stay in the shadows and never get much run.

The CVSS (Common Vulnerability Scoring System) is a system designed to score each vulnerability based on a number of factors.

Even flaws with critical ratings may not be of much use to an attacker if they're not in a widely deployed application. That's one of the reasons Guarnieri believes there needs to be a major shift in the way that the industry looks at vulnerabilities in general and their place in the security chain in particular. Bringing the probability of exploitation into the equation is one step in that direction.

"Right now we're relying on the CVSS score and broken metrics. They're purely technical evaluations of the vulnerabilities and don't you any absolute measurements of the likelihood of exploitation," Guarnieri said. "For cybercriminals, Java is the main thing. It's used for targeted attacks, but targeted intrusions come down to Office in a lot of cases. Java is the bad animal in the play for cybercrime. Knowing this gives you a lot of context and advantage when counteracting. Critical bugs are really only fifty percent of what's being used. The rest are low and medium severity. If you filter the CVE collection down to the ones that are actually being weaponized and used, it's a much smaller number."

Guarnieri estimates that there are roughly 100 vulnerabilities being used or sold on the underground at any given time, and the tens of thousands of others are mostly background noise.

"That gives you a very limited context of what's likely to happen when it comes to exploitation and helps with prioritization," he said. "Right now, we always base security on what might possibly happen, not on what's likely to happen."

Guarnieri, the creator of the Cuckoo Sandbox malware analysis tool, advocates a data- and intelligence-driven approach to vulnerability analysis and security, something that's also been espoused by others in the industry, including Dan Guido of Trail of Bits. That approach takes into account the relevance of a particular vulnerability to your specific organization, how likely it is to be exploited and what the effect would be on your organization if it was exploited. 

"People are too systematic about their security," he said. "We're being so exposed, it's a disaster. Data-driven security should be the next thing. Collect and analyze the data from the wild and provide a realistic assessment of what's going on."

Courtesy by Dennis Fisher