Does it Matter if It’s Black or White(listing)?

The fundamental principle that underpins all security is the need to stop bad people or processes while allowing good people or processes. Much of security is about access control, and access control starts with identity. Identity on its own, however, is not enough – we also need to understand purpose. We need to identify the person or process, and decide whether the intent is good or bad.

Consider passports in the physical world. They prove identity, but do not tell us intent. We can reinforce identity with lists of known intent: a whitelist of frequent flyers or VIPs whose intent is known to be good, and a blacklist of terrorists and bad people whose intent is known to be bad.

Cybersecurity is the same. Based on identity and intent we maintain whitelists of known good (or at least acceptable) behavior, and blacklists of known bad (or unacceptable) behavior. Security is determined based on how we use these lists. We either allow what’s on the whitelist and prevent everything else, or we prevent what’s on the blacklist, and allow everything else. We tend to concentrate on one approach or the other: whitelisting or blacklisting.

Keeping our computers clean is a good example. In the beginning, the anti-malware industry simply blacklisted the bad things. But now the alternative is gaining traction: whitelisting the good things. So which is best for achieving maximum security?

In Favor of Blacklisting

The foundation of anti-virus technology is a blacklist of all known malware. The technology is based on blacklisting because, in the beginning, there were very few viruses. A primary advantage of blacklisting is that it is conceptually simple to recognize a few bad things, stop them, and allow everything else.

A second argument in favor of blacklisting is ‘administrative ease’. The maintenance of blacklists is something we can delegate to trusted third parties – in this instance the anti-virus companies. They in turn, particularly with the advent of the internet, can automatically update the blacklist for us. Basically, we don’t have to do anything.

Whitelisting is different because it can be difficult to delegate the decision on which applications we need to delegate to a third party. “Whitelisting would be the perfect solution if people only have one computer that is never patched and never changed”, explains Dan Power, UK regional manager for anti-spam company Spamina. “Intellectually, it makes perfect sense to only allow execution of the files that you know to be good.”

Maintaining this whitelist is, however, very difficult. “The problem comes when you have to register or re-register every DLL (dynamic link library) each time you patch or install a new application. [You have to consider] who you allow to install their own software, and who you stop? Which bits of software can make changes and which can’t? It becomes more of an administrative rather than intellectual issue”, Power says.

David Harley, senior research fellow at ESET, agrees: “Whitelisting – which isn’t much different in principle to the integrity checking of yesteryear, requires more work by internal support teams and interferes with the end-users’ right to install anything they like; which is more of a problem in some environments than in others.”

That’s not to say that everyone considers such delegation to be impossible. Last year Microsoft’s Scott Charney proposed a form of whitelisting for access to the internet; that is, only users with an internet health certificate for their computer should be allowed to access the web. He has few supporters in the security industry. “If computers were like televisions, with just one base operating system that was never changed, then it’s doable”, says Spamina’s Power. “But in the real world there are just so many variables associated with Windows and all the bits of software that have ever been written for Windows, that it’s almost impossible to be able to say what is and what’s not a clean or healthy computer”.

Jennifer Gilburg, director of marketing at Intel, sees a different problem with this type of whitelisting. “Think of e-commerce”, she says. “An online trader would rather take the occasional fraudulent transaction than risk turning away a good transaction. So the thought of blocking a user from coming onto the internet until they are trusted would terrify many of the e-commerce providers who make their livelihood on the basis of the more users the better. I suspect that most of the e-commerce world would be lobbying very hard to put down this version of whitelisting.” Interestingly then, one of the strongest arguments in favor of blacklisting is the problems associated with whitelisting.

In Favor of Whitelisting

Henry Harrison, technical director at Detica, turns the spotlight onto blacklisting and raises a concern. “Anti-virus blacklisting”, he says, “is based on the idea of detecting things that are known to be bad and stopping them. But it simply cannot detect things that are bad, but not known.”

Zero-day threats are not known simply because they are zero-day threats – and blacklisting lets them in as if they were good. “What we are seeing today”, continues Harrison, “is a lot of targeted, covert attacks – external infiltration into corporate networks with a view to the theft of valuable information using techniques that are specifically designed to evade blacklisting. One possible response to zero-day threats is whitelisting.”

Lumension’s senior vice president, Alan Bentley, points to the sheer volume of malware as a problem for blacklisting. “Blacklisting”, he explains, “is threat centric. Whitelisting is completely the opposite: it’s trust centric. While blacklisting malware used to be adequate, the whole threat arena in the cyberworld has exploded to such an extent that we now have to question whether blacklisting alone is still good enough.”

Keep in mind, this is what Lumension does: it protects endpoints (such as the PC on your desk) by making it administratively easy to create and maintain a whitelist of acceptable applications while supporting that with a blacklist of malware. “We believe that if you look at the two things together, whitelisting should absolutely be the first line of defense for any organization, because it simply stops everything that isn’t approved”, Bentley continues. “But what it cannot do is remove malware once it has embedded itself into a machine.”

Bit9, like Lumension, is a company that concentrates on whitelisting. “The premise of application whitelisting is very simple”, says Harry Sverdlove, chief technology officer. “What you want running on your system is a much smaller set than what you don’t want. We apply this model to other aspects of security in our life. For example, who do you let into your home? You don’t keep a list of everyone bad in the world. Rather, you only allow people into your home whom you trust.”

The explosion in malware (in excess of two million new pieces of every month) is exactly what makes us question whether blacklisting remains realistic. “As a general rule, whitelisting is always more secure than blacklisting”, Sverdlove maintains. “But it requires you to think more about how software arrives on your systems and whether or not it is trustworthy. That’s why a software reputation database can be an invaluable aide in whitelisting – it provides a trust rating on software, like a trusted advisor or background security check service, that can make the process more manageable. If everything you run comes from a well-known third party, approving software almost exclusively from a cloud-based reputation service can be enough. In most cases, however, you also have your custom or proprietary software. An effective and robust whitelisting solution allows you to combine both your own policies along with those from a reputation database.”

So we should ask ourselves whether we can harness the power of cloud-based reputation systems to generate our whitelists? Spamina already uses this methodology to produce its blacklist of spam sources, calling on six separate reputation blacklists, but never relying on just one, and thus minimizing the chance of false positives.

An AV World

“I’ve never advocated AV as a single defensive layer”, says ESET’s Harley. “Whitelisting can − and does − work for businesses, though it works best where there’s an authoritarian IT culture, rather than laissez-faire: restricted privileges and so on. I wouldn’t generally recommend it as a complete substitute for AV, but if it’s implemented properly, it’s a rational multi-layering strategy”, he accepts. “It does, at a stroke, obviate most of the risk from social engineering-dependent threats. In fact, most AV nowadays does have some whitelisting ability, though how it’s done and to what extent varies enormously.”

Ram Herkanaidu, security researcher at Kaspersky Lab UK, has a similar viewpoint and acknowledges the increasing relevance of whitelisting. “As the amount of malware increases”, he says, “I can see at some point it could be more efficient to only allow whitelisted files to be run in an organization. The idea has been around for a while, but many things have to be taken into consideration, like software updates – especially Windows updates – remote users, smartphone and non-standard users. Ideally, as well as using the vendor’s whitelist, you could have a local whitelist too. So while the idea of having a ‘trusted environment’ is very appealing, in practice it is difficult to achieve.”

Kaspersky, like other anti-virus companies, is already looking into whitelisting. “We have been running a whitelist program to collect information about all known good files”, Herkanaidu reveals. “The files are sent to us by our whitelist partners and also through our Kaspersky Security Network. This is our ‘neighborhood watch’ [that] users become part of when they install Kaspersky internet security. Information about all unknown files is sent to our ‘in the cloud’ service and automatically analyzed. If malicious, all computers within the network are protected. If it is not malicious, it will be added to our whitelist.” Herkanaidu explains that this approach has two benefits for Kaspersky customers: “it will reduce the risk of false positives, and will increase scan speeds. In this way we have been able to collect information about millions of files.”

Six of One…

So what’s our conclusion? Whitelisting is fundamentally the better security solution. If something isn’t on the list, it gets stopped – the default position for whitelisting is secure. With blacklisting, if something isn’t on the list it gets allowed – the default position for blacklisting, therefore, is insecure.

Measured against this, the administrative effort involved in blacklisting is minimal compared to whitelisting; and the difference increases as the size of the whitelist increases. However, the efficiency of blacklisting decreases as its size increases. You could almost say that whitelisting is best with a small whitelist, while blacklisting is best with a small blacklist. In other words, it’s the relative size that’s the issue. If there are more blacks than whites, you should whitelist; if there are more whites than black, you should blacklist. However, since neither of these situations is likely to occur in the real world, our conclusion is simple: you need both.