The cybersecurity industry has long exalted those capable of breaking technology and coming up with fresh new attack techniques, but is it time to recognize those tasked with defending and recovering from incidents? Dan Raywood looks at the case for the blue team.
“There is much more status given to the ability to test or hack than there is for those that focus more on defensive techniques"
In last year’s Q3 issue of Infosecurity we looked at the concept of red teaming: exploring where it came from, who red teamers are, what services they typically offer and why businesses should care about red teaming. Now, a year on, it seems that there is a sea change in attitudes towards those on the other side of the test, the blue team.
The blue team is the security team, those tasked with securing the network, defending and hardening it against attacks and remediating when something goes wrong. For every new attack method, there’s a defender tasked with rolling out a patch or applying a rule or policy so their network is not vulnerable.
Blue Hats versus Red Hats
The blue team are now often being perceived as ‘the good guys’ who, whilst working tirelessly to defend networks in the face of attack, actually receive none of the praise they deserve.
There is also an argument that blue teaming is, in fact, more skilled than red teaming, highlighted by the defense ethos of needing to be ‘prepared all the time’ in an era of vulnerability disclosure and bug bounties. Ultimately, the industry is often guilty of praising those on the offensive side without recognizing the work of those doing the defending and repairing.
James Jardine is CEO of Jardine Software and is a co-host of the Down the Security Rabbithole podcast.
“I have spent time working both sides of the fence on the red and the blue side,” he tells Infosecurity. “There is much more status given to the ability to test or hack than there is for those that focus more on defensive techniques.
“Of course, one of those reasons is that offensive techniques are easily confirmed. If I find a SQL injection and exploit it to gain access to a cache of data, it is tough to dispute that. I know it happened, I have the data. This makes it easy to identify an accomplishment.”
On the defense side, Jardine adds, it is not as easy as you can assume that defensive techniques are working, “but we cannot be 100% sure.”
He says: “Maybe we are breached, we just don’t know it yet? Maybe the defense worked today, but will it be as effective tomorrow? I think these are some of the things that put offensive in the forefront over defensive.”
On episode 206 of the Southern Fried Security podcast, host Martin Fisher said that he was tired of people saying security is broken with no idea how to fix it and that “it is easy to break but harder to defend.” Guest speaker Wendy Nather, director of advisory CISOs at Duo Security, said that it is all too easy to say “you missed a spot,” but eventually it is time to grow up and do some defending.
Speaking to Infosecurity, Nather argues that it is time that those given the task of repairing the ‘damage’ are given more credit, adding it is actually about more than that, as running an enterprise security program requires much more than just blue teaming.
“The term blue team connotes a game or a match, one side against the other in a limited exercise, but most of the time, corporate security is not about attack and defense,” she says.
“Corporate security is not football. It is the ongoing process of designing and building secure systems, processes and procedures; it’s about helping the business do what it does in the most secure way possible without hampering its success,” says Nather.
“Every so often, you have to drop what you’re doing and respond to an incident, but most of the time you’re having a discussion with a colleague about finding the best way to build something or make it better.”
"Corporate security is not about attack and defense"
Are The Blues More Skilled?
The thing that seems to have tipped the balance in the favor of the blue team is the acknowledgement of the skills required to be a blue teamer. Nather points out that being a blue teamer “requires empathy, psychology, process re-engineering and a pragmatic understanding of how everything works, not just the best way to attack something.” You also need “a broad technical knowledge far beyond what a security researcher might choose to focus on.”
The issue seems to be that there is not so much glory in being on the defensive side. Jardine says that while there are plenty of people that work on the blue side and focus on helping to defend against and stop the bad guys, the appreciation for the job they do is just not there.
Quentyn Taylor, director of information security for Canon Europe, tells Infosecurity that the problem is “we venerate [too much] the red teamers and attackers” who claim they can break into a company in 10 minutes.
So is it time to recognize the defender? “Yes, as we celebrate offense with the Pwnie awards – which are hilarious – we need something like that for the blue teamers because so often the work of the blue team goes unrecognized.
“With red teaming, you need deep skills in one particular area, but with a blue team you need to have such a width of skills to be able to cover everything from PR to reverse engineering to everything else,” Taylor says. “I’ve been a blue teamer for 18 years, and I still don’t even think of myself as a blue teamer – it’s time to recognize ourselves for what we are.”
Nothing is Rosy on the Blue Side
Along with not being as recognized as those conducting offensive actions, another problem that blue teams face is in achieving a ‘business as usual’ baseline. Gemma Moore, director, information security consultant and penetration tester at Cyberis, thinks the blue team needs to understand “what normal activity looks like in order to be able to identify abnormalities,” and this is a key weakness in detecting unusual activity within the network.
So why is this a weakness? Moore says that you need to consider the variety of indicators that might be present within a standard corporate network. On a purely technical level, the blue team needs to consider all kinds of events to baseline: user authentication events, the use of system privileges, patterns of network traffic internally, CPU and memory consumption, changes in DNS querying activity, users conducting unusual operations within the network, and many more.
“Any changes in these metrics could be a result of a compromise in progress,” she says. “This is a lot of data to process and analyze, and it requires a really in-depth understanding of technical operations in an effective blue team.”
That is why a skilled blue team is so important, she argues, and the challenge for the blue team is to understand which indicators of compromise are more likely to provide a true positive in their particular business environment, so that these are properly monitored.
“Sifting through data and distinguishing a real incident from a false positive takes knowledge, skill and experience.”
"With a blue team you need to have such a width of skills to be able to cover everything"
Are You Normal?
Understanding what is ‘normal’ is a challenge for the blue team, who need to determine what they expect normal to look like, and when dealing with a persistent attack or anomaly, need to know what the problem is and be able to respond immediately. Nather says that this is why cybersecurity practitioners need to be promoted as role models, as this sort of work comes with its own unique challenges.
April C. Wright is CEO of Architect Security, and she believes it is not a case of the red or blue team needing more skills but that they “definitely need different skills and they need a totally different mindset as default.”
She adds: “Defenders must be able to see big-picture trends, anticipate events before they happen, and understand known risks to focus efforts on high-risk areas. Offensive teams need to have more of a ‘predator’ mindset, be able to find the forgotten system, exploit human weakness and understand the nature of vulnerabilities.
“In other words, you need a blue team that can ‘think like an attacker’ and a red team/offense that can ‘evade detection’. Without knowledge of both sides, either team is going to be limited in effectiveness.”
Taylor says that this is one of the common failings around red team reports; they do not take the time to explain findings to the blue team. “What frustrates me is a red team that doesn’t understand that they need to educate,” he adds. “If they get in they should sit down with the blue team and talk them through what they did and how to defend against it.”
Time for Recognition
All of this acknowledgement of the challenges of blue teaming and the skills required suggests that it is time the blue teamer was seen as an equal to the red teamer. Wright says that a good day for a blue teamer can be when nothing happens. This can be “extremely relieving and knowing that you’ve defended data against attack for another day is, in my opinion, a more difficult struggle and a bigger win.”
However, it does feel that there is more recognition growing about what blue teams do. Self-declared blue teamer and security technology lead at Entergy Keirsten Brager explains that the efforts to recognize blue teamers are very timely and if we’re going to get more people interested in the cybersecurity career field “the public needs to understand that it entails more than hacking.”
She agrees with the general consensus that while offensive skills are great for informing defensive strategies, the industry needs to move away from promoting the celebrity hacker as the de facto career choice because it is such a small subset of the industry.
“Companies need defensive talent, business acumen, communication skills, regulatory knowledge, incident response and a host of other overlooked capabilities that are required to run a successful security program,” Brager says. “The better approach is to promote how offensive and defensive skills are equally important to continuously improving the discipline as a whole. We need fewer examples of the lone hacker and more illustrations of the collaborative nature of our work.”
