Expert cybersecurity practitioners are intensely aware of how complex the field may seem to less experienced colleagues. An effective cyber defense function, for example, requires colleagues with technical expertise as well as colleagues a genuine understanding of the threat landscape, adversarial tactics, cyber strategy, and essential related concepts including legal or reputational impact of a cyber incident.
Conveying these ideas through training and education too often relies on the transfer of facts without context, in which users often may not genuinely understand why certain cybersecurity behaviors are more useful than others. The use of serious games highlights a promising and engaging avenue to educate users with facts placed into an appropriate and relevant context.
Simulation exercises are forms of serious games that allow large groups to practice incident response and resilience playbooks, something that benefits groups across a huge range of audiences, at every level from industry training to multinational military exercises.
Within NATO specifically, Locked Shields and Cyber Coalition are just some of the simulation-based exercises used to train military allies in red/blue-teaming, cyber strategy, and multinational collaboration. On the industry side, most of the large consultancy practices (and a number of smaller specialist firms) offer simulation exercises within their cyber-advisory practices.
However, the plethora of cyber educational attempts highlights more examples than not of exercises which do not stand up to scrutiny when it comes to achieving the goals of effective transfer of information.
Across most major cyber wargame providers, these authors found methodologies and pedagogical (teaching) goals are often either unspecified, underspecified, or, when present, are used to excuse or justify poor game design. Existing cybersecurity research has highlighted opportunities for improvement including additional business context.
We go one step further and argue that designing formal educational cybersecurity games without any acknowledgement or inclusion of professional game designers risks derailing the entire educational exercise.
Stop Designing Ineffective Cyber Exercises (Or, Why do I need a Game Designer?)
At first glance it may appear a simple task to design a game. After all, most of us have been playing games since we all were children. Unfortunately, designing an effective game that achieves a designer’s pedagogical goals is not so straightforward.
For cybersecurity scenarios and the ‘serious gaming’ space in particular, the issue comes down to how far the game should represent reality. Cybersecurity experts often bristle at what they see as simplification and design scenarios that ultimately overwhelm a player as their attempt to capture the full complexity of the system thus producing a game that, while incredibly accurate, is so complex that it cannot be engaged with by anyone other than themselves.
Ill-equipped designers may fail to address aspects including player personas, the game’s narrative structure, or may simply not understand the conventions of the genre that they are designing for. To overcome these issues, inexperienced designers will often engage in the ‘reskinning’ of existing games - in effect throwing a new coat of paint on a pre-existing system of abstraction, without understanding the trade-offs that they are making.
Limitations on existing cyber wargame design have been addressed in part by experts highlighting the need to adjusting wargaming components (including realism) depending on the game’s scope and objectives.
At the highest levels, one hopes that well-resourced institutions leverage the research undertaken at specialist institutes (such as the US Naval War College or the MITRE.ORG’s cyber wargaming framework).
However, as authors have previously lamented (at conferences which aim to ‘bridge the gap’ between ‘academic and/ or theoretical research’ and industry practice), more often than not industry actors do not base design decisions on sound game design principles, resulting in duplicated work and a less effective teaching mechanisms as a result.
Avoiding Pitfalls: Key Recommendations for Cyber Wargaming Design
‘You do not need to be an IT expert to design cyber wargames. You need to be familiar with wargame design and then study the cyber factors related to your scenario.’ - Dr Roger Mason (Designing Cyber Wargames)
Include the right subject matter experts in the room - Of course, cyber wargames must align with real-world threat landscapes, ensuring that simulations and narratives contain feasible scenarios that match the priorities of the end users.
As a teaching tool, it is also essential to include colleagues with experience in serious game design, a specialist who understands the mechanics, dynamics and aesthetics decisions that are required to design an effective learning game. They can map out the core loops of the play to ensure that they reflect the experts' model and they know how to playtest and iterate on the design thus improving the outcomes.
Understand Ludic vs Learning Goals - A learning game can fulfil many different learning goals (practice of skills, the transfer of knowledge, or the production of knowledge among others). A ludic goal describes the objectives within the game, for example, to successfully de-escalate a cyber incident, or protect the player’s information assets. However, the ludic goals must simultaneously be engaging, while complimenting the learning goals.
It is essential to make sure that the decisions of one do not overwhelm the needs of the other, for example, that the players do not treat the game as a ‘tick-box’ exercise to achieve the ludic goals, without engaging the desired learning goals, or vice versa.
Recognize the importance of debriefing
To paraphrase Dewey: all experiences teach, but not all experiences are educational. An effective learning game needs to be integrated into a learning program - not merely tacked on as a redundant (if not distracting) experience. The debriefing, or reflection, part of a game is the defining feature of a true serious game, and must be a key focus for cyber wargame designers. Without a planned debrief phase, the game no longer fits within Kolb’s experiential learning cycle.
Running an ineffective cybersecurity game risks feeding false assumptions to players, skewing analysis, stifling new ideas, and creating a false sense of security among participants. This can have dramatic effects on cyber defense performance as participants take ‘knowledge’ back into their roles (across industry, government, or military).
Effective teaching matters, and for this, project managers must employ effective specialist game designers who can apply the key principles outlined in this article.
Amy is an Information Security doctoral candidate at Royal Holloway, University of London. She is currently a Visiting Scholar at NATO Cooperative Cyber Defence Centre of Excellence and Cybersecurity Fellow at the Belfer Center, Harvard Kennedy School, where her research explores the security implications of AI-enabled technology in defence and the military. Amy has previously worked in cyber intelligence and holds both CISSP and CREST threat intelligence certifications. Most recently Amy contributes as a Scenario Designer with The CyberFish Company, providing cyber-psychology crisis management training.
Peadar is a serious game designer and lectures in learning game design and gamification at the University of Tallinn. His company, Integrated Game Solutions, provides consultancy and design services for serious games and simulations, with a focus on providing engaging training outcomes.