“By failing to prepare, you are preparing to fail.” - Benjamin Franklin
When it comes to cybersecurity, preparedness is your most valuable asset. By regularly testing your defenses through interactive cybersecurity exercises, your organization can identify weaknesses, refine your incident response plans, and ensure your team is ready to act in the event of an attack. With two main cybersecurity exercises — life-fire and tabletop — choosing the right approach can be difficult.
In this blog post, we will explore the benefits and limitations of both types of exercise, give recommendations on how to choose which is right for your organization, and discuss how combining these methods can create a more complete training program for your security team.
Have you ever faced a real cyber attack? Where your network is being assaulted and critical data is being exfiltrated? Live-fire cybersecurity exercises attempt to realistically simulate this scenario in a controlled environment, attacking your actual infrastructure and tools using modern tactics, techniques, and procedures, including live malware.
In a live fire exercise, the red team (attacking) would use simulated malware or other tactics, techniques, and procedures used by real threat actors to breach the defenses of the defending blue team, evade detection, and infiltrate the network. In these events, the red team would be adjusting their strategy to adapt to your defenses just as a real attacker would, creating a pressure cooker environment for participants.
Your team’s response is monitored in real-time, assessing their decision-making and execution of incident response plans in a high-pressure and time-constrained environment, much like the type they would experience during a real attack. The goal of this type of exercise is to identify vulnerabilities and gaps in your security posture.
The benefits of live fire exercises are undeniable:
However, live-fire exercises aren't without potential drawbacks. The cost and complexity often required to set up and execute live fire exercises require significant resources. You might need specialized expertise or dedicated infrastructure to avoid disrupting your actual operations. Even if cost is not an issue, simulated attacks can still impact business activities by taking time away from your security team. Careful planning and scheduling is required to minimize disruptions.
While live fire cybersecurity exercises offer a highly interactive experience with simulated attacks, tabletop exercises function more as thought experiments. In this type of exercise, teams gather around a table (not always literally) to discuss hypothetical attack scenarios and strategize their response.
Tabletop cybersecurity exercises often involve moderator-led discussions, where diverse attack scenarios are presented to the group. Participants might not be limited only to the security team, as tabletop role-play is just as beneficial for business executives, PR teams, and customer support as it is to security practitioners. These exercises provide a structured way to examine existing incident response plans and business continuity strategies through open discussion and debate.
Tabletop exercises have several advantages:
Of course, tabletop exercises also have limitations. The hypothetical nature of a tabletop exercise cannot fully replicate the pressure and uncertainty of a real attack. This could lead to unrealistic assumptions and responses that would not translate well to a live situation where attackers are unpredictable. Tabletop exercises are also vulnerable to bias, as preconceived notions or personal opinions can sway discussions. This could potentially neglect alternative perspectives and overlook critical aspects of the scenario.
The choice between live fire and tabletop exercises should not be an either-or proposition. If at all possible, performing both exercises could create a more comprehensive training approach. Tabletop exercises are excellent ways to prepare for, debrief, and analyze the results of live-fire exercises. This helps teams anticipate potential issues, analyze their response in hindsight, and refine their procedures for future live fire simulations. You could also consider basing your live fire exercises on specific vulnerabilities identified during tabletop sessions, maximizing the learning potential. By rotating between live fire exercises and tabletop exercises, you keep your team challenged by different, complimentary training styles that help ensure they are prepared for all types of cyber attacks.
If you are unable to do both types of exercises, consider the following to help decide which might be right for your organization:
Many cybersecurity experts recommend both live fire and tabletop exercises. Notably, the NIST CSF (National Institute of Standards and Technology Cybersecurity Framework) includes tabletop exercises as part of its “Identify” and “Protect” functions, providing recommendations on developing and conducting tabletop exercises. Additionally, the NIST Special Publication 800-84 provides a guide to designing, conducting, and evaluating both tabletop and live fire exercises.
Other respected government agencies, such as the United States Cybersecurity and Infrastructure Security Agency (CISA), also explicitly recommend tabletop exercises and allude to the value of live fire exercises. In their framework, CISA claims that “investments in planning and exercises support timely response and recovery actions, resulting in reduced impact to the delivery of services.”
It isn’t just government institutions that advocate for simulated cybersecurity exercises, however. Many renowned cybersecurity experts also support the value of live-fire exercises. For example, infamous hacker Kevin Mitnick was very outspoken in support of simulated exercises to test defenses. Although Kevin Mitnick passed away in July of 2023, his company Mitnick Security continues to offer vulnerability testing and red team operations similar to those conducted in many live fire exercises.
Every year, the NATO Cooperative Cyber Defense Centre of Excellence (CCDCOE) hosts two major cybersecurity exercises, Locked Shields and Crossed Swords. These large-scale live fire exercises bring together expert teams from various NATO nations to defend simulated networks against complex cyber attacks. Since 2016, Stamus has partnered with the CCDCOE at both Crossed Swords and Locked Shields, providing training and expertise while deploying the Stamus Security Platform (SSP) in support of the exercises. Notably, at Crossed Swords 2022 the 2 representative team members from Stamus Networks used SSP to identify 60% of the total threats discovered by the yellow team. This type of involvement from our team showcases our commitment and belief in these types of exercises. Peter Manev, co-founder and Chief Strategy Officer of Stamus Networks, recently discussed his experience at Crossed Swords in an article on tabletop exercises published by Dark Reading.
Ultimately, the choice between live fire and tabletop exercises is less about determining which is better, and more about creating a training program tailored to your organization’s unique needs and goals. By understanding the strengths and limitations of each type, employing a strategic framework, and considering a combined approach, you can equip your security team with the knowledge, skills, and confidence to face any cyber threat that may come their way. Remember, preparedness is your most valuable asset.
To be notified of new blog posts and other news, make sure to subscribe to the Stamus Networks blog and the Stamus Spotlight Monthly Newsletter, follow us on Twitter, LinkedIn, and Facebook, or join our Discord.