John Duda and Scott J. Glick
The manner in which a tabletop exercise (TTX) is designed will have a profound impact on the success of the TTX. For example, if an organization is seeking to exercise a policy, plan, or procedure during the TTX, it is essential that the policy, plan, or procedure have sufficient details and is ready to be tested. If the organization’s policies, plans, and procedures have not reached that level of development, then the organization should consider using a seminar or workshop to educate participants and fill in the necessary details before conducting the TTX. A TTX should also be designed to ensure that specific “big picture” goals and “SMART” – specific, measurable, achievable, relevant, and time-bound –objectives can be achieved.
Most important, a TTX should be designed and facilitated in ways that consider how adults learn best. Dr. Malcolm Knowles, who is known for his groundbreaking research concerning adult learners, and other scholars who have studied how to effectively engage adults, have emphasized that adults learn differently and are motivated differently. As a result, to maximize the learning that takes place during a TTX: (1) TTX participants should be represented during the planning and involved with how their training is delivered; (2) TTXs should be designed to take into account the impact of experiential learning on adults and draw upon their prior experiences and knowledge; (3) TTX participants should be asked to solve problems from the information that is presented; and (4) TTXs should be designed to have immediate relevance and impact to participants’ jobs. When TTXs are properly designed and executed, their use meets the principles which underlie adult learning.
Based on the foregoing, we recommend that organizations use the following checklist to ensure that the critical design elements for a successful TTX are considered.
- The exercise design team is small, manageable, experienced, and includes trusted agents from relevant participating organizations.
- The exercise design team has the necessary resources and support from senior leadership.
- The policy, plan, or procedure to be evaluated has sufficient details to be tested or exercised.
- Specific “big picture” goals have been identified (e.g., whether the policies, plans, and procedures have any gaps, and establishing relationships among responding organizations).
- “SMART” – specific, measurable, achievable, relevant, and time-bound –objectives have been identified.
- The exercise scenario is tied to the goals and objectives, as well as identified learning outcomes.
- The exercise scenario is realistic and includes appropriate script and data injects.
- The exercise scenario is designed to maximize the involvement of all participants.
- The exercise has been designed and includes facilitation techniques that draw upon the way that adults learn best (e.g., the appropriate use of multi-media).
- Sufficient time will be allocated for the exercise given the nature of the topics to be discussed, the number of participants, and how the facilitator intends to engage the participants.
- Exercise participants will be from all the appropriate agencies or organizations based on the exercise scenario and its goals, objectives, and learning outcomes.
- Exercise participants will be provided with short summaries of key exercise-related information (“one-pagers”) that they can use during the exercise.
- The exercise has an experienced and skilled facilitator with extensive knowledge about the organization, its mission, the subject matter, and the exercise objectives.
- Exercise planners have a detailed logistical checklist for the delivery of the exercise (e.g., space and seating requirements, audio and video equipment, microphones to enable all participants to be heard).
- Technology has been tested in the location of the exercise and is functioning as intended.
- There are enough skilled and knowledgeable exercise evaluators who can identify key takeaways for use during any immediate “hot wash” and who can record useful data for an after-action report.
John Duda
CEO
Scott Glick
General Counsel
Creating a New Standard for Evaluating Tabletop Exercises
By John Duda and Scott J. Glick
Even though tabletop exercises (TTXs) have been used for decades, an industry standard has not emerged on how to evaluate their effectiveness. Since 2012, John Duda, CEO of SummitET, has noted the lack of an industry standard for quantitative assessments of TTXs, which prompted him to develop a rubric for analyzing and measuring exercise effectiveness. Based on their extensive exercise experience in both the government and the private sector, Mr. Duda and Scott J. Glick, SummitET’s General Counsel, have refined the rubric and its scoring of various exercise factors into what they call the XF ScoreTM, which they discuss in their article published by the Domestic Preparedness Journal entitled Creating A New Standard for Evaluating Tabletop Exercises.
Access the full article published by DomesticPreparedness.com.
Resources
The Importance of Conducting Exercises
Attacks both domestically and internationally have pushed legislation into domestic resilience, combating terrorism, incident command, and disaster preparedness.
Types of Exercises and Training
The types of exercises we conduct from discussion-based to operations-based exercises.