Creating a New Standard for Evaluating Tabletop Exercises

Creating a New Standard for Evaluating Tabletop Exercises

By John Duda and Scott J. Glick

Even though tabletop exercises (TTXs) have been used for decades, an industry standard has not emerged on how to evaluate their effectiveness. Since 2012, John Duda, CEO of SummitET, has noted the lack of an industry standard for quantitative assessments of TTXs, which prompted him to develop a rubric for analyzing and measuring exercise effectiveness. Based on their extensive exercise experience in both the government and the private sector, Mr. Duda and Scott J. Glick, SummitET’s General Counsel, have refined the rubric and its scoring of various exercise factors into what they call the XF ScoreTM, which they discuss in their article published by the Domestic Preparedness Journal entitled Creating A New Standard for Evaluating Tabletop Exercises.

Access the full article published by DomesticPreparedness.com.

Tabletop exercise design checklist

Go to our checklist for organizations to use to ensure that the critical design elements for a successful TTX are considered.

John Duda, Chief Executive Officer and founding partner at SummitET

John Duda

CEO

Scott Glick VP and General Counsel

Scott Glick

General Counsel

Resources

No Results Found

The page you requested could not be found. Try refining your search, or use the navigation above to locate the post.

Checklist for Designing a Successful Tabletop Exercise

Checklist for Designing a Successful Tabletop Exercise

John Duda and Scott J. Glick

The manner in which a tabletop exercise (TTX) is designed will have a profound impact on the success of the TTX. For example, if an organization is seeking to exercise a policy, plan, or procedure during the TTX, it is essential that the policy, plan, or procedure have sufficient details and is ready to be tested. If the organization’s policies, plans, and procedures have not reached that level of development, then the organization should consider using a seminar or workshop to educate participants and fill in the necessary details before conducting the TTX. A TTX should also be designed to ensure that specific “big picture” goals and “SMART” – specific, measurable, achievable, relevant, and time-bound –objectives can be achieved.

Most important, a TTX should be designed and facilitated in ways that consider how adults learn best. Dr. Malcolm Knowles, who is known for his groundbreaking research concerning adult learners, and other scholars who have studied how to effectively engage adults, have emphasized that adults learn differently and are motivated differently. As a result, to maximize the learning that takes place during a TTX: (1) TTX participants should be represented during the planning and involved with how their training is delivered; (2) TTXs should be designed to take into account the impact of experiential learning on adults and draw upon their prior experiences and knowledge; (3) TTX participants should be asked to solve problems from the information that is presented; and (4) TTXs should be designed to have immediate relevance and impact to participants’ jobs.  When TTXs are properly designed and executed, their use meets the principles which underlie adult learning.

Based on the foregoing, we recommend that organizations use the following checklist to ensure that the critical design elements for a successful TTX are considered.

  • The exercise design team is small, manageable, experienced, and includes trusted agents from relevant participating organizations.
  • The exercise design team has the necessary resources and support from senior leadership.
  • The policy, plan, or procedure to be evaluated has sufficient details to be tested or exercised.
  • Specific “big picture” goals have been identified (e.g., whether the policies, plans, and procedures have any gaps, and establishing relationships among responding organizations).
  • “SMART” – specific, measurable, achievable, relevant, and time-bound –objectives have been identified.
  • The exercise scenario is tied to the goals and objectives, as well as identified learning outcomes.
  • The exercise scenario is realistic and includes appropriate script and data injects.
  • The exercise scenario is designed to maximize the involvement of all participants.
  • The exercise has been designed and includes facilitation techniques that draw upon the way that adults learn best (e.g., the appropriate use of multi-media).
  • Sufficient time will be allocated for the exercise given the nature of the topics to be discussed, the number of participants, and how the facilitator intends to engage the participants.
  • Exercise participants will be from all the appropriate agencies or organizations based on the exercise scenario and its goals, objectives, and learning outcomes.
  • Exercise participants will be provided with short summaries of key exercise-related information (“one-pagers”) that they can use during the exercise.
  • The exercise has an experienced and skilled facilitator with extensive knowledge about the organization, its mission, the subject matter, and the exercise objectives.
  • Exercise planners have a detailed logistical checklist for the delivery of the exercise (e.g., space and seating requirements, audio and video equipment, microphones to enable all participants to be heard).
  • Technology has been tested in the location of the exercise and is functioning as intended.
  • There are enough skilled and knowledgeable exercise evaluators who can identify key takeaways for use during any immediate “hot wash” and who can record useful data for an after-action report.
John Duda, Chief Executive Officer and founding partner at SummitET

John Duda

CEO

Scott Glick VP and General Counsel

Scott Glick

General Counsel

Creating a New Standard for Evaluating Tabletop Exercises

By John Duda and Scott J. Glick

Even though tabletop exercises (TTXs) have been used for decades, an industry standard has not emerged on how to evaluate their effectiveness. Since 2012, John Duda, CEO of SummitET, has noted the lack of an industry standard for quantitative assessments of TTXs, which prompted him to develop a rubric for analyzing and measuring exercise effectiveness. Based on their extensive exercise experience in both the government and the private sector, Mr. Duda and Scott J. Glick, SummitET’s General Counsel, have refined the rubric and its scoring of various exercise factors into what they call the XF ScoreTM, which they discuss in their article published by the Domestic Preparedness Journal entitled Creating A New Standard for Evaluating Tabletop Exercises.

Access the full article published by DomesticPreparedness.com.

Resources

No Results Found

The page you requested could not be found. Try refining your search, or use the navigation above to locate the post.

SummitET Experts to Attend 2023 National Homeland Security Conference (NHSC)

SummitET Experts to Attend 2023 National Homeland Security Conference (NHSC)

Summit Exercises and Training LLC (SummitET®) experts are attending the 2023 National Homeland Security Conference (NHSC) and trade show in Chicago, Illinois from July 24 to 27.

The NHSC brings together professionals in Homeland Security, Law Enforcement, Fire and Emergency Management to learn about emerging trends in homeland security and see the new equipment, technology, and resources available to enhance their mission objectives.

In support of the Urban Area Security Initiative (UASI)’s emergency management needs, SummitET leverages global experience and public, private, and military subject matter expertise to provide exercise and training support in the areas of counterterrorism, weapons of mass destruction (WMD), and chemical, biological, radiological, nuclear, and explosive materials (CBRNE).

National Homeland Security conference

SummitET’s trainers and program management professionals offer knowledge and know-how with real world event and response experience. Our expert backgrounds include law enforcement, fire services/EMT, emergency management, Federal Bureau of Investigation, Homeland Security, strategic communications, and more.

Conference attendees can connect with SummitET experts to discuss preparedness solutions for current and emerging threats including exercises, training, and plan development with interagency coordination.

Visit Our Booth

Find SummitET experts at the NHSC tradeshow booth #426 where you can learn more about how we can help prepare your agency or organization. Play our interactive game to win preparedness giveaways including t-shirts marked with the SummitET tagline “Preparedness is not an accident®”.

See Our Preparedness Capabilities

Experts at Our Booth

Come visit us at booth #426 in Chicago this July.

John Duda, Chief Executive Officer and founding partner at SummitET

John Duda

Chief Executive Officer and Founding Partner

Ron Crane

Ron Crane

Director of Programs Quality Assurance &
Program Manager, Isotope Crossroads

SummitET News

SummitET Appoints Adam Montella as Senior Vice President

SummitET Appoints Adam Montella as Senior Vice President

Summit Exercises and Training LLC (SummitET®), a Preparedness Solutions Company®, announced the addition of Adam Montella, MPA, MHS-V, CMP as the company’s new Senior Vice President. Adam is an internationally recognized subject matter expert, speaker, and writer on...

read more
SummitET Introduces Preparedness Podcast: Bridging the Gap

SummitET Introduces Preparedness Podcast: Bridging the Gap

Summit Exercises and Training LLC (SummitET®) is excited to announce the launch of a new preparedness video podcast befittingly entitled “Bridging the Gap”, in which subject matter experts (SMEs) discuss domestic and international emerging threats. “Bridging the Gap”...

read more

Bridging the Gap Episode 2: Stolen Radiation Sources

Bridging the Gap Episode 2: Stolen Radiation Sources

SummitET Subject Matter Experts interpret the emerging threat discussed in the following article. 

Three months, three missing radioactive items. Here’s what you need to know” by Tara Subramaniam

CNN.com, March 31, 2023 

non destructive testing

The recent articleThree months, three missing radioactive items. Here’s what you need to know provides an overview of the recent loss of regulatory control of three items containing radioisotopes in Thailand, Australia, and the United States. The article offers examples of how radioisotopes are used in commercial applications, discusses their associated hazards, and addresses the frequency of loss of regulatory control as seen in these three cases.

Radioactive Isotopes Uses

Radiological materials are used for beneficial purposes across the nation and the world daily – in medicine, research, and industrial applications. These materials are important for diagnosing and treating medical conditions like cancer or to ensure that the integrity of infrastructure like roads or pipelines are verified before beginning construction or use.  

The nature and use of radioactive materials in devices vary. In the U.S., radioactive materials are highly regulated, and the level of regulation, security, and oversight is based on the potential risk posed from the sources within the device. The U.S. Nuclear Regulatory Commission (NRC) and 39 Agreement States work together to ensure that all radioactive materials are properly licensed and used across all industries.

Industrial Radiography Camera

The industrial radiography camera, like the one that went missing recently in the U.S., is one common device that has greater regulatory oversight due its use as a high activity source. It is used in industrial settings for various types of non-destructive testing, but most commonly it is used to image welds or other structural integrity of pipes or metal structures. Most of these devices use an Iridium-192 (Ir-192) source which emits strong gamma photons that are needed to penetrate through the metal to create the images used to look for cracks or defects in the welds, among other purposes. Thousands of industrial radiography jobs are safely conducted across the nation every year.

The Ir-192 source is kept inside a very robust, shielded container, about the size of a shoebox, except when the imaging is occurring – then it is cranked out of the shield through a long tube to expose the film around the pipe or weld of interest. The source is only outside of the shielded housing for 30 seconds to a minute at a time and under very controlled settings carefully managed by the trained operators. This operator training, as well as the requirements for storage, security, transportation, and general use of the device, is outlined in regulations, and the NRC and Agreement States work closely with the licensees who are using these devices.

Theft or Loss of Radioactive Materials   

Bottom-line, individuals can’t just go and purchase or use an industrial radiography camera or other higher activity source without meeting stringent regulations that include significant site security and background checks on those employees with access to the materials.  

However, as indicated in the article, accidents sometimes happen. When they do occur, immediate notification is required, the regulatory agencies actively coordinate with the licensee and any other appropriate agencies to locate the source and oversee any regulatory enforcement actions necessary to prevent other issues from occurring.

Prevention 

Sources can go missing by accident or by act of terrorism but in either scenario it is important that agencies are prepared to prevent and mitigate this risk. Preparing via the development of intelligence information sharing pathways, by creating specialized plans for communications, and other lifesaving tactics are what SummitET’s experts focus on when creating tailored solutions including exercises and training. Learn more how experts can help your agency be at its most prepared.

Contributors:

Angela Leek, Director of Radiological Solutions and Regulatory Affairs

Angela Leek, CHP

Director of Radiological Solutions and Regulatory Affairs

Andrew Manson Law Enforcement and CBRNE Subject matter expert

Andrew Manson

Law Enforcement/CBRNE SME

Justin walker law enforcement and SWAT Subject matter expert

Justin Walker

Law Enforcement SME/Retired Phoenix Police Officer/SWAT Operator; Lead RSARex SME

Kevin Quigley, CBRN and WMD Subject Matter Expert at SummitET

Kevin Quigley

All Hazards SME

hand holding smart phone with application icons interface and globe networking system. concept technology social network communication.

Flattening the Social Media Monitoring Curve

Consider these 7 steps in monitoring social media in order to decipher your information requirements and facilitate the decision-making process.

people maintain trust on mountain

Maintaining Trust and Preparing for Change During a Crisis

Open, accurate, and consistent communication is one of the most important parts of an effective crisis response. Learn to facilitate trusting relationships with stakeholders.

Hospital workers moving quickly

Hospital Preparedness; The Four S’s of Surge Capacity

Surge capacity is unique for each hospital. However, it does not mean the larger hospital has an infinite surge capacity. Learn the 4 S’s of surge capacity.

illustration of map in the shape of a human head

Communication Through Message Mapping

Messages play an important role in determining the success of communication efforts before, during, and after a crisis. Learn to message map to communicate the major issues facing your organizations during a crisis.

SummitET Introduces Preparedness Podcast: Bridging the Gap

SummitET Introduces Preparedness Podcast: Bridging the Gap

Summit Exercises and Training LLC (SummitET®) is excited to announce the launch of a new preparedness video podcast befittingly entitled “Bridging the Gap”, in which subject matter experts (SMEs) discuss domestic and international emerging threats. “Bridging the Gap” provides insight to challenges that the preparedness and response community face before, during, and after crises. Whether it is communications, training or exercises, biological implications, and anything in between, SummitET SMEs engage agencies and organizations in conversation to work towards strong, practical preparedness solutions.

SummitET industry-leading experts are real-world practitioners who are uniquely qualified to assist with preparedness and response, as well as, offer tailored approaches to effective solutions prior to, during, and after a crisis. Their extensive experience and backgrounds make our SMEs exceptionally wellversed in today’s emerging threats facing agencies and communities domestically and abroad.

Bridging the Gap, a SME-led podcast

Bridging the Gap leverages our expansive network and areas of expertise to provide views with a multi-disciplinary and plain language approach to timely preparedness topics. Bringing SummitET and industry experts together to discuss these topics highlights the tremendous knowledge and experience that our team offers its clients while engaging in conversations on how preparedness can continually evolve,” states John Duda, SummitET CEO.

Bridging the Gap is hosted by Kevin Quigley, SME in CBRNE (Chemical, Biological, Radiological, Nuclear, and Explosives) and WMD (Weapons of Mass Destruction) and retired U.S. Marine Corps CBRNE defense officer. It is co-hosted by Andrew Manson, Law Enforcement SME and prior program manager for the FBI’s domestic WMD response.

Episodes will air monthly and will include real-world, experiential insights from experts in the preparedness and response communities, as well as lessons learned and best practices for real and emerging threats. The hosts will help audiences bridge the gap between emerging threats in the news and implications, response challenges, and learning opportunities.

Preparedness is not an accident®.

Meet the Hosts

Kevin Quigley, CBRN and WMD Subject Matter Expert at SummitET

Kevin Quigley

All Hazards SME and retired U.S. Marine Corps Chemical, Biological, Radiological, Nuclear, and Explosives (CBRNE) defense officer

Andrew Manson Law Enforcement and CBRNE Subject matter expert

Andrew Manson

Law Enforcement SME and prior program manager for the FBI’s domestic WMD response

About SummitET

Summit Exercises and Training® (SummitET®) is a Florida-based, veteran-owned preparedness solutions company that provides proven full spectrum solutions to threats facing U.S. and International government and non-government organizations, as well as private entities through industry-leading experts and impeccable customer service.

SummitET News

SummitET Appoints Adam Montella as Senior Vice President

SummitET Appoints Adam Montella as Senior Vice President

Summit Exercises and Training LLC (SummitET®), a Preparedness Solutions Company®, announced the addition of Adam Montella, MPA, MHS-V, CMP as the company’s new Senior Vice President. Adam is an internationally recognized subject matter expert, speaker, and writer on...

read more
SummitET Introduces Preparedness Podcast: Bridging the Gap

SummitET Introduces Preparedness Podcast: Bridging the Gap

Summit Exercises and Training LLC (SummitET®) is excited to announce the launch of a new preparedness video podcast befittingly entitled “Bridging the Gap”, in which subject matter experts (SMEs) discuss domestic and international emerging threats. “Bridging the Gap”...

read more