The Transportation Security Administration conducts covert testing programs to ensure security effectiveness of commercial aviation. One such program is the Index testing program, which involves attempts to covertly bring threat items into commercial airports. A RAND research team was asked to benchmark this program against analogues to see whether the program conforms to industry quality management standards and best practices.

Benchmarking the Transportation Security Administration’s Covert Index Testing Program

Joseph C. Chang, Michael W. Robbins, Vanya Barrer, Nikolay Maslov

ResearchPublished Apr 23, 2025

The Transportation Security Administration (TSA) conducts various covert testing programs as one of many ways to ensure the effectiveness of its workforce, operations, and programs for commercial aviation. One such program is TSA Inspection’s Red Team Index Division’s (RTID’s) covert testing program, or the Index testing program. Index testing involves teams of officers and specialists who attempt to covertly bring threat items through commercial airport checkpoints into secure areas at the airport. The program is intended to produce quality testing data suitable for a national-level analysis. 

A RAND research team was asked to benchmark Index testing against other testing analogues to see whether the program conforms to some industry quality management standards or best practices. After conducting a literature review and interviews, the research team was unable to find a direct analogue to Index testing but was able to find some applicable standards. Most of these standards are currently implemented in Index testing, but the research team could not establish that for a handful of standards. The research team makes recommendations on whether RTID should consider new steps toward implementing these standards and proposes some relevant new standards that RTID might consider to fill those gaps.

Key Findings

Key Findings

  • Only a few organizations conduct any type of covert testing. However, they are not direct analogues to Index testing because they are more ad hoc in nature, are on a smaller scale, and therefore cannot facilitate the same level of statistical rigor.
  • An analysis of interview notes reveals concerns in the following areas: limitations and constraints, testing variation, data quality and standards, and cooperation and information sharing.
  • The research team found four sources that provide standards, best practices, or general principles that are most relevant to Index testing. Other sources contain information already covered by these four sources, refer mainly to cybersecurity or acquisition support, focus on tactics, are overly generic, or are pedagogical.
  • The research team found that, of the 28 unique standards enumerated in the said four sources, Index testing has implemented 22; they were unable to establish that the remaining standards have been implemented. Therefore, Index testing appears to have implemented most of the relevant standards that were identified.

Recommendations

Recommendations

  • Consider the following actions, which pertain to implementing standards described in existing literature: Take extra steps to ensure covertness, prevent mirror imaging bias, improve the variety of skills and knowledge base of red teams, and increase the number of threat scenarios considered.
  • Consider the following actions, which pertain to developing new standards: Define allowable variations in standard threat scenarios, set clear analysis goals for Index testing data, establish targets for statistical precision, implement a hybrid approach that blends standard and broader categories of threat scenarios, prioritize between the competing objectives of training and evaluation (e.g., training might affect covertness, but evaluation requires true covertness), and publicize aspects of the Index testing program.

Topics

Document Details

Citation

RAND Style Manual

Chang, Joseph C., Michael W. Robbins, Vanya Barrer, and Nikolay Maslov, Benchmarking the Transportation Security Administration’s Covert Index Testing Program, Homeland Security Operational Analysis Center operated by the RAND Corporation, RR-A2269-1, 2025. As of April 23, 2025: https://www.rand.org/pubs/research_reports/RRA2269-1.html

Chicago Manual of Style

Chang, Joseph C., Michael W. Robbins, Vanya Barrer, and Nikolay Maslov, Benchmarking the Transportation Security Administration’s Covert Index Testing Program. Homeland Security Operational Analysis Center operated by the RAND Corporation, 2025. https://www.rand.org/pubs/research_reports/RRA2269-1.html.
BibTeX RIS

Research conducted by

This research was sponsored by DHS S&T and conducted in the Management, Technology, and Capabilities Program of the RAND Homeland Security Research Division.

This publication is part of the RAND research report series. Research reports present research findings and objective analysis that address the challenges facing the public and private sectors. All RAND research reports undergo rigorous peer review to ensure high standards for research quality and objectivity.

RAND is a nonprofit institution that helps improve policy and decisionmaking through research and analysis. RAND's publications do not necessarily reflect the opinions of its research clients and sponsors.