Can Artificial Intelligence Help Improve Air Force Talent Management?
An Exploratory Application
ResearchPublished Jan 19, 2021
In this report, RAND researchers explore an application of artificial intelligence that would enable the U.S. Air Force to leverage existing data for improved human resource management policies and practices. The main purpose of this report is to serve as a worked example for Air Force policymakers as they consider how to approach the potential ways in which artificial intelligence can improve human resource management processes.
Both private and public organizations are increasingly taking advantage of improvements in computing power, data availability, and analytic capabilities to improve business processes. These trends have prompted U.S. Department of Defense policymakers to become more interested in whether adopting data-enabled methods would facilitate more-effective management of department personnel. In this report, RAND researchers explore one such application that would enable the U.S. Air Force to leverage existing data for improved human resource management (HRM) policies and practices. Specifically, the researchers develop a performance-scoring system that uses artificial intelligence (AI) and machine learning, which would enable the expanded use of performance narratives in HRM processes. The main purpose of this report is to serve as a worked example (i.e., a step-by-step solution to a problem) for Air Force policymakers as they consider how to approach the potential ways in which AI can improve HRM processes.
Key Findings
- The researchers apply the cross-industry standard process for data-mining to the AI application explored in this analysis. This task requires two primary data inputs: (1) a sufficiently large sample of officer performance narratives and (2) outcome labels that provide information about which narratives indicate the best job performance.
- The researchers find that there is a business need for the AI performance-scoring system. The system could be used as a tool to facilitate policy analysis, assist development teams, enable professional development, or aid in competitive selection decisions.
- Extracting and digitizing large amounts of officer evaluation data from the existing archive is feasible, but, because the process of digitizing the text from older documents requires extensive tuning and computation time, the process could present a potential challenge to future efforts by Air Force practitioners.
- Initial model results are promising: Standard machine-learning algorithms accurately predicted an officer's performance quality by identifying known signals in the performance narrative text without explicit programming.
- Implementation concerns regarding privacy, fairness, explainability, and other unintended consequences are greatest if the system were used to make decisions that would alter officers' careers. These considerations could be less of a barrier to implementing the system for other purposes.
Recommendations
- Policymakers should position systems and policies to take advantage of analytic uses of data that HRM processes generate.
- Natural language processing techniques can reduce the need to pre-quantify information at the data collection stage.
- Human resources managers should consider AI applications to be an enabler of better policies rather than a substitute for human decisionmaking.
Topics
Document Details
- Copyright: RAND Corporation
- Availability: Available
- Year: 2021
- Print Format: Paperback
- Paperback Pages: 76
- Paperback Price: $20.00
- Paperback ISBN/EAN: 978-1-9774-0645-3
- DOI: https://doi.org/10.7249/RRA812-1
- Document Number: RR-A812-1
Citation
RAND Style Manual
Schulker, David, Nelson Lim, Luke J. Matthews, Geoffrey E. Grimm, Anthony Lawrence, and Perry Firoz, Can Artificial Intelligence Help Improve Air Force Talent Management? An Exploratory Application, RAND Corporation, RR-A812-1, 2021. As of April 8, 2025: https://www.rand.org/pubs/research_reports/RRA812-1.html
Chicago Manual of Style
Schulker, David, Nelson Lim, Luke J. Matthews, Geoffrey E. Grimm, Anthony Lawrence, and Perry Firoz, Can Artificial Intelligence Help Improve Air Force Talent Management? An Exploratory Application. Santa Monica, CA: RAND Corporation, 2021. https://www.rand.org/pubs/research_reports/RRA812-1.html. Also available in print form.
Research conducted by
Funding for this research was made possible by the independent research and development provisions of RAND’s contracts for the operation of its U.S. Department of Defense federally funded research and development centers. The research was conducted within the Manpower, Personnel, and Training Program of RAND Project AIR FORCE.
This publication is part of the RAND research report series. Research reports present research findings and objective analysis that address the challenges facing the public and private sectors. All RAND research reports undergo rigorous peer review to ensure high standards for research quality and objectivity.
This document and trademark(s) contained herein are protected by law. This representation of RAND intellectual property is provided for noncommercial use only. Unauthorized posting of this publication online is prohibited; linking directly to this product page is encouraged. Permission is required from RAND to reproduce, or reuse in another form, any of its research documents for commercial purposes. For information on reprint and reuse permissions, please visit www.rand.org/pubs/permissions.
RAND is a nonprofit institution that helps improve policy and decisionmaking through research and analysis. RAND's publications do not necessarily reflect the opinions of its research clients and sponsors.