Evaluating Pilot Performance

doi:10.1017/CBO9780511609817.014

11 - Evaluating Pilot Performance

Published online by Cambridge University Press: 04 August 2010

Brian T. Schreiber ,

Winston Bennett Jr. ,

Charles M. Colegrove ,

Antoinette M. Portrey ,

David A. Greschke and

Herbert H. Bell

Edited by

K. Anders Ericsson

Show author details

K. Anders Ericsson: Affiliation:
Florida State University

Book contents

Get access

Summary

Piloting an aircraft, especially in combat situations, requires a high level of performance. Air combat tactics involve dynamic, four-dimensional (x, y, z, and timing) maneuvering and positioning of the aircraft; done at high speed, often approaching or surpassing supersonic speed. Physical and cognitive capabilities and resources are frequently put to the test during flight. Poor performance in wartime can lead to death. Even peacetime military operations and civilian aviation are not incident or accident free. To best prepare our pilots, an understanding of their knowledge, skill competencies, and deficiencies is necessary. In order to accomplish this objective, their performance must be measured and their training progress tracked on various skills using objective outcome measures in both the training environment and in live-range exercises.

In many fields the benefits of task-specific and experiential training are beginning to experience investigation and consideration, with an integration of training and historically conventional methods of education (Boshuizen, Chapter 17; Ericsson, Chapter 18; Lajoie, Chapter 3; Schraagen, Chapter 7). This is not the case with the field of aviation, where the importance of task-specific training has been realized since the initiation of manned flight. As technologies within the aviation industry increased in complexity and sophistication over the past century, so did the demand for training and measuring pilot proficiency. This growing demand was the impetus for the development in flight simulation, within both the military and commercial industries.

Type: Chapter
Information: Development of Professional Expertise
Toward Measurement of Expert Performance and Design of Optimal Learning Environments
, pp. 247 - 270

DOI: https://doi.org/10.1017/CBO9780511609817.014 [Opens in a new window]

Publisher: Cambridge University Press

Print publication year: 2009

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

Book purchase

Temporarily unavailable

References

Bailey, M., & Woodhead, R. (1996). Current status and future developments of RAF aircrew selection. Selection and training advances in aviation: AGARD conference proceedings 588 (pp. 8–1–8–9). Prague, Czech Republic: Advisory Group for Aerospace Research and Development.Google Scholar

Bell, H. H., & Waag, W. L. (1998). Evaluating the effectiveness of flight simulators for training combat skills: A review. The International Journal of Aviation Psychology, 8(3),223–242.CrossRef Google Scholar

Bennett, W., Jr., Schreiber, B. T., & Andrews, D. H. (2002). Developing competency-based methods for near-real-time air combat problem solving assessment. Computers in Human Behavior, 18, 773–782.CrossRef Google Scholar

Brictson, C. A., Ciavarelli, A. P., Pettigrew, K. W., & Young, P. A. (1978). Performance assessment methods and criteria for the Air Combat Maneuvering Range (ACMR): Missile envelope recognition (Special Report No. 78–4 (Confidential)). Pensacola, FL: Naval Aerospace Medical Research Laboratory.Google Scholar

Bush, G. W., President. (2005). Budget of the United States government. Fiscal Year 2006. Retrieved February 15, 2007, from, http://www.whitehouse.gov/omb/budget/fy2006/.

Carolan, T., MacMillan, J., Entin, E. B., Morley, R. M., Schreiber, B. T., Portrey, A. M., et al. (2003). Integrated performance measurement and assessment in distributed mission operations environments: Related measures to competencies. In 2003 interservice/industry training, simulation and education conference (I/ITSEC) proceedingsOrlando, FL: National Security Industrial Association.Google Scholar

Carretta, T. R., & Ree, M. J. (2003). Pilot selection methods. In Tsang, P. S. & Vidulich, M. A. (Eds.), Principles and practice of aviation psychology (pp. 357–396). Mahwah, NJ: Lawrence Erlbaum Associates.Google Scholar

Colegrove, C. M., & Alliger, G. M. (2002, April). Mission essential competencies: Defining combat mission readiness in a novel way. Paper presented at the NATO RTO Studies, Analysis and Simulation (SAS) Panel Symposium. Brussels, Belgium.

Colegrove, C. M., & Bennett, W., Jr. (2005). Competency-based training: Adapting to warfighter needs. Paper presented at the meeting of the Royal Astronautic and Engineering Society, London, UK.

Crane, P. M., Robbins, R., & Bennett, W., Jr. (2000). Using distributed mission training to augment flight lead upgrade training. In 2000 Proceedings of the interservice/industry training systems and education conference. Orlando, FL: National Security Industrial Association.Google Scholar

Crosbie, R. J. (1995). The history of the dynamic flight simulator. Washington, DC: Department of the Navy.Google Scholar

Dickman, J. L. (1982). Automated performance measurement: An overview and assessment. In Proceedings of the 4th interservice/industry training equipment conference: Volume I (pp. 153–165). Washington, DC: National Security and Industrial Association.Google Scholar

Dillard, A. E. (2002). Validation of advanced flight simulators for human-factors operational evaluation and training programs. In Workshop on foundations for modeling and simulation (M&S) verification and validation (V&V) in the 21st Century (Foundations '02 V&V Workshop). Laurel, MD: Johns Hopkins University, Applied Physics Laboratory.Google Scholar

Dockeray, F. C., & Isaacs, S. (1921). Psychological research in aviation in Italy, France, England and the American Expeditionary Forces. Journal of Comparative Psychology, 1, 115–148.CrossRef Google Scholar

Gray, T. H., Chun, E. K., Warner, H. D., & Eubanks, J. L. (1981). Advanced flight simulator: Utilization in A-10 conversion and air-to-surface attack training (AFHRL-TR-80–20, AD A094 608). Williams Air Force Base, AZ: Air Force Human Resources Laboratory, Operations Training Division.Google Scholar

Gray, T. H., & Fuller, R. R. (1977). Effects of simulator training and platform motion on air-to-surface weapons delivery training (AFHRL-TR-77–29, AD A043 648). Williams Air Force Base, AZ: Air Force Human Resources Laboratory, Operations Training Division.CrossRef Google Scholar

Hagin, W. V., Durall, E. P., & Prophet, W. W. (1979). Transfer of training effectiveness evaluation: US Navy Device 2B35 (Seville Research Corporation Rep. No. TR79–06, ADA073669). Pensacola, FL: Chief of Naval Education and Training.Google Scholar

Hughes, R., Brooks, R. B., Graham, D., Sheen, R., & Dickens, T. (1982). Tactical ground attack: On the transfer of training from flight simulator to operational Red Flag exercise. In Proceedings of the 4th interservice/industry training equipment conference: Volume I (pp. 127–130). Washington, DC: National Security Industrial Association.Google Scholar

Jenkins, D. H. (1982). Simulation training effectiveness evaluation (TAC Project No. 79Y-OO1F). Nellis AFB, NV: Tactical Fighter Weapons Center.Google Scholar

Kellogg, R., Prather, E., & Castore, C. (1980). Simulated A-10 combat environment. In Proceedings of the Human Factors Society 24th annual meeting (pp. 573–577). Santa Monica, CA: Human Factors Society.Google Scholar

Krusmark, M., Schreiber, B. T., & Bennett, W., Jr. (2004). Measurement and analysis of F-16 4-ship team performance in a simulated distributed mission training environment (AFHRL-HE-AZ-TR-2004–0090). Mesa, AZ: Air Force Research Laboratory Human Effectiveness Directorate, Warfighter Readiness Research Division.Google Scholar

Leeds, J., Raspotnik, W. B., & Gular, S. (1990). The training effectiveness of the simulator for air-to-air combat (Contract No. F33615–86-C-0012). San Diego, CA: Logicon.Google Scholar

Lintern, G., Sheppard, D., Parker, D. L., Yates, K. E., & Nolan, M. D. (1989). Simulator design and instructional features for air-to-ground attack: A transfer study. Human Factors, 31, 87–100.CrossRef Google Scholar

Mayer, G. B., Jr. (1981). Determining the training effectiveness and cost-effectiveness of visual flight simulators for military aircraft (AD A104627, Master's thesis). Monterey, CA: Naval Postgraduate School.Google Scholar

McGuinness, J., Bouwman, J. H., & Puig, J. A. (1982). Effectiveness evaluation for air combat training (ADP000203). In Proceedings of the 4th interservice/industry training equipment conference: Volume I (pp. 391–396). Washington, DC: National Security Industrial Association.Google Scholar

McGuinness, J., Forbes, J. M., & Rhoads, J. E. (1984). Air combat maneuvering performance measurement system design (AFHRL-TP-83–56). Williams Air Force Base, AZ: Operations Training Division, Armstrong Laboratory.Google Scholar

Moroney, W. F., & Moroney, B. W. (1999). Flight simulation. In Garland, D. J., Wise, J. A., & Hopkin, V. D. (Eds.), Handbook of aviation human factors. Mahwah, NJ:Lawrence Erlbaum Associates.Google Scholar

Nullmeyer, R. T., & Spiker, V. A. (2003). The importance of crew resource management in MC-130P mission performance: Implications for training evaluation. Military Psychology, 15(1), 77–96.CrossRef Google Scholar

Payne, T. A., Hirsch, D. L., Semple, C. A., Farmer, J. R., Spring, W. G., Sanders, M. S., et al. (1976). Experiments to evaluate advanced flight simulation in air combat pilot training: Volume I. Transfer of learning experiment. Hawthorne, CA: Northrop Corporation.Google Scholar

Polzella, D. J., Hubbard, D. C., Brown, J. E., & McLean, H. (1987). Aircrew training devices: Utility and utilization of advanced instructional features. Phase 4. Summary report (AFHRL-TR-87–21; ADA188418). Brooks Air Force Base, TX: Air Force Human Resources Laboratory.Google Scholar

Robinson, J. C., Eubanks, J. L., & Eddowes, E. E. (1981). Evaluation of pilot air combat maneuvering performance changes during TAC ACES training. Nellis Air Force Base, NV: U.S. Air Force Tactical Fighter Weapons Center.Google Scholar

Schreiber, B. T., & Bennett, W., Jr. (2006). Distributed mission operations within-simulator training effectiveness baseline study: Summary report (AFRL-HE-AZ-TR-2006–0015-Vol I, ADA461866). Mesa, AZ: Air Force Research Laboratory, Warfighter Readiness Research Division.Google Scholar

Schreiber, B. T., Stock, W.A., & Bennett, W., Jr. (2006). Distributed mission operations within-simulator training effectiveness baseline study: Metric development and objectively quantifying the degree of learning (AFRL-HE-AZ-TR-2006–0015-Vol II, ADA461867). Mesa, AZ: Air Force Research Laboratory, Human Effectiveness Directorate, Warfighter Readiness Research Division.Google Scholar

Schreiber, B. T., Watz, E., & Bennett, W., Jr. (2007). DIS: Does interoperability suffice? A need to set a higher standard (07S-SIW-067, 2007 Spring SIWzie Award). In 2007 spring simulation interoperability workshop conference. Norfolk, VA: SISO.Google Scholar

Schreiber, B. T., Watz, E., Bennett, W., Jr., & Portrey, A. (2003). Development of a distributed mission training automated performance tracking system. In Proceedings of the 12th conference on behavior representation in modeling and simulation (BRIMS). Scottsdale, AZ.Google Scholar

Symons, S., France, M., Bell, J., & Bennett, W., Jr. (2005). Linking knowledge and skills to mission essential competency-based syllabus development for distributed mission operations (AFRL-HE-AZ-TR-2006–0014). Mesa, AZ: Air Force Research Laboratory, Warfighter Readiness Research Division.Google Scholar

Corp, Vreuls Research. (1987). Air combat maneuvering performance measurement system for SAAC/ACMI, Volume II, Appendices 5 & 6. Wright-Patterson Air Force Base, OH: Air Force Systems Command.Google Scholar

Waag, W. L. (1991). The value of air combat simulation: Strong opinions but little evidence. In The Royal Aeronautical Society flight simulator symposium: Training transfer – Can we trust flight simulation? Proceedings. London: The Royal Aeronautical Society.Google Scholar

Waag, W. L., & Bell, H. H. (1997). Estimating the training effectiveness of interactive air combat simulation (AL/HR-TP-1996–0039). Armstrong Laboratory, AZ: Aircrew Training Research Division.CrossRef Google Scholar

Waag, W. L., Houck, M., Greschke, D. A., & Raspotnik, W. B. (1995). Use of multiship simulation as a tool for measuring and training situation awareness. In AGARD Conference Proceedings 575 Situation awareness: Limitations and enhancement in the aviation environment (AGARD-CP-575). Neuilly-Sur-Seine, France: Advisory Group for Aerospace Research & Development.Google Scholar

Wickens, C. D. (2003). Aviation displays. In Tsang, P. S. & Vidulich, M. A. (Eds.), Principles and practice of aviation psychology (pp. 357–396). Mahwah, NJ: Lawrence Erlbaum Associates.Google Scholar

Wiekhorst, L. A., & Killion, T. H. (1986). Transfer of electronic combat skills from a flight simulator to the aircraft (AFHRL-TR-86–45, AD C040 549). Williams Air Force Base, AZ: Air Force Human Resources Laboratory, Operations Training Division.Google Scholar

Wiekhorst, L. A. (1987). Contract ground-based training evaluation: Executive summary (ADA219510). Langley Air Force Base, VA: Tactical Air Command.Google Scholar

Wooldridge, L., Obermayer, R. W., Nelson, W. H., Kelly, M. J., Vreuls, D., & Norman, D. A. (1982). Air combat maneuvering performance measurement state space analysis (AFHRL-TR-82–15; ADA121901). Brooks Air Force Base, TX: Air Force Human Resources Laboratory.CrossRef Google Scholar