Skip to main content Accessibility help

We use cookies to distinguish you from other users and to provide you with a better experience on our websites. Close this message to accept cookies or find out how to manage your cookie settings.

Close cookie message
×
Hostname: page-component-78c5997874-mlc7c Total loading time: 0 Render date: 2024-11-03T05:24:24.046Z Has data issue: false hasContentIssue false

11 - Evaluating Pilot Performance

Published online by Cambridge University Press:  04 August 2010

By
Brian T. Schreiber ,
Winston Bennett Jr. ,
Charles M. Colegrove ,
Antoinette M. Portrey ,
David A. Greschke  and
Herbert H. Bell
Edited by
K. Anders Ericsson
K. Anders Ericsson
Affiliation:
Florida State University
Get access

Summary

Piloting an aircraft, especially in combat situations, requires a high level of performance. Air combat tactics involve dynamic, four-dimensional (x, y, z, and timing) maneuvering and positioning of the aircraft; done at high speed, often approaching or surpassing supersonic speed. Physical and cognitive capabilities and resources are frequently put to the test during flight. Poor performance in wartime can lead to death. Even peacetime military operations and civilian aviation are not incident or accident free. To best prepare our pilots, an understanding of their knowledge, skill competencies, and deficiencies is necessary. In order to accomplish this objective, their performance must be measured and their training progress tracked on various skills using objective outcome measures in both the training environment and in live-range exercises.

In many fields the benefits of task-specific and experiential training are beginning to experience investigation and consideration, with an integration of training and historically conventional methods of education (Boshuizen, Chapter 17; Ericsson, Chapter 18; Lajoie, Chapter 3; Schraagen, Chapter 7). This is not the case with the field of aviation, where the importance of task-specific training has been realized since the initiation of manned flight. As technologies within the aviation industry increased in complexity and sophistication over the past century, so did the demand for training and measuring pilot proficiency. This growing demand was the impetus for the development in flight simulation, within both the military and commercial industries.

Type
Chapter
Information
Development of Professional Expertise
Toward Measurement of Expert Performance and Design of Optimal Learning Environments
 , pp. 247 - 270
Publisher: Cambridge University Press
Print publication year: 2009

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

Bailey, M., & Woodhead, R. (1996). Current status and future developments of RAF aircrew selection. Selection and training advances in aviation: AGARD conference proceedings 588 (pp. 8–1–8–9). Prague, Czech Republic: Advisory Group for Aerospace Research and Development.Google Scholar
Bell, H. H., & Waag, W. L. (1998). Evaluating the effectiveness of flight simulators for training combat skills: A review. The International Journal of Aviation Psychology, 8(3),223–242.CrossRefGoogle Scholar
Bennett, W., Jr., Schreiber, B. T., & Andrews, D. H. (2002). Developing competency-based methods for near-real-time air combat problem solving assessment. Computers in Human Behavior, 18, 773–782.CrossRefGoogle Scholar
Brictson, C. A., Ciavarelli, A. P., Pettigrew, K. W., & Young, P. A. (1978). Performance assessment methods and criteria for the Air Combat Maneuvering Range (ACMR): Missile envelope recognition (Special Report No. 78–4 (Confidential)). Pensacola, FL: Naval Aerospace Medical Research Laboratory.Google Scholar
Bush, G. W., President. (2005). Budget of the United States government. Fiscal Year 2006. Retrieved February 15, 2007, from, http://www.whitehouse.gov/omb/budget/fy2006/.
Carolan, T., MacMillan, J., Entin, E. B., Morley, R. M., Schreiber, B. T., Portrey, A. M., et al. (2003). Integrated performance measurement and assessment in distributed mission operations environments: Related measures to competencies. In 2003 interservice/industry training, simulation and education conference (I/ITSEC) proceedingsOrlando, FL: National Security Industrial Association.Google Scholar
Carretta, T. R., & Ree, M. J. (2003). Pilot selection methods. In Tsang, P. S. & Vidulich, M. A. (Eds.), Principles and practice of aviation psychology (pp. 357–396). Mahwah, NJ: Lawrence Erlbaum Associates.Google Scholar
Colegrove, C. M., & Alliger, G. M. (2002, April). Mission essential competencies: Defining combat mission readiness in a novel way. Paper presented at the NATO RTO Studies, Analysis and Simulation (SAS) Panel Symposium. Brussels, Belgium.
Colegrove, C. M., & Bennett, W., Jr. (2005). Competency-based training: Adapting to warfighter needs. Paper presented at the meeting of the Royal Astronautic and Engineering Society, London, UK.
Crane, P. M., Robbins, R., & Bennett, W., Jr. (2000). Using distributed mission training to augment flight lead upgrade training. In 2000 Proceedings of the interservice/industry training systems and education conference. Orlando, FL: National Security Industrial Association.Google Scholar
Crosbie, R. J. (1995). The history of the dynamic flight simulator. Washington, DC: Department of the Navy.Google Scholar
Dickman, J. L. (1982). Automated performance measurement: An overview and assessment. In Proceedings of the 4th interservice/industry training equipment conference: Volume I (pp. 153–165). Washington, DC: National Security and Industrial Association.Google Scholar
Dillard, A. E. (2002). Validation of advanced flight simulators for human-factors operational evaluation and training programs. In Workshop on foundations for modeling and simulation (M&S) verification and validation (V&V) in the 21st Century (Foundations '02 V&V Workshop). Laurel, MD: Johns Hopkins University, Applied Physics Laboratory.Google Scholar
Dockeray, F. C., & Isaacs, S. (1921). Psychological research in aviation in Italy, France, England and the American Expeditionary Forces. Journal of Comparative Psychology, 1, 115–148.CrossRefGoogle Scholar
Gray, T. H., Chun, E. K., Warner, H. D., & Eubanks, J. L. (1981). Advanced flight simulator: Utilization in A-10 conversion and air-to-surface attack training (AFHRL-TR-80–20, AD A094 608). Williams Air Force Base, AZ: Air Force Human Resources Laboratory, Operations Training Division.Google Scholar
Gray, T. H., & Fuller, R. R. (1977). Effects of simulator training and platform motion on air-to-surface weapons delivery training (AFHRL-TR-77–29, AD A043 648). Williams Air Force Base, AZ: Air Force Human Resources Laboratory, Operations Training Division.CrossRefGoogle Scholar
Hagin, W. V., Durall, E. P., & Prophet, W. W. (1979). Transfer of training effectiveness evaluation: US Navy Device 2B35 (Seville Research Corporation Rep. No. TR79–06, ADA073669). Pensacola, FL: Chief of Naval Education and Training.Google Scholar
Hughes, R., Brooks, R. B., Graham, D., Sheen, R., & Dickens, T. (1982). Tactical ground attack: On the transfer of training from flight simulator to operational Red Flag exercise. In Proceedings of the 4th interservice/industry training equipment conference: Volume I (pp. 127–130). Washington, DC: National Security Industrial Association.Google Scholar
Jenkins, D. H. (1982). Simulation training effectiveness evaluation (TAC Project No. 79Y-OO1F). Nellis AFB, NV: Tactical Fighter Weapons Center.Google Scholar
Kellogg, R., Prather, E., & Castore, C. (1980). Simulated A-10 combat environment. In Proceedings of the Human Factors Society 24th annual meeting (pp. 573–577). Santa Monica, CA: Human Factors Society.Google Scholar
Krusmark, M., Schreiber, B. T., & Bennett, W., Jr. (2004). Measurement and analysis of F-16 4-ship team performance in a simulated distributed mission training environment (AFHRL-HE-AZ-TR-2004–0090). Mesa, AZ: Air Force Research Laboratory Human Effectiveness Directorate, Warfighter Readiness Research Division.Google Scholar
Leeds, J., Raspotnik, W. B., & Gular, S. (1990). The training effectiveness of the simulator for air-to-air combat (Contract No. F33615–86-C-0012). San Diego, CA: Logicon.Google Scholar
Lintern, G., Sheppard, D., Parker, D. L., Yates, K. E., & Nolan, M. D. (1989). Simulator design and instructional features for air-to-ground attack: A transfer study. Human Factors, 31, 87–100.CrossRefGoogle Scholar
Mayer, G. B., Jr. (1981). Determining the training effectiveness and cost-effectiveness of visual flight simulators for military aircraft (AD A104627, Master's thesis). Monterey, CA: Naval Postgraduate School.Google Scholar
McGuinness, J., Bouwman, J. H., & Puig, J. A. (1982). Effectiveness evaluation for air combat training (ADP000203). In Proceedings of the 4th interservice/industry training equipment conference: Volume I (pp. 391–396). Washington, DC: National Security Industrial Association.Google Scholar
McGuinness, J., Forbes, J. M., & Rhoads, J. E. (1984). Air combat maneuvering performance measurement system design (AFHRL-TP-83–56). Williams Air Force Base, AZ: Operations Training Division, Armstrong Laboratory.Google Scholar
Moroney, W. F., & Moroney, B. W. (1999). Flight simulation. In Garland, D. J., Wise, J. A., & Hopkin, V. D. (Eds.), Handbook of aviation human factors. Mahwah, NJ:Lawrence Erlbaum Associates.Google Scholar
Nullmeyer, R. T., & Spiker, V. A. (2003). The importance of crew resource management in MC-130P mission performance: Implications for training evaluation. Military Psychology, 15(1), 77–96.CrossRefGoogle Scholar
Payne, T. A., Hirsch, D. L., Semple, C. A., Farmer, J. R., Spring, W. G., Sanders, M. S., et al. (1976). Experiments to evaluate advanced flight simulation in air combat pilot training: Volume I. Transfer of learning experiment. Hawthorne, CA: Northrop Corporation.Google Scholar
Polzella, D. J., Hubbard, D. C., Brown, J. E., & McLean, H. (1987). Aircrew training devices: Utility and utilization of advanced instructional features. Phase 4. Summary report (AFHRL-TR-87–21; ADA188418). Brooks Air Force Base, TX: Air Force Human Resources Laboratory.Google Scholar
Robinson, J. C., Eubanks, J. L., & Eddowes, E. E. (1981). Evaluation of pilot air combat maneuvering performance changes during TAC ACES training. Nellis Air Force Base, NV: U.S. Air Force Tactical Fighter Weapons Center.Google Scholar
Schreiber, B. T., & Bennett, W., Jr. (2006). Distributed mission operations within-simulator training effectiveness baseline study: Summary report (AFRL-HE-AZ-TR-2006–0015-Vol I, ADA461866). Mesa, AZ: Air Force Research Laboratory, Warfighter Readiness Research Division.Google Scholar
Schreiber, B. T., Stock, W.A., & Bennett, W., Jr. (2006). Distributed mission operations within-simulator training effectiveness baseline study: Metric development and objectively quantifying the degree of learning (AFRL-HE-AZ-TR-2006–0015-Vol II, ADA461867). Mesa, AZ: Air Force Research Laboratory, Human Effectiveness Directorate, Warfighter Readiness Research Division.Google Scholar
Schreiber, B. T., Watz, E., & Bennett, W., Jr. (2007). DIS: Does interoperability suffice? A need to set a higher standard (07S-SIW-067, 2007 Spring SIWzie Award). In 2007 spring simulation interoperability workshop conference. Norfolk, VA: SISO.Google Scholar
Schreiber, B. T., Watz, E., Bennett, W., Jr., & Portrey, A. (2003). Development of a distributed mission training automated performance tracking system. In Proceedings of the 12th conference on behavior representation in modeling and simulation (BRIMS). Scottsdale, AZ.Google Scholar
Symons, S., France, M., Bell, J., & Bennett, W., Jr. (2005). Linking knowledge and skills to mission essential competency-based syllabus development for distributed mission operations (AFRL-HE-AZ-TR-2006–0014). Mesa, AZ: Air Force Research Laboratory, Warfighter Readiness Research Division.Google Scholar
Corp, Vreuls Research. (1987). Air combat maneuvering performance measurement system for SAAC/ACMI, Volume II, Appendices 5 & 6. Wright-Patterson Air Force Base, OH: Air Force Systems Command.Google Scholar
Waag, W. L. (1991). The value of air combat simulation: Strong opinions but little evidence. In The Royal Aeronautical Society flight simulator symposium: Training transfer – Can we trust flight simulation? Proceedings. London: The Royal Aeronautical Society.Google Scholar
Waag, W. L., & Bell, H. H. (1997). Estimating the training effectiveness of interactive air combat simulation (AL/HR-TP-1996–0039). Armstrong Laboratory, AZ: Aircrew Training Research Division.CrossRefGoogle Scholar
Waag, W. L., Houck, M., Greschke, D. A., & Raspotnik, W. B. (1995). Use of multiship simulation as a tool for measuring and training situation awareness. In AGARD Conference Proceedings 575 Situation awareness: Limitations and enhancement in the aviation environment (AGARD-CP-575). Neuilly-Sur-Seine, France: Advisory Group for Aerospace Research & Development.Google Scholar
Wickens, C. D. (2003). Aviation displays. In Tsang, P. S. & Vidulich, M. A. (Eds.), Principles and practice of aviation psychology (pp. 357–396). Mahwah, NJ: Lawrence Erlbaum Associates.Google Scholar
Wiekhorst, L. A., & Killion, T. H. (1986). Transfer of electronic combat skills from a flight simulator to the aircraft (AFHRL-TR-86–45, AD C040 549). Williams Air Force Base, AZ: Air Force Human Resources Laboratory, Operations Training Division.Google Scholar
Wiekhorst, L. A. (1987). Contract ground-based training evaluation: Executive summary (ADA219510). Langley Air Force Base, VA: Tactical Air Command.Google Scholar
Wooldridge, L., Obermayer, R. W., Nelson, W. H., Kelly, M. J., Vreuls, D., & Norman, D. A. (1982). Air combat maneuvering performance measurement state space analysis (AFHRL-TR-82–15; ADA121901). Brooks Air Force Base, TX: Air Force Human Resources Laboratory.CrossRefGoogle Scholar

Save book to Kindle

To save this book to your Kindle, first ensure [email protected] is added to your Approved Personal Document E-mail List under your Personal Document Settings on the Manage Your Content and Devices page of your Amazon account. Then enter the ‘name’ part of your Kindle email address below. Find out more about saving to your Kindle.

Note you can select to save to either the @free.kindle.com or @kindle.com variations. ‘@free.kindle.com’ emails are free but can only be saved to your device when it is connected to wi-fi. ‘@kindle.com’ emails can be delivered even when you are not connected to wi-fi, but note that service fees apply.

Find out more about the Kindle Personal Document Service.

Available formats
×

Save book to Dropbox

To save content items to your account, please confirm that you agree to abide by our usage policies. If this is the first time you use this feature, you will be asked to authorise Cambridge Core to connect with your account. Find out more about saving content to Dropbox.

Available formats
×

Save book to Google Drive

To save content items to your account, please confirm that you agree to abide by our usage policies. If this is the first time you use this feature, you will be asked to authorise Cambridge Core to connect with your account. Find out more about saving content to Google Drive.

Available formats
×