An Assessment of General Aviation Pilot Performance During Simulated Flight


The primary aim of this study was the development of a set of normative data that captured the performance of a sample of general aviation pilots during a simulated flight from Wagga Wagga to Bankstown via Canberra, Goulburn and Mittagong. A secondary aim was to consider the impact of pilot qualification on the performance of pilots during the five legs of the flight.

Pilots were issued a completed flight plan and all the relevant documents necessary to complete the flight, including weather information, maps, and an aircraft checklist. A total of 34 pilots were recruited to undertake the flight and the exercise was conducted as it would be expected to occur within the operational environment. The experimenter acted as the Flightwatch operator and air traffic controller where necessary, and recorded the details of the flight.

Data pertaining to in-flight performance were recorded at a number of different levels of analysis, the first of which was pilots' own self-reports of their performance. Pilots' performance was also rated by an observer, and assessments were made on a number of different dimensions including the accuracy with which the aircraft was controlled, the accuracy of the track flown, the accuracy in maintaining the prescribed altitude, the level of fatigue management, and the appropriateness of the communication. The final level of analysis involved objective data that were recorded each second that the simulator was in operation. For each of the five legs of the flight, a set of geographic boundaries were identified and representative data that occurred with these boundaries were summarised using measures of central tendency1.

In relation to the self-report data, pilots considered their performance in the flight simulator poorer than their performance in general. This may be explained by the difficulties that some pilots perceived in exercising control over the simulated aircraft. Indeed, of the eight dimensions assessed, aircraft control was associated with the lowest rating during the simulation. However, it should also be noted that relatively lower ratings were recorded for other variables including fuel management, fatigue management, scanning, and decision-making.

The observations of pilot performance revealed differences between perceived behaviour during the five legs of the flight. Specifically, performance during leg 5, the last leg, tended to be rated at a level consistently lower than performance during the preceding legs. Comparative analyses using pilot qualification as a between-groups factor failed to explain the basis for this difference in perceived performance.

The differences between the perceived performance of pilots in leg 5 and perceived performance during the preceding legs of the flight were further examined using the data recorded by the flight simulator. While differences were anticipated for variables such as altitude, it appeared that performance deteriorated on a range of variables, including the mean range of the heading and the mean range of the pitch angle of the aircraft. The variability in performance during the final leg of the flight could not be explained on the basis of pilot qualification, and suggests that other factors may be impacting on performance. It was considered that these factors might include the impact of fatigue and/or the impact of the demands in conducting a stepped descent to avoid violations of controlled airspace during the approach to Bankstown airport.

Overall, the data acquired in the present study represent a useful normative dataset against which the performance of pilots can be assessed in the future. As expected, there is a significant level of variability in the performance of pilots who conducted the simulated approach. This variability was most evident during the final stage of the flight when the demands on pilots were most acute and when the impact of fatigue was most likely to occur. This represents an avenue for future research and development.

  1. Measures of central tendency are statistical summaries of a set of data. The most common measure of central tendency is the mean (average), followed by the median (the middle score in a series of rank-ordered data), and the mode (the most frequently occurring result).
Type: Research and Analysis Report
Author(s): Dr Mark Wiggins
Publication date: 23 May 2006
ISBN: 1 921092 49 1
Last update 07 April 2014
Download complete document
[ Download PDF: 2.17MB]