EUROCONTROL EXPERIMENTAL CENTRE EEC Report N° 350 · EUROCONTROL EXPERIMENTAL CENTRE Gaze analysis...

EUROPEAN ORGANISATIONFOR THE SAFETY OF AIR NAVIGATION

EUROCONTROL EXPERIMENTAL CENTRE

Gaze analysis and psychophysiological parameters: A tool for the design and the evaluation of Man-Machine Interfaces

Feasibility study

EEC Report N° 350

Project HRS-Z-05_PHYS

Issued: July 2000

The information contained in this document is the property of the EUROCONTROL Agency and no part shouldbe reproduced in any form without the Agency’s permission.

The views expressed herein do not necessarily reflect the official views or policy of the Agency.

EUROCONTROL

REPORT DOCUMENTATION PAGE

Reference:EEC Report N° 350

Security Classification:Unclassified

Originator:

EEC - IND

(INDependent Reseach)

Originator (Corporate Author) Name/Location:EUROCONTROL Experimental CentreCentre des Bois des BordesB.P.15F - 91222 Brétigny-sur-Orge CEDEXFRANCETelephone : +33 (0)1 69 88 75 00

Sponsor: Sponsor (Contract Authority) Name/Location:EUROCONTROL AgencyRue de la Fusée, 96B -1130 BRUXELLESTelephone : +32 (0)2 729 90 11

TITLE: GAZE ANALYSIS AND PSYCHOPHYSIOLOGICAL PARAMETERS : A TOOL FOR THE DESIGNAND THE EVALUATION OF MAN-MACHINE INTERFACES

FEASIBILITY STUDY

Authors P. Cabon, B. Farbos

R. MollardLAA, Université R. Descartes Paris V

Date

07/2000

Pages

x + 55

Figures/Photos

26 / 7

Tables

1

Annex

1

References

19

EATCHIP TaskSpecification

-

Project

HRS-Z-05_PHYS

Task No. Sponsor

-

Period

1999

Distribution Statement:(a) Controlled by: Head of IND(b) Special Limitations None(c) Copy to NTIS: Yes

Descriptors (keywords):

ATC, human factors, HMI, gaze analysis, perceptual aspect, information processing, EEG, HRV, workload,stress, fatigue.

Abstract:This study demonstrates the value of gaze analysis with simultaneous physiological data recording for theassessment of the human-computer interface in air traffic control.The literature on visual scanning was reviewed. Two studies using different eye-tracking systems, one head-mounted (I VIEW) and one remote (ASL 504) were conducted at EEC. In each study, a TRACON IIsimulator was used to present controllers with a representative ATC task and a set of relevant electro-physiological measures (EEG, HRV etc..) was employed.This study shows that selected eye-view and physiological parameters may be used together to evaluatestrain, fatigue, operational strategies of controllers in highly interactive situations.Further studies should investigate the effect of changes in the nature and extent of automation of the ATCtask, such electronic stripping, flat displays etc.

This document has been collated by mechanical means. Should there be missing pages, please report to:

EUROCONTROL Experimental CentrePublications Office

Centre des Bois des BordesB.P. 15

91222 - BRETIGNY-SUR-ORGE CEDEXFrance

v

GAZE ANALYSIS AND PSYCHOPHYSIOLOGICAL PARAMETERS :

A TOOL FOR THE DESIGN AND THE EVALUATION OF MAN-MACHINE INTERFACES

FEASIBILITY STUDY

By

P. Cabon, B. Farbos, R. Mollard LAA Université René Descartes - Paris V

SUMMARY

This study demonstrates the value of gaze analysis with simultaneous physiological datarecording for the assessment of the human- computer interface in air traffic control.

The literature on visual scanning was reviewed. Two studies using different eye-trackingsystems, one head-mounted (I VIEW) and one remote (ASL 504) were conducted at EEC.In each study, a TRACON II simulator was used to present controllers with a representativeATC task and a set of relevant electro-physiological measures (EEG, HRV etc…) wasemployed.

This study shows that selected eye-view and physiological parameters may be used togetherto evaluate strain, fatigue, operational strategies of controllers in highly interactive situations.

Further studies should investigate the effect of changes in the nature and extent of automationof ATC task, such electronc stripping, flat displays etc.. .

vii

FOREWORD

The need for more ‘objective’ measures of the effects of Air Traffic Control on controllers haslong been apparent. Various studies have been undertaken at EEC on potential methods ofmeasuring these effects (EEC Reports n° 64, 164, 183, 226, 228, 323, 334 and 339).

The LAA (Laboratoire d’Anthropologie Appliquée, Université Paris V René Descartes) underthe direction of Professor Alex Coblentz, has carried out studies of fatigue, sleep disorder andassociated strain among Airline Pilots, Car Drivers and Air Traffic Controllers among others.

We therefore commissioned the LAA to undertake a series of feasibility studies on the use ofCortisol analysis, Electroencephalography, in particular Cortical Evoked Potential and otherrelated methods, which were reported in EEC Report n° 323 and in EEC Note n° 16/98. Thesestudies used a TRACON II autonomous simulator. LAA subsequently undertook the transferof these methods to real-time simulation, as reported in EEC Reports n° 334 and 339.

This report covers further TRACON II studies of eye movement recording methods, operatedin association with electrophysiological studies.

On behalf of the Director EEC, the experimenters for the LAA and the EEC Project Officerswish to thank the controllers and the EEC staff involved for their active and dedicatedco-operation.

Françoise CalooHugh David

EEC Project Officers.

ix

TABLE OF CONTENTS

ABBREVIATIONS............................................................................................................................................... X

1 - INTRODUCTION AND AIMS - .................................................................................................................... 3

2 - LITERATURE REVIEW - ............................................................................................................................. 5

2.1 - THE EYE TRACKING TECHNIQUE -................................................................................................................ 52.2 - DATA PROCESSING METHODS - .................................................................................................................... 6

2.2.1 - Fixation chronology methods - ........................................................................................................... 62.2.2 - Statistical analysis of gaze in a specific area - ................................................................................... 82.2.3 - Analysis of fixation groups - ............................................................................................................... 92.2.4 Weighted Search Area (WSA) ............................................................................................................. 10

2.3 - APPLICATIONS OF VISUAL SCANNING IN ERGONOMICS - ............................................................................ 10

3 - SYNTHESIS OF THE IVIEW STUDY - ..................................................................................................... 14

4 - ASL STUDY - ................................................................................................................................................. 16

4.1 - AIMS - ....................................................................................................................................................... 164.2 - METHOD - ................................................................................................................................................. 16

4.2.1 - Controllers and sample - .................................................................................................................. 164.2.2 – Eye-tracking device description - .................................................................................................... 164.2.3 - Data collected - ................................................................................................................................ 18

4.3. - RESULTS - ................................................................................................................................................ 204.3.1 - Fixation frequency and duration - .................................................................................................... 204.3.2 - Physiological data -.......................................................................................................................... 214.3.3 - Controllers’ orders -......................................................................................................................... 224.3.4 - Perceived workload -........................................................................................................................ 23

5 - ERRORS AND VISUAL SCANNING ANALYZIS - ................................................................................. 24

5. 1 - THE IVIEW SYSTEM - .............................................................................................................................. 255.1.1 - Analyzis of errors - 5 minutes -......................................................................................................... 255.1.2 - Analyzis of errors - 2 minutes -......................................................................................................... 27

5.2 - THE ASL SYSTEM - ................................................................................................................................... 305.2.1 - Analyzis of errors - 5 minutes -......................................................................................................... 305.2.2 - Analyzis of errors - Weighted Search Area method -........................................................................ 32

6 - DISCUSSION AND CONCLUSION - ......................................................................................................... 41

7 - REFERENCES -............................................................................................................................................. 43

ANNEX................................................................................................................................................................. 45

FRENCH TRANLATION .................................................................................................................................. 49

Green pages: French translation of the foreword, the summary, the introduction, objectives, discussion and conclusion.Pages vertes : Traduction en langue française de l'avant-propos, du résumé, de l’introduction, des objectifs, de la discussion

et conclusion.

x

ABBREVIATIONS

ASL Applied Science Laboratories

Astrips Active Strips

ATC Air Traffic Controller

CRT Cathodic Ray Tube

CTTT Controllers Transition Trials Tools

EEC Eurocontrol Experimental Centre

EEG Electroencephalogram

EUROCONTROL European Organisation for the Safety of Air Navigation

HMI Human Machine Interface

HRV Heart Rate Variability

IVIEW System Video-based contact-free evaluation of gaze position

KeyB Key Board

LAA Laboratoire d'Anthropologie Appliquée

NAC Nippon Automatic Corporation

NASA National Aviation and Space Agency

NASA TLX Nasa Task Load Index

NOLDUS Noldus Information Technology Inc

OBSERVER video pro Tool for the study of human behavior

POPUP Window displayed in response to behavior mouse click

Pstrips Pending Strips

SMI Senso Motoric Instruments

TRACON Terminal Radar Approach Control ( Simulator )

WSA Weighted Search Area

1

A N T H R O P O L O G I E A P P L I Q U E E

45, rue des Saints-Pères 75270 PARIS Cedex 06Téléphone : 01 42 86 20 37- 01 42 86 20 41 -Télécopie : 01 42 61 53 80

E.mail : [email protected]

* * *

GAZE ANALYSIS AND PSYCHOPHYSIOLOGICAL PARAMETERS :

A TOOL FOR THE DESIGN AND THE EVALUATION OF MAN-MACHINE

INTERFACES FEASIBILITY STUDY

* * *

DOC AA 407/00 June 2000

2

This study was conducted with the fruitful collaboration of Mrs Françoise Caloo and MrHugh David.The laboratory expressed their grateful acknowledgements to the Air Traffic Controllers whowere actively involved in these experiments :

Ms Andy BarffFranck DowlingPeter EriksenRoger GerreauPaul HaworthYann KermarquerBernard KerstenneFrançois VergneAntoine VidalJean Paul Zabka

3

1 - INTRODUCTION AND AIMS -

The introduction of new technologies in the field of Air Traffic Control leads to

deep changes in the Controllers’ working procedures and tools. Traditional radar

displays and paper strips are progressively being replaced by cathodic displays,

electronic strips, a mouse and a keyboard. These changes can affect the performance of

Controllers in different ways and it is crucial to ensure that these new tools are capable

of helping Controllers in managing traffic. In this context, the previous studies

conducted at the Eurocontrol Experimental Centre were intended to validate a

methodology to address some important issues for the development of these new tools.

This methodology was based on physiological and subjective data concerning workload,

stress and fatigue.

The present study aims to complete these evaluations with gaze analysis. In fact,

visual scanning is one of the first steps of information processing. However, a basic

distinction should be made between “looking” and “seeing”. Considering that one can

“see” without “looking” gaze analysis itself is not sufficient for the evaluation of

human-machine interfaces. This is the reason why it is necessary to develop a

comprehensible approach that should allow to take into account some perceptual

aspects as well as the information processing of Controllers. Therefore, gaze analysis

and physiological data can maximise the evaluation both in terms of the impact of a new

device on the Controller’s state as well as for the design of new interfaces. In particular,

the integration of new tools using windows makes the use of large displays

(2000X2000) necessary. Thus, the way information is presented to Controllers is

changing. Because of these changes, the Controllers’ visual strategies are changing

drastically. For example, depending on its position on the display some information

could be neglected. It is thus useful to get some objective evaluations of the impact of

these new technologies on Controllers.

The main aim of the feasibility study was to evaluate the relevance of a

methodology using both gaze analysis and physiological data to assess interfaces. With

this aim in view, simple simulations using the TRACON software were presented to 10

Controllers. TRACON has already been used in previous studies (Cabon et al, 1997) for

the investigation of several issues such as the use of psychophysiological measures for

the evaluation of workload, learning process, or stress… Brookings et al (1996) also

used it on a study on workload. There were two main advantages to using TRACON for

the purpose of this feasibility study : it is easy to learn for Controllers and various

scenarios can be setup. Obviously, the simulations played with TRACON cannot be

4

considered either as reality or as the complex real-time simulations played at

EUROCONTROL.

However, as the aim of this study is not to provide some results on TRACON itself

but on the physiological measures and gaze data, TRACON appears to be very

adapted to this context. Moreover, in previous studies Controllers felt the simulations to

be realistic regarding the normal control activities and they were highly motivated in

using the software.

After a literature review on the topic of visual scanning, this report presents a synthesis

of the two feasibility studies conducted at the Eurocontrol Experimental Centre (EEC).

In these two studies, two eye tracking devices were used. In the first study, a head-

mounted device (IVIEW) was used. Its main results are summarised. In the second

study, a remote device was tested (ASL 504). Thanks to technical improvements, the

ASL device has allowed a very precise analysis of errors. The methodology and the

main results are presented.

5

2 - LITERATURE REVIEW -

The aim of this review is to summarize:

- the eye tracking technique,

- the data processing methods,

- the main applications of this method in ergonomics, in particular in aeronautics.

2.1 - The eye tracking technique -

The eye tracking use the following components (figure 1):

- a video camera and an infrared light that tracks the eye movements (eye system),

- a video camera recording the scene (scene camera),

- a real-time data processing system calculating the gaze direction.

Figure 1

Description of the principle for the acquisition of gaze

(adapted from Rousseau et coll., 1994)

Scenecamera

Eye ofsubject

Processingsystem of

data

Scenepicture

Eye camera

Infrared

Eye picture

6

This device displays the location of gaze points on the scene observed by the subject.

The device can be either head-mounted (e.g. IVIEW) or remote (e.g. ASL 504).

Depending on the device used, a head movement’s compensation is achieved. This

compensation permits to process automatically the gaze co-ordinates.

2.2 - Data processing methods -

Main data obtained from the eye tracking are the number and the duration of

fixations on specified area of the visual field. Depending on objectives, 4 classes of data

processing methods can be distinguished:

- fixation chronology,

- statistical analysis of gaze in a specific area,

- analysis of fixation groups,

- weighted search area.

2.2.1 - Fixation chronology methods -

These methods are based on the analysis of fixation sequences. These sequences

can be presented in two ways:

- by indicating the fixation points routes (figure 2),

- by indicating the fixation points succession in a specified area of the visual field

(figure 3).

The first method provides a spatial information whereas the latter gives a

temporal information.

7

Figure 2

Illustration of gaze point during one session

(adapted from Menu et al., 1994)

Figure 3

Fixation time in one selected area during a reading task

(adapted from Menu et al., 1994)

Y c

oord

inat

es

Time of session (s)

X coordinates

Fixa

tion

time

(ms)

Time of session (s)

8

2.2.2 - Statistical analysis of gaze in a specific area -

These methods are based on the statistical analysis of the fixation distribution in a

specific area (figure 4). This method allows an hierarchical organization of the various

areas in terms of information search.

Figure 4

Description of fixation frequency for three sectors defined from driving simulation

(adapted from Laya, 1994)

0

20

40

60

80

100

Left In front Right

Freq

uenc

y fi

xatio

n (s

)

9

2.2.3 - Analysis of fixation groups -

Two kinds of methods can be distinguished:

- Modelling of transition probability from one area to another (Papin, 1981)

- Factor analysis (Rousseau et al, 1994 (figure 5) which allows the separation of

some sectors which are statistically different and the evaluation of the chronology

of fixations and the quantity of fixations.

The width of the arrows indicate the number of fixation sequence

Figure 5

Illustration of factor analysis realized from sequence of fixations on a car

(adapted from Rousseau et al, 1994)

Sect

or 2

Sector 3

Sector 4Sector 4

Sector 1

sect

or 5

0

20

40

60

80

100

1 2 3 4 5

Sector

Num

ber

of f

ixat

ion

sequ

ence

Sector

10

2.2.4 Weighted Search Area (WSA)

The WSA developped by Chia and Fang-Tsan (1997) is based on the integration

of fixation frequency and duration. It is based on the description by vectors of visual

fixations and on assignment of weight to each vector by its respective fixation time.

Hence, the aim of WSA is to describe concentrations of visual strategies and potentially

quantify the amount of visual load (figure 6). This method was applied in our study of

error analysis (section 5.2.2).

Figure 6

Illustration of Weighted Search Area method (WSA) - This method is based on the

description by vectors of frequency and time of fixations

(adapted from Chia and Tsang, 1997)

2.3 - Applications of visual scanning in ergonomics -

For the operator, visual scanning consists of directing his gaze toward

informative areas. The analysis of these different fixation areas permits the evaluation

of the strategy used by the operator in taking the information and of the efficiency of

this strategy regarding his activity.

This efficiency is influenced by various factors. Drury and Sinclair (1983)

showed that visual scanning is influenced by the task difficulty. In that study, task

difficulty was mainly related to the size, the contrast and the variety of the objects.

11

The temporal organization, i.e. the duration and the rhythm of an activity is also

an important factor which influences the visual scanning pattern and the performance

(Moraal, 1974). Togami (1984) showed that when the activity rhythm is high, the

fixation number and duration increase. They also observed that the increase of the

fixation duration is more efficient to improve performance than the increase of the

fixation number.

In addition, Saito (1972) showed that the higher is the temporal demand, the less

the visual scanning is large (i.e., less objects are inspected). Simon et al (1993) observed

that the visual scanning is more structured when the workload increases. These results

are confirmed by Tole et al (1982) on aircraft pilots. When the pilot has to cope with a

high workload, he has less time to look at various instruments and therefore adopt a less

random visual scanning strategy. Other studies conducted in simulators found the same

results (Krebs et al, 1977, Harris et al, 1986).

These results are consistent with the conclusions of Sperandio (1991). Operators

have to adapt their strategy to cope with the increase of task demand by using a more

“economic” visual strategy to optimize the information taking.

In the field of aeronautics, visual scanning behaviour was shown to be very useful

for the interface design. (Table 1) summarized some recent studies conducted in this

field.

12

AUTHORS OBJECTIVE SAMPLE DATA

COLLECTIONRESULTS

Bellendes et al.(1997)

Examination of differences inattention control betweenexperienced and novice pilots frommeasurement of visual scanning

- Twelve students pilots

- Twelve Flight instructors

- Flight simulator

- Number of fixations- Mean dwell time

- Novices tended to dwell for a longer time aneach instrument

- Information is extracted more efficiently byexperts from nearly all instruments anparticularly from the most information withinstrument

Itoh (1990) Examination of the differences inscanning behaviour under conditionsof high mental workload betweenpilots using and integrated displayand pilots using the earlier electro-mechanical display

- Five experimented pilots - Number of fixations- Fixation time

- The proportion of fixation duration increases inresponse to the increase of mental workload

- In terms of the ease of obtaining informationfrom instruments the integrated display usingthe CRT might be more preferable for pilotstask

Tole et al.(1982)

Examination of the relationshipbetween pilot visual scan ofinstruments and mental workload

- Six subjects varying in skill-level from non pilot to ahighly experienced NASAtest pilot

- Flight simulation

- Time course of eyefixations

- Dwell time of eachfixation

- Scanning behaviour may serve as an indicatorof both workload and skill.

-As a pilot’s skills develop, his visual scanningbehaviour is less as less affected by non visualincrements in workload.

- Pilots with a high experienced skill present adecrease of the percentage of long dwells.

David(1977)

Provide a detail description of threemethods as means for measuring theController’s eye movement and thenecessary : direct observation, videotranscription and eye tracker system

- Two teams consisted foreach by 6 Controllers

- Video- Number and durationof fixation

- Eye mark recording (NAC Mark V) is precisebut intrusive.

- Video-recording requires about four times thelength of observation: it may be biased byomission.

- Direct observation is very "cheap" but requiresfour times also the length of observation: insomme circumstances it may not be acceptableto the Controller.

Table 1

13

For Air Traffic Control, the understanding of controllers visual search behaviour

becomes more and more important with the prevalent use of computer screens and other

visual display devices. In fact, the evaluation of controllers’ visual scanning patterns could

help to improve the design of these new interfaces.

In 1975, Karsten et al. found already that controllers spend approximately 80% of

their time looking at the radar display, 13% at flight strips and 5% at input devices.

However, in a real time simulation, David (1977) showed that the number of fixations

directed at various devices were not necessarily indications of the relative efficiencies of

displays. This observation seems to be related to the system limitation (NAC). New

generation eyes tracking systems (e.g. ASL) now overcome these limitations

More recently, the research of Stein (1987), showed also from a real time ATC

simulation that the longest fixation duration’s were observed for aircraft compared to other

objects on the radarscope. For the authors, the inclusion of other objects and scene planes

could drastically reduce the average duration of the fixations on aircraft.

However, only a few studies have associated gaze data and others objectives or

subjectives data to evaluate the information processing related to perception.

Itoh (1992) observed, in pilots, with the measure of heart rate variability, no

difference in mental workload between pilots using integrated CRT displays (Boeing 767)

and pilots using earlier electromechanical displays (Boeing 747-300) (table 1). Regarding

the visual scanning, they noticed that when the workload increased, the fixation duration

for certain instruments increased. In addition, they observed a significant difference of

visual scanning between the two interfaces during abnormal situation (e.g. taking off and

landing with engine failures). In fact, the changes in scanning pattern for B767 were found

to be smaller than those of the B747 pilots. Based on these results, the authors concluded

that the application of CRT helps pilots to obtain sufficient information more easily than

electromechanical displays do, even under abnormal situations.

Only one study has used both psychophysiological and visual scanning

measurements in Air Traffic Controller (Brookings et al, 1996). Although, the Brookings’

study did not evaluated the fixation duration and number of fixations but the vertical blink

and horizontal saccade activity, their results indicated that « as the visual demands

increased, the controllers blinked less often so as not to miss important events on the radar

scope ». For the author, this interpretation is supported by other investigations reporting

decreased blink rates when operators were under conditions of increased visual load.

14

3 - SYNTHESIS OF THE IVIEW STUDY -

The first step of this study was intended to evaluate the interest of using eye tracking

technique and physiological data for the evaluation of various aspects in the context of

interface design (photos 1 and 2). In the context of this study, it was decided to test the

following hypothesis: the use of different methods for entering data, the keyboard or the

trackball, changes the visual scanning of Controllers as well as their attention and

workload. In fact, it is assumed that the use of the keyboard forced the Controller to look

frequently at the keyboard and, consequently, reduced his visual inspection on the radar

and his attention towards the information displayed. The experimental protocol allowed to

compare 4 conditions combining the use of a trackball and a keyboard and 2 levels of

traffic:

- keyboard and low level of traffic,

- keyboard and high level of traffic

- trackball and low level of traffic,

- trackball and high level of traffic.

Photo 1

15

Photo 2

Eight subjects were involved in this study. The data collected were: fixation time

and frequency on various items of the visual field, Heart Rate Variability for mental effort

evaluation, the electroencephalogram (EEG) for the assessment of attention, the NASA-

TLX for perceived workload and simulator data for the Controllers’action and

performance.

The comparison between the Keyboard conditions and the Trackball conditions

shows that:

- changes in fixations are more frequent in the keyboard conditions but fixations are

longer in the Trackball conditions,

- performance is better in the Trackball conditions,

- workload is not affected,

- attention is higher in the Trackball conditions.

The comparison of the two levels of traffic indicates that:

- gaze frequency and duration are not affected,

- workload is higher in the 30-aircraft condition,

- handover errors are increased with traffic.

Therefore this study showed that the use of both gaze data and physiological

measures are able to evaluate the impact of different interfaces.

16

4 - ASL STUDY -

4.1 - Aims -

The aim of this second study is twofold:

- to examine the advantages of the ASL 504 device compared to IVIEW,

- to study the interest of the data collected as regards error analysis.

4.2 - Method -

4.2.1 - Controllers and sample -

Four Controllers were involved in this study. Among these 4 controllers, 2 had

already participated in the previous IVIEW study. Each Controller was presented with 2

simulations on the TRACON. The two Controllers who were not involved in the previous

study were presented a traffic of 15 aircraft in 30 minutes (session 1) and then a traffic of

30 aircraft in 30 minutes (session 2). As they were already familiar with TRACON, the 2

other Controllers were presented only with two sessions with 30 aircraft. For this study,

only the trackball was used by the Controllers as the previous study had demonstrated that

the graphical interface (trackball) was better than the numerical interface (keyboard

condition).

4.2.2 – Eye-tracking device description -

The eye-tracking device used in this study is the ASL 504. Contrary to IVIEW, the

ASL 504 is not head-mounted but totally remote (photos 3 and 4). It is composed of an

eye camera located beside the TRACON display. An infrared light, emitted by the camera,

is reflected on the controllers’cornea. The only thing that the controller wears on his

forehead is a transmitter that gives his head’s position. This information is sent to an aerial

located behind the Controller’s head and processed by a computer that allows to the

camera to track the eyes thanks to a motor.

17

Photo 3

Photo 4

18

Therefore, this device takes into account all the controllers’ eyes movements (photo 5).

The method to determine the gaze position is the same as for IVIEW. Compared to IVIEW,

one of the main advantages of ASL is that it compensates the head movements and

therefore provides for each gaze its co-ordinates relative to the display. The TRACON

display was filmed using a video camera during all the sessions.

Photo 5

4.2.3 - Data collected -

The data collected were the same as those collected in the previous study:

- gaze: fixation frequency and duration,

- physiological: Heart Rate Variability (HRV) and the electroencephalogram (EEG),

- simulator data: Controllers’ actions and errors,

- subjective: workload (NASA-TLX).

19

Contrary to the IVEW study, the ASL 504 allowed to process the data automatically.

In the IVIEW study, the fixation points were obtained by reviewing the video recording at

a lower speed (5 times less than the normal speed) and by manually coding the gaze

position using the OBSERVER (photo 6). In the present study, the OBSERVER was used

to collect the Controllers’ actions (photo 7).

Photo 6

Photo 7

20

4.3. - Results -

The results were analyzed in two ways:

- a global analysis in which the two levels of traffic were compared,

- an analysis focused on the errors.

4.3.1 - Fixation frequency and duration -

Figures 7 and 8 show the fixation frequency and duration on the menu, the radar,

the strips and the communication window for the two levels of traffic. It can be observed

that for all the items, the fixations frequency and duration are more frequent in 15-aircraft

condition. However, the fixations duration on the radar are longer in the 30-aircraft

condition. Therefore, the Controllers changed their fixation more in the lowest traffic

condition and spent less time looking at the radar. This result can be explained by the fact

that in the lowest traffic condition, the Controllers used were those with the least

experience in TRACON while the 30-aircraft condition included the 2 controllers who had

been involved in the first study. This also related to the fact that the traffic increase the

controller tend to focus to only a few important elements of the display.

Figure 7 – Fixation frequency as a function of items and traffic levels.

0

2

4

6

8

10

12

14

16

Menu Radar Strips Com Off

Traffic level : 15 aircraft (n = 2 ATC)


Fix

atio

n (m

n)

21

Figure 8. Fixation duration as a function of items and traffic levels.

4.3.2 - Physiological data -

HRV and EEG results are presented in figure 9. The results show that the power in

the middle frequency is increased (i.e. the effort is increased) in the 30-aircraft condition.

EEG data indicate also an increase of attention in the 30 aircraft condition for the 4

Controllers.

Figure 9. Power in the theta frequency band of the EEG and in the middle frequency of the HeartRate Variability as a function of traffic.

0

10

20

30

40

50

60

Menu Radar Strips Com Off



150

160

170

180

190

200

210

220

230

T15 T300

10

20

30

40

50

60

70

80

90

100

T15 T30

Pow

er in

the

mid

dle

freq

uenc

y (m

n)

Pow

er in

the

The

ta b

and

(mn)

Fix

atio

n du

rati

on (

%/m

n)

22

4.3.3 - Controllers’ orders -

The number of Controllers’ orders divided by the number of aircraft (figure 10)

show that Controllers changed the aircraft heading more than their speed or altitude. This

result is consistent with the results obtained in the previous study. It can also be observed

that Controllers provide more orders per aircraft at the highest level of traffic in order to

cope with the task difficulty.

Figure 10. Frequency of controllers’ orders as a function of traffic

0.00

0.10

0.20

0.30

0.40

0.50

0.60

0.70

0.80

0.90

1.00

Altitude Speed Heading



Con

trol

lers

ord

ers

(mn)

23

4.3.4 - Perceived workload -

Figure 11 shows the results of the workload assessment after the two sessions for

the Controllers who were familiar and for the Controllers who were not familiar with

TRACON. In the first sessions, it can be observed that perceived workload is higher for

Controllers who were not familiar although they were only presented with 15 aircraft. In

the second session, the less experienced Controllers perceived a higher workload but with

less differences than in the first session.

Figure 11Assessment workload as a function of traffic.

Mentaldemand

Temporaldemand

Performance Effort Frustrationlevel

Physicaldemand

Mentaldemand

Temporaldemand

Performance Effort Frustrationlevel

Physicaldemand

Arb

itra

ry U

nits

Arb

itra

ry U

nits

100

80

60

40

20

0

100

80

60

40

20

0

Not experienced / 30 aircraft Experienced / 30 aircraft

Not experienced / 15 aircraft Experienced / 30 aircraft

24

5 - ERRORS AND VISUAL SCANNING ANALYZIS -

Three kinds of situations were analysed with the IVIEW and ASL systems:

separation conflicts, missed approaches and handover errors. The number of these errors

was determined with the TRACON software, at the end of each session. The time of errors

occurrence was determined with video recordings. Finally, each error’ analysis was

realised by taking into account gaze movements, physiological data and the conditions of

each session.

For the IVIEW system, the analyses were carried out in two ways:

- a mean was calculated for each set of data 5 minutes before Controller’ errors,

- data were processed per second, 2 minutes before errors.

With the ASL system, two kinds of analyses were also achieved:

- gaze movements were automatically analysed for 16 areas of the radar screen, 5

minutes before errors (figures 12 and 13),

- the application of the Chia and Fang-Tsan’s method: the Weighted Search Area.

Figure 12

Description of the areas analyzed with the Iview system

Radar

Communication

Strips

25

Figure 13

Description of the areas analyzed with the ASL system

5. 1 - The IVIEW system -

5.1.1 - Analyzis of errors - 5 minutes -

Five minutes before an error, the visual scanning of each Controller was determined

from this fixation frequency. The fixation frequency was calculated for the following

items: Radar, Keyboard, Astrips, Pstrips, Popup and Others (figure 14).

No significant difference was observed per item and error. Moreover, no effect of

the trackball-keyboard and traffic level conditions was observed on missed approach,

handover and separation conflict errors.

Nevertheless, two results can be noticed:

- the number of fixation decreased on the Keyboard, before a missed approach,

- the fixation frequency increased for the Astrips before a separation conflict error.

Radar

Communication

Strips

Radar Radar Radar

Radar Radar Radar Radar

Radar Radar Radar Radar

RadarRadarRadarRadar

26

Figure 14

Mean fixation frequency 5 minutes before missed approach, handover and

separation conflict errors

- Iview system -

0

50

100

150

200

250

Radar Keyb Astrips Pstrips Popup Others

Fix

atio

n fr

eque

ncy

Missed approach Handover Separation conflict

27


One subject’s duration and frequency of fixation, 2 minutes before a separation

conflict is presented on figure 15. Data obtained from these two gaze parameters were

processed per second.

Figure n 15

Description of duration and frequency fixation and the HRV of one Controller 2 minutes before a

separation conflict – Iview system -

00:00 00:15 00:30 00:45 01:00 01:15 01:30 01:45 02:00

popup

radar

pstrips

astrips

keyb

mouse

others

Time

-150-100

-500

50

100150

Var

iati

on o

f H

RV

(%

)

28

This data processing allowed to compare simultaneously the visual scanning of a

Controller and his workload determined by HRV.

No particular visual scanning was observed 2 minutes before the separation conflict.

Generally, when the workload was increasing we observed that duration of fixation was

increasing. The same trend was observed for the other kinds of errors: missed approach

(figure 16) and handover errors (figure 17).

Time before error

Figure 16

Description of the visual scanning and evolution of HRV of Controller n°3

before a missed approach - Iview system -

-150-100

-500

50

100150

Var

iati

on o

f H

RV

(%

)

p o p u p

r a d a r

p s t r ip s

a s t r ip s

k e y b

m o u s e

o t h e r s

02:00 01:45 01:30 01:15 01:00 00:45 00:30 00:15 00:00

29

Time before error

Figure 17

Description of the visual scanning and evolution of HRV of Controller n°4

before a handover error - Iview system -

popup

radar

pstrips

astrips

keyb

mouse

others

-200

-100

0

100

200

Var

iati

on o

f H

RV

(%

)

02:00 01:45 01:30 01:15 01:00 00:45 00:30 00:15 00:00

30

Moreover, the data showed a great inter-individual variability of visual scanning

before the occurrence of an error. Two hypotheses could explain this variability:

- every Controller will present different visual scanning patterns whatever the kind of

error,

- IVIEW did not allow to determine one particular visual scanning. Hence, it was

impossible to use the IVIEW’s data to determine sectors of the radar covered by the

Controller’s visual scanning. In fact, data processing with IVIEW system is extremely

time-consuming.

In conclusion, the IVIEW system did not allow a description of visual search pattern

before error. These results were associated with a limitation of the IVIEW system.

Nevertheless, the IVIEW data suggest a relation between workload and gaze patterns: the

higher the workload, the longer the fixation time. This increase in workload could be due

to an increase in visual load.

5.2 - The ASL system -

With the ASL system, the analysis of errors and visual scanning of Controllers was

faster than with the IVIEW system. Because ASL allows to compensate the head

movements an automatic processing was possible.

While the TRACON’s screen was divided into three big sectors with the IVIEW

system (cf figure 12) the ASL system allowed a more detailed processing. Therefore, the

radar was divided into 16 sectors (cf figure 13).


Figures 18 and 19 show the evolution of duration and number of fixation of one

Controller (n°9), 5 minutes before a separation conflict.

This presentation of results shows that the gaze zone is localized mainly into radar

sectors 4, 7, 8, 11 and 12. These zones correspond to pending strips and to an airport being

often associated with many missed approaches.

It can also be observed that before this conflict, the duration and the number of

fixation were high: their increase suggests that a Controller’s gaze is better focussed on this

area.

31

Therefore, this method provides a global and a quantitative information on the

Controller’s gaze on a period. However, it does not permit a detailed analyzis of visual

pattern just before the error as regards the place where the error occurred.

Figure 18

Fixation duration over the 16 radar sectors 5 minutes before separation conflict

Controllers n°9 - ASL system -

Figure 19

Fixation number over the 16 radar sectors 5 minutes before separation conflict

Controllers n°9 - ASL system -

Fix

atio

n n

um

ber

0

10

20

30

40

50

60

70

Radar sectors

Radar 1

Radar 2

Radar 3

Radar 4

Radar 5

Radar 6

Radar 7

Radar 8

Radar 9

Radar10

Radar 11

Radar 12

Radar 16

Radar 15

Radar 14

Radar 1313

1415

16

4

8

12

Radar 1

Radar 2

Radar 3

Radar 4

Radar 5

Radar 6

Radar 7

Radar 8

Radar 9

Radar10

Radar 11

Radar 12

Radar 16

Radar 15

Radar 14

Radar 130

10

20

30

40

50

60

70

80

90

Radar sectors

1314

1516

4

8

12

32

5.2.2 - Analyzis of errors - Weighted Search Area method -

In order to determine the evolution of duration and frequency of fixation before the

occurrence of an error, a new method reflecting the visual area covered by a subject was

applied.

This method was developed by (Chia-Fen and Fang-Tsan, 1997) and named

Weighted Search Area (WSA). Its principle is to determine visual fixations by vectors on

an X-Y co-ordinate system. Vectors are hence assigned by a weight from theirs fixation

duration. Through various procedures of adding vectors and rotating axes, the WSA and

overall fixation vector can be calculated. This methodology is used as a quantitative

measure for visual load where a large amount of visual attention is necessary.

Figures 20 and 21 show examples of the WSA graph calculated for 2 subjects on

the whole of their session. The description of the totality of gaze points is also proposed

simultaneously.

Figure 22 shows for the same Controller the WSA graph 15, 10 and 5 minutes

before a separation conflict. The results show a progressive decrease of Controller’s visual

scanning 10 and 5 minutes before error.

In addition, five minutes before this error, WSAs per minute show that a Controller

does not look at the area where the conflict occurred (figure 23). One minute before the

error the WSA showed that the gaze of Controller n°9 was focussed on a zone, outside the

conflict zone.

The same result was obtained for another Controller who made the same error

(figures 24 and 25).

At the opposite, during a missed approach, the use of the WSA method showed that

the Controller looked at the zone where the missed approach occurred (figure 26).

Therefore, these results suggest that some errors are due to visual perception

whereas others are related to the information processing. However, this errors classification

could be better analyzed with the confrontation and interviews of the Controller with his

gaze pattern after the simulation.

33

Gaze points Gaze points and associated WSA

Figure 20

Description of the gaze points and associated-WSA for Controllers n°9 on the whole session (30 aircraft per 30 minutes)

- ASL system -

34

Gaze points Gaze points and associated WSA

Figure 21

Description of the gaze points and associated-WSA for Controllers 11 on the whole session (30 aircraft per 30 minutes)

- ASL system -

35

Figure 22

Description of the WSA 15, 10 and 5 minutes before separation conflict

- Controllers 9, condition 30 aircraft per 30 minutes - ASL system

15 minutes before error10 minutes before error5 minutes before error

36

Figure 23

Description of the WSA each minute, 5 minutes before separation conflict


5 minute before error4 minute before error3 minute before error2 minute before error1 minute before error

Separation conflict

37

Figure 24

Description of the WSA 15, 10 and 5 minutes before separation conflict


15 minutes before error10 minutes before error5 minutes before error

38

Figure 25

Description of the WSA each minute, 5 minutes before separation conflict


Separation conflict


39

Figure 26

Description of the WSA each minute, 5 minutes before a missed approach


Missed approach


41

6 - DISCUSSION AND CONCLUSION -

This study was intended to evaluate the feasibility of using both physiological and

gaze data for the evaluation of Human-Machine Interface in the field of Air Traffic

Control. It was conducted in two steps:

- in a first step, a head-mounted eye-tracking device (IVIEW), was used to test the

differences in the visual scanning patterns of Controllers using a keyboard or a

trackball,

- in a second step, a remote eye tracking device compensating the head movements was

used (ASL 504). This step was more focused on the analysis of errors. The data

collected with IVIEW were also processed regarding errors.

The simulations were presented on the TRACON software. The first study has

involved 8 controllers, the second 4 controllers. Two controllers were involved in the two

studies. In addition to gaze parameters, physiological data (Heart Rate Variability for

mental effort, EEG for attention process) and subjective data (NASA-TLX for perceived

workload) were collected.

The results of the IVIEW study showed the relevance of the data collected to

evaluate the impact of the keyboard versus the trackball conditions. However, it has also

raised the limitations of the eye-tracking device related to the fact that it does not

compensate the head movements. Due to these limitations, the data processing was manual,

time-consuming and cannot be enough detailed to conduct a precise analysis.

Opposite, the ASL 504 tested in the second study has two main advantages. First, as

it is a remote device, it reduces the constraints for the operators. Secondly, as the eye

camera movements are linked to the head movements it compensates all the movements of

the operator. Thanks to this compensation, the data processing is automatic and thus is less

time-consuming and more precise than the IVIEW device. Moreover, the ASL 504 allows

to work on multi-display environment. The only limitation that was noticed concerns the

operators who wear spectacles. For some of them, the gaze data are lost during a few

seconds.

The ASL 504 was particularly useful for the error analzsis. In fact, it allowed to

evaluate the area of the radar covered by the controllers’ visual scanning in the minutes

preceding the occurrence of the errors. This analysis allows to distinguish two kinds of

errors: those related to a lack of visual perception of the area where the error occurred and

those due to the processing of the situation by the Controller. In the first kind of error, the

42

visual scanning area does not cover the point where the error occurred whereas in the

second type the visual scanning area covers this point. Therefore, for this latter, the cause

of the error would be more related to the cognitive process of the situation than to the

perception of the situation. This errors analysis using eye tracking should be confirmed on

a larger sample. Also, it should be useful to have a debriefing with the controller at the end

of the session in order to better interpret the cause of the error.

The IVIEW data regarding the error, although less detailed than those collected with

ASL suggest a relationship between HRV and visual scanning. Most of the time, in the

minutes preceding an error an increase of the power in the middle frequency of HRV (i.e.

an increase of mental effort) is observed while the fixation duration increase. This result is

consistent with those of Itoh et al (1990) who found that higher fixation duration is

observed when the workload increased. This result can be explained by the fact the

increase in task load imposed that the pilot and the Controller take information and

concentrate their attention on a limited amount of information.

In conclusion, the feasibility study presented in this report had demonstrated the

interest of using both physiological and gaze data in the context of the evaluation of a

specific HMI.

Therefore, it is proposed to transpose this method in more specific simulation as a

tool for the evaluation of new Controller ‘ workplace. In this context, the data collected

should be useful for the evaluation of the CTTT and Skytool concepts (Controllers

Transition Trials Tools). More specifically, this transposition will focus on the impact of

the automation of the Controllers’ workplace as well as on the use of electronic stripping

presented on flat displays. This study will allow to take into account different levels of

automation, from the traditional Controllers’ workplace to a highly automated HMI using

flat displays. This study would permit to provide some recommendations for the

optimization of the information displayed to the Controller.

43

7 - REFERENCES -

BELLENKES (A.H.) ; WICKENS (C.D.) ; KRAMER (A.F.).- Visual scanning and pilot

expertise: the role of attentional flexibility and mental model development.- Aviation, Space and

Environmental Medicine, 1997, vol. 68, n° 7, pp. 569-579.

BROOKINGS (J.B.) ; WILSON (G.F.) ; SWAIN (C.R.).- Psychophysiological responses

to changes in workload during simulated air traffic control. -Biological Psychology, 1996, 42,

pp. 361-377.

CABON (P.) ; MOLLARD (R.) ; COINTOT (B.) ; MARTEL (A.) ; BESLOT (P.).-

Elaboration of a Method for the Assessment of psychophysiological states of Air Traffic

Controllers in Simulation. EEC Report N° 323, 1997, 64 p.

CHIA (F.C.) ; FANG-TSAN (F).- A new method for describing search patterns and

quantifying visual load using eye movement data.- International Journal of Industrial

Ergonomics, 1997, 19, pp. 249-257.

DAVID (H.).- Measurement of Air Traffic Controllers’ eye movements in Real-Time

Simulation. EEC Report N° 187, 1977, 35 p.

DRURY (C.G.) ; SINCLAIR (M.A.).- Human and machine performance in an inspection

task.- Human factors, 1983, vol. 25, n°4, pp. 391-399.

HARRIS (R.L.) ; GLOVER (B.J.) ; SPADY (A.A.).- Analytical Techniques of Pilot

Scanning Behaviour and Their Applications.- NASA papers 2525, 1986.

ITOH (Y.) ; HAYASHI (Y.) ; TSUKUI (I.) ; SAITO (S.).- The ergonomic evaluation of

eye movement and mental workload in aircraft pilots.- Ergonomics, 1990, vol. 33, n° 6, pp. 719-

733.

KREBS (M.J.) ; WINGERT (J.W.) ; CUNNINGHAM (T.).- Exploration of an

Oculometer-Based Model of Pilot Workload.- NASA CR-145153, 1977.

44

LAYA (O.).- De la situation réelle à la simulation : exemples d’analyses de l’exploration

visuelle de l’automobiliste.- Editions Octares, Collection Colloques, 1994, pp. 69-92.

MENU (J.P.) ; PLANTIER (J.) ; VALOT (C.).- Les stratégies oculomotrices d’exploration

d’images satelliales.- Editions Octares, Collection Colloques, 1994, pp. 25-36.

MOORAL (J.).- Visual inspection of sheet steel : an ergonomic study in the Dutch steel

industry.- Le Travail Humain, 1974, vol. 31, n°1, pp. 35-52.

ROUSSEAU (F.) ; LOSLEVER (P.) ; ANGUE (J.C.).- L’exploration visuelle d’images

publicitaires.- L’analyse ergonomique du travail par l’étude de l’exploration visuelle.- Editions

Octares, Collection Colloques, 1994, pp. 37-56.

SAITO (M.).- Studies on bottles inspection task.- Journal of Science of Labour, 1972, vol.

48, n°8, pp. 475-495.

SIMON (P.) ; ROUSSEAU (F.) ; ANGUE (J.C.). Quantitative analysis of mental

workload influence on eye scanning movements.- IEEE/SMC’93 Conference, Systems

engineering in the service of humans, Le touquet – France, 1993.

TOGAMI (H.).- Affects on visual search performance of individual differences in fixation

time and number of fixations.- Ergonomics, 1984, vol. 27, n° 7, pp. 789-799.

TOLE (J.R.) ; SYTEPHENS (A.T.) ; HARRIS (R.L.) ; EPHRATH (A.R.).- Visual

scanning behavior and mental workload in aircraft pilots.- Aviation, Space and Environmental

Medicine, 1982, pp. 54-60.

SPERANDIO (J.C.).- La recherche en psychologie (Domaines et méthodes).- Edition

DUNOD, 1991, pp. 199-262.

STEIN (E.S.).- Where will all our air traffic controllers be in the year 2001 ? Proceeding

of the Fourth International Symposium on aviation, Columbus, pp. 195-201.

45

ANNEX

Detailed operating procedures forthe Weighted search area (wsa)

47

1 - Acquire the horizontal and vertical coordinates of each fixation point (Xij ; Yij)

from the eye movement data, where i = 1, 2, 3, 4 denote the four quadrants on a two-

dimensional plane and j = 1, 2, 3, …., n equal the total number of fixations taken by a

subject. These coordinate values correspond to the fixation vector extending from the

origin.

2 – Multiply each fixation vector by its respective fixation time tij, then the vector

becomes r V (t ij X ij ; t ijYij), where tij > 0.

3 – Derive the compound vector r V i (Xi ; Yi) by summing up all the component

vectors on each quadrant : Xi = tijXijj =1

n

∑ and Yi = tijYijj =1

n

∑ where i = 1, 2, 3, 4 denote the

four quadrants. The lenght (Li) of the compound vector on each quadrant equals Li =(Xi

2+Yi2). The angle () of the compound vector

r V i on each quadrant can be derived by

ß = tan-1Xi

Yi where i = 1, 2, 3, 4 denote four quadrants.

4 – Project all the r V i onto each axis and sum up by the following rules : L0(X+) =

L1cosß1 + L4cosß4, L0(X-) = L2cosß2 + L3cosß3, L0(Y+) = L1sinß1 + L2sinß2, L0(Y-) =

L3sinß3 + L4sinß4 where L0(X+) is the total projected length on the X axis for X>0, L0(X-)

is the total projected length on the X axis for X<0, L0(Y+) is the total projected length on

the Y axis for Y>0, L0(Y-) is the total projected length on the Y axis for Y<0.

5 – Rotate the horizontal and vertical axes, where 00 < K0 < 900 . Both axes would

be rotated K degrees by R times, where R = (900 / K) –1. Then, reproject the compound

vector r V i onto the new references axes. Though the reprojection, Lr(X+), Lr(X-), Lr(Y+),

Lr(Y-) can be derived where Lr(X+) is the total projected length on the X axis for X>0,

Lr(X-) is the total projected length on the X axis for X<0, Lr(Y+) is the total projected

length on the Y axis for Y>0, Lr(Y-) is the total projected length on the Y axis for Y<0, in

the r rotated coordinate system. Procedures would be repeated R times for r = 1 until r = R.

49

Traduction en langue française de l’Avant-propos, du Résumé, del’Introduction, des Objectifs, de la Discussion et Conclusion.

ANALYSE DU REGARD ET MESURES PSYCHOPHYSIOLOGIQUES:

UNE METHODE POUR LE DEVELOPPEMENT ET L’EVALUATION DESINTERFACES HOMME-MACHINE.

ETUDE DE FAISABILITE

par

P. Cabon, B. Farbos, R. MollardLAA Université René Descartes - Paris V

AVANT - PROPOS

Le besoin d'évaluer plus objectivement l'impact du contrôle aérien sur les contrôleurs est unepréoccupation ancienne. De nombreuses études ont été conduites au CEE pour déterminer lesméthodes d'investigations les plus adaptées pour en mesurer les effets (CEE Rapports n° 64, 164,183, 226, 228, 323, 334 et 339).

Le LAA (Laboratoire d'Anthropologie Appliquée de l'Université Paris V, René Descartes) amené, sous la direction du Pr Alex Coblentz, de nombreuses études sur la fatigue, les troubles dusommeil et autres troubles associés sur les pilotes de lignes, les conducteurs d'automobiles et lescontrôleurs aériens.

C'est pourquoi, nous avons demandé au LAA, de faire une série d'études de faisabilité surl'utilisation d'analyse du cortisol, d'électro-encéphalographie en particulier la technique du PEA(Potentiel Evoqué Auditif) ainsi que d'autres méthodes ad hoc et qui sont décrites dans unrapport CEE n° 323 et une note CEE n°16/98. Ces études ont été réalisées par le LAA sur unsimulateur autonome TRACON II et ultérieurement sur des simulations temps-réel et fait l'objetdes rapports CEE n° 334 et 339.

Ce rapport couvre des études complémentaires d'analyses des mouvements oculaires associées àdes mesures électro-physiologiques, également réalisées sur TRACON II.

Le Directeur du Centre Expérimental, les chercheurs du Laboratoire d'Anthropologie Appliquéeainsi que les chefs de projets tiennent à exprimer leurs remerciements aux contrôleurs et auxpersonnels du CEE pour leur étroite et dévouée collaboration.

Françoise CalooHugh David

EEC project officers

51

RESUME

Cette étude démontre la pertinence d'une méthode reposant à la fois sur l'analyse des regardscouplée à des données physiologiques pour évaluer des interfaces Homme-Machine dans ledomaine du contrôle de la circulation aérienne.

Une revue de la littérature sur le thème des mouvements du regard a précédé deux études qui ontété conduites au Centre Expérimental Eurocontrol. Deux systèmes ont été utilisés, dans lapremière étude un système ( I VIEW) posé sur la tête et dans la deuxième, un système déporté(ASL504). Un logiciel TRACON II permettant de présenter aux contrôleurs des tâches ATCreprésentatives et une sélection de mesures electro-physiologiques (EEG, HRV etc….) ontégalement, été utilisés pour ces deux études.

Cette étude conforte l'intérêt d'associer des paramètres physiologiques et d'analyse du regardpour évaluer la charge de travail, la fatigue, les stratégies opérationnelles des contrôleurs dans unenvironnement hautement interactif.

Des études complémentaires devraient être poursuivies pour analyser l'impact des changementsdans les tâches ATC induit par l'introduction et la généralisation de nouvelles technologiesinformatiques tels que: strips électroniques, écrans plats etc…

53

1 - INTRODUCTION ET OBJECTIFS -

L’introduction de nouvelles technologies dans le domaine du contrôle aérien

entraîne des changements importants dans les procédures de travail et dans les outils

utilisés par les contrôleurs. Les écrans radar traditionnels et les strips papiers sont

progressivement remplacés par des écrans cathodiques, des strips électroniques, une

souris,... Ces changements peuvent affecter la charge de travail et la performance des

contrôleurs. Il apparaît essentiel de s’assurer que ces nouvelles technologies soient

capables d’aider les contrôleurs à gérer un trafic en constante augmentation. Dans ce

contexte, les travaux menés par le Laboratoire d’Anthropologie Appliquée en collaboration

avec le Centre Expérimental d’Eurocontrol ont pour objectif de valider des méthodes

d’évaluations permettant d’étudier l’impact de ces nouvelles technologies sur les

contrôleurs. Ces méthodes reposent sur le recueil de données physiologiques et subjectives,

concernant la charge de travail, le stress, la fatigue et l’activité des contrôleurs.

La présente étude vise à compléter ces évaluations avec l’utilisation de l’analyse des

mouvements du regard. Elle a pour but de prendre en compte les aspects perceptifs et de

traitement de l’information lors de l’évaluation d’un nouvel outil ou pendant la

conception d’une nouvelle interface. L’intégration de nouveaux outils informatiques se

traduit en effet fréquemment par l’utilisation d’écrans de grande taille (2000X2000), ce qui

peut influer sur la présentation des informations par rapport aux postes de travail

traditionnels mais aussi modifier les stratégies visuelles des contrôleurs. En fonction de

leur localisation sur l’écran, certaines informations peuvent éventuellement être négligées

par les contrôleurs.

L’objectif principal de cette étude consiste à évaluer la pertinence d’une méthode,

reposant à la fois sur l’analyse des regards couplée à des données physiologiques, pour

évaluer des interfaces. La démarche retenue repose sur l’utilisation de simulations

simplifiées à partir du logiciel TRACON, déjà utilisé dans des études précédentes (Cabon

et coll., 1997) pour évaluer la charge de travail, les processus d’apprentissage et le stress

des contrôleurs. Deux campagnes d’essais ont été conduites au Centre Expérimental

d’Eurocontrol. Dans la première campagne, un système (IVIEW) posé sur la tête du

contrôleur a été utilisé. Dans la seconde, un système déporté a été testé (ASL504). Ce

dernier est équipé d’un dispositif de compensation des mouvements de la tête permettant

de réaliser des traitements plus précis sur les stratégies oculaires adoptées par les

contrôleurs. Ce rapport présente à la fois une revue de la littérature sur le thème des

mouvements du regard ainsi qu’une synthèse des principaux résultats obtenus au cours de

ces deux campagnes.

54

2 - DISCUSSION ET CONCLUSION -

La première campagne de mesure a été effectuée avec la participation de 8

contrôleurs, la seconde avec 4 contrôleurs. Deux contrôleurs ont été impliqués dans les

deux campagnes. En complément des paramètres de regard (nombre et fréquence de

fixations), des données physiologiques (variabilité cardiaque pour l’évaluation de l’effort

mental, l’EEG pour les processus attentionnels) et des données subjectives (NASA-TLX

pour la perception de l’effort) ont été recueillies.

Les résultats de l’étude « IVIEW » (première campagne de mesure) montrent

l’intérêt des données recueillies pour évaluer l’impact de l’utilisation du clavier comparé à

une boule roulante. Cette étude a également mis en évidence les limites d’IVIEW en raison

de l’absence de compensation des mouvements de la tête. De ce fait, le traitement des

données reste manuel, long et n’est pas suffisamment détaillé pour effectuer des analyses

précises. A l’opposé, le système ASL 504, évalué dans la seconde étude, présente deux

avantages principaux. Le premier est qu’il réduit les contraintes pour le contrôleur

puisqu’il n’est pas posé sur la tête. Le second a trait au dispositif de compensation des

mouvements de la tête qui permet d’effectuer un traitement automatique, beaucoup plus

rapide et précis qu’avec IVEW. Par ailleurs, ASL504 peut être utilisé pour évaluer des

postes de travail qui comportent plusieurs écrans. La seule difficulté technique rencontrée

concerne les contrôleurs qui portent des lunettes. Dans ce cas, les données ne peuvent pas

être exploitables sur de courtes périodes (quelques secondes) d’enregistrements.

Le système ASL s’est révélé particulièrement utile pour l’analyse des erreurs. En

effet, il permet d’évaluer précisément la localisation du regard sur l’écran dans les minutes

précédant l’erreur. Cette analyse a permis de distinguer deux types d’erreurs : celles liées à

une absence de perception visuelle de la zone où l’erreur a eu lieu et celles associées à une

absence de traitement de la situation. Dans le premier cas, les regards ne couvrent pas la

zone de l’erreur alors que dans le second, le contrôleur regardait cette zone. Néanmoins,

cette analyse d’erreur doit être confirmée sur un échantillon plus important. Un entretien

d’auto-confrontation avec le contrôleur à la fin de la session, permettrait également de

mieux interpréter ce type d’erreur, en particulier sur le plan de la stratégie adoptée par le

contrôleur.

Les données concernant les erreurs collectées lors des essais avec le système IVIEW

bien que moins détaillées que celles recueillies avec ASL, suggèrent une relation entre la

variabilité cardiaque et les mouvements du regard. Généralement, dans les minutes

précédant une erreur, une augmentation de la puissance dans la bande de fréquence

moyenne (c’est-à-dire une augmentation de l’effort mental) s’observe pour la variabilité

55

cardiaque alors que la durée des fixations augmente. Ces résultats confirment ceux de Itoh

et coll. (1990) qui ont observé que l’augmentation de la durée de fixation se trouve

associée à une augmentation de la charge de travail. Ce résultat peut être expliqué par le

fait que l’augmentation de la difficulté de la tâche entraîne une focalisation de l’attention

sur un nombre limité d’informations.

En conclusion, cette étude de faisabilité démontre l’intérêt d’utiliser à la fois des

données physiologiques et de regard dans le contexte de l’évaluation d’une interface

spécifique. Il est donc proposé de transposer cette méthode à des simulations plus

complexes comme outil d’évaluation de nouveaux postes de travail des contrôleurs. Dans

ce contexte, les données recueillies peuvent s’avérer utiles pour l’évaluation des concepts

CTTT (Controllers Transition Trials Tools) et Skytools. Cette transposition pourrait être

centrée sur l’impact de l’automatisation et de l’utilisation des strips électroniques sur la

charge de travail des contrôleurs.

EUROCONTROL EXPERIMENTAL CENTRE EEC Report N° 350 · EUROCONTROL EXPERIMENTAL CENTRE Gaze analysis...

Documents

Transcript of EUROCONTROL EXPERIMENTAL CENTRE EEC Report N° 350 · EUROCONTROL EXPERIMENTAL CENTRE Gaze analysis...