Interactive and Immersive 3D Interactive and Immersive 3D Visualization for ATCVisualization for ATC

Matt Cooper & Marcus LangeMatt Cooper & Marcus LangeNorrköping Visualization and Interaction StudioNorrköping Visualization and Interaction StudioUniversity of Linköping, SwedenUniversity of Linköping, Sweden

Summary of last Summary of last presentationpresentation

A quick description of the application A quick description of the application as presented at last year’s workshopas presented at last year’s workshop

3D Visualization for ATM3D Visualization for ATM

Display flights in 3DDisplay flights in 3D–– Show waypointsShow waypoints–– Show Trajectories (flight plans)Show Trajectories (flight plans)Display Terrain in 3DDisplay Terrain in 3DUse 3D models to show flightsUse 3D models to show flightsTarget (SemiTarget (Semi--)Immersive displays)Immersive displays–– WorkbenchWorkbench–– VR Theatre (‘Reality Centre’)VR Theatre (‘Reality Centre’)

3D Visualization for ATM3D Visualization for ATM

Other information is also 3DOther information is also 3D–– Not just dodging the other planesNot just dodging the other planes

Aircraft have to be navigated around Aircraft have to be navigated around 3D obstacles:3D obstacles:–– Exclusion zonesExclusion zones–– Weather featuresWeather features

‘Chop reports’‘Chop reports’Storms and localized featuresStorms and localized features

Weather(3)Weather(3)

Conflict detectionConflict detection

Immersive solution: VoiceImmersive solution: Voice

Use voice controlUse voice control–– to provide access to complex featuresto provide access to complex features–– Use in any environmentUse in any environment–– (almost) No buttons required(almost) No buttons required

Use Voice feedback for confirmationUse Voice feedback for confirmation–– Also warnings and other informationAlso warnings and other information

Stay immersedStay immersed

Future work (as at 12/2003)Future work (as at 12/2003)

New features to addNew features to add–– More weather featuresMore weather features–– More powerful conflict detectionMore powerful conflict detection–– turbulence wakes for spacingturbulence wakes for spacing–– More flights between other airportsMore flights between other airportsMore work on the interaction modesMore work on the interaction modesMore work needed on voice interactionMore work needed on voice interactionBetter voice feedbackBetter voice feedback

Better information Better information displaydisplay

We have looked at displaying more information We have looked at displaying more information about flightsabout flights–– Used selfUsed self--sorting flagssorting flags

Flags carry information about each flightFlags carry information about each flight–– Flight numberFlight number–– DestinationDestination–– SpeedSpeed–– AltitudeAltitude

Flags sort themselves so that they are visible from Flags sort themselves so that they are visible from any angle as the camera view changesany angle as the camera view changes–– Overlaps are automatically avoided Overlaps are automatically avoided –– no clutterno clutter

Stereo makes it easy to match labels with flightsStereo makes it easy to match labels with flights

Better orientation displayBetter orientation display

Orientation information is hard to includeOrientation information is hard to includeBetter orientation is included using a Better orientation is included using a compass rosecompass rose–– Shown using transparency over existing terrain Shown using transparency over existing terrain

displaydisplayMoves with the user view in workbench Moves with the user view in workbench display display The ring described by the displayed rose The ring described by the displayed rose can also be used for selection where can also be used for selection where appropriateappropriate

Better use of dataBetter use of data--gloveglove

The data glove has been used to The data glove has been used to provide gestureprovide gesture--based interactionbased interactionThe combined handThe combined hand--eye interaction eye interaction approach allows for a more approach allows for a more sophisticated selection mechanismsophisticated selection mechanism–– Use coUse co--ordinated selection to give user a ordinated selection to give user a

natural selection processnatural selection process–– Picking with your handPicking with your hand

Picking using hand and Picking using hand and dominant eyedominant eye

Crosshair

GloveGlove--based pickingbased picking

WakeWake--vortex displaysvortex displays

Can display wake vortices according to more Can display wake vortices according to more powerful data representationpowerful data representationThe data is based on physical modelling of The data is based on physical modelling of the wake vortexthe wake vortex–– Data obtained from Peter Data obtained from Peter ChorobaChoroba at ERCat ERC

Provides wake vortex intensity across a grid Provides wake vortex intensity across a grid orthogonal to flight directionorthogonal to flight direction–– Depends on weather conditionsDepends on weather conditions

We display regions where the intensity is We display regions where the intensity is above some thresholdabove some threshold

WakeWake--vortex displaysvortex displays

Problems are that the variation of the Problems are that the variation of the wake vortex is small on the scale of wake vortex is small on the scale of the 3D displaythe 3D display–– We see straight lines behind the aircraftWe see straight lines behind the aircraft

Might be more appropriate where the Might be more appropriate where the resolution of the display is finerresolution of the display is finer

Wake vorticesWake vortices

Conflict detectionConflict detection

Previous work on conflict detection was very Previous work on conflict detection was very coarsecoarseRelied on checking flight’s exclusion zones Relied on checking flight’s exclusion zones at a number of specific points along flight at a number of specific points along flight path, ahead of the current positionpath, ahead of the current positionNew approach depends on checking the New approach depends on checking the zones described by each flight using zones described by each flight using intersections of the described volumesintersections of the described volumes

Conflict detection (2)Conflict detection (2)

The flight paths are tracked for a The flight paths are tracked for a number of minutes into the futurenumber of minutes into the futureEach line segment describes a volume Each line segment describes a volume which is checked against all others in which is checked against all others in the same time zonethe same time zoneThe results are checked in realThe results are checked in real--time as time as a flight is manipulateda flight is manipulated–– Allows realAllows real--time conflict resolutiontime conflict resolution

Better weather Better weather informationinformation

In previous version weather In previous version weather information was complexinformation was complex–– Included a great deal of information but Included a great deal of information but

in a form which was difficult to interpretin a form which was difficult to interpretWe have tried to reduce the We have tried to reduce the complexity of the information without complexity of the information without reducing it’s contentreducing it’s content–– Based on the idea of inclusion/exclusion Based on the idea of inclusion/exclusion

zones rather than detailed informationzones rather than detailed information

Better weather Better weather informationinformation

Automatic identification of zones of Automatic identification of zones of –– Low pressure Low pressure –– weather systemsweather systems–– High turbulenceHigh turbulence–– High wind speedHigh wind speedRepresented using translucent Represented using translucent surfaces which are automatically surfaces which are automatically extracted from the volume weather extracted from the volume weather datadata

Automatic weather ‘hotAutomatic weather ‘hot--spots’spots’

Audio feedback systemAudio feedback system

Sound alerts using speech to provide Sound alerts using speech to provide informationinformationPlaced in the user’s work environment using Placed in the user’s work environment using ‘positional sound’ based on‘positional sound’ based on–– Amplitude modulationAmplitude modulation–– Phase delaysPhase delays

Very realistic positional effect drawing user’s Very realistic positional effect drawing user’s attention to the correct view locationattention to the correct view location

More Airports and FlightsMore Airports and Flights

Original data starting to seem smallOriginal data starting to seem smallNumber of active flights was under 20 Number of active flights was under 20 at any one timeat any one timeNo flights between other airports No flights between other airports includedincludedNeeded expandingNeeded expanding

More airports and flightsMore airports and flights

Scenario SimulatorScenario Simulator

Work undertaken at Work undertaken at EurocontrolEurocontrol by by Magnus Magnus AxholtAxholt and Stephen and Stephen PeterssonPetersson–– Final year Students from Final year Students from LiULiU

Developed a scenario import system Developed a scenario import system for the 3D applicationfor the 3D application–– Should now be possible to expand our Should now be possible to expand our

repertoire of data and scenarios to repertoire of data and scenarios to evaluate the system more thoroughlyevaluate the system more thoroughly

Improved speech Improved speech recognition interfacerecognition interface

Speech recognition technology has Speech recognition technology has improved significantly over the last yearimproved significantly over the last year–– Allowed for development of a much improved Allowed for development of a much improved

interfaceinterface–– Based on continuous speech recognitionBased on continuous speech recognition

More commands have been addedMore commands have been addedPossible options have also been expandedPossible options have also been expandedReliability is such that the user can be freed Reliability is such that the user can be freed from almost all hand interaction except for from almost all hand interaction except for flight and waypoint selection and placementflight and waypoint selection and placement

AcknowledgementsAcknowledgements

The worker:The worker:–– Marcus LangeMarcus Lange

The management:The management:–– Anders Anders YnnermanYnnerman–– Vu Duong (Eurocontrol ERC)Vu Duong (Eurocontrol ERC)