Use-inspired basic research: Marrying humans & technology

Frank DursoGeorgia Tech

BASIC APPLIED

Science and application are bound

together like fruit and the tree

that bears it.

Without theory,

practice is but routine born of

habit

If I find 10,000 ways something

won’t work, I haven’t

failed…every wrong

attempt discarded is

a step forward

The opposite of a profound

truth may well be another

profound truth

Backoff man, I’m a

scientist

Peterson?

Pasteur in his quadrant• Basic: tartaric vs. paratartaric acid

– all organic crystals rotate light• Applied: Contamination in fermentation

– Crystals rotate light fermentation was organic (Germ theory)

• Basic: Germ theory experiments– Swan necks & beef broth

• Applied: beer, wine, silkworms, pasteurization, antiseptics

• Basic (Analogy): Fermentation::contagious disease• Applied: Vaccinations

Wright brothers in the quadrant

• Applied: Glider tests• Basic: Experimental data (lifting tables) of the

time were incorrect• Basic: Experiments

– Built a wind tunnel to use for aircraft design• Applied: The Flyer at Kitty Hawk

TECHNOLOGY CHANGES

The life of Besse CooperThe rise of the machines

Happy Birthday Besse

Model T Ford - 1908

Buick - 2011

1908 Model T vs. 2011 Buick

Model-T Buick• Weight (lbs) 1,200 3,671• Horsepower 20 220• Max speed (mph) 30 120• Gas mileage (mpg) 25 23• Cost (2008 $) 20,000 30,000

ENIAC - 1945

PDP-1 Computer - 1960

Apple’s Ipad - 2011

ENIAC vs. PDP-1 vs. Ipad(Nos. approx.)

ENIAC PDP-1 IpadDate 1945 1960 2011Weight 27 tons 1,200 lbs 1.3 lbsSpace (sq ft) 680 30 < 1Mem capacity --- 10 KB 64 GBSpeed (add/sec) 5K 100K 1GhzCost (2011 $) 6,000,000 875,000 700Cal/sec for $1k (2011 $) 0.8 115 1.4G

Putting the rate of change in information technology in perspective

“An ordinary notebook PC can run a database that is more powerful and almost 100 times as large as that of a major bank of the 1980s”

If the cost of an automobile had changed at the same rate as the cost of computing over 100 years, one would be able to buy about 10 million 2011 Buicks for about 3 cents.

If the speed of an automobile had changed at the same rate as the speed of computers over 100 years, an automobile would have attained the speed of light around 1965.

THE HUMAN-TECHNICAL SYSTEM: AN IRRESISTIBLE ATTRACTION?

Technology is attractive Promise of faster, better, cheaper; makes life

easier; makes us safer Technology impacts everything Technology centric: Because we can

Have cold feet?

• Resistance to new technology– Luddites– Samurai– Bicycle face

• What is being replaced?– Tradeoffs– Side effects– Evolution

Marrying humans and technology

WHY COGNITION?

The solution to every major societal problem of the next 50 years will require an understanding of how people think

The jobs are cognitive

The people are cognitive

The problems are cognitive

• Situation awareness and hazard detection• Information overload in air traffic control• Low altitude military combat• Automation and out of the loop• Medication errors• Managing chronic disease• Training health care workers• Global warming

The solutions are cognitive . . .

Tower Simulator

Situation dimension principle

Ground Control

Intentionality, 78

Controller Memory, 7

Protagonist, 286

Space, 180

Time, 25

Unclassified, 11Local Control

Intentionality, 219

Controller Memory, 3

Motion, 21

Protagonist, 455

Space, 326

Unclassified, 16

Time, 25

Environment, 3

Dimensions of situation understanding

Integrated display

Reverse engineering

• Chase & Simon

Some displays increase information complexity, some do not

Mean Number of Glances by Condition

158.67178.67

243.17 253.50

0

50

100150

200250

300

350

standard tma cpdlc tmacpldc

Condition

Mea

n Nu

mbe

r of G

lanc

es

Practice reduces information complexity

0

2

4

6

8

10

12

TRIAL 1 TRIAL 2

Mea

n nu

mbe

r of g

lanc

es

STATICDYNAMIC

Threat & Error Strategery

• Identify threat• What strategy would you use?• What cues suggest that strategy?• Participants

– Children’s intensive care nurses– Airline pilots [automation]– Air traffic controllers– Locomotive engineers

PICU Nurses

AUGMENTED COGNITION

What is the state of the operator?Modeling the state of the operatorMonitoring the state of the operator

Auditory P300 differential attentionEEG , heartrate variability workloadHeart rate arousalPupil diameter cognitive loadEyelid droop fatiguePosture imminent actionSaliva stressFacial expression emotionfNIR retrieval v. encoding

MB’s S-car

AugCog and loss of SA

• EMG• But how can we make sure they’re confused

Watching a parade from the 40th floor

• The view was breathtaking. From the window one could see the crowd below. Everything looked extremely small from such a distance, but the colorful costumes could still be seen. Everyone seemed to be moving in one direction in an orderly fashion and there seemed to be little children as well as adults. The landing was gentle and luckily the atmosphere was such that no special suits had to be worn. At first there was a great deal of activity. Later, when the speeches started, the crowd quieted down. The man with the television camera took many shots of the setting and the crowd. Everyone was very friendly and seemed glad when the music started.

Figure 4. Z-score transformed electromyograms for the experimental passage, classified by participant response and EMG activity. The x-axis of each graph is time in seconds and the y-axis is standard deviations.

Participant Self-Report

Confused Not Confused

Electromyographic Activity

Increase in amplitude

No increase in amplitude

Right Corrugator supercilii

Left Corrugator supercili

Right Depressor anguli oris

N = 5

N = 4

N = 11

N = 4

HOW TO PLAY IN PASTEUR’S QUADRANT

Leave the labRealize where you are and why you’re there

Leaving “the lab”

• I was a control freak

The broader context

• I thought cognitive psychology was all there was– Environment– Socio-political

The QWERTY story

In which quadrant are you?Chance favors the prepared

mind

Hell, there are no rules here, we’re

trying to accomplish something

Never express yourself more

clearly than you are able

to think

No job is too big, no fee is

too big

Thanks

Others in Pasteur’s Quadrant

• Jim Staszewski mine sweeper• Frank Drews anesthesia• Penny Sanderson sonification• The Habers—low altitude military combat• Many more

Thanks

Help with abduction• Looked for a rich literature to help me explain how operators in dynamic

environments understand their situations• Reading comprehension

– Strengths• Detailed processing structures underlying comprehension in dynamic environments• Plethora of clever methodologies that could, in principle, be brought into dynamic realms• Measurement techniques and discussion of strengths and weaknesses• Context to classify different approaches to SA (e.g., cognitive vs. cognitive engineering)

– Limits• Dynamic• Input• Reason for understanding is different• Strategies

• Information needs & search• Information complexity • Detecting confusion