6/19/2002 Applied Physics Laboratory/ Naval Research Laboratory1 EDSS Program Review: June 2002...
-
Upload
melinda-daniel -
Category
Documents
-
view
215 -
download
0
Transcript of 6/19/2002 Applied Physics Laboratory/ Naval Research Laboratory1 EDSS Program Review: June 2002...
6/19/2002 Applied Physics Laboratory/ Naval Research Laboratory
1
EDSS Program Review: June 2002
David Jones:APL-UWJim Ballas: NRL
6/19/2002 Applied Physics Laboratory/ Naval Research Laboratory
2
Outline
David Introduction Project Overview HCI Experience Progress Report
6/19/2002 Applied Physics Laboratory/ Naval Research Laboratory
3
Outline…
Jim UCD Process Task Analysis Evaluations Future Plans
6/19/2002 Applied Physics Laboratory/ Naval Research Laboratory
4
APL-UW Team Members
Bob Miyamoto- PI David Jones Troy Tanner & Bill Kooiman
6/19/2002 Applied Physics Laboratory/ Naval Research Laboratory
5
APL-UW Background
Miyamoto’s Group Env Effects on
Sensors TDA development Training Tools
Me
6/19/2002 Applied Physics Laboratory/ Naval Research Laboratory
6
Project Overview
General Philosophy- the design model fits the user’s mental model: UCD
Perform task analyses that feed into the interface design & its evaluation
Support EDSS developers with UCD and HCI standards and guidelines
Provide iterative feedback
6/19/2002 Applied Physics Laboratory/ Naval Research Laboratory
7
Example of UCD Application: DMARS
Study the user’s information needs
Study how the user performs given tasks
Create an intuitive process
Involve users in design
6/19/2002 Applied Physics Laboratory/ Naval Research Laboratory
8
High Seas Workflow System Helps produce the High
Seas Warning Heard from the supervisors Then heard from actual
users Different stories Created a system for the
users Flexible design
6/19/2002 Applied Physics Laboratory/ Naval Research Laboratory
9
HSW cont…
Funded by DARA & SPAWAR Written in JAVA Uses Polexis’ XIS for map
functions Running operationally at SD
METOC Center
6/19/2002 Applied Physics Laboratory/ Naval Research Laboratory
10
FNC Project: EVIS Studying human-system
component of METOC support
Worked closely with users and customers
Conducted experiments Performed Cognitive Task
Analysis Gone to sea for evaluation
RULE MISSION OPERATION EVOLUTION PARAMETER MARGINAL SEVERE T00 T12 T24 T36 T48 T60 T72
1 AIR AERIAL REFUEL ALL LEVELS VISIBILITY 1600-4800 M <1600 M 2 AIR AERIAL REFUEL ALL LEVELS TURBULENCE LIGHT MODERATE 3 AIR AERIAL REFUEL ALL LEVELS ICING LIGHT MODERATE 4 AIR AERIAL REFUEL ALL LEVELS THUNDERSTORMS WITHIN 25 NM WITHIN 10 NM 5 AIR AIR BASE LAUNCH/RECOVERY VISIBILITY 1600-4800 M <1600 M 6 AIR AIR BASE LAUNCH/RECOVERY CEILING 1000-3000 FT <1000 FT 7 AIR ATTACK CLOSE AIR SUP CEILING 1000-3000 FT <1000 FT 8 AIR ATTACK CLOSE AIR SUP VISIBILITY 1600-4800 M <1600 M 9 AIR ATTACK CLOSE AIR SUP TURBULENCE LGT-MDT HEAVY 10 AIR BOMBING LOW LVL +500 AGL CEILING 0500-1000 FT <500 FT 11 AIR BOMBING LOW LVL +500 AGL VISIBILITY 4800-5600 M <4800 M 12 AIR BOMBING LOW LVL +500 AGL TURBULENCE LIGHT MODERATE 13 AIR BOMBING LOW LVL +500 AGL ICING LIGHT MODERATE 14 AIR BOMBING MID-HI LEVEL CLOUD COVER 40-60% >60% 15 AIR RECON GROUND LEVEL VISIBILITY 1000-3000 M <1000 M 16 AIR RECON HIGH LEVEL CLOUD COVER 50-60% >60% 17 AIR RECON HIGH LEVEL VISIBILITY 4800-5600 M <4800 M 18 AIR RECON LOW LEVEL CEILING 1000-3000 FT <1000 FT 19 AIR RECON UAV (DRONES) CEILING 4000-6000 FT <4000 FT 20 AIR RECON UAV (DRONES) VISIBILITY 4800-5600 M <4800 M 21 AIR RECON UAV (DRONES) ALTITUDE WIND 40-60 KT >60 KT 22 AIR RECON UAV (DRONES) PRECIPITATION 0.1-0.3 IN/HR >0.3 IN/HR 43 C4ISR HIGH ALTITUDE ISR CLOUD COVER 40-60% >60%
6/19/2002 Applied Physics Laboratory/ Naval Research Laboratory
11
Progress Report
Initial Task Analysis -Jim Gaining Domain Knowledge Training Observations- David Organizing a UCD Workshop
6/19/2002 Applied Physics Laboratory/ Naval Research Laboratory
12
Gaining Domain Knowledge
EDSS User’s Guide Draft Mission Needs Statement for
Distributed Collaborative Planning Systems for Expeditionary Forces
COMPHIBGRU THREE 041605Z OCT99 4.X Tiger Team User Input Spreadsheet Pubs: NWP 3—02.1; ATP 3 ch 6, JP 3-02
6/19/2002 Applied Physics Laboratory/ Naval Research Laboratory
13
Training on the USS Tarawa
Attended training in San Diego: Apr
Enthusiastic users
Hands-on training well received
Great audience for usability evaluation
6/19/2002 Applied Physics Laboratory/ Naval Research Laboratory
14
Example of User reactions Staff personnel were excited
about creating overlays for planning
But.. Menu headings caused
some confusion “Are Assault Plans part of
AOA Mgmt?” “When do I use the Env
DB?”
6/19/2002 Applied Physics Laboratory/ Naval Research Laboratory
15
User reactions
Navigation among the different windows was difficult at times
Some windows require expertise that all users might not have or forgot
6/19/2002 Applied Physics Laboratory/ Naval Research Laboratory
16
Quick Thoughts after Training
Most users want it to look like Windows DII UIS provides HCI guidance Some ideas
Back Arrows & Undo command Web-based and searchable user’s guide Tooltip help- On mouse of menu title Workflow wizards or web-based training
6/19/2002 Applied Physics Laboratory/ Naval Research Laboratory
17
UCD Workshop
Scheduled for 30 July 2002 At SAIC Tysons Corner office Our ideas
UCD Processes DII HCI Standards HCI Design Principles with examples HCI Evaluation
SAIC Ideas?
6/19/2002 Applied Physics Laboratory/ Naval Research Laboratory
18
Jim Ballas
UCD & HCI Task Analysis Evaluations Future Plans
6/19/2002 Applied Physics Laboratory/ Naval Research Laboratory
19
NRL-WDC
Team Members Jim Ballas
Ph.D. in Applied Experimental Psychology Derek Brock
M.S. in Computer Science, HCI emphasis Beth Kramer
M.S. in Human Factors Psychology Janet Stroup
B.A., some graduate CS coursework
6/19/2002 Applied Physics Laboratory/ Naval Research Laboratory
20
NRL-WDC: Interface Design and Evaluation Section
4 Ph.Ds on staff of 10Expertise in HCI, Human factors, Cognitive Science, Computer Science, Auditory perceptionProjects include AEGIS (with LMC), DDX (with Raytheon), NATO S&T, KSA EVIS Management6.1 to 6.3 projects
6/19/2002 Applied Physics Laboratory/ Naval Research Laboratory
21
NRL-WDC: Interface Design and Evaluation Section
HCI Research cited in Major reference documents
Handbook of Human Computer Interaction
ACM CHI Conference 2001 paper: “Demystifying
Direct Manipulation”
Wrote and Revised Operator Workstation Evaluation section for IUSW-21 at Sea Test this Sept
6/19/2002 Applied Physics Laboratory/ Naval Research Laboratory
22
User Centered Design
Following approach outlined in NATO COADE document
Additional principles: HCI as an instance of language use
6/19/2002 Applied Physics Laboratory/ Naval Research Laboratory
23
User Centered Design: COADE
6/19/2002 Applied Physics Laboratory/ Naval Research Laboratory
24
User Centered Design: COADE
6/19/2002 Applied Physics Laboratory/ Naval Research Laboratory
25
Viewing HCI as an Instance of Language Use The design and implementation of an
effective software application and its user interface is ultimately a communication problem that always involves both the designer’s meaning and the user’s understanding
The principles at work in people’s use language form a comprehensive framework for the design of human-computer interaction
6/19/2002 Applied Physics Laboratory/ Naval Research Laboratory
26
Principles of Language Use Any form of communication between people
is an instance of language use; language is used to do things together
Language use requires people to coordinate their actions and their attention (cognition); it always involves “speaker’s” meaning and “addressee’s” understanding.
Meaning and understanding require common ground
6/19/2002 Applied Physics Laboratory/ Naval Research Laboratory
27
Common Ground
Common ground is knowledge that people establish they can use with each other on the basis of shared experience
When common ground is missing, meaning and understanding breakdown
Building common ground is always a serial process - even though the resulting shared knowledge may contain gaps
6/19/2002 Applied Physics Laboratory/ Naval Research Laboratory
28
Layers in Language Use and in HCI Language use frequently involves more than one
conceptual layer of activity; telling a story, for instance, involves at least two layers:
The story teller and the listener participate as themselves in the first layer
In the second layer, the events of the story take place Similarly, HCI has two principal layers of activity:
The designer and user participate as themselves in the first layer
In the second layer, the user interacts with the computer as if it (and not the designer) were the user’s counterpart
Each layer in an instance of language use makes different demands of the user’s language use skills
6/19/2002 Applied Physics Laboratory/ Naval Research Laboratory
29
Language Use Issues in HCI In the first layer of HCI, designers, through
the software’s presentation, must help users to compensate for gaps that direct access (through menus, etc.) imposes on the process of building coherent common ground
In the second layer, wherever possible, interfaces should be designed to allow users to establish and use common ground with the interface itself as a regular part of their interaction with the computer
6/19/2002 Applied Physics Laboratory/ Naval Research Laboratory
30
Guidelines, Standards, and Relevant Literature
DII User Interface Standards MilStd2525
Research on distributive planning: Klein & Miller Work directed by NRL Cited in MCDP-5
General human factors and HCI literature
6/19/2002 Applied Physics Laboratory/ Naval Research Laboratory
31
Work to Date: Initial Task Analysis Partially Complete
Initial Task List:EDSS planning
Make Basic Decisions Create Operational Area Determine Landing Craft Complete Default Craft Parameters Table Make Navigation Decisions Design Sea Echelon Areas
Free Hand 4W Grid
Select HLZ Select Beach Center and Boat Lane Design Routes Design Display Determine Ship-to-Shore Movement
1 2 3 4 5
Make Basic Decisions Create Operational Area Determine Landing Craft
Complete Default Craft Parameters Table Make Navigation Decisions Free Hand
Goal
Amphibious Task Force objectives and plan general courses of action
Make master plan: amphibious landing plans for a variety of different geographic situations
Determine landing craft availability as basis for landing craft assignment to the ship-to-objective movement
Establish default planning speeds (speed made good) of the landing craft and the helicopters
determine the unit of measurement that will be used
used when the geography does not fit a 4W grid pattern, e.g., in confined waters
Product
Landing plan directory which can contain numerous landing plans
Table of landing craft available for each ship Table
decision to promulgate unit of measurement in the OPTASK Amphib to ensure that all staffs and ships plot the same information
Performance Requirements
Type of chart being used; degree of accuracy that can be plotted on the paper chart; degree of the accuracy required
Information Requirements Default craft parameters
types and number of landing craft available for each ship Default craft parameters table
Actions Fill in table
Decisions
Determine beachheads, landing areas, landing sites, beaches, helicopter landing zones, etc., H-Hour
Asset type (choices are surface, air, infantry platoon & company, mech infantry platoon & company, and armor platoon & company) Decision re. changes to default parameters
Nav Units-Sea Min (DDMM.MM); Nav Units-Sea Secs (DDMMSS; Land-measurement in Grid format
DESIGN SEA ECHELON AREAS
Task Name
6
6/19/2002 Applied Physics Laboratory/ Naval Research Laboratory
32
Task Analysis: other tasks Administrative/file operation tasks
Log On And Initializing Log Off Installing New EDSS Software Exporting Plans Importing Plans
GCCS-M Tasks Saving Slides Deleting Slides Exporting Slides Installing Maps And Charts Retrieving Maps Or Charts Removing Maps Or Charts Uninstalling Maps, Charts, And Imagery Line Of Sight (Los) Profile
6/19/2002 Applied Physics Laboratory/ Naval Research Laboratory
33
Future Work
Complete task analysisIncluding cognitive analysis using Critical Decision Method
Workshop:To include illustrations of design issues, e.g. illustration of lighting/ filtering effect on color images
6/19/2002 Applied Physics Laboratory/ Naval Research Laboratory
34
Future Work: Evaluation
Approach: evaluate user performance and compare to desired performance
Examples: DMARS EVIS Software Development Tools and
Processes
6/19/2002 Applied Physics Laboratory/ Naval Research Laboratory
35
DMARS Evaluation Summary Observe and compare use of three media
Paper (NAVOCEANO Mine Warfare Pilot publication) WWW (based on MWP, so called RP-WEB UC-CD (User Centered Digital METOC Acoustic
Reference Manual - DMARS) Five METOC tasks: prepare brief and answer 4
questions 12 METOC officers and enlisted personnel
NAS Patuxent River NAS North Island
Each person tested on five tasks using three media
6/19/2002 Applied Physics Laboratory/ Naval Research Laboratory
36
DMARS Evaluation: Process Measures Task timing logged with Activity Catalog Tool
a NASA sponsored tool Coding scheme distinguished following tasks
Acquire information Browse: search for topic in the METOC document by
navigating from one section to another to another (Search for and Move To)
Interpret: read information from a specific section; Assemble briefing document;
Compose/edit: generate and/or modify the document using a word processor or presentation software;
Copy/paste: copy material from the METOC document into the briefing document.
6/19/2002 Applied Physics Laboratory/ Naval Research Laboratory
37
DMARS Evaluation: Outcome measures
Accuracy On Problem 1, Subject presented brief
and experimenter graded eight items Required analysis, not just picture
Problems 2-5, Subject supplied answers.
Number of images used in briefing Preference
6/19/2002 Applied Physics Laboratory/ Naval Research Laboratory
38
DMARS Evaluation: Time to Find Information to Prepare Briefing
Significant effect of document type
F(2, 16) = 5.62, p < .01 RP-WEB> [UC-CD, PAPER]
Discussion Effect on key problem
Only on browse time No briefing preparation
differences (when time to manually prepare slides added to PAPER condition
No interpretation time differences
Magnitude RP-WEB 160s longer than UC-
CD on a task which overall takes 1260 s (13% time increase)
0
200
400
600
800
1000
Browse time (s)
UC-CD PAPER RP-WEB
6/19/2002 Applied Physics Laboratory/ Naval Research Laboratory
39
DMARS Evaluation: Accuracy
Significant effect f document type F(2,16) = 3.48 , p =.055
UC-CD > [PAPER, RP-WEB] @ p = .087
Discussion UC-CD errors on problems
4 and 5 due to omission of location selection
UC-CD errors on winds in Problem 1 due to user inexperience with new form of wind vector--only a short training period used.
0
0.2
0.4
0.6
0.8
1
UC-CD PAPER RP-WEB
6/19/2002 Applied Physics Laboratory/ Naval Research Laboratory
40
CCTV
Desk
Door
Storage (document)
IT21
Doo
r
NITESServer
NITESNT
NITESNT
NITESNT
IT21
IT21
Printer
SPA-25(Radar)
: Video Camera
CCTV
F: Forecaster
T: Technician
O: Project Observer
NITESServerRAID
Desk
: Workstation
NOWCOPIER Coffee
FILE
T
O
O
Wind
FILE
FILE
SMOOS
SMQ-11(Satellite)
: Chair
FIT21
T
T
: Port Hole
Example of At Sea Observations: USS CARL VINSON during Battlegroup training (COMPTUEX)
6/19/2002 Applied Physics Laboratory/ Naval Research Laboratory
41
At Sea Observation Methodology Office Environment
Observed forecasters workflow (not-interfering basis) Two watches a day (12 on, 12 off) Each session is about 12 hours long Two Observers for each watch; two observations per watch
Equipment Three video cameras Note taking Palm Pilot for recording the timing of the task Performance Questionnaires Interviews
6/19/2002 Applied Physics Laboratory/ Naval Research Laboratory
42
Future Work Summary
Workshop Task Analysis Design recommendations Evaluation