Joske Houtkamp100510

Usability of GeoVisualisationOutline

1. Usability. Usability evaluation.

2. Usability engineering. User centered design. Requirements and task analysis.

3. Benefits of 3D visualizations (and pitfalls).

4. Trends and developments : UX and affect.

5. Conclusions.

Usability

• Usability means making products and systems easier to use, and matching them more closely to user needs and requirements.

• International standard, ISO 9241-11:

The extent to which a product can be used by specified users to achieve specified goals with effectiveness, efficiency and satisfaction in a specified context of use.

www.usabilitynet.org

Usability

• Effectiveness – can users complete tasks, achieve goals with the product, i.e. do what

they want to do? • Efficiency

– how much effort do users require to do this? (Often measured in time) • Satisfaction

– what do users think about the products ease of use?

• The users - who is using the product? – highly trained and experienced users, or novices?

• Their goals - what are the users trying to do with the product – does it support what they want to do with it?

• The usage situation (or 'context of use') – where and how is the product being used?

www.usabilitynet.org

Usability goals

Focus on performance-based facts

Effective to use

Efficient to use

Safe to use

Have good utility

Easy to learn

Easy to remember how to use

(effectiveness)

(efficiency)

(safety; protect from making serious mistakes, recover)

(utility; functionality for required tasks)

(learnability)

(memorability)

(Preece e.a, Interaction Design, 2002)

Example: Nederland Later (Netherlands Environmental Assessment Agency (PBL))

• Maps show land use transitions, based on the simulation of land use development (Land Use Scanner )

• trends in spatial development, and estimates of the effectiveness of spatial policy.

• maps for 2000, 2010, 2020 and 2040 • text separate

• information for discussions on the impact of land use changes and their environmental, ecological and spatial effects.

GESO application (Google Earth visualization of Sustainable Outlook)

Interactive application using the Google Earth viewer

• viewers can navigate over the country and zoom to high level of detail, which facilitates recognition of the area

• switch layers on/off

• colours for current land use

• future land use types are represented by coloured squares (cells)

Usability evaluation

Approaches • Usability testing• Field studies• Analytical evaluation

Methods and techniques• Quantitative or qualitative• Formative or summative • Laboratory or field• Users or experts

Characteristics of approaches

expertnaturalisticappliedType

problemsdescriptionsmeasures & errors

Feed back

problemsqualitativequantitativeData

prototypeearlyprototypeWhen

anywherenaturalcontrolledLocation

not involvednaturaldo taskUsers

AnalyticalField studies

Usability testing

[Interaction Design]

Evaluation approaches and methods

xModeling

xTesting

xxAsking experts

xxAsking users

xxObserving

AnalyticalField studies

Usability testing

Method

[Interaction Design]

Usability testing

Usually lab experiments

• Performance metrics

• Issues based metrics

• Self-report measures

• Behavioral and physiological metrics

T. Tullis, B. Albert. Measuring the user experience, 2008

Performance metrics

Best way to evaluate effectiveness and efficiency.Calculated based on specific user behaviors, using scenarios or tasks

• Task succes (can user complete task, binary or levels)• Time-on-task (how quickly can user perform task)• Errors (number of mistakes made during task)• Efficiency (amount of effort, cognitive/physical, to complete

task/number of steps required to complete a task/combination task success and time)

• Learnability (change of efficiency metric over time)

Issues based metrics

Usability issues (important for design recommendations)Anything that – prevents task completion– creates confusion– produces an errorPerforming the wrong action, missing information, misinterpreting

information, not understanding navigation, etc etc.

• Real issues? • How severe?• How frequent?

User groups perform tasks in lab

Self-report measures

Ask users about their perception of the application and their interaction with it

Semantic differential scales Beautiful o o o o o o o Ugly

Likert scales“The information was easy to find”

Strongly disagree – disagree - neither agree nor disagree – agree - strongly agree

Open-ended questions(Which 5 elements did you like the least/most? Reasons for assessments)

Standard questionnaires: SUS (System usability scale), QUIS (user interface satisfaction), USE (Usefulness, Satisfaction and ease of Use)

Behavioral and physiological metrics

• Observation of (non)verbal behavior

• Facial expressions (video, electromyogram sensors)

• Eye-tracking (analysis of fixations to create a heat map)

• Skin conductance, heart rate (arousal)

• Pressure on mouse, on seat

Laboratory experiments

• assess the effect of independent variable(s) on dependent variable(s)

• controlled conditions• selection of tasks, participants• experimenters’ influence• high precision

Which type of visualization leads to better performance?

Experiments and validity

– Conclusion validity – Internal validity– Construct validity

– External validity – can we generalize the study results to other people/places/times?

– Ecological validity- degree to which the experimental situation reflects the type of environment in which the results will be applied (McGrath: realism)

(Sh. Carpendale, Evaluating Information Visualizations 2008)

Field studies and experiments

Field study: • on location, observer does not affect process• provides rich description (not always relevant data)

Field experiment:• on location, but control (specific task, measuring tools)• interpretation of results easier

Generalizability, precision, realism

Inspection method: heuristic evaluation

• Experts inspect a system or design using a set of heuristics or guidelines.

• Developed Jakob Nielsen in the early 1990s.

• Traditional set is based on heuristics distilled from an empirical analysis of 249 usability problems.

• Empirical evidence suggests that on average 5 evaluators identify 75-80% of usability problems

• Heuristics are developed for mobile devices, wearables, virtual worlds, games, and for information visualizations (Zuk et al 2006).

Nielsens Heuristics

Visibility of system status

Match between system and the real world

User control and freedom

Consistency and standards

Help users recognize, diagnose and recover from errors

Error prevention

Recognition rather than recall

Flexibility and efficiency of use

Aesthetic and minimalist design

Help and documentation

(Nielsen 2001)

Hierarchical grouping

Heuristics must be tailored to visualization aspects

Examples of perceptual and cognitive heuristics

• Color perception varies with size of colored item • Local contrast affects color & gray perception • Consider people with color blindness • Quantitative assessment requires position

or size variation • Consider Gestalt Laws • Provide multiple levels of detail • Integrate text wherever relevant

Examples of information seeking heuristics

• Overview first • Zoom and filter• Details on demand • Relate • Extract • History • Expose uncertainty• Formulate cause & effect

Other inspection methods

• Cognitive walkthrough, stepping through a pre-planned scenario noting potential problems.

• Expert is told the assumptions about user population, context of use, task details, and a scenario..

• Guided by 3 questions:

– Will the correct action be sufficiently evident to the user?

– Will the user notice that the correct action is available?

– Will the user associate and interpret the response from the action correctly?

Outline

1. Usability. Usability evaluation.

2. Usability engineering. User centered design. Requirements and task analysis.

3. Benefits of 3D visualizations (and pitfalls).

4. Trends and developments : UX and affect.

5. Conclusions.

Usability Engineering/User centered design

Methods and techniques in design and development that focus on usability of a product.

• Usability evaluation in all phases of the development cycle, anditerative.– conceptual model– early prototypes of a new system – later, more complete prototypes.

• User centered.• Some of the same techniques are used in evaluation as for

establishing requirements, differently.• Different evaluation approaches and methods are often combined in

one study.

• NB: Constraints in budget and time.

Visualisation design

Applying a user-centered approach to interactive visualisation design,

Wassink et al 2008

Visualisation design phases (iterative)

1. Early envisioning phaseAnalysis of current situation (users, environments, tasks) - profiles and requirements

2. Global specification phaseProposal of solutions, present to users and other stakeholders

3. Detailed specification phaseProposal of solutions, present to users and other stakeholders

Activities:1. Analysis2. Design – scenarios, later prototypes3. Evaluation of prototypes; visual representation and interaction

styles

Requirements

• Statement about an intended product that specifies what it should do or how it should perform (Preece et al, 2001)

• Requirements describe the formal specifications required to implement the system

Goals of requirements analysis:• Understand about users, tasks, context• Produce a stable set of requirements

• getting requirements right is crucial• the stage where failure occurs most commonly• mistakes in a final product are expensive• try to understand underlying needs• do not decide for the user, but check with the users

Requirements

• Functional requirements – What should the application do?

• What do users want to do with the application?• How do users perform tasks now, and with the new application?

• Non-functional requirements– Data requirements (type and properties of the data manipulated

in the system)– User requirements (user profiles and needs)– Usability requirements (usability goals, plus targets)– Environment, context of use

Data gathering techniques

• Questionnaires (many users, difficult to design)• Interviews (exploring, time consuming)• Existing documentation (trustworthy?)• Observation (creates understanding, time consuming)• Participation (ethnography)• Focus groups

• Use different methods, involve all stakeholder groups• Use template for requirement description (e.g. Volere template)

SurveyExample: screenshot study for GIS (Haklay &Zafiri 2008)

Goals: • How do GIS users organise and customise the interface?• study common users in daily usage

Users were asked to send a screenshot of their entire screen when working on routine tasks

Questionnaire to provide additional information• Sent to GIS mailing lists

Analysis: • proportion of interface assigned to map-other parts of interface (e.g.

toolbars)• User experience• Screen resolution

Results: simple technique to understand how GIS is used in situ

User profile and requirements

• Psychological characteristics: cognitive style, motivation

• Knowledge and experience (novices, experts?)

• Physical : colour blind

• Task related: frequency of use

Task descriptions

• Scenarios (Carrol, 1995, 2000)

― an informal narrative story, simple, ‘natural’, personal, not generalisable

― describe tasks and system interaction from user perspective

― details of technology not necessary

• Use cases

• Essential use cases

Task analysis

• Task descriptions are often used to envision new systems or devices

• Task analysis is used mainly to investigate an existing situation• It is important not to focus on superficial activities

What are people trying to achieve? Why are they trying to achieve it?How are they going about it?

• Many techniques, the most popular is Hierarchical Task Analysis (HTA)

Hierarchical Task Analysis

• Involves breaking a task down into subtasks, then sub-sub-tasks and so on. These are grouped as plans which specify how the tasks might be performed in practice

• HTA focuses on physical and observable actions, and includes looking at actions not related to software or an interaction device

• Start with a user goal which is examined and the main tasks for achieving it are identified

• Tasks are sub-divided into sub-tasks

• Task analysis techniques such as HTA help to investigate existing systems and practices

Example Hierarchical Task Analysis (graphical)

Prepare advice on location of power plant

Determine regulations

Determine optional areas

Check for best match

Report on results321 4

0

Locate areas size > …

Check vicinity of..

Check availability of…

Acceptable? Note location2.1 2.2 2.3 2.4 2.5

Prototyping: low fidelity – high fidelity

Basic functionality,

screen layout, workflow

Prototyping

More or less interactive representations of the design ideas (Wizard-of-Oz)

• used for early evaluations• easy to understand• supports communication by team members• allows alternatives• no distraction of interface design details (colours, shape

of buttons etc.)

Outline

1. Usability. Usability evaluation.

2. Usability engineering. User centered design. Requirements and task analysis.

3. Benefits of 3D visualizations (and pitfalls).

4. Trends and developments : UX and affect.

5. Conclusions.

3D visualizations

Increasing demand for 3D, “realistic”,visualizations, with high level of detail and animation.

• familiarity with new technology (games)• 3D, realistic visualizations are aesthetically pleasing• expected benefits of realism

– general expectations that realistic depictions minimize interpretive effort

– real world feels complete, accurate, easy (available instantly and constantly)

3D better than 2D?

2D and 3D displays support different classes of spatial tasks2D:• tasks requiring precise relative position, such as metric judgements

of distances, sizes, and angles• poor for shape-understanding tasks3D:

• tasks requiring shape understanding, such as line-of-sight judgements, gross scene layout, and object identification,

• poor for relative-position tasks.

• Hegarty et al: for many tasks (visual search, memorization etc) 2D icons are better than 3D-icons

• “Display should highlight task-relevant information, and this process of highlighting inevitably entails paring down reality.”

Why?

3D integrates all three dimensions into a single view and provide depth cues;

• depths compress faster than widths in these displays (unsuited to relative-position tasks)

2D displays separate the three dimensions of space into multiple planar views with no depth cues

• displays do not differentially compress depth and width (good for relative-position tasks)

St. John et al (2001)

Preference for realism

Hegarty et al (2008) Decoupling of Intuitions and Performance in the Use of Complex Visual Displays

• extraneous realism slowed response time and lead to more eye fixations on both task-relevant and task-irrelevant regions of the displays. • some participants persisted in favoring these realistic displays over non-realistic maps.

Preference for 3D visualizations: Naïve Realism and Naïve Cartography

• Users prefer realistic, complex and high-fidelity displays, even when their performance is lower (extra information is not task relevant, and distracting)

• Users have more confidence in data presented in realistic displays

• Appreciation of the 3D visualization may transfer to the content of the data

• User preferences, even those of domain experts, are not a good indication of display effectiveness; testing required.

(Smallman &St.John 2005, Hegarty et al 2009)

3D visualizations

If the data themselves are 3D, the third dimension communicates important information

3D useful for:• visualizing volumes, and sightlines (instead of making mental model

combining 2D visualizations)– showing inaccuracies, missing information, misunderstanding

• communication• navigating through areas

Realistic texturing, illumination:• may facilitate feeling of ‘presence’ in a location• may allow affective appraisal of an area

Outline

1. Usability. Usability evaluation.

2. Usability engineering. User centered design. Requirements and task analysis.

3. Benefits of 3D visualizations (and pitfalls).

4. Trends and developments : UX and affect.

5. Conclusions.

Importance of affect in interaction

• Affective aspects are concerned with how interactive systems make people respond in emotional ways

• Well-designed interfaces can elicit good feelings in users

• Expressive interfaces can provide reassuring feedback

• Badly designed interfaces make people angry and frustrated

• Models of affect provide a way of conceptualizing emotional and pleasurable aspects of interaction design

User experience goals

• satisfying • aesthetically pleasing• enjoyable • engaging• supportive of creativity• pleasurable • rewarding• exciting • fun• entertaining • provocative• helpful • surprising• motivating • enhancing sociability• emotionally fulfilling • challenging

• boring • annoying• frustrating

Quality Duality

Hassenzahl et al. (2001):Judgment of applications quality is based on ergonomic as

well as affective or hedonic qualities.

Ergonomic qualities: traditional usabilityHedonic qualities (pleasure, enjoyment, fun) seem to have

little to do with the goals of the user

• Hedonic qualities may interfere with performance• Users sometimes appreciate ergonomic, and sometimes

certain hedonic qualities

User experience/quality in use

Bentley, Johnston, Baggo 2005

Attractive things work better

Donald Norman (2002)“.. any pleasure, derivable from the appearance or functioning of the

tool increases positive affect, broadening the creativity and increasing the tolerance for minor difficulties and blockages. ..The changes in processing style released by positive affect aid in creative problem solving that is apt to overcome both difficulties encountered in the activity and those created by the interface design.

In other words, when we feel good, we overlook design faults.

“Tools that are meant to support serious, concentrated effort, in which the task is well specified and the approach relatively well understood, are best served by designs that emphasize function and minimize irrelevancies. “

Here the normal tensions of the situation are beneficial. The design should not get in the way; it must be carefully tailored for the task.

User experienceTraditional focus of evaluation of

geospatial systems : usability with emphasis on completing

application tasks and goals.

• ignores the nonfunctional aspects that complete a user’s experience.

Hodza 2009

User experience (UX) is alternative view

• encompasses functional (e.g. usability, ergonomics, etc.) and nonfunctional (e.g. aesthetics, emotions, pleasure, cognitive stimulation, etc.) dimensions

• was adopted to evaluate an innovative Experiential GIS (EGIS).

Aesthetics and perceived usability

Tractinsky e.a. 2000

Experiments to assess relation between perception of aesthetical value and usability of an application

Pre-experimental measures indicate strong correlations between system's perceived aesthetics and perceived usability.The degree of system's aesthetics affected the post-use perceptions of both aesthetics and usability.The degree of actual usability had no such effect.

Aesthetics and usability

Results resemble those found by social psychologists regarding the effect of physical attractiveness on the valuation of other personality attributes.

Experiment: visualisation techniques compared

Representations of 17 types of land use (a subset of the original 45 classes), selected for this experiment.

3 techniques:• coloured squares• 2D icons• 3D icons (objects)

• colours are identical• icons are designed for this experiment• in questionnaire focus on affective and aesthetic effects

2D icons developed at the PBL for this experiment

• easy recognition• level of detail is low, the representation has no resemblance to the

actual appearance of the future landscape (tool supports strategic decision making on regional and national levels)

3D objects, made in Sketchup

• level of detail is low, to stress that the representation is iconic• but: the height of the 3D objects, gives an impression of the effect on

the landscape from an aerial oblique view

Methodology

Participants• 45 (12 women), were randomly divided over the 3 conditions

(between subjects design). – 18 students Human Geography and Planning– 27 were employees, mainly researchers, of the Netherlands

Environmental Assessment Agency (experience with cartography, geographical and political issues, etc)

• Mean age 32,5 years (SD=11.4)

Equipment• desktop computers and 24” widescreen monitors

Tasks

• representative of the actions of policymakers

• context of four scenarios – for instance, exploring locations for new industrial areas in a

densely populated area, and considering the effects for other types of land use

• compare maps of current and future land use

• assess differences in distribution of land use types over time, and estimating the area they cover

Questionnaires/measures

• time required to complete scenario• use of functionalities (zoom, rotate)• questions on appreciation interface and visualisation• visualisation objects were appraised separately

– recognizability– aesthetic appreciation

printed screenshots of seven environments

• assessment of the quality of the areas for living and recreation for the individual (7 point Likert scale)

• air pollution, traffic noise, spaciousness, visual attractiveness (maps do not contain data on these topics)

• added for the purpose of this experiment, to reveal effects of visualisationtechniques on the affective appraisal of an area

Example questions

Imagine you live in the location you see here

•the environment is quiet

•the quality of the air is good

•there is little traffic noise

•the area is attractive

•etc.

Answers on 7 point scales, and option: “not enoughinformation to judge this”.

Hypothesis

• 2D icons and 3D objects improve the performance of the participants (speed and correctness)

• 2D icons offer visual diversity → higher appreciation

• 3D objects are considered more novel, realistic, and attractive →

– representation with 3D objects appreciated most – transfer to the appraisal of the environment

Results

• 2D icons and 3D objects did not improve the accuracy or speed of the task performance significantly

• no significantly higher score on recognizability than the colored squares. In the application, the color differences were apparently clear enough for the number of land use types presented.

• the appreciation of the interface, and of the visualization of the environment in its entirety: similar for all conditions

– the visualisation of future land use was (almost sign.) considered more beautiful in the condition with 3D objects (F = 3.067, p = 0,057)

Results

• 3D objects for depictions of land use were appreciated most

The 3D objects ‘blend in’ better with the Google Earth visualization and are thus more aesthetically pleasing. They add expected variations in height, thereby increasing the degree of perceived realism.

Perceived quality of the environment

• Out of 50 items, 30 times the condition with 3D objects showed the highest scores.

• the differences were small: 5 times significant (p<.05), 2 timesalmost (p≤.075)

• only once the condition with 2D icons scored significantly higher • the condition with colored squares not once.

• The 3D objects led to more convergence in the answers.

All conditions contained the same information.

Conclusion of the experiment

The visualisation technique may influence the perceived quality of the visualized environments, also on aspects that can not be directly inferred from the visualisations.

Validity of this experiment?

• Will the real users (policy makers) show the same response?• Will the effect wear off quickly?• Is the effect stronger/more worrying than with current visualizations

(colour effects)?

Finally …

• Develop with the users (User centered design).

• Ask the users (but test their performance).

• Affect is important.

• Perform field studies with real users: context and domain knowledge are relevant.