EPFL, spring 2010 - week 12!UX evaluation

today’s schedule

➝ Review of UX evaluation plans ➝ Test run of scripts ➝ Guest talk ➝  Individual walk throughs

overview

➝  about UX evaluations ➝ UX evaluation criteria ➝ UX evaluation methods ➝  tips and hints ➝  for more information

Attribution: parts of this presentation are based on Forum Nokias ÚX Evaluation series

why UX evaluation? ➝  UX is a critical success factor for products and services

➝  We need confirmation during design and development that we are on the right track

➝  It is challenging for the software developer to keep an objective view of the developed software ➝  It is human nature to become “blind” of your own

outcomes ➝  Thus it is important to get a second opinion of the UX,

from outside of the development team

➝  Preferably from real users trying it in a realistic environment

The designer is not the user

Success factor

Simulate real use situations

Functionality Reliability Efficiency Usability Portability Maintainability

Suitability Maturity Time behavior Operability Installability Testability

Accuracy Fault tolerance Resource utilization

Learnability Adaptability Stability

… … ... … … …

usability and product quality

➝  there are many metrics to measure software performance and quality

Code coverage

Source lines of code

Function point analysis

Number of sw bugs

Implemented functional

requirements

SW Product Quality (ISO-9126)

UX and product quality

User Experience

Expectations, motives, actions, interpretations,

…

➝  measuring UX is not straight-forward

➝  there are many intervening issues

➝  product quality has a strong impact on UX

In this definition, UX is not a part of product quality.

A SW engineering

point-of-view?

Functionality Reliability Efficiency Usability Portability Maintainability

Suitability Maturity Time behavior Operability Installability Testability

Accuracy Fault tolerance Resource utilization

Learnability Adaptability Stability

… … ... … … …

SW Product Quality (ISO-9126)

UX evaluation criteria 1/5

basic usability heuristic UX drivers derived from the UX elements

As such Issues

detailed for your product

Drivers tailored for your product

UX evaluation criteria 2/5

➝  Evaluate the design against the specified UX artefacts

UX evaluation criteria 3/5

➝  evaluate against benchmark ➝  products and experiences

UX evaluation criteria 4/5

➝  Run your tests with all the target devices and form factors

Bar Slide

Clamshell Swivel

Touch screen Touch only screen

Full hw qwerty Full sw qwerty

Resistive screen Capacitative screen

UX evaluation criteria 5/5

➝  the ultimate criterion: ➝  potential users try the product in their mobile context

Direct, spoken criteria Indirect, unspoken criteria

- originating from the context

planning the UX evaluations 1/2

➝  for any UX artifacts, you can think can and should it be evaluated, somehow

Specify needs and context of use

Define the UX concept Create UI designs

Evaluate concepts and designs

A step-by-step process would

be easy to manage.

In practice, design and

evaluation are often in a parallel

and iterative relationship.

planning the UX evaluations 2/2

Evaluate concepts and designs

➝  What artefacts do we want to evaluate ➝  Sketches, visualisations, prototypes,...

➝  What issues are we evaluating? ➝  Concept, interaction, grahical design, ...

➝  Who will do the evaluation? ➝  Experts, end users,...

My cool travel- mate concept

UX evalution: expert evaluation

➝  Usability experts evaluate a system based on common heuristics (i.e. design principles) and personal expertise ➝  Common heuristics are based on knowledge gained

through expertise and experience ➝  Knowledge about perceptional and cognitive processes

- such as the function of memory – are utilized ➝  At least two experts take part in the evaluation to

ensure reliable results ➝  At the beginning of an expert evaluation, the value of

the system to its user and buyer/provider is determined ➝  Based on these values, usability criteria are defined

➝  At the end, the findings documented and prioritized. Solutions proposals are made

➝  Also user tasks can be defined and walkthroughs conducted as part of the evaluation

UX evaluation: focus groups ➝  a group of people (4-7) that has

something in common will be invited to semi-structured group discussion session to share their views on certain topic. Lasts usually 2 hours

➝  face-to-face sessions ➝  Sometimes on-line

➝  variations of the method ➝  Exploratory ➝  Feature prioritization ➝  Competitive analysis ➝  Trend explanation

UX evaluation: usability testing ➝  the test user is asked to conduct certain tasks with the

user interface under design ➝  paper prototypes can be used ➝  users are asked to think aloud

➝  sometimes good to test the UI with pairs of users to

trigger more discussion or to test social aspects ➝  usability problems are identified and prioritized based on

the user data gathered during the usability tests

Moderator

Test user Test cases

Observer

Tested sw

Special equipment can be useful to catch the

actions on mobile devices

UX evaluation: observation ➝  the researcher does not interrupt the user

but just observes the user’s actions ➝  the user is followed usually shorter time

than when shadowing in her/his own environment, and usually only certain activities/time period are observed

➝  good for studying users that cannot be interrupted

➝  reveals task flows and possible detectable problems in them and behavioral patterns but not motivations, attitudes, values, concerns

If observation is not possible (e.g. in hospital) video

recording can be alternative. Video can be gone through

with the participant(s) afterwards

UX evaluation: make a mix

➝  an expert evaluation and a usability test can be used as complementary methods

➝  an expert evaluation require less time and preparation than a usability test

➝  expert evaluations are recommended before usability tests in order to support designing the test tasks

➝  a satisfaction questionnaire can be combined with a usability test

UX evaluation: tips and hints ➝  the UX evaluation criteria should be in-line with

the defined UX drivers, targets and requirements ➝  make sure to share your UX evaluation criteria

with the designers before they start their work ➝  do evaluations with real users

➝  expert evaluations can provide quick useful findings ➝  try to have a end user mindset

➝ ideally your evaluation methods should scale to weekly (or even daily) evaluations ➝  design & evaluation iteration loops do not have to be

big monolithic steps

➝  UX evaluation(s) should be an integral part of software development process, just like software testing