Usability EvaluationLon Barfield (26 slides)

Evaluation of anything

... that is designed for external consumption, needs to be evaluated by external agents.Can be a front end, film, pepsi challenge…

or a piece of writing.Is the work fit for purpose?Should be a no-brainer, but...

Reasons not to evaluate

It is a hassleYou lay your work open to external opinionAlterations will mean more work

Some people think it is better just to assume it’s great and stick your head in the sand...

Ways of evaluating

Rules/guidelines/patterns (‘heuristics’)Can be difficult to applyUse them in initial design

Expert opinionWhere do you get an unbiased expert from?

Real usersBest but hard work

Real users - usability testing

Some usersA thing for them to useThey have to actually use itA test; questions, and results

Some users

One subjectTwo subjectsGroup

Representative users

A thing to use

Finished systemCompetitors systemPrototype/simulation

Prototypes/simulations

Story boardsPaper basedScreen based (Photoshop, Flash)Single track/walk throughWizard of Oz

Paper based

Photocopoier, spreadsheet, tipex, scissors, bluetacBy hand or printouts from layout programEarly and fast = incredible savings in effort‘Paper psychology’

Good can be small and fast

Big and slow can be bad. “Our site needs redesigning its rubbish, get 10 people into the usability lab and conduct in depth analyses of whats wrong with it…”

Here’s an idea; hand sketch it, small group, ‘where would you click for more information?’.

They have to use it!

Not about just asking a flatmate ‘is it user friendly?’ They will say ‘Yeah, what’s on TV?’.

‘Do you want to stay my friend, and does this look like I’ve put some work into it?’

Use it… Task oriented

Don’t ask:Is it easy to use? Is it quick to use ?Do ask:How long does it take the user get an average ticket if they already know how to use the machine? When new users use it for the first time can they all get the ticket they require?

Quantitive

Observations or questions‘How many seconds to click it?’‘Does the site have events information?’Can do manipulations with the dataVey focussed and might be measuring the wrong thing

Qualitative

Interviews, chatty or open questionsMore difficultData is less maleableInteresting

Questions

Questions have to be:PlannedUsefulNeutral

What to ask, how to ask it

What to ask - Life on Mars

‘Well, what do you think; is it easy and fast to use?’Mars probe. ‘Well, what do you think; is there life on Mars?’Break the big quesitons down into the detailed quesitons that we can actually get answers to.

Life on Mars...

The National Research Council panel nicknamed the "weird life" committee. The group worries that scientists may be too Earth-centric when looking for extraterrestrial life. The problem for scientists is that "you only find what you're looking for," said Penn State University geosciences professor Katherine Freeman

How to ask it?

Can be very trickyThe Elderberry Experiments 60’s (more later)

Neutral questioning

Societal and personal pressuresYour best friend likes your design - its the Bradley effect (Tom Bradley 1982 mayor)‘Do you mind if I smoke?’ Vs ‘Would you prefer it if I didn’t smoke’Scots referendum. “Do you agree that…”

Capturing results - which method

Method has to be appropriateObservation doing a task

Metrics, talk throughQuestions in a formQuestions in an interview

Capturing results

Video them and ask them ‘did you see the help button on the page?’.... ‘Yes’Simple closed questions use fill in forms with data capture.

Open/closed questions

What do you think of the help function?Did you use the help function?I found the help function

a) interestingb) boring…

Be very careful with multiple choices

Audio and video

Transcription/analysis is VERY time consumingAudio for interviewsVideo for visual oriented investigations

Interviews

Already a large amount of good practiceClearAsset managementProcessing the recordingsPresenting results

… and competitor analysis

… is analysing your competitors to see how you stack up against them… is looking at what others are doing

it’s not about copy and pasting bits here and there to create your own product. Not the ‘perfect solution’.

Conclusions

AppropriateConsideredFast and small and many

Slides on slideshare.net/lonbarfield