Download - Managed Learning Environment Project User Issues - Formative and Summative Assessment Mark Simpson How did we specify the system - specification? How did.

Managed Learning Environment Project

User Issues - Formative and Summative Assessment

MarkMark Simpson Simpson

How did we specify the system - specification?How did we decide on content- requirements?How did we verify content?Develop conventions for pages?Usability trade-offs and compromises?


What is UsabilityPerhaps it is:Learnability, Efficiency, MemorabilityAccessibility, Recoverability, Satisfaction

OrIf you use the system enough you will be able to use it

OrIf you are too stupid to work it out it isn’t my problem and why are you here.


Usability how hard is it?Everybody has an opinion

“I hate …… …… …”

“Frames / Flash / PDF are rubbish”

“we know what we want, don’t waste anymore time exploring and understanding, just get on and create it”. Steve Krug, Don’t make me think

Anything new will not satisfy X % of users, until they have used it.


Difference between screen and paper

Sun

Absorbed & Reflected constant light

Electron Gun

Direct & Emittedflickering light

What is this, in your opinion ?

RED

BLUE

What colour is this, in your opinion?


Development cycle

Changing login & password

October 2000

March 2001

September2001

December2001

February 2002

April 2002

June 2002

July 2002

August 2002

Web Site Requirements / Specification

Develop Content List

PreparingContent / Layout

Designing initial prototype

1st user testing card sort

Modifying design menus

2nd user testing4th user testingNavigation &Logic Testing

CorrectingNavigation Issues

3rd user testing

Modifying design

WritingHelp and Search

Project evaluation report

Implement uPortal


SpecificationWhat does the system do?

What are the users likeWhat do they want the system to doWhat can we do from the users want listWhat are the usability issues with current systemsCan the MLE solve these issues

Project team produced a list of what they wanted as indicative student users


User Requirements / Content How do you start

Ask yourself / project team / steering committee Ask users Research other systems or similar sites What is unique to your project – DMU it was QLS Produce a list of content areas Refine list to remove duplication / impossible content Record list with reasons


Sources of Requirements1. DMU PC ownership and use

survey 1997, 98, 99, & 00

2. Collaboration in Research Support by Academic Libraries in Leicestershire survey [CORSALL]

3. ANGEL user analysis results

4. DMU Nursing School, online course survey

5. DMU Library survey of student information needs

6. DMU Research papers on using the Internet for teaching.

7. DMU Keyskills survey of all first year students

8. Student Experience Unit feedback from students leaving the university

9. DMU Student focus group meeting

10. Usage reports from similar systems

11. User interviews with users of similar existing systems


10. User tracking data User Logs for six months from a similar system

Logon Data, route through site, user names

Course being studied, Self assessments completed, discussion involvement

Didn’t match interview or focus group reports


Results of Specification Consistent increase in PC ownership and use of online resources – 34 % in 2001

All users ask for more content than they use

Academic users want the MLE to be integrated in university and not isolated

Navigation done using mouse and back arrow

Content and feature list for student users


Version 1


Card Sorting

menus, content and priority


Dendrogram

Cluster Analysis – grouping of content


MLE Version 2


Further login trials, testing navigation

Login trials - user name and password

Simple content location

Logging in and out

Interface issues

Don’t look at too much each time


Testing login &

photographs


Login trials with HNC, BA, MA students

12 first year students HNC Housing Part-time

7 second year students BA Accounting and Finance Full-time

26 postgraduates taking MA degrees Full-time

6 postgraduates last part of MA Full-time

Between each test correct problems and develop a few new features for testing


How many times do you test it, how many subjects do you need?

As many times as will yield useful data

Between 4 and 10 subjects at a time

What are you testing ?what is the questionwhat will you do with the resultswhat resources are available

Don’t leave it hoping it will be alright


MLE Version 3


Future UsabilityTraining and Help pages

Focus groups – face to face

Online questionnaires

Feedback from modules with teaching resources online

User tracking and what people actually do


Benefits of the methods

Provide a handle on the university administration information issues

Publicise the project – making it visible

Resolve problems as they were discovered

Kept in touch with all types of users


How would we do it differently

Have an off-line non development system

Use external evaluation only in the second half of the project

When carrying out usability testing freeze the design and clearly name versions

A recorded and agreed method of tracking changes and developments in the interface.


What should you do? Set-up a steering group or panel

Think about what the site / page / system is providing and so make your site user centred, not content centred

Do testing of content and navigation regularly

Always have a usability issue to hand

Use conventions and standard layouts – they work

– what did the users tell you


Summary

6 Formative and 3 Summative methods of assessment

Ask users what they want

Test navigation regularly

Implement changes and test again

Don’t expect to please everybody – usability is opinions based on evidence


Where am I now ?