Subjective questionnaires

Click here to load reader

  • date post

    05-Dec-2014
  • Category

    Education

  • view

    176
  • download

    0

Embed Size (px)

description

 

Transcript of Subjective questionnaires

  • 1. Leonardo da Vinci Partnerships Project GUI USABILITY AND ACCESSIBILITY: EXCHANGING KNOWLEDGE AND EXPERIENCES Usability Testing Subjective Questionnaires Cristina Cachero This project has been funded with support from the European Commission under the Lifelong Learning Programme

2. Leonardo da Vinci Partnerships Project GUI USABILITY AND ACCESSIBILITY: EXCHANGING KNOWLEDGE AND EXPERIENCES Inquiry method: you ask your users about what they do, but you do not observe it directly. There are several proposals that have shown enough internal and external validity, and that can be applied at different times during the performance test After all the tasks have been finished (test-level questionnaires) Before/after each task is performed (task-level questionnaires) Also, questionnaires may include open questions that can be codified and analyzed. This project has been funded with support from the European Commission under the Lifelong Learning Programme 2 Subjective Questionnaires 3. Leonardo da Vinci Partnerships Project GUI USABILITY AND ACCESSIBILITY: EXCHANGING KNOWLEDGE AND EXPERIENCES Test-level (post-test) questionnaires (1/2): CSUQ: Computer System Usability Questionnaire [Lewis 1995]: 19 sentences, all positive. They measure four dimensions: system utility, information quality, interface quality and general satisfaction. It was devised to be administered off- line PSSUQ: Post-Study System Usability Questionnaire: variation of CSUQ to be administered in person QUIS: Questionnaire for User Interface Satisfaction. 27 sentences divided in five groups: general reaction, screen, terminology/system info, learning, system capabilities. EUCS : End-user computing satisfaction [Abdinnour-Helm 2006]. 12 items representing five dimensions: content, accuracy, format, ease of use and timeliness. All the items are combined into a global EUCS measure. 5-point scale (Almost never almost always) USE: Usefulness, Satisfaction and Ease of Use [Lund 2001]. 30 items divided in four categories: utility, satisfaction, ease of use and ease of learning. For each one, seven-point scale items..This project has been funded with support from the European Commission under the Lifelong Learning Programme 3 Subjective Questionnaires 4. Leonardo da Vinci Partnerships Project GUI USABILITY AND ACCESSIBILITY: EXCHANGING KNOWLEDGE AND EXPERIENCES Test-level (post-test) questionnaires (2/2): SUS: Systems Usability Scale [Brooke 1996]: 10 questions, 5 formulated in positive and 5 formulated in negative. It gives a global measure of the site usability Net promoter Scores (NPS) : A single score regarding the fidelity of the user SUMI [Kirakowski 1996] Questionnaires that rank sites (based on proprietary DB) SUPR-Q: UX questionnaire . Includes usability (4 items), credibility (trust, value & comfort, 5 items), loyalty (2 items, one of which is the Net Promoter Score) and Appearance (2 items). Five-point scale from strongly disagree to strongly agree except for Item 10 (net promoter question) WAMMI, ACSI, Opinion Lab Proprietary (beware of reliability and validity!!) This project has been funded with support from the European Commission under the Lifelong Learning Programme 4 Subjective Questionnaires Source: ISO 13407 5. Leonardo da Vinci Partnerships Project GUI USABILITY AND ACCESSIBILITY: EXCHANGING KNOWLEDGE AND EXPERIENCES This project has been funded with support from the European Commission under the Lifelong Learning Programme 5 Post-test SQ: EUCS [Abdinnour-Helm 2006]. 6. Leonardo da Vinci Partnerships Project GUI USABILITY AND ACCESSIBILITY: EXCHANGING KNOWLEDGE AND EXPERIENCES This project has been funded with support from the European Commission under the Lifelong Learning Programme 6 Post-test SQ: USE [Lund 2001] http://hcibib.org/perlman/question.cgi?form=USE 7. Leonardo da Vinci Partnerships Project GUI USABILITY AND ACCESSIBILITY: EXCHANGING KNOWLEDGE AND EXPERIENCES SUS (Systems Usability Scale) [Brooke 1996] 10 questions, 5 formulated in positive and 5 formulated in negative. It gives a global measure of the site usability (it does not distinguish among different components). It is the most reliable [Tullis and Stetson, 2004] This project has been funded with support from the European Commission under the Lifelong Learning Programme 7 Post-test SQ: SUS [Brooke 1996] The mean SUS value is 66%, the 25 percentile is 57% and the 75 percentile is 77%. This means that we should get a SUS greater than 80% (taking into account confidence intervals) to be reasonably sure of the global satisfaction of our users. 8. Leonardo da Vinci Partnerships Project GUI USABILITY AND ACCESSIBILITY: EXCHANGING KNOWLEDGE AND EXPERIENCES Activity: Read the original SUS paper, published by Brooke in 1996. You have the paper available in the Moodle platform. This project has been funded with support from the European Commission under the Lifelong Learning Programme 8 Post-test SQ: SUS 9. Leonardo da Vinci Partnerships Project GUI USABILITY AND ACCESSIBILITY: EXCHANGING KNOWLEDGE AND EXPERIENCES This project has been funded with support from the European Commission under the Lifelong Learning Programme 9 Post-test SQ: SUS: Exercise 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5 1. I think that I would like to use this system frequently 2. I found the system unnecessarily complex 3. I thought the system was easy to use 4. I think that I would need the support of a technical person to be able to use this system 5. I found the various functions in this system were well integrated 6. I thought there was too much inconsistency in this system 7. I would imagine that most people would learn to use this system very quickly 8. I found the system very cumbersome to use 9. I felt very confident using the system 10. I needed to learn a lot of things before I could get going with this system Strongly Strongly Disagree Agree 10. Leonardo da Vinci Partnerships Project GUI USABILITY AND ACCESSIBILITY: EXCHANGING KNOWLEDGE AND EXPERIENCES This project has been funded with support from the European Commission under the Lifelong Learning Programme 10 Post-test SQ: SUS: Score? 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5 1. I think that I would like to use this system frequently 2. I found the system unnecessarily complex 3. I thought the system was easy to use 4. I think that I would need the support of a technical person to be able to use this system 5. I found the various functions in this system were well integrated 6. I thought there was too much inconsistency in this system 7. I would imagine that most people would learn to use this system very quickly 8. I found the system very cumbersome to use 9. I felt very confident using the system 10. I needed to learn a lot of things before I could get going with this system Strongly Strongly Disagree Agree 11. Leonardo da Vinci Partnerships Project GUI USABILITY AND ACCESSIBILITY: EXCHANGING KNOWLEDGE AND EXPERIENCES This project has been funded with support from the European Commission under the Lifelong Learning Programme 11 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5 1. I think that I would like to use this system frequently 2. I found the system unnecessarily complex 3. I thought the system was easy to use 4. I think that I would need the support of a technical person to be able to use this system 5. I found the various functions in this system were well integrated 6. I thought there was too much inconsistency in this system 7. I would imagine that most people would learn to use this system very quickly 8. I found the system very cumbersome to use 9. I felt very confident using the system 10. I needed to learn a lot of things before I could get going with this system Strongly Strongly Disagree Agree Post-test SQ: SUS. Steps Reverse items 2,4,6,8,10 Sum positions for each item (0 to 4) Multiply by 2.5 4 1 1 4 1 2 1 1 4 3 TOTAL SCORE: 22 SUS SCORE: 22*2.5=55 12. Leonardo da Vinci Partnerships Project GUI USABILITY AND ACCESSIBILITY: EXCHANGING KNOWLEDGE AND EXPERIENCES This project has been funded with support from the European Commission under the Lifelong Learning Programme 12 Post-test SQ: SUS. Threshold value Which should be our SUS objective? 13. Leonardo da Vinci Partnerships Project GUI USABILITY AND ACCESSIBILITY: EXCHANGING KNOWLEDGE AND EXPERIENCES This project has been funded with support from the European Commission under the Lifelong Learning Programme 13 Post-test SQ: SUS. Adaptation The SUS vocabulary can be adapted to the idyosyncrasy of the particular system Example: OHIM: system->web site 14. Leonardo da Vinci Partnerships Project GUI USABILITY AND ACCESSIBILITY: EXCHANGING KNOWLEDGE AND EXPERIENCES This project has been funded with support from the European Commission under the Lifelong Learning Programme 14 Post-test SQ: SUS. Calculator Calculator: http://www.measuringux.com/SUS-scores.xls 15. Leonardo da Vinci Partnerships Project GUI USABILITY AND ACCESSIBILITY: EXCHANGING KNOWLEDGE AND EXPERIENCES Net Promoter Scores: measure obtained through a single question (loyalty of the user to the app): How likely is it that youll recommend this product to a friend or colleague? (0..10) Three segments: Promoters: Responses from 9 to 10 Passives: Responses from 7 to 8 Detractors: Responses from 0 to 6 Promoter score (-100..+100): %Promoters-%Detractors This project has been funded with support from the European Commission under the Lifelong Learning Programme 15 Post-test SQ: Net Promoter Score 16. Leonardo da Vinci Partnerships Project GUI USABILITY AND ACCESSIBILITY: EXCHANGING KNOWLEDGE AND EXPERIENCES Questionnaires that provide rankings to compare your results against the results from other similar apps SUPR-Q (Sauro) WA