ASSESSING DESIGN - Stanford HCI...
Transcript of ASSESSING DESIGN - Stanford HCI...
![Page 1: ASSESSING DESIGN - Stanford HCI grouphci.stanford.edu/publications/2012/ScalingStudioCritique-UWMSR/...HCI Online CS 147 5 weeks. 10 weeks. 2 ... • 124 countries • all ages Wednesday,](https://reader030.fdocuments.net/reader030/viewer/2022020303/5b2d53617f8b9ad76e8bc6ef/html5/thumbnails/1.jpg)
ASSESSING DESIGN
Scott Klemmer
Wednesday, July 25, 12
![Page 2: ASSESSING DESIGN - Stanford HCI grouphci.stanford.edu/publications/2012/ScalingStudioCritique-UWMSR/...HCI Online CS 147 5 weeks. 10 weeks. 2 ... • 124 countries • all ages Wednesday,](https://reader030.fdocuments.net/reader030/viewer/2022020303/5b2d53617f8b9ad76e8bc6ef/html5/thumbnails/2.jpg)
Wednesday, July 25, 12
![Page 3: ASSESSING DESIGN - Stanford HCI grouphci.stanford.edu/publications/2012/ScalingStudioCritique-UWMSR/...HCI Online CS 147 5 weeks. 10 weeks. 2 ... • 124 countries • all ages Wednesday,](https://reader030.fdocuments.net/reader030/viewer/2022020303/5b2d53617f8b9ad76e8bc6ef/html5/thumbnails/3.jpg)
Wednesday, July 25, 12
![Page 4: ASSESSING DESIGN - Stanford HCI grouphci.stanford.edu/publications/2012/ScalingStudioCritique-UWMSR/...HCI Online CS 147 5 weeks. 10 weeks. 2 ... • 124 countries • all ages Wednesday,](https://reader030.fdocuments.net/reader030/viewer/2022020303/5b2d53617f8b9ad76e8bc6ef/html5/thumbnails/4.jpg)
=
Wednesday, July 25, 12
![Page 5: ASSESSING DESIGN - Stanford HCI grouphci.stanford.edu/publications/2012/ScalingStudioCritique-UWMSR/...HCI Online CS 147 5 weeks. 10 weeks. 2 ... • 124 countries • all ages Wednesday,](https://reader030.fdocuments.net/reader030/viewer/2022020303/5b2d53617f8b9ad76e8bc6ef/html5/thumbnails/5.jpg)
Wednesday, July 25, 12
![Page 6: ASSESSING DESIGN - Stanford HCI grouphci.stanford.edu/publications/2012/ScalingStudioCritique-UWMSR/...HCI Online CS 147 5 weeks. 10 weeks. 2 ... • 124 countries • all ages Wednesday,](https://reader030.fdocuments.net/reader030/viewer/2022020303/5b2d53617f8b9ad76e8bc6ef/html5/thumbnails/6.jpg)
Wednesday, July 25, 12
![Page 7: ASSESSING DESIGN - Stanford HCI grouphci.stanford.edu/publications/2012/ScalingStudioCritique-UWMSR/...HCI Online CS 147 5 weeks. 10 weeks. 2 ... • 124 countries • all ages Wednesday,](https://reader030.fdocuments.net/reader030/viewer/2022020303/5b2d53617f8b9ad76e8bc6ef/html5/thumbnails/7.jpg)
How can we make grading ‘fairer’?
Wednesday, July 25, 12
![Page 8: ASSESSING DESIGN - Stanford HCI grouphci.stanford.edu/publications/2012/ScalingStudioCritique-UWMSR/...HCI Online CS 147 5 weeks. 10 weeks. 2 ... • 124 countries • all ages Wednesday,](https://reader030.fdocuments.net/reader030/viewer/2022020303/5b2d53617f8b9ad76e8bc6ef/html5/thumbnails/8.jpg)
Wednesday, July 25, 12
![Page 9: ASSESSING DESIGN - Stanford HCI grouphci.stanford.edu/publications/2012/ScalingStudioCritique-UWMSR/...HCI Online CS 147 5 weeks. 10 weeks. 2 ... • 124 countries • all ages Wednesday,](https://reader030.fdocuments.net/reader030/viewer/2022020303/5b2d53617f8b9ad76e8bc6ef/html5/thumbnails/9.jpg)
Wednesday, July 25, 12
![Page 10: ASSESSING DESIGN - Stanford HCI grouphci.stanford.edu/publications/2012/ScalingStudioCritique-UWMSR/...HCI Online CS 147 5 weeks. 10 weeks. 2 ... • 124 countries • all ages Wednesday,](https://reader030.fdocuments.net/reader030/viewer/2022020303/5b2d53617f8b9ad76e8bc6ef/html5/thumbnails/10.jpg)
problem-findingas well as
problem solving
Wednesday, July 25, 12
![Page 11: ASSESSING DESIGN - Stanford HCI grouphci.stanford.edu/publications/2012/ScalingStudioCritique-UWMSR/...HCI Online CS 147 5 weeks. 10 weeks. 2 ... • 124 countries • all ages Wednesday,](https://reader030.fdocuments.net/reader030/viewer/2022020303/5b2d53617f8b9ad76e8bc6ef/html5/thumbnails/11.jpg)
Grading Rubric
http://cs147.stanford.edu
Wednesday, July 25, 12
![Page 12: ASSESSING DESIGN - Stanford HCI grouphci.stanford.edu/publications/2012/ScalingStudioCritique-UWMSR/...HCI Online CS 147 5 weeks. 10 weeks. 2 ... • 124 countries • all ages Wednesday,](https://reader030.fdocuments.net/reader030/viewer/2022020303/5b2d53617f8b9ad76e8bc6ef/html5/thumbnails/12.jpg)
cs147: Weekly Ritual• Assignments due each Friday 8am• Participate in studio• Reflect on & self-assess your work
Wednesday, July 25, 12
![Page 13: ASSESSING DESIGN - Stanford HCI grouphci.stanford.edu/publications/2012/ScalingStudioCritique-UWMSR/...HCI Online CS 147 5 weeks. 10 weeks. 2 ... • 124 countries • all ages Wednesday,](https://reader030.fdocuments.net/reader030/viewer/2022020303/5b2d53617f8b9ad76e8bc6ef/html5/thumbnails/13.jpg)
Courseware: Submit, Share
Wednesday, July 25, 12
![Page 14: ASSESSING DESIGN - Stanford HCI grouphci.stanford.edu/publications/2012/ScalingStudioCritique-UWMSR/...HCI Online CS 147 5 weeks. 10 weeks. 2 ... • 124 countries • all ages Wednesday,](https://reader030.fdocuments.net/reader030/viewer/2022020303/5b2d53617f8b9ad76e8bc6ef/html5/thumbnails/14.jpg)
Providing “Bumpers”
Wednesday, July 25, 12
![Page 15: ASSESSING DESIGN - Stanford HCI grouphci.stanford.edu/publications/2012/ScalingStudioCritique-UWMSR/...HCI Online CS 147 5 weeks. 10 weeks. 2 ... • 124 countries • all ages Wednesday,](https://reader030.fdocuments.net/reader030/viewer/2022020303/5b2d53617f8b9ad76e8bc6ef/html5/thumbnails/15.jpg)
a game-theoretic analysis
Wednesday, July 25, 12
![Page 16: ASSESSING DESIGN - Stanford HCI grouphci.stanford.edu/publications/2012/ScalingStudioCritique-UWMSR/...HCI Online CS 147 5 weeks. 10 weeks. 2 ... • 124 countries • all ages Wednesday,](https://reader030.fdocuments.net/reader030/viewer/2022020303/5b2d53617f8b9ad76e8bc6ef/html5/thumbnails/16.jpg)
results
Wednesday, July 25, 12
![Page 17: ASSESSING DESIGN - Stanford HCI grouphci.stanford.edu/publications/2012/ScalingStudioCritique-UWMSR/...HCI Online CS 147 5 weeks. 10 weeks. 2 ... • 124 countries • all ages Wednesday,](https://reader030.fdocuments.net/reader030/viewer/2022020303/5b2d53617f8b9ad76e8bc6ef/html5/thumbnails/17.jpg)
high performers low performers
Wednesday, July 25, 12
![Page 18: ASSESSING DESIGN - Stanford HCI grouphci.stanford.edu/publications/2012/ScalingStudioCritique-UWMSR/...HCI Online CS 147 5 weeks. 10 weeks. 2 ... • 124 countries • all ages Wednesday,](https://reader030.fdocuments.net/reader030/viewer/2022020303/5b2d53617f8b9ad76e8bc6ef/html5/thumbnails/18.jpg)
Other universities leveraging these materials• Berkeley, CMU, U Mass, Harvard,
Maryland, Michigan, MIT, UNC, Olin, Pitt, Utah, Virginia Tech, Yale, UMD, Hawaii
• American university of Beirut, FAST National University, Islamabad, Sydney, Tehran Polytechnic, NTU Singapore, National University of Singapore, American University in Cairo, Open University in the Netherlands
Wednesday, July 25, 12
![Page 19: ASSESSING DESIGN - Stanford HCI grouphci.stanford.edu/publications/2012/ScalingStudioCritique-UWMSR/...HCI Online CS 147 5 weeks. 10 weeks. 2 ... • 124 countries • all ages Wednesday,](https://reader030.fdocuments.net/reader030/viewer/2022020303/5b2d53617f8b9ad76e8bc6ef/html5/thumbnails/19.jpg)
How can we scale design assessment?
Wednesday, July 25, 12
![Page 20: ASSESSING DESIGN - Stanford HCI grouphci.stanford.edu/publications/2012/ScalingStudioCritique-UWMSR/...HCI Online CS 147 5 weeks. 10 weeks. 2 ... • 124 countries • all ages Wednesday,](https://reader030.fdocuments.net/reader030/viewer/2022020303/5b2d53617f8b9ad76e8bc6ef/html5/thumbnails/20.jpg)
The limitations of multiple choice
Wednesday, July 25, 12
![Page 21: ASSESSING DESIGN - Stanford HCI grouphci.stanford.edu/publications/2012/ScalingStudioCritique-UWMSR/...HCI Online CS 147 5 weeks. 10 weeks. 2 ... • 124 countries • all ages Wednesday,](https://reader030.fdocuments.net/reader030/viewer/2022020303/5b2d53617f8b9ad76e8bc6ef/html5/thumbnails/21.jpg)
Calibrated Peer Assessment
Wednesday, July 25, 12
![Page 22: ASSESSING DESIGN - Stanford HCI grouphci.stanford.edu/publications/2012/ScalingStudioCritique-UWMSR/...HCI Online CS 147 5 weeks. 10 weeks. 2 ... • 124 countries • all ages Wednesday,](https://reader030.fdocuments.net/reader030/viewer/2022020303/5b2d53617f8b9ad76e8bc6ef/html5/thumbnails/22.jpg)
Wednesday, July 25, 12
![Page 23: ASSESSING DESIGN - Stanford HCI grouphci.stanford.edu/publications/2012/ScalingStudioCritique-UWMSR/...HCI Online CS 147 5 weeks. 10 weeks. 2 ... • 124 countries • all ages Wednesday,](https://reader030.fdocuments.net/reader030/viewer/2022020303/5b2d53617f8b9ad76e8bc6ef/html5/thumbnails/23.jpg)
Topics Covered• Needfinding & observation• Rapid prototyping• Interface principles• Visual design• Evaluating interfaces
Wednesday, July 25, 12
![Page 24: ASSESSING DESIGN - Stanford HCI grouphci.stanford.edu/publications/2012/ScalingStudioCritique-UWMSR/...HCI Online CS 147 5 weeks. 10 weeks. 2 ... • 124 countries • all ages Wednesday,](https://reader030.fdocuments.net/reader030/viewer/2022020303/5b2d53617f8b9ad76e8bc6ef/html5/thumbnails/24.jpg)
Video Lectures
Wednesday, July 25, 12
![Page 25: ASSESSING DESIGN - Stanford HCI grouphci.stanford.edu/publications/2012/ScalingStudioCritique-UWMSR/...HCI Online CS 147 5 weeks. 10 weeks. 2 ... • 124 countries • all ages Wednesday,](https://reader030.fdocuments.net/reader030/viewer/2022020303/5b2d53617f8b9ad76e8bc6ef/html5/thumbnails/25.jpg)
Wednesday, July 25, 12
![Page 26: ASSESSING DESIGN - Stanford HCI grouphci.stanford.edu/publications/2012/ScalingStudioCritique-UWMSR/...HCI Online CS 147 5 weeks. 10 weeks. 2 ... • 124 countries • all ages Wednesday,](https://reader030.fdocuments.net/reader030/viewer/2022020303/5b2d53617f8b9ad76e8bc6ef/html5/thumbnails/26.jpg)
In-video Quizzes
Wednesday, July 25, 12
![Page 27: ASSESSING DESIGN - Stanford HCI grouphci.stanford.edu/publications/2012/ScalingStudioCritique-UWMSR/...HCI Online CS 147 5 weeks. 10 weeks. 2 ... • 124 countries • all ages Wednesday,](https://reader030.fdocuments.net/reader030/viewer/2022020303/5b2d53617f8b9ad76e8bc6ef/html5/thumbnails/27.jpg)
Weekly Quizzes
Wednesday, July 25, 12
![Page 28: ASSESSING DESIGN - Stanford HCI grouphci.stanford.edu/publications/2012/ScalingStudioCritique-UWMSR/...HCI Online CS 147 5 weeks. 10 weeks. 2 ... • 124 countries • all ages Wednesday,](https://reader030.fdocuments.net/reader030/viewer/2022020303/5b2d53617f8b9ad76e8bc6ef/html5/thumbnails/28.jpg)
Forum
Wednesday, July 25, 12
![Page 29: ASSESSING DESIGN - Stanford HCI grouphci.stanford.edu/publications/2012/ScalingStudioCritique-UWMSR/...HCI Online CS 147 5 weeks. 10 weeks. 2 ... • 124 countries • all ages Wednesday,](https://reader030.fdocuments.net/reader030/viewer/2022020303/5b2d53617f8b9ad76e8bc6ef/html5/thumbnails/29.jpg)
Assignments
Wednesday, July 25, 12
![Page 30: ASSESSING DESIGN - Stanford HCI grouphci.stanford.edu/publications/2012/ScalingStudioCritique-UWMSR/...HCI Online CS 147 5 weeks. 10 weeks. 2 ... • 124 countries • all ages Wednesday,](https://reader030.fdocuments.net/reader030/viewer/2022020303/5b2d53617f8b9ad76e8bc6ef/html5/thumbnails/30.jpg)
Rubrics
Wednesday, July 25, 12
![Page 31: ASSESSING DESIGN - Stanford HCI grouphci.stanford.edu/publications/2012/ScalingStudioCritique-UWMSR/...HCI Online CS 147 5 weeks. 10 weeks. 2 ... • 124 countries • all ages Wednesday,](https://reader030.fdocuments.net/reader030/viewer/2022020303/5b2d53617f8b9ad76e8bc6ef/html5/thumbnails/31.jpg)
Evaluation
Wednesday, July 25, 12
![Page 32: ASSESSING DESIGN - Stanford HCI grouphci.stanford.edu/publications/2012/ScalingStudioCritique-UWMSR/...HCI Online CS 147 5 weeks. 10 weeks. 2 ... • 124 countries • all ages Wednesday,](https://reader030.fdocuments.net/reader030/viewer/2022020303/5b2d53617f8b9ad76e8bc6ef/html5/thumbnails/32.jpg)
From crowds helping others...
Wednesday, July 25, 12
![Page 33: ASSESSING DESIGN - Stanford HCI grouphci.stanford.edu/publications/2012/ScalingStudioCritique-UWMSR/...HCI Online CS 147 5 weeks. 10 weeks. 2 ... • 124 countries • all ages Wednesday,](https://reader030.fdocuments.net/reader030/viewer/2022020303/5b2d53617f8b9ad76e8bc6ef/html5/thumbnails/33.jpg)
... to helping each other• Fosters empathy and equality• Scales naturally• Requires serious community buy-in.
And that turned out to be a benefit.
Wednesday, July 25, 12
![Page 34: ASSESSING DESIGN - Stanford HCI grouphci.stanford.edu/publications/2012/ScalingStudioCritique-UWMSR/...HCI Online CS 147 5 weeks. 10 weeks. 2 ... • 124 countries • all ages Wednesday,](https://reader030.fdocuments.net/reader030/viewer/2022020303/5b2d53617f8b9ad76e8bc6ef/html5/thumbnails/34.jpg)
Cheating?
Wednesday, July 25, 12
![Page 35: ASSESSING DESIGN - Stanford HCI grouphci.stanford.edu/publications/2012/ScalingStudioCritique-UWMSR/...HCI Online CS 147 5 weeks. 10 weeks. 2 ... • 124 countries • all ages Wednesday,](https://reader030.fdocuments.net/reader030/viewer/2022020303/5b2d53617f8b9ad76e8bc6ef/html5/thumbnails/35.jpg)
HCI Online CS 147
5 weeks. 10 weeks.
2 hours of online lectures per week, broken up into 8-12 minute segments.
2 hours of in-person lectures per week, broken up into 1-hour class periods.
Weekly, individual assignments. Weekly, group assignments.
1 quiz per week with mini-quizzes embedded in lecture videos. 3 quizzes in total, taken in class.
Final project: complete design cycle to create a website.
Final project: complete design cycle to create mobile web app.
Peer- and self- assessments. Staff- and self- assessments.
Statement of Accomplishment (“Apprentice” or “Studio”). Stanford University Credit.
No prerequisites. Enrollment at Stanford.Wednesday, July 25, 12
![Page 36: ASSESSING DESIGN - Stanford HCI grouphci.stanford.edu/publications/2012/ScalingStudioCritique-UWMSR/...HCI Online CS 147 5 weeks. 10 weeks. 2 ... • 124 countries • all ages Wednesday,](https://reader030.fdocuments.net/reader030/viewer/2022020303/5b2d53617f8b9ad76e8bc6ef/html5/thumbnails/36.jpg)
hci-online: Weekly Ritual• Assignments due most tuesdays at
11pm PDT• Chinmay/Robi graded 8-10 examples• Calibrated Peer Assessment
• Each student graded five assignments...• ...and then their own• Grade: median of the five
Wednesday, July 25, 12
![Page 37: ASSESSING DESIGN - Stanford HCI grouphci.stanford.edu/publications/2012/ScalingStudioCritique-UWMSR/...HCI Online CS 147 5 weeks. 10 weeks. 2 ... • 124 countries • all ages Wednesday,](https://reader030.fdocuments.net/reader030/viewer/2022020303/5b2d53617f8b9ad76e8bc6ef/html5/thumbnails/37.jpg)
who enrolled?
Wednesday, July 25, 12
![Page 38: ASSESSING DESIGN - Stanford HCI grouphci.stanford.edu/publications/2012/ScalingStudioCritique-UWMSR/...HCI Online CS 147 5 weeks. 10 weeks. 2 ... • 124 countries • all ages Wednesday,](https://reader030.fdocuments.net/reader030/viewer/2022020303/5b2d53617f8b9ad76e8bc6ef/html5/thumbnails/38.jpg)
Student numbers• 29,105 students watched video(s)• 6,853 submitted quiz(zes)• 2,470 completed an assignment
• and 791 completed all 5
• From the 10,114 who filled out the initial survey...• 124 countries• all ages
Wednesday, July 25, 12
![Page 39: ASSESSING DESIGN - Stanford HCI grouphci.stanford.edu/publications/2012/ScalingStudioCritique-UWMSR/...HCI Online CS 147 5 weeks. 10 weeks. 2 ... • 124 countries • all ages Wednesday,](https://reader030.fdocuments.net/reader030/viewer/2022020303/5b2d53617f8b9ad76e8bc6ef/html5/thumbnails/39.jpg)
Age
0
1250
2500
3750
5000
Under 13 13 - 17 18 - 24 25 - 34 35 - 44 45 - 54 55 - 64 65 +
Wednesday, July 25, 12
![Page 40: ASSESSING DESIGN - Stanford HCI grouphci.stanford.edu/publications/2012/ScalingStudioCritique-UWMSR/...HCI Online CS 147 5 weeks. 10 weeks. 2 ... • 124 countries • all ages Wednesday,](https://reader030.fdocuments.net/reader030/viewer/2022020303/5b2d53617f8b9ad76e8bc6ef/html5/thumbnails/40.jpg)
Employment
0
1250
2500
3750
5000
K-12 St
udent
Underg
raduat
e
Graduat
e Stud
ent
Recent
gradu
ate
Self-e
mployed
Working
full-tim
e
Working
part-ti
me
Unemplo
yed
Retired
Other
Wednesday, July 25, 12
![Page 41: ASSESSING DESIGN - Stanford HCI grouphci.stanford.edu/publications/2012/ScalingStudioCritique-UWMSR/...HCI Online CS 147 5 weeks. 10 weeks. 2 ... • 124 countries • all ages Wednesday,](https://reader030.fdocuments.net/reader030/viewer/2022020303/5b2d53617f8b9ad76e8bc6ef/html5/thumbnails/41.jpg)
10 self-organized meetup locations
• London, England• Palo Alto, CA• San Francisco, CA• New York City, NY• Buenos Aires, Argentina• Bangladesh• Portland, OR• Mountain View, CA• San Jose, CA• Aachen, Germany
Wednesday, July 25, 12
![Page 42: ASSESSING DESIGN - Stanford HCI grouphci.stanford.edu/publications/2012/ScalingStudioCritique-UWMSR/...HCI Online CS 147 5 weeks. 10 weeks. 2 ... • 124 countries • all ages Wednesday,](https://reader030.fdocuments.net/reader030/viewer/2022020303/5b2d53617f8b9ad76e8bc6ef/html5/thumbnails/42.jpg)
Students transcribed (some) lectures in 13 languages
• English, Spanish, Brazilian Portuguese, Russian, Bulgarian, Japanese, Korean, Slovak, Vietnamese, Chinese Simplified, Chinese Traditional, Persian, and Catalan
Wednesday, July 25, 12
![Page 43: ASSESSING DESIGN - Stanford HCI grouphci.stanford.edu/publications/2012/ScalingStudioCritique-UWMSR/...HCI Online CS 147 5 weeks. 10 weeks. 2 ... • 124 countries • all ages Wednesday,](https://reader030.fdocuments.net/reader030/viewer/2022020303/5b2d53617f8b9ad76e8bc6ef/html5/thumbnails/43.jpg)
Wednesday, July 25, 12
![Page 44: ASSESSING DESIGN - Stanford HCI grouphci.stanford.edu/publications/2012/ScalingStudioCritique-UWMSR/...HCI Online CS 147 5 weeks. 10 weeks. 2 ... • 124 countries • all ages Wednesday,](https://reader030.fdocuments.net/reader030/viewer/2022020303/5b2d53617f8b9ad76e8bc6ef/html5/thumbnails/44.jpg)
Wednesday, July 25, 12
![Page 45: ASSESSING DESIGN - Stanford HCI grouphci.stanford.edu/publications/2012/ScalingStudioCritique-UWMSR/...HCI Online CS 147 5 weeks. 10 weeks. 2 ... • 124 countries • all ages Wednesday,](https://reader030.fdocuments.net/reader030/viewer/2022020303/5b2d53617f8b9ad76e8bc6ef/html5/thumbnails/45.jpg)
what happened?
Wednesday, July 25, 12
![Page 46: ASSESSING DESIGN - Stanford HCI grouphci.stanford.edu/publications/2012/ScalingStudioCritique-UWMSR/...HCI Online CS 147 5 weeks. 10 weeks. 2 ... • 124 countries • all ages Wednesday,](https://reader030.fdocuments.net/reader030/viewer/2022020303/5b2d53617f8b9ad76e8bc6ef/html5/thumbnails/46.jpg)
Wednesday, July 25, 12
![Page 47: ASSESSING DESIGN - Stanford HCI grouphci.stanford.edu/publications/2012/ScalingStudioCritique-UWMSR/...HCI Online CS 147 5 weeks. 10 weeks. 2 ... • 124 countries • all ages Wednesday,](https://reader030.fdocuments.net/reader030/viewer/2022020303/5b2d53617f8b9ad76e8bc6ef/html5/thumbnails/47.jpg)
Assignment 1 - Needfinding
Observations of subject preparing for IT certifications
Wednesday, July 25, 12
![Page 48: ASSESSING DESIGN - Stanford HCI grouphci.stanford.edu/publications/2012/ScalingStudioCritique-UWMSR/...HCI Online CS 147 5 weeks. 10 weeks. 2 ... • 124 countries • all ages Wednesday,](https://reader030.fdocuments.net/reader030/viewer/2022020303/5b2d53617f8b9ad76e8bc6ef/html5/thumbnails/48.jpg)
Assignment 1 - Needfinding
Design inspirations for personal finance displays
Wednesday, July 25, 12
![Page 49: ASSESSING DESIGN - Stanford HCI grouphci.stanford.edu/publications/2012/ScalingStudioCritique-UWMSR/...HCI Online CS 147 5 weeks. 10 weeks. 2 ... • 124 countries • all ages Wednesday,](https://reader030.fdocuments.net/reader030/viewer/2022020303/5b2d53617f8b9ad76e8bc6ef/html5/thumbnails/49.jpg)
Assignment 2 - Prototyping
Prototype of IT certification website
Wednesday, July 25, 12
![Page 50: ASSESSING DESIGN - Stanford HCI grouphci.stanford.edu/publications/2012/ScalingStudioCritique-UWMSR/...HCI Online CS 147 5 weeks. 10 weeks. 2 ... • 124 countries • all ages Wednesday,](https://reader030.fdocuments.net/reader030/viewer/2022020303/5b2d53617f8b9ad76e8bc6ef/html5/thumbnails/50.jpg)
Assignment 2 - Prototyping
Prototype of mobile banking app
Wednesday, July 25, 12
![Page 51: ASSESSING DESIGN - Stanford HCI grouphci.stanford.edu/publications/2012/ScalingStudioCritique-UWMSR/...HCI Online CS 147 5 weeks. 10 weeks. 2 ... • 124 countries • all ages Wednesday,](https://reader030.fdocuments.net/reader030/viewer/2022020303/5b2d53617f8b9ad76e8bc6ef/html5/thumbnails/51.jpg)
Assignment 3 - Start Building
Implementation plan for IT certification website
Wednesday, July 25, 12
![Page 52: ASSESSING DESIGN - Stanford HCI grouphci.stanford.edu/publications/2012/ScalingStudioCritique-UWMSR/...HCI Online CS 147 5 weeks. 10 weeks. 2 ... • 124 countries • all ages Wednesday,](https://reader030.fdocuments.net/reader030/viewer/2022020303/5b2d53617f8b9ad76e8bc6ef/html5/thumbnails/52.jpg)
Assignment 3 - Start Building
Implementation plan for mobile banking app
Wednesday, July 25, 12
![Page 53: ASSESSING DESIGN - Stanford HCI grouphci.stanford.edu/publications/2012/ScalingStudioCritique-UWMSR/...HCI Online CS 147 5 weeks. 10 weeks. 2 ... • 124 countries • all ages Wednesday,](https://reader030.fdocuments.net/reader030/viewer/2022020303/5b2d53617f8b9ad76e8bc6ef/html5/thumbnails/53.jpg)
Assignment 4 - Ready to Test
“[My] website will help to find a certification program which fits to a person, check[s] the progress of study and encourage[s] person to get certified.”
Wednesday, July 25, 12
![Page 54: ASSESSING DESIGN - Stanford HCI grouphci.stanford.edu/publications/2012/ScalingStudioCritique-UWMSR/...HCI Online CS 147 5 weeks. 10 weeks. 2 ... • 124 countries • all ages Wednesday,](https://reader030.fdocuments.net/reader030/viewer/2022020303/5b2d53617f8b9ad76e8bc6ef/html5/thumbnails/54.jpg)
Assignment 4 - Ready to Test
TextText
“Mybank mobile users can check their bank account movements, evolution (balance changes on time) and stats (what are the main expenditures and incoming types in each account). As well as make money transfers from one account to another in a fast, easy way... everywhere.”
Wednesday, July 25, 12
![Page 55: ASSESSING DESIGN - Stanford HCI grouphci.stanford.edu/publications/2012/ScalingStudioCritique-UWMSR/...HCI Online CS 147 5 weeks. 10 weeks. 2 ... • 124 countries • all ages Wednesday,](https://reader030.fdocuments.net/reader030/viewer/2022020303/5b2d53617f8b9ad76e8bc6ef/html5/thumbnails/55.jpg)
Wednesday, July 25, 12
![Page 56: ASSESSING DESIGN - Stanford HCI grouphci.stanford.edu/publications/2012/ScalingStudioCritique-UWMSR/...HCI Online CS 147 5 weeks. 10 weeks. 2 ... • 124 countries • all ages Wednesday,](https://reader030.fdocuments.net/reader030/viewer/2022020303/5b2d53617f8b9ad76e8bc6ef/html5/thumbnails/56.jpg)
Assessments
HCI Online CS 147
Correlation between self- and staff- assessments.
Correlation between self- and peer- assessments.
% of students who received their own grade.
0.81 0.91
0.78 N/A
52% 69%
Wednesday, July 25, 12
![Page 57: ASSESSING DESIGN - Stanford HCI grouphci.stanford.edu/publications/2012/ScalingStudioCritique-UWMSR/...HCI Online CS 147 5 weeks. 10 weeks. 2 ... • 124 countries • all ages Wednesday,](https://reader030.fdocuments.net/reader030/viewer/2022020303/5b2d53617f8b9ad76e8bc6ef/html5/thumbnails/57.jpg)
• How much did you feel like you learned from assessing others' work?
• How much did you feel like you learned from assessing your own work?
Wednesday, July 25, 12
![Page 58: ASSESSING DESIGN - Stanford HCI grouphci.stanford.edu/publications/2012/ScalingStudioCritique-UWMSR/...HCI Online CS 147 5 weeks. 10 weeks. 2 ... • 124 countries • all ages Wednesday,](https://reader030.fdocuments.net/reader030/viewer/2022020303/5b2d53617f8b9ad76e8bc6ef/html5/thumbnails/58.jpg)
Wednesday, July 25, 12
![Page 59: ASSESSING DESIGN - Stanford HCI grouphci.stanford.edu/publications/2012/ScalingStudioCritique-UWMSR/...HCI Online CS 147 5 weeks. 10 weeks. 2 ... • 124 countries • all ages Wednesday,](https://reader030.fdocuments.net/reader030/viewer/2022020303/5b2d53617f8b9ad76e8bc6ef/html5/thumbnails/59.jpg)
Patriotic Grading
Wednesday, July 25, 12
![Page 60: ASSESSING DESIGN - Stanford HCI grouphci.stanford.edu/publications/2012/ScalingStudioCritique-UWMSR/...HCI Online CS 147 5 weeks. 10 weeks. 2 ... • 124 countries • all ages Wednesday,](https://reader030.fdocuments.net/reader030/viewer/2022020303/5b2d53617f8b9ad76e8bc6ef/html5/thumbnails/60.jpg)
Peer assessment• Students made it work!• Variation in effort, standards,
interpretation• Need for richer feedback...
• to acknowledge hard work• to help students improve
Wednesday, July 25, 12
![Page 61: ASSESSING DESIGN - Stanford HCI grouphci.stanford.edu/publications/2012/ScalingStudioCritique-UWMSR/...HCI Online CS 147 5 weeks. 10 weeks. 2 ... • 124 countries • all ages Wednesday,](https://reader030.fdocuments.net/reader030/viewer/2022020303/5b2d53617f8b9ad76e8bc6ef/html5/thumbnails/61.jpg)
Students praised• Lectures• Peer assessment• Design without programming
Wednesday, July 25, 12
![Page 62: ASSESSING DESIGN - Stanford HCI grouphci.stanford.edu/publications/2012/ScalingStudioCritique-UWMSR/...HCI Online CS 147 5 weeks. 10 weeks. 2 ... • 124 countries • all ages Wednesday,](https://reader030.fdocuments.net/reader030/viewer/2022020303/5b2d53617f8b9ad76e8bc6ef/html5/thumbnails/62.jpg)
Criticism• Assignments: examples & more clarity• More time between assignments• Make all project work & deadlines clear
at beginning• Platform (and usability) bugs• All the kinks of a first offering• More qualitative feedback
Wednesday, July 25, 12
![Page 63: ASSESSING DESIGN - Stanford HCI grouphci.stanford.edu/publications/2012/ScalingStudioCritique-UWMSR/...HCI Online CS 147 5 weeks. 10 weeks. 2 ... • 124 countries • all ages Wednesday,](https://reader030.fdocuments.net/reader030/viewer/2022020303/5b2d53617f8b9ad76e8bc6ef/html5/thumbnails/63.jpg)
The benefits and challenges of timeshifted lectures
Wednesday, July 25, 12
![Page 64: ASSESSING DESIGN - Stanford HCI grouphci.stanford.edu/publications/2012/ScalingStudioCritique-UWMSR/...HCI Online CS 147 5 weeks. 10 weeks. 2 ... • 124 countries • all ages Wednesday,](https://reader030.fdocuments.net/reader030/viewer/2022020303/5b2d53617f8b9ad76e8bc6ef/html5/thumbnails/64.jpg)
The benefits and challenges of timeshifted lectures
Wednesday, July 25, 12
![Page 65: ASSESSING DESIGN - Stanford HCI grouphci.stanford.edu/publications/2012/ScalingStudioCritique-UWMSR/...HCI Online CS 147 5 weeks. 10 weeks. 2 ... • 124 countries • all ages Wednesday,](https://reader030.fdocuments.net/reader030/viewer/2022020303/5b2d53617f8b9ad76e8bc6ef/html5/thumbnails/65.jpg)
Faculty collaborations: an unexpected benefit
Wednesday, July 25, 12
![Page 66: ASSESSING DESIGN - Stanford HCI grouphci.stanford.edu/publications/2012/ScalingStudioCritique-UWMSR/...HCI Online CS 147 5 weeks. 10 weeks. 2 ... • 124 countries • all ages Wednesday,](https://reader030.fdocuments.net/reader030/viewer/2022020303/5b2d53617f8b9ad76e8bc6ef/html5/thumbnails/66.jpg)
Students Took Ownership• Sharing cool interfaces, resources,
articles• Collating reading lists, creating
assignment aids• Doing really creative work• Helping other students
• heuristic evaluation feedback• answering forum questions• extra peer assessment
Wednesday, July 25, 12
![Page 67: ASSESSING DESIGN - Stanford HCI grouphci.stanford.edu/publications/2012/ScalingStudioCritique-UWMSR/...HCI Online CS 147 5 weeks. 10 weeks. 2 ... • 124 countries • all ages Wednesday,](https://reader030.fdocuments.net/reader030/viewer/2022020303/5b2d53617f8b9ad76e8bc6ef/html5/thumbnails/67.jpg)
Wednesday, July 25, 12
![Page 68: ASSESSING DESIGN - Stanford HCI grouphci.stanford.edu/publications/2012/ScalingStudioCritique-UWMSR/...HCI Online CS 147 5 weeks. 10 weeks. 2 ... • 124 countries • all ages Wednesday,](https://reader030.fdocuments.net/reader030/viewer/2022020303/5b2d53617f8b9ad76e8bc6ef/html5/thumbnails/68.jpg)
`
Wednesday, July 25, 12
![Page 69: ASSESSING DESIGN - Stanford HCI grouphci.stanford.edu/publications/2012/ScalingStudioCritique-UWMSR/...HCI Online CS 147 5 weeks. 10 weeks. 2 ... • 124 countries • all ages Wednesday,](https://reader030.fdocuments.net/reader030/viewer/2022020303/5b2d53617f8b9ad76e8bc6ef/html5/thumbnails/69.jpg)
The Medium is the Message
Wednesday, July 25, 12
![Page 70: ASSESSING DESIGN - Stanford HCI grouphci.stanford.edu/publications/2012/ScalingStudioCritique-UWMSR/...HCI Online CS 147 5 weeks. 10 weeks. 2 ... • 124 countries • all ages Wednesday,](https://reader030.fdocuments.net/reader030/viewer/2022020303/5b2d53617f8b9ad76e8bc6ef/html5/thumbnails/70.jpg)
Wednesday, July 25, 12
![Page 71: ASSESSING DESIGN - Stanford HCI grouphci.stanford.edu/publications/2012/ScalingStudioCritique-UWMSR/...HCI Online CS 147 5 weeks. 10 weeks. 2 ... • 124 countries • all ages Wednesday,](https://reader030.fdocuments.net/reader030/viewer/2022020303/5b2d53617f8b9ad76e8bc6ef/html5/thumbnails/71.jpg)
From CoimbatoreHello Scott,
This is Arun Martin, a usability analyst from Coimbatore, India. I have enrolled for your online HCI course last week and require your suggestions for the Design Brief on "Change."
I plan to share my thoughts for commuters who start early from their offices, park their two-wheelers at a parking stand, and then commute to their offices through cabs or other modes of transport. I'm looking at the following challenge: How can technology help commuters to ascertain whether there are available parking spaces during morning hours so they can spend few minutes at home/parking area to monitor their fitness levels and improve their health? Is this something that I can work on as a project?
Thanks,Arun
Wednesday, July 25, 12
![Page 72: ASSESSING DESIGN - Stanford HCI grouphci.stanford.edu/publications/2012/ScalingStudioCritique-UWMSR/...HCI Online CS 147 5 weeks. 10 weeks. 2 ... • 124 countries • all ages Wednesday,](https://reader030.fdocuments.net/reader030/viewer/2022020303/5b2d53617f8b9ad76e8bc6ef/html5/thumbnails/72.jpg)
Wednesday, July 25, 12
![Page 73: ASSESSING DESIGN - Stanford HCI grouphci.stanford.edu/publications/2012/ScalingStudioCritique-UWMSR/...HCI Online CS 147 5 weeks. 10 weeks. 2 ... • 124 countries • all ages Wednesday,](https://reader030.fdocuments.net/reader030/viewer/2022020303/5b2d53617f8b9ad76e8bc6ef/html5/thumbnails/73.jpg)
what’s next?
Wednesday, July 25, 12
![Page 74: ASSESSING DESIGN - Stanford HCI grouphci.stanford.edu/publications/2012/ScalingStudioCritique-UWMSR/...HCI Online CS 147 5 weeks. 10 weeks. 2 ... • 124 countries • all ages Wednesday,](https://reader030.fdocuments.net/reader030/viewer/2022020303/5b2d53617f8b9ad76e8bc6ef/html5/thumbnails/74.jpg)
MOOC is a terrible name
Wednesday, July 25, 12
![Page 75: ASSESSING DESIGN - Stanford HCI grouphci.stanford.edu/publications/2012/ScalingStudioCritique-UWMSR/...HCI Online CS 147 5 weeks. 10 weeks. 2 ... • 124 countries • all ages Wednesday,](https://reader030.fdocuments.net/reader030/viewer/2022020303/5b2d53617f8b9ad76e8bc6ef/html5/thumbnails/75.jpg)
Beta 2: Fall 2012
Wednesday, July 25, 12
![Page 76: ASSESSING DESIGN - Stanford HCI grouphci.stanford.edu/publications/2012/ScalingStudioCritique-UWMSR/...HCI Online CS 147 5 weeks. 10 weeks. 2 ... • 124 countries • all ages Wednesday,](https://reader030.fdocuments.net/reader030/viewer/2022020303/5b2d53617f8b9ad76e8bc6ef/html5/thumbnails/76.jpg)
Integrating Guidelines Into the Coursera ForumNisha Masharani
Wednesday, July 25, 12
![Page 77: ASSESSING DESIGN - Stanford HCI grouphci.stanford.edu/publications/2012/ScalingStudioCritique-UWMSR/...HCI Online CS 147 5 weeks. 10 weeks. 2 ... • 124 countries • all ages Wednesday,](https://reader030.fdocuments.net/reader030/viewer/2022020303/5b2d53617f8b9ad76e8bc6ef/html5/thumbnails/77.jpg)
New features: A list of guidelines relevant to browsing the forumA “Start New Thread” button
Wednesday, July 25, 12
![Page 78: ASSESSING DESIGN - Stanford HCI grouphci.stanford.edu/publications/2012/ScalingStudioCritique-UWMSR/...HCI Online CS 147 5 weeks. 10 weeks. 2 ... • 124 countries • all ages Wednesday,](https://reader030.fdocuments.net/reader030/viewer/2022020303/5b2d53617f8b9ad76e8bc6ef/html5/thumbnails/78.jpg)
Clicking on “Start New Thread” brings the user to this page. Guidelines show up above the portion of the page to which it is relevant.To post in a forum, the user clicks the name of the forum (mandatory).
Wednesday, July 25, 12
![Page 79: ASSESSING DESIGN - Stanford HCI grouphci.stanford.edu/publications/2012/ScalingStudioCritique-UWMSR/...HCI Online CS 147 5 weeks. 10 weeks. 2 ... • 124 countries • all ages Wednesday,](https://reader030.fdocuments.net/reader030/viewer/2022020303/5b2d53617f8b9ad76e8bc6ef/html5/thumbnails/79.jpg)
The user can hover over “example” to view an example of the rule in action.
Wednesday, July 25, 12
![Page 80: ASSESSING DESIGN - Stanford HCI grouphci.stanford.edu/publications/2012/ScalingStudioCritique-UWMSR/...HCI Online CS 147 5 weeks. 10 weeks. 2 ... • 124 countries • all ages Wednesday,](https://reader030.fdocuments.net/reader030/viewer/2022020303/5b2d53617f8b9ad76e8bc6ef/html5/thumbnails/80.jpg)
Why Do These Rules 1. Keep it short. Shorter posts have __ more responses
than longer posts. [Need data from Coursera]2.Use the search function. 35.2% of posts surveyed are
duplicates.3. Draft your post before you choose a subforum. 24.8% of
posts are in the wrong subforum.4.Use tags wisely. 50.5% of posts don’t have any tags. 5. Give your post a good title. 24.8% of posts have titles
that don’t correspond to the actual post.6.Be civil. 6.7% of posts are rude or inappropriate. 7. Use votes to bring attention to good posts. [Need data
from Coursera]
Wednesday, July 25, 12
![Page 81: ASSESSING DESIGN - Stanford HCI grouphci.stanford.edu/publications/2012/ScalingStudioCritique-UWMSR/...HCI Online CS 147 5 weeks. 10 weeks. 2 ... • 124 countries • all ages Wednesday,](https://reader030.fdocuments.net/reader030/viewer/2022020303/5b2d53617f8b9ad76e8bc6ef/html5/thumbnails/81.jpg)
Instructor role in re-runs?
Wednesday, July 25, 12
![Page 82: ASSESSING DESIGN - Stanford HCI grouphci.stanford.edu/publications/2012/ScalingStudioCritique-UWMSR/...HCI Online CS 147 5 weeks. 10 weeks. 2 ... • 124 countries • all ages Wednesday,](https://reader030.fdocuments.net/reader030/viewer/2022020303/5b2d53617f8b9ad76e8bc6ef/html5/thumbnails/82.jpg)
Encouraging Richer Feedback• How could this student best improve his/her user evaluation?
From among the following, copy one or more pieces of advice that would help the student. Paste your advice in the feedback box below.• Test with more users• Test in a different location/environment• Test with a more functional prototype• Create a more thorough script/plan• Reflect on findings and find underlying reasons• Test with a different alternate prototype• Other
• Example:• Test with more users: since a lot of the feedback about look and feel
wasn't consistent across all users, testing with more might help.• Test with a more functional prototype: especially for the part about
looking inside an item, where functionality is not implemented
Wednesday, July 25, 12
![Page 83: ASSESSING DESIGN - Stanford HCI grouphci.stanford.edu/publications/2012/ScalingStudioCritique-UWMSR/...HCI Online CS 147 5 weeks. 10 weeks. 2 ... • 124 countries • all ages Wednesday,](https://reader030.fdocuments.net/reader030/viewer/2022020303/5b2d53617f8b9ad76e8bc6ef/html5/thumbnails/83.jpg)
Wednesday, July 25, 12
![Page 84: ASSESSING DESIGN - Stanford HCI grouphci.stanford.edu/publications/2012/ScalingStudioCritique-UWMSR/...HCI Online CS 147 5 weeks. 10 weeks. 2 ... • 124 countries • all ages Wednesday,](https://reader030.fdocuments.net/reader030/viewer/2022020303/5b2d53617f8b9ad76e8bc6ef/html5/thumbnails/84.jpg)
Different peer assessment approaches
• Multiple languages?• Homogenous v. Heterogeneous• One large circle or many small ones• Tournaments• Norm-setting, tight/loose cultures
Wednesday, July 25, 12
![Page 85: ASSESSING DESIGN - Stanford HCI grouphci.stanford.edu/publications/2012/ScalingStudioCritique-UWMSR/...HCI Online CS 147 5 weeks. 10 weeks. 2 ... • 124 countries • all ages Wednesday,](https://reader030.fdocuments.net/reader030/viewer/2022020303/5b2d53617f8b9ad76e8bc6ef/html5/thumbnails/85.jpg)
Hard deadlines?
Wednesday, July 25, 12
![Page 86: ASSESSING DESIGN - Stanford HCI grouphci.stanford.edu/publications/2012/ScalingStudioCritique-UWMSR/...HCI Online CS 147 5 weeks. 10 weeks. 2 ... • 124 countries • all ages Wednesday,](https://reader030.fdocuments.net/reader030/viewer/2022020303/5b2d53617f8b9ad76e8bc6ef/html5/thumbnails/86.jpg)
Hard deadlines?
Wednesday, July 25, 12
![Page 87: ASSESSING DESIGN - Stanford HCI grouphci.stanford.edu/publications/2012/ScalingStudioCritique-UWMSR/...HCI Online CS 147 5 weeks. 10 weeks. 2 ... • 124 countries • all ages Wednesday,](https://reader030.fdocuments.net/reader030/viewer/2022020303/5b2d53617f8b9ad76e8bc6ef/html5/thumbnails/87.jpg)
Recording: production values & pedagogical style
Wednesday, July 25, 12
![Page 88: ASSESSING DESIGN - Stanford HCI grouphci.stanford.edu/publications/2012/ScalingStudioCritique-UWMSR/...HCI Online CS 147 5 weeks. 10 weeks. 2 ... • 124 countries • all ages Wednesday,](https://reader030.fdocuments.net/reader030/viewer/2022020303/5b2d53617f8b9ad76e8bc6ef/html5/thumbnails/88.jpg)
Back to bricks:(How to) flip the class?
Wednesday, July 25, 12
![Page 89: ASSESSING DESIGN - Stanford HCI grouphci.stanford.edu/publications/2012/ScalingStudioCritique-UWMSR/...HCI Online CS 147 5 weeks. 10 weeks. 2 ... • 124 countries • all ages Wednesday,](https://reader030.fdocuments.net/reader030/viewer/2022020303/5b2d53617f8b9ad76e8bc6ef/html5/thumbnails/89.jpg)
Evaluators->Coaches
Wednesday, July 25, 12
![Page 90: ASSESSING DESIGN - Stanford HCI grouphci.stanford.edu/publications/2012/ScalingStudioCritique-UWMSR/...HCI Online CS 147 5 weeks. 10 weeks. 2 ... • 124 countries • all ages Wednesday,](https://reader030.fdocuments.net/reader030/viewer/2022020303/5b2d53617f8b9ad76e8bc6ef/html5/thumbnails/90.jpg)
What is the ‘studio’ in the 21st century?
Wednesday, July 25, 12
![Page 91: ASSESSING DESIGN - Stanford HCI grouphci.stanford.edu/publications/2012/ScalingStudioCritique-UWMSR/...HCI Online CS 147 5 weeks. 10 weeks. 2 ... • 124 countries • all ages Wednesday,](https://reader030.fdocuments.net/reader030/viewer/2022020303/5b2d53617f8b9ad76e8bc6ef/html5/thumbnails/91.jpg)
Let’s build practical theory
Wednesday, July 25, 12
![Page 92: ASSESSING DESIGN - Stanford HCI grouphci.stanford.edu/publications/2012/ScalingStudioCritique-UWMSR/...HCI Online CS 147 5 weeks. 10 weeks. 2 ... • 124 countries • all ages Wednesday,](https://reader030.fdocuments.net/reader030/viewer/2022020303/5b2d53617f8b9ad76e8bc6ef/html5/thumbnails/92.jpg)
“Nothing is as practical as a good theory”
“The best way to understand something is to try and change it”
–Kurt Lewin
Wednesday, July 25, 12
![Page 93: ASSESSING DESIGN - Stanford HCI grouphci.stanford.edu/publications/2012/ScalingStudioCritique-UWMSR/...HCI Online CS 147 5 weeks. 10 weeks. 2 ... • 124 countries • all ages Wednesday,](https://reader030.fdocuments.net/reader030/viewer/2022020303/5b2d53617f8b9ad76e8bc6ef/html5/thumbnails/93.jpg)
Thanks to...• Stanford: Jane, Robi, Alex, Chinmay,
Kathryn...• Coursera: Huy, Daniel, Pang Wei,
Ngiam, Daphne, Andrew...• Colleagues who let me show their
work• Students going above and beyond: Amy...
Wednesday, July 25, 12