Combatting survey fatigue: uncovering the student learning experience using the
-
Upload
tuende-varga-atkins -
Category
Education
-
view
758 -
download
0
description
Transcript of Combatting survey fatigue: uncovering the student learning experience using the
![Page 1: Combatting survey fatigue: uncovering the student learning experience using the](https://reader036.fdocuments.net/reader036/viewer/2022081413/5462e26eaf79599e2c8b4d5d/html5/thumbnails/1.jpg)
ID 164. Combatting survey fatigue: uncovering the student learning experience using the
Nominal Group Technique
Funded by ELESIG
Tunde Varga-Atkins, eLearning Unit Jaye McIsaac, Educational Development
University of Liverpool11-13 Sep 2012, ALT-C
![Page 2: Combatting survey fatigue: uncovering the student learning experience using the](https://reader036.fdocuments.net/reader036/viewer/2022081413/5462e26eaf79599e2c8b4d5d/html5/thumbnails/2.jpg)
Outcomes
•To consider the use of Nominal Group Technique as a student evaluation strategy •To question the use (and concept) of technology in relation to an evaluation process
![Page 3: Combatting survey fatigue: uncovering the student learning experience using the](https://reader036.fdocuments.net/reader036/viewer/2022081413/5462e26eaf79599e2c8b4d5d/html5/thumbnails/3.jpg)
HE context: surveys
![Page 4: Combatting survey fatigue: uncovering the student learning experience using the](https://reader036.fdocuments.net/reader036/viewer/2022081413/5462e26eaf79599e2c8b4d5d/html5/thumbnails/4.jpg)
One solution: face-to-face Nominal Group Technique sessions
![Page 5: Combatting survey fatigue: uncovering the student learning experience using the](https://reader036.fdocuments.net/reader036/viewer/2022081413/5462e26eaf79599e2c8b4d5d/html5/thumbnails/5.jpg)
What is theNominal Group Technique?
• Structured group activity • One given topic• Facilitates group
decision-making• Immediate results• Reducing researcher/participant
bias- Delbecq & Van de Ven (1971)
![Page 6: Combatting survey fatigue: uncovering the student learning experience using the](https://reader036.fdocuments.net/reader036/viewer/2022081413/5462e26eaf79599e2c8b4d5d/html5/thumbnails/6.jpg)
Nominal Group Technique Stages
1) Individual responses 2) Clarification and consolidation
3) Ranking responses
‘Nominal’ Group = only in name group, most activities are individual
Example question: ‘What to keep doing (feedback)?’
![Page 7: Combatting survey fatigue: uncovering the student learning experience using the](https://reader036.fdocuments.net/reader036/viewer/2022081413/5462e26eaf79599e2c8b4d5d/html5/thumbnails/7.jpg)
What to keep doing? (feedback)Stage 1: individual responses
Annotated ,personalised feedback.
Model answers
Giving model answers for essay titles
Audio feedback for everything!
Audio feedback giving extensive comments
Drafts that must be submitted and marked prior final work
1 2 3
45 6
![Page 8: Combatting survey fatigue: uncovering the student learning experience using the](https://reader036.fdocuments.net/reader036/viewer/2022081413/5462e26eaf79599e2c8b4d5d/html5/thumbnails/8.jpg)
What to keep doing? (feedback)Stage2 : clarification and
consolidation
Model answers
Giving model answers for essay titles
Audio feedback for everything!
Audio feedback giving extensive comments
Drafts that must be submitted and marked prior final work
35
24
Annotated ,personalised feedback.
6
1
Audio feedback
Model answers
Personalised feedback
![Page 9: Combatting survey fatigue: uncovering the student learning experience using the](https://reader036.fdocuments.net/reader036/viewer/2022081413/5462e26eaf79599e2c8b4d5d/html5/thumbnails/9.jpg)
What to keep doing? (feedback)Stage3 : ranking responses
Item no. Item description
5 points 1 Annotated, personalised feedback4 points 2 Giving model answers3 points 4 Audio feedback
2 points 7 etc.1 point 10 etc.
![Page 10: Combatting survey fatigue: uncovering the student learning experience using the](https://reader036.fdocuments.net/reader036/viewer/2022081413/5462e26eaf79599e2c8b4d5d/html5/thumbnails/10.jpg)
The top five qualities of good feedback
Timing, quick
Detailed, on how to improve Applicabl
e to future work Personal
Consistent bw markers
1
2
3
4
5
![Page 11: Combatting survey fatigue: uncovering the student learning experience using the](https://reader036.fdocuments.net/reader036/viewer/2022081413/5462e26eaf79599e2c8b4d5d/html5/thumbnails/11.jpg)
is a useful student evaluation technique
Conclusion: The Nominal Group Technique
![Page 12: Combatting survey fatigue: uncovering the student learning experience using the](https://reader036.fdocuments.net/reader036/viewer/2022081413/5462e26eaf79599e2c8b4d5d/html5/thumbnails/12.jpg)
… (minor) problems of efficiency
Credit by Nathan T. Baker
![Page 13: Combatting survey fatigue: uncovering the student learning experience using the](https://reader036.fdocuments.net/reader036/viewer/2022081413/5462e26eaf79599e2c8b4d5d/html5/thumbnails/13.jpg)
Can technology improve (time & resource)
efficiencyof the NGT process?
Presumption!
![Page 14: Combatting survey fatigue: uncovering the student learning experience using the](https://reader036.fdocuments.net/reader036/viewer/2022081413/5462e26eaf79599e2c8b4d5d/html5/thumbnails/14.jpg)
No single technology could
replicateall the 3 stages of
NGT.Technologies we tried: ID275, Thu 9am
![Page 15: Combatting survey fatigue: uncovering the student learning experience using the](https://reader036.fdocuments.net/reader036/viewer/2022081413/5462e26eaf79599e2c8b4d5d/html5/thumbnails/15.jpg)
and…
![Page 16: Combatting survey fatigue: uncovering the student learning experience using the](https://reader036.fdocuments.net/reader036/viewer/2022081413/5462e26eaf79599e2c8b4d5d/html5/thumbnails/16.jpg)
Our assumption:
Technology can improve (time)
efficiency of the NGT process.
![Page 17: Combatting survey fatigue: uncovering the student learning experience using the](https://reader036.fdocuments.net/reader036/viewer/2022081413/5462e26eaf79599e2c8b4d5d/html5/thumbnails/17.jpg)
revisited…
learning (evaluation)
technology
+
= efficiency ?
Problem of lack of theorising technology (Oliver 2012)
![Page 18: Combatting survey fatigue: uncovering the student learning experience using the](https://reader036.fdocuments.net/reader036/viewer/2022081413/5462e26eaf79599e2c8b4d5d/html5/thumbnails/18.jpg)
Paper
• ubiquitous• manipulable• overlay• multi-items• legible• visible (Luff et al 2007)
• text entry• legibility• visibility• scoring• manipulable• multiitems
Digital
![Page 19: Combatting survey fatigue: uncovering the student learning experience using the](https://reader036.fdocuments.net/reader036/viewer/2022081413/5462e26eaf79599e2c8b4d5d/html5/thumbnails/19.jpg)
Conclusion: can technology enhance efficiency of the process?
• ubiquitous• manipulabl
e• overlay• multi-items• text entry• legibility• visibility
Image by Paul Duvall
![Page 21: Combatting survey fatigue: uncovering the student learning experience using the](https://reader036.fdocuments.net/reader036/viewer/2022081413/5462e26eaf79599e2c8b4d5d/html5/thumbnails/21.jpg)
References • Delbecq, A., Van de Ven, Andrew, & Gustafson, D. (1975).
Group techniques for program planning : a guide to nominal group and Delphi processes. Glenview Ill.: Scott Foresman.
• Luff et al (2007) Augmented Paper: Developing relationships between digital content and paper
• Oliver, M. (2012) Learning with technology as a coordinated sociomaterial practice: digital literacies as a site of praxiological study
• Varga-Atkins, McIsaac et al (2011) Using the nominal group technique with clickers to research student experiences of e-learning: a project report [http://slidesha.re/xQlBCg ]
• Varga-Atkins, McIsaac et al (2011) The Nominal Group Technique – a practical guide for facilitators [http://slidesha.re/AmYOgv]
![Page 22: Combatting survey fatigue: uncovering the student learning experience using the](https://reader036.fdocuments.net/reader036/viewer/2022081413/5462e26eaf79599e2c8b4d5d/html5/thumbnails/22.jpg)
Guide to NGT & Project report
See References
On slideshare.net , search for Nominal Group Technique