Investigating the Pedagogical Push and Technological Pull of Computer Assisted Formative Assessment
2012 Computer-Assisted Assessment
description
Transcript of 2012 Computer-Assisted Assessment
![Page 1: 2012 Computer-Assisted Assessment](https://reader034.fdocuments.net/reader034/viewer/2022051209/54803694b4af9fd3158b5e0f/html5/thumbnails/1.jpg)
CMA in Moodle:Past, Present and Future
Tim HuntLeading Technical DeveloperThe Open University
![Page 2: 2012 Computer-Assisted Assessment](https://reader034.fdocuments.net/reader034/viewer/2022051209/54803694b4af9fd3158b5e0f/html5/thumbnails/2.jpg)
Overview of this talkThis talk will focus on the ‘Present’
1 Overview
2 Questions
3 Activities
4 Conclusions
![Page 3: 2012 Computer-Assisted Assessment](https://reader034.fdocuments.net/reader034/viewer/2022051209/54803694b4af9fd3158b5e0f/html5/thumbnails/3.jpg)
What are we trying to achieve?A CMA system must address• pedagogic• administrative• technical
considerations
![Page 4: 2012 Computer-Assisted Assessment](https://reader034.fdocuments.net/reader034/viewer/2022051209/54803694b4af9fd3158b5e0f/html5/thumbnails/4.jpg)
Overview
![Page 5: 2012 Computer-Assisted Assessment](https://reader034.fdocuments.net/reader034/viewer/2022051209/54803694b4af9fd3158b5e0f/html5/thumbnails/5.jpg)
Demonstration
http://v.gd/quizexampleou
http://v.gd/quizexample
![Page 6: 2012 Computer-Assisted Assessment](https://reader034.fdocuments.net/reader034/viewer/2022051209/54803694b4af9fd3158b5e0f/html5/thumbnails/6.jpg)
Map of Moodle’s CMA tools
![Page 7: 2012 Computer-Assisted Assessment](https://reader034.fdocuments.net/reader034/viewer/2022051209/54803694b4af9fd3158b5e0f/html5/thumbnails/7.jpg)
Questions
![Page 8: 2012 Computer-Assisted Assessment](https://reader034.fdocuments.net/reader034/viewer/2022051209/54803694b4af9fd3158b5e0f/html5/thumbnails/8.jpg)
The quiz activity
![Page 9: 2012 Computer-Assisted Assessment](https://reader034.fdocuments.net/reader034/viewer/2022051209/54803694b4af9fd3158b5e0f/html5/thumbnails/9.jpg)
The question engine
![Page 10: 2012 Computer-Assisted Assessment](https://reader034.fdocuments.net/reader034/viewer/2022051209/54803694b4af9fd3158b5e0f/html5/thumbnails/10.jpg)
The question bank
![Page 11: 2012 Computer-Assisted Assessment](https://reader034.fdocuments.net/reader034/viewer/2022051209/54803694b4af9fd3158b5e0f/html5/thumbnails/11.jpg)
The quiz activity
![Page 12: 2012 Computer-Assisted Assessment](https://reader034.fdocuments.net/reader034/viewer/2022051209/54803694b4af9fd3158b5e0f/html5/thumbnails/12.jpg)
Administrative parts of Moodle
![Page 13: 2012 Computer-Assisted Assessment](https://reader034.fdocuments.net/reader034/viewer/2022051209/54803694b4af9fd3158b5e0f/html5/thumbnails/13.jpg)
Other graded activities
![Page 14: 2012 Computer-Assisted Assessment](https://reader034.fdocuments.net/reader034/viewer/2022051209/54803694b4af9fd3158b5e0f/html5/thumbnails/14.jpg)
Plugins• The various conceptual parts of Moodle are made up of -plugins-core Moodle functionality
• Plugins make it easy to add custom functionality• They break down the functionality into units that can be
combined in different ways
![Page 15: 2012 Computer-Assisted Assessment](https://reader034.fdocuments.net/reader034/viewer/2022051209/54803694b4af9fd3158b5e0f/html5/thumbnails/15.jpg)
Questions
![Page 16: 2012 Computer-Assisted Assessment](https://reader034.fdocuments.net/reader034/viewer/2022051209/54803694b4af9fd3158b5e0f/html5/thumbnails/16.jpg)
Question behaviours• How the student interacts with a question• For example
1 The Student gives a response and clicks ‘Check’
2 They immediately get told if they were right, with feedback
3 If they were wrong, the can click ‘Try again’
4 If they get it right on a later try, they get less marks
5 After 3 incorrect tries, they are just told the answer• Some parts of this are configurable
![Page 17: 2012 Computer-Assisted Assessment](https://reader034.fdocuments.net/reader034/viewer/2022051209/54803694b4af9fd3158b5e0f/html5/thumbnails/17.jpg)
Question behaviours (to date)• Interactive with multiple tries – as on last slide• Deferred feedback• Immediate feedback• Deferred feedback with CBM• Immediate feedback with CBM• Adaptive mode• Adaptive mode (no penalties)• Manually graded
![Page 18: 2012 Computer-Assisted Assessment](https://reader034.fdocuments.net/reader034/viewer/2022051209/54803694b4af9fd3158b5e0f/html5/thumbnails/18.jpg)
Question types – run-time• I described the behaviour without giving any details of
the question
• Question types say what the question is-what is shown-how the student responds-how it is graded-how the feedback is generated
![Page 19: 2012 Computer-Assisted Assessment](https://reader034.fdocuments.net/reader034/viewer/2022051209/54803694b4af9fd3158b5e0f/html5/thumbnails/19.jpg)
Question types – authoring• Moodle questions are authored using web forms• Teachers can define the question, within the template
provided by the question type
• Questions are stored in Moodle’s question bank-an area for each course / quiz-hierarchical categories in each area
![Page 20: 2012 Computer-Assisted Assessment](https://reader034.fdocuments.net/reader034/viewer/2022051209/54803694b4af9fd3158b5e0f/html5/thumbnails/20.jpg)
Question types (to date)• All question types combine a few basic interactions-entering words-entering numbers / symbolic notations-selection-positioning things – points, molecules, parts of a diagram
• Moodle now has question types for all of these separately-… but what about combinations?
![Page 21: 2012 Computer-Assisted Assessment](https://reader034.fdocuments.net/reader034/viewer/2022051209/54803694b4af9fd3158b5e0f/html5/thumbnails/21.jpg)
Question import/export formats• A way to get questions-in to Moodle from other systems-out of Moodle
• Also useful for bulk authoring
![Page 22: 2012 Computer-Assisted Assessment](https://reader034.fdocuments.net/reader034/viewer/2022051209/54803694b4af9fd3158b5e0f/html5/thumbnails/22.jpg)
Question display options• Information should, or should not, be visible at given times-marks-right/wrong-different types of feedback
• There are settings for this• and natural limits-e.g. no feedback until an answer is submitted
![Page 23: 2012 Computer-Assisted Assessment](https://reader034.fdocuments.net/reader034/viewer/2022051209/54803694b4af9fd3158b5e0f/html5/thumbnails/23.jpg)
Putting questions together• In principle, any question can use with any behaviour• In practice, it is not that simple-e.g. essays can only be manually graded
• Still, many interactive questions can be assembled
![Page 24: 2012 Computer-Assisted Assessment](https://reader034.fdocuments.net/reader034/viewer/2022051209/54803694b4af9fd3158b5e0f/html5/thumbnails/24.jpg)
Activities
![Page 25: 2012 Computer-Assisted Assessment](https://reader034.fdocuments.net/reader034/viewer/2022051209/54803694b4af9fd3158b5e0f/html5/thumbnails/25.jpg)
The quiz – set up• Sequence of questions to be answered-specific question or random selection-fixed order or shuffled-answered in sequence, or freely
• Which students should have access• When and where attempts can be made
![Page 26: 2012 Computer-Assisted Assessment](https://reader034.fdocuments.net/reader034/viewer/2022051209/54803694b4af9fd3158b5e0f/html5/thumbnails/26.jpg)
Quiz access rulesAll the different ways students may, or may not, be restricted from accessing the quiz• Open / close dates• Time limit• Number of attempts• IP address• Secure browser• Enforced delays between attempts• Acknowledge academic honesty statement first
![Page 27: 2012 Computer-Assisted Assessment](https://reader034.fdocuments.net/reader034/viewer/2022051209/54803694b4af9fd3158b5e0f/html5/thumbnails/27.jpg)
The quiz – results• Question outcomes combined to give overall results• Results reported• Final grades sent to the grade-book
![Page 28: 2012 Computer-Assisted Assessment](https://reader034.fdocuments.net/reader034/viewer/2022051209/54803694b4af9fd3158b5e0f/html5/thumbnails/28.jpg)
Quiz reportsAllow the outcomes to be viewed, analysed and downloaded• Grades• Responses• Statistics – standard test psychometrics – CTT• Efficient manual grading
![Page 29: 2012 Computer-Assisted Assessment](https://reader034.fdocuments.net/reader034/viewer/2022051209/54803694b4af9fd3158b5e0f/html5/thumbnails/29.jpg)
Lesson activity• Like a “choose your own adventure” book• Since Moodle 1.2 – not yet using the question
bank/question engine
![Page 30: 2012 Computer-Assisted Assessment](https://reader034.fdocuments.net/reader034/viewer/2022051209/54803694b4af9fd3158b5e0f/html5/thumbnails/30.jpg)
Other possibilities• Could be built in Moodle• Or IMS LTI could be used to connect to external tools
![Page 31: 2012 Computer-Assisted Assessment](https://reader034.fdocuments.net/reader034/viewer/2022051209/54803694b4af9fd3158b5e0f/html5/thumbnails/31.jpg)
Summary
![Page 32: 2012 Computer-Assisted Assessment](https://reader034.fdocuments.net/reader034/viewer/2022051209/54803694b4af9fd3158b5e0f/html5/thumbnails/32.jpg)
How does Moodle measure up?A CMA system must address• pedagogic• administrative• technical
considerations
![Page 33: 2012 Computer-Assisted Assessment](https://reader034.fdocuments.net/reader034/viewer/2022051209/54803694b4af9fd3158b5e0f/html5/thumbnails/33.jpg)
Pedagogy• Behaviours allow a range of learning tasks-Gibbs and Simpson conditions for feedback to promote
learning → Interactive behaviour-Certainty Based Marking
• Question types-designed to allow detailed feedback
• Options for constructing quizzes• Quiz reports
![Page 34: 2012 Computer-Assisted Assessment](https://reader034.fdocuments.net/reader034/viewer/2022051209/54803694b4af9fd3158b5e0f/html5/thumbnails/34.jpg)
Administration• Moodle provides a lot of this-courses, users, groups, authentication, enrolment-course as an sequence of activities-gradebook, event calendar, notifications
• Question bank• Quiz set-up UI• Quiz access rules• Quiz reports
![Page 35: 2012 Computer-Assisted Assessment](https://reader034.fdocuments.net/reader034/viewer/2022051209/54803694b4af9fd3158b5e0f/html5/thumbnails/35.jpg)
Technical design• Small core + plugins• Moodle as a framework-runs on Windows/Linux/Mac, Mysql/Postgres/...-concepts of courses, users, groups, uploaded files...
• Simple but flexible database design for questions• Open source project-software as as a way to capture the learning of a
community of practice
![Page 36: 2012 Computer-Assisted Assessment](https://reader034.fdocuments.net/reader034/viewer/2022051209/54803694b4af9fd3158b5e0f/html5/thumbnails/36.jpg)
Where should Moodle do more?• More different activities using questions-lesson should use question bank / engine-question practice-question authoring / peer review as a learning task
• Better facilities for collaborative authoring of questions-sharing between quizzes / courses / sites-versioning
• More flexibility for building quizzes out of questions
![Page 37: 2012 Computer-Assisted Assessment](https://reader034.fdocuments.net/reader034/viewer/2022051209/54803694b4af9fd3158b5e0f/html5/thumbnails/37.jpg)
Three things to remember• Question = behaviour + type• Plugins to organise software with lots of functionality• Moodle quiz comes from 10 years of community of practice-Future contributions to the open source project welcome
Documentationhttp://docs.moodle.org/en/Quiz_module
eAssessment at the OU with open source software
http://v.gd/oueassessment
![Page 38: 2012 Computer-Assisted Assessment](https://reader034.fdocuments.net/reader034/viewer/2022051209/54803694b4af9fd3158b5e0f/html5/thumbnails/38.jpg)
Map of Moodle’s CMA tools