Using a VLE for Efficient and Effective Feedback

Nick LundNick Lund

Manchester Metropolitan Manchester Metropolitan UniversityUniversity

Introduction Introduction Knight and Yorke (2003) – traditional Knight and Yorke (2003) – traditional

assessment has more to do with assessment has more to do with accountability and quality control than with accountability and quality control than with providing feedback for learning.providing feedback for learning.

Falchikov (2005) – similarly notes the Falchikov (2005) – similarly notes the conflicting demands on assessment. She conflicting demands on assessment. She also points out “there is little other than also points out “there is little other than custom and familiarity to suggest traditional custom and familiarity to suggest traditional methods have worked ‘well’.” Further she methods have worked ‘well’.” Further she claims it produces passive learners with claims it produces passive learners with lower levels of cognitive activities.lower levels of cognitive activities.

IntroductionIntroduction Bloxham and Boyd (2007) – ‘feedback is the Bloxham and Boyd (2007) – ‘feedback is the

most important aspect of the assessment most important aspect of the assessment process for raising achievement, yet process for raising achievement, yet currently students express considerable currently students express considerable dissatisfaction with much feedback’ dissatisfaction with much feedback’

Hulme and Forshaw (2009) - traditional Hulme and Forshaw (2009) - traditional written feedback is often not the most written feedback is often not the most effective for students or efficient for staff.effective for students or efficient for staff.

Heinrich et al (2009) – e-tools are used to Heinrich et al (2009) – e-tools are used to increase efficiency of marking (with time increase efficiency of marking (with time freed being invested in quality freed being invested in quality improvements).improvements).

Background Background HEA Psychology Network workshopHEA Psychology Network workshop

The Double EE’s: Giving students feedback The Double EE’s: Giving students feedback effectively and efficientlyeffectively and efficiently

Details at Details at http://www.psychology.heacademy.ac.uk/html/event_reporhttp://www.psychology.heacademy.ac.uk/html/event_reports_2009.aspts_2009.asp

Phil Denton’s electronic feedback softwarePhil Denton’s electronic feedback software Discussed at the conferenceDiscussed at the conference Used in Psychology at MMU Cheshire for 5 yearsUsed in Psychology at MMU Cheshire for 5 years

Denton's electronic Denton's electronic feedbackfeedback

System for producing detailed feedbackSystem for producing detailed feedback Can use different types of comment:Can use different types of comment:

general comments (to whole class)general comments (to whole class) standard comments (to a proportion of class)standard comments (to a proportion of class) personal comments (to single person)personal comments (to single person) criterion comments (performance on each criterion comments (performance on each

assessment criterion)assessment criterion) All general, standard and criterion All general, standard and criterion

comments input to file & then available for comments input to file & then available for use.use.

For details seeFor details seehttp://www.psychology.heacademy.ac.uk/docs/ppt/http://www.psychology.heacademy.ac.uk/docs/ppt/

p20090715_Feedback_Software_Denton.pptp20090715_Feedback_Software_Denton.ppt

Denton's electronic Denton's electronic feedbackfeedback

Problems of softwareProblems of software Those of us who use it find the system to be Those of us who use it find the system to be

an excellent tool. However there are some an excellent tool. However there are some problemsproblems

Complexity Complexity Some staff find it too complex to set up and to useSome staff find it too complex to set up and to use

Set up timeSet up time For learning to use the systemFor learning to use the system For establishing comment bank for grades, specific For establishing comment bank for grades, specific

criteriacriteria Support Support

Produced by an individual not a software companyProduced by an individual not a software company Phil Denton always tries to overcome problems but Phil Denton always tries to overcome problems but

he is not paidhe is not paid

Reason this ‘project’ came about was some conflict Reason this ‘project’ came about was some conflict in the set up on my computer last year in the set up on my computer last year

Simple system using WebCTSimple system using WebCT Adapted for an experimental report assignment.Adapted for an experimental report assignment. Took all the descriptors from the electronic marking Took all the descriptors from the electronic marking

system & made them available on WebCT.system & made them available on WebCT. Students could see all descriptors of all mark bands Students could see all descriptors of all mark bands

and weighting for every part of the report before and weighting for every part of the report before and after submission.and after submission.

Feedback consisted of a brief personal comment Feedback consisted of a brief personal comment followed by a mark for each section of the report.followed by a mark for each section of the report.

Students told to compare their marks with grade Students told to compare their marks with grade comments comments andand to look at comments of grade/s to look at comments of grade/s above.above.

Invited to discuss/challenge marks in individual Invited to discuss/challenge marks in individual tutorial slots tutorial slots

Example of feedbackExample of feedback Overall this is a very good report with an excellent Overall this is a very good report with an excellent

method and results section. Your abstract was method and results section. Your abstract was accurate but not concise, try to keep it brief. The accurate but not concise, try to keep it brief. The introduction was good but was a little sketchy on introduction was good but was a little sketchy on the face inversion effect (one of the IVs).the face inversion effect (one of the IVs).

AbstraAbstract ct

IntroductioIntroductionn

MethoMethodd

ResultsResults DiscussiDiscussionon

RefsRefs TOTALTOTAL

5555 6060 7070 7070 6565 6060 6464

Extract from WebCT Extract from WebCT informationinformation

Introduction Introduction (20%)(20%) 00 There is no introduction to this assignment. You should include an There is no introduction to this assignment. You should include an

introduction detailing relevant theory and research, appropriately referenced introduction detailing relevant theory and research, appropriately referenced and providing an introduction to the research you are undertaking in the and providing an introduction to the research you are undertaking in the assignment. It should contain background information to both of the assignment. It should contain background information to both of the independent variables of the experiment.independent variables of the experiment.

2020 There is only a very superficial introduction to this assignment. You There is only a very superficial introduction to this assignment. You should include an introduction detailing relevant theory and research, should include an introduction detailing relevant theory and research, appropriately referenced and providing an introduction to the research you appropriately referenced and providing an introduction to the research you are undertaking in the assignment. It should contain background information are undertaking in the assignment. It should contain background information to both of the independent variables of the experiment.to both of the independent variables of the experiment.

3030 There is only a superficial introduction to this assignment. You There is only a superficial introduction to this assignment. You should include an introduction detailing relevant theory and research, should include an introduction detailing relevant theory and research, appropriately referenced and providing an introduction to the research you appropriately referenced and providing an introduction to the research you are undertaking in the assignment. It should contain background information are undertaking in the assignment. It should contain background information to both of the independent variables of the experiment.to both of the independent variables of the experiment.

4040 There is a fair introduction to this assignment. You should include There is a fair introduction to this assignment. You should include an introduction detailing relevant theory and research, appropriately an introduction detailing relevant theory and research, appropriately referenced and providing an introduction to the research you are referenced and providing an introduction to the research you are undertaking in the assignment. It should contain background information to undertaking in the assignment. It should contain background information to both of the independent variables of the experiment.both of the independent variables of the experiment.

Evaluation Evaluation Three perspectivesThree perspectives

StaffStaff External examinerExternal examiner StudentsStudents

Staff – quick and efficient.Staff – quick and efficient.

EE – liked the amount of information given EE – liked the amount of information given to students and the transparency of the to students and the transparency of the system. Made special mention of system in system. Made special mention of system in the Examinations Boardthe Examinations Board

EvaluationEvaluation Students – 44 on unitStudents – 44 on unit Sources of evidenceSources of evidence

Questionnaire given after the feedback Questionnaire given after the feedback WebCT recordsWebCT records Use of consultancy with unit leaderUse of consultancy with unit leader

Student questionnaire - Student questionnaire - resultsresults

31 from 44 responded.31 from 44 responded. 24 favourable24 favourable

System more informative, encouraged System more informative, encouraged them to review work more, more them to review work more, more helpful, more transparent.helpful, more transparent.

7 not entirely favourable7 not entirely favourable Liked information but found system Liked information but found system

impersonalimpersonal

Student evaluationStudent evaluation WebCT accessWebCT access

All 39 who collected the assignment viewed All 39 who collected the assignment viewed the relevant WebCT pages within three days.the relevant WebCT pages within three days.

Use of consultancyUse of consultancy 15 used the opportunity to discuss their marks15 used the opportunity to discuss their marks 6 challenged the marking of at least one 6 challenged the marking of at least one

section section

Discussion Discussion Evidence suggests feedback wasEvidence suggests feedback was

Efficient – took less staff time than writing long Efficient – took less staff time than writing long commentscomments

Effective – students understood feedback and Effective – students understood feedback and were able to act on itwere able to act on it

Engaging – it required active participation. Engaging – it required active participation. Allowed students to make their own Allowed students to make their own judgements about their work & challenge judgements about their work & challenge marksmarks

Transparent – everything used in marking was Transparent – everything used in marking was available to students & it allowed them to available to students & it allowed them to study all mark bandsstudy all mark bands

Questions Questions Should we encourage students to Should we encourage students to

challenge marks?challenge marks?

How can it be adapted for other How can it be adapted for other types of assignments?types of assignments?

Does it have any advantages over Does it have any advantages over Denton’s more sophisticated system Denton’s more sophisticated system of feedback?of feedback?

References References Bloxham, S & Boyd, PBloxham, S & Boyd, P (2007). (2007). Developing effective Developing effective

assessment in Higher Educationassessment in Higher Education. Maidenhead: OU Press. Maidenhead: OU Press Denton, P., Roberts, M., Madden, J. & Rowe, P. (2008). Denton, P., Roberts, M., Madden, J. & Rowe, P. (2008).

Students’ response to traditional and computer Students’ response to traditional and computer assisted formative feedback: a comparative case assisted formative feedback: a comparative case study. study. British Journal of Educational Technology, 39, British Journal of Educational Technology, 39, 486-500.486-500.

Falchicov, N (2005). Falchicov, N (2005). Improving assessment through Improving assessment through student involvement. student involvement. Abingdon: RoutledgePalmerAbingdon: RoutledgePalmer

Heinrich, E., Milne, J., Ramsay, A., & Morrison, D. Heinrich, E., Milne, J., Ramsay, A., & Morrison, D. (2009). Recommendations for the use of e-tools for (2009). Recommendations for the use of e-tools for improvements around assignment marking quality. improvements around assignment marking quality. Assessment & Evaluation in Higher EducationAssessment & Evaluation in Higher Education, , 3434, 469-, 469-479. 479.

Hulme, J. & Forshaw, M. (2009). Effectiveness of Hulme, J. & Forshaw, M. (2009). Effectiveness of feedback provision for undergraduate students. feedback provision for undergraduate students. Psychology Learning and TeachingPsychology Learning and Teaching, , 88, 34-38., 34-38.

Knight, P.T. & Yorke, M. (2003). Knight, P.T. & Yorke, M. (2003). Assessment, Learning Assessment, Learning and Employability. and Employability. Maidenhead: OU PressMaidenhead: OU Press