EVALUATION OF THREE AUDIO FEEDBACK

METHODSDR I-CHANT CHIANG

IAC@ABER.AC.UK

2 FEBRUARY 2011

PROBLEMS WITH WRITTEN FEEDBACK

LEAST SATISFIED AREA IN NATIONAL STUDENT SURVEY (BBC, 2007; UNISTATS, 2009)

LACK OF DEPTH, SIMPLY STATE THE PROBLEM

ILLEGIBLE HANDWRITING

OVEREMPHASIS ON AREAS OF WEAKNESS

INABILITY TO PRIORITIZE

BENEFITS OF AUDIO FEEDBACK

STUDENT

EASIER TO UNDERSTAND

MORE DEPTH

RICHER FORM OF FEEDBACK

NUANCED FEEDBACK

TONE OF VOICE

MORE GENUINE, ACCESSIBLE, PERSONAL

BETTER WORK, BETTER MARKS(KING, MCGUGAN, BUNYAN, 2008;

ICE, CURTIS, PHILLIPS, WELLS, 2007; MERRY AND ORSMOND, 2008)

BENEFITS OF AUDIO FEEDBACK

TUTOR

MORE DEPTH

FLEXIBILITY

TIME SAVINGS LONG-TERM

GIVING FEEDBACK

HIGHER QUALITY WORK

(KING, MCGUGAN, BUNYAN, 2008; ICE, CURTIS, PHILLIPS, WELLS, 2007;

MERRY AND ORSMOND, 2008)

DIFFERENT TYPES OF STUDENTSDIFFERENT TYPES OF TUTORS

RESEARCH QUESTIONS

HOW DO DIFFERENT STUDENTS RESPOND TO AUDIO FEEDBACK?

HOW DO DIFFERENT METHODS OF AUDIO FEEDBACK COMPARE?

RESEARCH QUESTIONS

HOW DO DIFFERENT STUDENTS RESPOND TO AUDIO FEEDBACK?

LOW-PERFORMING STUDENTS DO NOT TAKE FEEDBACK ON BOARD LIKE HIGH-PERFORMING STUDENTS(VAN DER ZEE, 2009)

LISTENING AND READING SKILLS DEVELOP DIFFERENTLY WHEN LEARNING IN A NON-NATIVE LANGUAGE (HERRON, MORRIS, SECULES, & CURTIS, 1995)

RESEARCH QUESTIONS

HOW DO DIFFERENT METHODS OF AUDIO FEEDBACK COMPARE?

IMPLEMENTATION, EFFECTIVENESS, AND PREFERENCE

TUTORS AND STUDENTS

ACROSS ASSESSMENT TYPES

THREE METHODS

AUDIO ONLY

TUTOR: RECORDS ON DIGITAL RECORDER

STUDENT: RECEIVES MP3 FILE

AUDIO-VISUAL ASYNCHRONOUS

TUTOR: RECORDS VIA HEADSET AND ADOBE ACROBAT 9.0

STUDENT: RECEIVES PDF FILE

AUDIO-VISUAL SYNCHRONOUS

TUTOR: RECORDS VIA HEADSET AND SCREENCAST

STUDENT: RECEIVES LINK TO WEBSITE

HTTP://SCREENCAST.COM/T/PBQZFECJ744

STUDY METHODOLOGY

5 ACADEMIC STAFF MEMBERS

33 PSYCHOLOGY HONOURS STUDENTS

“HIGH-PERFORMING” AND “LOW-PERFORMING”

11 NON-NATIVE ENGLISH SPEAKERS

14 ASSESSMENTS

EVALUATION

QUESTIONNAIRES

STUDENT PERFORMANCE

INTERVIEWS

FOCUS GROUPS

RESULTS

TIME PER METHOD

0%

25%

50%

75%

100%

Audio Only A-V Asynchronous A-V Synchronous0

3

6

9

12

Percent Listened Average Minutes Recorded

% LISTENED TO FEEDBACK

0%

25%

50%

75%

100%

Audio Only A-V Asynchronous A-V Synchronous

35%48%

60%

96%94%100%

High Performing Students Low Performing Students

AUDIO ONLY

STUDENT PERSPECTIVE

VERY USEFUL, LEAST CHALLENGING, LISTENED IN A VARIETY OF SETTINGS

PREFERENCE: 2ND AMONG LO STUDENTS, 3RD AMONG HI STUDENTS

“IT IS ALL IN ONE GO RATHER THAN HAVING TO CLICK ON SOUND BITES INDIVIDUALLY”

AUDIO ONLY

TUTOR PERSPECTIVE

VERY EASY TO USE, PORTABLE

GAVE BROADER, LESS SPECIFIC FEEDBACK

LEAST PREFERRED METHOD

“IT WAS VERY STRAIGHTFORWARD”

BEST FOR POSTERS, NON-ELECTRONIC WORK

AUDIO-VISUAL ASYNCHRONOUS

STUDENT PERSPECTIVE

VERY USEFUL, SPECIFIC AND DETAILED, SELF-PACED, NO WAY TO STOP MID-COMMENT, MANY CLICKS, SOMETIMES INCONSISTENT

PREFERENCE: 1ST AMONG LO STUDENTS, 2ND AMONG HI STUDENTS

“IT PINPOINTS EXACTLY WHERE THEY’RE SPEAKING ABOUT AND IT ALSO OFFERS THE MARKER AN OPPORTUNITY TO BE MORE ELABORATE, EXPRESS THEMSELVES BETTER”

AUDIO-VISUAL ASYNCHRONOUS

TUTOR PERSPECTIVE

CLEAR, SIMPLE, STRAIGHTFORWARD, EXPANSIVE

TEDIOUS, SPENDING MORE TIME

HALF THE TUTORS PREFER THIS METHOD MOST

“I LIKED IT, BUT I TOOK WAY TOO LONG”

BEST FOR REPORTS, PROPOSALS

AUDIO-VISUAL SYNCHRONOUS

STUDENT PERSPECTIVE

FELT LIKE SITTING WITH THE TUTOR, GOOD FOR FORMULATING ARGUMENTS, MUST HAVE GOOD INTERNET CONNECTION, COULD NOT SAVE THE FILE

PREFERENCE: 3RD AMONG LO STUDENTS, 1ST AMONG HI STUDENTS

“I CAN SEE THE LECTURERS’ THOUGHT PROCESS”

AUDIO-VISUAL SYNCHRONOUS

TUTOR PERSPECTIVE

CANNOT WAFFLE, FOCUS ON IMPORTANT POINTS

PINPOINT SPECIFICS, RE-RECORD SOMETIMES

HALF THE TUTORS PREFER THIS METHOD MOST

“IT IS MORE LIKE A (ONE-SIDED) CONVERSATION”

BEST FOR ESSAYS, FORMAL PRESENTATIONS

STUDENT TYPES

NO DIFFERENCES BETWEEN NATIVE AND NON-NATIVE ENGLISH SPEAKERS

A-V Synchronous

A-V Asynchronous

Audio Only

Overall High Performing Low Performing

MUCH LESS

SOMEWHAT LESS

SLIGHTLY LESS

ABOUT THE

SAME

SLIGHTLY MORE

SOMEWHAT MORE

MUCH MORE

OVERALL THEMES

TUTORS ENJOYED GIVING FEEDBACK THROUGH AUDIO MEANS BUT SOME ARE PUT OFF BY TECHNOLOGY

END OF YEAR EVALUATIONS WERE HIGH FOR FEEDBACK

LANGUAGE COMPETENCE MAY BE A FACTOR IN 1ST YEAR

DIFFERENCES IN HIGH AND LOW PERFORMING STUDENTS

OVERALL THEMES

MARKS INCREASED MOST WITH A-V ASYNCHRONOUS

MATCH METHOD TO ASSESSMENT TYPE

AUDIO ONLY: POSTER, NON-ELECTRONIC WORK

A-V ASYNCHRONOUS: REPORTS, PROPOSALS

A-V SYNCHRONOUS: ESSAYS, FORMAL PRESENTATIONS

FOR MORE INFORMATION ABOUT THIS PROJECT:

HTTP://SITES.GOOGLE.COM/SITE/AUDIOFEEDBACKUK/