Developing evaluation instruments for electronic resources © alecbrunei 24/05/01

Developing evaluation instruments Developing evaluation instruments for electronic resourcesfor electronic resources

Presentation FormatPresentation Format

1.1. Course contentCourse content

2.2. Evaluation issuesEvaluation issues

3.3. Reflective evaluationReflective evaluation

1. Course content1. Course content

• Three componentsThree components

– Theories of learningTheories of learning

– Software evaluationSoftware evaluation

– Website evaluationWebsite evaluation

Theories of learningTheories of learning

• ““the disciplinary authority of theory … as a the disciplinary authority of theory … as a condition for enquiry” condition for enquiry” (Widdowson, 1994)(Widdowson, 1994)

• Levy (1997)Levy (1997) computers as tools/as tutors computers as tools/as tutors

• behaviourismbehaviourism

• constructivismconstructivism

• Vygotskyan socio-cultural learning theoryVygotskyan socio-cultural learning theory

• (Jones & Mercer, 1993)(Jones & Mercer, 1993)

Software evaluationSoftware evaluation

• collaborativecollaborative exploration, assessment & exploration, assessment & evaluation of ESL softwareevaluation of ESL software

• construct instruments (online examples, construct instruments (online examples, lecturer input)lecturer input)

• Hubbard’s (1988) evaluation parametersHubbard’s (1988) evaluation parameters

• trial instrumenttrial instrument

• evaluate application, effectiveness & utility of evaluate application, effectiveness & utility of instrumentinstrument

Website evaluationWebsite evaluation• individual exploration, assessment & individual exploration, assessment &

evaluation of ESL websitesevaluation of ESL websites

• remote (online) & resident lecturer inputremote (online) & resident lecturer input

• flexibility of delivery paradigmflexibility of delivery paradigm

• benefits of text-based chat environmentbenefits of text-based chat environment

• additional issues for evaluationadditional issues for evaluation

2. Evaluation Issues2. Evaluation Issues

• General evaluation issuesGeneral evaluation issues

• Software evaluation issuesSoftware evaluation issues

• Website evaluation issuesWebsite evaluation issues

General evaluation issuesGeneral evaluation issues

•• ““necessary enabling conditions” necessary enabling conditions” (Widdowson, 1994) (Widdowson, 1994)

•• EL texts/coursebooks v electronic resourcesEL texts/coursebooks v electronic resources

•• the importance of critical, evaluative toolsthe importance of critical, evaluative tools

• • lack of spending on teacher developmentlack of spending on teacher development

•• ““rich learning tasks” (Luke, 2000)rich learning tasks” (Luke, 2000)

•• for ICT for ICT people need to be trained to learn people need to be trained to learn to change (Luey, 1997)to change (Luey, 1997)

•• avoidance of the hype of ICTavoidance of the hype of ICT

Software Evaluation IssuesSoftware Evaluation Issues

• what to evaluate, how to communicate findingswhat to evaluate, how to communicate findings

• difficulty of overview and detail - linear paths of difficulty of overview and detail - linear paths of informationinformation

• the tunnel vision of the computerthe tunnel vision of the computer

• categories for evaluationcategories for evaluation

• ““many of the components in each category are many of the components in each category are interrelated both inside and outside the category” interrelated both inside and outside the category” (Norazimawati Ahmad, 2000)(Norazimawati Ahmad, 2000)

Website Evaluation IssuesWebsite Evaluation Issues• what to evaluate, how to communicate what to evaluate, how to communicate

findings given data communications findings given data communications capacity of websitescapacity of websites

• overview & the importance of navigationoverview & the importance of navigation

• new evaluation categoriesnew evaluation categories

• conflicts between quantity of information conflicts between quantity of information and quality of information, between and quality of information, between information & knowledge (Eastment, 2001)information & knowledge (Eastment, 2001)

• evaluating as an Internet competency evaluating as an Internet competency (Eastment, 2001)(Eastment, 2001)

3. Reflective evaluation3. Reflective evaluation• contemplate problem, think about solutions contemplate problem, think about solutions

(Widdowson’s “necessary enabling conditions”) (Widdowson’s “necessary enabling conditions”)

• autonomy not dependence (Wainryb, 1992)autonomy not dependence (Wainryb, 1992)

• how evaluation deals with features of software how evaluation deals with features of software or web-siteor web-site

• communicating value judgements to readerscommunicating value judgements to readers

• student-teacher evaluations are the beginnings student-teacher evaluations are the beginnings of a process of thinking critically about of a process of thinking critically about electronic teaching resourceselectronic teaching resources