Ian Lucas Executive Director ETS Europe UK CRITERION ® Online Writing Evaluation.

Ian LucasExecutive DirectorETS Europe UK

CRITERION ®

Online Writing Evaluation

Criterion . . .

• is a web-based service that utilises automated scoring technologies to evaluate writing samples and provide students with an instant holistic score and annotated diagnostic feedback that analyses elements of grammar, usage, mechanics, style, and organisation and development.

CRITERION

Criterion gives students valuable self-paced writing practice, diagnostic feedback and provides instant holistic scores on their essay writing performance - whether in the classroom or at home.

How Criterion Works

• Technology developed using scored student writing, rubrics, and linguistic and discourse analysis tools

• Holistic score feedback based on analysing syntax, organisation of ideas, and vocabulary

• Diagnostic feedback on errors in student writing through a linguistic model that identifies grammatical, mechanical, and usage errors

How Criterion Was Developed

• Technology derived from ETS research on natural language processing and information retrieval

• Essays scored by trained faculty readers• 465 samples across the range of score points per

topic• Since 1999, 1.5 million GMAT essays have been

scored with over 98% agreement with human scorers

• Writing analysis tools developed at ETS provide diagnostic feedback

CRITERION SUPPORTS TEACHING

• Offers the opportunity to practice writing

• Analyse essay development

• Customise topics & writing options

• Provides instant diagnostic feedback

• Summary reports of student performance

• Easy access to class performance

CRITERION SUPPORTS LEARNING

• Real-time in a Web-based application• Use the writer’s handbook• Revise essays based on review & feedback• Opportunities to work from home• Feedback on:

• Grammar• Usage • Mechanics• Style• Organisation & Development

What Criterion Can Provide

• A self-paced, risk-free learning environment• Immediate (within 15 seconds) evaluation on

writing performance• Holistic score and diagnostic feedback• Library of topics and sample essays for

student and instructor review• Individual, interactive student writing

portfolios• Intuitive Writer’s Handbook

What Criterion Cannot Do

• Criterion technology does not actually read essays; it uses natural language processing techniques

• Criterion is not perfect; it sometimes makes mistakes scoring essays

• Holistic scoring technology can be fooled• Diagnostic feedback makes mistakes and

does not catch all errors• Criterion should not replace an instructor

Criterion . . .

• is a web-based service that utilises automated scoring technologies to evaluate writing samples and provide students with an instant holistic score and annotated diagnostic feedback that analyses elements of grammar, usage, mechanics, style, and organisation and development.

Thank you

• More information

• http://www.ets.org/criterion/tour

• Extra information

• [email protected]

• [email protected]

http://www.ets.org/criterion/tour





mailto:[email protected]




Ian Lucas Executive Director ETS Europe UK CRITERION ® Online Writing Evaluation.

Documents

Transcript of Ian Lucas Executive Director ETS Europe UK CRITERION ® Online Writing Evaluation.