The Machine Scoring of Essays: Redefining Writing Pedagogy?

Deborah CrusanWright State University

My talk this morning will

focus on one of the programs typically used for classroom writing/writing assessment - MY Access!

define MY Access! discuss a small study in which a few ELLs

used MY Access! report their attitudes toward MY Access! wonder about the impact of MY Access! and

similar programs on second language writers and writing.

But first, we must remember ~

"For there is nothing either good or bad, thinking makes it so."

~ William Shakespeare (1564-1616), Hamlet, II.ii

My translation: We are (well at least I am) very good at creating dichotomies of good and bad

So just for now, let’s suspend our biases . . .

So . . . What is MY Access?

MY Access! is a “web-based instructional writing product that provides students enrolled in grade 4 through higher education with the opportunity to develop their writing skills within an electronic portfolio-based environment” (Vantage Learning, 2007, p. 1).

MY Access! grades students’ responses to select writing prompts and offers suggestions for improvement to the text in a matter of seconds.

MY Access claims that their program

decreases writing instructors’ grading burden

that the almost instantaneous feedback provided by MY Access! motivates students to revise more.

Problems Warschauer & Grimes (2008) found that

students rarely use programs like MY Access! to revise anything other than surface errors

Teachers can create their own prompts; however, MY Access! will score only those prompts included in the program

In many cases, longer essays receive higher scores; consequently, MY Access! assumes that length is a proxy for fluency.

The study Graduate students worked with ELLs, asking them to

respond three times to two different prompts Students responded to a survey about their

experiences with the program Additionally, grads interviewed ELLS regarding their

attitudes toward MY Access! For the purposes of this presentation, I’ll highlight

comments from six of the students who agreed to be interviewed

Overall, students’ opinions about MY Access! were mixed; students found useful aspects as well as aspects they termed less helpful.

Student attitudes - Hannah

Hannah reported that she was excited about the program, but because she had little computer experience, she had difficulty learning the program. She had problems remembering what to do each time she logged on because she didn’t use the program very often. She found that the MY Access! prompts were not very interesting.

Xiao Xiao admitted that it was very easy

to find and insert information from other sources into his writing – even if it wasn’t his ideas. He found MY Access! feedback useful for making surface corrections – the program pointed them out, so he could correct them – but he wasn’t motivated to revise his work beyond fixing the surface errors.

HeecheonHeecheon believed that he could

get the same kind of information from Microsoft Word. Initially, he liked the instant feedback My Access! provided, but he began to feel overwhelmed as he read through the massive amount of information offered to him by the program.

MohamedMohamed believed that MY Access! would be beneficial in the classroom. He found it to be a great tool to use during the writing process because it gave him a variety of tools and activities to aide in the revision of his writing.

Farin After writing and submitting his

first draft, Farin thought it was scored too low, so he went to the Internet and found some material loosely related to his topic. When he inserted the material into his essay, his score went up even though the essay was not as coherent as it had been when he first wrote it.

WafaWafa felt that MY Access! helped her get into the habit of doing multiple drafts. She liked the editing tools and found some of the feedback very helpful in the revision of her essays.

Consensus Students enjoyed writing with a computer They liked the immediate feedback provided However, they often found the feedback overwhelming Students appreciated help finding grammar errors Many became bored Many students did only surface editing, not revision Several students admitted adding text from the

Internet Students did not use features in MY Access.

Should you use MY Access! in your second language writing classroom?

It depends! Teachers need to ask themselves questions

about these programs and how their use might impact their pedagogy

However, this technology is here to stay, so . . .

Teachers must understand this technology and strive to keep assessments locally controlled, context-specific, and accessible