08 assessing reading and writing
Transcript of 08 assessing reading and writing
A framework for assessing readingRecommendations for reading
assessment
• Step 1: Write test specifications
• Step 2: Select an appropriate text
• Step 3: Modify the text
• Step 4: Write test questions
• Step 5: Decide on weighting
• Step 6: Validate and re-validate
Lloyd & Davidson (2009) en Coombe, Folse & Hubley (2007)
• Step 7: Ensure standardized test
administration
• Step 8: Ensure reliable rating/marking
• Step 9: Rescale (if necessary)
• Step 10: Provide students with diagnostic
feedback
• Step 11: Evaluate your test
• Step 12: Recycle your test
Lloyd & Davidson (2009) en Coombe, Folse & Hubley (2007)
Coombe, Folse & Hubley (2007)
1. Use specificationsfor even coverage of curricular outcomes
to promote content validity
2. Avoid skill contamination by limiting writing
3. Choose text to fit purpose of assessment
and to match students’ level.
4. Make items less difficult than passage
level
5. Test items in the order of the passage.
6. Sample skills with range of formats.7. Keep a file of authentic material from
newspapers, magazines, etc.8. Avoid texts with controversial or biased
material.9. For objective formats, try to make all
statements positive.10. Rephrase material using synonyms to
avoid students looking for verbatim matches.
Coombe, Folse & Hubley (2007)
indirect vs. direct assessmentfree writing vs. guided writing
writing assessment scalesresponding to student writing
10 things to remember
Indirect writing assessment
• It measures correct usage in sentence level constructions and focuses on spelling and punctuation via objective formats like MCQs and cloze tests.
• Measures determine the S’s knowledge of writing sub-skills such as grammar and sentence construction which are assumed to constitute components of writing ability
• Measures are largely concerned with accuracy rather than communication.
Coombe, Folse & Hubley (2007)
Direct writing assessment
• It measures the S’s ability to communicate through the written mode based on the production of written texts.
• It requires Ss to come up with the content, find a way to organize the ideas and use appropriate vocabulary, grammatical conventions and syntax.
• Measures integrate all elements of writing.
Coombe, Folse & Hubley (2007)
free writing guided writing
requires Ss to read a
prompt that poses a
situation and write a
planned response
based on a combination
of background
knowledge and
knowledge learned from
the course.
requires Ss to
manipulate content that
is provided in the
prompt, usually in the
form of a chart or
diagram. It’s a bridge
between objective and
subjective formats.
Coombe, Folse & Hubley (2007)
• Be clear about expected form/length of response (1 paragraph, 250 words, a letter)
• Clearly specify what you want in the prompt(3 causes & effects, two supporting details)
• Specify discourse patterns students are expected to use(i.e. compare/contrast, cause/effect, description etc.)
• Ask Ss to provide something beyond the prompt (i.e., opinion, inference, prediction.
Coombe, Folse & Hubley (2007)
• Considerations for GW are useful for free writing.
• Use of multiple raters. Agree on grading criteria in
advance & calibrate before grading).
• Decide on which scale to use beforehand.
• Acquaint Ss with marking scheme in advance by
using it for teaching, grading homework and
providing feedback.
• Provide Ss with enough space for an outline, a draft
and the finished product.
• Avoid issues that might offend or disadvantage Ss.Coombe, Folse & Hubley (2007)
Holistic marking Analytical marking
It is based on marker's
total impression of the
essay. It is variously
termed impressionistic,
global or integrative
marking.
It requires Ts to give
separate ratings for the
different components of
language ability. It has
been termed discrete
point marking.
Coombe, Folse & Hubley (2007)
Advantages Disadvantages
• Reliable if done under no time constraints & if Ts receive training.
• Generally perceived to be quicker than other types of marking
• Since overall writing ability is assessed, Ss are not disadvantaged by one lower component, i.e. poor grammar.
• Scores emphasize the writer’s strengths
• Unreliable if marking is done under short time constraints & with untrained Ts.
• Longer essays often tend to receive higher marks
• Tendency for marker to overlook sub-skills that make up writing.
• It does not provide a profile of the student's writing ability and to provide feedback.
• Difficult to interpret composite score
Coombe, Folse & Hubley (2007)
Advantages Disadvantages
• Generally more effective with inexperienced teachers.
• provide Ts with a "profile" of their Ss' strengths & weaknesses
• Training raters is easier because the scales are more explicit and detailed.
• Perceived to be more time consuming
• A set of specific criteria has to be written and markers need to be trained: attend frequent calibration sessions.
• Because Ts look at specific areas in a given essay, the most common being content, organization, grammar, mechanics and vocabulary, marks are often lower
Coombe, Folse & Hubley (2007)
• Feedback provides opportunities for Ss to learn & make improvements
• Most common type of written teacher feedback is handwritten comments, usually at the end of the paper or in the margins
• Some Ts like to use correction codes to provide formative feedback and to facilitate marking
• Electronic feedback is particularly valuable because it can be used to give a combination of handwritten comments and correction codes.
Coombe, Folse & Hubley (2007)
sp Spelling
vt Verb tense
ww Wrong word
wv Wrong verb
☺ Nice idea/content!
₪ Switch placement
¶ New paragraph
? I don’t understand
Coom
be,
Fols
e&
Huble
y(2
007)
1. Give students multiple writing assessment opportunities.
2. Test a variety of writing skills and create tasks of varying lengths.
3. Develop prompts that are appropriate for the Ss.
4. Evaluate all answers to one question before going on to the next.
5. Mark only what the student has written.
Coombe, Folse & Hubley (2007)
6. Have a systematic approach for dealing with marking discrepancies: average of the two raters for a small discrepancy and to utilize a third rater if there is a big discrepancy.
7. Get students involved. 8. Provide students with diagnostic
feedback.9. Practice blind or double blind marking. 10. Calibrate and recalibrate. The best way to achieve inter-rater reliability is to practice.
Coombe, Folse & Hubley (2007)
Coombe, C., Folse. K, & Hubley, N. (2007) A Practical
Guide to Assessing English Language Learners. Ann
Arbor, MI: University of Michigan Press.