Assigning Writing and Using Rubrics to Get Student …wac/History-Legacy/CurriculumResources/...•...

Assigning Writing and Using Rubrics to Get Student Essays that Are Worth Reading and Grading...

...and also to do program assessment, save grading time, increase fairness and consistency, and level the playing field

Presented to the WAC Week Workshop

Prepared and Presented by Chuck Paine, Ph.D., and Whitney Myers, Ph.D. Candidate in English

Writing and Student Engagement

The results are stunning. The relationship between the amount of writing for a course and students’ level of engagement—whether engagement is measured by time spent on the course, or the intellectual challenge it presents, or students’ level of interest in it—is stronger than the relationship between students’ engagement and any other course characteristic. —Richard J. Light, Making the Most of College: Students Speak their Minds (Harvard UP, 2001), 55.

Workshop Abstract

The premise of this workshop is that students will produce better writing—and develop the critical skills they need to succeed in their academic pursuits—if we give them writing assignments that pose interesting problems and if we clearly and completely specify the criteria we’ll use for assessing their work.

We obviously won’t try to cover all the material in this long handout, but we hope it will be useful for later consultation.

Using Rubrics, page i

Using Rubrics, page ii

ContentsAssigning Writing ......................................................................................................................................... 1

Some General Principles for Using Writing to Promote Learning.................................................................................. 1 Students’ Stages of Development as Writers: Novice to Expert .................................................................................... 2 A Variety of Forms of Writing to Assign ......................................................................................................................... 2 Four Effective Ways to Create Problem-Based Tasks................................................................................................... 3

More Examples of Problem-Based Tasks .................................................................................................................. 4 Features of an Effective Assignment ............................................................................................................................. 6

Questions for Peer-Reviewing an Assignment Handout............................................................................................. 7 Individual Activity: Sketching a Course’s Learning Outcomes and Major Assignments ................................................ 8

Using Rubrics................................................................................................................................................ 9 A Rationale for Using Rubrics?...................................................................................................................................... 9 The Elements of a Rubric ............................................................................................................................................ 10 Types of Rubrics.......................................................................................................................................................... 11 Rubric Examples.......................................................................................................................................................... 11

Holistic Rubrics ......................................................................................................................................................... 11 Holistic Scale for Grading Article Summaries (Bean) ............................................................................................ 11 Holistic Scale for Journals in Beginning Spanish Class (Walvoord and Anderson) .............................................. 12 Holistic Scale for Grading Physics Microthemes (Bean) ....................................................................................... 12 Holistic Scale for Quantitative Literacy (Stevens and Levi) ................................................................................... 13 Holistic Scale for Inquiry and Critical Thinking (Stevens and Levi) ....................................................................... 14

Non-Grid Designs ..................................................................................................................................................... 15 Simple Scoring Guide: Argument Essay ............................................................................................................... 15 Ethical Issues and Social Responsibility Portfolio (Stevens and Levi) .................................................................. 16 Diversity of Human Experience Portfolio (Stevens and Levi) ................................................................................ 17 Economics: Analysis of a Proposed Law (Walvoord and Anderson).................................................................... 18 Art History: Hypothetical Newspaper Article (Walvoord and Anderson)................................................................ 20 Education Poster Presentation (Walvoord and Anderson) .................................................................................... 21

Scoring Guides (Single Scale with Only Highest Level of Performance Described) ................................................ 22 Scoring Guide for Changing Communities in Our City (Huba and Freed)............................................................. 22 Leading a Class Discussion (Stevens and Levi) ................................................................................................... 23 Scoring Guide for Abstract for Master’s in Public Health Class............................................................................. 24

Multi-Level Rubrics ................................................................................................................................................... 25 Generic Multi-Level Rubric (Stevens and Levi) ..................................................................................................... 25 Book Review (Stevens and Levi)........................................................................................................................... 26 The Critical Thinking Rubric from Washington State (http://wsuctproject.wsu.edu/ctr.htm) .................................. 27 Summary/Strong Response in First-Year Writing Course (Bean) ......................................................................... 28 Multi-Level Rubric for Formal Oral Performance in Graduate Program (Huba and Freed) ................................... 29

Constructing Rubrics ................................................................................................................................................... 31 Student Collaboration and Student-Writing Artifacts ................................................................................................ 31

Suggestions for Making the Process Work............................................................................................................ 31 A Metarubric (Stevens and Levi) .............................................................................................................................. 32

Classroom Assessment Techniques (CATs).............................................................................................33 One-Minute Papers................................................................................................................................................... 33 Email Minute ............................................................................................................................................................. 33 Direct Paraphrasing (or, the “Dear Aunt Hepsy” letter)............................................................................................. 34 Application Cards...................................................................................................................................................... 34 Faculty Concerns...................................................................................................................................................... 34 Other Rapid Assessment Techniques ...................................................................................................................... 34

Sources.........................................................................................................................................................35

Using Rubrics, page 1

Assigning Writing1

Some General Principles for Using Writing to Promote Learning 1. Take an inventory of course goals (often stated in syllabus as learning outcomes):

• Subject matter goals—essential concepts and knowledge, • Critical thinking goals—disciplinary processes of inquiry, analysis, and argument, • Other goals set by the professor or connected to program’s core outcomes, • Design critical thinking problems connected to your course goals, • Problems should require students to use course knowledge while helping them learn disciplinary

ways of thinking, analyzing, and arguing; • Problems should engage student interest and promote inquiry; • Highest level of critical thinking typically comes from “messy,” “ill-structured,” or open-ended

problems with no algorithmically attained “right answers”—problems that lead to a claim with supporting arguments.

2. Develop a repertoire of ways to give critical thinking problems to students: • As thought provokers for exploratory writing (e.g., one-page “thinking pieces,” in-class

freewrites; cats (see page 33 of this handout); journal entries; other kinds of informal, non-graded writing);

• As short (2-3 page) assignments or very short (one-paragraph) “microtheme” assignments; • As longer, formal writing assignments or as options for research paper topics; • As tasks for small-group problem solving; • As opening questions for a whole-class discussion; • As questions for in-class debates; • As essay exam questions or practice exam questions.

3. Think of writing assignments as a crucial part of course design: • “Reverse engineer”' your course by designing the final assignment first (principle of “backward

design”); • Create earlier assignments that develop the skills needed for the final assignment; • Consider breaking longer assignments into stages; • Consider adding informal non-graded writing to help students explore ideas and promote

learning; • Encourage the value of a writing center for all writers.

4. When assigning formal writing, treat writing as a process: • Create a rhetorical context for assignments giving students a sense of audience, purpose, and

genre;

1 This section is adapted from John C. Bean’s Engaging Ideas and various unpublished workshop handouts.


• Emphasize exploration, reflective research, good talking, multiple perspectives; • Encourage imperfect first drafts; • Stress substantial revision reflecting increased complexity and elaboration of thought and

increased awareness of readers’ needs; • Where possible, allow rewrites; write comments that encourage revision and that emphasize the

higher order concerns of ideas, thought content, organization, and development; • Consider instituting peer review workshops; • Encourage the value of a writing center for all writers.

5. Develop clear scoring criteria and give them to students in advance: • Simple numerical or +/check/- scales for exploratory writing; • Grading rubrics or scoring guides for formal writing.

Students’ Stages of Development as Writers: Novice to Expert2

Stage 1: Non-academic or pseudo-academic writing [what students bring from high school].

Stage 2: Generalized academic writing concerned with stating claims, offering evidence, respecting others’ opinions, and learning how to write with authority [goal of first-year composition].

Stage 3: Novice approximations of particular disciplinary ways of making knowledge [early courses in the major].

Stage 4: Expert, insider prose within a discipline [advanced courses in the major].

A Variety of Forms of Writing to Assign Consider assigning these kinds of writing:

• Academic or scholarly writing in a specific field, • Professional workplace writing (proposals, reports, memos, technical papers, or other

disciplinary kinds of professional public-health writing), • Writing about the discipline or research for general or lay audiences, • Civic or public argument related to the discipline, • Other kinds of writing or communication projects specific to public health (creative projects,

websites, multi-media presentations, PowerPoint presentations, and so forth).

2 Adapted from Susan Peck MacDonald, Professional Writing in the Humanities and Social Sciences (Carbondale: Southern Illinois UP, 1994), 187.

Four Effective Ways to Create Problem-Based Tasks 1. Give students a problematic thesis to defend or attack.

• Recent advertising has (has not) made enormous gains in portraying women as strong, independent, and intelligent.

• Pesticides are/are not absolutely necessary for modern agriculture. • The overriding religious view expressed in Hamlet is (is not) an existential atheism similar to

Sartre’s. • Prescribing Ritalin and other psychotropic medications is (is not) an appropriate treatment for

behavioral problems of children.

2. Give students a problem-laden question. • Based on your review of the professional literature and your own critical thinking, how might a

classroom teacher best accommodate a student with....? • In class we discussed a way to estimate percentage of the wage gap that is attributable to labor

market discrimination. How does Durden and Gaynor’s approach differ from the approach covered in class? Do not focus on the technical details but on the bigger picture. (Dr. Bridget Heideman, Seattle University)

• What should epidemiologist Chris Johnson recommend in response to... Write a coherent 500-word memo that summarizes the problem and describes the most important one or two recommendations, briefly explaining why these recommendations will be most effective.

3. Give students raw data (lists, graphs, tables, etc.) and ask them to write an argument or analysis based on the data. • To what extent do the attached data support the hypothesis “Children from single parent

families”....”? First, create a scattergram as a visual test of the hypothesis. Then create a verbal argument analyzing whether the date support the hypothesis.

• Your friend and you are looking over Table 1 and note that ... but that the.... Your friend was confused about the difference but had to leave for work. Send your friend a coherent, well-structured email message about one screen in length that explains the difference between ... and ..., and that speculates about the factors that lead to such differences between.

4. Let students develop their own question. • Generic method for assigning a problem-based research paper: Task: Write a 7-10 page research

paper on a significant question related to any aspect of [course subject]. The introduction to your paper should pose the question or problem that your paper will address and motivate your readers to agree that your problem is a significant problem that needs addressing. Your proposed answer to this question (summarized in a single sentence) will serve as the thesis statement for your paper. You will deliver this paper to members of your small group where you will role-play participants in an undergraduate research conference. Prospectus: Midway through the course, you will submit to the instructor a prospectus that describes the problem or question that you plan to address and shows why the question is (1) problematic and (2) significant.


More Examples of Problem-Based Tasks Here are some examples of problem-based writing from various undergraduate courses.

History [Developed by Professor A1 Mann, Seattle U.] Your next formal assignment will be to create an argument that supports, rejects, or modifies one of the following theses and that supports your argument with historical evidence. For this thinking piece, select one of the theses and explore your initial ideas.

• The essential theme of the French Revolution was human freedom; Napoleon Bonaparte killed the French Revolution by reversing its thrust toward freedom.

• The industrial revolution created unprecedented wealth at the expense of brutalizing European labor and colonial producers.

• The ultimate victors in the English Revolution of 1688, the American Revolution, and the French Revolution were the economically conservative property-owning classes.

Literature • Freewrite on the poem’s title (2-3 minutes). Given this title, what do you think the poem may be

about? What associations does it raise for you? Then read the poem very carefully at least six times. Circle three significant images that seem most vivid, important, or puzzling to you. Then fieewrite for 2-3 minutes about each image. Describe the image and explore what it means for you. Bring your exploration to class, where you will share your discoveries with other members of your small group.

• Othello begins in the conventional, rational world of Venice. Within this world fathers want their daughters to many the “wealthy, curled darlings” of their own nation-to preserve Venice from the threat of “outsiders.” Desdemona chooses to marry an “outsider” in both race and class. Does the play reveal her choice as foolish or not? To put it another way, does Shakespeare’s Othello perpetuate Elizabethan stereotypes of the ethnic outsider or does it undermine them? Does it warn daughters to marry within their race and class or does it serve to critique and undermine the world view and values of “insiders”?

Introduction to Psychology [Montana State University, Professor Wes Lynch]: In this course students were asked to do a 15 minute thinking piece as part of homework for each class period. TASK: Suppose you had a theory that laboratory rats fed a steady diet of beer and hot dogs could learn to find their way through a maze faster than rats fed a steady diet of squash, spinach, and broccoli. How would you design a scientific experiment to test this hypothesis? In your discussion, use the terms experimental group, control group, independent variable, dependent variable. Before beginning this task, review pages 17-23. Explain in your own words the difference between a variable interval schedule of reinforcement and a variable ratio schedule of reinforcement. Then explore this question: Is a skilled trout fisherman on a variable interval or a variable ratio schedule or reinforcement while fishing?

Nursing: Using layperson’s language, explain to a new insulin-dependent diabetic what is meant by the glycemic index and glycemic load of foods and why knowing about glycemic index and load is important.

A variation on this assignment: Explain the difference between “glycemic index” and “glycemic load” to different audiences using appropriate language: (a) A six year old child; (b) an adult with high school equivalency education; (c) an adult college graduate with strong science background.

Mechanical Engineering: For the design application we have been studying, your design team has proposed four alternative solutions: conventional steel roller bearings, ceramic bearings, air bearings, and magnetic bearings. As a team, make a list of the advantages and disadvantages of each solution. Then make a joint case.


Economics [Developed by Dr. Bridget Hiedemann, Seattle University]: TASK: As illustrated in Figures 1 and 3 in the Goldin and Rouse article, the sex composition of major symphony 1 orchestras changed dramatically over time. The increase in the percentage of female musicians corresponded with a change in audition policies. Goldin and Rouse use econometric analysis to assess the effect of “blind” auditions on female musicians. What do you see as the advantage of using econometrics over simply examining trends?

Finance [Developed by Dr: Peter Brous, Seattle U.]: The situation: One day you receive the following memo from your boss: Our current CEO, Mark Pigott, wants to examine the possibility of selling our Financial Services division. To help with our decision-planning, I need from you a memo that recommends for or against divestiture based on your analysis of key data. I am providing you with a set of reasonable assumptions regarding the future if we sell the assets of this division. Please compare the financial performance of Paccar if we divest this division versus the financial performance of Paccar if we do not (status quo plan). In addition to your memo, please provide me for each scenario (divestiture versus status quo) a set of Projected Financial Statements for the next 5 years (2000-2004). These statements should include income statements, balance sheets, common size income statements and balance sheets, statements of cash flow, and a complete set of ratios. Your task: Write the required memo and attach to it the complete statements requested by Richard Fox. In your memo itself, recommend a course of action regarding divestiture. Drawing on key data from your attachments, support your plan with reasons, explanations, and evidence. Your memo should be succinct (no more than two pages) and clearly written and argued.

Chemistry [From Jeffrey Kovac and Donna W. Shemood, ,Writing Across the Chemistrv Curriculum: An Instructor's Handbook. Prentice Hall: Upper Saddle River, NJ, 2001, p.681]:

• Using your understanding of the chemical bond, construct an argument either supporting or refuting the statement that van der Walls interactions are bonds.

• You are a principal investigator on a research project and receive the following e-mail from your chemical technician, Rogelio:

It’s been a long time since I took general chemistry. I only remember the concepts of the structure of an atom, electronegativity, and atomic radius. When I did the experiments, I noticed two tendencies: (1) The chemical reactivity of alkali metals increases down a group. (2) The chemical reactivity of halogens decreases down a group. Can you explain why these trends occur?

Reply to Rogilio's question (2-3 typed pages)


Features of an Effective Assignment An effective assignment handout should include the following:

Task. Describe the writing task: Explain what you want students to do in the paper. In most kinds of academic prose, writers address an unresolved or controversial problem or other kind of question that interests the reader. The assignment often describes the problem or question-at-issue or asks the student to formulate his or her own problem or question. Sometimes the task is given as thesis to support or attack, or the instructor might provide other specific instructions such as summarize and critique an article, write an analysis of a text, or write a technical report on an experiment.

Rhetorical Context: Specify a rhetorical context for the task by describing the writer’s role, the audience, and the format or genre:

• Role and audience. The student’s paper should bring something new, surprising, interesting, challenging, or useful to a specified audience. It is best to identify that audience in the assignment. (The instructor can then role play being that audience.) Typical audiences might be “students confused about X because they missed last week’s classes,” “newspaper readers undecided about issue X and considering alternative views,” or “participants in an undergraduate research conference interested in your analysis of X.” Also identify the writer’s role or purpose such as to inform, to explain, to critique, to analyze, or to persuade. Specifying a role and an audience creates a real world exigency for writing. When students think of their teacher as the audience, they are placed in the awkward position of addressing a person who knows more about the topic than they do and who has no reason to read the paper other than to grade it.

• Format and genre. Specify expected length, manuscript form, margins, spacing, and similar details. Wherever possible, connect these requirements to real-world genres such as an experimental report, scholarly article or conference paper, op-ed piece, memo, proposal, white paper, and so forth. Students need to learn the constraints and reader expectations associated with different genres.

Process. Specify expectations about the process to be followed: Specify a time schedule for completion of first drafts, peer review workshops, revisions, and final drafts. Consider asking students to save all drafts and submit these in a folder along with the final draft. Also consider asking students to include an “acknowledgements page,” in which they thank persons who discussed the topic with them or responded to drafts. (These requirements can discourage plagiarism and encourage students to treat writing as a process.)

Purpose. Consider explaining the purpose of the assignment: Often students appreciate why you have designed the assignment in a certain way. What concepts or skills is the assignment trying to teach? How does the assignment connect to overall learning goals for the course? How does it connect to the program’s competencies?

Criteria. Explain criteria for evaluation: Explain how the final product will be graded. How much weight will be given to ideas? To organization and development? To sentence style and readability? To grammatical correctness? To manuscript form and appearance? Ideally, provide a rubric or scoring guide for the assignment.


Questions for Peer-Reviewing an Assignment Handout 1. Is the assignment clear? How might a student misread the assignment and do something not

anticipated?

2. Does the assignment specify an audience and a role for the writer (that is, a purpose for writing to this audience)?

3. Are my grading criteria clear? Have I adequately explained them to students?

4. If you were a student, would you find the assignment interesting and challenging?

5. If you were a student, how difficult would this assignment be? How long do you think it would take?

6. To what extent does this assignment stimulate critical thinking? Does it cause students to wrestle with key concepts or key thinking skills in the course?

7. If the assignment is quite difficult, could it be preceded by a simpler “skill-building assignment” that would serve as scaffolding?

8. Is the purpose of the assignment clear? Does it seem to tie into my course goals? Would it seem like busy work to some students?

9. Are the mechanics of the assignment clear (due dates, expected length, single versus double spacing, typed versus handwritten, manuscript form, etc.?)

10. Is the process students should go through as explicit as possible?

11. Should I build any checkpoints built into the assignment to verify that students are on track? (e.g., submission of a thesis, title, and introduction? Mandatory conference? Annotated bibliography?)


Individual Activity: Sketching a Course’s Learning Outcomes and Major Assignments Take 15 minutes to do this activity. If you’re creating a course with a colleague who is present, work with her/him. (This activity is adapted from Walvoord and Anderson.)

1. List specifically what you want students to be able to do at the end of the course.

2. Select types of major tests or assignments that will measure whether students can accomplish those objectives.

3. Compose a course skeleton, beginning with what you want your students to learn and then sequencing your major assignments. Include just the major assignments.

4. Ask yourself the following questions:

a. Fit: is there a good fit between the learning I want and the assignments I have chosen?

b. Feasibility: Is this work load reasonable, strategically placed, and sustainable for me and for my students.


Using Rubrics

A Rationale for Using Rubrics? Rubrics take time and energy to construct, but most teachers who have tried them find them worthwhile because they:

• Are appreciated by and helpful to students, who in turn will appreciate your efforts to make your grading criteria clear and understandable;

• Induce students to work harder because students are provided a clear understanding about what constitutes excellence;

• Help students understand what you want them to focus on—e.g., organization, correctness, complexity, risk-taking, etc.;

• Save you time in the grading process and thereby help you provide timely feedback—and save your psychic and emotional energy;

• Help you explain what you care about, what you believe are the important competencies they need to learn;

• Help you stay on track with what you need to teach;

• Help students evaluate their own work and the work of their peers;

• Help students participate in their own learning, because they know what is important to aim for; students develop their critical thinking because they are empowered to self-assess and self-improve;

• Save you from having to explain your criteria to students after they’ve handed in their work;

• Help mentors, tutors, and fellow students provide focused, constructive feedback;

• Help team teachers or teaching assistants grade consistently;

• Help departmental colleagues communicate with each other about their courses’ content, standards, and criteria;

• Help you refine your teaching skills by allowing you to monitor what students are doing well and what they need more help with;

• Level the playing field by announcing to all students (not just the academically adept) what for you counts as excellent, satisfactory, and unsatisfactory;

• Form the basis of departmental assessment.


The Elements of a Rubric Task Description: Describe the task (as covered in the first part of this handout).

Scale: Describe how well or poorly each dimension of the task has been performed. They can be qualitative descriptors, numbers, or grades. A rubric that contains only the highest level of performance is a “scoring guide.” Some commonly used labels are:

• Sophisticated, competent, partly competent, not yet competent

• Exemplary, proficient, marginal, unacceptable

• Advanced, intermediate, intermediate novice

• Distinguished, proficient, intermediate, novice

• Accomplished, average, developing, beginning

• A, B, C, D, unacceptable

• 20 18 16, 14 12 10, 8 6 4, 2 0 Dimensions: Divide the task into distinct dimensions as simply and completely as possible. This is where you communicate what you care about—and what you want your students to attend to.

Description of Dimensions: For each dimension, describe what constitutes minimum performance for each scale. For a “scoring guide,” just describe the highest level of performance.

Task Scales

Dimensions

Descriptions of Dimensions

Three-level rubric with parts labeled.


Types of Rubrics Holistic Rubrics provide the criteria for an all-at-once assessment of a performance, without breaking it down into specific discrete, repeating dimensions. Holistic rubrics are especially appropriate for informal writing assignments that need to be graded quickly. They are also useful for program assessment, when it’s important to evaluate “discourse-based assessment instruments” (e.g., portfolios) efficiently and reliably. I’ve included five examples of holistic rubrics on pages 11–14.

Analytic Rubrics break down the dimensions of the performance into discrete, repeating categories (like the one on page 10). They can be further broken down into:

• Non-grid designs (pages 15–21)

• Scoring-Guides (with a single scales describing only the highest level of performance) (pages 22–24)

• Multi-level rubrics (the most difficult to create, providing complete information to students) (pages 25–30)

Rubric Examples Holistic Rubrics Holistic Scale for Grading Article Summaries (Bean)


Holistic Scale for Journals in Beginning Spanish Class (Walvoord and Anderson)

Holistic Scale for Grading Physics Microthemes (Bean)


Holistic Scale for Quantitative Literacy (Stevens and Levi)


Holistic Scale for Inquiry and Critical Thinking (Stevens and Levi)


Non-Grid Designs

Simple Scoring Guide: Argument Essay 1. Does the introduction effectively present the issue and the thesis, while evoking reader’s interest?

(10 points)

2. Are the ideas sufficiently complex? Are there good reasons in support of the thesis? Is the argument logical? (30 points)

3. Are opposing or alternative views adequately and fairly summarized? Are the responses to the opposing views effective? (20 points)

4. Is their appropriate and sufficient evidence? Is the argument well-developed with appropriate details? (20 points)

5. Is the essay well organized into a unified whole? Are there good transitions? Do paragraphs have topic sentences? (20 points)

6. Is language style effective? Is language well chosen for the intended audience? Is the tone appropriate? (1 0 points)

7. Are sentences well constructed? Is the paper carefully edited? (20 points)


Ethical Issues and Social Responsibility Portfolio (Stevens and Levi)


Diversity of Human Experience Portfolio (Stevens and Levi)


Economics: Analysis of a Proposed Law (Walvoord and Anderson)


Using Rubrics, page 19 Using Rubrics, page 19

Art History: Hypothetical Newspaper Article (Walvoord and Anderson)


Education Poster Presentation (Walvoord and Anderson)


Scoring Guides (Single Scale with Only Highest Level of Performance Described)

Scoring Guide for Changing Communities in Our City (Huba and Freed)


Leading a Class Discussion (Stevens and Levi)



Scoring Guide for Abstract for Master’s in Public Health Class

Outcome

Exe

mpl

ary

Ade

quat

e

Pre

sent

but

in

adeq

uate

Not

pre

sent

Conventions

Contains no more than 250 words.

If the study is experimental or quasi-experimental study, uses the standard structure (OMReC, etc.); otherwise, follows a logical and easy-to-comprehend structure.

25 %

of G

rade

Use conventions of grammar, mechanics, and style associated with American scientific English.

Information, Understanding, Structure

Subject: Makes clear early in the abstract what the article/study is about, general scope and boundaries of the work, including, as appropriate, time periods, specific populations, treatments, general problems.

Purpose: Makes clear what specific problem is addressed, issue is raised, or opportunity is described and seized upon. What question are they trying to answer?

Main point: makes clear what are the aims, hypotheses, and/or the one or two main findings.

OMReC, Objective-Methods-Results-Conclusions, (or similar) structure: includes the crucial information from each of the OMReC sections; if sections do not have headings, the abstract provides key words to locate the reader (e.g. for methods: “A such-and-such design was used….”).

Summarizes the study quickly and accurately; allows the reader to intelligently choose whether to read the article.

50%

of G

rade

Makes clear what happened: what did researchers did and how they did it, what they found and how, generally, they interpreted the results.

Style, Clarity, Concision

In general, the abstract is written clearly and concisely; little or no verbiage.

Actions: Most verbs convey the crucial action of a sentence; not weak “to be” verbs.

Cohesion and Coherence: Where appropriate, sentences begin with contextual information (short, simple, familiar, consistent) and end with new information (long, complex, exciting).

25%

of G

rade

Concision: words are deleted that 1) mean little or nothing, 2) repeat the meaning of other words, 3) are implied by other words. Where appropriate, phrases are replaced with words and negatives are changed to affirmatives.

Multi-Level Rubrics

Generic Multi-Level Rubric (Stevens and Levi)


Book Review (Stevens and Levi)


The Critical Thinking Rubric from Washington State (http://wsuctproject.wsu.edu/ctr.htm) 1) Identifies and summarizes the problem/question at issue (and/or the source's position). Emerging Mastering Does not identify and summarize the problem, is confused or identifies a different and inappropriate problem. Does not identify or is confused by the issue, or represents the issue inaccurately.

Identifies the main problem and subsidiary, embedded, or implicit aspects of the problem, and identifies them clearly, addressing their relationships to each other. Identifies not only the basics of the issue, but recognizes nuances of the issue.

2) Identifies and presents the STUDENT'S OWN hypothesis, perspective and position as it is important to the analysis of the issue. Emerging Mastering Addresses a single source or view of the argument and fails to clarify the established or presented position relative to one's own. Fails to establish other critical distinctions.

Identifies, appropriately, one's own position on the issue, drawing support from experience, and information not available from assigned sources.

3) Identifies and considers OTHER salient perspectives and positions that are important to the analysis. Emerging Mastering Deals only with a single perspective and fails to discuss other possible perspectives, especially those salient to the issue.

Addresses perspectives noted previously, and additional diverse perspectives drawn from outside information.

4) Identifies and assesses the key assumptions. Emerging Mastering Does not surface the assumptions and ethical issues that underlie the issue, or does so superficially.

Identifies and questions the validity of the assumptions and addresses the ethical dimensions that underlie the issue.

5) Identifies and assesses the quality of supporting data/evidence and provides additional data/evidence related to the issue. Emerging Mastering

Merely repeats information provided, taking it as truth, or denies evidence without adequate justification. Confuses associations and correlations with cause and effect. Does not distinguish between fact, opinion, and value judgments.

Examines the evidence and source of evidence; questions its accuracy, precision, relevance, completeness. Observes cause and effect and addresses existing or potential consequences. Clearly distinguishes between fact, opinion, & acknowledges value judgments.

6) Identifies and considers the influence of the context * on the issue. Emerging Mastering Discusses the problem only in egocentric or sociocentric terms. Does not present the problem as having connections to other contexts-cultural, political, etc.

Analyzes the issue with a clear sense of scope and context, including an assessment of the audience of the analysis. Considers other pertinent contexts.

7) Identifies and assesses conclusions, implications and consequences. Emerging Mastering Fails to identify conclusions, implications, and consequences of the issue or the key relationships between the other elements of the problem, such as context, implications, assumptions, or data and evidence.

Identifies and discusses conclusions, implications, and consequences considering context, assumptions, data, and evidence. Objectively reflects upon the their own assertions.


Summary/Strong Response in First-Year Writing Course (Bean) Title 5 4 3 2 1 0 Academic title forecasts content Has title, but doesn’t accurately forecast content Either has no title or a course-based

title (“Summary/ Response”)

Introduction 10 9 8 7 6 5 4 3 2 1 0 Hooks readers interest; provides background and context; states question that Rauch and rest of paper will address

Provides some background and context, but less clearly and effectively than an 8+ paper; may assume reader already knows the assignment; question is fuzzy

Introduction lacks focus; doesn’t try to hook reader or provide context; doesn’t identify the question at issue

Summary of Rauch’s article 20 18 16 14 12 10 8 6 4 2 0 Provides concise condensation of Rauch’s argument; well balanced, accurate, clear, and complete; good use of attributive tags and short quotations; identifies original source; no wasted words

Shows that writer understands the article but not as helpful to new reader as a 16+; may omit or misrepresent key ideas; may be wordy or unclear in places; may summarize sequence of topics rather than summarize argument; may be weak in attribution

May be too unclear or inaccurate to help new reader understand the article; may fail to use attributive tags so that reader can’t distinguish article from writer’s own ideas

Rhetorical analysis 20 18 16 14 12 10 8 6 4 2 0 Uses rhetorical principles from AGBW (Ch 4 and 6) to analyze Rauch’s angle of vision and identify and critique his rhetorical strategies for making the argument persuasive

Shows general understanding of rhetorical principles and applies them to Rauch but with less insight, clarity, and development than a 16+

Doesn’t clearly apply principles of rhetorical analysis to Rauch’s argument; too fuzzy, unclear, or undeveloped to be effective for readers

Writer’s own views on GMOs in conversation with Rauch and other writers in coursepack 40 36 32 28 24 20 16 12 8 4 0 Shows engaged response to Rauch’s ideas; draws on insights from other articles in packet (cites at least two sources other than Rauch); shows strong wrestling with disagreements among authors; surprises reader with points and details; high level of critical thinking

Shows engaged response to Rauch but is not as clear, well-developed, complex, or surprising as a 32+; shows less wrestling with questions and ideas; may not use required sources as effectively; may tend toward a surpriseless “good points” and “bad points” catalogue

Ideas may be either very thin or unclear; may not include required sources; may be a series of opinions (“I agree with this; I disagree with that”) rather than arguments; inadequate wallowing in complexity

Closed-form organization 20 18 16 14 12 10 8 6 4 2 0 Has clear thesis statement that summarizes writer’s strong response; follows reader expectation theory: points-first structure; good transitions; old/new contract; unity and coherence

Has thesis statement but may be more fuzzy or incomplete than in a 16+; may cause some reader confusion about structure; generally follows reader expectation theory but may fail in places to have clear transitions, use point-first structure, or follow old/new contract

Thesis may be missing or very unclear or fuzzy; confusing structure; may frequently fail to follow points-first structure, use transitions, or follow old/new contract; lacks unity and coherence

Clarity, gracefulness, ease of reading 20 18 16 14 12 10 8 6 4 2 0 Clear, grammatical sentences; clear pronoun reference; fluent sentence structure; prose is concise, graceful, easy to read; reader can hear writer’s voice

Occasionally unclear sentences or confusing pronoun reference; prose may be occasionally wordy or convoluted

Frequent patches of unclear sentences; sentence structure may be non-grammatical; reader often gets lost at the sentence level

Manuscript form 10 9 8 7 6 5 4 3 2 1 0 Follows APA manuscript form; uses APA parenthetical citation for page numbers; has “References” page

Has some features of APA form but omits others

Makes no effort to follow APA form

Deductions for rule-based errors in grammar punctuation, usage, or spelling POSITIVE ETHOS ANNOYING NOISE ....... ERRORS DESTROY ETHOS +5 0 -5 -8 -10 ........ -12 -15 -20 -25


Multi-Level Rubric for Formal Oral Performance in Graduate Program (Huba and Freed)


Constructing Rubrics Stevens and Levi identify “four key stages in constructing rubrics”:

1. Reflecting: Take time to reflect on what you want from the students, why you believe the assignment will build the skills and knowledge you think they should have, and generally what you expect from them.

2. Listing: Make a list of the learning objectives you hope the assignment will help them attain.

3. Grouping and Labeling: Organize the stage-2 list into categories (which will become your “dimensions”).

4. Application: Build the rubric.

Student Collaboration and Student-Writing Artifacts Consider including your students in the assessment process by bringing student work to the table. When you and your class are examining a concrete student-writing artifact, they can help you refine the quality of the rubric, and you can help them understand how your rubric translates to the kind of writing they’re doing. If you treat writing assignments as a process—a sequence of assignments, multiple drafts and revision, peer review, etc.—you can enlist the help of your students, who will often be more adept than you at finding a rubric’s gaps and the weak spots.

For instance, you can copy an early draft of student work (or student writing from a previous semester) and lead your class in investigating how well the draft meets the criteria of the rubric. Along the way, you can investigate the rubric itself, determining how clearly it communicates your standards, and with your students you can make the rubric better. It also gives you an opportunity to articulate why you’ve chosen your criteria.

We’ve followed this process in classes with first-year students, upper-level undergraduates, and advanced Ph.D. candidates. For advanced students especially, they tend to script standards that are more rigorous than those we’ve created. Students will become more invested in the process of assessment, more trusting, and more knowledgeable. And they’ll perform better.

Suggestions for Making the Process Work Whether bringing in a current student’s draft or polished work from a previous semester, we have found that “A” or ideal papers are for this process not ideal. Work in the “B” range invites students to engage in serious critique as they’ll have suggestions for improvement.

We follow a process something like this: 1. Encourage thoughtful and serious criticism by reminding your students (or asking them to

imagine) that the person/group who created the work at hand is present in the class. 2. Read the paper silently or aloud (aloud is better if you have the time), or have them read

it before class. 3. Ask first “what works?” “What has this writer done that is particularly effective,

thoughtful, creative, insightful, etc.?” Ask whether the rubric spells out clearly what the writer has done well.

4. Then ask for helpful criticism for improving the paper. Again, go back and forth between the work at hand (what advice you’d give the writer) and the rubric (how it might be amended or added to).

5. Alter your rubric to reflect your students’ input.


A Metarubric (Stevens and Levi)


Classroom Assessment Techniques (CATs)3

Classroom Assessment Techniques (CATs) can provide instant feedback about student learning. The basic idea is this: get feedback about the class and student learning during the semester, while you can still do something about it.

There are three steps to implementing a CAT: 1. Decide which CAT will provide the information you need. 2. Implement the CAT. 3. Respond to the CAT, sharing what you found with the class and, when appropriate,

making changes.

One-Minute Papers Get a qualitative response for determining how well students understand the material or assessing their attitudes toward the class or your instruction, have students write a “minute paper” (for some activities, they’ll need more time). You can do these before class to assess how well prepared students are or what’s puzzling them, in the middle of class to allow students to gather their thoughts, or at the end of class.

Some topics: • “What was the most important thing you learned during today’s/last session’s class?” • “What important questions remain unanswered?” • “What was the muddiest point? “ • “Summarize such-and-such concept in a single sentence.” • “Paraphrase (for a specified audience) such-and-such concept.” • Application Cards: Hand out cards and ask students to apply a principle to a real-world

situation. Or you can solicit feedback about how they’re responding to your teaching methods:

• “How was the pace of today’s class?” • “Were the examples clear?” • “Were the topics presented sufficiently?” • “What other activities could we do that would help you learn the material?”

Email Minute This is a modification of the minute paper that utilizes technology to create a less threatening manner of communication. It can be approached in two ways:

• For the last two to three minutes of class, professor asks students to respond anonymously to two questions. The professor then reads through comments and looks for similar concerns, themes, or comments. Then, a summary of the themes is sent to the class list via email, or

• Distribute two questions at the end of class and ask all students to respond via email

3 From Huba & Freed and Angelo & Cross.


Benefits to the email-minute CAT include: • Gets students acquainted with using email to communicate with professors instead of just

friends or family • Allows time for a reflection • Extends office hours by allowing more time for communication between professor and

student • One message can be sent simultaneously by using a distribution list • Offers an important “teaching moment” in terms of discussing audience, tone, and

construction of ethos

Direct Paraphrasing (or, the “Dear Aunt Hepsy” letter) Basically, this CAT asks students to write to a specific audience (e.g., a lay audience like Uncle George or Aunt Hepsy) explaining a particular point or theme that was part of a lecture, article, or class conversation. This encourages students to eschew or embrace technical terms and figure out how to convey what they’ve learned in their own words to different people with different perspectives. In these responses you might look for:

• Accuracy • Relevancy and appropriateness for audience • Effectiveness in satisfying the assignment

Application Cards This CAT asks students to write down at least one way an important theory, concept, or principle could be applicable outside the classroom. We like this one because it anticipates the often heard complaint, “When will I need to know this for anything else in my life?” Share responses the next class period as a way to initiate discussion.

Faculty Concerns All of these classroom assessment techniques ask that teachers be open to student advice and feedback throughout the semester. We often feel pretty comfortable giving our students feedback about their learning (aka “grades”) but less comfortable with our students reciprocating. You might consider having a discussion with your class about the difference between “feedback” and “criticism,” asking for anonymous student feedback, focusing questions/responses on behaviors and processes rather than personalities, and lastly, remembering to consciously use the data for improvement. Some important questions to consider before implementing CATS in your classroom include:

• How do I feel about accepting suggestions for change from students? • How do I typically react when students propose changes? • How can I develop an approach for discussing assessment results with students that is

comfortable to me? • How can I ensure students that I am open to listening to their suggestions?

Other Rapid Assessment Techniques Consult Huba and Freed (chapter 5) for a wonderful and quick collection of instant assessment techniques. Consult Angelo and Cross for a much more detailed account.


Sources These are the books we used to create this handout.

Angelo, Thomas A. and K. Patricia Cross. Classroom Assessment Techniques: A Handbook for College Teachers.

Bean, John C. Engaging Ideas: The Professor’s Guide to Integrating Writing, Critical Thinking, and Active Learning in the Classroom.

Diamond, Robert M. Designing & Assessing Courses and & Curricula: A Practical Guide.

Huba, Mary E. and Jann E Freed. Learner-Centered Assessment on College Campuses: Shifting the Focus from Teaching to Learning.

Stevens, Dannelle D. and Antonia J. Levi. Introduction to Rubrics: An Assessment Tool to Save Grading Time, Convey Effective Feedback, and Promote Student Learning.

Walvoord, Barbara E. and Virginia Johnson Anderson. Effective Grading: A Tool for Learning and Assessment.

Walvoord, Barbara E. Assessment Clear and Simple: A Practical Guide for Institutions, Departments, and General Education.


Assigning Writing and Using Rubrics to Get Student …wac/History-Legacy/CurriculumResources/...•...

Documents

Transcript of Assigning Writing and Using Rubrics to Get Student …wac/History-Legacy/CurriculumResources/...•...