CONSTRUCTING ESSAY QUESTIONS

MEASURING COMPLEX ACHIEVEMENT with ESSAY QUESTIONS

Essay

a short literary composition on a particular theme

or subject, usually in prose, and generally analytic, speculative, or interpretative.

What’s the diff?

Objective Test items measure a variety of simple learning outcomes ex. recall of memorized information

Interpretive exercises measures complex achievement.

Higher mental processes Understanding Reasoning

Critical thinking Scientific thinking Creative thinking

Problem solving

Essay Question:

Especially useful for measuring those aspects of complex achievement that cannot be measured well by more objective means:

  the ability to supply rather than identify interpretations and applications of data

  the ability to organize, integrate, and express ideas in a general attack on a problem.

The BIG IDEA OF ESSAY IS...

INTEGRATE BIG IDEAS.

Forms of Essay

Restricted – Response Essay Question Usually limits both the content and response. The content is usually restricted by the scope and the topic to be discussed.

It is most useful for measuring learning outcomes requiring the interpretation and application of data in a specific area.

Example: ON MEASURING THE ABILITY TO INTERPRET AND APPLY DATA

1) Describe TWO situations that demonstrate the law of Supply and Demand.

2) Explain how the RH bill is both an bane and a boon in the country.

Extended Response Essay   Allows students to select any factual information that

they think is pertinent, to organize the answer in accordance with their best judgment, and to integrate and evaluate ideas they deem appropriate.

  However, too creative expression and other higher-‐order skills makes this type inefficient for measuring more specific learning outcomes.

Example:

Compare media’s freedom of speech in the 70s to the present day. Cite examples.

  The restricted-response question usually limits both the content and the response. The content is usually restricted by the scope of the topic to be discussed. Limitations on the form of response are generally indicated in the question.

Restricted-Response Essay Questions

  Examples   1. Describe two situations that

demostrate the application of the law of supply and demand. Do not use those examples discussed in class.

  2. Why is the barometer one of the most useful instreuments for forecasting weather? Answer in a brief paragraph.

Restricted-Response Essay Questions

  The extended-response question or assignment allows students to select any factual information that they think is pertinent, to organize the answer in accordance with their best judgment, and to integrate and evaluate ideas as they deem appropriate.

Extended-Response Essays

  Advantage:   - ability to analyze problems, organize

their ideas, describe in own words, and/or develop coherent argument.

  Disadvantage:   - inefficient for measuring more

specific learning outcomes and introduces scoring difficulties.


  Example   1. Compare developments in

international relations in the administration of President Gloria Macapagal-Arroyo and President Ejercito Estrada. Cite examples when possible.


  Example   2. Evaluate the significance of the sea

captain’s pursuit of the white whale in Moby Dick.

  3. Describe the influence of Mendel’s law of heredity on the development of biology as a science.


  Restricted-Response

  VS

  Extended-Response

Learning Outcomes Measured

  Restricted-response is similar to objective interpretive exercise in measuring complex learning outcome.

  Difference:

  Objective Interpretive –select answer   Restricted –supply answer

•  Explain cause-and-effect relationships •  Describe applications of principles •  Present relevant arguments •  Formulate tenable hypotheses •  Formulate valid conclusions •  State necessary assumptions •  Describe the limitations of data •  Explain methods and procedure

Restricted-response essay measures ability to:

  Extended-response essay assessments measure more general learning outcome.

•  Produce, organize, and express ideas •  Integrate learning in different areas •  Create original forms (e.g., designing an

experiment) •  Summarize (e.g., writing a summary of a story) •  Construct creative stories (e.g., narrative essays) •  Explain concepts or principles (e.g., expository

essay) •  Persuade a reader (e.g., persuasive essay)

Extended-response essay measures ability to:

Restricted- Response Essay

•  Asks the learner to demonstrate an understanding of the given material

•  Measures comprehension •  More structured, more specific •  Useful for measuring learning

outcomes requiring the interpretation and application of data in a specific area

Extended-Response Essay

•  Students use judgment; integrates and evaluates ideas they deem appropriate

•  Emphasizes integration, organization, and originality

•  Creative process, places more value on higher-order skills

•  Measures complex learning outcomes that cannot be measured by other means

•  Emphasizes the integration and application of thinking and problem-solving skills

•  Enables the direct evaluation of writing skills

•  Easy Construction •  Direct contribution to student learning

Advantages of Essay Questions

•  Unreliability of scoring   X clear identification of learning

outcomes being measured   X well-defined scoring rubrics •  Amount of time required for scoring the

responses •  Limited sampling of content that essay

questions provide

Limitations of Essay Questions

  1. Restrict the use of essay questions to those learning outcomes that cannot be measured satisfactory by objective items.

2.  Construct questions that will call forth the skills specified in the learning standards.

  3. Phrase the question so that the student’s task is clearly indicated.

Suggestions for constructing essay questions

  4. Indicate an appropriate time limit for each question.

  5. Avoid the use of optional questions.

•  Clear specification of criteria leads to an improved Reliability and Validity of Assessment

•  Informing the students of these criteria before administering the exam will give them an idea what to expect and what to do.

Scoring Criteria

  “Rubrics is a way of describing evaluation criteria based on the expected outcomes and performances of students.” (Ebert-May, 1997)

    Anchor Responses -- most clearly

correspond to the levels of scoring rubric.

Scoring Rubrics (grading tools)

•  Must be descriptive (clear) rather than judgmental in nature

•  Reliability, Comparability and Fairness of Scores

Scoring Rubrics (grading tools)

  Poor:   “Excellent”

  Better:   “Writing is clear and thoughts are

complete.”

Restricted-response essay questions   1. Why is the barometer one of the most

useful instruments for forecasting weather? Answer in a brief paragraph.

  2. Describe two situations that demonstrate the Application of the law of supply and demand. Do not use those examples discussed in class.

Restricted-response essay questions

  Make a list of possible answers

  Full credit   Partial credit

  1. What is geography? Don’t use the definition discussed in class.

  “Study of Earth as the home of man.”

  study of the surface of the earth and its relation to the activities of man.

  study of man and earth

  study dealing with man and his relationship with his natural environment

Extended-response essay questions

  1. Evaluate the significance of the sea captain’s pursuit of the white whale in the Philippine deep.

  2. Describe the influence of Mendel’s laws of heredity on the development of biology as a science.

Analytic Scoring Rubrics

•  Separate scores for certain characteristics gives students clearer feedback on their strengths and weaknesses.

•  Finer distinctions or specifications are useful especially in scoring expository essays and descriptive summaries.


  State of Oregon statewide writing assessment rubrics:

1.  Ideas 2.  Organization 3.  Voice 4.  Word Choice 5.  Sentence Fluency 6.  Conventions 7.  Content


  Narrative Essay rubrics developed by Wilf, Gearhart, Quellmalz and Whittaker (1992)

1.  Theme 2.  Character 3.  Setting 4.  Plot 5.  Communication

Holistic Scoring Rubrics

•  Can be constructed more rapidly

•  Can be used to score a set of essay responses more rapidly than analytic scoring rubrics.

•  Doesn’t provide accurate feedback, but teachers could write comments.

http://www.azed.gov/wpcontent/uploads/PDF/OfficialScoringGuide6ptHolistic_6_10.pdf

 1. Prepare an outline of the expected answer in advance.

Suggestions for scoring essay questions

  Example: Rubric for oral presentation (http://www.cbd.int/ibd/2008/resources/teachers/appendix3.shtml)

2. Use a scoring rubric appropriate to the type of question used.

  3. Control the influence of irrelevant factors such as:

  a. Legibility of handwriting   b. Spelling   c. Sentence structure   d. Punctuation   e. Neatness

  4. Evaluate answers without looking at the student’s name

  5. Obtain two or more independent ratings when important decisions are to be made.

 

1. Prepare an outline of the expected answer in advance.

Suggestions for scoring essay questions

2. Use a scoring rubric appropriate to the type of question used.

3. Control the influence of irrelevant factors such as:

a. Legibility of handwriting b. Spelling c. Sentence structure d. Punctuation e. Neatness

4. Evaluate all responses to one question before going on to the next one.

5. Evaluate answers without looking at the student’s name.

6. Obtain two or more independent ratings when important decisions are to be made.

CONSTRUCTING ESSAY QUESTIONS

Documents

Transcript of CONSTRUCTING ESSAY QUESTIONS