Efficacy of Pretest/Posttest Instruments

Technology Instructional AssessmentBeth Teuteberg Weed

SUMMATIVE ASSESSMENT FORMATIVE ASSESSMENT

Administered at the end of a unit of instruction or at a predetermined time (semester/testing week)

Administered before or during a unit of instruction

Typically results in a grade Is not graded (but may be acted upon by students and teachers)

Indicates learning of material, but does not change the course of instruction for tested population

Indicates prior knowledge, can change the course of instruction or learning

DEFINITIONS

Formative Assessment In The Classroom

• Self and Peer (Clear goals essential)– Review/grade each other’s work– Reflect on learning ****Research slide from Richard J. Shavelson

– Informal• Observations Science Notebooks• Group talk Interviews• Assessment Conversations

• Teacher– Formal

• Curriculum provided assessments• Teacher provided assessments

Clinical Settings◦ Pretest given to establish baseline◦ Intervention administered (control group/test

group)◦ Post test administered◦ Compare results

Educational Settings using Pretest PosttestPretest givenInstruction Post test given after instructionCompare results

Pretest Posttest Research Studies

Audience •Fourth year surgical students-9-week clinicals, lessons•Given pretest on semester concepts

Methods •Some groups of students were given pre-tests to study •Some groups also given additional online information•One group received no additional materials

Premise •Students will use the pretest to as a self-study tool•Students will also be motivated to use additional online materials

RESEARCH CASE ONE

On the post-test all groups of students made gains

No significant difference between those with access to study materials and not

The majority of students didn’t use pretest or materials

In this case, the pretest/posttest was used as a summative assessment

Students saw the materials as not relevant to “the exam”

Teachers wasted time on creating unused materials

ResultsImplications

Audience •District sixth grade students studying science unit on Forces and Motion•Two research groups using different science methods also part of study

Methods• Teacher was not involved in making test,

administering test, results• Pretest given to one sixth of the students• Pretest given before unit –Posttest at unit

completion 4-6 months later

Premise •“Pretest affect” would be shown for students taking test•Students might learn from the test, scaffold ideas, or recall test items

RESEARCH CASE TWO

No pre-test affect No statistical

difference between the students who took pretest and those who didn’t

Consistent with other research

Students with more prior knowledge did better on posttest

In this case, the pretest/posttest was used as a summative assessment

There was no link between pretest information and teachers, students, and learners

Missed opportunity

ResultsImplications

Audience •Undergrad political science students•Taking four consecutive political science courses

Methods

• Four Pretests created based upon course syllabi, proposed goals, course objectives and administered on Blackboard

• Posttest given after 15 weeks of instruction of each course

• Consecutive pretests/posttests contained questions from previous courses

Premise •Pretest/Posttest would be an effective tool to measure learning of course content in individual course, and over a period of consecutive courses

RESEARCH CASE THREE

On the post-tests all students made gains

On following pretests/posttests retention of previous coursework was evident

Who created the test?

Did instructor see pretest results?

Were results made available to students?

Were pretests used as a teaching tool?

ResultsQuestions

QUESTION

Can pretest/posttest be used as a valid assessment tool to measure knowledge or learning?

ANSWER

Yes, the pretest/posttest tool can be used as an effective summative assessment tool as the research reveals.

RESEARCH SUMMARY

Pretest information used as Formative assessment tool!

According to Black and William “Improving formative assessment can raise standards”

Inside the Black Box: Raising Standards Through Classroom Assessment-1998

Working Inside the Black Box: Assessment for Learning-2004

We can do better than that!

You do not need to use a posttest Pretests can have a wide or narrow focus Wide focus(over the course of a unit) Can be a drawing at the beginning of class

to demonstrate a concept Can be a mini-test of 3 questions reviewing

previous lesson Can be an entry in a science notebook or

journal

Pretest Forms (Formative)

Start with the end in mind (objectives) Question for depth of knowledge and

applications, not just vocabulary Include 1 or 2 essay or drawings so

students can demonstrate either depth of prior knowledge or to show gains in posttest

Add a self-assessment toolStudents can relate confidence- level of their responses

Designing Pretest (Formative)

1. Document results. Note high areas of prior knowledge

2. Note weak areas, and note misconceptions

3. Note confidence indicated in student responses

4. Match instruction to meet needs of group as a whole and for individuals.

Assessing Pretests(Formative)

Pretest to ImproveLearning (Formative)Review the pretest as a class---Use it as an overview of upcoming learningEncourage the students to think about what they know and don’t understand throughout the unit—demonstrate it!

METACOGNITIONEncourage working in groups or pairs “talk out” their thinkingDuring instruction and discussion ask open ended questions “Why do you think that?” Encourage multiple responses and respectful disagreements that use credible support

ASSESSMENT TOOLChecklist

Does your aquarium picture include… CHECK-OFF

ElodeaCabombaAccurate water levelGravel2 Snails2 Female guppies2 Male guppies

Labels for all items

Good for younger children

Can be used for drawings or models

Encourages students to focus on elements

of importance

CHECKLIST

Project Summative Assessment (given/explained before

assignment)

Did you learn anything?

No!I’m lost!

I need help to improve

Yes!I’m

confident

Black, P. and Wiliam, D. (1998). Inside the black box: Raising standards through classroom assessment. Phi Delta Kappan, 80(2):139-148.

Black, P., Harrison, C., Lee, C., Marshall, B., & Wiliam, D. (2004). Working Inside the Black Box: Assessment for Learning in the Classroom.  Phi Delta Kappan, 86, 1. p.8. 

Shavelson, R. (2005). On Science Inquiry and Formative AssessmentKeynote Address NSTA Science Assessment ConferenceResearch & Practical Approaches for Classroom Teachers, School Administrators, & School Districts.

RESOURCES