Part #3 © 2014 Rollant Concepts, Inc.2 Assembling a Test #

Part #3

Part #3 2014 Rollant Concepts, Inc.

2

2014 Rollant Concepts, Inc.Assembling a Test

#

2014 Rollant Concepts, Inc.Assembling a TestFirst page = easier questions

Test should progress from easier to more difficult (Nitko, 2004)#

2014 Rollant Concepts, Inc.Assembling a TestUnnecessary to group content topics

#

2014 Rollant Concepts, Inc.Assembling a TestAvoid the correct answer to be the same letter for more than 3 or 4 questions in a row

Vary the difficulty level within a content area#

2014 Rollant Concepts, Inc.Assembling a TestEdit w/ having a colleague proofread for punctuation, grammar, reading level, makes sense, clarity and content

Ensure stem and options are on same page#

2014 Rollant Concepts, Inc.Assembling a TestPrint on one side of the page if two sided use 2 staples like a book

Provide margins for making notes these can be reviewed for areas that students may have fuzzy thinking

#

2014 Rollant Concepts, Inc.Are you tense?

#

10 2014 Rollant Concepts, Inc.



Test ConstructionCurriculum = 100, 200, 300, 400

Within a Course Formative Evaluation =Summative Evaluation =Exam Matrix for curriculum/courseExam Blueprint Revisions Verbs for Objectives Types of questions = structure/level

Decision Points

2014 Rollant Concepts, Inc.13

Test Analysis Thought for the day

There are three kinds of lies: lies, damned lies and statistics. Mark Twain

Statistical findings in isolation are meaningless. Informed interpretation is the goal without distortion.14 2014 Rollant Concepts, Inc.

Where to Analyze?#1 Exam overall [KR-20]

#2 Items for Correct Responses

#3 Items for discrimination

#4 Option A as incorrect select

#5 OPTIONAL Option discrimination15 2014 Rollant Concepts, Inc.

Summary Steps to AnalysisExam desired - KR-20 of 0.65 to 0.85 2. & 3. Items** mark & review ? that haveP-value of < 0.30 or > 0.80Point Biserial Index [PBI]If negative check first [may be keyed incorrectly]Then if time review items with 0.80STEP #3 Discrimination Item discrimination ratio [IDR] = % of upper 27% minus the lower 27% who answered item correctly Point Biserial Index [PBI] = considers the total group variance



Step #2 Step #3


Step #3

What to Analyze in Options? STEP #4Options Which were selected by test-taker?Distractor analysisResponse frequencies & patterns = # number of students who selected each option Who selects option A as a pattern for incorrect items?



Step #4

What to Analyze in Options? STEP #5Options Which were selected by test-taker?Distractor analysis - Point biserial for options = indicated the average performance of those selecting the option Positive [+] correlates w/ correct option Negative [-] correlates w/ other options



Step #5

Summary 3 Steps to AnalysisExam desired - KR-20 of 0.65 to 0.85 Items** mark & review ? that haveP-value of < 0.30 or > 0.80Point Biserial Index [PBI]negative first [check for being correctly keyed]Then

Part #3 © 2014 Rollant Concepts, Inc.2 Assembling a Test #

Documents

Transcript of Part #3 © 2014 Rollant Concepts, Inc.2 Assembling a Test #