Why Scale -- 1
description
Transcript of Why Scale -- 1
![Page 1: Why Scale -- 1](https://reader035.fdocuments.net/reader035/viewer/2022062501/568163f5550346895dd582c7/html5/thumbnails/1.jpg)
Why Scale -- 1
• Summarising data–Allows description of developing
competence• Construct validation
–Dealing with many items• rotated test forms
– check how reasonable it is to summarise data (through sums, or weighted sums)
![Page 2: Why Scale -- 1](https://reader035.fdocuments.net/reader035/viewer/2022062501/568163f5550346895dd582c7/html5/thumbnails/2.jpg)
What do we want to achieve in our measurement?
Locate students on a line of developing proficiency that describe what they know and can do.
================================So, we need to make sure that• Our measures are accurate (reliability);• Our measures are indeed tapping into the
skills we set out to measure (validity);• Our measures are “invariant” even if
different tests are used.
![Page 3: Why Scale -- 1](https://reader035.fdocuments.net/reader035/viewer/2022062501/568163f5550346895dd582c7/html5/thumbnails/3.jpg)
Properties of an Ideal Approach
• Scores we obtained are meaningful.
Ann Bill Cath
What can each of these students do? Scores are independent of the sample of items
used If a different set of items are used, we will get the
same results.
![Page 4: Why Scale -- 1](https://reader035.fdocuments.net/reader035/viewer/2022062501/568163f5550346895dd582c7/html5/thumbnails/4.jpg)
Using Raw Scores?
• Can raw scores provide the properties of an ideal measurement?
• Distances between differences in scores are not easily interpretable.
• Difficult to link item scores to person scores.
![Page 5: Why Scale -- 1](https://reader035.fdocuments.net/reader035/viewer/2022062501/568163f5550346895dd582c7/html5/thumbnails/5.jpg)
Equating raw scores - 2
0 100%Score on the easy test
Scor
e on
the
hard
test
100%A
A
A
BB B
C
C C
![Page 6: Why Scale -- 1](https://reader035.fdocuments.net/reader035/viewer/2022062501/568163f5550346895dd582c7/html5/thumbnails/6.jpg)
Link Raw Scores on Items and Persons
single digit addition
Task Difficulties
multi-step arithmetic
word problems
arithmetic with vulgar fractions
25%
50%
70%
90%?
Object Scores
?
?
?
90%
70%
50%
25%
![Page 7: Why Scale -- 1](https://reader035.fdocuments.net/reader035/viewer/2022062501/568163f5550346895dd582c7/html5/thumbnails/7.jpg)
Item Response Theory (IRT)
• Item response theory helps us address the shortcomings of raw scores– If item response data fit and IRT (Rasch)
model, measurement is at its most powerful level.• Person abilities and item difficulties are calibrated
on the same scale.• Meanings can be constructed to describe scores• Student scores are independent of the particular set
of items in the test.– IRT provides tools to assess the extent to which
good measurement properties are achieved.
![Page 8: Why Scale -- 1](https://reader035.fdocuments.net/reader035/viewer/2022062501/568163f5550346895dd582c7/html5/thumbnails/8.jpg)
IRT
• IRT models give the probability of success of a person on items.
• IRT models are not deterministic, but probablistic.
• Given the item difficulty and person ability, one can compute the probability of success for each person on each item.
![Page 9: Why Scale -- 1](https://reader035.fdocuments.net/reader035/viewer/2022062501/568163f5550346895dd582c7/html5/thumbnails/9.jpg)
Building a Model
Probability of Success
Very low achievement Very high achievement
1.0
0.0
0.5
![Page 10: Why Scale -- 1](https://reader035.fdocuments.net/reader035/viewer/2022062501/568163f5550346895dd582c7/html5/thumbnails/10.jpg)
Imagine a middle difficulty task
Probability of Success
Very low achievement Very high achievement
1.0
0.0
0.5
![Page 11: Why Scale -- 1](https://reader035.fdocuments.net/reader035/viewer/2022062501/568163f5550346895dd582c7/html5/thumbnails/11.jpg)
Item Characteristic Curve
Probability of Success
Very low achievement Very high achievement
1.0
0.0
0.5
![Page 12: Why Scale -- 1](https://reader035.fdocuments.net/reader035/viewer/2022062501/568163f5550346895dd582c7/html5/thumbnails/12.jpg)
Item Difficulty -- 1
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
-4 -3 -2 -1 0 1 2 3 4
![Page 13: Why Scale -- 1](https://reader035.fdocuments.net/reader035/viewer/2022062501/568163f5550346895dd582c7/html5/thumbnails/13.jpg)
Variation in item difficulty
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
-4 -3 -2 -1 0 1 2 3 41 23
![Page 14: Why Scale -- 1](https://reader035.fdocuments.net/reader035/viewer/2022062501/568163f5550346895dd582c7/html5/thumbnails/14.jpg)
Variation in item difficulty
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
-4 -3 -2 -1 0 1 2 3 4
![Page 15: Why Scale -- 1](https://reader035.fdocuments.net/reader035/viewer/2022062501/568163f5550346895dd582c7/html5/thumbnails/15.jpg)
Estimating Student Ability
10 34 76 39 67 29 3 7 89 5 56 40 2 8 11 13 27 66 77 64 4 9 1 45 46 14 35 21 23 81 75 6 12
![Page 16: Why Scale -- 1](https://reader035.fdocuments.net/reader035/viewer/2022062501/568163f5550346895dd582c7/html5/thumbnails/16.jpg)
Estimating Student Ability
10 34 76 39 67 29 3 7 89 5 56 40 2 8 11 13 27 66 77 64 4 9 1 45 46 14 35 21 23 81 75 6 12
![Page 17: Why Scale -- 1](https://reader035.fdocuments.net/reader035/viewer/2022062501/568163f5550346895dd582c7/html5/thumbnails/17.jpg)
Estimating Student Ability
10 34 76 39 67 29 3 7 89 5 56 40 2 8 11 13 27 66 77 64 4 9 1 45 46 14 35 21 23 81 75 6 12
![Page 18: Why Scale -- 1](https://reader035.fdocuments.net/reader035/viewer/2022062501/568163f5550346895dd582c7/html5/thumbnails/18.jpg)
Estimating Student Ability
10 34 76 39 67 29 3 7 89 5 56 40 2 8 11 13 27 66 77 64 4 9 1 45 46 14 35 21 23 81 75 6 12
![Page 19: Why Scale -- 1](https://reader035.fdocuments.net/reader035/viewer/2022062501/568163f5550346895dd582c7/html5/thumbnails/19.jpg)
Estimating Student Ability
10 34 76 39 67 29 3 7 89 5 56 40 2 8 11 13 27 66 77 64 4 9 1 45 46 14 35 21 23 81 75 6 12
![Page 20: Why Scale -- 1](https://reader035.fdocuments.net/reader035/viewer/2022062501/568163f5550346895dd582c7/html5/thumbnails/20.jpg)
3 | | | | X| | X| | XX| | 2 XX| |9 22 XXX| | XXX| |6 16 XXXXX| |8 11 27 29 1 XXXXX| | XXXXXXX|* |31 XXXXXXX|* |2 30 XXXXXXXXX|* * * |13 XXXXXXXXXX|* * * * * |19 0 XXXXXXX|* * * * * * |5 32 XXXXXXXX|* * * * * |7 15 28 XXXXXXX|* |4 14 21 XXXXXXXX|* * |3 17 20 23 XXXXXXXXX| |10 18 24 -1 XXXXXX| | XXXX|* |1 XXXX| | XX| |12 26 -2 XXX| |25 XX| | X| | X| | X| | -3 X| |
![Page 21: Why Scale -- 1](https://reader035.fdocuments.net/reader035/viewer/2022062501/568163f5550346895dd582c7/html5/thumbnails/21.jpg)
3 | | | | X| | X| | XX| | 2 XX| |9 22 XXX| | XXX| |6 16 XXXXX| |8 11 27 29 1 XXXXX| | XXXXXXX|* |31 XXXXXXX|* |2 30 XXXXXXXXX|* * * |13 XXXXXXXXXX|* * * * * |19 0 XXXXXXX|* * * * * * |5 32 XXXXXXXX|* * * * * |7 15 28 XXXXXXX|* |4 14 21 XXXXXXXX|* * |3 17 20 23 XXXXXXXXX| |10 18 24 -1 XXXXXX| | XXXX|* |1 XXXX| | XX| |12 26 -2 XXX| |25 XX| | X| | X| | X| | -3 X| |
Tasks at level 1 require mainly recall of knowledge, with little interpretation or reasoning.
Tasks at level 3 require doing mathematics in a somewhat "passive way", such as manipulating expressions, carrying out computations, verifying propositions, etc, when the modelling has been done, the strategies given, the propositions stated, or the needed information is explicit.
Tasks at level 5 require doing mathematics in an active way: finding suitable strategies, selecting information, posing problems, constructing explanations and so on.
![Page 22: Why Scale -- 1](https://reader035.fdocuments.net/reader035/viewer/2022062501/568163f5550346895dd582c7/html5/thumbnails/22.jpg)
3 | | | | X| | X| | XX| | 2 XX| |9 22 XXX| | XXX| |6 16 XXXXX| |8 11 27 29 1 XXXXX| | XXXXXXX|* |31 XXXXXXX|* |2 30 XXXXXXXXX|* * * |13 XXXXXXXXXX|* * * * * |19 0 XXXXXXX|* * * * * * |5 32 XXXXXXXX|* * * * * |7 15 28 XXXXXXX|* |4 14 21 XXXXXXXX|* * |3 17 20 23 XXXXXXXXX| |10 18 24 -1 XXXXXX| | XXXX|* |1 XXXX| | XX| |12 26 -2 XXX| |25 XX| | X| | X| | X| | -3 X| |
Distance between the location of items and students fully describe students’ chances of success on the item
This property permits the use of described scales
Why a Rasch Model?