Use SP Xpress to analyze students’ performance & evaluate the question items setting
description
Transcript of Use SP Xpress to analyze students’ performance & evaluate the question items setting
![Page 1: Use SP Xpress to analyze students’ performance & evaluate the question items setting](https://reader035.fdocuments.net/reader035/viewer/2022062309/56815a83550346895dc7f162/html5/thumbnails/1.jpg)
St. Anthony’s School
16 Jan 2013 1
![Page 2: Use SP Xpress to analyze students’ performance & evaluate the question items setting](https://reader035.fdocuments.net/reader035/viewer/2022062309/56815a83550346895dc7f162/html5/thumbnails/2.jpg)
Background information
2
![Page 3: Use SP Xpress to analyze students’ performance & evaluate the question items setting](https://reader035.fdocuments.net/reader035/viewer/2022062309/56815a83550346895dc7f162/html5/thumbnails/3.jpg)
Enhance the effectiveness of learning and teaching
3
![Page 4: Use SP Xpress to analyze students’ performance & evaluate the question items setting](https://reader035.fdocuments.net/reader035/viewer/2022062309/56815a83550346895dc7f162/html5/thumbnails/4.jpg)
Evaluation after Assessment
Why did students
choose A, not B, C or D?
What are the follow-up actions?
Are the questions
too difficult or too easy?
Have the students learned? Have they
studied hard?
4
![Page 5: Use SP Xpress to analyze students’ performance & evaluate the question items setting](https://reader035.fdocuments.net/reader035/viewer/2022062309/56815a83550346895dc7f162/html5/thumbnails/5.jpg)
5
![Page 6: Use SP Xpress to analyze students’ performance & evaluate the question items setting](https://reader035.fdocuments.net/reader035/viewer/2022062309/56815a83550346895dc7f162/html5/thumbnails/6.jpg)
Student MCIItem MCICronbach AlphaGraph shape
Data – Good to start
6
![Page 7: Use SP Xpress to analyze students’ performance & evaluate the question items setting](https://reader035.fdocuments.net/reader035/viewer/2022062309/56815a83550346895dc7f162/html5/thumbnails/7.jpg)
2 Modified Caution IndexesStudent Modified Caution Index
Item Modified Caution Index
Students’ Performance
Question items’
Performance
7
![Page 8: Use SP Xpress to analyze students’ performance & evaluate the question items setting](https://reader035.fdocuments.net/reader035/viewer/2022062309/56815a83550346895dc7f162/html5/thumbnails/8.jpg)
Student MCIxP Chart
2012-13 P.4 1st
Reading Exam
8
![Page 9: Use SP Xpress to analyze students’ performance & evaluate the question items setting](https://reader035.fdocuments.net/reader035/viewer/2022062309/56815a83550346895dc7f162/html5/thumbnails/9.jpg)
Student MCIxP Chart
2012-13 P.4 1st
Reading Exam
9
![Page 10: Use SP Xpress to analyze students’ performance & evaluate the question items setting](https://reader035.fdocuments.net/reader035/viewer/2022062309/56815a83550346895dc7f162/html5/thumbnails/10.jpg)
More focused strategiesType A Type B
The teaching and learning are suitable for them. Encourage them to keep on.
Know the difficult questions Miss the simple questionsCareless mistakesDon’t rely on luck
Type D Type C
Adjust the teaching contentRaise their learning motivation Need more consolidation
Answering techniquesCareless mistakesFocus on the foundationDon’t rely on luck
10
![Page 11: Use SP Xpress to analyze students’ performance & evaluate the question items setting](https://reader035.fdocuments.net/reader035/viewer/2022062309/56815a83550346895dc7f162/html5/thumbnails/11.jpg)
Item MCI xP Chart
2012-13 P.4 1st
Reading Exam
11
![Page 12: Use SP Xpress to analyze students’ performance & evaluate the question items setting](https://reader035.fdocuments.net/reader035/viewer/2022062309/56815a83550346895dc7f162/html5/thumbnails/12.jpg)
Item MCI xP Chart
Need improveme
nt
Difficult question
s
Problem question
s
12
![Page 13: Use SP Xpress to analyze students’ performance & evaluate the question items setting](https://reader035.fdocuments.net/reader035/viewer/2022062309/56815a83550346895dc7f162/html5/thumbnails/13.jpg)
MCI item P-value chartItem Type
1112 P.3 Final Exam
1213 P.4 First Exam
A 20 17B 1 2C 1 0D 3 7Total no. of questions
25 26According the Item Modified Caution Index, most of the items set are normal.
13
![Page 14: Use SP Xpress to analyze students’ performance & evaluate the question items setting](https://reader035.fdocuments.net/reader035/viewer/2022062309/56815a83550346895dc7f162/html5/thumbnails/14.jpg)
Cronbach Alpha (Reliability)
Cronbach Alpha
Reliability
0 – 0.3 low
0.7-0.9 acceptable
> 0.9 high
14
![Page 15: Use SP Xpress to analyze students’ performance & evaluate the question items setting](https://reader035.fdocuments.net/reader035/viewer/2022062309/56815a83550346895dc7f162/html5/thumbnails/15.jpg)
2011-2012 P.3 First Exam
Slightly Easy
15
![Page 16: Use SP Xpress to analyze students’ performance & evaluate the question items setting](https://reader035.fdocuments.net/reader035/viewer/2022062309/56815a83550346895dc7f162/html5/thumbnails/16.jpg)
2012-2013 P.4 First Exam
Appropriate
16
![Page 17: Use SP Xpress to analyze students’ performance & evaluate the question items setting](https://reader035.fdocuments.net/reader035/viewer/2022062309/56815a83550346895dc7f162/html5/thumbnails/17.jpg)
Which question
items should be improved?
17
![Page 18: Use SP Xpress to analyze students’ performance & evaluate the question items setting](https://reader035.fdocuments.net/reader035/viewer/2022062309/56815a83550346895dc7f162/html5/thumbnails/18.jpg)
Which question items are difficult to
our students?
18
![Page 19: Use SP Xpress to analyze students’ performance & evaluate the question items setting](https://reader035.fdocuments.net/reader035/viewer/2022062309/56815a83550346895dc7f162/html5/thumbnails/19.jpg)
SP Xpress analysis on ‘2011-2012 Primary 3 Third Exam Paper’SP Xpress analysis results:
Type A: 20 itemsType B: 1 itemType C: 1 itemType D: 3 items
Let’s examine one item from each category (Type B, C, D)
19
![Page 20: Use SP Xpress to analyze students’ performance & evaluate the question items setting](https://reader035.fdocuments.net/reader035/viewer/2022062309/56815a83550346895dc7f162/html5/thumbnails/20.jpg)
Exam item example: Type B
On which floor of the centre, can Grandpa find presents for Jack?A. 1/F B. 2/F C. 3/F D.4/F
20
![Page 21: Use SP Xpress to analyze students’ performance & evaluate the question items setting](https://reader035.fdocuments.net/reader035/viewer/2022062309/56815a83550346895dc7f162/html5/thumbnails/21.jpg)
Possible reasons for item being Type BType B: Item is not difficult but requires
amendments because many students couldn’t answer it.
Possible reasons:Item design flawsNot close to what teacher taught in classStudents lack specific training in reading
comic strips
21
![Page 22: Use SP Xpress to analyze students’ performance & evaluate the question items setting](https://reader035.fdocuments.net/reader035/viewer/2022062309/56815a83550346895dc7f162/html5/thumbnails/22.jpg)
Teacher examined the Type B itemStudents found it difficult because …
Need to process and connect many ideas all at the same timeQuestion Comic strip
Which floor? 1/F,2/F,3/F,4/Ffloor directory
… of the Centre … Star Shopping Centre
… can Grandpa find presents …
We can buy all the gifts …
presents? presentsrobotstoys(4/F)
… can Grandpa … ‘ We’ refers to ‘grandparents’
22
![Page 23: Use SP Xpress to analyze students’ performance & evaluate the question items setting](https://reader035.fdocuments.net/reader035/viewer/2022062309/56815a83550346895dc7f162/html5/thumbnails/23.jpg)
Recommended solutionsIncrease students’ chances in reading
English comic strips in classroom teaching or leisure reading
Teacher needs to illustrate more clearly in connecting ideas
Teacher needs to teach more explicitly which means more visual demonstrations
23
![Page 24: Use SP Xpress to analyze students’ performance & evaluate the question items setting](https://reader035.fdocuments.net/reader035/viewer/2022062309/56815a83550346895dc7f162/html5/thumbnails/24.jpg)
Exam item example: Type CIn which step does Jack’s mum add ‘soy sauce’ and ‘wasabi’ to the sushi?A. Step 1 B. Step 3 C. Step 6 D. Step 10
24
![Page 25: Use SP Xpress to analyze students’ performance & evaluate the question items setting](https://reader035.fdocuments.net/reader035/viewer/2022062309/56815a83550346895dc7f162/html5/thumbnails/25.jpg)
Possible reasons for item being Type CType C: Item is difficult and requires
amendments because many students couldn’t answer it.
Possible reasons:Confusing descriptionQuestion misleading studentsQuestion containing error(s)
25
![Page 26: Use SP Xpress to analyze students’ performance & evaluate the question items setting](https://reader035.fdocuments.net/reader035/viewer/2022062309/56815a83550346895dc7f162/html5/thumbnails/26.jpg)
Teacher examined the Type C itemStudents did not have the relevant personal
experience in making and/or eating sushiThe recipe did not mention dipping sushi into ‘soy
sauce’ or ‘wasabi’ after the finishWithout experience, students could not interpret the
correct answer which is the final stepThis question contains the error which requires
outside knowledge uncommon among the students
26
![Page 27: Use SP Xpress to analyze students’ performance & evaluate the question items setting](https://reader035.fdocuments.net/reader035/viewer/2022062309/56815a83550346895dc7f162/html5/thumbnails/27.jpg)
Exam item example: Type DAfter cutting the vegetables and crab meat, Jack’s mum ____.C.takes some rice wine vinegar and mixes it with the sushi rice
This is the answer but the option rephrased the instructions.
27
![Page 28: Use SP Xpress to analyze students’ performance & evaluate the question items setting](https://reader035.fdocuments.net/reader035/viewer/2022062309/56815a83550346895dc7f162/html5/thumbnails/28.jpg)
Features of Type D itemType D: Item is difficult and doesn’t require
amendments. It is suitable for screening the high ability students. Low and mid- ability students can’t answer it.
Type D items should be kept at a low proportion in an assessment.
The rephrasing is difficult for students to interpret the correct answer.
28
![Page 29: Use SP Xpress to analyze students’ performance & evaluate the question items setting](https://reader035.fdocuments.net/reader035/viewer/2022062309/56815a83550346895dc7f162/html5/thumbnails/29.jpg)
Analysis of TSA P.3 Reading PaperPredict P.3 students’ results in the upcoming TSA
reading assessmentCompare the overall students’ ability from different
yearsExamine the neglected elements in our teaching Assumption: TSA assessment covers the basic
competency that all students are supposed to have acquired by the end of a learning stage
29
![Page 30: Use SP Xpress to analyze students’ performance & evaluate the question items setting](https://reader035.fdocuments.net/reader035/viewer/2022062309/56815a83550346895dc7f162/html5/thumbnails/30.jpg)
SP Xpress analysis on ‘2011 TSA P.3 Reading Paper (3ER1)’
2011-2012 P.3
2012-2013 P.3
Mean 19.28 18.95
SD 4.78 4.76
Type A 18 20
Type B 6 3
Type C 0 1
Type D 2 230
![Page 31: Use SP Xpress to analyze students’ performance & evaluate the question items setting](https://reader035.fdocuments.net/reader035/viewer/2022062309/56815a83550346895dc7f162/html5/thumbnails/31.jpg)
An Interesting Fact: A common Type B item across 2 yearsPart 1B Question #1
How many chapters are there in this book?‘Chapter / Table of Contents’ can be an obstacleStudents never encounter ‘Chapter’ in storybook
teaching (replacing textbook teaching)
31
![Page 32: Use SP Xpress to analyze students’ performance & evaluate the question items setting](https://reader035.fdocuments.net/reader035/viewer/2022062309/56815a83550346895dc7f162/html5/thumbnails/32.jpg)
An Interesting Fact: A common Type B item across 2 years Sam buys two sandwiches with the Rainbow Fast Food
coupon. How many sandwiches can he get?C. three sandwiches (= Buy 2 + 1 Free)
Students are not familiar with ‘Buy 2 Get 1 Free’ concept even though they can understand every single word
32
![Page 33: Use SP Xpress to analyze students’ performance & evaluate the question items setting](https://reader035.fdocuments.net/reader035/viewer/2022062309/56815a83550346895dc7f162/html5/thumbnails/33.jpg)
An Interesting Fact: A common Type D item across 2 years “ … too. In the end, John wins the race. All the
people cheer loudly …” Read lines 11-12. What do the people do when John
wins the race?C. shout happily
Students are not familiar with synonyms
cheer loudly
shout happily33
![Page 34: Use SP Xpress to analyze students’ performance & evaluate the question items setting](https://reader035.fdocuments.net/reader035/viewer/2022062309/56815a83550346895dc7f162/html5/thumbnails/34.jpg)
Personal reflection1. Teachers become aware of the quality of the
assessments Accumulate professional experience in setting
better assessments
2. Teachers prepared for the unexpected and unlearned
Teachers become cautious and adjust their teaching
34
![Page 35: Use SP Xpress to analyze students’ performance & evaluate the question items setting](https://reader035.fdocuments.net/reader035/viewer/2022062309/56815a83550346895dc7f162/html5/thumbnails/35.jpg)
Things to do Suggest Ts reflect on their exam paper
using SP Xpress Facilitate Ts process the data with
technical supportsSet up a school based question bankNurture a culture on giving a more
constructive feedback to both Ts and Ss after assessment
35
![Page 36: Use SP Xpress to analyze students’ performance & evaluate the question items setting](https://reader035.fdocuments.net/reader035/viewer/2022062309/56815a83550346895dc7f162/html5/thumbnails/36.jpg)
Thank you so much for your time!