FOR CHRISTIANS looking to improve critical thinking skills ...
Critical thinking with school data: Looking back to look forwardsCritical thinking with school data:...
Transcript of Critical thinking with school data: Looking back to look forwardsCritical thinking with school data:...
![Page 1: Critical thinking with school data: Looking back to look forwardsCritical thinking with school data: Looking back to look forwards Presenter A/Prof Robyn Pierce,University of Melbourne](https://reader033.fdocuments.net/reader033/viewer/2022051321/5ff8391596e8cf309e40d558/html5/thumbnails/1.jpg)
Critical thinking with school data: Looking back to look forwards
PresenterA/Prof Robyn Pierce, University of Melbourne
MAV-MEG 2019
![Page 2: Critical thinking with school data: Looking back to look forwardsCritical thinking with school data: Looking back to look forwards Presenter A/Prof Robyn Pierce,University of Melbourne](https://reader033.fdocuments.net/reader033/viewer/2022051321/5ff8391596e8cf309e40d558/html5/thumbnails/2.jpg)
Data Piñata
2
DiagnosticsNAPLANPAT MathsRelative growth reportsRecording student achievementTeachers’ recordsSMARTMathleticsStudent interviewsNumeracyLiteracySchool ReportsState DataStudent ReportsACARA reportsMy Schools Well beingReporting software
Department of EducationStaffStudentsCurrent ParentsParent support groupsSchool CouncilBoard of DirectorsThe PressAlumniParents for Potential enrolments
Data SourcesWho wants to know?
![Page 3: Critical thinking with school data: Looking back to look forwardsCritical thinking with school data: Looking back to look forwards Presenter A/Prof Robyn Pierce,University of Melbourne](https://reader033.fdocuments.net/reader033/viewer/2022051321/5ff8391596e8cf309e40d558/html5/thumbnails/3.jpg)
Informed by researchAustralian Research Council –Linkage Project
Statistical literacy in the education workplace:Barriers and enablers for school personnel making data-driven decisions
3
University of Melbourne
A/Professor Robyn Pierce, Mr Roger Wander
Professor Ian Gordon , Ms Sue HelmeUniversity of TasmaniaA/Professor Helen Chick, Professor Emerita Jane Watson
VCAA
Mr Michael Dalton, Dr Magdalena Les
DEECD
Ms Sue Buckley
![Page 4: Critical thinking with school data: Looking back to look forwardsCritical thinking with school data: Looking back to look forwards Presenter A/Prof Robyn Pierce,University of Melbourne](https://reader033.fdocuments.net/reader033/viewer/2022051321/5ff8391596e8cf309e40d558/html5/thumbnails/4.jpg)
System Report on Student Achievement are:
• useful for identifying topics in the curriculum that need attention in our school. 82%
• useful to inform whole school planning. 80% .
• helpful for grouping students according to ability. 67%
• helpful for planning my lessons. 58%
• telling me things about my students that I had not realised. 55%
Attitudes - Agree or Strongly Agree
![Page 5: Critical thinking with school data: Looking back to look forwardsCritical thinking with school data: Looking back to look forwards Presenter A/Prof Robyn Pierce,University of Melbourne](https://reader033.fdocuments.net/reader033/viewer/2022051321/5ff8391596e8cf309e40d558/html5/thumbnails/5.jpg)
% Agree or Strongly Agree
• I don’t feel I can adequately interpret the SRSA I receive at our school. 15%
• SRSA take too long to interpret. 29%
• The amount of data presented in the SRSA I see is overwhelming. 33%
Perceived difficulties
5
![Page 6: Critical thinking with school data: Looking back to look forwardsCritical thinking with school data: Looking back to look forwards Presenter A/Prof Robyn Pierce,University of Melbourne](https://reader033.fdocuments.net/reader033/viewer/2022051321/5ff8391596e8cf309e40d558/html5/thumbnails/6.jpg)
6
Data: not just numbers
(Pierce & Chick)
![Page 7: Critical thinking with school data: Looking back to look forwardsCritical thinking with school data: Looking back to look forwards Presenter A/Prof Robyn Pierce,University of Melbourne](https://reader033.fdocuments.net/reader033/viewer/2022051321/5ff8391596e8cf309e40d558/html5/thumbnails/7.jpg)
Local context
7
Snap shots at a moment in time
Add your own photos - same people on different days – creating a different impression
![Page 8: Critical thinking with school data: Looking back to look forwardsCritical thinking with school data: Looking back to look forwards Presenter A/Prof Robyn Pierce,University of Melbourne](https://reader033.fdocuments.net/reader033/viewer/2022051321/5ff8391596e8cf309e40d558/html5/thumbnails/8.jpg)
Professional context
8
• The curriculum
• Content knowledge
• Knowledge of common misconceptions
• Mathematics Pedagogical knowledge
![Page 9: Critical thinking with school data: Looking back to look forwardsCritical thinking with school data: Looking back to look forwards Presenter A/Prof Robyn Pierce,University of Melbourne](https://reader033.fdocuments.net/reader033/viewer/2022051321/5ff8391596e8cf309e40d558/html5/thumbnails/9.jpg)
Now for the graphs and numbers
9
Read the valuesCheck the scale, read the values
Compare the valuesNo lonely numbers (Hans Rosling)
Analyse the dataFrame useful questions
– analyse for a purpose
Purposes:
Reporting – to students , parents, school council…
System data
Inform teachingSystem data from assessment of learning: a littleGood questions: assessment for learning: a lot
![Page 10: Critical thinking with school data: Looking back to look forwardsCritical thinking with school data: Looking back to look forwards Presenter A/Prof Robyn Pierce,University of Melbourne](https://reader033.fdocuments.net/reader033/viewer/2022051321/5ff8391596e8cf309e40d558/html5/thumbnails/10.jpg)
MAKING SENSE OF DATA
Reports : looking back to look forward10
![Page 11: Critical thinking with school data: Looking back to look forwardsCritical thinking with school data: Looking back to look forwards Presenter A/Prof Robyn Pierce,University of Melbourne](https://reader033.fdocuments.net/reader033/viewer/2022051321/5ff8391596e8cf309e40d558/html5/thumbnails/11.jpg)
This is James
James is in Year 7 at Example College
James’ NAPLAN results
Reading: 523Writing: 582Spelling: 595Grammar & Punctuation: 557Numeracy: 541
11
![Page 12: Critical thinking with school data: Looking back to look forwardsCritical thinking with school data: Looking back to look forwards Presenter A/Prof Robyn Pierce,University of Melbourne](https://reader033.fdocuments.net/reader033/viewer/2022051321/5ff8391596e8cf309e40d558/html5/thumbnails/12.jpg)
Here is James with the other Example College Year 7 students
12
![Page 13: Critical thinking with school data: Looking back to look forwardsCritical thinking with school data: Looking back to look forwards Presenter A/Prof Robyn Pierce,University of Melbourne](https://reader033.fdocuments.net/reader033/viewer/2022051321/5ff8391596e8cf309e40d558/html5/thumbnails/13.jpg)
Look at the distribution for the class members’ results: The number line scale
![Page 14: Critical thinking with school data: Looking back to look forwardsCritical thinking with school data: Looking back to look forwards Presenter A/Prof Robyn Pierce,University of Melbourne](https://reader033.fdocuments.net/reader033/viewer/2022051321/5ff8391596e8cf309e40d558/html5/thumbnails/14.jpg)
Here’s Jamesa lonely number
![Page 15: Critical thinking with school data: Looking back to look forwardsCritical thinking with school data: Looking back to look forwards Presenter A/Prof Robyn Pierce,University of Melbourne](https://reader033.fdocuments.net/reader033/viewer/2022051321/5ff8391596e8cf309e40d558/html5/thumbnails/15.jpg)
James and two others
![Page 16: Critical thinking with school data: Looking back to look forwardsCritical thinking with school data: Looking back to look forwards Presenter A/Prof Robyn Pierce,University of Melbourne](https://reader033.fdocuments.net/reader033/viewer/2022051321/5ff8391596e8cf309e40d558/html5/thumbnails/16.jpg)
More students placed against their results
![Page 17: Critical thinking with school data: Looking back to look forwardsCritical thinking with school data: Looking back to look forwards Presenter A/Prof Robyn Pierce,University of Melbourne](https://reader033.fdocuments.net/reader033/viewer/2022051321/5ff8391596e8cf309e40d558/html5/thumbnails/17.jpg)
The whole class: What do we know about James?What do we know about the class?
National Minimum Standard
17
![Page 18: Critical thinking with school data: Looking back to look forwardsCritical thinking with school data: Looking back to look forwards Presenter A/Prof Robyn Pierce,University of Melbourne](https://reader033.fdocuments.net/reader033/viewer/2022051321/5ff8391596e8cf309e40d558/html5/thumbnails/18.jpg)
Building a box-and-whisker plot (1)
Divide the class into 4 quarters
![Page 19: Critical thinking with school data: Looking back to look forwardsCritical thinking with school data: Looking back to look forwards Presenter A/Prof Robyn Pierce,University of Melbourne](https://reader033.fdocuments.net/reader033/viewer/2022051321/5ff8391596e8cf309e40d558/html5/thumbnails/19.jpg)
Building a box-and-whisker plot (2)
19Insert a “box” over the middle 50% ie half of the class
![Page 20: Critical thinking with school data: Looking back to look forwardsCritical thinking with school data: Looking back to look forwards Presenter A/Prof Robyn Pierce,University of Melbourne](https://reader033.fdocuments.net/reader033/viewer/2022051321/5ff8391596e8cf309e40d558/html5/thumbnails/20.jpg)
Building a box-and-whisker plot (3)
20
For the plots sent from the Victorian NAPLAN team: Turn over, remove the students who top and bottom 10% of students Place “whiskers” to the highest and lowest remaining students
![Page 21: Critical thinking with school data: Looking back to look forwardsCritical thinking with school data: Looking back to look forwards Presenter A/Prof Robyn Pierce,University of Melbourne](https://reader033.fdocuments.net/reader033/viewer/2022051321/5ff8391596e8cf309e40d558/html5/thumbnails/21.jpg)
Building a box-and-whisker plot (4)
21
![Page 22: Critical thinking with school data: Looking back to look forwardsCritical thinking with school data: Looking back to look forwards Presenter A/Prof Robyn Pierce,University of Melbourne](https://reader033.fdocuments.net/reader033/viewer/2022051321/5ff8391596e8cf309e40d558/html5/thumbnails/22.jpg)
22
![Page 23: Critical thinking with school data: Looking back to look forwardsCritical thinking with school data: Looking back to look forwards Presenter A/Prof Robyn Pierce,University of Melbourne](https://reader033.fdocuments.net/reader033/viewer/2022051321/5ff8391596e8cf309e40d558/html5/thumbnails/23.jpg)
Box-and-whisker plots do not show top and bottom 10% of cohort results
Note Key
80% of the National cohort
80% of the State cohort
80% of your School cohort
![Page 24: Critical thinking with school data: Looking back to look forwardsCritical thinking with school data: Looking back to look forwards Presenter A/Prof Robyn Pierce,University of Melbourne](https://reader033.fdocuments.net/reader033/viewer/2022051321/5ff8391596e8cf309e40d558/html5/thumbnails/24.jpg)
Box-and-whisker graphs break the cohort into fixed percentages
Same proportion of students, represented by each whisker
![Page 25: Critical thinking with school data: Looking back to look forwardsCritical thinking with school data: Looking back to look forwards Presenter A/Prof Robyn Pierce,University of Melbourne](https://reader033.fdocuments.net/reader033/viewer/2022051321/5ff8391596e8cf309e40d558/html5/thumbnails/25.jpg)
Pay attention to the big picture:
Compare with the State, compare with Similar SchoolsCompare with the past 5 years
25
:
Your local knowledge is importantDon’t spend time on small differences
Expect variation:
Pay attention to success:
![Page 26: Critical thinking with school data: Looking back to look forwardsCritical thinking with school data: Looking back to look forwards Presenter A/Prof Robyn Pierce,University of Melbourne](https://reader033.fdocuments.net/reader033/viewer/2022051321/5ff8391596e8cf309e40d558/html5/thumbnails/26.jpg)
Are we meeting the learning needs of all of our students?
Relative Growth/Gain
![Page 27: Critical thinking with school data: Looking back to look forwardsCritical thinking with school data: Looking back to look forwards Presenter A/Prof Robyn Pierce,University of Melbourne](https://reader033.fdocuments.net/reader033/viewer/2022051321/5ff8391596e8cf309e40d558/html5/thumbnails/27.jpg)
Why was it needed?q Progress on the NAPLAN scale is not linear. q The rate of growth diminishes as students progress along
the NAPLAN scale. q This occurrence is observed in various assessments
programs around the world. It is understood that students achieve larger educational milestones lower on the assessment scale more easily compared to more subtle milestones further along the assessment scale.
![Page 28: Critical thinking with school data: Looking back to look forwardsCritical thinking with school data: Looking back to look forwards Presenter A/Prof Robyn Pierce,University of Melbourne](https://reader033.fdocuments.net/reader033/viewer/2022051321/5ff8391596e8cf309e40d558/html5/thumbnails/28.jpg)
What is Relative Growth?
qStudents performance is compared to similar ability students.
qNAPLAN definition:A Student performance in the current year is compared to all students in Victoria who had the same score two years prior. E.g. A Year 5 student in 2019 is compared to those students who had the same score in Year 3 in 2017.
![Page 29: Critical thinking with school data: Looking back to look forwardsCritical thinking with school data: Looking back to look forwards Presenter A/Prof Robyn Pierce,University of Melbourne](https://reader033.fdocuments.net/reader033/viewer/2022051321/5ff8391596e8cf309e40d558/html5/thumbnails/29.jpg)
In 2014 there were 60,000 students in Year 7 Reading
Of these students 1,000 had a scale score for Reading of 380 in Year 5 in 2012 (2 years Prior)
To determine the level of relative Growth for this subset
of students …
![Page 30: Critical thinking with school data: Looking back to look forwardsCritical thinking with school data: Looking back to look forwards Presenter A/Prof Robyn Pierce,University of Melbourne](https://reader033.fdocuments.net/reader033/viewer/2022051321/5ff8391596e8cf309e40d558/html5/thumbnails/30.jpg)
Relative growth
30
Year 5 students on same score
Year 7 low medium high
25% middle 50% 25%
![Page 31: Critical thinking with school data: Looking back to look forwardsCritical thinking with school data: Looking back to look forwards Presenter A/Prof Robyn Pierce,University of Melbourne](https://reader033.fdocuments.net/reader033/viewer/2022051321/5ff8391596e8cf309e40d558/html5/thumbnails/31.jpg)
Lowest 25% of scores
Middle 50% of scores Highest 25% of
scores
Low Growth
Medium Growth HighGrowth
2014 Reading Scale Scores in Year 7 for students that had a Year 5 reading score in 2012 of 380.
![Page 32: Critical thinking with school data: Looking back to look forwardsCritical thinking with school data: Looking back to look forwards Presenter A/Prof Robyn Pierce,University of Melbourne](https://reader033.fdocuments.net/reader033/viewer/2022051321/5ff8391596e8cf309e40d558/html5/thumbnails/32.jpg)
Tabular report
Relative Growth in the NAPLAN Data Service
![Page 33: Critical thinking with school data: Looking back to look forwardsCritical thinking with school data: Looking back to look forwards Presenter A/Prof Robyn Pierce,University of Melbourne](https://reader033.fdocuments.net/reader033/viewer/2022051321/5ff8391596e8cf309e40d558/html5/thumbnails/33.jpg)
Relative Growth in the NAPLAN Data Service
Graphical summary report
![Page 34: Critical thinking with school data: Looking back to look forwardsCritical thinking with school data: Looking back to look forwards Presenter A/Prof Robyn Pierce,University of Melbourne](https://reader033.fdocuments.net/reader033/viewer/2022051321/5ff8391596e8cf309e40d558/html5/thumbnails/34.jpg)
0
10
20
30
40
50
60
70
80
90
100
Below Range Band 4-5 Band 6 -7 Band 8 - 9 Above Range
Perc
enta
ge o
f stu
dent
s
Year 7 NAPLAN Bands
Relative Growth of students - Year 7 to Year 9 Reading
LowMediumHigh
Graphical summary report - detail
![Page 35: Critical thinking with school data: Looking back to look forwardsCritical thinking with school data: Looking back to look forwards Presenter A/Prof Robyn Pierce,University of Melbourne](https://reader033.fdocuments.net/reader033/viewer/2022051321/5ff8391596e8cf309e40d558/html5/thumbnails/35.jpg)
Example … comparing growth statistics
Year5NAPLANresultsNAPLAN Band <3 3 4 5 6 7 8 >8 Mean
School A 0 0 2 5 3 5 4 6 570- - 8% 20% 12% 20% 16% 24%
School B1 5 7 8 3 1 0 0
4234% 20% 28% 32% 12% 4% - -
NAPLANMeanScaledScoresYear 3
(2 years prior)Year 5
(current year) Difference
School A 468.4 569.9 101.5School B 309.8 423.4 113.6
State 431.8 504.5 72.7
![Page 36: Critical thinking with school data: Looking back to look forwardsCritical thinking with school data: Looking back to look forwards Presenter A/Prof Robyn Pierce,University of Melbourne](https://reader033.fdocuments.net/reader033/viewer/2022051321/5ff8391596e8cf309e40d558/html5/thumbnails/36.jpg)
Example …comparing Relative Growth patterns
36
Year 3 Student Scores Relative Growth for
YR-3 to YR-5
Year 5 Student ScoresScaled Score Band Scaled Score Band
346 3 Medium 467 5346 3 Medium 477 5356 3 Low 416 4367 3 Medium 467 5367 3 Low 405 4377 4 Low 426 5398 4 Medium 509 6408 4 Low 447 5440 5 Medium 520 6464 5 Medium 557 7477 5 Medium 509 6477 5 Medium 532 7491 6 High 622 8491 6 High 603 8491 6 Medium 544 7506 6 High 800 >8506 6 High 716 >8524 6 Medium 557 7524 6 High 622 8544 >6 Medium 603 8544 >6 High 716 >8544 >6 Medium 571 7544 >6 High 645 >8571 >6 High 716 >8608 >6 High 800 >8
Year 3 Student Scores Relative Growth for
YR-3 to YR-5
Year 5 Student Scores
Scaled Score Band Scaled Score Band
157 1 Medium 360 3181 1 High 416 4236 1 Medium 372 3236 1 High 426 5250 1 Medium 372 3264 1 Medium 384 4264 1 Low 320 <3277 2 High 447 5277 2 Medium 384 4290 2 Medium 436 5302 2 Low 334 3302 2 Medium 395 4302 2 High 457 5313 2 High 477 5313 2 Medium 405 4324 3 Medium 405 4324 3 High 467 5346 3 High 488 6346 3 Low 360 3356 3 Medium 477 5387 4 Medium 457 5408 4 High 520 6419 4 High 532 7419 4 Low 395 4452 5 Medium 498 6
School A School B
School A: HIGH relative growth has been achieved only by the higher ability students.
School B: HIGH relative growth is evident from a range of Year 3 starting points
![Page 37: Critical thinking with school data: Looking back to look forwardsCritical thinking with school data: Looking back to look forwards Presenter A/Prof Robyn Pierce,University of Melbourne](https://reader033.fdocuments.net/reader033/viewer/2022051321/5ff8391596e8cf309e40d558/html5/thumbnails/37.jpg)
37
Identifying strengths and weaknesses by topicItem analysis
![Page 38: Critical thinking with school data: Looking back to look forwardsCritical thinking with school data: Looking back to look forwards Presenter A/Prof Robyn Pierce,University of Melbourne](https://reader033.fdocuments.net/reader033/viewer/2022051321/5ff8391596e8cf309e40d558/html5/thumbnails/38.jpg)
38
Look at the actual items on the Test item
MCD
Non-Calc
![Page 39: Critical thinking with school data: Looking back to look forwardsCritical thinking with school data: Looking back to look forwards Presenter A/Prof Robyn Pierce,University of Melbourne](https://reader033.fdocuments.net/reader033/viewer/2022051321/5ff8391596e8cf309e40d558/html5/thumbnails/39.jpg)
% Correct National % Correct State% Correct Group (selected)
![Page 40: Critical thinking with school data: Looking back to look forwardsCritical thinking with school data: Looking back to look forwards Presenter A/Prof Robyn Pierce,University of Melbourne](https://reader033.fdocuments.net/reader033/viewer/2022051321/5ff8391596e8cf309e40d558/html5/thumbnails/40.jpg)
±5% : ‘Similar’ or ‘Different’?
Number Correct
Number Incorrect
% Correct
360 40 90340 60 85320 80 80
Number Correct
Number Incorrect
% Correct
18 2 9017 3 8516 4 80
Number Correct
Number Incorrect
% Correct
9 1 9085
8 2 80
10 students sat the test
20 students sat the test
400 students sat the test
![Page 41: Critical thinking with school data: Looking back to look forwardsCritical thinking with school data: Looking back to look forwards Presenter A/Prof Robyn Pierce,University of Melbourne](https://reader033.fdocuments.net/reader033/viewer/2022051321/5ff8391596e8cf309e40d558/html5/thumbnails/41.jpg)
Numeracy
a) Items where the % correctwas very different to the state
b) Items where a large proportion of students chose a (wrong) distracter
Are there any obvious commonalities in those items that scored above or below the State? Are they in the same topic area?
41
Similar
Above
Below
Item 15Item 7
Item 3
Item 22Item 11
Item 4
Item 25Item 17
Item 8
Item 1
What can we learn from looking at:
![Page 42: Critical thinking with school data: Looking back to look forwardsCritical thinking with school data: Looking back to look forwards Presenter A/Prof Robyn Pierce,University of Melbourne](https://reader033.fdocuments.net/reader033/viewer/2022051321/5ff8391596e8cf309e40d558/html5/thumbnails/42.jpg)
Tutorials can be found at
http://usingassessmentdata.vcaa.vic.edu.au/naplan/index.aspx
Robyn Pierce [email protected]
Roger Wander [email protected]
Michael Dalton [email protected]
![Page 43: Critical thinking with school data: Looking back to look forwardsCritical thinking with school data: Looking back to look forwards Presenter A/Prof Robyn Pierce,University of Melbourne](https://reader033.fdocuments.net/reader033/viewer/2022051321/5ff8391596e8cf309e40d558/html5/thumbnails/43.jpg)
![Page 44: Critical thinking with school data: Looking back to look forwardsCritical thinking with school data: Looking back to look forwards Presenter A/Prof Robyn Pierce,University of Melbourne](https://reader033.fdocuments.net/reader033/viewer/2022051321/5ff8391596e8cf309e40d558/html5/thumbnails/44.jpg)
Thank youA/Prof Robyn PierceMelbourne Graduate School of Education
[email protected] first line
•Second line