HKASME

Science Assessment Test 2019

June 28, 2019

(Debriefing Seminar)

HKASME Science Assessment Test

Purpose:

• To develop and to implement a science assessment test for S2&3 students in Hong Kong

• To analyse the test results and to provide feedback to schools/the administration on the strengths and weaknesses of students in learning science

• Observing, measuring and recording data (SP1)• Comparing and classifying (SP2)• Planning and designing (SP3)• Experimenting (SP4)• Interpreting data / Inferring (SP5)• Communicating (SP6)

Science Process Skills (SPS)

HKASME Science Assessment Test

• Duration: 1 hour

• Paper Structure:

Section A: 24 MCQs

Section B: 2 Short-response questions

• Coverage: Science knowledge in

topics 1 to 11 of CDC Science Curriculum

S1 to S3

Participations in 2019 SAT

No. of Participating Schools: 86

Participants S2 S3 Total

Boys 779 605 1384

Girls 674 446 1120

Total 1453 1051 2504

733

1842

2402

25922504

22

62

88

77

86

0

10

20

30

40

50

60

70

80

90

100

0

500

1000

1500

2000

2500

3000

2015 2016 2017 2018 2019

no

. of

sch

oo

ls

no

. of

can

did

ates

year

No. of schools & candidates participated in SAT

candidates

schools

Participation in SAT across Year

Analysis of 2019 SAT Results

Mean SD

Section A 11.6 (48%)

3.73 (15.5%)

B1 5.7 (57%)

2.19 (21.9%)

B2 3.8 (48%)

1.57 (19.7%)

Section B 9.6 (53%)

3.10 (17.2%)

Note: The percentage in parenthesis represents % marks equivalent.

14.313.1

14.714.2

11.6

8.2

9.8

7.7

9.6 9.6

29.7 29.5 29.730.8

27

0

5

10

15

20

25

30

35

2015 2016 2017 2018 2019

mar

ks

year

Candidates' performance in SAT (Mean Score) across year

MC

B_total

Paper Score

Comparison of Performance of S2 and S3 Students

Section A Section B

Mean S.D. Mean S.D.

S211.4

(47%)3.63

(15.1%)9.3

(52%)3.13

(17.4%)

S312.0

(50%)3.83

(16.0%)10.0

(55%)3.01

(16.7%)

Note: The percentage in parenthesis represents % marks equivalent.

Grading of 2019 SAT

• Bronze Award based on a criteria-based model

• Diamond Award (top 5%)

• Expert judgment for Gold, Silver and Bronze Award

• QP and students’ performance on individual questions available to participating schools

• More informative school reports

2019 SAT Award Scheme

Bronze Silver Gold Diamond

Paper score / marks

21.0 – 28.5 29.0 – 33.0 33.5 – 39.5 >40.0

(% marks) (39-53%) (54-61%) (62-73%) (≥74%)

Section A score / MCQs

> 8 > 8 > 8 > 8

Section B score / marks

> 5 > 5 > 5 > 5

(Paper score = 1.5 × Mark in Section A + Mark in Section B)

HKASME set up an expert panel to determine the cut-off based on the performance of the participants.

2019 SAT Awards

Award type Number

Diamond130

(5.2%)

Gold406

(16.2%)

Silver503

(20.1%)

Bronze873

(34.9%)

Appreciation (or Participation)592

(23.6%)

Diamond Gold Silver BronzeAppreciation & Participation

S2 65(4.5%)

205(14.1%)

261(18.0%)

542(37.3%)

380(26.2%)

S3 65(6.2%)

201(19.1%)

242(23.0%)

331(31.5%)

212(20.2%)

WholeGroup

130(5.2%)

406(16.2%)

503(20.1%)

873(34.9%)

592(23.6%)

Comparison of Awards received by S2 & S3 Candidates

Note: The percentage in parenthesis represents % of candidates receiving the awards.

School-based Report –Appendix I Analysis of Participants’ Responses to the Multiple-choice

Questions (MCQs)

Q. No Skills assessed Key% correct Strength /

weaknessWG School

1 Identifying variables D 56.8 57.1 -

2 Classifying C 88.3 85.7 -

3 Making hypothesis B 42.1 57.1 S

4 Understanding; Inferring B 68.8 75.0 -

5 Understanding C 25.4 28.6 D

6 Understanding C 62.7 46.4 W

7 Understanding A 37.9 35.7 -

8 Inferring C 22.8 42.9 D, S

9 Understanding; Predicting A 39.5 42.9 -

10 Understanding D 33.4 42.9 -

11 Understanding; Predicting A 44.2 42.9 -

12Planning & design, Fair testing

A 56.3 78.6 S

Strength in a question (indicated by “S”) means that the school got a higher correct % by 15% or more on that question compared to the whole group in 2016 SAT.

Weakness in a question (indicated by “W”) means that the school got a lower correct % by 15% or more on that question compared to the whole group in 2016 SAT.

Questions that are poorly answered by the whole group (1/3 correct) are represented by “D”

20 Interpreting graph; Inferring A 28.2 7.1 D, W

21 Making hypothesis B 39.1 39.3 -

22 Understanding; Inferring D 52.2 82.1 S

23 Understanding; Inferring D 81.9 85.7 -

24 Interpreting graph; Inferring C 50.8 57.1 -

Question(s) showing “weaker” performance as compared to WG

WG School

Q. No. Key Most Popular Distractor Key Most Popular Distractor

Q1 D(56.8%)

C(28.2%)

D(33.3%)

C(46.7%)

Q17 B(49.1%)

D(33.6%)

B(30.0%)

D(40.0%)

Q19 B(35.4%)

D(32.5%)

B(20.0%)

D(46.7%)

Q21 B(39.1%)

D(23.8%)

B(20.0%)

A(40.0%)

Analyzing student responses & identifying their problems

Q. No. A B C D Right Ans.Correct

%Wrong

%Diff. Index

Disc. Index

01 5.67% 9.15% 28.23% 56.83% D 56.83% 43.05% 57.96% 0.53

Analysis of Participants’ Performance in Short-response Questions – Appendix II

Mean and standard deviation

Whole Group (WG) School

Mean SD Mean SD

Question 1(out of 10)

5.7 (57%)

2.19 (21.9%)

5.2 (52%)

2.37 (23.7%)

Question 2(out of 8)

3.8 (48%)

1.57 (19.7%)

3.1 (38%)

1.95 (24.4%)

Section B(out of 18)

9.6 (53%)

3.10 (17.2%)

8.3 (46%)

3.77 (20.9%)

Double-digit coding system for Q1(b)(ii)“ Construct a graph to show the relationship of the length of spring and the mass loaded on it.”

CODE ITEMNumber of Response

WG School

Correct Responses :100

(4.0%)0

(0%)

G10A best straight line is drawn through all points (i.e. data points areevenly distributed above & below the line)

100(4.0%)

0(0%)

Incorrect Responses/Nil Response 2404

(96.0%)35

(100%)

G90 No line drawn on the data points40

(1.6%)1

(2.9%)

G91 A point-to-point zigzag line graph/curve is drawn1389

(55.5%)28

(80.0%)

G92 All or some of the points are incorrectly plotted439

(17.5%)3

(8.6%)

G93 A straight line/curve which passes through the origin is drawn.263

(10.5%)0

(0%)

G94 A graph showing a histogram/bar chart/pie chart26

(1.0%)1

(2.9%)

G98 Other mistakes related to graph (e.g. multiple lines)59

(2.4%)0

(0%)

G99 Unattempt188

(7.5%)2

(5.7%)

A99+S99+G99

Unattempt in ALL aspects137

(5.5%)1

(2.9%)

0

0.2

0.4

0.6

0.8

1

1.2

1.4

1.6

1.8

0

5

10

15

20

25

2017 2018 2019

mar

ks

Per

cent

year

Candidates' responses & mean score in plotting graph

scale direct entry

histogram/bar chart/pie chart

mean score

Mean Score of Candidates in Plotting Graph across Year

School-based Report – Appendix IV

(1) Diamond, Gold, Silver and Bronze Awards in 2019

(2) Overall Performance of School

Award type Number

Diamond 130 (5.2%)

Gold 406 (16.2%)

Silver 503 (20.1%)

Bronze 873 (34.9%)

S2 S3Whole Group

(WG)School

Mean score:(out of 54)

26.3 (49%)

28.0 (52%)

27.0 (50%)

26.3 (49%)

Standard deviation:

7.48 (13.8%)

7.75 (14.3%)

7.64 (14.1%)

7.48 (13.8%)

BQ.1 BQ.2

Chinese Version 51.3% 51.6%

English Version 45.5% 45.5%

Mixed 2.5% 2.0%

Unattempt 0.8% 0.9%

Language version adopted by participants in answering questions of Section B:

Student Performance Report (Sample)

Documents received by participating schools

• Student Performance Report

• Certificate of Award for students

• 2019 SAT Question Paper

• Marking Scheme for School

• 2019 SAT Score Summary for School

• School-based Report/ General Report of 2019 SAT