Topic 11Policy Process Studies:
Policy Evaluation Studies
PEDU 6209
Policy Studies in Education
Conceptualization of evaluation Fitzpatrick, Sanders and Worthen define “evaluation
as the identification, clarification, and application of defensible criteria to determine an evaluation object’s value (worth or merit) in relation to those criteria.” (2004, p.5)
Conceptualization of Policy Evaluation
Conceptualization of policy evaluation: Accordingly, the constituents of evaluation consist of
The evaluation object: It is of course public policy or more specifically education policy.
The evaluation criteria: Define the criteria upon which the evaluations are based. They may include • Policy objectives: To assess to what extent the identified policy
objectives have been fulfilled.
• Policy measures and programs: To investigate to what extent the designed policy measures and programs have been implemented
• Policy outcomes: To measure to what extent the targeted outcomes have been realized
• Cost and benefit of the policy: To assess the cost of the implementation of policy means against the benefits entailed by the policy outcome
Conceptualization of Policy Evaluation
Conceptualization of policy evaluation: the constituents of policy evaluation consist of …
The judgment: The evaluation can be studies and judge by means of
• Analytical-technical study
• Interpretive-political study
• Discursive-critical study
Conceptualization of Policy Evaluation
Typology of policy evaluation In regards to the purpose of policy evaluation, policy
evaluation can be categorized into Formative evaluations aim to provide information for
personnel implementing the policy or carrying out the program. Hence, their primary purposes are for the development and improvement of the policy and/or program
Summative evaluations aim to pass judgment on the performance of the policy and/or program. They are primarily carried out as accountability mechanism or auditing measures.
Conceptualization of Policy Evaluation
Typology of policy evaluation In relation to the policy stages, policy evaluation can
be categorized into Need assessments: They the studies conducted at the
formative stages of the policy process. They are conducted either to evaluate the urgencies of the policy problems or the needs that are required if the identified desirable states of affairs are to be realized.
Process-monitoring studies: They are studies carried out to monitor to what extent are the designed policy measures .and programs are implemented in real world situations. They also aim to provide information for the improvement of the effectiveness of the policy instruments in use.
Conceptualization of Policy Evaluation
Typology of policy evaluation In relation to the policy stages, policy evaluation can
be categorized into Outcome studies: They are carried out to assess to what
extent have the preconceived policy outcomes been attained. They can also provide information to the revelations of unanticipated consequences (desirable, undesirable or both) of the policy
Conceptualization of Policy Evaluation
Typology of policy evaluation In terms of the policy context, policy evaluation can
be classified intoEndogenous policy evaluation: They are studies carried out
to assess the internal effectiveness and coherence among various parts of the policy constituents. The most common types of the endogenous evaluation studies are those to assess whether the policy outcomes are in congruence with the policy objectives.
Exogenous policy evaluation: They are studies assessing whether the policy substance are in congruence with the policy environment. These type of evaluation studies are especially informative in ever changing policy environment.
Conceptualization of Policy Evaluation
Frank Fischer’s Conception of Four Levels of Policy Evaluation
The framework of practical logical of policy evaluation: Post-positivist methodological approach to policy evaluation:
By post-positivism, it refers to "a contemporary school of social science that attempts to combine the discourse of social and political theory with the rigor of modern science. It calls for a marriage of scientific knowledge with interpretive and philosophical knowledge about norms and values." (Fischer, 1995, p. 243) Accordingly, Fischer advocates that policy evaluation research should extend beyond the dominance of empirical-positivism and incorporate practical discourse about rightness of normative claims into policy
evaluation research..
Frank Fischer’s Conception of Four Levels of Policy Evaluation
The framework of practical logical of policy evaluation: Fischer names such a post-positivist approach in policy
evaluation the framework of the "Practical Logic of Policy Evaluation".
"It is design to illuminate the basic discursive components of a full or complete evaluation, one which incorporates the full range of both the empirical and normative concerns that can be brought to bear on an evaluation. Structured around four interrelated discourses, the approaches extends from concrete empirical questions pertinent to a particular situation up to the abstract normative issues concerning a way of life." (Fischer, 1995, p. 18)
Frank Fischer’s Conception of Four Levels of Policy Evaluation
The framework of practical logical of policy evaluation: Accordingly, the framework of practical logic of policy
evaluation is made up of First-order evaluation level:
• Technical-analytical discourse
• Contextual discourse
Second-order evaluation level• Systems discourse
• Ideological discourse
Frank Fischer’s Conception of Four Levels of Policy Evaluation
Technical-analytical discourse: Program verification By program verification, it refers to concerns "with
measurement of the efficiency of program outcome. …The basic questions of verification are Does the program empirically fulfill its stated
objective(s)? Does the empirical analysis uncover secondary or
unanticipated effects that offset the program objectives?
Does the program fulfill the objectives more efficiently than alternative means available?" (p. 20)
Frank Fischer’s Conception of Four Levels of Policy Evaluation
Technical-analytical discourse: Program verification "Program verification employs such methodologies as
experimental research and cost-benefit analysis. The goal is to produce a quantitative assessment of the degree to which a program fulfills a particular objective (standard or rule) and a determination (in terms of a comparison of input and output) of how efficiently the objective is fulfill (typically measured as a ratio of costs to benefits) compared to other possible means." (p. 20)
Frank Fischer’s Conception of Four Levels of Policy Evaluation
Contextual discourse: Situational validation Validation focuses on whether or not the particular
program objectives are relevant to the situation. …Instead of measuring program objectives per se, validation examines the conceptualizations and assumptions underlying the problem situation which the program is designed to influence. Validation centers around the following questions: Is the program objective(s) relevant to the problem
situation? Are there circumstances in the situation that require an
exception to be made to the objective(s)? Are two or more criteria equally relevant to the problem
situation?" (Pp. 20-21)
Frank Fischer’s Conception of Four Levels of Policy Evaluation
Contextual discourse: Situational validation "Validation is an interpretive process of reasoning that
takes place within the framework of the normative belief systems brought to bear on the problem situation. Validation draw in particular on qualitative methods, such as those developed for sociological anthropological research." (p.21)
Frank Fischer’s Conception of Four Levels of Policy Evaluation
Systems discourse: Societal vindication "In vindication… the basic task is to show that a policy
goal (from which specific program objectives were drawn) addresses a valuable function for the existing societal arrangement. Vindication is organized around the following questions: Does the policy goal have instrumental or contributive
value for the society as a whole? Does the policy goal result in unanticipated problems
with important societal consequences? Does a commitment to the policy goal lead to
consequence (e.g. benefits and costs) that are judged to be equitably distributed?" (p. 21)
Frank Fischer’s Conception of Four Levels of Policy Evaluation
Systems discourse: Societal vindication As second-order vindication, this type of policy
evaluation "steps outside of the situational action context in which program criteria are applied and implemented in order to assess empirically the instrumental consequences of a policy goal in terms of the system as a whole." (p. 21) Hence, the methodological approach adopted by societal vindication is macroscopic-institutional analysis which usually takes the forms of comparative and/or historical-sociological methods.
Frank Fischer’s Conception of Four Levels of Policy Evaluation
Ideological discourse: Social choice "Social choice seeks to establish and examine the
selection of a critical basis for marking rationally informed choices about societal systems and their respective ways of life. …Social choice raises the following types of questions: Does the fundamental ideal (or ideology) that organize the
accepted social order provide a basis for a legitimate resolution of conflicting judgments?
If the social order is unable to resolve basic values conflicts, do other social orders equitably accommodate the relevant interests and needs that the conflicts reflect?”
Do normative reflection and empirical support the justification and adoption of an alternative ideology and the social order it prescribes?” (p. 22)
Frank Fischer’s Conception of Four Levels of Policy Evaluation
Ideological discourse: Social choice "Social choice involves the interpretive tasks of social
and political critique, particularly as practiced in political theory and philosophy. Most fundamental are the concepts of a 'rational way of life' and 'good society'. Based on the identification and organization of a specific configuration or values ― such as equality, freedom, or community ― models of the good society serve as a basis for the adoption of higher level evaluative criteria." (p. 22) The typical methodological approach adopted by this type of evaluative studies is critical approach to political, social and philosophical inquiry.
Frank Fischer’s Conception of Four Levels of Policy Evaluation
Ideological Choice
Because of
SystemsVindicationBecause of
SituationalValidation
Because of
WarrantSince
Technical VerificationData So, (Qualifier), Conclusion
Critical Discursive
Perspective
Interpretive Political
Perspective
AnalyticTechnical
Perspective
Evaluation Study of MOI Policy for Secondary Schools in HKSAR
Topic 11Policy Process Studies: Policy Evaluation Studies
A policy measure in search of an objective: Summative or even judgmental evaluation of the
efficiency of CMI in comparison with EMI Informative evaluation of the efficiency of preparing
secondary-school leavers to become trilingual and bilateral
Upholding mother-tongue instruction Summative evaluation of the effectiveness of
eliminating mixed-coded teaching in secondary schools in HKSAR
Technical-analytical Discourse: Program verification
“The Objectives of the Longitudinal Study (1999-2002) are
to trace the academic & personal development of students in schools adopting either Chinese or English as the MOI;
to compare the degree of improvement of students’ academic and personal development in schools adopting either Chinese or English as the MOI;
to compare the language ability (in both Chinese & English) of students in schools adopting either Chinese or English as the MOI; and
to identify facilitating & hindering factors affecting students learning in school adopting Chinese as the MOI.”
(p.2 of Tender ref. ED/PR/EMICM/99)
The Objectives of the Longitudinal Study (2002-2004)
Study are to find out The effects of different MOI arrangements at senior levels on
students’ learning, e.g. their academic performance, personal development, language ability and high-order thinking skills;
the major factors that enhance/hinder students’ learning in Chinese or English MOI at senior secondary levels; and
the major factors that enhance/hinder the introduction of English as MOI or in part only at senior secondary levels.
based on the findings, to design/recommend measures that support students’ learning under different MOI arrangement for their whole secondary schooling.
(Tender ref. ED/P&R/EMICMI/02)
To compare the academic and personal development outcomes of students in EMI and CMI schools: Summative, outcome and endogenous evaluation research
Instructional paradox in MOI policy outcomes Socio-psychological paradox in MOI policy outcomes School-organizational paradox in MOI policy outcomes
Technical-analytical Discourse: Program verification
SubjectsSubjects
Cohorts of Students under Study
1999/00 2000/01 2001/02 2002/03 2003/04 2004/05 2005/06
98-Cohort S.2 S.3 S.4 S.5 S.6 S.7
99-Cohort S.1 S.2 S.3 S.4 S.5 S.6 S.7
98 Cohort
99-00
98AAI
F. 2Tests
HKCEE
99AAI
F. 3Tests
F. 4Tests
F. 1Tests
F. 2Tests
F. 3Tests
F. 4Tests
HKCEE
00-01 01-02 02-03 03-04
99 Cohort
HKALE
HKALE
04-05 05-06
Implementation of the MOI Guidance
Conceptual Framework of the Project
Students’ Academic & Socio-economic Background
CHIG
CMID
CLOW
EMI
Students’ Performance & Perception of Learning Environment
A. Academic Development a. Language Subject - Chinese Language - English Language b. Content Subject - Mathematics - Science - Social Subjects
B. Personal Development a. Self-esteem b. Citizenship Development c. Social Efficacy d. Language & learning attitudes
C. Perception of Learning Environment
a. Learning process & motivation b. Teacher’s efficacy c. Classroom climate d. Quality of school life
Teachers’ Perception of Learning Environment A. Sense of efficacy B. Sense of administrative and collegial support C. Sense of reward & recognition from work D. Sense of autonomy & empowerment E. Feeling of stress in work
School Administration & Learning Environment A. Principal’s Leadership B. Organizations of Academic & Disciplinary Orders C. Organizational Climate & Collaborative Culture.
EMI
CMI
EMI
EMI
CMI
CMI
HKCEEResults
Instructional paradox in MOI policy outcomes
(Junior Forms)
Technical-analytical Discourse: Program verification
Summary of HLMs of Effects of Pre-entry Achievement, Socioeconomic Background, Gender &
Sampling Strata on F.3 Achievement Score of English Language in 2000-2001, 98-Cohort
Baseline
Model
Strata
Model 1
Strata
Model 2
Fixed Effects on Intercept
Pupil Level (L1)
Intercept -0.196*** 0.237*** 0.164
Individual AAI 0.497*** 0.499*** 0.499***
Individual SES 0.085*** 0.085*** 0.085***
Being Female 0.334*** 0.335*** 0.336***
School Level (L2)
School Means-AAI 0.361*** 0.184*** 0.257**
School Means-SES 0.181* 0.042 0.032
CMI -- -0.564*** --
CHIG -- -- -0.511***
CMID -- -- -0.522***
CLOW -- -- -0.374*
Random Effects
% of within-school variance explained 24.207% 24.151% 24.157%
% of between-school variance explained 85.873% 92.443% 92.400%
% of total variance explained 65.360% 69.727% 69.699%
* Significant at 0.05 level; ** Significant at 0.01 level; *** Significant at 0.001 level
Summary of HLMs of Effects of Pre-entry Achievement, Socioeconomic Background, Gender &
Sampling Strata on F.3 Achievement Score of English Language in 2001-2002, 99-Cohort
Baseline
Model
Strata
Model 1
Strata
Model 2
Fixed Effects on Intercept
Pupil Level (L1)
Intercept -0.200*** 0.311*** 0.295**
Individual AAI 0.483*** 0.484*** 0.484***
Individual SES 0.089*** 0.089*** 0.089***
Being Female 0.362*** 0.364*** 0.364***
School Level (L2)
School Means-AAI 0.210*** 0.080 0.096
School Means-SES 0.234* 0.023 0.024
CMI -- -0.662*** --
CHIG -- -- -0.657***
CMID -- -- -0.644***
CLOW -- -- -0.625**
Random Effects
% of within-school variance explained 24.535% 24.468% 24.474%
% of between-school variance explained 84.915% 90.864% 90.641%
% of total variance explained 64.290% 68.185% 68.040%
* Significant at 0.05 level; ** Significant at 0.01 level; *** Significant at 0.001 level
Summary of HLMs of Effects of Pre-entry Achievement, Socioeconomic Background, Gender &
Sampling Strata on F.3 Achievement Score of Science (English Version) in 2000-2001, 98-Cohort
* Significant at 0.05 level; ** Significant at 0.01 level; *** Significant at 0.001 level
Baseline
Model
Strata
Model 1
Strata
Model 2
Fixed Effects on Intercept
Pupil Level (L1)
Intercept -0.128** -0.699*** -0.785***
Individual AAI 0.510*** 0.525*** 0.526***
Individual SES -0.002 -0.004 -0.004
Being Female -0.011 -0.025 -0.026
School Level (L2)
School Means-AAI -0.165** -0.053 0.022
School Means-SES -0.052 0.110 0.123
CMI -- 0.729*** --
CHIG -- -- 0.820***
CMID -- -- 0.748***
CLOW -- -- 0.955**
Random Effects
% of within-school variance explained 12.871% 12.855% 12.836%
% of between-school variance explained 13.867% 42.288% 45.657%
% of total variance explained 13.049% 18.123% 18.711%
Summary of HLMs of Effects of Pre-entry Achievement, Socioeconomic Background, Gender &
Sampling Strata on F.3 Achievement Score of Science (English Version) in 2001-2002, 99-Cohort
Baseline
Model
Strata
Model 1
Strata
Model 2
Fixed Effects on Intercept
Pupil Level (L1)
Intercept -0.062 -0.510*** -0.760***
Individual AAI 0.498*** 0.512*** 0.511***
Individual SES 0.001 -0.001 -0.001
Being Female 0.010 0.011 0.011
School Level (L2)
School Means-AAI -0.119* -0.018 0.224*
School Means-SES -0.002 0.086 0.078
CMI -- 0.553*** --
CHIG -- -- 0.684***
CMID -- -- 0.772***
CLOW -- -- 1.160***
Random Effects
% of within-school variance explained 12.777% 12.732% 12.743%
% of between-school variance explained 48.090% 64.704% 71.490%
% of total variance explained 19.487% 22.607% 23.906%
* Significant at 0.05 level; ** Significant at 0.01 level; *** Significant at 0.001 level
Summary of HLMs of Effects of Pre-entry Achievement, Socioeconomic Background, Gender &
Sampling Strata on F.3 Achievement Score of Social Study (English Version) in 2000-2001, 98-Cohort
Baseline
Model
Strata
Model 1
Strata
Model 2
Fixed Effects on Intercept
Pupil Level (L1)
Intercept -0.127** -0.538*** -0.650**
Individual AAI 0.477*** 0.473*** 0.472***
Individual SES -0.005 -0.005 -0.005
Being Female 0.092*** 0.091*** 0.090***
School Level (L2)
School Means-AAI -0.332*** -0.168* -0.057
School Means-SES 0.172 0.253* 0.258*
CMI -- 0.534*** --
CHIG -- -- 0.603***
CMID -- -- 0.614**
CLOW -- -- 0.827*
Random Effects
% of within-school variance explained 12.805% 12.757% 12.751%
% of between-school variance explained 6.561% 27.558% 26.079%
% of total variance explained 11.665% 15.461% 15.185%
* Significant at 0.05 level; ** Significant at 0.01 level; *** Significant at 0.001 level
Summary of HLMs of Effects of Pre-entry Achievement, Socioeconomic Background, Gender &
Sampling Strata on F.3 Achievement Score of Social Study (English Version) in 2001-2002,
99-Cohort
Baseline
Model
Strata
Model 1
Strata
Model 2
Fixed Effects on Intercept
Pupil Level (L1)
Intercept -0.092 -0.547*** -0.640***
Individual AAI 0.428*** 0.421*** 0.420***
Individual SES -0.038** -0.032** -0.032**
Being Female 0.141*** 0.139*** 0.139***
School Level (L2)
School Means-AAI -0.252*** -0.043 0.052
School Means-SES 0.028 0.109 0.098
CMI -- 0.571*** --
CHIG -- -- 0.600***
CMID -- -- 0.684***
CLOW -- -- 0.786**
Random Effects
% of within-school variance explained 16.714% 16.730% 16.723%
% of between-school variance explained 13.091% 49.154% 48.878%
% of total variance explained 15.926% 23.781% 23.716%
* Significant at 0.05 level; ** Significant at 0.01 level; *** Significant at 0.001 level
Instructional paradox in MOI policy outcomes
(Senior Forms; HKCEE)
Technical-analytical Discourse: Program verification
Summary of HLMs of Effects of Pre-entry Achievement, F.3 Corresponding Achievement, Socioeconomic Background,
Gender & Sampling Strata on HKCEE-English Language (Syllabus B) (by score) in 2002-2003, 98-Cohort
Five-Year
Baseline
Model
Five-Year
Strata
Model
Senior-Form
Baseline
Model
Senior-Form
Strata
Model
Fixed Effects on Intercept
Pupil Level (L1)
Intercept -0.294*** 0.047 -0.083*** -0.068*
Individual AAI 0.626*** 0.629*** 0.821*** 0.822***
Individual SES 0.123*** 0.124*** 0.068*** 0.068***
Being Female 0.409*** 0.414*** 0.179*** 0.179***
F.3 English Achievement -- -- 0.444*** 0.444***
School Level (L2)
School Means-AAI 0.298*** 0.145** 0.113*** 0.106**
School Means-SES 0.189* 0.091 0.014 0.010
CMI -- -0.432*** -- -0.018
Random Effects
% of within-school variance explained 28.219% 28.247% 52.918% 52.924%
% of between-school variance explained 91.304% 94.233% 97.107% 97.098%
% of total variance explained 66.486% 68.274% 79.723% 79.720%
* Significant at 0.05 level; ** Significant at 0.01 level; *** Significant at 0.001 level
Summary of HLMs of Effects of Pre-entry Achievement, F.3 Corresponding Achievement, Socioeconomic Background,
Gender & Sampling Strata on HKCEE-English Language (Syllabus B) (by score) in 2003-2004, 99-Cohort
Five-Year
Baseline
Model
Five-Year
Strata
Model
Senior-Form
Baseline
Model
Senior-Form
Strata
Model
Fixed Effects on Intercept
Pupil Level (L1)
Intercept -0.307*** -0.003 -0.110*** -0.073
Individual AAI 0.632*** 0.638*** 0.808*** 0.809***
Individual SES 0.132*** 0.130*** 0.068*** 0.068***
Being Female 0.395*** 0.403*** 0.186*** 0.186***
F.3 English Achievement -- -- 0.461*** 0.461***
School Level (L2)
School Means-AAI 0.266*** 0.092 0.110** 0.087*
School Means-SES 0.184* 0.150** 0.037 0.034
CMI -- -0.360*** -- -0.042
Random Effects
% of within-school variance explained 27.121% 27.117% 50.206% 50.208%
% of between-school variance explained 88.278% 92.119% 95.819% 95.983%
% of total variance explained 62.962% 65.211% 76.937% 77.034%
* Significant at 0.05 level; ** Significant at 0.01 level; *** Significant at 0.001 level
Summary of HLMs of Effects of Pre-entry Achievement, F.3 Corresponding Achievement, Socioeconomic
Background, Gender & Sampling Strata on HKCEE-Physics (by score) in 2002-2003, 98-Cohort
Five-Year
Baseline
Model
Five-Year
Strata
Model
Senior-Form
Baseline
Model
Senior-Form
Strata
Model
Fixed Effects on Intercept
Pupil Level (L1)
Intercept -0.073* 0.178* -0.066* 0.290***
Individual AAI 0.486*** 0.484*** 0.491*** 0.492***
Individual SES 0.022 0.023 0.026* 0.027*
Being Female -0.128*** -0.132*** -0.112*** -0.115***
F.3 Science Achievement -- -- 0.258*** 0.260***
School Level (L2)
School Means-AAI 0.379*** 0.221** 0.346*** 0.171**
School Means-SES 0.069 0.083 0.102 0.084
CMIE -- -0.224** -- -0.381***
CMIC -- -0.341** -- -0.441***
Random Effects
% of within-school variance explained 14.787% 14.754% 24.937% 24.716%
% of between-school variance explained 81.105% 82.220% 83.305% 86.925%
% of total variance explained 45.203% 45.696% 51.707% 53.248%
* Significant at 0.05 level; ** Significant at 0.01 level; *** Significant at 0.001 level
Summary of HLMs of Effects of Pre-entry Achievement, F.3 Corresponding Achievement, Socioeconomic
Background, Gender & Sampling Strata on HKCEE-Physics (by score) in 2003-2004, 99-Cohort
Five-Year
Baseline
Model
Five-Year
Strata
Model
Senior-Form
Baseline
Model
Senior-Form
Strata
Model
Fixed Effects on Intercept
Pupil Level (L1)
Intercept -0.078* 0.228** -0.094** 0.263***
Individual AAI 0.453*** 0.458*** 0.469*** 0.475***
Individual SES 0.022* 0.022* 0.021* 0.020
Being Female -0.101*** -0.099*** -0.090*** -0.091***
F.3 Science Achievement -- -- 0.239*** 0.241***
School Level (L2)
School Means-AAI 0.236*** 0.173** 0.256*** 0.181**
School Means-SES 0.194** 0.135* 0.171* 0.098
CMIE -- -0.331*** -- -0.433***
CMIC -- -0.361** -- -0.388***
Random Effects
% of within-school variance explained 10.685% 10.512% 19.683% 19.588%
% of between-school variance explained 81.260% 82.729% 82.347% 84.746%
% of total variance explained 44.161% 44.766% 49.406% 50.494%
* Significant at 0.05 level; ** Significant at 0.01 level; *** Significant at 0.001 level
Summary of HLMs of Effects of Pre-entry Achievement, F.3 Corresponding Achievement, Socioeconomic
Background, Gender & Sampling Strata on HKCEE-Chemistry (by score) in 2002-2003, 98-Cohort
Five-Year
Baseline
Model
Five-Year
Strata
Model
Senior-Form
Baseline
Model
Senior-Form
Strata
Model
Fixed Effects on Intercept
Pupil Level (L1)
Intercept -0.112** 0.115 -0.112** 0.257***
Individual AAI 0.493*** 0.494*** 0.496*** 0.502***
Individual SES 0.011 0.011 0.016 0.017
Being Female -0.003 -0.002 0.017 0.021
F.3 Science Achievement -- -- 0.273*** 0.277***
School Level (L2)
School Means-AAI 0.237*** 0.153* 0.210*** 0.099
School Means-SES 0.132 0.084 0.173* 0.094
CMIE -- -0.313** -- -0.501***
CMIC -- -0.231* -- -0.389***
Random Effects
% of within-school variance explained 12.125% 11.997% 22.163% 21.912%
% of between-school variance explained 78.541% 79.499% 81.218% 84.833%
% of total variance explained 39.172% 39.486% 46.212% 47.535%
* Significant at 0.05 level; ** Significant at 0.01 level; *** Significant at 0.001 level
Summary of HLMs of Effects of Pre-entry Achievement, F.3 Corresponding Achievement, Socioeconomic
Background, Gender & Sampling Strata on HKCEE-Chemistry (by score) in 2003-2004, 99-Cohort
Five-Year
Baseline
Model
Five-Year
Strata
Model
Senior-Form
Baseline
Model
Senior-Form
Strata
Model
Fixed Effects on Intercept
Pupil Level (L1)
Intercept -0.108** 0.112 -0.131** 0.176*
Individual AAI 0.444*** 0.446*** 0.464*** 0.471***
Individual SES 0.003 0.005 0.000 0.003
Being Female 0.034 0.033 0.044 0.044
F.3 Science Achievement -- -- 0.256*** 0.260***
School Level (L2)
School Means-AAI 0.103 0.062 0.107* 0.074
School Means-SES 0.259* 0.222* 0.296** 0.225*
CMIE -- -0.247* -- -0.394***
CMIC -- -0.259* -- -0.322*
Random Effects
% of within-school variance explained 6.979% 6.894% 16.685% 16.537%
% of between-school variance explained 69.789% 71.792% 72.046% 75.738%
% of total variance explained 33.770% 34.576% 40.299% 41.789%
* Significant at 0.05 level; ** Significant at 0.01 level; *** Significant at 0.001 level
Summary of HLMs of Effects of Pre-entry Achievement, F.3 Corresponding Achievement, Socioeconomic
Background, Gender & Sampling Strata on HKCEE-Geography (by score) in 2002-2003, 98-Cohort
Five-Year
Baseline
Model
Five-Year
Strata
Model
Senior-Form
Baseline
Model
Senior-Form
Strata
Model
Fixed Effects on Intercept
Pupil Level (L1)
Intercept -0.150** -0.239** -0.142** -0.201*
Individual AAI 0.500*** 0.514*** 0.517*** 0.530***
Individual SES 0.011 0.012 0.019 0.018
Being Female 0.155*** 0.160*** 0.137*** 0.145***
F.3 Social Study Achievement -- -- 0.219*** 0.220***
School Level (L2)
School Means-AAI -0.038 0.147* -0.030 0.143*
School Means-SES 0.198 0.053 0.190 0.042
CMIE -- -0.514*** -- -0.563***
CMIC -- 0.313** -- 0.279**
Random Effects
% of within-school variance explained 10.632% 10.419% 15.923% 15.827%
% of between-school variance explained 30.977% 48.979% 37.644% 57.305%
% of total variance explained 17.850% 24.098% 23.628% 30.541%
* Significant at 0.05 level; ** Significant at 0.01 level; *** Significant at 0.001 level
Summary of HLMs of Effects of Pre-entry Achievement, F.3 Corresponding Achievement, Socioeconomic
Background, Gender & Sampling Strata on HKCEE-Geography (by score) in 2003-2004, 99-Cohort
Five-Year
Baseline
Model
Five-Year
Strata
Model
Senior-Form
Baseline
Model
Senior-Form
Strata
Model
Fixed Effects on Intercept
Pupil Level (L1)
Intercept -0.072 -0.260** -0.081 -0.223*
Individual AAI 0.557*** 0.561*** 0.581*** 0.585***
Individual SES 0.029* 0.028* 0.033* 0.032*
Being Female 0.169*** 0.175*** 0.135** 0.140**
F.3 Social Study Achievement -- -- 0.227*** 0.230***
School Level (L2)
School Means-AAI -0.066 0.053 -0.064 0.042
School Means-SES 0.156 0.110 0.205* 0.124
CMIE -- -0.128 -- -0.222
CMIC -- 0.321** -- 0.270**
Random Effects
% of within-school variance explained 10.757% 10.561% 17.798% 17.580%
% of between-school variance explained 39.065% 51.107% 42.312% 55.538%
% of total variance explained 20.021% 23.830% 25.820% 30.001%
* Significant at 0.05 level; ** Significant at 0.01 level; *** Significant at 0.001 level
Summary of HLMs of Effects of Pre-entry Achievement, F.3 Corresponding Achievement, Socioeconomic
Background, Gender & Sampling Strata on HKCEE-History (by score) in 2002-2003, 98-Cohort
Five-Year
Baseline
Model
Five-Year
Strata
Model
Senior-Form
Baseline
Model
Senior-Form
Strata
Model
Fixed Effects on Intercept
Pupil Level (L1)
Intercept -0.014 -0.394*** -0.019 -0.335**
Individual AAI 0.525*** 0.539*** 0.541*** 0.556***
Individual SES 0.016 0.018 0.022 0.025
Being Female 0.034 0.045 0.020 0.029
F.3 Social Study Achievement -- -- 0.233*** 0.236***
School Level (L2)
School Means-AAI -0.002 0.167* 0.027 0.188*
School Means-SES -0.070 -0.081 0.003 -0.050
CMIE -- -0.119 -- -0.202
CMIC -- 0.582*** -- 0.499***
Random Effects
% of within-school variance explained 9.360% 9.436% 16.035% 16.098%
% of between-school variance explained 39.962% 64.495% 44.970% 68.124%
% of total variance explained 18.227% 25.388% 24.418% 31.172%
* Significant at 0.05 level; ** Significant at 0.01 level; *** Significant at 0.001 level
Summary of HLMs of Effects of Pre-entry Achievement, F.3 Corresponding Achievement, Socioeconomic
Background, Gender & Sampling Strata on HKCEE-History (by score) in 2003-2004, 99-Cohort
Five-Year
Baseline
Model
Five-Year
Strata
Model
Senior-Form
Baseline
Model
Senior-Form
Strata
Model
Fixed Effects on Intercept
Pupil Level (L1)
Intercept 0.081 -0.209* 0.081 -0.181
Individual AAI 0.551*** 0.556*** 0.567*** 0.570***
Individual SES 0.027 0.027 0.041* 0.041*
Being Female 0.063 0.069 0.017 0.022
F.3 Social Study Achievement -- -- 0.243*** 0.241***
School Level (L2)
School Means-AAI -0.045 0.102 -0.029 0.116
School Means-SES -0.048 -0.026 -0.022 -0.028
CMIE -- -0.067 -- -0.107
CMIC -- 0.408*** -- 0.369**
Random Effects
% of within-school variance explained 10.681% 10.506% 17.287% 17.153%
% of between-school variance explained 45.140% 55.356% 48.419% 57.613%
% of total variance explained 21.315% 24.348% 26.895% 29.639%
* Significant at 0.05 level; ** Significant at 0.01 level; *** Significant at 0.001 level
Instructional paradox in MOI policy outcomes
(Access to University Education & HKALE)
Technical-analytical Discourse: Program verification
Educational Attainment Measure: JUPAS Minimum entry requirement:1. Grade E or above in either 2 AL subjects or 1 AL subject + 2 AS subjects (other than UE and Chi Lang & Culture)2. Grade E or above in AS UE3. Grade E or above in AS Chi Lang & Culture
Summary of HLMs of Effects of Pre-entry Achievement, Socioeconomic Background, Gender & Sampling Strata on Meeting the Minimum
Entrance Requirement of the 8 JUPAS Participating Institutions in 2004-2005, 98-Cohort
Baseline Model 1 Strata Model 1 Baseline Model 2 Strata Model 2
B Odds Ratio B Odds Ratio B Odds Ratio B Odds Ratio
Fixed Effects on Intercept
Pupil Level (L1)
Intercept -2.391*** 0.092 -2.196*** 0.111 -1.984*** 0.138 -1.645*** 0.193
Individual AAI 1.353*** 3.870 1.364*** 3.912 1.038*** 2.824 1.180*** 3.254
Individual SES -0.075** 0.928 -0.072** 0.931 -0.055* 0.946 -0.039 0.962
Being Female 0.366*** 1.442 0.372*** 1.450 0.246*** 1.279 0.289*** 1.335
School Level (L2)
School Means-AAI 0.576*** 1.779 0.450*** 1.568 0.709*** 2.032 0.328** 1.388
School Means-SES 0.124 1.132 0.090 1.094 0.200 1.222 0.275* 1.316
CMI -- -- -0.227* 0.797 -- -- -- --
CMIE -- -- -- -- -- -- -0.150 0.861
CMIC -- -- -- -- -- -- -0.694*** 0.500
* Significant at 0.05 level; ** Significant at 0.01 level; *** Significant at 0.001 level
Summary of HLMs of Effects of Pre-entry Achievement, Socioeconomic Background, Gender & Sampling Strata on Meeting the Minimum
Entrance Requirement of the 8 JUPAS Participating Institutions in 2005-2006, 99-Cohort
Baseline Model 1 Strata Model 1 Baseline Model 2 Strata Model 2
B Odds Ratio B Odds Ratio B Odds Ratio B Odds Ratio
Fixed Effects on Intercept
Pupil Level (L1)
Intercept -2.607*** 0.074 -2.394*** 0.091 -2.013*** 0.134 -1.687*** 0.185
Individual AAI 1.511*** 4.531 1.518*** 4.564 1.094*** 2.985 1.181*** 3.258
Individual SES -0.086** 0.918 -0.084** 0.919 -0.059* 0.942 -0.048 0.953
Being Female 0.512*** 1.669 0.510*** 1.665 0.330*** 1.391 0.382*** 1.465
School Level (L2)
School Means-AAI 0.674*** 1.962 0.536*** 1.708 0.784*** 2.191 0.484*** 1.622
School Means-SES 0.022 1.023 0.015 1.015 0.148 1.159 0.196* 1.217
CMI -- -- -0.239* 0.787 -- -- -- --
CMIE -- -- -- -- -- -- -0.171 0.843
CMIC -- -- -- -- -- -- -0.690*** 0.501
* Significant at 0.05 level; ** Significant at 0.01 level; *** Significant at 0.001 level
Educational Attainment Measure: JUPAS Maximum entry requirement:1. Grade E or above in either 2 AL subjects or 1 AL subject + 2 AS subjects (other than UE and Chi Lang & Culture)2. Grade D or above in AS UE3. Grade E or above in AS Chi Lang & Culture
Summary of HLMs of Effects of Pre-entry Achievement, Socioeconomic Background, Gender & Sampling Strata on Meeting the Maximum
Entrance Requirement of the 8 JUPAS Participating Institutions in 2004-2005, 98-Cohort
Baseline Model 1 Strata Model 1 Baseline Model 2 Strata Model 2
B Odds Ratio B Odds Ratio B Odds Ratio B Odds Ratio
Fixed Effects on Intercept
Pupil Level (L1)
Intercept -3.118*** 0.044 -2.499*** 0.082 -2.839*** 0.058 -1.948*** 0.143
Individual AAI 0.977*** 2.657 1.089*** 2.973 0.837*** 2.309 1.117*** 3.056
Individual SES -0.099** 0.906 -0.082* 0.921 -0.041 0.960 -0.017 0.983
Being Female 0.443*** 1.557 0.476*** 1.609 0.417*** 1.518 0.512*** 1.669
School Level (L2)
School Means-AAI 1.011*** 2.748 0.494*** 1.640 1.009*** 2.742 0.096 1.101
School Means-SES 0.271 1.311 0.183 1.201 0.349* 1.418 0.405** 1.499
CMI -- -- -0.783*** 0.457 -- -- -- --
CMIE -- -- -- -- -- -- -0.708*** 0.493
CMIC -- -- -- -- -- -- -1.527*** 0.217
* Significant at 0.05 level; ** Significant at 0.01 level; *** Significant at 0.001 level
Summary of HLMs of Effects of Pre-entry Achievement, Socioeconomic Background, Gender & Sampling Strata on Meeting the Maximum
Entrance Requirement of the 8 JUPAS Participating Institutions in 2005-2006, 99-Cohort
Baseline Model 1 Strata Model 1 Baseline Model 2 Strata Model 2
B Odds Ratio B Odds Ratio B Odds Ratio B Odds Ratio
Fixed Effects on Intercept
Pupil Level (L1)
Intercept -4.001*** 0.018 -3.367*** 0.034 -3.042*** 0.048 -2.678*** 0.069
Individual AAI 1.710*** 5.531 1.736*** 5.674 1.193*** 3.297 1.433*** 4.190
Individual SES 0.018 1.018 0.020 1.020 0.042 1.043 0.059 1.061
Being Female 0.605*** 1.832 0.616*** 1.852 0.394*** 1.483 0.520*** 1.682
School Level (L2)
School Means-AAI 1.092*** 2.979 0.599*** 1.820 0.899*** 2.458 0.498*** 1.646
School Means-SES 0.212 1.236 0.157* 1.170 0.324* 1.382 0.325*** 1.385
CMI -- -- -0.609*** 0.544 -- -- -- --
CMIE -- -- -- -- -- -- -0.357** 0.700
CMIC -- -- -- -- -- -- -1.185*** 0.306
* Significant at 0.05 level; ** Significant at 0.01 level; *** Significant at 0.001 level
Summary of HLMs of Effects of Pre-entry Achievement, F.3 Corresponding Achievement, Socioeconomic Background, Gender & Sampling
Strata on HKAS-Use of English (by score) in 2004-2005, 98-Cohort
Seven-Year
Baseline
Model
Seven-Year
Strata
Model 1
Seven-Year
Strata
Model 2
Senior-Form
Baseline
Model
Senior-Form
Strata
Model 1
Senior-Form
Strata
Model 2
Fixed Effects on Intercept
Pupil Level (L1)
Intercept -0.304*** 0.128 0.448*** -0.003 0.101 0.240**
Individual AAI 0.535*** 0.535*** 0.515*** 0.761*** 0.759*** 0.744***
Individual SES 0.109*** 0.108*** 0.107*** 0.061*** 0.061*** 0.063***
Being Female 0.240*** 0.238*** 0.241*** 0.058* 0.056* 0.060*
F.3 English Achievement -- -- -- 0.460*** 0.456*** 0.441***
School Level (L2)
School Means-AAI 0.326*** 0.136* -0.112 0.160** 0.102 -0.005
School Means-SES 0.190 0.111 0.076 0.085 0.072 0.071
CMI -- -0.486*** -- -- -0.117 --
CMIA -- -- -1.440*** -- -- -0.586***
CMIB -- -- -0.613*** -- -- -0.170**
Random Effects
% of within-school variance explained 11.134% 11.228% 11.913% 30.783% 30.843% 30.933%
% of between-school variance explained 89.330% 90.730% 93.120% 94.634% 93.998% 94.522%
% of total variance explained 52.601% 53.388% 54.977% 64.643% 64.334% 64.654%
* Significant at 0.05 level; ** Significant at 0.01 level; *** Significant at 0.001 level
Summary of HLMs of Effects of Pre-entry Achievement, F.3 Corresponding Achievement, Socioeconomic Background, Gender & Sampling
Strata on HKAS-Use of English (by score) in 2005-2006, 99-Cohort
Seven-Year
Baseline
Model
Seven-Year
Strata
Model 1
Seven-Year
Strata
Model 2
Senior-Form
Baseline
Model
Senior-Form
Strata
Model 1
Senior-Form
Strata
Model 2
Fixed Effects on Intercept
Pupil Level (L1)
Intercept -0.446*** 0.024 0.308*** -0.164*** 0.009 0.182**
Individual AAI 0.495*** 0.518*** 0.499*** 0.748*** 0.748*** 0.724***
Individual SES 0.108*** 0.105*** 0.098*** 0.055*** 0.055*** 0.055***
Being Female 0.174*** 0.195*** 0.185*** 0.029 0.036 0.027
F.3 English Achievement -- -- -- 0.489*** 0.484*** 0.462***
School Level (L2)
School Means-AAI 0.310*** 0.121 -0.094 0.189*** 0.085 -0.048
School Means-SES 0.319** 0.234*** 0.161** 0.066 0.056 0.037
CMI -- -0.526*** -- -- -0.195** --
CMIA -- -- -1.391*** -- -- -0.752***
CMIB -- -- -0.560*** -- -- -0.226***
Random Effects
% of within-school variance explained 10.321% 10.037% 10.369% 29.549% 29.632% 29.732%
% of between-school variance explained 78.098% 85.385% 93.157% 91.664% 91.645% 94.428%
% of total variance explained 46.424% 50.173% 54.468% 62.636% 62.665% 64.194%
* Significant at 0.05 level; ** Significant at 0.01 level; *** Significant at 0.001 level
To compare the academic and personal development outcomes of students in EMI and CMI schools: Summative, outcome and endogenous evaluation research
Instructional paradox in MOI policy outcomes Paradox in socio-psychological outcomes School-organizational paradox in MOI policy outcomes
Technical-analytical Discourse: Program verification
Paradox in Socio-psychological Outcomes
Academic Self-concept Attitudes towards English & Motives & Strategies
in English Learning Quality of School Life School Identity and Education Aspiration
Summary of HLMs of Effects of Sampling Strata on Academic Self-Concept in English throughout
Secondary School years, 99-Cohort
F1-English F2-English F3-English F4-English F5-English
Fixed Effects on Intercept
Individual Level
Intercept-grand means 0.123* 0.152* 0.184* 0.139 0.260***
Individual AAI 0.137*** 0.114*** 0.115*** 0.193*** 0.162***
Being Female 0.076** 0.061** 0.041 0.082** 0.037
School Level
School-Means AAI -0.033 -0.017 -0.025 -0.024 -0.088
Sampling Strata
CHIG effect -0.198*** -0.177* -0.249*** -0.264*** -0.339***
CMID effect -0.231*** -0.221** -0.274** -0.181* -0.358***
CLOW effect -0.242* -0.323* -0.335* -0.192 -0.390**
Random Effects
% of within- school variance Explained
1.03% 0.87% 1.02% 1.538% 1.067%
% of between- school variance Explained
42.88% 54.12% 48.97% 64.645% 63.814%
* Significant at 0.05 level; ** Significant at 0.01 level; *** Significant at 0.001 level
Summary of HLMs of Effects of Sampling Strata on Academic Self-Concept in English
throughout Secondary School years, 98-Cohort
F2-English F3-English F4-English F5-English
Fixed Effects on Intercept
Individual Level
Intercept-grand means 0.216** 0.163** 0.256** 0.172**
Individual AAI 0.174*** 0.214*** 0.220*** 0.208***
Being Female 0.047* 0.070** 0.058** 0.082***
School Level
School-Means AAI -0.067 -0.059 -0.138* -0.038
Sampling Strata
CHIG effect -0.318*** -0.280*** -0.355*** -0.297***
CMID effect -0.289*** -0.199** -0.297** -0.232**
CLOW effect -0.324* -0.247* -0.428** -0.268*
Random Effects
% of within-school variance Explained
0.98% 1.36% 1.328% 1.375%
% of between-school variance Explained
68.94% 69.94% 70.284% 82.180%
* Significant at 0.05 level; ** Significant at 0.01 level; *** Significant at 0.001 level
Paradox in Socio-psychological Outcomes
Academic Self-concept Attitudes towards English & Motives & Strategies
in English Learning Quality of School Life School Identity and Education Aspiration
Table 5.16 Summary 1 of HLMs of Effects of Sampling Strata on Language Attitude of learning English in 2001-02, 98-Cohort (F.4)
Attitude towards Learning English Orientation of Learning English Motivational Intensity of Learning English Attitude
Towards
Bilingualism
Interest Importance Instrumental
Orientation Communicative
Orientation
In-school
Motivational
Intensity
Outside
Motivational
Intensity
Eng-Lesson
Motivational
Intensity
Fixed Effects on Intercept
Intercept-grand means -0.091 -0.008 -0.159 -0.155 0.001 -0.051 0.150* 0.186*
Individual AAI 0.109*** 0.197*** 0.167*** 0.122*** 0.074** 0.147*** 0.033 0.194***
Being Female 0.287*** 0.372*** 0.382*** 0.354*** 0.274*** 0.313*** 0.096*** 0.163***
School-Means AAI -0.027 -0.074 0.012 0.009 -0.072 -0.021 -0.036 -0.049
Sampling Strata
CHIG effect -0.003 -0.215** -0.040 -0.015 -0.133* -0.084 -0.230*** -0.336***
CMID effect -0.122 -0.224* -0.085 -0.086 -0.212* -0.144 -0.239** -0.351***
CLOW effect -0.111 -0.212 0.011 -0.018 -0.217 -0.160 -0.301* -0.377*
Random Effects
% of within-school
variance Explained
1.98% 3.72% 3.56% 2.98% 1.60% 2.48% 0.29% 1.43%
% of between-school
variance Explained
25.55% 52.65% 31.86% 18.68% 15.97% 56.42% 44.22% 78.02%
Note 1: Summarized from Sampling Strata Models
* Significant at 0.05 level; ** Significant at 0.01 level; *** Significant at 0.001 level
Table 5.8a Summary 1 of HLMs of Effects of Sampling Strata on Language Attitude of Learning English in 2002-2003, 98-Cohort F5
Attitude towards Learning English
Attitude towards
Bilingualism Model 4
Attitude towards
Bilingualism Model 5
Interest Model 4
Interest Model 5
Essentiality Model 4
Essentiality Model 5
Difficulties Model 4
Difficulties Model 5
Fixed Effects on Intercept
Individual Level (L1)
Intercept-grand means -0.163*** -0.183* -0.158*** 0.012 -0.205*** -0.181* -0.065** 0.189**
Individual AAI 0.112*** 0.111*** 0.136*** 0.136*** 0.123*** 0.122*** 0.055* 0.056*
Being Female 0.313*** 0.313*** 0.319*** 0.318*** 0.403*** 0.402*** 0.122*** 0.123***
School Level (L2)
School-Means AAI 0.032 0.049 0.041 -0.027 0.050 0.062 0.071* -0.048
Sampling Strata
CHIG effect -- 0.061 -- -0.227*** -- -0.032 -- -0.356***
CMID effect -- -0.072 -- -0.249** -- -0.123 -- -0.300***
CLOW effect -- 0.091 -- -0.182 -- 0.067 -- -0.353**
Random Effects
% of within-school variance explained
2.457% 2.445% 2.664% 2.686% 3.938% 3.931% 0.394% 0.397%
% of between-school variance explained
29.206% 36.023% 32.511% 55.067% 38.851% 48.550% 24.944% 71.586%
% of total variance explained
3.734% 4.048% 4.007% 5.043% 5.786% 6.293% 1.264% 2.921%
Note 1: Summarized from ??? * Significant at 0.05 level; ** Significant at 0.01 level; *** Significant at 0.001 level
Table 5.15 Summary 1 of HLMs of Effects of Sampling Strata on Language Attitude of learning English in 2001-02, 99-Cohort (F.3)
Attitude towards Learning English Orientation of Learning English Motivational Intensity of Learning English Attitude
Towards
Bilingualism
Interest Importance Instrumental
Orientation Communicative
Orientation
In-school
Motivational
Intensity
Outside
Motivational
Intensity
Eng-Lesson
Motivational
Intensity
Fixed Effects on Intercept
Intercept-grand means 0.003 0.090 -0.058 -0.051 0.015 0.053 0.174* 0.150*
Individual AAI 0.106*** 0.111*** 0.131*** 0.107*** 0.079*** 0.083*** -0.027 0.099***
Being Female 0.165*** 0.251*** 0.263*** 0.236*** 0.185*** 0.199*** 0.007 0.143***
School-Means AAI -0.020 -0.052 -0.001 -0.000 -0.040 0.005 0.036 0.035
Sampling Strata
CHIG effect -0.103 -0.234** -0.073 -0.062 -0.093 -0.152* -0.222** -0.307***
CMID effect -0.126 -0.297** -0.136 -0.121 -0.166* -0.218* -0.206* -0.321**
CLOW effect -0.171 -0.379* -0.136 -0.141 -0.196 -0.295* -0.284 -0.320*
Random Effects
% of within-school
variance Explained
1.21% 2.58% 2.58% 1.94% 1.17% 1.76% 0.38% 1.29%
% of between-school
variance Explained
35.54% 44.89% 38.36% 35.71% 26.38% 38.55% 0.97% 73.79%
Note 1: Summarized from Sampling Strata Models
* Significant at 0.05 level; ** Significant at 0.01 level; *** Significant at 0.001 level
Table 5.8b Summary 1 of HLMs of Effects of Sampling Strata on Language Attitude of Learning English in 2002-2003, 99-Cohort F4
Attitude towards Learning English
Attitude towards
Bilingualism Model 4
Attitude towards
Bilingualism Model 5
Interest Model 4
Interest Model 5
Essentiality Model 4
Essentiality Model 5
Difficulties Model 4
Difficulties Model 5
Fixed Effects on Intercept
Individual Level (L1)
Intercept-grand means -0.194*** -0.259** -0.182*** -0.029 -0.236*** -0.207* -0.067** 0.160*
Individual AAI 0.148*** 0.149*** 0.136*** 0.134*** 0.155*** 0.155*** 0.059** 0.057**
Being Female 0.343*** 0.344*** 0.340*** 0.335*** 0.441*** 0.441*** 0.130*** 0.128***
School Level (L2)
School-Means AAI -0.001 0.023 -0.009 -0.040 -0.002 -0.009 0.040 -0.020
Sampling Strata
CHIG effect -- 0.101 -- -0.222** -- -0.028 -- -0.309***
CMID effect -- 0.071 -- -0.212* -- -0.089 -- -0.298***
CLOW effect -- 0.065 -- -0.103 -- 0.023 -- -0.248
Random Effects
% of within-school variance explained
3.265% 3.275% 3.070% 3.144% 4.647% 4.652% 0.566% 0.575%
% of between-school variance explained
-0.874% -2.829% -4.980% 15.582% 19.799% 21.674% -7.313% 32.474%
% of total variance explained
3.006% 2.893% 2.770% 3.608% 5.518% 5.629% 0.322% 1.565%
Note 1: Summarized from ??? * Significant at 0.05 level; ** Significant at 0.01 level; *** Significant at 0.001 level
Table 5.8c Summary 1 of HLMs of Effects of Sampling Strata on Language Attitude of Learning English in 2003-2004, 99-Cohort F5
Attitude towards Learning English
Attitude towards
Bilingualism Model 4
Attitude towards
Bilingualism Model 5
Interest Model 4
Interest Model 5
Essentiality Model 4
Essentiality Model 5
Difficulties Model 4
Difficulties Model 5
Fixed Effects on Intercept
Individual Level (L1)
Intercept-grand means -0.212*** -0.284** -0.189*** -0.016 -0.193*** -0.205** -0.087*** 0.212**
Individual AAI 0.104*** 0.104*** 0.112*** 0.112*** 0.110*** 0.109*** 0.025 0.024
Being Female 0.342*** 0.342*** 0.317*** 0.315*** 0.372*** 0.372*** 0.134*** 0.133***
School Level (L2)
School-Means AAI 0.109** 0.169* 0.092* 0.039 0.071* 0.124 0.109** -0.009
Sampling Strata
CHIG effect -- 0.068 -- -0.257*** -- -0.041 -- -0.398***
CMID effect -- 0.066 -- -0.244** -- -0.038 -- -0.368***
CLOW effect -- 0.159 -- -0.169 -- 0.136 -- -0.397*
Random Effects
% of within-school variance explained
2.907% 2.904% 2.888% 2.922% 3.108% 3.107% 0.518% 0.518%
% of between-school variance explained
27.018% 26.129% 29.194% 44.712% 34.244% 38.227% 38.560% 72.174%
% of total variance explained
4.693% 4.624% 4.482% 5.454% 4.771% 4.982% 2.329% 3.929%
Note 1: Summarized from ??? * Significant at 0.05 level; ** Significant at 0.01 level; *** Significant at 0.001 level
Table 5.9 Summary 1 of HLMs of Effects of Sampling Strata on Use of English in 2003-2004, 99-Cohort F5
Use of English with Foreigners
Model 4
Use of English with Foreigners
Model 5
Use of English in Mass Media
Model 4
Use of English in Mass Media
Model 5
Use of English in Class Model 4
Use of English in Class Model 5
Fixed Effects on Intercept
Individual Level (L1)
Intercept-grand means -0.069** 0.095 -0.019 0.111 0.007 0.079
Individual AAI 0.030 0.029 -0.024 -0.025 0.032 0.030
Being Female 0.101*** 0.103*** -0.010 -0.007 -0.023 -0.022
School Level (L2)
School-Means AAI 0.190*** 0.148** 0.180*** 0.149* 0.085** 0.085
Sampling Strata
CHIG effect -- -0.237*** -- -0.217*** -- -0.133*
CMID effect -- -0.254*** -- -0.154 -- -0.113
CLOW effect -- -0.148 -- -0.129 -- -0.026
Random Effects
% of within-school variance explained
0.392% 0.395% 0.292% 0.318% 0.298% 0.350%
% of between-school variance explained
49.925% 68.321% 40.763% 52.952% 7.747% 19.823%
% of total variance explained 3.044% 4.032% 1.883% 2.388% 0.560% 1.034%
Note 1: Summarized from ??? * Significant at 0.05 level; ** Significant at 0.01 level; *** Significant at 0.001 level
Paradox in Socio-psychological Outcomes
Academic Self-concept Attitudes towards English & Motives & Strategies
in English Learning Quality of School Life
Linguistic efficacy Learning Opportunities
School Identity and Education Aspiration
Table 5.18 Summary 1 of HLMs of Effects of Sampling Strata on Quality of School Life in 2000-01, 98-Cohort
General
Satisfaction Teacher-Students
Relations Social
Integration Achievement Adventure
Opportunity
Opportunity
Relating to MOI
Linguistic
Efficacy
Fixed Effects on Intercept
Intercept-grand means -0.065 -0.140 -0.009 -0.052 -0.004 0.160 0.417*** -0.666***
Individual AAI 0.087* 0.163*** 0.205*** 0.174*** 0.142*** 0.133*** -0.004 0.194***
Being Female 0.087* 0.050 0.213*** -0.065* 0.102** 0.017 -0.105*** 0.130***
School-Means AAI 0.067 -0.035 -0.117 -0.116 -0.073 -0.128 -0.016 -0.013
Sampling Strata
CHIG effect -0.027 0.165* -0.088 0.116 -0.051 -0.180* -0.564*** 0.709***
CMID effect -0.001 0.095 -0.155 0.156 -0.058 -0.237* -0.546*** 0.846***
CLOW effect 0.112 0.273 -0.081 0.152 -0.025 -0.224 -0.398* 0.969***
Random Effects
% of within-school
variance Explained
0.47% 1.31% 1.78% 0.85% 0.62% 0.40% 0.51% 1.43%
% of between-school
variance Explained
53.69% -- 14.25% 35.41% 12.72% 35.35% 73.75% 49.76%
Note 1: Summarized from Sampling Strata Models
* Significant at 0.05 level; ** Significant at 0.01 level; *** Significant at 0.001 level
Table 5.17 Summary 1 of HLMs of Effects of Sampling Strata on Quality of School Life in 2000-01, 99-Cohort
General
Satisfaction Teacher-Students
Relations Social
Integration Achievement Adventure
Opportunity
Opportunity
Relating to MOI
Linguistic
Efficacy
Fixed Effects on Intercept
Intercept-grand means -0.032 -0.030 0.035 -0.098 -0.086 0.143 0.418*** -0.610***
Individual AAI 0.014* 0.014* 0.035*** 0.033*** 0.022** 0.053*** 0.011 0.095***
Being Female 0.037*** 0.017* 0.052*** 0.006 0.031*** 0.031** -0.010 0.080***
School-Means AAI 0.178** 0.056 0.119 0.039 0.167* -0.040 -0.055 0.012
Sampling Strata
CHIG effect -0.047 0.048 -0.056 0.086 0.061 -0.168** -0.560*** 0.813***
CMID effect 0.006 -0.017 -0.123 0.122 0.064 -0.256** -0.611*** 0.740***
CLOW effect 0.020 0.038 -0.149 0.165 0.153 -0.290 -0.587*** 0.792***
Random Effects
% of within-school
variance Explained
0.17% 0.11% 0.30% 0.16% 0.14% 0.14% 0.16% 0.90%
% of between-school
variance Explained
38.10% -- 37.77% -- 14.51% 29.78% 49.40% 68.34%
Note 1: Summarized from Sampling Strata Models
* Significant at 0.05 level; ** Significant at 0.01 level; *** Significant at 0.001 level
Paradox in Socio-psychological Outcomes
Academic Self-concept Attitudes towards English & Motives & Strategies
in English Learning Quality of School Life
Linguistic efficacy Learning Opportunities
School Identity and Education Aspiration
99-cohort 2001/2002 (F.3)
Table 4.38 99-Cohort 01F3 Students’ Responses to the statement “If I could choose freely once again, I would choose CMI school.” (%)
Strongly Disagree & Disagree
Either or Strongly Agree & Agree
EMI Students 71.9 18.2 9.9
CHIG Students 39.7 36.5 23.8
CMID Students 32.9 36.8 30.3
CLOW Students 23.9 36.7 39.4
Table 4.39 99-Cohort 01F3 Students’ Responses to the statement “If I could choose freely once again, I
would choose EMI school.” (%)
Strongly Disagree & Disagree
Either or Strongly Agree & Agree
EMI Students 10.0 21.3 68.7
CHIG Students 26.3 37.1 36.6
CMID Students 32.2 36.2 31.5
CLOW Students 42.7 35.3 22.0
98-cohort 2001/2002 (F.4)
Table 4.40 98-Cohort 01F4 Students’ Responses to the statement “If I could choose freely once again, I would choose
CMI school.” (%)
Strongly Disagree & Disagree Either or Strongly Agree & Agree
EMI Students 75.6 16.6 7.8
CHIG Students 46.7 30.3 23.0
CMID Students 35.6 30.9 28.0
CLOW Students 25.8 26.3 36.4
Table 4.41 98-Cohort 01F4 Students’ Responses to the statement “If I could choose freely once again, I would choose
EMI school.” (%)
Strongly Disagree & Disagree Either or Strongly Agree & Agree
EMI Students 8.3 18.4 73.4
CHIG Students 23.1 34.0 42.9
CMID Students 32.2 35.0 32.8
CLOW Students 40.5 35.8 23.4
99-cohort students’ responses to long-term educational aspiration at Form 5 (%)
EMI CHIG CMID CLOW
Graduate from degree program 62.9 41.9 28.3 13.1
Finish Associate-degree
program 8.4 9.4 10.3 7.0
Finish Form-7 program 21.1 30.1 31.9 29.4
Finish Springboard or
equivalent program
2.8 5.5 8.2 15.0
Finish Form-5 program 4.8 13.1 28.3 35.5
Educational Aspiration
98-cohort students’ responses to long-term educational aspiration at Form 5 (%)
EMI CHIG CMID CLOW
Graduate from degree program 62.6 37.6 25.8 14.6
Finish Associate-degree
program 9.2 9.8 10.1 8.2
Finish Form-7 program 20.3 31.6 31.6 28.8
Finish Springboard or
equivalent program
2.3 5.8 9.3 14.3
Finish Form-5 program 5.6 15.2 23.2 34.2
Educational Aspiration
Educational Aspiration
99-cohort students’ responses to next-year’s educational expectation (%)
EMI CHIG CMID CLOW
Continuing to F.6 88.5 74.7 59.3 38.4
Studying in Springboard or
equivalent programs 6.1 11.3 19.4 29.0
Repeating F5 2.0 6.3 8.1 8.9
Seeking employment 3.4 7.7 13.1 23.7
Educational Aspiration
98-cohort students’ responses to next-year’s educational expectation (%)
EMI CHIG CMID CLOW
Continuing to F.6 87.2 68.6 50.7 35.3
Studying in Springboard or
equivalent programs 3.8 6.4 10.1 9.7
Repeating F5 6.0 16.3 24.4 28.8
Seeking employment 2.9 9.0 14.8 26.2
To compare the academic and personal development outcomes of students in EMI and CMI schools: Summative, outcome and endogenous evaluation research
Instructional paradox in MOI policy outcomes Paradox in socio-psychological outcomes School-organizational paradox in MOI policy outcomes
• Paradox between instructional effectiveness and educational advancement
• Paradox between intake-process-output in different phases of secondary-school education
Technical-analytical Discourse: Program verification
Junior-SecondaryInstructional Process
Senior-SecondaryInstructional Process
Six-FormInstructional Process
Form-1 Intakes
HKCEE Results
HKALE Results
To prepare of secondary-school leavers to be trilingual & bilateral Typology of bilingual
Balance bilingual Chinese dominant bilingual English dominant bilingual
Dimensions of bilingual proficiency Cognitive demanding or undemanding Context reduced or embedded
Levels of bilingual proficiency Conversational proficiency Context-reduced and cognitive undemanding proficiency Context-embedded and cognitive demanding proficiency Academic language proficiency
Technical-analytical Discourse: Program Verification
Cognitively Undemanding
Cognitively Demanding
Context Embedded
Context Reduced
A C
B D
Jim Cummins’ Conceptualization of language proficiency
Upholding mother-tongue instruction Elimination of mixed-coded teaching in secondary
schools in HKSAR
at the expenses of Opportunity to learn Opportunity for educational advancement
Technical-analytical Discourse: Program Verification
To identify the stakeholders (e.g. students, parents, teachers and/or administrators in secondary schools, employers, higher-education institutions, etc.) and their “definition of the situations” and “systems of relevance” in regards to the MOI policy
Contextual Discourse: Situational Validation of the Relevance of the MOI Policy
Situational paradoxes facing stakeholders Students and Parents: Situational paradox between immediate
learning efficacy and long-term prospects in educational or even socioeconomic advancement
School teachers and administrators: Situational paradox between instructional efficacy and
graduates' prospects in educational advancements Situation paradox between current instructional efficacy and
future efficacy of attracting "high-quality" primary-school leavers or even future prospect of survival of the schools.
Employers: The situational paradox is between the prospects of local and global markets.
Higher-education institution: The situation paradox is between short-term instructional efficacy and long-term development of universally applicable scholarship and/or professional competence
Contextual Discourse: Situational Validation of the Relevance of the MOI Policy
To validate whether or to what extent each of the stakeholders’ “systems of relevance” have been fulfilled by the MOI policy measure.
Contextual Discourse: Situational Validation of the Relevance of the MOI Policy
Systemic vindication of value orientation of the MOI policy To define the institutional features of the social, political
and economic systems of HKSAR: Biglossic or even triglossic structures of a post-colonial society, one-country-two-system polity, and open and small economy in global-informational capitalism.
To evaluate whether or to what extent that MOI policy have instrumental or contributive value for these systemic features of HKSAR
System Discourse: Societal Vindication of the MOI Policy
Localcommunityof Mother Tongue
Global professional-intellectual community
Global- informationaleconomy
Global- politicalarena
Localtertiary educationinstitution
Intra-nationaltertiary educationinstitution
PrimarySchoolSystem
Internationaltertiary educationinstitution
Intra-nationalpolitical &economicsystems
Localpolitical &economicsystems
SecondarySchoolSystem
Institutional features of the MOI policy environment (socio-cultural, political and economic systems) of HKSAR
Localcommunityof Mother Tongue
Global professional-intellectual community
Global- informationaleconomy
Global- politicalarena
Localtertiary educationinstitution
Intra-nationaltertiary educationinstitution
PrimarySchoolSystem
Internationaltertiary educationinstitution
Intra-nationalpolitical &economicsystems
Localpolitical &economicsystems
SecondarySchoolSystem
MonolingualLocalcommunityof Cantonese
Trilingual systemwithCantoneseDominance
BilingualSystemwithEnglishDominance
Trilingual systemwithEnglish/PutonghuaDominance
To vindicate the intensification effect of MOI policy on the contradictory structure of HKSAR Biglossic or even triglossic structures of the open and small
economy of HK within the global-informational capitalism. Biglossic structure of the political system of HKSAR within the
context of One Country Two System Biglossic or even troglossic structure of the cultural system of
a post-colonial society
To vindicate the unequalizing effect of MOI policy on the structural disparity among the 18 school districts in the SSPA system in terms of distribution of EMI school places
System Discourse: Societal Vindication of the MOI Policy
香港中學學位分配辦法中十八個學校網之間的總學額及英文學額分佈情況(1998) 升讀英中機會率
本網學生可享用的
總學額
本網學生可享用的
總英文學額
本網小學生升讀
英中的機會率
本網的英中學額
(0.27)差距比率
本網的英中學生
(0.4)差距比率
香港
中西區 2957 1460 0.4937 +0.8341 +0.2344
灣仔區 2535 1210 0.4773 +0.7731 +0.1933
東區 5460 1317 0.2412 -0.1040 -0.3970
南區 2142 436 0.2035 -0.2439 -0.4911
九龍
油尖旺區 3604 1307 0.3627 +0.3471 -0.0934
深水 區 3683 1266 0.3437 +0.2769 -0.1406
九龍城區 6117 2491 0.4072 +0.5127 +0.0181
黃大仙區 4038 1262 0.3125 +0.1610 -0.2187
觀塘區 5187 1255 0.242 -0.1012 -0.3951
新界
葵青區 5182 942 0.1818 -0.3247 -0.5455
荃灣區 2841 665 0.2341 -0.1305 -0.4148
屯門區 6045 1212 0.2005 -0.2552 -0.4988
元朗區 5068 876 0.1728 -0.3579 -0.5679
北區 3786 576 0.1521 -0.4348 -0.6197
大埔區 4475 1159 0.2590 -0.0379 -0.3525
沙田區 6885 2164 0.3143 +0.1676 -0.2142
西貢區 2642 232 0.0878 -0.6738 -0.7805
離島區 1068 12 0.0112 -0.9583 -0.9719
香港中學學位分配辦法中十八個學校網之間的總學額及英文學額分佈情況(2005) 升讀英中機會率
本網學生可享用的
總學額
本網學生可享用的
總英文學額
本網小學生升讀
英中的機會率
本網的英中學額
(0.26)差距比率
本網的英中學生
(0.4)差距比率
香港
中西區 2370 1196 0.5046 +0.9178 +0.2616
灣仔區 2077 1052 0.5065 +0.9248 +0.2662
東區 3988 921 0.2309 -0.1224 -0.4226
南區 1463 301 0.2057 -0.2181 -0.4856
九龍
油尖旺區 3012 1155 0.3835 +0.4573 -0.0413
深水 區 3317 1295 0.3904 +0.4837 -0.0240
九龍城區 5477 2142 0.3911 +0.4862 -0.0223
黃大仙區 3639 854 0.2347 -0.1082 -0.4133
觀塘區 4892 1060 0.2167 -0.1766 -0.4583
新界
葵青區 4444 884 0.1989 -0.2441 -0.5027
荃灣區 2361 483 0.2046 -0.2226 -0.4886
屯門區 5456 1057 0.1937 -0.2638 -0.5157
元朗區 6200 936 0.1510 -0.4263 -0.6226
北區 3341 560 0.1676 -0.3630 -0.5810
大埔區 2756 896 0.3251 +0.2355 -0.1872
沙田區 5098 1602 0.3142 +0.1942 -0.2144
西貢區 2732 277 0.1014 -0.6147 -0.7465
離島區 849 31 0.0365 -0.8612 -0.9087
香港中學學位分配辦法中十八學校網之間總學額及英文學額分佈情況(2007)
升讀英中機會率
本網學生可享用的
總學額
本網學生可享用的
總英文學額
本網小學生升讀
英中的機會率
本網的英中學生
(.28) 差距比率
本網的英中學生
(0.4)差距比率
香港
中西區 1918 1012 0.5276 +0.9012 +0.3191
灣仔區 1564 901 0.5761 +1.0758 +0.4402
東區 2899 746 0.2573 -0.0728 -0.3567
南區 1051 239 0.2274 -0.1806 -0.4315
九龍
油尖旺區 2270 890 0.3921 +0.4128 -0.0198
深水埗區 2626 932 0.3549 +0.2789 -0.1127
九龍城區 4142 1727 0.4169 +0.5024 +0.0424
黃大仙區 2651 674 0.2542 -0.0839 -0.3644
觀塘區 3750 774 0.2064 -0.2563 -0.4840
新界
葵青區 3318 646 0.1947 -0.2984 -0.5133
荃灣區 1850 384 0.2076 -0.2521 -0.4811
屯門區 3809 741 0.1945 -0.2990 -0.5137
元朗區 4555 797 0.1750 -0.3695 -0.5626
北區 2438 470 0.1928 -0.3053 -0.5180
大埔區 2196 801 0.3648 +0.3143 -0.0881
沙田區 3794 1288 0.3395 +0.2233 -0.1513
西貢區 2068 140 0.0677 -0.7561 -0.8308
離島區 629 28 0.0445 -0.8396 -0.8887
To vindicate the institutionalized effect of MOI policy on the inequality of mobility chances between CMI and EMI students in competing access to higher education.
System Discourse: Societal Vindication of the MOI Policy
Figure 1: Summary of Odd Ratio of CMI (with reference to EMI) in Models accounting for Educational Attainment Levels in Table 2 to 6
2AL+Chi(E)+UE(D)
2AL+Chi(E)+UE(E)
98 cohort
CMI .424
99 cohort
CMI .560
14-pts+Chi+Eng
98 cohort
CMI .774
99 cohort
CMI .774
8-pts+Chi+Eng
98 cohort
CMI .647
99 cohort
CMI .779
5-pts+Chi+Eng
98 cohort
CMI .331
99 cohort
CMI .488
Sit for
HKCEE
98 cohort
CMI .298
99 cohort
CMI .439
All odd ratios are statistically significant except those are in italic
Figure2: Summary of Odd Ratio of CMIC and CMIE (with reference to EMI) in Models accounting for Educational Attainment Levels in Table 2 to 6
2AL+Chi(E)+UE(D)
2AL+Chi(E)+UE(E)
98cohort
CMIE .494
CMIC .218
99 cohort
CMIE .697
CMIC .304
14-pts+Chi+Eng
98cohort
CMIE .863
CMIC .500
99 cohort
CMIE .845
CMIC .508
8-pts+Chi+Eng
98cohort
CMIE .647
CMIC .516
99 cohort
CMIE .806
CMIC .574
5-pts+Chi+Eng
98 cohort
CMIE .374
CMIC .237
99 cohort
CMIE .602
CMIC .328
Sit for
HKCEE
98 cohort
CMIE .353
CMIC .323
99 cohort
CMIE .564
CMIE .280
All odd ratios are statistically significant except those are in italic
To identify the ideological stance underlying the MOI policy discourse Orientations to bilingualism
Bilingualism as problem• Linguistic imperialism• Linguistic nationalism
Bilingualism as rights Bilingualism as resource
Power implications of bilingualism Bilingualism as coercive power Bilingualism as collaborative power
Structure-agent stance in bilingualism Structural imposition stance Agent resistance stance Agent appropriation stance
Ideological Discourse: Social Choice underlying the MOI policy
To identify the ideological stance underlying the MOI policy discourse The myth of mother tongue
Ideological Discourse: Social Choice underlying the MOI policy
Definitions of Mother Tongue Criteria Definition
Origin the language one learned first (The language one has
established the first long-lasting verbal contacts in)
Identification
a. Internal
b. External
a. the language one identifies with/ as a native speaker of
b. the language one is identified with/as a native speaker of
by others
Competence The language one knows best
Function The language one uses most
Source: Skutnabb-Kangas, Tove (2000) Linguistic Genocide in Education or Worldwide Diversity and Human Rights? Mahwah, NJ: Lawrence Erlbaum Associates. P. 106
To reflect and choose the ideological stances toward Cantonese, Poutonghau and English, which have been embedded in the MOI policy discourse in post-1997 HK.
Ideological Discourse: Social Choice underlying the MOI policy
Evaluation Study of MOI Policy for Secondary Schools in HKSAR
Mother Tongue or
Other Tongue
END
Topic 11Policy Process Studies:
Policy Evaluation Studies
Top Related