Educational Administration Quarterly 1 –35
© The Author(s) 2016Reprints and permissions:
sagepub.com/journalsPermissions.nav DOI: 10.1177/0013161X16638430
eaq.sagepub.com
Review
Gender Differences in Instructional Leadership: A Meta-Analytic Review of Studies Using the Principal Instructional Management Rating Scale
Philip Hallinger1,2, Li Dongyu3, and Wen-Chung Wang3
AbstractPurpose: Instructional leadership has assumed steadily increasing importance within the general role set of principals over the past 60 years. One persisting finding within this corpus of studies concerns the consistently higher ratings obtained by female principals on instructional leadership when compared with their male counterparts. This article employed meta-analysis first to test if there are significant differences in perceptions of the instructional leadership practices of male and female principals. Method: Then, the results were further analyzed to describe the nature of differences that were revealed in the first-stage analysis. The database for the meta-analysis consisted of 40 data sets drawn from 28 studies that had used the Principal Instructional Management Rating Scale (PIMRS) in studies of gender and instructional leadership. The data sets comprised perception data collected variously from principals and teachers on more than 2,000 principals between 1983 and 2014. The data were analyzed at three different construct levels measured by the PIMRS instrument:
1Chulalongkorn University, Bangkok, Thailand2University of Johannesburg, Johannesburg, South Africa3Hong Kong Institute of Education, Tai Po, Hong Kong
Corresponding Author:Philip Hallinger, Faculty of Education, Chulalongkorn University, 254 Phayathai Road, Bangkok 10330, Thailand. Email: [email protected]
638430 EAQXXX10.1177/0013161X16638430Educational Administration QuarterlyHallinger et al.research-article2016
by guest on March 16, 2016eaq.sagepub.comDownloaded from
2 Educational Administration Quarterly
PIMRS Total Score, three dimensions, 10 functions. Results: The meta-analysis yielded statistically significant gender differences in instructional leadership (Cohen’s d = .288, p < .01) when the results of all studies were combined. The direction of the findings suggested that female principals engaged in more active instructional leadership than male counterparts. Implications: The gender differences in instructional leadership are consistent with the broader results of the meta-analytic literature on gender and leadership. Implications are discussed for both research and practice.
Keywordsinstructional leadership, gender, principal, school leadership, meta-analysis, PIMRS
In 2015, the requirement for principals to assume central responsibility for instructional leadership pervades education systems throughout the world (Hallinger & Lee, 2013; Hallinger & Wang, 2015; Robinson, Lloyd, & Rowe, 2008). A key line of empirical inquiry in educational leadership has focused on identifying how “personal characteristics” of principals influence their role behavior (e.g., Bossert, Dwyer, Rowan, & Lee, 1982; Goldring, Huff, May, & Camburn, 2008; Hallinger, 2011; Hallinger & Heck, 1996; Leithwood, Begley, & Cousins, 1990; Leithwood & Jantzi, 2008). One principal charac-teristic that has maintained the interest of scholars for more than 50 years is gender (Charters & Jovick, 1981; Collard, 2001; Frasher & Frasher, 1979; Krüger, 1996, 2008; Shakeshaft, 2006).
Interest in how gender shapes leadership is linked to both equity and instrumental concerns (Krüger, 1996, 2008; Pounder & Coleman, 2002; Shakeshaft, 2006). In summarizing the early literature in this domain, Eagly, Karau, and Johnson (1992) concluded that females tend to employ a more participatory and task-focused style of leadership than male principals. Moreover, they observed that
occupying a role that is congenial in gender-relevant terms may have gains for school leaders in terms of some increase in their tendency to organize activities to accomplish relevant tasks. As school principals, women may encounter role expectations that are especially congenial with their own gender role. (p. 92)
Subsequent research has focused more specifically on gender differences in the enactment of instructional leadership (Hallinger, 2011). Although scholars have suggested that female principals may be more active instruc-tional leaders, this latter generation of studies has never been systematically reviewed (Hallinger, 2011; Hallinger & Murphy, 1985; Krüger, 2008).
by guest on March 16, 2016eaq.sagepub.comDownloaded from
Hallinger et al. 3
In this review of research, we sought to determine if there were differences in the levels and patterns of instructional leadership practiced by male and female principals. We used meta-analysis to synthesize results from a corpus of 40 independent data sets drawn from 28 studies.1 All the studies had employed the Principal Instructional Management Rating Scale (PIMRS; Hallinger, 1982/1990) for data collection. Our sample included data collected from more than 6,000 teachers and 2,800 principals in three countries.
Meta-analysis is playing an increasingly important role in demarking knowledge accumulation in educational leadership (e.g., Hallinger, Wang, & Chen, 2013; Leithwood & Sun, 2012; Robinson et al., 2008; Scheerens, 2012; Sun & Leithwood, 2015; Witziers, Bosker, & Kruger, 2003). Meta-analysis offers a more systematic means of integrating findings from a body of empirical studies than traditional methods of research review and is, there-fore, increasingly prominent in policy-oriented research (Hallinger, 2014; Lipsey & Wilson, 2001). Thus, we undertook this study in the hope that our findings would advance a topic of continuing interest in our field.
Theoretical Perspective
We begin by providing an overview of research on principal instructional leadership. The study focuses specifically on studies that employed the PIMRS, so we also devote some attention to discussing its underlying con-ceptual framework. Finally, we provide an overview of previous research on the relationship between gender and principal instructional leadership.
General Perspectives on Principal Instructional Leadership
Among the global trends in educational leadership and management that emerged over the past 30 years, few have been more significant, widespread, or persistent than the focus on understanding linkages between leadership and learning (Bossert et al., 1982; Hallinger & Heck, 1996; Heck & Hallinger, 2014; Robinson et al., 2008; Sun & Leithwood, 2015; Witziers et al., 2003). The genealogy of scholarship on “instructional leadership” can be traced to the conceptual efforts of Bridges (1967) and the empirical research of Gross and Trask (1964) in the United States. However, it was the “effective schools movement” (Edmonds, 1979) that served as a catalyst for more concerted and sustained efforts to understand if and how principal leadership makes a dif-ference in student learning (Bossert et al., 1982; Erickson, 1979; Hallinger & Murphy, 1985; Leithwood et al., 1990). For example, in 1979, Donald Erickson authored a pointed and, strange as it may seem today, controversial proposal for the field of educational administration.
by guest on March 16, 2016eaq.sagepub.comDownloaded from
4 Educational Administration Quarterly
Three years ago I opined that the most promising relevant work, largely ignored by scholars identified with “educational administration,” was the work on “school effects.” The literature during the last three years has further reinforced my dual conviction that “school effects” studies, broadly defined, represent the current leading edge in the research domain I am assessing, and that few scholars affiliated with “educational administration” are taking note of them, though nothing could be more profoundly pertinent than the school effects studies to the consequence of educational organization. (Erickson, 1979, p. 10)
With hindsight, Erickson’s judgments 30 years ago were nothing less than prescient. Reviews of research conducted during this watershed period resulted in a significant uptick in both the quantity and quality of studies of principal instructional leadership (e.g., Bossert et al., 1982; Bridges, 1982; Erickson, 1979, Leithwood & Montgomery 1982). Among the most influential conceptu-alizations of leadership and learning to emerge from this era was the “instruc-tional management model” developed by Bossert et al. (1982). They proposed that the effects of principal instructional management on learning were both moderated by characteristics of the principal and his/her environment and medi-ated by features of the school organization and learning climate.
Consistent with this model, other scholars identified a number of relevant personal characteristics of principals including gender, personality type, self-efficacy, years of teaching and administrative experience, prior training, and race (e.g., Bridges, 1982; Eagly et al., 1992; Erickson, 1979; Hallinger & Murphy, 1985; Leithwood & Montgomery, 1982). During the ensuing decades, these personal characteristics have been studied extensively in rela-tion to their role in shaping the leadership behavior of school principals (e.g., Coleman, 2007; Goldring et al., 2008; Hallinger, 2011; Hallinger & Heck, 1996; Leithwood et al., 1990). The current study’s attempt to examine the relationship between gender and principal instructional leadership is located within this broader perspective on leadership and learning.
The PIMRS Conceptual Framework
Concurrent with publication of the Bossert model, Hallinger and Murphy (1985) developed a more explicit conceptual framework aimed at defining the instructional leadership role of the principal (see also Hallinger, Murphy, Weil, Mesa, & Mitman, 1983). The framework proposed three “dimensions” for this leadership role: Defines the School’s Mission, Manages the Instructional Program, and Develops a Positive School Learning Climate (see Figure 1). The first dimension, Defines the School’s Mission, centers on the principal’s role working to ensure that the school has a clear mission
by guest on March 16, 2016eaq.sagepub.comDownloaded from
Hallinger et al. 5
focused on the academic progress of students. Murphy and Torre (2014) assert that clear mission and goals represent “essential scaffolding” for school improvement. While this dimension does not assume that the principal defines the school’s mission alone, it does propose that the principal is responsible for ensuring that an academic mission exists and ensuring its effective communication to staff, students, and the community (Edmonds, 1979; Hallinger & Heck, 1996, 2002; Hallinger et al., 1983; Robinson et al., 2008; Scheerens, 2012; Sun & Leithwood, 2015).
The second dimension, Manages the Instructional Program, focuses on the principal role in “managing the technical core” of the school, learning, and teaching (Hallinger & Murphy, 1985). Although the principal must share and delegate many tasks involved in monitoring and developing the school’s instructional program, overall coordination remains a key leadership respon-sibility of the principal (Bossert et al., 1982; Edmonds, 1979; Hallinger & Heck, 1996; Hallinger & Murphy, 1985, 2012; Robinson et al., 2008). The third dimension, Develops a Positive School Learning Climate, conforms to the notion that successful schools create an “academic press” through the development of a school climate characterized by high standards and expec-tations, capacity development, and continuous improvement (Edmonds, 1979; Hallinger & Heck, 1996; Hallinger et al., 1983; Leithwood & Montgomery, 1982; Leithwood & Sun, 2012; Robinson et al., 2008).
Figure 1. PIMRS instructional leadership model (Hallinger & Murphy, 1985).
by guest on March 16, 2016eaq.sagepub.comDownloaded from
6 Educational Administration Quarterly
Gender and School Leadership
Researchers first reported differences in patterns of leadership exercised by male and female school principals in the 1960s (e.g., Gross & Trask, 1964; Hemphill, Griffiths, & Frederiksen, 1962). Subsequent scholarship con-ducted during the 1970s and 1980s sought to amplify initial findings using a variety of leadership constructs including task/interpersonal leadership orien-tation, democratic/autocratic leadership orientation, initiating structure/con-sideration, and participatory/directive leadership style (e.g., Adkinson, 1981; Arcy, 1980; Charters & Jovick, 1981; Fischel & Pottker, 1977; Frasher & Frasher, 1979; Gross & Trask, 1976; Salley, McPherson, & Baehr, 1979). In 1992, Eagly and colleagues conducted a meta-analytic synthesis of this early generation of research and concluded that female principals tended to adopt more participatory, democratic, task-focused leadership styles than their male counterparts. In a subsequent meta-analysis conducted a decade hence, Eagly, Johannesen-Schmidt, and van Engen (2003) found that female leaders tended to achieve stronger ratings on transformational leadership as well as engag-ing in more contingent reward behaviors associated with transactional leadership.
Another parallel line of inquiry into gender and school leadership evolved over time focusing on how gender shapes the principal’s exercise of “instruc-tional leadership.” Taking a cue from the research of Gross and Trask (1976), Hallinger and Murphy (1985) reported that “the similarity in findings con-cerning the gender variable in this and previous [school leadership] studies suggests that it is worth examining more closely in future research” (p. 234). This stimulated subsequent empirical inquiry into gender differences in prin-cipal instructional leadership (e.g., Cunningham, 2004; Geiselman, 2004; Howell, 1989; Krüger, 1996, 2008; McCabe, 1993; Miller, 1991; Munroe, 2009; Nogay & Beebe, 2008; O’Donnell, 2002; Schoch, 1992; Trout, 1985). Thus, 25 years later, Hallinger (2011) recommended that this topic was also ready for meta-analytic review.
Methodology
When a domain of research evidences a concentration of quantitative studies that have analyzed the relationship between two variables, meta-analysis offers a powerful means of quantitatively integrating substantive findings (Glass, 1977). Lipsey and Wilson (2001) noted that the analytical power of meta-analysis is magnified when studies have used the same measures of the relevant variables. The current study used meta-analysis to synthesize quan-titative findings drawn from a corpus of studies all of which had employed
by guest on March 16, 2016eaq.sagepub.comDownloaded from
Hallinger et al. 7
the same means of measuring the independent variable (i.e., gender) and the dependent variable of instructional leadership (i.e., PIMRS ratings).
The Principal Instructional Management Rating Scale
Hallinger and Murphy’s (1985) instructional leadership framework paved the way for subsequent development of the PIMRS (Hallinger, 1982/1990). The PIMRS consists of 50 “behaviorally anchored” items (Latham & Wexley, 1981) designed to measure the 3 dimensions and 10 functions shown earlier in Figure 1. Three parallel forms of the PIMRS have been developed: Principal Form (self-assessment), Teacher Form, and Supervisor Form. The items that comprise each form are identical; only the stems change to reflect the differing perspectives of the role groups (Hallinger & Murphy, 1985; Hallinger & Wang, 2015).
For each item, the rater assesses the frequency with which the principal has been observed by the respondent to enact a particular instructional leadership behavior. The items employ a Likert-type scale ranging from (1) almost never to (5) almost always. The instrument can be scored by calculating scores for the full test and/or for the dimension/function-level constructs. The resulting data is used to generate profiles that describe the level of principal engage-ment in the dimensions comprising this role (Hallinger & Wang, 2015).
The PIMRS Principal and Teacher Forms have been tested extensively for reliability and validity (Hallinger & Wang, 2015; Hallinger et al., 2013). A recent meta-analytic study of 40+ PIMRS studies found a full-scale alpha reliability (Cronbach, 1951) of .96 for the Principal Form and a Generalizability Theory reliability rho hat (Kane, Gilmore, & Crooks, 1976) of .99 for the Teacher Form (Hallinger et al., 2013). All three dimension-level reliability coefficients were found to exceed .90 for both forms. Function-level reliabil-ity coefficients ranged from .75 to .86 for the Principal Form and from .90 to .95 for the Teacher Form (Hallinger et al., 2013).
Hallinger and Wang (2015) assessed the validity of the PIMRS using five separate validation procedures (Cronbach, 1988; Nunnally & Bernstein, 1994). “Content validity” was supported through judgments of constructs and items by school professionals (Hallinger & Wang, 2015). Construct validity was assessed at the “scale level” through school document analysis, analysis of intercorrelations among subscales, and confirmatory factor analy-sis (Kline, 2013). Construct validity was further assessed at the “item level” using Rasch analysis (Wright, Linacre, Gustafen, & Martin-Lof, 1994). Hallinger and Wang (2015) reported that 90% of the items comprising the 3 dimensions and 10 functions “fit” the proposed subscale structure. Further analysis demonstrated measurement invariance, implying that PIMRS scores
by guest on March 16, 2016eaq.sagepub.comDownloaded from
8 Educational Administration Quarterly
can be equated for principals across school levels. In sum, the PIMRS meets standards of reliability and validity commonly applied for research instru-ments (Cronbach, 1988; Nunnally & Bernstein, 1994).
Identification of Sources for Meta-Analysis
The authors established the following criteria for inclusion of studies in the meta-analysis.
•• Studies of gender and principal instructional leadership that had used the PIMRS for data collection
•• PIMRS studies produced as master theses, doctoral dissertations, or journal articles
•• Studies beginning in 1983, the publication year of the first PIMRS study, and continuing to 2014.
A variety of approaches were employed to identify relevant studies. We examined recently published reviews of related literature, and also employed Google Scholar™ to identify relevant journal articles. Knowing that many PIMRS studies have been conducted as doctoral dissertations, we also searched ProQuest’s Dissertations Express™. Finally, we obtained a number of raw data sets from the publisher.
After a careful review of potential sources, we identified 28 studies con-taining suitable data for meta-analysis. All were doctoral dissertations, though three had also been published as journal articles (i.e., Hallinger & Murphy, 1985; Nogay & Beebe, 2008; O’Donnell & White, 2005). Several authors had administered the PIMRS to both teachers and principals. Therefore, we concluded the search with a larger number of data sets (i.e., 40) than studies (i.e., 28; see Table 2). The total sample for this analysis consisted of 2,807 principals (1,586 males and 1,221 females) and 6,175 teacher respondents covering elementary, middle, and high schools from three countries (i.e., United States, Thailand, Taiwan).
Thirteen data sets had been collected with the PIMRS Teacher Form and 27 data sets with the PIMRS Principal Form. Since Hallinger and Wang (2015) noted that scores generated from different PIMRS forms cannot be treated as equivalent, our approach to using data collected with the two dif-ferent forms requires elaboration. First, our meta-analysis was based on tests of difference and significance rather than comparisons of absolute score lev-els (Glass, 1977; Hedges & Olkin, 1985; Lipsey & Wilson, 2001). Second, even when the teacher and principal data sets had been obtained from a single study the data were collected from different respondents. Therefore, we were
by guest on March 16, 2016eaq.sagepub.comDownloaded from
Hallinger et al. 9
able to treat the data sets as “independent samples” and combined for the purposes of analysis. Nonetheless, concerns over the equivalency of data obtained from the PIMRS Teacher and Principal Forms implied the need to examine this issue empirically (see data analysis section below).
There were 17 data sets where results were reported for the full scale (i.e., PIMRS Total Score), 16 for the three dimensions, and 31 for the 10 functions (see Table 1). The varying approaches to reporting PIMRS results represented a practical constraint since one cannot transform dimension- or function-level scores (e.g., means and SDs) extracted from data tables into full scale scores. This constraint is common in meta-analysis and reduced the sample size for our meta-analytic procedures (Lipsey & Wilson, 2001).
In any research synthesis or meta-analysis, the composition of the studies is a salient consideration both for the researchers and readers (Hallinger, 2014; Hallinger & Wang, 2015; Lipsey & Wilson, 2001). Confidence in the findings of meta-analysis is only as high as the quality of the studies that contributed data toward the results. Doctoral dissertations, while vetted by the granting universities, do not typically employ review procedures as rigor-ous as those used by refereed journals (e.g., double-blind peer review). Although scholars actually recommend the inclusion of dissertations in meta-analysis to reduce publication bias (Sutton, Song, Gilbody, & Abrams, 2000), meta-analyses that rely solely or heavily on doctoral dissertations come under greater scrutiny with respect to data quality. The current meta-analytic study relied largely on unpublished doctoral dissertations (89% of sources; see Table 2), a feature that we need to place in perspective for the reader.
Over the past 50 years, scholars have substantiated that the bulk of empiri-cal research on educational leadership is conducted by doctoral students (Bridges, 1982; Hallinger, 2011; Hallinger & Heck, 1996). If this is the case, then one would expect research syntheses, including meta-analytic reviews, in our field to rely heavily on doctoral research. The authors examined this assumption empirically by examining the composition of “all meta-analytic reviews of educational leadership published in refereed journals.” As it is only recently that meta-analysis has become more commonly used in educa-tional leadership, our Google scholar search identified only seven studies (see Table 2).
Data presented in Table 2 indicate that the high percentage of doctoral dis-sertations comprising the database in our study (i.e., 89%) was reasonably consistent with the percentage of dissertations featured in other meta-analytic reviews of educational leadership. The average percentage of dissertations per study was 65%, and doctoral dissertations comprised 90% of all sources included in these published studies. Moreover, it should be noted that these figures actually underestimate the actual percentage of doctoral dissertations.
by guest on March 16, 2016eaq.sagepub.comDownloaded from
10
Tab
le 1
. Su
mm
ary
of S
elec
ted
Stud
ies
and
Stud
y C
hara
cter
istic
s.
No.
Aut
hor
Yea
rD
ata
Typ
eD
ata
Sour
ceSc
hool
Le
vel
Sam
ple
Size
PIM
RS
Leve
ls o
f D
ata
Ana
lysi
sPr
inci
pals
Mal
e Pr
inci
pals
Fem
ale
Prin
cipa
lsT
each
er
1A
tkin
son
2013
Raw
TA
LL75
1065
1,31
1A
ll le
vels
2Ba
bcoc
k19
91Ex
tP
E21
312
786
—Fu
nctio
ns 3
Baue
r20
13R
awP
H77
2156
—A
ll le
vels
4Ba
uer
2013
Raw
TH
7721
5615
4A
ll le
vels
5C
arr
2011
Raw
PE
62
4—
All
leve
ls 6
Dun
n20
10Ex
tP
M&
H12
875
53—
Func
tions
7D
urye
a19
88Ex
tP
ALL
289
19—
Dim
ensi
ons
8G
allo
n19
98Ex
tT
ALL
154
1170
4Fu
nctio
ns 9
Gei
selm
an20
04Ex
tP
E18
076
104
—D
imen
sion
s10
Gei
selm
an20
04Ex
tT
E18
076
104
799
Dim
ensi
ons
11G
roff
2002
Ext
PE
110
5951
—Fu
nctio
ns12
Hal
linge
ra19
83Ex
tT
E6
33
104
Func
tions
13H
owel
lb19
89Ex
tP1
E20
214
359
—Fu
nctio
ns14
How
ell
1989
Ext
P2E
202
143
59—
Func
tions
15H
owel
l19
89Ex
tP3
E37
307
—Fu
nctio
ns16
How
ell
1989
Ext
P4E
3730
7—
Func
tions
17Je
nnin
gs20
13Ex
tP
E30
921
—Fu
nctio
ns18
Lehl
1989
Ext
PE
167
9—
Tot
al s
core
19M
arin
2013
Ext
PA
LL13
974
65—
Func
tions
20M
cCab
e19
93Ex
tP
M&
H5
32
—A
ll le
vels
21M
cDon
ald
2012
Raw
PE
142
12—
All
leve
ls (con
tinue
d)
by guest on March 16, 2016eaq.sagepub.comDownloaded from
11
No.
Aut
hor
Yea
rD
ata
Typ
eD
ata
Sour
ceSc
hool
Le
vel
Sam
ple
Size
PIM
RS
Leve
ls o
f D
ata
Ana
lysi
sPr
inci
pals
Mal
e Pr
inci
pals
Fem
ale
Prin
cipa
lsT
each
er
22M
cDon
ald
2012
Raw
TE
142
1210
4A
ll le
vels
23M
iller
c19
91Ex
tP1
M44
3410
—T
otal
Sco
re24
Mill
er19
91Ex
tP2
M27
243
—T
otal
Sco
re25
Mun
roe
2009
Raw
PE
3514
21—
All
leve
ls26
Nog
aya
1995
Ext
PM
&H
6133
28—
All
leve
ls27
Nog
aya
1995
Ext
TM
&H
6133
2842
4A
ll le
vels
28O
’Don
nela
2002
Ext
PM
7557
18—
Dim
ensi
ons
29O
’Don
nella
2002
Ext
TM
7557
1830
0D
imen
sion
s30
Pear
iso
2011
Ext
PE
3624
12—
Func
tions
31Po
ovat
anik
ul19
93Ex
tP
M&
H44
3113
—T
otal
Sco
re32
Prat
ley
1992
Ext
PM
&H
7664
12—
Func
tions
33Pr
atle
y19
92Ex
tT
M&
H62
5210
420
Func
tions
34R
ose
1991
Ext
PA
LL71
5516
—Fu
nctio
ns35
Ruz
icsk
a19
89Ex
tT
E&M
118
319
2A
ll le
vels
36Sc
hoch
1992
Ext
TE
7040
3053
7Fu
nctio
ns37
Tro
ut19
85Ex
tP
M&
H28
1414
—T
otal
& F
unct
ions
38T
rout
1985
Ext
TM
&H
2814
1427
5T
otal
& F
unct
ions
39Y
ang
1996
Ext
PE
106
5353
—Fu
nctio
ns40
Yan
g19
96Ex
tT
E10
653
5385
1Fu
nctio
ns
Not
e. P
IMR
S =
Pri
ncip
al In
stru
ctio
nal M
anag
emen
t R
atin
g Sc
ale;
T =
tea
cher
; P =
pri
ncip
al; E
= e
lem
enta
ry s
choo
l; M
= m
iddl
e sc
hool
; H =
hig
h sc
hool
; ALL
= e
lem
enta
ry, m
iddl
e, a
nd h
igh
scho
ol; R
aw =
raw
dat
a fr
om a
utho
r; E
xt =
dat
a ex
trac
ted
from
dis
sert
atio
n.a P
ublis
hed
stud
ies.
bIn
the
How
ell s
tudy
, we
obta
ined
four
sep
arat
e se
ts o
f effe
ct s
ize
estim
ates
. cIn
the
Mill
er s
tudy
, we
obta
ined
tw
o se
ts o
f effe
ct
size
est
imat
es fr
om t
wo
grou
ps o
f pri
ncip
als
(P1
&P2
).
Tab
le 1
. (c
ont
inue
d)
by guest on March 16, 2016eaq.sagepub.comDownloaded from
12 Educational Administration Quarterly
The column “published papers” in Table 2 includes not only refereed articles but also book chapters and conference papers. It would be difficult to argue that the latter two sources undergo more rigorous review than doctoral stud-ies. In sum, we conclude that the composition of the database of studies ana-lyzed in this study is consistent with other published meta-analyses in our field.
Data Collection
Data extraction. Each study was reviewed to identify key characteristics. Descriptive (e.g., sample sizes, school level, year) as well as statistical data were extracted from each study and entered into a MS Excel spreadsheet. In cases where we had obtained a raw data set, we generated relevant statistics for all three construct levels. The final database used for meta-analysis included 40 data sets drawn from 28 studies containing a variety of different statistics organized by construct level.
Table 2. Distribution of Dissertations and Published Studies in Meta-Analyses in Educational Leadership Published in Refereed Journals.
Studies Year Journal
Unpublished Dissertations
Pub & Confa
Total Sources
Raw % Raw Raw
1. Eagly et al. 1992 EAQ 112 90% 13 1252. Witziers et al. 2003 EAQ 2 5% 35 373. Chin 2007 APER 28 100% 0 284. Robinson et al. 2008 EAQ 0 0% 22 225. Leithwood and Sun 2012 EAQ 79 100% 0 796. Hallinger et al. 2013 EAQ 129 94% 6 1357. Sun and Leithwood 2015 SESI 79 72% 31 1108. Marzano, Waters,
and McNulty2005 Book 54 80% 15 69
Across 8 studiesb 483 80% 122 605Average per studyb 60 67% 15 76Current study 2016 25 89% 3 28
Note. EAQ = Educational Administration Quarterly; APER = Asia Pacific Education Review; SESI = School Effectiveness and School Improvement.aThis column contains both papers published in refereed journals as well as conference papers and book chapters. bThe summary “across the 8 studies” totals the sources for all studies reports the average within this meta-analytic literature (e.g., grand mean dissertations across 8 studies = 80%). The next line lists the average per study where the mean dissertations per study = 66%.
by guest on March 16, 2016eaq.sagepub.comDownloaded from
Hallinger et al. 13
Data quality. Our reliance on doctoral dissertations, although justified above, still raises issues with respect to “data quality.” Historically, doctoral studies in educational leadership have tended to rely on measurement instruments of dubious reliability and validity, and simple statistics (Bridges, 1982; Hal-linger, 2011). First, as noted above, in this meta-analysis all the studies employed the same validated instrument, the PIMRS, for the dependent vari-able. This not only reduced concerns over data quality, but meant that we could focus on the same set of conceptual constructs across studies. Second, the independent variable of interest, gender, was easily measured as a discrete variable (i.e., male/female). Together, these features enabled the authors to achieve a larger sample size for many of our analyses than has been the case in other meta-analyses (e.g., Robinson et al., 2008)—even those where the total sample of studies was larger (e.g., Eagly et al., 1992; Witziers et al., 2003). Finally, for the purposes of our meta-analysis, we only required rela-tively simple statistics to compute effect sizes. This contrasts, for example, with the corpus of studies comprising meta-analyses of leadership effects on student achievement (e.g., Robinson et al., 2008; Scheerens, 2012; Witziers et al., 2003). In sum, after examining these features of the composite studies, we did not view the reliance on doctoral dissertations as a major constraint with respect to quality for this meta-analysis.
Data Analysis
Our data analysis procedures examined differences in the direction, magni-tude, and significance of differences in the level of instructional leadership exercised by male and female principals across the studies. Then we analyzed the effect sizes (ES) to determine if there were identifiable patterns of differ-ence in how male and female principals enacted the various dimensions and functions of this role.
Transforming statistics into a common metric. We began by transforming statis-tics generated from different tests (e.g., t test, ANOVA, regression) into a common effect size metric. Researchers conducting meta-analyses in educa-tional leadership have employed a variety of effect size metrics including Cohen’s d, z1, r, and g (e.g., Chin, 2007; Eagly et al., 1992; Leithwood & Sun, 2012; Robinson et al., 2008; Scheerens, 2012; Sun & Leithwood, 2015; Wit-ziers et al., 2003). The leadership effects literature often reports “variance explained” as a measure of the predictive power of one variable on another (e.g., Hallinger & Heck, 1996; Robinson et al., 2008; Witziers et al., 2003).
In this study, however, we had a different objective. Rather than testing a set of variables for their predictive power, we sought to determine whether
by guest on March 16, 2016eaq.sagepub.comDownloaded from
14 Educational Administration Quarterly
and to what extent there was a “difference” in the leadership exercised by male and female principals. For this purpose, we selected Cohen’s d, one of the most widely used metrics in meta-analysis. In this study, Cohen’s d repre-sents the “standardized mean difference” between the scores of male and female principals on the PIMRS (Cohen, 1992; Hedges & Olkin, 1985).
Tables presenting mean and standard deviation, correlation r, and ANOVA eta-squared (η2) statistics had been collected from the 28 studies. We trans-formed the r and η2 statistics into Cohen’s d using formulas specified in the literature (e.g., Cohen, 1988; Rosenthal, 1994). When working with means and standard deviations as well as with the raw data sets, we computed Cohen’s d using the following formula (Cohen, 1992):
dX X
S=
−M F
p
, (1)
where X M is the mean for male principals, X F is the mean for female prin-cipals, and Sp is the pooled standard deviation. The standard error of the effect size estimate is
SE dn n
n n
d
n n( )
( ).=
++
+M F
M F M F
2
2 (2)
Using these formulae, we produced a Cohen’s d statistic for each available construct level (i.e., PIMRS Total Score, Dimension, Function) across the corpus of studies (see Tables 3 and 4). To identify the direction of the effect size, we examined the coding of the studies and redefined the d values as either positive or negative. Throughout our analyses a positive d indicates more active instructional leadership from male principals, and a negative d indicates more active instructional leadership from the female group.
Testing for homogeneity. The next step in data analysis involved testing homo-geneity of the data set. This involves examining whether the distribution of effect sizes across data sets may have been influenced by sources other than sampling error (Lipsey & Wilson, 2001). We employed the Q statistic in our test for homogeneity. Q was computed as
Q SE d dSE d d
SE di i i
i i i
ii
= × −×
−−
−∑ ∑∑
( ( ) )[ ( ( ) )]
( ),2 2
2 2
2 (3)
where i indexes data sets, Q is asymptotically chi-square distributed with degrees of freedom of the number of studies minus one, and the others as defined previously. If homogeneity of the study sample is rejected, then
by guest on March 16, 2016eaq.sagepub.comDownloaded from
Hallinger et al. 15
additional tests are required to explore other potential sources of error in the sample (Leithwood & Sun, 2012; Lipsey & Wilson, 2001). If the homogene-ity assumption is not rejected, then it is possible to “combine” effect sizes obtained from multiple data sets into a weighted mean effect size (Lipsey & Wilson, 2001).
Weighting the effect sizes. Since sample size differs across studies, researchers commonly “weight” the effect size by the study variance. This produces an additional statistic, weighted mean effect size. We computed this statistic, using the following formula:
dSE d d
SE d
i ii
ii
=−
−
∑∑
( )
( ),
2
2 (4)
with a standard error of
SE dSE dii
( )( )
.=−∑
12
(5)
We computed the weighted mean effect size and its standard error to assess the direction, magnitude, and significance of gender differences on 14 PIMRS constructs (i.e., Total Score, 3 Dimensions, 10 Functions). These analyses also informed our analysis of patterns of instructional leadership exercised by male and female principals on the PIMRS dimensions and functions.
Combining effect sizes. Lipsey and Wilson (2001) demonstrated that when a study reports multiple effect sizes representing the same construct (e.g., three PIMRS Dimensions), they can be averaged to create a “total effect size” sta-tistic for the study (p. 101). Rosenthal and Rubin (1986) further supported the use of this approach when the dependent measures are highly correlated. This was the case in the current study (Hallinger & Murphy, 1985; Hallinger & Wang, 2015).
In the 16 studies where we possessed a d statistic for the PIMRS Total Score, this was used to represent a “total effect size” for the study. For studies that did not report results based on the total score, we averaged ds reported at either the dimension or function level to produce a “total effect size” statistic for the study. Using Babcock’s (1991) study as an example, we extracted principal means and standard deviations reported at the function level. We computed 10 function-level ds and standard errors. We then computed a total mean effect size for the study (i.e., −0.552, SE = 0.142; reported in Table 4).
by guest on March 16, 2016eaq.sagepub.comDownloaded from
16 Educational Administration Quarterly
A similar approach was employed for the other 20 data sets. Finally, we aver-aged the 40 study-level effect sizes into an overall effect size (Lipsey & Wilson, 2001).
Examining differences between principal and teacher data sets. In meta-analy-sis, we can treat the data source as a “study characteristic” and test the between-group variance, even if the assumption of homogeneity has not been rejected. Therefore, as a follow-up test, we calculated the between-groups homogeneity statistic (QB), by conducting the analysis of analog to the ANOVA, under the fixed-effect model. This approach is widely used by researchers when testing the effect of categorical variables, in this case teach-ers or principals as the data source (see Eagly et al., 1992; Lipsey & Wilson, 2001).
We first calculated the Qj for each group (i.e., Q statistics for principals [QPrin] and for teachers [QTch]). We then obtained the within-groups homoge-neity (Qw) by summing up the Qj (j represents the number of groups). We then computed QB using the Q total (QT), which we obtained from the previous homogeneity analysis, minus Qw. If QB is significant, it means the difference of mean effect sizes across groups cannot be explained only by sampling error and there is a significant difference between the groups.
Assessing differences in patterns of instructional leadership practice. The final analytical step involved descriptive analysis of patterns in the instructional leadership practices of the male and female principals. This analysis was aimed at determining if observed differences formed a general pattern or if differences were concentrated in particular instructional leadership dimen-sions or functions. This involved re-examining data generated in the earlier analyses conducted at the dimension and function levels.
Interpreting the magnitude of effect sizes. Following these meta-analytic tests, we faced the task of interpreting the results. As noted above, in meta-analysis different statistics have been used by researchers for measuring effect size (d, r, r2, etc.). Moreover, for a given statistic, researchers have applied different “standards” (i.e., cutoff points) for interpreting the practical meaning of effect sizes (Glass, 1977; Hedges & Olkin, 1985; Lipsey & Wilson, 2001). In selecting a “standard” for this study, we referred to Cohen’s (1992) discus-sion of this issue.
My intent was that medium ES represent an effect likely to be visible to the naked eye of a careful observer. (It has since been noted in effect size surveys that it approximates the average size of observed effects in various fields.) I set
by guest on March 16, 2016eaq.sagepub.comDownloaded from
Hallinger et al. 17
small ES to be noticeably smaller than medium but not so small as to be trivial, and I set large ES to be the same distance above medium as small was below it. . . . For this test [i.e., Cohen’s d], the Ho is that d = 0 and the small, medium, and large ESs (or H1s) are d = .20, .50, and .80. (Cohen, 1992, pp. 156-157)
Cohen (1992) further noted that interpretation of the magnitude of effect sizes varies across different fields of inquiry. Relevant factors to consider include the nature of the phenomenon being studied (e.g., leadership) as well as the research methods used by researchers in the sample of studies. Eagly et al. (1992) observed that widely varying and largely uncontrolled condi-tions under which school leadership studies are typically conducted tend to reduce the magnitude of effect sizes. This constraint was relevant in the cur-rent study in which 100% of the data had been obtained through nonexperi-mental research conducted by doctoral students. Eagly and colleagues further concluded: “As a consequence, neither sex nor other variables would ordinar-ily produce large effect sizes in studies of principals’ leadership style” (Eagly et al., 1992, pp. 92-93). This assertion has been borne out in subsequent meta-analytic studies of school leadership conducted over the succeeding 20 years, which typically report “small effects” (e.g., Chin, 2007; Leithwood & Sun, 2012; Robinson et al., 2008; Scheerens, 2012; Sun & Leithwood, 2015; Witziers et al., 2003).
We kept these factors in mind when interpreting the magnitude of effect sizes in our study. Operationally, we followed Cohen’s (1992) recommenda-tion and coded d > .20 as a “small effect,” d > .50 as a “moderate effect,” and d > .80 as a “large effect.” These standards are applied in all subsequent tables and guide our discussion of the meta-analytic results. In addition to the d statistic, it is common to use Cohen’s U3 statistic to describe the mean dif-ference between two groups. This represents the percentage of one group that will be above the mean of the other group. For example, ds of .20, .50, and .80 indicate that 58%, 69%, and 79% of one group will be above the mean of the other group, respectively.
Results
We used the analytical procedures described above to determine the level and pattern of differences in instructional leadership exercised by male and female principals. We begin by describing the direction and magnitude of effect sizes reported from the studies at different construct levels. Next, we report the homogeneity analysis used to determine the approach required for subsequent meta-analytic procedures. Then we report the meta-analyses con-ducted for different construct levels as well as for the overall data set. Finally,
by guest on March 16, 2016eaq.sagepub.comDownloaded from
18 Educational Administration Quarterly
we examine the pattern of differences between male and female principals with respect to the instructional leadership dimensions and functions.
Descriptive Results
The 12 studies (17 data sets) that reported PIMRS Total Scores (i.e., full scale score) included 623 principals (273 males, 350 females). The direction of d suggested more active instructional leadership from female principals in 14 studies and males in 3 studies (see Table 2). The magnitude of d exceeded the cutoff representing a “practical effect” (i.e., .20) in 10 of the 14 studies where the direction suggested more active instructional leadership from female principals, and in only one study where the direction favored male principals (see Table 3).
Table 4 presents 48 ds drawn from 16 studies that reported dimension-level scores for 974 principals (424 males, 550 females). Forty of the dimen-sion-level ds suggested more active instructional leadership among female principals, 27 of which offered evidence of a practical effect (i.e., the d statis-tic exceeded −.20). In contrast, only two of the eight ds suggesting more active instructional leadership from male principals reached the level indicat-ing a practical effect (i.e., >. 20).
Analysis of 31 relevant data sets yielded 287 PIMRS function-level scores for 2,138 principals (1,215 males, 923 females; data not shown). Among the 231 function-level ds in the direction of female principals, 150 indicated a practical effect (e.g., 40 large effect, 36 moderate effect, and 74 small effect). In contrast, 16 indicated a difference in favor of male princi-pals at a level that could be interpreted as small. In sum, the direction of ds at all three construct levels suggested that female principals were per-ceived as exercising more active instructional leadership. The magnitude of ds, however, suggested that the effect was small to moderate in size. Subsequent analyses sought to further refine and test the significance of these descriptive findings.
Homogeneity Analysis
As discussed earlier, the appropriate sequence of meta-analytic procedures depends on the homogeneity of the study sample. A significant Q statistic indicates that the variability of the effect sizes cannot be explained only by sampling error. In this event, a single weighted mean effect size is not capable of estimating the population mean. Thus, our next step involved computing Q statistics for all 14 constructs.
by guest on March 16, 2016eaq.sagepub.comDownloaded from
19
Tab
le 3
. Ef
fect
Siz
e Es
timat
es fo
r 16
Stu
dies
Bas
ed o
n th
e PI
MR
S T
otal
Sco
re.
No.
Aut
hor
Yea
rD
ata
Typ
eD
ata
Sour
ceSc
hool
Le
vel
Sam
ple
Size
Tot
al P
IMR
S
Prin
cipa
lsM
ale
Fem
ale
Tea
cher
sd
se
Mag
nitu
de
and
Dir
ectio
n of
d
1A
tkin
son
2013
Raw
TA
LL75
1065
1311
−0.
391
0.21
4Fe
m+
2Ba
uer
2013
Raw
PH
7721
56−
0.07
50.
104
Fem
3Ba
uer
2013
Raw
TH
7721
5615
40.
062
0.08
6M
ale
4C
arr
2011
Raw
PE
62
4—
−1.
892
1.02
4Fe
m+
++
5Le
hl19
89Ex
tP
E16
79
−0.
995
0.26
8Fe
m+
++
6M
cCab
e19
93Ex
tP
M&
H5
32
−0.
013
0.24
0Fe
m 7
McD
onal
d20
12R
awP
E14
212
−0.
532
0.39
9Fe
m+
+ 8
McD
onal
d20
12R
awT
E14
212
104
−1.
414
0.30
3Fe
m+
++
9M
iller
1991
Ext
P1a
M44
3410
−0.
690
5.70
2Fe
m+
+10
Mill
er19
91Ex
tP2
M27
243
0.73
012
.134
Mal
e++
11M
unro
e20
09R
awP
E35
1421
−0.
042
0.14
1Fe
m12
Nog
ay19
95Ex
tP
M&
H61
3328
−0.
303
0.11
3Fe
m+
13N
ogay
1995
Ext
TM
&H
6133
2842
4−
0.35
00.
143
Fem
+14
Poov
atan
ikul
1993
Ext
PM
&H
4431
13−
0.85
00.
175
Fem
++
+15
Ruz
icsk
a19
89Ex
tT
E&M
118
315
5−
2.65
20.
189
Fem
++
+16
Tro
ut19
85Ex
tP
M&
H28
1414
0.11
80.
214
Mal
e17
Tro
ut19
85Ex
tT
M&
H28
1414
560
−0.
055
0.29
5Fe
m
Tot
al62
327
335
02,
708
Not
e. P
IMR
S =
Pri
ncip
al In
stru
ctio
nal M
anag
emen
t R
atin
g Sc
ale;
T =
tea
cher
; P =
pri
ncip
al; E
= e
lem
enta
ry s
choo
l; M
= m
iddl
e sc
hool
; H =
hig
h sc
hool
; ALL
= e
lem
enta
ry, m
iddl
e, a
nd h
igh
scho
ol; R
aw =
raw
dat
a fr
om a
utho
r; E
xt =
dat
a ex
trac
ted
from
dis
sert
atio
n. P
ositi
ve d
s es
timat
es
indi
cate
mor
e ac
tive
inst
ruct
iona
l lea
ders
hip
from
mal
e pr
inci
pals
and
neg
ativ
e ds
from
fem
ale
prin
cipa
ls. I
n th
e la
st c
olum
n, F
em in
dica
tes
that
the
di
rect
ion
of d
favo
red
fem
ales
, Mal
e in
dica
tes
that
it fa
vore
d m
ales
. Whe
re t
his
is fo
llow
ed b
y a
“+”
it in
dica
tes
a sm
all e
ffect
, “+
+”
a m
oder
ate
effe
ct, a
nd “
++
+”
a la
rge
effe
ct b
ased
on
d. If
no
“+”
is p
rese
nt, i
t in
dica
tes
that
the
mag
nitu
de o
f d s
ugge
sted
no
prac
tical
effe
ct.
a Thi
s st
udy
gath
ered
dat
a fr
om t
wo
diffe
rent
sam
ples
of p
rinc
ipal
s.
by guest on March 16, 2016eaq.sagepub.comDownloaded from
20
Tab
le 4
. Ef
fect
Siz
e Es
timat
es a
t th
e PI
MR
S D
imen
sion
Lev
el fo
r 16
Stu
dies
.
No.
Aut
hor
Yea
r
Sam
ple
Size
of
Prin
cipa
lsD
efin
es t
he S
choo
l’s
Mis
sion
Man
ages
the
In
stru
ctio
nal P
rogr
amPr
omot
es a
Pos
itive
Sc
hool
Clim
ate
Tot
alM
ale
Fem
dSE
Dir
/Mag
dSE
Dir
/Mag
dSE
Dir
/Mag
1A
tkin
son
2013
7510
65−
0.21
30.
294
Fem
+−
0.51
30.
213
Fem
++
−0.
384
0.22
8Fe
m+
2Ba
uer
2013
7721
56−
0.07
70.
133
Fem
−0.
135
0.11
7Fe
m−
0.05
30.
256
Fem
3Ba
uer
2013
7721
56−
0.04
40.
127
Fem
0.11
00.
108
Mal
e0.
072
0.08
0M
ale
4C
arr
2011
62
4−
2.11
11.
059
Fem
++
+−
0.82
80.
898
Fem
++
+−
2.45
61.
119
Fem
++
+ 5
Dur
yea
1988
289
19−
0.93
81.
236
Fem
++
+−
0.56
12.
308
Fem
++
−0.
080
4.38
1Fe
m 6
Gei
selm
an20
0418
076
104
−0.
583
0.07
5Fe
m+
+−
0.49
40.
075
Fem
+−
0.28
30.
075
Fem
+ 7
Gei
selm
an20
0418
076
104
−0.
161
0.03
5Fe
m−
0.14
00.
035
Fem
−0.
100
0.03
5Fe
m 8
McC
abe
1993
53
2−
0.18
00.
425
Fem
0.14
00.
256
Mal
e0.
270
0.25
1M
ale+
9M
cDon
ald
2012
142
12−
0.29
50.
453
Fem
+−
0.45
40.
526
Fem
+−
0.57
40.
402
Fem
++
10M
cDon
ald
2012
142
12−
1.29
10.
325
Fem
++
+−
1.50
80.
328
Fem
++
+−
1.29
40.
316
Fem
++
+11
Mun
roe
2009
3514
210.
158
0.16
6M
ale
−0.
122
0.17
3Fe
m−
0.06
20.
153
Fem
12N
ogay
1995
6133
28−
0.02
60.
178
Fem
−0.
158
0.13
3Fe
m−
0.40
70.
108
Fem
+13
Nog
ay19
9561
3328
−0.
543
0.16
1Fe
m+
+−
0.21
50.
150
Fem
+−
0.28
20.
145
Fem
+14
O’D
onne
ll20
0275
5718
0.22
70.
167
Mal
e+0.
110
0.17
1M
ale
0.14
80.
128
Mal
e15
O’D
onne
ll20
0275
5718
−0.
320
0.16
9Fe
m+
−0.
361
0.15
7Fe
m+
−0.
430
0.14
5Fe
m+
16R
uzic
ska
1989
118
3−
4.20
40.
153
Fem
++
+−
2.89
60.
168
Fem
++
+−
1.70
00.
265
Fem
++
+
N16
974
424
550
Not
e. P
ositi
ve d
s es
timat
es in
dica
te m
ore
activ
e in
stru
ctio
nal l
eade
rshi
p fr
om m
ale
prin
cipa
ls a
nd n
egat
ive
ds fr
om fe
mal
e pr
inci
pals
. In
the
last
co
lum
n, F
em in
dica
tes
that
the
dir
ectio
n of
d fa
vore
d fe
mal
es, M
ale
indi
cate
s th
at it
favo
red
mal
es. W
here
thi
s is
follo
wed
by
a “+
” it
indi
cate
s a
smal
l effe
ct, “
++
” a
mod
erat
e ef
fect
, and
“+
++
” a
larg
e ef
fect
bas
ed o
n d.
If n
o “+
” is
pre
sent
, it
indi
cate
s th
at t
he m
agni
tude
of d
sug
gest
ed n
o pr
actic
al e
ffect
.
by guest on March 16, 2016eaq.sagepub.comDownloaded from
Hallinger et al. 21
Q for the PIMRS Total Score is 24.854, which is not significant (p < .05, critical value of chi-square at α = .05, df = 16, is 26.296). Moreover, as shown in Table 5, 12 of the 14 Q statistics are not significant. These results indicate that the assumption of homogeneity is not rejected, and the data sets can be analyzed without additional testing of moderators. Therefore, we proceed next to the meta-analytic synthesis where we computed the weighted mean effect size for all 14 constructs to determine the relationship between princi-pal gender and instructional leadership.
Meta-Analysis of Effect Sizes by Construct Level
In Table 6 we report the weighted mean effect sizes for the PIMRS Total Score, 3 Dimensions, and 10 Functions. These were all negative (ranging from −0.159 to −0.358, M = −0.257), “small” in magnitude, and statistically significant (p < .01). The confidence intervals were not overly large and the upper level of all 14 confidence intervals still fell in the negative effects domain. These results further affirm the finding of small but statistically sig-nificant gender differences favoring more active instructional leadership from the female principals.
Table 5. Q Statistics for 14 PIMRS Constructs.
Instructional Leadership Constructs No. of ds Q df
PIMRS Total Score 17 24.854 16Dimensions Defines the School’s Mission 16 32.350* 15 Manages the Instructional Program 16 20.011 15 Develops the School Learning Climate 16 16.061 15Functions Frames the School’s Goals 29 30.870 28 Communicates the School’s Goals 28 31.468 27 Supervises and Evaluates Instruction 28 24.388 27 Coordinates the Curriculum 28 31.322 27 Monitors Student Progress 29 256.276* 28 Protects Instructional Time 30 34.156 29 Maintains High Visibility 28 15.020 27 Provides Incentives for Teachers 29 34.743 28 Promotes Professional Development 29 29.425 28 Provides Incentives for Learning 29 26.762 28
Note. PIMRS = Principal Instructional Management Rating Scale.*p < .05.
by guest on March 16, 2016eaq.sagepub.comDownloaded from
22
Tab
le 6
. Su
mm
ary
of W
eigh
ted
Mea
n Ef
fect
Siz
e Es
timat
es a
nd R
elat
ed S
tatis
tics
by C
onst
ruct
Lev
el.
Inst
ruct
iona
l Lea
ders
hip
Con
stru
cts
Dat
a Se
tsd
SED
ir &
Mag
Upp
er C
ILo
wer
CI
Z S
core
PIM
RS
Tot
al S
core
17−
0.29
50.
091
Fem
+−
0.11
6−
0.47
43.
236*
*D
imen
sion
s
D
efin
es t
he S
choo
l’s M
issi
on16
−0.
281
0.07
0Fe
m+
−0.
143
−0.
419
3.99
0**
M
anag
es t
he In
stru
ctio
nal P
rogr
am16
−0.
258
0.07
0Fe
m+
−0.
121
−0.
396
3.68
9**
Pr
omot
es a
Pos
itive
Sch
ool C
limat
e16
−0.
206
0.07
0Fe
m+
−0.
069
−0.
343
2.95
2**
Func
tions
Fram
es t
he S
choo
l’s G
oals
29−
0.24
20.
050
Fem
+−
0.14
3−
0.34
14.
809*
*
Com
mun
icat
es t
he S
choo
l’s G
oals
28−
0.27
80.
050
Fem
+−
0.17
9−
0.37
75.
503*
*
Supe
rvis
es a
nd E
valu
ates
Inst
ruct
ion
28−
0.24
40.
050
Fem
+−
0.14
5−
0.34
34.
844*
*
Coo
rdin
ates
the
Cur
ricu
lum
28−
0.25
90.
050
Fem
+−
0.16
1−
0.35
85.
146*
*
Mon
itors
Stu
dent
Pro
gres
s29
−0.
358
0.05
2Fe
m+
−0.
255
−0.
460
6.83
3**
Pr
otec
ts In
stru
ctio
nal T
ime
30−
0.21
40.
048
Fem
+−
0.11
9−
0.30
84.
409*
*
Mai
ntai
ns H
igh
Vis
ibili
ty28
−0.
159
0.05
0Fe
m−
0.06
1−
0.25
83.
175*
*
Prov
ides
Ince
ntiv
es fo
r T
each
ers
29−
0.28
00.
049
Fem
+−
0.18
4−
0.37
65.
738*
*
Prom
otes
Pro
fess
iona
l Dev
elop
men
t29
−0.
332
0.04
9Fe
m+
−0.
237
−0.
428
6.80
1**
Pr
ovid
es In
cent
ives
for
Lear
ning
29−
0.19
80.
050
Fem
−0.
099
−0.
296
3.93
5**
Not
e. P
ositi
ve e
ffect
siz
e es
timat
es in
dica
te m
ore
activ
e in
stru
ctio
nal l
eade
rshi
p fr
om m
ale
prin
cipa
ls a
nd n
egat
ive
effe
ct s
izes
from
fem
ales
.**
Whe
n Z
sco
re e
xcee
ds 1
.96
it is
sig
nific
ant
at p
< .0
1 (t
wo
taile
d).
by guest on March 16, 2016eaq.sagepub.comDownloaded from
Hallinger et al. 23
Overall Effect Size Analyses
Next, we averaged the weighted mean effect size into a single mean value for each study (see Table 7). In 33 of the 37 data sets (89%), the direction of the study’s mean effect size indicated more active instructional leadership by female principals. Finally, we computed a summary mean effect size for the full set of studies of −0.303 (SE = 0.043; CI = [−0.206, −.0371]). The Z value of 7.046 indicates that the overall effect size statistic is significant at the p < .01 level (see Table 7). A d of −0.303 suggests that 62% of females were above the mean of males. This analysis again supports the conclusion that there is a small but statistically significant difference in principal instruc-tional leadership, with females demonstrating more active engagement in this role.
Comparing Teacher and Principal Perceptions of Principal Instructional Leadership
Next, for reasons discussed earlier, we reanalyzed the teacher and principal data sets separately. We obtained a weighted mean effect size of −0.327 (upper CI −0.428, lower CI −0.227) for the principal data sets, and −0.208 (upper CI −0.353, lower CI −0.064) for the teacher respondent data sets. These d statistics, −0.327 and −0.208, suggest that 63% and 58% of females were above the mean of males, respectively. Although the results suggest somewhat larger differences perceived within the principal self-report data, both ds indicated small but statistically significant differences in favor of female principals.
To determine whether these differences were statistically significant, we calculated the between-groups homogeneity statistic (QB) using procedures specified earlier. QB, was 1.755, with 1 degree of freedom (i.e., the number of categories minus 1).2 Q was not statistically significant at p < .05, thereby indicating no significant between-groups effect based on the data source (the critical value of chi-square at α = .05 and df = 1 is 3.84). This affirmed the acceptability of our meta-analytic procedures that combined the teacher and principal data sets into a single database for analysis.
Nature of Gender Differences in Principal Instructional Leadership
Our next task was to examine the pattern of differences observed between the instructional leadership of the male and female principals. Simply stated, we wished to determine if differences in the instructional leadership of male and
by guest on March 16, 2016eaq.sagepub.comDownloaded from
24 Educational Administration Quarterly
Table 7. Effect Size Estimates on PIMRS Total Score by Study.
No. Author YearData Type
Data Source
School Level
N Principals
Total Effect Size
d SE Dir/Mag
1 Atkinson 2013 Raw T ALL 31 −0.391 0.341 Fem+ 2 Babcock 1991 Ext P E 215 −0.552 0.142 Fem++ 3 Bauer 2013 Raw P H 77 −0.075 0.256 Fem 4 Bauer 2013 Raw T H 77 0.062 0.163 Male 5 Carr 2011 Raw P E 6 −1.892 1.024 Fem+++ 6 Dunn 2010 Ext P M&H 128 −0.215 0.180 Fem+ 7 Duryea 1988 Ext P ALL 28 −0.527 0.411 Fem++ 8 Gallon 1998 Ext T ALL 15 −0.061 0.584 Fem 9 Geiselman 2004 Ext P E 180 −0.454 0.153 Fem+10 Geiselman 2004 Ext T E 180 −0.134 0.151 Fem11 Groff 2002 Ext P E 110 −0.515 0.194 Fem++12 Hallinger 1983 Ext T E 10 −0.533 0.831 Fem++13 Howell 1989 Ext P E 202 −0.296 0.155 Fem+14 Jennings 2013 Ext P E 30 −0.024 0.398 Fem15 Lehl 1989 Ext P E 16 −0.995 0.534 Fem+++16 Marin 2013 Ext P ALL 139 −0.119 0.170 Fem17 McCabe 1993 Ext P M&H 6 −0.013 0.913 Fem18 McDonald 2012 Raw P E 15 −0.532 0.770 Fem++19 McDonald 2012 Raw T E 15 −1.414 0.809 Fem+++20 Miller 1991 Ext P1 M 44 −0.690 0.367 Fem++21 Miller 1991 Ext P2 M 27 0.730 0.620 Male++22 Munroe 2009 Raw P E 35 −0.042 0.345 Fem23 Nogay 1995 Ext P M&H 61 −0.303 0.258 Fem+24 Nogay 1995 Ext T M&H 61 −0.350 0.259 Fem+25 O’Donnell 2002 Ext P M 75 0.162 0.271 Male26 O’Donnell 2002 Ext T M 75 −0.370 0.272 Fem+27 Peariso 2011 Ext P E 36 −0.109 0.354 Fem28 Poovatanikul 1993 Ext P M&H 44 −0.850 0.343 Fem+++29 Pratley 1992 Ext P M&H 76 −0.470 0.317 Fem+30 Pratley 1992 Ext T M&H 62 −0.532 0.349 Fem++31 Rose 1991 Ext P ALL 71 −0.239 0.285 Fem+32 Ruzicska 1989 Ext T E&M 11 −2.652 0.882 Fem+++33 Schoch 1992 Ext T E 70 −0.386 0.244 Fem+34 Trout 1985 Ext P M&H 28 0.118 0.378 Male35 Trout 1985 Ext T M&H 28 −0.055 0.378 Fem++36 Yang 1996 Ext P E 106 −0.358 0.196 Fem+37 Yang 1996 Ext T E 106 −0.133 0.194 Fem
(continued)
by guest on March 16, 2016eaq.sagepub.comDownloaded from
Hallinger et al. 25
female principals were broadly distributed or concentrated in particular PIMRS dimensions and functions. To address this question, we analyzed the direction and magnitude of effect sizes across the PIMRS dimensions and functions.
Figure 2 shows data previously arrayed in Table 5 in graphic form. The visual data indicate a fairly similar pattern of results across the 3 dimensions and the 10 functions. More specifically, the direction and magnitude of effects suggest small but significant differences on each of the three PIMRS dimensions (i.e., d(D1) = −0.281, d(D2) = −0.258, d(D3) = −0.206). Although there was more varia-tion in the magnitude of effect sizes on function-level constructs (i.e., −0.159 to −0.358), all 10 ds suggested more active instructional leadership from the female principals (p < .01). Thus, gender-related differences appeared to be a “general effect” rather than concentrated in a few specific functions such as Supervising and Evaluating Instruction or Communicating the School’s Goals.
Discussion
Differences in the leadership styles employed by male and female principals were identified in the literature on educational leadership as far back as the 1950s and 1960s (e.g., Gross & Trask, 1964; Hemphill et al., 1962; Ramseyer, 1955). Over time, an accumulating body of research has yielded anecdotal references to female principals engaging the instructional leadership role more actively than male principals (Hallinger, 1983, 2011). Given the increas-ing importance assigned to instructional leadership in educational policy and practice, it appeared timely to review the empirical literature on this issue.
We employed meta-analysis to quantitatively integrate findings from 40 independent data sets drawn from 28 studies that had used the PIMRS to assess the instructional leadership of more than 2,500 principals from three countries.
No. Author YearData Type
Data Source
School Level
N Principals
Total Effect Size
d SE Dir/Mag
Weighted mean ES 2,531 −0.303 0.043 Z for mean ES 7.046**Q (df = 36) 38.824
Note. Positive ds estimates indicate more active instructional leadership from male principals and negative ds from female principals. In the last column, Fem indicates that the direction of d favored females, Male indicates that it favored males. Where this is followed by a “+” it indicates a small effect, “++” a moderate effect, and “+++” a large effect based on d. If no “+” is present, it indicates that the magnitude of d suggested no practical effect.**The ES is considered significant at p < .01 (two-tailed) when the Z score exceeds 2.58.
Table 7. (continued)
by guest on March 16, 2016eaq.sagepub.comDownloaded from
26 Educational Administration Quarterly
In sum, the results indicated a “small but statistically significant effect” of gen-der on instructional leadership, with more active instructional leadership from female principals. This conclusion was reflected in the direction, magnitude, and significance of the total weighted mean effect size (Cohen’s d = −0.303; U3 = 62%; p < .01). Analysis of variation in effect sizes across the 3 PIMRS dimen-sions and 10 functions further indicated that gender differences were “general” rather than concentrated in specific areas of leadership practice.
To place these results in perspective, we refer to Eagly’s meta-analytic stud-ies of gender differences in leadership (e.g., Eagly & Carli, 2003; Eagly et al., 1992; Eagly et al., 2003). Although Eagly’s research has focused on gender differences using leadership constructs other than instructional leadership, she employed “the common metric of effect sizes, which in reviews of gender and leadership generally take the form of a standardized difference (or d), defined as the difference between the mean scores of women and men” (Eagly & Carli, 2003, p. 811). With this in mind, it should be noted that the effect sizes (i.e., Cohen’s d) reported in the current study were generally as large or larger than those reported in Eagly’s rigorous meta-analytic studies of gender and leader-ship (e.g., Eagly & Carli, 2003; Eagly et al., 1992; Eagly et al., 2003).
A third issue concerns our interpretation of the summary effect size (i.e., d = −0.303, U3 = 62%) as “potentially meaningful.” In fact, the meaning of the gender-related differences in instructional leadership identified in this study can only be determined when the gender and leadership relationship is stud-ied in conjunction with relevant dependent variables of interest to scholars and policy makers (e.g., teacher efficacy, teacher learning, teaching and learning quality, teacher commitment). This suggests the need for future mul-tivariate studies that examine both moderators of gender (e.g., years of
Figure 2. Weighted mean effect sizes and confidence intervals.
by guest on March 16, 2016eaq.sagepub.comDownloaded from
Hallinger et al. 27
teaching experience of the principal) as well as mediators of leadership effects on student learning. With this in mind, we cautiously characterize the “small effect” identified in this study as “potentially meaningful.” Thus, the importance of this study lies in documenting and advancing the resolution of a question that has retained its relevance in the educational leadership litera-ture for more than 50 years.
Limitations of the Study
As is the case in all meta-analyses (Lipsey & Wilson, 2001), key limitations derive from the sample of studies. For example, scholars have observed that master theses and doctoral dissertations meet a less rigorous and consistent standard than peer-reviewed journal articles (e.g., Bridges, 1982; Hallinger, 2011; Robinson et al., 2008). Nonetheless, as described earlier, almost all research syntheses and meta-analytic reviews published in our field’s top journals have relied heavily on doctoral dissertations. Moreover, quality con-cerns within the dissertations included in this study were ameliorated, to a degree, by two features of the studies. They all employed a common vali-dated instrument, the PIMRS, for data collection, and the statistics needed from the studies for the purpose of this meta-analysis were relatively basic (e.g., correlations). Therefore, we suggest that our database of studies was neither atypical nor a reason to disqualify the results.
A second limitation lies in the concentration of data sets (37 of 40) that came from studies conducted in the United States. This limited geographic range is relevant given the diverse roles and status accorded to women across different societies (Shakeshaft, 2006). It is, however, interesting to note that results from studies conducted in Thailand (Poovatanikul, 1993) and Taiwan (Yang, 1996) were quite consistent with results from the United States. Nonetheless, gener-alization of our findings beyond the United States awaits confirmation based on a larger sample of principals working a broader set of societies.
A third limitation concerns the exclusive focus on studies that used the PIMRS. Although deployment of a common instrument in all 40 data sets clearly enabled a more powerful application of meta-analysis, it introduced an “implicit bias” toward research conducted from an “American perspective on instructional leadership” (see Robinson et al., 2008; Scheerens, 2012; Witziers et al., 2003). This further highlights the need to be cautious in gen-eralizing the results beyond the United States.
A fourth limitation also follows from the use of the PIMRS as the depen-dent measure. This instrument uses a response scale designed to assess the “relative frequency” of observed leadership behavior (Hallinger & Wang, 2015; Latham & Wexley, 1981). The instrument was not designed to yield scores that, by themselves, measure the quality, utility, suitability, or
by guest on March 16, 2016eaq.sagepub.comDownloaded from
28 Educational Administration Quarterly
effectiveness of the principal’s instructional leadership. Thus, our character-izations of gender differences throughout the article have emphasized the level of principal activity or engagement in the instructional leadership role. Although, we assert that this approach is both justifiable and useful, it only offers a one-dimensional perspective on how principals enact their instruc-tional leadership role (Hallinger & Wang, 2015).3
The final limitation concerns the delimited scope of this study. This research neither addressed why female principals may engage the instruc-tional leadership role more actively than males, nor whether these perceived differences carry over into relevant teaching and learning processes and out-comes. Testing explanations for causes and impact of gender differences are critical for the purposes of policy and practice. Nonetheless, the first step in this research process was to establish the nature of gender differences in the enactment of the instructional leadership role. Having taking a step toward accomplishing that objective, future research should undertake multivariate studies that advance our understanding of the causes and effects of these per-ceived gender differences in principal instructional leadership. Thus, we emphasize the need for multivariate studies that examine potential modera-tors of gender (e.g., years of prior teaching experience of the principal, school size) as well as measures of the impact of principal instructional leadership (e.g., teacher behavior, school improvement).
Implications for Practice
When shifting the focus toward practice, the findings cohere into a leadership orientation that seems remarkably well-suited to 21st-century schools. Eagly and Carli (2003) described a leadership orientation among females as task-focused and democratic, while also exhibiting key features of transforma-tional leadership. Our results extend this assertion to include a stronger disposition to engage the principal’s role as an instructional leader.
Nonetheless, the small size of the gender differences found in this study warrants caution when moving from research findings to practice. Although our results do not support a call for wholesale changes to principal selec-tion, they do call into question the passive approach to principal recruit-ment and selection that continues to predominate in many education systems around the world. For example, in the United States, it is only recently that the overall gender distribution of principals has shifted slightly in favor of females (52% to 48%) at the primary school level. Even so, 70% of America’s high school principals continue to be male (Bitterman, Goldring, & Gray, 2013). Elsewhere in the world, in both developing and industrialized societies, the percentage of female principals consistently lags well behind males (Shakeshaft, 2006): Australia (20%/80%), France
by guest on March 16, 2016eaq.sagepub.comDownloaded from
Hallinger et al. 29
(30%/70%), Italy (30%/70%), Philippines (15%/75%), the United Kingdom (25%/75%). Our findings offer an “instrumental rationale” to complement equity-based arguments for increasing the density of female principals in the population of school leaders in contexts where they continue to lag male counterparts (see Coleman, 2007; Collard, 2001; Krüger, 1996, 2008; Nogay & Beebe, 2008; Pounder & Coleman, 2002; Shakeshaft, 2006).
This study was conducted within the lineage of a half-century effort to understand linkages between school leadership, teaching, and learning (Bossert et al., 1982; Bridges, 1967; Erickson, 1979; Hallinger & Heck, 1996; Scheerens, 2012). Our decision to focus on gender differences was stimulated by a series of reports that female principals appeared to engage in more active instructional leadership than male counterparts. We hope that this meta-analytic study has both brought greater clarity to the boundaries of cur-rent empirically grounded knowledge and identified potentially fruitful direc-tions for future research and practice.
Declaration of Conflicting Interests
The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The authors disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: The authors wish to acknowledge the funding support of the Research Grant Council (RGC) of Hong Kong for its support through the General Research Fund (GRF 841711).
Notes
1. The discrepancy between the sample size of Principal Instructional Management Rating Scale studies examined in this report and the number identified by Hallinger (2011) lies in the fact that not all the studies that he had identified contained data suitable for our analyses.
2. QPrin = 24.185; QTch = 15.770; Qw = QPrin + QTch = 39.956; QB = QT − QW = 41.711 − 39.956 = 1.755.
3. Robinson et al., 2008 make a useful clarification of this distinction in their dis-cussion of leadership capabilities.
References
Adkinson, J. A. (1981). Women in school administration: A review of the research. Review of Educational Research, 51, 311-343.
Arcy, J. (1980). Self-perceptions of leader behavior of male and female elemen-tary school principals in selected school districts in the Midwest United States (Unpublished doctoral dissertation). Iowa State University, Ames, IO.
by guest on March 16, 2016eaq.sagepub.comDownloaded from
30 Educational Administration Quarterly
Atkinson, R. (2013). An assessment of the perceived instructional leadership behaviors of assistant principals (Unpublished doctoral dissertation). Virginia Commonwealth University, Richmond, VA.
Babcock, C. (1991). Perceived instructional leadership behavior: Gender differ-ences, female principals, male principals (Unpublished doctoral dissertation). Western Michigan University, Kalamazoo, MI.
Bauer, A. (2013). A comparative study of the leadership behaviors of the assistant prin-cipal (Unpublished doctoral dissertation). St. John’s University, New York, NY.
Bitterman, A., Goldring, R., & Gray, L. (2013). Characteristics of public and private elementary and secondary school principals in the United States: Results from the 2011-12 Schools and Staffing Survey. Washington DC: National Center for Education Statistics, U.S. Department of Education.
Bossert, S., Dwyer, D., Rowan, B., & Lee, G. (1982). The instructional management role of the principal. Educational Administration Quarterly, 18(3), 34-64.
Bridges, E. (1967). Instructional leadership: A concept re-examined. Journal of Educational Administration, 5, 136-147.
Bridges, E. (1982). Research on the school administrator: The state-of-the-art, 1967-1980. Educational Administration Quarterly, 18(3), 12-33.
Carr, T. (2011). An examination of leadership styles in implementing instructional technology: A case study to examine the elementary school principal perspective (Unpublished doctoral dissertation). North Central University, Minneapolis, MN.
Charters, W. W., Jr., & Jovick, T. D. (1981). The gender of principals and principal/teacher relations in elementary schools. In P. A. Schmuck, W. W. Charters, & R. O. Carlson (Eds.), Educational policy and management: Sex differentials (pp. 307-331). Cambridge, MA: Academic Press.
Chin, J. M. C. (2007). Meta-analysis of transformational school leadership effects on school outcomes in Taiwan and the USA. Asia Pacific Education Review, 8, 166-177.
Cohen, J. (1988). Statistical power analysis for the behavioral sciences. Hillsdale, NJ: Erlbaum.
Cohen, J. (1992). A power primer. Psychological Bulletin, 112, 155-159.Coleman, M. (2007). Gender and educational leadership in England: A comparison
of secondary headteachers’ views over time. School Leadership & Management, 27, 383-399.
Collard, J. L. (2001). Leadership and gender: An Australian perspective. Educational Management Administration & Leadership, 29, 343-355.
Cronbach, L. J. (1951). Coefficient alpha and the internal structure of tests. Psychometrika, 16, 297-334.
Cronbach, L. J. (1988). Five perspectives on validity argument. In H. Wainer & H. Braun (Eds.), Test validity (pp. 3-17). Hillsdale, NJ: Erlbaum.
Cunningham, J. (2004). The effect of gender-role identity of female principals in Texas on teacher perceptions of instructional leadership behaviors (Unpublished doctoral dissertation). Texas A&M University, Corpus Christi, TX.
Dunn, A. (2010). A quantitative study of the perceptions of school leadership prep-aration in the State of Georgia (Unpublished doctoral dissertation). Walden University, Minneapolis, MN.
by guest on March 16, 2016eaq.sagepub.comDownloaded from
Hallinger et al. 31
Duryea, M. (1988). The relationship between perceived principal instructional leadership behavior and locus of control (Unpublished doctoral dissertation). University of Houston, Houston, TX.
Eagly, A. H., & Carli, L. L. (2003). The female leadership advantage: An evaluation of the evidence. Leadership Quarterly, 14, 807-834.
Eagly, A. H., Johannesen-Schmidt, M. C., & van Engen, M. (2003). Transformational, transactional, and laissez faire leadership styles: A meta-analysis comparing women and men. Psychological Bulletin, 95, 569-591.
Eagly, A. H., Karau, S., & Johnson, B. (1992). Gender and leadership style among school principals: A meta-analysis. Educational Administration Quarterly, 28, 76-102.
Edmonds, R. (1979). Effective schools for the urban poor. Educational Leadership, 37, 15-24.
Erickson, D. A. (1979). Research on educational administration: The state-of-the-art. Educational Researcher, 8(3), 9-14.
Fischel, A., & Pottker, J. (1977). Performance of women principals: A review of behavioral and attitudinal studies. In J. Pottker & A. Fischel (Eds.), Sex bias in the schools (pp. 289-299). London, England: Associated University Press.
Frasher, J., & Frasher, R. (1979). Educational administration: A feminine profession. Educational Administration Quarterly, 2, 1-13.
Gallon, S. (1998). A study of teachers’ perceptions of the principal as an instructional leader: A comparative analysis of elementary, middle and senior high school principals (Unpublished doctoral dissertation). Florida International University, Miami.
Geiselman, S. (2004). Predicting elementary school student achievement: The impact of principal gender and principal leadership skills (Unpublished doctoral disser-tation). University of Louisville, Louisville, KY.
Glass, G. V. (1977). Integrating findings: The meta-analysis of research. Review of Research in Education, 5, 351-379.
Goldring, E., Huff, J., May, H., & Camburn, E. (2008). School context and individ-ual characteristics: What influences principal practice? Journal of Educational Administration, 46, 332-352.
Groff, M. (2002). Elementary instructional leadership: Factors that affect perfor-mance (Unpublished doctoral dissertation). Saint Louis University, St. Louis, MO.
Gross, N., & Trask, A. (1964). Men and women as elementary school principals. Eugene, OR: ERIC.
Gross, N., & Trask, A. (1976). The sex factor and the management of schools. New York, NY: Wiley.
Hallinger, P. (1983). Assessing the instructional management behavior of principals (Unpublished doctoral dissertation). Stanford University, Stanford, CA.
Hallinger, P. (1990). Principal instructional management rating scale. Sarasota, FL: Leading Development Associates. (Original work published 1982)
Hallinger, P. (2011). A review of three decades of doctoral studies using the Principal Instructional Management Rating Scale: A lens on methodological progress in educational leadership. Educational Administration Quarterly, 47, 271-306.
by guest on March 16, 2016eaq.sagepub.comDownloaded from
32 Educational Administration Quarterly
Hallinger, P. (2014). Reviewing reviews of research in educational leadership: An empirical analysis. Educational Administration Quarterly, 50, 539-576.
Hallinger, P., & Heck, R. H. (1996). Reassessing the principal’s role in school effec-tiveness: A review of empirical research, 1980-1995. Educational Administration Quarterly, 32, 5-44.
Hallinger, P. & Heck, R. H. (2002). What do you call people with visions? The role of vision, mission and goals in school leadership and improvement. In K. Leithwood, P. Hallinger, J. Chapman, D. Corson, & A. Hart (Eds.), The second international handbook of educational leadership and administration (pp. 9-40). Dordrecht: Kluwer.
Hallinger, P., & Lee, M. S. (2013). Exploring principal capacity to lead reform of teaching and learning quality in Thailand. International Journal of Educational Development, 33, 305-315.
Hallinger, P., & Murphy, J. F. (1985). Assessing the instructional leadership behavior of principals. Elementary School Journal, 86, 217-248.
Hallinger, P., & Murphy, J. F. (2012). Running on empty? Finding the time and capacity to lead learning. NASSP Bulletin, 97, 5-21.
Hallinger, P., Murphy, J., Weil, M., Mesa, R. H. P., & Mitman, A. (1983). Effective schools: Identifying the specific practices and behaviors of the principal. NASSP Bulletin, 67(463), 83-91.
Hallinger, P., & Wang, W. C. (2015). Assessing instructional leadership with the Principal Instructional Management Rating Scale. Dordrecht, Netherlands: Springer.
Hallinger, P., Wang, W. C., & Chen, C. W. (2013). Assessing the measurement prop-erties of the Principal Instructional Management Rating Scale: A meta-analysis of reliability studies. Educational Administration Quarterly, 49, 272-309.
Heck, R. H., & Hallinger, P. (2014). Modeling the effects of school leadership on teach-ing and learning over time. Journal of Educational Administration, 52, 653-681.
Hedges, L. V., & Olkin, I. (1985). Statistical methods for meta-analysis. Orlando, FL: Academic Press.
Hemphill, J., Griffiths, D., & Frederiksen, N. (1962). Administrative performance and personality. New York, NY: Teachers College, Columbia University.
Howell, V. (1989). Analyses by gender and by school enrollment, ratings of elemen-tary school principals’ instructional leadership behaviors (Unpublished doctoral dissertation). Tennessee State University, Knoxville, TN.
Jennings, T. C. (2013). School principals’ personality types and instructional lead-ership practices in selected distinguished and non-distinguished Title One ele-mentary schools in South Carolina (Unpublished doctoral dissertation). South Carolina State University, Orangeburg, SC.
Kane, M. T., Gilmore, G. M., & Crooks, T. J. (1976). Student evaluations of teach-ing: The generalizability of class means. Journal of Educational Measurement, 13, 171-183.
Kline, R. B. (2013). Exploratory and confirmatory factor analysis. In Y. Petscher & C. Schatsschneider (Eds.), Applied quantitative analysis in the social sciences (pp. 171-207). New York, NY: Routledge.
by guest on March 16, 2016eaq.sagepub.comDownloaded from
Hallinger et al. 33
Krüger, M. L. (1996). Gender issues in school headship: Quality versus power? European Journal of Education, 31, 447-461.
Krüger, M. L. (2008). School leadership, sex and gender: Welcome to difference. International Journal of Leadership in Education, 11, 155-168.
Latham, G., & Wexley, K. (1981). Increasing productivity through performance appraisal. Menlo Park, CA: Addison Wesley.
Lehl, A. (1989). A comparison study of the instructional knowledge and the instruc-tional leadership of elementary principals in Nebraska (Unpublished doctoral dissertation). University of Nebraska, Lincoln, NE.
Leithwood, K., Begley, P., & Cousins, B. (1990). The nature, causes and consequences of principals’ practices: An agenda for future research. Journal of Educational Administration, 28(4), 5-31.
Leithwood, K., & Jantzi, D. (2008). Linking leadership to student learning: The con-tributions of leader efficacy. Educational Administration Quarterly, 44, 496-528.
Leithwood, K., & Montgomery, D. J. (1982). The role of the elementary school prin-cipal in program improvement. Review of Educational Research, 52, 309-339.
Leithwood, K., & Sun, J. P. (2012). The nature and effects of transformational school leadership: A meta-analytic review of unpublished research. Educational Administration Quarterly, 48, 387-423.
Lipsey, M. W., & Wilson, D. B. (2001). Practical meta-analysis. Thousand Oaks, CA: Sage.
Marin, J. (2013). The relationship between self-efficacy, ethnicity, and beliefs about diversity to instructional leadership and transformational leadership practices of urban principals (Unpublished doctoral dissertation). University of Southern California, Los Angeles, CA.
Marzano, R. J., Waters, T., & McNulty, B. A. (2005). School leadership that works: From research to results. Arlington, VA: Association for Curriculum and Staff Development.
McCabe, B. (1993). Antecedents to principal instructional management behavior (Unpublished doctoral dissertation). University of California-Santa Barbara, Santa Barbara, CA.
McDonald, J. (2012). Instructional leadership and student achievement: The role of Catholic identity in supporting instructional leadership (Unpublished doctoral dissertation). Drexel University, Philadelphia, PA.
Miller, L. (1991). Perceptions of Alabama middle school teachers and princi-pals regarding the principal’s instructional leadership role and competencies as developed by effective school studies (Unpublished doctoral dissertation). University of Alabama, Tuscaloosa, AL.
Munroe, M. D. (2009). Correlation of emotional intelligence and instructional leadership behaviors (Unpublished doctoral dissertation). University of Phoenix, Phoenix, AZ.
Murphy, J. F., & Torre, D. (2014). Vision: Essential scaffolding. Educational Management Administration & Leadership, 43, 177-197.
Nogay, K. (1995). The relationship of superordinate and subordinate gender to the perceptions of leadership behaviors of female secondary principals (Unpublished doctoral dissertation). Youngstown State University, Youngstown, OH.
by guest on March 16, 2016eaq.sagepub.comDownloaded from
34 Educational Administration Quarterly
Nogay, K., & Beebe, R. J. (2008). Gender and perceptions: Females as secondary principals. Journal of School Leadership, 18(6), 583-602.
Nunnally, J. C., & Bernstein, I. H. (1994). Psychometric theory (3rd ed.). New York, NY: McGraw-Hill.
O’Donnell, R. (2002). Middle-level principals’ instructional leadership behaviors and student achievement (Unpublished doctoral dissertation). Lehigh University, Lehigh, PA.
O’Donnell, R., & White, G. (2005). Within the accountability era: Principals’ instruc-tional leadership behaviors and student achievement. NASSP Bulletin, 89(645), 56-71.
Peariso, J. F. (2011). A study of principals’ instructional leadership behaviors and beliefs of good pedagogical practice among effective California high schools serving socioeconomically disadvantaged and English learners (Unpublished doctoral dissertation). Liberty University, Lynchburg, VA.
Poovatanikul, V. (1993). Analyses of the perceptions of Thai principals and teachers as to the principal’s role as instructional leader (Unpublished doctoral disserta-tion). Southern Illinois University, Carbondale, IL.
Pounder, J., & Coleman, M. (2002). Women—Better leaders than men? In general and educational management it still “all depends.” Leadership & Organization Development Journal, 23, 122-133.
Pratley, D. (1992). A study of middle level principals’ involvement in instructional leadership (Unpublished doctoral dissertation). Michigan State University, East Lansing, MI.
Ramseyer, J. A. (1955). Factors affecting educational administration: Guideposts for research and action (The School-Community Development Study; Monograph Series No. 2). Columbus, OH: College of Education, Ohio State University.
Robinson, V. M., Lloyd, C., & Rowe, K. (2008). The impact of leadership on student outcomes: An analysis of the differential effects of leadership types. Educational Administration Quarterly, 44, 635-674.
Rose, M. (1991). Perceptions of selected principals in South Carolina concerning their instructional management behaviors (Unpublished doctoral dissertation). University of Southern Mississippi, Hattiesburg, MS.
Rosenthal, R. (1994). Parametric measures of effect size. In H. Cooper & L. V. Hedges (Eds.), The handbook of research synthesis (pp. 231-244). New York, NY: Sage.
Rosenthal, R., & Rubin, D. B. (1986). Meta-analytic procedures for combining stud-ies with multiple effect sizes. Psychological Bulletin, 99, 400-406.
Ruzicska, J. (1989). The relationships among principals’ sense of efficacy, instruc-tional leadership, and student achievement (Unpublished doctoral dissertation). University of San Francisco, San Francisco, CA.
Salley, C., McPherson, R., & Baehr, M. (1979). What principals do: A preliminary occupational analysis. In D. Erikson & T. Reller (Eds.), The principal in metro-politan schools (pp. 42-66). Berkeley, CA: McCutchan.
Scheerens, J. (2012). School leadership effects revisited: Review and meta-analysis of empirical studies. Dordrecht, Netherlands: Springer.
by guest on March 16, 2016eaq.sagepub.comDownloaded from
Hallinger et al. 35
Schoch, A. (1992). The relationship between instructional leadership behavior, school effectiveness, school size, gender, race, and years of principalship experi-ence in elementary schools in South Carolina (Unpublished doctoral disserta-tion). University of South Carolina, Columbia, SC.
Shakeshaft, C. (2006). Gender and educational management. In C. Skelton, B. Francis, & L. Smulyan (Eds.), The Sage handbook of gender and education (pp. 497-511). Thousand Oaks, CA: Sage.
Sun, J. P., & Leithwood, K. (2015). Direction-setting school leadership practices: A meta-analytical review of evidence about their influence. School Effectiveness and School Improvement, 26, 499-523.
Sutton, A. J., Song, F., Gilbody, S. M., & Abrams, K. R. (2000). Modeling publica-tion bias in meta-analysis: A review. Statistical Methods in Medical Research, 9, 421-445.
Trout, K. (1985). The relationship between gender and selected characteristics associ-ated with instructional leadership for senior high school principals (Unpublished doctoral dissertation). University of South Carolina, Columbia, SC.
Witziers, B., Bosker, R., & Kruger, M. (2003). Educational leadership and student achievement: The elusive search for an association. Educational Administration Quarterly, 34, 398-425.
Wright, B. D., Linacre, J. M., Gustafen, J. E., & Martin-Lof, P. (1994). Reasonable mean square fit values. Rasch Measurement Transactions, 8, 370.
Yang, C.-S. (1996). Instructional leadership behaviors of elementary school princi-pals in Taiwan, Republic of China (Unpublished doctoral dissertation). University of Northern Colorado, Greeley, CO.
Author Biographies
Dr. Philip Hallinger is professor of educational management at Chulalongkorn University (Thailand) and Distinguished Visiting Professor of educational leadership and management at the University of Johannesburg (South Africa). In 2014, he received the Excellence in Research Award from AERA and the Roald F. Campbell Award for Lifetime Achievement in Educational Administration from UCEA. His research interests include instructional leadership, school leadership effects and prob-lem-based leadership development.
Dr. Li Dongyu is a post-doctoral fellow in the Faculty of Education and Human Development, the Hong Kong Institute of Education. Her research interests include educational leadership, gender differences, and meta-analysis.
Dr. Wen-Chung Wang is chair professor of educational psychology and director of the Assessment Research Centre in the Hong Kong Institute of Education, Hong Kong SAR. He is a fellow of the AERA and associate editor of several psychology journals. His research interests include Rasch measurement, item response theory, computer-ized adaptive testing, cognitive diagnosis modeling, educational and psychological measurement, psychometrics.
by guest on March 16, 2016eaq.sagepub.comDownloaded from
Top Related