An Evaluation of the Promotional Processes in a
Large Texas Metropolitan Police Department
by
Stephen A. Bishopp
March 2010
CHAPTER ONE
INTRODUCTION
Police officers make many decisions which can profoundly affect human rights
and often do so without direct supervision (Maher, 2003; Klinger, 2004; Lersch &
Mieczkowski, 2005). The officer’s behavior is often only constrained by the utility of
some form of social control (Black, 1976). Organizationally, police leaders should be
able to effectively oversee operations; enforce policies and motivate personnel and
provide an atmosphere of professional conduct (Stafford, 1989). Therefore,
understanding the process by which a person is chosen to be in a position of leadership
within the police organization becomes vital.
This current research was undertaken as an examination of the process by which a
police sergeant is promoted to the rank of lieutenant in a large metropolitan police
department; in particularly one utilizing an assessment center (AC) in determining
promotability. To study this issue, data were taken from a large Texas metropolitan
police department (LTMPD)1. In the LTMPD, the rank of lieutenant is an upper
management, command level rank. The data were examined in order to assess how a
person, through a dyadic promotional process (written examination and AC participation)
is able to arrive at this rank. This thesis provides participant/researcher insight into the
LTMPD’s promotional process in order to inform decisions concerning its continued use
for identifying successful police leaders.
The rationalization for analysis at this level lies in the fact that the rank of
lieutenant is the last or highest rank within the LTMPD which utilizes a test based
promotional process where numeric ordering provides a hierarchical list. Candidates who
take the lieutenant’s test and finish at or near the top of the final list are a revered cohort
and are the most qualified for promotion. The data will suggest the results of this dyadic
promotional process to be largely serendipitous.
Leaders are chosen to establish departmental goals and be able to lead officers to
successfully carry out those objectives (Scarborough et al, 1999; Berkshire Advisors, Inc,
2004; Klinger, 2004). Data from the lieutenant testing and promotional processes at
LTMPD during the past eight and a half year period were analyzed. The data consist of
three lieutenant promotional exams which have heretofore received neither critical review
nor transparent evaluation. A better understanding of what increases the odds for
promotion and, therefore, the behavior of the organization itself is provided. Because of
the organizational importance of the police leader, a discussion on police organization
and police behavior theories is necessary.
CURRENT THEORY
Police Organizational Theory
Lavigna (2008) described the importance of leadership to an organization as
“critical to organizational success” (p. 9). Leadership is particularly salient in policing
where preservation of laws and protection of human rights are of primary concern.
Shafer (2009) states plainly “the need for effective leadership in policing is quite evident”
(p. 238). This need is both external and internal to the police department in the way in
which it affects general operations. Externally, society should feel that the police
department has the legitimate right to use necessary power or authority over the citizenry
and that the police can do the job asked of them in a fair and proper manner. Internally, a
police leader must be able to direct work groups efficiently, ensure compliance to
policies, and generally see that the mission of the police department is carried out (also
see Jermier and Berkes, 1979). This is true for all levels of leadership, from front line
supervisors (such as patrol sergeants) to top departmental commanders and chiefs of
police. Shafer (2009) conducted a survey of 769 police commanders at the Federal
Bureau of Investigation’s National Academy. In his subsequent report, he noted that,
“ineffective leaders demoralize subordinates and co-workers, presented a poor image of
the agency within the community, and served as poor role models for future generations”
(pg. 248).
Police Behavior Theory
Moving beyond simple organizational only views, Klinger (2004) observed that
police officers act based on the interplay of two dynamics; the organizational structure of
the department they work for and the environment in which they are expected to perform.
This interplay has been largely ignored, as evidenced in the paucity of research literature
on this relationship (Klinger, 1997; 2004). Klinger addresses the matter head on and
develops a theory which has very important underpinnings for this current research study.
Briefly, he described organizational determinants of police behavior as properties such as
degree of professionalism and bureaucratization; department size, presence of specialized
units, and policy implementation. Environmental factors include judicial decisions (e.g.
rulings in evidentiary hearings or search warrant requirements), the political landscape of
those governing the police, and social ecology. These multi-faceted determinants are
important to understand when conducting an assessment of police behavior. An effective
police leader must be able to effectively implement policy while being responsive to the
needs of community in which he or she directs officer behavior.
The social ecology of the police is also important to understand in the context of
police leadership. Klinger (1997) notes the relationship between officers’ behavior and
“the level of social deviance in the patrol district in which they are situated” (pg. 300).
Klinger is saying that officers function within the context of the division or district in
which they work. For the purpose of the current research paper, Klinger’s patrol districts
are synonymous to the patrol divisions described in this current research in which daily
operations are largely reflective of the lieutenant’s leadership skills. In his article,
Klinger described the actions of officers as vigorous if they were likely to extend their
authority by making arrests, conducting investigations and so forth. The actions of
responding officers are viewed as lenient behavior when they are likely to do the opposite
and take no action. Hence, directing and monitoring the behavior of the officers and
setting their performance expectations are reflective of the leadership capabilities of the
lieutenants in command of the officers.
Within the LTMPD, lieutenants are the key in the supervision and management of
patrol practices within divisional boundaries. Larger cities are divided up into smaller
parcels in which police patrol (Wilson, 2003; Klinger, 2004). The LTMPD is broken up
into seven patrol divisions; each division is further divided by five to six sectors, and
each sector segregated further into beats. A lieutenant is generally responsible for the
entire division (having on the average between 200 and 300 subordinate sergeants,
corporals and patrol officers) much like the patrol division’s Deputy Chief. The Deputy
Chief has twenty-four hour responsibility for all divisional functions such as
administration, investigations, property maintenance, accountability, and so forth. The
lieutenant oversees these same patrol level functions during his or her eight-hour shift
(each division divided temporally by watches or shifts; day shift, evening shift and late
night shift and spatially by physical street boundaries, reporting areas, political
boundaries and, so forth). The lieutenant is responsible for the division’s officers’
service delivery, order maintenance, enforcement of state laws, and local ordinances
(Klinger, 2004). Additionally, lieutenants are responsible for subordinate officers’
adherence to departmental policy and discipline (Stafford, 1989). This is a tremendous
responsibility. Besides daily, routine operational oversight, lieutenants are expected to be
able to have boots on the ground in unexpected and extraordinary events which occur,
and direct line personnel during stressful events (e.g. hostage situations or riots). They
must be involved in planning and logistics during scheduled events to ensure effective
police presence at large gatherings (e.g. charity events or marathons). The police
lieutenant is also expected to direct the supervision of officers and accomplish established
goals during devastating natural disasters such as hurricanes, tornados or large fires.
The Management Efficiency Study of 2004
The number of lieutenants promoted in such a short period of time within this
urban police department demonstrates the need for an extensive review of promotional
processes. It must also be understood that nearly as many senior lieutenants would have
had to retire or otherwise vacate their rank to create those open positions. Berkshire
Advisors, Inc., (2004) provided a critical assessment of the management and efficiency of
LTMPD. They produced an aggregate level critical assessment of all operations and
processes within the police department which resulted in the publication of a
Management Efficiency Study (MES).
There are several key issues within the report which impact this research paper.
The deficiencies brought to light in the MES regarding promotion; training, competence,
and morale surrounding mid-level management (lieutenants) are disconcerting.
Examples of deficiencies in these areas include a near lack of past performance (positive
or negative) being reviewed and considered for promotion; lack of required
“supplemental supervisory training”, being out of touch with line personnel, and micro-
managing. Also, the MES discussed the potential for mass promotions in the mid and
upper management level ranks. Regarding this point, it stated, “The department is facing
a significant leadership vacuum over the next five years” (Berkshire Advisors, Inc., 2004;
Chapter II, p. 12). Two of the three tests occurred after 2004; the premonition of a
leadership vacuum appears to have come true.
The MES was initiated in order to evaluate the effectiveness of many areas with
the LTMPD. It identified policies and practices which needed improvement, and
suggested ways in which to make those improvements. Of particular interest to the
current research is the promotional schema of the LTMPD in light of the report because
the promotional process has not changed since the release of the MES.
CHAPTER TWO
CURRENT RESEARCH
Although it gives rise to a number of important recommendations for change
throughout LTMPD, the MES is not the sole motivation for the current research paper.
There is a paucity of academic research specifically reviewing the police management
promotion predictors or the after effects. Much of the prior research focused on methods
of identifying officers worthy of promotion via AC (Ross, 1980; Feltham, 1988; Lowry,
1997 Brown, 2000; Ho, 2005; Lance, 2008; Rupp et al., 2008) promotions via
demographic or age related variables (Prewett-Livingston et al., 1996; Adams 2002;
O’Brien, 2003; Ho, 2005; Krajewski, H. and Rothstein, M., 2007; White, 2008; Archbold
& Schulz, 2008; Archbold, C., & Hassell, K., 2009) or other motivations for seeking (or
not seeking) to advance up the chain of command due to multi-cultural, financial, or
familial reasons (Whetstone, 2001).
The vast majority of research was conducted externally which means by an
observer or researcher outside of the process. None were done by someone who was
directly involved in and affected by the promotional process. The researcher on this
project was first an active participant in the three promotional processes, and then later an
academic researcher examining the process in which he had participated. It is from this
methodological approach that current research will shed light on this promotional
process. Furthermore, it is the intent of this project to contribute quantitatively to the
extant literature on police promotions and provide a benchmark by which further
academic analysis of promotional processes can be conducted. The current research also
suggests a theory for police organizational success. In order to obtain effective police
leaders, an organization must promote those with the ability to efficaciously provide a
positive work environment for subordinate police officers as well as build police
legitimacy within the community.
Based on the importance of leadership and considering the supervisory finesse
needed to successfully function organizationally and environmentally, one would expect
promotions to be based on merit or performance. The LTMPD’s use of the lieutenant’s
assessment center is used to gauge a candidate’s managerial merit and predict one’s
ability to successfully perform as a lieutenant. Some may argue that the AC process is
able to effectively evaluate candidates and predict future success and therefore justify its
high financial cost2 (Krause et al., 2006). Those who are able to display leadership skill in
the AC must surely have merit as a manager. Many municipalities appear to have bought
into this concept of the AC’s ability to assess leadership qualities. A survey found 62%
of cities with populations over 50,000 used the assessment center for police and fire
promotions (Lowry, 1997). However, despite the AC’s abundant use the current
literature is far from congruent on the issue of AC validity3.
In light of the extant literature this research hypothesized that many variables are
unconnected to merit but are connected to promotion. This paper reviews the literature
on predictors of promotion. Next, there will be a discussion of the importance of
effective police leadership by describing what the dangers of poor leadership can be.
Following that, methodology and results are presented. Finally, this paper wraps up by
discussing the findings and the variables which either influences or has no influence on
being promoted to lieutenant within the LTMPD.
VALIDITY AND ASSESSMENT CENTERS
The LTMPD Assessment Center
There are a number of variables being analyzed in this thesis as a predictor of
promotion. Chief among these variables is the AC score earned by the promotional
candidate. The reason for choosing this as the most important independent variable is
simply because of the formula used by the City of Dallas to determine an overall score.
That formula, written score (40%) + assessment center score (60%) = 100% of the final
score. Hence, it is reasonable to assume that the greatest predictor variable will be the
assessment score. The following section discusses and reviews the research on
assessment centers and the validity of those measures within the context of the law
enforcement promotional process.
This research utilizes Ross’ definition (1980) of the term ‘assessment center:’
“The term ‘assessment center’ describes the simultaneous assessment of several individuals conducted by a group of trained evaluators using a variety of exercises and providing evaluation of a number of abilities.” (p. 89)
Simply stated, this means a number of people (assessors) evaluate candidates in
several scenarios (in this case study of a LTMPD it is three scenarios) to
determine or judge the candidates possibility of success as a police lieutenant.
During each assessment session, the candidate is provided a timed problem or
situation and is graded on his/her ability to perform as a lieutenant in the context
of several specified dimensions (Ross, 1980; Feltham, 1988; Rockhill & Halton,
Inc, 2007). As an example, the lieutenant’s test for the LTMPD measures six
dimensions; written communication, oral communication, supervisory
skills/administrative skills, problem analysis, interpersonal relationships, and
leadership. What should also be understood is what the assessors are not
supposed to be grading. These dimensions are; job knowledge, understanding of
the job effectiveness under stress, and attention to detail. The unevaluated
dimensions are purported to be embedded in the exercise but they are not graded
(Rockhill & Halton, Inc, 2007).
The AC for the LTMPD takes place over three consecutive days in which
a tactical exercise, oral presentation exercise, and an in-basket exercise3 are
presented; one exercise per day. The oral presentation and tactical exercise are
structured in the same way; thirty minutes to review written material and take
notes, a ten minute break, and then ten minutes of interview by two or three
assessors. For the in-basket exercise the candidate’s exercise time is extended;
one hour to review material and take notes, five minutes to put the reviewed
material in a stack with the highest priority item first. The candidate then gets a
thirty minute break after which twenty minutes is allotted for the assessors to
interview the candidate and ask questions concerning the in-basket. What this
essentially represents is a period of time no longer than forty minutes spread out
over three days by which the assessors decide who has the skill to be a lieutenant.
Another important aspect of the AC should be brought to light. During each
allotted time period; ten minutes on day one, ten minutes on day two, and twenty
minutes on day three there are a different group of assessors during each exercise.
That means no set of assessors sees the same candidate over the entire forty
minutes. Essentially, leadership/managerial skill decisions are made in
increments of ten minutes, ten minutes, and twenty minutes.
With the AC explicated, the discussion moves necessarily on to the
validity of the ACs and the information they provide via this portion of the
promotional process. The validity of police selection processes has been
challenged many times in the past concerning their predictive value for future
police performance (Ho, 2005). The next section reviews at some length the
literature on AC validity.
The Types of Validity Related to Assessment Center Measurement
LTMPD trusts that AC’s are valid for police promotions. More
specifically, they rely on the strength of the ACs capability to accurately measure
and determine the level of leadership skills in tested individuals, thereby
predicting a high likelihood of success at the rank of Lieutenant. Additionally,
this is the highest rank currently attainable on the LTMPD for which one
competes for promotion by means of testing (the next level is the deputy chief
rank and is a command level position appointed solely at the discretion of the
Chief of Police). Analyzing the testing process to determine how one becomes a
lieutenant is more crucial for this position because this is the rank by which the
police department’s chief level officers are selected. Because of the assumed
strength of the relationship between AC scores and promotion, it becomes
imperative to review the research on the validity of assessment centers.
Ross (1980) conducted a criterion related validity analysis of AC use in
law enforcement promotional processes. Criterion validity refers to the ability of
the AC to rate variables such as judgment, oral expression, problem analysis and,
leadership (Ross, 1980; Feltham, 1988; Rockhill & Halton, Inc, 2007). This is
accomplished through “focus on the correlation of the test being validated with
some well-respected outside measure of the same objectives” (Brown, 2000; p. 8).
Ross’ (1980) analysis was of a southern California police agency which rated 49
police promotional candidates. The purpose of the study was to determine the
degree to which an AC was capable of predicting effective and ineffective
management behavior. The determination was made via a two year follow-up
using performance evaluations as a measurement guide.
This study revealed several key points. Ross (1980) found that the
Pearson’s correlation between job performance and AC ratings was .47, a
moderate to high level of correlation (Maxfield and Babbie, 1995). Ross then
compared this correlation to the correlation between written test scores and job
performance. Ross (1980) found that written tests were correlated with job
performance at a level no higher than .30 (also see Feltham, 1988). Secondly,
verbal skills, rather than job knowledge, organization, writing skills, etc., were
heavily relied on (Ross, 1980). She stated that “it appears that the assessors relied
mostly on evidence of interpersonal skills and communication skills to make their
final rating” (Ross, 1980, p. 94). The study demonstrates the value of rating the
AC process post-promotion in order to judge the predictability of the AC.
One other important concern discussed by Feltham (1988) was the
cognitive frame through which assessors rated promotional candidates. For
instance, the author found that assessors rated individuals similarly. However, the
assessors had differential definitions of the various job performance criterion used
to rate the candidate. Furthermore, he states, “another explanation could be that
factor analyses of performance ratings indicate more about the cognitive structure
of the raters than the behavior patterns of the ratees” (p. 139). This appears to
point to a realization that raters perceive the same job performance differently
because of their own perceptions. Therefore, one’s overall rating may have much
more to do with the raters than with how the promotional candidate actually
functions during the AC.
Content validity describes the ability of the AC exercise to evaluate the
objective it is meant to gauge. In other words, if the candidate is being assessed
on his or her ability to display leadership and interpersonal skills, does the subject
matter contained within the exercise actually target those dimensions? Content
validity is based on the particular tasked being performed. Lowry (1997)
purported this approach to AC testing as a better way to actually gauge a
candidate’s ability at getting a specific job done. For example, when rating a
candidate’s organizational skills, a scenario may present several issues to the
candidate; notify personnel under their command of meeting, discuss a possible
agenda with peers, pick the place for the meeting and, brief the chain of command
(i.e., the deputy chief under whom the lieutenant serves) of the content and
outcome of the meeting. A check list of typical actions could be used to indicate
when each action or task is accomplished (Lowry, 1997). He goes on to state,
“The task-specific assessment center is concerned primarily with evaluating the subject’s performance in tasks (work samples) related to the job. Therefore, the desired outcome from such a job analysis should be the tasked performed on the job, rank ordered based on the importance, frequency and need to perform on entry to the position” (Lowry, 1997, p. 55).
Lance (2008) also argues for the advancement of content validity rated
AC’s. He posits that when assessing dimensions (as occurs with construct and
criterion validity measures) such as leadership or communications skills, the
candidate is often assessed based on how well he or she performs within the
context of the exercise setting rather than the actual portrayal of the dimension.
In his investigation of assessment center validity Lance (2008) observed “factor
analysis almost always support robust exercise factors and not dimension factor”
(p. 87). Rather than capture the behavioral dimension for which a particular
exercise is designed to measure, the exercise itself becomes the evaluated task.
This is an important finding when taking into account the LTMPD AC
process. In each exercise scenario the candidate is put into a role and presented a
problem. For example, he or she may be told to act as though they are at a city
council meeting and addressing council members. The candidate is given the
problem to address and a maximum of ten minutes (Rockhill & Halton, Inc, 2007)
to “act” out the scenario. It can be easily deduced that a person who is better at
acting as a manager (lieutenant) may fair better than the person who is, in fact, the
better manager but with poor acting skills. Furthermore, Lance (2008) makes the
argument that the AC does not measure those dimensions for which they were
designed. He found that assessment centers “evaluate candidate behavior with
respect to potentially content valid simulated performance situations” (p. 93), and
not psychometric dimensions.
Rupp et al. (2008) dispute Lance’s finding and observe that AC’s do have
the ability to measure the dimensions of the candidates. They go one to argue that
the dimensions are the sum total of numerous abilities needed to affect good job
performance. They state that, “knowledge, skills, abilities, and so on that are
identified as (a) job relevant (b) most appropriately accessed via behavioral
methods.” Furthermore, one could expect the role of the manager to be
multidimensional when performing a specific task. Assigning personnel certain
days off while keeping in mind each person’s unique life situation vis-a-vis
departmental needs and goals. In an exercise of this type the candidate would be
assessed within several dimensions; leadership, interpersonal skills,
communication, and so on. Rupp et al. (2008) state that it is peculiar to measure
one task at a time when one task at a time is not how the ‘real world’ works.
Construct validity concerns the utility of an AC in transforming a
dimension or construct into a quantitative and measurable variable. Brown (2000)
describes construct validity as “experimental demonstrations that a test is
measuring the construct it claims to be measuring” (p. 8). The question here is
whether or not an exercise aimed at measuring a construct is able to actually judge
and measure the candidate’s ability. The AC used by the LTMPD rates the
following dimensions or constructs; written communication, oral communication,
supervisory skills/administrative abilities, problem solving, interpersonal skills,
and leadership. (Rockhill & Halton, Inc, 2007). Brown (2000) suggests that there
is no single way to measure the construct validity of a test. Furthermore, construct
validity measures are likely to have problems of ambiguity caused by attempting
to define the construct by which a candidate will be evaluated. Lowry (1997)
observed that there are over 8,600 definitions for the dimension of leadership. He
also suggests using task-specific measures rather than trying to quantify complex
constructs.
Importance of Validity
According to Kroecker (2000) assessment centers are a way in which
police departments can more effectively measure those abilities not captured
through written tests or performance plans (job evaluations)4. Kroecker indicates
the AC is the preferred method and should be given greater weight in predicting
of future behavior over performance plans, written scores, and even extensive
service. The reason is that a promotional candidate may have superior strengths
in all of these areas but lack the ability to think critically or manage personnel.
Even though a heavier weight being given to the AC is suggested, Kroecker
(2000) advances the opinion that AC score should not be the stand-alone decider
for promotion. Other indicators of performance (i.e., interviews, minimum
number in current job, or overall tenure) should be considered as equal pieces of
the promotional pie.
Made clear in the article on AC benefit, Kroecker (2000) explicated the
insurmountable importance of validity. It is absolutely essential for an AC to
provide valid results in order for it to predict leadership qualities. Feltham (1988)
offered a caveat when evaluating assessment center validity. Psychometric
problems (the measurement of personality traits for example) can create undesired
results. He posits that the halo effect, for instance, is the most intractable.
Feltham (1988) further suggests incorporating a more comprehensive assessment
of validity from such sources as subordinates and peers to allow for a more
complete picture of promotability.
DEMOGRAPHIC PREDICTORS
Age
The influence of age has been analyzed in recent research (Adams, 2002;
Krajewski, & Rothstein, 2007; White, 2008). Of particular interest in the articles
is how the age differential effects decisions to promote within a police
department. One may expect minimal promotional effect in an organization
operating within the constraints of a paramilitary rank structure. The ADEA (Age
Discrimination in Employment Act) places federal restrictions on employment
status related to age. The law also affects the use of age as a variable utilized as a
decision factor for promotion. That is not say age has zero effect. For instance,
one must be at least 19 ½ years of age6 in order to be hired; serve a specific period
of time at each rank before being considered for promotion to the next level, and
so forth. Therefore, while age necessarily has an effect on time-in-grade (amount
of time spent at a rank) and time-in-service (aggregate seniority on the
department), chronological age should have no effect on who does and does not
get promoted. According to the analysis of the date from the 202 promotional
candidates of this thesis, this does not appear to be the case.
The discussion in recent research indicates age may very well play a role
in who is deemed worthy of promotion. This effect can be negatively correlated
to promotion (Adams, 2002), or have a positive effect on AC scores for older
individuals (Krajewski, & Rothstein, 2007). Krawjewski and Rothstein (2007)
tested the effect of age on AC performance and found that older individuals may
enjoy an advantage. An examination of 371 police managers revealed that while
the effects where moderate; there was support for use of age as a predictor for
promotability of police managers They observe, “the current study provides some
support for this suggestion by showing that certain traits are more highly related
to AC scores when age is taken into account” (pg. 30). Further noted by
Krawjewski and Rothstein (2007) was the suggestion that AC assessors may be
using stereotypical processes when faced with large numbers of candidates. They
posit a halo effect by the assessors by which older candidates were predetermined
as more capable “in order to facilitate the complex decision making and rating
tasks required of them” (pg. 29).
Interestingly, an opposite argument was offered by Krawjewski and
Rothstein (2007) with stirs the debate on the mitigation the age factor. Possibly,
older candidates may be stereotypically viewed as being less energetic or slower
to learn; hence the older individual is less desirable for consideration of
promotion. However, should a candidate be able to cross the age stereotype, their
AC score would be considerably higher.
The argument of the age factor being utilized and inversely correlating to
promotion was analyzed in an article by Adams (2002). While not within the
context of policing, the research clearly points out the age effect during the
promotional decision making process. Adams (2002) suggests older workers are
passed over for promotion but economic reasons drive those decisions. Career
events such as retirement packages for older workers and the creation of job
attachment through promotion of younger employees may be driving factors in
career advancement decision choices by company leaders.
The studies by Krawjewski and Rothstein (2007), and Adams (2002),
make viable cases concerning age mediating the decision to promote. Both are
important to this current research because they demonstrate age may be a
controllable factor for analyzing a person’s promotion within an organization. It
may be an economically based option for a company or the employee, the effect
of AC assessors’ preference or the candidate’s superior demonstration of
managerial traits; either scenario demonstrates that age cannot be simply
dismissed as a predictor of promotion.
Gender
There was paucity of research available specifically addressing the subject
of female promotion rates within a police department. Participation in testing and
eventually getting promoted is a self-selected process and is necessarily affected
by the decision to not seek promotion. While it has been suggested that males fair
better during academy training (White, 2008), the issue of promotions and the
gender effect is largely unexplored (Archbold & Schulz, 2008; Archbold &
Hassell, 2009).
The gender question on promotional outcomes was examined within the
context of making a decision not to promote. Tokenism was seen as one such
reason. Archbold and Schulz (2008) suggested that tokenism in the police
workplace is more complex and could not be seen as a simple numbers game. A
female who is seen as a token promotion (or view themselves as a token) “are
faced with high visibility, isolation, problems assimilating into the group, and
performance pressures that stem from a combination of these factors” (Archbold
& Schulz, 2008; p. 52). Females may choose not to promote even if they believe
they are competent and capable of fulfilling the role if promoted, they decide not
to risk being seen as a person promoted based solely on their gender.
In the similar vein of opting out of the promotional process, Archbold and
Hassell (2009) found that females’ reasons for deciding not to promote were very
closely related to a man’s reasons. In a list of top five reasons for women and
men declining to participate in the promotional process, the lone difference was
administrative bias. The bias is related to tokenism and that females may decide
against promotion because they are being pushed by their bosses to participate
and promote. Therefore, it is felt as though they are being singled out and
eventually promoted because of gender, not ability.
Archbold and Hassell (2009), however, point out that it is the “marriage
tax” that may be inhibiting female officers from seeking promotion. They must
consider child care, family schedules, and pay when deciding to advance their
rank. Organizationally, females are faced with (or more aware of) anti-nepotism
rules. These policies “restrict the promotional opportunities for women who are
married to male officers” (p. 70). While this certainly may be the case, especially
in smaller departments, a large metropolitan area consisting of more than 3,000
officers (e.g. the LTMPD) would certainly have a greater number of rank specific
career opportunities; thereby, nepotism would be of less concern. The gender
variable, however, cannot be overlooked.
Race
Affirmative action promotions and race quotas have been analyzed in
several recent articles (Prewett-Livingston et al., 1996; O’Brien, 2003; Ho, 2005).
Although not specifically addressing promotions to managerial ranks, this
research is insightful concerning hiring policies that can be directly related to
departmental promotional polices. For instance, the predictability of race on
determining employment would naturally be transferable to policies directing
promotional processes. Therefore, affirmative action requirements and quotas
used to increase employment among certain demographically situated individuals
should be assumed to increase minority representation within a particular rank.
O’Brien (2003) found that this may not always be the case. He analyzed the
initial employment of minority group members regressed on variables found
within the population (number of minorities in population; also see Ho, 2005),
and governmental structures (race of mayor and race of city council
representatives). Not unexpectedly, he found populations with a greater number
of minorities “had the most consistent, positive but modest, effect on minority
employment” (p. 194).
Interestingly, O’Brien (2003) found that policies utilizing affirmative
action without using quota goals were not found to increase the representation of
any particular minority group. However, used in conjunction, Black and Hispanic
employment increased. In addition, higher pay actually accounted for decreased
minority employment (O’Brien, 2003). In contrast, Ho (2005) found “race-based
favoritism toward ethnic minority applicants might not exist in the police
departments recruitment decisions” (p. 470). But, affirmative action was found to
increase metropolitan minority police officer ranks. Ho (2005) also found
psychological recommendations were not favorable to minority applicants.
Recruiters did not place a high importance on those recommendations so the
impact on minority hiring was not significant. This possibly reflects the
importance of affirmative action goals being placed over psychological
evaluations. Either way, in light of these current analyses, a possible race based
paradigm among those designing and executing hiring examinations exists.
Therefore, it can be naturally assumed that race plays a role in the promotability
of a particular person based on demographics.
Further muddying the waters of the race component and promotions,
Prewett-Livingston et al. (1996) provide evidence that race and final AC scores
are related. They contend that AC assessor race influences final ratings.
Particularly, they investigate possible similarities in race between assessor and
candidate influence final score outcomes. They posit “members of groups (e.g., a
racial group) will tend to evaluate members of their group more positively than
members of other groups” (p. 179).
This study specifically addressed a police manager AC and the final scores
assigned to candidates being a function of assessor panel race distribution.
Prewett-Livingston et al. (1996) analyzed the a sergeant’s AC (N=135) and offer
the summation,
“In summary, despite its limitations, the present field study offers evidence that rated performance of Black candidates and White candidates in a selection panel interview was a function of the Black-White composition of the interview panels and the race of the panel interviewers.” (p. 185).
While the limitations in this study included being unable to completely control the
racial makeup of the panel interviews (assessors) and accuracy of the panel scores
being unavailable, this study certainly points a statistical finger in the direction of
“race matters” for promotional considerations. Hence; race must be included in
any examination of police promotional processes.
Education
Since August Vollmer was Chief of Police in Berkley, CA, at the beginning of the
twentieth century, higher education has been proposed and debated as a requirement of
employment as a police officer. Education is seen as something that should be ongoing
(Travis, 1995); essential to effective community policing (Breci, 1994); or as having little
or no influence on successful police training (White, 2008). The federal government
weighed in on the issue of higher education for police officers through The President’s
Commission on Law Enforcement and the Administration (1967) which made the
following observation and recommendation,
“A policeman today is poorly equipped for his job if he does not understand the legal issues involved in his everyday work, the nature of the social problems he constantly encounters, the psychology of those people whose attitudes toward the law differ from his. Such understanding is not easy to acquire without the kind of broad general knowledge that higher education imparts, and without such understanding a policeman's response to many of the situations he meets is likely to be impulsive or doctrinaire. Police candidates must be sought in the colleges and especially among liberal arts and social science students” (p. 107)
The question of the level of education obtained by a police officer and its effect
on the officer’s performance and/or suitability for a policing career continues to be
discussed extensively in more recent literature. (Bayley, 1994; Breci, 1994; Travis,
1995; Mastrofski, 2006). More precisely, the quandary has been directed at how much
education is needed to have a positive influence on the officer. Mastrofski (2006) wrote
that there was no evidence that there was an impact on officer decision making (an
integral dimension for successful policing). Mastrofski asked several questions which
helped explicate the gap in the research evaluating the positive or negative effect of
higher education on police officers. Two questions are particularly poignant: (1) “What
skills and habits, if any, does college develop?” and (2) “Do some police assignments
benefit when college-educated officers perform them but others do not?” (p. 7).
Neither of these questions will be specifically answered by the current research.
The use of educational attainment as a predictor of police promotions, however, may
shed some light on the extent to which education plays a part in the promotional process.
Certainly, skill and the ability to perform a particular duty (Breci, 1994), such as one
required of a lieutenant, are important, it would be beneficial to know to what extent
higher education plays in one’s promotion to lieutenant. This debate takes us a step
closer to understanding the importance and connection of obtaining a higher education
and its influence on the promotion of a police lieutenant within the observation group.
The next logical step would be to gauge the effectiveness of a college education in
predicting the promotion of police commanders. Or, just as importantly, the
ineffectiveness.
Higher education is thought by some to be the future of law enforcement (Bayley,
1994; Breci, 1994). Such reviews speculate that the “complexities of policing would
require higher levels of education” (Breci, 1994, p. 1). Bayley (1994) recognized that a
higher education is an important variable for senior police promotions on numerous
police departments throughout the world. He observed the average educational level of a
police officer rose from a high school diploma in 1969 to at least a two-year college
degree by 1990.
On the other hand, White (2008) analyzed police officers’ performance by way of
measuring their performance in the academy. The premise is that officers who
demonstrate superior performance in the police academy will most likely be better
officers on the street. That being said, White (2008) attempts to predict officers’
academy performance by controlling for several variables, college education being one of
those. There can be little argument against a police officer gaining more education.
Unquestionably an academically well rounded officer could be expected to make better
and more objective decisions. Through a multivariate analysis, he finds that the
possession of a college education is not statistically significant in the prediction of
successful academy performance.
The literature does not quantify the extent to which education is able to actually
predict a skillful leader. There is a questionable gain by obtaining a college education
when the effect may not last a significant amount of time (Mastrofski, 2006). However, a
criteria or qualification which is important for initially hiring a person can be inferred as
being even more important when one is promoted to a managerial rank. Obtaining a
college education (or at least some college credit) is used by some police departments as
a hiring requirement and seen as facet of future policing. However, it is hotly debated as
to the importance or influence education should have in hiring or promotion of police
officers. More study is needed to properly quantify the effect education has on successful
police leadership. This research attempts to take a step in that direction, albeit small, by
controlling for education and operationalizing it as a dummy variable. This analysis will
allow for a measure of the effect of education on the promotional process without being
confounded by data showing number of hours of higher education one may have.
Past Behavior, Tenure, and the Number of Times Tested for Promotion
Danner et al (2008) studied the effect of habits as predictors of future behavior.
They found that habit attenuated the effect of intention-driven behavior. The incidences
of commendations (good behavior), and sustained complaints (bad behavior) indicates
frequency of past events; therefore, it describes the candidates past behavior. The
number of times tested indicates a past behavior that is intentional but not necessarily
habitually performed. This research tests commendations and sustained complaints, and
the number of times tested to analyze their effect on the promotional process. Further,
the promotional candidate’s tenure is expected to have a direct correlation to times tested
and the number of commendations and sustained complaints due to temporal conditions.
Those who have been on the LTMPD longer had more opportunity to take numerous
tests. Their longevity also afforded them more opportunity to receive commendations or
sustained complaints. Additionally, there is very little quantitative research on the effect
of tenure as a predictor of future behavior. Therefore, tenure is not used utilized as an
independent variable but is provided in Figure 1 for illustrative purposes.
DEMONSTRATING THE NEED FOR EFFECTIVE POLICE LEADERSHIP
Hays et al. (2007) accurately and succinctly state, “Strong leadership is essential
for effective policing” (p. 3). Strong leadership, however, creates several connotations in
what it takes to be a strong leader. Rowe (2006) and Densten (2003) note the importance
of portraying leadership qualities to encourage “followership” within subordinates.
Among some of the issues addressed in assessing effective (or strong) leadership are such
variables as time in grade (Rowe, 2006), education (Hays et al., 2007), and behavior
modification (Trautman, 2002; Hubert et al., 2007), Rowe (2006) explicated a common
theme in via an ethnographic study of lower rank officers’ perceptions of senior
management. He found that on average the street level officer saw senior management as
detached from day to day operations or “real police work.” This leads to a loss of
legitimacy in decision making skills of supervising officers and creates a milieu of
subordinate officers who have little faith in their commanders. Rowe (2006) further
suggests a time in grade requirement ought to be considered. Officers should be expected
to stay at a certain rank for a substantial period of time in order to gain experience before
moving up to the next level. He found, “That the rapid ascent of some officers up the
rank scale meant that they did not have time to develop operational experience was often
noted as problematic by PCs [police constable or officer]” (p. 765).
Beyond the subjective and sometimes narrow view taken by lower ranking police
officers there lie some real, objective problems. When there is a failure of proper
supervision or appropriate oversight by senior police managers there can be
departmentally damaging misconduct. This misconduct can range from overlooking
obvious ethical problems, improper or inefficient training program, refusing to be held
accountable for decisions and so on (Trautman, 2002) which may lead to systemic morale
issues and ethical problems with personnel.
Illuminating the damaging effect of poor supervision, Pogarsky and Piquero (2004)
applied deterrence theory to the study of police conduct. They found perceived sanctions
which come certainly and quickly lowered instances of police misconduct. In other
words, if an officer knows for sure that his or her misconduct will be found out and
quickly punished, they will most likely be deterred from it. A common theme found in
the literature is a police manager who is involved with his own misconduct is far less
likely to observe or punish the misconduct of others.
Promoting a person who has demonstrated past performance problems or created
disciplinary issues through their own misconduct is not conducive to good leadership.
While there some argument as to how much or to what extent past behavior predicts
future performance (Ouellette & Wood, 1998; Ajzen, 2002), there can be little doubt that
one does not magically gain police knowledge, morals, or virtues simply by being
promoted. Strong leaders possess these tools before they come to the table.
Huberts et al. (2007) identified three leadership qualities and their effects on
police integrity violations. They found that role modeling, strictness, and openness of the
manager have a significant effect on most of the twenty integrity violations analyzed in
this study. The police leader must display those characteristics he or she wishes to see in
officers under their command (Densten, 2003) and lead by example (Engel, 2003). Engel
(2003) identified four styles of supervision, each having a positive and negative
characteristic. The commonalities in the positives were such concepts as the ability to
form relationships with subordinates and being supportive of decisions made by lower
ranking officers. In addition, the most productive style of supervision was one in which
the supervisor was actively engaged. Engel (2003) describes this style of supervisor as,
“Active supervisors embrace a philosophy of leading by example. Their goal is to be heavily involved in the field alongside subordinates while controlling patrol officer behavior, thus performing the dual function of street officer and supervisor” (p. 6).
HYPOTHESES
This evaluation of the promotional process is done in order to answer the basic
question: what does it take to get promoted? Organizational theory posits that the best
and the brightest should rise to the leadership roles. This ensures the organization’s
success and is true for any organization, private or public. It is a theory not specific to
police organizations. Klinger’s (1997) integrated police behavior theory gives a very
insightful glance at police behavior at the individual level as well as at the organizational
level. Considering these two theories together, the importance of having a means by
which to choose effective police leaders can easily be seen. Theoretically, the police
leader has to be able to successfully function in the police milieu while simultaneously
juggling the environmental factors external to the police organization. For this analysis
the lieutenant rank is purposefully targeted because it is the highest position for which
one may test. Most importantly, though, it is this rank where the success of the
organization relies on how well the organizational rubber meets the environmental road.
Following previous research (i.e., Ross, 1980), this study examined the following
hypotheses. The first set of hypotheses tested in this current research attempts to
examine what impacts the AC outcome (See Table 1 Below for Summary). AC’s
assessors rate promotional candidates on acting skills (Lance, 2008), job specific
behaviors (Lowry, 1997), as well as interpersonal skills and communication skills (Ross,
1980).
What is known about the current data is that one must score high enough on the
written test to qualify for the assessment center. Candidates are given a 100 point written
test and a passing score is assessed; therefore, the written score coefficient is expected to
be in the positive direction. All candidates who score at or above the passing score are
invited to continue with the second step of the promotional process, the AC. It is
hypothesized that the written score will have a positive influence on the AC score. All
other variables are not factored into the final score and should have no effect on the AC
score. However, due to the extant literature, final AC score will likely be influenced by
those variables.
Education plays no direct role in one’s qualifications and eligibility to be
promoted to Lieutenant. However, a certain level of education must be obtained to be
hired on the LTMPD so education is expected to impact promotability in a positive
direction. Therefore, it is hypothesized that education will have a positive effect on ones
promotion.
The percentage of females on the LTMPD remains remarkably similar to state and
national percentages of representation on the police force as an entire unit. All police
departments are expected to maintain representative diversity in their hiring and
promotion practices (EEOC, 2010). Thus, this research expects that being a female
sergeant will positively correlate with promotion. However, much of the extant literature
concerns candidate demographics such as age and minority status. For instance, age
related to assessor perceptions of promotability can both negatively and positively affects
final scores (Adams, 2002; Krajewski and Rothstein, 2007). It is expected that age will
have an effect; however, it is unclear from the conflicts in the extant literature whether
the impact will be in a positive or negative direction. Additionally, race is expected to
affect the promotion decisions. Prewett-Livingston et al (1996) found that race
(minority) also has a discernable effect on how raters score candidates during AC’s (also
see O’Brien, 2003; Ho, 2005).
Sustained complaints and Commendations are used as proxy measures of the
candidate’s past behavior. Behavioral dimensions (e.g., leadership, communication,
organizational, and interpersonal skills) are assessed during the AC and some
measurement of one’s past is appropriate. While it is arguable that past behavior is a
good predictor of future behavior (Ouillette and Wood, 1998; Ajzen, 2002), the proxies
utilized here are the written documentation of positive and negative behavior. As such, it
is expected that commendations will have a positive effect on AC scores while sustained
complaints should have the opposite effect. The number of times tested is regressed on
AC score because literature suggests that one is capable of learning how to do the
assessment center (Kroecker, 2000; Rockhill and Halton, Inc, 2007). This learning takes
place via feedback from prior AC as well as training classes prior to the AC. Each time a
person participates in the AC process he/ she learns how to better himself/herself and
prepare for the next promotional process in which they participate. The number of times
one takes test is expected to improve promotional odds.
Table 1: Summary of Expected Direction of Influence on Assessment Center Scores
Variable Direction of Expected RelationshipWritten Score + Education + Female + Minority + Age in Years - Commendations + Sustained Complaints - Times Tested +
The second set of hypotheses concerns the weight given to the AC score in the
final score, this research hypothesizes that attaining a higher AC score will better predict
who is to be promoted. Furthermore, written score should also report as significant and
in the positive direction. All other variables shown in Table 2 are expected to have
similar predictability in strength and direction on promotions as they had on high
assessment center score. Because the AC score is weighted so heavily in the final score
outcome, anything affecting the AC score should similarly predict promotion. That being
said, it is believed that the AC score, following prior research, will be the largest
contributing factor in the promotion decisions at LMTPD.
Table 2: Summary of Expected Direction of Influence on Promotion
Variable Direction of Expected RelationshipAC Score +Written Exam Score + Education + Female + Minority + Age in Years - Commendations + Sustained Complaints - Times Tested +
CHAPTER THREE
DATA AND METHODS
DATA
The data were collected from a series of open record requests for information.
The information collected was all deemed to be theoretically relevant to the study at
hand. It should be noted that the dataset is a population rather than a sample. That is, the
individuals that are represented in the data include every person that participated in at
least one (or as many as all three) of the promotional process examinations. Data
included information about the sergeants’ demographics, their work records, and their
testing records. (See Table 3 below for a synopsis of the descriptive statistics of all
variables.)
Table 3: Descriptive Statistics of Lieutenant Promotional Population Data
Variable NameObservatio
ns Mean Std. Dev. Minimum MaximumPromoted 202 0.3465 0.4770 0.00 1.00High Written Score 202 73.3069 8.0047 57.00 97.00High AC Score 202 7.3037 0.9446 4.52 9.36Education 202 0.7277 0.4462 0.00 1.00Female 202 0.1733 0.3794 0.00 1.00Minority 202 0.4703 0.5004 0.00 1.00Commendations 202 55.2079 24.6132 12.00 151.00Sustained Complaints 202 0.8119 1.4573 0.00 7.00Times Tested 202 1.3218 0.5467 1.00 3.00Age of Sergeant 202 41.3502 5.3048 30.20 58.94
Source: Open Records request via Public information Act (Texas Open Records Act), Chapter 552, Texas Government Code (2009).
The present study is an amalgamation of testing procedures, formulas, and
individual level data from the entire population of 202 police sergeants who tested for the
rank of lieutenant. The data presented in this research is collected from three temporally
separated lists of promotional prospects. Promotional processes represented in this study
are dated by the date of the written test. Those dates are November 1, 2001; October, 20,
2005, and October 30, 2007 tests and naturally provide excellent quantitative data. The
written test date is chosen as the beginning of the promotional testing course because
after the process is complete and a list of candidates provided, actual promotion dates
vary greatly. For instance, a list of the numerically ranked candidates is given to the
Chief of Police. He may promote the first 2 one month after the final list, the next 5 ten
months after, etc. The actual promotion dates may vary greatly even within the same list
of candidates. However, the written test date remains the same for everyone on the list.
Dependent Variables
Two statistical models were completed during the analyses, and thus this research
had two dependent variables. The first dependent variable is the AC Score, which is
measured as the value assigned to the promotional candidate by the Assessment Center.
The second dependent variable was promotion. This was measured simply as whether or
not the Sergeant was promoted to Lieutenant after completion of the testing process.
Promoted is coded simply as a dichotomous variable (0=Not Promoted and 1=Promoted).
Independent Variables
Several variables were used to examine the various influences on both of the final
outcome measures of this research study. First, the written test is a test where the
individual can score up to 100 point test. The data from this population shows that actual
scores ranged from 57 to 97 and had a mean score of 73.31. In order to qualify for the
assessment center, the candidate must score higher than a set minimum score. The
qualifying minimum scores were different for each test and were not announced until all
scores have been tallied; only those with the higher scores are allowed to proceed in the
testing process7. Assessment center scores were weighted and then transformed into
scores that could range from zero to ten. The population showed a mean score of 7.30 and
a range of 4.52 to 9.36. All three AC scores were transformed to be equivalent measure
for the performance of the analyses in this research paper (See Appendix 1 for the
formulas used to make this transformation).
Demographic variables were also included about each individual officer.
Education was coded as a dummy variable whether a candidate completed some college
hours (1) or had not taken any college credit hours (0). Race was coded as a dummy
variable with the candidates that self identify as having minority status coded as minority
(1) and those who were Caucasian coded as 0. The age of the candidate at the time of
participation in the promotional process is operationalized as age in years. While age is
not always directly linked to tenure on the police force, it has been used as a proxy for the
length of tenure at LTMPD. The argument espoused by the literature suggests that age
may be a determining factor in assessment centers and promotion. In other words, a
younger candidate who is less than thirty years old will not be expected to have as many
years of tenure as a 50 year old candidate. Gender was also tested as a demographic
predictor. For the purposes of this research, female was coded as (1) and male (0).
Commendations and sustained complaints are used as measures of promotional
candidate past behavior. Commendations are arguably the results of good behavior being
rewarded and have a mean of 55.21 with a range from 12 commendations to 151.
Conversely, complaints are the results of wrongful behavior and occur after an Internal
Affairs investigation where the officer is deemed to have violated a rule. Also, by using
sustained complaints, only the most serious violations are recorded. While the actual
discipline received from a sustained complaint can range from termination to verbal
counseling, internal affairs investigates grievous acts; often sending less serious matters
back to the officer’s chain of command for a more informal review. The data indicate
that some candidates had as many as seven sustained complaints or as few as zero with a
mean of .81.
The times tested variable (coded as the number of times tested during the last 3
testing opportunities) represents the number of times a person participated in the
promotional process. This variable is included to take into account the possibility of a
learning effect as suggested by the literature. Only eight the candidates took the test each
time it was offered (total three). Interestingly, no one who participated in all three
assessment centers has been promoted. Furthermore, only nine of the forty-nine who
took the test two times were promoted. The remaining sixty-one were promoted the first
time they participated in the promotional process.
As this research is done on the amalgamation of three testing procedures, another
variable that is slightly altered (or dependent upon time tested, etc.) is tenure. (See Figure
1 for an illustration of tenure and its relationship to promotion for each of the three
testing scenarios.
Figure 1: Tenure of Promotional Candidates
Source: Open Records request via Public information Act (Texas Open Records Act), Chapter 552, Texas Government Code (2009).
Surprisingly, there is a dearth of literature explicating the effect of tenure alone. It is
provide here in order to show that there is very little difference in the average amount of
tenure each candidate has at the sergeant rank among the promoted and not promoted.
Within this police department, this is not an unexpected consequence. The time spent as
a sergeant (the rank just below my analysis rank) is expected to have little effect on the
actual promotion because any person holding their rank for at least 6 months can enter
the promotional process for the next rank. Figure 1 represents the average tenure in
months for each test and then all tests combined: 2001 test average tenure for promoted
was 71 months, not promoted 72 months; 2005 test average tenure for promoted was 79
months, not promoted 80 months, and the 2007 test average tenure for promoted was 70
months, not promoted 67 months. The total average tenure for all three tests combined
show that overall those who were promoted had 74 months of tenure compared to an
0102030405060708090
Time in Months
2001 2005 2007 All Years
Test Year
Tenure as an LTMPD Sergeant
Promoted
Not Promoted
average 71 months for those not promoted.
LIMITATIONS
One of the main limitations of this analysis is inherent of any study revolving
around one particular group of people; limited external validity. This analysis applies to
one police department in which individuals were testing for one specific rank. Increasing
this test to analyze the police candidates testing for the rank sergeant or to data outside of
the LTMPD may provide differential findings. Furthermore, this is a self selected
population and inferences concerning subordinate officers as well as those who chose not
to participate in the process cannot accurately be made.
Another limitation is the restricted number of observations from which data were
collected. Because of the limited number of minority candidates, it was not possible to
break the racial component down any further. The merging of all non-white officers into
the minority variable, for instance, was needed in order to keep the model stable.
Analysis of a larger group such as the lieutenants and sergeants may result in the ability
to tease out single race effects. It should be noted here that there was one observation
dropped from the 202 officers represented in this analysis. One person from the first
testing group retired during the process and only incomplete data were available. The
impact from dropping this person is minimal because it was a white male and, therefore a
member of two of the largest groups represented in this study; white and male.
Lastly, past behavior was represented by sustained complaints and
commendations. Proxies for past behavior are problematic because individuals could
have received a generous portion of commendations of sustained complaints AFTER
being promoted. The temporal aspect of the past behavior was not entirely gleaned from
the data. Performance reviews are available for consideration however there are several
notable problems with doing so. First, supervisors must do little documentation or a
“Meets Standards” or “Exceeds Standards”. However, for a rating of “Does Not Meet
Standards” a considerable amount of documentation must be done. Secondly, this rating
can be appealed should the person being rated feel the need to do so. Therefore, it is
profoundly unlikely that supervisors created more work for themselves when a passing
performance score could be more easily given. Secondly, a considerable amount of time
would have to be dedicated to reading each three or four page review for all 202
candidates and this is an overwhelming time constraint for this study. What is more,
when coupled with the likely invariability of the data for reasons stated above, a lengthy
review of performance plans would have likely been unproductive.
CHAPTER FOUR
ANALYSES AND RESULTS
Stata (Statacorp, 2007) was used to perform two sets of analyses in this research
project. First, using Ordinary Least Squares (OLS) regression, this research paper
examined the effects of several important variables on High Assessment Center (AC)
Score. These variables included the following: written test results, work history
indicators, and demographics. The OLS analyses were used to test hypotheses regarding
which attributes were significantly related to candidates receiving higher overall AC
scores.
In Table 4, high AC score was the dependant variable. In this model, written high
score (β = .02) was significant at P < 0 .013. Age had a larger coefficient than written
high score (β= -.05), and was significant at P > 0.00. The commendations variable (β
= .01) was also significant, P < 0.04. The OLS regression model was able to predict 17%
of the variance found in the dependant variable which is very modest.
Table 4: Regression Model of Predictors for a High Assessment Center Score (N=202)
High Assessment Center Score CoefficientsStd. Err. t P>[t]
Written High Score 0.020 0.010 2.490 0.013**Education 0.070 0.140 0.500 0.620Female 0.120 316.000 0.730 0.470Minority -0.210 0.130 -1.590 0.110Age in Years -0.050 0.010 -4.270 0.00***Commendations 0.010 0.002 2.060 0.04**Sustained Complaints -0.040 0.040 -0.870 0.390Times Tested -0.110 0.120 -0.910 0.370Constant 7.810 0.870 9.000 0.00***
Prob > F = 0.00, R2 = .17. Significance P < .05**, P < .01***
Source: Open Records request via Public information Act (Texas Open Records Act), Chapter 552, Texas Government Code (2009).
Despite the literature, education, gender, and minority status failed to rise to a level of
significance and did not appear to be related to a high AC score.
Diagnostics of the OLS model run for this analysis showed the data to be
normally distributed. A variance inflation factor analysis assessed multicollinearity. The
mean VIF was 1.09 and this did not present a problem (Acock, 2008). Stepwise
regressions were performed and the model was robust.
Table 5 reports the logistic regression model utilized the same variables as in
Table 4. Promoted was the dependent variable. The AC coefficient is both positive (β =
1.04) and highly significant (p < 0.00). According to Acock (2008) this can be
interpreted as a 4 percent increase for every one-hundredth of a point, or a 48 percent
increase in odds of promotion with every tenth of a point improvement in score. The
odds ratio for written high score was 1.31 and it was significant indicating that for every
point higher in the written score, a candidate’s odds of being promoted increased 31
percent. This is not surprising and is to be expected given the equation used for the final
score: Written Score (40%) + Assessment Score (60%) = Final Score.
Table 5: Logistic Regression model of Predictors of Promotion (N = 202)
PROMOTED Odds Ratio
Std. Err.
Z P>[Z] [95% Conf.
Interval]
High Assessment Center Score
1.04 0.01 5.51 0.00***
1.03 1.06
Written High Score 1.31 0.08 4.42 0.00***
1.16 1.48
Education 0.47 0.34 -1.05 0.29 0.12 1.91Female 0.78 0.69 -0.28 0.79 0.14 4.45Minority 0.78 0.52 -0.38 0.71 0.21 2.92Age in Years 1.06 0.07 0.82 0.41 0.93 1.21Commendations 0.99 ..01 -0.91 0.36 0.96 1.01Sustained Complaints 0.3 0.13 -2.74 0.01**
*0.12 0.71
Times Tested 0.03 0.03 -4.25 0.00***
0.01 0.16
Significance P < .05**, P < .01***.
Source: Open Records request via Public information Act (Texas Open Records Act), Chapter 552, Texas Government Code (2009).
A diagnostic test was done on the logistic model by testing the area under the receiver
operating characteristic (LROC). For this model, LROC = .98 which suggests a stable
model.
The number of commendations that the promotional participant receives is
significant (p < 0.04) and positive in the AC model. These findings do not represent any
previous research findings on past behavior effects on AC’s and may warrant further
research. Additionally, the results on age provide proof of its effect on the AC rating.
Age was negative and significant (-0.05; p < 0.000) and demonstrates a candidate’s age is
inversely related to obtaining a high AC score.
Concerning past behavior, the odds ratio for sustained complaints is 0.30 and it is
significant. This result suggests that for every sustained complaint a candidate’s odds of
being promoted decrease by 70 percent. The number of times tested had a similar inverse
effect on the odds of promotion. Evidence provided by the promotion mode
demonstrated the odds of being promoted decrease by 97 percent with each additional
test taken.
For the purpose of testing both research questions (AC Score and Promotability),
the variable for candidate’s status as a minority was a dichotomized. Minority (non-
white) was coded as 1 and non-minority equaled 0. This was done to ensure a proper
number of observations in order to keep the models stable. However, Pearson chi-
squared tests were performed to test the relationship between and individual race
characteristics and promotion. In these tests, being Black was the only significant race
relationship, P < 0.02. All other races proved to have no statistically significant
relationship to promotion. Chi-squared tests showed that minority status and promotion
were significantly related at the .05 level (see Table 6).
Table 6: Chi-Squared Tests of Individual Race Characteristics and Minority Status
Chi-Squar
e
d.f. Prob.
Black 5.285 1 0.022
Hispanic 0.337 1 0.562
Asian 0.424 1 0.052
Native American
0.210 1 0.647
Minority 4.203 1 .040
CHAPTER FIVE
DISCUSSION AND CONCLUSION
Discussion
There are several key findings which can be gleaned from the data analysis. It
was expected that AC score would be a strong predictor of promotion; there is a 4 percent
increase in odds of being promoted with every one-hundredth of a point increase in score.
Likewise, it is not surprising that the written high score variable in the promotion model
reported a 31 percent increase in the odds of promotion. However, a 31 percent increase
in odds accounts for a whole point difference; a mere tenth of a point increase in AC
score raises the odds of promotion by 48 percent. Thus, AC score increases the odds of
promotion to a much greater extent than does the written score.
Written high score is statistically significant in predicting promotional outcome;
however, the coefficient was rather small. This suggests that even though written test
scores account for 40 percent of the overall grade given at the end of the promotional
testing process, it is the assessment process that truly influences how promotional
decisions are made. In short, doing very well on the written portion of the test makes no
“real” impact on the overall final score. Written score is merely a qualifier to move onto
the AC portion of the testing process.
The independent variables of minority status and being female failed to achieve
significance in either model. This actually indicates that the department as a whole and
possibly the AC specifically uses determinants for promotion other than race and gender.
This is often a concern with any organization; a person may be promoted because of the
“good ole boy system” or because the person was a minority or female (i.e. through
affirmative action initiatives). By controlling for these independent variables, it was
shown that race and gender have no significant influence on promotability. This suggests
to some extent that the LTMPD promotions are blind to one’s physical characteristics as
organizational theory suggests it should be.
The age of the individual is negative and significant in the OLS model but not in
the model testing its relationship to being promoted. This provides evidence of the
argument that younger promotional candidates are being sought (Adams, 2002), and
lends proof to the argument that the AC scores younger candidates higher than they do
the more seasoned veterans. Younger candidates are being scored higher than older
candidates and tenure appears show almost no differentiation between promoted and non-
promoted. This suggests there are other reasons for younger officers being promoted
other than the possibility of having more experience in the rank. This is an important
finding and should be investigated further.
Past behavior variables which were operationalized as the number of
commendations and the number of sustained complaints offered some insight.
Commendations were significantly related to high AC score but not to being promoted.
In contrast, sustained complaints did not prove significant in the OLS model but it was
significant and inversely related to being promoted. This insight is somewhat constrained
because both are subjectively applied to the officer by their chain of command. How to
interpret this remains unclear but provides some groundwork for future research on the
effects of past behavior on AC’s and promotion.
Times tested is not significant in the OLS model, however the negative coefficient
would suggest that the outcomes leans towards the notion that with every test taken the
odds of being promoted are significantly lowered. The number of times tested becomes
significant in model testing the dependant variable promoted, and indicates that repeated
participation in the entire promotional process diminishes the candidate’s odds of
promotion vastly. Additionally, the logistic regression model indicates that education
still has no significance which provides robust proof that one’s educational level plays no
role in promotability. In other words, the non-significance of the coefficient for
education in both models indicates that for the LTMPD, education is not a valued
characteristic and has no predictive effect on one’s promotion.
There is a caveat to the gender questions which is implicit in the findings.
Females comprise 18.57% of the 70 lieutenants that have been promoted over the
course three promotional processes analyzed in thesis. They make up 18.81% of
the 101 lieutenants on the LTMPD. The data suggest females are evenly
represented in promotions and are not advanced in rank at a rate greater than their
male counterparts. Again, this may suggest that the LTMPD is gender neutral
with respect to promotions. However, the data can also be interpreted as the
female promotion rates being purposefully kept within a specific percentage.
Having some college education has no predictive effect on one’s AC score
or promotion. Furthermore, the level of advanced education either does not help
one to obtain a high AC score. Additionally, the number of times tested appears
to hurt one’s chances of promotion in the logistic regression model but has no
effect on a high AC score in the OLS model. This would suggest that feedback
from prior tests, contrary to the literature (Kroecker, 2000), has little or no impact
on subsequent AC outcomes and numerous attempts at getting promoted
significantly decrease the odds of it occurring. Furthermore, there seems to be no
difference in the amount of tenure among those promoted and those not promoted.
Conclusion
The results from this study offer other intriguing insights for answers to the
question of what it takes to be promoted. Currently, there are just over 100 officers at the
rank of lieutenant within this police department, including the 70 that have been
promoted since November 2001. It is one of the top ten largest metropolitan police
departments in the United States (Hickman and Reaves, 2006); serving more than
1,000,000 people and seemingly has a large turnover of its management level officers. In
a metropolitan police department of this size and given the importance of this upper-
management position, there can be little argument against efficient, thorough and,
repeated testing of the effectiveness and efficiency of the promotional processes.
This current analysis of the testing procedures in the LTMPD answers the basic
question; what does it take to get promoted? It has been empirically shown that doing
well in the AC strongly affects final scores. Because of the strength of the AC score on
final score, AC high score predictors become highly important.
The significance in the findings provided by the AC model are vividly apparent
when one recognizes there are no known validity tests of the AC process done by the
LTMPD. Experientially known, the AC proctors go to great lengths to ensure the
candidates “stay in character” when addressing the assessors suggests that role playing is
a significant part of the process. It would appear that the ability to play the role is the
most important aspect of the AC, a point well made by Lance (2008).
An aggregate view of the data strongly suggests this department lends little or no
value to maturity (age), experience (tenure), education (college hours), or perseverance
(times tested). This is astounding considering that the rank of lieutenant is crucial to the
successful relationship between the organization and its personnel as well as the
organization and the society it serves. However, this research paper does not argue for
the abandonment of the AC process. Indeed, continuing to utilize the AC would be
advantageous. Using the AC as one of many tools in the promotional toolkit and would
certainly be useful for determining performance and capabilities of the promotional
candidate. Nevertheless, holding out AC rating as the best, stand alone indicator of
future performance is highly unrealistic.
Considering the importance of organizational theory of promoting the best to
guarantee organizational success and the theory of police behavior offered by Klinger
(1997; 2001), promotion of those who are most likely to be successful is absolutely
necessary. That is not to say those currently in leadership positions on the LTMPD are
not the best and brightest; the fact is there is no way to know at this point. However, we
now have a clearer understanding of the promotional process and what it takes to get
promoted in the LTMPD. Predictors of promotion have been empirically tested and lead
to a question ripe for further analysis. Explicitly, do those skills, behaviors, or
characteristics that it takes to get promoted on the LTMPD positively correlate with what
it takes to be a successful leader? Tacitly, this current research suggests promotions are
serendipitous and not related to true ability beyond one’s stellar performance at the AC,
as well as having the good fortune to have lenient or favorably assessors.
American police agencies are paramilitary organizations (Jermier and Berkes,
1979) in which the rank of lieutenant is a position of leadership. Several predictors of
promotion were uncovered and provide enlightenment into the behavior and structure of
this police organization. Through the data analysis, we see that the command structure is
driven almost entirely by AC score. Simply stated, a high AC score significantly raises
the odds of being promoted to lieutenant. The LTMPD’s chief level officers are chosen
from the lieutenant ranks. Because of this, the variables affecting the AC score become
the most important when discussing organizational behavior and its effect on police
personnel as well as on society.
In light of the importance of promotion to the lieutenant rank and the financial
cost of using the AC, the implications of the findings is glaring. Fervidly suggested by
the current research is that organizations utilizing and heavily relying on AC’s better be
absolutely convinced of their ability to predict future leadership behavior. It will be those
AC scores that greatly influences the list of who is worthy of promotion. From that list,
leaders will be chosen and they will contribute substantially to the societal and
organizational success (or failure) of their police department.
ENDNOTES
1. All data presented for written and assessment center test scores, assessment center attendance dates and overall scores as well as all candidate demographic information were obtained by way of open records request sent in accordance to the Public information Act (Texas Open Records Act), Chapter 552, Texas Government Code. This law was extensively used and is imperative in order to legally and ethically acquire the reported data.
2. The combined cost to the LTMPD for all three assessment centers was in excess of $100,000.
3. There is an argument that Assessment Centers may indeed be able to accurately predict successful leadership behavior, and thus are valuable tools in the promotional procedures of police departments (Ross, 1980; Kroecker, 2000).
4. The tactical exercise is one that includes a scenario in which the candidate would be expected to apply his/her knowledge of street tactics such as crime scene protection, mass arrest or large accident scene. The oral presentation is one in which the candidate’s scenario would be a situation in which the candidate would have to act as if they are giving a speech to the city council, command staff, etc. An in-basket exercise simulates a full in-basket in which there are multiple scenarios ranging from personnel issues to complaints against officers or crime reduction operations. The candidate is expected to address and prioritize each incident.
5. Kroecker’s article appeared in The Police Chief and was written for chiefs of police who would be assumed to have been benefactors of the AC and therefore more inclined to promote its use.
6. The LTMPD hires people as young as 19 ½ years old with the condition that they have at least 60 hours of credit at an accredited college. Once hired a recruit must endure a 32-week academy training program followed by 24 weeks of field training. After field training, the recruit is in probationary status for another six months. Therefore, a person would be 21 before being a full, non-probationary police officer.
7. All candidates with a written score at or above the cut off score was allowed to attend the assessment center for that particular test:
a. Test 1cut off: score of 61.
b. Test 2 cut off: score of 57.
c. Test 3 cut off: score of 62.
APPENDIX
The formulas used by the LTMPD to determine the candidates’ positions on the
final list are as follows:
Test 1- November 1, 2001 Statistical Formulas
Written score: (((Raw Written Score-74.344)/7.524) x 5)+85)=STANDARIZED WRITTEN SCORE
Assessment score: (((Raw AC Score-7.552)/.721) x 5)+85)=STANDARIZED AC SCORE
Final Score: (Stnd. Written x .40)=( Stnd. AC x .60)=FINAL SCORE
Test 2- October 20, 2005 Statistical Formulas
Written score: (((Raw Written Score-67.33)/7.068) x 5)+85)=STANDARIZED WRITTEN SCORE
Assessment score: (((Raw AC Score-3.409)/.0561) x 5)+85)=STANDARIZED AC SCORE
Final Score: (Stnd. Written x .40)=( Stnd. AC x .60)=FINAL SCORE
Test 3-October 30, 2007 Statistical Formulas
Written score: (((Raw Written Score-73.705)/7.826) x 5)+85)=STANDARIZED WRITTEN SCORE
Assessment score: (((Raw AC Score-3.445)/.447) x 5)+85)=STANDARIZED AC SCORE
Final Score: (Stnd. Written x .40)=(Stnd. AC x .60)=FINAL SCORE
As can be seen here, the denominator used in the written score and assessment
center score calculations vary greatly. The basic sequence is the raw score-average and
the result is divided by an unknown number. The product is multiplied by 5 and added to
85. This researcher was unable to find the reason for the disparity however one scoring
model remains consistent. The final score calculations are 40% written plus 60%
assessment center. It is easy to see here why the assessment center has such a large effect
on the final outcome. Not only is it afforded a larger percentage of the overall score, it is
a much smaller number in the standardized formula compared to the written score. Any
variation would have a much greater influence in the final outcome. As a result of those
final outcomes, a hierarchical list is produced with the top score as number one, the next
score as number two and so forth.
Top Related