Post on 17-Sep-2018
The Determinants and Performance Effects of Supervisor Bias
Jasmijn C. Bol
University of Illinois at Urbana-Champaign
bol@uiuc.edu
January 2008
Abstract
This paper examines the determinants and performance effects of leniency and centrality bias.
An empirical analysis of a compensation plan for low-level employees with both objective and
subjective performance measures leads to two key results. First, the causes of supervisor bias
include: employee performance, the difference between the organizational level of the supervisor
and the employee, the financial position of the firm, the duration of the employee-supervisor
relationship, and supervisor characteristics. This indicates that supervisors take their own
personal preferences into account when appraising employee performance. Second, supervisor
bias affects future employee incentives. Contrary to previous assumptions, the results show that
biased performance ratings can have both positive and negative effects on incentives. Leniency
bias positively affects performance improvement, while centrality bias has a negative effect on
performance.
Acknowledgements: I want to thank Stan Baiman, Antonio Dávila, Henri Dekker, Chris Ittner,
Frank Moers, Fernando Peñalva, Joan Enric Ricart, Stan Veuger and seminar participants at the
2006 Global Management Accounting Research Symposium in Copenhagen, the 2006 AAA
Annual Meeting in Washington D.C., the University of Pennsylvania, the London School of
Economics, the University of Colorado at Boulder, the University of Illinois at Urbana-
Champaign, the London Business School, the Harvard Business School and the University of
Southern California for their comments and suggestions.
1
I. INTRODUCTION
This paper provides empirical evidence on the determinants and performance effects of
supervisor bias. Knowledge of supervisor bias is important for managers in charge of incentive
design. Understanding the causes of supervisor bias, and the consequences of supervisor bias on
future performance, can help compensation system designers make better-informed decisions
about how much supervisor discretion to allow for in performance-based compensation plans,
thereby improving incentive contracting.
In this paper, I focus on two well-known forms of supervisor bias: leniency bias and
centrality bias (Prendergast, 1999). Leniency bias is the tendency to provide employees with
inflated subjective performance ratings (Bretz, Milkovich, & Read, 1992), while centrality bias is
the tendency to compress performance ratings, creating less variance in performance ratings than
in actual performance. Although there is considerable empirical evidence indicating that the use
of subjective performance measures leads to lenient and compressed ratings (Landy & Farr,
1980; Murphy & Cleveland, 1991; Bretz et al., 1992; Prendergast & Topel, 1993; Jawahar &
Williams, 1997; Prendergast, 1999; Moers, 2005), empirical evidence on how supervisor bias
actually influences employee behavior is almost entirely absent. This paper extends the current
literature by focusing both on what causes a supervisor to bias performance ratings and on how
biased ratings affect the incentive effects of performance-based compensation contracts.
I study a compensation plan for low-level employees of a financial service provider. The
incentive system includes both objective and subjective performance measures and gives
supervisors some freedom to make ex post modifications to financial rewards. I use contract
design and performance data from five branch offices for 2003 and 2004. I begin my analysis by
establishing that the performance ratings on the subjective dimension are subject to leniency and
2
centrality bias. I then examine why the performance ratings of some employees are subject to
more leniency and centrality bias than those of others. Finally, I examine how these biases affect
performance change between 2003 and 2004.
The results indicate that supervisors do not behave as pure contract-driven robots when
appraising performance. I find that differences in organizational level between employee and
supervisor, and the financial performance of the company, are determinants of leniency and
centrality bias. This is consistent with supervisors taking information-gathering costs and
attitudes towards rating accuracy into consideration when rating behavior. Supervisor
characteristics, the duration of the employee-supervisor relationship, and employee performance
influence the extent of leniency bias, suggesting that supervisors consider the cost of
communicating evaluations and how employee ratings reflect on their own management
capabilities.
Finally, performance ratings influence incentive provision. Leniency bias has a positive
effect on performance improvement, while centrality bias has a negative effect, for both above-
average and below-average performers.
The study makes several important contributions to the performance evaluation and
compensation literature. This is the first paper that empirically investigates how biased
performance ratings influence the effectiveness of a performance-based compensation contract.
Prior studies have established the existence of leniency and centrality bias, but not the
consequences of these biases on future performance. Moreover, this paper does not only present
evidence indicating that the use of subjective performance measures leads to biased performance
ratings; it also provides insights into the determinants of supervisor bias. Finally, this paper
offers a detailed description of the salient features of an incentive system for low-level
3
employees and of the supervisors’ and employees’ reactions to this system. This gives us insight
into the compensation practice for low-level employees, an area in which detailed information is
scarce (Indjejikian, 1999).
This paper consists of five sections. Section II reviews research on supervisor bias in
compensation contracts and develops the hypotheses. The research design is presented in section
III and in section IV the results are analyzed. The final section summarizes the results and
discusses future research possibilities.
II. HYPOTHESES
Most papers dealing with subjectivity in compensation contracts focus on the
determinants of subjectivity. These studies examine the role of subjectivity in incentive systems
and indicate how introducing subjectivity improves contracts by mitigating incentive distortions
or reducing risk (Baker, Gibbons, & Murphy, 1994; Baiman & Rajan, 1995; Hayes & Schaefer,
2000; Ittner, Larcker, & Meyer, 2003; Gibbs, Merchant, Van der Stede, & Vargus, 2004).
However, the fact that performance evaluation is subject to supervisor discretion can also give
rise to a number of problems, the most prominent of which is rating inaccuracy.1
Since the correctness of subjective performance evaluations cannot be assessed by
outside parties, supervisors can take their own preferences into consideration when rating
employee performance. One documented consequence of allowing supervisors to apply
discretion is biased performance ratings (Landy & Farr, 1980). A long line of research, mainly in
human resources management, investigates performance rating accuracy in subjective
1 Inaccurate performance ratings are clearly not the only potential negative consequence of introducing subjectivity
into explicit compensation contracts. Other concerns, such as unclear measurements criteria, are also likely to
influence the effectiveness of the compensation contract. In this paper, I limit the discussion to examining the effects
of biased performance ratings.
4
performance appraisal and presents considerable evidence on biased performance ratings (e.g.,
Murphy & Cleveland, 1991; Bretz et al., 1992; Jawahar & Williams, 1997; Moers, 2005). These
papers find that supervisors tend to rate leniently and that they do not differentiate strongly
between employees when evaluating performance.
The extent to which ratings are biased is not the same for different supervisors or for
different employees evaluated by the same supervisor. Supervisors are predicted to vary the
amount of bias applied to the performance ratings depending on how rating bias influences their
own utility.2 In determining the extent of bias, supervisors consider the potential negative
consequences of communicating performance ratings. Communicating harsh but accurate ratings
to employees will likely damage personal relationships and lead to discussions and criticism. By
offering employees lenient and compressed ratings, supervisors can reduce the real and
psychological cost of communicating evaluations (McGregor, 1957; Bernardin & Buckley,
1981). This leads to the following hypotheses:
Hypothesis 1a: The costs of communicating evaluations have a positive effect on the extent of
leniency bias that is applied to performance ratings.
Hypothesis 2a: The costs of communicating evaluations have a positive effect on the extent of
centrality bias that is applied to performance ratings.
The extent of bias that supervisors apply to performance ratings is also influenced by the
time and effort needed to gather the appropriate performance information. High information-
gathering costs will make supervisors less willing to invest the required time in information
collection (Harris, 1994). This is predicted to lead to leniency and centrality bias, because
2 In this paper I focus on incentive-consistent bias by supervisors. However, supervisors can also unconsciously bias
performance ratings due to their cognitive limitations. See, for example, Lipe and Salterio (2000) and O’Connor,
Deng and Shields (2006).
5
biasing imprecise performance evaluations reduces the chance that resulting imprecise ratings
lead to painful discussions and dissatisfied employees. This leads to the following hypotheses:
Hypothesis 1b: Information-gathering costs have a positive effect on the extent of leniency bias
that is applied to performance ratings.
Hypothesis 2b: Information-gathering costs have a positive effect on the extent of centrality
bias that is applied to performance ratings.
Since supervisors’ compensation and promotion possibilities are often linked to employee
performance, supervisors are also expected to take the effects of their rating decisions on
employee incentives into account (Prendergast & Topel, 1993), as well as how the performance
ratings reflect on the supervisors’ own management capabilities (Longenecker, Sims, & Gioia,
1987). By rating poorly performing employees more leniently, supervisors may try to hide
departmental problems in order to come across as more capable managers.
Hypothesis 1c: Employee performance has a negative effect on the extent of leniency bias that
is applied to performance ratings.
Supervisors are also predicted to take their superiors’ attitude towards rating accuracy
into account. Supervisors adjust their biasing behavior based on the expected rewards for
accurate ratings and/or the expected consequences of being perceived as a biased rater by their
own superiors (Longenecker et al., 1987). This results in the following hypotheses:
Hypothesis 1d: A positive attitude of the supervisor’s superiors towards rating accuracy has a
negative effect on the extent of leniency bias that is applied to performance
ratings.
Hypothesis 2d: A positive attitude of the supervisor’s superiors towards rating accuracy has a
negative effect on the extent of centrality bias that is applied to performance
ratings.
6
Finally, rating behavior is influenced by the utility supervisors get out of (dis)favoring
employees (Prendergast & Topel, 1996). Favoring and disfavoring certain employees leads to
biased ratings, although favoritism does not necessarily lead to leniency and centrality bias.
Employee Incentives and Supervisor Bias
Performance ratings are an important element of incentive contracts because they
summarize the outcome of the performance evaluation process and link performance to pay by
indicating how much compensation the employee will receive. Bias in performance ratings is
likely to influence the incentives created by the compensation contract. In this paper I identify
three ways in which bias in performance ratings is expected to influence employee incentives: 1)
through the effect bias has on the link between pay and performance, 2) through the effect bias
has on the perceived fairness of the compensation system, and 3) through the effect bias has on
the performance information received by the employees.
The Link between Pay and Performance
One of the main objectives of using a performance-based compensation system is
motivating employees to supply effort on the right job dimensions.3 According to agency theory,
linking pay to performance motivates employees to exert increased effort to improve
performance, because increased performance results in increased pay (Holmstrom, 1979).
Performance measures play a crucial role in this process because employees direct their attention
to those actions that are measured.
However, the incentive effect of a compensation plan does not depend solely on what is
measured; it also depends on how these actions are measured. Employees will not be motivated
3 Another important objective of performance measurement is to differentiate between highly skilled and less skilled
employees. Although the effect of less than optimal personnel decisions might be severe, the aim of this paper is not
to investigate the effect of bias on selection issues.
7
to increase effort unless improved performance is actually expected to translate into more
compensation.4 When performance is assessed subjectively, this may not take place consistently.
Bias clouds the link between pay and performance, affecting the incentive provision of the
compensation plan.
The Perceived Fairness of the Compensation System
Employee incentives are not solely determined by the relation between pay and
performance. The extent to which employees are motivated by a compensation system is also
influenced by the perceived fairness of the compensation plan (Akerlof & Yellen, 1988; Blinder
& Choi, 1990; Colquitt, Conlon, Wesson, Porter, & Yee Ng, 2001). Employees not only care
about how their rating compares to their performance, but also about how the received
performance rating compares to their expectations and to the ratings received by others. In the
organizational justice literature, two types of subjective perceptions of fairness are distinguished:
the fairness of the outcome distributions, or distributive justice, and the fairness of the
procedures used to determine these outcome distributions, or procedural justice (Greenberg,
1990). Inaccuracies in performance ratings caused by supervisor bias are predicted to influence
the perceived fairness of the compensation system as bias changes the outcome distribution of
the compensation plan.
Performance Information
Compensation systems also influence employee incentives through their effect on self-
perception. Performance ratings provide employees with information about how their
performance is perceived by the supervisor. This information influences employee incentives by
affecting the employee’s self-perceived marginal productivity of effort (Fang & Moscarini,
4 In expectancy theory (Vroom, 1964; Heneman & Schwab, 1972) this is referred to as the degree to which
performance is instrumental for the attainment of certain outcomes, and research shows that it is important in
motivating individuals.
8
2002). Higher self-confidence enhances motivation by affecting the expected result of effort,
which provides supervisors with clear incentives to provide employees with performance ratings
that increase or maintain their self-esteem (Bénabou & Tirole, 2002; 2003).
The Performance Effects of Leniency and Centrality Bias
Supervisor bias is studied extensively in the HR literature in the hope of finding ways to
reduce it, under the maintained assumption that inaccurate ratings per se are bad for
compensation contracting (see Landy & Farr, 1980; Rynes, Gerhart, & Parks, 2005). In this
paper, I argue that this assumption may be incorrect. There are several ways in which bias is
expected to influence the functioning of performance-based compensation contracts and they are
not all negative. In the following sections I discuss how the two biases considered in this paper,
leniency and centrality bias, are predicted to influence employee incentives.
Leniency Bias
Higher ratings can affect incentive provision positively by increasing congruence
between performance rating expectations and received performance ratings. Individuals have a
tendency to overestimate themselves5 and therefore to rate themselves higher than their
supervisors do (McFarlane Shore & Thornton, 1986; Harris & Schaubroeck, 1988). Employees
who believe they have received a lower performance rating (and consequently less
compensation) than they deserve, are expected to lower their performance in order to restore a
feeling of equity (Akerlof & Yellen, 1988; Colquitt et al., 2001). More lenient performance
5 There exists abundant evidence in the psychology literature that most people overestimate their abilities and past
achievements (e.g. Larwood & Whittaker, 1977; Arkin, Cooper, & Kolditz, 1980), that they tend to recall their
success more than their failures (e.g. Mischel, Ebbesen, & Zeiss, 1976), and that they have the tendency to be
unrealistically optimistic about their future life events (e.g. Weinstein, 1980).
9
ratings may minimize these feelings of unfairness and the resulting negative effects on
performance.
Moreover, more lenient performance ratings can maintain or increase the employee’s
confidence in his ability and efficacy, which can be valuable to the firm because it increases the
employee’s motivation to undertake ambitious projects and persist even when faced with
adversity (Bénabou & Tirole, 2002).
On the other hand, since performance ratings give employees a signal about how their
performance is perceived by their supervisor, lenient ratings may lead employees to falsely
conclude that they are performing their tasks as desired by their supervisor. Providing employees
with this mistaken impression will have a negative effect on performance because it motivates
them to continue to take wrong/suboptimal actions. Hence, leniency bias is predicted to have a
positive effect on effort, but this will only lead to improved performance if the behavior that is
stimulated is desirable. Since supervisors are expected to be cautious about stimulating
undesirable actions, the overall effect of leniency bias is expected to be positive. I state the
hypothesis as follows:
Hypothesis 3: Leniency bias has a positive effect on the effectiveness of a compensation
contract as an incentive provider.
Centrality Bias6
The lack of distinction between performance ratings of different employees is expected to
influence employee incentives in several ways. First, above-average performers are likely to feel
disenchanted when employees who perform worse are rewarded almost equally. This is expected
to negatively affect above-average employees’ future performance (Lazear, 1991). Moreover,
6 Although both biases will move up ratings for below-average performance, centrality bias also decreases the
variance in reported employee performance.
10
compression reduces the probability that the value of a marginal performance rating increase will
outweigh the cost of the extra effort needed to improve performance sufficiently to receive this
marginal performance rating increase. Centrality bias is thus predicted to have a negative effect
on the incentives of above-average performers. This results in the following hypothesis:
Hypothesis 4: Centrality bias has a negative effect on the effectiveness of a compensation
contract as an incentive provider for above-average performers.
The situation is different for below-average performers, as compression influences their
ratings in a positive way. Because of centrality bias, their performance seems similar to that of
top performers. Since most employees consider their own performance to be above-average
(Meyer, 1975), providing below-average employees with their true comparative position will be
a deflating experience for most of them, which can have a disruptive effect on their performance
(Pearce & Porter, 1986; Gabris & Mitchell, 1988). Compressed ratings might therefore have a
positive effect on the incentives of below-average performers as they protect them from
information that will lower their self-perception and the perceived prospects from providing
effort in the future (Prendergast & Topel, 1993; Fang & Moscarini, 2002).
On the other hand, just as with above-average performers, the lack of variance in
performance ratings might also negatively affect the motivation of below-average performers as
their marginal cost to improve performance ratings may be higher than the marginal benefit they
receive from the performance rating improvement.
Since I cannot predict which of these arguments explains the relation between centrality
bias and the performance change of below-average performers, their relative importance must be
empirically determined.
11
III. RESEARCH DESIGN
Research Setting
In this paper I use data provided by a Dutch financial service provider (FSP). FSP is one
of the main financial service providers in the Netherlands, and it has a considerable stake in the
European market. It serves over nine million customers and its assets have a total value of some
€500 billion (in 2005).
Recently, competition in the financial sector has become more intense. More specialized
firms have managed to increase their marketshare substantially, jeopardizing FSP’s market
leading position in the Dutch market. In order to defend its market position, FSP decided to
move to more results-oriented management, to strengthen the relationships with current
customers and to acquire new customers. To accomplish these objectives FSP restructured its
front-offices, introduced a more pro-active way of approaching potential new customers and
designed a variety of new management control systems. As part of this initiative, FSP introduced
a new incentive system for its branch offices in 2003. The compensation plan covers all branch
office employees, except for the local management teams.
The New Compensation Contract
The new compensation system consists of a fixed salary and a bonus, where the fixed
salary depends on the employee’s job and the periodic fixed salary increase (up to a stated
maximum), and the bonus on the employee’s overall performance rating.7 In the incentive
system, performance is not only objectively measured but also subjectively assessed. More
specifically, half of the overall rating is determined by an output-based performance rating and
half by a competence-based performance rating.
7 Although the performance ratings are taken into account when making promotion decisions, the system was
designed to determine yearly compensation, not promotions.
12
The output-based section of the system consists of three to six subjective or objective
performance measures that measure specific employee output. Examples of objective
performance measures are: ‘the number of insurance policies sold,’ ‘the percentage increase in
portfolio growth,’ or ‘the number of appointments made.’ Examples of subjective outcome
performance measures are ‘the value of the front-office support,’ ‘the quality of the management
information reports,’ or ‘timely preparation of customer reports.’ The designers of the system
have provided the supervisors with examples of performance measures, but the supervisors
decide wich performance measures to use. 8
Although the intention of the new compensation system is to improve output, the
designers of the system wanted to avoid a blind focus on improving measured performance. The
compensation system therefore includes the subjective assessment of competencies that are
considered essential to performing the employee’s tasks in an efficient and correct manner. The
supervisor must choose between one and four competence measures, which are subjective
assessments of specific behavior that the employee must demonstrate. Hence, the system not
only indicates what type of output improvement is expected, it also informs the employees how
output improvements are expected to be accomplished. The competencies ‘cooperative behavior’
and ‘customer focus’ are included in all performance documents (see Figure 1).
The weight on each performance measure is determined by the number of measures
chosen within each section. Each section determines half of the overall rating and the weights of
the measures within each section are equal. The supervisor also sets specific targets and margins
8 Although supervisors individually select the performance measures and set the targets, before presenting the
contracts to the employees, all supervisors and management meet to discuss and compare the chosen performance
measures and targets. In this way FSP hopes to increase the consistency in the use of the system.
13
for each of the selected measures.9 At the end of the year, the supervisor evaluates performance
according to these measures and targets, and assigns a rating between one and five (not
necessarily integers) to each measure. This leads to an overall rating which determines a)
whether the employee receives the periodic fixed salary increase, and b) the bonus percentage
range. The employee only receives the periodic fixed salary increase if the overall rating is above
1.5.10
Additional Discretion in the Compensation System
The use of subjective performance measures is not the only way in which the FSP has
introduced supervisor discretion into its compensation system. Supervisors also have discretion
over the awarded bonus percentage. The overall rating is linked only to a bonus percentage
range,11
not to a concrete percentage and, if deemed necessary, supervisors are allowed to
provide employees with a bonus percentage outside the stated range. These discretionary bonus
adjustments are intended to ensure that important actions that are not foreseen ex ante can still be
rewarded.
In sum, the incentive system links pay to performance and puts part of the compensation
at risk. It does not rely exclusively on objective measures, but combines objective performance
measurement with subjective performance appraisal, thereby giving supervisors a relatively wide
scope for discretion.
The New versus the Old Compensation System
Although both the new and the old system consist of a fixed salary and possibly a bonus
and/or a periodic fixed salary increase, the new compensation system is different in two
9 Margins indicate when a performance target is not met, almost met, met, surpassed and generously surpassed
indicated by the scores ‘bad,’ ‘regular,’ ‘good,’ ‘very good’ and ‘excellent,’ respectively. 10
This is only true for employees who have not yet reached the maximum of their salary scale. 11
A ‘bad’ rating (0-1.5) corresponds to a bonus of 0%, a ‘regular’ rating (1.5-2.5) to 1-3%, a ‘good’ rating (2.5-3.5)
to 4-8%, a ‘very good’ rating (3.5-4.5) to 9-12% and an ‘excellent’ rating (4.5-5) to 13-15%. See Figure 1.
14
important ways. First, the old system did not use any objective performance measures; only
subjective assessments were used to evaluate performance. Second, in the old system a higher
performance rating did not consistently translate into a higher bonus percentage. In theory, there
was a link between pay and performance, where a higher performance rating should result in a
higher bonus. In practice, however, supervisors seemed to simply increase the bonus percentage
with one or two percentage points when performance was acceptable. This resulted in a situation
where some employees would, for example, get a bonus of four percent when rated ‘excellent’,
while others received fourteen percent when rated ‘good’.
The lack of a clear link between pay and performanc was the main reason for FSP to
abandon the old system. By creating a clear link, the new system is expected to increase
employee incentives and focus on output.12
The new system was not introduced to reduce
compensation costs: FSP increased employee compensation so that no employee would receive
less compensation after the introduction of the new system.
Introduction and Implementation of the New Compensation System
FSP took the introduction of the new compensation system very seriously. All employees
and supervisors received information (through presentations, folders, letters, and internet sites)
on the functioning of the system, its formal steps, and the timelines of its implementation. The
directors of the branch offices also wrote a formal letter to all employees indicating top
management’s support for the new system. Supervisors were provided with workshops which
explained how to select the performance measures, how to set the performance targets, and how
to rate performance. In order not to destroy the incentive effect of the system, as had happened
with the old system, supervisors where explicitly asked to rate performance accurately and to
12
An internal study that examined the consequences of the introduction of the new compensation system finds that
both supervisors and employees think that employee effort and focus on output have increased after the introduction
of the system.
15
uphold the link between pay and performance.13
Moreover, empirical evidence (e.g.
Longenecker et al., 1987; Burney, Henle, & Widener, 2006) indicates that attaching importance
to the implementation and use of a compensation system reduces the raters’ tendency to bias
performance ratings. The setting of this study is therefore biased against finding the existence of
supervisor bias, and therefore their determinants and consequences.
Data
The analyses employ two years of proprietary archival data on the incentive plans for
employees of five offices of FSP14
. I limit my study to employees who were employed by the
bank in both 2003 and 200415
and to departments that made significant use of objective
performance measures (at least 25%). This results in 396 complete performance documents, 198
from 2003 and 198 from 2004.16
The documents (see Figure 1) provide information about the chosen performance
measures, the targets, the ratings per measure, the total rating, and the old and new total
compensation (salary scale, periodic salary increase and bonus). Furthermore, the documents
indicate the employee’s job, department and supervisor, and an extensive description of the
chosen performance measures. Based on these descriptions I classify each output-based
performance measure as being either objective or subjective, while the competencies are by
13
Although the supervisors were asked to rate accurately, the concepts of leniency and centrality bias were not
explained and supervisors were not explicitly asked to refrain from applying these biases. 14
The five studied offices provide a representative mix of different regions, different office sizes and different types
of areas (ranging from rural to major cities). 15
Using employees that were employed by FSP in 2003 and 2004 might lead to selection bias because employees
that left the company in 2003 are not included in the sample. To examine this issue, I have asked the HR managers
of each office whether there were employees that had left the company because of the new compensation system.
Four of the five Hr managers said that this had not happened, while one HR-manager mentioned that there were
some instances of employees leaving because of the new system but that this was a very small percentage (less than
3%). 16
The data gathered does not cover all the employees meeting these conditions because some of the performance
documents were incomplete or were never received by the human resource department. I have checked whether
certain departments were underrepresented in the sample but found no such bias.
16
definition subjective. This results in two performance dimensions: objective and subjective.17
I
also obtained personal characteristics of the supervisors and employees such as gender and age,
and I interviewed several employees, supervisors, and designers of the system to get a better
understanding of the compensation system and the company background. Finally, I used data
provided by an internal study that examined employee and supervisor reactions to the
compensation system.
Summary statistics for the main variables used in this paper (see Tables 1, 2 and 3)
indicate that supervisors in 2003, on average, provided higher ratings on the subjective than on
the objective dimension. Both the mean (2.94 versus 2.80) and the median (3.00 versus 2.75)
were significantly higher in 2003 (p < 0.005 two-tailed). This relationship reversed in 2004; both
the mean (3.14 versus 3.28) and the median (3.13 versus 3.33) subjective rating were
significantly lower (p < 0.005 two-tailed). The descriptive statistics also show that the variance
in the objective ratings was significantly larger (p < 0.001 two-tailed) than the variance in the
subjective ratings in both 2003 and 2004. As predicted, the subjective ratings seem to be more
lenient (although only in 2003) and more compressed, suggesting the presence of leniency and
centrality bias. In the next section, I examine the existence of leniency and centrality bias more
formally.
17
For 63 percent of the employees in the sample the subjective dimension solely consists of competencies, while for
the remaining employees only one or two subjective output-based measures were chosen. I examine whether the
results presented in this paper are influenced by the type of subjective measures that are used (only competencies or
also subjective output-based measures), by re-estimating all models for only those employees that have no
subjective output-based measures and by re-estimating all models while excluding the subjective output-based
measures. In both cases the results are similar to those presented.
17
Leniency and Centrality Bias
In order to examine the determinants and performance effects of supervisor bias, I first
examine whether the performance ratings are subject to leniency and centrality bias, as predicted
by previous research. To detect leniency, I test whether performance ratings on the subjective
dimension are higher, on average, than performance ratings on the objective dimension, after
controlling for other relevant influences. Similar to Moers (2005), a higher score on the
subjective dimension is considered to be evidence consistent with more lenient ratings. Leniency
bias can only be adequately captured in this way if the performance ratings on the objective
dimension are unbiased or at least significantly closer to the true performance value than the
subjective ratings. Another underlying assumption is that employees have similar ability levels
on the two dimensions. After controlling for effort allocation effects, actual employee
performance on the objective and subjective dimension (as opposed to supervisors’ potentially
biased assessments) is therefore assumed to be very similar. There should thus, on average, be no
difference between the ratings on the different dimensions unless the ratings are biased, after
controlling for other relevant influences.
As the dependent variable, I use the employees’ performance ratings on the objective and
subjective dimensions (RATING). This means that each employee is included twice for each
year, once with his objective rating and once with his subjective rating. The influence of
discretion on the ratings is examined by including a dummy variable that equals one if the
observation is a subjective rating and zero otherwise (DSubjectivity). To account for differences in
contract design that may cause different effort allocations, I control for the contractual weight on
the specific dimension (WEIGHT), as a higher weight is expected to lead to higher effort and
consequently to higher ratings. I also include the number of performance measures (NR_PM), as
18
a larger number of performance measures can lead to incentive dilution. To control for year
effects, I include a year dummy (Y2003) that is one if the observation relates to 2003 and zero
otherwise.
In order to determine whether leniency bias is specific to a certain group of employees, or
is a more general consequence of using subjective performance measures, I include several
additional control variables. First, supervisors are not expected to behave in identical ways when
it comes to rating performance. I include supervisor dummies (DSupervisor) to control for these
differences. Since rating behavior might also be dissimilar for different types of employees, I
also control for three employee variables: age (AGE), gender (SEX), and number of designated
contract hours per week (HOURS). Finally, the way the system is used (e.g., the type of
measures that are chosen) and/or the importance attached to the system (e.g., the time that is
provided for instructions and evaluations) might not be identical for different departments and
offices. To control for these differences, office (DOffice) and department dummies (DDepartment) are
included.18
To control for likely correlation of regression model errors for a given employee, I use a
random effects model with robust standard errors. This leads to the following specification:
�������� �� ��������������� ���������� ����_���� � !����� �"����� �#$�%� �&�'(�$��
∑ �� *�+* ���,�-��./-0 ∑ �1
"�1+ 2 �344���5 ∑ �6
#�6+" �7�,8-�9�:�;
<� ε=> ?1A
where i indicates performance ratings (i= 1, …, 792), t time, j supervisors, k offices, and l
departments.
18
Employees have different tasks and duties that are logically translated into different performance measures. I
partly control for these differences by adding supervisor, office, and department dummies. The chosen performance
measures per supervisor are, however, also not identical per se. To examine the influence of performance measure
differences within reference groups, I redo the analyses for only those observations where the chosen performance
measures in the reference group are at least 90% the same in the two consecutive years. The results show similar
inferences as those in Table 5, implying that the results are not driven by differences in performance measures
within reference groups.
19
I use random effects19
to obtain consistent estimates of all parameters, including coefficients of
time invariant regressors (Greene, 1993).
I investigate whether the use of subjective measures leads to centrality bias, by
calculating the ratio between employees’ ratings on the objective (subjective) dimension and the
mean rating on the objective (subjective) dimension (R_RATING = Max ((rating/mean rating),
(mean rating/rating)). To justify this method, I assume that the variance in performance is similar
over the two dimensions and that the variance in the objective performance ratings is similar to
the performance variance in the sample. All other variables are the same as in model 1, and a
random effects model with robust standard errors is used, leading to the following empirical
specification:
�_�������� �� ��������������� ���������� ����_���� � !����� �"����� �#$�%� �&�'(�$��
∑ �� *�+* ���,�-��./-0 ∑ �1
"�1+ 2 �344���5 ∑ �6
#�6+" �7�,8-�9�:�;
<� ε=> ?2A
Since the descriptive statistics show different patterns when comparing the objective and
subjective ratings for the two consecutive years, I examine the existence of leniency and
centrality on a yearly basis, in addition to the pooled sample.
The Determinants of Bias
In this section, I examine the determinants of leniency and centrality bias. Although all
the compensation contracts are subject to supervisor discretion, the extent to which ratings are
biased is expected to vary with supervisor preferences. In Table 4, descriptive statistics on
leniency and centrality bias and the discretionary bonus adjustment are presented. They show
that the average amount of leniency bias was larger in 2003 than in 2004, while the average
amount of centrality bias was very similar in the two consecutive years. Supervisors also took
19
The Hausman test indicates that random effects estimation is adequate for models 1, 2, 3 and 4.
20
advantage of their ability to make discretionary adjustments, especially in 2003. None of these
adjustments was negative, and the average adjustment was 2.7% (1.5%) in 2003 (2004).
I measure the extent of leniency bias in the subjective rating of a specific employee by
comparing the objective rating to the subjective rating of that employee. As differences between
the objective and subjective rating can be caused by differences in effort allocation on the
different dimensions, I control for both the contractual weight and the number of performance
measures. Moreover, the difference between the objective and subjective rating captures both
leniency and compression applied to the rating (even after controlling for effort allocation
effects). In order to control for the effort allocation effect, and to separate leniency bias from
centrality bias, I use a residual model. I regress the ratio between subjective and objective ratings
on the weights, the number of performance measures and on the centrality bias (explained
below), and use the residuals of this regression as a measure of leniency bias (LEN_BIAS).20
This
proxy for leniency bias will likely contain some measurement error because possible differences
in ability levels on the two dimensions are not taken into account. However, since measurement
error that is caused by different ability levels is not correlated with the independent variables, the
regression estimates will be unbiased. Measurement error in the dependent variable will however
decrease the power of the test, which will work against finding any results.
Several variables are predicted to affect the extent of leniency bias applied by
supervisors. Since supervisors prefer to avoid negative consequences related to rating
performance, and painful confrontations are more likely to occur when performance is
inadequate, employee performance is predicted to have a negative effect on the extent of
leniency bias. Supervisors might also rate more leniently, especially when employee
20
A problem with this measure is that it assumes leniency bias and centrality bias to be independent. However, if I
use the simple ratio between subjective and objective ratings and control for the extent of centrality bias in the
regression, I get similar results.
21
performance is poor, to hide departmental problems. I capture employee performance by
including the objective performance ratings (OBJ_R) (see Table 6 for variable definitions).
Confronting employees with harsh but accurate ratings might be less costly to the
supervisor when the personal relationship with the employee is not that strong. I therefore predict
that ratings are less lenient when the employee and supervisor have only started working together
recently. Moreover, since supervisors care about the performance of their department, they are
expected to reduce leniency when rating leniently might give employees the mistaken idea that
they are performing their duties correctly when they are not. Since misunderstandings on how
tasks should be preformed are more likely to occur when the employee and supervisor have
worked together for a short period of time, supervisors are also expected to be less lenient in the
initial stage of the relationship to avoid stimulating dysfunctional behavior. To capture the
influence of the duration of the employee-supervisor relationship, I include a dummy variable
that equals one if the employee joined the company within the last three years, if the employee
recently changed jobs within the company or if a new supervisor was assigned, and zero
otherwise (NEW_R).
The extent of leniency bias is also influenced by the time and effort the supervisor must
invest in collecting performance information. High information-gathering costs are predicted to
result in less precise evaluations, which are expected to lead to more lenient ratings because
leniency decreases the possibility that imprecise ratings lead to problems. I capture information-
gathering costs using the supervisor’s position in the organizational hierarchy. More specifically,
22
I proxy for information-gathering costs (INFO_C) using the difference between the
organizational level of the supervisor and the employee.21
The effect of the attitude of the supervisor’s superiors towards rating accuracy is
analyzed by examining the financial performance of the offices, as financial conditions are
expected to affect attitudes towards leniency bias. More specifically, poor financial performance
is predicted to put pressure on the office to keep compensation costs down, making the superior
less tolerant of lenient ratings (Longenecker et al., 1987). I measure the financial position of the
offices by including their profit growth (GROWTH) and the difference between their budgeted
and actual profit (BUDGET).
Supervisor and employee characteristics are also expected to influence supervisor bias as
they influence the supervisor’s preferences. I include supervisor gender (SUP_SEX) and
employee gender (SEX) as independent variables because some evidence in the psychology
literature indicates that female supervisors rate more leniently (Tsui & O'Reilly, 1989), and that
female employees are rated lower than male colleagues (Rosen & Jerdee, 1974). Finally, the age
difference between the supervisor and employee (DIF_AGE) is included because research on
relational demography shows that individuals with comparative demographic characteristics are
more likely to develop a close relationship (Tsui & O'Reilly, 1989). A closer relationship is
expected to have a positive effect on leniency bias (Varma, DeNisi, & Peters, 1996), because it
can increase the utility received from favoring the employee, and/or increase the pain suffered
when the employee needs to be confronted.
Discretionary bonus adjustments provide supervisors with an additional way to influence
compensation. To control for a possible substitution effect, I include the number of percentage
21
The influence of the supervisor’s position in the organizational hierarchy on rating behavior has been analyzed in
the psychology literature, and several studies indicate little interrater agreement on performance ratings provided by
supervisors from different organizational levels (Berry, Nelson, & McNally, 1966; DeCotiis & Petit, 1978).
23
points that the bonus is adjusted beyond the stated range (DIS_ADJ). Finally, I include year
(Y2003) and department dummies (DDepartment) as additional controls.22,
.23
A random effects model
at the employee level24
with robust standard errors is used to test the following empirical
specification:
C��_D��$�� �� ��'DE_��� �����_��� ����F'_G�� � ��'����� �"D(������ �#$(�_$�%�
�&$�%� �*��F_����� �2��$_��E�� ���!����� ∑ �6�&6+�� �7�,8-�9�:�;
<� ε=> ?3A
where i relates to employee ratings (i= 1, …, 396), t to time and l to departments.
In order to measure centrality bias, I calculate the ratio between the standard deviation of all
ratings on the objective dimension and the standard deviation of all ratings on the subjective
dimension for each reference group25
(CEN_BIAS). This ratio is the same for all employees in a
reference group, and a high ratio indicates that the subjective ratings are compressed relative to
the objective ratings.
Since the extent of centrality bias is measured per reference group, I examine the
determinants of centrality bias at the supervisor level. I include the same explanatory variables as
in model 3 except that I use the median objective performance rating of the reference group
instead of the individual performance ratings (OBJ_R), and I include supervisor age (SUP_AGE)
instead of the age difference between supervisor and employee (DIF_AGE). Employee gender
(SEX) is also excluded. The predicted signs of the explanatory variables are similar to those in
model 3 apart from the expected sign on office performance. Pressure to keep compensation
22
Due to high correlation between the variables measuring office performance and the office dummies, the office
dummies are not included in the model. 23
The introduction of additional variables that control for the number of employees evaluated by the supervisor and
contract choices made by the supervisor, such as the contractual weight placed on subjective measures, the number
of subjective measures included and the consistency in the chosen subjective measures within the reference group,
leads to similar results. 24
Random effects at the supervisor level lead to similar results except for the age difference between the supervisor
and the employee, which in this specification no longer has a negative effect on the extent of leniency bias. 25
A reference group consists of all employees who have a similar function and who are evaluated by the same
supervisor.
24
costs down is predicted to lead to more, instead of less, compression. Furthermore, the standard
deviation of all objective performance ratings per reference group (REF_SD) is included as an
additional control variable, since the extent to which ratings can be compressed depends on
existing performance variation in the reference group. This leads to the following specification
that is tested with a random effects model with robust standard errors:26, 27
G��_D��$�� �� ��'DE_��� �����_��� ����F'_G�� � ��'����� �"D(������ �#$(�_$�%�
�&$(�_����� �*��$_��E�� �2��F_$��� ���!����� <� ε=> ?4A
where i relates to supervisor rating decisions (i= 1, …, 67), and t to time.
Performance Effects
In this part of the analysis, I investigate how leniency and centrality bias affect the
effectiveness of the compensation system as an incentive provider. I capture the differences in
incentives provided by the compensation system by examining the change in employee
performance following exposure to supervisor discretion. I measure the change between 2003
and 2004 as the supervisors’ influence on incentive provision becomes apparent after the
evaluation of 2003. Changes in objective performance (∆PERF_O), subjective performance
(∆PERF_S), and total performance (∆PERF_T) are examined because bias is hypothesized to
have an overall influence on employee incentives. Performance change is expected on both the
objective and the subjective dimensions. However, since prior analyses indicate that the
subjective (and therefore the total) ratings are biased, they must be interpreted with care.
26
Estimating the model at the employee level with random effects at the supervisor level leads to similar results. 27
As with employee performance, I use the median bonus adjustments per reference group instead of the individual
bonus adjustments (DIS_ADJ). Moreover, because of the limited number of observations at the supervisor level (n =
67) the department dummies are not included.
25
Existing variation in the biases provides an opportunity to examine the influence of these
biases on future performance.28
Leniency (LEN_BIAS) and centrality bias (CEN_BIAS) are
measured as before. However, in order to analyze the separate effects of centrality bias on above-
average and below-average performers, I split CEN_BIAS into two variables: CEN_BIASA and
CEN_BIASB. CEN_BIASA (CEN_BIASB) takes the value of the ratio when the employee
performs above (below) the average of his reference group on the objective dimension, and zero
otherwise. As mentioned before, whether I am able to adequately capture leniency and centrality
bias depends on the validity of the assumption that the objective performance rating is a good
benchmark. This assumption is however less essential when examining employee reactions
because even if the objective performance rating is not a good benchmark of actual performance,
the objective performance rating is likely to be used by employees as a benchmark to compare
their subjective rating to and subsequently to form their perceptions of bias with. Thus, since
employees react to their perceptions, this analysis is less dependent on the validity of the earlier
made assumptions.
I control for several variables that might influence the change in employees’ performance
ratings. First, discretionary bonus adjustments are expected to have a positive effect on employee
incentives. By adjusting ‘incorrect’ ratings, the supervisor signals that providing employees with
a fair reward is considered important. This reinforces employees’ beliefs that improving
performance will result in higher rewards, which is essential in motivating employees to enhance
performance (Dubinsky & Levy, 1989). Moreover, discretionary adjustments are expected to
28
When developing hypotheses on the influence of centrality bias on incentives, I assume that the employees had at
least some knowledge of the rating distribution of their reference group. During the interviews, I asked several
employees about this matter and they confirmed that they had some, though not full, knowledge of the performance
ratings of their peers. Moreover, through the internal study conducted by the company (see page 15), employees
confidentially provided information on the perceived fairness of the evaluation procedures and the outcome
distributions of the new compensation system, which implies that they possessed knowledge of how rewards were
distributed.
26
have a positive effect on incentives by creating a feeling of reciprocity.29
I control for the effect
of discretionary adjustments using the DIS_ADJ variable defined earlier. Discretionary
adjustments are not only expected to influence the motivation of the employees who receive
them, but also the motivation of those who do not but are confronted by their existence.
Discretionary bonus adjustments create a lack of consistency in rewarding procedures. For some
employees actions not included in the compensation contract are taken into account, while for
others they are not. This lack of consistency influences incentives by influencing employee
perceptions of procedural justice within their organization (Parker & Kohlmeyer, 2005). To
control for the influence of discretionary bonus adjustments on employees who do not receive
them, I use a dummy variable that equals one if an adjustment has taken place within the
reference group and zero otherwise (REF_DISADJ).
Third, I control for the maximum amount the performance of the employee can still
improve (the difference between the maximum performance rating (5) and the obtained
performance rating in 2003, MGROWTH), for changes in the contractual incentive weight on the
objective dimension (DIF_WO) and for changes in the number of performance measures
(DIF_NR).
I also include several personal conditions that might make an employee less sensitive to
the incentives provided by the compensation contract. First, employees who have reached the
upper limit of their salary scale might be less receptive to changes in provided incentives as
performance improvement can only lead to a bonus increase, not to a periodic salary increase. I
also include contract hours and age, as I suspect that part-time and older employees are not
motivated in the same way (e.g., they are expected to be less concerned about their future
29
The employee receives more from the company than the required minimum, making the employee willing to
provide more than the minimum effort required (Hannan, Kagel, & Moser, 2002).
27
career). Some employees have occupied a different position and/or were appraised by a different
supervisor in 2004. To ensure this does not drive the results, two dummy variables are included
(DDIF_FUN & DDIF_SUP) that equal one if the employee’s function and supervisor, respectively,
have changed, and zero otherwise. Finally, I add office (DOffice) and department dummies
(DDepartment) to control for office-specific and job-specific elements, such as local market
conditions, that might influence employee performance. This leads to the following empirical
specification:30
∆���F� L� L�C��_D��$� L�G��_D��$�� L�G��_D��$D� L ��$_��EM L"��F_��$��E�
L#���'���� L&��F_�'� L*��F_��� L2�7MN_�OPQ L���7MN_R3SQ L����%_$G�
L���'(�$� L������ ∑ L1�*1+� �344���5 ∑ L6
�"6+�2 �7�,8-�9�:�;
T� ?5A
where i relates to employees (i= 1, …, 198) and ∆PERF can be ∆PERF_T, ∆PERF_O or ∆PERF_S.
IV. RESULTS
Leniency and Centrality Bias
As a first step in my analysis, I examine whether the performance ratings are, on average,
subject to leniency and centrality bias (Table 5). The results show that the ratings on the
subjective dimension are higher than the ratings on the objective dimension, after controlling for
relevant influences, implying that subjective performance ratings are subject to leniency bias.
However, the results of the annual analyses show that the subjective performance ratings are not
more lenient in 2004. Subjectivity even has a marginally significant negative effect on the
ratings. These results indicate that supervisors in general use the leeway they have in subjective
30
I have chosen to model the relationship between centrality and leniency bias and performance change as a linear
relationship because I expect the effect of more leniency and centrality bias to be relatively constant over the limited
range found in this setting.
28
performance appraisal to bias performance ratings, but that this does not necessarily result in
more lenient ratings.
Consistent with earlier empirical work, I find that the ratings on the subjective
performance dimensions are more compressed than the rating on the objective dimension. The
number of performance measures has a negative effect on the ratings, suggesting that a higher
number of measures leads to lower performance ratings, consistent with the dilution argument.
The Determinants of Bias
Determinants of leniency and centrality bias are investigated in Table 6. Employee
performance is found to be one of the main determinants of leniency bias. The objective
performance ratings negatively affect the extent of leniency bias, which indicates that
supervisors rate more leniently when employee performance is low. The performance of the
reference group has no effect on centrality bias.
Ratings are found to be less lenient when the employee and supervisor only recently
started to work together, indicating that supervisors are more willing to be accurate when
personal ties are not that strong, and when lenient ratings are more likely to provide a wrong
signal on the correctness of employee actions. Although the lack of a strong personal relationship
was predicted to make supervisors more willing to differentiate between employees, ratings are
found not to be less, but more compressed in the initial period of the employee-supervisor
relationship. A possible explanation for this is that supervisors refrain from differentiating
strongly until they have more information on employee performance.
The proxy for information-gathering costs positively affects the extent of leniency and
centrality bias, indicating that supervisors provide more lenient and compressed ratings when
29
gathering information is more costly. The evidence also indicates that the office’s financial
position affects supervisor bias. As predicted, both financial performance variables have a
positive effect on the extent of leniency bias and a negative effect on the extent of centrality bias.
This indicates that supervisors become more lenient and compress less when the office is
performing better financially.
Supervisor gender is also found to influence rating behavior. I find that male supervisors
are more lenient, not female supervisors, as predicted by previous studies. A speculative
explanation for this finding is that female supervisors are aware of the stereotype and
consequently exercise more caution when rating. Employee gender was not found to influence
the extent of bias.31
Finally, age differences between supervisors and employees negatively affect
the extent of leniency bias, suggesting that it is more costly for supervisors to rate accurately
when they have a strong relationship with the employees.
In sum, the results indicate that supervisors vary the amount of bias they apply to
performance ratings, and that more bias is applied when biased performance ratings increase the
utility the supervisor receives out of rating employee performance.
Performance Effects
The performance effects of leniency and centrality bias are examined in Table 7. The
results32
indicate that supervisor bias has both positive and negative effects on future
31
Most studies that find that female employees receive lower ratings than male employees are conducted in settings
where the occupation would likely be perceived as masculine (see Landy & Farr, 1980), while this setting has both
masculine- and feminine-type positions. However, even after interacting the position’s stereotype with employee
gender, I find no evidence of employee gender affecting the extent of bias applied to the ratings. 32
The Breusch-Pagan and White's tests for heteroskedasticity indicate that the data does not suffer from severe
heteroskedasticity problems and the Variance Inflation Factors (VIF) indicate the absence of multicollinearity
problems.
30
performance.33
First, leniency bias has a positive effect on objective and total performance
changes, indicating that employees who receive more lenient ratings on the subjective dimension
show more performance improvement on the objective and total dimensions in the following
year. The results also show that centrality bias, for above- and below-average performers, has a
negative effect on performance change on all dimensions, indicating that more compressed
ratings have a negative influence on the performance of all employees.
Regarding the impact of discretionary bonus adjustments, I find that discretionary
adjustments have a positive effect on performance change, while discretionary adjustments made
in the reference group have no effect. This implies that receiving a discretionary adjustment
increases future performance. An alternative explanation is that the ratings in 2004 now portray
the true level of overall performance. Instead of needing an ex post adjustment to give the
employee the fair amount of compensation, the true value is now directly captured by the
measures, resulting in a higher rating. However, even if we cannot be sure that incentives will
improve after employees have received an adjustment, the results show that employees who do
not receive an adjustment do not show a significant negative performance reaction, thereby
taking away an important reason for not adjusting rewards.
The results also show that the employee’s growth potential (MGROWTH) has a positive
effect on performance change. This shows that employees with more room for improvement
improve their ratings to a greater extent. Furthermore, I find that changes in weights and number
of performance measures influence performance change. Being evaluated by a different
supervisor is also found to influence performance; it negatively affects performance changes.
33 I also examine the performance effects on a more aggregate level. More specifically, I investigate whether the
average performance change of all employees who were appraised by the same supervisor is larger (smaller) when
that supervisor has a stronger tendency to apply leniency (centrality) bias. To test this I use model 5, but measure the
performance changes and the biases at the supervisor level. The results show that the average extent of leniency
(centrality) bias is positively (negatively) related to the average performance change.
31
Finally, the results show that age has a negative effect on performance change, implying that
younger employees are more sensitive to incentives.
Robustness Checks
The results show that performance change on the subjective dimension is not
significantly affected by leniency bias. A plausible explanation for this finding is that the
motivational effect caused by the leniency bias in 2003 is offset by the less lenient average
subjective performance ratings provided in 2004. An alternative explanation is that employees
who score higher on the subjective dimension than on the objective dimension (the majority of
the employees) reallocate their effort to the objective dimension because improving a high rating
even further requires more than a linear effort increase. I examine this possibility by including
the quadratic term of the maximum growth potential (MGROWTH) to control for the possible
non-linear effort requirements. The results (not reported) show that the quadratic terms have no
significant effect on performance change and all other inferences remain the same.
I also investigate the influence of effort reallocation on performance change. I do this by
examining the group of employees who are most likely to reallocate their effort from the
subjective to the objective dimension because of non-linear effort requirements; i.e., employees
scoring high on the subjective dimension (top 20%) and scoring higher on the subjective than on
the objective dimension. I use model 5 and replace the leniency bias variable by a dummy
variable that indicates one if the employee belongs to this group and zero otherwise. The results
(not reported) show no relation between the dummy variable and performance change on the
objective dimension, suggesting that the positive relationship between leniency bias and
32
performance change on the objective dimension is not driven by the reallocation of effort from
the subjective to the objective dimension.
Previously, I implicitly assumed that the performance ratings are exactly proportional to
the bonus percentage. The ratings are, however, linked to a bonus range, not to a concrete
percentage. The actual bonus might therefore not be exactly proportional to the rating, even if no
discretionary bonus adjustment has occurred. To ensure that this is not driving the results, I
calculate the difference between the actual bonus percentage and the bonus percentage that
would have been paid if the relation between the rating and the bonus percentage had been
completely proportional. After replacing the discretionary adjustment variable (DIS_ADJ) with
this newly constructed difference variable, I re-estimate specification 5. The results (not
reported) are similar to those in Table 7.34
Additional Analysis
In order to capture the incentive effect, I measure the performance change by comparing
the performance rating of 2003 with the rating of 2004. However, the subjective and therefore
the total rating are biased and the amount of bias applied by the supervisor is not necessarily
consistent over the years. The measurements of the performance change on the subjective and
total dimension are therefore clouded.
To address this limitation, I preform an additional analysis with data collected through a
questionnaire that was part of an internal study performed by FSP.35
In this analysis, instead of
the change in performance, I use the change in effort to capture the incentive effects. The change
in effort is measured using the response to a survey question that asked employees how their
34
The results of model 3 and 4 also remain unchanged when DIS_ADJ is replaced by this new difference variable. 35
The questionnaire was conducted in august of 2004 and administrated by the designers of the system. I had no
influence on the design or execution of the questionnaire.
33
effort was affected by the new incentive system (see Table 8 for variable definition). The
questionnaire also asked the employees to provide details on the design of their performance
document and the outcome of the performance measurement process, which allows me to
measure both leniency and centrality bias. The information from the questionnaire is less precise
and elaborate than the archival data (the questionnaire was kept anonymous so I was unable to
link the questionnaire to the archival data). The calculations of the biases are therefore not
identical to those performed for the main analysis.36
Finally, the questionnaire provides
information on some of the control variables included in the main model. This leads to the
following specification that is tested with an ordered probit regression model:
�FF'��� L� L�C��_D��$� L�G��_D��$�� L�G��_D��$D� L ��%_$G� L"F(CC�����
L#���� L&$�%� T� ?6A
where i relates to employees (i= 1, …, 70).
The results, presented in Table 8, confirm the findings of the main analysis. Leniency
bias has the hypothesized positive effect on effort, while centrality bias has the expected negative
effect. In summary, I find strong support for the prediction that leniency and centrality bias affect
the effectiveness of the compensation system as an incentive provider.
V. CONCLUSIONS AND FUTURE RESEARCH
Rating inaccuracy caused by supervisor bias is perceived to be one of the main problems
from introducing subjectivity into compensation contracts. This is due to its assumed negative
36
For the calculation of leniency bias in the main analysis I used the percentage of objective and subjective
performance measures. Since the questionnaire does not provide any information that allows me to distinguish
between objective and subjective output-based measures (competences are by definition subjective), I only include
those employees that indicated that their output-based measures were very measurable (hence, assuming that for
these employees all output-based performance measures were objective). The way centrality bias is calculated is
identical but since the questionnaire was only completed by a limited number of employees, the information on the
performance of the reference group is very limited.
34
effect on the compensation system’s ability to motivate employees and provide valuable
information for personnel decisions. Our knowledge of the influence of supervisor bias on the
effectiveness of compensation contracting is, however, extremely limited. Empirical studies
examining the consequences of biased performance ratings are especially lacking.
This paper contributes to the literature by investigating what causes supervisors to bias
performance ratings and by analyzing how biased ratings affect employee incentives. The results
indicate that the use of subjective measures leads to more lenient and compressed ratings and
that supervisors bias ratings to a greater extent when bias enhances the utility received out of
rating performance. The results also provide strong support for the prediction that supervisor bias
not only affects current performance ratings, but also future employee incentives. I find that
leniency bias has a positive effect on performance improvement, while centrality bias has a
negative performance effect for both above- and below-average performers.
The empirical finding that supervisor bias can have a positive influence on employee
incentives provides an explanation for earlier empirical findings that show that supervisors in
general do not receive rewards for rating accurately (e.g., Napier & Latham, 1986). Companies
seem to be interested in the effectiveness of performance-based compensation contracts in
increasing employees’ future performance, not necessarily in the accuracy of the performance
appraisal as such (at least not for incentive purposes). Motivation to manage the performance
appraisal process in an efficient manner seems to be provided by linking the supervisors’
compensation and promotion possibilities to their unit’s performance, not to the accuracy of their
unit’s performance ratings.
The findings from this study are not without limitations. First, although there are no
theoretical reasons to expect that the results would not extend to other settings, its
35
generalizability is reduced by relying on data from one firm. Moreover, I was only able to obtain
information for two consecutive years. As a consequence, I was unable to examine whether the
documented effects of leniency and centrality bias persist over the years. More extensive time-
series data from multiple organizations would remove these limitations. Second, the focus of this
study has been on intentional bias. Supervisors can also unintentionally bias performance ratings
due to their cognitive limitations. I was unable to distinguish the extent of distortion caused by
these cognitive limitations and therefore unable to investigate its effect.
Another opportunity for future research is to examine the effect of inaccurate ratings on
other functions of performance ratings. I have limited the study to incentive provision, but
performance ratings are also likely to influence training and promotion decisions, among others.
Finally, this study focuses only on the effects of biased performance ratings on future
performance. Subjectivity in compensation contracting is likely to give rise to additional
concerns, e.g., uncertainty about performance criteria. Investigating how these other concerns, in
combination with biased ratings, influence the effectiveness of compensation contracts would
make an important contribution to the literature.
36
REFERENCES
Akerlof, G. A. & J. L. Yellen, 1988, Fairness and Unemployment, The American Economic
Review, 78(2): 44-49.
Arkin, R., H. Cooper, & T. Kolditz, 1980, A Statistical Review of the Literature Concerning the
Self-Serving Attribution Bias in Interpersonal Influence Situations, Journal of Personality, 48:
435-48.
Baiman, S. & M. V. Rajan, 1995, The Informational Advantages of Discretionary Bonus
Schemes, The Accounting Review, 70(4): 557-79.
Baker, G. P., R. Gibbons, & K. J. Murphy, 1994, Subjective Performance Measures in Optimal
Incentive Contracts, Quarterly Journal of Economics, 109(4): 1125-56.
Bénabou, R. & J. Tirole, 2003, Intrinsic and Extrinsic Motivation, Review of Economic Studies,
70(3): 489-520.
Bénabou, R. & J. Tirole, 2002, Self-Confidence and Personal Motivation, The Quarterly Journal
of Economics, 117(3): 871-915.
Bernardin, J. H. & R. M. Buckley, 1981, Strategies in Rater Training, Academy of Management
Review, 6(2): 205-12.
Berry, N. H., P. D. Nelson, & M. McNally, 1966, A Note on Supervisor Ratings, Personnel
Psychology, 19: 423-26.
Blinder, A. S. & D. H. Choi, 1990, A Shred of Evidence on Theories of Wage Stickiness, The
Quarterly Journal of Economics, 105(4): 1003-15.
Bretz, R. D., G. T. Milkovich, & W. Read, 1992, The Current State of Performance Appraisal
Research and Practice: Concerns, Directions, and Implications, Journal of Management, 18(2):
321-52.
Burney, L. L., C. A. Henle, & S. K. Widener, 2006, Do Characteristics of Strategic Performance
Measurement Systems Used in Incentives Enhance Organizational Fairness?, Working Paper: 1-
41.
Colquitt, J. A., D. E. Conlon, M. J. Wesson, C. O. L. H. Porter, & K. Yee Ng, 2001, Justice at
the Millennium: A Meta-Analytic Review of 25 Years of Organizational Justice Research,
Journal of Applied Psychology, 86(3): 425-45.
DeCotiis, T. & A. Petit, 1978, The Performance Appraisal Process: A Model and Some Testable
Propositions, Academy of Management Review, 3(3): 635-46.
37
Dubinsky, A. J. & M. Levy, 1989, Influence of Organizational Fairness on Work Outcomes of
Retail Salespeople, Journal of Retailing, 65(2): 221-52.
Fang, H. & G. Moscarini, 2002, Overconfidence, Morale and Wage-Setting Policies: Mimeo,
Yale University.
Gabris, G. T. & K. Mitchell, 1988, The Impact of Merit Raise Scores on Employee Attitudes:
The Matthew Effect of Performance Appraisal, Public Personnel Management, 17(4): 369–89.
Gibbs, M., K. A. Merchant, W. A. Van der Stede, & M. E. Vargus, 2004, Determinants and
Effects of Subjectivity in Incentives, The Accounting Review, 79(2): 409-36.
Greenberg, J., 1990, Organizational Justice: Yesterday, Today, and Tomorrow, Journal of
Management, 16(2): 399-432.
Greene, W. H., 1993, Econometric Analysis, 5 ed, New Jersey: Pearson Education, Inc.
Hannan, L. R., J. H. Kagel, & D. V. Moser, 2002, Partial Gift Exchange in an Experimental
Labor Market: Impact of Subject Population Differences, Productivity Differences, and Effort
Requests on Behavior, Journal of Labor Economics, 20(4): 923-51.
Harris, M. M., 1994, Rater Motivation in the Performance Appraisal Context: A Theoretical
Framework, Journal of Management, 20(4): 737-56.
Harris, M. M. & J. Schaubroeck, 1988, A Meta-Analysis of Self-Supervisor, Self-Peer, and Peer-
Supervisor Ratings, Personnel Psychology, 41: 43 - 62.
Hayes, R. M. & S. Schaefer, 2000, Implicit Contracts and the Explanatory Power of Top
Executive Compensation for Future Performance, Rand Journal of Economics, 31(2): 273-93.
Heneman, H. G. & D. P. Schwab, 1972, Evaluation of Research on Expectancy Theory
Predictions of Employee Performance, Psychological Bulletin, 78(1): 1-9.
Holmstrom, B., 1979, Moral Hazard and Observability, Bell Journal of Economics, 10(1): 74-91.
Indjejikian, R. J., 1999, Performance Evaluation and Compensation Research: An Agency
Perspective, Accounting Horizons, 13(2): 147-57.
Ittner, C. D., D. F. Larcker, & M. W. Meyer, 2003, Subjectivity and the Weighting of
Performance Measures: Evidence from a Balanced Scorecard, The Accounting Review, 78(3):
725-58.
Jawahar, I. M. & C. R. Williams, 1997, Where All the Children Are above Average: The
Performance Appraisal Purpose Effect, Personnel Psychology, 50: 905-26.
Landy, F. J. & J. L. Farr, 1980, Performance Rating, Psychological Bulletin, 87(1): 72-107.
38
Larwood, L. & W. Whittaker, 1977, Managerial Myopia: Self-Serving Biases in Organizational
Planning, Journal of Applied Psychology, 62(2): 194–98.
Lazear, E. P., 1991, Labor Economics and the Psychology of Organizations, Journal of
Economic Perspectives, 5(2): 89-110.
Lipe, M. G. & S. E. Salterio, 2000, The Balanced Scorecard: Judgmental Effects of Common and
Unique Performance Measures, The Accounting Review, 75(3): 283-98.
Longenecker, C. O., H. P. Sims, & D. A. Gioia, 1987, Behind the Mask: The Politic of Employee
Appraisal, The Academy of Management Executive, 1(3): 183-93.
McFarlane Shore, L. & G. C. Thornton, 1986, Effects of Gender on Self-and Supervisory
Ratings, Academy of Management Journal, 29(1): 115 - 29.
McGregor, D. M., 1957, An Uneasy Look at Performance Appraisal, Harvard Business Review,
35(3): 89-94.
Meyer, H. H., 1975, The Pay-for-Performance Dilemma, Organizational Dynamics, 3: 39-50.
Mischel, W., E. B. Ebbesen, & A. M. Zeiss, 1976, Determinants of Selective Memory About the
Self, Journal of Consulting and Clinical Psychology, 44(1): 92-103.
Moers, F., 2005, Discretion and Bias in Performance Evaluation: The Impact of Diversity and
Subjectivity, Accounting, Organizations and Society, 30(1): 67-80.
Murphy, K. R. & J. Cleveland, 1991, Performance Appraisal: An Organizational Perspective,
Boston: Allyn and Bacon.
Napier, N. K. & G. P. Latham, 1986, Outcome Expectancies of People Who Conduct
Performance Appraisals, Personnel Psychology, 39(4): 827-37.
O'Connor, N., F. J. Deng, & M. D. Shields, 2006, Determinants of the Subjective Performance
Measurement of Managerial Behavior, Working Paper: 1-33.
Parker, R. J. & J. M. Kohlmeyer, 2005, Organizational Justice and Turnover in Public
Accounting Firms: A Research Note, Accounting, Organizations and Society, 30: 357-69.
Pearce, J. L. & L. W. Porter, 1986, Employee Responses to Formal Performance Appraisal
Feedback, Journal of Applied Psychology, 71(2): 211-18.
Prendergast, C., 1999, The Provision of Incentives in Firms, Journal of Economic Literature,
37(1): 7-63.
Prendergast, C. & R. H. Topel, 1993, Discretion and Bias in Performance Evaluation, European
Economic Review, 37(2-3): 355-65.
39
Prendergast, C. & R. H. Topel, 1996, Favoritism in Organizations, Journal of Political Economy,
104(5): 958-78.
Rosen, B. & T. H. Jerdee, 1974, Influence of Sex Role Stereotypes on Personnel Decisions,
Journal of Applied Psychology, 59(1): 9-14.
Rynes, S. L., B. Gerhart, & L. Parks, 2005, Personnel Psychology: Performance Evaluation and
Pay for Performance, Annual Review of Psychology, 56: 571-600.
Tsui, A. S. & C. A. O'Reilly, 1989, Beyond Simple Demographic Effects: The Importance of
Relational Demography in Superior-Subordinate Dyads, The Academy of Management Journal,
32(2): 402-23.
Varma, A., A. S. DeNisi, & L. H. Peters, 1996, Interpersonal Affect and Performance
Appraisals: A Field Study, Personnel Psychology, 49: 341-60.
Vroom, V. H., 1964, Work and Motivation: Wiley New York.
Weinstein, N. D., 1980, Unrealistic Optimism About Future Life Events, Journal of Personality
and Social Psychology, 39(5): 806-20.
40
FIGURE 1
The performance measurement document
Name:
Department:
Position:
Supervisor:
EvaluationBad Regular Good Very Good Excellent
1) Ο Ο Ο Ο Ο
2) Ο Ο Ο Ο Ο
3) Ο Ο Ο Ο Ο
4) Ο Ο Ο Ο Ο
5) Ο Ο Ο Ο Ο
6) Ο Ο Ο Ο Ο
Comments
EvaluationBad Regular Good Very Good Excellent
1) Ο Ο Ο Ο Ο
2) Ο Ο Ο Ο Ο
3) Ο Ο Ο Ο Ο
4) Ο Ο Ο Ο Ο
5) Ο Ο Ο Ο Ο
6) Ο Ο Ο Ο Ο
Comments
Total rating output dimension 50%
Total rating competence dimension 50%
Total score
Salary Scale Growth % Bonus
Salary Scale Growth % Bonus
PERFORMANCE DOCUMENT
Salary Determination
Good Very Good
Output performance measures
Competence performance measures
Current Salary
New Salary
Excellent
0% 1-3% 4-8% 9-12% 13-15%
Bad Regular
41
TABLE 1
Descriptive statistics: Part I
2003 2004
Variable Mean SD Median Range n Mean SD Median Range n
Overall performance rating 2.87 0.52 2.86 1.6 – 4.7 198 3.20 0.50 3.23 1.6 – 4.4 198
Objective performance rating 2.80 0.76 2.75 1.0 – 5.0 198 3.28 0.74 3.33 1.0 – 5.0 198
Subjective performance rating 2.94 0.46 3.00 1.5 – 4.3 198 3.14 0.45 3.13 2.1 – 4.6 198
Total # of performance measures 9.70 1.77 10.00 5 – 12 198 10.68 1.49 11.00 5 – 12 198
# of objective performance measures 4.04 1.35 4.00 1 – 6 198 4.48 1.30 5.00 2 – 6 198
# of subjective performance measures 5.66 1.36 6.00 2 – 9 198 6.20 1.06 6.00 3 – 8 198
% of total rating objectively determined 44.33 8.72 50.00 25 – 50 198 44.37 7.57 50.00 25 – 50 198
% of total rating subjectively determined 55.67 8.72 50.00 50 – 75 198 55.63 7.57 50.00 50 – 75 198
42
TABLE 2
Descriptive statistics: Part II
Variable Mean SD Median Range n
Age 38.39 8.60 37 23 – 60 198
Supervisor age 39.76 7.54 37 28 – 57 41
Designated contract hours in 2003 32.68 6.52 36 12 – 40 198
Designated contract hours in 2004 32.27 6.80 36 12 – 40 198
Profit growth in 2003a
1.09 2.62 1.60 -6.2 – 11.9 198
Profit growth in 2004a
5.68 18.21 1.27 -20.3 – 67.2 198
Budgeted versus actual profit in 2003b
3.61 3.56 1.64 1.6 – 12.6 198
Budgeted versus actual profit in 2004b
4.14 4.05 4.24 -4.2 – 7.81 198
Variable 2003 2004
% of employees at their salary scale maximum 26.26 31.31
% of female employees 56.06 56.06
Number of supervisors 35 32
% of female supervisors 28.57 40.63
a The percentage difference between the offices’ profit this year and the offices’ profit last year.
b The percentage difference between the offices’ budgeted profit growth and the actual profit growth.
43
TABLE 3
Correlation between variables
198 observations for 2003 1 2 3 4 5 6 7 8 9 10
1. Overall performance rating
2. Objective performance rating 0.90***
3. Subjective performance rating 0.80*** 0.47***
4. Total # of performance m. -0.14** -0.11 -0.13*
5. # of objective performance m. -0.12* -0.17** -0.02 0.65***
6. # of subjective performance m. -0.06 0.03 -0.15** 0.66*** -0.15**
7. % objectively determined -0.10 -0.16** 0.02 0.10 0.52*** -0.39***
8. Employee age -0.02 -0.03 -0.03 -0.04 -0.02 -0.03 -0.03
9. Employee contract hours 0.18** 0.19*** 0.12* 0.30*** 0.13* 0.26*** 0.04 -0.04
10. Profit growth -0.16** -0.15** -0.12 -0.12* -0.05 -0.10 -0.01 0.04 -0.17**
11. Budgeted versus actual profit 0.24*** 0.12* 0.35*** -0.06 0.17** -0.25*** 0.09 -0.16** 0.02 -0.11
198 observations for 2004
1. Overall performance rating
2. Objective performance rating 0.88***
3. Subjective performance rating 0.78*** 0.41***
4. Total # of performance m. -0.08 -0.08 -0.05
5. # of objective performance m. 0.02 -0.03 0.07 0.72***
6. # of subjective performance m. -0.14** -0.08 -0.16** 0.53*** -0.21***
7. % objectively determined 0.18*** 0.10 0.19*** -0.05 0.57*** -0.77***
8. Employee age -0.22*** -0.25*** -0.09 0.04 -0.02 0.08 -0.06
9. Employee contract hours 0.14** 0.15** 0.11 0.10 0.08 0.04 -0.01 0.00
10. Profit growth -0.08 -0.09 -0.04 -0.28*** -0.19*** -0.16** -0.10 -0.01 0.08
11. Budgeted versus actual profit 0.12 0.13* 0.03 0.36*** 0.10 0.39*** -0.02 0.07 0.14* -0.30***
***, **, * is statistically significant at respectively the 1%, 5%, and 10% level (two-tailed).
44
TABLE 4
Descriptive statistics: Part III
Variable Mean SD Median Range n
2003
Leniency biasa 0.06 0.32 0.00 -0.44 – 1.75 198
Centrality biasb 2.18 1.41 1.92 0.32 – 5.95 198
Leniency bias per supervisorc 0.06 0.22 0.03 -0.29 – 0.80 35
Centrality bias per supervisor 2.22 1.36 2.17 0.40 – 5.95 35
2004
Leniency bias -0.06 0.27 -0.11 -0.50 – 1.61 198
Centrality bias
2.20 1.15 1.92 0.52 – 6.05 198
Leniency bias per supervisor -0.06 0.14 -0.10 -0.25 – 0.40 32
Centrality bias per supervisor 2.10 1.00 1.88 0.68 – 5.10 32
Variable 2003 2004
Discretionary bonus adjustment 35% 19%
Average bonus adjustmentd 2.7% 1.5%
a Leniency bias is captured by regressing the ratio between the subjective and objective ratings on the
weights, the number of performance measures and on the centrality bias, and using the residuals of this
regression as a measure of leniency bias. b Centrality bias is measured as the ratio between the standard deviation of all performance ratings on the
objective dimension and the standard deviation of all performance ratings on the subjective dimension for
each reference group. c The average amount of bias applied per supervisor.
d The average bonus adjustment once an adjustment is made. All adjustments were of a positive nature.
45
TABLE 5
The impact of subjectivity on the performance rating and on its variation
2003
2004
Independent variables a
RATING R_RATING RATING R_RATING RATING R_RATING
DSubjectivity
0.10**
(2.11)
-0.14***
(-7.35)
0.29***
(4.80)
-0.15***
(-5.70)
-0.09
(-1.44)
-0.11***
(-4.82)
WEIGHT 0.00
(0.20)
0.00
(1.33)
-0.00
(-0.86)
0.01**
(2.04)
0.01
(0.88)
0.00
(1.54)
NR_PM -0.06***
(-2.76)
-0.00
(-0.25)
-0.06**
(-2.31)
-0.01
(-1.27)
-0.07
(-1.19)
-0.02
(-1.32)
Y2003 -0.40***
(-9.80)
0.05***
(3.28)
AGE -0.01**
(-2.47)
0.01*
(1.76)
-0.01
(-1.34)
-0.00
(-0.20)
-0.01***
(-2.69)
0.01***
(2.66)
SEX -0.12
(-1.38)
0.05*
(1.81)
-0.10
(-1.14)
0.04
(1.17)
-0.17
(-1.55)
0.07*
(1.87)
HOURS 0.01
(1.43)
-0.00
(-1.14)
0.01
(1.50)
-0.00
(-1.12)
0.01
(1.08)
-0.00
(-1.05)
R2
Within
Between
Overall
0.22
0.41
0.31
0.14
0.32
0.20
0.09
0.58
0.43
0.22
0.42
0.33
0.05
0.37
0.26
0.14
0.32
0.24
***, **, * indicate statistical significance at respectively the 1%, 5% and 10% level (two-tailed). Z-values are in parentheses
and n = 792 for the first two regressions and n = 396 for the last four. (Continued on next page)
46
TABLE 5 (continued)
RATING Performance rating, on the objective or the subjective dimension
R_RATING Performance ratings variation, measured as the ratio between employees’ performance ratings on the objective
(subjective) dimension and the mean performance rating on the objective (subjective) dimension: Max
((rating/mean rating), (mean rating/rating))
DSubjectivity Dummy variable that equals 1 if the observation refers to a subjective performance rating and 0 otherwise
WEIGHT Contractual incentive weight on the dimension (objective or subjective) to which the observation relates
NR_PM Number of performance measures on the dimension (objective or subjective) to which the observation relates
Y2003 Dummy variable that equals 1 if the observation relates to 2003 and 0 otherwise
AGE Employee age
SEX Employee gender (0 for female and 1 for male)
HOURS Employee designated contract hours per week a An intercept, as well as supervisor, department and office dummies are included but not reported.
47
TABLE 6
The determinants of leniency and centrality bias
Independent variables
a
LEN_BIAS
CEN_BIAS
OBJ_R
b -0.33***
(-16.07)
0.19
(0.52)
NEW_R -0.05**
(-2.12)
0.64*
(1.78)
INFO_C 0.04*
(1.78)
0.48*
(1.69)
GROWTH 0.02**
(2.08)
-0.02**
(-2.54)
BUDGET 0.01***
(3.44)
-0.04
(-1.54)
SUP_SEX 0.11***
(4.05)
-0.08
(-0.23) SEX 0.03
(1.26)
DIF_AGE -0.01*
(-1.73)
SUP_AGE 0.02
(1.38) DIS_ADJ
b 0.00
(0.44)
-0.23
(-1.33) Y2003 -0.06***
(-3.22)
0.58*
(1.67) REF_SD 2.30***
(3.71)
R2
Within
Between
Overall
0.73
0.63
0.67
0.55
0.29
0.33
***, **, * indicate statistical significance at respectively the 1%, 5% and 10% level (two-
tailed). Z-values are in parentheses and n = 396 for the first and n = 67 for the second
regression .
(Continued on next page)
48
TABLE 6 (continued)
LEN_BIAS Leniency bias, measured as the residuals of the regression of the ratio
between the subjective performance rating and the objective
performance rating on the contractual weights, the number of
performance measures and the centrality bias
CEN_BIAS Centrality bias, measured as the ratio between the standard deviation of
all performance ratings on the objective dimension and the standard
deviation of all performance ratings on the subjective
dimension for each reference group
OBJ_R Objective performance rating
NEW_R Dummy variable that equals 1 if the employee joined the company in the
last three years, if the employee recently changed jobs within the
company or if a new supervisor was assigned, and 0 otherwise
INFO_C Dummy variable that is 0 if the supervisor and the employee have
the same organizational level and 1 if the supervisor has a higher
organizational level
GROWTH Percentage difference between the profit of this year and the profit of
last year calculated per office per year
BUDGET Percentage difference between budgeted profit growth and actual profit
growth calculated per office per year
SUP_SEX Supervisor gender (0 for female and 1 for male)
SEX Employee gender (0 for female and 1 for male)
DIF_AGE Absolute difference between supervisor age and employee age
SUP_AGE Supervisor age
DIS_ADJ Difference between the rewarded bonus percentage and the maximum
of the stated bonus range
Y2003 Dummy variable that equals 1 if the observation relates to 2003 and 0
otherwise
REF_SD Standard deviation of all objective performance ratings within the same
reference group a An intercept and department dummies are included but not reported.
b Since the determinants of centrality bias are examined at the supervisor level the median
objective performance rating per reference group and the median bonus adjustment per
reference group are used in the second regression.
49
TABLE 7
Regression of the effects of supervisor discretion on incentive provision (I)
Independent variables
a
∆PERF_O
∆PERF_S
∆PERF_T
LEN_BIAS 0.73***
(3.00)
-0.10
(-0.94)
0.22*
(1.89) CEN_BIASA -0.14***
(-3.12)
-0.04*
(-1.72)
-0.08***
(-3.08) CEN_BIASB -0.13***
(-3.15)
-0.06***
(-2.69)
-0.09***
(-3.69) DIS_ADJ 0.06*
(1.86)
0.07***
(4.01)
0.07***
(3.65) REF_DISADJ 0.01
(0.11)
0.07
(0.93)
0.02
(0.27) MGROWTH
b 0.27**
(2.26)
0.45***
(6.78)
0.36***
(4.66) DIF_WO 0.01**
(2.07)
0.00
(0.67)
0.01**
(2.36) DIF_NR
c -0.02
(-0.53)
-0.06*
(-1.79)
-0.02
(-1.34) DDIF_ SUP -0.18
(-1.46)
-0.13*
(-1.91)
-0.16**
(-2.14) DDIF_JOB 0.17
(0.85)
-0.04
(-0.32)
0.07
(0.56) MAX_SC -0.18
(-1.42)
0.05
(0.68)
-0.06
(-0.80)
HOURS
-0.01
(-0.73)
0.00
(0.10)
-0.00
(-0.42)
AGE
-0.02***
(-2.64)
-0.01**
(-2.27)
-0.01***
(-3.18)
R2-ADJ
0.47
0.46
0.51
***, **, * indicate statistical significance at respectively the 1%, 5%, and 10% (two-
tailed). T-values are in parentheses and n = 198 for all three regressions.
(Continued on next page)
50
TABLE 7 (continued)
∆PERF_O Difference between the objective performance rating of 2003 and the
objective performance rating of 2004
∆PERF_S Difference between the subjective performance rating of 2003 and
the subjective performance rating of 2004
∆PERF_T Difference between the total performance rating of 2003 and the total
performance rating of 2004
LEN_BIAS Leniency bias, measured as the residuals of the regression of the ratio
between the subjective performance rating and the objective
performance rating on the contractual weights, the number of
performance measures and the centrality bias
CEN_BIASA Variable that equals CEN_BIAS if the observation relates to an
above-average performer and 0 otherwise
CEN_BIASB Variable that equals CEN_BIAS if the observation relates to a below-
average performer and 0 otherwise
DIS_ADJ Difference between the rewarded bonus percentage and the maximum
of the stated bonus range
REF_DISADJ Dummy variable that equals 1 if a bonus adjustment was made
within the reference group and 0 otherwise
MGROWTH Growth potential, measured as the difference between the maximum
performance rating (5) and the obtained performance rating in 2003
DIF_WO Difference between the contractual incentive weight on the objective
dimension in 2003 and in 2004
DIF_NR Difference between the number of performance measures included in
2003 and the number included in 2004
DDIF_SUP Dummy variable that equals 1 if the employee’s supervisor is
different in 2004 than in 2003 and 0 otherwise
DDIF_JOB Dummy variable that equals 1 if the employee’s function is different
in 2004 than in 2003 and 0 otherwise
MAX_SC Dummy variable that equals 1 if the employee has reached the
maximum of his salary scale and 0 otherwise
HOURS Employee designated contract hours per week
AGE Employee age a An intercept, as well as department and office dummies are included but not reported.
b For regressions with the dependent variable ∆PERF_O, ∆PERF_S and ∆PERF_T I use
the objective rating, the subjective rating and the total rating of 2003, respectively, to
calculate MGROWTH. c
For regressions with the dependent variable ∆PERF_O, ∆PERF_S and ∆PERF_T I use
the number of objective, subjective and total performance measures, respectively, to
calculate DIF_NR.
51
TABLE 8
Regression of the effects of supervisor discretion on incentive provision (II)
Independent variables
a
EFFORT
LEN_BIAS 0.39*
(1.65) CEN_BIASA -0.27
†
(-1.44) CEN_BIASB -0.72**
(-2.24) MAX_SC -0.58
(-1.43) FULLTIME 0.39
(0.97) AGE -0.02
(-1.19) SEX -0.29
(-0.81)
McFadden’s Pseudo R2
0.10
***, **, *, † indicate statistical significance at respectively the 1%, 5%, 10% and 15%
(two-tailed). T-values are in parentheses and n = 70.
EFFORT Change in employee effort. The change is measured by using the
employee’s assessment (captured on a 5-item Likert scale) of how the
new incentive system has affected his effort.
LEN_BIAS Leniency bias, measured as the residuals of the regression of the ratio
between the performance rating on the competence dimension and the
performance rating on the output-based dimension on the number of
performance measures and the centrality bias, for all those employees
who have indicated in the questionnaire that their output-based
performance measures were very measurable
CEN_BIASA Variable that equals CEN_BIAS if the observation relates to an
above-average performer and 0 otherwise
CEN_BIASB Variable that equals CEN_BIAS if the observation relates to a below-
average performer and 0 otherwise
(Continued on next page)
52
TABLE 8 (continued)
MAX_SC Dummy variable that equals 1 if the employee has reached the
maximum of his salary scale and 0 otherwise
FULLTIME Dummy variable that equals 1 if the employee works fulltime and 0
otherwise
AGE Employee age
SEX Employee gender (0 for female and 1 for male) a The intercept is included, but not separately reported.