Factors Influencing Employee Intentions

8/3/2019 Factors Influencing Employee Intentions

http://slidepdf.com/reader/full/factors-influencing-employee-intentions 1/18

Factors Influencing Employee Intentions to Provide HonestUpward Feedback Ratings

Austin F. R. Smith Æ Vincent J. Fortunato

Published online: 24 January 2008

Ó Springer Science+Business Media, LLC 2008

Abstract Based on current research involving rater

motivation, we examined several factors hypothesized toinfluence employee intentions to provide honest upward

feedback. Survey data were collected from a demographi-

cally diverse sample of hospital employees (n = 203). In

summary, we found empirical support for generalizing

extant models of rater motivation to an upward feedback

context: cynicism towards upper management and the

upward feedback process, understanding upward feedback,

and opportunity to observe their supervisors were the pri-

mary predictors of employee intentions to provide honest

upward feedback ratings, mediated by the (a) extent to

which employees perceived positive benefits would result

from rating their supervisors honestly, (b) the extent to

which employees feared retaliation by their supervisors,

and (c) rater self-efficacy.

Keywords Upward feedback Á Multisource feedback ÁMulti-rater feedback Á Performance appraisal ÁRater motivation Á Rater participation Á Rater accuracy

Introduction

In recent years, multisource feedback has become a popular

performance appraisal tool used by an estimated 90% of

Fortune 1000 companies (Bracken et al. 2001; Edwards

and Ewen 1996, 2001; Tornow and Tornow 2001). Also

referred to as 360° and multi-rater feedback, multisource

feedback has been called the ‘‘vital sign of the modern

organization’’ (Church and Waclawski 1998, p. 81).

Unfortunately, the utility of multisource feedback, partic-

ularly upward feedback, may be attenuated when raters

provide inaccurate performance ratings (Bernardin and

Tyler 2001; Bracken and Timmreck 2001). Subordinates

may inflate ratings to avoid retribution, limit interpersonal

conflict, or to ingratiate themselves with the supervisor or

they may deflate ratings to retaliate against a supervisor

who has disciplined or given a low performance rating to

the subordinate (Longenecker et al. 1987; Longenecker

1989; London 2001; Waldman and Atwater 2001).

Thus, the importance of identifying and examining

empirical factors that may influence subordinates inten-

tions to provide honest upward feedback ratings cannot be

understated. In this investigation, we draw from several

models of rater motivation (DeCotiis and Petit 1978; Harris

1994; Mohrman and Lawler 1983; Murphy and Cleveland

1995) and the rater participation literature (Avis and

Kudisch 2000; Kudisch et al. 2006; Westerman and Rosse

1997) in identifying several variables that might be just as

important in predicting employee intentions to honestly

rate their supervisors as they are in predicting supervisor

rating accuracy (see Fig. 1). Specifically, we examined the

extent to which employees (a) are cynical about the

motives of management (cynicism toward top manage-

ment) and the upward feedback process (cynicism toward

upward feedback), (b) believe that providing upward

A. F. R. Smith

Department of Psychology, University of Southern Mississippi,

Hattiesburg, MS, USA

Present Address:

A. F. R. Smith

Personnel Decisions International, New York, NY, USA

V. J. Fortunato

Department of Psychology, Boise State University,

Boise, ID 83725-1758, USA

Present Address:

V. J. Fortunato (&)

People Business LLC, Boise, ID, USA

e-mail: [email protected]

123

J Bus Psychol (2008) 22:191–207

DOI 10.1007/s10869-008-9070-4



feedback ratings is a role appropriate activity (role appro-

priateness), (c) understand the upward feedback process

(knowledge of upward feedback), (d) believe that upward

feedback would be used for either developmental, or

administrative purposes, or both (rating purpose), (e) had

ample opportunity to observe their supervisor (opportunity

to observe), (f) believe that positive benefits would result

from implementing an upward feedback program (per-

ceived benefits), (g) fear retaliation by their supervisors

(fear of retaliation), and (h) believe they have the skills,abilities, and confidence to rate their supervisors accurately

(rater self-efficacy).

Consistent with extant rater motivation theories (i.e.,

DeCotiis and Petit 1978; Harris 1994; Mohrman and

Lawler 1983; Murphy and Cleveland 1995), the model

presented in Fig. 1 indicates that employees will be more

likely to provide honest upward feedback ratings when

they believe that the organizational context in which the

upward feedback system takes place supports an environ-

ment in which employees (a) understand the benefits

inherent in providing honest upward feedback ratings, (b)

do not fear retaliation by their supervisors if they do pro-vide honest upward feedback ratings, and (c) believe that

they are capable of rating their supervisors accurately.

Thus, the model presented in Fig. 1 indicates that per-

ceived benefits, fear of retaliation, and rater self-efficacy

will mediate the relationships between cynicism, rating

purpose, role appropriateness, knowledge of upward feed-

back, and opportunity to observe, with employee intentions

to provide honest upward feedback ratings. For mediation

to occur, we must demonstrate that (a) the independent

variables shown in Fig. 1 relate to the dependent variable

and the mediating variables as hypothesized, (b) the

hypothesized mediating variables relate to the dependent

variable, and (c) variance explained by the independent

variables is substantially reduced when controlling for the

mediating variables (see Baron and Kenny 1986; MacK-

innon et al. 1995).

Hypothesized Relations Between the IndependentVariables and Hypothesized Mediating Variables with

the Dependent Variable

Cynicism

Organizational cynicism has been defined as a ‘‘negative

attitude towards one’s employing organization, comprising

three dimensions: (1) a belief that the organization lacks

integrity; (2) negative affect toward the organization; and

(3) tendencies to engage in disparaging and critical

behaviors toward the organization that are consistent with

these beliefs and affect’’ (Dean et al. 1998, p. 345) thatresult from a pattern of unsuccessful and unsupported

change initiatives (Abraham 2000; Waldman and Atwater

1998). Researchers generally agree that the success of

multi-rater systems depends, in part, on the extent to which

employees trust the motives of upper management (Church

1995; Pollack and Pollack 1996; Wimer and Nowack

1998). Moreover, DeCotiis and Petit (1978), Mohrman and

Lawler (1983) and Murphy and Cleveland (1995) sug-

gested that raters will rate more honestly when they believe

Perceived

Benefits

Rater

Self-Efficacy

Employee

Intentions to

Provide Honest

Upward Feedback

Ratings

Cynicism

Role

Appropriateness

Knowledge of

Upward Fdbk

Opportunity to

Observe

Rating Purpose:

DevelopmentalPerceived

Risk: Fear of

Retaliation

Rating Purpose:

Administrative

Fig. 1 Proposed model based on

current extant models of rater

motivation

192 J Bus Psychol (2008) 22:191–207

123



in the integrity of performance appraisal initiatives. Thus,

employees who are cynical about the motives of manage-

ment, in general, and especially about upward feedback

initiatives, are not likely to provide honest feedback ratings

(Tziner et al. 2005). Therefore, we hypothesize the

following:

Hypothesis 1: Cynicism in top management and cyni-cism with respect to upward feedback will relate negatively

with employee intentions to provide honest feedback

ratings.

Rating Purpose

There is consistent agreement that rating purpose affects

supervisors’ intentions to rate their employees honestly. In

general, less rating distortion occurs when ratings are used

for developmental purposes (i.e., ratings used only for

employee development) than for administrative purposes(i.e., ratings used to determine pay and promotion: DeC-

otiis and Petit 1978; Longenecker et al. 1987; Mohrman

and Lawler 1983; Murphy and Cleveland 1995; Tziner

et al. 2001, 2002). Rating purpose may also be an impor-

tant factor that influences subordinate motivation in an

upward feedback context. For example, subordinates have

been shown to be more willing to participate in upward

feedback programs if they believe it to be used for devel-

opmental purposes, rather than administrative purposes

(Avis and Kudisch 2000; Westerman and Rosse 1997).

Moreover, when subordinates believe that upward feed-

back will be used for administrative purposes, they tend toprovide distorted and inaccurate ratings (e.g., Dalton 1998;

Lepsinger and Lucia 1997; Van Velsor 1998). Therefore,

we hypothesize the following:

Hypothesis 2a: Employee perceptions regarding the

extent to which upward feedback will be used for devel-

opmental purposes will relate positively with employee

intentions to provide honest upward feedback ratings.

Hypothesis 2b: Employee perceptions regarding the

extent to which upward feedback will be used for admin-

istrative purposes will relate negatively with employee


Role Appropriateness

Providing supervisory ratings of subordinates is standard

practice in organizations and is typically seen as an

appropriate supervisory job responsibility (Murphy and

Cleveland 1995). However, switching from a supervisor-

only to a multisource feedback system violates traditional

organizational structure and may place both raters and ra-

tees in unfamiliar and uncomfortable situations

(Funderburg and Levy 1997; Murphy and Cleveland 1995).

Research has shown that employees who believe that rating

their supervisors is a role appropriate activity are more

likely to participate in an upward feedback system than are

those who do not (Avis and Kudisch 2000; Kudisch et al.

2006; Westerman and Rosse 1997). Therefore, it is alsopossible that subordinates who believe that evaluating their

supervisor is a role appropriate activity might also be more

likely to provide honest ratings than those who do not

believe that such ratings are appropriate. Therefore, we

hypothesize the following:

Hypothesis 3: Role appropriateness will relate positively

with employee intentions to provide honest upward feed-

back ratings.

Knowledge of Upward Feedback

Rater motivation theories indicate that knowledge of the

rating system and format by which individuals are rated

influence employee motivation to rate honestly (DeCotiis

and Petit 1978; Mohrman and Lawler 1983; Tziner and

Murphy 1999; Tziner et al. 2001, 2002). For example,

when a rating format is misunderstood, completion time

and rating inaccuracies increase (DeCotiis and Petit 1978;

Harris 1994; Mohrman and Lawler 1983). Similarly,

research has shown that rating quality relates positively to

raters’ level of comfort with the rating process (i.e., Tziner

and Murphy 1999; Tziner et al. 2001, 2002). Similarly,

rater participation research has shown that knowledge of

upward feedback relates positively to employee willing-

ness to provide upward ratings (Avis and Kudisch 2000;

Kudisch et al. 2006).

With respect to upward feedback, several researchers

have suggested that participants’ understanding and

knowledge of the upward feedback process is essential for

it to be effectively implemented (Bracken 1994; Church

and Bracken 1997; Church and Waclawski 1998; Milliman

et al. 1995; Rotolo and Kudisch 1996; Waldman and

Atwater 2001). For example, Church and Waclawski

(1998) noted that people are likely to be uncomfortable inrating their supervisors if they do not understand the con-

cept of upward feedback and Antonioni and Woehr (2001)

suggested that communicating the purpose of, and rationale

behind, upward feedback is an important factor in ensuring

the success of the system. Therefore, we hypothesize the

following:

Hypothesis 4: Knowledge of upward feedback will relate

positively with employee intentions to provide honest

upward feedback ratings.

J Bus Psychol (2008) 22:191–207 193

123



Opportunity to Observe

One advantage of upward feedback is that subordinates

often have a better opportunity to observe some aspects of

their manager’s performance than do those who tradition-

ally provide ratings (Murphy and Cleveland 1995). Several

researchers have suggested that it is essential for the suc-

cess of upward feedback programs to have raters who havehad an adequate opportunity to observe their supervisor’s

performance (Bernardin and Tyler 2001; Bracken and

Timmreck 2001; Farr and Newman 2001). Indeed, Wes-

terman and Rosse (1997) found that employees who

indicated having a greater knowledge of their supervisor’s

performance were more willing to participate in an upward

feedback system than were those people who had less

knowledge of their supervisor’s performance. Therefore,

we hypothesize the following:

Hypothesis 5: Opportunity to observe will relate posi-

tively with employee intentions to provide upwardfeedback ratings.

Perceived Benefits

There is overwhelming support that raters consider the

benefits of rating honestly when rating others (Avis and

Kudisch 2000; Davis 2000; DeCotiis and Petit 1978; Harris

1994; Kudisch et al. 2006; Mohrman and Lawler 1983;

Murphy and Cleveland 1995; Tziner et al. 1998, 2001,

2002; Tziner 1999; Tziner and Murphy 1999; Westerman

and Rosse 1997). Providing an honest appraisal ‘‘is likely

to happen only when the perceived consequences of doingso outweigh the advantages of not doing so’’ (Mohrman

and Lawler 1983, p. 187) and raters consider both the

benefits and risks of rating honestly (Murphy and Cleve-

land 1995). According to Murphy and Cleveland, upward

feedback raters may receive several indirect benefits of

providing honest ratings. For example, providing honest

feedback may be perceived as leading to improved super-

visory performance, enhancing communication and

teamwork, creating a more enjoyable and productive work

environment, and improving organizational functioning.

Based on the above, we hypothesize the following:

Hypothesis 6: The perceived benefits of providing hon-

est ratings will relate positively with employee intentions

to provide honest upward feedback ratings.

Fear of Retaliation

Rater motivation theories specify and research shows that

raters also consider the potential negative consequences of

rating honestly when asked to rate others (e.g., Murphy and

Cleveland 1995). In an upward feedback context, one

potential risk of rating a supervisor honestly is retaliation

on the part of the supervisor (Atwater and Waldman 1998;

Waldman and Atwater 2001). Unlike downward ratings, in

which the subordinate has no legitimate power over the

rater (i.e., his or her supervisor), a supervisor has the power

to retaliate by taking such actions as assigning an unde-sirable job, preventing advancement, or even firing the

rater (Church and Waclawski 1998). Retaliation may also

consist of verbal and non-verbal manifestations of resent-

ment that might exist on the part of the supervisor

(Mohrman and Lawler 1983; Murphy and Cleveland 1995).

Although organizations have attempted to limit employees’

fear of retaliation by keeping rater identities confidential,

many employees still fear that their ratings will be identi-

fied and used against them (Tornow and London 1998;

Waldman and Atwater 2001). Therefore, we hypothesize

the following:

Hypothesis 7: Fear of retaliation for providing honest

ratings will relate negatively with employee intentions to

provide honest upward ratings.

Rater Self-efficacy

Self-efficacy has been defined as an individual’s belief

that he or she is capable of executing a specific course

of action (Bandura 1986). In the performance appraisal

literature, rater self-efficacy has been defined as an

individual’s belief that he or she is capable of rating

another accurately (Mohrman and Lawler 1983; Murphy

and Cleveland 1995; Napier and Latham 1986). Harris

(1994, p. 747) suggested that ‘‘lower self-efficacy will be

associated with decreased rater motivation’’, and Mohr-

man and Lawler (1983) suggested that raters will be

motivated to rate others honestly when they believe that

they have the ability to rate accurately. Research has

supported these views. For example, rater self-efficacy

has been shown to relate positively to the quality and

accuracy of performance appraisal ratings (Mohrman and

Lawler 1983; Tziner et al. 1998, 2001, 2002; Tziner andMurphy 1999). Similarly, rater self-efficacy has also

been shown to relate positively with employee willing-

ness to participate in an upward feedback program (Avis

and Kudisch 2000; Westerman and Rosse 1997). Based

on the above, we hypothesize the following:

Hypothesis 8: Rater self-efficacy will relate positively


back ratings.

194 J Bus Psychol (2008) 22:191–207

123



Hypothesized Relations Between the Independent

Variables and the Hypothesized Mediating Variables of

Mediation

Perceived Benefits and Fear of Retaliation

We hypothesize that perceived benefits andfear of retaliation

will mediate the relationships between cynicism, ratingpurpose, role appropriateness, knowledge of upward feed-

back, and opportunity to observe with employee intentions to

provide honest upward feedback ratings. First, employees

who are cynical towards their organization tend to be skep-

tical regarding the utility of upward feedback (Atwater et al.

2000). Theyare not likely to invest their time and effort into a

program that is perceived as temporary, unimportant, and of

little utility (Church and Waclawski 1998; Kudisch et al.

2006). Similarly, if raters distrust the integrity of the orga-

nization, they may also distrust the organization’s ability to

professionally and ethically implement a successful upward

feedback system and maintain rater confidentiality (Abra-ham 2000; Bracken and Timmreck 2001; Kudisch et al.

2006). As a result, raters with cynical attitudes toward their

organization and the upward feedback process may be

fearful of the consequences of providing candid upward

ratings (i.e., fear retaliation), and consequently, may be

unwilling to provide honest upward assessments. Thus, we

hypothesize the following:

Hypothesis 9a: Cynicism will relate negatively with

perceived benefits.

Hypothesis 9b: Perceived benefits will mediate the

relationship between cynicism and employee intentions toprovide honest upward feedback ratings.

Hypothesis 9c: Cynicism will relate positively with fear

of retaliation.

Hypothesis 9d: Fear of retaliation will mediate the rela-

tionship between cynicism and employee intentions to

provide honest upward feedback ratings.

Second, research indicates that rating purpose may affect

raters’ perceptions of the consequences of providing an

honest upward rating, and, hence, their motivation to do so.

For example, Waldman and Atwater (1998) suggested thatraters view administrative upward feedback systems as more

politically biased and less beneficial than developmentally

based systems. Similarly, because important outcomes are

typically linked to administrative ratings, fear of retaliation

becomes amplified in these settings (Waldman and Atwater

1998). Thus, we hypothesize the following:

Hypothesis 10a: The extent to which upward feedback

ratings are viewed as being administrative in scope will

relate negatively with perceived benefits, whereas the

extent to which upward feedback ratings are viewed as

being developmental in scope will relate positively to

perceive benefits.


relationship between rating purpose (both administrative

and developmental) and employee intentions to provide

honest upward feedback ratings.

Hypothesis 10c: The extent to which upward feedback

ratings are viewed as being administrative in scope will

relate positively with fear of retaliation, whereas the extent

to which upward feedback ratings are viewed as being

developmental in scope will relate negatively to fear of

retaliation.

Hypothesis 10d: Fear of retaliation will mediate the

relationship between rating purpose (both administrative

and developmental) and employee intentions to provide


Third, employees who believe that providing upward

feedback is a role appropriate activity tend to believe that

upward feedback will have beneficial consequences (Wes-

terman and Rosse 1997). Conversely, employees also tend to

perceive fewer risks involved with providing an upward

rating if they view rating their supervisor as role appropriate

activity (Kudisch et al. 2006). Thus, employees who hold

perceptions that providing upward feedback is role appro-

priate should perceive the benefits of providing honest

upward feedback ratings and be less likely to fear supervisor

retaliation. Therefore, we hypothesize the following:

Hypothesis 11a: Role appropriateness will relate posi-tively with perceived benefits.


relationship between role appropriateness and employee


Hypothesis 11c: Role appropriateness will relate nega-

tively with fear of retaliation.




Fourth, when raters understand the content, instructions,and rationale behind the actual rating form, they are more

likely to view the rating process as beneficial and, as a

result, rate more honestly than when they don’t understand

the process (DeCotiis and Petit 1978; Mohrman and Lawler

1983). Conversely, Church and Waclawski (1998) sug-

gested that enhancing raters’ knowledge of the process and

philosophy behind upward feedback is critical in lessening

raters’ fears that negative outcomes, such as retaliation,

will occur. Therefore, we hypothesize the following:

J Bus Psychol (2008) 22:191–207 195

123



Hypothesis 12a: Knowledge of upward feedback will

relate positively with perceived benefits.


relationship between knowledge of upward feedback and

employee intentions to provide honest upward feedback

ratings.

Hypothesis 12c: Knowledge of upward feedback willrelate negatively with fear of retaliation.


relationship between knowledge of upward feedback and


ratings.

Fifth, research shows that raters who have had an ample

opportunity to observe their supervisors’ performance

tended to view the rating process as more beneficial than

did raters who did not (Lehr and Facteau 1992). Con-

versely, employees who had fewer opportunities to observe

their supervisors’ performance were more fearful of retal-iation than were those who had ample opportunities to do

so (Lehr and Facteau 1992). Therefore, we hypothesize the

following:

Hypothesis 13a: Opportunity to observe will relate pos-

itively with perceived benefits.


relationship between opportunity to observe and employee


Hypothesis 13c: Opportunity to observe will relate neg-

atively with fear of retaliation.


relationship between and employee intentions to provide


Rater Self-efficacy

We hypothesize that rater self-efficacy will mediate the

relationships between role-appropriateness, knowledge of

upward feedback, and opportunity to observe with

employee intentions to provide honest upward feedback ratings. People who view upward feedback as a role

appropriate activity, understand the process and content of

upward feedback, and have ample opportunity to observe

their supervisor, are likely to believe in their ability to

provide honest upward feedback ratings, and as a result,

provide honest upward feedback ratings (DeCotiis and Petit

1978; Harris 1994; Heneman and Wexley 1983; Kudisch

et al. 2006). Based on the above, we hypothesize the

following:

Hypothesis 14a: Role appropriateness will relate posi-

tively with self-efficacy.

Hypothesis 14b: Rater self-efficacy will mediate the



Hypothesis 14c: Knowledge of upward feedback will

relate positively with self-efficacy.

Hypothesis 14d: Rater self-efficacy will mediate the

relationship knowledge of upward feedback and employee


Hypothesis 14e: Opportunity to observe will relate pos-

itively with self-efficacy.

Hypothesis 14f: Rater self-efficacy will mediate the

relationship between opportunity to observe and employee


Participants

Questionnaires were disseminated to all employees of a

behavioral health center (n = 288). Two-hundred and

twenty-five completed surveys were returned. However, 22

surveys were dropped from further consideration because

all items were answered with the same response (e.g.,

answer ‘‘strongly agree’’ to every item), including reverse

scored items, resulting in a net 70% response rate. Sixty-

eight percent of respondents (n = 137) were Caucasian,

32% (n = 65) were African-American, and 0.5% (n = 1)

were of Hispanic or Asian descent. Sixty-six percent of

respondents were women (65.5%). Average job tenure was5 years ( M = 5.16, SD = 4.47). Participants’ educational

levels were as follows: high school degree (23.3%,

n = 47), technical degree (4.0%, n = 8), associates degree

(14.4%, n = 29), bachelors degree (25.7%, n = 52), mas-

ters degree (25.2%, n = 51), and doctoral degree (7.4%,

n = 15). Participants also represented a wide variety of

occupational areas within the behavioral health division:

Alcohol and Drug (19.2%), Administration (6.4%),

Assessment/Business Office (4.4%), Adult Inpatient

(11.8%), Adolescent Day Treatment (6.9%), Child and

Adolescent Inpatient (7.4%), Eating Disorders (6.9%),

Food and Nutrition (6.4%), Outreach/Marketing (3.9%),

Outpatient Counseling (6.4%), Outpatient Psychiatry

(13.8%), Transportation (3.0%), and other (3.5%). Finally,

28% of respondents were in supervisory positions.

Procedure

Administration of the questionnaire was part of a study

to determine the feasibility of implementing an upward

196 J Bus Psychol (2008) 22:191–207

123



feedback system at the behavioral health center. The first

author met individually with center administrators to

solicit support for the data collection process. The

director of the center sent out emails to lower level

administrators and supervisors encouraging participation

in and support for the data collection process. Although

completion of the survey was voluntary, it was strongly

encouraged by the director.Each survey packet included a cover letter, question-

naire, and a sealable interoffice envelope preaddressed to

the first author. The cover letter introduced the

researcher, provided an explanation as to the purpose of

the study, described the upward feedback process, and

provided instructions for returning the completed

surveys.

Surveys were distributed in the following four ways: (a)

at group data collection meetings, (b) hand delivered by the

unit liaison, (c) hand delivered by the first author, or (d)

placed in the participant’s interoffice mailbox. Approxi-

mately 85% of the surveys were collected using thefollowing procedure: The first author identified one or two

liaisons from most of the center’s departmental units to

assist in the data collection process. Data collection

meetings were then scheduled at which the first author gave

each attendee a survey packet. Attendees were asked to

read the cover letter, complete the survey, seal the enve-

lope, and give their completed surveys directly to the first

author. The names of participants who participated in data

collection efforts during these meetings were recorded so

that each departmental liaison could distribute surveys to

those employees who were absent from the meetings.

Liaisons received weekly reminders to distribute and col-

lect surveys from those who did not attend the group

meetings. Liaisons returned completed surveys to the sec-

ond author via interoffice mail.

Approximately 15% of participants received surveys

directly from the first author or had surveys placed in their

office mailboxes. The cover letter received by these indi-

viduals included instructions to send the completed and

sealed surveys directly to the first author via interoffice

mail. Email and/or personal reminders to complete the

surveys were periodically sent to those persons who did not

receive their surveys in group meetings or from their

liaisons.

Measures

The survey included a total of 31 items. All scale items

were rated using a 5-point Likert-type scale ranging from 1

(strongly disagree) to 5 (strongly agree). Survey items

were randomized and approximately 30% of the items were

reverse-coded.

Predictor Variables

Cynicism toward top management was measured using four

items adapted from Vance, Brooks, and Tesluk (1995: e.g.,

‘‘Efforts to make improvements are recognized within

[center’s name]’’ [reverse scored]). Cynicism toward

upward feedback was measured with one item adapted

from Murphy and Cleveland (1995: ‘‘I doubt that [center’sname] upper administration really believes in the impor-

tance of upward feedback’’) and two items adapted from

Kudisch et al. (2006: e.g., ‘‘I trust that [center’s name]

upper administration would commit the time and resources

that are necessary to successfully implement an upward

feedback system [reverse scored]’’; ‘‘I trust that [center’s

name] upper administration would take the steps necessary

to guarantee the anonymity and confidentiality of upward

raters’’ [reverse scored]).

Rating purpose was measured using two items that were

adapted from Avis and Kudisch (2000). One item measured

the extent to which the purpose was viewed as develop-mental (‘‘If an upward feedback system were implemented at

[center name], I trust that [center name] would use the

upward ratings exclusively for the purpose of helping

supervisors develop their management skills’’), and one item

measured the extent to which the purpose was viewed as

administrative (‘‘If an upward feedback system were

implemented at [center name], I believe that [center name]

administrators would use the upward ratings to make deci-

sions about supervisors’ promotion, retention, and/or pay’’).

A three-item measure developed by Avis and Kudisch

(2000) was used to measure role appropriateness (e.g., ‘‘It

is not my place to rate the performance of my supervisor’’

[reverse scored]). Knowledge of upward feedback was

measured using three items adapted from Kudisch et al.

(2006: e.g., ‘‘Performance information that is collected

from multiple raters will provide a more comprehensive

and accurate view of performance than will information

that is collected only from a single rater’’). Opportunity to

observe supervisory behavior was measured using three

items adapted from Lehr and Facteau (1992; e.g., ‘‘I

observe important aspects of my immediate supervisor’s

job performance’’).

Perceived benefits was measured using a 10-item scale

adapted from Kudisch et al. (2006). Five items asked

employees to assess their own perception of the perceived

benefitsto the organization if the survey respondent provided

an honest upward feedback rating (e.g., ‘‘If I provided honest

performance feedback to my immediate supervisor, I believe

that open communication in my department would

increase’’) and five items asked employees to assess their

perceptions of benefits if others’ provided honest upward

feedback ratings (e.g., ‘‘If others at [center’s name] provided

honest performance feedback to their immediate

J Bus Psychol (2008) 22:191–207 197

123



supervisors, I believe that open communication at [center

name] would increase’’). Fear of retaliation was measured

using a three-item scale adapted from Kudisch et al. (2006;

e.g., ‘‘I will be able to provide candid feedback to my

supervisor without fear of retaliation’’ [reverse scored]).

Rater self-efficacy was measured using a three item scale

adapted from Avis and Kudisch(2000; e.g., ‘‘I have the skills

and abilities that are necessary to accurately rate my imme-diate supervisor’s performance’’).

Employee Intentions to Provide Honest Upward Feedback

Ratings

A 2-item scale adapted from Davis (2000) was used to

assess employee intent to provide honest upward feedback

ratings (i.e., If an upward feedback system was imple-

mented at [center name] and I was asked to rate the

performance of my immediate supervisor : (a) ‘‘I would

share my honest opinions about my immediate supervisor;’’(b) ‘‘I would provide ratings that reflect what I really think

about my immediate supervisor’s performance’’).

Results

Factor Analyses

Exploratory factor analysis using oblimin rotation was per-

formed on the predictor items. Six factors were derived,

explaining 64% of the variance. The first factor explained

33%of the variance andincluded all of the perceived benefitsitems. The second factor explained 20% of the variance and

included all of the cynicism items, the fear of retaliation

items andthe purpose—developmental item. Thethird factor

explained 7% of the variance and included the role appro-

priateness and knowledge of upward feedback items. The

three other factors consisted of the opportunity to observe

items, self-efficacy items, and the purpose—administrative

item, respectively. Because the cynicism toward top man-

agement and cynicism toward upward feedback loaded on a

single factor and represent similar constructs, these items

were combined into a single scale we refer to as cynicism.

However, although role appropriateness and knowledge of upward feedback items also loaded on a single factor,

because we believe that they represent different constructs

we did not combine these items.

Test for Common Method Variance

Because we were concerned that method bias may have

played a role in our analyses, we examined our data

using Harmon’s one-factor test (see Podsakoff and Organ

1986). The assumption of this test is that if common

method variance was present, then (a) one general factor

would account for a large portion of the covariance

among the measured variables or (b) a single factor

would be found. However, our analysis indicated that

three factors with eigenvalues over 1.0 explained 69% of

the variance among scores on our measures. Moreover,as shown in Table 1, we found both positive and nega-

tive statistically significant correlations as well as several

statistically non-significant correlations. Thus, although

common method variance was not completely ruled out,

our findings indicate that it was not a major cause for

concern in our study.

Demographic Variables

We examined the relationships between our sample char-acteristics (i.e., age, race, sex, and education) and the study

variables. Few statistically significant differences were

found. Age correlated positively with perceived benefits,

r = 0.23, p\ 0.01 and negatively with cynicism, r = -

0.22, p\ 0.01; Men ( M = 3.81) reported higher levels of

perceived benefits than women ( M = 3.52), t (200) = 2.56,

p\ 0.05; and Caucasians reported higher levels of

knowledge of upward feedback ( M = 4.10) and lower

levels of perceived benefits ( M = 3.53) than African-

Americans ( M s = 3.70 and 3.81, respectively), t s(200) =

2.47 and -4.53, ps\ 0.05. We also found statistically

significant Analyses of Variance with respect to educationand (a) role appropriateness, F (5, 196) = 2.80, p\ 0.05,

and (b) knowledge of upward feedback, F (5, 196) = 2.53,

p\ 0.05. Follow-up Tukey’s post hoc tests indicated that

individuals with high school degree’s reported lower levels

of role appropriateness ( M = 3.74) and knowledge of

upward feedback ( M = 3.77) than people with Master’s

degrees ( M s = 4.17 and 4.17, respectively). None of the

sample characteristics related to employee intentions to

provide honest upward feedback ratings were statistically

significant.

Descriptive Statistics, Scale Reliabilities, and

Correlation Coefficients

Coefficient alpha estimates of reliability were computed on

all study variables. Table 1 shows scale means, standard

deviations, coefficient alpha estimates of reliabilities for

each variable, and correlation coefficients among the pre-

dictor and criterion variables.

198 J Bus Psychol (2008) 22:191–207

123



Hypothesized Relations between the Independent

Variables and Hypothesized Mediating Variables with

the Dependent Variable

Hypothesis 1 stated that cynicism would relate negatively


back ratings. This hypothesis was supported, r = -0.22,

p\ 0.01. Hypotheses 2a and 2b stated that rating pur-

pose—developmental would relate positively and rating

purpose—administrative would relate negatively, respec-

tively, with employee intentions to provide honest upward

feedback ratings. However, neither variable correlated

statistically significantly with employee intentions to pro-

vide honest upward feedback ratings. Thus, Hypothesis 2

was not supported.We were concerned that the lack of support for

Hypothesis 2 might have been the result of range restric-

tion. However, as shown in Table 1, respondents’ scores on

the two rating purpose items fell near the mid-point of the

five point scale with standard deviations actually higher

than those reported for the other predictor variables (rating

purpose—administrative: M = 2.94; SD = 1.02; rating

purpose—developmental: M = 3.53; SD = 0.88). Thus,

range restriction does not appear to have affected statistical

conclusion validity.

However, it is possible that slight differences in the

wording of the items might have accounted for this unex-pected finding. For rating purpose—developmental, we

asked ‘‘…I trust that [the organization] would use the

upward feedback ratings exclusively for the purpose of

helping supervisors develop their managerial skills’’.

Conversely, for rating purpose—administrative, we asked

‘‘…I believe that [the organization’s administrators] would

use the upward ratings to make decisions about supervi-

sors’ promotion, retention, and/or pay’’ might have

accounted for the unexpected findings. Having trust and

believing are two different cognitive points of view and as

a result, affected individuals’ responses to the two items

differently.

1

Hypotheses 3 and 4 stated that role appropriateness and

knowledge of upward feedback, respectively, would relate

positively with employee intentions to provide honest

upward feedback ratings. These hypotheses were sup-

ported: Role appropriateness and knowledge of upward

feedback both correlated positively with employee inten-

tions to provide honest upward feedback ratings, r s = 0.35

and 0.35, ps\ 0.01, respectively. Hypothesis 5 stated that

opportunity to observe would relate positively with


ratings. This hypothesis was supported, r = 0.27,

p\ 0.01.Hypotheses 6–8 stated that perceived benefits and rater

self-efficacy would relate positively and fear of retaliation

would relate negatively with employee intentions to pro-

vide honest upward feedback ratings, respectively. These

hypotheses were supported, r = 0.38, p\ 0.01 for per-

ceived benefits; r = 0.38; p\ 0.01 for rater self-efficacy;

and r = -026, p\ 0.01 for fear of retaliation.

Hypothesized Relations Between the Independent

Variables and Hypothesized Mediating Variables

Hypotheses 9a and 9c stated that cynicism would relate

negativelywith perceived benefitsand positively with fear of

retaliation, respectively. Both hypotheses were supported:

r = -0.55, p\ 0.01 and r = 0.69, p\ 0.01. Hypothesis

10a stated that rating purpose—administrative will relate

negatively with perceived benefits, whereas rating pur-

pose—developmental will relate positively with perceive

Table 1 Zero-order correlations among study variables and scale reliabilities (n = 203)

Variable M SD 1 2 3 4 5 6 7 8 9 10

1. Cynicism 2.68 0.78 (0.86)

2. Purpose: Administrative 2.94 1.02 -0.05 –

3. Purpose: Developmental 3.53 0.88 -0.53** -0.02 –

4. Role Appropriateness 3.98 0.63 -0.15* -0.11* 0.21** (0.71)

5. Knowledge of Up. Fdbk. 3.98 0.62 -0.16* -0.07 0.21** 0.45** (0.55)

6. Opportunity to observe 3.67 0.74 -0.19* 0.03 0.26** 0.27** 0.17* (0.60)

7. Perceived benefits 3.62 0.76 -0.54** 0.07 0.40** 0.21** 0.29** 0.31** (0.97)

8. Fear of retaliation 2.51 0.93 0.69** 0.08 -0.43** -0.19** -0.16* -0.25** -0.39** (0.85)

9. Rater self-efficacy 3.94 0.63 -0.17** 0.03 0.25** 0.49** 0.36** 0.46** 0.37** -0.20** (0.70)

10. Intentions to rate 4.12 0.67 -0.22** -0.02 0.10 0.35** 0.35** 0.27** 0.38** -0.26** 0.38** (0.70)

Note: Scale values range from 1 = ‘‘Strongly Disagree’’ to 5 = ‘‘Strongly Agree’’; Coefficient alpha estimates of reliability are in parentheses

* p\ 0.05, one-tailed; ** p\ 0.01, one-tailed

1 We thank an anonymous reviewer for this possible explanation.

J Bus Psychol (2008) 22:191–207 199

123



benefits and Hypothesis 10c stated that rating purpose—

administrative will relate positively with fear of retaliation,

whereas rating purpose—developmental will relate nega-

tively with fear of retaliation. Hypotheses 10a and 10c were

supported only with respect to rating purpose developmen-

tal, r = 0.40, p\ 0.01 for perceived benefits and r = -0.43,

p\ 0.01 for fear of retaliation. Rating purpose—adminis-

trative did not correlate statistically significantly with eitherperceived benefits or fear of retaliation.

Hypotheses 11a and 12a stated that role appropriateness

and knowledge of upward feedback, respectively, would

correlate positively with perceived benefits, whereas

Hypotheses 11c and 12c stated that role appropriateness

and knowledge of upward feedback, respectively, would

relate negatively with fear of retaliation. These hypotheses

were supported: Role appropriateness and knowledge of

upward feedback both correlated positively with perceived

benefits, r s = 0.21 and 0.19, ps\ 0.01, respectively, and

negatively with fear of retaliation, r s = -0.19 and -0.16,

ps\ 0.01, respectively.Hypotheses 13a and 13c stated that opportunity to

observe will relate positively with perceived benefits and

negatively with fear of retaliation, respectively. These

hypotheses were supported, r = 0.31, p\ 0.01 for per-

ceived benefits and -0.25, p\ 0.01 for fear of retaliation.

Hypotheses 14a, 14c, 14e stated that role appropriate-

ness, knowledge of upward feedback, and opportunity to

observe, respectively, would all relate positively with rater

self-efficacy. These hypotheses were supported: r s = 0.49,

0.36, and 0.46, ps\ 0.01, for role appropriateness,

knowledge of upward feedback, and opportunity to

observe, respectively.

Mediation

According to Baron and Kenny (1986), four steps need to

be satisfied to determine whether a relationship between an

independent variable and a dependent variable is mediated

by a third variable. First, the independent variable must

relate statistically significantly with the dependent variable

(Model 1). Second, the hypothesized mediating variable

must relate statistically significantly with independent

variable (Model 2). Third, the mediating variable must

relate statistically significantly with the dependent variable.

Fourth, for complete mediation, the relationship between

the independent variable and the dependent variable must

become statistical non-significant when controlling for the

variance of the mediating variable on the dependent vari-

able (Model 3). Otherwise, partial mediation occurs when

the path coefficient between the independent variable and

dependent variable is substantially reduced and the pro-

portion of the total effect that is accounted for by the

indirect effect (via the hypothesized mediator) differs sta-

tistically significantly from zero (MacKinnon et al. 1995;

Sobel 1986). We followed the procedures outlined by

Mackinnon et al. (1995) in order to calculate the proportion

of the total effect accounted for by the hypothesized

mediating variable. Specifically, we calculated the pro-

portion of the indirect effect to the total effect:

ab=ðs0 þ abÞ

where a is the regression coefficient between the inde-

pendent variable and the mediating variable, b is the

regression coefficient between the mediating variable and

the dependent variable, and s0 is the regression coefficient

between the independent variable and the dependent vari-

able, controlling for the mediating variable. Next, we used

an on-line interactive tool developed by Preacher and

Leonardelli (2001, March) based on Sobel’s (1986) method

for calculating whether the indirect effect differed statis-

tically significantly from zero.

Table 2 shows the results of the regression analyses forall of the models tested. Note that with the exception of

rating purpose—administrative and rating purpose—

developmental, neither of which related statistically sig-

nificantly to either the dependent or hypothesized

mediating variables, all of the tests of Model 1 (dependent

variable on the independent variable) and Model 2

(hypothesized mediating variable on the independent var-

iable) were statistically significant. Thus, with the

exception of the rating purpose measures, the first two

conditions of mediation were satisfied. Note also that in all

of the tests of Model 3 (dependent variable regressed on

both the independent variable and hypothesized mediatingvariable), the hypothesized mediating variable related sta-

tistically significantly with the dependent variable. Thus,

the third condition was satisfied for all of the variables.

Finally, as shown in Table 2, the path coefficients between

the independent variables and dependent variable were

substantially reduced in all of the equations. In all cases,

the proportion of the total effect accounted for by the

indirect effect was statistically significant. A discussion of

each test with respect to the hypothesized mediating effects

is presented in the subsequent paragraphs.

Hypotheses 9b and 9d stated that perceived benefits and

fear of retaliation, respectively, would mediate the rela-tionship between cynicism and employee intentions to

provide honest upward feedback ratings. One hundred

percent of the total effect between cynicism and employee

intentions to provide honest upward feedback was medi-

ated by perceived benefits, t = -4.26, p\ 0.05). Similarly,

100% of the total effect between cynicism and employee

intentions to provide honest upward feedback was medi-

ated by fear of retaliation, t = -2.39, p\ 0.05. Thus,

Hypotheses 9b and 9d were supported.

200 J Bus Psychol (2008) 22:191–207

123



Hypotheses 10b and 10d indicated that the relationship

between rating purpose and employee intentions to provide

honest upward feedback ratings would be mediated by

perceived benefits and fear of retaliation, respectively.

However, neither variable related statistically significantly

with the dependent variable in Model 1. Thus, the first

condition necessary for mediation was not satisfied. As a

result, Hypotheses 10b and 10d were not supported.

Hypotheses 11b and 12b indicated that the relationships

between role appropriateness and knowledge of upward

feedback and employee intentions to provide honest

upward feedback ratings would be mediated by perceived

benefits. As shown in Table 2, 23% of the total effect

between role appropriateness and employee intentions to

provide honest upward feedback ratings was mediated by

perceived benefits, t = 2.62, p\ 0.05, and 14% of the total

Table 2 Hierarchical regression analyses used to assess whether (a) perceived benefits, (b) fear of retaliation, and (c) rater self-efficacy mediated

the relationships between cynicism, rating purpose, understanding upward feedback, and opportunity to observe with employee intentions to

provide honest upward feedback ratings (n = 203)

Independent variable Model 1 Model 2 Model 3 % Mediationa t -testb

r 2

B SE b r 2

B SE b sr 2

B SE b Mediating variable

Cynicism 0.04* 0.19 0.06 -0.20* 0.00 0.00 0.07 -0.01 100 -4.26*

Perceived benefits 0.30* -0.53 0.06 -0.54* 0.10* 0.34 0.07 0.38*Cynicism 0.04* 0.19 0.06 -0.20* 0.00 0.00 0.09 -0.03 100 -2.39*

Fear of retaliation 0.47* 0.82 0.06 0.69* 0.03* -0.17 0.07 -0.24*

Rating purpose—Adm 0.00 -0.01 0.05 -0.01 0.00 -0.03 0.04 -0.03 0 0.99

Perceived benefits 0.00 0.05 0.05 0.07 0.14* 0.34 0.05 0.38*

Rating purpose—Adm 0.00 -0.01 0.05 -0.01 0.00 0.00 0.05 0.00 0 -1.09

Fear of retaliation 0.01 0.08 0.07 0.08 0.07* -0.19 0.05 -0.26*

Rating purpose—devel. 0.01 0.07 0.05 0.10 0.00 -0.05 0.05 -0.07 169 4.12*

Perceived benefits 0.16* 0.34 0 .06 0.40* 0 .14* 0.36 0 .06 -0.41*

Rating purpose—devel. 0.01 0.07 0.05 0.10 0.00 -0.02 0.06 -0.02 127 2.97*

Fear of retaliation 0.19* -0.46 0.07 -0.43* 0.06* -0.20 0.06 -0.27*

Role appropriateness 0.12* 0.36 0.07 0.35* 0.08* 0.29 0.07 0.28* 23 2.62

Perceived benefits 0.04* 0.25 0.08 0.21* 0.11* 0.29 0.06 0.32*

Role appropriateness 0.12* 0.36 0.07 0.35* 0.10* 0.33 0.07 0.31* 14 -2.18*


Knowledge of Up. Fdbk. 0.11* 0.35 0.07 0.33* 0.11* 0.26 0.07 0.24* 30 3.11*

Perceived benefits 0.08* 0.35 0.08 0.29* 0.15* 0.28 0.06 0.32

Knowledge of Up. Fdbk. 0.11* 0.35 0.07 0.33* 0.09* 0.31 0.07 0.29* 12 1.22


Opportunity to observe 0.07* 0.24 0.06 0.27* 0.02* 0.15 0.06 0.16* 38 3.22*

Perceived benefits 0.10* 0.32 0.07 0.31* 0.10* 0.29 0.06 0.33*

Opportunity to observe 0.07* 0.24 0.06 0.27* 0.04* 0.19 0.06 0.21* 20 2.29*


Understanding of UF 0.16* 0.50 0.08 0.39* 0.05* 0.34 0.09 0.27* 30 3.01* Rater self-efficacy 0.20 0.54 0.08 0.54 0.06* 0.27 0.08 0.25*

Opportunity to observe 0.07* 0.24 0.06 0.27* 0.01 0.10 0.07 0.11 58 3.90*

Rater self-efficacy 0.21 0.39 0.05 0.46 0.09* 0.36 0.08 0.33*

Note: The predictor is listed in regular font. The mediator is listed in italic font. Model 1: The dependent variable is regressed on the independent

variable. Model 2: The hypothesized mediating variable is regressed on the independent variable. Model 3: The dependent variable is regressed

on both the independent and hypothesized mediating variablesa % Indirect = Percentage of total effect that is mediated (ab /(s0

+ ab)), where a = unstandardized regression coefficient of the independent

variable on the mediating variable, b = unstandardized regression coefficient of the mediating variable on the dependent variable, and

s0= unstandardized regression coefficient of the independent variable on the dependent variable, controlling for the mediating variable

bt-test examining whether the indirect effect (mediation) differs statistically significantly from zero (Preacher and Leonardelli 2001; Sobel

1986)

* p\ 0.05

J Bus Psychol (2008) 22:191–207 201

123



effect between role appropriateness and employee inten-

tions to provide honest upward feedback ratings was

mediated by fear of retaliation, t = -2.18, p\ 0.05. Thus,

Hypotheses 11b and 12b were supported. Similarly,

Hypotheses 11d and 12d indicated that the relationships

between role appropriateness and knowledge of upward

feedback and employee intentions to provide honest

upward feedback ratings would be mediated by fear of retaliation. As shown in Table 2, 30% of the total effect

between role appropriateness and employee intentions to


perceived benefits, t = 3.11, p\ 0.05. Thus, Hypothesis

11d was supported. However, although 12% of the total

effect between role appropriateness and employee inten-

tions to provide honest upward feedback ratings was

mediated by fear of retaliation, this result was not statis-

tically significant, t = -1.22, p[ 0.05. Thus, Hypotheses

12d was not supported.

Hypotheses 13b and 13d indicated that the relationship

between opportunity to observe and employee intentions toprovide honest upward feedback ratings would be mediated

by perceived benefits and fear of retaliation, respectively.

Thirty-eight percent of the total effect between opportunity

to observe and employee intentions to provide honest

upward feedback ratings was mediated by perceived ben-

efits, t = 3.22, p\ 0.05. Similarly, 20% of the total effect

between opportunity to observe and employee intentions to

provide honest upward feedback ratings was mediated fear

of retaliation, t = 2.29, p\ 0.05. Thus, Hypotheses 13b

and 13d were supported.

Hypotheses 14b and 14d stated that rater self-efficacy

would mediate the relationships between role appropriate-

ness and knowledge of upward feedback, respectively.

Thirty percent of the total effect between understanding

upward feedback and employee intentions to provide

honest upward feedback ratings was mediated by rater self-

efficacy, t = 3.01, p\ 0.05. Thus, Hypotheses 14b and

14d were supported.

Finally, Hypothesis 14f stated that rater self-efficacy

would mediate the relationship between opportunity to

observe and employee intentions to provide honest upward

feedback ratings. Fifty-eight percent of the total effect

between opportunity to observe and employee intentions to


rater self-efficacy, t = 3.90, p\ 0.05. Thus, Hypothesis

14f was supported.

Structural Equation Modeling

Structural equation modeling was used to examine the

overall fit of the hypothesized model presented in Fig. 1

and the relative importance of the various predictor and

mediating variables. Testing was based on the covariance

matrix and used maximum likelihood estimation as

implemented in LISREL VIII (Joreskog and Sorbom

1992). For predictor scales that had more than five items,

two or more subscales were created by randomly assigning

three or more items to each subscale. The unreliability of

variables based on single indicators was computed and

specified by fixing the error term for the indicator to oneminus the reliability of the measure (estimated using

Cronbach’s coefficient alpha) times the variance of scores

on the measure (1 - r xx)*s2. Several measures of model fit

were assessed: v2, comparative fit index (CFI), goodness-

of-fit index (GFI), adjusted-goodness-of-fit index (AGFI),

root-mean-square error of approximation (RMSEA), and

standardized mean square residual (RMSR).

Structural equation modeling results indicated a perfect

fit between the hypothesized model and the data

(v2 = 246.61, df = 595, p = 1.00; RMSEA = 0.00;

GFI = 1.00; CFI = 1.00; NFI = 1.00; RMSR = 0.00).

Standardized and unstandardized path coefficients for thestructural model are presented in Fig. 2. When statistically

controlling for the variability explained for by the other

mediators, perceived benefits (b = 0.20, p\ 0.05) and

rater self-efficacy (b = 0.43, p\ 0.05) accounted for

unique variance in self ratings of employee intentions to

provide honest upward feedback ratings.

When statistically controlling the variability accounted

for by the other predictors, employee cynicism explained

unique variance in perceived benefits (b = -0.49,

p\ 0.05) and fear of retaliation (b = 0.81, p\ 0.05);

opportunity to observe explained unique variance in per-

ceived benefits (b = 0.27, p\ 0.05), fear of retaliation

(b = -0.25, p\ 0.05), and rater self-efficacy (b = 0.62,

p\ 0.05); and understanding of upward feedback explained

unique variance in rater self-efficacy (b = 0.40, p\ 0.05).

In summary, the structural equation analysis indicated that

the most important predictors of employee intentions to

provide honest upward feedback ratings were cynicism,

understanding upward feedback, and opportunity to observe

mediated by perceived benefits and rater self-efficacy.

Discussion

Although multisource feedback has become a powerful and

popular performance appraisal tool (Bracken et al. 2001;

Edwards and Ewen 1996, 2001; Tornow and Tornow

2001), its utility may be limited by raters who intentionally

distort or provide inaccurate ratings (Bernardin and Tyler

2001; Bracken and Timmreck 2001). Based on extant

models of rater motivation (DeCotiis and Petit 1978; Harris

1994; Mohrman and Lawler 1983; Murphy and Cleveland

1995), we hypothesized that cynicism, rating purpose,

202 J Bus Psychol (2008) 22:191–207

123



knowledge of upward feedback, role appropriateness,

opportunity to observe, perceived benefits, fear of retalia-

tion, and rater self-efficacy would correlate with employee


Moreover, we hypothesized that perceived benefits, fear of

retaliation, and rater self-efficacy would mediate the rela-

tionships between the other predictor variables and

employee intentions to provide upward feedback ratings.

In general, our findings supported our hypotheses andsuggest that extant rater motivation models generalize to an

upward feedback context. Specifically, cynicism, knowl-

edge of upward feedback, role appropriateness, and

opportunity to observe, all correlated as hypothesized with


ratings, mediated in part or in full by perceived benefits,

fear of retaliation, and rater self-efficacy.

One interesting finding is that despite common practice

that suggests that multi-rater feedback should be used

primarily for developmental and not administrative pur-

poses (Dalton 1998; Lepsinger and Lucia 1997; Van Velsor

1998), rating purpose did not influence employee intentions

to provide honest ratings. It is typically assumed that raters

are likely to distort their ratings for both political and

personal gain, often retaliating against supervisors and

coworkers with whom they do not get along or inflating

ratings of those they like, especially when raters know that

their ratings can affect administrative decisions regarding

their supervisors or peers. Our findings are encouraging in

that they suggest raters desire to be fair when rating others,

regardless of how multisource ratings are to be used.

We also note that although fear of retaliation correlated as

predicted with employee intentions to provide honest upward

feedback ratings, it did not account for unique variance over

and above that explained by perceived benefits and rater self-

efficacy in the structural model. However, the lack of statis-

tical significance for fear of retaliation as a mediator was in

part due to the strong relationship between organizational

cynicism and fear of retaliation (r = 0.69). That is, multico-

linearity between the two variables might have attenuated thepotential mediating effect of fear of retaliation. Most likely,

employees who held cynical attitudes about the motives of

management were also those who were skeptical that their

supervisors would not retaliate against them for providing

honest, and potentially negative, feedback. Our findings also

indicated that althoughemployees were informed that upward

feedback would be used for developmental purposes, those

who were most cynical of upward feedback were also more

likely to believe that it wouldbe not be usedfor developmental

purposes (r = -0.53 between rating purpose—develop-

mental and cynicism). Thus, in general, it appears that

employees who held littletrust in the manner in whichupward

feedback would be used not only feared retaliation, but also

believed that upward feedback would not be used for its

intended purpose.

Practical Implications

Church and Waclawski (2001, p. 93) stated the following:

‘‘What is needed now in the field is a detailed set of

Perceived

Benefits

Cynicism

Knowledge of

Upward Fdbk

Opportunity to

Observe

Purpose:

Developmental

Employee Intentions

to Provide Honest

Upward Feedback

Ratings: Self

Purpose:

Administrative

-.48* (-.65)

.79* (1.35)

-.03 (-.02)

.01 (.00)

.03 (.02)

-.16 (.28)

-.19 (-.30)

.10 (.09)

.27* (.45)

-.29* (-.59)

.62* (.76)

.21* (.20)

-.07 (-.05)

.43* (.54)

-.12 (-.07)

Role

Appropriateness

.38 (.81)

.32 (.40)

Rater

Self-

Efficacy

.38 (.81)

Fear of

Retaliation

Fig. 2 Structural model

showing standardized and

unstandardized (in parentheses)

path coefficients among

measured variables,

v2 = 384.15, df = 222,

p\ 0.01; RMSEA = 0.06;

GFI = 0.88; CFI = 0.94;

NFI = 0.88; RMSR = 0.04.

Although not shown,

correlations among the predictor

variables and among the

mediating variables were

estimated. * p\ 0.05

J Bus Psychol (2008) 22:191–207 203

123



criteria, recommendations for best practice, and alterna-

tives with valences that consulting practitioners can use for

decision-making around multisource feedback implemen-

tations’’. Although some suggestions for soliciting honest

rater responses are given in The Handbook of Multisource

Feedback (i.e., use for developmental purposes only,

gather anonymous ratings), the number of practical sug-

gestions provided in the literature is small relative to thenegative impact that dishonest ratings have on the validity

of multisource feedback systems. Currently, practitioners

attempting to encourage honest ratings are left to rely more

on common sense and guesswork than research-backed

guidelines.

Our findings provide several practical tips for soliciting

more honest ratings. Most importantly, organizations

should build employee trust and minimize cynical atti-

tudes, perhaps by having senior-level executives

demonstrate and articulate visible and enthusiastic support

for the upward feedback or multisource process as sug-

gested by Church (1995), Milliman et al. (1995), Pollack and Pollack (1996), and Wimer and Nowack (1998).

However, communication supportive of the process is a

necessary but not sufficient condition when implementing a

multisource feedback system. Our findings also indicate

that employees are likely to provide honest ratings when

they do not fear retaliation by their supervisors or top

management. Thus, it is critical that organizations imple-

ment strategies to protect rater confidentiality, discourage

retaliation, and provide a grievance process to report any

instances of retaliation that might occur. Moreover, our

findings suggest that organizations develop training pro-

grams that describe multisource feedback and articulate

how multisource feedback is beneficial for both organiza-

tional productivity and for developing employees. Training

programs should also include strategies on how to effec-

tively observe and rate supervisor behavior.

Although training programs designed to communicate

the what, why, and how of multisource feedback can

improve rating honesty and although implementing strate-

gies that ensure confidentiality of responses can mitigate

fear of retaliation, and although raters may ideally intend to

rater others honestly regardless of the purpose of the rat-

ings, organizational political and personal factors may still

affect rater honesty. One way to minimize the potential for

dishonest ratings is to incorporate validity and reverse-

coded items into multisource feedback questionnaires. For

example, questionnaires may include ‘‘Yes/No’’ items such

as the following: (a) ‘‘Do you fear that the ratee and/or

others will retaliate against you for providing completely

honest responses to this survey?’’ (b) ‘‘Do you have enough

knowledge of the ratee’s work behavior to accurately

complete this survey?’’ and (c) ‘‘Do your responses on this

survey reflect your honest opinions about the ratee’s

behavior at work?’’ A ‘‘no’’ response on any of the above

three items would suggest less-than-honest-and-accurate

responses. Further, multisource feedback surveys should

include at least one item designed to detect socially

desirable ratings (e.g., ‘‘My supervisor is perfect’’).

Reverse-coded items are also needed to reduce method

bias, whether due to an unmotivated rater, careless, or halo

errors (e.g., marking ‘‘Strongly Agree’’ for every item on a5-point Likert-type scale). Incorporating the aforemen-

tioned issues into survey design would provide

practitioners with a method of detecting less-than-honest

response sets, whether intentional or unintentional. Practi-

tioners could then exclude questionable ratings from the

multisource feedback report.

In summary, organizations should attempt to create a

rating context in which persons trust the organization,

understand the upward feedback process, have ample

opportunity to observe their supervisor’s performance,

perceive the process to be beneficial, have little fear of

retaliation, and have the ability to rate accurately. Also, therating form should include items that can assess the validity

of individuals responses, identify social desirable response

sets, and detect less than honest and/or accurate response

sets.

Strengths and Limitations

A couple of strengths of the current study deserve

mention. First, our overall response rate was 70%, sug-

gesting that the relationships observed in our sample

adequately represented the views the employees of the

behavioral health center from which we drew our sam-

ple. Second, consistent with Longnecker’s (1986)

recommendation for more field research on rater moti-

vation, this study was conducted in an organizational

setting, thereby enhancing the external conclusion

validity of both our findings and the theoretical frame-

works from which our hypotheses were drawn.

This study also had a few limitations. One limitation is

that we measured rater intentions rather than actual

behavior. Although it doesn’t necessary follow that some-

one will actually behave in a manner consistent with their

stated intentions, Armitage and Conner (2001) found meta-

analytic support for the relationship between individuals’

stated intentions and their actual behavior. This is consis-

tent with Ajzen’s Theory of Reasoned Action, which states

that a ‘‘person’s intention to perform (or not perform) a

behavior is the immediate determinant of action, and bar-

ring unforeseen events, people are expected to act in

accordance with their intentions’’ (Ajzen 1985, p. 12).

A second limitation of this study was that the reliability

estimate for scores on the opportunity to observe variable

204 J Bus Psychol (2008) 22:191–207

123



was only 0.60. Unreliability of scores in a measure atten-

uates observed correlations, thereby threatening the

statistical conclusion validity of research findings. How-

ever, estimates of reliability, such as Cronbach’s alpha

used in the present study are influenced not only by how

well items correlate with one another, but also by how

many items exist in the measure. In this study, the oppor-

tunity to observe scale consisted of only three items, itemsthat correlated at acceptable levels among themselves (r s

between 0.30 and 0.38: see Nunnally and Bernstein 1994).

Using the Spearman–Brown prophecy formula, had we

doubled the scale to six or to nine items, we would have

increased our reliability estimates to 0.75 and 0.82,

respectively. As a result, all of our observed correlations

would have increased had we used a measure with more

than three items. For example, the correlation between

opportunity to observe and employee intentions to provide

honest upward feedback ratings (r = 0.27) would have

increased to r = 0.37 if we had used a six-item measure

instead of a three-item measure (see Nunnally and Bern-stein 1994).

More importantly, despite a reliability estimate of 0.60,

we did find substantive and statistically significant corre-

lations in the hypothesized direction between scores on the

opportunity to observe variable and scores on measures of

other variables: all of the observed correlations were

greater than r = 0:25j j. Thus, statistical conclusion validity

was not a problem in the present study.

Conclusion and Future Research

The main challenge in implementing a successful upward

system is ensuring that raters rate their supervisors hon-

estly. In this study, we examined several factors

hypothesized to influence employee intentions to rate their

supervisors honestly. In summary, we found support for

generalizing current rater motivation models to an upward

feedback context. More importantly, our findings provide

empirical support for many of the recommendations

researchers and practitioners have been making for years

regarding the requirements necessary for successfully

implementing upward feedback programs.

However, because of the paucity of research on rater

motivation applied to multisource feedback systems, sev-

eral areas for future research are suggested. In addition to

replicating the current study using a different employee

sample, future research should identify and examine

additional contextual predictors of rater motivation in

upward feedback systems, such as computerized versus

non-computerized systems, rating source, organizational

commitment, rater accountability, size of rater group,

organizational climate, and power distance, as well as

additional individual difference variables, such as goal

orientation, mood, and perceived openness to feedback.

Acknowledgements The authors would like to thank Dr. Tony

Calabrese, Director of Psychology Services at Pine Grove Recovery

Center, for his assistance in facilitating data collection. This article

was based on Austin F. R. Smith’s dissertation under the supervision

of Vincent J. Fortunato. The authors contributed equally to this

article.

References

Abraham, R. (2000). Organizational cynicism: Bases and conse-

quences. Genetic, Social, and General Psychology Monographs,

126 , 269–292.

Ajzen, I. (1985). In J. Kuhl & J. Beckman (Eds.), From intentions to

actions: A theory of planned behavior . Heidelberg: Springer.

Antonioni, D., & Woehr, D. J. (2001). Improving the quality of

multisource rater performance. In D. W. Bracken, C. W.

Timmreck, & A. H. Church (Eds.), The handbook of multisource

feedback: The comprehensive resource for designing and

implementing MSF processes (pp. 114–129). San Francisco,

CA: Jossey-Bass.

Armitage, C. J., & Conner, M. (2001). Efficacy of the theory of

planned behaviour: A meta-analytic review. British Journal of

Social Psychology, 40, 471–499.

Atwater, L., & Waldman, D. (1998). Accountability in 360-degree

feedback. HR Magazine, 43, 96–102.

Atwater, L. E., Waldman, D. A., Atwater, D., & Cartier, P. (2000). An

upward feedback field experiment: Supervisors’ cynicism,

reactions, and commitment to subordinates. Personnel Psychol-

ogy, 53, 275–297.

Avis, J. M., & Kudisch, J. D. (2000). Factors influencing subordi-

nates’ willingness to participate in upward feedback. Paper

presented at the 21st annual Industrial & Organizational

Behavior Graduate Conference, Knoxville, TN.Bandura, A. (1986). Social foundations of thought and action: A

social cognitive theory. Englewood Cliffs, NJ: Prentice Hall.

Baron, R. E., & Kenny, D. A. (1986). The moderator-mediator

variable distinction in social psychological research: Conceptual,

strategic, and statistical considerations. Journal of Personality

and Social Psychology, 51, 1173–1182.

Bernardin, H. J., & Tyler, C. L. (2001). Legal and ethical issues in

multisource feedback. In D. W. Bracken, C. W. Timmreck, & A.

H. Church (Eds.), The handbook of multisource feedback: The

comprehensive resource for designing and implementing MSF

processes (pp. 447–462). San Francisco, CA: Jossey-Bass.

Bracken, D. W. (1994). Straight talk about multirater feedback.

Training and Development, 48, 44–51.

Bracken, D. W., & Timmreck, C. W. (2001). Success and sustain-

ability: A systems view of multisource feedback. In D. W.Bracken, C. W. Timmreck, & A. H. Church (Eds.), The

handbook of multisource feedback: The comprehensive resource

for designing and implementing MSF processes (pp. 478–494).

San Francisco, CA: Jossey-Bass.

Bracken, D. W., Timmreck, C. W., & Church, A. H. (Eds.). (2001).

The handbook of multisource feedback: The comprehensive

resource for designing and implementing MSF processes . San

Francisco: Jossey-Bass.

Church, A. H. (1995). First-rate multirater feedback. Training and

Development, 49, 42–43.

Church, A. H., & Bracken, D. W. (1997). Advancing the state of the

art of 360-degree feedback: Guest editors’ comments on the

J Bus Psychol (2008) 22:191–207 205

123



research and practice of multirater assessment methods. Group

and Organizational Management, 22, 149–161.

Church, A. H., & Waclawski, J. (1998). Making multirater feedback

systems work. Quality Progress, 31, 81–89.

Church, A. H., & Waclawski, J. (2001). A five-phase framework for

designing a successful multisource feedback system. Consulting

Psychology Journal: Practice and Research, 53, 82–95.

Dalton, M. (1998). Best practices: Five rationales for using 360-

degree feedback in organizations. In W. W. Tornow & M.

London (Eds.), Maximizing the value of 360-degree feedback: A

process for successful individual and organizational develop-

ment (pp. 59–77). San Francisco, CA: Jossey-Bass.

Davis, L. T. (2000, April). The effect of organizational factors on

intended rating behavior . Poster session presented at the 15th

annual meeting of the Society for Industrial and Organizational

Psychology, New Orleans, LA.

Dean, J. W., Brandes, P., & Dharwadkar, R. (1998). Organizational

cynicism. Academy of Management Review, 23, 341–352.

DeCotiis, T., & Petit, A. (1978). The performance appraisal process:

A model and some testable propositions. Academy of Manage-

ment Review, 3, 635–646.

Edwards, M. R., & Ewen, A. (1996). 360-degree feedback: The

powerful new model for employee assessment and performance

improvement . New York: AMACON.

Edwards, M. R., & Ewen, A. (2001). Readiness for multisource

feedback. In D. W. Bracken, C. W. Timmreck, & A. H. Church

(Eds.), The handbook of multisource feedback: The comprehen-

sive resource for designing and implementing MSF processes

(pp. 33–47). San Francisco, CA: Jossey-Bass.

Farr, J. L., & Newman, D. A. (2001). Rater selection: Sources of

feedback. In D. W. Bracken, C. W. Timmreck, & A. H. Church

(Eds.), The handbook of multisource feedback: The comprehen-

sive resource for designing and implementing MSF processes

(pp. 96–113). San Francisco, CA: Jossey-Bass.

Funderburg, S. A., & Levy, P. A. (1997). The influence of individual

and contextual variables on 360-degree feedback system

attitudes. Group and Organizational Management, 22, 210–235.

Harris, M. M. (1994). Rater motivation in the performance appraisal

context: A theoretical framework. Journal of Management, 20,

737–756.

Heneman, R. L., & Wexley, K. N. (1983). The effects of delay in

rating and amount of information observed on performance

rating accuracy. Academy of Management Journal, 26 , 677–686.

Joreskog, K. G., & Sorbom, D. (1992). LISREL VIII: Analysis of linear

structural relations. Mooresville, IN: Scientific Software, Inc.

Kudisch, J. D., Fortunato, V. J., & Smith, A. F. R. (2006). Contextual

and individual difference factors predicting individuals’ desire to

provide upward feedback. Group and Organization Manage-

ment, 31, 509–529.

Lehr, C. O., & Facteau, J. D. (1992, May). Individual and contextual

factors related to subordinate appraisal system effectiveness.

Paper presented at the 7th annual conference for the Society of

Industrial and Organizational Psychology, Montreal.

Lepsinger, R., & Lucia, A. D. (1997). The art and science of 360degree feedback . San Francisco, CA: Jossey-Bass.

London, M. (2001). The great debate: Should multisource feedback

be used for administrative or development only? In D. W.

Bracken, C. W. Timmreck, & A. H. Church (Eds.), The




Longnecker, C. O. (1989). Truth or consequences: Politics and

performance appraisals. Business Horizons, 32, 76–82.

Longnecker, C. O., Sims, H. P., & Gioia, D. A. (1987). Behind the

mask: The politics of employee appraisal. The Academy of

Management Executive, 1, 183–193.

MacKinnon, D. P., Warsi, G., & Dwyer, J. H. (1995). A simulation

study of mediated effect measures. Multivariate Behavioral

Research, 30(1), 41–62.

Milliman, J. F., Zawacki, R. A., Schulz, B., Wiggins, S., & Norman,

C. A. (1995). Customer service drives 360-degree goal setting.

Personnel Journal, 74, 136–142.

Mohrman, A. M., & Lawler, E. E. (1983). Motivation and

performance appraisal behavior. In F. Landy, S. Zedeck, & J.

Cleveland (Eds.), Performance measurement and theory (pp.

173–198). Hillsdale, NJ: Erlbaum.

Murphy, K. R., & Cleveland, J. N. (1995). Understanding perfor-

mance appraisal: Social, organizational, and goal-based

perspectives. Thousand Oaks, CA: Sage.

Napier, N. K., & Latham, G. P. (1986). Outcome expectancies of

people who conduct performance appraisals. Personnel Psy-

chology, 39, 827–837.

Nunnally, J. C., & Bernstein, I. H. (1994). Psychometric theory (3rd

edn.). New York: McGraw Hill.

Podsakoff, P. M., & Organ, D. W. (1986). Self-reports in organization

research: Problems and prospects. Journal of Management, 12,

531–544.

Pollack, D. M., & Pollack, L. J. (1996). Using 360-feedback in

performance appraisal. Public Personnel Management, 25, 507–

529.

Preacher, K. J., & Leonardelli, G. J. (2001, March). Calculation for

the Sobel test: An interactive calculation tool for mediation tests

(Computer software). Available from http://www.unc.edu/

*preacher/sobel.htm.

Rotolo, C. T., & Kudisch, J. D. (1996). Important considerations in

developing and implementing a successful multi-source assess-

ment system. International Personnel Management Association

News, 14, 18–19.

Sobel, M. E. (1986). Some new results on indirect effects and their

standard errors in covariance structural models. In N. Tuma

(Ed.), Sociological methodology 1986 (pp. 159–186). Washing-

ton DC: American Sociological Association.

Tornow, W. W., & London, M. (1998). Maximizing the value of 360-

degree feedback: A process for successful individual and

organizational development . San Francisco, CA: Jossey-Bass.

Tornow, W. W., & Tornow, C. P. (2001). Linking multisource

feedback content with organizational needs. In D. W. Bracken,

C. W. Timmreck, & A. H. Church (Eds.), The handbook of

multisource feedback: The comprehensive resource for design-

ing and implementing MSF processes (pp. 48–62). San

Francisco, CA: Jossey-Bass.

Tziner, A. (1999). The relationship between distal and proximal

factors and the use of political considerations in performance

appraisal. Journal of Business and Psychology, 14, 217–231.

Tziner, A., & Murphy, K. R. (1999). Additional evidence of

attitudinal influences in performance appraisal. Journal of

Business and Psychology, 13, 407–419.

Tziner, A., Murphy, K. R., & Cleveland, J. N. (2001). Relationships

between attitudes toward organizations and performance apprai-

sal systems and rating behavior. International Journal of Selection and Assessment, 9, 226–241.

Tziner, A., Murphy, K. R., & Cleveland, J. N. (2002). Does

conscientiousness moderate the relationship between attitudes

and beliefs regarding performance appraisal and rating behavior?

International Journal of Selection and Assessment, 10, 218–224.

Tziner, A., Murphy, K. R., & Cleveland, J. N. (2005). Contextual and

rater factors affecting rater behavior. Group and Organization

Management, 30, 89–98.

Tziner, A., Murphy, K. R., Cleveland, J. N., Beaudin, G., &

Marchand, S. (1998). Impact of rater beliefs regarding perfor-

mance appraisal and its organizational context on appraisal

quality. Journal of Business and Psychology, 12, 457–467.

206 J Bus Psychol (2008) 22:191–207

123

http://www.unc.edu/~preacher/sobel.htm







Vance, R. J., Brooks, S. M., & Tesluk, P. E. (1995, May).

Organizational cynicism, cynical cultures, and organizational

change efforts. Paper presented at the 10th Annual Conference of

the Society of Industrial and Organizational Psychology,

Orlando, FL.

Van Velsor, E. (1998). Designing 360-degree feedback to enhance

involvement, self-determination, and commitment. In W. W.

Tornow & M. London (Eds.), Maximizing the value of 360-

degree feedback: A process for successful individual and

organizational development (pp. 149–195). San Francisco, CA:

Jossey-Bass.

Waldman, D. A., & Atwater, L. E. (1998). The power of 360 degree

feedback: How to leverage performance evaluations for top

productivity. Houston: Gulf Publishing.

Waldman, D. A., & Atwater, L. E. (2001). Confronting barriers to

successful implementation of multisource feedback. In D. W.

Bracken, C. W. Timmreck, & A. H. Church (Eds.), The




Westerman, J. W., & Rosse, J. G. (1997). Reducing the threat of rater

nonparticipation in 360-degree feedback systems: An explor-

atory examination of antecedents to participation in upward

ratings. Group and Organizational Management, 22, 288–309.

Wimer, S., & Nowack, K. M. (1998). Thirteen common mistakes

using 360-degree feedback. Training and Development, 52,

69–78.

J Bus Psychol (2008) 22:191–207 207

123



Reproducedwithpermissionof thecopyrightowner. Further reproductionprohibitedwithoutpermission.

Factors Influencing Employee Intentions

Documents

Transcript of Factors Influencing Employee Intentions