The Effect of Performance Reporting Frequency on Employee ...

The Effect of Performance Reporting Frequency on Employee Performance

Gary Hecht [email protected]

Jessen L. Hobson

[email protected]

Laura W. Wang [email protected]

University of Illinois at Urbana-Champaign

March 2016

This is a preliminary draft. Please do not circulate or cite without the authors’ permission.

mailto:[email protected]

The Effect of Performance Reporting Frequency on Employee Performance

ABSTRACT In environments where many decision rights reside at the lower levels of the organization, managers often rely on self-reported performance information from lower-level employees for planning and decision making. In addition, managers can also use this information for employee evaluation (i.e., assessing employee ability for promotion or training purposes). In these environments, the accuracy and timeliness of self-reported performance information is of important concern. As a result, many recommend that managers collect self-reported performance information frequently and avoid tying it directly to financial rewards. However, little is known about the performance implications of this recommended reporting practice. In this study, we use an experiment to examine the effect of performance-reporting frequency on employee performance in a setting where reported performance does not affect current or future financial rewards and employees cannot misreport their performance information. Using goal orientation theory from social psychology, we predict and find that even in this setting, frequent performance reporting can have detrimental effects. More specifically, frequent performance reporting undermines performance when employees know that reported performance will be used to evaluate their ability. Our study answers the call for research on the motivational effects of the mere act of performance reporting and has important practical implications.

Keywords: reporting frequency, performance evaluation, goal orientation, avoidance orientation, learning orientation

1

I. INTRODUCTION

In environments where decision rights reside at the lower levels of the organization,

lower-level employees often hold private information about their own performance. In this

environment, managers often rely on self-reported information provided by lower-level

employees for planning and decision making. For example, the project manager of a software

development team collects performance reports (i.e., progress updates) from the programmers

and engineers to monitor progress and coordinate efforts. These reports provide vital information

for important decisions such as resource allocation, scheduling, and announcing software release

dates. Of course, managers can also use employee-provided performance information for

employee evaluation (e.g., assessing employee ability for promotion or training purposes). That

is, performance reports provided by programmers and engineers can be used by managers to

formally or informally evaluate the employees for promotion or training purposes.

In this environment, managers face at least two important decisions: (1) the frequency

with which to obtain performance reports from lower-level employees (hereafter, reporting

frequency) and (2) whether to use information contained in these reports to evaluate employee

ability (hereafter, evaluative purpose). Conventional wisdom suggests that when the accuracy of

the reported information is reasonably assured, more frequent reports are beneficial as they allow

managers to identify potential problems and/or opportunities on a timely basis. As a result, some

researchers suggest that managers should obtain performance reports frequently and avoid tying

performance reports directly to financial rewards, so as to mitigate employees’ propensity to

misreport performance (e.g., Simons 1994). However, little is known about the performance

implications of this recommended practice. In this study, we use an experiment to examine the

effect of reporting frequency and evaluative purpose on employee performance in a setting

2

where misreporting is not possible. In doing so, we assure the accuracy of the reported

performance information, which is the focus of a large prior literature on performance reporting

(e.g., Guidry, Leone, and Rock 1999; Evans, Hannan, Krishnan, and Moser 2001; Maas and Van

Rinsum 2013; Hales, Hobson, and Resutek 2015), and isolate the performance effect of reporting

frequency and evaluative purpose.

We model an environment where employees work on a challenging task and have to

accurately report their private performance information. Compensation is fixed and is not

affected by the employee’s actual or reported performance. In this setting, conventional

economic reasoning suggests that reporting frequency should have no effect on employee

motivation or performance. However, using goal orientation theory from social psychology, we

predict that performance-reporting frequency can affect task performance through its influence

on individuals’ goal orientation toward the task. Two types of goal orientation are relevant to our

theory development: avoidance orientation refers to an individual’s tendency to focus on

avoiding bad performance or unfavorable judgments of competence; learning orientation refers

to an individual’s tendency to focus on learning and developing competence (Elliot and

Harackiewica 1996; Elliot and Church 1997; Elliot 1999). These goal orientations are not

mutually exclusive and can be influenced by personal as well as situational factors (Elliot 1999).

Reporting frequency represents a situational factor that can influence individuals’ goal

orientation. Frequent performance reporting reduces the performance horizon of the task. This

reporting-induced “short-termism” reduces the perceived value of learning and perceived

achievability of great performance, which in turn decreases their learning orientation and

increases their avoidance orientation. Prior literature provides evidence that learning orientation

enhances task performance while avoidance orientation undermines task performance (see Elliot

3

(1999) for a review). As a result, we predict that frequent performance reporting reduces task

performance by decreasing individuals’ learning orientation and increasing their avoidance

orientation.

Furthermore, we predict that using performance reports for employee evaluation, even

when such evaluation has no current or future financial implications, exacerbates the negative

performance effect of frequent performance reporting. Recall that avoidance orientation refers to

an individual’s tendency to focus on avoiding poor performance or negative judgment about

his/her competence. This tendency is suppressed when evaluation is not a concern, leaving little

room for reporting frequency to have an effect. On the contrary, when evaluation is salient, the

extent to which individuals worry about negative judgments can vary greatly, leaving much room

for reporting frequency to affect avoidance orientation. As a result, using performance reports for

employee evaluation exacerbates the effect of frequent reporting on avoidance orientation, and

therefore the negative effect on task performance.

We conduct an experiment to test our predictions. In our experiment, participants work

on GMAT critical reasoning questions for 30 minutes in exchange for a fixed wage. All

participants receive private performance feedback (i.e., the number of correctly answered

questions) from the computer every 10 minutes and have to truthfully report their private

performance information to the experimenter. Thus, we hold constant across conditions the level

and frequency of private performance feedback participants receive. We manipulate two factors

between subjects. First, we manipulate the frequency with which employees report their task

performance. Participants in the Low Frequency condition are asked to report their performance

(i.e., the number of correctly answered questions) to the administrator after 30 minutes.

Participants in the High Frequency condition are asked to report their performance to the

4

administrator every 10 minutes. Importantly, regardless of their reporting condition, all

participants have exactly 30 minutes to work on the task. Second, we manipulate whether

participants receive information indicating that we will use their performance reports to evaluate

their critical reasoning skill.

We measure participants’ task performance as the number of questions they correctly

answer. We find that the main effect of reporting frequency on task performance is negative but

not significant. However, we find a significant interaction between reporting frequency and

evaluative purpose, indicating that using self-reported performance information for employee

evaluation moderates the effect of reporting frequency on task performance. Specifically,

frequent performance reporting harms task performance when reported performance is used for

employee evaluation, but does not harm performance when reported performance is not used for

employee evaluation. We conduct a path analysis to shed light on the mechanism through which

frequent performance reporting harms performance when used for employee evaluation. Overall,

the negative performance effect of frequent performance reporting is fully mediated by the

increase in participants’ avoidance orientation.

Our study answers the call for research on “the motivational effects related to the mere

act of collecting evaluation data” (Sprinkle and Williamson 2007). Specifically, we investigate

the effect of collecting performance data directly from employees with private information about

their own performance. Related prior research focuses primarily on mechanisms to increase the

accuracy of this information when employees have financial incentives to under- or over-report

their true performance (e.g., Guidry, Leone, and Rock 1999; Evans, Hannan, Krishnan, and

Moser 2001; Maas and Van Rinsum 2013; Fisher, Maines, Peffer, and Sprinkle 2002; Chow,

Cooper, and Waller 1988; Chow, Cooper, and Haddad 1991; Waller 1988). Our results suggest

5

that even when misreporting is not possible, reporting frequency and whether self-report

performance information is used for employee evaluation (i.e., basic aspects of the performance-

reporting function) have important implications for employee performance.

Our study also has important implications for reporting practices. While frequent

performance reports allow managers to identify potential problems and/or opportunities on a

timely basis, these reports can increase employees’ avoidance orientation and undermine

performance in environments where information contained in these reports is used for employee

evaluation. As a result, to the extent that obtaining frequent and timely performance information

is critical for planning and decision making, decoupling financial incentives from reported

performance information (i.e., to mitigate the incentive to misreport) as suggested by prior

research may not be sufficient to alleviate undesirable consequences. Our results suggest that

committing to not using this information for employee evaluation is vital to avoiding lower

actual performance. More generally, our study highlights the importance of managing the actual

and perceived use of self-reported performance information when such information has to be

frequently collected.

Next, section II provides background and develops our hypotheses. Section III describes

the method used to test these hypotheses. Section IV presents the results, and Section V

concludes.

II. THEORY AND HYPOTHESES

Background

In environments where decision rights reside at the lower levels of the organization,

lower-level employees endowed with decision rights often have private information about their

own performance. As a result, managers have to obtain performance reports from these

6

employees, resulting in the upward self-reporting of performance information. Self-reported

performance information can take many forms. One example is individuals’ reporting of

progress toward a long-term project (Amabile and Kramer 2011). Employees responsible for a

long-term project often have useful private information regarding the project’s progress, and

managers likely rely on employees to self-report this progress. Other firms leverage frequent

employee, team and/or department meetings where progress on projects is shared. At a more

aggregate level, subsidiaries, business units, and departments report performance to higher levels

of the organization. For example, senior managers at USA Today get weekly performance

reports containing information that ranges from year-to-date performance figures (e.g.,

advertising volume against plan, committed future volume by issue) to account-specific

information (Simons and Weston 1990).

Regardless of its form, managers often rely on this information for planning and decision

making. For example, project managers rely on self-reported performance information from

employees for coordination and resource allocation. In addition, managers can also use self-

reported performance information for employee evaluation. Ideally, the firm’s performance

measurement system captures outcomes associated with employees’ actions, thereby providing

information necessary for employee evaluation. However, performance measures are often noisy

indicators of “true” performance, as agents’ effort is often unobservable (Holmstrom 1979).

Further, performance measures may simply be unavailable in certain situations. For these

reasons, managers may also rely on performance information provided by employees for

employee evaluation (e.g., Arnold, Hannan, and Tafkov 2015).

Performance reporting is a fundamental accounting function and has generated much

accounting research. However, prior studies have primarily focused on how characteristics of the

7

reporting, evaluation, and reward systems jointly affect employees’ propensity to misreport their

private performance information (e.g., Guidry, Leone, and Rock 1999; Evans, Hannan, Krishnan,

and Moser 2001; Maas and Van Rinsum 2013; Fisher, Maines, Peffer, and Sprinkle 2002; Chow,

Cooper, and Waller 1988; Chow, Cooper, and Haddad 1991; Waller 1988). This focus on

truthful reporting is well justified as the integrity of self-reported performance information is of

vital importance for both planning and evaluation purposes. However, in this study, we create an

experimental setting where employees cannot misreport their private performance information.

In doing so, we are able to show that even when misreporting is not a concern, characteristics of

the performance reporting function can have important motivation implications and affect actual

performance.

Conventional wisdom suggests that when misreporting is not a concern, more frequent

performance reports are beneficial as they allow managers to identify potential problems and/or

opportunities on a timely basis. As a result, some researchers suggest that managers should

obtain performance reports frequently as long as they avoid tying performance reports directly to

financial rewards (e.g., Simons 1994). However, little is known about the performance

implications of high reporting frequency, which is the focus of our current study. Notably, our

dimension of self-reporting frequency is distinct from the construct of performance-feedback

frequency, a focus of an extensive prior literature (e.g., Bilodeau 1966; Cook 1968; Lam, DeRue,

Karam, and Hollenbeck 2011; Lurie and Swaminathan 2009; Salmoni, Schmidt, and Walter

1984). This literature focus on an environment where managers hold private information about

employee performance and examines the performance implications of the frequency with which

this information is communicated to employees.1 As discussed earlier, we focus on an

1 This literature concludes that frequent feedback can have both a positive and negative effect on task performance. On one hand, more frequent performance feedback leads to more timely learning and superior task performance

8

environment in which employees hold private information about their own performance, and

managers have to obtain this information from the employees. In this environment, more

frequent performance reporting does not alter the timing or content of information the employees

can use for the task, and therefore should have no effect on task performance according to

conventional economic reasoning. However, psychology theory suggests otherwise.

Hypotheses Development

We rely on prior social psychology research to develop our hypotheses about the

performance implications of reporting frequency. In particular, we leverage literature on

individuals’ goal orientation. Goal orientation refers to an individual’s disposition or propensity

toward developing or demonstrating knowledge, ability, or skills when engaging in task

performance (e.g., VandeWalle 1997; DeGeest and Brown 2011). Goal orientation has been

found to affect individuals’ motivation and effort, perspectives adopted when performing a task,

and performance in a variety of contexts (VandeWalle 1997).

Related research has identified three types of goal orientation adopted by individuals

(VandeWalle 1997; Elliot 1999).2 When individuals adopt an approach goal orientation, they

desire to prove their competence, knowledge, and/or ability in some positive manner. Individuals

frame tasks and effort with the potential for success as a key mindset. Conversely, when

adopting an avoidance goal orientation, individuals desire to avoid poor performance and

negative judgements about their competence. Individuals frame tasks and effort with the

potential for failure as a key mindset. Somewhat distinct from these performance-focused

(e.g., Bilodeau 1966; Cook 1968; Salmoni, Schmidt, and Walter 1984). On the other hand, frequent performance feedback can lead to information overload or cause individuals to overreact to noise contained in the feedback, which reduces task performance (e.g., Lam, DeRue, Karam, and Hollenbeck 2011; Lurie and Swaminathan 2009). 2 Recent literature considers the extent to which goal orientation is influenced by stable personality traits and situational or environmental factors (Elliot 1999). As such, we adopt the perspective that individuals likely have a general tendency toward one or more orientation types, but that environmental or situational factors also influence individuals’ propensity to exhibit a certain goal orientation.

9

approaches to tasks, individuals can adopt a learning goal orientation. Under this approach,

individuals worry less about current exhibition of competence, knowledge, and/or ability, and

more about their development of knowledge, skills, and abilities.

Prior literature provides consistent theory and empirical evidence suggesting that learning

orientation enhances task performance while avoidance orientation undermines task

performance, but theory and empirical evidence on the performance effect of approach

orientation is mixed (Elliot 1999). As a result, we focus our theory development on avoidance

and learning orientation.3 In particular, prior research suggests that avoidance orientation

increases performance anxiety and decreases help-seeking, which in turn undermines task

performance (e.g., Elliot and Church 1997; Elliot and McGregor 1999; Middleton and Midgley

1997). Learning orientation, on the other hand, leads to deeper processing of information,

increased task absorption, enhanced retention of information, appropriate help-seeking behavior,

and higher intrinsic motivation. Thus, learning orientation improves task performance, although

such improvement tends to occur over a longer time horizon (e.g., Elliot 1999; Harackiewicz,

Barron, and Elliot 1998; Urdan 1997).

Ultimately, greater reporting frequency can undermine performance by decreasing

individuals’ learning orientation and increasing their avoidance orientation. To elaborate,

consider that greater reporting frequency reduces the time available between instances of

reporting. In turn, this leads employees to adopt a short-term perspective, which influences their

perceptions of the value of learning and the achievability of great performance. More

specifically, since learning in general entails short-term costs and long-term benefits, a short-

term perspective leads employees who must report performance more frequently to perceive

3 Despite the lack of theory to develop predictions related to individuals’ approach orientation, we measure and analyze individuals’ approach orientation and its implications in our experiment (see Sections III and IV).

10

opportunities to learn as less valuable. Thus, these employees are less likely to exhibit a learning

orientation. Further, in the short-term, high levels of performance are less achievable, thus

increasing the salience of the potential threat of having to report low performance, and resulting

observers’ perceptions that they are incompetent or lack knowledge, skills, or ability. As a result,

individuals increase their avoidance orientation relative to when reporting frequency is lower.

This shift away from a learning orientation, and towards an avoidance orientation, leads to lower

levels of performance, especially relative to when reporting frequency is lower. This leads to our

first hypothesis:

H1: Higher reporting frequency reduces performance. Furthermore, we predict that using self-reported performance information for employee

evaluation (i.e., evaluation of employee knowledge, skills, ability, competence, etc.) even when

such evaluation has no current or future financial implications can exacerbate the negative

performance effect of frequent performance reporting. Recall that avoidance orientation refers to

an individual’s tendency to focus on avoiding poor performance or negative judgement about

his/her competence. This tendency is likely muted when evaluation is not a concern, leaving little

room for reporting frequency to have an effect on avoidance orientation. That is, when

employees believe that their skills or ability are not being evaluated, they will be less likely to

worry about observers’ judgement and perceptions of incompetence regardless of how

frequently they have to report their performance. Conversely, when evaluation of one’s skills or

ability is salient, the extent to which individuals worry about negative judgments from observers

can vary greatly, leaving much room for reporting frequency to affect avoidance orientation, an

thus performance. Therefore, frequent reporting will be more likely to increase avoidance

11

orientation and reduce task performance when reported information is used for employee

evaluation. This discussion leads to our second hypothesis:

H2: Using self-reported performance for employee evaluation exacerbates the negative effect of reporting frequency on performance.

We do not make a prediction about the main effect of employee evaluation on

performance because it can have two countervailing effects. On one hand, evaluation provides a

non-monetary incentive, motivating individuals to work harder and perform better. On the other

hand, it provides the context for individuals to adopt an avoidance orientation, which undermines

performance. As such, the effect of employee evaluation on performance depends on whether the

former or latter effect dominates.4

III. METHOD

Participants

Given the abstract nature of the task and the absence of any requirement for participants

to have institutional knowledge or skills, we use undergraduate students at a large U.S.

University as participants (Libby, Bloomfield, and Nelson 2002). Eighty-seven participants

participated in our study and 44 percent of the participants are female. On average, participants

are 19 years old and have less than a year of work experience.

Experimental Design

Our experiment models a setting where employees work on a challenging task that uses

an important skill. We use GMAT critical reasoning questions as the experimental task because

the skill required for this task (i.e., critical reasoning skill) is one that business students (i.e., our

participant pool) value and seek to improve.5 Within the experimental task, all participants

4 We explore these potential effects in our analyses, as reported in Section IV. 5 In the post-experimental questionnaire, we ask participants to rate the extent to which they agree with the following statement, “I think critical reasoning is an important skill for my academic performance” on a seven-point

12

receive performance feedback as part of the task itself (i.e., directly from the computer program,

as opposed to from a superior or other individuals), thereby modeling an environment in which

employees have information regarding their performance that is then reported to a superior.

In this setting, we manipulate two factors using a 2 × 2, full-factorial, between-subjects

design. First, we manipulate the frequency with which employees report their task performance.

Participants in the Low Frequency condition are told: “Your third feedback screen is also a

performance report that must be reported to the administrator. After the entire task is over and

you have reviewed your performance information on the final (after 30 minutes) feedback

screen, please raise your hand and the administrator will come to your workstation to record your

performance information on a piece of paper.” Participants in the High Frequency condition are

told: “Each of the three feedback screens is also a performance report that must be reported to

the administrator. After you have reviewed your performance information in each feedback

screen (after 10 minutes, 20 minutes and 30 minutes), please raise your hand and the

administrator will come to your workstation to record your performance information on a piece

of paper.” Second, we manipulate whether we tell participants that we will use reported

performance information to evaluate their critical reasoning skill. Specifically, participants in the

No Evaluation condition are told: “the administrator will use your performance report to assess

the quality of the critical reasoning questions in the question bank for potential future use.”

Participants in the Evaluation condition are told: “the administrator will use your performance

report to students’ critical reasoning skill at (the participants’ institution).”

Likert scale with 1 labeled “strongly Disagree” and 7 labeled “Strongly Agree.” Average response to this item is 6.34 (untabulated) indicating that on average, participants agree that critical reasoning is an important skill for their academic performance. In addition, participants’ response to this item does not differ across reporting frequency (F1,

86 = 0.09, p = 0.76) or evaluation purpose (F1, 86 = 0.03, p = 0.86) conditions.

13

Notably, our evaluation manipulation design choice balances a variety of dimensions.

One, our evaluation manipulation could be stronger, as we could have framed the purpose of the

performance report as a mechanism to evaluate individual-specific critical reasoning skill (as

opposed to the business student population in general). However, in order to keep the experiment

anonymous, and to avoid deceiving participants, we opt for a weaker manipulation of employee

evaluation. Further, ideally, we would tell participants in the No Evaluation condition that their

performance reports will not be used to assess their critical reasoning skill, and this statement

would eliminate their concerns about evaluation. However, doing so may make evaluation salient

for participants who are otherwise not thinking about performance evaluation. As a result, we tell

participants in the No Evaluation condition that the administrator will use their performance

reports to assess the quality of critical reasoning questions in our question bank. This statement

diverts participants’ attention away from evaluation concerns.6

Procedures

After participants arrive in the lab, we provide them with extensive instructions on

laboratory computers. The experimental instrument is administered using z-Tree software

(Fischbacher 2007). First, we inform participants that they will work on GMAT critical-

reasoning questions for 30 minutes, and they will be paid $20 for their time.7 Second, we show

participants an example critical-reasoning question accompanied by a detailed solution. Third,

we tell participants that their goal for the experimental task is to answer as many critical

6 This manipulation is also a more active one, relative to a more passive approach in which statements in the Evaluation condition are simply absent in No Evaluation condition. A self-reporting environment likely induces some baseline perceptions of evaluation, and thus, we chose our more active approach to strengthen our evaluative purpose manipulation. 7 We pay participants $20 upon completion of the post-experimental questionnaire, and facilitate provision of extra credit after they complete a follow-up survey approximately one week after the main session. All participants were offered the same compensation and extra credit so as to inspire participation, but avoid inducing a particular goal orientation via a factor (i.e., pay-for-performance incentives) other than reporting frequency and evaluative purpose.

14

reasoning questions correctly as possible. In doing so, we define good performance for all

participants and minimize noise associated with idiosyncratic performance definition. Fourth, we

inform participants that they will have access to a short article that describes useful strategies to

improve their critical reasoning skill and they can access this article as often as they like. The

opportunity to access this article makes learning possible and salient. Fifth, we inform

participants that every 10 minutes, the computer will provide them feedback on the cumulative

number of questions they have answered correctly. Thus, we keep feedback frequency constant

across all conditions. We then inform participants about the required performance-reporting

frequency and the intended purpose of the performance reports they provide (i.e., participants’

specific experimental conditions). Participants then answer comprehension questions about the

manipulations before working on the task.

We provide participants 30 minutes to answer critical reasoning questions in three

separate 10-minute increments. They are able to skip any question and can access a helpful

article that describes useful strategies they can use to improve their critical reasoning skill at any

time. At the end of the first 10-minute period, the computer informs the participant how many

questions they answered correctly. Participants in the High Frequency conditions then raise their

hand and wait until an administrator comes to view and record their score.8 Because their score

is visible on the computer screen, they are unable to misreport their performance. The second

and third 10-minute periods proceed in exactly this same manner. Participants in the Low

Frequency condition receive computer-based feedback at the same 10-minute increments, but

8 To make competence evaluation salient in a setting where participants and the administrators do not know each other, we require participants to make face-to-face performance reports. However, our theory extends to alternative channels of performance reporting (e.g., email, phone, etc.) as long as it is salient to the employees that the recipient of the performance reports can make judgements about their skills or ability, and they care about such judgements. To the extent that employees in natural settings care about the judgments of their superior more than participants care about the judgments of the administrators, our setting biases against us finding support for our hypotheses.

15

only report their performance, by raising their hand and waiting for an administrator, at the end

of the task – after all three periods. Next, participants complete a post-experiment questionnaire

to capture their goal orientation toward the experimental task. In addition, approximately 7 days

later, we asked several follow up questions via a web-based survey to capture personality

measures and other variables that potentially influence performance.9

IV. RESULTS

Performance Effects

H1 predicts that higher reporting frequency reduces performance. H2 predicts that the

effect of reporting frequency on performance is exacerbated when self-reported performance

information is used for employee evaluation. We measure participants’ performance

(Performance) as the number of critical reasoning questions they answer correctly. To test our

hypotheses, we conduct an ANCOVA using Performance as the dependent variable, Reporting

Frequency (Low Frequency vs. High Frequency) and Evaluative Purpose (No Evaluation vs.

Evaluation) as independent variables, and participants’ self-reported GPA (GPA) as a covariate.

We use GPA to control for participants’ general academic proficiency; however, all results are

robust to not including GPA as a covariate.

Figure 1 and Panel A of Table 1 present mean Performance by condition. When

participants are explicitly told that we will use their performance reports to evaluate their critical

reasoning skill, participants answer an average of 17.33 critical reasoning questions correctly in

the Low Frequency condition and an average of 13.14 in the High Frequency condition. When

participants are told that we will use their performance reports to evaluate the quality of

9 The web-based survey includes measures of participants’ Big Five personality traits, general tendency for goal orientation, general fear of evaluation, risk preference, and overconfidence. All results reported in the Results section are robust to including these measures of individual differences as control variables. Thus, we do not discuss these variables further.

16

questions in the question bank, they answer an average of 15.10 questions correctly in the Low

Frequency condition and an average of 17.00 questions correctly in the High Frequency

condition.

[Insert Table 1 and Figure 1 here]

Panel B of Table 1 presents the results of the ANCOVA. Neither Reporting Frequency

(F1,81 = 0.39, p = 0.53) nor Evaluative Purpose (F1,81 = 0.35, p = 0.56) has a significant main

effect, but they interact to affect Performance (F1,81 = 5.58, p = 0.02). Panel C of Table 1 reports

follow-up simple effects analyses of this interaction. Higher reporting frequency decreases

performance in the Evaluation condition (p = 0.03), but has an opposite but insignificant effect

on performance in the No Evaluation condition (p = 0.24). These results suggest that the effect of

reporting frequency on performance depends on whether performance reports are used for

employee evaluation.

Goal-Orientation Effects

Next, we perform more detailed tests of the underlying theory. More specifically, we test

whether reporting frequency affects participants’ goal orientation as predicted. To measure

participants’ goal orientation, we adapt Elliot and Church’s (1997) goal orientation scale. Elliot

and Church (1997) developed the scale to measure students’ goal orientation toward a college-

level psychology class. We modify their scale to measure participants’ goal orientation toward

our experimental task. For example, we change items such as “My fear of performing poorly in

this class is often what motivates me” to “My fear of performing poorly on the critical reasoning

questions motivated me to work hard on the questions.” In addition, we drop items that do not

apply to our experimental setting, such as “I’m afraid that if I ask my TA or instructor a ‘dumb’

question, they might not think I’m very smart.”

17

The modified scale contains eight items: three for approach orientation, three for

avoidance orientation, and two for learning orientation.10 Each item is measured using a seven-

point Likert scale with “1” labeled “strongly disagree” and “7” labeled “strongly agree” (see

Appendix). Results of a confirmatory factor analysis provide evidence that these items load as

expected onto three factors, each representing one of the three types of goal orientation

(untabulated). Given this, we use the average of the three avoidance-orientation items to measure

participants’ avoidance orientation toward the experimental task (Avoidance Orientation) and the

average of the two learning-orientation items to measure participants’ learning orientation

toward the experimental task (Learning Orientation).

Recall our theory suggests that higher reporting frequency decreases participants’

learning orientation and increases their avoidance orientation. In addition, the effect of reporting

frequency on avoidance orientation is exacerbated when performance reports are used for

competence evaluation. To test these predictions, we conduct separate ANOVAs using

Avoidance Orientation and Learning Orientation as the dependent variables. In both analyses,

Reporting Frequency (Low Frequency vs. High Frequency) and Evaluative Purpose (No

Evaluation vs. Evaluation) are the independent variables. Table 2 and 3 present the results of the

ANOVA for Avoidance and Learning Orientation, respectively.

As shown in Table 2, Reporting Frequency does not have a significant main effect on

Avoidance Orientation (F1,81 = 0.77, p = 0.38). Evaluation Purpose has a significant main effect

on Avoidance Orientation (F1,81 = 3.68, p = 0.06), suggesting that evaluative pressure increases

participants’ level of avoidance orientation. More importantly, the Reporting Frequency ×

Evaluative Purpose interaction is significant (F1,81 = 3.66, p = 0.06), suggesting that the effect of

10 Although we do not theorize about the effect of reporting frequency on individuals’ approach orientation, we included the approach-orientation items for completeness.

18

reporting frequency on participants’ level of avoidance orientation is moderated by whether

performance reports are used for employee evaluation. Panel C of Table 2 reports follow-up

analyses of this interaction. Although reporting frequency increases participants’ level of

avoidance orientation in the Evaluation condition as predicted (p = 0.03), it does not have a

significant effect on participants’ level of avoidance orientation in the No Evaluation condition

(p = 0.51). These results suggest that the effect of reporting frequency on participants’ level of

avoidance orientation depends on whether performance reports are used for competence

evaluation. Higher reporting frequency increases avoidance orientation when self-reported

performance information is used for employee evaluation, and this effect is suppressed when this

information is not used to evaluate competence or ability.

[Insert Table 2 and Figure 2 here]

As shown in Table 3, neither Reporting Frequency (F1,81 = 1.47, p = 0.23) nor Evaluative

Purpose (F1,81 = 0.29, p = 0.59) has a significant main effect on Learning Orientation.

Furthermore, the Reporting Frequency × Evaluation interaction is also insignificant (F1,81 = 1.06,

p = 0.31). These results suggest that higher reporting frequency does not decrease participants’

level of learning orientation in our experiment as predicted by our theory.11

[Insert Table 3 here]

Path Analysis

We employ a structural equations-based path analysis (Schumacher and Lomax 2004) to

shed light on the mechanism through which reporting frequency affects performance in the

Evaluation condition. Panel A of Figure 2 specifies the predicted path model and Panel B reports

11 For completeness, we also conduct an ANOVA (not tabulated) using Approach Orientation as the dependent variable. Results suggest that Reporting Frequency (F1,81 = 0.03, p = 0.86), Evaluation (F1,81 = 0.86, p = 0.36), and the Reporting Frequency × Evaluation interaction (F1,81 = 1.38, p = 0.24) are all insignificant.

19

the results. Consistent with our expectations, we find that high reporting frequency increases

participants’ level of avoidance orientation (p < 0.05), and participants’ level of avoidance

orientation is negatively correlated with their task performance (p < 0.01). These results are

robust to controlling for GPA, which is positively correlated with participants’ task performance

(p < 0.01). However, inconsistent with our predictions, we find no significant relation between

reporting frequency and participants’ level of learning orientation, and between their level of

learning orientation and task performance (all p > 0.50).

Finally, the direct effect of reporting frequency on participants’ task performance after

controlling for participants’ level of avoidance and learning orientation is insignificant (p >

0.50). This suggests that the negative effect of reporting frequency on participants’ performance

is fully mediated by an increase in participants’ avoidance orientation. Overall, while we find no

evidence suggesting that higher reporting frequency reduces participants’ learning orientation,

we find strong evidence suggesting that higher reporting frequency increases participants’ level

of avoidance orientation, which in turn reduces their task performance when performance reports

are used for employee evaluation, even when such evaluation has no current or future financial

implications.

[Insert Figure 3 here]

V. CONCLUSION

In this study, we examine the effect of performance-reporting frequency on employee

performance and investigate whether this effect is moderated by the use of these reports for

employee evaluation. Using an experiment to test our theory, we find that frequent performance

reporting increases participants’ avoidance orientation, which in turn decreases their task

performance. Contrary to our theory, however, we do not find evidence that frequent

20

performance reporting decreases participants’ learning orientation. Rather, in our experiment, the

negative performance effect of frequent performance reporting is fully mediated by the increase

in participants’ avoidance orientation.

Our study provides a “fundamental” perspective on the reporting function within

accounting (see also Basu and Waymire (2006) and Basu, Dickhaut, Hecht, Towry, and

Waymire (2009)). Specifically, prior related research primarily focuses on employees’

propensity to misreport performance information. We create an environment where employees

know their own performance and must report honestly. Our results suggest that even in this

environment, elements of the fundamental reporting function can influence actual performance.

Thus, we answer the call for research on “the motivational effects related to the mere act of

collecting evaluation data” (Sprinkle and Williamson 2007).

Our results have important implications for reporting practices. We highlight a potential

consequence of using frequent reporting when performance reports are used for employee

evaluation. While frequent evaluative performance reports allow managers to identify potential

problems and/or opportunities on a timely basis, these reports influence employees’ goal

orientation, ultimately undermining their performance. As a result, to the extent that obtaining

frequent and timely performance information from lower-level employees is important for

planning and decision making, our results suggest that firms should manage the actual or

perceived use of the reporting function. For instance, it may be important for managers to

commit to not using self-reported performance information for employee evaluation. If such

commitment is not feasible or credible, organizations should consider separating the decision-

making and employee-evaluation functions.

21

Our paper is subject to limitations that create opportunities for future research. We

manipulate reporting frequency at two levels. However, the question remains regarding the linear

nature of the effect of reporting frequency. In some settings, higher reporting frequency may

“immunize” an individual from the avoidance orientation. With greater reporting frequency, each

individual reporting instance may no longer be reflective of the employee’s ability. That is,

reporting may occur so frequently, individuals no longer feel the same level of evaluative

pressure during individual instances of reporting. As a result, the effect of reporting frequency

may diminish. Future research could investigate this potential non-linear effect of reporting

frequency.

Another research opportunity relates to our choice of experimental task. Our task reflects

an ability that is important to our subjects, but one that can be performed in a reasonable amount

of time as to facilitate a laboratory experiment and internally valid test of theory. Our lack of

results related to participants’ learning orientation may be (in part) a function of our research

method and/or the task we chose. That is, despite our effort to create a setting where learning is

possible and salient, the short-term nature of our experiment and task may have precluded

observation of effects on participants’ learning orientation. Thus, we must caution against over-

interpreting our results related to learning orientation. Future research could further test our

theory in a setting more conducive to long-term development of knowledge, skills, and abilities

and/or where such development is more immediately beneficial.

Other research opportunities relate to settings where misreporting is possible. The

negative effects of reporting frequency and evaluation purpose on actual performance may

exacerbate misreporting. That is, employees who exhibit poor performance as a result of

increased reporting frequency and evaluation pressure may feel a heightened need to misreport

22

their actual performance. In addition, the opportunity to misreport may mitigate the effect of

reporting frequency on actual performance, especially among individuals more likely to

misreport (i.e., those who experience less personal cost from misreporting, and thus are more

likely to be dishonest). Honest employees who are not willing to misreport will suffer the

negative performance effects of frequent reporting, while dishonest employees will be immune

to such effects because they can resort to misreporting. To the extent that such effects sustain

over time, dishonest employees may perform better than honest employees.

Relatedly, while our study focuses on performance reporting within a firm, the theory we

test can also inform the global debate about the frequency with which publicly traded companies

should report the results of their operations to the capital markets. For example, in June 2013, the

EU parliament voted to exempt all companies listed on EU stock exchanges from quarterly

financial reporting. Our theory suggests that when managers feel evaluative pressure from

external stakeholders, frequent external reporting can increase avoidance orientation and

undermine managerial performance, which in turn heightens the need to engage in accrual or real

earnings management (e.g., Bhojraj and Libby 2005; Gigler, Kanodia, Sapra, and Venugopalan

2014).

23

APPENDIX

Goal Orientation Questions about the Experimental Task

1. When I worked on the critical reasoning questions, it was important to me to perform well.

2. When I worked on the critical reasoning questions, it was important to me to do better than other people in this study.

3. My desire to perform well on the critical reasoning questions motivated me to work hard on the questions.

4. When I worked on the critical reasoning questions. I worried about the possibility of performing poorly.

5. When I worked on the critical reasoning questions, I worried about the possibility performing worse than other people in this study.

6. My fear of performing poorly on the critical reasoning questions motivated me to work hard on the questions.

7. When I worked on the critical reasoning questions, it was important to me to learn and improve my critical reasoning skill.

8. My desire to become more competent at critical reasoning motivated me to work hard on the questions.

Questions 1-3 capture participants’ approach goal orientation toward the experimental task, questions 4-6 capture participants’ avoidance goal orientation, and questions 7 and 8 capture participants’ learning goal orientation.

24

REFERENCES Amabile, T.M., and S.J. Kramer. 2011. The power of small wins. Harvard Business Review 89

(5): 70-80. Arnold, M., R. L., Hannan, I. D. Tafkov. 2015. Non-verifiable communication in homogeneous

and heterogeneous teams. Working paper, University of Bern, Tulane University, and Georgia State University.

Basu, S., J. Dickhaut, G. Hecht, K.L. Towry, and G. Waymire. 2009. Recordkeeping alters

economic history by promoting reciprocity. Proceedings of the National Academy of Sciences 106 (4): 1009-1014.

Basu, S., and G. Waymire. 2006. Recordkeeping and human evolution. Accounting Horizons 20

(3): 1-29. Bhojraj, J., and R. Libby. 2005. Capital market pressure, disclosure frequency-included

earnings/cash flow conflict, and managerial myopia. The Accounting Review 80: 1-20. Bilodeau, E.A. 1966. Acquisition of skill. New York, NY: Academic Press. Chow, C.W., J.C. Cooper, and K. Haddad. 1991. The effects of pay schemes and ratchets on

budgetary slack and performance: A multiperiod experiment. Accounting, Organizations and Society 16 (1): 47-60.

Chow, C.W., J.C. Cooper, and W.S. Waller. 1988. Participative budgeting: Effects of a truth-

inducing pay scheme and information asymmetry on slack and performance. The Accounting Review 63: 111-122.

Cook, D. M. 1968. Impact on managers of frequency of feedback. Academy of Management

Journal 11: 263-277. DeGeest, D., and K.G. Brown. 2011. The role of goal orientation in leadership development.

Human Resources Development Quarterly 22 (2): 157-175. Elliot, A. J. 1999. Approach and avoidance motivation and achievement goals. Educational

Psychologist 34 (3): 169-189. Elliot, A. J., and M. A. Church. 1997. A hierarchical model of approach and avoidance

achievement motivation. Journal of Personality and Social Psychology 72: 218-232. Elliot, A. J., and J. M. Harackiewicz. 1996. Approach and avoidance achievement goals and

intrinsic motivation: a mediational analysis. Journal of Personality and Social Psychology 70: 968-980.

25

Elliot, A.J., and H. McGregor. 1999. Test anxiety and the hierarchical model of approach and avoidance achievement motivation. Journal of Personality and Social Psychology 76: 628 – 644.

Evans, J. H., R. L. Hannan, R. Krishnan, and D. V. Moser. 2001. Honesty in managerial

reporting. The Accounting Review 76: 537-559 Fischbacher, U. 2007. z-Tree: Zurich toolbox for ready-made economic experiments.

Experimental Economics 10 (2): 171-178. Fisher, J., L.A. Maines, S.A. Peffer, and G.B. Sprinkle. 2002. Using budgets for performance

evaluation: Effects of resource allocation and horizontal information asymmetry on budget proposals, budget slack, and performance. The Accounting Review 77 (4): 847 – 865.

Gigler, F., C. Kanodia, H. Sapra, and R. Venugopalan. 2014. How frequent financial reporting

can cause managerial short-termism: an analysis of the costs and benefits of increasing reporting frequency. Journal of Accounting Research 52(2): 357-387.

Guidry, F., A. J. Leone, and S. Rock. 1999. Earnings-based bonus plans and earnings

management by business unit managers. Journal of Accounting and Economics 26: 113-42.

Harackiewicz, J.M., K.E. Barron, and A.J. Elliot. 1998. Rethinking achievement goals: When are

they adaptive for college students and why? Educational Psychologist 33: 1-21. Hales, J., J. L. Hobson, and R. J. Resutek. 2015. Does One Size Fit All? An Experimental

Investigation of Narcissism and Management Control Systems. Working Paper, Georgia Institute of Technology, University of Illinois at Urbana-Champaign, and University of Georgia.

Holmstrom, B. 1979. Moral hazard and observability. The Bell Journal of Economics 10 (1): 74-

91. Lam C.F., D. S. Derue, E. P. Karam, and J. R. Hollenbeck. 2011. The impact of feedback

frequency on learning and task performance: challenging the “more is better” assumption. Organization Behavior and Human Decision Processes 116: 217-228.

Libby, R., R.J. Bloomfield, and M.W. Nelson. 2002. Experimental research in financial

accounting. Accounting Organizations and Society 27 (8):777-812. Lurie, N. H., and J. M. Swaminathan. 2009. Is timely information always better? The effect of

feedback frequency on decision making. Organizational Behavior and Human Decision Processes 108: 315-329.

26

Maas, V. S., and M. van Rinsum. 2013. How control system design influences performance misreporting. Journal of Accounting Research 51 (5): 1159-1186.

Middleton, M., and C. Midgley. 1997. Avoiding the demonstrating of lack of ability: An

underexplored aspect of goal theory. Journal of Educational Psychology 89: 710-718. Salmoni, A. W., R. A. Schmidt, and C. B. Walter. 1984. Knowledge of results and motor

learning: a review and critical reappraisal. Psychological Bulletin 95: 355-386. Schumacker, R. and R. Lomax. 2004. A Beginner’s Guide to Structural Equation Modeling.

Mahwah, New Jersey: Erlbaum. Simons, R. 1994. How new top managers use control systems as levers of strategic renewal.

Strategic Management Journal 15 (March): 169-189. Simons, R., and H. Weston. 1990. USA Today. Case Study 9-191-004. Boston: Harvard Business

School. Sprinkle G. B., and M. G. Williamson. 2007. Experimental research in managerial accounting. In

Handbook of Management Accounting Research, edited by C. Chapman, A. Hopwood and M. Shields, 415-444. Elsevier Science.

Urdan, T. 1997. Achievement goal theory: past results, future directions. Advances in Motivation

and Achievement 10: 99-141. Vandewalle, D. 1997. Development and validation of a work domain goal orientation instrument.

Educational and Psychological Measurement 8: 995-1015. Waller, W.S. 1988. Slack in participative budgeting: The joint effects of a truth-inducing pay

scheme and risk preferences. Accounting, Organizations and Society 13: 87-98.

27

Panel A: Sumamry Statistics for Performancea

Low Frequencyb High Frequencyb

Evaluationc

Mean 17.33 13.14 Standard Deviation (6.89) (4.59) N 24 22No Evaluationc

Mean 15.10 17.00 Standard Deviation (6.21) (6.14) N 21 20

Panel B: Analysis of CovarianceFactors df MSE F -statistic p- valuee

Reporting Frequency (RF) 1 12.93 0.39 0.53

Evaluative Purpose (EP) 1 11.58 0.35 0.56

RF x EP 1 184.20 5.58 0.02

GPAd 1 283.96 8.61 <0.01

Error 81 32.99

Pannel C: Simple Effects of Reporting Frequency on Performancedf MSE F-statistic p-value e

within No Evaluation 1 46.24 1.40 0.24

within Evaluation 1 158.65 4.81 0.03

a

b

c

d

e All reported p -values are two-tailed. GPA represents participants' self-reported GPA.

Performance represents the number of critical reasoning questions participants answered correctly. Reporting Frequency is manipulated between subjects at two levels: Low Frequency and High Frequency. Participants in the Low Frequency condition were asked to report their performance to the administrator at the end of the task (i.e., after 30 minutes). Participants in the High Frequency condition were asked to report their performance to the administrator every 10 minutes.

Table 1The Effects of Reporting Frequency and Evaluative Purpose on Performance

Evaluative Purpose is manipulated between subjects at two levels: No Evaluation and Evaluation. Participants in the No Evaluation condition were informed that the administrator will use their performance report(s) to assess the quality of critical reasoning questions in the question bank. Participants in the Evaluation condition were informed that the administrator will use their performance report(s) to assess students' critical reasoning skill at the participants' institution.

28

Panel A: Summary Statistics for Avoidance Orientationa


Evaluationc



Panel B: Analysis of VarianceFactors df MSE F -statistic p- valued

Reporting Frequency (RF)b 1 1.52 0.77 0.38

Evaluative Purpose (EP)c 1 7.26 3.68 0.06

RF x EP 1 7.21 3.66 0.06

Error 81 1.97

Pannel C: Simple Effects of Reporting Frequency on Avoidance Orientationdf MSE F-statistic p-value d



a

b

c

d All reported p -values are two-tailed.

Table 2The Effects of Reporting Frequency and Evaluative Purpose on Avoidance Orientation

Avoidance Orientation is participants' average score of the three avoidance items in the goal orienation questionnaire (see Appendix) measured using a 7-point Likert scale.Reporting Frequency is manipulated between subjects at two levels: Low Frequency and High Frequency. Participants in the Low Frequency condition were asked to report their performance to the administrator at the end of the task (i.e., after 30 minutes). Participants in the High Frequency condition were asked to report their performance to the administrator every 10 minutes.

Evaluative Purpose is manipulated between subjects at two levels: No Evaluation and Evaluation. Participants in the No Evaluation condition were informed that the administrator will use their performance report(s) to assess the quality of critical reasoning questions in the question bank. Participants in the Evaluation condition were informed that the administrator will use their performance report(s) to assess students' critical reasoning skill at the participants' institution.

29

Panel A: Summmary Statistics for Learning Orientationa


Evaluationc



Panel B: Analysis of VarianceFactors df MSE F -statistic p- valued

Reporting Frequency (RF)b 1 3.94 1.47 0.23

Evaluation Purpose (EP)c 1 0.78 0.29 0.59

RF x E 1 2.84 1.06 0.31

Error 81 2.69

Pannel C: Simple Effects of Reporting Frequency on Learning Orientationdf MSE F-statistic p-value d



a

b

c

d All reported p -values are two-tailed.

Table 3The Effects of Reporting Frequency and Evaluative Purpose on Learning Orientation

Learning Orientation is participants' average score of the two learning items in the goal orienation questionnaire (see Appendix) measured using a 7-point Likert scale.Reporting Frequency is manipulated between subjects at two levels: Low Frequency and High Frequency. Participants in the Low Frequency condition were asked to report their performance to the administrator at the end of the task (i.e., after 30 minutes). Participants in the High Frequency condition were asked to report their performance to the administrator every 10 minutes.Evaluative Purpose is manipulated between subjects at two levels: No Evaluation and Evaluation. Participants in the No Evaluation condition were informed that the administrator will use their performance report(s) to assess the quality of critical reasoning questions in the question bank. Participants in the Evaluation condition were informed that the administrator will use their performance report(s) to assess students' critical reasoning skill at the participants' institution.

30

a

b

c

Figure 1

Reporting Frequency is manipulated between subjects at two levels: Low Frequency and High Frequency. Participants in the Low Frequency condition were asked to report their performance to the administrator at the end of the task (i.e., after 30 minutes). Participants in the High Frequency condition were asked to report their performance to the administrator every 10 minutes.Evaluative Purpose is manipulated between subjects at two levels: No Evaluation and Evaluation. Participants in the No Evaluation condition were informed that the administrator will use their performance report(s) to assess the quality of critical reasoning questions in the question bank. Participants in the Evaluation condition were informed that the administrator will use their performance report(s) to assess students' critical reasoning skill at the participants' institution. Performance represents the number of critical reasoning questions participants answered correctly.

The Effects of Reporting Frequencya and Evaluative Purposeb on Performancec

31

a

b

c

Figure 2

Reporting Frequency is manipulated between subjects at two levels: Low Frequency and High Frequency. Participants in the Low Frequency condition were asked to report their performance to the administrator at the end of the task (i.e., after 30 minutes). Participants in the High Frequency condition were asked to report their performance to the administrator every 10 minutes.Evaluative Purpose is manipulated between subjects at two levels: No Evaluation and Evaluation. Participants in the No Evaluation condition were informed that the administrator will use their performance report(s) to assess the quality of critical reasoning questions in the question bank. Participants in the Evaluation condition were informed that the administrator will use their performance report(s) to assess students' critical reasoning skill at the participants' institution. Avoidance Orientation is participants' average score of the three avoidance items in the goal orienation questionnaire (see Appendix) measured using a 7-point Likert scale.

The Effects of Reporting Frequencya and Evaluative Purposeb on Avoidance Orientationc

32

Panel A: Theoretical model

Panel B: Test of the theoretical model

**, *** Significant at the ≤0.05, and ≤0.01 levels, respectively.

Figure 3Path Analysis

Reporting Frequency

Avoidance Orientation

Learning Orientation

Performance

GPA

+ -

+ +-

Reporting Frequency

Avoidance Orientation

Learning Orientation

Performance

GPA

0.31** -0.44***

0.36***

-0.02 0.05

-0.14

The Effect of Performance Reporting Frequency on Employee ...

Documents

Transcript of The Effect of Performance Reporting Frequency on Employee ...