FUR XII, LUISS, Roma, June 24, 2006 Financially Stimulated Effort Hits Individual Cognitive...

FUR XII, LUISS, Roma, June 24, 2006

Financially Stimulated Effort HitsIndividual Cognitive Constraints:

Evidence From a Forecasting Task with Varying Working Memory Load

by Ondrej Rydval, CERGE-EI, Prague

Acknowledgements: invaluable comments especially from Andreas Ortmann and Nat Wilcox, financial support from

Grant Agency of the Czech Republic and Hlavka Foundation

COMMENTS WELCOME!!!

MOTIVATION

I examine how performance-contingent financial incentives interact with intrinsic motivation and cognitive constraints in determining individual differences in cognitive performance.Camerer and Hogarth (1999, JRU) propose a capital-labor framework describing how financial incentives may interact in non-trivial ways with intrinsic motivation to induce cognitive effort (labor), and how cognitive effort productivity may vary across individuals due to their different cognitive constraints (capital).

Even if salient financial incentives induce high effort, both financial and cognitive resources may be wasted for individuals whose cognitive constraints inhibit performance improvements. This prediction, if warranted, calls for attention to individual cognitive constraints in designing efficient incentive schemes in firms, experimental settings, and elsewhere.

The next slide shows the main blocks of the capital-labor framework…

financialincentives

cognitiveperformance

CognitiveProduction

cognitiveeffort

cognitivecapital

intrinsicmotivation

Labor theory of cognitionCapital-LaborFramework

Labor theory of cognition: Smith &

Walker (1993), Wilcox (1993)

Camerer & Hogarth (1999)

Benabou & Tirole trilogy,

Cacioppo et al. (1996)

Experimental psychology:Engle & Kane (2004)

Crowding out?

Here is how one can think of the capital-labor framework.

I briefly outline the literature and the main research questions:

Degree of complementarit

y?

DESIGN

I provide an initial empirical test of the capital-labor framework, focusing on the complementarity of cognitive capital and effort.

To impose theoretical structure on the framework, one can broadly think of cognitive constraints as a vector composed of general cognitive capital and task-specific capital.Drawing on contemporary cognitive psychology, I measure individual differences in general cognitive capital by a working memory span test – a strong and robust predictor of general fluid intelligence as well as performance in cognitive tasks requiring controlled information processing.Since pre-existing task-specific capital (think of expertise) is vital for performance in many field cognitive tasks but is hard to measure, I intentionally minimize its potential relevance by designing a controlled laboratory experiment where working memory is itself the main component of task-specific capital, aside expertise acquired endogenously through on-task learning.

The next slide shows what working memory is and why it is a useful measure of individual differences in cognitive capital…

Working memory is a domain-general ability to control and rapidly reallocate attention among competing cognitive uses, over and beyond domain-specific short-term memory capacity. People with high working memory are better able to code and store a limited amount of task-relevant information and keep this information accessible during the execution of complex, information-interfering cognitive and behavioral tasks.

A typical working memory span test has two interacting components:- processing component: e.g. calculating simple equations “(9/3)-2=?”- memory component: e.g. memorizing a sequence of letters

1. In the test, subjects observe several sequences with alternating processing and memory components (sequences have various length).

2. After a given sequence, subjects must recall the memory components in correct order (e.g. letters).

3. Throughout the test, subjects must also maintain accuracy/speed on the processing component.

4. Subjects’ working memory score depends on the number of memory components recalled in correct order.

Note: A typical short-term memory test only has the memory component.

DESIGN cont.

To identify the impact of working memory on cognitive performance, I supply “external memory” to subjects as a treatment:

In a computerized time-series forecasting task, two screens with forecast-relevant information are presented either concurrently or sequentially => two between-subject treatments.

HYPOTHESIS to be tested: Since the Sequential (Concurrent) treatment offers less (more) “external memory” to subjects and hence features a higher (lower) working memory load, working memory should be a stronger (weaker) determinant of forecasting performance, after controlling for other between-subject cognitive, motivational and personality differences.

The next several slides show how I experimentally implemented the time-series forecasting task and in particular the Sequential and Concurrent treatments…

Time-series forecasting task (a la Klayman, 1988)

Per

iod

Sig

nal

Rep

eatin

g pa

tter

n

Err

or

Om

ega

1 40 46 4 902 20 34 -8 463 30 18 8 564 10 46 0 565 20 34 4 586 40 18 0 587 30 46 -4 728 10 34 8 529 20 18 8 4610 10 46 0 5611 40 34 -4 7012 10 18 -8 2013 30 46 0 7614 20 34 4 5815 10 18 -4 2416 30 46 -8 6817 20 34 4 5818 30 18 8 5619 10 46 0 5620 40 34 -4 70

Subjects repeatedly forecast Omega, a sum of Signal, Repeating pattern and Error.

They observe 8-period history windows of Signal and Omega (see the green columns).

Subjects are told that to accurately forecast Omega, they need to discover the Repeating (seasonal) pattern from successive values of Omega and Signal by subtracting them. The unobserved, random Error makes this task harder but subjects know the Error distribution (uniform discrete).

Subjects forecast next-period value of Omega. To be able to do that (as Signal is unpredictable), they are shown next-period value of Signal.

Concurrent treatment subjects observe Signal and Omega on one screen.

Sequential treatment subjects observe Signal and Omega on two successive screens.

Financial incentives to forecast accurately are high: subjects can earn up to 70-80 PPP Dollars.

Current period

15 of 100 Time remaining 15

Signal Omega

Period 8 10 64

Period 9 20 50

Period 10 10 24

Period 11 40 90

Period 12 10 36

Period 13 30 52

Period 14 20 66

Current period 15 10 48

Next period 1630

?

Here is what Concurrent treatment subjects observe on a typical screen (for period 15): subjects combine Signal and Omega values to forecast period-16 Omega.

Current period


Signal

Period 8 10

Period 9 20

Period 10 10

Period 11 40

Period 12 10

Period 13 30

Period 14 20

Current period 15 10

Next period 1630

By contrast, Sequential treatment subjects first observe a screen with Signal values only: they memorize Signal values and wait for the corresponding Omega screen…

Current period


Omega

Period 8 64

Period 9 50

Period 10 24

Period 11 90

Period 12 36

Period 13 52

Period 14 66

Current period 15 48

Next period 16 ?

…and once the corresponding Omega screen appears, Sequential treatment subjects combine the previously memorized Signal values with the observed Omega values to forecast period-16 Omega.

Period

ma12error1 ma12error2

10 20 30 40 50 60 70 80 90 100

0

5

10

15

20

Forecasting performance – absolute forecast errors

The graph below shows 12-period moving averages of absolute forecast errors, averaged across subjects in each forecasting period, separately for the Sequential and Concurrent treatment.

Sequential (average)

Concurrent (average)

EARLY LATE

abso

lute

for

ecas

t er

rors

Period

ma12error1 ma12error2

10 20 30 40 50 60 70 80 90 100

0

5

10

15

20

EARLY LATE

Forecasting performance – absolute forecast errors

The Concurrent treatment (with lower working memory load) has lower absolute forecast errors (on average) throughout the task…

…but statistically significant learning occurs in both treatments between EARLY and LATE stages of the task.



abso

lute

for

ecas

t er

rors

Period

ma12error1 ma12error2 p10ma12error1 p90ma12error1

10 20 30 40 50 60 70 80 90 100

0

5

10

15

20

EARLY LATE

Heterogeneity in forecasting performance

Same graph, with 10th and 90th

percentiles added to averages, reveals substantial across-subject heterogeneity in absolute forecast errors in both treatments.

As hypothesized, working memory should better explain the heterogeneity in the Sequential treatment (with higher working memory load). I focus on the LATE stage.


90th percentiles for Concurrent (o) and

Sequential (+)

10th percentiles for Concurrent (o) and

Sequential (+)


abso

lute

for

ecas

t er

rors

Med

late

OspanTotal20 30 40 50 60 70 80

0

5

10

15

20

Med

late

OspanTotal20 30 40 50 60 70 80

0

5

10

15

20

Correlations between forecasting performance and working memoryL

AT

E a

bso

lute

fo

reca

st e

rro

rL

AT

E a

bso

lute

fo

reca

st e

rro

r

working memory

working memory

Concurrent treatment:spearman = -0.0006 (p=0.997)pearson = -0.2208 (p=0.155)

Sequential treatment:spearman = -0.3028 (p=0.048)pearson = -0.4540 (p=0.002)

As hypothesized, working memory is much stronger negatively correlated with subjects’ LATE absolute forecast errors in the Sequential treatment (with higher working memory load).

By contrast, other measured cognitive, motivational, personality and demographic individual differences cannot explain between-subject performance variation in the Sequential treatment. I nevertheless control for these in the formal analysis that follows…

Testing the hypothesis that working memory is a stronger determinant of forecasting performance in the Sequential treatment

• I regress LATE absolute forecast error on working memory and other potential determinants of forecasting performance: short term memory, math ability, intrinsic motivation, etc. The figure below shows only several selected specifications as explained by the labels.• As some subjects’ performance is top-bounded, I use Censored Least Absolute Deviations (CLAD) estimator. So far I have 43 observations per treatment (students from Prague non-selective universities).• The bars are coefficient estimates for working memory. The estimates for the Sequential treatment generally have economically meaningful magnitude. A yellow bar indicates that working memory has a significant impact on forecasting performance (at 10% level). A green bar indicates that, in addition, the impact of working memory is significantly larger in the Sequential treatment (at 10% level).

Sequential

Concurrent

Estimation with working memory only

…with short term memory and math ability added

…with intrinsic motivation further added

…with EARLY performance added as a proxy for intrinsic forecasting ability (covariates partialled out of the proxy)…same, but with an alternative working memory measure that has short term memory and math ability partialled out

Estimate for the hardest forecasting season only, with covariates added

…same, but with EARLY performance proxy added

Testing the hypothesis that working memory is a stronger determinant of forecasting performance in the Sequential treatment

Sequential

Concurrent

Estimation with working memory only

…with short term memory and math ability added

…with intrinsic motivation further added

…with EARLY performance added as a proxy for intrinsic forecasting ability (covariates partialled out of the proxy)…same, but with an alternative working memory measure that has short term memory and math ability partialled out

Estimate for the hardest forecasting season only, with covariates added

…same, but with EARLY performance proxy added

• Conclusion: The impact of working memory (cognitive capital) on forecasting performance is clearly stronger in the Sequential than in the Concurrent treatment…•…but establishing this result more robustly may require more observations.

Working memory, or rather lack thereof, clearly presents a cognitive constraint on forecasting performance.

In additional treatments (not completed), the working memory constraint is interacted with variation in financial incentives by offering subjects to purchase “external memory” at different relative prices: subjects start in the harder Sequential treatment but can purchase switching to the easier Concurrent treatment.

What individual characteristics, beside working memory, will determine buying behavior? Will more external memory be bought under higher financial incentives?

In each period, subjects also bet on the quality of their forecasts, prior to placing a forecast. Financial return to betting is decreasing in subjects’ absolute forecast error.

Especially psychologists have argued that performance in cognitive tasks is likely to be also affected by people’s confidence in their abilities. Bets provide a measure of confidence in one’s forecasting abilities. Initial results suggest that working memory affects how quickly bets respond to improvements in forecasting accuracy.

I will next investigate a two-equation system where both bets and performance are treated as a result of dynamic learning processes, using exogenous variation in the quality of forecasting feedback to identify the impact of bets on performance.

CONCLUSION / ROAD AHEAD

FUR XII, LUISS, Roma, June 24, 2006 Financially Stimulated Effort Hits Individual Cognitive...

Documents

Transcript of FUR XII, LUISS, Roma, June 24, 2006 Financially Stimulated Effort Hits Individual Cognitive...