Experimental Psychology Society

Experimental Psychology Society

Intertemporal choice, hyperbolic discounting, and mental time travel:

A comparative and evolutionary discussion

Stephen E. G. Lea, University of Exeter

Lancaster Conference, April 2013Symposium on Mental Time Travel

organised by Tony Dickinson

S. E. G. Lea 2

The Economic Problem:The stylised facts about inter-temporal choice

Humans are terrible at making choices between outcomes that appear at different times in the future

Everyday examples:• Extra sleep now vs. getting to work punctually in an

hour’s time• Having a washing machine now vs. being debt-free next

year• Luxury expenditure now vs. enough income in retirement• A cigarette now vs. living 20 years longerThis is by far the biggest deviation from rational choice in the whole of economic behaviour

EPS Lancaster, 11th April 2013

S. E. G. Lea 3

Experimental data support this, under many different names:

• Failure to delay gratification• Lack of self control• Weakness of will• Impulsiveness• Myopia• Short time-horizon• Inability to consider future consequences• [excessive] Delay discounting• Failure of self-regulation• ...etc


S. E. G. Lea 4

In what ways are we bad at intertemporal choice?

1. Impatience (Fisher, 1930): We choose the outcome that will arrive sooner even when it is much less valuable than the more delayed one, beyond any point that could be justified by a rational analysis

2. Inconsistency (Ainslie, 1974): We change our preferences between immediate and delayed outcomes as they (both) come closer in time – though this cannot happen if delay is discounted exponentially, as rationally it should be


S. E. G. Lea 5

In terms of discount rates:

1. We have to set a “subjective discount rate” at an absurdly high level (Friedman: 33%) to account for the (econometric) facts of human saving (and even higher for experimental data)

2. No single discount rate will account for all the inter-temporal choices that an individual makes even in one situation, let alone across situations


S. E. G. Lea 6

The psychological solution

• Suppose that the value of a future outcome declines according a hyperbolic rather than an exponential function of the time until it will be realised

• This automatically generates a discount rate that is higher for outcomes further in the future

• Hyperbolic discounting is now widely accepted by (behavioural) economists. It is arguably the most important concept to have been imported into economics from psychology


S. E. G. Lea 7

What is the psychological evidence for hyperbolic discounting?

• Herrnstein’s matching law, formulated to describe the behaviour of pigeons working on concurrent variable interval schedules of reinforcement asserts that:

R1/R2 = Rf1/Rf2

• In this free-operant situation, Ri is a rate of responding

(pecks/unit time) and Rfi is a rate of reinforcement (food

deliveries/unit time)• In discrete trial choice, there is exactly one reinforcement, after a

delay Di ; and the natural analogue of reinforcement rate is then simply 1/Di – leading to the prediction of hyperbolic discounting


S. E. G. Lea 8

Commitment• Ainslie and Rachlin used this analysis to design

experiments on pigeons, in which the birds showed both impatience and preference reversal, and choices were qualitatively consistent with the discrete-trial version of the matching law

• Mazur shows that the following form is more precise:Vi = Ai/(1+KDi)

Where Vi is “subjective value”, Ai is reward size, Di is delay, and K is a fitting constant

• Vi values can be used to predict choice across a wide range of animal choice experiments


S. E. G. Lea 9

Evolutionary argument No. 1• Matching is a consequence of basic processes of operant

conditioning / associative learning, common to all vertebrates, and at least some invertebrates, e.g. bees, molluscs

• It predicts “near enough” optimal behaviour in standard foraging situations, e.g. the optimal diet problem, faced by organisms of virtually all taxa

• We therefore predict that the laws of associative learning, in general, and matching in particular, will be evolutionarily conservative

• Humans should be no exception to any of thisEPS Lancaster, 11th April 2013

S. E. G. Lea 10

If you aren’t getting worried, you haven’t been paying attention…

1. We are explaining life-time, one-off, deliberative, human intertemporal choices by means of phenomena demonstrated in short-term, highly repetitive, contingency-governed, choices made by rats and pigeons.

2. A key step in predicting the phenomena is the use of the matching law, which in general does not hold for human deliberative choices (Horne and Lowe)

3. The discount rates required to account for commitment choices in the laboratory would be grotesque if extrapolated to the life-time scale – even by the standards of human intertemporal irrationality


S. E. G. Lea 11

…and how reliable is hyperbolic discounting anyway?

• Not all direct experimental tests of discount functions show a hyperbolic form in all participants

• Not all market data are consistent with hyperbolic rather than exponential discounting

• Not all demonstrations of preference reversal show reversal in the direction predicted by hyperbolic discounting (Read et al 2012)

• Intertemporal choices can be radically affected by framing of the question (Read et al 2013)…

• …or by the comparison set made available (Vlaev et al 2007)


S. E. G. Lea 12

ParadoxAs psychologists, what we have to explain is not why humans are so

bad at intertemporal choice, but why we are so (relatively) good at it… and

not why hyperbolic discounting occurs, but why it sometimes doesn’t


S. E. G. Lea 13

How, other than from the matching law, might we get hyperbolic discounting (or something like it)?

Two basic kinds of possibility1. Humans have some mechanism, either

specific instincts or a specific cognitive capacity, arising from our evolutionary past, that deals with the far future (i.e. more than a few seconds/minutes ahead)

2. Decisions about the far future are outside our evolutionary experience, so we deal with them as best we can using general-purpose cognitive mechanismsEPS Lancaster, 11th April 2013

S. E. G. Lea 14

Adaptations for long-term choices• Examples: – birds gaining weight before migration– hibernators gaining weight in the autumn– hoarders (both scatter hoarders e.g. some corvids,

some squirrels, and larder hoarders e.g. hamsters) spending much time and energy creating caches in the food-rich season

• These behaviours are subserved by specific physiological or instinctual mechanisms and/or specific cognitive capacities (e.g. seasonal change in size of stomach or hippocampus)

Red knot. Photograph by Jan van de Kam from PLoS, used with permission


S. E. G. Lea 15

A common selective pressure: Seasonal variation

• Such mechanisms have evolved because they enable animals to exploit particular niches, and in all these examples, it is seasonal variation in food supply

• It’s conceivable that1. During human evolution we had to cope with

such seasonal variations2. The mechanism for doing so is capable of being

extended beyond its natural one-year limitation


S. E. G. Lea 16

Life history strategies• However many animals also make “choices” between

patterns of behaviour that affect the entire course of their lives, not just survival in the coming season

• In evolutionary theory these patterns are referred to as life-history strategies

• A very common field for such choices is the timing of reproduction (the r/K decision axis)

• Although these differences in strategy are clearly facultative, to what extent can we regard them as being chosen? It’s plausible that they are subserved by evolved instincts


S. E. G. Lea 17

Human reproductive choice• Humans are long-lived, reproduce slowly, and

occupy many different ecological niches: these all predict multiple life-history strategies

• In existing hunter-gatherer societies, birth rate is well below the maximum achievable, implying some kind of trade-off to achieve maximum fitness (e.g. Kaplan, 1996)

• In all modern societies, birth rate has crashed (the demographic transition), implying that the trade-off is labile


S. E. G. Lea 18

Reproductive choice and economic choices

• Modern humans vary in speed of reproduction strategies• These variations are correlated with social class, economic status

and other indicators of environmental harshness and unpredictability (e.g. Quinlan, 2008; Ellis et al., 2009; Nettle, 2010)

• Increased use of credit and reduced saving are associated with the fast-reproduction (r) strategy

• However direct experimental tests or a correlation between life history strategies and high discount rates do not reliably support predictions (e.g. Wilson & Daly, 2006)


S. E. G. Lea 19

What about general cognitive mechanisms?

• In delayed gratification tasks, whether the contingencies are hypothetical or real, people need to imagine the future

• This is closely linked to recollecting the past – episodic memory

• Suddendorf & Corballis (1997, 2007) call these two capacities “Mental time travel” and claim that they constitute a uniquely human cognitive capacity (autonoetic consciousness: Tulving, 1985)

• The forward looking version of it is referred to as anticipation or episodic future thought


S. E. G. Lea 20

Temporal discounting and mental time travel

• Stewart, Chater & Brown (2003) propose Decision by Sampling (DbS) theory

• According to DbS, we compare a proposed delay with a sample of delays drawn from memory

• Because long delays are experienced less often than short delays, the distribution in the sample follows a power law (cf. Zipf’s law, etc)

• This yields (sub)hyperbolic discounting even with random sampling from memory, without any underlying psychoeconomic scale


S. E. G. Lea 21

Evolutionary argument No. 2• For whatever reason (there are many hypotheses),

humans, and perhaps a few other species, have evolved the capacity for reflective thought.

• This gives us a consciousness of the self – and thereby makes mental time travel over long periods possible

• This allows us to compare an imagined future with present reality; this might be the selective pressure that caused the capacity to evolve

• But whatever pressures produced this capacity, they did not involve modern, finance-based, choices, so there is no reason to expect these to be made optimally or accurately


S. E. G. Lea 22

An inevitable complication: the need for self-control

• Associative learning still works in humans, even if we have other cognitive processes available in addition

• So we have two mechanisms available for making many kinds of decisions – and they will often lead in opposite directions

• When we use reflective thought to override the response that would be made on the basis of the laws of conditioning, we call that self-control or self-regulation (though it would be better to call it control by the self)

• This “mental tool” is used in many situations – including inter-temporal choice

• There is much evidence that it is cognitively and even physiologically expensive to use


S. E. G. Lea 23

But (lack of) self-control is not the core problem

• Myopia and preference reversals occur even when self-control has not broken down. When operant conditioning completely takes over, the degree of myopia is even worse

• There are plausible evolutionary accounts of mental time travel, and its use to make value comparisons over months or years

• Hyperbolic discounting can be predicted by a general cognitive mechanisms, Decision by Sampling theory

• No mechanism has been proposed that would lead to the “economically rational” solution, exponential discounting


S. E. G. Lea 24

So how do humans discount?To assess the value of far distant rewards, we have to rely on:• Self control, to avoid the hyper-myopia

predicted by the matching law• Episodic future thought, to imagine the future

situation• And to assess its value– Either Decision by Sampling (or something like it)– Or rational calculation, if we have the theoretical knowledge

and computational ability , not to mention the required faith in economists


S. E. G. Lea 25

This means we are likely to observe:

• Substantial individual differences• Substantial intra-individual inconsistencies• Varying forms of the discount function• Little correspondence between actual and

rational discounting, either in terms of discount functions or discount rates

QED


S. E. G. Lea 26

Thanks to…

• Detlef Fetchenhauer, Erik Hölzl and their students at Köln, and Stephanie Preston and all at the Michingan Meeting on the Interdisciplinary Science of Consumption, for discussion of earlier versions of this paper

• Paul Webley, Lesley Newson, Avril Mewse, Wendy Wrapson, Gordon Brown, Peter Fischer and David Crelley for discussion of essential ideas


S. E. G. Lea 27

References• Ainslie, G. (1974). Impulse control in pigeons. Journal of the Experimental Analysis of Behavior, 21, 485-489.• Ashby, F. G., Alfonso-Reese, L. A., Turken, A. U., & Waldron, E. M. (1998). A neuropsychological theory of multiple systems in category learning. Psychological Review,

105, 442-481.• Barron, G., & Erev, I (2003). Small feedback-based decisions and their limited correspondence to description-based decisions. Journal of Behavioral Decision Making,

16, 215-233.• Baumeister, R. F., Bratslavsky, E., Muraven, M., & Tice, D. M. (1998). Ego depletion: Is the active self a limited resource? Journal of Personality and Social Psychology,

74, 1252-1265.• Clayton, N. S., Bussey, T. J., & Dickinson, A. (2003). Can animals recall the past and plan for the future? Nature Reviews Neuroscience, 4, 685-691.• Conger, R., & Killeen, P. (1974). Use of concurrent operants in small group research - demonstration. Pacific Sociological Review, 17, 399-416.• Fisher, I. (1930). The theory of interest. New York: Macmillan.• Frank, M. J., Cohen, M. X., & Sanfey, A. G. (2009). Multiple systems in decision making: A neurocomputational perspective. Current Directions in Psychological

Science, 18, 73-77.• Friedman, M. (1953). The methodology of positive economics. In M. Friedman (Eds.), Essays in Positive Economics, pp. 3-43. Chicago: University of Chicago Press.• Gigerenzer, G., & Todd, P. M. (1999). Fast and frugal heuristics: The adaptive toolbox. In G. Gigerenzer, P. M. Todd, and the ABC Research Group (Eds.), Simple

heuristics that make us smart. New York: Oxford University Press.• Herrnstein, R. J. (1961). Relative and absolute strengths of response as a function of frequency of reinforcement. Journal of the Experimental Analysis of Behavior, 4,

267-272.• Horne, P. J., & Lowe, C. F. (1993). Determinants of human performance on concurrent schedules. Journal of the Experimental Analysis of Behavior, 59, 29-60.• Kaplan, H (1996). A theory of fertility and parental investment in traditional and modern human societies. Yearbook of Physical Anthropology, 39, 91-135.• Katona, G. (1975). Psychological economics. New York: Elsevier.• Lea, S. E. G (1979). Foraging and reinforcement schedules in the pigeon: Optimal and non-optimal aspects of choice. Animal Behaviour, 27, 875-886.• Lea, S. E. G. (2001). Anticipation and memory as criteria for special welfare consideration. Animal Welfare, 10, S195-S208.• Nettle, D. (2010). Dying young and living fast: variation in life history across English neighborhoods. Behavioral Ecology, 21, 387-395.• Mazur, J. E. (1984). Tests of an equivalence rule for fixed and variable reinforcer delays. Journal of Experimental Psychology: Animal Behavior Processes, 10, 426-436.• Piersma, T., Dietz, M. W., Dekinga, A., Nebel, S., van Gils, J. A., Battley, P. F. & Spaans, B. (1999). Reversible size-changes in stomachs of shorebirds: when, to what

extent, and why? Acta Ornithologia, 34, 175-181.• Quinlan, R. J. (2008). Human Pair-Bonds: Evolutionary Functions, Ecological Variation, and Adaptive Development. Evolutionary Anthropology, 17, 227-238.• Rachlin, H., & Green, L. (1972). Commitment, choice, and self-control. Journal of the Experimental Analysis of Behavior, 17, 15-22.• Roberts, W. A. (2002). Are animals stuck in time? Psychological Bulletin, 128, 473-489.• Schneider, W., & Shiffrin, R. M. (1977). Controlled and automatic human information-processing: 1: Detection, search, and attention. Psychological Review, 84, 1-66.• Suddendorf, T., & Corballis, M. C. (1997). Mental time travel and the evolution of the human mind. Genetic Social and General Psychology Monographs, 123, 133-

167.• Suddendorf, T., & Corballis, M. C. (2007). The evolution of foresight: What is mental time travel, and is it unique to humans? Behavioral and Brain Sciences, 30, 299-

351.• Thaler, R. H., & Shefrin, H. M. (1981). An economic theory of self-control. Journal of Political Economy, 89, 392-406.• Wilson, M., & Daly, M (2006). Are juvenile offenders extreme future discounters? Psychological Science, 17, 989-994.


Experimental Psychology Society

Documents

Transcript of Experimental Psychology Society