Chapter 6 – Schedules or Reinforcement and Choice Behavior Outline –Simple Schedules of...

52
Chapter 6 – Schedules or Reinforcement and Choice Behavior Outline Simple Schedules of Intermittent Reinforcement Ratio Schedules Interval Schedules Comparison of Ratio and Interval Schedules Choice Behavior: Concurrent Schedules Measures of Choice Behavior The Matching Law Complex Choice Concurrent-Chain Schedules Studies of “Self Control”

Transcript of Chapter 6 – Schedules or Reinforcement and Choice Behavior Outline –Simple Schedules of...

Page 1: Chapter 6 – Schedules or Reinforcement and Choice Behavior Outline –Simple Schedules of Intermittent Reinforcement Ratio Schedules Interval Schedules Comparison.

Chapter 6 – Schedules or Reinforcement and Choice Behavior

• Outline – Simple Schedules of Intermittent

Reinforcement• Ratio Schedules• Interval Schedules• Comparison of Ratio and Interval Schedules

– Choice Behavior: Concurrent Schedules• Measures of Choice Behavior• The Matching Law

– Complex Choice• Concurrent-Chain Schedules• Studies of “Self Control”

Page 2: Chapter 6 – Schedules or Reinforcement and Choice Behavior Outline –Simple Schedules of Intermittent Reinforcement Ratio Schedules Interval Schedules Comparison.

• Simple Schedules of Intermittent Reinforcement

• Ratio Schedules– RF depends only on the number of responses

performed• Continuous reinforcement (CRF)

– each response is reinforced• barpress = food• key peck = food

• CRF is rare outside the lab.– Partial or intermittent RF

Page 3: Chapter 6 – Schedules or Reinforcement and Choice Behavior Outline –Simple Schedules of Intermittent Reinforcement Ratio Schedules Interval Schedules Comparison.

• Partial or intermittent Schedules of Reinforcement

• FR (Fixed Ratio)– fixed number of operants (responses)

• CRF is FR1

– FR 10 = every 10th response RF• originally recorded using a cumulative

record– Now computers

• can be graphed similarly

Page 4: Chapter 6 – Schedules or Reinforcement and Choice Behavior Outline –Simple Schedules of Intermittent Reinforcement Ratio Schedules Interval Schedules Comparison.
Page 5: Chapter 6 – Schedules or Reinforcement and Choice Behavior Outline –Simple Schedules of Intermittent Reinforcement Ratio Schedules Interval Schedules Comparison.

• The cumulative record represents responding as a function of time– the slope of the line represents rate of

responding.• Steeper = faster

Page 6: Chapter 6 – Schedules or Reinforcement and Choice Behavior Outline –Simple Schedules of Intermittent Reinforcement Ratio Schedules Interval Schedules Comparison.

• Responding on FR scheds.– Faster responding = sooner RF

• So responding tends to be pretty rapid

– Postreinforcement pause• Postreinforcement pause is directly related to FR.

– Small FR = shorter pauses• FR 5

– large FR = longer pauses • FR 100

– wait a while before they start working.

– Domjan points out this may have more to do with the upcoming work than the recent RF• Pre-ratio pause?

Page 7: Chapter 6 – Schedules or Reinforcement and Choice Behavior Outline –Simple Schedules of Intermittent Reinforcement Ratio Schedules Interval Schedules Comparison.
Page 8: Chapter 6 – Schedules or Reinforcement and Choice Behavior Outline –Simple Schedules of Intermittent Reinforcement Ratio Schedules Interval Schedules Comparison.
Page 9: Chapter 6 – Schedules or Reinforcement and Choice Behavior Outline –Simple Schedules of Intermittent Reinforcement Ratio Schedules Interval Schedules Comparison.

• how would you respond if you received $1 on an FR 5 schedule?

• FR 500?– Post RF pauses?

• RF history explanation of post RF pause– Contiguity of 1st response and RF

• FR 5 – 1st response close to RF– only 4 more

• FR 100 – 1st response long way from RF – 99 more

Page 10: Chapter 6 – Schedules or Reinforcement and Choice Behavior Outline –Simple Schedules of Intermittent Reinforcement Ratio Schedules Interval Schedules Comparison.

• VR (Variable ratio schedules)– Number of responses still critical– Varies from trial to trial

• VR 10 – reinforced on average for every 10th

response. – sometimes only 1 or 2 responses are required– other times 15 or 19 responses are required.

Page 11: Chapter 6 – Schedules or Reinforcement and Choice Behavior Outline –Simple Schedules of Intermittent Reinforcement Ratio Schedules Interval Schedules Comparison.

• Example (# = response requirement)

VR10 FR10• 19 RF 10 RF• 2 RF 10 RF• 8 RF 10 RF• 18 RF 10 RF• 5 RF 10 RF• 15 RF 10 RF• 12 RF 10 RF• 1 RF 10 RF

• VR 10– (19+2+8+18+5+15+12+1)/8 = 10

Page 12: Chapter 6 – Schedules or Reinforcement and Choice Behavior Outline –Simple Schedules of Intermittent Reinforcement Ratio Schedules Interval Schedules Comparison.

• VR = very little postreinforcement pause– why would this be?

• Slot machines– very lean schedule of RF– But - next lever pull could result in a payoff.

Page 13: Chapter 6 – Schedules or Reinforcement and Choice Behavior Outline –Simple Schedules of Intermittent Reinforcement Ratio Schedules Interval Schedules Comparison.

• FI (Fixed Interval Schedule)– 1st response after a given time period has

elapsed is reinforced. • FI 10s

– 1st response after 10s RF.• RF waits for animal to respond• responses prior to 10-s not RF.

• scalloped responding patterns– FI scallop

Page 14: Chapter 6 – Schedules or Reinforcement and Choice Behavior Outline –Simple Schedules of Intermittent Reinforcement Ratio Schedules Interval Schedules Comparison.
Page 15: Chapter 6 – Schedules or Reinforcement and Choice Behavior Outline –Simple Schedules of Intermittent Reinforcement Ratio Schedules Interval Schedules Comparison.
Page 16: Chapter 6 – Schedules or Reinforcement and Choice Behavior Outline –Simple Schedules of Intermittent Reinforcement Ratio Schedules Interval Schedules Comparison.

• Similarity of FI scallop and post RF pause?– FI 10s?– FI 120s?

• The FI scallop has been used to assess animals’ ability to time.

Page 17: Chapter 6 – Schedules or Reinforcement and Choice Behavior Outline –Simple Schedules of Intermittent Reinforcement Ratio Schedules Interval Schedules Comparison.

• VI (variable interval schedule)– Time is still the important variable – However, time elapse requirement varies around a set

average• VI 120s

– time to RF can vary from a few seconds to a few minutes

• $1 on a VI 10 minute schedule for button presses?– Could be RF in seconds– Could be 20 minutes

• post reinforcement pause?

Page 18: Chapter 6 – Schedules or Reinforcement and Choice Behavior Outline –Simple Schedules of Intermittent Reinforcement Ratio Schedules Interval Schedules Comparison.

• Produces stable responding at a constant rate– peck..peck..peck..peck..peck– sampling whether enough time has passed

• The rate on a VI schedule is not as fast as on an FR and VR schedule– why?– ratio schedules are based on response.

• faster responding gets you to the response requirement quicker, regardless of what it is?

– On a VI schedule # of responses don’t matter,• steady even pace makes sense.

Page 19: Chapter 6 – Schedules or Reinforcement and Choice Behavior Outline –Simple Schedules of Intermittent Reinforcement Ratio Schedules Interval Schedules Comparison.

• Interval Schedules and Limited Hold– Limited hold restriction

• Must respond within a certain amount of time of RF setup

– Like lunch at school• Too late you miss it

Page 20: Chapter 6 – Schedules or Reinforcement and Choice Behavior Outline –Simple Schedules of Intermittent Reinforcement Ratio Schedules Interval Schedules Comparison.

• Comparison of Ratio and Interval Schedules– What if you hold RF constant

• Rat 1 = VR • Rat 2 = Yoked control rat on VI

– RF is set up when Rat 1 gets to his RF• If Rat 1 responds faster, RF will set up sooner for

Rat2• If Rat 1 is slower, RF will be delayed

Page 21: Chapter 6 – Schedules or Reinforcement and Choice Behavior Outline –Simple Schedules of Intermittent Reinforcement Ratio Schedules Interval Schedules Comparison.

• Comparison of Ratio and Interval Schedules

Page 22: Chapter 6 – Schedules or Reinforcement and Choice Behavior Outline –Simple Schedules of Intermittent Reinforcement Ratio Schedules Interval Schedules Comparison.

• Why is responding faster on ratio scheds?– Molecular view

• Based on moment x moment RF• Inter-response times (IRTs)

– R1……………R2 RF» Reinforces long IRT

– R1..R2 RF » Reinforces short IRT

• More likely to be RF for short IRTs on VR than VI

Page 23: Chapter 6 – Schedules or Reinforcement and Choice Behavior Outline –Simple Schedules of Intermittent Reinforcement Ratio Schedules Interval Schedules Comparison.

• Molar view– Feedback functions

• Average RF rate during the session is the result of average response rates

– How can the animal increase reinforcement in the long run (across whole session)?• Ratio - Respond faster = more RF for that day

– FR 30– Responding 1 per second RF at 30s– Respond 2 per second RF at 15s

Page 24: Chapter 6 – Schedules or Reinforcement and Choice Behavior Outline –Simple Schedules of Intermittent Reinforcement Ratio Schedules Interval Schedules Comparison.

• Molar view continued– Interval - No real benefit to responding faster

• FI 30• Responding 1 per second RF at 30 or 31 (30.5)• What if 2 per second 30 or 30.5 (30.25)

– Pay • Salary?• Clients?

Page 25: Chapter 6 – Schedules or Reinforcement and Choice Behavior Outline –Simple Schedules of Intermittent Reinforcement Ratio Schedules Interval Schedules Comparison.

• Choice Behavior: Concurrent schedules– The responding that we have discussed so far

has involved schedules where there is only one thing to do.

– In real life we tend to have choices among various activities

– Concurrent schedules • examines how an animal allocates its responding

among two schedules of reinforcement?• The animals are free to switch back and forth

Page 26: Chapter 6 – Schedules or Reinforcement and Choice Behavior Outline –Simple Schedules of Intermittent Reinforcement Ratio Schedules Interval Schedules Comparison.
Page 27: Chapter 6 – Schedules or Reinforcement and Choice Behavior Outline –Simple Schedules of Intermittent Reinforcement Ratio Schedules Interval Schedules Comparison.

• Measures of choice behavior– Relative rate of responding

• for left key

BL .

(BL + BR)

– BL = Behavior on left

– BR = Behavior on right

We are just dividing left key responding by total responding.

Page 28: Chapter 6 – Schedules or Reinforcement and Choice Behavior Outline –Simple Schedules of Intermittent Reinforcement Ratio Schedules Interval Schedules Comparison.

• This computation is very similar to the computation for the suppression ratio.– If the animals are responding equally to each

key what should our ratio be? 20 . = .50

20+20

– If they respond more to the left key? 40 . = .67

40+20

– If they respond more to the right key? 20 . = .33

20+40

Page 29: Chapter 6 – Schedules or Reinforcement and Choice Behavior Outline –Simple Schedules of Intermittent Reinforcement Ratio Schedules Interval Schedules Comparison.

• Relative rate of responding for right key– Will be reciprocal of left key responding, but also

can be calculated with the same formula

BR .

(BR + BL)

• Concurrent schedules?– If VI 60 VI 60– The relative rate of responding for either key will

be .5• Split responding equally among the two keys

Page 30: Chapter 6 – Schedules or Reinforcement and Choice Behavior Outline –Simple Schedules of Intermittent Reinforcement Ratio Schedules Interval Schedules Comparison.

• What about the relative rate of reinforcement?– Left key?

• Simply divide the rate of reinforcement on the left key by total reinforcement.

rL .

(rL + rR)

• VI 60 VI 60?– If animals are dividing responding equally?– .50 again

Page 31: Chapter 6 – Schedules or Reinforcement and Choice Behavior Outline –Simple Schedules of Intermittent Reinforcement Ratio Schedules Interval Schedules Comparison.

• The Matching Law– relative rate of responding matches relative

rate of RF when the same VI schedule is used• .50 and .50

– What if different schedules of RF are used on each key?

Page 32: Chapter 6 – Schedules or Reinforcement and Choice Behavior Outline –Simple Schedules of Intermittent Reinforcement Ratio Schedules Interval Schedules Comparison.

• Left key = VI 6 min (10 per hour)• Right key = VI 2 min (30 per hour)

Left key relative rate of responding

BL . = rL . 10 =.25 left

(BL + BR) (rL + rR) 40

Right key?

simply the reciprocal

.75

Can be calculated though

BR . = rR . 30 =.75 right

(BR + BL) (rR + rL) 40

Thus - three times as much responding on right key .25x3 = .75

Page 33: Chapter 6 – Schedules or Reinforcement and Choice Behavior Outline –Simple Schedules of Intermittent Reinforcement Ratio Schedules Interval Schedules Comparison.

Matching Law continued: Simpler computation.

BL . = rL .

BR rR

10

30

again – three times as much responding on right key

Page 34: Chapter 6 – Schedules or Reinforcement and Choice Behavior Outline –Simple Schedules of Intermittent Reinforcement Ratio Schedules Interval Schedules Comparison.

• Herrnstein (1961) compared various VI schedules – Matching Law.

• Figure 6.5 in your book

Page 35: Chapter 6 – Schedules or Reinforcement and Choice Behavior Outline –Simple Schedules of Intermittent Reinforcement Ratio Schedules Interval Schedules Comparison.
Page 36: Chapter 6 – Schedules or Reinforcement and Choice Behavior Outline –Simple Schedules of Intermittent Reinforcement Ratio Schedules Interval Schedules Comparison.

• Application of the matching law– The matching law indicates that we match our behaviors to

the available RF in the environment.– Law,Bulow, and Meller (1998)

• Predicted adolescent girls that live in RF barren environments would be more likely to engage in sexual behaviors

• Girls that have a greater array of RF opportunities should allocate their behaviors toward those other activities

• Surveyed girls about the activities they found rewarding and their sexual activity

• The matching law did a pretty good job of predicting sexual activity

– Many kids today have a lot of RF opportunities.• May make it more difficult to motivate behaviors you want them to do

– Like homework» X-box» Texting friends» TV

Page 37: Chapter 6 – Schedules or Reinforcement and Choice Behavior Outline –Simple Schedules of Intermittent Reinforcement Ratio Schedules Interval Schedules Comparison.

• Complex Choice– Many of the choices we make require us to

live with those choices• We can’t always just switch back and forth

– Go to college?– Get a full-time job?

• Sometimes the short-term and long-term consequences (RF) of those choices are very different

– Go to college» Poor now; make more later

– Get a full-time job» Money now; less earning in the long run

Page 38: Chapter 6 – Schedules or Reinforcement and Choice Behavior Outline –Simple Schedules of Intermittent Reinforcement Ratio Schedules Interval Schedules Comparison.

• Concurrent-Chain Schedules• Allows us to examine these complex choice behaviors

in the lab

– Example• Do animals prefer a VR or a FR?

– Variety is the spice of life?

Page 39: Chapter 6 – Schedules or Reinforcement and Choice Behavior Outline –Simple Schedules of Intermittent Reinforcement Ratio Schedules Interval Schedules Comparison.
Page 40: Chapter 6 – Schedules or Reinforcement and Choice Behavior Outline –Simple Schedules of Intermittent Reinforcement Ratio Schedules Interval Schedules Comparison.

• Choice of A– 10 minutes on VR 10

• Choice of B– 10 minutes on FR 10

• Subjects prefer the VR10 over the FR10– How do we know?

• Subjects will even prefer VR schedules that require somewhat more responding than the FR – Why do you think that happens?

Page 41: Chapter 6 – Schedules or Reinforcement and Choice Behavior Outline –Simple Schedules of Intermittent Reinforcement Ratio Schedules Interval Schedules Comparison.

• Studies of Self control– Often a matter of delaying immediate

gratification (RF) in order to obtain a greater reward (RF) later.• Study or go to party?• Work in summer to pay for school or enjoy the time

off?

Page 42: Chapter 6 – Schedules or Reinforcement and Choice Behavior Outline –Simple Schedules of Intermittent Reinforcement Ratio Schedules Interval Schedules Comparison.

• Self control in pigeons?– Rachlin and Green (1972)

• Choice A = immediate small reward• Coice B = 4s Delay large reward

– Direct choice procedure• Pigeons choose immediate, small reward

– Concurrent-chain procedure• Could learn to choose the larger reward

– Only if a long enough delay between initial choice and the next link.

Page 43: Chapter 6 – Schedules or Reinforcement and Choice Behavior Outline –Simple Schedules of Intermittent Reinforcement Ratio Schedules Interval Schedules Comparison.
Page 44: Chapter 6 – Schedules or Reinforcement and Choice Behavior Outline –Simple Schedules of Intermittent Reinforcement Ratio Schedules Interval Schedules Comparison.

• This idea that imposing a delay between a choice and the eventual outcomes helps organisms make “better” (higher RF) outcomes works for people to.

• Value-discounting function V = M .

(1+KD)• V-value of RF• M- magnitude of RF• D – delay of reward• K – is a correction factor for how much the animal is influenced by

the delay

– All this equation is saying is that the value of a reward is inversely affected by how long you have to wait to receive it.

– IF there is no delay D=0• Then it is simply magnitude over 1

Page 45: Chapter 6 – Schedules or Reinforcement and Choice Behavior Outline –Simple Schedules of Intermittent Reinforcement Ratio Schedules Interval Schedules Comparison.

• If I offer you– $50 now or $100 now?

50 . = 50 100 . = 100

(1+1x0) (1+1x0)

– $50 now or $100 next year?

50 . = 50 100 . = 7.7

(1+1x0) (1+1x12)

Page 46: Chapter 6 – Schedules or Reinforcement and Choice Behavior Outline –Simple Schedules of Intermittent Reinforcement Ratio Schedules Interval Schedules Comparison.
Page 47: Chapter 6 – Schedules or Reinforcement and Choice Behavior Outline –Simple Schedules of Intermittent Reinforcement Ratio Schedules Interval Schedules Comparison.

• As noted above K is a factor that allows us to correct these delay functions for individual differences in delay-discounting

• People with steep delay discounting functions will have a more difficult time delaying immediate gratification to meet long-term goals– Young children– Drug abusers

• Madden, Petry,Badger, and Bickel (1997)– Two Groups

• Heroin-dependent patients• Controls

– Offered hypothetical choices• $ smaller – now• $ more – later

– Amounts varied • $1,000, $990, $960, $920, $850, $800, $750, $700, $650, $600, $550, $500, $450,

$400, $350, $300,$250, $200, $150, $100, $80, $60, $40, $20, $10, $5, and $1– Delays varied

• 1 week, 2 weeks, 2 months, 6 months, 1 year, 5 years, and 25 years.

Page 48: Chapter 6 – Schedules or Reinforcement and Choice Behavior Outline –Simple Schedules of Intermittent Reinforcement Ratio Schedules Interval Schedules Comparison.
Page 49: Chapter 6 – Schedules or Reinforcement and Choice Behavior Outline –Simple Schedules of Intermittent Reinforcement Ratio Schedules Interval Schedules Comparison.
Page 50: Chapter 6 – Schedules or Reinforcement and Choice Behavior Outline –Simple Schedules of Intermittent Reinforcement Ratio Schedules Interval Schedules Comparison.

•  • It has been described mathematically in the following way (Baum, 1974)•  • RA = b rA a

• RB rB•  • RA and RB refer to rates of responding on keys A and B (i.e. left and right)

•  • rA and rB refer to the rates of reinforcement on those keys

•  • When the value of exponent a is equal to 1.0 a simple matching relationship occurs

where the ratio of responses perfectly match the ratio of reinforcers obtained.•  • The variable b is used to adjust for response effort differences between A an B

when they are unequal, or if the reinforcers for A and B were unequal.

Page 51: Chapter 6 – Schedules or Reinforcement and Choice Behavior Outline –Simple Schedules of Intermittent Reinforcement Ratio Schedules Interval Schedules Comparison.
Page 52: Chapter 6 – Schedules or Reinforcement and Choice Behavior Outline –Simple Schedules of Intermittent Reinforcement Ratio Schedules Interval Schedules Comparison.