Lecture 17: Instrumental Conditioning (Associative Structures)

Learning, Psychology 5310

Spring, 2015

Professor Delamater

Instrumental Learning: Associative Structures

1. S – R association2. R – O association3. S – O association4. S – [R–O] association

• Lots of different types of learning could occur and account forwhy the animal increases its responding with training.

S – R – O (3-term Contingency): What associations do they learn?

Thorndike’s Law of Effect (revisited)

• Cats learned to escape puzzle box to obtain food reward.

• Thorndike assumed this was caused by the development of an S-R association.

• Reinforcement “stamped in” this association, without itself being learned about.

Evidence for R – O Associations in Instrumental Learning

Reward Devaluation Experiment

• During the test phase, subjects chose to work for the outcome that had not beendevalued.

• This implies that they “knew” what they were working for, i.e., they acquired anR – O association.

R1-O1 Devalue O1 R1 vs R2 (R1 < R2)R2-O2 No Deval of O2

R1 – Tobacco Devalue Tobacco R1 vs R2R2 – Chocolat No Deval of Choc

Instr Train Outcome Deval Test

Hogarth & Chase, 2011 Study

Evidence for R – O Associations in Instrumental Learning

Reward Devaluation Experiment

• During the test phase, subjects chose to work for the outcome that had not beendevalued.

• This implies that they “knew” what they were working for, i.e., they acquired anR – O association.

R1-O1 Devalue O1 R1 vs R2 (R1 < R2)R2-O2 No Deval of O2

R1 – Tobacco Devalue Tobacco R1 vs R2R2 – Chocolat No Deval of Choc

Instr Train Outcome Deval Test

Hogarth & Chase, 2011 Study

Evidence for R – O & S – R Associations in Instrumental Learning

Adams (1982) Experiment: Extensive vs Limited training

• One set of groups given 100 Lever-Pellet rewards, another given 500.

Lever Press – PelletsInstr Train (100 or 500) Outcome Deval Test

Evidence for R – O & S – R Associations in Instrumental Learning

Adams (1982) Experiment: Extensive vs Limited training

• One set of groups given 100 Lever-Pellet rewards, another given 500.• One of each set then was given pellet devaluation, and the other no pellet devaluation.

Lever Press – Pellets Pellets – Illness Lever Press ?Lever Press – Pellets Pellets | Illness Lever Press ?

Instr Train (100 or 500) Outcome Deval Test

Evidence for R – O & S – R Associations in Instrumental Learning

Adams (1982) Experiment: Extensive vs Limited training

• One set of groups given 100 Lever-Pellet rewards, another given 500.• One of each set then was given pellet devaluation, and the other no pellet devaluation.• Interestingly, the devaluation effect goes away with extensive instrumental training.• This suggests that as the instrumental response becomes “habitual” the S-R association

more strongly controls the response.• However, with more limited instrumental training, the response is more strongly

controlled by the R-O association.

Lever Press – Pellets Pellets – IllnessLever Press ?

Lever Press – Pellets Pellets | IllnessLever Press ?

Instr Train (100 or 500) Outcome Deval Test

Evidence for S – O Associations in Instrumental Learning

1. Rescorla & Solomon (1967) Two Process Theory:S – O & S – R associations are learned.

The stimulus (S) comes to “motivate” responding.

It does this because S associates with the emotional

aspects of the reinforcing outcome.

Evidence for S – O Associations in Instrumental Learning

1. Rescorla & Solomon (1967) Two Process Theory:S – O & S – R associations are learned.

The stimulus (S) comes to “motivate” responding.

It does this because S associates with the emotional

aspects of the reinforcing outcome.Pavlovian – instrumental transfer test

• The Pavlovian CS (Tone) increases instrumental lever pressing when it is paired with food.• But it decreases instrumental lever pressing when it is paired with foot shock (CER).

Evidence for S – O Associations in Instrumental Learning

1. Rescorla & Solomon (1967) Two Process Theory:S – O & S – R associations are learned.

2. Reward-specific Outcome expectancies:S – O associations can motivate instrumental

responding because the stimulus associates with the specific features of the reinforcing outcome.

• The Pavlovian CSs exert a highly specific effect on instrumental responding.• This implies that each CS associated with the sensory-specific features of the Outcome.

R1-O1 CS1-O1 CS1: R1 vs R2 (R1 > R2)R2-O2 CS2-O2 CS2: R1 vs R2 (R1 < R2) R1 vs R2 (R1 = R2)

Instr Train Pav Train TestKruse et al, 1983 Study

Evidence for S – [R – O] Associations in Instrumental Learning

S1: R1-O1, R2-O2

Tone: R1-Pel, R2-Sucr

R1 = Left lever pressR2 = Right lever press

Instr Train Outcome Deval Test

Colwill & Rescorla, 1990 Study

Evidence for S – [R – O] Associations in Instrumental Learning

S1: R1-O1, R2-O2S2: R1-O2, R2-O1

Tone: R1-Pel, R2-SucrLight: R1-Sucr, R2-Pel

Instr Train Outcome Deval Test

Colwill & Rescorla, 1990 Study

Now, each of two Rs are reinforced with each of the two Os, but differentR-O relations are signaled by different Ss.

Evidence for S – [R – O] Associations in Instrumental Learning

S1: R1-O1, R2-O2 Devalue O1S2: R1-O2, R2-O1 No Deval of O2

Tone: R1-Pel, R2-Sucr Pel – IllnessLight: R1-Sucr, R2-Pel Sucr – No Illness

Instr Train Outcome Deval Test

Colwill & Rescorla, 1990 Study

What would be the effect of devaluing one of the reinforcing outcomes?

Evidence for S – [R – O] Associations in Instrumental Learning

S1: R1-O1, R2-O2 Devalue O1 S1: R1 vs R2S2: R1-O2, R2-O1 No Deval of O2 S2: R1 vs R2

Tone: R1-Pel, R2-Sucr Pel – Illness Tone: R1 vs R2Light: R1-Sucr, R2-Pel Sucr – No Illness Light: R1 vs R2

Instr Train Outcome Deval Test

Colwill & Rescorla, 1990 Study

This was assessed in a test phase conducted under extinction conditions….

Evidence for S – [R – O] Associations in Instrumental Learning

• The rats selectively avoided the R when it signaled the devalued O in the presence of a specific S.

• The behavior must have been controlled by hierarchical S-[R-O] associations.

S1: R1-O1, R2-O2 Devalue O1 S1: R1 vs R2(S1: R1 < R2)S2: R1-O2, R2-O1 No Deval of O2 S2: R1 vs R2 (S2: R1 > R2)

Tone: R1-Pel, R2-Sucr Pel – Illness Tone: R1 vs R2Light: R1-Sucr, R2-Pel Sucr – No Illness Light: R1 vs R2

Instr Train Outcome Deval Test

Colwill & Rescorla, 1990 Study