1

Classical and InstrumentalConditioning

Lecture 8

2

Basic Procedure for Classical Conditioning

CS(Bell)

US(Meat Powder)

UR(Salivation)

CR(Salivation)

3

Major Phenomena ofClassical Conditioning

• Acquisition– CR to CS Reinforced by US– Response Gains Strength

• Magnitude of CR• Probability of CR

4

Acquisition: Trial 1

CS

US

CR

Bell

Food

Drops ofSaliva

Time

5

Acquisition: Trial 5

CS

US

CR

Bell

Food

Drops ofSaliva

Time

6

Acquisition: Trial 10

CS

US

CR

Bell

Food

Drops ofSaliva

Time

Acquisition of a Conditioned ResponseCS ==> US

0

0.2

0.4

0.6

0.8

1

1 2 3 4 5 6 7 8 9 10 11 12

Prob

abili

ty o

f CR

Trials

Negative Acceleration

Positive Acceleration

The “Sigmoidal” (S-Shaped) Learning Curve is Also Known as an Ogive

7

Two Shapes to the Learning Curve

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15

Res

pons

e S

treng

th

Trials

Naïve Organism/Complex Behavior

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15

Res

pons

e S

treng

th

Trials

Experienced Organism/Simple Behavior

8

9

Major Phenomena ofClassical Conditioning

• Acquisition• Extinction

– No Reinforcement– Response Loses Strength

10

Extinction: Trial 1

CS

US

CR

Bell

Food

Drops ofSaliva

Time

11

Extinction: Trial 5

CS

US

CR

Bell

Food

Drops ofSaliva

Time

12

Extinction: Trial 10

CS

US

CR

Bell

Food

Drops ofSaliva

Time

13

Extinction of a Conditioned ResponseCS ==> No US

0

0.2

0.4

0.6

0.8

1

1 2 3 4 5 6 7 8 9 10 11 12

Prob

abili

ty o

f CR

Trials

14

Major Phenomena ofClassical Conditioning

• Acquisition• Extinction• Spontaneous Recovery

– Rest After Extinction– Retest CS alone

15

Spontaneous Recovery After ExtinctionCS ==> No US

0

0.2

0.4

0.6

0.8

1

Prob

abili

ty o

f CR

Trials

Acquisition Extinction

SpontaneousRecovery

(Rest)

FurtherExtinction

16

Re-Acquisition of Extinguished ResponseCS ==> US

0

0.2

0.4

0.6

0.8

1

Prob

abili

ty o

f CR

Trials

Acquisition Extinction Re-Acquisition

SpontaneousRecovery

Savings inRelearning

(Rest)

17

Extinction Below Zero

• Extinction Trials Continued After CR Disappears Entirely

• Reduced Spontaneous Recovery• Less Savings in Relearning

– Slower Reacquisition

18

Major Phenomena ofClassical Conditioning

• Acquisition• Extinction

– Extinction Below Zero• Spontaneous Recovery• Re-Acquisition

– CS Reinforced by US– Savings in Relearning

19

Implications of Savingsand Spontaneous Recovery

• Conditioned Response Not “Lost”• Rather, Inhibited or Suppressed

– In Line with Changing Circumstances• Conditioned Stimulus No Longer Reinforced

• CR Can be Disinhibited– In Line with Changing Circumstances

• CS Reinforced Once Again

20

Major Phenomena ofClassical Conditioning

• Acquisition• Extinction• Spontaneous Recovery• Re-Acquisition• Generalization

– CS0 vs. CS1…CSn

– Generalization Gradient

21

The Generalization Gradient

00.10.20.30.40.50.60.70.80.9

1

-5 -4 -3 -2 -1 0 1 2 3 4 5

Prob

abili

ty o

f CR

Test Stimulus

OriginalCS

Lower Higher

250cps

200cps 300cps

350cps150cps

22

Major Phenomena ofClassical Conditioning

• Acquisition• Extinction• Spontaneous Recovery• Re-Acquisition• Generalization• Discrimination

– CS+ (Reinforced)– CS- (Unreinforced)

23

Discrimination LearningCS+ ==> US

CS- ==> No US

0

0.2

0.4

0.6

0.8

1

1 2 3 4 5 6 7 8 9 10 11 12

Prob

abili

ty o

f CR

Trials

CS+CS-

250 cps

200 cps

24

Review of Classical Conditioning

• Vocabulary– Unconditioned Stimulus– Unconditioned Response– Conditioned Stimulus– Conditioned Response

• Phenomena– Acquisition

• Reinforcement– Extinction– Spontaneous Recovery– Savings in Relearning– Generalization

• Generalization Gradient– Discrimination

25

Sensory Preconditioning

CS2

US

CR

Bell

Food

Saliva

Light

CS2

CS1

Bell

Saliva

LightCS1

CR

1

2

3

26

Higher-Order Conditioning

CS1

US

CR

Bell

Food

Saliva

Light

CS1

CS2

Bell

Saliva

LightCS2

CR

1

2

3

27

Significance of Classical Conditioning

• Extends Control of Reflexes to Other Environmental Events– Associations between Events

• Ubiquitous (Nervous System)• Pavlov: All Learning is Classical

Conditioning (?)• Laws of Classical Conditioning are the

Laws of Emotional Life

28

Thorndike’s Puzzle Box

29

Thorndike’s Results

30

Thorndike’s Laws of Learning

Law of Readiness

Law of Effect

Law of Exercise

31

Instrumental Conditioning(Operant Conditioning)

• Learn Adaptive Behavior– Through Experience of Success, Failure

• Organism Operates on Environment– Behavior Changes Environment

• Behavior Instrumental– Obtains Desired State of Affairs

• Associations between Behaviors and Outcomes

32

B.F. Skinner’s Operant Chamber

33

Instrumental Conditioning Procedure

• Phase 1: Baseline Behavior• Phase 2: Acquisition Phase• Phase 3: Discrimination Learning• Phase 4: Extinction

34

Vocabulary of Instrumental Conditioning

• Conditioned Response (No URs)• Conditioned Stimulus (No USs)• Reinforcement

– Positive– Negative (Not Punishment)

• Acquisition• Extinction• Generalization• Discrimination

35

Schedules of Reinforcement• Continuous• Partial• Intermittent

– Fixed Ratio (FR)– Variable Ratio (VR)– Fixed Interval (FI)– Variable Interval (VI)

• Differential Reinforcement– Of Low Rates (DRL)– Of High Rates (DRH)

36

Intermittent Reinforcement

The Matching LawHerrnstein (1970)

• Concurrent VI Schedules– Give Organism a Choice

• Key A: VI3 • Key B: VI1

• Response Rate is Proportional to the Frequency of Reinforcement– Also Magnitude, Delay of Reinforcement

• Basic Principle of Microeconomics– Supply and Demand

• Relative Value of Reinforcers 37

38

Significance of Instrumental Conditioning

• Voluntary Behaviors Come Under Control of Environmental Events– Behavior-Outcome Associations

• Ubiquitous (Vertebrates)• Thorndike, Skinner: All Learning is

Instrumental/Operant Conditioning (?)• Laws of Instrumental Conditioning Are

the Laws of Adaptive Behavior– Habits– Incentives