Post on 16-Jan-2016
Operant vs. Classical
Operant responses understood by comprehending the consequences they produce
vs.
Pavlovian responses may have environmental effects, but this is not what controls them
Edward Thorndike
Acquisition of Goal-directed responses.
Law of Effect - behavior occurs in a random, trial-and-error fashion.
Consequences of a behavior can increase or decrease the future probability of that behavior. 1874-1949
Trials
Esc
ap
e T
ime
Cat Puzzle Box
Law of Effect
Responses followed by a satisfying outcome will strengthen the association between the situation and the response.
Responses followed by a dissatisfying outcome will weaken the association between the situation and the response.
Law of Effect
Press lever Escape from boxTime 1:
Time 2: Placed in box higher probability of pressing lever
Strengthens box-lever press association
Pull string Remain in boxTime 1:
Time 2: Placed in box lower probability of pulling string
Weakens box-string pulling association
Discrete-Trial ProceduresW. S. Small (1900) – used the maze as a tool to study operant learning in rats.
“complex” maze; measures = running speed, latency to reach goal.
Hampton Court Palace Maze
T-Maze
Start
FoodNo Food
Straight Alley “Maze”
Start Food
Operant (Instrumental) Conditioning
The Skinner Box:
Lever
2 Stages of training: 1. Magazine training
2. Shaping = reinforcement of successive approximations
Lever
Lever
Lever
Lever
Lever
Free-Operant Procedures
Discrete-trial procedures provide animal with limited opportunities to respond.
Free-operant procedures allow the animal to perform the instrumental response repeatedly without constraint.
Discrete-Trial Operant Procedure
Lever
Light = Discriminative stimulus (SD)
Acquisition and Extinction of Lever Press Response
Fixed
Variable
RatioInterval
Schedules of Reinforcement
Post-reinforcement pause
FI scallop
time
# re
spon
ses
Size of post-reinforcement pause determined by response-reinforcement ratio
PREE
What can be an operant?
Practically any behavior or behavioral parameter!
rate of responsetime of responsevariability of responsepushes and pullsposturestudy habitsathletic performancearts and craftscreativitybad habits and behavioral disorders
Allen et al. (1964)
• The case of Ann, a nursery school student
• Problems: asocial behavior, complaining, weird habits.
• Solution: Stop paying attention to her when she does these things.
Extinction of temper tantrums
Reinforcement without awareness
•Conditioning in amesics – eyeblink in HM
“Memento”
•Radio static study
Subjects told it was an experiment on stress.
Twitching of very small thumb muscle actually terminated the aversive stimulus (harsh noise)
•Students condition professor
Pigeon in a Pelican
Ginger
Marian Breland Bailey – How to train a chicken
The famous dancing chicken
Limits on Operant Conditioning
• Instinctive drift – “misbehavior.”
• Belongingness effects – reinforcer/behavior specificity e.g., behavior systems effects.
• Unintended consequences –Doing one thing precludes doing other things.
• Some responses cannot be conditioned.– E.g., yawning or sneezing.
Factors that affect O.C.
• The response– Maze running vs. lever pressing– Running vs. throwing football
• Temporal contiguity
Delayed reinforcement
Overcoming the effects of delay
• Secondary reinforcers
• “Marking” procedure
Factors that affect O.C.
• The response– Maze running vs. lever pressing– Running vs. throwing football
• Temporal contiguity
• Contingency
“Superstitious Behavior”
• Suggested that temporal contiguity more important than contingency
• 15-s FT, no response requirement
• “adventitious reinforcement”
“In 6 out of 8 cases the resulting responses were so clearly defined that two observers could agree perfectly in counting instances. One bird was conditioned to turn counter-clockwise about the cage, making 2 or 3 turns between reinforcements. Another repeatedly thrust its head into one of the upper corners of the cage….”
Orienting toward feeder
Pecking near feeder
Moving along wall
¼ turn
Degraded Contingency Effect
= bar press = food
Perfect contingency
Degraded contingency
Signal extra food
= light CSResponse
Strong
Strong
Weak
Factors that affect O.C.
• The response– Maze running vs. lever pressing– Running vs. throwing football
• Temporal contiguity• Contingency• The reinforcer
– Magnitude, quality– Upshifts and downshifts
Anticipatory Contrast - Crespi (1942)
00.5
11.5
22.5
33.5
44.5
2 4 6 8 10 12 14 16 18 20 2 4 6 8
Trials
Run
ning
Spe
ed (f
t/se
c)
256-16 Pellets16-16 Pellets1 - 16 Pellets
Rats run down maze to find food pellets in goal arm.
Reinforcement of Variability