Instrumental Conditioning

Also called Operant Conditioning

Pavlovian Conditioning = Stimulus learningInstrumental Conditioning = Response learning

Instrumental behavior = behavior that occurs because it was previously instrumental in producing certain consequences

Also called ‘goal-directed’ behavior

Early work in this area done by Thorndike (late 1800s) put cats in puzzle boxes the cats had to manipulate a latch to open the door,escape and get food initially the cats would thrash about randomly untilthey accidentally opened the door then the latency to escape and get food would decreaseover successive trials

Trials

Latency

The behavior, or response, is instrumental in, or responsible for, the outcome (i.e., escaping and getting food).

Thus, this type of learning became known as ‘instrumental conditioning’

Thorndike interpreted the results of his experiment as reflecting the learning of an S-R association

Thorndike believed the cats learned an associationbetween the stimuli inside the puzzle box and the escape response

The consequence of the successful response – escaping the box – strengthened the association between the boxstimuli and that response

On the basis of his work, Thorndike formulated the law of effect

Law of effect

this law states that if a response in the presence of a stimulus is followed by a satisfying event, the association between the stimulus (S) and the response (R) is strengthened if the response is followed by an annoying event, the S-R association is weakened according to the law of effect, animals learn an association between the response and the stimuli present at the time of the response – the consequence of the response is not one of the elements in the association the satisfying or annoying consequence simply serves to strengthen or weaken the association between the response and the stimulus situation

Modern approaches to the study of Instrumental Conditioning

Discrete-Trial Procedures involves a single response performed only at a certain time

Goal

Start

rat is put in start arm and runs togoal arm where it receives food

only single action (or response sequence) and reward is given

then rat removed from the apparatus after ITI the animal is placed in the startarm again for another trial each response is a discrete action. The speed and onset of thebehavior is determined by the subject. The experimenter determines when the subject may begin the action (usually by putting the rat in the start arm)

Modern approaches to the study of Instrumental Conditioning

Free-Operant Procedures more like what you will do in the lab with bar-pressing here the experimenter decides which behavior is correct butthe subject determines when the behavior will be executed your rats will be put in an operant chamber and allowed to respond at their own pace

subjects can make the response and receive reward more thanonce an operant response (such as a bar-press) is defined in terms ofthe effect that it has on the environment – the critical thing is not the muscles involved in the behavior but the way in which the behavior ‘operates’ on the environment

Cumulative Recorder

Cumulative Record

way of presenting data in free-operant procedures one response builds on the previous response

Response Measures with discrete-trial procedures can measure speed, and latency to make the response with free-operant procedures can measure rate of responding (#BP/min)

Studies can also combine elements of both discrete-trialand free-operant procedures

For ex., L tells subject when to respond and the subject can then respond at its own pace

Responses are freely emitted by the subject, but somecontrol by the experimenter

Magazine Training and Shaping

in order for instrumental conditioning to occur, the subject must make the desired response prior to receiving the reward

how do we train rats to make the response (i.e., bar-press)?

the preliminary phase is called magazine training• the food-delivery device is called the food magazine• when food is delivered the device makes a noise• after enough pairings of this noise with food delivery, the rat will go to the food cup when it hearsthe noise

Magazine Training and Shaping

after magazine training, the animal is ready to learn theinstrumental response

we ‘teach’ the response through a process called responseshaping

shaping involves:• reinforcement of successive approximations to therequired response• and nonreinforcement of earlier response forms

Instrumental Conditioning Procedures

There are 4 basic types of instrumental conditioning

These 4 types are categorized according to:

1. Nature of the outcome controlled by the behavior

a. Appetitive stimulus – pleasant outcome (food)b. Aversive stimulus – unpleasant outcome (shock)

2. Relationship or contingency between the responseand the outcome

a. Positive contingency – R produces O

b. Negative contingency – R eliminates/prevents O

Instrumental Conditioning Procedures

Appetitive Aversive

PositiveContingency(R produces O)

NegativeContingency(R prevents O)

PositiveReinforcement

Punishment

NegativeReinforcement

OmissionTraining

Instrumental Conditioning Procedures

1. Positive reinforcement – also called reward training

• response produces an appetitive outcome that is notas likely to occur otherwise

• positive contingency between the response and anappetitive stimulus• example, rat bar-press for food• result: response increases

Instrumental Conditioning Procedures

2. Punishment

• positive contingency between the response and anaversive stimulus• if the subject performs the response, it receives the aversive outcome• if the subject does not perform the response, it doesnot receive the aversive outcome• example, shock a rat whenever it makes a BPor mother scold child for playing in the street• result: response decreases

Instrumental Conditioning Procedures

3. Negative reinforcement

• negative contingency between the response and anaversive stimulus• the response terminates or prevents the delivery of an aversive stimulus

• example, put a rat in a box and give him continuousshocks. The rat could jump over a barrier to turn offor escape the shock.

• this is called an escape procedure • subjects can also avoid an aversive stimulus that isscheduled to occur

Instrumental Conditioning Procedures

3. Negative reinforcement (Escape/Avoidance)

• example, students can study before an exam to avoidgetting a bad grade • example, a rat may be scheduled to receive a shock at the end of a warning stimulus. If he makes a certain response (BP or jump over a barrier) before the warning stimulus is over, he avoids getting shocked• result: response increases

Instrumental Conditioning Procedures

4. Omission Training

• negative contingency between the response and anappetitive stimulus• the response prevents the delivery of a pleasantevent• if the subject does not make the response, the reward is delivered• example, a child is told to go to his room when he doessomething bad • example, suspending someone’s driver’s license forimpaired driving • result: response decreases

Instrumental Conditioning Procedures

4. Omission Training

• usually with omission training, the subject can receivethe reward for engaging in other behaviors• sometimes referred to as DRO•Differential Reinforcement of Other behaviors