Operant Conditioning

Pavlovianlearning how stimuli provide informationabout other stimuli

Operantlearning how your behavior leads to goodand bad results

Natural selection effective organisms survive other ones die out results in complex well adapted organisms

Behavioral selection behavior leading to good outcomes

is strengthened other behavior dies out results in complex behavior that is

suitable for the environment

The Law of Effect(voluntary responses, not reflexes)

Responses followed by good thingsare strengthened

Responses followed by bad thingsare weakened

R Re responses (reward or reinforcer)

GoodConsequence

Bad Consequence

Behaviorproducesconsequences

Behavioreliminatesconsequences

Positivereinforcement

Omission

Punishment

NegativeReinforcement

The Law of Effect(voluntary responses, not reflexes)

Responses followed by good thingsare strengthened

Responses followed by bad thingsare weakened

R Re responses (reward or reinforcer)

GoodConsequence

Bad Consequence

Behaviorproducesconsequences

Behavioreliminatesconsequences

Positivereinforcement

Omission

Punishment

NegativeReinforcement

Increaseprobabilityof response

The Law of Effect(voluntary responses, not reflexes)

Responses followed by good thingsare strengthened

Responses followed by bad thingsare weakened

R Re responses (reward or reinforcer)

GoodConsequence

Bad Consequence

Behaviorproducesconsequences

Behavioreliminatesconsequences

Positivereinforcement

Omission

Punishment

NegativeReinforcement

Decreaseprobabilityof response

bar

pre

ss r

ate

acquisition

time

extinction

ratoutofbox

spontaneousrecovery

stopreinforcement

Operant conditioning may also be subjectto inhibition.

Contingency vs. Contiguity

Does the response have to predict the consequence?

Hammond (1980)

nofood food

positive

press bar 5 95 contingency

Rat learns

don’t press 0 100 to press!

nofood food

zero

press bar 5 95 contingency

Rat stops

don’t press 5 95 pressing or doesn’t learn

Supports Contingency

Conditioned Reinforcement

Skinner (1938)

Phase 1“click” . . . . . food

conditioned primaryreinforcer reinforcer

Phase 2Rats learn to press a bar to hearthe “click”

Other examples praisemoneygrades

Conditioned reinforcers only develop when theypredict primary reinforcer(more evidence for contingency)

Token Reinforcers

First train animals that tokens can be

exchanged for food

(e.g., vending machine)

Then see if animals will learn other responses

to get tokens

Chimpanzees

•tokens work as well as “real” reinforcers

•fight over the tokens

panel

Learned Helplessness (Seligman)

Phase I - Learning to Escape

Control Dogs Yoked Dogs

Shock

•A long lasting shock is given to both groups

every once in a while

•Control dogs can turn shock off by pushing a

panel

•Yoked dogs’ shock turns off too, when control

dog pushes panel

•Yoked dogs can do nothing themselves to escape

shock

Phase 2 - Avoidance Learning

•shock delivered to one side of box•if dog jumps hurdle to other side there is no shock

Control dogs learn to avoid shockYoked dogs don’t

Yoked dogs have learned that they can’t stop shockThey have learned to be helpless

hurdle

“Curing” learned helplessness

Curing learned helplessness requires the animal toexperience success

Depression is like learned helplessness

•passive•lose weight•have trouble learning new tasks

•Drag the dog over the hurdle a few times•Eventually learns to avoid shock

Preventing Learned Helplessness

If dogs are first exposed to shock that they can

turn off and then exposed to uncontrollable

shock, they don’t learn to be helpless.

Early Experience!

Methods for Studying Memory

STUDY TEST retention interval

Recall Tests

Free recall

Serial recall

Cued recall

steel thread

tights universe

cob batteries

eye cow

pear clarinet

nail pitch

airplane mittens

bunny sprinkler

spaghetti saucer

canary microphone

Recognition

DRUM

CORD Study

KITE

PARROT

yes-no Test

CLASS

KITE

CORD CORN Forced

Choice

KITE BIRD

HIT MISS

FALSEALARM

CORRECTREJECTION

OLD

ITEM

NEW

RESPONSE“YES” “NO”

Experiments with no study phase

Autobiographical Memory

•Recall all of your elementary school teachers

•Where were you when “Challenger” spaceshuttle blew up?

How do you know if recall is accurate?

Semantic Memory

Tests of general knowledge

Easy questions“Is an oak a tree?”measure response time

Hard questionsWhich is bigger, an eagle or a raccoon?

Savings in Relearning

Learn Relearnsomething the same thing

How many If it takestrials? few trials, you

have memory

% savings =

trials for trials fororiginal learning relearning

trials for original learning

Examplelearning ancient Greek passages

STUDY - 1-1/2 year old has Greek passages readto him

TEST - Age 8

TEST

10 new passages 30 trials

10 old passages 21 trials

% savings = = 30%

Conclude:

There is some memory for old passages

30-2130

“Classic” or Standard Theory of Memory

Selectivity of Encoding

Only some information is storedpermanently

ABCDEFGHIJKL

C DE ]FG

H

E

Rejects

conveyor

For closer examination

not good

enough

Hi-grade

Rejects

conveyor

For closer examination

not good

enough

Hi-grade

Information from sense organs

Sensory Memory

AttentionSystem

Short-Term Store

Forgotten

LongTermStore

Classic Theory

SensoryMemories

IconicStore

EchoicStore

Attention Filter

Short-TermStore

Long-TermStore

•large capacity•rapid turnover•“raw” information

•small capacity•symbolic information

•almost unlimitedcapacity

eyes ears

Short-Term Store

some metaphors

1. Box with Slots

(limitedcapacity)

2. A work bench

wherethinkingtakes place

parrot pizza jet

parrot jet

3. STS is the activated part of long-term store

4. STS is your consciousness

jetparrotpizza

parrot jetheli-

copter pizzaham-

burger

birdfly

plane food

eats

What is in STS?

Mental Symbols

sound of the word “dog”appearance of the letter “A”

world

STS LTSperception

retrieval from LTS

What happens to items in STS?

LTS

SensoryMemory

STS

forgottenif not

rehearsed

rehearsalkeeps item

in STS

transferredto LTS ifrehearsed

a lot or ifimportant

What do symbols inSTS stand for?

(Coding)

Classic Theory

words & letters are coded by sound(acoustic)

ORinternal articulationnot by meaning

Conrad (1964)serial recall of letters

confusions are acoustic

code must be acoustic

F D K Y X C M

right rite wright right write rite

B C P V E C T

How is STS Limited?

1. Number HypothesisSTS can hold only a fixed numberof symbols

7 2 (Miller)

2. Time HypothesisAny symbol in STS will be lost inaround 2 seconds if it is notreactivated by rehearsal

Span of STS is determined by decay rate (2 sec)and the rate of rehearsal

Baddeley, Thomson & Buchanan (1975)

tested memory span for words

memory span = number of items where immediate serial recall is perfect

short words “pig” “bet”long words “noon “rose” (take longer

to say toyourself)

Memory span was greater for short words

Conclude:Your memory span is the number of wordsyou can rehearse in 2 seconds

The Time Hypothesis is correct

Improving STS Capacity

1. Chunking with recoding

combine many symbols into one

T H E D O G

“the dog”

0 0 1 0 1 0 1 1 1

1 2 7

2. Chunking without recoding

organize symbols into groups

7 1 6 - 2 7 5 - 7 2 6 1

Are STS and LTS Really Different?

Classic Theory says “yes”!

time

capacity

cause offorgetting

nature ofsymbols(coding

STS

a fewseconds

a fewitems

decay or“gettingbumped

out”

acoustic/articulatoryfor words

LTS

“permanently”

“unlimited”

retrievalfailure

anything

Immediate Free-Recall Serial Position Curve

100%

recall

1 2 3 25

primacy

asymptote

recency

| | |

serial position

Immediate Free-Recall Serial Position Curve

100%

recall

1 2 3 25

primacy

asymptote

recency

| | |

serial position

nonrecencyLTS

STS

Dissociations in free recall serial position curve

1. No dissociation

Condition A iseasier than Bover all positions

2. A is better thanB for nonrecencyonly

A is better for LTS

3. A is better than Bfor recency only

A is better for STS

Recall A

B

A

B

A

B

Actual DissociationsSupporting STS-LTS Distinction

Independentvariable

presentationrate

SLOW vs. FAST

intervening taskbefore recall

orno task

old-age vs.college-age

subjects

amnesicvs.

normalsubjects

Effect onRecency

SLOW =FAST

NO TASK >TASK

YOUNG ≈OLD

AMNESIC ≥NORMAL

Effect onNonrecency

SLOW > FAST

NO TASK = TASK

YOUNG > OLD

AMNESIC = 0

Immediate Free-Recall Serial Position Curve

100%

recall

1 2 3 25

primacy

asymptote

recency

| | |

serial position

Immediate Free-Recall Serial Position Curve

100%

recall

1 2 3 25

primacy

asymptote

recency

| | |

serial position

Craik (1970)

Recency items are recalled worse than nonrecencyitems in final free recall

Supports STS-LTS distinction

NegativeRecencyEffect

FINAL FREE RECALL

Effects of Marijuana on Memory

Darley et al. (1973)

start •presentation and immediate free recall often lists

•administration of drug or placebo

after one hour -Phase 1 Final free recall of ten lists

Recall .4 -.2 -

serial position

Phase 2 Presentation and immediate free recall of ten new lists

Recall .8 -.6 -.4 -.2 -

serial position

Conclude: Drug hurts transfer from STS to LTS

No differenceDrug doesn’teffect retrievalfrom LTS

Drug isworse onnonrecencyonly

placebo

drug

drug

placebo

Problems with Classic Theory

(1) You can get recency even when STS isn’tinvolved

•Baddeley & Hitch (1977) “Rugby” experiment

•Roediger & Crowder (1976) “President”experiment

100%

serial position of presidents

Washington Lincoln Ford

Nixon

Recency

(2) The Classic Theory’s view of rehearsal is wrong

Craik & Watkins (1973)

Phase 1

keep track of most recent “b” word

“basket” “spoon” “telephone” “baby”

report “baby”

Rehearsal time

basket - 2 items

Phase 2

recall all “b” words

%recall

| | | | | | |0 1 2 3 4 5 6

Rehearsal Time

20—

10 —Actual Data

(2) The Classic Theory’s view of rehearsal is wrong

Craik & Watkins (1973)

Phase 1

keep track of most recent “b” word

“basket” “spoon” “telephone” “baby”

report “baby”

Rehearsal time

basket - 2 items

Phase 2

recall all “b” words

%recall

| | | | | | |0 1 2 3 4 5 6

Rehearsal Time

20—

10 —Actual Data

(sheer amount of rehearsal doesn’t matter always)

If classic theoryis right

Levels of ProcessingTheory of Memory

A modification of the Classic Theory

Craik & Lockhart (1972)

•memory is determined by encoding

•the “deeper” the processing during encoding, thebetter the memory

•amount of rehearsal is not important

•distinction between STS (primary memory) andLTS (secondary memory) is still present

Craik & Lockhart (1972)

2. Kinds of rehearsal

maintenance - keep repeating word

at shallow (phonemic) level

KEEPS WORD AVAILABLE, but DOES NOT improve long-term memory for word

elaborative - think about the meaning of the word in lots of different ways

(semantic)

does improve long-term memory

Relation Between Rehearsal and Perception

What happens when you perceive a word?

D O G

Stage 1 detect parts of letters(visual)

recognize letters “D” “O” “G”

Stage 2 convert letters to “phonemes”(phonemic)

D = d O = o G = hard g

Stage 3 convert phonemes to meaning(semantic)

“d o g” = pet that barks

Event:

D O G

letters(visual)

sound(phonemic)

meaning(semantic)

concentrateon letters

poormemory

concentrateon sound

fairlyshort

memory

concentrateon meaning

longlasting

memory

shallow

deep

1. Personally significant? 2. 2 syllables? 3. Animate? 4. Personally significant? 5. 2 syllables? 6. Animate? 7. Personally significant? 8. 2 syllables? 9. Animate?10. Personally significant?11. 2 syllables?12. Animate?13. Personally significant?14. 2 syllables?15. Animate?16. Personally significant?17. 2 syllables?18. Animate?19. Personally significant?20. 2 syllables?21. Animate?

Watkins & Watkins, 1974

Supports the idea that negative recency in finalrecall is due to shallow processing when list length is known (maintenance rehearsal)

listlengthknown

listlength

unknown

serial position

immediate recall

final recall

Support for Levels of Processing

Craik & Tulving (1975)

Orienting questions

Case word in capital letters

TABLEsubject responds yes

Rhyme word rhymes with weight?

MARKETsubject responds no

Sentence word fits in sentence

“He met a _______”

FRIENDsubject responds yes

Probabilityof “Hit” inRecognition

.8

.6

.4

.2

sentence rhyme case

Deeper means better memory

“yes” better than “no” (elaboration)

yes

yes

yesno

no

no

Some Criticisms

1. Theory fails to emphasize retrieval cues.

2. Theory fails to say exactly what is “deeper”processing

Fisher & Craik (1977)Shows that retrieval cues are as important as depth of processing during encoding.

STUDY

TEST

context - itempail - HAILmilk - COWmat - CAT

leaf - TREE

pail? (HAIL)dog? (CAT)

milk? (COW)free? (TREE)

rhyme contextsemantic contextrhyme contextsemantic context

cue encodingsame cue - rhymesemantic - rhymesame cue - semanticrhyme - semantic