Engaging students outside of class as a tool for enhanced learning

55
Engaging students outside Engaging students outside of class as a tool for of class as a tool for enhanced learning enhanced learning Dan Bernstein Dan Bernstein University of Kansas University of Kansas UBC Centre for Teaching and Academic Growth 22 November 2006 [email protected]

description

Engaging students outside of class as a tool for enhanced learning. Dan Bernstein University of Kansas UBC Centre for Teaching and Academic Growth 22 November 2006 [email protected]. Unhappy with answers to questions. - PowerPoint PPT Presentation

Transcript of Engaging students outside of class as a tool for enhanced learning

Engaging students outside of Engaging students outside of class as a tool forclass as a tool for

enhanced learning enhanced learning

Dan BernsteinDan BernsteinUniversity of KansasUniversity of Kansas

UBC Centre for Teaching and Academic Growth 22 November 2006

[email protected]

Unhappy with answers to questions• What is the major point of quantum mechanics What is the major point of quantum mechanics

that makes prediction impossible? What level of that makes prediction impossible? What level of prediction is affected? What level is not?prediction is affected? What level is not?

• What effect do quantum mechanics have on What effect do quantum mechanics have on scientific objectivity as part of our process of scientific objectivity as part of our process of knowing?knowing?

• Describe the Heisenberg uncertainty principle Describe the Heisenberg uncertainty principle (briefly). What relevance do you think it has for (briefly). What relevance do you think it has for contemporary research and clinical practice in contemporary research and clinical practice in psychology?psychology?

• Does quantum theory render Newtonian Does quantum theory render Newtonian mechanics useless to humans in their interactions mechanics useless to humans in their interactions with the physical environment? Why or why not?with the physical environment? Why or why not?

Simple problem led me onto a path of continuous inquiry that

has lasted a decade

Started with Experimental Studies• In search of the best way to generate high

level student answers to those questions

• Compared multi-media presentation with lecture, reading, and irrelevant information

• Measured rote and complex understanding with multiple choice, fill-in, short essay

• Advantages of random assignment and controlled conditions

• Goal ultimately a practical one

Basic Procedures• Commercial and home-made multi-media

• Used video and live lecture with same content as multi-media

• Created text samples with same content

• Assessment of levels of understanding with mixed methods of responding

• Individual instruction with unlimited time

• Participants from introductory psychology

Learning from Lecture

0

5

10

15

20

25

30

35

40

<= 69 70-79 80-89 90-100

Lecture

Learning from Lecture v. Reading

0

10

20

30

40

50

60

<= 69 70-79 80-89 90-100

LectureReadings

Lecture, Reading, and Computer

0

10

20

30

40

50

60

<= 69 70-79 80-89 90-100

LectureReadingsComputer

Percentage Correct by Instructional Mode and Level of

Understanding

0

10

20

30

40

50

60

70

Computer Lecture Readings

RoteApplicationEvaluative*

Percentage Correct by Instructional Mode and Question Type

0

10

20

30

40

50

60

70

80

Computer Lecture Readings

Multiple-choice*Fill-in-the-blankEssay

Mean Instructional Time by Instructional Mode

0

5

10

15

20

25

30

Computer Lecture Readings

Instructional time*

Upgraded the lecture component

• Live lecture to portion of introductory class

• Topic is reactive measurement and quantum mechanics

• Matched to locally authored multi-media and text -- also irrelevant text control

• Motivation sustained by possibility of test

• Assessment layered by type of understanding and format of question

Distribution of total test performance resulting from four methods of

instruction

0%

10%

20%

30%

40%

50%

60%

<= 59

60-69

70-79

80-89

90-100

<= 5960-6970-7980-8990-100

Number of Correct Responses Across Levels of Understanding

0

2

4

6

8

10

12

14

rote application evaluation

lecturecomputerreadingcontrol

Time on task (in minutes)

0

10

20

30

40

50

60

70

Computer Lecture Reading

What do you make of these data?

• How would it help me teach?

• Any ideas for next innovation?

• Critical reactions to the evidence?

My take home messages• Students learning by computer or reading

showed greater and deeper understanding• Multi-media has potential but very high

cost to get started• Reading consistently took the least time• Reading is a good way to learn if you have Reading is a good way to learn if you have

a way to assure that it is donea way to assure that it is done

Next problem arose when I presented my data

• Challenged by Tom Hatch, from Gardner’s Project Zero group

• Described “Deep Understanding”– Use ideas in a context not taught– Ideas retained without memorization

• I presumed that abstract was more generalI presumed that abstract was more general

• Agreed to test the idea in my class

Tracking understanding over time

• Learning and motivation• Sophomore level course• 20 to 45 students• Taught 9 times in 7 years• Professional grade reading• Hybrid online and F2F• Largely discussion in class• All essay examinations• Conceptual writing required

Wanted Richer Essay Exams• Why is magazine training so critical to the process of Why is magazine training so critical to the process of

shaping a new response class? Sometimes the sound of shaping a new response class? Sometimes the sound of the operation of the food magazine is called a bridging the operation of the food magazine is called a bridging stimulus; what might this metaphor mean?stimulus; what might this metaphor mean?

• What is the evolutionary mechanism for the cultural What is the evolutionary mechanism for the cultural survival of religious taboos against slaughtering and eating survival of religious taboos against slaughtering and eating cattle? You should identify the conditions that make this cattle? You should identify the conditions that make this evolution most likely.evolution most likely.

• Explain the notion of selection by consequences as a Explain the notion of selection by consequences as a description of the change in distribution of behavior that description of the change in distribution of behavior that takes place during shaping. How is this process different takes place during shaping. How is this process different from selection that takes place under naturally occurring from selection that takes place under naturally occurring conditions?conditions?

Reasonably Conceptual Items• Describe the results of [Razran’s] second experiment on Describe the results of [Razran’s] second experiment on

semantically mediated generalization (numbers) done semantically mediated generalization (numbers) done with a 13-year old boy. Be sure to identify the with a 13-year old boy. Be sure to identify the respondent behavior, the training stimuli, and the test respondent behavior, the training stimuli, and the test stimuli. Why is this example semantically rather than stimuli. Why is this example semantically rather than physically mediated generalization?physically mediated generalization?

• What do chicken scratches, raccoon rubbings and pig What do chicken scratches, raccoon rubbings and pig rooting have in common? What stimuli have pre-existing rooting have in common? What stimuli have pre-existing relations these bits of behavior? What does the relations these bits of behavior? What does the “misbehavior of organisms” have to do with the generality “misbehavior of organisms” have to do with the generality of principles of learning? of principles of learning?

Transform into Problems• Old question:

– What were the reinforcing consequences in the Welsh, Bernstein & Luthans (restaurant) study? How were the consequences identified?

• Problem-based Assessment :– Suppose you were asked to implement a motivational

program in a business with 25 employees engaged in the following activities: production planning, inventory delivery, direct production, packaging, and marketing. Based on your understanding of the restaurant study, how would you improve the quality of the employees work by using access to activities as a motivator? Describe the costs and benefits of the program and make a recommendation about implementation.

• Old question:– The behavioral context into which punishment is

added must be considered in evaluating the likely effect of a given stimulus. What conditions are essential in this analysis?

• New question:– Suppose it is your task to decrease the frequency of

college students engaging in binge drinking or games involving high levels of intoxication. You have been asked to set up a punishment program to eliminate this program before someone is hurt. What context for binge drinking would you identify first? What punishing consequence would you use? What characteristics would you include in your punishing system to maximize effectiveness? Please give an example of an additional element you would need to include to make the plan maximally effective.

Points for Question 1• 4pts - reinforcement for drinking

• 5 pts - ID specific severe punishing consequence

• 4 pts - Practical plan for measurement of abusive drinking

• 12 pts - Immediate consequence (4 pts); severity (more than change action) (4 pts); consistent system (4pts)

• 5 pts - reinforcement plan for alternative actions that are preferred, if possible using original class of reinforcers

• Old question:

• Describe the results of the experiment on semantically mediated generalization done with a 13-year-old Soviet boy. Be sure to identify the respondent behavior, the training stimuli, and the test stimuli. Why is this example semantically rather than physically mediated generalization?

• New question:

• Generate your own example of classical conditioning using the neutral stimuli “idea” and “rock” along with the eliciting relation between a very loud noise and increased heart rate. Your example should include all of the following components: a description of a conditioning procedure that would produce differing reactions to the two stimuli, a description of the procedure that tests for the direct effects of conditioning, a description of a procedure that would test for physically mediated generalization, a description of a procedure that would test for semantically mediated generalization, and the likely results of the three test procedures.

Points for Question 2• 4 pts - specific conditioning -- say “idea” (2)

and then puff of air in eye (2)

• 7 pts - Direct test: Present “idea” (3) w/o air (2) and see blink (2) follows

• 7 pts - Physically mediated generalization: Present sound similar to “idea” (3) w/o air (2) and see weak or little blink (2)

• 7 pts - Semantically mediated generalization: Present sound or picture or word related to “idea” (3) w/o air (2) and see strong blink (2) more than physically mediated

Use existing work:Keep an archive of evidence

Distribution on 1st Attempt

05

10152025303540

Percent Students

<= 69 70-79 80-89 90-100

Levels of Achievement

Old questions

Change of Questions

05

101520253035404550

Percent Students

<= 69 70-79 80-89 90-100

Levels of Achievement

Old questions New questions

What do you make of the data?

• Is this a surprise?• What examples do you

have in your own work?• What does it take to

“problematize” a concept for assessment?

• Keep going in this direction or turn back to abstraction?

I relied on my experimental results

• The key is getting students to read

• If only they were prepared, we would all be better off

• Post hoc argument from fluency -- free up mental resources for problem solving [away from details]

Organizing units by topics

Online questions on reading

Feedback until mastery of topics

Students consider reading

Complete roster services

Click on name for full record

See items, make comments

Aggregate Performance 1st Try

0

5

10

15

20

25

30

35

Percent Students

<=69 70-79 80-89 90-100

Levels of Achievement

New Questions

Addition of EDU Out of Class

05

10152025303540

Percent Students

<=69 70-79 80-89 90-100

Levels of Achievement

New Questions Web-aided

What do you make of these data?Would you keep using it? I did.

Next idea came from Grant Wiggins

• Welding teacher story

• Generate internalized criteria

• Like composition studies and editing

• Used prior student work with permission

• No feedback from me

Bb - consider others’ work

Students share reactions w/o me

Exam - Eliminate Problem Actions

0

10

20

30

40

50

60

70

Percent Students

(0-3) (4-6) (7-9) (10-12) (13-15)

Categories of Achievement

No Web W/ Examples

Exam Question onCultural Anthropology

01020304050607080

Percent Students

(0-4) (5-8) (9-12) (13-16) (17-20)

Categories of Achievement

No Web W/ Examples

Take home complex essay

0

10

20

30

40

50

60

70

Percent Students

(0-5) (6-10) (11-15) (16-20) (21-25)

Categories of Achievement

Without With Examples

Add Generation to Recognition• Posted another variation on the question • Each student was to write an answer• Also expected to comment on one answer• Collected a sample of answers• They were focus of class discussion

– Students comment on completeness– Offer suggestions to refine answers

Learning With Full Techno BoostReport:

http://www.unl.edu/peerrev/examples/bernstein/index.html

05

101520253035404550

Percent Students

<= 69 70-79 80-89 90-100

Levels of Achievement

Old questions New questions Most recent

Comparison with last three offerings (percent of students)

0

10

20

30

40

50

60

<=69 70-79 80-89 90-100

Fall 00Fall 01Spring 03Spring 04

Distribution of PerformanceMedian 80/90

0

1

2

3

4

5

6

7

8

<=69 70-79 80-89 90-100

spring 03spring 05

General Pattern of Improvement

• Increased frequency and care of student reading

• Dramatically improved class climate

• Increased recognition of examples of understanding

• Improved percentage of students reaching high levels of understanding

Note use of equivalent problems

• Same idea with different contextual cues

• Generate up to eight problems per concept

• Use several to teach, then test in new context

• Teach again as needed in new examples

• Retest in yet additional new context

• Final exam is yet another generalization test

Inductive and deductive teaching

• Active discovery is important learning

• Didactic instruction can be efficient

• Bransford -- “time for telling”

• My research => advantages for both

• Lendol Calder is a nice synthesis

Comments? Examples? Comments? Examples? Questions?Questions?