The Learning Research Quiz Show - Will at Work...

37
The Learning Research Quiz Show Bridging Gap between Research and Practice

Transcript of The Learning Research Quiz Show - Will at Work...

Page 1: The Learning Research Quiz Show - Will at Work Learningwillthalheimer.typepad.com/files/...learning-research-quiz-show...The Learning Research Quiz Show ... Slides available at: ...

The

Learning Research

Quiz Show

Bridging Gap between Research and Practice

Page 2: The Learning Research Quiz Show - Will at Work Learningwillthalheimer.typepad.com/files/...learning-research-quiz-show...The Learning Research Quiz Show ... Slides available at: ...

Slides available at:     www.is.gd/willstuff  

Research

Page 3: The Learning Research Quiz Show - Will at Work Learningwillthalheimer.typepad.com/files/...learning-research-quiz-show...The Learning Research Quiz Show ... Slides available at: ...

Simplicity is the Ultimate Sophistication

Leonardo da Vinci

Engagement &Understanding

Remembering

Application

The Decisive Dozenfor Learning Design and Learning Measurement 

http://is.gd/ddResearch

Baseline1. Content

2. Exposure

3. Guiding Attention

4. Creating Correct Conceptions

5. Repetition

6. Feedback 

7. Variation

8. Retrieval Practice

9. Context Alignment

10. Spacing

11. Persuasion

12. Perseverance

http://is.gd/DecisiveDozen

Page 4: The Learning Research Quiz Show - Will at Work Learningwillthalheimer.typepad.com/files/...learning-research-quiz-show...The Learning Research Quiz Show ... Slides available at: ...

http://Work-Learning.com/catalog

Page 5: The Learning Research Quiz Show - Will at Work Learningwillthalheimer.typepad.com/files/...learning-research-quiz-show...The Learning Research Quiz Show ... Slides available at: ...

Quite simply, the BEST book on smile sheet creation and utilization, Period!

Karl M. KappProfessor of Instructional Technology

Bloomsburg University

Thoughtful and sensible advice for feedback tools that will provide valid and actionable data.

Robert O. BrinkerhoffProfessor Emeritus, Western Michigan 

University & Director, Brinkerhoff Evaluation Institute

Evidence‐based practice at the master level.

Julie DirksenAuthor of Design For How People Learn

Question 1How much are smile-sheet results correlated with learning outcomes?

A. High marks on smile sheets indicate that the training was likely VERY SUCCESSFUL in creating learning.

B. High marks indicate that the training was likely to be AT LEAST SOMEWHAT SUCCESSFUL in creating learning.

C. High marks on smile sheets tell us VIRTUALLY NOTHING about the success of our training programs in creating learning.

Page 6: The Learning Research Quiz Show - Will at Work Learningwillthalheimer.typepad.com/files/...learning-research-quiz-show...The Learning Research Quiz Show ... Slides available at: ...

Level 1to

Level 2

r=.09

Alliger, Tannenbaum, Bennett, Traver, & Shotland (1997).

A meta-analysis of the relations among training criteria.

Personnel Psychology, 50, 341-357.

1990’s

Level 1to

Level 2

r=.09

Sitzmann, T., Brown, K. G., Casper, W. J., Ely, K., & Zimmerman, R. D.

(2008). A review and meta-analysis of the nomological network of

trainee reactions. Journal of Applied Psychology, 93, 280-295.

2000’s

www.SmileSheets.com

Question 2When designing learning, should we base our learning methods on the

preferences of our learners?

A. Learners are fairly good judges of their own learning, so we should trust their learning-design intuitions.

B. Learners are not always good judges of their own learning, so we should utilize other design imperatives.

C. Another answer would be obviously better.

Page 7: The Learning Research Quiz Show - Will at Work Learningwillthalheimer.typepad.com/files/...learning-research-quiz-show...The Learning Research Quiz Show ... Slides available at: ...

We’d like to trust our learners…

But the research shows that they don’t always know their own learning…

Learners are Overly Optimistic Zechmeister & Shaughnessy (1980).

Learners can’t always OvercomeFaulty Prior KnowledgeKendeou & van den Broek (2005).

Learners Fail to Properly Use ExamplesRenkl (1997).

Learners Fail to Give ThemselvesRetrieval PracticeKarpicke, Butler, & Roediger (2009).

Two Recent Reviews Emphasize Learners’ Lack of Knowledge of LearningBrown, Roediger & McDaniel (2014); Kirschner & van Merriënboer (2013).

Page 8: The Learning Research Quiz Show - Will at Work Learningwillthalheimer.typepad.com/files/...learning-research-quiz-show...The Learning Research Quiz Show ... Slides available at: ...

Question 3You’ve got a 45 minute elearning program just about

developed. Then, your client says they want a 60 minute program instead. What’s more potent,

EXTRA REVIEW or EXTRA QUESTIONS?

A. Extra Review will better support remembering.

B. Extra Questions (without feedback) will better support remembering.

C. Extra Questions (with feedback) will better support remembering.

0

10

20

30

40

50

60

70

Jones, H. E. (1923-1924). Experimental studies of college teaching: The effect of examination on permanence of learning. Archives of Psychology, 10, 1-70.

60 minutesClassroom

55 minutesLecture

---5 minutesAnsweringQuestions

Retrieval Practice – Research

Page 9: The Learning Research Quiz Show - Will at Work Learningwillthalheimer.typepad.com/files/...learning-research-quiz-show...The Learning Research Quiz Show ... Slides available at: ...

Retrieval Practice – Research

0

1

2

3

4

Nungester, R. J., & Duschastel, P. C. (1982). Testing versus review: Effects on retention. Journal of Educational Psychology, 74, 18-22.

Learn-Learn

Learn-Retrieve

Allen, G. A., Mahler, W. A., & Estes, W. K. (1969). Effects of recall tests on long-term retention of paired associates. Journal of Verbal Learning and Verbal Behavior, 8, 463-470.Anderson, R. C., & Biddle, W. B. (1975). On asking people questions about what they are reading. In G. H. Bower (Ed.) The psychology of learning and motivation: Advances in research and theory (Vol. 9). New York: Academic Press.Bahrick, H. P. (1979). Maintenance of knowledge: Questions about memory we forgot to ask. Journal of Experimental Psychology: General, 108, 296-308.Bahrick, H. P., & Hall, L. K. (1991). Preventive and corrective maintenance of access to knowledge. Applied Cognitive Psychology, 5, 1-18.Bahrick, H. P., & Phelps, E. (1987). Retention of Spanish vocabulary over 8 years. Journal of Experimental Psychology: Learning, Memory, and Cognition, 13, 344-349.Bahrick, H. P., & Phelps, E. (1988). The maintenance of marginal knowledge. In U. Neisser & E. Winograd (Eds.), Remembering reconsidered: Ecological and traditional approaches to the study of memory (pp. 178-192). Cambridge, UK: Cambridge University Press.Bahrick, H. P., Bahrick, L. E., Bahrick, A. S., & Bahrick, P. E. (1993). Maintenance of foreign language vocabulary and the spacing effect. Psychological Science, 4, 316-321.Bangert-Drowns, R. L., Kulik, J. A., & Kulik, C. C. (1991). Effects of frequent classroom testing. Journal of Educational Research, 85, 89-99.Berger, S. A., Hall, L. K., & Bahrick, H. P. (1999). Stabilizing access to marginal and submarginal knowledge. Journal of Experimental Psychology: Applied, 5, 438-447.Bjork, R. A. (1988). Retrieval practice and the maintenance of knowledge. In M. M. Gruneberg, P. E. Morris, R. N. Sykes (Eds.), Practical Aspects of Memory: Current Research and Issues, Vol. 1., Memory in Everyday Life (pp. 396-401). NY: Wiley.Carrier, M., & Pashler, H. (1992). The influence of retrieval on retention. Memory & Cognition, 20, 633-642.Craik, F. I. M., & Lockhart, R. S. (1972). Levels of processing: A framework for memory research. Journal of Verbal Learning and Verbal Behavior, 11, 671-684.Crooks, T. J. (1988). The impact of classroom evaluation practices on students. Review of Educational Research, 58, 438-481.Crowder, R. G. (1976). Principles of learning and memory. Hillsdale, NJ: Erlbaum.Cuddy, L. J., & Jacoby, L. L. (1982). When forgetting helps memory: An analysis of repetition effects. Journal of Verbal Learning and Verbal Behavior, 21, 451-467.Cull, W. L. (2000). Untangling the benefits of multiple study opportunities and repeated testing for cued recall. Applied Cognitive Psychology, 14, 215-235.Cull, W. L., Shaughnessy, J. J., & Zechmeister, E. B. (1996). Expanding understanding of the expanding-pattern-of-retrieval mnemonic toward confidence in applicability. Journal of Experimental Psychology: Applied, 2, 365-378.d’Ydewalle, G. D., Swerts, A., & De Corte, E. (1983). Study time and test performance as a function of test expectations. Contemporary Educational Psychology, 8, 55-67.Dempster, F. N. (1987). Effects of variable encoding and spaced presentations on vocabulary learning. Journal of Educational Psychology, 79, 162-170.Ebbinghaus, H. (1913). Memory: A contribution to experimental psychology, (Translated by H. A. Ruger and C. E. Bussenius). New York: Teachers College, Columbia University. (Also available 1964 and 1987, New York: Dover Publications. Original published in 1885).Foos, P. W., & Clark, M. C. (1983). Learning from text: Effects of input order and expected test. Human Learning, 2, 177-185.Foos, P. W., & Fisher, R. P. (1988). Using tests as learning opportunities. Journal of Educational Psychology, 80, 179-183.Gall, M. D., Ward, B. A., Berliner, D. C., Cahen, L. S., Winne, P. H., Elashoff, J. D., & Stanton, G. C. (1978). Effects of questioning techniques and recitation on student learning. American Educational Research Journal, 15, 175-199.Gates, A. I. (1917). Recitation as a factor in memorizing. Archives of Psychology, 26, 1-104.Glover, J. A. (1989). The “testing” phenomenon: Not gone but nearly forgotten. Journal of Educational Psychology, 81, 392-399.Hamaker, C. (1986). The effects of adjunct questions on prose learning. Review of Educational Research, 56, 212-242.Hogan, R. M., & Kintsch, W. (1971). Differential effects of study and test trials on long-term recognition and recall. Journal of Verbal Learning and Verbal Behavior, 10, 562-567.Izawa, C. (1992). Test trials contributions to optimization of learning processes: Study/test trials interactions. In A. F. Healy, S. M. Kosslyn, & R. M. Shiffrin (Eds.) From Learning Processes to Cognitive Processes: Essays in Honor of William K. Estes (Volume 2, pp. 1-33). Hillsdale, NJ: Erlbaum.Jacoby, L. L. (1978). On interpreting the effects of repetition: Solving a problem versus remembering a solution. Journal of Verbal Learning and Verbal Behavior, 17, 649-667.Jones, H. E. (1923-1924). Experimental studies of college teaching: The effect of examination on permanence of learning. Archives of Psychology, 10, 1-70.Kuo, T. M., & Hirshman, E. (1996). Investigations of the testing effect. American Journal of Psychology, 109, 451-464.Lachman, R., & Laughery, K. R. (1968). Is a test trial a training trial in free recall learning? Journal of Experimental Psychology, 76, 40-50.Landauer, T. K., & Ainslie, K. I. (1975). Exams and use as preservatives of course-acquired knowledge. Journal of Educational Research, 69, 99-104.Landauer, T. K., & Bjork, R. A. (1978). Optimum rehearsal patterns and name learning. In M. M. Gruneberg, P. E. Morris, & R. N. Sykes, (Eds.), Practical Aspects of Memory (pp. 625-632). London: Academic Press.Landauer, T. K., & Eldridge, L. (1967). Effect of tests without feedback and presentation-test interval in paired-associate learning. Journal of Experimental Psychology, 75, 290-298.Meyer, G. (1934). An experimental study of the old and new types of examination: I. The effect of the examination set on memory. Journal of Educational Psychology, 25, 641-661.Meyer, G. (1935). An experimental study of the old and new types of examination: II. Methods of study. Journal of Educational Psychology, 26, 30-40.Modigliani, V. (1976). Effects on a later recall by delaying initial recall. Journal of Experimental Psychology: Human Learning and Memory, 2, 609-622.Modigliani, V. (1978). Effects of initial testing on later retention as a function of the initial retention interval. In M. M. Gruneberg, P. E. Morris, & R. N. Sykes, (Eds.), Practical Aspects of Memory (pp. 652-659). London: Academic Press.Modigliani, V., & Hedges, D. G. (1987). Distributed rehearsals and the primacy effect in single-trial free recall. Journal of Experimental Psychology: Learning, Memory, and Cognition, 13, 426-436.Modigliani, V., Rea, C. P., & Hedges, D. G. (1988). Within-story recalls and the retention of ideas in prose processing. Canadian Journal of Behavioural Science, 20, 73-84.

Retrieval Practice – Research

0

10

20

30

40

50

60

Butler, A. C., & Roediger III, H. L. (2007). Testing improves long-termretention in a simulated classroom setting. European Journal of Cognitive Psychology, 19, 514-527.

NoTests

Test---No

Feedback

Test---

WithFeedback

Page 10: The Learning Research Quiz Show - Will at Work Learningwillthalheimer.typepad.com/files/...learning-research-quiz-show...The Learning Research Quiz Show ... Slides available at: ...

Question 4When should we give learners feedback on quiz questions?

A. Immediately after each question.

B. Immediately after the end of the quiz.

C. After a delay of an hour or more.

D. After a delay of several days or more.

Question 5If an organization switches from

classroom training to elearning, can they expect cost savings in general?

A. Yes. eLearning is generally LESS COSTLY to develop and deploy.

B. No. eLearning is generally MORE COSTLY to develop and deploy.

C. No. eLearning generally COSTS ABOUT THE SAME as classroom training.

Page 11: The Learning Research Quiz Show - Will at Work Learningwillthalheimer.typepad.com/files/...learning-research-quiz-show...The Learning Research Quiz Show ... Slides available at: ...

While the common wisdom argues that e‐learning is less costly:

“Survey data suggest that training costs across organizations remain relatively constant as training shifts from face‐to‐face to technology‐based methods.” (p. 87)

http://is.gd/TrainingResearch2012

Question 6To brainstorm new ideas, what’s the

best way to support creativity?

Having groups of people brainstorm TOGETHER

Having groups of people brainstorm 

AS INDIVIDUALS

Mullen, B., Johnson, C., & Salas, E. (1991). Productivity loss in brainstorming groups: A metaanalytic integration. 

Diehl, M., & Stroebe, W. (1987). Productivity loss in brainstorming groups: Toward the solution of a riddle. 

Page 12: The Learning Research Quiz Show - Will at Work Learningwillthalheimer.typepad.com/files/...learning-research-quiz-show...The Learning Research Quiz Show ... Slides available at: ...

Question 7Is it a best practice that we present learners with learning objectives?

A. YES, by presenting learners with well-designed learning objectives, we give them an ADVANCE ORGANIZER.

B. YES, by presenting learners with well-designed learning objectives, we GUIDE LEARNER ATTENTION to the most critical information.

C. NO, learning objectives are JUST ONE TOOL in the instructional-design toolbox that can accomplish the same exact thing.

Question 8Compared to NOT giving learning objectives to learners, what is the

likely result of presenting objectives targeting half of the content?

A. The content targeted by the learning objectives will be better remembered.

B. The content targeted will be better remembered, but the content not targeted will be less remembered.

C. Both the targeted and non-targeted content will be better remembered.

Page 13: The Learning Research Quiz Show - Will at Work Learningwillthalheimer.typepad.com/files/...learning-research-quiz-show...The Learning Research Quiz Show ... Slides available at: ...

Rothkopf, E. Z., & Billington, M. J. (1979). Goal-guided learning from text: Inferring adescriptive processing model from inspection times and eye movements. Journal of Educational Psychology, 71, 310-327.

0

10

20

30

40

50

60

70

80

No Objectives

NotTargeted

Targeted

0

10

20

30

40

50

60

70

80

Objectives

NotTargeted

Targeted

Question 9Is it okay to present learners with learning objectives that utilize the

word “UNDERSTAND?”

A. No, “UNDERSTAND” is not measurable, so it shouldn’t be used.

B. No, “UNDERSTAND” is too general, so it shouldn’t be used.

C. Yes, “UNDERSTAND” is fine to use as long as other words in the objective are clear and specific.

Page 14: The Learning Research Quiz Show - Will at Work Learningwillthalheimer.typepad.com/files/...learning-research-quiz-show...The Learning Research Quiz Show ... Slides available at: ...

Rothkopf, E. Z., & Billington, M. J. (1979). Goal-guided learning from text: Inferring adescriptive processing model from inspection times and eye movements. Journal of Educational Psychology, 71, 310-327.

0

10

20

30

40

50

60

70

80

No Objectives

NotTargeted

Targeted

0

10

20

30

40

50

60

70

80

Objectives

NotTargeted

Targeted

You will UNDERSTAND that WHEAT was once HARVESTED by HITTING it with a FLAIL.

“An objective that generally identifies the information to be learned … will produce robust effects. 

Including other information (per Mager’s, 1962, definition) will not significantly help and it may hinder the effects of the objectives.”

Mager’s Criteriafor Instructional Objectives:

The performancethe learners 

should be able to do. 

1

The conditionsunder which they should be able to achieve that performance. 

2

The criteriafor acceptable performance.

3

Hamilton, 1985, p. 78

Hamilton, R. J. (1985). A framework for the evaluation of the effectiveness of adjunct questions and objectives. Review of Educational Research, 55, 47‐85.

Page 15: The Learning Research Quiz Show - Will at Work Learningwillthalheimer.typepad.com/files/...learning-research-quiz-show...The Learning Research Quiz Show ... Slides available at: ...

Who are our objectives for?Learning 

Professionals

Learners

Video on Learning Objectives:www.is.gd/LOBJ_VIDEO

Question 10In designing learning, we should take learner characteristics into account?

A. YES, We should provide different learning interactions based on learners’ different LEARNING STYLES.

B. YES, We should provide MULTIPLE TYPES of learning interactions to support all learners.

C. YES, We should provide different learning interactions based on learners differing prior knowledge.

D. We should do ALL OF THE ABOVE.

Page 16: The Learning Research Quiz Show - Will at Work Learningwillthalheimer.typepad.com/files/...learning-research-quiz-show...The Learning Research Quiz Show ... Slides available at: ...

Science has proven, the learning-styles approach is not effective!

Science has proven, the learning-styles approach is not effective!

Pashler, H., McDaniel, M., Rohrer, D., & Bjork, R. (2008). Learning styles: Concepts and evidence. Psychological Science in the Public Interest, 9(3), 105‐119.

“We conclude therefore, that at present, there is no adequate evidence base to justify incorporating learning‐styles assessments into general educational practice. Thus, limited education resources would better be devoted to adopting other educational practices that have a strong evidence base, of which there are an increasing number.” 

Page 17: The Learning Research Quiz Show - Will at Work Learningwillthalheimer.typepad.com/files/...learning-research-quiz-show...The Learning Research Quiz Show ... Slides available at: ...

Science has proven, the learning-styles approach is not effective!

Are learning‐styles predictions validated by research?

"No. Several reviews that span decades have evaluated the literature on learning styles (e.g., Arter & Jenkins, 1979; Kampwirth& Bates, 1980; Kavale & Forness, 1987; Kavale, Hirshoren, & Forness, 1998; Pashler et al., 2009; Snider, 1992; Stahl, 1999; Tarver & Dawson, 1978), and each has drawn the conclusion that there is no viable evidence to support the theory.”

Willingham, D. T., Hughes, E. M., & Dobolyi, D. G. (2015). The scientific status of learning styles theories. Teaching of Psychology, 42(3), 266‐271.

Question 11Which of the following

statements is true?

A. People remember 10% of what they READ.

B. People remember 20% of what they HEAR.

C. People remember 30% of what they SEE.

D. People remember 50% of what they SEE and HEAR.

E. All of the above.

F. None of the above.

Page 18: The Learning Research Quiz Show - Will at Work Learningwillthalheimer.typepad.com/files/...learning-research-quiz-show...The Learning Research Quiz Show ... Slides available at: ...

1020

30

50

7080

Reading Seeing Hearing Seeing &Hearing

Collaboration Doing

Chi, M. T. H., Bassok, M., Lewis, M. W., Reimann, P., & Glaser, R. (1989). Self-explanations: How students study and use examples in learning to

solve problems. Cognitive Science, 13 , 145-182.

RetentionRates

“People remember 10% of what they read…”

This is bogus data !!

Page 19: The Learning Research Quiz Show - Will at Work Learningwillthalheimer.typepad.com/files/...learning-research-quiz-show...The Learning Research Quiz Show ... Slides available at: ...
Page 20: The Learning Research Quiz Show - Will at Work Learningwillthalheimer.typepad.com/files/...learning-research-quiz-show...The Learning Research Quiz Show ... Slides available at: ...
Page 21: The Learning Research Quiz Show - Will at Work Learningwillthalheimer.typepad.com/files/...learning-research-quiz-show...The Learning Research Quiz Show ... Slides available at: ...

Will’s Note:To save paper, I’m omitting the other dozen examples of this

learning myth.

www.Debunker.Club

www.is.gd/BadCone

Page 22: The Learning Research Quiz Show - Will at Work Learningwillthalheimer.typepad.com/files/...learning-research-quiz-show...The Learning Research Quiz Show ... Slides available at: ...

Question 4 -- RevisitedWhen should we give learners feedback on quiz questions?

A. Immediately after each question.

B. Immediately after the end of the quiz.

C. After a delay of an hour or more.

D. After a delay of several days or more.

0

10

20

30

40

50

60

Butler, A. C., & Roediger III, H. L. (2008). Feedback Enhances the Positive Effects and Reduces the Negative Effects of Multiple-Choice Testing. Memory & Cognition. 36 (3), 604-616.

NoQuiz

Quiz,But No

Feedback

Immed.Feedback

After-Test

Feedback

Feedback Research

Page 23: The Learning Research Quiz Show - Will at Work Learningwillthalheimer.typepad.com/files/...learning-research-quiz-show...The Learning Research Quiz Show ... Slides available at: ...

0

10

20

30

40

50

60

Brosvic, Epstein, Cook, Dihoff (2005). Efficacy of error for the correction of initiallyincorrect assumptions and of feedback for the affirmation of correct responding: Learning in the classroom. Psychological Record, 55, 401-418.

No Feedback

FeedbackImmed.

FeedbackAfter1 Day

Feedback Research

10

11

12

13

14

15

16

Results (retrieval after 3 days)

More, Arthur, J. (1969). Delay of feedback and the acquisition and retention of verbalmaterials in the classroom. Journal of Educational Psychology, 60, 339-342.

FeedbackImmed.

FeedbackAfter2.5

Hours

FeedbackAfter

1Day

FeedbackAfter

4Days

Feedback Research

Page 24: The Learning Research Quiz Show - Will at Work Learningwillthalheimer.typepad.com/files/...learning-research-quiz-show...The Learning Research Quiz Show ... Slides available at: ...

• Immediate Feedback Creates Understanding

•More Feedback Needed

•More Elaborate Feedback

•More Emotional Support

•More Important to Give Feedback on Correct Answers

Early in Learningor

Complex Material

•Delayed Feedback Supports Remembering

• Less Feedback Needed

• Less Elaborate Feedback

•No/Less Emotional Support

• Less Important to Give Feedback on Correct Answers

Later in Learningor

Simple Material

www.work‐learning.com/catalog.html

Page 25: The Learning Research Quiz Show - Will at Work Learningwillthalheimer.typepad.com/files/...learning-research-quiz-show...The Learning Research Quiz Show ... Slides available at: ...

Question 12Which of the following

statements is true?

A. People forget about 90% of what they learn one week after learning it.

B. People forget at a rate first discovered by Hermann Ebbinghaus.

C. People forget about half of what they learn within 24 hours.

D. All of the above.

E. None of the above.

The Truth:

Forgetting depends on many things!

www.work‐learning.com/catalog.html

Page 26: The Learning Research Quiz Show - Will at Work Learningwillthalheimer.typepad.com/files/...learning-research-quiz-show...The Learning Research Quiz Show ... Slides available at: ...

I’ve learned that people will forget

what you said, people will forget what you did, but people will never forget how you made them feel.

Maya Angelou

http://quoteinvestigator.com/2014/04/06/they‐feel/

They may forget what you said — but they will

never forget how you made them feel.

Carl W. Buehner

On Forgetting

You know what's right, just do right. You don't really have to ask anybody. The truth is—right may not be expedient, it may not be profitable, but it will satisfy your soul...

Maya Angelou

https://www.youtube.com/watch?v=bxrV2J_OjGo

Page 27: The Learning Research Quiz Show - Will at Work Learningwillthalheimer.typepad.com/files/...learning-research-quiz-show...The Learning Research Quiz Show ... Slides available at: ...

Rhetorical QuestionWhat did Robert Frost mean when

he wrote, “Good fences make good neighbors”?

A. Good fences make good neighbors.

B. Good fences make bad neighbors.

C. Good neighbors make good fences.

D. Some other answer?

E. He meant for us to think more deeply before jumping to conclusions.

Question 13Today, which type of research offers the most

critical information for learning design?

A. Research surveys of learner preferences.

B. Research examining learning methods and their outcomes.

C. Research on brain-based learning and neuroscience.

D. Training needs analysis, especially cognitive task analysis.

Page 28: The Learning Research Quiz Show - Will at Work Learningwillthalheimer.typepad.com/files/...learning-research-quiz-show...The Learning Research Quiz Show ... Slides available at: ...

Glial Grandeur

From Researchers at the University of Edinburgh

http://is.gd/grandeur

Someday, neuroscience may have a ton to teach us about learning.

Page 29: The Learning Research Quiz Show - Will at Work Learningwillthalheimer.typepad.com/files/...learning-research-quiz-show...The Learning Research Quiz Show ... Slides available at: ...

"I don't think brain science has anything to say for business practice."

"We still don't really know how the brain works."

"The state of our knowledge [of the brain] is childlike."

"The human brain was not built to learn. It was built to survive.“

John MedinaNeuroscientist

June 2015

November 2012:

“Neuroscience Applied to Education: Mostly Unimpressive”

January 2014:

“I’ve often written that it’s hard to bring neuroscientific data to bear on issues in education… Hard, but not impossible.

Daniel WillinghamResearch Psychologist

Page 30: The Learning Research Quiz Show - Will at Work Learningwillthalheimer.typepad.com/files/...learning-research-quiz-show...The Learning Research Quiz Show ... Slides available at: ...

Daniel Busso & Courtney Pollock:

“There is little doubt that our knowledge of the developing brain is poised to make important contributions to the lives of parents, educators and policymakers…

Some have voiced concerns about the viability of educational neuroscience, suggesting that neuroscience can inform education only indirectly…

“Others insist that neuroscience is only one small component of a multi-pronged research strategy to address educational challenges, rather than a panacea…”

Busso, D. S., & Pollack, C. (2015). No brain left behind: Consequences of neuroscience discourse for education. Learning, Media and Technology, 40(2), 168‐186.

“As the first review of its type [looking at the intersection of neuroscience and elearning], it is perhaps unsurprising that we find few, if any, examples of replication…

“Caution is, therefore, required in relating such findings to real-world contexts involving other types of participant[s]…

“The relevance of cognitive neuroscience to education is increasingly undisputed, and it may be within the field of [elearning] that its early impact will be greatest.”

Howard‐Jones, P., Ott, M., van Leeuwen, T., & De Smedt, B. (2015). The potential relevance of cognitive neuroscience for the development and use of technology‐enhanced learning. Learning, Media and Technology, 40(2), 131‐151. 

Page 31: The Learning Research Quiz Show - Will at Work Learningwillthalheimer.typepad.com/files/...learning-research-quiz-show...The Learning Research Quiz Show ... Slides available at: ...

Question 14Are we as learning professionals more susceptible

to recommendations if they mention “neuroscience” or “brain science?”

A. Yes, there is scientific evidence that people are more likely to believe arguments if they use scientific terminology.

B. Yes, there is scientific evidence that people are more likely to believe arguments if they use the term “neuroscience.”

C. No, this is an urban legend.

“Brain Science” “Neuroscience”“Several highly cited studies have shown that superfluous neuroscience information may bias the judgement of non-experts….”

“However, the idea that neuroscience is uniquely persuasive has been met with little empirical support….

Nevertheless, excitement about neuroscience is concerning because it can be exploited by unscrupulous consumer–product manufacturers.” (p. 4)

Busso, D. S., & Pollack, C. (2015). No brain left behind: Consequences of neuroscience discourse for education. Learning, Media and Technology, 40(2), 168‐186.

“Powerful, often self-interested, commercial forces serve as mediators between research and practice, and this raises some pressing questions for future work in the field: what does responsible [research-to practice] translation look like?” (p. 12).

Page 32: The Learning Research Quiz Show - Will at Work Learningwillthalheimer.typepad.com/files/...learning-research-quiz-show...The Learning Research Quiz Show ... Slides available at: ...

Question 15Repetition has powerful benefits for learning, if it

isn’t rote, boring, or irrelevant. What’s the best way to support remembering with repetitions?

A. Repeat concepts fairly close in time so that learners can easily process the material.

B. Repeat concepts at fairly wide intervals to strengthen long-term memory.

C. Repeat concepts at a slight delay, getting the benefits of both short and long delays.

Page 33: The Learning Research Quiz Show - Will at Work Learningwillthalheimer.typepad.com/files/...learning-research-quiz-show...The Learning Research Quiz Show ... Slides available at: ...

Krug, D., Davis, T. B., & Glover, J. A. (1990). Massed versus distributed repeated reading:A case of forgetting helping recall? Journal of Educational Psychology, 82, 366-371.

Research Example

0

10

20

30

40

ImmediateRepetition

SpacedRepetition

Research Example

Karpicke, J.D. & Roediger, H.L. (2007). Expanding retrieval practice promotes short-term retention, but equally spaced retrieval enhances long-term retention. Journal of Experimental Psychology: Learning, Memory and Cognition, 33, 704-719.

0

10

20

30

40

50

No Spacing

Expanding Spacing

Equal Spacing

Page 34: The Learning Research Quiz Show - Will at Work Learningwillthalheimer.typepad.com/files/...learning-research-quiz-show...The Learning Research Quiz Show ... Slides available at: ...

Research Example

Dellarosa, D., & Bourne, L. E. (1985). Surface form and the spacing effect. Memory & Cognition, 13, 529-537. From Experiment 1.

10

20

30

40

50

60

SinglePresentation

Non-SpacedRepetition

SpacedRepetition

Widely-Spaced

Repetition

Question 16At the end of your elearning program, what’s the

best way to get learners to actually put what they’ve learned into practice in their work?

A. Have the learners make a written commitment to implement a few key goals.

B. Have the learners make a commitment to a few key goals, and have them commit to a time and place where they’ll work on the goals.

C. Have the learners make a commitment to a few key goals and inform their manager of those goals.

Page 35: The Learning Research Quiz Show - Will at Work Learningwillthalheimer.typepad.com/files/...learning-research-quiz-show...The Learning Research Quiz Show ... Slides available at: ...

Setting a Goal = “I intend to Lose Weight”

Setting a Trigger =

“I intend to Walk for 30 Minutes on Monday, Wednesday, and Friday as soon as I wake up.”

IF Situation, THEN Action

Goal Setting vs. Trigger Setting

Review Article: Gollwitzer, P. M., & Sheeran, P. (2006). Implementation intentions and goal achievement: A meta‐analysis of effects and processes. Advances in Experimental Social Psychology, 38, 69‐119.

Reviewed 94 separate experiments and found a medium‐to‐high magnitude (d = .65) for the benefits of implementation intentions. 92 of 94 experiments showed positive results!!

Attending a workshop, self‐examination, buying organic, recycling, exercise, diet, solving law cases, taking vitamins.

Page 36: The Learning Research Quiz Show - Will at Work Learningwillthalheimer.typepad.com/files/...learning-research-quiz-show...The Learning Research Quiz Show ... Slides available at: ...

Last QuestionWhat’s your current practice in terms of the

scientific research on learning?

A. I am a ravenous reader of research-related sources.

B. I try to keep abreast of the major research findings.

C. I don’t always pay close attention.

D. I learned what I need to know in my graduate program, in other schooling, or in my work.

E. I don’t believe research is very valuable.

Maya Angelou

https://www.youtube.com/watch?v=bxrV2J_OjGo

Try to live your life in a way that you will not regret years of useless virtue and inertia and timidity. Take up the battle. Take it up. It's yours, this is your life. This is your world...

Make it a better world…

Page 37: The Learning Research Quiz Show - Will at Work Learningwillthalheimer.typepad.com/files/...learning-research-quiz-show...The Learning Research Quiz Show ... Slides available at: ...

Phone: 888‐579‐9814Email:  info@work‐learning.comWebsite: Work‐Learning.comAudits: LearningAudit.comSmiles: SmileSheets.comBlog: WillAtWorkLearning.comTwitter: @WillWorkLearn

Will Thalheimer, PhDPresidentWork‐Learning Research, Inc.Somerville, Massachusetts, US

The Learning Research Quiz Show

DevLearn – September/October 2015