Making Learning Visible Through the Scholarship of Teaching and Learning University of Alaska,...

Making Learning VisibleThrough the Scholarship of Teaching

and Learning

University of Alaska, Anchorage

March 8-9, 2007

Randy Bass and Heidi Elmendorf

Georgetown University

Comments on the Readings for Thursday November 13th Current Forum: Read23 timesWed Nov 12 2003 8:17 pm Date:Bastos, John Michael < > Author: [email protected]: Water... Subject:Not totally sure this is right, but I'll take a chance... My guess is that themicrobes "digest" the contaminants andinternally break them down into another substance. Although digest might notbe the best word, because it's not likethey eat them (they dont have mouths). But I suppose they look for a chemicalor protein and bind or absorb it, andthen break it down internally with other enzymes. The enzymes take one thingand then break it down into morespecific parts; then use some of it for food or energy to make more cells\parts -and then it probably just excreteswhatever else it doesn't need.That's my take at least.

Ventures answer whenuncertain

Good detail

Comments on the Readings for Thursday November 13th Current Forum: Read24 timesWed Nov 12 2003 8:31 pm Date:Whitehurst, Celadon Charles < > Author: [email protected]: Water... Subject:yeah, it seems like the microbes strip the pollutants of important chemicals thatchange its chemical makeup and makeit into a completely different compound all together.

confirmation

Comments on the Readings for Thursday November 13th Current Forum: Read27 timesWed Nov 12 2003 8:34 pm Date:Whitehurst, Celadon Charles < > Author: [email protected] things... Subject:It was really refreshing to hear how a microbe mutated in a way that benefittedus. The PCB resistant microbes thatactually eat the harmful pollutants are a welcome break from all of the antibioticresistant, disease-causing microbes wehave studied so far!One question i had was whether the Arabian Gulf was so rich in oil-eatingmicrobes b/c of the large amounts of oilconstantly being dumped... Did the pollution actually cause the proliferation ofthese microbes b/c they had toconstantly deal with the oil and then they were ready when Iraqi forces dumpedlarger amounts?I think the idea of producing and using the enzymes that break down thepollutants rather than using the microbes reallydoes seem much safer, but I am slightly confused as to the logistics of how theywould actually implement the plan.

Appreciation of therange of science!

Great attempt to answerown question

Comments on the Readings for Thursday November 13th Current Forum: Read22 timesWed Nov 12 2003 8:47 pm Date:Condon, Kelly C. < > Author: [email protected] cleaners Subject:The information I read in today's selections sounds really hopeful, but thearticles also bring up a few questions andpersonal reservations. It is very fortunate that the PCBs are helping to clean theHudson, but that should not take our

Link out to regulationand to peer comment

1. What do you see?

2. What do you know?

3. What don’t you know? What questions do you have?

4. If you were going to find out more, how would you start? In the library? On the Web?

5. What are you bringing from other learning experiences that helps you make sense of this artifact?

Baseline Reflection Exercise:

What do we know about the difference between novice and

expert learners?

Historical Thinking and Other Unnatural Acts by Sam Wineburg

Metacognitive CognitiveAffective

Expertise and Adaptive Expertise

• Use of Knowledge• self-defined goals • exploration• re-examination

• Awareness of knowledge• epistemological• multiple perspectives• assumptive• limited

• Qualities of Knowledge• organized• contextualized• retrievable• flexible

A Traditional View…

NOVICE MIRACLE EXPERT

product product

product

NOVICEprocesses

LEARNING EXPERTprocesses

product

With a Focus on Learning Processes

productproduct

Even with attention to active learning and developmental processes, there is a tendency to see processes as inevitably leading to a version of expert product. And students still seem to believe in the miracle…

But what does expertise look like when seen through the lens of traditional expert

product?

Expert product is missing the evidence of context and process

•Formal and tidy •Loss of complexity and uncertainty

•Focus on cognitive•Emphasis on knowledge

product

NOVICEprocesses

LEARNING EXPERTprocesses

productproduct ≠

Processes that are effective in developing expert thinking do not

necessarily lead to the ability to produce expert product.

Is our problem then a conflation of the products of expertise with the products

of a developmental process?

Instead we need to expect learning products appropriate to learning

processes

NOVICEprocesses

LEARNINGprocesses

EXPERTprocesses

productsproductsproducts

LEARNINGprocesses

LEARNINGprocesses

How can we better understand these intermediate processes?

How might we design to foster and capture them?

Using the Scholarship of Teaching and Learning to make the most of this expansion of learning

“Reflecting on one’s practice is what practice is all about.”

Lee Shulman, President, Carnegie Foundation for the Advancement of Learning

Three Cases

• Authentic questions and the criteria of excellence (Bernstein, Psychology)

• Capturing the making of conversational biologists (Elmendorf, Biology)

• Close Reading (Bass, English)

Case #1: Dan Bernstein, University of Kansas

Traditional Abstract Questions

05

10152025303540

Percent Students

<= 69 70-79 80-89 90-100

Levels of Achievement

Old questions

Dan Bernstein, Psychology, University of Kansas

Changing to ‘authentic’ questions

Old: What were the reinforcing consequences in the Welsh, Bernstein & Luthans [restaurant] study? How were the consequences identified?

New: Suppose you were asked to implement a motivational program in a business setting. It is a small production unit with 25 employees who can engage in the following activities: production planning, inventory delivery, direct production, packaging, and marketing. Based on your understanding of the study of the fast food restaurant, how would you proceed to improve the quality of the employees' work by using access to activities as a motivator. Your answer should include the assessment of the relative values of activities, the establishment of contingencies, and a simple design for evaluating the project. Finally, describe the costs and benefits of the program for the business and make a recommendation about whether or not it should be implemented.


Comparison of achievement after exam with new ‘authentic’ questions


The story of the seven weldsfrom Grant Wiggins

Blackboard Discussion Board

• Learners should recognize quality work– Posted generalized example problems

• Posted another variation on the question

• Each student was to write an answer

• Also expected to comment on one answer


Learning With Full Range of Techniques

05

101520253035404550

Percent Students

<= 69 70-79 80-89 90-100

Levels of Achievement

Old questions New questions Most recent


Documented and made public through a course portfolio

Alignment:

•Goals

•Activities & Assessments

•Student Work


Conversational Biologists

Blackboard Discussion

Forum

In the Lab

Research Projects

In the Classroom

Case #2: Heidi Elmendorf, Georgetown University

Students’ Self-Perception:‘outside of their comfort zone’

70% self-identify as having had a poor or deficient preparation in science or as being lacking in science

ability.

“My basic thought was, yes, there are scientists and what they study is highly complicated stuff, ‘I can’t understand all those facts’.”

“I have generally felt inadequate in previous science courses, as humanities subjects have always been more ‘my thing’.”

“I have taken several science courses, and somehow managed to pass them through a combination of memorization and lots of studying, but am the first to admit that I have difficulty thoroughly understanding concepts.”

There is also something about science that is scary. Maybe this is my own personal problem, based on my experience with the subject, and my previous perceptions about science – but science is an intimidating field to me.”

Heidi Elmendorf, Biology, Georgetown University

Dichotomous Views of Science

•Inquiry•Uncertainty

•Problem-solving•Creativity•Exciting

•Promising

•Facts•Certainty

•Memorization•Rules

•Intimidating•Dangerous

Scientists: Public:


Mandate for Pedagogical Innovation

• Is there a role for non-scientists in science?– “It is vital that the public has a better working knowledge of the science and

technology that defines our very existence on the planet.” Rita Colwell

• Then what should they learn?– “We know full well that you will forget most of the facts that you have learned.

But what we hope you will retain is the capacity to integrate ideas and exercise thoughtful judgments across many aspects of human endeavor.” Shirley Tilghman

• How well do K-12 approaches address this need?– “Fully half the students who had not taken a course in biology did as well or

better than 40% of the students who had taken such a course."

Mullis and Jenkins, 1988


Layers of Course Structure:conversation and evidence

Reading &

Blackboard

Group Projects

Weekly Quizzes

Lab / Teaching

Class


“Structuring” the Conversation• The assignment

• Join in the conversation once a week• Roles? ‘summarizer’ ‘responder’ ‘confused one’ ‘criticizer’ ‘connector’

• I am invisible on-line• Follow-through

• Starting point for class:• Reading print outs of on-line discussion• On-line discussion integrated into class

• Worked into research papers

• Credit for participation; ungraded


How to Look at the Data?The Method… “Coding”

• Close reading and re-reading of Blackboard conversations

• Being selective & the mountain of evidence • Categorize comments… • Use color and notes to ‘code’• Tabulate


Emergent PatternsComments on the Readings for Thursday November 13th Current Forum: Read23 timesWed Nov 12 2003 8:17 pm Date:Bastos, John Michael < > Author: [email protected]: Water... Subject:Not totally sure this is right, but I'll take a chance... My guess is that themicrobes "digest" the contaminants andinternally break them down into another substance. Although digest might notbe the best word, because it's not likethey eat them (they dont have mouths). But I suppose they look for a chemicalor protein and bind or absorb it, andthen break it down internally with other enzymes. The enzymes take one thingand then break it down into morespecific parts; then use some of it for food or energy to make more cells\parts -and then it probably just excreteswhatever else it doesn't need.That's my take at least.


Good detail


confirmation






• expressing interest/enthusiasm• asking questions• answering questions • referring to text• noting science content• referring to scientific process• bringing in outside information

(linking out) • referring to each other (building

community)• taking intellectual risks

Heidi Elmendorf, Biology, Georgetown

Patterns of on-line conversations

– Fostering community

– Venturing ideas

– Referencing science as a process

Discussion Board - Early in Term

showing interest30%

asking a question20%

focus on the scientific process

6%

science information16%

linking out 7%

building community 9%

venturing an idea/answer12%

Discussion Board - Late in Term

showing interest13%

asking a question19%

focus on the scientific process

11%

science information9%

linking out 10%

building community 13%

venturing an idea/answer25%

–Giving summaries

–Stating science Facts

–Expressing naive interest

What Attributes of Learning Can On-line Conversations Foster?

• Support initial encounters with information• Encourage slower and more responsive

conversations• Permits students to revisit and reflect on

conversations• Emphasize process over product• Build ‘safe’ intellectual communities

Questions emerge from observed tensions

Evidence makes the claim more complicated

Talks through why the theme is important

Case #3: Randy Bass, Georgetown University

-Developed Pedagogies that allowed me to gather evidence of their reading processes.

-Made conversation and thinking aloud an integral part of the course.

This led to an inquiry

Randy Bass, English, Georgetown

Using analog video and an online discussion board to transform a “close reading” exercise

Group think-aloud captured on videotape

“TranscriptionOutline”

Online paper:Close readingOf own tape

Step #1

Step #2

Step #3


Breakdown of a Learning Activity

• What are the component activities that someone has to do well to be successful at this?

• _____• _____• _____• _____• _____• _____

• Where does this breakdown for students? What are the component obstacles?

• _____• _____• _____• _____• _____• _____


Learning Activity: Reading a literary text to generate

interesting questions about context and complexity.

• Level One: Reading

• Level Two: Recognizing complexity

• Level Three: Generating researchable questions


• Level One: Reading– Novice close reading strategies

• ______• ______

– Questioning strategies: • ______• ______

• Level Two: Recognizing complexity– Close rereading strategies:

• ______• ______

– Deferral of meaning: • ______• ______

• Level Three: Generating researchable questions about context and meaning– Strategies for Inquiry

• ______• ______

Obstacles

What does it look like when each of these breaks down?


Learning Activity: Reading a literary text to generate

interesting questions about context and complexity.

• Level One: Reading– Novice close reading strategies (“schoolish”):– Questioning strategies:

• Level Two: Recognizing complexity

– Close rereading strategies: – Deferral of meaning:

• Level Three: Generating researchable questions about context and meaning

– Strategies for Inquiry

Intermediate reading/analysis strategies for opening up possibilities and deferring meaning


“Oral midterms”

• 45 minute oral midterm

• Compress video and put on cd-rom

• Comments linked to time codes, asking them to rewatch and reflect.


What kinds of findings has this work yielded?

• New pedagogical strategies and course designs– e.g. “Think aloud” exercise– New ways to make use of online

discussions

• New framework: – “Learning Activity Breakdown”– Basis for collaborative inquiry?

• A theory or concept about a particular way of reading. – “Protocol of Deferral”: Intermediate reading

strategies


What are the three cases a case of?

All three motivated by discrepancy between course design and long-term learning values

What works?

Focuses on new activities to improve performance of summative assessments

Tracking grades over time

What’s possible?

Focuses on new ungraded course element related to process

Coding to track intellectual development in conversation

What is?

Created new course element in order to make process visible

Close reading of think alouds and related student writing

Comments on the Readings for Thursday November 13th Current Forum: Read23 timesWed Nov 12 2003 8:17 pm Date:Bastos, John Michael < > Author: [email protected]: Water... Subject:Not totally sure this is right, but I'll take a chance... My guess is that themicrobes "digest" the contaminants andinternally break them down into another substance. Although digest might notbe the best word, because it's not likethey eat them (they dont have mouths). But I suppose they look for a chemicalor protein and bind or absorb it, andthen break it down internally with other enzymes. The enzymes take one thingand then break it down into morespecific parts; then use some of it for food or energy to make more cells\parts -and then it probably just excreteswhatever else it doesn't need.That's my take at least.


Good detail


confirmation






Exploring a range of learning products appropriate to learning

processes

NOVICEprocesses

LEARNINGprocesses

EXPERTprocesses


LEARNINGprocesses

LEARNINGprocesses

“Thin slices” of online discussion or blog

Classroom assessment techniques

Interviews, think alouds, performance

Reflections, justifications

The Teaching Commons

“The teaching commons is a conceptual space in which communities of educators committed to inquiry and innovation come together to exchange ideas about teaching and learning and use them to meet the challenges of educating students for personal, professional, and civic life.”

Pat Hutchings and Mary Taylor Huber,

The Advancement of Learning: Building the Teaching Commons

Exploring Social Pedagogies:Using Collaborative Inquiry in SoTL

• a set of strategies for creating educational environments in which learning occurs in the context of a community

• added emphasis on activities that ask learners to represent knowledge for others

Exploring Social Pedagogies:Using Collaborative Inquiry in SoTL

•Through the close examination of the evidence of student learning, this project :

– will create a framework for designing better education experiences

– where students develop flexible and integrative thinking– in communication-intensive contexts

•Our purpose is to:– capture the essential assignment structures of social

pedagogies– create a clear, commonsense schema – demonstrate parallels between pedagogical practices and

assessment issues

Faculty conduct sustained scholarship of teaching projects on technology and learning in the humanities.

70 Faculty

21 Campuses

Five years

A Digital Story

Digital Story from Cultural History

QuickTime™ and aAnimation decompressor

are needed to see this picture.





another response (faculty collaborator)

“I was watching the digital story, which I’d seen before, and I wrote in my notes ‘digital book report’. I mean, there isn’t anything there that you wouldn’t have learned from a couple of hours in the Civil Rights Museum in Birmingham.”

Where is the full

evidence of Charea’s learning?

From the Charea Batiste

interview

[19:41] Because I was never involved in the civil rights movement, as I said that was a long time ago for me. I feel that I don’t have… can’t in my own words describe what happened. I was never there, I didn’t experience any of those things, so my words are just from an outside point of view.

But the pictures are first hand. These are people who actually went through the pain, who went through the torture, and their stories are told through these still images.

My voice was used I guess to give life to those pictures, but the pictures itself [sic] they told the story.

And my voice, I remember listening, I would get very angry telling the story. And I think that’s what added to the images. Because the anger in my voice--although still in tune with the digital story--without being irate was enough to make the images real, relevant, so you could feel the anger that was, you know, produced from those acts of violence. [20:52]

“On the relationship of the images to her voice”

The Development of Expertise?

Knowledge Immersion

Problem recognition

Confidence

Personal connection

Sense of audience and purpose

Sense of play and control

Mix of cognition, affect, and metacognition

Defining ‘Learning by Teaching’

“Enhancing disciplinary understanding by acting as teachers of that discipline.”

“Shifting the role of the students to help them gain the perspective of teachers.”

Equating Pedagogical Content Knowledge and Adaptive Expertise

“blending of content and pedagogy into an understanding of how particular topics, problems, or issues are organized, represented, and adapted to the diverse interests and abilities of learners, and presented for instruction” - Lee Shulman

My Challenge:

• Do students learn a subject by teaching that subject?

• Does their understanding of the subject differ when they have learned it as teachers rather than when they have learned it as students?

The Questions I Ask:

• Finding and interpreting evidence

Georgetown University

McKinley TechnologyHigh School

Instead of laboratory research, GU biology seniors develop and teach inquiry-based science

curriculum in a DC high school, study the learning of school students, and write extensive SoTL case

studies as senior theses.

Contrasting Perspectives

What Faculty Say

Simply appealing to their emotions

Missing authentic science experience

= laboratory

Learning K-12 science knowledge (content & analysis)

Activities distract from science focus

Learning about teaching = not learning about science

What Students Say

Increases awareness of relevance of science

Highly motivating

Stimulates creative thinking

Solidifies foundational knowledge

Being scientists

The Student:

“It was only after I had started researching the history of blood typing did I realize that the concept was not as clear-cut as imagined… First, I had not anticipated the depth behind the alleged simplicity of blood typing as it had been presented to me for eight years. And second, I faced a similar challenge as to how I could present this discovery to my students.”

The Faculty:

“I think that the theses are terrific. But we still fundamentally disagree about whether these students are learning any science.”

“He claims he learned about the biology of blood typing - and speaks about ‘discovering’ the information. But it is really just figure 13.4 in the Genetics textbook. He took that course with me and earned an A in it. He had already learned this material.”

Les

son

Pla

ns

Tea

chin

g M

ater

ials

Cas

e S

tud

ies

Ref

lect

ion

s &

Th

eses

Su

rvey

s &

Inte

rvie

ws

“Fly

on

th

e W

all”

Vid

eo

“Integration of

knowledge”

“The basic process of learning is universal across all disciplines; it is the intricacies specific to a subject that make it unique. Generally, educational experiences are focused on teaching learners these specifics before establishing a strong appreciation of learning, particularly a comprehension for why concepts are taught the way they are. Having approached learning as a teacher, I have asked and answered the why question for myself, enriching my teaching ability, my learning of science, and of every other discipline.”

“Ability to translate

knowledge”

“There is a distinct difference between being able to understand a foreign language, or a collegiate biology course, and having the deeper level of understanding necessary to act as a translator -- to know what has to be said and in what order, what should be emphasized, and what can be eliminated without affecting the meaning.”

“Flexible knowledge”

“We both knew that the college level version of the answer was that chemical messengers are responsible, but we realized that we didn’t understand their operation well enough to break it down in sixth grade terms…. I have discovered that in order to explain a concept to my students, I first have to have a complete understanding of its more complicated aspects so that I can identify what is most important, and then I have to be able to break the concept down and explain it clearly and concisely.”

Do Bacteria talk?

“Awareness of knowledge”

“When I was listening to some of these students talk at the Science Fair about how they really didn’t have any idea why they did the experiment they chose, all I could think was ‘that was me in Organic Chemistry. My whole philosophy in Organic Chemistry was ‘follow directions, don’t blow anything up, get out of there alive.’ I never had any idea why I did any experiment in Organic Chemistry.”

“Using the scientific method and reflecting about it as I try to teach it has made me once again see science as a process. I have found that the idea of science as a process of continual learning has implanted itself not only in my science learning, but also in other aspects of my life both academic and non-academic.”

Social Pedagogies

• Strong sense of purpose• Students develop a sense of voice

– A concrete product or authentic task, – the creation of a sense of community or

audience, – and a process that gives students critical

feedback from sources other than faculty.

Social Pedagogies & Expertise

Flexibility: Flexibility with their knowledge; apply knowledge to novel situations, take intellectual risks, manage uncertainty

Translation: Practice communication that involves translating knowledge into forms that are comprehensible and accessible to others. Acts of distillation and structuring.

Integration: Practice evaluating and assimilating diverse ideas shaped by their understanding of the social learning situation; pattern recognition.

Awareness: Develop self-awareness of their own knowledge and assumptions; confront the thinness of their learning.

Instead we need to expect learning products appropriate to learning

processes

NOVICEprocesses

LEARNINGprocesses

EXPERTprocesses


LEARNINGprocesses

LEARNINGprocesses

Flexibility

Translation

Integration

Awareness

This slide intentionally left blank

In Memory of James Slevin, 1945-2006English, Georgetown University

The Right to Literacy, with Andrea Lunsford (MLA, 1990).

Introducing English: Essays in the Intellectual Work of Composition (Pittsburgh, 2001).

Key References

• Visible Knowledge Project, online gallerieshttp://crossroads.georgetown.edu/vkp/

• Course Portfolio repository (Dan Bernstein’s portfolio)http://www.courseportfolio.org

• Knowledge media Lab, Carnegie Foundation for the Advancement of Teachinghttp://kml.carnegiefoundation.org/

• Charea Batiste, CSUMB ‘04; Cecilia O’Leary, history, CSUMB• Students in the Biology Teaching Thesis, GU• James Slevin, Francis March Award Address, available at:

http://english.georgetown.edu

http://crossroads.georgetown.edu/vkp/



http://www.courseportfolio.org/

http://kml.carnegiefoundation.org/

Making Learning Visible Through the Scholarship of Teaching and Learning University of Alaska,...

Documents

Transcript of Making Learning Visible Through the Scholarship of Teaching and Learning University of Alaska,...