Research Tools and Methods: Investigating Student · PDF fileResearch Tools and Methods:...

Research Tools and Methods: Investigating Student Success in Calculus-based Physics IKATHY KOENIG AND ALEX MARIES, PHYSICS DEPT

UNIVERSITY OF CINCINNATI

Higher education is “now reaching a major tipping point,” Coleman added. “We cannot condone poor teaching of introductory STEM courses because we are trying to weed out the weaker students in the class or simply because a professor, department and/or institution fails to recognize and accept that there are, in fact, more effective ways to teach.”

AAU Five-year Status Report 10-2-17

In 2011, it launched its Undergraduate STEM Initiative to encourage systemic reforms to science education to improve teaching and learning, especially in first- and second-year courses.

Failing to adopt evidence-based teaching practices “must be viewed as irresponsible, an abrogation of fulfilling our collective mission to ensure that all students …..are provided with the maximum opportunity to succeed”.

Department-wide innovations are becoming institutional priorities, teaching and learning centers are being redesigned, and data and analytics are being used to monitor and improve student learning.

Investigating Student Success in Calculus-based Physics I

Setting:

Spring Semester 2017

7 sections of Calculus-based physics (~130 students each)

6 faculty taught across these sections (various teaching styles)

Data Collection:

Common exam scores and final course grades (DFW rates)

Force Concept Inventory (FCI) – pre-test (and post-test)

ACT Math Scores

Student engagement survey

Conversations with faculty to better understand teaching style

Teaching Styles2 Flipped Classrooms

“Flipped” and “Flipped (New)”

videos assigned outside of class (selected from internet)

essentially no lecture

in-class problem solving (Powerpoint slides)

frequent use of clickers

Teaching Styles (cont’d)

“Active” for Active Learning

assigned textbook reading (no videos assigned)

essentially ~30%-40% lecture

in-class problem solving (similar problems to Flipped)

frequent use of clickers

Teaching Styles (cont’d)

“Lec-to-Active” initial instructor went on medical leave after 1st Exam (~week 6)assigned textbook reading used clickers and group problems solving (similar to others)

“Lec-Based 1”, “Lec-Based 2” Lec-Based 1 assigned textbook reading essentially ~75% lecture (reported by students) some in-class problem solving (similar problems to others) one section included use of clickers

0%

5%

10%

15%

20%

25%

Active Flipped 1 Flipped(New)

Lec-to-Active Lec-Based 1 Lec-Based 2

DFW

Rat

eDFW Rates Spring 2017

What goes into the Calc-Based Physics grade?

50% common exams

20% quizzes and other points up to the instructor

15% recitation attendance and homework15% WebAssign Online Homework

Red font represents common elements across courses.

26.5

27.0

27.5

28.0

28.5

29.0

29.5

30.0

Active Flipped Flipped(New)

Lec-to-Active Lec-based 1 Lec-based 2

Mea

n AC

T M

ath

(36

max

)Mean ACT Math Scores

65% is mastery

0

5

10

15

20

25

30



Pre-

FCI M

ean

(of 3

0)Force Concept Inventory Pre-test

50

55

60

65

70

75

80

85



Exam

Mea

n (%

)All Exams

(Avg of 3 Exams Plus Final)

45

50

55

60

65

70

75

80

85

90

95

1 2 3 4

Mea

n of

All

Exam

s (%

)

Quartile

Mean Exam Score by Quartiles

Active

Flipped

Flipped (New)

Lec-to-Active

Lec-based 1

Lec-based 2

45

50

55

60

65

70

75

80

85

90

95

1 2 3 4

Mea

n of

All

Exam

s (%

)

Quartile

Mean Exam Score by Quartiles

Active

Flipped

Flipped (New)

Lec-to-Active

Lec-based 1

Lec-based 2

13% spread

12% spread

10% spread

7% spread

How might these patterns be explained?

Measuring Classroom Climate

The 4 Properties of Powerful Teachers

Personality - approachable, humor, good-naturedPresence – “own” the pedagogy, connect with content

and studentsPreparation - the obvious plus reassess b/n each classPassion

– Rob Jenkins, Chronicle of Higher Education, March 16, 2015

Student Engagement Survey Survey given to all students last two weeks of the term

Survey was anonymous but tagged to instructor

Extra credit tied to survey completion (91% completed it!)

Objectives: student background info not available through

Institutional Researchhow students engaged in course componentshow useful students thought those components were how each faculty conducted class and set up classroom

climate

How often did you read the text? Active

Flipped(both*)

Lec-based(all 3**)

Read text >75% of the time 6% 25% 2%-8%Read text ~50% of the time 8% 14% 1%-9%Read text ~25% of the time 36% 33% 24%-40%Read none of the text 51% 28% 33%-66%

Of those who read the text:~10% indicated text extremely useful all the time18-33% indicated text never useful

*responses similar **Lec-based 2 did not assign textbook reading according to students

How often did you do the online homework? Active

Flipped(both)

Lec-based(all 3)

Most of the time 86% 78%-86% 79%-86%75% of the time 12% 11%-12% 8%-17%50% of the time 0% 1%-8% 2%-4%25% or less of the time 2% 1%-3% 1%-3%Was the HW useful? Active FL New Lec-basedVery useful 38% 24% 19% 18%-23%Somewhat useful 58% 62% 55% 52%-62%Not at all useful 4% 14% 26% 17%-24%

Instructor of Active class assigned HW for all sections to complete.

Attendance (self-reported) 80% attended every class (another 10% attended 3/4 of classes)

What motivated to attend? Choose ALL. Active Flip

Flip(New)

Lec-Active

Lec1

Lec2

Instructor took attendance. 72% 37% 39% 65% 90% 4%Clickers-participation grade. 80% 83% 73% 36% 2% 7%Lecture was useful. 73% 34% 8% 65% 30% 70%Instructor worked problems. 74% 63% 9% 55% 32% 63%Students worked problems. 56% 58% 11% 37% 23% 32%None of above motivated me. 0% 1% 9% 6% 5% 13%

When didn’t understand something or couldn’t do a HW problem, where did you generally go for help? (check ALL)

Choice (select ALL that apply) ResponseTextbookKhan Academy or online lecture sourceOther studentsSI LeaderLearning Assistance Center (tutoring, etc)Office Hours with instructorOffice hours with LA or graduate TAGoogle/internetI didn’t seek out helpOther

When didn’t understand something or couldn’t do a HW problem, where did you generally go for help? (check ALL)

Choice (select ALL that apply) ResponseGoogle/internet 87%Other students 77%Khan Academy or online lecture source 51%Textbook 45%SI Leader 17%Other 10%Learning Assistance Center (tutoring, etc) 7%Office Hours with instructor 5%Office hours with LA or graduate TA 3%I didn't seek out help 0%

Ranged from 0%-11%

When didn’t understand something or couldn’t do a HW problem, where did you generally go for help? (check ONE)

Choice (select ALL that apply) ResponseGoogle/internet 51%Other students 23%Khan Academy or online lecture source 10%Textbook 9%SI Leader 1%Other 1%Learning Assistance Center (tutoring, etc) 1%Office Hours with instructor 1%Office hours with LA or graduate TA 1%I didn't seek out help 0%

27%38% Flip (New)

Of the time you spent on physics each week, how much time was spent working with other students?

a. I never worked with other students outside of class.**b. I occasionally worked with other students outside of class.c. I often worked with other students but it was informal without

regular meeting times or necessarily the same students.d. I often worked with the same set of students but we never set

specific meeting times. e. I often worked with the same set of students and we set regular

study group meeting times.

Measuring Classroom ClimateMy physics instructor made me feel comfortable in asking questions either during or after class.

Active FlipFlip

(New)Lec-

ActiveLec1

Lec2

Strongly Disagree/Disagree 7% 18% 34% 9% 17% 8%Disagree Little/Agree Little 17% 41% 52% 41% 47% 37%Agree/Strongly Agree 76% 41% 14% 50% 36% 55%

Attended Office Hours: 11% 4% 0% 6% 2% 3%

Measuring Classroom ClimateMy physics instructor encouraged students to seek help if needed (i.e. office hours, Physics Learning Center, MASS, SI, etc.).

Active FlipFlip

(New)Lec-

ActiveLec1

Lec2

Strongly Disagree/Disagree 3% 5% 24% 2% 10% 7%

Disagree Little/Agree Little 12% 17% 38% 27% 41% 34%

Agree/Strongly Agree 85% 77% 40% 71% 50% 59%

Measuring Classroom ClimateMy physics instructor made me feel like I could succeed in this course.

Active FlipFlip

(New)Lec-

ActiveLec1

Lec2


Measuring Classroom ClimateMy instructor was clear and specific about what he/she expected students to learn.

Active FlipFlip

(New)Lec-

ActiveLec1

Lec2


“Active Learning” instructor posts a Weekly Pacing Guide (“Roadmap”) on Bb the week before so students know what to read and when HW is due, etc.

If you were to take this course again, list three things you would like to see kept the same.

If you were to take this course again, list three things you would like to see done differently.

Preferred Learning Environment in Physics Class

*

Choice ResponseListen to instructor lecture/solve problems ~50%*Blend of lecture, examples, group/individual work ~50%All group work with content via video/read text 2%-6%Listen to instructor lecture/NO solve problems 1%-5%

*74% for Flipped (New)

How might this be useful?

Data-driven decision making Reflective teaching (improve personal teaching style) Know student behavior (revise assignments, due dates) Know student needs (revise topics covered, time spent)

Should OneStop designate AL/Flipped versus LEC? Should students with certain attributes be

automatically placed with certain instructors?

Questions?

Kathy Koenig

[email protected]

0

5

10

15

20

25FC

I Mea

n Sc

ores

(30

max

)Pre- and Post-FCI Mean Scores

Pre-FCIPost-FCI

65% is considered mastery

Other questions to consider asking…

Indicate the percent of class time your instructor typically spent on each activity (the total should add to 100%).

My instructor tried hard to make sure students understood the physics concepts discussed in class.

My physics instructor provided explanations of course content that were understandable.

My physics instructor seemed to genuinely care about whether or not I learned the material.

7 Learning Principles#1-- All learning involves connecting up with and building on prior thinking and knowledge base.

#2 Motivation to learn is essential. Is necessary element of teaching.

#3– Engagement - People must think hard about a subject to learn it.

#4 Need effective feedback--Guidance that shapes thinking and learning.

#5 Expert thinking involves more than knowing information, is also how information is organized, applied, and learned.

#6 Part of memory that remembers and process ideas on short times scales extremely limited capacity! More loaded down, less well it processes, less learned.

#7 Requirements for retention well established.

Research Tools and Methods: Investigating Student · PDF fileResearch Tools and Methods:...

Documents

Transcript of Research Tools and Methods: Investigating Student · PDF fileResearch Tools and Methods:...