Statistics (and SoTL Projects)

Jim SmithMichigan State UniversityLyman Briggs College

It’s as much who you know, as what you know!

“Lies, Damned Lies, and Statistics”

A Case Study in Statistics

Use of a Stats Analytical Framework

Statistics 101

Some Basics

Which Statistical Test Should You Use in Your SoTL Work?

Some Examples

Learning Goals

By the end of this session, scholars will (have):

• Understand the need for and the role(s) of statistical tests in science education research;

• Be able to explain some basic tenets of statistics;

• Explored some literature examples of the application of statistics to SoTL work; and

• Begin to formulate a statistical framework for their Biology Scholars research project.

Statistics and SoTL Projects

Did student learning increase as a result of my cell model assignment?

Do representations in comic book form improve student understanding of photosynthetic processes?

Are there differences in the preparedness and motivation levels of students who W/D/F in Introductory Biology?

Does increased metacognition decrease the W/D/F rate in Introductory Biology?

Statistics are important components of strategies to answer these questions!

With respect to Statistics:

Where would you place yourself on the following scale?

1. From my perspective, there’s “Lies, Damned Lies, and Statistics”

2. I agree with the above statement somewhat, but not strongly.

3. I am halfway between “Damned Lies” and “Comfort”.

4. I agree with the below statement, but not strongly.

5. I am completely comfortable with statistics as they apply to SoTL work.

The Problem – Young and Fry (2008)

“While the results are reported to be statistically significant, I am not familiar enough with the Spearman’s Rho nonparametric correlation analysis to know whether the data are truly convincing. ”

ASM Bio Scholar Summer 2012

Young and Fry (2008)

Framework for Analysis of Statistical Inference

Identify Variables of Interest

Formulate

Hypotheses Design

Sampling Strategy

Collect Raw Data

Perform

Statistical Test(s)

Inference(Level 1)

Inference (Level 2+)

Tasks

• Identify the appropriate elements in the article of interest and map them onto the boxes

• Decide if the statistical test was appropriately chosen and carried out

• Decide if the claims (inferences) are supportable at each level

• Tie the framework to your project

Smith, Valles & Zeleke, in prep.

An Application – Young and Fry (2008)

What is the MAI, and how was it used to measure aspects of metacognition?

An Application – Young and Fry (2008)

What is the difference between metacognitive knowledge and metacognitive regulation?

Framework for Analysis of Statistical Inference

Identify Variables of Interest

Formulate

Hypotheses Design

Sampling Strategy

Collect Raw Data

Perform

Statistical Test(s)

Inference(Level 1)

Inference (Level 2+)

Tasks

• Identify the appropriate elements in the article of interest and map them onto the boxes

• Decide if the statistical test was appropriately chosen and carried out

• Decide if the claims (inferences) are supportable at each level

• Tie the framework to your project

Smith, Valles & Zeleke, in prep.

ASM BSP S12: STATS 101

What do I really need to know about Statistics?

ASM BSP S12: STATS 101

What do I really need to know about Statistics?

If you are going to do SoTL research, you really do need a basic working knowledge of statistical inference

Fun, but no help for today

Extremely helpful!

What the difference between:

Experimental (or Research) Hypothesis

and

Statistical Hypothesis

What the difference between:

Experimental (or Research) Hypothesis

HA: Metacognition improves student learning

and

Statistical Hypothesis

What the difference between:

Experimental (or Research) Hypothesis

and

Statistical Hypothesis

HA: Course grade is correlated with MAI total

What the difference between:

Experimental (or Research) Hypothesis

and

Statistical Hypothesis

H0: No correlation of Course grade with MAI total

The Problem – Young and Fry (2008)

The data allow us to reject the hypothesis that there is no correlation between Course Grade and MAI total.

What is the difference between significance level and effect size?

What the difference between:

Experimental (or Research) Hypothesis

HA: Metacognition improves student learning

and

Statistical Hypothesis

What the difference between:

Discrete Numbers

and

Continuous Numbers

With respect to Statistics:

Where would you place yourself on the following scale?

1. From my perspective, there’s “Lies, Damned Lies, and Statistics”

2. I agree with the above statement somewhat, but not strongly.

3. I am halfway between “Damned Lies” and “Comfort”.

4. I agree with the below statement, but not strongly.

5. I am completely comfortable with statistics as they apply to SoTL work.

What the difference between:

Discrete Numbers

Categorical; membership known without error

and

Continuous Numbers

What the difference between:

Discrete Numbers

and

Continuous Numbers

Not known without error; have a distribution

Overall: Mean = 74.70 (74.70%) +/- 11.45 SD TH: Mean = 24.47 (81.57%) +/- 3.91 SD ER: Mean = 28.96 (72.40%) +/- 5.52 SD MC: Mean = 21.27 (70.90%) +/- 4.45 SD

Discrete or Continuous?

What the difference between:

Non-parametric Statistics

For discrete data; for data that violate parametric assumptions

and

Parametric Statistics

What the difference between:

Non-parametric Statistics

e.g., Chi-squared tests, Mann-Whitney U test, Wilcoxon Rank Sums test

and

Parametric Statistics

What the difference between:

Non-parametric Statistics

and

Parametric Statistics

What the difference between:

Non-parametric Statistics

and

Parametric Statistics

e.g., t-test, ANOVA, Pearson’s r

Assumptions of Parametric Statistics(for continuous data)

Observations are unbiased and independent

Data are normally distributed

Equal variances across groups

Sta

tist

ical

Dec

isio

n T

ree

From: Robert Gerwien, “A Painless Guide to Statistics: Online Resources for Biology”, http://abacus.bates.edu/~ganderso/biology/resources/statistics.html

Sta

tist

ical

Dec

isio

n T

ree

Was the use of Spearman’s Rank Correlation test appropriate?

Work with your group

Yes/No, and why

Young and Fry: Revisited

Young and Fry: Was theirs the appropriate statistical test?

REM - You will find disagreement in the ranks (including among manuscript reviewers)

Test Time: What kind of data? Test?

Encouragingly, using t test analysis, we found that in four of our courses (both BSCI 223 courses, BSCI 424, and BSCI 422) there was significant improvement on the concept inventory scores from presurvey to postsurvey.

From Marbach-Ad et al. 2009

Test Time: What kind of data? Test?

From Drew & Triplett 2008

Test Time: What kind of data? Test?

From Steck et al. 2012

Test Time: What kind of data? Test?

From Steck et al. 2012

Test Time: What kind of data? Test?

From Steck et al. 2012

“I’m baffled…”

You may (probably) need help!!

Become friends/colleagues with a really smart biologist or social scientist who uses a statistical approach in their own work

• A lot of these folks have strong experimental design skills

• Ecologists and Evolutionary Biologists seem to be particularly strong

On-campus statistical consulting service?