1 Psychology 242 Introduction to Research Course Overview Module This module is best used as a...

1Psychology 242Introductionto Research

Course Overview Module

This module is best used as a PowerPoint “Show”.

Best way to print this: Click ‘File” “Print’; In the dialogue box click

“print what?”. Select “Handouts (3 slides per page)”

Go to “slide show” and click “run show”

Click anywhere

© Dr. David J. McKirnan, 2014The University of Illinois [email protected] not use or reproduce without permission

5/1/14

mailto:[email protected]



Psychology 242, Dr. McKirnan

Final Exam Review

Cranach, Tree of Knowledge [of Good and Evil] (1472)

This module is best used as a PowerPoint “Show”.

Best way to print this: Click ‘File” “Print’; In the dialogue box

click “print what?”.

Select “Handouts (3 slides per page)”

Go to “slide show”, click “run show”

© Dr. David J. McKirnan, 2014The University of Illinois [email protected] not use or reproduce without permission



3Psychology 242Introductionto Research What is science?

What is science?

Content Empirical findings: Facts Ways of classifying nature Well supported theories

Methods Core empirical approach Basic experimental design Specific research procedures Statistical reasoning

Values Critical thought Theory: Why? or How? Evidence: How do you know? Discover the natural world

4Psychology 242Introductionto Research Irrational beliefs

Psychology 242, Dr. McKirnan Week 12-13, quasi-experimental designs.

Wish fulfilling, emotion-based beliefs:

• Spurious correlations• Evaluating evidence

Critical thought – rational, empirical-based analysis – is cognitively effortful

Our brains may be “hard wired” for irrational beliefs.

Cognitive biases:

Rationalism & science have a tough row to hoe

• …self-satisfying; confirmatory bias

• …differentiating facts from opinions

• …emotional responses precede thought


Intuition:

Rationalism:

Empiricism:


Four basic sources of knowledge or information:

Authority: Credible / powerful peopleSocial institutionsTradition

Emotionality or a “hunch”

“Emotional IQ”

Simple sensation / perceptionDirect observation; data

Logical coherenceArticulation with other ideas

Most central to Science

How do we know things?


What does science do?

Describe the world Initial approach to scientific study: “what is it” Leads to hypotheses

Predict events Core feature of a hypothesis: if “X” then “Y”. Often still descriptive rather than experimental.

Test theories Cause and effect questions involving hypothetical

constructs. Often controlled experiments or complex correlation

designs. Test applications of theories

Psychology 242, Dr. McKirnan Week 2: Role & structure of science.

Using theory to model change Testing interventions or policy

What does science do?


Psychology 242, Dr. McKirnan Weeks 1 & 2; Introduction to science.

Basic features of a research study

Basic features of research; Theory Hypothetical construct Hypothesis Replication Operational definition Internal & external validity Confound Independent v. Dependent variables

Which is the “cause” & which is the “effect”? Which is measured & which is manipulated?

Measurement v. experimental studies

Click through and be sure you can define each of these.


Basic Elements of a Research Project

MethodsMeasurement v.

experimental

ConclusionsFuture research?

PhenomenonBig picture / question

Theory Hypothetical Constructs

Causal explanation

Hypothesis Operational definition

Specific prediction

Data / Results• Descriptive data• Test hypothesis

DiscussionImplications for theory

Then specific methods, the core of a scientific study.

…and derive concrete hypotheses.

Then actual data & results…

… articulate a clear theory

Begin with the “big question”

…and larger issues.

… implications for the theory

Core elements of a research study



Core features of a research study:

Hypothesis

Theory

Methods

Data & Analysis

Discussion

Hypothetical constructs In important relationship

More specific variables Falsifiable prediction

Operational definition Internal & external validity

Numerical representation Normal distribution Probability

Meaning of these results for the theory Study Limitations:

Internal validity? External validity?

Results Descriptive: Empirical question or exploration Hypothesis: Statistical significance

Know these key terms & concepts.


Psychology 242, Dr. McKirnan Weeks 1 & 2; Introduction to science.

Section 1 study guide

Core elements of the research flow

Each component of the research flow corresponds to a later component…


Psychology 242, Dr. McKirnan Week 3; Experimental designs

Research process: The Big Picture

PhenomenonBig picture question.

Theory 2Alternate explanation,

invoking other hypothetical constructs.

Hypothesis 2Another prediction that tests

the same theory.

Theory 1Possible explanation,

invoking one set of hypothetical constructs.

Methods 1Operationally define the

variables & test the hypothesis.

Methods 2An alternate operational definition & way of testing

the hypothesis.

Hypothesis 1A prediction that logically

flows from – and tests – the theory.



Basics of Design: Internal Validity

Internal Validity: Can we validly determine what is causing the results of

the experiment?

General Research Hypothesis: the experimental outcome (values of the Dependent Variable) is caused only by the experiment itself (Independent Variable).

Confound: a “3rd variable” (unmeasured variable other than

the Independent Variable) actually led to the results.

Core Design Issues:

1. Appropriate control group

2. Equivalent experimental & control groups (except

for the Independent Variable).



External validity: summary

The study structure & context

The research Setting:

The Dependent

Variable

The research Sample:

Is the sample typical of the larger population?

Is this typical of

“real world”

settings where the phenomenon occurs?

Is the outcome measure represen-tative, valid &

reliable?

Does the experimental manipulation (or measured predictor) actually create (validly

assess…) the phenomenon you are interested in?

The Independent

Variable

External Validity: Can we validly generalize from this experiment to the

larger world?



Validity & Research approaches

Observation or Measurement Experiments

Simple Description Correlational Studies

Quasi-experiment

s

“True” experiment

sQualitative Quantitative

Explore the actual process of a behavior.

Describe a behavioral or social trend.

Relate measured variables to each other to test hypotheses.

Test hypotheses in naturally occurring events or field studies.

Test specific hypotheses via controlled “lab” conditions.

External validity Internal validity

Less control:

Observe / test phenomenon under natural conditions.

More accurate portrayal of how it works in nature

Less able to interpret cause & effect

More control:

Create the phenomenon in a controlled environment

Address specific questions or hypotheses

Better interpret cause & effect

Know what these research strategies represent & how they differ.

Understand the trade-off of internal & external validity across them.



Quasi-experiments

2. Evaluate existing groups or program(s) Simple survey of an intervention that already occurred Non-equivalent designs, due to Time series designs, often with archival data

1. Study naturally occurring events that could not be brought into a lab or a true experiment. Measurement studies Retrospective designs

Understand these two forms of quasi-experiments.

Understand these forms of non-equivalent designs.

Self-selection Non-random assignment Use of existing groups Participants not blind

Quasi-experimental designsExperimental designs for “studies in nature”.



True v. quasi-experimental designs, 3

True experiments: Quasi-experiments:

Emphasize Internal Validity Assess cause & effect (in relatively artificial

environment) Test clear, a priori hypotheses

Emphasize External Validity Describe “real” / naturally occurring events Clear or exploratory hypotheses

Groups Equivalent at baseline Random Assignment (or matching). Participants & experimenter Blind to

assignment.

Non-equivalent groups Non-random assignment Existing groups Self-selection Participants not blind.

Control study procedures Create / manipulate the independent variable Control procedures & measures

Complete Control not Possible May not be able to manipulate the independent

variable Partial control of procedures & measures

Know clearly how quasi-experiments differ from true experiments. In that light, know the core characteristics of an experiment and why

those characteristics are important.



Maturation

Reactive measures

Statistical regression

Mortality / drop-out

HistoryHistorical / cultural events occur between baseline & follow-up.

Individual maturation or growth occurs between baseline & follow-up.

People respond to being measured or being a measured a second time.

Extreme scores at baseline “regress” to a more moderate level over time.

People leave the experiment non-randomly (i.e., for reasons that may affect the results…).

Quasi-experiments that do not have a control group:

Group Intervention or event Observe2Observe1

Confound Observe2Observe1

Threats to internal validity (confounds):

Know these! What is a confound? Why is that

important?



Non-equivalent (quasi-experimental) designs

Group

Group

Observe1

Observe1

Two Group Pre- Post- Design

Non-equivalent groups Self-selection Non-random assignment Use of existing groups Participants not blind

Observe2

Intervention or event Observe2

Intervention & Assessments often controlled by researcher in these designs.

Similar to true experimental design, except for non-equivalent groups

Contrast group

Understand this slide.



Sampling overview

Who do you want to generalize to? Who is the target population?

broad – external validity narrow – internal validity

How do you decide who is a member? demographic / behavioral criteria? subjective / attitudinal?

What do you know about the population already – what is the “sampling frame”?

Will you use a:

Probability or random sample?

What does this mean?

Why does this make a difference?

Sampling

Most externally valid & representative Assumes:

Less valid for hidden groups.

• Clear sampling frame• Population is available

Non-probability or convenience sample

targeted / multi-frame snowball…

Less externally valid Best when:

No clear sampling frame Hidden / avoidant population.

20Psychology 242Introductionto Research Ethics


Research Ethics:The Tuskegee StudyThe Common Rule

The Belmont Report

21Foundations of REsearch Tuskegee Study: Overview

Tuskegee study begin as a potentially valuable trial of treatment outcomes

Begun – and should have remained – a natural history of participants’ response to treatment.

Became a wholly unethical no-treatment history. Based on spurious – and racist – scientific reasoning about

differences between Africans and Caucasians Investigators took advantage of participants economic and social

vulnerability to exploit and harm them. Note: Tuskegee participants were not actually given syphilis; they

were not given treatment.

Tuskegee led to many of our research norms and institutional controls.

22Psychology 242Introductionto Research Ethics procedures stemming from Tuskegee

Dr. David J McKirnan, [email protected]

Informed consent

Non-coercive enrollment & retention

Led to the 1979 Belmont Report

Indirectly to core elements of the “Common Rule”.

Ethical review & monitoring

Led to establishment of the Federal Office for Human Research Protections (OHRP)

Led to laws requiring Institutional Review Boards (IRBs)

All Federally funded research must be reviewed and monitored by a local IRB

Most institutions (e.g., UIC) require IRB approval of all research, federally funded or not.

Have a general sense of why Tuskegee was unethical, and how it influenced our ethics decision making now


Dr. David J McKirnan

The Common Rule

Minimize risks

Risks must be reasonable

Recruit participants equitably

Informed consent

Document consent

Monitor for safety

Protect vulnerable participants & maintain confidentiality

The “Common Rule” criteria for Human Subjects Protection

Understand what each of these mean.

24Psychology 242Introductionto Research Belmont Report (CITI training)

1. Respect For Persons

Exercise autonomy & make informed choices.

2. Beneficence

Minimize risk & maximize of social/individual benefit.

3. Justice Do not unduly involve groups who are unlikely to benefit.

Include participants of all races & both genders

Communicate results & develop programs/ interventions

Dr. David J McKirnan

You know these from your CITI training.

Generally understand them; be able to recognize these key values.


Psychology 242, Dr. McKirnan Descriptive Research.

Descriptive research

Quantitative Qualitative or Observational Existing data

Describe an issue via valid & reliable numerical measures

Simple: frequency counts of key behavior

“Blocking” by other variables

Correlational research: “what relates to what”

Study behavior “in nature” (high ecological validity).

Qualitative

Interviews

Focus groups

Textual analysis

Observational Direct

Unobtrusive

Use existing data for new quantitative (or qualitative) analyses

Accretion Study “remnants” of

behavior

Wholly non-reactive

Archival Use existing data to

test new hypothesis

Typically non-reactive

What does it mean for research to be ‘reactive’?


Descriptive data


Testing hypothesis with Archival, Time Series data

Archival data: Already exist, collected for another reason

Time series: “Snapshots” of a variable over time, sampling different people each time

Longitudinal: Follow the same cohort of people over time.

Quasi-independent variable: naturally occurring event, e.g. Magic Johnson testing positive for HIV HIV testing rates?

See next slide:

27Psychology 242Introductionto Research Archival, time series data example: Magic Johnson




Correlation designs: Drawbacks & fixes

Confounds!; unmeasured 3rd variable problem

Causality; a simple correlation may confuse cause & effect.

Dealing with confounds: Use complex measurements or samples to eliminate alternate hypotheses.

Alcohol consumption

Depression

Stock marketHemlines

General optimism

?

?

This slide illustrated the “3rd variable problem” in interpreting correlational data.

What does that refer to? Why is that important? Can you generate an example of that in a few words?



Descriptive Research: Overview

Reliability Test – retest Split – half Alpha (internal)

Validity Face Content Predictive Construct Ecological

Basic design issues:

Time frame Cross sectional Longitudinal Case study

Know what these terms mean. Go back to the lecture notes or your book for definitions & examples.

30Psychology 242Introductionto Research Statistics: an introduction

Using numbers in science

Number scales & frequency distributions

Central Tendency: Mode, Median, Mean

Variance: Standard Deviation

The Z score and the normal distribution

Using Z scores to evaluate data

Testing hypotheses: critical ratio.


Psychology 242, Dr. McKirnan Exam #3 study guide

Distributions

Normal distribution: mean = mode = median at center of the distribution

Mean

Median

Mode

Median

Mode

MeanBimodal distribution Mean & median

are similar, at the center.

Mode

Mean

Median

Skewed distribution: Extreme scores in one direction make the median, and mean larger than the mode.

What are examples of data that might fall into these distributions?

32Psychology 242Introductionto Research Scales


Types of numerical scales



Ratio zero point grounded in physical property; values

are “absolute”

continuous & equal intervals

physical description: elapsed time, height

Intervalno zero point; scale values relative

continuous with equal interval

behavioral research, e.g., attitude or rating scales.

Ordinalrank order with non-equal intervals; no ‘0’ point

Simple finish place, rank in organization...

Categorical ‘values’ = categories only

inherent categories: ethnic group, gender, zip code

Types of numerical scales

Continuous scales (scores on a continuum)

Be able to provide or recognize examples of these scale types

34Psychology 242Introductionto Research Scales and Central Tendency


Measure of Central tendency Typically used for:

Mode (most common score) categorical variablesoften: bimodal distributions

Median (middle of distribution) categorical or continuous variables

highly skewed data

Mean (average score) continuous variables onlymore “normal” distributions

use different measures of central tendency.



Measures of Dispersion or Variance

1. Range of the highest to the lowest score.

Provides simple idea of where scores fall

Very sensitive to any extreme score(s) (“outliers”).

2. Standard deviation of scores around the Mean

Similar to “average” amount each score deviates from the M.

“Standardizes” scores to a normal curve, allowing for basic statistics.

More accurate & detailed than range

You should know these by now

Two measures of variance

36Psychology 242Introductionto Research z


How far is your score (X) from the mean (M)

How much variance is there among all the scores in the sample [standard deviation (S)]

Z = X–M

S=

You must know the Z score

It is the core form of the critical ratio. It represents the:

Strength of the experimental effect

Adjusted by the amount of error variance

Z



The normal distribution is a hypothetical distribution of cases in a sample

It is segmented into standard deviation units.

Each standard deviation unit (Z) represents a fixed % of cases

We use Z scores & associated % of the normal distribution to make statistical decisions about whether a score might occur by chance.

Z and the normal distribution

If you do not fully understand this slide go back to the Statistics 1 lecture notes and figure it out!!

Remember approximations of these numbers



Normal distribution; Z scores

1. Calculate how far the score (X) is from the mean (M); X–M.

2. “Adjust” X–M by how much variance there is in the sample via standard deviation (S).

3. Z = X–M / S

How “good” is a score of ‘6' in two groups?

Table 1, high varianceMean (M) = 4, Score (X) = 6

Standard Deviation (S) = 2.4.

(X-M = 6 - 4 = 2)

Z (X-M/S) = 2/2.4 = 0.88

Table 2, low(er) varianceMean (M) = 4, Score (X) = 6

Standard Deviation (S) = 1.15.

(X-M = 6 - 4 = 2)

Z (X-M/S) = 2/1.15 = 1.74

Use Z to evaluate a scoreDistance from M / “error” variance



Evaluating scores using Z

-3 -2 -1 0 +1 +2 +3Z Scores

(standard deviation units)

X = 6, M = 4, S = 2.4, Z = .88

X = 6, M = 4, S = 1.15, Z = 1.74

70% of cases

90% of cases

C. Criterion for a “significantly good” score

I need you to understand the logic of this approach.

If your criterion for a “good” score is that it surpass 90% of all scores…

With high variance a ‘6’ is not “good”.

With lower variance ‘6’ is good.



Core research questions

One participant’s score

Means for 2 or more groups

Scores on two measured variables

Does this score differ from the M for the group by more than chance?

Is the difference between these Means more than we would expect by chance? -- more than the M difference between any 2 randomly selected groups?

Is the correlation (‘r’) between these variables more than we would expect by chance -- more than between any two randomly selected variables?

Data Statistical Question

Analyze with Z score

Analyze with t score

Analyze with r


Psychology 242, Dr. McKirnan Statistics introduction 1

Numbers are important for representing “reality” in science (and other fields).

Different measures of central tendency are useful & accurate for different data;

Mean is the most common.

Median useful for skewed data

Mode useful for simple categorical data

Variance (around the mean) is key to characterizing a set of numbers.

We understand a set of scores in terms of the:

Central tendency – the average or Mean score

The amount of variance in the scores, typically the Standard Deviation.

Summary



Z is the prototype critical ratio:

Summary

Statistical decisions follow the critical ratio:

How far is your score (X) from the mean (M)

How much variance is there among all the scores in the sample [standard deviation (S)]

Z =X–M

S=

t is also a basic critical ratio used for comparing groups:

How different are the two group Means

How much variance is there within each the two groups; (“standard error of the mean”)

t =M1 – M2=

grp2

grp2

grp1

grp1

n

Variance

n

Variance

You must understand what a critical ratio is.

This slide needs to make perfect sense to you!!


Statistics Introduction 2.

Dr. McKirnan, Psychology 242

Introduction to statistics # 2

Revised 4/5/09

What can Plato’s Allegory of the Cave tell us about scientific reasoning?

Was our hypothesis supported? The critical ratio and the logic of the t-test.

The central limit theorem and sampling distributions

Correlations and assessing shared variance

"The Allegory of the Cave" by Allison Leigh Cassel

44Psychology 242Introductionto Research Plato’s Cave, 6

We cannot observe “nature” directly, we only see its manifestations or images:


We are trapped in a world of immediate sensation;

Our senses routinely deceive us (they have error).

We cannot get outside our limited sensations to see the underlying “form” of nature

What does Plato’s Allegory of the Cave tell us about scientific reasoning?

45Psychology 242Introductionto Research Plato’s Cave, 2

e.g., evolution, gravity, learning, motivation…


We study hypothetical constructs; basic “operating principles” of nature

Processes that we cannot “see” directly…

…that underlie events that we can observe.

We test hypotheses about what we can see and use rational analysis – theory – to deduce what the “form” of these processes must be, and how they work.



Why can’t we just observe “nature” directly?

1. We can only observe the effects of hypothetical constructs, not the processes themselves.

2. We examine only a sample of the world; no sample is 100% representative of the entire population

3. Our theory helps us develop hypotheses about what we should observe if our theory is “correct”.

4. We test our hypotheses to infer how nature works.

5. Our inferences contain error: we must estimate the probability that our results are due to “real” effects versus chance. You must understand these

basic concepts and terms!



“Statistical significance”

We assume that a score with less than 5% probability of occurring (i.e., higher or lower than 95% of the

other scores) is not by chance alone … p < .05)

Z > +1.98 occurs < 95% of the time (p <.05).

If Z > 1.98 we consider the score to be “significantly” different from the mean

To test if an effect is “statistically significant”…

Compute a Z score for the effect

Compare it to the critical value for p<.05; + 1.98

Really important

Testing statistical significance



About 95%of cases

Statistical significance & Areas under the normal curve

95% of scores are between Z = -1.98 and Z = +1.98.

2.4% of cases2.4% of

cases

-3 -2 -1 0 +1 +2 +3Z Scores


Z = +1.98Z = -1.98



-3 -2 -1 0 +1 +2 +3

Z Scores (standard deviation units)

34.13% of

cases

34.13% of

cases

13.59% of

cases

2.25% of

cases

13.59% of

cases

2.25% of

cases

2.4% of cases

2.4% of cases

Z = +1.98Z = -1.98

In a hypothetical distribution:

2.4% of cases are higher than Z = +1.98

2.4% of cases are lower than Z = -1.98

Thus, Z > +1.98 or < -1.98 will occur < 5% of the time by chance alone.

Statistical significance & Areas under the normal curve

95% of cases

With Z > +1.98 or < -1.98 we reject the null hypothesis & assume the results are not by chance alone.



Critical Ratio



Critical ratio

Critical ratio =

The strength of the results (our

direct observation of nature)

Amount of error variance (the odds that our observation is due to chance)

Difference between Ms for the two groups

Variability within groups (error)

control group

experimental group

Mgroup2

Mgroup1

Within-group variance, group2Within-group

variance, group1

t =



The Critical Ratio in action

Low variance

All three graphs have = difference between groups.

They differ in variance within groups.

The critical ratio helps us determine which one(s) represent a statistically significant difference.

Medium variance

High variance

Be able to answer these:

How do the between group variance & within group variance constitute the critical ratio.

t represents the critical ratio for group comparisons: how does t vary for these three examples?

Which might reflect a statistically significant difference?



The Central Limit Theorem; small samples

<-- smaller M larger --->

True Population M “True” normal

distribution

ScoreScore Score

ScoreScore

Score Score Score

Score

Score

Score

With few scores in the sample a few extreme or “deviant” values have a large effect.

The distribution is “flat” or has high variance.

Central limit theorem



The Central Limit Theorem; larger samples

With more scores the effect of extreme or “deviant” values is offset by other values.

Central Limit Theorem

The distribution has less variance & is more normal.

ScoreScoreScore Score

ScoreScore

Score Score

Score

Score



distribution

Score

Score ScoreScore

Score Score ScoreScoreScoreScore

Score

Score

ScoreScore

ScoreScore

Score

Score

Score Score



The Central Limit Theorem; large samples

Central Limit Theorem

ScoreScoreScore Score

ScoreScore

Score Score

Score

Score



distribution

Score

Score ScoreScore

Score ScoreScoreScoreScore

Score

Score

Score

ScoreScore

ScoreScore

Score

Score

Score Score

ScoreScore

Score

Score Score

Score

Score

ScoreScore

ScoreScore

Score

Score

Score

Score

Score

Score

Score

ScoreScore

Score

ScoreScore With many scores “deviant” values are completely offset by other values.

The distribution is normal, with low(er) variance.

The sampling distribution better approximates the population distribution

Be able to apply the central limit theorem logic to evaluating t.

Translate that to using the t table.



Central limit theorem & evaluating t scores

1. Smaller samples (lower df) have more variance.

2. So, t must be larger for us to consider it statistically significant (< 5% likely to have occurred by chance alone).

3. Compare t to a sampling distribution based on df.

4. Critical value for t with p <.05 goes up or down depending upon sample size (df)


1.860 2.306 2.896 3.355 5.041

1.833 2.262 2.821 3.250 4.781

1.812 2.228 2.764 3.169 4.587

1.796 2.201 2.718 3.106 4.437

1.782 2.179 2.681 3.055 4.318

1.771 2.160 2.650 3.012 4.221

1.761 2.145 2.624 2.977 4.140

1.753 2.131 2.602 2.947 4.073

1.734 2.101 2.552 2.878 3.922

1.725 2.086 2.528 2.845 3.850

1.708 2.060 2.485 2.787 3.725

1.697 2.042 2.457 2.750 3.646

1.684 2.021 2.423 2.704 3.551

1.671 2.000 2.390 2.660 3.460

1.658 1.980 2.358 2.617 3.373

1.645 1.960 2.326 2.576 3.291

A t-table specifies Critical Values:

Alpha Levels

0.10 0.05 0.02 0.010.001

Critical values for testing whether an effect is

Statistically Significant

df8 9

101112131415182025304060

120

Alpha = .05, df = 8

Alpha = .05, df = 18

Alpha = .05, df = 120

Know how to use a t table.

What is ‘Alpha’?

What are Degrees of Freedom (df)?

What is a ‘Critical Value’?

Alpha = .01, df = 40


df = 8,

-2 -1 0 +1 +2 Z Score


Central Limit Theorem; variations in sampling distributions

df = 18,

As samples sizes ( df ) go down…

the estimated sampling distributions of t scores based on them have more variance,

giving a more “flat” distribution.

This increases the critical value

for p<.05.

df = 120, t > ±1.98, p<.05

t > ± 2.10, p<.05

t > ± 2.31, p<.05

Get this! -- Be able to go to a t table and apply this logic.

Give yourself the Statistics Lectures 2 notes for details.


t-test We create group differences

on the Independent Variable. …and assess how the groups

differ on the Dependent Var.

Taking a correlation approach


Difference between groups standard error of M

Correlation; We measure individual

differences on the predictor variable…

and see if they are associated with differences on the outcome.

Σ (Z var1* Z var2)Df (n-1)

Correlations


Are people a given amount above (or below) the mean of one variable equally above (or below) the M of the 2nd variable?

We measure distance from M using Z scores.

r can range from -1.0 to +1.0

E.g., if participants who have Z = +1.5 on variable 1 also have Z = 1.5 on variable 2, etc., r = +1.0.


Statistics summary: correlation

Pearson Correlation (r): measures how similar the variance is between two variables (“shared variance”) within a group of participants.

r: Σ (Z var1* Z var2)

Df (n-1)r =

For each participant multiply the Z scores for the two variables

Sum across all participants

Divide by df:



Type I and Type II errorsKnow what the Null Hypothesis is!*

*Any effect is due to chance alone



Type I v. Type II errors

“Reality”

Ho true[effect due to chance

alone]

Ho false[real experimental

effect]

Decision

Accept HoCorrect decision

Type II error

Reject Ho Type I errorCorrect decision



Statistical Decision Making: Errors

Type I error; Reject the null hypothesis [Ho] when it is actually true: Accept as ‘real’ an effect that is due to chance only

Type I error rate determined by Alpha (.10, .05, .01…)

More “liberal” alpha (e.g., .10)

reject Ho more often.

Worst form of error: statistical conventions are designed to prevent type I errors




“Reality”


alone]


effect]

Decision


Type II error




Statistical Decision Making: Errors

Type II error; Accept Ho when it is actually false;

Assume as chance an effect that is actually real.

Type II most strongly affected by statistical power (df):

Central Limit Theorem:

Smaller samples Assume more varianceMore conservative

critical value*

*within a given alpha…

Too conservative a critical value Type II error




“Reality”


alone]


effect]

Decision


Type II error


Understand the logic of Type I & Type II errors.

Be able to map these on to alpha levels and df in your study.



Inferential statistics: summary, Key terms

Plato’s cave and the estimation of “reality” Hypothetical constructs actual observations

Sample population

Inferences about our observations: Deductive v. Inductive link of theory / hypothetical constructs

& data

Generalizing results beyond the experiment

Critical ratio / Z You will be asked to produce and describe this.

Variance, variability in different distributions

Degrees of Freedom [df]

68Psychology 242Introductionto Research Inferential statistics, cont.


t-test, between versus within –group variance

Sampling distribution, M of the sampling distribution

Alpha (α), critical value

t table, general logic of calculating a t-test

“Shared variance”, positive / negative correlation

General logic of calculating a correlation (mutual Z scores).

Null hypothesis, Type I & Type II errors.



Multiple independent variables4/14/09

Testing hypotheses about > 1 independent variable

Factorial Designs:

Main effects,

Additive Effects,

Interactions



> 1 independent variable

Designs with > 1 Independent Variable

Why have more than one IV? Include a ‘control’ variable

Test 2 (or more) Independent variables




Include a ‘control’ variable as a second I.V. 1. Block the data by gender, age, race, attitudes, etc.

2. Test if the main Independent Variable has the same effect within both groups

What is the effect of self-reflection on stress reduction?

Hypothesis: training in self-reflection helps buffer the stress of exams.

2nd Question: is that effect the same in women and men? [old v. young, etc…]

Main effect: Self-reflection training less stress

Interaction: training less stress worked for women, not men.

Conclusion: Including a ‘control’ variable helped clarify the results.

E X

A M

P L

E




A. Test separate, ‘main effects’ of each I.V. (Do each of these variables significantly affect the outcome?)

B. Test ‘additive’ effects of > 1 I.V.s simultaneously (What is the combined effect of these variables?)

C. Test interaction of 2 or more I.V.s (Does the effect of one I.V. on the outcome depend upon a second variable...?)

Testing more than one Independent Variable

Know the difference between a main effect, an additive effect, and an interaction.



Interaction example: Genetics, stress and depression

Participants’ genotype and level of childhood trauma interact in depression.

There is a general (main) effect whereby more trauma leads to greater likelihood of adult depression



Interaction example: Genetics, stress and depression, 2

However … the effect of trauma interacts with genetics

Childhood trauma has no effect in people who have no genetic vulnerability.

With increasing vulnerability, increasing trauma increases the likelihood of depression

Understand clearly why/how this is an interaction, not a main effect or additive effect.

Also understand how the interaction tells us much more than the simple main effect.


Psychology 242, Dr. McKirnan Multiple independent variables

Figure 3 Mean ratings of subjective stimulation and sedation on the BAES under 0.65 g/kg alcohol and placebo in women and men.

Example of a 3-way interaction

Alcohol (v. placebo) made men much more stimulated.

Alcohol made women much more sedated



M B

AE

S s

ubsc

ale

scor

es

Alternate portrayal of 3-way mood interaction

0

5

10

15

20

25

30

35

40

45

50

Stimulation Sedation

Men, AlcoholMen, PlaceboWomen, AlcoholWomen, Placebo

Placebo conditions do not show much effect

The alcohol conditions show a classic “cross-over” effect for gender & mood;

Men get aroused

Women get sedated

Why/how is this an interaction?



Multiple IVs; summary 2

Are critical to theory development and testing:

Alcohol makes it more difficult to inhibit behavior, but primarily among men.

Stress or other environmental events can “switch on” genes that create psychological or other problems; genetic dispositions and environment are not separate processes.

Multiple Independent Variables / Predictors:

Establish key “boundary conditions” to theory: when and among whom does a basic psychological process operate?

78Psychology 242Introductionto Research Summary

Key terms: Main effect Additive effect Interaction Cross-over interaction Factorial design Repeated measure




Complex experiments: Within- subjects & blocking designs

Own control

Reversal designs

Repeated measures & Randomized block designs



Basic forms of within-subjects designs, 1

1. Own control Each participant in control and experimental group.

Optimally, order is counter-balanced

2. Reversal designs

3. Repeated measures & Randomized block designs

Basic forms of within subjects designs;




1. Own control

2. Reversal designs Hypothesis: behavior controlled by clearly bounded condition

Design: “A – B – A”; impose – withdraw – impose condition

3. Repeated measures & Randomized block designs

Basic forms of Within subjects designs;




1. Own control

2. Reversal designs

3. Repeated measures Multiple treatment conditions: each participant gets each

treatment. Longitudinal / time sampling: each participant assessed over

multiple time periods Randomized block designs: Repeated measure combined with

between-groups variable.




Within subjects designs; own control, 2

1. Own Control Repeated Measures Design

All participants get the Control Condition and measurement

All participants then get the experimental intervention and measurement.

Single Group

Experimental Condition

Observe2Observe1Control Condition

Hypothesis tested by differences between conditions (Observation1 v. Observation2) within group.

Internal validity: eliminate possible confound of group differences at baseline, since there is only one group.



Reversal designs

2. “REVERSAL” DESIGNS

Test again under normal state.

Test under temporary experimental condition

Test at baseline in normal state,

Examples: Role of incentives in enhancing performance Impact of anti-depressant drug on mood Effect of self-awareness on following social norms




1. Own control

2. Reversal designs

3. Repeated measures & Randomized block designs Combine a blocking variable with repeated measures. Common for:

Biomedical research Behavioral intervention evaluations




Blocking Variable; between - subjects factor

e.g., age or ethnic groups, groups based on an attitude measure… Person variables are not “true” IVs; people not randomly assigned.

Or: Experimental condition; drug dose, treatment, etc.

A “true” IV with random assignment

Repeated measure: within-subjects factor

Multiple treatment conditions: Each participant is observed after each treatment condition E.g., high v. low incentives, different instructional sets…

Longitudinal / time sampling: Measure D.V. over multiple time periods (Cohort studies).

Here both the blocking variable and the repeated measures are considered IVs.

Or:

Groups may be formed around a “Person” variable;

Randomized block design



Baseline assessment prior to intervention or experimental condition.

Within subjects designs; own control, 3

Group 1 Control Condition

Group 2

Baseline Measure

Baseline Measure

Experimental Condition

Repeated measures / randomized block design

AssignmentRandomly or via natural “blocks”

Treatment vs. Placebo. Primary Independent Variable.

Measure2 M3 M4..

Measure2 M3 M4..

Follow-up. Repeated Measures assessment of the Dependent Variable.

Time is a 2nd Independent Variable.



There are two Independent Variables:

Experimental treatment(e.g., drug dose v. placebo)

Time(Repeated measures of the outcome variable)

Each IV may have a main effect on the outcome

If both IVs have main effects the two together would have an additive effect on the outcome

The core hypothesis would be supported by an interaction effect of treatment group by time.



Mea

n sy

stol

ic B

lood

Pre

ssur

e

Blocking variable

Base-line

1 2 3 4 5 640

60

80

100

120

140

160

180

200

Placebo

Treatment

Month of study visit

Effect of drug treatment on systolic blood pressure:

The treatment group has overall lower Bp, independent of time.

Main effect example

M = 160

M = 106

Imagine we are testing a new Statin drug for high blood pressure.

The study hypothesis is that drug treatment will help lower Bp, with stronger effects over time.

Here are some (made up) randomized block, repeated measures data.

Main Effect.This shows a

These data do not support the hypothesis that drug treatment helps lower Bp:

The treatment group was lower at baseline (before treatment), and stayed lower over time.

These data would suggest a problem with the randomization: the groups were not equivalent at baseline.



Mea

n sy

stol

ic B

lood

Pre

ssur

e

Blocking variable

Base-line

1 2 3 4 5 640

60

80

100

120

140

160

180

200

Placebo

Treatment


Effect of drug treatment on systolic blood pressure:

Both the treatment and control groups show lower Bp over time.

Main effect example

M = 147

M = 105

Main Effect.This also shows a

These data also do not support the hypothesis:

Both groups got better over time.

Drug vs. placebo treatment made no difference.



Mea

n sy

stol

ic B

lood

Pre

ssur

e

Blocking variable

Base-line

1 2 3 4 5 640

60

80

100

120

140

160

180

200

Placebo

Treatment


Drug treatment & systolic blood pressure:

Both groups get better over time,

and the treatment group has overall lower Bp.

This ‘adds’ to a strong effect of treatment at the later study visits.

Additive effect example

Additive Effect.Here is an example of an

These data also do not support the hypothesis:

Both groups did get better, and the additive effect of group & time yielded the best outcome.

However, the treatment group was lower at baseline, prior to treatment.

These data suggest that people just get better over time, plus a problem with the randomization.



Mea

n sy

stol

ic B

lood

Pre

ssur

e

Blocking variable

Base-line

1 2 3 4 5 660

80

100

120

140

160

180

200

Placebo

Treatment


Drug treatment & systolic blood pressure:

The treatment group gets better over time.

The control group stays stable.

Interaction effect example

Interaction Effect.Here is an

The core hypothesis the this study is supported by this interaction effect.

The groups are equivalent at baseline.

The treatment group shows an effect over time, the control group does not.



Summary

Within – subjects designs are somewhat common in psychological research;

Own control designs create a strong contrast for the Independent Variable.

Since everyone gets all treatments, they eliminate problems in creating experimental v. control groups.

Very common in biomedical or public health studies; Most clinical studies are longitudinal; participants are

followed over time

The intervention or experimental treatment is I.V. #1 (blocking or grouping variable).

Stability or change over time is I.V. # 2 (repeated measure).

1 Psychology 242 Introduction to Research Course Overview Module This module is best used as a...

Documents

Transcript of 1 Psychology 242 Introduction to Research Course Overview Module This module is best used as a...