IA Research IA Research Method & DesignMethod & Design

Year 2 IB Psych OnlyYear 2 IB Psych Only

Methodology• The research method used.– Does more than outline the researchers’ methods • “We conducted a survey of 50 people over a two-week period

and subjected the results to statistical analysis”

The Scientific Method • 1. Define a research problem• 2. Propose a hypothesis and make predictions

• Hypothesis: A testable prediction• Must have operational definitions (a statement of the procedures used to

define the variables)• Ex: human intelligence is defined as what an intelligence test measures. (teacher

without a rubric?)• Must be replicable. (repeatable)

• 3. Design and conduct a research study• 4. Analyze the data• 5. Communicate the results, build new theories (modify and try

again)

Correlational Research• Detects relationships

between 2 variables (X & Y, dogs & cats)

• Does NOT say that one variable causes another.

• # of books read= $$ salary

Correlational Research• perfect positive correlation perfect positive correlation (+1.00)

high scores on 1 set are associated with high scores on another set – (ex: between children’s age and height)

• perfect negative correlation perfect negative correlation (-1.00) high negative correlation – (dancing accidents and amount of alcohol

drunk)

45 degree angle +1

45 degree angle -1

X (IV) horizontal line

How to Read a Correlation Coefficient

CorrelationsCorrelations• Independent variable on X axisX axis• Dependant variable on Y axisY axis

X axisX axis

Y axisY axis Y axisY axis

ScatterplotScatterplot• A graphed cluster of dots; each represents the

value of 2 variables– The slope of the points indicates the relationship– The amount of scatter suggests the strength of

correlation (high or low)

High Low

No Correlation No Correlation (scatterplot) correlation of 0.00 shows the 2 sets are not related

Illusory CorrelationIllusory Correlation• Sometime we see

relationships which do not exist– We believe there is relationship

and so we recall instances which confirm our belief• Ex: Length of marriage relates to

male baldness?

Mr. Pointy always gives me a better math

score!

Experimental Research• The researcher manipulates 1

or more factors – Explore cause and effect

relationships.– Observe the effect on some

behavior or mental process– Controlled ObservationControlled Observation

• You control & manipulate the environment and the variables– Mozart causes depression – Bananas cause constipation

RESEARCH DESIGN

Research DesignResearch Design• Skepticism: Skepticism: • A researcher needs to be skeptical (doubt until

proven)• Let the data speak for itself

Research DesignResearch Design• OverconfidenceOverconfidence: confirmation bias (if you are not

skeptical) “Of course it will be X” • We tend to think we know more than we do.

– 82% of U.S. drivers think they are the top 30% in safety

– 81% of new business owners believe their business will succeed. Their peers? Only 39%. (Now that’s overconfidence!!!)

Research DesignResearch Design• Hindsight bias: Hindsight bias: “I Knew It All Along”• The tendency to believe, after learning the

outcome, that you knew it all along.– Looking backwards– Solving a puzzle, once it’s done – “Oh, that was obvious.”– “Of course, ANY dummy could see Sept. 11,

2001 would happen.” Not Sept 10th!

Research DesignResearch Design• ReplicationReplication– You are able to repeat the experiment– You will get similar results no matter how often you

repeat• Operational Definition Operational Definition – Procedures used in defining research variables– Narrow down the focus of the study– Study Ritalin, all kids? ADHD? Boys? WS? All

schools? 1 school?

Research DesignResearch Design• Hypothesis:Hypothesis: what problem or situation do you want

to solve, test, or discover?

• Example: I want to see the results of Ritalin on young boys (age 13-16) who have ADHD, and live in Winston Salem, NC.

Research DesignResearch Design• Population Population - large group– The subjects or people to be studied

• Sample: Sample: draw from your Population– Age, gender, geographic location– Random from population– Random selection and assignment

• RepresentativenessRepresentativeness– Sample accurately

reflects the population

Random Sampling

Research DesignResearch Design• Representativeness:Representativeness:• What percentage do you need?• Depends on the populationpopulation in the study.– 100 out of 500= 20% (larger pop, lower %)– 10-15 out of a class of 25= 40-50% (smaller pop,

larger %)– Netherlands studies 800,000, only need 4-5% (40,000

people in study)

Research DesignResearch Design

• Random Assignment to GroupsRandom Assignment to Groups– Do not categorize based on gender, age, size, GPA,

etc.

• Control Group: Control Group: No change (not exposed to IV)• Experimental Group: Experimental Group: Change 1 variable (only 1)

Research DesignResearch Design• Independent Variable (IV)Independent Variable (IV)– What is being introduced, what is

new, what are you changing?– What is being manipulated?

• Dependent Variable (DV)Dependent Variable (DV)– What is being measured?– The change caused (or not caused)

by the indep. variable.

Confounding VariablesConfounding Variables• Any extraneous variables that could cause data

contamination• False consensus effect False consensus effect - we tend to overestimate

the extent others share our beliefs and behaviors.

• Reactivity:Reactivity: When a subject’s behavior is changed because s/he knows that s/he is being observed

Confounding VariablesConfounding Variables• Demand CharacteristicsDemand Characteristics: When a subject behaves

in the way that s/he thinks the experimenter wants, rather than in a natural fashion

• Experimenter BiasExperimenter Bias: Certain behaviors from the researcher bias the subject’s behavior (*)

• Experimental condition Experimental condition - actual setting, paint color, music, noise, time or day, what the subject ate for breakfast, the weather, the season of year

Research DesignResearch Design• Prevention of ContaminationPrevention of Contamination– How to stop those Confounding Variables– Single BlindSingle Blind: subject is unaware (Persistent/Stubborn)– Double BlindDouble Blind: Assistant & Subject Unaware

Research DesignResearch Design• Statistics & DataStatistics & Data• T-test, CHI-square, Z-score• Psychometrics

• Statistical SignificanceStatistical Significance– “I want to prove that my independent variable causes

my dependent variable 95% 95% of the time”– 95% to be valid– Probability= P<.05(5%) P<.05(5%) chance, random, chaos theory

STATISTICAL METHODOLOGYT-Test, Chi-Squared, Mean, Median, Mode

2 Types of Statistics2 Types of Statistics• 2 types of analysis techniques:• 1. Descriptive statisticsDescriptive statistics: techniques

that help summarize large amounts of info. Include measures of variability and measures of correlation (Describe the data)

• PopulationPopulation, Bag of M&MsBag of M&Ms• 2. Inferential statisticsInferential statistics: techniques

that help researchers make generalizations about a finding, based on a limited number of subjects

• SampleSample, Handful of M&MsHandful of M&Ms

Descriptive Statistics Descriptive Statistics

– Frequency distribution Frequency distribution - organizational technique that shows the number of times each score occurs, so that the scores can be interpreted• Graph depictions– frequency polygon frequency polygon - curve– frequency histogram frequency histogram - bars

Descriptive Statistics Descriptive Statistics – Central Tendency Central Tendency - a number that represents the

entire group or sample – Tend to hover towards the center• Average IQ score, around 100• 2 genius parents tend to have average IQ child• Politicians (Dem or Rep) dance in the center for max. votes• Weight distribution

Descriptive Statistics Descriptive Statistics – The Bell CurveThe Bell Curve–Grades, IQ, Poverty – Link between intelligence and salary• When did a C become an F?• Is a C acceptable? C=average• Does everyone get a trophy, ribbon?• Can everyone get an A?

Descriptive Statistics Descriptive Statistics • meanmean - the arithmetic average • medianmedian - middle score when arranged lowest to highest• modemode - the most frequent score in a distribution–unimodalunimodal - one high point–bimodalbimodal - two high points

Set: 2, 2, 3, 5, 8 Median: 3 Mode: 2Mean: Add up (20), divide by 5= 4

Descriptive Statistics Descriptive Statistics –bimodalbimodal - two high points–The more overlap in the bimodal arches, the higher

the variable link between the data–The less overlap, the lower the connection

Descriptive MeasuresDescriptive Measures• 2 ways we measure:• 1. RangeRange: Highest score minus the lowest score--

tells how far apart the scores are – simplest measures of variability to calculate.• (weakness of range: it can easily be influenced by one extreme

score, Savant IQ of 220)

• Set: 2, 2, 3, 5, 8• Range: 8 - 2 = 66

Ex: Age Range 15-17, Difference 2 7-17, Difference 10

Child prodigies, Dougie Houser, Chess, sci, art, music

Descriptive MeasuresDescriptive Measures• The other way to measure is:• 2. Standard DeviationStandard Deviation: measure of variability that describes

how scores are distributed around the mean. – (1 SD, 2 SD, -1, -2)

– Central Tendency: tend to hover near the center.

Standard DeviationStandard Deviation

34% 34%13.5%13.5%2% 2%

68%

95%

99%

1% outliersSavant, 2201 in 30 million

Standard DeviationStandard DeviationTo calculate standard deviation (SD):• 1. find the meanmean of the distribution 44• 2. subtractsubtract each score from the meanmean 4-2, 4-2, 4-3, 4-5, 4-84-2, 4-2, 4-3, 4-5, 4-8• 3. squaresquare each result – “deviations” 4-2=24-2=2 2 2

squared=4squared=4• 4. addadd the squared deviations 4 + 4 + 1 + 1 + 16 = 264 + 4 + 1 + 1 + 16 = 26• 5. dividedivide by the total numbernumber (n - 1) of scores; this result is called

the variancevariance 26 / 4 (5 – 1) = 6.5 (V)26 / 4 (5 – 1) = 6.5 (V)• 6. find the square root square root of the variancevariance; this is the standard standard

deviation (SD)deviation (SD) 2.55 (SD)2.55 (SD)• 7. n = biased sample n = biased sample – does not accurately represent population

being tested (out of the norm, get rid of out-liers) 55• 8. (n - 1) (n - 1) = unbiased sampleunbiased sample 44• 9. now you can compare distributions with different means and

standard deviations ((ex: 3 different class scores, 78, 80, 92ex: 3 different class scores, 78, 80, 92))

Set: 2, 2, 3, 5, 8

Sigma Sigma Σ• ΣΣ the symbol for standard deviation (SD) is s. – Greek letter “sigma” (lower case form)

• SS upper case letter (other Greek “sigma”) – Standard meaning in mathematics, “add up a list

of numbers.” – Represents Sum, i.e. add together

Z-ScoreZ-ScoreZ-scoresZ-scores: a way of expressing a score’s distance from the

mean in terms of the standard deviation (SD)• to find a Z-score for a number in a distribution, subtractsubtract

the meanmean from that number, and dividedivide the result by the standard deviation standard deviation 8 – 4 (M)= 4 / 2.55 (SD) = 8 – 4 (M)= 4 / 2.55 (SD) = 1.561.56• a positivepositive Z-score shows that the number is higher than

the mean (You’re OK, IQ, health average or higher)• a negativenegative Z-score allows psychologists to compare

distributions with different means and standard deviations (Below average, health, psych concerns)• Sometimes Z-scores are necessary to explain standard

deviation in an experiment’s results/discussionPOS ZPOS ZNEG ZNEG Z

Skewed ResultsSkewed Results• When there are more scores at

the highhigh or lowlow end of a distribution it is said to be skewedskewed–tail signifies the extreme score–Single tailed Single tailed = extreme score

on either side–Which direction are the

“outliers?” –Called Right/Left Skew–Also Pos./Neg. Skew

Majority

Majority

Outliers: fringe, oddball, genius, bad egg

A Skewed Distribution

Are the results positively or negatively skewed?

Positive Skew orSkewed Right

Inferential StatisticsInferential StatisticsTests of Significance Tests of Significance - used for determining

whether the difference in scores between the experimentalexperimental and control groups control groups is really due to the effects of the independent independent variable variable or just due to random chancerandom chance

• If p < .05 p < .05 (95%) (95%) the outcome (or the difference between experimental and control groups) has a probability of occurring by random chance less than

5 x per 100– Researchers conclude the effect of the

independent variable is significant (real).

Inferential StatisticsInferential Statistics• Statistically Significant Statistically Significant – • It is concluded that the independent variable

made a real difference between the experimental group and the control group

– Ritalin really Ritalin really DOESDOES help ADHD help ADHD– Raising serotonin levels Raising serotonin levels DOESDOES

help Depression (yoga)help Depression (yoga)

Null HypothesisNull Hypothesis• Null Hypothesis:Null Hypothesis: any alternative hypothesis, if yours is

wrong!• Significance testsSignificance tests are used to acceptaccept or rejectreject the null null

hypothesishypothesis. – If the probability of observing your result is < .05 (95%)< .05 (95%)– Your theory is truetrue, reject the null hypothesisreject the null hypothesis• Meaning that your original hypothesis is possible

(without chance, random, chaos)– If the probability of observing your result is > .05> .05 accept

the null.• Meaning that your original hypothesis is not possible

(too much left to chance, random events)• You need a backup, alternative hypothesis

Null Hypothesis PracticeNull Hypothesis Practice• Accept or Reject the Null?Accept or Reject the Null?• My hypothesis: Drug X will stop sleep walking 95%.• Do the testing. Do the data. • Drug X has a probability of 63%.• Is it greater than or less than 5%5% chance? <>.05? • Do you accept the Null or reject the Null Hypothesis?• ACCEPT the NULL! My theory was wrong!• 37% chance, error, random– Maybe it’s the patients I chose? – Maybe too much caffeine before bed? – Maybe drug was contaminated in the lab?– Start over, new test, new drug, new data

Null Hypothesis PracticeNull Hypothesis Practice• Accept or Reject the Null?Accept or Reject the Null?• My hypothesis: Stress causes mice to gain weight.• Do the testing. Do the data. • The “stressed mice” gained weight 97%.• The “control group” of mice showed no weight gain.• Is it greater than or less than 5%5% chance? <>.05? • Do you accept the Null or reject the Null Hypothesis?• REJECT the NULL! My theory was right!• 3% chance, error, random– Good Job! Bonus and a raise!

Types of TestsTypes of Tests

•1. T-Test1. T-Test•2. Chi-Square Test2. Chi-Square Test•3. Mann-Whitney U3. Mann-Whitney U•4. Sign Test4. Sign Test•5. Wilcoxon Matched-Pairs Signed-5. Wilcoxon Matched-Pairs Signed-

RankRank Test Test

When to Use the When to Use the T-TestT-Test??• T-TestT-Test – when 11 variablevariable is used in 2 2

situationssituations-- Ex: Ritalin effects in either ADHD males or ADHD females-- Ex: subject has to pick out a letter in a round list or a square list

• Common situation in psychology:• Randomly assign people to an “experimental” “experimental”

group or a “control” “control” group to study the effect– In this situation, we are interested in the meanmean

difference between the 2 conditions2 conditions.– The significance test significance test used in this kind of scenario is

called a t-test.

• Used to determine whether the observedobserved mean mean differencedifference is within the range (less that.05) that would be expected if the null hypothesis were true.

When to Use the T-Test?When to Use the T-Test?

How to Use the T-Test?How to Use the T-Test?

• T-TestT-Test• 1. SubtractSubtract meanmean from each score• 2. Rank items• 3. Sum of PositivePositive Ranks• 4. Sum of NegativeNegative Ranks• 5. Smallest score = T• If tt > 1.96 > 1.96 or < - 1.96< - 1.96, then p p < .05< .05 (Test is Valid)

GIRLSBOYS74522314415364

3021

How to Use the T-Test?How to Use the T-Test?• T-TestT-Test• 1. SubtractSubtract meanmean from each score

Mean= 21 divided by 7 = 34-3, 2-3, 3-3, 4-3, 1-3, 3-3, 4-31 , -1, 0, 1, -2, 0, 1

• 2. Rank items1, 1, 1, 0, -1, -2

• 3. Sum of PositivePositive Ranks1+1+1+0=3

• 4. Sum of NegativeNegative Ranks-1 + -2 = -3

• 5. Smallest score = t (-3)• If tt > 1.96 > 1.96 or < - 1.96< - 1.96, then p p < .05< .05

BOYS4234134

21

7scores

Test is VALID

We have to redo our

hypothesis ???

That bites

Awesome Calculators!Awesome Calculators!

• www.graphpad.com/quickcalcs/index.cfm

• T-Test• Chi-Square

When do I Use Chi-Square?When do I Use Chi-Square?• A common situation in psychology is when a researcher is

interested in the relationship between 2 nominal or categorical variables.

• The significance test used in this kind of situation is called a chi-square (2).

• Ex: We are interested in whether single Ex: We are interested in whether single men men vs. vs. womenwomen are more likely to own are more likely to own catscats vs. vs. dogsdogs..

• Notice that both variables are categorical.– Kind of pet– Gender male or female.

Chai-squared

Example Data: Observed (Actual Data)Example Data: Observed (Actual Data)

• MalesMales are more likely to have dogsdogs as opposed to cats

• FemalesFemales are more likely to have catscats than dogs

Cat Dog

Male 20 30 50

Female 30 20 50

50 50 100

NHSTNHST (Null Hypothesis Significance Testing)

Question: Are these differences best accounted for by the null hypothesis?

Is there is a real relationship between gender and pet ownership?

Chi-Square Test – Chi-Square Test – when there are when there are 2 variables2 variables

– The closer your results (Experimental and Control), the harder to prove if indep. variable (IV) really worked.

– Further apart, you can see definite difference.

Example Data: Example Data: Expected DataExpected Data

• To find expected valueexpected value for a cell of the table, multiply the corresponding row total by the column total, and divide by the grand total

• For the first cell (and all other cells)

• (50 x 50)/100 = 25• Thus, if the two variables

are unrelated, we would expect to observe 25 people in each cell

Cat Dog

Male 25 25 50

Female 25 25 50

50 50 100

Example Data: Example Data: Expected vs. ObservedExpected vs. Observed

• The differences between these (E) expected values expected values (25) (25) and the (O) observedobserved valuesvalues (see boxes) (see boxes) are aggregated according to the Chi-square formula:

Cat Dog

Male 20 30 50

Female 30 20 50

50 50 100

E

EO 22

25

2520

25

2530

25

2530

25

2520 22222

4111125

25

25

25

25

25

25

252

• Once you have the chi-squarechi-square statistic, it can be evaluated against a chi-square sampling distributionchi-square sampling distribution

• The sampling distribution characterizes the range of chi-square values we might observe if the null hypothesis is trueif the null hypothesis is true, but sampling error is giving rise to deviations from the expected values.

• In our example in which the chi-square was 4.04.0, the associated p-p-value was >.05value was >.05

• Accept the Null HypothesisNull Hypothesis, need an Alternative Hypothesis, You did NOT prove your experiment

Do We Do We AcceptAccept or or RejectReject the Null? the Null?

Mann-Whitney U TestMann-Whitney U Test• Skewed results? Are they from the same

distribution?– Use to determine if there were problems with

sampling, population, contamination– Use for 2 groups (samples)– Sub. For T-Score (T-Test)– Ex: Experimental & Control

How To Use Mann-Whitney U TestHow To Use Mann-Whitney U Test– Ex: Experimental & Control– Lay out all of your scores (in both groups)– Rate them Rank 1Rank 1 (lowest) - Rank 15 Rank 15 (highest)

• Experimental GroupExperimental Group Control GroupControl Group• Time (min) Rank Time (min) Rank• 140 4 130 11• 147 6 135 2• 153 8 138 3• 160 10 144 5• 165 11 148 7• 170 13 155 9• 171 14 168 12• 193 1515

How To Use Mann-Whitney U TestHow To Use Mann-Whitney U Test• Add up the sum of both groups (+)• Experimental GroupExperimental Group Control GroupControl Group• Time (min) Rank Time (min) Rank• 140 4 130 1• 147 6 135 2• 153 8 138 3• 160 10 144 5• 165 11 N1=8 N1=8 148 7• 170 13 155 9• 171 14 168 12• 193

15________________________________• R1 =81, N1=8 R2 =39, N2=7R1 =81, N1=8 R2 =39, N2=7

N2=7N2=7

How To Use Mann-Whitney U TestHow To Use Mann-Whitney U Test• Experimental GroupExperimental Group Control GroupControl Group• R1 =81, N1=8 R1 =81, N1=8 R2 =39, N2=7R2 =39, N2=7

• Formula to find U Formula to find U (Hypothetical Data Statistics)• U=N1N2 + N1(N1+1)-R1

2• U=(8)(7) + 8(9) -81

2• U= 56 + 36 – 81

• U= 11

How To Use Mann-Whitney U TestHow To Use Mann-Whitney U Test• Experimental GroupExperimental Group Control GroupControl Group• R1 =81, N1=8 R2 =39, N2=7R1 =81, N1=8 R2 =39, N2=7 U=11U=11• Is 11 in between the N1-N2 range of #s on the chart? (6-50) YESIs 11 in between the N1-N2 range of #s on the chart? (6-50) YES• Go to the Mann-Whitney Chart (Table 1)

N1 2 3 4 5 6 7 8N22

3

4

5

6

7 6/50

How To Use Mann-Whitney U TestHow To Use Mann-Whitney U Test• Experimental GroupExperimental Group Control GroupControl Group• R1 =81, N1=8 R2 =39, N2=7R1 =81, N1=8 R2 =39, N2=7 U=11U=11• Is 11 in between the N1-N2 range of #s on the chart? (6-50) Is 11 in between the N1-N2 range of #s on the chart? (6-50)

• If YES, If YES, reject the Null Hypothesis, your data is acceptable to use• Your distribution and population is acceptable, even though a skew

has occurred, you are within the acceptable range

• If NOIf NO, accept the Null Hypothesis, your data is not acceptable• Something has contaminated your population or your data, you must

go to a Null, or Alternate Hypothesis.