Its All Too Much

Conducting Psychological ResearchSlides Prepared by Alison L. OMalleyPasser Chapter 2

1

What is good science?

Jot down 3 characteristics2Origins of Research QuestionsPersonal experience and daily events Prior research and theoryReal-world problemsSerendipityGenerate an example associated with each source.

3Conducting a Literature Search

Where to begin?4Conducting a Literature Search Online databases: PsycInfo, Google Scholar Boolean operators: AND, NOT, OR to narrow results ***Peer-reviewed articles***Full text access? If not, try authors personal websites or interlibrary loan (allow plenty of time!) 5Conducting a Literature Search Research Question: Are pet owners happier than non pet owners?

Whats the optimal Whats the optimal Way way to enter this question into a search database?In search database?

Manuscript componentBrief descriptionAbstractShort summary of studyIntroductionBackground and rationale for hypothesesMethodParticipants, procedure, materials/measuresResultsData analysis statistical tests reveal support or lack thereof for hypotheses DiscussionNon-statistical review of findings, implications, limitations, avenues for future researchReferencesList of all in-text citations formatted in APA styleMaking Sense of What You Find 7Manuscript componentBrief descriptionAbstractShort summary of studyIntroductionBackground and rationale for hypothesesMethodParticipants, procedure, materials/measuresResultsData analysis statistical tests reveal support or lack thereof for hypotheses DiscussionNon-statistical review of findings, implications, limitations, avenues for future researchReferencesList of all in-text citations formatted in APA styleMaking Sense of What You Find 8Manuscript componentBrief descriptionAbstractShort summary of studyIntroductionBackground and rationale for hypothesesMethodParticipants, procedure, materials/measuresResultsData analysis statistical tests reveal support or lack thereof for hypotheses DiscussionNon-statistical review of findings, implications, limitations, avenues for future researchReferencesList of all in-text citations formatted in APA styleMaking Sense of What You Find 9Manuscript componentBrief descriptionAbstractShort summary of studyIntroductionBackground and rationale for hypothesesMethodParticipants, procedure, materials/measuresResultsData analysis statistical tests reveal support or lack thereof for hypotheses DiscussionNon-statistical review of findings, implications, limitations, avenues for future researchReferencesList of all in-text citations formatted in APA styleMaking Sense of What You Find 10Manuscript componentBrief descriptionAbstractShort summary of studyIntroductionBackground and rationale for hypothesesMethodParticipants, procedure, materials/measuresResultsData analysis statistical tests reveal support or lack thereof for hypotheses DiscussionNon-statistical review of findings, implications, limitations, avenues for future researchReferencesList of all in-text citations formatted in APA styleMaking Sense of What You Find 11Manuscript componentBrief descriptionAbstractShort summary of studyIntroductionBackground and rationale for hypothesesMethodParticipants, procedure, materials/measuresResultsData analysis statistical tests reveal support or lack thereof for hypotheses DiscussionNon-statistical review of findings, implications, limitations, avenues for future researchReferencesList of all in-text citations formatted in APA styleMaking Sense of What You Find 12Manuscript componentBrief descriptionAbstractShort summary of studyIntroductionBackground and rationale for hypothesesMethodParticipants, procedure, materials/measuresResultsData analysis statistical tests reveal support or lack thereof for hypotheses DiscussionNon-statistical review of findings, implications, limitations, avenues for future researchReferencesList of all in-text citations formatted in APA styleMaking Sense of What You Find Note. Review papers (e.g., Annual Review of Psychology) will deviate from this format13Forming a Hypothesis Inductive: Specific facts general conclusionData driven; bottom up E.g., medical diagnosis based on symptoms

Deductive: General principle specific conclusion Theory driven; top down E.g., All people have ___. Pat is a person. Therefore, Pat has ___.

Is one logical approach better than the other? REMEMBER: Above all else, hypotheses must be TESTABLE!!

14Research Approaches: Key Distinctions Describe the characteristics of a recent happy episode in your life.

How happy are you?

Qualitative vs. Quantitative

12 3 4 5Research Approaches: Key Distinctions Research Scenario 1: Employees randomly assigned to receive cookies or not receive cookies while completing a job satisfaction questionnaire (Brief, Butcher, & Roberson, 1995)

Research Scenario 2: Employees complete a questionnaire containing questions about mood and job satisfaction Experimental vs. DescriptiveResearch Approaches: Key Distinctions Research Scenario 1: Employees randomly assigned to receive or not receive cookies while completing a job satisfaction questionnaire (Brief, Butcher, & Roberson, 1995)

Research Scenario 2: Employees complete a questionnaire containing questions about mood and job satisfaction Experimental vs. DescriptiveWhat can we conclude on the basis of each research scenario? Why? Research Design: Mind Your VariablesIndependent variable: Systematically manipulated by the researcher in experimental research

Dependent variable: Outcome of interest; what we design research to assess/measure

Research Design: Mind Your Variables

Identify the IV(s) and DV(s) in this scenario:Employees randomly assigned to receive cookies or not receive cookies while completing a job satisfaction questionnaireMastering IVs and DVsGenerate and describe a good strategy for distinguishing independent variables from dependent variables in research scenarios. Research Approaches: Key Distinctions Did employees complete the job satisfaction questionnaire under the same conditions (i.e., in identical environments), or did they take the questionnaire online at a time and place of their choosing?

Laboratory vs. FieldLab settings = CONTROLResearch Approaches: Key Distinctions Field experiments still entail manipulation of an IV, but occur in a natural setting as opposed to a lab setting.

Researchers often mention the tradeoff between internal and external validity. What exactly does this mean, and why does such a tradeoff occur?

Laboratory vs. FieldResearch Approaches: Key Distinctions Cross-sectional vs. Longitudinal 20 year olds 40 year olds 60 year olds If all three age groups are measured and compared in summer 2013, the designis cross-sectional.

Research Approaches: Key Distinctions Cross-sectional vs. Longitudinal 20 year olds 40 year olds 60 year olds If all three age groups are measured and compared in summer 2013, the designis cross-sectional.

Beware of cohort effectsdifferent age groups have different histories. Are observeddifferences due to age differences or the groups different historical experiences? Research Approaches: Key Distinctions Cross-sectional vs. Longitudinal 20 years old Summer 2013 40 years old Summer 2033 60 years old Summer 2053If a group of participants is measured repeatedly over time, the design is longitudinal.

Research Approaches: Key Distinctions Cross-sectional vs. Longitudinal 20 years old Summer 2013 40 years old Summer 2033 60 years old Summer 2053If a group of participants is measured repeatedly over time, the design is longitudinal.

Sequential research designs examine several age cohorts longitudinally.

Research Approaches: Key Distinctions Cross-sectional vs. Longitudinal 20 years old Summer 2013 40 years old Summer 2033 60 years old Summer 2053 What are the advantages and disadvantages of longitudinal and sequentialresearch designs?

Research Design: Mind Your VariablesInternal validity is compromised by the presence of confounds, a particularly pesky type of extraneous variable.

Research Design: Mind Your VariablesExample: Do participants prefer stimuli associated with the first letter of the English alphabet?

If random assignment is used such that half the participants see the object on the left and half see the object on the right, whats the problem?

ABThe Role of Sampling What is a population? The entire group of scores that a researcher desires tolearn about (e.g., all U.S. college students)What is a sample? A subset of scores from the population (e.g., 1,000 college students from a variety of colleges)Population vs. Sample Analyzing Data and Drawing Conclusions

Quantitative and qualitative analysis Descriptive Statistics Measures of central tendency address the typicality of a score:

Mode: most frequent scoreMedian: middle score (of an ordered set) Mean: mathematical center of distribution

Organize and summarize a set of data Descriptive StatisticsBuild a dataset comprised of how many siblings each of your classmates has.

Establish the mode, median, and mean for this dataset.

Central TendencyDescriptive Statistics: Central Tendency

Is it more appropriate to report the mean or the median for men and women in this dataset? Why?

MD = medianSP = sexual partners Apologies for the fuzzy image 34Descriptive Statistics: Measures of Dispersion Measures of dispersion address the spread (i.e., the variability) of a set of scores.

Organize and summarize a set of data

Sketch the distribution associated with each of the three parties. Descriptive Statistics: Measures of Dispersion Measures of dispersion address the spread (i.e., the variability) of a set of scores.

Organize and summarize a set of data Range: distance between highest and lowest score Variance: spread of scores relative to mean Standard deviation: square root of varianceInferential Statistics An oft heard question is whether research findings are statistically significant. Are our findings merely due to random errorto chance?

Inferential statistics reveal the probability that our findings are due to chance.

We use sample data to infer the nature of the population

Inferential Statistics Psychological scientists traditionally maintain that findings are statistically significant if the probability is less than 5% that the results are due to random error. We use sample data to infer the nature of the population

p < .05 = Inferential Statistics: Drawing Conclusions Statistically significant findings mean that weve proven how the world works, right?

We use sample data to infer the nature of the population

Inferential Statistics: Drawing Conclusions Statistically significant findings mean that weve proven how the world works, right?

WRONG.

We use sample data to infer the nature of the population

Inferential Statistics: Drawing Conclusions Our results may not be practically importantor perhaps there were confounding variables at play.Good research design is critical! And even with solid research design, maybe our conclusion is downright wrong. We use sample data to infer the nature of the population

Drawing ConclusionsTwo errors: False alarms and missed opportunities An innocent person is found guilty False alarm (Type I error)

In research terms, we mistakenly conclude that two variables are associatedwhen they really have nothing to do with each other.

Drawing ConclusionsTwo errors: False alarms and missed opportunities A guilty person is found innocent Missed opportunity (Type II error)

In research terms, we mistakenly conclude that two variables are not associatedwhen they really are related.

Drawing ConclusionsTwo errors: False alarms and missed opportunities Apply the false alarm and missed opportunity scenarios to the cookie experiment (Brief et al., 1995).

How to Tell Your Research StorySo we all speak the same language! Run, dont walk, to access the 6th edition of the APA publication manual!

http://www.apastyle.org/Helpful for students to see this is the most recent version 45Building Knowledge and Theories Contemplate the distinction between a theory and a hypothesis

Now, why does theory building matter?

What Makes a Good Theory? Testability and specificity Does theory lend itself to testable hypotheses and specific predictions? Internal consistency and clarity Does theory avoid contradictory predictions? Can it be falsified? Is it clear to experts how components of the theory relate to each other?

47What Makes a Good Theory? Empirical supportCan theory be reconciled with current knowledge base? If not, can it debunk current fact? Does high quality research support new hypotheses derived from theory? ParsimonyLaw of parsimony: Explanations should use the minimum number of principles necessary to account for the maximum number of facts.

48What Makes a Good Theory? Last, but not least: Does the theory have an impact on the field? Proof and Disproof Science values lively debate. There is no tolerance for the notion of absolute proof. Its always possible that our results are due to chance. Similarly, a single set of results cannot disprove a hypothesis derived from a theory.

Science is forward-moving, and theories are strengthened or weakened as supportive or unsupportive findings continually emerge.

Research is more probabilistic than absolute (Baumeister, 2008)