Study Design Research Methods Professional Development Institute Kali Trzesniewski December 4, 2015.
-
Upload
susan-kelly -
Category
Documents
-
view
220 -
download
0
Transcript of Study Design Research Methods Professional Development Institute Kali Trzesniewski December 4, 2015.
Study Design
Research Methods Professional Development Institute
Kali TrzesniewskiDecember 4, 2015
Study
DesignData
Collection Method
Study
DesignData
AnalysisMethod
Study
Design
A Blueprint
Before you start planningNeed to know where you are going
If you were going to build a hotel
BEFOREdeciding the look and function of the hotel or making a blueprint
buy
materialsMake an action plan
So, don’t start working on
• Measures• Sample• Design (e.g., cross-sectional, experimental)• Tasks (e.g., activities)• Etc
Until you have a clear goal
What type of evidence will allow
me to reach my goal in a
convincing way?
Study Design Function
The function of a study design is to ensure that the evidence obtained
enables us to answer the initial question as unambiguously as
possible
So, when designing a study, ask:
Given my research question, what type of evidence is needed to
answer the question in a convincing way?
Keep in Mind
• Part of answering your research question in a convincing way…
• is addressing alternative, rival explanations
Types of Research Designs
• Descriptiveo Documents
frequencies or levels
• Correlationo Tests relations
between two variables
o Focused on individual differences
o May be cross-sectional or longitudinal
• Experimento Controlled
context to test causal impact
o Focused on group differences
o Requires random assignment to control for individual differences
• Quasi-Experimento Naturally
occurring experiment
o Weaker causal impact
o Focused on group differences
o Requires control group, but not random assignment
Types of Research DesignsTime
in EFNE
PAmount of
Vegetables
Consumed
Re-enroll in 4-H
Participation in County
Fair
Better
Goal Settin
gRandomly
assigned to 4-H Thrive!
Bad Okay Great!02468
1012
How was the Research Methods Institute?
Research Design Strengths and Limitations
• Descriptiveo Quick and easyo Interpretation not always clear
• Correlationo Quick and easyo Cannot inform causation
Research Design Strengths and Limitations
• Experiment o Can inform causation o Harder to doo No one wants to be the controlo Might not reflect real life
• Quasi-experimento Easier than experimento Cannot make strong causation conclusionso With statistical adjustments (e.g., propensity matching)
causation can be suggested
The design you use will determine the conclusions you
can make
Research Question:Is my program effective?
Hypothesis:Youth who participate in
my program will make better food choices
• Do youth increase on self-reported food choices pre to post?
• Do youth who have participated in my program make better food choices than youth who have not participated in the program?
• Does my program change the food choices youth make?
Operationalizing your hypothesis
• Research design: correlation• Conclusion: program resulted in better food
choices• Why people might not trust the conclusion:
change might reflect naturally-occurring growtho Need a control sample to show change isn’t
happening in a similar population of non-participants
Do youth increase on self-reported food choices pre to post?
• Research design: quasi-experiment & descriptive
• Conclusion: program resulted in better food choices
• Why people might not trust the conclusion: youth who ended up in program might have always had better food choiceso Need random assignment or statistical controls
Do youth who have participated in my program make better food choices than youth who have
not participated in the program?
• Research design: experiment• Conclusion: program resulted in better food
choices• Why people might not trust the conclusion:
people generally trust experimentso Questions usually revolve around generalizability
and scaling up
Does my program change the food choices youth make?
Choosing Which Design to Use
What do I want to learn?What will I want to report?• Participants, leaders, community members,
funders, etc.What might I want to know for program refinement?What will my take-home message be?What figures or tables will I want?
Jump to the End
What can I feasibly do?
Common Mistakes in Research
• Insufficiently motivated research questionso Is this something people other than me care
about?o Will it generate new knowledge?o Does it address a real problem?o Does it impact a sufficient portion of the
population?
Common Mistakes in Research
• Unresearchable problemso Can this question be answered through ethically
and valid research practices?o Is this feasible for me to do?
• Favored research methodso Don’t design a study around a research method
• Blind data miningo Don’t collect data then figure out what to do with
it
Go forth and research!