Research Methods: Design and Analysis
-
Upload
james-lani-phd -
Category
Education
-
view
371 -
download
8
description
Transcript of Research Methods: Design and Analysis
![Page 1: Research Methods: Design and Analysis](https://reader035.fdocuments.net/reader035/viewer/2022062312/555cb2f9d8b42aad358b546f/html5/thumbnails/1.jpg)
Research Methods: Design and Analysis
by Dr. James Lani
![Page 2: Research Methods: Design and Analysis](https://reader035.fdocuments.net/reader035/viewer/2022062312/555cb2f9d8b42aad358b546f/html5/thumbnails/2.jpg)
Today’s Webinar
To schedule a consultation call 877-437-8622 or email [email protected]
• Why Do Research?• The Research Cycle• Research Questions and (Testable) Hypotheses
• Operationalization of Variables• Literature Review Resources• Research Designs• Sampling Method and Sample Assignment• Instrumentation• Method of Data Collection (Procedures)• Validity
• Internal Validity• External Validity
• Reliability• Data Analysis Plan• Sample Size• Q & A
![Page 3: Research Methods: Design and Analysis](https://reader035.fdocuments.net/reader035/viewer/2022062312/555cb2f9d8b42aad358b546f/html5/thumbnails/3.jpg)
Why Do Research?
To schedule a consultation call 877-437-8622 or email [email protected]
OK, to get out of school….and also to examine questions we care about. We want to test a theory, evaluate a program or intervention, describe a phenomenon.
Does Violence on TV cause children to be more or less aggressive?Do opposites attract or birds of a feather flock together?Does treatment X predict academic achievement?Is there a difference on depression scores by group (exercise vs. not)?
What is True???Goal of research: describe, explain, and predict. To establish knowledge, you need methods and statistics.
![Page 4: Research Methods: Design and Analysis](https://reader035.fdocuments.net/reader035/viewer/2022062312/555cb2f9d8b42aad358b546f/html5/thumbnails/4.jpg)
The Research Cycle
To schedule a consultation call 877-437-8622 or email [email protected]
LiteratureReview
MethodHypotheses and
Operational Variables
MethodDesign Study and
Collect Data
ResultsAnalyze data and
accept or reject Ho’s
Discussion
![Page 5: Research Methods: Design and Analysis](https://reader035.fdocuments.net/reader035/viewer/2022062312/555cb2f9d8b42aad358b546f/html5/thumbnails/5.jpg)
Research Questions and (Testable) Hypotheses
To schedule a consultation call 877-437-8622 or email [email protected]
• Hypotheses: State relationships among variables (constructs). Typically in terms of Differences, Relationships, or Prediction.
• Keep research questions and hypotheses aligned (in number and language).
• Questions must be testable.• Keep it tentative: no proof—just support
or non-support
![Page 6: Research Methods: Design and Analysis](https://reader035.fdocuments.net/reader035/viewer/2022062312/555cb2f9d8b42aad358b546f/html5/thumbnails/6.jpg)
Operationalization of Variables
(or how do you measure what you’re studying?)
To schedule a consultation call 877-437-8622 or email [email protected]
Constructs—objects that are not directly observable• Constructs are measured with variables• Should be quantifiable • (e.g., depression can be measured with a Beck
depression scale or by the observation of the number of times someone laughs)
Confounding Variables• Provide alternative explanation of response in the DV
Levels of Measurement: Nominal, Ordinal, Interval, and Ratio
![Page 7: Research Methods: Design and Analysis](https://reader035.fdocuments.net/reader035/viewer/2022062312/555cb2f9d8b42aad358b546f/html5/thumbnails/7.jpg)
Literature ReviewResources
To schedule a consultation call 877-437-8622 or email [email protected]
See: UW—Madison—the writing centerUC Santa Cruz—UNW library—write a literature reviewWashington and Lee University—literature reviewStatistics Solutions’ Free membership—sample dissertation
![Page 8: Research Methods: Design and Analysis](https://reader035.fdocuments.net/reader035/viewer/2022062312/555cb2f9d8b42aad358b546f/html5/thumbnails/8.jpg)
Research Designs
To schedule a consultation call 877-437-8622 or email [email protected]
EXPLORATORY
EXPERIMENTALDescriptive
Longitudinal
CAUSAL Cross-sectionalCorrelational Cohort
designs
Non-experimental
Sequential
![Page 9: Research Methods: Design and Analysis](https://reader035.fdocuments.net/reader035/viewer/2022062312/555cb2f9d8b42aad358b546f/html5/thumbnails/9.jpg)
Research Designs
To schedule a consultation call 877-437-8622 or email [email protected]
Its about how you structure your inquiry
![Page 10: Research Methods: Design and Analysis](https://reader035.fdocuments.net/reader035/viewer/2022062312/555cb2f9d8b42aad358b546f/html5/thumbnails/10.jpg)
Research Designs
To schedule a consultation call 877-437-8622 or email [email protected]
Experimental vs. Non-experimental Designs
Non-Experimental DesignsCorrelational studies, observational
studies, quasi-experimental studies, and surveys
Experimental (or True) DesignsManipulate the IV(e.g., Control vs. Experimental groups)Random assignmentConfounds are controlled or eliminated
![Page 11: Research Methods: Design and Analysis](https://reader035.fdocuments.net/reader035/viewer/2022062312/555cb2f9d8b42aad358b546f/html5/thumbnails/11.jpg)
Research Designs
To schedule a consultation call 877-437-8622 or email [email protected]
Experimental Designs
One shot case studies X O
One group pretest posttest O1 X O2
Statistic group X O1 O2 R=random assignment, O=observation, X=treatment group
![Page 12: Research Methods: Design and Analysis](https://reader035.fdocuments.net/reader035/viewer/2022062312/555cb2f9d8b42aad358b546f/html5/thumbnails/12.jpg)
Research Designs
To schedule a consultation call 877-437-8622 or email [email protected]
True Experimental DesignsPretest-posttest control group design
R O1 X O2
R O3 -- O4
R=random assignment, O=observation, X=treatment groupGroup Time 1 Time 2 Time3
Random Treatment Mean X Mean
Assignment Control Mean Mean
No diff Sig diff
Pretest Posttest0
5
10
ControlTreatment
![Page 13: Research Methods: Design and Analysis](https://reader035.fdocuments.net/reader035/viewer/2022062312/555cb2f9d8b42aad358b546f/html5/thumbnails/13.jpg)
Research Designs
To schedule a consultation call 877-437-8622 or email [email protected]
For great research design references, see:
Cook and Campbell ‘79 Campbell and Stanley ‘63 Creswell ‘05 Leedy and Ormrod ‘10 SocialResearchMethods.net
![Page 14: Research Methods: Design and Analysis](https://reader035.fdocuments.net/reader035/viewer/2022062312/555cb2f9d8b42aad358b546f/html5/thumbnails/14.jpg)
A word about Sampling Method and Sample Assignment
To schedule a consultation call 877-437-8622 or email [email protected]
Random sample: each individual in the population has an equal chance to become a participant of the sample.
Random assignment: each participant of the sample has an equal chance of being assigned to a group (treatment or control).
![Page 15: Research Methods: Design and Analysis](https://reader035.fdocuments.net/reader035/viewer/2022062312/555cb2f9d8b42aad358b546f/html5/thumbnails/15.jpg)
Instrumentation
To schedule a consultation call 877-437-8622 or email [email protected]
These are the tools that measure the outcomes—typically variables in the tools are categorized into independent variables (IV’s) and dependent variables (DV’s).
IV’s (or treatment variables) are typically manipulated by researcher.
DV’s (also called outcome variables or criterion variables) are responses to IV, caused by IV, or predicted by IV.
The variables should be reliable and valid.
Don’t make your own.
See Google scholar, review articles, Statistics Solutions directory.
![Page 16: Research Methods: Design and Analysis](https://reader035.fdocuments.net/reader035/viewer/2022062312/555cb2f9d8b42aad358b546f/html5/thumbnails/16.jpg)
Instrumentation
To schedule a consultation call 877-437-8622 or email [email protected]
Questionnaires Interviews Observations Rating scales Achievement tests Personality
inventories Historical reports or
data
![Page 17: Research Methods: Design and Analysis](https://reader035.fdocuments.net/reader035/viewer/2022062312/555cb2f9d8b42aad358b546f/html5/thumbnails/17.jpg)
Method of Data Collection (Procedures)
To schedule a consultation call 877-437-8622 or email [email protected]
After describing the sample and the instruments, tell the reader how you are going to systematically and objectively administer these instruments to these participants.
ParticipantsMeasures
![Page 18: Research Methods: Design and Analysis](https://reader035.fdocuments.net/reader035/viewer/2022062312/555cb2f9d8b42aad358b546f/html5/thumbnails/18.jpg)
Validity
To schedule a consultation call 877-437-8622 or email [email protected]
You measure what you say you are measuring.
(Validity is more important than reliability)
![Page 19: Research Methods: Design and Analysis](https://reader035.fdocuments.net/reader035/viewer/2022062312/555cb2f9d8b42aad358b546f/html5/thumbnails/19.jpg)
Internal Validity
To schedule a consultation call 877-437-8622 or email [email protected]
Eliminate confounds or competing explanations of differences on DV (E.g., Differences on Self-efficacy by gender…but the woman were 35 y.o. and men were 75 y.o.)
Threats to internal validity• Selection bias—you think they represent the population (Safeguard tip: randomly sample and randomly assign)• Pre-post studies without a control group
o History/maturation—something else occurred between the assessments
• Drop-out/attrition • Reliability of measure• Low power• Order effect• Multiple tests of significance • John Henry Effect
![Page 20: Research Methods: Design and Analysis](https://reader035.fdocuments.net/reader035/viewer/2022062312/555cb2f9d8b42aad358b546f/html5/thumbnails/20.jpg)
External Validity
To schedule a consultation call 877-437-8622 or email [email protected]
Def: The ability to generalize outcomes to the population
Threats• Population validity—ability to generalize to
population. (E.g., Kohlberg moral development—males value individual rights, where Gilligan found that females valued relationships).
• Ecological validity—ability to generalize from lab to everyday environments.
![Page 21: Research Methods: Design and Analysis](https://reader035.fdocuments.net/reader035/viewer/2022062312/555cb2f9d8b42aad358b546f/html5/thumbnails/21.jpg)
Reliability
To schedule a consultation call 877-437-8622 or email [email protected]
Inter-rater reliability (kappa coefficient)
Test-retest reliability Internal consistency—
Cronbach’s alpha
![Page 22: Research Methods: Design and Analysis](https://reader035.fdocuments.net/reader035/viewer/2022062312/555cb2f9d8b42aad358b546f/html5/thumbnails/22.jpg)
Data Analysis Plan(what to include)
To schedule a consultation call 877-437-8622 or email [email protected]
Reliability of scales: Cronbach’s alpha
Descriptive statistics: Means, standard deviations, frequency and percentages.
A data analysis plan for EACH research hypothesis.
• Inferential: ANOVA, MANOVA, regression, time-series, SEM, Chi-square, correlation
• Non-inferential: descriptives (e.g., M, SD, %)
Assumptions of analyses and justification for analyses
![Page 23: Research Methods: Design and Analysis](https://reader035.fdocuments.net/reader035/viewer/2022062312/555cb2f9d8b42aad358b546f/html5/thumbnails/23.jpg)
Sample Size
To schedule a consultation call 877-437-8622 or email [email protected]
N = ?
1. Effect Size—importance of differences
2. Power—likelihood of finding differences (typically .80)
3. Alpha (typically .05)
![Page 25: Research Methods: Design and Analysis](https://reader035.fdocuments.net/reader035/viewer/2022062312/555cb2f9d8b42aad358b546f/html5/thumbnails/25.jpg)
We’re interested… Webinar Feedback and
What You’d LikePlease provide feedback on the webinar and tell us what
else you’d like to learn about in upcoming webinars.
Send comments to: [email protected]
![Page 26: Research Methods: Design and Analysis](https://reader035.fdocuments.net/reader035/viewer/2022062312/555cb2f9d8b42aad358b546f/html5/thumbnails/26.jpg)
Thank You for your Participation
and Attention!
Join us for our next webinar on Wednesday, November 20th at
8:30pm ETResults Chapter:
Conducting, Interpreting, and Writing