Research Methodology

Research Traditions

What is research?

Research is “the systematic approach to finding answers to questions.”

“Questions” comes first – questions drive inquiry; questions will inform the kind of research we do.

Quantitative and Qualitative research traditions

Both are “empirical,” involving the collection of original data (from human subjects).

Both are drive by identifiable conceptual and methodological assumptions.

Research Traditions

Quantitative research emphasizes:

•the systematic measurement and quantification of variables

•the statistical analysis of data

•the use of mathematic models and causal inferencing

Qualitative research emphasizes:

•thick description of the interactions of individuals and interpretations of these interactions

•heuristic (discovery oriented analysis of data)

•leading to “grounded theory.”

Research Traditions

Assumptions about:

1.discovery

2.variables

3.data collection

4.data analysis

5.use of data

Research Traditions

Quantitative

There is a “T”ruth that can be discovered.

Qualitative

Individuals socially construct meaning / “t”ruth, so truth is as relative as the number of individuals involved. Qualitative research is sometimes referred to as “interpretive,” for this reason.

Discovery

Research Traditions

Quantitative

Few, parsimonious – goal is to isolate a single variable so that causation can be established.

Qualitative

Variables

Research Traditions

Quantitative

Parsimonious

L. parsimonia "sparingness, frugality," from pars-, stem of parsi, perf. tense of parcere "to spare, save" + -monia, suffix signifying action or condition

Qualitative

“Thick description.” Goal is to consider all variables that might be pertinent.

Margaret Mead

Claude Lévi-Strauss

Hunter Thompson

Variables

Research Traditions

Quantitative

Deductive: Researchers bring a theory to the study setting which is then tested in that setting.

Qualitative

Inductive: Theory arises from data – hence “grounded theory” – grounded in data. Theories inform researchers’ preparation for study, but doesn’t lock them into particular ways of understanding.

Data Collection

Research Traditions

Competence data:

•error recognition / correction

•grammatical judgments

•cord sorting

Affective data:

•questionnaires

•matched-guise techniques

•diary studies

•focused introspection

Data CollectionPerformance data:•reading aloud•completion tasks•elicited imitation•reconstruction•role plays•oral interviews•composition

Research Traditions

Quantitative

Statistical. Searching for statistically significant differences among samples around specific variables, in order to understand effects of interventions.

Qualitative

Heuristic. Searching data variously (taxonomic analysis, semantic relationships [x is a kind of y, etc.]) in order to understand a specific culture.

Data Analysis

Research Traditions

Quantitative

By random sampling, generalize results from a small sample to a large population.

Qualitative

Not interested in generalizability. Results inform further study and help build theory (our interpretive understanding of a cultural setting).

Use of Data

Statistics

1.Descriptive Statistics

The use of numbers to describe results.

•mean = average scores (etc.) from all participants

•SD = standard deviation ≈ average distance from the mean of all points in a data set

Statistics

1 SD = 68 % of population (red)

2 SD = 95 % of population (red and green)

3 SD = 99 % of population (red, green, and blue)

1 SD = 68 % of population (green)

2 SD = 95 % of population (green and blue)

3 SD = 99 % of population (green, blue, and red)

Statistics

mean = 63.5 in. SD = 2.5 in. 61 in. 66 in.

58.5 in. 68.5 in.

Statistics: Height of US Women 18–74 (n =

6,588)

Statistics: Example

2.42

1.28 3.56

Statistics: English Students

2.30

1.43 3.17

Statistics: ESL Students

Statistics: Example

Statistics

2. Inferential Statistics

Analysis of descriptive statistics. How do we know if the difference between two data sets (means, frequencies, correlations, and so on) is significant – i.e., attributable to something other than chance?

A variety of tests used to determine significance:

•Anova (analysis of variance)

•chi-square

•F-test

•t-test

•Pearson product moment correlation

Statistics

Significance – means “attributable to something other than chance.”

Example: old teaching method v. new teaching method

Experimental method semester-end test scores are significantly higher than control method tests

If we have set up the study correctly, we may be able to conclude that the new method is the cause of higher student achievement.

Statistics

Probability. Significance is usually expressed in terms of probability, “p” – for example:

p < .01

p < .05

refer to the probability that differences occurred through chance alone (as opposed to the influence of the experimental method).

Statistics

p < .01 means there is less than 1 percent probability that chance alone explained our result; there is more than 99 percent probability that something other than chance explained our result.

p < .05 means there is less than 5 percent probability that chance alone explained our result; there is more than 95 percent probability that something other than chance explained our result.

p = 0.026 means that our inferential analysis has determined there to have been a 2.6 percent probability that chance alone explained our result; and a 97.4 percent probability that something other than chance explained our result.

Statistics: Example

Statistics: Example

So, while the results may be interesting …

Statistics: Example

… they are not significant – i.e., we can’t contribute them to anything other than chance.

Research Traditions

Quantitative

Correlation studies

Survey research

Experimental research

Qualitative

Case studies

Ethnographic research

Research Methods

Research Methods

Correlation Studies (Quantitative tradition)

Purpose: to understand relationships among characteristics.

Issue: Correlation is NOT causation. Examples?

Cancer

clusters

Research Methods

Correlation Studies (Quantitative tradition)

Analysis: Statistical, based on calculation of correlation coefficients.

Research Methods

Correlation Studies (Quantitative tradition)

Analysis: Statistical, based on calculation of correlation coefficients.

Research Methods

Correlation Studies (Quantitative tradition)

Example: Goldstein, L.M. (1987). Standard English: The only target for nonnative speakers of English? TESOL Quarterly, 21, 417-436.

The author examined the use of African American English (BVE) among Hispanic students. She found that those who had extensive contact with African American students used features of BVE more than students with less contact.

http://www.youtube.com/watch?v=kkcTpfcno-E

Research Methods

Survey Research (Quantitative tradition)

Purpose: to learn about the characteristics of an entire group (population) by asking questions of a small component of that group (sample).

Issue: Random sampling.

Analysis: statistical, both descriptive (percentages, etc.) and inferential (i.e., looking at the relationships or correlations in the results.

Research Methods

Survey Research (Quantitative tradition)

Example: Duran, R.P. (1987). Factors affecting development of second language literacy. In S.R. Goldman and H.T. Trueba (Eds.), Becoming literate in English as a second language (pp. 33-55). Norwood NJ: Ablex.

The author surveyed Hispanic freshmen concerning their basic language characteristics, their own rating of their academic skills, and their SAT scores. As a result of correlation analyses, the author found a positive relationship between high self-ratings and high SAT scores.

Research Methods

Experimental Research (Quantitative tradition)

Purpose: to establish a cause / effect relationship between an independent variable (IV; the cause agent) and a dependent variable (DV; the agent acted upon).

Requires experiment and control groups and careful control of all but the IV so that causation (if any) can be established.

Research Methods

Experimental Research (Quantitative tradition)

Issue: “true-ness” of experimentControl and

Experimental Groups?Random Sampling

Pre-Experiment No. Only 1 group, pre- and post-test structure

No. Use of extant group

Quasi-Experiment Yes. No. Use of extant groups

True Experiment Yes. Yes.

Research Methods

Experimental Research (Quantitative tradition)

Analysis: Inferential statistics to determine if difference between groups are significant – that is, caused by the IV, not by chance alone.

Example: Carrell, P.L. (1985). Facilitating ESL reading by teaching text structure. TESOL Quarterly, 19, 461-481.

The author found that teaching rhetorical structures to ESL freshmen resulted in improved ability to recall information in related texts.

Research Methods

Case Study Research (Qualitative tradition)

Purpose: to focus attention on a single entity (or a very small number) in a naturalistic setting.

Data is gathered through interviews, observations, diary accounts, text analysis, video / audio recording, etc.

Analysis: heuristic, piecing together patterns in the data.

Research Methods

Case Study Research (Qualitative tradition)

Example: Kravin, H. (1992). Erosion of a language in bilingual development. Journal of Multilingual and Multicultural Development, 13, 307-325.

The author analyzed the bilingual development of a six-year-old Finnish-American child. The study is primarily concerned with the processes by which one of the child’s languages begins to dominate the other. The author speculated in conclusion that the subordination of Finnish in the child is the result of limited linguistic input in that language.

Research Methods

Ethnographic Research (Qualitative tradition)

Purpose: to examine culture, society, schooling, etc. for the purpose of understanding the processes of acculturation, socialization, learning, etc. from the perspective of individuals in the setting.

Data is gathered through interviews, observations, diary accounts, text analysis, video / audio recording, and participant observation.

Analysis: heuristic, discovery oriented

Research Methods

Ethnographic Research (Qualitative tradition)

Example: Hornberger, N.H. (1987). Bilingual education success, but policy failure. Language in Society, 16, 205-226.

The author investigated bilingual education in a particular community in Peru through a two-year ethnographic study. Findings focus on the perceptions of the value of Spanish and Quechua among students and their families.

• Are there male/female differences in how invitations are extended between native speakers? How does non-native speaker behavior compare?

• Is there a sequence in which second language pronouns are acquired? If so, what is it?

• Does practice with sentence-combining result in learners producing longer T-units in their classroom compositions?

• What word-attack skills do learners naturally use when they encounter a word they don’t know?

• Is there a relationship between the age at which second language instruction begins and the level of SL proficiency achieved?