Mixed-methods data analysis

Graduate Seminar in English Language Studies

Suranaree, March 2011Richard Watson ToddKMUTT

http://arts.kmutt.ac.th/crs/research/mmda.ppt

Overview

Pure quantitative research Pure qualitative research Mixed-methods research

Collecting both QUANT and QUAL data using different instruments

Mixed-methods data analysis Usually only QUAL data collected Data is treated both quantitatively and

qualitatively

Quantitative or qualitative?

QUANT – QUAL distinction in applied linguistics research

QUANT: data is numbers; uses statistics Experimental research; surveys

QUAL: data is words; uses thematic or narrative interpretation Conversation analysis; ethnography

Mixed-methods research

“A mixed methods study involves the collection or analysis of both quantitative and qualitative data in a single study with some attempts to integrate the two approaches at one or more stages of the research process” (Dörnyei, 2007)

Purposes: Achieve a fuller understanding Triangulate findings

Examples of mixed-methods research Poor example

Research into attitudes: survey a large number and interview a predetermined small number of subjects

Purpose: unclear Similar, slightly better example

Research into attitudes: survey a large number of subjects, then, selecting based on questionnaire responses, interview a small number

Purpose: follow-up on interesting results

Examples of mixed-methods research An example of the opposite

Interview a small number to gain insights to design a questionnaire, then survey a large number

Purpose: informing instrument design Another similar example

Research into beliefs: interview 4 teachers but survey 80 students

Purpose: accounting for practicality in using instruments

Examples of mixed-methods research An example focusing on triangulation

Research into strategies: comparing results from different instruments

Much strategy research involves the use of SILL SILL asks respondents to identify how often they

use a particular strategy Strategy use is context-dependent Research question: Will recent context of learning

change responses to SILL?

Examples of mixed-methods research Method

Single subject Time 1: read academic articles Time 2: read short stories for pleasure Responded to SILL twice Interviewed 4 times (background interview, after

SILL responses, summary interview)

Examples of mixed-methods research SILL responses

Showed major differences between 2 times e.g. “If I guess the meaning of a word, later I will check whether my guess is correct by using a dictionary.” rated Always at Time 1; Never at Time 2

Interview responses Showed that recent learning contexts influenced different

ratings Triangulation to confirm results or triangulation to

provide different perspectives

Mixed-methods data analysis

“The most common perception of mixed methods research is that it is a modular process in which qualitative and quantitative components are carried out either concurrently or sequentially. Although this perception is by and large true, it also suggests that the analysis of the data should proceed independently for the QUANT and QUAL phases and mixing should occur only at the final interpretation stage. This conclusion is only partially true … we can also start integrating the data at the analysis stage, resulting in what can be called mixed methods data analysis”

Dörnyei (2007)

Mixed-methods data analysis (MMDA) From Dörnyei, MMDA means

Quantitising qualitative data Qualitising quantitative data

Quantitising qualitative data

Quantitising is often done unconsciously Conducting a keyword analysis Use of IELTS scores in research

Quantitising helps a qualitative analysis by allowing a reliability check

Quantitising can be used to count and compare frequency of themes

Quantitising allows further statistical analysis of data, but information is always lost when converting QUAL to QUANT

Qualitising quantitative data

Not common Narrative profile formation

Using quantitatively obtained questionnaire data in a qualitative description of a subject

More complex MMDA

Nature of QUANT data Concise Allows further analysis (inferential statistics) Provides summary information

Nature of QUAL data Detailed and informative Allows insight into cases Provides in-depth information

More complex MMDA

What purposes can mixing QUANT and QUAL data analysis serve? Illustration for insight Concise summary to give overview Preliminary overview to inform analysis Providing a more well-rounded and more

persuasive analysis

MMDA: Illustration for insight In many QUANT studies, it is easy to get lost

in the numbers and forget what they mean If the numbers are derived from QUAL data, it

is useful to give a QUAL example to concretise the QUANT findings

In Case 1, the original data is QUAL; this is quantitised for analysis; a QUAL example is given to concretise the data and to show how the quantitative analyses was applied

MMDA: Summarising for an overview In some QUAL research (primarily involving

categorisation or thematisation), the lengthy, detailed data make it difficult to see the overall pattern

It can be useful to provide a QUANT summary as an overview

In Case 2, the data is QUAL and analysed in a QUAL way, but the overall pattern of results is presented as QUANT

MMDA: Preliminary overview to inform analysis In QUAL studies with large amounts of data, it is

difficult for the researcher to ensure that all relevant issues have been identified

It is also difficult to see underlying patterns that can be drowned in the sheer quantity of data

It is useful to conduct a preliminary QUANT analysis to ensure all issues and underlying patterns are identified

In Case 3, QUAL data is treated qualitatively to find keywords which then inform a QUAL thematic analysis

MMDA: Providing a more well-rounded and more persuasive analysis In QUAL studies with large amounts of data,

restricting analysis to either QUANT or QUAL cannot provide a full picture of the data

QUAL provides detailed description of the data

QUANT provides generalisations of patterns to the whole data set

In Case 4, QUAL and QUANT analyses are used together to produce a fuller description of the data

Uses of MMDA

Use

Illustration for insight Summarise for

overview Inform analysis Provide full picture

Pattern

QUANT → QUAL QUAL → QUANT

QUANT → QUAL Mix of QUANT and

QUAL