The research process, surveys, questionnaires & interviews Communication Research Week 3.

The research process, surveys, questionnaires & interviews

Communication Research

Week 3

Communication Research Spring 2005 2

Steps in the research process 1) Establish the need for research 2) Define the problem 3) Establish research objectives 4) Determine research design 5) Identify information types and sources 6) Determine methods of accessing data


Steps in the research process 7) Design data collection forms 8) Determine sample plan and size 9) Collect data 10) Analyse data 11) Prepare final research report


The research process

PR

OB

LEM

S

TA

TE

ME

NT

Purpose of the study

Exploration

Description

Hypothesis

Types of investigation

Establishing:

Causal relationships

Correlations

Group differences, ranks etc

Extent of researcher

interference

Minimal: Studying events as they normally occur

Manipulation and/or control

and/or simulation

Study setting

Contrived

Non contrived

Measurement & measures

Operational defnItems (measure)

ScalingCategorising

Coding

Unit of analysis (pop to be studied)

Individuals

Dyads

Groups

Organisations

Machines

Sampling design

Probability/non

probability

Sample size (n)

Time Horizon

One-shot (cross-

sectional)

Longitudinal

Data-collection method

Observation

Interview

Questionnaire

Physical measurement

Unobtrusive

Data-analysis

1.Feel

for data

2.Goodness

of data

3. Hypothesis

testing

DETAILS OF STUDY MEASUREMENT


The Classical Ideal of Science

Generalization

Theories

Hypotheses

Observation

Deduction

OperationalizationMethod & measurement

Induction


Deductive vs Inductive Reasoning

DEDUCTIVE Argues from the general

to the particular Eg you observe that all

deciduous trees lose their leaves …

You therefore reason that your bare tree is deciduous

INDUCTIVE Argues from the

particular to the general Eg if you burn one finger

on a hot stove … You therefore reason

that you could burn all of them


Preparing an hypothesis or research question

An hypothesis is a ‘reasonable scientific proposal’ or a ‘statement of expected results’

It is not a statement of fact but a declarative statement which tells the reader what you are going to do NOT how you plan to do it

An hypothesis is common to scientific research methodologies while qualitative research more commonly uses research questions to focus


Writing a hypothesis A well written hypothesis

Is stated in declarative form Posits a relationship between variables Reflects a theory or body of literature upon

which it is based Is brief and to the point Is testable


Developing an hypothesis An hypothesis should be developed from a well-

researched body of knowledge which is both logical and feasible eg Inland waterways are becoming polluted through fuel

discharge from petrol-driven powerboats Powerboats are noisy thus creating auditory pollution Alternatives to petrol engines exist

Electric (battery-powered) boats will decrease pollution on

inland waterways


Ideas, questions & hypothesesResearch Interest or

IdeasResearch Problem or

QuestionsHypothesis

Open classroom and academic success

What is the effect of open vs traditional classrooms on reading level?

Children taught reading in open classroom settings will read at a higher grade level than children taught reading in a traditional setting.

Test-taking skills and grades

Will students who how to “take” a test improve their scores?

Students who receive training in the “Here Today – Gone Tomorrow” method will score higher on the SATs than students who do not receive the training.

Television and consumer behaviour

How does watching television affect buying behaviour of adolescents?

Adolescent boys buy more of the products advertised on television than do adolescent girls.

Drug abuse and child abuse

Is drug abuse related to child abuse?

There is a positive relationship between drug abuse among adults and their physical and psychological abuse as children

Adult care How have many adults adjusted to the responsibility of caring for their aged parents?

The number of children who are caring for their parents in the child’s own home has increased over the past ten years.


Rats & scientific research


Problems with asking questions as evidence (from Vance Packard (1956) The Hidden Persuaders)

1. You can’t assume that people know what they want In a survey of male drinkers the men expressed a

strong preference for a ‘nice dry beer’. When they were then asked how a beer could be dry, they were stumped.Those able to offer any answers at all revealed widely different notions.



2. You can’t assume people will tell you the truth about their wants and dislikes even they know them. Psychologists at the McCann-Erikson ad agency

asked a sampling of people why they didn’t buy one client’s product – kippered herring.The main reason the people gave under direct questioning was that they just didn’t like the taste of kippers. More persistent probing however uncovered the fact that 40% of the people who said they didn’t like kippers had never in their entire lives tasted kippers.



3. It is dangerous to assume that people can be trusted to behave in a rational way. A test was designed to establish the influence of the

package on the product. It gave housewives three different boxes filled with detergent and requested that they try them all out for a few weeks and then report which was the best for delicate clothing (Note: actually only the boxes were different, the detergents were identical – one box was predominantly yellow; the second was blue and the third was blue with splashes of yellow).

Result: the detergent in the yellow box was too strong, in the blue box it left the clothes dirty and in the blue and yellow it was ‘fine’ and ‘wonderful’.


Survey research

Purpose is to determine the current status of a population with respect to one or more variables

Can be qualitative or quantitative, depending on data sought


Advantages of surveys Inexpensive Can obtain current information Enable the researcher to obtain a great deal of

information at one time Provide quantitative or numerical data Very common and so some of the info you seek

may have already been gathered eg ABS


Problems with surveys People often don’t tell the truth, especially

about personal matters People make mistakes about what they’ve

done Obtaining representative samples is frequently

difficult People often refuse to participate Relatively small percentages of people answer

and return questionnaires Writing good survey questions is difficult


Self-administered questionnaires

ADVANTAGES Inexpensive No interviewer bias to worry

about You can ask about very

personal matters You can ask complex, detailed

questions

DISADVANTAGES People may misinterpret

questions Low response rates the norm You don’t know who actually

filled out the questionnaire Sampling errors frequent


Stages of a survey 1. Define your information needs

what information? purpose of information? who knows the answers? practical issues eg time

2. Carry out background research – what work has been done before in this area?

3. Choose a survey technique questionnaires diaries individual interviews group discussions


Stages of a survey 4. Define and test your methodology

Open or closed questions? Pre-test questions Ordering of questions Trial questioning techniques

5. Administer your survey – consider any problems By mail/email Is target group representative? Were all questions answered satisfactorily?

6. Analyse your results Did enough people reply? Is target group representative? Were all questions answered satisfactorily?


Stages of a survey 7. Present your findings

Usually in report format Outline parameters of survey critically Do your results prove or imply results? Acknowledge limitations eg time, sample size,

demographics of sample etc


Problems with Survey Research

Inadequate response

Including unrelated items on questionnaire

Poorly worded items

Complex items

Leading questions

Assuming facts not necessarily in evidence

Analysing open-ended questions


Problems with Survey Research People often don’t tell the truth about

themselves People make mistakes even if they are

trying to tell the truth eg ratings books Obtaining representative samples is

difficult Relatively small percentage of people

answer and return questionnaires


Survey & Questionnaire design

When designing a survey or questionnaire you need to consider the type of question which will give you the most accurate data

There are five main types of questions 1. Close-ended questions which use yes/no

responses Q. I have good communications with my supervisor

A. Yes No


Survey & Questionnaire design

2. Open-ended questions which allow the respondents to give an unlimited answer

Q. What problems are you having with your supervisor?


Survey & Questionnaire design 3. Checklist which present a list of items where

participants are asked to check those items that apply to their particular situation.

Q. Please check the following types of communications that you have with your supervisor.

Informal meetings After hours discussions

Formal meetings Telephone

Written reports Social gatherings

Emails Committee meetings


Survey & Questionnaire design 4. Multiple-choice questions which offer several

choices and the respondent is asked to select the most correct one. Ensure the choices presented cover all the possible options.

Q. How often do you purchase items from the company vending machines?

a) Once a dayb) 2-3 times a dayc) 3-5 times a dayd) 6 or more times a day


Survey & Questionnaire design 5. Ranking scales which require the participants

to rank order a list of items. Q. Of the following list of five types of communication that you

might have with your supervisor, rank from 1 (most important) to 5 (least important).

— Formal meetings— Informal conversations— Written reports— Letters or emails— Telephone discussions


Survey & Questionnaire design 6. Likert scales which usually measure attitude toward a

concept or idea. It allows the respondent to indicate the degree of agreement usually on a 5 or 7 point scale.

Q. Please indicate the degree to which you agree or disagree with this statement. “The 1995 Ford Falcon is a substantial improvement on the 1994 model.”

Strongly agree Agree Neither agree Disagree Stronglynor disagree

Disagree

5 4 3 21


Survey & Questionnaire design 7. Semantic differential scales measures attitudes by displaying

pairs of opposite terms and asking respondents to check which term best describes their feeling toward the concept or topic.

Q. Place an X in the space between the two terms that best describes how you see the XYZ Corporation Office.

XYZ CORPORATION DISTRICT OFFICE[the topic or entity being evaluated]

Pleasant - - - - - - - - - - - - -- - - - - UnpleasantEfficient - - - - - - - - - - - - -- - - - - InefficientNot helpful - - - - - - - - - - - - -- - - HelpfulProfessional - - - - - - - - - - - - -- - UnprofessionalInsensitive - - - - - - - - - - - - -- - - SensitiveFriendly - - - - - - - - - - - - -- - - - - UnfriendlySlow - - - - - - - - - - - - -- - - - - FastRigid - - - - - - - - - - - - -- - - - - Flexible


Other types of qualitative research – depth interviews

one-on-one interview which is relatively unstructured with a subject by a trained interviewer.

The direction of the interview is guided by the responses to the questions


Other types of qualitative research – personal interviews

ADVANTAGES Interviewer can explain Qs in

detail Interviewer can use a variety of

data collection methods Interviewer can spend a lot of

time with respondents You know who is answering the

questions A higher likelihood of achieving

the desired response rate Not intimidating

DISADVANTAGES Can be intrusive (too personal) Time-consuming and expensive Hard to find people in sample at

times People are reluctant to answer

some questions Needs well-trained interviewers Possible language barriers


Other types of qualitative research – focus groups

Often used in market research A group is asked a series of structured

questions and guided through structured sessions

Designed to probe their attitudes and feelings about a range of issues


Other types of qualitative research – projection techniques

Such as word association tests, sentence and story completion, cartoon tests, consumer drawings, photo sorts

These techniques belong to the field of clinical psychology

They are designed to probe and penetrate a person’s defense mechanisms and allow true feelings to emerge

The research process, surveys, questionnaires & interviews Communication Research Week 3.

Documents

Transcript of The research process, surveys, questionnaires & interviews Communication Research Week 3.