Quantitative Research Method (Research Design)
description
Transcript of Quantitative Research Method (Research Design)
BY TEMTIM ASSEFAOCTOBER, 2013
Quantitative Research Method (Research Design)
Overview of Quantitative Research
Uses deductive method of knowledge acquisition
Intends to falsifies an existing theory Tries to generate generalizable knowledge
Accepts objectivity of knowledge Uses a standard measurement instruments The researcher is independent in the process
of knowledge constructionIntends with prediction and controlling
Research Process
Problem
Conceptual FW, Hypothesis
Develop Instrument
Research Design Data
Collection & Analysis
Data Interpretation
Step 1: Problems/Questions
A question/problem occurs or is posed to the researcher for which that researcher has no and can find no answer(s).
Example – usabilityThe software development industry has invested in the development of a variety of Usability Inspection Methods (UIMs) to address Web usability issues (Matera et al., 2006). UIMs are procedures in which inspectors examine usability-related aspects of a user interface (Rocha and Baranauska, 2003). However, in (Rivero et al., 2013), we identified that there is a lack of UIMs that are able to find usability problems in early stages of the development process.
Step 2: Literature, Theory, Logic
Available literature/theory is reviewed to determine if there is already a solution to the problem.
Existing solutions do not always explain new conditions.
The existing solution might require some revision or even be discarded.
Develops a hypothesis as a tentative solution to the problem
Stone (1956) defines hypothesis as a tentative explanation about as the relationship between two or more variables
Empirical to theoretical mapping
Conceptual Framework - User Involvement in system development
Participation
Conflict Conflict resolution
Influence
Types of Quantitative Research Design
A research Design is nothing more than a plan for conducting a research in such a way as to answer a research questions.
A number of questions arise to select the research design – who, what shall be the focus of the study? What attributes of individuals, organizations, systems, etc, In what contexts observations will be made (organizations, laboratories, etc), what methods employed to analyze the data?
Cont’d …
Researchers are constrained with people, time, money, etc to select the best the research design
A prime consideration is designing cost effective research
Selecting a research design Quantitative research method design includes Survey /field study field experiment, laboratory experiment, simulation, and case study
Decide form of data collection
QuestionnairesInterviewsObservation
Questioners
Most frequently used measurement instrument in organization studies
Includes sets of questions to which the subject responds
It is used to measure: Attitude Opinions Demographic characteristics of the subjects
Questionnaires design
Questionnaires make use of lists and rating scalesBehaviors and attitudes are complex and cannot
be easily evaluated and quantifiedCheck list is a list of behaviors, characteristics
or other entities that a researcher is investigatingEither the researcher or participants simply
check(s) items from the listWhat are features of user friendly software
1. Graphical interface2. Clear navigation direction3. Immediate feedback4. Other specify ____________
Questionnaire …
A rating scale is more useful when a behavior, attitude, or other phenomena of interest needs to be evaluated on a continuum scale
It is designed with the following scales 1. “inadequate” to “excellent”, 2. “never” to “always” or 3. “strongly disapprove” to “strongly
approve”
Guideline for Questionnaire
1. Keep it short• What do I do with the information?• Is it absolutely essential to have this information
to solve part of the research problem2. Use simple, clear, unambiguous language3. Check for unwarranted assumptions implicit in
your questions How many cigarette do you smoke each day?
Good to add a choice 25 ___ 25-16 ____ 15-5 ___ <5 ____ None ____
4. Word your questions in ways that don’t give clues about preferred or more desirable responses
What strategies have you used to try to quit smoking?
Leads him to list strategies he did not try
Guide line …
Check consistency – that leads to give contradictory answer for two questions
Determine in advance how you will code the response
Keep the respondents task simpleProvide clear instructionsMake the questionnaire attractive and professional
looking Conduct a pilot test
Give for half a dozen friends to see they have difficulty understanding any items
Scrutinize the almost final product carefully to make sure it address your needs
Advantage of Questionnaire
Easily to collect data from large number of people, including those who live thousands of miles away
Can be administered by unskilled data collectorsPresents a uniform stimulus to all subjectsCan be collected online or through mailResearcher does not have contact with the
respondentsThe anonymity of the respondents helps
respondent to provide more truthful information than they would be in personal interviews, especially on sensitive and controversial issue
Draw back of questionnaire
Majority of people may not return, if it is mailed the return rate is not more than 50%
Returned responses may not be a true representative of the originally selected samples
Cannot be used with illiterate or persons who cannot read
Most questioners are inflexible, respondents answer by choosing only from the given lists
Interviews
Interviews are meetings in which the interviewers directs questions at the interviewee and records the obtained responses
It is direct face to face interactionsCan be undertaken through telephoneHelp to captured more detailed information
than questionersAllow to include the respondents views
Design of Interview
Can be structured and unstructured Unstructured interview is designed with open
ended questions The researchers asks and the responds reply
to the questions The same questions may not be asked for
different respondents Examples
Can you list down the benefit of the new software for your organization?
What are the main drawbacks of the new software? What do you suggest to improve software to solve
the current problems?
Structured interviews
The researcher asks standard set of questions and nothing moreIn semi structured interview, the researcher may follow the standard questions with
one or more individually tailored questions to get clarification or probe a persons reasoning
In structured interview, you can have checklist of responses Example
What are the benefits of LAN in the organizationa. Facilitates communications among employeesb. Improves information accessc. Improves relationship with customers d. Other specify
What kind of the support the management provides to the success of the Intranet system
Advantages of Interview
Interview is formal and emotionally neutralHelp to develop hypothesis at early stage of
the research than testing hypothesis It has a distinct advantage of establishing
rapport with potential participants and gain cooperation
Has higher response rateFlexible to include additional views and
opinionsHas better validity, the researcher can
further ask probing questions on some biased response
Disadvantage
It is generally expensive than other data collection methods
The researcher is the main interviewer otherwise training of interviewer is very expensive and time taking
The interviewer presents the questions in different manners that lead to different responses – affects the validity and reliability of the data
Characteristics of the respondents (age, sex, ethnicity, etc) may affect the value of measured variables
Respondents may not free on some controversial issues
Data Analysis and Interpretation
Does the observed data supports the previously postulated research hypothesis/relationships between variables
There are different data analysis (statistical ) methods for data analysis
Commonly mentioned as descriptive and inferential statistical methods
Descriptive methods include: Frequency, mean, median, correlation, Charts and graphs, etc
•
Data analysis …
Inferential methods include: Parametric test: t-test, ANOVA, linear regression, Non-parametric test: Chai square test, Man Whitney test
Benefits
Theoretical propositions can be tested in an objective fashionEnable generalizationsEasy to use (although may be an illusion)Established and accepted research method across multiple fieldsAided by statistical toolsImmensely helped by advancement computingStrong in external validity if sampling errors are minimized.
Field Experiment
Field Experiment
A field experiment is an experimental research method which is performed outside the laboratory in the natural settings
It follows all steps of scientific process1) selection and determination of a problem,
2) selection of participants and measuring instruments,
3) selection of a research plan,
4) execution of the plan,
5) analysis of data and
6) formation of conclusions
Features of FE
It has three unique features1. the research takes place in
natural setting
2. the experimenter manipulates one or more independent variables while exerting much control over confounding variables; and
3. the effect of the manipulations on one or more dependent variables is systematically observed.
FE Design
There three type of designs
Controlling Nuisance in Field Experiment
Solomon Four Groups in True ExperimentExperimental group 1: R O X O Control group 1: R O O Experimental group 2: R X O Control group 2: R O
Removing statistically the effect of suspected confounding variables
Example – Interface Design and Computer use
System use is low among employees. One reason is poor interface design. You divide the employees into two groups (Group 1 & 2)Redesign the Interface and let Group 1 work on the new
system - Treatment GroupLet Group 2 work on the old Interface– Controlled
GroupAfter six or some months after Interface design,
measure, the computer use.If computer use is increased for Group 1, you can
conclude that Interface design and computer use has association
Data collection and Analysis
The following methods are used to collect data Questionnaire Structured interview Computer log files
Data analysis Descriptive statistical methods Inferential statistical methods
Advantages of FE
Conducted in natural settings and has higher experimental validity
Help to clearly identify antecedents of observed effects in casual relationships
Used for the development of theory as well as for the solution of applied problems;
The logic of FE can be applied in the analysis of many naturally occurring changes
Help for testing of broad hypotheses dealing with complex social process in lifelike situations.
Disadvantage
The methods used to control confounding variables in FE are not sufficient
Manipulation of variables cause legal and ethical problemsDifficult to control dynamically changing environment
during the course of the experimentDifficult to precisely measure dependant variables in the
field settingsExpensive to conduct compared to lab experimentNeeds highly skilled person to design and conduct
Role of FE in computer Science
It has many applications in IS To develop theory To test hypothesis To evaluate IS tools and techniques
Practical Examples
Fukada et al (n.d.) used field experiment to study the effect
of Road Facility Management Support System on asset
management of the public infrastructure.
Mayur et al (2000) made study to compare and suggest the
best training method for companies between instructor
based training and computer based training methods.
Chen et al (2007) made a field experiment study to identify
effects of different types of social information on
contributions to an online communities.
Review Questions
Explain the different quantitative research design methods
What is the strength and weakness of each methodWhat are the different survey research methods What is the main data analysis methodswhat problem can you address with quantitative
research methods What are the procedures in quantitative Research How do you use Quantitative research in computer
science?