Longitudinal and Evaluational Research

8/12/2019 Longitudinal and Evaluational Research

1/39

Longitudinal and Evaluational

Research

By: John Amada, Jyulier Malco,

Jason Mangilit


2/39

Longitudinal research outline

I. What is Longitudinal research

II. Method of data gathering

III. Pros/AdvantagesIV. Cons/Disadvantages

V. Types

VI. Journal


3/39

Longitudinal Research

A type of research method used to

discover relationships between variables

that are not related to various background

variables.

Researches study the same group over a

long period of time hence the term,

longitudinal.


4/39

How is data gathered?

asking people questions, observing

people's behavior, and observing the

physical traces or results of people's

behavior for a long period of time.


5/39

Pros/Advantages

Useful on researches about development

life span issues.


6/39

Cons/Disadvantages

Requires a long and enormous amount of

time.


7/39

Types of Longitudinal Research

Panel Study/Cohort Study:

Involves sampling a cross section of

individuals.

Example: people within a common age-band,

and groups with some other date-specific

common experience, such as people

graduating from a university, having a firstchild, or migrating to another country in a

given year or band of years


8/39

Retrospective study

Involves looking to the past by looking athistorical information such as medical

records. Studies groups of individuals who are alike in

many ways but differ by a certaincharacteristic (for example, bodybuilders who

do cardiovascular exercise and those whodon't) are compared for a particular outcomesuch as heart attacks.


9/39

Journal Article


10/39

Evaluation research


11/39

Evaluation research outline

I. What is evaluation

a. Definition

b. Nature of Evaluation

II. Types of Evaluation

a. Formative

b. Summative

III. Quantity and Quality: Paradigmatic Choices in Evaluation Methodology

a. The Paradigm Problem

b. Quantitative Approach

c. Qualitative Approach

d. Paradigm of Choices

IV. Methods of Data Collection

a. Types of Data

b. Methods of Collecting Data

c. Mixing Methods

d. Method and Theory

V. Journal Article


12/39

What is evaluation?

Evaluation is a systematic review of

someone or somethings worth,

significance, merit using a criteria based

on a set of standards


13/39

Methods of evaluation

Evaluation is influenced by the situation,

how the evaluation is conducted.

Evaluation helps find the right methods to

use.


14/39

Types of data

Qualitative

used to obtain details of the subjective

experiences of programme planners and

participants in process-orientedevaluations.

Quantitative

objective factual data that are not onlyreplicable, but also amenable to statistical

analysis.


15/39

How they are acquired

Primary data

Involves the use of research instruments.

Secondary data

Data collected other researchers,

organizations or government departments for

their own purposes.


16/39

Types of data

Design

I. Structured

- it imposes a structure on the data at the point of collection

- it has predetermined nature of the response categoriesaffords respondents little opportunity to express their individualfeelings.

Ex. Standardized assessment schedules or attitudinalscales

II. Unstructured- provide scope for individuals to respond in their own terms

Ex. Informal interviews and open-endedquestionnaires


17/39

Questionnaires

- Can be used to provide the main source ofdata in a study

- Concerned with description and

measurement- Can include assessment schedules,

sociometric tests and attitude rating scales

1. use an existing scale

2. construct a new scale


18/39

PRINCIPLES in using

questionnaires to collect data:1. The questionnaire should be delivered with a

covering letter explaining the purpose of theresearch.

2. It is important that the document itself looks

attractive and that all instructions are easy tofollow.

3. Questionnaires should not be too long

4. The order in which questions are asked is

important5. Question wording should be clear and

unambiguous.


19/39

Closed questions

- Ones which require the respondent to

select one or more responses from a

predetermined list of possible answers.- Yes or No

- Strong Agree or Disagree

- Choose a number

- Symbols and Pictures


20/39

Open-Ended Questions

- Allow respondents to answer in their own words,rather than being restricted to choosing from alist of pre-coded categories.

- They give respondents leeway to elaborate theiranswers.

- They call for greater deliberation and a writtenresponse.

- They can generate wide range of response- they are best placed towards the end of the

questionnaire.


21/39

Formulating Open-Ended

Questions1. Use language appropriate to the target

audience

2. Avoid leading questions

X Do you prefer..? Dont you agree that?

3. Always make sure that the frame of

reference for a question is clearly stated


22/39

General advice in developing an

interview schedule1. It begins with the evaluator drawing up a

preliminary list of categories deemed to berelevant given the evaluation situation.

- program literature- related documentary materials

- data collected from unstructured,exploratory interviews with principal

stakeholders2. The evaluator proceeds to generate lists of

questions relevant to each category


23/39

Internal Interviewers

-using programme staff to conductinterviews

Advantage:

Can help to keep costs down as well ashaving other potential advantages

Disadvantage:

Programme staff can find it difficult to keepthe roles of service provider and researchinterviewer separate


24/39

Group Interview or Focus Group

- provides relatively inexpensive and

efficient way of collecting data particularly

when the investigator is primarily

concerned with obtaining an insight intothe attitudes and opinions of groups,

rather than acquiring specific information

about individuals.


25/39


Group Size

There is no fixed size for a focus group interview, but the groupmust nt be too large as this may inhibit some members from joiningin the discussion.

A typical focus group involves around six to twelve people whoare brought together by the researcher for the purpose of answeringa number of questions.

Group Facilitator

Encourages the participants to respond in their own termswhile simultaneously ensuring that the focus of the group ismaintained.

Group SettingAllows the participants to qualify their original responses in thelight of comments made by other group members.


26/39


Advantage:

1. It can generate a lot of qualitative data fairly quickly; normally afocus group session lasts one to two hours

2. It is relatively inexpensive to run and therefore a popular choicewhen financial resources are limited.

Disadvantages:

1. Individuals may suppress or modify their true feelings when in thepresence of others.

2. Individuals may feel particularly inhibited when in a group where

participants not only know each other, but also have to workalongside each other.

3. There is the question of the status of individual participants.


27/39


Preliminary stage

- It can help identify what participants see as the keyissues.

Critical stage

- It can be used to obtain feedback from participantsas to what problems are encountered and they areresolved.

End of the programme

- It can be used to obtain participants perceptions ofthe impact and overall effectiveness of the programme.


28/39

Observation

Systematic Observation

- when observation and recording are doneaccording to explicit procedures which permitreplication and rules are followed which permitthe use of logic of scientific inference

- involves much more than the looking andlistening as part of everyday life.

- the evaluator becomes the main instrumentof data collection


29/39

Roles of the evaluator

I. Participant observation- involves the evaluator entering the social world of those engaged in

programme activities in order to provide a full and detailed account of theprogramme

a. Complete participant

- the role adopted by the researcher is a covert or secret one; those

people under observation are unaware of the identity or intentions of theobserver

- there is a possibility that the covert participants may feelconstrained from asking questions in field setting for fear of appearingunnaturally inquisitive, thereby running the risk of being exposed.

b. Participant-as-observer

- the reasons for the researchers presence are made explicit,

consequently, there is no threat of exposure.- Overt observers might have the advantage of being able to freelyquestion informants, but the fact that they are seen as researchers mayinfluence how informants behave and respond to their questioning.


30/39

Roles of the evaluator

I. Non-participatory roles

a. Observer-as-participant

- contact with informants is brief, and although the roleis an overt one there is no attempt on the part of the researcher

to develop relationships with informantsb. Complete observer

- there is no social interaction whatsoever between theobserver and the informant during the course of data collection

- as is the case with the systematic eavesdropping orwhen the behavior of subjects in laboratory-based experimentsis observed through a one-way mirror


31/39

Advantages:1. Observational fieldwork is essential if the evaluator is to provide a suitably

descriptive account of the core features of any programme

2. When there is strong emphasis on participation, personal experience becomes animportant resource

3. Direct experience of programme activities enables the evaluator to draw on tacit,as well as propositional, knowledge, in order to describe a situation or series of

events4. In some situations, observational methods can succeed where other methods fail

or prove infeasible.

Disadvantages:

1. Gathering, recording, and analysing field data are costly activities, both in terms oftime and money.

2. There is concern about the reliability and validity of observational data.3. There is always the possibility that individuals will consciously modify their

behavior if aware that they are under observation.

4. Fieldworker gets so caught up in the activities under observation that it affectswhat is actually observed.


32/39

Documentary Sources

Kinds of documents1. Primary sources

- documents compiled by individuals who have firsthand experience of the eventsdescribed

2. Secondary sources

- materials consists of documents produced by individuals who do not possess personalknowledge of the situation

Ex. Written synopsis of a lengthy official report

3. Public documents- administrative records held by national and local governments, official statistics and

reports of government select committees. Many of these documents are freely available

4. Private documents

Ex. File notes, records, case files

5. Solicited documents

- includes documents the individual is asked to produce by the researcher; hence theresearcher can exercise some control with respect to what information is included and themanner in which it is recorded.

6. Unsolicited documents

- the researcher is unable to influence the kind of information that is collected and mustmake do with what is available. The information will have been originally collected for apurpose other than an evaluation study and is therefore referred to as secondary data.


33/39

Mixing Methods

Mixed-methods research designs, in some

cases integrating quantitative and

qualitative methods, are now an

established feature of programmeevaluation research and policy evaluation

studies.


34/39

Mixing Methods

Triangulation

- borrowed from surveying or navigation,where it refers to the practice of

establishing the exact position of a givenobject by taking readings ormeasurements from multiple view-points.Using more than one reference pointenables greater accuracy ofmeasurement.


35/39

Mixing Methods

Types of Triangulation1. Data

- involves the creation of multiple data sets by collecting data in a variety of contextsand settings at different points in time.

2. Investigator

- occurs when more than one researcher or evaluator investigates the same situation.Using multiple investigators ensures that a number of different viewpoints are

represented.3. Theory

- entails making use of a number of alternative or competing theories in examining thedata

4. Methodological

a. within-methodentails applying the same method on different occasions or usingmultiple techniques within a given method

Ex. Administering the same questionnaire to

respondents on separate occasionsb. between-methodsrefers to the actual mixing of methods in a single researchdesign

Ex. A questionnaire survey of programme activities can be combined withnon-participant observation of programme activities and qualitative interviews withsamples of recipients and key workers.


36/39

Mixing Methods

Advantages:1. Allows the researcher to have greater

confidence in the research findings than isthe case when a single method is used

2. Way of guarding against threats to bothreliability and validity.

3. Involves cross-checking for internalconsistency (within-method) and tests thedegree of external validity (between-method)


37/39

Method and Theory

Types of Evaluation

1. Method-driven

- rigid adherence to a particular method, or

collection of methods, determines the focusand scope of an evaluation

2. Theory-driven

- the evaluator chooses those research

methods deemed best suited for testing theparticular theory in question


38/39


39/39

Thank you for your time. :)

Credits to Mam Rieta for the Evaluation

research portion of our power point

Longitudinal and Evaluational Research

Documents

Transcript of Longitudinal and Evaluational Research