Smith/Davis (c) 2005 Prentice Hall Chapter Nineteen Nonexperimental Methods PowerPoint Presentation...

Smith/Davis (c) 2005 Prentice Hall

Chapter Nineteen

Nonexperimental Methods

PowerPoint Presentation created by Dr. Susan R. BurnsMorningside College


Descriptive Methods

Do not involve manipulation of an independent variable

When we use descriptive methods, we can only speculate about causation that may be involved.


Descriptive Methods

Archival and Previously Recorded Sources of Data– Refers to use of data recorded by other

individuals for other purposes (e.g. public health and census data)

– The General Social Survey (GSS) is an archival source that can be accessed online

http://www.icpsr.umich.edu/gss/home.htm


Descriptive Methods

Archival and Previously Recorded Sources of Data– Potential Problems

You will not know exactly who left the data you are investigating.

The participants may have been selective in what they chose to write.

– This problem is also known as selective deposit. Archival and previously recorded sources of data may

not survive long enough for you to make use of them.


Descriptive Methods

Comparisons with the Experimental Method– We are not able to exercise any control with

regard to gathering these data and cannot make any statements regarding cause-and-effect.


Descriptive Methods

Observational Techniques– Case studies

Involves intense observation and recording of behavior of a single (perhaps two) participant(s) over an extended period of time.

– There are no guidelines for conducting a case study and the procedures employed, behaviors observed, and reports produced may vary substantially.

– Frequently used in clinical settings to help formulate ideas and hypotheses for further research.

– Although case studies often provide interesting data, their results may be applicable only to the individual participants who was observed. That is, the researcher should not generalize beyond the individual participant who was observed.


Descriptive Methods

Observational Techniques– Naturalistic Observation

Involves seeking answers to research questions by observing behavior in the real world.

– The first goal of naturalistic observation is to describe behavior as it occurs in the natural setting without the artificiality of the laboratory.

– The second goal of naturalistic observation is to describe the variables that are present and the relations among them.


Psychological Detective

Why should the researcher be concealed or unobtrusive in a study using naturalistic observation?– The reactance or reactivity effect

Refers to biasing of the participants’ responses because they know they are being observed.

The reactivity effect is also known as the Hawthorne effect because of the location of the original study.


Descriptive Methods

Participant Observation– The researcher becomes part of the group being studied.

Often used when the goal of the research project is to learn something about a specific culture or socioeconomic group.

Ethnography is a form of participant observation based on the anthropological tradition of research.

– “Observer as participant” refers to a researcher who primarily observes a situation but who interacts with the others (Glesne, 1999).

– “Participant as observer” refers to the researcher who becomes a part of the culture by working and interacting extensively with the others (Glesne, 1999).



What are the drawbacks and weaknesses of the participant observer technique?– An extended period of time may be necessary

before the participant observer is accepted as a member of the group that is under study.

– Cannot make cause-and-effect statements.


Descriptive Methods

Clinical perspective– Schein (1987) argued convincingly that the clinical perspective or

model is not a subcategory of participant observation because: A client typically chooses the clinician, whereas the participant observer

chooses the others to be studied. Unlike participant observers, clinicians cannot be unobtrusive because they

have been asked to participate in the situation. Although the participant observer can remain passive, clinicians must

intervene in the situation. The participant observer’s goal is understanding, whereas the clinician’s

goal is helping. Participant observers validate their findings by replication while clinicians

validate their findings by being able to predict the results of a given intervention.


Descriptive Methods

Choosing Behaviors and Recording Techniques.– Time sampling

Involves making observations at different time periods in order to obtain a more representative sampling of the behavior of interest.

– Selection of time periods may be determined randomly or in a more systematic manner.

The use of time sampling may apply to the same or different participants.


Descriptive Methods

Choosing Behaviors and Recording Techniques.– Situation sampling

Involves observing the same behavior in several different situations. This techniques offers the researcher two advantages:

– By sampling behavior in several different situations, you are able to determine whether the behavior in question changes as a function of the context in which you observed it.

– You are likely to observe different participants in the different situations and because different individuals are observed, your ability to generalize any behavioral consistencies across the various situations is increased.


Descriptive Methods

Deciding how to present the results of your research project.– Qualitative presentation of results:

Report consists of a description of the behavior in question (a narrative record) and the conclusions prompted by this description.

– Quantitative or numerical presentation of results: Need to know how behavior under investigation is going

to be measured and how these measurements will be analyzed.


Descriptive Methods

Using More than One Observer– There are two main reasons for using more than

one observer: One observer may miss or overlook a bit of behavior. There may be some disagreement concerning exactly

what was seen and how it should be rated or categorized.


Descriptive Methods

Using More than One Observer– When two individuals observe the same behavior,

it is possible to see how well their observations agree.

– The extent to which the observers agree is called interobserver reliability.

Low interobserver reliability indicates that the observers disagree about the behavior(s) they observed.

High interobserver reliability indicates agreement.


Descriptive Methods

The formula for calculating interobserver reliability is as follows:

85% agreement is generally considered to be an acceptable minimum level for interobserver reliability.


Qualitative Research

Qualitative research– “is defined as an inquiry process of understanding a social

or human problem, based on building a complex, holistic picture, formed with words, reporting detailed views of informants, and conducted in a natural setting” (Creswell, 1994, p. 2)

– The qualitative research style is much less formal and impersonal, and the reader of a qualitative research report can expect to find such additions as “definitions that evolved during a study” (Creswell, 1994, p. 7)


Qualitative Research

The qualitative researcher believes that a full description of human behavior includes people’s feelings in addition to what they are doing and how they are doing it.

The qualitative researcher is committed to studying particular people in specific settings.

Qualitative researchers prefer to use inductive logic. Qualitative research begins with guiding hypotheses

reflecting a global issue of interest (Marshall & Rossman, 1989).

Qualitative researchers typically analyze their data “simultaneously with data collection, data interpretation, and narrative reporting writing” (Creswell, 1994, p. 153).


Correlational Studies

A correlational study involves the measurement and determination of the relation between two variables.

In terms of control, empirical measurement, and statistical an analysis, a correlational study is likely to be more rigorous than one of the descriptive methods.

Researchers use correlational studies when data on two variables are available, but we can only measure, rather than manipulate, either variable.

Although we can determine the degree of relation that exists between these two variables, we are not able to offer a cause-and-effect statement concerning these two variables.


Ex post facto – is a Latin phrase meaning “after the fact.”

When we conduct an ex post facto study, we are using an IV “after the fact” – it has already varied before we arrived on the scene.

Ex Post Facto Studies


Surveys, Questionnaires, Tests, and Inventories

Surveys typically request our opinion on some topic or issue that is of interest to the researcher.

Types of Surveys– Descriptive – seeks to determine what percentage of the

population has a certain characteristic, holds a certain opinion, or engages in a particular behavior.

Examples:– The Gallup Poll– Nielsen television ratings

– Analytic –seeks to determine what the relevant variables are and how they might be related.

Questions need to be chosen carefully. Pilot testing is generally necessary to determine the type of

questions that will be used in the final survey instrument.



Step 1 – decide what type of instrument to use. How will the information be gathered?

– Will you use a mail survey, conduct a phone interview, or conduct the survey in person? Will you use trained interviewers?

Step 2 – identify the types of questions to use.– Yes-No Questions

The respondent answers yes or no to the items.– Forced Alternative Questions

The respondent must select between two alternative responses.– Multiple-Choice Questions

The respondent must select the most suitable response from among several alternatives.

– Likert-Type Scales The individual answers a question by selecting a response alternative from a

designated scale. A typical scale might be the following: (5) strongly agree, (4) agree, (3) undecided, (2) disagree, or (1) strongly disagree.

– Open-Ended Questions A question is asked to which the respondent must construct his or her own

answer.



Step 3 – Write the items: They should be clear, short, and specific.– Use familiar vocabulary and be at the reading level of the individuals you

intend to test. Step 4 – Pilot-test and seek opinions from knowledgeable others.

– It is important to ask others, especially professionals who have expertise in your area of research interest, to review your items. They may be able to detect biases and unintended wordings that you had not considered.

Step 5 – determine the relevant demographic data to be collected.– Includes items such as age, sex, annual income, size of community,

academic major, and academic classification. Step 6 – determine administration procedures and develop

instructions.– Instructions must be clear, concise, easy to follow, and thoroughly

rehearsed (for in-person or phone interviews).– How will informed consent be obtained?



Survey Administration Procedures– Mail Surveys

Advantages– The researcher does not have to be present while the survey is

being completed.– Surveys can be sent to a much larger number of participants than

a single researcher could ever hope to contact in person. Disadvantages

– The researcher cannot be sure who actually completes the survey.

– There is no guarantee the respondent answered the questions in the same order in which they appeared in the survey.

– Low return rate.



Assume that you are planning to conduct a mail survey project. You are concerned about the possibility of having a low response rate and want to do everything to ensure the return of your surveys. What can you do to increase your response rate?

– The initial mailing should include a letter that clearly summarizes the nature and importance of the research project, how the respondents were selected, and the fact that all responses are confidential. You should include a prepaid envelope for the return of the completed survey.

– It may be necessary to send an additional mailing(s) to respondents. Because the original survey may have been misplaced or lost, it is important to include a replacement. One extra mailing may not be sufficient; you may find it necessary to send two or three requests before you achieve an acceptable response rate. These extra mailings are typically sent at two- to three-week intervals.



Personal Interviews– Advantages

When a trained interviewer administers a survey in a respondent’s home, the response rate climbs dramatically.

The trained interviewer is able to clarify ambiguous questions, making sure that all questions are answered in the proper sequence, and generally assisting with any problems that the respondents may experience.

– Disadvantages The potential for considerable expenditure of time and money

exists. People not always at home.



Telephone Interviews– Advantages

It is easy to create random samples with random-digit dialing. 95% of households in the U.S. have telephones. Computer technology allows responses to be entered directly

as they are made.– Disadvantages

Incoming calls may be screened and potential respondents lost .

It is easy to say no to a phone interviewer. Cannot use visual aids to clarify questions. Cannot evaluate nonverbal cues such as facial expressions,

gestures, and posture. It is more difficult to establish rapport.



Tests and Inventories– Tests and inventories are designed to assess a

specific attribute, ability, or characteristic possessed by the individual being tested.

Characteristics of Good Tests and Inventories– Validity – a test or inventory has validity when it

actually measures what it is supposed to measure.



Validity (ways to establish)– Content validity – indicates that the test items actually

represent the type of material they are supposed to test. A panel of expert judges is often used to assess the content validity of test items. Their degree of agreement is known as interrater reliability.

– Concurrent validity – is established when a score on a test or inventory agrees with another measure (e.g. aggression score on a test agrees with a clinician’s assessment).

– Criterion validity – is established when a test score compares favorably with a predicted future outcome (e.g. SAT score and college success).



Reliability - refers to the extent that the test or inventory is consistent in its evaluation of the same individuals over repeated administrations.

The greater the similarity between scores produced by the same individuals on repeated administrations, the greater the reliability of the test or inventory.



Reliability (assessment)– Test-retest procedure

The test is simply given a second time and scores from the two tests are compared; the greater the similarity, the higher the reliability

– Split-Half technique Involves dividing a test or inventory into two halves or

subtests and then administering them to the same individuals on different occasions or by administering the entire test and then splitting it into two halves.



On the surface, the test-retest procedure appears to be quite straightforward and reasonable. However, there may be a problem with establishing reliability in this manner. What is it?

– The main problem with test-rest concerns the fact that the participants are repeatedly administered the same test or inventory. Having already taken the test or inventory may result in the individuals remembering the questions and answers the next time the instrument is administered (thus allowing potential bias).



Types of Tests and Inventories– Achievement tests

Are given when an evaluation of an individual’s level of mastery or competence is desired.

The score that distinguishes passing from failing determines the minimum level of achievement that must be attained.

– Aptitude tests Are used to assess an individual’s ability or skill in a particular

situation or job.– Personality tests or inventories

Measure specific aspects of an individual’s motivational state, interpersonal capability, or personality.

Smith/Davis (c) 2005 Prentice Hall Chapter Nineteen Nonexperimental Methods PowerPoint Presentation...

Documents

Transcript of Smith/Davis (c) 2005 Prentice Hall Chapter Nineteen Nonexperimental Methods PowerPoint Presentation...