Second Language Research Chapter 3

By Abolfazl Ghanbari

AimTo know about Pilot Testing To know different ways of data gathering

3.1 Pilot Testing

A pilot testing is a small-scale trial of the proposed procedure, materials, and methods.

The goal is to test, and often to revise, and then finalized the materials and methods.

It is carried to uncover any problems, and address them before the main study Can help avoid costly and time-consuming problems during the data collection procedure as well as the loss of valuable, potentially useful, and often irreplaceable data (Gass & Mackey, 2000).Sometimes pilot studies result in data that might be useable for the main study

3.2 The significance of data collection measures

Findings in second language research are highly dependent on the data collection measures used.

The choice of one measure over another is highly dependent on : the research questions askedAnd may also related to the theoretical framework within which research is conducted.

For example: sample of Japanese passives and the experts at avoidance

3.3 Researching formal models of language

(1) Acceptability Judgment: Materials Procedure

(2) Elicited imitation (3) Magnitude estimation (4) Truth-value judgments and other interpretation tasks (5) Sentence matching

(1) Acceptability Judgment: 1. UG-based second language research seeks to determine the extent to

which second language learners have the same abstract representations as do native speakers. One needs to determine, then, what learners believe to be grammatical in the language being learned and what they consider ungrammatical.

2. 2. Because UG researchers need to understand the nature of abstract representations and because abstractness is only inferable from surface phenomena, researchers are often in a position of needing to force learners into stating what is possible and what is not possible in their second language. Acceptability judgments, a common elicitation tool in linguistics, are often used for this purpose.

(1) Acceptability Judgment: 3. the general using acceptability judgment is to give a list of target language sentences to be judged and to ask for corrections of any sentences judged incorrect. There are numerous practical considerations when dealing with collecting judgment data, some of which we address below:

Materials

Procedure

(1) Acceptability Judgment:

(1) Materials:Order of presentationNumber of sentences

TimingContext Comparison Modality

(1) Acceptability Judgment: Order of presentation: Counterbalance so that order of presentation does not affect results (see Gass, 1994, for a study in which each participant received a different sentence ordering).Number of sentences: Balance grammatical and ungrammatical sentences. Judging sentences can become tiresome and judgments may become unreliable. One way of countering this is to make sure that participants have different orders (see previous point). It is also important to make sure that there are sufficient numbers of filler or distractor sentences so that participants in a study cannot easily guess what the study is about.Timing: With tests given orally or on the computer, the timing between sentences can be controlled. Questions about how much time should elapse have not been seriously addressed in the literature, but could potentially affect the results (see Murphy, 1997).

(1) Acceptability Judgment:

Context: So that the imagination of a context does not enter into the picture, sentences can be embedded in a situational context Comparison: At times, one wants to have sophisticated judgments of acceptability involving subtleties in language. In such instances, it might be advisable to ask participants to judge sentences as they relate to one another (e.g., We didn't dare to answer him back versus We didn't dare answer him back—which is better?). This technique is not often used in second language research .

(1) Acceptability Judgment:

Modality: Are sentences given orally, on paper, or on a computer? Examples of each type abound, although the paper and pencil task is probably the most common, with computer testing increasing in recent years. The least common is oral administrationPictures: Although not actual judgments of acceptability, a slight variation on acceptability involves the use of pictures. In these situations, learners might have to match up a sentence with a picture or provide a judgment about a sentence in relation to an accompanying picture. Information from these tasks may lead to inferences regarding grammaticality.

(1) Acceptability Judgment:

(2) Procedures: Corrections Instructions Scoring

(1) Acceptability Judgment:

Corrections: Certain assumptions that are often made with native speaker judgments cannot

always be made with nonnative speaker judgments. Thus, in order to ensure that the research focus is the same as the nonnative

speakers' focus, it is necessary to ask learners to correct those sentences they judge to be unacceptable. For example, given the sentence *She knows the woman whom is the sister of my friend, learners might judge this as incorrect.

But without correction, we do not know what they think is incorrect about it.

(1) Acceptability Judgment:

Instructions: Instructions, therefore, and the examples provided need to be carefully crafted.

For example, Ionin and Wexler (2002) provided explanation orally: "The investigator talked the practice items over with the child and ensured that the child was responding to the grammaticality and not to the meaning of the sentences" (p. 121).

It is often beneficial if instructions can be translated into the native languages of the participants. The Bley-Vroman et al. instructions would, of course, need to be modified according to the specific circumstances.

(1) Acceptability Judgment:

Scoring: Scoring will depend on how the task is set up. One sample is showing "goodness" on a “Likert scale” 1: -2 -1 0 1 2 (Five point Likert scale)

2:

3: 1 2 3 4 5 6 7 (positive scale )

(2) Elicited imitation

It is used to determine grammaticality. If a given sentence is part of one's grammar, it will be relatively easy

to repeat. In elicited imitation tasks, sentences are presented to participants

auditorily (i.e., either taped or orally by the researcher), and the participants are then asked to repeat them.

The sentences are typically designed to manipulate certain grammatical structures, and a person's ability to repeat the sentences accurately is a reflection of his or her internal grammatical system.

(3) Magnitude estimation

It is useful when one wants not only to rank items in relation to one another, but also to know how much better X is than Y. It has been used when eliciting grammatical knowledge not as an absolute (yes or no), but as a matter of gradation (i.e., which sentence is more acceptable than another?).

(4) Truth-value judgments and other interpretation tasks

Truth-value judgments are a way of understanding how people interpret sentences. These have been used extensively in the study of reflexives by L2 learners.

(5) Sentence matching

Sentence matching tasks are usually performed on a computer. Participants are seated in front of a computer and are presented with a sentence that is either grammatical or ungrammatical. After a short delay, a second sentence appears on the screen, with the first sentence remaining in place.

3.4 Processing research

1) Sentence interpretation

2) Reaction time

3) Moving window

4) Eye-tracking

1) Sentence interpretation (SI)

One model is dealing with SI is Competition Model.

it focuses on how learners process information

The major concern is what information people use in coming to an understanding of the relationships of words in a sentence.

It uses sentences with various cues.

For example,

when we see or read a sentence such as Sally kissed John, how do we come to

an interpretation of who kissed whom? In English, we rely on word order (the

first noun is typically the subject), meaning and animacy status of lexical

items (if the sentence were The pencil kissed John, we would be confused as

to the relationship), and morphological agreement. Some languages use case

markings as the dominant cue with word order being less important.

Not all languages use these same criteria (called cues), and not all languages

assign the same degree of importance or strength to each criterion.

2) Reaction time

It is assumed that the more time it takes to respond to a sentence, the more processing "energy" is required.Reaction time measures are often used in conjunction with other kinds of research already discussed. For example, they can be an integral part of sentence matching experiments because the framework underlying sentence matching relies on comparing reaction times between grammatical matched sentences and ungrammatical matched sentences.

3) Moving window

is another elicitation measure that is typically carried out on a computer.In a moving window experiment, words are presented on a screen one at a time, with each successive word appearing after a participant indicates that she or he is ready. In other words, it is the participant who controls the reading speed. Once a new word appears on the screen, the previous word disappears. After the entire sentence has appeared, the participant presses a button to indicate whether the sentence is grammatical or ungrammatical.Moving window techniques can provide information about processing times for various parts of the sentence.

4) Eye-tracking

It provides documentation on eye gaze and/or duration while reading/watching something on a screen. The assumption behind eye-tracking is the eye-mind link, which assumes that where one focuses one’s eyes is a reflection of thought processing longer duration assume some sort of difficulty.

3.5 Interaction-based research

the focus is learners' conversational interactions with others (e.g., other learners, native speakers, and teachers) and the developmental benefits of such interactions.Within interaction-based research, the goal is usually to manipulate the kinds of interactions in which learners are involved, the kind of feedback they receive during interaction, and the kind of output they produce in order to determine the relationship between the various components of interaction and second language learning.The most common way of gathering data is to involve learners in a range of carefully planned tasks.

3.5 Interaction-based research

one-way and two-way task:In a one-way task, the information flows from one person to the other, as when a learner describes a picture to his or her partner.In a two-way task, there is an information exchange whereby both parties (or however many participants there are in a task) hold information that is vital to the resolution of the task.

3.5 Interaction-based research

a closed task and an open task: Another way to classify tasks is to consider the resolution of the task. Is there one correct outcome, as in a closed task (e.g., when two learners need to identify exactly five differences between two pictures), or do the participants need to agree on a common outcome or conclusion, as in an open task such as a discussion activity?

3.5 Interaction-based research

Picture Description Tasks: Many picture description tasks are information-gap tasks. It is important to ensure that if someone is describing a picture to another, the describer's picture cannot be seen.Spot the Difference:Participants are asked to find the differences, and the number of differences can be prespecified so that the participants have a goal toward which to work.it is crucial that participants not see their partners' pictures.

1. Find a picture that contains items that can easily be described, but that includes vocabulary that is likely to cause some lack of understanding, and hence, some negotiation. This might involve physical objects or the placement of objects (above, on top of).2. Separate individuals by a barrier or at least ensure that the picture is not visible to the other person in the pair.3. Ensure that the picture contains appropriate items for description and/or appropriate location of items in the picture. For example, a picture with a car on top of a house would add another element of difficulty to the task.4. If relevant, make sure that the task elicits the linguistic structures or forms of interest.5. Ensure that there are sufficient opportunities for interactional modifications, feedback, and output based on the research question.6. For picture description, make sure the participants understand that the person drawing should not see the picture until the task is completed. For spot the difference, make sure that no participant shows his or her picture to the other(s). Inform participants about the number of differences, if necessary.7. As usual, carefully pilot test the task.

guidelines for spot the difference and picture description tasks

3.5 Interaction-based research Jigsaw Tasks:is a two-way taskindividuals have different pieces of information. In order to solve the task, they must orally interact to put the pieces together.Consensus(a general agreement) Tasks:Consensus tasks generally involve pairs or groups of learners who must come to an agreement on a certain issue.it does not guarantee that there will be interaction.Another type of consensus task is a dictogloss task. In this type of task, learners work together to reconstruct a text that has been read to them.

3.5 Interaction-based research Consciousness-Raising Tasks:are intended to facilitate learners' cognitive processes in terms of awareness of some language area or linguistic structure.In these tasks, learners are often required to verbalize their thoughts about language on their way to a solution.Computer-Mediated Research:involves learners in communicative exchanges using the computer.

3.6 Strategies and cognitive processesStrategy-based research is aimed at determining the strategies used when learning a second language together with the variables that determine the selection of strategies.Asking learners (directly or through a questionnaire) which strategies they use in general, or which strategies they use when attempting a particular task.Observing learners while they work at their language learning tasks.Asking learners to give a retrospective commentary on how they learn (e.g., through keeping diaries or through dialogue journals).Asking learners to provide a synchronic commentary on how they accomplished a task (i.e., to talk about their thoughts to an interviewer while they were doing the task).Tracking learners (through the use of a computer) on a variety of measures.

3.6 Strategies and cognitive processes3.6.1. Observations: frequently take place within a classroom context.3.6.2. Introspective Measures: which tap participants' reflections on mental processes, originated in the fields of philosophy and psychology. Verbal reporting is a special type of introspection:Self-report: With self-report data, one can gain information about general approaches to something.Self-observation: In self-observation, a learner reports on what she or he has done. In self-observation, a learner reports on what she or he has done.Self-revelation (also known as "think-aloud"): A participant provides an ongoing report of his or her thought processes while performing some task.

3.6 Strategies and cognitive processes3.6.2. Introspective Measures: Stimulated Recall: It is a means by which a researcher, in an effort to explore a learner's thought processes or strategies, can prompt the leaner to recall and report thoughts that she or he had while performing a task or participating in an event.Think-Alouds or Online Tasks: individuals are asked what is going through their minds as they are solving a problem or completing a task.Immediate Recalls: used to elicit data immediately after the completion of the event to be recalled. It can be distinguished from stimulated recall in that it must occur immediately after the event, and it can be distinguished from online recall in that it does not occur simultaneously with the event.

3.7. Sociolinguistic/pragmatics-based research Both sociolinguistics and pragmatics are the study of language in context. Thus, they emphasize social and contextual variables as they affect the learning and production of a second language. Naturalistic Settings: Researchers can, of course, attempt to set up situations in which certain language events will recur. Elicited Narratives: There are a number of ways to do this. One can, of course, ask participants to tell a story about some past event or future plans in order to elicit these tenses.

3.7. Sociolinguistic/pragmatics-based research Silent Film: One way to elicit data is through the retelling of a silent film. The idea is to give learners a uniform prompt from which to speak. Film Strips With Minimal Sound: These can be used in the same way as silent films by turning the sound off. It is important that learners not be influenced by the speech of either their native language or the target language. Picture Tasks: This type of task can also be used by a single individual whose task it is to put the story together on his or her own.

3.7. Sociolinguistic/pragmatics-based research Discourse Completion Test (DCT): DCTs are implemented most frequently in writing, with the participants being given a description of a situation in which the speech act occurs. After the description, there is usually blank space where the response is required. (it is in written format)

Role Play: In general, there are two types of role plays: open and closed. Closed role plays are similar to discourse completion tasks but in an oral mode. Participants are presented with a situation and are asked to give a one-turn oral response. Open role plays, on the other hand, involve interaction played out by two or more individuals in response to a particular situation.

3.7. Sociolinguistic/pragmatics-based research Video Playback for Interpretation: In pragmatics research we might be interested in how people react to pragmatic infelicities. For example, how might the native speaker professor have reacted in the following situation and why? NNS: I have a favor to ask you. NS: Sure, what can I do for you? NNS: You need to write a recommendation for me.

3.8. Questionnaires and surveysBrown (2001) defined questionnaires as “any written instruments that present respondents with a series of questions or statements to which they are to react either by writing out their answers or selecting them among existing answers”. Survey, typically in the form of questionnaire, is one of the most common methods of collecting data on attitudes and opinion from a large group of participants. It has been used to investigate a wide variety of questions in second language researchQuestionnaires allow researchers to gather information that learners are able to report about themselves, such as their beliefs and motivations about learning or their reactions to learning and classroom instruction and activities

3.8. Questionnaires and surveysIn addition to different varieties of questionnaires, two types of questionnaire items may be identified: closed and open ended.A closed-item question is one for which the researcher determines the possible answers, whereas an open-ended question allows respondents to answer in any manner they see fit. Closed-item questions typically involve a greater uniformity of measurement and therefore greater reliability. They also lead to answers that can be easily quantified and analyzed.Open-ended items, on the other hand, allow respondents to express their own thoughts and ideas in their own manner, and thus may result in more unexpected and insightful data.

Concerns related to the analysis of questionnaire data

Questionnaire usually do not provide a complete picture of the complexities of individual contexts.

Even though it is often assumed that researchers can control or eliminate bias by using questionnaires, it is also possible, as with any type of elicitation device, that the data elicited will be an artifact of the device

To maximize the effectiveness of the questionnaire, researchers should try to achieve the following: Simple, uncluttered formats Unambiguous, answerable questions Review by several researchers Piloting among a representative sample of the research population

Existing databasesIn addition to the many elicitation techniques we have discussed in this chapter, there are existing databases as well, consisting of data that have already been collected, transcribed, and often analyzed. CHILDES (MacWhinney 2000), which focuses on spoken

language.IRIS