Intro Chapter II

7/29/2019 Intro Chapter II

http://slidepdf.com/reader/full/intro-chapter-ii 1/105

1

Chapter One

INTRODUCTION

Listening has gained much attention both in research and in language pedagogy

as it has changed its role from a passive activity which deserved less class time to an

active process through which language acquisition takes place (Vandergrift, 2004).

Listening is now widely accepted as an essential skill that enables language

acquisition to take place, both in mother tongue and in second or foreign language

(Rost, 2002). The learning environment in second language acquisition (SLA) or

foreign language learning is not as supportive as first language acquisition, though. It

is stressed that for a person to learn a second language, three conditions were required:

1) motivation to learn the language, 2) speakers of the target language who are able to

provide support and input, and 3) a social setting which provide the learners with

sufficient exposures to the target language (Wong-Fillmore, 1991). As listening is

required in two of the three requirements, it is evidently that it is worth to be taught

explicitly in its own right. To help the students become proficient in listening,

taxonomies of listening skills that underlie the process of listening comprehension

were proposed. While the taxonomies of listening skills help us understand the

mysterious listening comprehension process, it also sheds light in listening pedagogy

(Buck, 2001). Following the taxonomies of listening skills, the teachers are able to

know which part of listening process is emphasized by the researchers and will pay

more attention to it in their teaching.

With the help of advanced computer technology and the advent of Internet, it is

now easier for both the teachers and learners to have access to spoken input of the

target language, in our case, English. Many listening websites and multimedia

CD-ROMs are now available for teachers to include as teaching materials and for the



2

learners to use as self-study aids. There have been studies investigating the usefulness

of incorporating CALL and listening instruction. Cheng (2006), for example, after

implementing a listening website (Randall’s ESL cyber listening lab) into her listening

instruction, it was found that the learners held very positive attitudes to the website

and also became motivated to learn. Likewise, Ramirez and Alonso (2007)

incorporated digital stories in the language course of a group of young learners,

attempting to investigate how the technology can enhance the learners’ listening

comprehension. A positive finding was obtained in a pretest and posttest comparison.

Students who had access to digital stories performed significantly better than those

who did not. These studies warranted the benefits that CALL are able to bring to

language teaching and learning, especially in the field of listening.

CALL is not enough to make language learning and teaching effective, though.

To be so, a proper learning syllabus has to be designed. That is to say, the materials

should be sequenced in an appropriate order and systematic training should be carried

out. While grading mechanisms are available for selecting reading texts (e.g., Flesch

reading ease score in Flesch, 1949 and Textladder in Ghadirian, 2003), and are even

extended to foster vocabulary acquisition (i.e., Text Grader in Haung, 2004), those

specifically for ‘listening texts’ are still virtually unavailable. Without such a

mechanism, some studies adopted readability measures to determine the difficulty

levels of listening texts. Smidt and Hegelheimer (2004), for example, used Flesch

reading ease score to grade the difficulty levels of three academic lectures, which

were used as assessment tools in their study to ensure text comparability. It seems that

to adopt the approach in reading research to audio materials is feasible.

Given the importance of listening in terms of second language acquisition, the

advantages CALL brings to language learning and teaching, and the pedagogical

needs of a grading mechanism for listening texts, this study attempts to address the



3

following issues: first of all, the development and the assessment of a grading formula

manipulating the weights of three difficulty measures (i.e., speech rate , academic

word ratios, and syntactic complexity); secondly, the impact of incorporating graded

online listening materials with a listening course on the students’ development of

listening ability; and thirdly, the students’ attitudes towards the use of the online

listening materials. A grading formula for listening texts was thus developed, and with

its help, a website was constructed in which sixty-six graded listening texts were

sequenced from easy to difficult with their MP3 files, transcripts, and comprehension

check questions. Meanwhile, the comprehension questions were designed to provide

the learners with systematic training of five target listening skills: listening for main

ideas, listening for details, interpreting the speaker’s intent, making inferences, and

summarizing. Each of them was considered an important element underlying

successful listening comprehension (Buck, 2001; Weir, 1993). Then, we infused this

website into a listening and speaking course for a group of EFL college level learners

to do self-study. It is expected that this study could shed a dawning light on grading

listening texts, and provide a preliminary understanding of how online graded

listening materials could help development of listening skill, and how CALL can

benefit listening instruction.



4

Chapter Two

REVIEW OF LITERATURE

2.1 Overview

Listening is the most essential skill of language learning as virtually all children

learn to listen as part of their first language (L1) acquisition process. Just as the

fundamental role listening plays in L1 acquisition, it is by no means less important in

second language (L2) learning (Rost, 2002). Due to the growing emphasis on

communicative competence in language learning and teaching in recent decades,

more and more studies were carried out to enhance the teaching and learning of

listening. Along the same vein, much research effort has been devoted to exploring

the process of listening comprehension (e.g., Anderson, & Lynch, 1988; Lynch, 1998;

Rost, 1990, 2002), identifying the factors that affect listening comprehension

(Brindley, & Slatyer, 2002; Chiang, & Dunkel, 1992; Flowerdew, 1994; Shohamy, &

Inbar, 1991; Teng, 2002), and finding efficient teaching approaches of listening

(Berne, 2004; Hinkle, 2006; Mendelsohn, 1998; Vandergrift, 1999). At the same time,

with the rapid development of technology, computers and the World Wide Web have

been applied to language courses in various ways. In terms of the teaching of listening,

the computer technology has brought several benefits to the teaching and learning of

listening. First of all, the access to listening materials increases substantially; this, in

turn, creates considerable chances for the learners to learn and practice. In addition,

availability of various websites also makes it much easier for instructors to get

supplementary listening materials. Secondly, listening materials become more

authentic, more reflective to the real-world spoken language. Fianlly, learners can

engage in their learning more actively. With a multimedia computer equipped with

internet service, learners can have access to listening materials anytime and anywhere



5

as many times as they wish. Hence, compared to traditional listening courses where

instructors play tapes or CDs, and learners listen and answer comprehension questions,

computer-based and web-based listening instruction can obviously create a more

meaningful and active learning environment that can in turn contribute to better

listening.

In this chapter, a brief introduction of listening comprehension will be offered,

followed by a discussion of factors that contribute to listening difficulties and thus

affect listening comprehension. Then, an overview of the changes in listening

instruction will be given. Additionally, the recent effort of incorporating computer

technology and the teaching and learning of listening will be introduced. Last, a

discussion on material selection of listening materials will be presented.

2.2 An Overview of Listening Comprehension

Many researchers have given their definitions of listening comprehension (e.g.,

Anderson, & Lynch, 1988; Brown, & Yule, 1983; Rost, 1990, 2002). For example,

Brown and Yule (1983) explained listening comprehension as a process of

understanding, repeating what was heard, figuring out the meaning of an exact word,

and then knowing what an expression refers to. Rost (2002) described listening

comprehension as a process of trying to understand what spoken language refers to in

one’s experience or in the real world.

In order to understand an utterance, various types of knowledge must be applied

to decode and interpret the incoming information. Buck (2001) concludes that the

knowledge involved in the listening process is of two types: linguistic knowledge (i.e.,

lexis, syntax, semantics, and discourse structures) and non-linguistic knowledge (i.e.,

general knowledge of the world, knowledge of the listening context, and personal

experience). How these two types of knowledge and their sub-knowledge are applied



6

to the acoustic information has aroused much debate. Among various perceptions, the

bottom-up view and the top-down view are the most important and classic. These two

perspectives are different mainly in their beliefs of the order in which different types

of knowledge are applied during the comprehension process. In the bottom-up view,

listening comprehension is seen as a process of passing information from stage to

stage. Acoustic input is first decoded into phonemes (the smallest sound unit), then it

is identified as an individual word, followed by the sentential level and then semantic

level. Finally, the listeners pull in their own experience or the communicative context

and understand what the input refers to. In other words, the order of the knowledge

applied in the bottom-up listening process is like a one-way street, which is not

changeable. Under this view, listening for details is usually trained by teachers. On the

other hand, in the top-down view, the order in which different types of knowledge

come into play is not in a fixed manner, rather, they may appear in any order or even

simultaneously. In addition, they are all capable of interacting and influencing with

one another. To illustrate, when hearing a sentence such as ‘It’s raining heavily,

remember to take an ________’ we can confidently expect that the next word being

umbrella. While we are applying our linguistic knowledge to comprehend the

utterance, our general knowledge of the world comes into play, and we know that

when it is raining, we use umbrellas to avoid getting wet. At the same time, our

linguistic knowledge tells us ‘an’ is the article for vowel-initiated words. Thus, as the

example shows, we do not have to process everything in the acoustic input to get its

idea as opposed to the bottom-up view in which listeners have to process every sound

to comprehend the information. In the top-down view, we only take what we need and

what is necessary.



7

2.3 Listening Difficulties

As have been noted by Rost (2002), listening is essential to language

development. Yet, it has been very challenging for L2 learners to learn. To help

learners to become skillful listeners, factors contributing to learners’ listening

difficulties have been identified in many studies, for example, text types (e.g.,

Brindley, & Slatyer, 2002; Shohamy, & Inbar, 1991; Su, 2003), speech rate (e.g., Blau,

1990; Flaherty, 1979; Griffiths, 1992; Rixon, 1986; Teng, 2002), and task types (e.g.,

Cheng, 2003, 2004; Teng, 1998a), syntactical complexity (e.g., Chiang & Dunkle,

1992; Dunkel, 1988), topic familiarity (Carrell, 1983; Carrell & Eisterhold, 1983,

Connor, 1984), and English proficiency (e.g., Vandergrift, 2006). In this paper, speech

rate, text types, syntactic complexity, and also task types, are discussed.

2.3.1 Speech Rate

Speech rate has profound effect on listening comprehension (Flowerdew, 1994,

Rixon, 1986). Concerning temporal factors such as speech rate and pausing time, both

Teng (2002) and Flowerdew (1994) noted that the slower the speech is, and the more

pauses a passage has, the better comprehension level listeners would be able to

achieve. This notion is easy to understand in the sense that with faster speech rate,

listeners would have less time to process the incoming information, hence, results in

unsuccessful comprehension. Based on Rixon’s (1986) remark, L2 learners need more

time than the native speakers to process each piece of information even when they

have no difficulties in understanding, as a result, “any very fast delivery will therefore

place an extra strain on the learner” (p. 58). To examine how speech rate can affect

learners’ comprehension, Griffiths (1992) compared the comprehension levels of 24

low-intermediate Japanese EFL learners who were assigned to listen to three passages

each in three different speech rates—slow, average, and fast—measured in the unit of



8

“words per minute” (wpm). The lower rates ranged from 126 to 128 wpm, average

from 188 to 189 wpm, and fast from 245 to 257 wpm. The results showed that

learners who listened to the slow version of the passages obtained significantly higher

scores than those who listened to normal and fast versions in the post-listening test.

To answer the question of at what rate a listening text is regarded as fast,

different listening specialists have proposed different standards for distinguishing fast

speech from slow speech. For example, Griffiths (1992) adopted the criteria in which

an average rate 125 wpm is slow, 185 wpm is normal, and 250 wpm is fast. Rubin

(1994), in her well-known review, noted that most research accepts 165 to 180 wpm

as normal speech rate. However, regardless of different interpretations of speech rate,

most research accepts 165 to 180 wpm as the normal speech rate (Rubin, 1994).

2.3.2 Text Types and Syntactic Complexity

Text type has also been noted as a factor affecting listening comprehension

(Brindley, & Slatyer, 2002; Rubin, 1994; Shohamy, & Inbar, 1991; Su, 2003).

Listening texts are often categorized into conversation and monologue, or

interactional and transactional (Rixon, 1986). Based on Rixon, “conversation” is

defined as “any series of spoken exchanges among a small group of people in which

the contributions are reasonably balanced in terms of quantity and are reasonably

coherent,” monologue, on the other hand, is “a speech by a single person” (p. 6).

However, to distinguish interactional and transactional, rather than the number of

people involved in, the purpose of the speech is the focus. Brown and Yule (1983)

maintained that the emphasis of a transactional passage is on the exchange of

information or the use of language to get things done, whereas in an interactional

passage it is on the use of language for establishing or maintaining social contact.

Conversations, or dialogues, as they contain information of lower density compared to



9

monologues such as lectures or news broadcasts, are considered to be easier to

listeners than monologues. In Brindley and Slatyer (2002), it is disclosed that

dialogues are easier than lectures as well as news broadcast.

One factor that comes along with text type is syntactic complexity, which is also

documented as an attribute of listening difficulty. While written texts generally

contain more syntactically complex sentences and less redundancies or repetitions

compared to spoken texts, it is assumed that texts similar to written texts are of higher

difficulty level than texts similar to spoken texts (Rubin, 1994; Chang, 2004).

Similarly, Rubin and Raforth (1984, cited in Shohamy & Inbar, 1991) introduced the

notion of ‘listenability,’ which is defined as a ‘function of orality in language,’ or the

degree to which a passage exhibits features common in oral language. Accordingly, it

seems to suggest that the more listenable or orally-oriented a text is, the easier it will

be for the listeners to comprehend. Along this line, Shohamy and Inbar (1991)

conducted a study investigating to what extent the ‘orality’ elements could affect

learners’ performance in listening comprehension tests. In their study, 150 secondary

school EFL learners were randomly assigned to listen to passages of two different

topics. For each topic, three different text types were prepared: news broadcast,

lecture and consultative dialogue. Of these three types of texts, news broadcast is near

to the literate end, lecture at the middle and consultative dialogue close to the oral end

(see Figure 2.1 below).



10

Figure 2.1 Correspondence of text types and the degree of ‘orality.’ (Shohamy & Inbar, 1991, p. 29).

LITERATE ORAL

1 2 3

After listening, the learners were given an immediate posttest to measure their

comprehension level. The results showed that, regardless of the topical differences,

learners who listened to the dialogue version of the topic got the highest scores in the

posttest; the news broadcast group, on the other hand, got the lowest scores. Hence,

the amount of ‘orality’ in a text does seem to be an important factor concerning the

difficulty level of listening texts, namely, the more oral a text is, the easier it will be,

and vice versa. Accordingly, as oral texts are syntactically less complex, the finding

also seemed to suggest the notion that higher syntactic complexity contributes to

higher difficulty level of listening text, and vice versa.

2.3.3 Task Types

Anderson and Lynch (1988) maintain that “[d]ifferent tasks present the listener

with varying degrees of complexity” (p. 59). Therefore, when considering factors

influencing the listeners’ performance, the nature of the task itself is also one that we

need to bear in mind (e.g., Brindley, & Slatyer, 2002; Teng, 1998a; Rubin, 1994;

Shohamy, & Inbar, 1991). Buck (2001), for example, concludes several guidelines of

controlling task difficulties (p. 151):

y Tasks that require processing less information tend to be easier than tasks

which require processing more information.

y Tasks that require processing information from just one location in the text

tend to be easier than tasks which require integrating information scattered

newsbroadcast

lecture consultativedialogue



11

throughout the text.

y Tasks that require recalling exact content tend to be easier than tasks which

require extracting the gist, or making a summary.

y Tasks that require simply selecting information tend to be easier than tasks

which require separating fact from opinion.

y Tasks that require information that is relevant to the main theme tend to be

easier than tasks which ask for irrelevant detail.

y Tasks that require immediate responses tend to be easier than tasks which

require a delayed response.

While these guidelines offer directions for test developers, they also help us to see

how different types of task may affect listening difficulties. For example, based on the

third guideline “tasks that require recalling exact content tend to be easier than tasks

which require extracting the gist, or making a summary”, it is logical to assume that

multiple choice questions are easier than short answer questions and questions

requiring listeners to summarize the text. Similarly, Brindley and Slatyer (2002) also

suggest that “items requiring only recognition are easier than those requiring retrieval

and production” (p. 377). Several studies have also confirmed this proposition (e.g.,

Brindley & Slatyer, 2002; Cheng, 2004; Teng, 1998a; Shohamy, & Inbar, 1991).

Shohamy and Inbar, in their investigation of the influence of text type and task type,

found that regardless of the topic differences, listeners performed significantly better

on questions requiring them to locate details, understand words with contextual

support, and recognize facts (‘local questions,’ Shohamy, & Inbar, 2002) than

questions requiring synthesizing information, drawing conclusions, or making

inferences (‘global questions,’ Shohamy, & Inbar, 2002). Likewise, Teng’s (1998a)

investigated how task type affected 186 low-intermediate level college students’

listening performance. The students were randomly assigned to listen to two types of



12

texts, and were assessed their comprehension using a three-version posttest:

multiple-choice questions, short answer questions, and cloze questions. Among the

three types of questions investigated, students who did the multiple-choice version

were found to get the highest score, and those who did the cloze version got the

lowest.

2.4 Teaching Listening

As communicative competence has been underscored in language teaching and

learning, listening has received increasing attention in language classrooms. The note

from Morley below can describe the change of the status of listening in language

learning and teaching:

At one time, listening was assumed to be a passive activity, meriting

little classroom attention. Now listening is recognized as an active process,

critical to L2 acquisition and deserving of systematic development as a

skill in its own right. (Morley, 1999, cited in Vandergrift, 2004, p. 3)

As the role of listening changed, so did the approach applied to teach listening.

From the “listening to repeat” audio-lingual approach, “question-answer”

comprehension approach, to “real-life listening” communicative approach, listening

instruction has become to underscore the ‘process’ (learning to listen) instead of the

‘product’ (listening to learn) of listening (Vandergrift, 2004). In the 1970s, listening

instruction largely emphasized learners’ development of the bottom-up process, that is,

the ability to identify words, sentence boundaries, individual sounds, and sound

combination. Then, in the following decade, because of the shifted view of second

language listening from language-based to schema-based, the teaching of listening

changed its focus to learners’ development of top-down process, namely, activating

their background knowledge such as topic familiarity, discourse clues, and pragmatic

conventions. However, none of these two teaching foci was proved to obtain



13

successful consequences by itself. In recent years, listening pedagogy has come to

underscore the instruction of learners’ use of strategies to enhance their learning

process (Hinkel, 2006).

2.4.1 Listening Strategies

Since the early 1990s, research on strategy training of second language learners

has become very active. Oxford (1990), for example, identified and categorized the

general language learning strategies. The strategies are classified to two groups: direct

strategies and indirect strategies. On the other hand, O’Malley and Chamot (1990) had

different categorizations: cognitive strategies, metacognitive strategies, and

socio-affective strategies. Regarding listening strategies, many researchers have tried

to identify efficient and beneficial strategies used by more proficient listeners in order

to enhance strategy-based listening instruction (e.g., Berne, 2004; Smidt, &

Hegelheimer, 2004; Teng, 1998b). Rost (2002), for example, summarized five most

commonly recognized strategies used by successful second language listeners:

predicting, inferencing, monitoring, clarifying, responding, and evaluating.

As the strategy-based approach has become more active in language teaching,

more and more studies started to highlight the importance of strategy teaching in

listening courses (e.g., Mendelsohn, 1995, 1998; Vandergrift, 1999, 2004; Vogely,

1995).

2.4.2 Listening Skills

Listening is perceived as a complex and multidimensional process. To

understand the complex process, a number of theorists have attempted to describe

listening in terms of taxonomies of skills (Buck, 2001). One common taxonomy is

dividing listening into two stages: comprehending and application (Carrol, 1972;

Clark & Clark, 1977). At the first stage, the learners process the linguistic information



14

in the message, and in the second stage, they utilize the information in a

communicative context. Given the growing interest in the communicative approach in

language teaching, there are also many taxonomies of listening skills described in

communicative terms. Compared to the available taxonomies to date, Weir’s

classification (1993) seems to be more comprehensive (See Table 2.1). While many of

the components in the taxonomies are essential in listening, there has not been

evidence suggesting that any of the taxonomies contains a complete description of

listening process. These taxonomies are important because they tell us what are

considered important in listening comprehension, and hence shed light in the teaching

of listening.

Table 2.1 A list of listening skills (Weir, 1993, cited from Buck, 2001, pp. 54-55)

Direct meaning comprehension

Listen for gist

Listening for main idea(s) or important information; and distinguishing that from

supporting detail, or examples

Listening for specifics, including recall of important details

Determining a speakers’ attitude or intention towards a listener or a topic

Inferred meaning comprehension

Making inferences and deductions

Relating utterances to their social and situational context

Recognizing the communicative function of utterances

Deducing meaning of unfamiliar lexical items from context

Contributory meaning comprehension

Understanding phonological features

Understanding grammatical notions such as comparison, cause, result, degree etc.

Understanding discourse markers

Understanding the main syntactic structure of clauses or idea units

Understanding cohesion, especially reference

Understanding lexical cohesion, especially lexical set membership and collocations

Understanding lexis

Listening and taking notes

Ability to extract salient points to summarize the textAbility to select relevant key points



15

As shown in Table 2.1, Weir (1993) divided listening process into four main

sections, and each with further detailed description: direct meaning comprehension,

inferred meaning comprehension, contributory meaning comprehension, and listening

and taking notes. The list is not only a description of listening process, but also a

comprehensive checklist of operations for listening tests.

2.5 CALL and listening instruction

Since 1960s, computers have been used in language education. The use of

computers could be divided into three types: behaviorist computer assisted language

learning (CALL), communicative CALL, and integrated CALL (Warschaur, & Healey,

1998). Behaviorist CALL, as the name suggests, was derived from the behaviorist

language learning theory. Where the tasks learners were required to perform via

computers were repetitive drill practices. Communicative CALL, which is derived

from the communicative teaching approach, allowed learners more freedom to engage

in the learning activities whenever they want via their own computers in their own

houses, and thus learners become more active in their language learning. The tasks in

communicative CALL mainly focused on manipulations of linguistic forms.

Integrated CALL, on the other hand, aims at integrating the four skills and pulling in

fuller technology support into learning process. Learners can download audio or video

clips from the World Wide Web (WWW) into their computers. With the rapid

technology improvement, World Wide Web has become the most popular facilities

available to students to obtain information and help them in their learning process

(Flowerdew, & Miller, 2005). Under this circumstance, learners can free themselves

from the constraints of learning time and space as opposed to traditional learning;

they are also granted more opportunities to learn. The type of CALL in this current

study is between the communicative CALL and the integrated CALL as the study



16

concerns the training of a single skill; yet, the CALL materials offered via the internet

is meant to provide learners with more opportunities to learn and practice.

Regarding listening instruction, there have been some examples of integrating

technology into language classes (e.g., Grgurovic & Hegelheimer, 2007; Ramirez &

Alonso, 2007). In Grgurovic and Hegelheimer, multimedia learning materials were

infused into an EFL course to help learners comprehend academic lectures. Two types

of text support were provided: subtitles and transcripts. While the results showed that

learners tend to use subtitles more than transcripts, the researchers further warranted

that instructors should actively encourage and train their students in using such help

options to become more proficient. Likewise, Ramirez and Alonso incorporated

digital stories in the language course of a group of young learners, attempting to

investigate how the technology can enhance the learners’ listening comprehension. A

positive finding was obtained in a pretest and posttest comparison. Students who had

access to digital stories performed significantly better than those who did not. These

studies warranted the benefits that CALL are able to bring to language teaching and

learning.

To move further, there were also studies which incorporated listening websites

into listening classes. Cheng (2006), for example, integrated an existing topic-based

listening websites—Randall’s ESL Cyber Listening Lab

(http://www.esl-lab.com) —into a listening class as supplementary materials after

class. The website offers a considerable amount of audio clips for listening practice,

and scripts of the audio clips along with several multiple-choice comprehension

questions. The purpose of the integration was to understand how learners of different

proficiency levels perceive the role of such CALL materials in their learning of

listening. Participants were 162 freshmen who enrolled in a listening course at a

private university in Taiwan. These learners were categorized into three groups—high,



17

average, and low—in accordance with their scores of the listening section of

intermediate General English Proficiency Test (GEPT). Participants were required to

go to the website once a week and listen to the passages assigned by the instructor that

were related to the weekly topic of the course. In addition to listening to the passages,

learners were also asked to complete the comprehension questions on the website.

Also, they had to tape-record the scripts with their peers. A questionnaire was used to

measure learners’ attitudes toward the supplementary learning materials. The results

showed no significant differences among the learners’ attitude toward the CALL

materials with respect to their proficiency levels. Nonetheless, regardless of learners’

proficiency levels, most of the participants express positive attitudes toward the use of

the website. Some learners reported that using such CALL materials after class was

less stressful than doing exercises during class and thus enabled them to enjoy the

learning process more. Another similar study was conducted by Chen (2004), who

also set up a web-based listening center for learners to practice. When listening,

learners could adjust the speech rate of the audio clips. And post to the listening

passage, they would be presented a set of comprehension questions and a vocabulary

quiz. Most users found themselves satisfied with the system. Chen also suggested that

his listening website be incorporated into language classes.

These studies warranted the benefits CALL are able to bring to language

teaching and learning.

2.6 “Listenabiliry”—selecting appropriate listening materials

Computer and internet have brought about many advantages to learners in their

learning of listening. While the online materials offer the learners with considerable

learning opportunities, they also pose a challenge to the instructors of selecting

suitable materials for learners in different proficiency levels. Despite that everal



18

factors affecting the difficulty levels of listening texts have been identified (e.g.,

Chiang & Dunkel, 1992; Griffiths, 1992; Shohamy & Inbar, 1991; Teng, 1998a, 2002),

very few of them are operationalized as measures for material selection. For the

teaching of vocabulary through extensive reading, a similar problem has also been

identified. For example, Huang (2003), in her study of the effects of graded reading

on vocabulary learning, indicated that preparing suitable reading texts is often a

challenge for teachers as the materials chosen will significantly affect the learners’

learning results. When learners are faced with overwhelming complex authentic texts

in extensive reading, difficulties in comprehension, let alone vocabulary acquisition,

can be expected. Hence, choosing materials that is suitable for a particular group of

learners is of great importance. Some initial efforts have been devoted to addressing

the problem. Huang designed an extensive reading module, named Text Grader

(http://candle.cs.nthu.edu.tw, under Reading) , as a self-access basis for 38 college

participants in Taiwan to investigate their vocabulary gains through the program. The

module was developed based on an earlier and similar program named Textladder

(Ghadirian, 2003). With the help of word lists research and quantitative corpus

analyses using word frequency computer programs, Textladder was able to screen a

large number of texts and choose appropriate materials that meet learners’ levels.

However, Ghadirian did not use the materials on learners to look at the effectiveness.

With modification and a test on real EFL learners in Text Grader , Huang (2003)

filtered the texts with four word lists and selected 16 articles: the General Service

Word List, a local Senior High School Students’ Word List, the University Word List,

and an Exposed Word List. The articles were sequenced from easy to difficult with

another control of times of targeted word exposure. Familiar words and unfamiliar

targeted words are identified for the study. The reading program was used by 38 EFL

college learners at home for a period of 12 weeks. The results suggest that the reading



19

materials facilitate learning of vocabulary in an extensive reading context; in addition,

learners also held a positive attitude toward the program. The two studies provided

preliminary but useful methods of manipulating learning materials based on word

frequency and sequencing of texts.

2.7 Purposes of the Study

Given the importance of listening in terms of second language acquisition, and

the benefits brought by computer technology to language learning, we attempt to

investigate the benefit of CALL, specifically the Internet use, on the development of

listening skill. Additionally, even though online listening materials are easily

accessible, their quality poses a question to language teachers who are motivated to

apply such materials. And, while factors contributing to listening difficulties have

been identified in much literature, there is virtually none, to our limited knowledge,

attempting to operationalize the factors for pedagogical concerns. Last, as grading

mechanisms are available for reading texts, the same mechanisms for listening texts

are still unavailable.

In light of the pedagogical needs and the research gaps, the purposes of the

present study are as follows: first, to examine to what extent online listening materials

could benefit the learners’ listening ability; second, to provide a pedagogical model

for listening teachers to choose supplementary listening materials; and finally, to

investigate the learners’ perceptions of using online listening materials in enhancing

listening ability.



20

Chapter Three

METHOD

3.1 Overview

In the present study, a listening website (called Freshmen Listening,

http://formoosa.fl.nthu.edu.tw/moodle2) using a free course management system was

constructed and infused into a freshmen listening and speaking course for eleven

weeks in the fall semester of 2006. The goal of the course was to enhance the learners’

speaking and listening abilities. Learners in this study were asked to visit the website

to practice their listening skills by doing the listening tasks provided in the website.

With assessment data and the learners’ logs, we hope to understand the feasibility of

sequencing listening materials with the help of a proposed formula as a criterion of

“difficulty” of texts by manipulating the weights of three difficulty measures. Also,

we hope to understand the change the learners might show after using the website for

an eleven-week period, namely, the usefulness of the online training program. In this

study, a one-group pretest and posttest research design was adopted.

This chapter presents the research method we used to construct the website, and

the design of the experiment we adopted to probe into the research questions. The

details of the participants, instructional materials, research instruments, process of

data collection, as well as methods of data analyses are reported.

3.2 Participants

Participants involved in this study were thirty-one EFL learners enrolling in a

freshmen listening and speaking course given by the Department of Foreign

Languages and Literatures in a local university. The class met once a week, each

lasting for 100 minutes. Thirty of the learners were freshmen who just graduated from

high schools and started their first semester in the university, the other one was a



21

junior student in the same department. Proficiency levels of the learners ranged from

low-intermediate to advanced as everyone of them had received formal English

instruction in high school for at least 6 years.

The access to computer and Internet was easy as the majority of the participants

had their own personal computers or notebooks. Moreover, the school dormitory, at

which all first-year students stayed, was equipped with broadband Internet service.

Learners were fairly familiar with using computer and connecting to the Internet, yet,

their previous experience of using listening website differed substantially. In a

questionnaire which tapped into their former experience about listening websites, only

five of them reported having such experience. The websites they used were mainly

international and local news websites such as CNN, BBC, and ICRT. More, English

learning websites like Studio Classroom were also included.

3.3 Development of the Listening Website

The listening website, Freshmen Listening , which was infused into the listening

and speaking course was set up on an open-source course management software,

MOODLE (Modular Object-Oriented Dynamic Learning Environment). MOODLE,

an e-learning platform designed to help material developers create online course, can

provide a user-friendly interface for both the instructors and the learners as the links

were well constructed and clearly described (see Figure 3.1 for a screenshot of the

website).



22

Figure 3.1 A screenshot of “Freshmen Listening”—the listening website

In the listening website, sixty-six listening texts were uploaded for the use of the

eleven-week training project. To gather the 66 passages, more than 66 listening texts

and their MP3 files were collected first from various resources. They included English

learning magazine (CNN Interactive English Magazine, Live ABC; ALL plus

Interactive English Magazine, Live ABC), an online listening website (Randall’s

cyber listening lab http://www.esl-lab.com), and TOEFL CBT and iBT preparation

books ( Master the TOEFL 2005, 2004, Thompson Learning, Inc. and The official

guide to the new TOEFL iBT , 2006, Educational Testing Service). Among the

materials that were collected, those which were too easy or too difficult for the

participants were weeded out, so did those whose topics were not interesting enough.

With these 66 listening passages and their MP3 files, a formula formed based on

a pre-service English teacher’s perspective to generate a difficulty score representing



23

the difficulty levels of the listening texts was developed and adopted to arrange the

texts in proper order, namely, from easy to difficult, for the participants to practice. As

mentioned in the previous chapter, factors that cause listening difficulty were

identified in the literature, yet, very little, to our limited knowledge, tried to

operationalize them for pedagogical concerns. Our study attempted to bridge the gap

by operationalizing three difficulty measures—speech rate, academic word ratios, and

the Flesch Reading Ease score—to develop a formula that could serve as a grading

mechanism for our listening texts as in Flesch Reading Ease score, a readability

measure, to reading texts. The three measures as well as the formula are discussed in

detail in the following section.

Speech rate: Speech rate, namely, the average word number spoken in a minute,

is the basic index reporting how fast a text is spoken. Flowerdew (1994) noted that the

speech rate of a listening passage has profound effects on listeners’ comprehension

level in the way that higher rate results in worse comprehension while lower rate

contributes to better comprehension. In the present study, the rate of each listening

text was computed by dividing the total words by the total time—in the unit of

minute—of the listening excerpt. For example, the first or the easiest passage had 160

words in total, and one minute and twenty-nine seconds duration, making its speech

rate 107.87 words per minute (wpm, i.e., 160/89*60). The rate range for the 66 texts

was between 90 to 242 wpm.

Academic word ratios: As the name suggest, this measure is an indication of the

density of academic word in the texts. In this study, the academic words were defined

using the Academic Word List (AWL, Coxhead, 2000). The AWL has around 10%

occurrences of the total words in written academic texts. It is suggested that by

learning the AWL words, the learners can develop their academic literacy more

efficiently (Nation, 2001). In the current study, it is hypothesized that the more



24

academic word a text contains, the more difficult it becomes. Therefore, of each

listening passage, the percentage of academic words was computed. This was done by

dividing the number of academic words in a passage by the number of total words.

For example, the first/easiest passage on the website has one academic word out of its

116 words (Note what we mean by ‘word’ here is word ‘type’ instead of word ‘token’),

thus its ratio was 0.94 % (i.e., 1/106). The range for the 66 texts was from 1% to 12%

approximately.

Flesch reading ease score: Flesch Reading Ease Score (Flesch, 1949) is a

well-known readability measure which generates scores for reading texts indicating

how easy or difficult a reading text is to comprehend. The score, ranges from 0-100,

are computed with considerations of the number of syllables, words, and sentences,

with higher score indicating easier passages and vice versa. In the case of English,

texts that score from 90 to 100 are believed to be considerably easy for average native

fifth graders, 60-70 for eighth and ninth graders, and 0-30 for college students (for

more detailed information, refer to ‘Wikipedia,’

http://en.wikipedia.org/wiki/Flesch-Kincaid_Readability_Test). The scores for the 66

texts were calculated with the help of the inherent function in Microsoft WORD,

which can be found in grammar/spelling check under Tool. For example, the

first/easiest passage among the 66 obtained an ease score of 51.6 using MS WORD

(see Figure 3.2 for the readability computation). The scores for the 66 passages

ranged from 32 to 92.



25

Figure 3.2 The readability computation of the first/easiest passage

Each of the sixty-six passages was computed concerning its speech rate,

academic word ratio, and reading ease score using the aforementioned methods. Table

3.1 summarizes the ranges of the three measures of the sixty-six listening texts.

Table 3.1 The ranges of the three measures of the 66 listening texts

Speech Rate Academic Word Ratios Flesch Reading Ease score

90 - 242 (wpm) 1% - 12% 32 - 92

3.3.1 The Proposed Formula—“Difficulty Score”

In the present study, we tried to operationalize the concept of difficulty in a

listening text with the three measures. With the three measures, the ‘difficulty

score’—a composite variable by weighting differently on the three measures—was

generated by using a proposed formula:

Difficulty score＝speech rate × 0.5＋academic word ratio × 0.3－reading ease

score × 0.2



26

In the formula, speech rate was assigned 50% of the weights as it was the

measure better supported in the literature (Flowerdew, 1994; Griffiths, 1992; Teng,

2002). As academic word ratios and reading ease score were borrowed from

readability measure, they were assigned 30% and 20% of the weights respectively

(see Figure 3.3 below for a summary of the manipulation of the weights of the three

measures). Even as a readability measure, one study (Smidt & Hegelheimer, 2004)

adopted Flesch Reading Ease score as a mean to ensure the comparability of three

different listening texts. Smidt and Hegelheimer used Flesch reading ease score and

Flesch-Kincaid grade level—another well-known readability measure—to verify the

comparability of different video lectures used in their pretest, posttest and delayed

posttest in their study which aimed at examining the effects of online academic

lectures on ESL learners’ listening comprehension. As the scores for the three texts

were similar to each other, the researchers concluded that the different videos used in

the three tests were comparable.

Difficulty scores generated from the formula ranged from 0 to 100, with higher

difficulty scores signifying harder texts, and lower, easier. After we computed the

score for each of the 66 texts, the texts were sequenced from 1 to 66—easy to

difficult—and presented to the learners on the website every week, six for each, based

on the sequence. Table 3.2 presents the computation results for all the 66 passages in

terms of the three measures.



27

Figure 3.3 The manipulation of the weights of the three measures

50%

30%

20%

Table 3.2 The computation results of the three measures of the 66 texts

Reading

ease score

Speech

rate

Adjusted

rate

Academic

word ratios

Adjusted

ratios

Difficulty

score

Presentation

sequence

TXT.69 51.6 107.87 44.29 0.94 7.93 14.21 No.1

TXT.18 61 112.44 46.46 1.88 15.79 15.77 No.2

TXT.25 62.5 112.81 46.62 2.46 20.72 17.02 No.3

TXT.48 92.3 173.29 71.61 0.00 0.00 17.34 No.4

TXT.30 56.9 100.79 41.65 3.14 26.42 17.37 No.5

TXT.23 52.4 106.56 44.03 2.84 23.86 18.69 No.6

TXT.21 59.3 105.96 43.79 4.09 34.43 20.36 No.7

TXT.36 44.9 121.40 50.16 2.17 18.28 21.59 No.8

TXT.26 50.1 101.75 42.05 4.49 37.80 22.34 No.9

TXT.52 80.4 177.09 73.18 1.37 11.52 23.96 No.10

TXT.28 54.7 115.77 47.84 4.66 39.22 24.75 No.11

TXT.17 41.7 97.71 40.38 5.30 44.60 25.23 No.12

TXT.54 75.9 177.25 73.24 1.53 12.87 25.30 No.13

TXT.22 42.9 104.58 43.21 5.07 42.63 25.82 No.14

TXT.24 59.7 105.21 43.48 6.45 54.26 26.08 No.15

TXT.57 68.8 184.52 76.25 0.79 6.62 26.35 No.16

TXT.29 35.7 90.27 37.30 6.10 51.28 26.90 No.17

TXT.13 70 167.42 69.18 2.68 22.53 27.35 No.18

TXT.20 42.4 103.92 42.94 5.78 48.61 27.58 No.19

TXT.16 50.4 101.60 41.99 6.74 56.70 27.92 No.20

Speech rate

Flesch Reading

Ease score

Academic word

ratios



28

TXT.5 55.5 148.24 61.25 3.50 29.44 28.36 No.21

TXT.51 73.5 176.79 73.05 2.81 23.62 28.91 No.22

TXT.35 37.8 133.41 55.13 3.62 30.43 29.13 No.23

TXT.2 60.1 166.10 68.64 2.78 23.36 29.31 No.24

TXT.66 77.7 153.42 63.40 5.26 44.27 29.44 No.25

TXT.34 61.8 169.30 69.96 2.75 23.11 29.55 No.26

TXT.10 57.9 154.81 63.97 3.80 31.94 29.99 No.27

TXT.9 87.7 213.88 88.38 1.47 12.37 30.36 No.28

TXT.27 50.3 99.81 41.25 7.95 66.90 30.63 No.29

TXT.42 83.2 201.82 83.40 2.22 18.69 30.66 No.30

TXT.47 84.5 215.29 88.96 1.24 10.45 30.72 No.31

TXT.7 52.2 160.27 66.23 3.43 28.84 31.32 No.32

TXT.53 64.3 195.69 80.86 1.75 14.76 32.00 No.33

TXT.70 77.5 198.98 82.22 2.70 22.73 32.43 No.34

TXT.50 77.3 222.86 92.09 0.81 6.78 32.62 No.35

TXT.65 79.4 226.40 93.55 0.74 6.23 32.77 No.36

TXT.56 77.9 214.52 88.64 1.69 14.25 33.02 No.37

TXT.37 47.4 133.48 55.16 5.99 50.34 33.20 No.38

TXT.8 60.2 175.46 72.51 3.66 30.77 33.44 No.39

TXT.43 79 203.89 84.25 2.88 24.20 33.59 No.40

TXT.49 80.4 199.57 82.46 3.40 28.61 33.73 No.41

TXT.11 82.4 225.63 93.24 1.52 12.74 33.96 No.42

TXT.41 89.7 242.00 100.00 0.85 7.13 34.20 No.43

TXT.40 82.2 224.57 92.80 1.74 14.63 34.35 No.44

TXT.63 84.9 235.00 97.11 1.80 15.15 36.12 No.45

TXT.1 83.3 207.57 85.77 4.46 37.55 37.49 No.46

TXT.38 53.7 167.55 69.24 5.60 47.07 38.00 No.47

TXT.3 44.7 160.00 66.12 5.71 48.06 38.54 No.48

TXT.45 55.4 144.31 59.63 8.02 67.49 38.98 No.49

TXT.4 45.6 142.25 58.78 7.42 62.44 39.00 No.50

TXT.6 84.6 222.71 92.03 4.72 39.67 41.00 No.51

TXT.12 84.7 224.48 92.76 4.90 41.23 41.81 No.52

TXT.68 49.3 158.21 65.38 7.76 65.25 42.40 No.53

TXT.67 63.1 223.40 92.32 4.38 36.80 44.58 No.54

TXT.44 64.8 197.87 81.77 6.64 55.84 44.67 No.55

TXT.J58.2 144.00 59.50 10.68 89.82

45.06 No.56

TXT.JJ 44.6 135.00 55.79 10.42 87.61 45.26 No.57



29

TXT.K 60.1 218.44 90.26 5.07 42.66 45.91 No.58

TXT.14 70.1 213.21 88.11 6.44 54.14 46.28 No.59

TXT.kk 54.3 168.09 69.46 9.27 77.95 47.25 No.60

TXT.ii 47.5 164.48 67.97 9.24 77.69 47.79 No.61

TXT.nn 36.8 163.88 67.72 8.61 72.43 48.23 No.62

TXT.B 50.7 152.13 62.86 10.70 90.00 48.29 No.63

TXT.H 32.1 130.83 54.06 11.56 97.23 49.78 No.64

TXT.46 59.5 166.74 68.90 11.89 99.98 52.54 No.65

TXT.Y 42.4 163.71 67.65 11.11 93.45 53.38 No.66

3.3.2 The Online Listening Exercises

In the eleven weeks, the 66 listening texts with their MP3 files and

comprehension check questions were presented to the learners based on their

difficulty levels. For each listening passage, there were three to six comprehension

questions probing into the participants’ comprehension levels, and also aiming at

training the five target listening skills of the learners (see Appendix A for a sample of

online listening task, containing texts, comprehension questions). The five target

listening skills—listening for main ideas, listening for details, interpreting the

speaker’s intent, making inferences, and summarizing the listening texts—were

chosen from the text book of the listening and speaking course in order to tie in the

online resources with the course. Comprehension questions of each passage were

written based on these target skills. For the first four skills, the question type was in

multiple-choice format. As for the last skill, summarization, the learners were asked to

write a summary of 120 to 150 words that included both the main idea and detailed

information. The total number of the questions for each skill across the eleven weeks

is presented in Table 3.3 below. The sample questions for each skill are shown in

Table 3.4.



30

Table 3.3 The number of questions for each skill

Listening

for main

ideas

Listening

for details

Interpreting

the speaker’s

intent

Making

inferencesSummarizing Total

72 118 26 79 11 306

Table 3.4. Sample questions for each target listening skill

Listening skills Sample questions

Listening for main

ideas

-What is the passage about?

-What is the main idea of this listening excerpt?

Listening for

details

-According to the conversation, what didn’t the speaker buy?

-What are the experts’ suggestions mentioned in the excerpt?

Interpreting the

speaker’s intent

-What’s the speaker’s attitude toward student credit cards?

-What does the man imply by saying “That’ll be a long time?”

Making inferences

-What does the speaker imply how she is going to pay back

the money?

-Based on the excerpt, select correct statements below.

Summarizing-Please write a short summary in about 150 words. In your

summary, both important information and details should be

included.

Each week, a set of six listening texts were presented, with three required and

three optional. The learners were required to perform the required set in which one

summary task was embedded. Access for each set of exercises was available for one

week. The learners had to finish the exercises in one week as the system was set to

reject attempts once the exercises were closed. While doing the exercises, the learners

were able to download the MP3 files and listen to it as many times as needed before

they virtually submitted their answers.

The MOODLE system was able to provide instant feedback every time after the

participants submitted their answers. For multiple-choice questions (MCQs), the



31

system would highlight the correct answer. As for the summary tasks, the idea units 1

of the listening passage and a summary model which were written by the research and

a master student majoring in teaching English as a foreign language (TEFL) were also

shown immediately after the submission as a reference for the learners to learn and

improve. Figure 3.4 and 3.5 show the instant feedback of MCQs and summary task

given by the system, respectively.

Figure 3.4 The instant feedback for MCQs

Note. The highlighted item was the correct answer.

Figure 3.5 The instant feedback for summary tasks

Note. The first block is the summary written by the learner, and the second part is the

idea units, and the final block is the sample summary.

1 The idea units of each summarization text were written independently by the researcher and a master

student majoring in TEFL independently. After the two versions were finished, a discussion was carriedout to finalize a version for grading. For the summary tasks in the pretest and posttest, the ratings of the

two writers reached a Cronbach alpha of 0.83 and 0.93, respectively.



32

3.4 Research Instruments

Three types of instruments were designed in this study. First, a listening test was

designed and used in the pretest and posttest in order to assess the participants’

listening ability before and after the eleven-week practice. Secondly, two sets of

questionnaires, the weekly questionnaire, and a final evaluation questionnaire were

developed to understand the participants’ perceptions of the difficulty levels of the

listening materials and their attitudes toward the usefulness of the website as well as

the online listening exercises. Finally, the participants’ logs and scores on the weekly

exercises on the website recorded by the tracker program during the eleven-week

period were used to understand the usefulness of both the online materials and the

grading formula.

3.4.1 Pretest and Posttest

An English listening test was designed to serve as the pretest and posttest, which

aimed at measuring the participants’ overall listening ability, also their strengths and

weaknesses in terms of the five target listening skills. The test comprised five

listening passages with twenty-four items taken from the last four units of the course

book which would not be covered until the second semester of the school year.

Among the twenty-four items, twenty-three of them were true/false and

multiple-choice questions under four listening passages, and the other one was a

productive task that asked the participants to summarize one listening passage. A

sample of the test items is shown Table 3.5. For the complete version, see Appendix B.

Despite the identical multiple-choice questions used in the two versions of the test,

different prompts for the summarizing task were used in the pretest and posttest. In

the pretest, a video clip under the topic of “The Age of E-mail” was used, which

presented a discussion on the good and bad of the technology. As for the posttest,



33

another video clip which concerned the harmful consequences of tobacco

use—Smoking—was used (see Appendix C and D for the video scripts). Due to the

shortage of language labs in the university, the tests were administered in a traditional

paper-based format.

To ensure the comparability of the two versions used in the pretest and posttest,

items in the posttest were identical with those in the pretest but with different order.

The posttest was designed to assess the participants’ listening ability concerning the

five target skills after receiving the eleven-week online practice. To avoid practice

effects, the test papers were collected immediately after the pretest; also, correct

answers to all the items were not reported. Hence, learners did not have chance to

have access to the test items until eleven weeks later.

Table 3.5 An excerpt of the pretest

A. Listen to the excerpt. Mark the sentences T (true) or F (false).

_____ 1. The speaker is an employer.

_____ 2. The speaker does not trust the people who work at his business.

_____ 3. Trusting his employees gives the speaker satisfaction.

B. Listen to the excerpt again. Circle the answer that best completes each

sentence.

1. According to the speaker, an employer should ________.

a. tell his employees what is right and wrong

b. watch his employees closely

c. let his employees decide what is right and wrong

2. According to the speaker, being “Big Brother” is ________.

a. an effective management strategy

b. an ineffective management strategy

c. neither effective nor ineffective

3. When the speaker says “go that extra mile,” he means ________.

a. exercising by running an extra time

b. getting satisfaction and rewards

c. working extra hard



34

3.4.2 The Weekly Questionnaire

The weekly questionnaire probed into the participants’ receptions of the

difficulty level of the easiest passage each week. The questionnaire contained five

items, each representing one aspect of difficulty level of the listening passage:

vocabulary, speech rate, syntactic complexity, topic familiarity, and overall difficulty

(see Table 3.6 for an excerpt of the questionnaire. For the complete version, refer to

Appendix E). The questionnaires were uploaded to the website each week, hence, the

learners, after submitting the required listening tasks, could respond to the

questionnaire immediately. As can be understood, there were eleven questionnaires

across the practice period, it was expected that the learners’ grading could serve as a

mean for us to understand the correspondence of the designed sequence and the

learners’ perceptions.

Table 3.6 An excerpt of the weekly questionnaire

1. I can understand _____ percent of the words in this listening task. □ almost 100%

□ More than 80%

□ 50% to 80%

□ Less than 50%

□ Less than 20%

2. I think the speed of this listening is ______.

□ Too fast to understand

□ Fast but understandable

□ About right for me

□ Slow

□ Too slow

3.4.3 The Final Evaluation Questionnaire

The final evaluation questionnaire was designed to tap into the participants’

background information, previous experiences of using online listening websites,

attitudes towards the website as well as the usefulness of the sequencing and the



35

online training. The questionnaire comprised twenty-three items and was divided into

two sections. The first section (from items 1 to 6) probed into the learners’

background information such as self-rated listening proficiency and previous

experiences of using listening websites. The other section (from items 7 to 23)

concerned the learners’ perceptions of the usefulness of the website, including the

layout, the quality, the role it played in their learning of listening, and the usefulness

of the material sequencing. Table 3.7 presents a sample of the evaluation

questionnaire (The complete version can be found in Appendix F).

Table 3.7 An excerpt of the evaluation questionnaire

The first

section

。 My English listening proficiency is at _____ level.

Primary Low-intermediate Intermediate

High-intermediate Advanced

。 I am eager to improve my listening ability

SA A U D SD

。 I have practiced English listening via other listening websites

like the one used this semester.Yes No (skip Q5 and Q6)

The second

section

。 I think the layout of the website (the color and the

arrangement) is clear and easy to follow.

SA A U D SD

。 The topics of the online listening tasks are interesting and

motivating.

SA A U D SD

。 The website and the online listening tasks are useful to

improve my overall listening ability.

SA A U D SD

。 The online tasks are more and more challenging from the

beginning to the end of the 11 weeks.

SA A U D SD

。 I would like to use such a listening website to enhance my

listening ability in the future.

SA A U D SD



36

3.4.4 Logs of Usage and Scores of the Online Tasks

During the semester, every learner’s logs of usage (i.e., the duration of time and

frequency of use) as well as their scores on the online tasks were recorded

automatically as an index of their progress toward the tasks with the help of the

tracker program embedded in MOODLE (see Figure 3.6).

Figure 3.6 The usage and score of the learner recorded by the system

MOODLE was very powerful in recording the learners’ activities on the website,

yet, the records were scattered in the database. To obtain usable records, we invited a

programmer to help us do the data mining, and pull useful data together. With the help

of the programmer, we were able to get the learners’ average score of each of the 66

tasks, also each learner’s scores across the eleven weeks. In addition, we were also

able to track the learners’ performance on the website in terms of the five target

listening skills. These data helped us understand the learners’ changes in terms of their

listening ability during the eleven weeks. Moreover, it was also expected that the

records could offer clues to the question of the usefulness of the materials in the way

they were designed.

3.5 Procedures

The experimental procedures proceeded from September to December of 2006.

At the first class meeting, a consent form (see Appendix G) was distributed to the

participants, and the pretest was administered. The learners were told that the 11-week

online practice would account for ten percent of their final grades for the course. Also,



37

they were acknowledged that their performance on the pretest, posttest, and the online

listening tasks would not affect their final course grade. In the second meeting, the

learners were instructed to use the website and were informed that the time available

for each set of exercises was one week. After the class was dismissed, the learners

started their first weekly listening practice on Freshmen Listening . Learners did the

required exercises and optional exercises, and then responded to the weekly

questionnaire. In the following weeks, learners followed the same pattern to the last

week of online listening practice. The third week was an exception, because in that

week, the server where the website was constructed went down. The breakdown cut

all the connection to the website for a few days. Hence, to grant the learners more

time to perform the tasks, the schedule was postponed for one week. In the following

weeks, the system worked steadily, and so did the online training project. Then, post

to the online training project, in week 14, the learners were distributed the posttest and

also the final evaluation questionnaire. A summary of the research procedures is

shown below (Table 3.8).

Table 3.8 The summary of the research procedures

Time/Period Research Procedures

Preparation

。 Collected materials

。 Developed the formula and sequenced the

materials

。 Designed comprehension questions

。 Designed pretest and posttest

。 Designed the questionnaires

Week 1 (9/11)。 Distributed the consent form

。 Distributed the pretest

Week 2 to 13 (9/18 to 12/4)

。 Instructed the learners to use the website

。 Online listening practice started

。 Weekly questionnaire

Week 14 (12/11) 。 Distributed the posttest。 Distributed the final evaluation questionnaire



38

3.6 Data Analysis

There were three main research directions in this study. First, we investigated to

what extent the online listening practice was useful or helpful to the learners in their

development of listening ability. Second, with the learners’ report in the final

evaluation questionnaire, we attempted to explore the learners’ attitudes toward the

online listening exercises. Third, with the learners’ weekly questionnaire responses

and scores on the tasks, we tried to examine the usefulness of the proposed formula

and the sequencing of the listening materials. Hence, the research questions we

attempted to answer are phrased as follows:

1. After learning from the online listening materials, did the students perform better

in the posttest than in the pretest in terms of the overall scores and items for each

skill?

2. Were there learners differences found in this project?

– Were the materials equally beneficial to learners of different proficiency

levels?

– Were there differences in achievement, given the unequal number of passages

finished?

– What type(s) of skill practice could best predict the learners’ posttest

performance?

3. How did the learners perceive the role of the materials in their learning?

– Were the learners’ perceptions of the material difficulty congruent with their

performance?

4. Was the proposed formula useful based on the learners’ weekly performance?



39

To answer the first question, the learners' pretest and posttest performance was

compared. For the second research question, the learners were further classified into

two groups based on their proficiency level and diligence level, and then group

performance was compared. Additionally, correlation among the five skills and the

posttest scores was computed to see if the skill practice across the eleven weeks could

predict the learners' performance in the posttest. To analyze data for question three,

learners' responses to the final evaluation questionnaire were reported using

descriptive statistics by assigning 0 to 5 to indicate the level of agreement among the

learners. Question four tapped into the usefulness of the proposed formula. We

computed correlation between the predefined "difficulty scores" and the learners'

scores of the online tasks. Also, to examine the congruency between the learners'

perceptions and their weekly performance in terms of the difficulty levels of the texts,

correlation was adopted again.

Table 3.9 The summary of data analysis methods of each research question

Research Questions Methods

1. Did the students perform better in the

posttest than in the pretest?

Pretest and posttest comparison

2. Were there learner differences found

in this project?

Pretest and posttest comparison based on

the learners’ proficiency and diligence

levels

Correlation

3. How did the learners perceive the role

of the materials in their learning?

Descriptive statistics

Correlation

4. Was the proposed formula useful

based on the learners’ weekly

performance?

Correlation



40

Chapter Four

RESULTS AND DISCUSSION

4.1 Overview

By constructing a MOODLE based listening website and infusing it into an EFL

college listening and speaking course, this study addressed the following research

questions:



skill?


- Were the materials equally beneficial to learners of different proficiency

levels?

- Were there differences in achievement, given the unequal number of passages

finished?

- What type(s) of skill practice could best predict the learners’ posttest

performance?


- Were the learners’ perceptions of the material difficulty congruent with their

performance?


Subjects in this study were thirty-one EFL college freshmen who were taking a

listening and speaking course. In the study, the subjects were required to do at least

three out of the six listening exercise each week across the eleven-week period during

the semester. To ensure the validity of the data, students who failed to complete more



41

than thirty-three tasks were excluded from the following analyses. Learners’ logs on

the listening website showed that eighteen out of the thirty-one students completed

more than thirty-three listening tasks. The valid subject number in this study therefore

dropped to eighteen. According to Hatch and Lazaraton (1991), when the subject

number is smaller than 30, a non-parametric analysis should be applied. Hence, the

results reported in the following section were all based on such analysis. However, the

thirty-one students were asked to fill in a weekly questionnaire, and the results of the

questionnaire responses were recorded anonymously; hence, the data of the valid

eighteen subjects were unable to be identified. Consequently, findings related to the

weekly questionnaire are therefore reported in the original group manner.

In this chapter, the results from the pretest, the posttest, the weekly questionnaire,

learners’ weekly task scores, and the final evaluation questionnaire were analyzed and

presented first. Then the usefulness of the online materials, the learners’ attitudes to

the online materials, and the feasibility of the proposed formula are discussed.

4.2 Results

4.2.1 The Learners’ Background

Several questions in the final evaluation questionnaire were designed to elicit the

learners’ background information (items 1 to 6) concerning their English proficiency

levels (item 1) and English listening proficiency levels (item 2), the attitudes to the

learning of listening (item 3) and their experience in using listening websites to

enhance listening ability (item 4 to 6). Regarding English and English listening

proficiency level, the majority of the subjects rated themselves as at intermediate level.

Among the eighteen students, three of them rated their own English proficiency level

as at high-intermediate level, twelve as intermediate, and three as low-intermediate.

Concerning listening proficiency, two out of the eighteen students rated themselves as



42

at high-intermediate level in listening, twelve as intermediate, and four as

low-intermediate. Table 4.1 summarizes the distribution.

Table 4.1 The learners’ self-evaluation of their English proficiency and English

listening proficiency levels

English Proficiency

Level N %

High-intermediate 3 16.7%

Intermediate 12 66.7%

Low-intermediate 3 16.7%

Total 18 100%

English Listening Proficiency

High-intermediate 2 22.2%

Intermediate 12 66.7%

Low-intermediate 4 11.1%

Total 18 100%

Concerning the learners’ experience of using listening websites, six of them

(33.3%) reported having used such websites to learn about English listening before

they were introduced to Freshmen Listening , the website used in this present study.

The websites used were mainly international and local news websites such as CNN ,

BBC , and ICRT ; and English learning websites such as the Studio Classroom. The

results showed that more than half of the subjects lacked experiences of learning

English listening via internet. For those who did use listening websites, the websites

they had used were not able to provide comprehension check questions and instant

feedback. Therefore, using the MOODLE website to practice listening skills seemed

to be a comparatively new experience to the subjects.



43

4.2.2 The Results from the Pretest and Posttest

Prior and post to the eleven weeks of online listening practice, an independent

listening test containing twenty-four items was administered. The tests were designed

to tap into the learners’ beginning level and their end-of-project level of their listening

ability and of their five target listening skills after the 11-week practice. Comparisons

of the total scores and the individual scores of the five skills were made. In order to

know whether different learners benefited distinctly from the listening practice, they

were classified according to their listening proficiency and their diligence level. The

question of whether more diligent students benefited more from the online practice

was also explored by operationalizing “diligence” as the total listening passages they

completed throughout the eleven weeks. Last, the question of which skill performance

across the eleven weeks is more strongly associated with the posttest performance was

also examined.

For both listening tests used in the pretest and posttest stages, every learner had

five individual scores for each of the five skills, and an overall score (i.e., the sum of

the five scores). The full scores for skill 1 to 5 are “10, 22, 6, 8, 10,” making the full

overall score 56 (see Table 4.2).

Table 4.2 The full scores for the five target listening skill

Skill

1.

Listening

for main

ideas

2.

Listening

for details

3.

Interpreting

the speaker’s

intent

4.

Making

inferences

5.

Summarizing Total

Full

score10 22 6 8 10 56

To investigate the changes the learners made in terms of their overall listening

ability and the five listening skills, both tests were graded by the researcher. Of the

twenty-four items in the tests, twenty-three of them were in true/false or



44

multiple-choice format whose answer keys were readily available. As to the rating of

the summaries, another rater, with the researcher (both were MA-TEFL students) was

invited to grade in order to ensure the rating reliability. Idea units (Johns & Mayes,

1990) of the two summaries were written by the two raters independently based on

their listening comprehension of the passages. Then, the two raters discussed on the

units to finalize a version for each of the summaries as the answer keys (see Appendix

H and I for the complete version of the idea units and the sample summaries of the

tests). Ten units were written for each of the two summaries, making the full score of

each task 10. With the finalized idea units, the two raters graded the learners’

summaries independently. In the pretest, Cronbach alpha reached 0.93, and in the

posttest 0.83. Both alphas indicated fairly high reliability. Thus, Learners’ grades were

computed by averaging the scores given by the two raters.

4.2.2.1 The Usefulness of the Online Materials

To answer the question of the usefulness of the online materials, we compared

the subjects’ performance in the pretest and the posttest in terms of: (1) the overall

score; and (2) the individual scores of the five listening skills.

A higher mean score was obtained in the posttest (45.17) than in the pretest

(42.75). To examine the significance of the difference, the Wilcoxon matched pairs

test (i.e., a non-parametric alternative test to the t-test for dependent samples) was

applied. Results showed that the learners’ change from the pretest to the posttest

reached a statistically significant level (p<0.05, see Table 4.3).



45

Table 4.3 The Wilcoxon matched pairs test of the learners’ performance in the pre-test

and post-test

Valid N T Z P level

Pretest-posttest 18 34.5 1.99 0.05*

Note: *Sig. p<0.05

Table 4.4 displays the learners’ scores on the five listening skills—1) listening

for main ideas, 2) listening for details, 3) interpreting the speaker’s intent, 4) making

inferences; and 5) summarizing—in the pretest and posttest. In terms of the learners’

change in the five skills, as shown in Table 4.5, the Wilcoxon matched pairs test

demonstrated that the learners improved significantly in skill 2 and 5 (p=0.04, p=0.02,

respectively), whereas no gain was found in skill 1, 3 and 4. The improved listening

skills were listening for details and summarizing.

Table 4.4 The learners’ score of the five target skills in the pretest and posttest

Skill 1 Skill 2 Skill 3 Skill 4 Skill 5

Pretest 8.89 18.78 4.67 6.11 4.31

Posttest 8.89 20.33 5.33 5.22 5.39



46

Table 4.5 The Wilcoxon matched pairs test of the learners’ performance on each skill

in the pretest and posttest

Skill 1 Listening for main ideas

Valid N T Z P value

Pretest-posttest 18 22.5 0.00 1.00

Skill 2 Listening for details

Valid N T Z P value


Skill 3 Interpreting the speaker’s intent

Valid N T Z P value


Skill 4 Making inferences

Valid N T Z P value


Skill 5 Summarizing

Valid N T Z P value


Note: *Sig. p＜0.05

To sum up, after using the online materials for eleven weeks, the learners were

found to make progress in (1) their overall listening proficiency, and also in (2)

specific aspects of listening skill—listening for details and summarizing.

4.2.2.2 Proficiency Level and the Final Achievement

To understand if the online materials were equally beneficial to learners of

different proficiency levels, the learners were categorized into a high proficiency

group (high-pro) and a low proficiency group (low-pro) based on their performance in

the pretest. With the mean score of 42.75, and SD being 5.34, learners who got a score



47

higher than 48.09 (i.e., mean+1 SD=42.75+5.34) were placed into the high-pro group

and those who got a lower score than 37.41 (i.e., mean-1 SD=42.75-5.34) were in the

low-pro group. The high-pro group (N=4, mean=49, as shown in Table 4.6) and the

low-pro group (N=4, mean=35.88) were compared of the progress they made from the

pretest to the posttest. The Wilcoxon matched pairs test indicated that in spite of the

score gain in the posttest of both groups (see Table 4.7), neither were statistically

significant. Namely, our online materials seemed to be equally beneficial to learners

of different proficiency levels.

Table 4.6 A summary of mean scores of the pretest and posttest of the high-pro and

the low-pro groups

Pretest Posttest

The high-pro group 49 49.5

The low-pro group 38.88 42.25

Table 4.7 The Wilcoxon matched paired test of the posttest performance of each groupValid N T Z p-level

High proficient

(pre & post)4 1.50 0.80 0.42

Low proficient

(pre & post)4 0.00 1.83 0.07

P<0.05

4.2.2.3 Diligence and Posttest Performance

To address the question concerning the relation between the learners’ diligence

level and their posttest performance, the learners were divided into two groups based

on the number of tasks they completed during the eleven weeks. In this study, every

learner was required to complete at least three out of the six required online tasks

every week. Yet, in spite of the requirement, the number of completed tasks differed

from one to another depending on the learners’ motivation and interests. To classify



48

the learners based on their diligence levels, the average numbers of the passages

finished by all the participants were calculated (see Figure 4.1 for the learners’

completion rates. A detailed picture of the learners’ task completion condition can be

found in Appendix J). The calculation obtained an average number of 43.6 (SD=11.8).

Using this number as a criterion, learners who were above the average were classified

as the more diligent group (N=7, mean number=33.6), while those below the average

were put into the less diligent group (N=5, mean number=60.4). The more diligent

group obtained a mean score of 44.1 in the pretest and 45 in the posttest (see Table 4.8

for the summary of the mean scores of each group). Though the scores increased from

the pretest to the posttest, the Wilcoxon matched pairs test showed that the

improvement was not statistically significant (see Table 4.9). The less diligent group,

on the other hand, obtained a mean score of 41.6 in the pretest and 44.5 in the posttest.

The same with the more diligent group, despite the score gain, the matched pairs test

did not find significant improvement.

Table 4. 8 A summary of the mean scores of the pretest and posttest of the more

diligent group and the less diligent group

Pretest Posttest

More diligent 44.1 45.0

Less diligent 41.7 44.5



49

Figure 4.1 The distribution of completion rates among the 18 learners

Table 4.9 The Wilcoxon matched pairs test of the pretest and posttest performance of

the less diligent and the more diligent group

Valid N T Z p-level

Less diligent

(pre & post)7 7.00 1.18 0.24

More diligent

(pre & post)5 5.00 0.00 1.00

4.2.2.4 Skill Performance and the Final achievement

The learners’ scores of each of the five skills across the eleven weeks were

retrieved from the tracker program and analyzed using correlation analysis to probe

into the relations between the learners’ posttest scores and their scores of each skill

across the eleven weeks. Table 4.10 displays the learners’ mean scores2 of each skill

across the eleven weeks, and Table 4.11 presents the mean scores of each learner

across the eleven weeks.

2

The mean score here indicates the average number of items correctly answered by the learners. Toillustrate, if there were 10 questions in “listening for main ideas,” and student A did five of them and

answered each one correctly, the mean score for student A was 10 (5/5*10) instead of 5 (5/10*10).

32

3

10

0

2

4

6

8

10

12

60-66 50-60 40-50 33-40

Number of finished tasks

N u m b e r o f l e a

r n e r s

17% 17%11%

55%



50

Table 4.10 The learners’ performance on the five target skills across the 11 weeks

Week 1 2 3 4 5 6 7 8 9 10 11

Skill1 9.33 9.17 8.06 8.46 7.87 8.61 7.50 5.65 6.67 7.78 4.72

Skill2 6.67 8.09 9.35 8.36 8.83 8.53 7.97 7.16 7.48 7.77 7.28

Skill3 7.78 * 5 5.56 5 4.44 9.44 6.11 8.33 * *

Skill4 8.33 8.41 8.80 3.98 7.87 8.89 7.32 7.78 7.92 1.57 6.11

Skill5 5.74 3.55 7.14 6.39 6.27 7.94 7.59 5.93 6.56 4.88 4.81

Table 4.11 The learners’ mean scores of the five target skills and their posttest scores

across the 11 weeks

Skill 1 Skill 2 Skill 3 Skill 4 Skill 5 PosttestS1 7.77 6.50 4.17 6.90 3.12 48

S2 7.42 8.96 7.71 7.70 6.98 51

S3 6.69 6.57 6.88 5.86 5.94 46.5

S4 7.74 7.82 6.88 7.89 6.54 50

S5 6.74 8.57 8.13 7.69 8.10 50

S6 8.30 7.62 4.58 6.34 5.89 38

S7 5.79 6.11 4.79 5.97 3.29 40

S8 7.61 7.97 6.88 6.97 8.08 49

S9 8.30 8.40 6.88 7.49 1.99 48

S10 9.03 7.42 6.46 7.14 5.40 42.5

S11 6.38 7.24 2.08 6.92 3.68 41

S12 7.06 8.11 7.08 6.62 5.93 40

S13 7.60 8.42 8.13 6.50 6.67 50.5

S14 7.41 9.12 5.63 7.42 6.79 45

S15 8.06 8.50 6.46 6.38 7.08 41.5

S16 9.39 8.09 6.67 8.50 8.39 42.5

S17 7.39 8.79 9.58 7.27 7.96 48

S18 8.64 8.95 7.29 6.39 6.79 41.5

Spearman correlation (i.e., a non-parametric alternative test of correlation

analysis) was performed to explore the relations between the learners’ performance on

each of the five skills and on the posttest. As shown in Table 4.12, it was found that

the learners’ performance on skill 3 (interpreting the speaker’s intent) and skill 4

(making inferences) were significantly related to their performance on the posttest. In



51

other words, learners’ performance on skill 3, interpreting the speaker’s intent (r=0.58,

p<0.05), and skill 4, making inferences (r=0.55, p<0.05), across the eleven-week

training period was found helpful in predicting the learners’ posttest performance.

Table 4.12 Spearman correlation between the learners’ performance on each of the

skill and on the posttest

Skill 1 Skill 2 Skill 3 Skill 4 Skill 5

Corr. with the posttest -0.01 0.34 0.58* 0.55* 0.34

Note: *Sig. p<0.05

4.2.3 Results from the Final Evaluation Questionnaire

In this section, learners’ responses to the evaluation questionnaire containing

twenty-three items were gathered and analyzed. The questionnaire was divided into

two sections, the first section, items 1 to 6, concerned the learners’ background

information. And the second section, items 7 to 23, was about the learners’

perceptions of the usefulness of the website and online materials. The majority of the

questions were in 5-point Likert scale format in which the five options were “strongly

agree, agree, undecided, disagree, and strongly disagree.” For each item, 5 to 1 point

was assigned to each option for data coding. For each Likert-scale question, the

learners’ ratings were computed by dividing the sum of points given by the learners

by the total number of the learners (i.e., rating=5×a+4× b+3×c+2×d+1×e/a+b+c+d+e,

“a” to “e” represents the number of the learners choosing a particular option). As a

result, higher ratings signify more positive attitudes from the learners.



52

4.2.3.1 Design of the Website

In our evaluation questionnaire, items 7 to 23 elicited learners’ perceptions about

the usefulness of the website. For item 7, a rating of 4.0 was obtained, indicating that

the learners felt fairly positive about the statement, “I think the layout of the website

is clear and easy to follow.” Item 8 also got a rating of 4.0, indicating that most

learners agreed on the statement that the website was easy to use. In item 9, “the

topics of the online listening tasks are interesting and motivating,” a rating of 3.00

was obtained, implying that the learners were indecisive about the statement. Namely,

our learners found the topics of the listening tasks neutral to them, neither interesting

nor boring. Table 4.13 below presents the statements as well as the percentage of

agreement and the rating of each item.

Table 4.13 The learners’ attitudes toward the design of the website

Item Statement% of

agreementRating

#7I think the layout of the website is clear and easy to

follow.

15/18

(83.5%)4.0

#8The website is easy to use, and I get used to it very

easily.

14/18

(77.8%)4.0

#9The topics of the online listening tasks are interesting

and motivating.

9/18

(50%)3.0

4.2.3.2 The Usefulness of the Online Materials

Items 13 to 22 in the evaluation questionnaire were designed to probe into the

learners’ perceptions of the usefulness of the online materials in terms of improving

their overall listening ability as well as the five listening skills. Item 13, 21, and 22

concerned the learners’ views of the helpfulness of the website in improving their

overall English ability. For item 13, “the website and the online listening tasks are



53

useful to improve my overall listening ability,” a fairly high rating (4.0) was obtained.

The learners’ supportive view was also evidenced from their responses to item 21, “It

is beneficial to have resources like this listening website accompanying with listening

courses” (rating=4.4), and item 22, “I would like to use such a listening website to

enhance my listening ability in the future” (rating=4.1). Table 4.14 provides a

summary of these findings. These findings may be due to the friendly online

environment of the website, as shown in learners’ responses to item 7—the layout is

clear—and item 8—the website is easy to use (both ratings are 4.0).

Items 14 to 18 explored learners’ attitudes of the usefulness of the materials

concerning the five target listening skills. For all the five items, the ratings were

satisfactory. Item 14 concerned the skill of listening for main ideas, the rating, 4.1,

indicated the learners found the materials fairly useful in terms of enhancing this

specific skill. The ratings for the other four skills were 3.8, 4.0, 3.7, and 3.6,

respectively (see also Table 4.14). The findings suggest that the learners did agree that

the online materials were useful for them to enhance their ability of the five individual

skills.

More supportive attitudes to the material usefulness were found in the learners’

comments which were also reported in the final evaluation questionnaire. One of the

learner reported that “the website gave her extra opportunities to practice her

listening.” Many other stated that “the materials did help enhance their listening

ability.” Furthermore, there was even one learner asking where she could find free

listening website like Freshmen Listening for her to practice; and another asking if we

could keep Freshmen Listening open so that she could have extra practice in the

future. All these comments given by the learners provided strong evidence of the

learners’ positive attitudes toward the online materials and the website.



54

Table 4.14. The learners’ perceptions of the usefulness of the online materials

Item Statement% of

agreementRating

#13The website and the online listening tasks are useful to

improve my overall listening ability.

14/18

(77.8%)4.0

#14The online practice improves my ability of listening for

the main idea of the passages.

17/18

(94.4%)4.1

#15The online practice improves my ability of listening for

detailed information of the passages.

14/18

(77.8%)3.8

#16The online practice improves my ability of

understanding the speaker’s intention or thoughts.

16/18

(88.9%)4.0

#17The online practice improves my ability of

understanding the implied information of the passages.

13/18

(72.3%)

3.7

#18The online practice improves my ability of summarizing

the listening content.

12/18

(66.7%)3.6

#21It is beneficial to have resources like this listening

website accompanying with listening courses.

17/18

(94.4%)4.4

#22I would like to use such a listening website to enhance

my listening ability in the future.

15/18

(83.4%)4.1

4.2.3.3 The Learners’ Perceptions about Attributes to Listening Difficulty

Items 10 to 12 tapped into the learners’ perceptions about factors causing their

listening difficulties. Item 10 dealt with text type, “There were two types of texts

during the eleven weeks—conversation between people and lectures/ news reports by

one person—which one do you find more challenging?” Among the thirty learners,

77.8% of them considered lectures and news reports more challenging than

conversation. On the question regarding “task type,” item 12 asked the learners to

grade their level of agreement to the statement, “the summary task is more difficult

than all the multiple-choice questions.” A very high grade, 4.6, was obtained from this

item, indicating that the learners did regard task type—specifically multiple-choice

questions and essay questions—a factor influencing their performance.



55

Item 11 investigated how the learners rated the importance of the potential

causes of listening difficulties. Five difficulty factors were listed in the item: speech

rate, vocabulary, text types, topic familiarity, and sentence structure. The learners

were asked to rank them from the most important to the least important; rank 1

indicated the most important and 5 the least important. The learners’ rankings were

averaged to understand how they perceived the five factors. Findings showed that the

learners generally regarded speech rate as the most influential factors causing

listening difficulties (rank=1.8) and sentence structure as the least influential

(rank=3.7). For the other factors, vocabulary was ranked as the second place (2.6),

followed by text types (3.2) and topic familiarity (3.6).

4.2.4 Results from the Weekly Questionnaire and Weekly Tasks

One of the main goals in this study was to propose and test a formula which

generated a “difficulty score” that determined the difficulty levels of our listening

materials. The formula was as follows:

Difficulty score＝speech rate×0.5＋academic word ratio×0.3+reading ease

score×0.2

In the formula, three listening difficulty factors—speech rate, syntactical

complexity, and vocabulary—were involved. The range of difficulty score was from 1

to 100; the higher the score was, the more difficult the passage would be, and vice

versa. With the difficulty scores generated from the formula, the 66 listening passages

in Freshmen Listening were sequenced from easy to difficult.

To verify the usefulness of the proposed formula, data was gathered from three

sources: a) the learners’ perceptions of the difficulty levels reported in the weekly

difficulty questionnaire, b) items concerning the usefulness of the formula in the final

evaluation questionnaire, and c) the learners’ performance on the required exercises



56

across the 11 weeks. Based on these data sources, an analysis of the usefulness of the

formula is presented in the following sections.

4.2.4.1 The Learners’ Perceptions

In the evaluation questionnaire, item 19 and 20 examined whether the learners’

perceptions of the difficulty levels of the 66 texts corresponded to the pre-determined

sequence. As is clear in Table 4.15, item 19, “the online tasks are more and more

challenging from the beginning to the end of the eleven weeks” was assigned a rating

of 3.4, indicating slightly positive attitudes from the learners. Item 20, “I performed

better on the listening tasks at the beginning than at the end in the past eleven weeks”

was given an average grade of 2.9, suggesting that the learners were indecisive on or

slightly disagreed with this statement. Based on the learners’ report in the evaluation

questionnaire, it seemed that the formula was not as useful as we expected in

determining the difficulty levels of listening texts.

Table 4.15 The learners’ perceptions of the usefulness of the predefined difficulty

scores

Item Statement% of

agreementRating

#19The online tasks are more and more challenging from

the beginning to the end of the 11 weeks.

7/18

(38.9%)3.4

#20I performed better on the listening tasks at the beginning

than at the end in the past 11 weeks

3/18

(16.7%)2.9

Another way we used to probe into the learners’ perceptions of the usefulness of

formula was the weekly questionnaire (Note that, due to the anonymous nature of the

questionnaire, it was not possible for us to identified the responses of the 18 learners

from the whole class, hence, as far as the weekly questionnaire is concerned, the

analysis was based on the responses of the 31 learners.) After the learners completed

the weekly required tasks, a weekly questionnaire was provided. In the questionnaire,



57

the learners were asked to rate the difficulty level of the easiest passage of the week

from five perspectives—speech rate, syntactical complexity, vocabulary, topic

familiarity, and overall difficulty. Throughout the eleven-week practice, the learners’

responses to the surveys were collected and analyzed. On one hand, we expected that

the learners’ perceptions could correlate with the pre-determined sequence. Namely,

we expected that the learners could rate the texts more and more difficult across the

eleven weeks. On the other hand, however, we did hope that the learners could gain

progress with the help of the practice, which would then lead to a reverse

consequence—disagreement between the learners’ perceptions and the predefined

sequence. Transformation of the learners’ ratings of the easiest passage across the

11-week period to a curve diagram is shown in Figure 4.2. The curve seems to suggest

that the learners’ ratings of the selected passages fluctuated across the eleven weeks. A

pattern moving from a lower score to a higher score was not found. The findings may

be in part due to the learners’ maturation and improvement during the practice period.

However, it may also show that the proposed formula needs further adjustment or

revision.

Figure 4.2 Learners’ ratings of the easiest passage each week throughout the practice

period

0

1

2

3

4

5

1 2 3 4 5 6 7 8 9 10 11

weeks

p o i n t s

vocabulary

speech rategrammar

topic familiarity

overall difficulty



58

Weekly Task Performance

0.00

10.00

20.00

30.00

40.00

50.00

1 2 3 4 5 6 7 8 9 10 11

Week

S c o r e

4.2.4.2 The Learners’ Weekly Performance

Each week, the learners were required to complete three out of the six listening

tasks. To investigate whether the online tasks were beneficial for the learners in

enhancing their listening ability, the learners’ scores on the required tasks each week

were averaged. Throughout the eleven weeks, eleven mean scores were obtained. The

eleven scores were presented in Table 4.16 and were plotted into Figure 4.3. As in the

figure of the learners’ weekly ratings, there was no clear pattern regarding the

learners’ weekly performance.

Table 4.16 The learners’ mean scores of the weekly required exercises

Week 1 2 3 4 5 6 7 8 9 10 11

Task

scores37.39 40.32 40.82 34.74 39.04 38.14 42.41 35.18 31.61 32.28 34.77

Note: Maximum score=50

Figure 4.3 The learners’ mean scores of the weekly required exercises



59

4.2.4.3 The Relationship between Learners’ Weekly Rating and Weekly Performance

To understand whether the learners’ perceptions of the difficulty levels of the

texts were congruent with their weekly performance, correlation between the learners’

ratings of difficulty levels and their weekly scores of the online tasks was computed.

Table 4.17 below depicts the learners’ grading from the five perspectives and the

results of the correlation computation. Overall, the correlation was not strong.

Moderate correlation was found between vocabulary and the weekly performance

(r=0.40) as well between grammar and the weekly performance (r=0.41). For the

other three factors, fairly weak relations were found: 0.17, 0.14, and 0.18 for speech

rate, topic familiarity, and overall difficulty, respectively.

Table 4.17 Pearson correlation between the learners’ rating and their weekly

performance

Weekly

performanceVocabulary Speech rate Grammar

Topic

familiarity

Overall

difficulty

1 36.70 3.5 2.6 2.9 3.2 2.82 41.04 3.2 2.6 2.7 2.8 2.3

3 41.28 4 3.1 3.7 3.7 3.2

4 33.49 3.2 2.6 2.8 2.7 2.6

5 37.85 4 2.8 3.3 3.6 3.1

6 36.92 3.6 3.3 3 3 2.9

7 39.05 4.1 2.7 3.4 4 3.4

8 35.69 3.4 1.8 2.9 3.4 2.5

9 40.54 3.1 2.1 2.5 2.4 2.4

10 31.63 3.3 2.5 2.8 3.2 2.7

11 33.69 3 2.6 2.3 2.9 2.6

Corr. 0.40 0.17 0.41 0.14 0.18

To tap into the relations among the five difficulty factors from the learners’

perspectives, correlation was also computed. The correlation matrix is shown in Table

4.18. From the table, it is clear that learners’ grading on vocabulary was highly related



60

to that on grammar, topic familiarity, and overall difficulty. Further, grading on

grammar was also highly related to that on topic familiarity and overall difficulty. The

grading on topic familiarity was highly related to that on grammar and overall

difficulty, and the grading on overall difficulty was found to be highly related to

vocabulary, grammar, and topic familiarity. Speech rate, however, was not found

significantly related to any of the other four aspects.

Table 4.18 The correlation among the learners’ rating from the five perspectives

Vocabulary Speech rate Grammar Topic

familiarity

Overall

difficulty

Vocabulary 1.00 0.49 0.95* 0.88* 0.92*

Speech rate 0.45 1.00 0.49 0.26 0.59

Grammar 0.95* 0.49 1.00 0.82* 0.84*

Topic

familiarity0.88* 0.26 0.82* 1.00 0.84*

Overall

difficulty

0.92* 0.59 0.84* 0.84* 1.00

Note: *Sig. p<.05

4.2.4.4 The Relation between the Predefined Text Difficulty Score and the Learners’

Weekly Performance

To tap into the usefulness of the proposed formula or the predefined sequence of

the 66 listening texts, we have looked at the learners’ perceptions reported both in the

weekly questionnaire and the final evaluation questionnaire, and the learners’

performance on the weekly tasks. To explore the question in more depth, we further

looked at the relation between the predefined text difficulty scores and the learners’

scores on the tasks using Pearson correlation. As only thirty-three required tasks were

completed by more than half of the learners, the computation of the two

variables—the predefined difficulty scores and the learners’ weekly scores—was



61

based on the thirty-three tasks (as listed in Table 4.19). In the Table, difficulty scores

and the learners’ scores and attempts of the thirty-three required online tasks were

displayed. The computation obtained a correlation value of -0.273. The finding

showed that very weak correlation existed between the two variables. In other words,

when taking learners’ performance as an indication of the difficulty levels of texts, the

predefined sequence did not seem to be useful.

3

Higher score had different meanings in each variable. In predefined text difficulty score, it meant amore difficulty text. Yet, in learners’ performance, it meant an easier text. Consequently, a negative

value was expected in the computation.



62

Table 4.19 A summary of the difficult scores of the required tasks and the learners’

performance on the tasks

Text

Presentation

Text difficulty

scoreSs scores

Number of

students who did

the textTXT.69 No.1 14.21 32.78 18

TXT.18 No.2 15.77 35.30 17

TXT.25 No.3 17.02 42.59 18

TXT.48 No.4 17.34 45.42 16

TXT.21 No.7 20.36 46.50 6

TXT.28 No.11 24.75 44.48 18

TXT.17 No.12 25.23 36.11 18

TXT.54 No.13 25.3 41.90 18

TXT.29 No.17 26.9 42.64 18TXT.13 No.18 27.35 29.48 18

TXT.20 No.19 27.58 32.18 18

TXT.35 No.23 29.13 40.28 18

TXT.2 No.24 29.31 38.54 18

TXT.66 No.25 29.44 35.71 7

TXT.27 No.29 30.63 38.33 18

TXT.42 No.30 30.66 31.30 18

TXT.47 No.31 30.72 37.73 18

TXT.7 No.32 31.32 44.79 18TXT.65 No.36 32.77 44.49 17

TXT.56 No.37 33.02 38.82 17

TXT.37 No.38 33.2 35.00 17

TXT.41 No.43 34.2 31.46 16

TXT.40 No.44 34.35 35.94 16

TXT.63 No.45 36.12 38.13 16

TXT.45 No.49 38.98 38.13 16

TXT.4 No.50 39 38.15 16

TXT.6 No.51 41 49.06 16TXT.44 No.55 44.67 34.89 18

TXT.J No.56 45.06 32.41 18

TXT.JJ No.57 45.26 31.76 17

TXT.II No.61 47.79 31.37 17

TXT.NN No.62 48.23 33.92 17

TXT.B No.63 48.29 39.41 17

Note: Maximum score for each text=50. N=18



63

4.2.5 Summary of the Results

Based on the data analysis, the answers to the research questions are summarized

as follows:



skill?

As far as overall listening ability is concerned, the learners indeed made progress

from the pretest to the posttest after using the online materials for eleven weeks. As to

the specific listening skills, we found that the learners became better at the skill of

listening for main ideas and summarizing. Concerning the other three target

skills—listening for main ideas, interpreting the speaker’ intent, and making

inferences,—no significant improvement was identified.


a. Were the materials equally beneficial to learners of different proficiency

levels?

Although the low proficiency group made more progress in the posttest than the

high proficiency group did, the differences were not statistically significant in either

group. Hence, the online materials seemed to be equally beneficial to the learners

regardless their proficiency levels.

b. Were there differences in achievement, given the unequal number of passages

finished?

There was no difference found concerning the learners’ diligence level. Namely,

learners who completed more tasks did not improve more than those who completed

fewer tasks. Both the more diligent group and the less diligent group improved in the

posttest, yet, the differences were not large enough to reach the significance level.



64

c. What type(s) of skill practice could best predict the learners’ posttest

performance?

Based on Spearman correlation, skill 3 and skill 4—interpreting the speaker’s

intent and making inference—were found to be able to predict the learners’ posttest

performance. In other words, when the learners were good at these two skills, their

posttest scores were also higher, and vice versa.


The learners expressed very supportive view to the use the online materials as

supplementary resources in listening courses. Moreover, many of them reported their

willingness to the use of listening websites as the one used in the study. Also, the

majority of the learners considered that the materials were beneficial to their learning

of listening, especially in “overall listening ability,” “listening for main ideas,” and

“interpreting the speaker’s intent.”

a. Were the learners’ perceptions of the material difficulty congruent with their

performance?

Overall, the learners’ perceptions were not very congruent with their weekly

performance. However, when rating the difficulty levels in terms of vocabulary and

grammar (or syntactic complexity), the learners’ perceptions were found moderately

related to their weekly performance. Concerning rating from the other two

aspects—speech rate and topic familiarity, the relation between perceptions and

performance was very weak.


Because no clear pattern was found in the learners’ weekly performance across the

eleven weeks, the formula did not seem to be useful in determining the difficulty

levels of the listening texts used in this study. This indicates that the formula needs

further refinement.



65

4.3 Discussion

In this study, we proposed a formula to grade our listening texts, and with its

help, we constructed a listening website in which the graded listening texts were

sequenced based on difficulty scores generated from the formula for our participants

to enhance their listening ability. The findings of the study were presented in the

previous section. In this section, a discussion on the findings is given.

4.3.1 The Usefulness of the Online Materials

Based on the implications drawn from previous listening studies, we constructed

a website containing 66 listening texts with their MP3 files and comprehension

questions and incorporated the website to a listening and speaking course for the

purpose of eleven-week listening training. The website, Freshmen Listening, had

provided the learners extra opportunities to learn about listening with the help of

several useful functions embedded in the chosen MOODLE platform. To reiterate,

Freshmen Listening was able to track the learners’ activities on it; additionally, it also

offered instant feedback for the learners to check their learning after each attempt on

the tasks. Moreover, transcripts of the listening texts were also provided to promote

comprehension and language acquisition (Chapelle, 2003).

The usefulness of the online materials was first demonstrated by comparing the

learners’ scores in the pretest and the posttest. Analyses were carried out in two

directions, learners’ overall listening performance and their performance in the

separate listening skills. The data analysis showed that the learners improved

significantly in their overall listening ability in the posttest after the eleven weeks

(45.17 in the posttest and 42.75 in the pretest, T=34.5, Z=1.99, p<0.05). Although it is

not clear whether it was the online materials themselves, a maturation effect, or the

combined effect of the materials and the classroom teaching that led to the progress,



66

this finding still provided supportive evidence to the use of such a supplementary

website as learning and teaching aids, which some of the previous studies were not

able to do. For example, Cheng (2006) applied a readily available listening website

(Randall’s Cyber ESL Listening Lab http://www.esl-lab.com) for her learners as extra

practice opportunities. While the learners were found to be highly motivated about the

use of the website, their performance was not measured in the study. In Chen (2004),

a similar situation was also found. In other words, the two studies conducted in a

similar context as that of the current study, though their learners’ attitudes were

positively reported, the advantages of using listening websites in terms of enhancing

listening ability were not confirmed. Measurement of the learners’ performance after

they have used the CALL material is essential to effect evaluation research.

In addition to the overall listening ability, the learners were also found to benefit

in the skills of “listening for details,” and “summarizing.” Namely, the learners

became better at locating and memorizing detailed information, and summarizing the

texts after listening. Concerning their improvement in listening to details, it may be

attributed to the reinforced short-term memory due to the extra listening opportunities

provided by the website. And as the learners were able to recall detailed information

better from the listening, the progress they made in summarizing in the posttest should

not be too surprising. For the other three skills—listening for main ideas, interpreting

the speaker’s intent, and making inferences—significant improvement was not found.

The finding here may be in part due to the uneven arrangement of the online tasks and

the unequal distribution of the question number for each skill. In the online tasks,

“summarizing” was arranged in the required tasks in each week, hence the learners

were more likely to finish all the summarizing exercises throughout the eleven weeks

(the completion rate was 88.9%, see Table 4.20 for detailed information). For the

other four skills, such arrangement was not practiced, which made it possible for the



67

learners to miss questions of certain skills of the week and thus fail to improve in

those skills (See Table 4.21 for the distribution of question number of each skill). Skill

2—listening for details—was an exception because it has the largest number of

practice items, which enabled the learners to practice on it every week. This is another

explanation for the significant progress the learners made skill 2 in the posttest. The

finding here also indicated one limitation in the design of the tasks, namely, the

unequal distribution of the skill questions. Significant progress in the unimproved

three skills may be found once equal number of skill questions was assigned to the

required tasks each week. The assumption is based on a belief of the amount of

practice on the training effects.

Table 4.20 The number of learners who did the summary tasks across weeks

Week 1 2 3 4 5 6 7 8 9 10 11 Ave

Number 18 18 17 18 16 18 16 15 15 14 12 16

% 100 100 94.4 100 88.9 100 88.9 83.3 83.3 77.8 66.7 88.9

Note: The total student number=18

Table 4.21 The distribution of question number of each skill

Skill 1 Skill 2 Skill 3 Skill 4 Skill 5 Total

Question number 72 118 26 79 11 306

To explore more deeply the usefulness of the online listening materials, we

looked into the learners’ performance in the pretest and posttest in many other ways in

addition to comparing the learners’ pretest and posttest scores as a whole and in

individual skills. First of all, to understand whether the online materials were equally

beneficial to learners of different proficiency levels, the achievement of the

more-proficient group and the less-proficient group were compared. The findings



68

suggested that neither group improved significantly in the posttest, implying that the

materials were equally beneficial to both groups. Regardless the fact that our learners

were all college level EFL freshmen, individual differences still existed. Among our

eighteen participants, there were some who were very advanced, and also some below

average. Our online listening materials were selected based on the researcher’s

experience to fit the level of college level EFL freshmen in general. Since there were

both easy and difficulty materials in the website, the online materials seemed to meet

the levels of students in both groups.

A more in-depth investigation of the material usefulness was carried out by

looking at the relationship between the learners’ diligence levels (i.e., the number of

listening tasks completed throughout the practice period) and their posttest

achievement. Based on the learners’ logs on Freshmen Listening , every learner

completed 43.5 tasks on average across the eleven weeks. The learners were divided

into two groups based on their diligence levels. Despite that more improvement from

the diligent group was expected, statistical analysis did not show a significant

difference between groups. The finding indicated that regardless the number of tasks

finished by the learners, their achievement in the posttest did not vary as a function of

diligence levels. This may be ascribed to the small subject number in the study. As the

total subject number was not big (only 18 learners), further grouping made the

sub-group number even smaller, hence their performance is difficult to reach

significant differences. Moreover, looking closer to the achievement differences

between the two groups, more achievement was actually found in the less diligent

group, which was totally contradictory to our expectation. One explanation for this

finding may be the different start points of the both groups in the pretest (the more

diligent group=44.1, and the less diligent group=41.7, p<0.05). As the diligent

learners’ were relatively high in their pretest scores, there was less room for them to



69

improve to a statistically significant level. On the other hand, while some of less

diligent learners had a lower start in the pretest (28.5 and 38.5), it was more likely for

them to show greater improvement.

Another way to assess the material usefulness was by means of the learners’

performance on the online tasks throughout the eleven weeks. A dilemma was posed

here: on one hand, we expected to see a descending curve as conditioned by the texts’

increasing difficulty throughout the eleven week. Yet, when the difficulty levels of

texts were held as constant, an ascending curve could signify the learners’ learning, on

the other hand. The finding reported in the result section indicated that neither

condition was identified; instead, a more complicated curve was found. The finding

was not surprising because learners’ scores on the tasks may not be determined only

by the difficulty levels of the texts, but also by the students’ learning either from the

online tasks or the classroom-based teaching, their motivation, and other

environmental or learner factors. To reiterate, the texts across weeks were arranged in

accordance to their difficulty levels, namely, from easy to difficult. Hence, as the texts

became difficult, the learners were likely to perform worse. However, while learners

were practicing during the eleven weeks, they were also receiving classroom-based

instruction; and their learning from both sources was doubtlessly another factor

influencing the learners’ performance. This may also explain why the learners’

performance did not go down as the materials became more difficult. Based on the

findings in the part, it is clear that learning of listening skills is more complicated than

we have understood.

All in all, we have tried to probe into the usefulness of the online materials via

various means. Whereas pretest and posttest comparisons concerning the learning

listening performance had revealed the usefulness of the online materials,

comparisons of learner performance regarding proficiency and diligence grouping did



70

not show significance possibly due to small subject numbers. In addition, strong

evidence was not found in the learners’ weekly performance, either. This may be due

to other learner or environmental factors, or the combined learning effect from the

class and the website.

4.3.2 The Usefulness of the Proposed Formula or the Predefined Sequence of the

Materials

Material development is a critical issue in language teaching and learning. As in

all area of language learning, it is important that the materials presented to the

learners meet the learners’ proficiency levels. To measure difficulty levels of listening

texts, we had proposed a formula to determine the difficulty levels of our listening

texts, and by putting our materials into use, we tested the feasibility of using the

formula in grading listening texts. The usefulness of the formula was demonstrated in

three ways. First, we looked at the learners’ performance on the online tasks in the

website. Secondly, we examined the learners’ report in the weekly questionnaire and

in the final evaluation questionnaire. Last, we looked at the relation between the

learners’ weekly performance and the predefined sequence of the texts.

The learners’ weekly performance, as have been mentioned, was not supportive

enough to approve of the usefulness of the formula since a steadily descending curve

was not found. To repeat, a descending curve, which stands for the learners’ scores

across weeks, was expected in order to prove that the predefined sequence of the texts

indeed followed the manner—from easy to difficult. Nevertheless, as have been noted

previously, difficulty was not the only factor that could influence the learners’

performance. Instead, learner factors (e.g., the learners’ interests, their motivation,

their familiarity to the materials), and environmental factors (e.g., noise, time), may

also play roles in determining the learners’ scores. To illustrate, when listening to a

text with a boring or unfamiliar topic, the learners may not be able to concentrate as



71

much as when listening to a text with an interesting topic. Also, it is possible that the

learners might have been less willing to do the exercises during certain period of time

such as the mid-term week and at the end of the semester.

In addition to the learners’ weekly performance, their reports in both the final

evaluation questionnaire and the weekly difficulty questionnaire also provided insight

into the usefulness of the formula. In the final evaluation questionnaire, the learners

were asked if the texts across the eleven weeks were more and more difficult from the

beginning to the end. A moderate rating was given by the learners to the question (3.4

out of 5 in 5-point Likert scale question), which indicated that the learners were more

indecisive than positive to the effectiveness of the predefined sequence. In the weekly

difficulty questionnaire, the learners graded the easiest text of each week from five

aspects—vocabulary, speech rate, syntactic complexity, topic familiarity, and overall

difficulty. A similar dilemma was also found when dealing with this question, that is,

on one hand, we expected the learners’ grading showed a descending tendency which

indicated the texts’ increasing difficulty. Nevertheless, bearing in mind the

expectation of seeing improvement from the learners, a descending curve was

absolutely a finding we did not wish to obtain. Results in this part were similar to

those in the learners’ performance across weeks, that is, no clear pattern was found to

approve of the effectiveness of the sequence. Several reasons may provide

explanation to the finding. One straightforward reason contributing to the finding may

be that our sequencing using the formula was indeed ineffective, indicating needs for

further refinement of the proposed formula. Moreover, as when the learners started to

participate in the online training project, almost all of them (except one) were fresh

out the senior high school, and practicing listening online was a fairly new experience

for them. This may explain why the learners graded the texts in the first two weeks as

difficult compared to those in the following few weeks (see Figure 4.2 in the result



72

section). And as the learners became more experienced in online listening, they thus

graded the texts from week 3 to 7 as easier than week 1 and 2, even though the

grading within the weeks was also turbulent. As is clear in the figure, it is undeniable

that there was a tendency of grading the texts in the last few weeks as more difficult

than in the previous weeks. The finding suggested that even though we did not find a

curve supporting the effectiveness of the sequencing in a week-by-week manner; by

zooming out the curve, or looking at it in a more general way, it was demonstrable

that the texts in the last few weeks were indeed more difficult than those in the

previous weeks.

One interesting finding worth mentioning is that, by looking at the learners’

grading and weekly performance, a moderately positive correlation was found

between the learners’ performance and their difficulty rating in terms of vocabulary

(r=0.40) as well as syntactic complexity (r=0.41). Namely, when the learners graded a

text as difficult in vocabulary and syntactic complexity, they actually did worse on the

task. On the other hand, learners’ grading on speech rate, topic familiarity, and overall

difficulty was not found to be related to their task performance. These findings were

found to confirm but also contradict some previous research concerning listening

difficulties. In terms of syntactic complexity, our finding confirmed previous studies

in that, syntactic modified texts (less syntactic complicated) were beneficial to

comprehension (Chiang & Dunkel, 1990; Rubin, 1994). Yet, with much literature

indicating speech rate a profound factor affecting listening comprehension

(Flowerdew, 1994; Griffiths, 1992; Rubin, 1994; Teng, 2001), our finding showed no

clear relation between speech rate and task performance. It may be in part due to that

the speech rate in some of the texts was too slow to be beneficial to the learners, as

speech rate that is too slow comparing to the normal speech may also hinder

comprehension (Griffith, 1992). Hence, whereas the learner may have graded a text as



73

easy because of slow speech rate, they were not necessarily able to show the benefit

they got from the lower rate in their task performance because the perceived slow rate

was actually a factor that hindered their comprehension.

To sum up this section, in the current study, we have tried to test the feasibility of

the proposed formula via various means. Strongly supportive attitudes to the

effectiveness of the predefined sequence (or the formula) were not found in the

evaluation questionnaire. As to the weekly questionnaire, a supportive pattern was not

found in a week-by-week examination, either. However, by looking at learners’

grading from a more general view, the learners’ grading seemed partially congruent

with the sequencing. Either case, however, suggested that the proposed formula may

need further refinement to be effective.

4.3.3 Learners’ Perceptions of the Online Materials

Additionally to the satisfactory findings concerning the usefulness of the online

materials obtained based on the pretest and posttest comparison, learners also reported

very positive attitudes toward the use of Freshmen Listening as supplementary

practice materials to the classroom-based learning. The finding was coherent with the

findings in Cheng (2006) and Chen (2004), in which the learners expressed highly

positive attitudes to the use of online listening materials. Based on their report in the

final evaluation questionnaire, satisfactory ratings were given by the learners to the

statement “the website and the online listening tasks are useful to improve my overall

listening ability,” and statements concerning the usefulness of the website to their five

listening skills (see Table 4.16 in the result section). More positive attitudes were

reported when being asked, “is it beneficial to have resources like the listening

website accompanying with listening courses?” (rating=4.4) and in “would you like to

use such listening website to enhance your listening ability in the future?”



74

(rating=4.1). The fairly high rating given by the learners suggested that learners

believed that by practicing on the website their listening proficiency would improve in

time.

Based on the learners’ reports from the evaluation questionnaire, the beneficial of

CALL to the learners’ learning process was further confirmed. From the learners’

background information, we found that though all of the learners showed high

motivation in improving their English listening proficiency, yet about two-thirds of

the learners have not tried to use listening websites as a practice tool. In the project,

the learners were introduced a listening website, and they indicated in the evaluation

questionnaire that they had became aware of the potential benefits embedded in this

kind of listening websites. To the questions asking whether the learners improved

their overall listening ability and the five target listening skills, fairly high ratings

were given by the learners (4.0 for overall listening ability and interpreting the

speaker’s intent, 4.1 for listening for main ideas, and 3.8 for listening for details). The

learners’ awareness of the benefits of the website was also evidenced from their

comments reported in the questionnaire: “the website did give me extra opportunities

to practice my listening,” and “the online materials did help enhance my listening

ability.” One learner even asked in the questionnaire where she could find free

listening websites like the one used in this project for her to practice. The learners’

reports are incontestably encouraging to the use of online listening materials to supply

regular listening courses. With the easily accessible materials, the learners were able

to attend to the extra practice whenever they have time, and wherever there were

computer facility.



75

Chapter Five

CONCLUSION

5.1 Overview

With regard to the importance of listening and the pedagogical need of a grading

mechanism for listening texts, this study tried to address the issue of how useful it can

be to infuse online listening materials into a listening and speaking course with the

purpose of enhancing the learners’ listening ability. Additionally, it also attempted to

develop a grading mechanism specifically for “listening texts,” and based on it, a

listening website consisting of sixty-six graded listening texts sequenced from easy to

difficult and their comprehension questions was constructed and infused into a

listening and speaking class. Apart from the usefulness of the online listening

materials and the proposed grading formula, another concern in this study is to

understand if there were learner differences that had impacted on the learners’ final

achievement. Finally, the study also takes an interest in exploring the learners’

attitudes toward the website and the online materials after the use for eleven weeks.

Eighteen EFL college freshmen were recruited as participants of this study and were

asked to do three out of six listening tasks each week for eleven weeks. Additionally

to the weekly tasks, which the learners’ performance was automatically recorded by

an embedded tracker program, a pretest, a posttest, a weekly difficulty questionnaire

and a final evaluation questionnaire were used as instruments for data collection.

The results of this study showed that it is useful and beneficial to infuse online

listening materials into listening and speaking classes in two ways. Concerning the

learners’ achievement after the eleven-week practice, they improved not only in their

overall listening ability, but in specific listening skills as well (i.e., listening for details

and summarizing). In terms of their perceptions of material usefulness, most learners



76

agreed that the online materials were beneficial in enhancing their overall listening

ability and specific listening skills. However, based on the learners’ weekly

performance throughout the eleven weeks, definite answer to the materials usefulness

was not found. The finding indicates that learning of listening skills is more

complicated than we have understood. To the question of the formula usefulness, no

definite answer was found either from the learners’ weekly performance on the

website or from their weekly rating of the texts’ difficulty levels. This was because,

first, the learners’ scores on the weekly tasks, which we used to concretize the text

difficulty, were actually not precise enough to reflect the difficulty levels of the

materials. Moreover, the learners’ performance on the tasks may also rely on other

factors such as learner factors (e.g., motivation, familiarity, interest) and

environmental factors (e.g., time). Regarding learner differences, by grouping the

learners based on their proficiency and diligence levels, we did not find apparent

differences that affected the learners’ achievement in the posttest. In other words, the

online listening materials seemed to be equally beneficial to all of the learners,

regardless their proficiency levels. In terms of the relation between the learners’

diligence levels and their achievement, no conclusive answer was found due to the

small subject number in this study, and the different proficiency levels of the learners

in both groups. Finally, With regards to the learners’ attitudes to the online materials,

very positive responses were found from the evaluation questionnaire. In the

questionnaire, the learners not only regarded that the materials were beneficial for

them to improve their listening skills, but also expressed high motivation of using

listening websites to get extra learning opportunities.

In the following section, the limitations of this study, directions recommended

for the future research, and the pedagogical implications are presented.



77

5.2 Limitations of the Study

The participants recruited in this study were from an intact class of college level

EFL freshmen. To address how online materials may affect their development of

listening skill, the texts used in the listening websites were selected depending on our

specific learners’ proficiency levels. Therefore, the tasks in the website may not be

readily usable for learners of other proficiency levels. Moreover, because the

experiment started at the very beginning of the teaching year of 2006, and the learners

were all first-year students, we were not able to know their proficiency levels until the

administration of the pretest. Additionally, during the selection of target texts, the

decision of which texts to include was based on the researcher’s experience

exclusively. Since such selection should rely on the teacher’s understanding of the

learners’ proficiency levels, by gathering target texts before knowing the learners, we

actually ran a risk in the materials selection process in that we may have included

texts of inappropriate difficulty levels and topics.

Secondly, to train the learners of the five target listening skills, comprehension

questions representing each of the five skills were designed. However, as the

development of the skill questions largely depended on the nature of the texts, equal

question numbers for each skill was not achieved. Accordingly, the questions

representing each skill were not distributed equivalently in the required tasks and

optional tasks. That is to say, learners may not be able to encounter every question

type in the required tasks each week with equal amount of practice. This limitation

therefore resulted in the learners’ insignificant achievement in some the skill of

listening for main ideas, interpreting the speaker’s intent, and making inferences.

Apart from the concerns in the materials development phase, incompleteness of

research design may also in part contribute to some of the insignificant findings

regarding the learners’ improvement. To begin with, the subject number was small,



78

making it hard to achieve statistical significance. Secondly, this study adopted the

one-group pretest and posttest design; in other words, a control group which served as

the baseline for comparison was not included. A more influential limitation was the

non-intensive nature of the online training project. During the training period, the

learners were asked to do three tasks each week. Based on the learner logs in the

website, it was found that more than half of the learners did the weekly required tasks

at the time when the tasks were about to be closed. This condition suggested that most

learners had access to the website only once a week, hence, a period of eleven weeks

thus did not seem to be sufficient enough to make differences in some specific

listening skills.

Finally, in this study, we made a pioneer effort in developing a grading

mechanism specially for listening texts. However, as there were many factors that

play roles in listening difficulty other than speech rate, syntactic complexity, and

vocabulary, our formula, which concerned only these three factors, may seem too

shallow. There were many other factors that should be taken into consideration such

as accents of the speakers, intonation, text type, and topic familiarity.

5.3 Directions for Future Research

Given the limitations of the current study mentioned in the previous section, we

recommend some directions for future research.

First of all, concerning the research design, more participants should be recruited,

and a control group should also be included as a basis of comparison. Moreover, in

order to achieve a more conclusive answer to the question of how online listening

materials could benefit EFL learners’ learning of listening, a more intensive training

syllabus is recommended. In other words, the training syllabus should be able to

engage the students in practicing for a longer period. Further, since uneven



79

distribution of the item number of each listening skill may in part account for the

learners unimproved performance in some of the listening skills, it is suggested that

future studies try to reach a balance in item number among the target skills.

Secondly, it is suggested that a listening test containing the materials used during

the eleven-week period be used in the pretest and in the posttest. In this study, one

independent listening test adopted from the course book of the listening and speaking

class was used in the pretest and posttest. However, the test itself did not consist any

of the texts from the 66 online practice texts, regardless that the items in the test were

also measuring the five target listening skills as the eleven-week practice. The

unmatched content between the test and the practice was in fact a threat to the validity

of the test and thus to the findings in this study. Hence, future studies were

encouraged to adopt more tests as a measurement of the learners’ achievement,

especially one that is reflective to the training materials.

Thirdly, as we did not reach a conclusive answer concerning the usefulness of the

proposed formula, and the incomplete design of the weekly difficulty questionnaire

seemed to play a role in such findings, it is encouraged that a refined version of

questionnaire be used. Our weekly difficulty questionnaire asked the learners’

perceptions of the difficulty level of the easiest text each week from five aspects:

vocabulary, speech rate, syntactical complexity, topic familiarity, and overall difficulty.

Since the perceptions of difficulty is comparative by nature, such questions may not

be specific and concrete enough. We recommend that future researchers who are

interested in this issue use more concrete questions to elicit the learners’ perceptions.

For example, instead of asking the difficulty level of a certain text, it may be better to

ask the learners to rank the difficulty levels of the required texts each week. Hence,

questions may be phrased as “please rank the texts from easy to difficult using 1 to 3,”

or “please mark the easiest text of this week.”



80

Finally, the formula proposed in this study was a pioneer effort we made on the

development of grading mechanism for listening texts, and weights assigned to each

difficulty measure in the formula were also tentatively proposed. As we were not

certain about how influential each difficulty measure was in determining listening

difficulty, it is recommended that future studies try to figure out the importance of

each difficulty measure, and then, based on the understanding, refine the current

formula. Future studies which are interested in the issue may try to sequence the

materials from easy to difficult separately based on their speech rate, syntactic

complexity, and academic word ratios. And by examining the relation between the

learners’ performance throughout the practice period and the three different sequences,

we may be able to figure out the importance of each difficulty measure.

5.4 Pedagogical Implications

In the present study, positive attitudes to the online materials and the listening

website were found in the learners’ responses to the final evaluation questionnaire.

This indicated that learners enjoyed learning via online materials which are accessible

anytime and anywhere. Accordingly, a paradigm of incorporating listening websites as

supplementary learning materials and listening instruction was recommended.

To begin with, a website equipped with a tracker program should be constructed

to enable the teachers to hold control on the students’ learning. Secondly, appropriate

listening materials with their MP3 files and transcripts should be gathered. And

comprehension questions should also be offered for the learners to check their

comprehension level and to locate listening problems. Thirdly, an orientation session

will be necessary before letting go the project in order to familiarize the learners with

the online learning environment. That is to say, learners should be informed explicitly

which elements (i.e., listening exercises, transcripts, and downloadable MP3 files) in



81

the website to use to foster their learning.

Based on the findings of this study, it is certain that the learning of listening is

more complicated than we have understood, there are so many factors influencing the

learners’ performance and achievement other than the materials’ difficulty levels. Yet,

as online listening materials or listening websites did seem to motivate the learners in

their learning of listening skill, it is strongly suggested carefully selected online

materials be used to supplement with the formal classroom-based instruction to the

purpose of enhancing the learners’ listening ability.



82

REFERENCES

Anderson, A., & Lynch, T. (1988). Listening. Oxford: Oxford University Press.

Berne, J. E. (2004). Listening Comprehension Strategies: A review of the literature.

Foreign Language Annals, 37, 521-533.

Blau, E. K. (1990). The effect of syntax, speed, and pauses on listening

comprehension. TESOL Quarterly, 24, 746-753.

Brindley, G., & Slatyer, H. (2002). Exploring task difficulty in ESL listening

assessment. Language Testing, 19, 369-394.

Brown, G., & Yule, G. (1983). Teaching the spoken language. Cambridge: Cambridge

University Press.

Buck, G. (2001). Assessing Listening. Cambridge: Cambridge University Press.

Carrell, P. L. (1983). Three components of background knowledge in reading

comprehension. Language Learning, 33, 183-207.

Carrell, P. L., & Eisterhold, J. C. (1983). Schema theory and ESL reading pedagogy.

TESOL Quarterly, 17, 553-573.

Carrol, J. B. (1972). Defining language comprehension. In R. D. Freedle and J. B.

Carrol (eds.) Language comprehension and the acquisition of knowledge. New

York: John Wiley and Sons.

Chang, C. S. (2004). An investigation of listening support in test takers’ performace in

EFL listening tests. In Proceedings of the Thirteenth Conference on English

Teaching and Learning in the Republic of China, (pp. 239-249). Taipei: Crane.

Chen, H. J. (2004). Developing a web-based listening center for language teachers

and learners. The Proceedings of the 2004 International Conference and

Workshop on TEFL & Applied Linguistics, (pp. 87-95). Taipei: Crane.

Cheng, C. L. (2006). Implementation and evaluation: Integrating English learning

websites into college listening comprehension instruction. The Proceedings of



83

the 23rd International Conference on English Teaching and Learning in the

Republic of China-changes, chances, and challenge in English Teaching and

Learning, (pp. 83-95). Taipei: Kuan Tang International Publications Ltd.

Cheng, H. F. (2004). A comparison of multiple-choice and open-ended responses

formats for the assessment of listening proficiency in English. Foreign

Language Annals, 37, 544-555.

Chiang, C. S., & Dunkel, P. (1992). The effect of speech modification, prior

knowledge, and listening proficiency on EFL lecture learning. TESOL

Quarterly, 26, 345-374.

Clark, H. H., & Calrk, E. V. (1977). Psychology and language: An introduction to

psycholinguisitics. New York: Harcourt Brace Jovanovich.

Connor, U. (1984). Recall of text: Differences between first and second language

readers. TESOL Quarterly, 18, 239-256.

Coxhead, A. (2000). A new academic word list. TESOL Quarterly, 34, 213-238.

Dunkel, P. (1988). The effects of delivery modification on notetaking and

comprehension of Ls lectures. Journal of Intensive English Programs, 2, 41-52.

ETS. (2006). The official guide to the new TOEFL iBT. New York: McGraw-Hill.

Flaherty, S. E. (1979). Rate-controlled speech in foreign language education. Foreign

Language Annals, 12, 275-280.

Flesch, R. (1949). The art of readable writing. New York: Harper.

Flowerdew, J. (1994). Research of relevance to second language lecture

comprehension—An overview. In J, Flowerdew (Ed.) Academic listening:

Research perspectives. Cambridge: Cambridge University Press. 7-29.

Flowerdew, J., & Miller,L. (2005). Second language listening: Theory and practice.

Cambridge: Cambridge University Press.

Ghadirian, S. (2003). Providing controlled exposure to target vocabulary through the



84

screening and arranging of texts. Language Learning & Technology, 6 (1),

147-164

Grgurovic, M., & Hegelheimer, V. (2007). Help options and multimedia listening:

students’ use of subtitles and the transcript. Language Learning & Technology,

11, 45-66.

Griffiths, R. (1992). Speech rate and listening comprehension: Further evidence of the

relationship. TESOL Quarterly, 26, 385-391.

Hatch, E., Lazaraton, A (1991). The research manual: Design and statistics for

applied linguistics. New York: Newbury House Publishers.

Hinkle, E. (2006). Current perspectives on teaching the four skills. TESOL Quarterly,

40 (1), 109-131.

Huang, H. T. (2003). Effects of Graded Texts on EFL College Students’ Incidental

Vocabulary Learning: Text Difficulty and Exposure Amount. Unpublished MA

thesis. National Tsing Hua University, R.O.C.

Johns, A. M., & Mayes, P. (1990). An analysis of summary protocols of university

ESL students. Applied linguistics, 11, 253-271

Krashen, S. (1985). The input hypothesis. New York: Longman.

Lynch, T. (1998). Theoretical perspectives on listening. Annual Review of Applied

Linguistics, 18, 3-19.

Mendelsohn, D. J. (1995). Applying learning strategies in second/foreign language

listening comprehension lesson. In D. J. Mendelsohn, & J. Rubin (Eds.), A

guide for the teaching of second language listening, (pp. 132-150). San Diego,

CA: Dominie Press.

Mendelsohn, D. J. (1998) Teaching listening. Annual Review of Applied Linguistics,

18, 81-101.

Morley, J. M. (1999). Current perspectives on improving aural comprehension.



85

Retrieved from http://www.eslmag.com/MorleyAural Story.htm.

Nation, P. (2001). Learning vocabulary in another language. Cambridge, U.K.:

Cambridge University Press.

O’Malley, J. M., & Chamot, A. U. (1990). Learning strategies in second language

acquisition. Cambridge: Cambridge University Press.

Oxford, R. L. (1990). Language learning strategies: What every teacher should know.

New York :Newbury House Publisher.

Priess, S. (2004). NorthStar: Listening and speaking, Advanced. NY: Pearson

Education Inc.

Ramirez, D. V., & Alonso, I. B. (2007). Using digital stories to improve listening

comprehension with Spanish young learners of English. Language Learning &

Technology, 11, 87-101.

Rixon, S. (1986). Developing listening skills. London: Macmillan.

Rost, M. (1990). Listening in Language Learning. Harlow: Longman.

Rost, M. (2002). Teaching and Researching Listening. London: Longman.

Rubin, J. (1994). A review of second language listening comprehension research. The

Modern Language Journal, 78, 199-221.

Rubin, D., & Rafourth, B. (1984). Oral language: a criterion for selecting listenable

materials (Unpublished paper).

Shohamy, E., & Inbar, O. (1991). Validation of listening comprehension tests: the

effect of text and question type. Language Testing, 8, 23-40.

Smidt, E., & Hegelheimer, V. (2004). Effects of online academic lectures on ESL

listening comprehension, incidental vocabulary acquisition, and strategy use.

Computer Assisted Language Learning, 17 (5), 517-556.

Su, S. W. (2003). The effects of text types on listening comprehension. Proceedings of

2003 International Conference on English Teaching and Learning in the



86

Republic of China, ( pp, 301-310). Taipei: Crane.

Sullivan, P. N., Brenner, G. A., & Zhong, Y. (2004). Master the TOEFL 2005. New

York: Thomson Arco.

Teng, H. C. (1998a). The effects of text types and task types on English listening

comprehension. English Teaching Journal, 23, 5-18.

Teng, H. C. (1998b). A study of EFL listening comprehension strategies. Paper

presented at the Annual Convention and Exposition of the Teachers of English

to Speakers of Other Languages. Retrieved Aug 3, 2006, from

http://www.eric.ed.gov/ERICDocs/data/ericdocs2/content_storage_01/0000000

b/80/11/0b/45.pdf

Teng, H. C. (2002). Effects of syntactic modification and speech rate on listening

comprehension. In Pr oceedings of the Tenth Conference on English Teaching

and Learning in the Republic of China, (pp. 588-597). Taipei: Crane.

Vandergrift, L. (1999). Facilitating second language listening comprehension:

acquiring successful strategies. ELT Journal, 55, 168-176.

Vandergrift, L. (2004) Listening to learn or learning to listen. Annual Review of

Applied Linguistics, 24, 3-25.

Vandergrift, L. (2006). Second language listening: listening ability or language

proficiency? The Modern Language Journal, 90, 6-18.

Vogely, A. J. (1995). Perceived strategy use during performance on here authentic

listening comprehension tasks. Modern Language Journal, 79(1), 41-56.

Warshauer, M., & Healey, D. (1998). Computers and language learning: An overview.

Language Teaching, 31, 57-71.

Weir, C. (1993). Understanding and developing language tests. New York: Prentice

Hall.

Wong-Fillmore, L. (1991). Second language learning in children: a model of language



87

learning in social context. In Bialystok, E. (ed.), Language processing in

bilingual children. Cambridge: Cambridge University Press.



88

APPENDICES

Appendix A

A Sample of the Online Task

Discovery’s space mission

This week, the space shuttle Discovery flew to the International Space Station as

NASA struggles to meet an important date. A plan to complete the station by 2010 is

at risk.

This is only the second shuttle flight since two thousand three. In February of that

year, the space shuttle Columbia broke apart as it prepared to land. The accident killed

the seven crew members.

Now, just short of a year has passed since the return to flight.

Plans call for sixteen shuttle flights by two thousand ten. NASA, the National

Aeronautics and Space Administration, has fallen behind in its effort to reach that

goal.

The goal is part of a plan that President Bush announced two and a half years ago to

send astronauts to the moon again.

Government money would finance a new spaceship that could take people to the

moon by two thousand twenty. The last time anyone went there was in nineteen

seventy-two. The plan also calls for traveling to Mars.

But Mister Bush said the first goal was to finish the space station by two thousand ten,

to study the long-term effects of living in space. Fifteen other nations are also

involved in the space station.

NASA plans to retire its three remaining shuttles once the station is completed.

This week, Discovery became the first shuttle launched on America’s Independence

Day. It lifted off with a crew of seven on Tuesday from the Kennedy Space Center in

Florida.

Bad weather had delayed the launch. Also, there had been some concerns about the

safety of the foam protective material on the external fuel tank.During the Columbia launch, a piece of material fell off the fuel tank and struck a

wing. The piece weighed more than seven hundred grams. It put a hole in the heat

shields and the shuttle came apart on re-entry.

A small amount of foam did come loose from the fuel tank on the Discovery. But

officials decided it was not enough to be dangerous. Also, astronauts are examining

the heat shields while at the space station.

If any damage were serious, an emergency plan calls for the astronauts to remain on

the station. NASA would then send up another shuttle to return them to Earth.



89

Discovery carried up thousands of kilograms of equipment and supplies. On Friday,

crew members connected a big storage container to the station. The Italian-made

container is called Leonardo.

The shuttle also brought a German astronaut who will remain on the station for six

months. The arrival of Thomas Reiter means a full three-person crew for the first time

since May of two thousand three.

The other two crew members, Pavel Vinogradov of Russia and American Jeff

Williams, arrived on a Russian Soyuz spacecraft in March.

Comprehension Questions

In this listening task, you will hear a passage talking about a space mission. Listen

carefully and answer the following questions.

You can download the file and listen as many times as you need, but only ONE

attempt on the task is allowed.

1. What is the main idea of this passage? (Listening for main ideas)

Recent NASA space project

Discovery’s flight to the moon

NASA’s plan of traveling to Mars

Columbia’s crash during their returning flight2. How many shuttles does NASA plan to send to the space by the year of 2010?

(Listening for details)

13

14

15

16

3. Based on the passage, what is the main goal of Discovery and NASA’s space plan

by 2010? (Making inferences)

Taking people to the moon

Taking people to the Mars

Finishing the space station

Building more spaceships

4. According to the passage, what caused Columbia to crash? (Making inferences)

The crew members were not well trained.

The shuttle was not well built.

The loading was too heavy for the shuttle.

The connection between the shuttle and NASA was in a mess.



90

5. According to the passage, which of the following statement(s) is/are true?

(Making inferences)

Columbia broke apart in the year of 2003.

The long-term goal of NASA’s recent space project is to study the effects of

living in the space.

Discovery was the second shuttle launched on the Independence Day.

NASA will send another shuttle to pick up Discovery’s crew members no matter

what.

6. Please summarize the passage in a short paragraph in less than 120 words. In your

summary, both important information and details should be provided.

(Summarizing)



91

Appendix B

The Pretest and the posttest

TEST 1

C. Listen to the excerpt. Mark the sentences T (true) or F (false).

_____ 1. The speaker is an employer.

_____ 2. The speaker does not trust the people who work at his business.

_____ 3. Trusting his employees gives the speaker satisfaction.

D. Listen to the excerpt again. Circle the answer that best completes each

sentence.

1. According to the speaker, an employer should ________.

a. tell his employees what is right and wrong

b. watch his employees closely

c. let his employees decide what is right and wrong

2. According to the speaker, being “Big Brother” is ________.

a. an effective management strategy

b. an ineffective management strategy

c. neither effective nor ineffective

3. When the speaker says “go that extra mile,” he means ________.

E. exercising by running an extra time

F. getting satisfaction and rewards

G. working extra hard

TEST 2

H. Listen to the excerpt. Mark the sentences T (true) or F (false).

_____ 1. According to the speaker, the Red Cross believes that war can be

abolished.

_____ 2. The Red Cross equates war with barbarism.

_____ 3. According to the speaker, war is sometimes desirable.

I. Listen to the excerpt again. Circle the answer that best completes each

sentence.

1. According to the speaker, oppressed groups ________.

a. should never use war to free themselves

b. use barbarism in war

c. sometimes must use war to free themselves

2. A rule that the Red Cross does NOT try to enforce is: ________.

a. Don’t shoot prisoners

b. Don’t kill people



92

c. Don’t make war on noncombatants

3. According to the speaker, it is important to distinguish between ________.

d. oppression and war

e. ethnic groups

f. war and barbarism

TEST 3

J. Listen to the excerpt. Mark the sentences T (true) or F (false).

_____ 1. Children are born with mathematical ability.

_____ 2. Exposure to music helped preschoolers with their spatial reasoning.

K. Listen to the excerpt again. Circle the answer that best completes each

sentence.

1. The speaker describes music as ________.

a. not very logical

b. highly mathematical

c. geometric

2. Preschoolers were taught ________.

a. to solve mazes

b. to copy geometric shapes

c. singing and piano

3. The wiring for mathematics and music is located ________.

a. on the right side of the brain

b. on the left side of the brain

c. at the base of the brain

TEST 4

L. Listen to the excerpt. Mark the sentences T (true) or F (false).

_____ 1. Broadcasters support the V-chip/ratings system.

_____ 2. Broadcasters trust the public to use the V-chip. _____ 3. Supporters of freedom of speech feel that V-chip/ratings system is

censorship.

M. Listen to the excerpt again. Circle the answer that best completes each

sentence.

1. Broadcasters think the public ________.

a. can be trusted to use the V-chip consistently

b. is hypocritical

c. wants less sex and violence on TV



93

2. The Telecommunications Act ________.

a. ordered the establishment of a TV ratings system

b. ordered broadcasters to control their own programming

c. ordered the establishment of the V-chip system

3. The FCC (Federal Communications Commission) can ________.

a. control the pretests of free speech supporters

b. set up its own ratings system

c. reject the Telecommunications Act



94

Appendix C

Pretest Video Script

The Age of E-mail

(3:27 minutes)

Narrator: When did it happen? How did it happen? It seems like just yesterday we

put a stamp on an envelope and sent it off in the mail. We’d sometimes wait days or

even weeks for a reply. Nowadays, our letters, messages, and business transactions are

instantaneous. And we can get the answers we need immediately.

There are very few people who can tell you who invented e-mail or exactly when

it became such a permanent part of our lives. But in 1971 an engineer created the first

system for sending an e-mail between computers. What seemed like a very small

invention currently has more than 125 million users and has profoundly changed the

way we lived and work.

Woman 1: It’s replaced the telephone. It’s replaced—what we call those

things?—teletypes, quixes, (mail) faxes, (mail) mail, snail mail.

Man 1: On the positive side you can, you know, instead of trying to voice-mail back

and forth and play telephone tag, you can reach a person pretty quickly.

Man 2: I guess it makes everything easier even when you can’t get in touch with

someone, you can always e-mail them, even if they’re, like, moved out of states or friends that go away to college and stuff like that.

Woman 1: I spend a couple of hours a day doing e-mail. It’s changed quite a lot. But I

also can communicate with twenty-seven people at one time and either give or collect

information very quickly.

Man 3: I think it's the perfect way of communicating because you don't have to reach

people. They can get there, get your e-mail when they want to download it—

Narrator: But why is it so popular? In the last ten years e-mail has caught on in a big

way. At work and at home, kids or seniors, we are all able to keep in touch with

co-workers, friends, and family in over 150 countries.

Despite the positive aspects of using e-mail at work, some professionals have

complaints.

Man 4: You can spend your whole day responding to e-mails and you really get

anything done that you want to get done.

Man 5: When you come in on a Monday morning, at least when I do, I probably get

150 garbage mails that I have to go through, get each one of them out there, so you’re

like, you have to go on vacation ‘cause there’ no way to just shut all that off. It’s a

small downside, but it is a downside.



95

Man 4: Well, I think it replaces some of the real personal communication. I think that,

you know, it isn't as effective as having a conversation. I think things can be

misconstrued. If I say something to you with a laugh in my voice that might—you

know, it's not what you say, it's how you say it. So when you are typing, it’s hard to

get across what your true feeling is.

Narrator: According to journalist and author Malcolm Gladwell, when little things

make a big difference in out lives, that constitutes “the tipping point.” Gladwell’s

book explains why change can happen quickly and unexpectedly.

He might even call e-mail a “social epidemic.” With access to e-mail available

almost anywhere, there is no way to escape it.

As e-mail becomes more rooted in today’s society, we will have to accept the

good and bad that go along with its growth.



96

Appendix D

Posttest Video Script

Smoking

(3:31 minutes)

Narrator: We all know that smoking can ruin your health, but a lot of us smoke

anyway. What’s the attraction? Why did we start? And how do we stop?

As you know, addiction is an unhealthy continued involvement with a

mood-altering substance or activity in spite of harmful consequences.

Tobacco use is the single most preventable cause of death in the United States.

Each year over 419,000 people in the United States die of smoking related illnesses.

Another 53,000 die of illnesses caused by secondhand smoke. Smoking contributes to

about one in every seven deaths in the United States—all of them unnecessary.

Dr. Jack Dillenberg: We find that the amount pain and suffering in illness that

tobacco causes is extraordinary. There is no one thing that a human being can do to

hurt their health more than to use tobacco. It is—that is statistically above everything

else.

Narrator: People who start using tobacco sometimes think, “I won’t get addicted.”

But addiction to nicotine, the drug that is found in tobacco products, is virtually

impossible to avoid if you use tobacco regularly. Nicotine is found in all tobacco products including cigarettes, cigars, pipe, tobacco,

chewing tobacco, and snuff.

The risks associated with tobacco are not limited to the physiological or

emotional addiction. Short-term effects include increased heart rate and blood

pressure, interference with blood flow and airflow to the lungs, and smoker’s breath.

Of more serious concerns are the long-term effects of tobacco use. These can be

deadly. They include heart disease, stroke, cancer, and emphysema, just to name a

few.

Today many college students have gotten the message about harmful effects of

tobacco. They consider smoking a thing of the past that has damaged the health of

their parents and older relatives.

Girl 1: I had an uncle who died from lung cancer, I saw what he had to go through

and I don't think anyone should have to go through that.

Narrator: But, unfortunately, smoking has not yet gone the way of the dinosaur.

About 25 percent of the adult population in the United States continues to smoke. And

3000 teenagers start smoking every day, with all the evidence about the hazards of

tobacco, cancer, cardiovascular disease, respiratory disease, gum disease, and



97

premature death, why do so many people continue to take up the habit?

Boy: It takes my mind off other things. It calms me down.

Girl 2: Cause I started and I couldn't stop.

Narrator: What can be done so that teenagers, and especially minority teenagers, get

the message that using tobacco is not cool? One strategy is anti-smoking ads and

slogans aimed at educating preteens and teenagers about the health consequences of

tobacco.

The evidence is clear. If you use tobacco, it’s worth it to quit and it’s worth it to

quit right now.



98

Appendix E

The Weekly Questionnaire

i. I can understand _____ percent of the words in this listening

task.

□ almost 100%

□ More than 80%

□ 50% to 80%

□ Less than 50%

□ Less than 20%

ii. I think the speed of this listening is ______.

□ Too fast to understand

□ Fast but understandable

□ About right for me□ Slow

□ Too slow

iii. I think the grammar in this listening text is ______.

□ Too complex to understand

□ Complex to understand


□ Easy to understand

□ Very easy to understandiv. The topic of this listening text is ______.

□ Very familiar to me

□ Fairly familiar to me


□ Unfamiliar

□ Very unfamiliar to me

v. Overall, I think this listening text is ______.

□ Too difficult

□ Difficult

□ Intermediate (about right for me)

□ Easy

□ Too easy



99

Appendix F

The Evaluation Questionnaire

Name: _______________ Student ID: ______________ Male/Female

Age:________

SA=strongly agree, A=agree, U=undecided, D=disagree, SD=strongly disagree

Part A: Personal Information

1. My overall English proficiency is at _____ level.

Primary Low-intermediate Intermediate High-intermediate Advanced

2. My English listening proficiency is at _____ level.

Primary Low-intermediate Intermediate High-intermediate Advanced

3. I am eager to improve my listening ability

SA A U D SD 4. I have practiced English listening via other listening websites like the one used this

semester.

Yes No (skip Q5 and Q6)

5. What websites did you used to improve your listening? (e.g., ICRT, Studio

Classroom, etc.)

_____________________________________________________________________

6. How often do you use those websites?

_____________________________________________________________________

Part B: The usefulness of the website we designed

I. The layout of the website (color, arrangement, user friendliness, clearness of

instruction, etc.)

7. I think the layout of the website (the color and the arrangement) is clear and easy to

follow.

SA A U D SD

8. The website is easy to use, and I get used to it very easily.

SA A U D SD

Thank you very much for completing the online tasks in the past 11 weeks. Weappreciate your efforts. Now we would like to know about what you think about

the online materials in order to help us revise them in the future. Please circle the

answers that best describe your thoughts.



100

II. The online exercises

9. The topics of the online listening tasks are interesting and motivating.

SA A U D SD

10. There are two types of excerpts during the 11 weeks— conversation between

people and lectures/ news reports by one person —which one do you find more

challenging?

Conversation between people Lectures/ news reports

11. There are some factors that may influence the difficulty level of the listening

materials. Based on the listening materials that we have used in this semester,

please rate the following five factors from 1-5 according to the level of their

influences. (Note: ‘1’ indicates the most influential and ‘5’ the least

influential)□ speech rate □ text types (dialogue or lectures/news report) □

vocabulary

□ sentence structure □ topic familiarity

12. The summary task is more difficult than all the multiple-choice questions.

SA A U D SD

13. The website and the online listening tasks are useful to improve my overall

listening ability.

SA A U D SD14. The online practice improves my ability of listening for the main idea of the

passages.

SA A U D SD

15. The online practice improves my ability of listening for the detailed information

of the passages.

SA A U D SD

16. The online practice improves my ability of understanding the speaker’s intention

or thoughts.

SA A U D SD

17. The online practice improves my ability of understanding the implied information

of the passages.

SA A U D SD

18. The online practice improves my ability of summarizing the listening content .

SA A U D SD

19. The online tasks are more and more challenging from the beginning to the end of

the 11 weeks.

SA A U D SD



101

20. I performed better on the listening tasks at the beginning than at the end in the past

11 weeks.

SA A U D SD

21. It is beneficial to have resources like this listening website accompanying with

listening courses.

SA A U D SD

22. I would like to use such a listening website to enhance my listening ability in the

future.

SA A U D SD

23. Please write down you comments or suggestions about the website or the online

listening materials for our future revision.

_____________________________________________________________________

_____________________________________________________________________

_____________________________________________________________________

Please make sure that you answer EVERY question before you submit

the questionnaire. Thank you again for completing the questionnaire!



102

Appendix G

The Consent Form

Dear students:

Welcome to the Department of Foreign Languages and Literature in Tsing Hua.

Judy Lo, my advisee, is conducting her thesis project and need to use your online

performance and some in-class tests for the research. By the time of reporting the

research, no participants' names or the institution name will be revealed. We need

your help and agreement of doing this. Thank you in advance for the attention. We

will be happy to answer any questions concerning data collection. Later, we will

provide a small gift as a token reward for you. Thank you very much.

Judy, Lo

Hsien-Chin Liou, Sept, 2006

Signature______________ Cell phone________________ Date _______________

If you have further questions about this project, you are welcome to send emails toJudy at [email protected].



103

Appendix H

Answer Key (Idea Units) to the Pretest Summary

Your summary should cover the following points. Check if you include all

these in your summary.

1. This excerpt talks about the use, and the good and bad of email. / Email has

profoundly changed the way we live and work.

2. In 1971, a group of engineers invented the first system of sending email through

computers.

3. Over 100 million people are using email to contact people in life.

4. Email replaces older forms of communication.

5. Email has made it easier to contact people (far away) and get instant reply.

6. Email allows people to communicate with more than one person at the same time.

7. People are spending too much time checking and responding their emails

8. Email seems to replace real personal communication, making it hard to get the

true feelings of the interlocutors.

9. Email has been rooted into contemporary life.

10. People need to accept both the good and bad of email.

Summary:

Ever since a group of engineers created the email system in 1971, there’s a

tremendous growth in its use. Today, at least 25 million people are use emails

regularly. The email system has profoundly changed the way we live and work. It

replaces the older forms of communication such as telephone, fax, and regular mail.

The positive side of email is that it enables us to easily contact friends who live far

away and get instant reply; it allows us to talk to many people at the same time; and it

lets us approach anyone without direct contact. However, email also has several down

sides. First of all, checking emails can be very time-consuming. We also get too many

garbage mails that need to be erased. Most of all, email replaces face-to-face

communication and makes it hard to get across our true feelings. No matter what,

email is here to stay and we have to accept both its advantages and disadvantages.



104

Appendix I

Answer Key (Idea Units) to the Posttest Summary

1. Everybody knows smoking is harmful to health, but many people smoke anyway.

2. In the US, many people die of smoking- and secondhand smoking-related

diseases.

3. Regular tobacco use will cause tobacco addiction inevitably.

4. Tobacco use has short term effects such as increased heart rate and blood pressure,

and smoker’s breath/ Tobacco use has long term and deadly effects such as heart

disease, stroke, and cancer.

5. Many college students have come to realize that smoking is bad.

6. Many teenagers start to smoke everyday.

7. Teenagers and preteens should be properly educated through anti-smoking ads and

slogans.

8. Tobacco is found more harmful than other substances. (Lily)

9. People continue smoking because they think it makes them feel relived, and

because they just couldn’t stop. (Judy)

10. It’s worth for Smokers’ to quit smoking as soon as possible. (Judy)

SummaryThis video clip showed a serious problem American society now faces: smoking

addition. We all know that smoking can ruin one’s health, in short term effect, it

caused illnesses like increased heart rate, blood pressure, and smoker’s breath; and in

long term, more deadly diseases such as heart disease, stroke, and cancer are expected.

Yet, many people continue smoking anyway. In the US, many people die of smoking-

or secondhand smoking-related diseases each year. Some people start smoking

because they think they won’t get addicted. However, the poisonous substances like

nicotine found in tobacco would inevitably make people addicted to it if tobacco is

used regularly.

Nowadays, many college students have come to realize that smoking is bad. Still,

smoking is not gone. There is still 25% of American adults continues smoking either

because they think it makes them feel relieved or because they just couldn’t stop.

More seriously, it is reported that 3,000 teenagers start to smoke everyday. To solve

this problem, it is suggested anti-smoking ads and slogans be made so that teenagers

and preteens would not regard smoking cool anymore.



Appendix J

The Task Completion Situation of the Learners

Ss

Number of

finished tasks

(n=66)

Completion

rate (%)Ss

Number of

finished tasks

(n=66)

Completion

rate (%)

S1 66 100.00 S10 57 83.36

S2 43 65.15 S11 39 59.09

S3 33 50.00 S12 33 50.00

S4 33 50.00 S13 60 90.91

S5 47 71.21 S14 39 59.09

S6 33 50.00 S15 37 56.06

S7 34 51.52 S16 66 100.00

S8 36 54.55 S17 42 63.64

S9 33 50.00 S18 53 80.30

Intro Chapter II

Documents

Transcript of Intro Chapter II