Questions and Answers from ETS about the TOEFL iBT

IEP/HE SIG ForumThe TOEFL iBT: Three Years LaterCoTESOL “Welcoming the World”October 23, 2008

The TOEFL iBT

The TOEFL iBT

Reading

The TOEFL iBT

ReadingListening

The TOEFL iBT

ReadingListeningBreak

The TOEFL iBT

ReadingListeningBreakSpeaking

The TOEFL iBT

ReadingListeningBreakSpeakingWriting

The TOEFL iBT

ReadingListeningBreakSpeakingWriting

Total test time:

4 hours

The TOEFL iBT

Reading: 3-5 passages, 20 minutes each

ListeningBreakSpeakingWriting

The TOEFL iBT

Reading: 3-5 passages, 20 minutes each

Listening: 6-9 passages, 60-90 minutes total

BreakSpeakingWriting

The TOEFL iBT

Reading: 3-5 passages, 20 minutes each

Listening: 6-9 passages, 60-90 minutes total

Break: 10 minutes

SpeakingWriting

The TOEFL iBT

Reading: 3-5 passages, 20 minutes each

Listening: 6-9 passages, 60-90 minutes total

Break: 10 minutes

Speaking: 6 questions, 45-60 second responses

Writing

The TOEFL iBT

Reading: 3-5 passages, 20 minutes each

Listening: 6-9 passages, 60-90 minutes total

Break: 10 minutes

Speaking: 6 questions, 45-60 second responses

Writing: 2 questions, 50 minutes total

TOEFL Score Comparisons

PBT310

400-403

500

550

600-603

677

CBT0-40

97

173

213

250

300

iBT0-9

32

61

79-80

100

120

General Questions

Question 1

1. Does ETS recommend minimum TOEFL iBT scores for students entering undergraduate and graduate programs? If not, how do universities choose minimum entry scores for the TOEFL iBT?

Answer 1

No, we don’t. Because the needs of different universities and programs are so varied, ETS does not recommend entry scores, but provides information to institutions in setting scores for their particular circumstances.

Answer 1

Here is a link to the information on the ETS website: http://www.ets.org/portal/site/ets/menuitem.1488512ecfd5b8849a77b13bc3921509/?vgnextoid=39282275562cb110VgnVCM10000022f95190RCRD&vgnextchannel=ee76d0b941abb110VgnVCM10000022f95190RCRD

Answer 1

At the highest level, ETS offers institutions 3 alternative methods of setting scores:

Answer 1

Use the TOEFL iBT standard-setting CD-ROM. E-mail TOEFLNews@ets.org or call +1-609-683-2008 to obtain the CD.

Answer 1

Use the TOEFL Score Comparison Tables to find which TOEFL iBT scores are comparable to your score requirements on TOEFL PBT. We also encourage universities to look at the annual TOEFL test and Score Data Summary which includes student performance on the test by native language, native country, gender and purpose for taking the test as well as percentile data. This can be very helpful when comparing iBT score requirements with previous CBT or PBT requirements.

Answer 1

Score Data Summary: http://www.ets.org/portal/site/ets/menuitem.c988ba0e5dd572bada20bc47c3921509/?vgnextoid=20beaf5e44df4010VgnVCM10000022f95190RCRD&vgnextchannel=d35ed898c84f4010VgnVCM10000022f95190RCRD

Answer 1

OR

Go to  www.ets.org/toefl Select "TOEFL Research" on the left

hand side of the screen

Answer 1

Consider the score requirements set by peer institutions.

Question 2

2. Are universities satisfied with the score setting or do they think it needs some adjustment with consideration given to the different sections?

Answer 2We are encouraging universities to review their score requirements now that students have been admitted with TOEFL iBT scores, and ETS had carried out academic studies, so there is actual test taker performance data in the annual Test and Score Data Summary. This is good score use practice and we support it.

Question 3

3. What type of feedback is ETS receiving from ESL professionals, universities, and students on the TOEFL iBT?

Answer 3

Overall, the feedback is quite positive. In the beginning, we had some problems getting enough seats to meet the demand, but that problem is easing now. We get positive feedback about some of the changes we have made in TOEFL iBT, such as adding the speaking measure, adding integrated items, eliminating discrete grammar items, and allowing note taking.

Answer 3

Here is a link to information on our website, with further links to news stories about TOEFL iBT: http://www.ets.org/portal/site/ets/menuitem.3c27603a8b4042d69a77b13bc3921509/?vgnextoid=811ed898c84f4010VgnVCM10000022f95190RCRD

 Some market research we recently conducted brought us the complaint from a student that you actually had to learn English to pass TOEFL, while with other tests you could just do test prep. We took that as a compliment!

Question 4

4. What percentage of test takers challenge their scores? Does ETS have a percentage for successful challenges?

Answer 4

Sorry, this is proprietary information that we don’t publish.

Answer 4

Here is information on how to apply for a rescore:

http://www.ets.org/portal/site/ets/menuitem.1488512ecfd5b8849a77b13bc3921509/?vgnextoid=1b01fa5947946110VgnVCM10000022f95190RCRD&vgnextchannel=c025a1e13bf36110VgnVCM10000022f95190RCRD

Answer 4

OR

Go to www.ets.org/toefl Select "Test Scores" Select "Internet-based Test" Pick the tab that says "Cancel,

Reinstate, Rescore"

Question 5

5. When are the PBT and CBT going to be phased out in the U.S? Are there plans to phase out these tests abroad?

Answer 5

The CBT test ended in July of 2006 and because scores are valid for 2 years, institutions should not be receiving CBT scores any longer. So CBT for TOEFL is now history.

 95% of all TOEFL testing today is Internet-based.

Answer 5

The PBT test is currently offered in countries without the infrastructure to support the iBT and for the foreseeable future we will need to continue with it. We monitor this situation regularly because our goal is to bring TOEFL iBT to those countries as quickly as possible. Until we can, we’ll continue to offer the paper-based TOEFL, in order to serve students in those locations.

Answer 5

In addition, we used the paper-based TOEFL in a very limited way to support the TOEFL iBT test center network. Now that the TOEFL iBT testing network has matured, 3 years into the process, we don’t use it very much since 95% of all our testing is Internet-based. However, students can still be admitted to a university with a PBT score.

Answer 5

The PBT is still offered in the U.S. 6 times per year in cities such as San Francisco, New York, and Boston. You can go to ETS’ website and search for PBT sites by clicking on each state individually and searching for “paper-based test” on the list of testing centers: http://www.ets.org/bin/getprogram.cgi?test=toefl

Question 6

6. Have you had native English speakers take the TOEFL iBT? If so, does ETS have any data how native speakers perform on the test?

Answer 6

Yes, we have administered TOEFL to native speakers, but the report has not been published yet. There is a published research report on the performance of native speakers on CBT, available in PDF format at: http://www.ets.org/portal/site/ets/menuitem.c988ba0e5dd572bada20bc47c3921509/?vgnextoid=43b1af5e44df4010VgnVCM10000022f95190RCRD&vgnextchannel=d35ed898c84f4010VgnVCM10000022f95190RCRD

Answer 6

Some high level findings in the research on the iBT and native speakers (though not published yet) were: Overall, native speakers performed better on the TOEFL

than non-native speakers, but did not “ace” it as a group; The Reading and Writing Sections showed moderate

differences between native speakers and non-native speakers;

The Listening section showed greater differences between native speakers and non-native speakers;

The Speaking section showed the greatest differences between the native speakers and non-native speakers. The lowest score on the rubric for Speaking was rarely given to non-native speakers.

Answer 6

The results of this study are still being analyzed and the report being written, but research reports are put on the ETS website in PDF format when they are published, so those who want more detail could check there periodically for this report: www.ets.org.

Answer 6

If you are interested in ETS’ ongoing research for TOEFL, here’s a link for a framework of recent research. You’ll see it’s on a variety of topics to ensure test quality. http://www.ets.org/Media/Research/pdf/Framework_Recent_TOEFL_Research.pdf

Question 7

7. How successful does ETS consider the TOEFL iBT? What are some challenges ETS faces with this new format?

Answer 7

We’re very pleased with the TOEFL iBT test. Overall test reliability ranges between .94 and .95 and the new integrated questions are performing as we expected in terms of discriminating those with greater or lesser proficiency. The new integrated questions and the way TOEFL does scoring for the Speaking and Writing sections are what we are probably most proud of.

Answer 7

All test results, including the spoken and written responses, are sent to ETS via the Internet after the test and are then distributed to our online scoring network for scoring. Each written response is scored by two raters and spoken responses are distributed to between 3-6 different scorers. ETS separates the testing and scoring environments to maintain security, validity and reliability. Thus, there is no possibility the scorer may know the test taker, and scorers evaluate worldwide responses.

Answer 7

Some of the challenges we have faced with TOEFL iBT are:

Answer 7

Building a test center network for TOEFL iBT. For CBT we had 617 test centers. Today we have grown to more than 4000. Recruiting, training and supporting that number of secure test centers to ensure we had sufficient test center space required great effort. Three years on, the network is now mature and we’ve managed to move beyond the initial learning curves.

Answer 7

Ensuring we have sufficient raters. On TOEFL CBT, test takers were asked to write one essay and there was no speaking section. Now we need scores for 2 essays and 6 spoken responses for each student.

Answer 7

Reliable electricity supply. As mentioned earlier, we need to have a reliable electrical supply in-country and at least intermittent Internet supply to offer the iBT test.

Answer 7

Helping universities and other score users understand TOEFL iBT scores and use them well. We’ve spent a great deal of effort helping score users understand the new score scale, how to set new score requirements, and to have good practices regarding using TOEFL scores. That includes looking for other evidence of English proficiency in the application. A TOEFL score provides very useful information but it is not the full story. We’ve encouraged universities to think beyond setting a total score requirement, and to consider section score information as well.

Questions on the Writing Section

Question 1

1. What are the most common errors students make in the Writing section?  What can teachers do to help students improve their responses to these tasks?

Answer 1

Most important is that students understand that on the integrated task they are not being asked for their opinions. They should be working with the information given.

For "errors", there is absolutely nothing ETS could zero in on. The independent essay task is little changed from the original TWE and assesses degrees of competence in language, development and organization.

Question 2

2. In practicing for the test, would it be more effective for students to write summaries of what they read and heard, or should they practice writing about how the points of view compare?

Answer 2

They could practice a variety of approaches to see what is easiest/most effective for them. They could summarize reading points one by one

and follow each point with information about how the lecture point relates.

They could summarize the lecture points one by one and follow each point by how it responds to the reading point.

They could briefly summarize the entire reading and then summarize the lecture being sure that their summarization includes how the lecture responds. 

All of these approaches are quite valid.

Extra!

Initially, ETS planned to introduced a variety of accents, but through a pilot test, they found that students performed worse when there was variety of accents compared to a single, standard American accent.

Thank you!

These slides are available on CoTESOL’s new wiki at http://cotesol.pbwiki.com/FrontPage

As the forum progresses, if there are any questions the participants cannot answer, we will send them into ETS for answers. These answers will also be posted on CoTESOL’s wiki.

Example: University of Colorado at Boulder

Minimum TOEFL Scores Required

Undergraduate Majors

Engineering majors 560 (220*) (83**)

All other majors 500 (173*) (61**)

* Equivalent score based on the computer-administered TOEFL.

** Scores on the internet based TOEFL.

Minimum TOEFL Scores Required

Graduate Programs

Aerospace engineering 560 (220*) (83**)

Biological sciences — ecology and evolutionary biology

550 (213*) (79**)

Biological sciences — molecular, cellular, and developmental biology

600 (250*) (100**)

Business (doctoral) 620 (260*) (105**)

Chemical Engineering 600 (250*) (100**)

Chemistry 600 (250*) (100**)

Civil engineering 530 (200*) (71**)

Comparative literature 550 (213*) (79**)

Computer science 560 (220*) (83**)

Dance 550 (213*) (79**)

Economics 525 (197*) (71**)

Education 650 (280*) (114**)

Electrical engineering 600 (250*) (100**)

Journalism 550 (213*) (79**)

Linguistics 550 (213*) (79**)

MBA 600 (250*) (100**)

Mechanical engineering (master's) 550 (213*) (79**)

Mechanical engineering (doctoral) 600 (250*) (100**)

Music (doctoral) 550 (213*) (79**)

Physics 575 (233*) (89**)

Spanish 550 (213*) (79**)

Telecommunications 600 (250*) (100**)

Theater 550 (213*) (79**)

All others 500 (173*) (61**)

*Equivalent score based on the computer-administered TOEFL.** Score on the internet based TOEFL.