Benchmarking in European Service of public Transport (BEST)

Changing the BEST Survey questionnaire?

Changing the BEST questionnaire? 2

Background

The BEST Steering committee discussed changes in the questionnaire in a meeting the 25th of August 2008 – based on an evaluation made by the Helsinki University of Technology

The steering committee decided that a small group should propose an “improved” questionnaire – where the number of statements are reduced

In addition the steering committee discussed:

To add more background questions to make it easier to assess sample quality, like:

Household size

No of persons at least 16 years old

Non-response rates – how it is calculated and the effect of decreasing response rates

How to improve documentation of the BEST Survey regarding

Sampling methods

% of interviews conducted by mobile and fixed line

Definition of non-response

This document presents some alternative ways the BEST Questionnaire could be changed. The other issues will be discussed later.

AlternativesAdvantages and disadvantages per alternative

Changing the BEST questionnaire? 4

Alternative ways the BEST Questionnaire could be changed

Five alternatives are proposed:

Alternative 0: No change in the questionnaire

Alternative 1: Only minor changes in wording

Alternative 2: Reduction in number of statements – but keeping most of the indexes

Alternative 3: Reduction in number of statements – and removal of the “index-level”

Alternative 4: Change the wording of the statements – and reducing the number of statements

Changing the BEST questionnaire? 5

Alternative 0: No change in the questionnaire

Advantages:

We continue to track a quite detailed information about the public transport service in each city

No need to change the use of BEST data in participating companies

Some participants has earlier stated that it is important that the survey stays “as it is” – because they need the information. And they want the benchmark data on all items from all the BEST cities.

Disadvantages:

Does not reduce costs

Does not allow adding CIG specific questions without increasing costs

The index level makes it difficult to spot changes in a quality dimension – because one item in an index might increase and another item might decrease. Makes it more difficult to see what is happening in each city

The index level make it more difficult to discuss changes – because a change in an index can consist of several minor and major changes

When respondents are asked to which extent they agree to a specific statement (as in this survey) – we have to define a level of quality that ideally should be comparable across quality items. That is not always easy

Changing the BEST questionnaire? 6

Alternative 1: Only minor changes in wording

Advantages:

We continue to track a quite detailed information about the public transport service in each city

No need to change the use of BEST data in participating companies

Some participants has earlier stated that it is important that the survey stays “as it is” – because they need the information. And they want the benchmark data on all items from all the BEST cities.

Disadvantages:

Does not reduce costs

If the score on the changed items increases / decreases – we do not know if it is because of the rephrasing or if the actual level of satisfaction has changed

Does not allow adding CIG specific questions without increasing costs

The index level makes it difficult to spot changes in a quality dimension – because one item in an index might increase and another item might decrease. Makes it more difficult to see what is happening in each city

The index level make it more difficult to discuss changes – because a change in an index can consist of several minor and major changes

When respondents are asked to which extent they agree to a specific statement (as in this survey) – we have to define a level of quality that ideally should be comparable across quality items. That is not always easy

Changing the BEST questionnaire? 7

Alternative 2: Reduction in number of statements – but keeping most of the indexes

Advantages:

No changes in wording means we have the history of the remaining quality items

Might allow some extra CIG-specific questions

Disadvantages:

All previous indexes from 2001 to 2008 have to be recalculated (not a very big problem)

Does not reduce costs significantly

Could be difficult to decide which items that should be removed, and which that should be kept

If we want to make a change – it should be more than just a small adjustment.

Changing the BEST questionnaire? 8

Alternative 3: Reduction in number of statements – and removal of the “index-level”

Advantages:

Fewer statements removes the need for an index level. This will make it more interesting and easier to compare results (the index might hide significant changes in the underlying statements)

No changes in wording means we have the history of the remaining quality items

Might reduce costs significantly (has to be verified) – or make it possible add more CIG specific or questions related to a field of special interest (fare evasion, graffiti/tagging, pricing issues, real-time information etc.)

Disadvantages:

Could be difficult to decide which items that should be removed, and which that should be kept

The level of information is reduced

Changing the BEST questionnaire? 9

Alternative 4: Ask the respondents to state how satisfied they are – and reducing the number of statements.

Advantages:

We do not need to define a “quality level” for each item in the survey (i.e. “Public transport mostly runs on schedule”). Instead the respondents are asked to state how satisfied they are with for example “Punctuality / Ability to be on time”)

Fewer statements removes the need for an index level. This will make it more interesting and easier to compare results (the index might hide significant changes in the underlying statements)

Might reduce costs significantly (to be verified?)

Disadvantages:

A change in wording might make it difficult to compare with previous results – but we might try to convert old results into a new index score

Could be difficult to decide which items that should be included

Description of the alternatives

Items that might be included in the different alternatives

Changing the BEST questionnaire? 11

Alternative 0: No change in the questionnaire

Traffic supplyA. Public transport is good for work/school tripsB. Public transport is good for trips e.g.

shopping, leisure etcC. Public transport is good for trips in the (CITY)

city centreD. Public transport is good for trips outside the

(CITY) city centreE. Nearest stop is close to where I liveF. Travel time on public transport is reasonableG. Waiting time is short at transfersH. I am satisfied with the number of departures

ReliabilityA. Public transport mostly runs on schedule

InformationA. It is easy to get the information needed when

planning a tripB. The information is good when traffic problems

occurC. The information is good in stops and

terminals Staff

A. The staff answers my questions correctlyB. The staff behaves nicely and correctly

Personal security and safetyA. I feel secure at stations and bus stopsB. I feel secure on board busses and trainsC. I am not afraid of traffic accidents when using

public transport Comfort

A. Traveling with public transport is comfortableB. Transfers are easyC. The busses and trains are modernD. The busses and trains are cleanE. I normally get a seat when I travel with public

transport Social Image:

A. More people will travel with public transport in the future

B. Public transport is good for the environmentC. Public transport is beneficial to society

Value for moneyA. Public transport gives value for moneyB. Public transport fares are reasonable

LoyaltyA. I gladly recommend travelling by public

transport to others Citizen satisfaction

A. How satisfied are you with public transport in general?

Changing the BEST questionnaire? 12

Alternative 1: Slight changes in wording – the actual changes to be decided

Traffic supplyA. Public transport is good for work/school tripsB. Public transport is good for trips e.g. shopping,

leisure etcC. Public transport is good for trips in the (CITY)

city centreD. Public transport is good for trips outside the

(CITY) city centreE. Nearest stop is close to where I liveF. Travel time on public transport is reasonableG. Waiting time is short at transfersH. I am satisfied with the number of departures

ReliabilityA. Public transport mostly runs on schedule

InformationA. It is easy to get the information needed when

planning a tripB. The information is good when traffic problems

occurC. The information is good in stops and terminals

StaffA. The staff answers my questions correctlyB. The staff behaves nicely and correctly

Personal security and safetyA. I feel secure at stations and bus stopsB. I feel secure on board busses and trainsC. I am not afraid of traffic accidents when using public

transport Comfort

A. Traveling with public transport is comfortableB. Transfers are easyC. The busses and trains are modernD. The busses and trains are cleanE. I normally get a seat when I travel with public transport

Social Image:A. More people will travel with public transport in the

futureB. Public transport is good for the environmentC. Public transport is beneficial to society

Value for moneyA. Public transport gives value for moneyB. Public transport fares are reasonable

LoyaltyA. I gladly recommend travelling by public transport to

others Citizen satisfaction

A. How satisfied are you with public transport in general?

Changing the BEST questionnaire? 13

Alternative 2: Reduction in number of statements – but keeping (most of) the indexes

Traffic supply

A. Travel time on public transport is reasonable

B. Waiting time is short at transfers

C. I am satisfied with the number of departures

Reliability

A. Public transport mostly runs on schedule Information

A. It is easy to get the information needed when planning a trip

B. The information is good when traffic problems occur

C. The information is good in stops and terminals

Staff

A. The staff behaves nicely and correctly*

Personal security and safety

A. I feel secure at stations, bus stops and on board the vehicle

B. I am not afraid of traffic accidents when using public transport

Comfort

A. Traveling with public transport is comfortable

B. Transfers are easy

C. The busses and trains are clean

D. I normally get a seat when I travel with public transport

Value for money

A. Public transport gives value for money

Citizen satisfaction

A. How satisfied are you with public transport in general?

Changing the BEST questionnaire? 14

Alternative 3: Reduction in number of statements – but keeping the same wording – and removal of the “index-level”

1. I am satisfied with the number of departures

2. Public transport mostly runs on schedule

3. It is easy to get the information needed when planning a trip

4. The information is good when traffic problems occur

5. The staff behaves nicely and correctly

6. I feel secure at stations, bus stops and on board the vehicle

7. Traveling with public transport is comfortable

8. Waiting time is short at transfers

9. The busses and trains are clean

10. I normally get a seat when I travel with public transport

11. Public transport gives value for money / The fare level / ticket price is reasonable

------------------------------------------------------

12. How satisfied are you with public transport in general?

Changing the BEST questionnaire? 15

Alternative 4: Ask the respondents to state how satisfied they are – and reducing the number of statements.

1. How satisfied are you with the public transport service in your area regarding…..?

2. The number of departures

3. Punctuality / ability to run on time

4. Information when delays occur

5. The behavior of public transport staff

6. The level of security provided at stations, bus stops and on board the vehicle

7. The comfort when travelling by public transport

8. Waiting time at transfers

9. The cleanliness of busses and trains

10. The possibility to get a seat when travelling by public transport

11. The fare level

----------------------------------

12. How satisfied are you with public transport in general?

Changing the BEST questionnaire? 16

Questions for group wise discussions:

1. Discuss the advantages and disadvantages of each of the 5 alternatives:

Do you agree on the listed advantages / disadvantages?

Should some advantages / disadvantages be removed or added to the list?

2. Rank the alternatives from the most to the least desirable alternative

3. Discuss the list of included statements in the preferred alternative – is there a need for:

Adding or removing statements?

Minor changes of wording?

4. Are one or more of the alternatives totally unacceptable to one or more members in the group – and why?

5. If time allows – are there other changes in the BEST Survey that should be considered?