A.1 – Cognitive testing recruitment instructions Wellcome...

Wellcome Trust Monitor wave 2 – Technical report Appendix A – Cognitive testing and pilot survey documents A.1 – Cognitive testing recruitment instructions

Wellcome Trust Monitor Wave 2 Cognitive Testing Recruitment Instructions (round 1)

BACKGROUND:

Thank you for agreeing to work on the recruitment for this piece of research. We would like you to recruit respondents for the initial phase of a large survey which has been commissioned by Wellcome Trust. For this initial stage of the research we are asking you to recruit a small number of respondents to take part in ‘cognitive interviews’. These are a type of depth interview that serves to test HOW individual questions or entire questionnaires work and respondents’ feedback will serve to improve the questionnaire. For this purpose we will be interviewing respondents face to face in the Borough office. We will then ask them about specific aspects of individual questions as well as the questionnaire as a whole, i.e. their thoughts on particular words used, different thoughts that may have influenced their response to a question etc.

The questionnaire is short and asks about attitudes to science and medical research.

RECRUITMENT:

These cognitive interviews are part of a larger project and are vital for the development of a strong questionnaire. We would therefore ask you to be particularly aware of the MRS and MRQSA guidelines and take care to only recruit people who have not been involved in research in at least the last 6 months. We will be conducting four stages of cognitive interviews, and will be speaking to a mix of adults (18+) and young people (defined as being aged 14-18). Overall, we would like you to recruit a total of:

5 adults with the aim of being able to conduct 4 interviews on the 23rd February 15 respondents (10 adults, 5 young people) with the aim of being able to conduct 12

interviews on the 28th and 29th February 15 respondents (10 young people, 5 adults) with the aim of being able to conduct 12

interviews on the 6th and 7th March 5 young people with the aim of being able to conduct 4 interviews on the 8th March

Adult respondents will receive an incentive of £30, young people will receive £20 of high street vouchers. Young people will also be reimbursed for travel expenses, up to £40. We would like for these interviews to take place in our Ipsos MORI Borough Road offices in London. Interviews should be recruited to be representative of the general public. The interview will take approx. 1 hour. DETAILS

UK Data Archive Study Number 7315 - Wellcome Trust Monitor 2, 2012

Wellcome Trust Monitor wave 2 – Technical report Appendix A – Cognitive testing and pilot survey documents

Cognitive Interviews: First round (Borough Rd)Date: 23rd February Time: Daytime, interview start between 11.00am and 5.30pm.

Recruit 5 adults for 4 interviews. Quotas: Adults:

- all aged 19+, of which at least 1 aged 19-39, at least 1 aged 40-64, at least 1 aged 65+ - mix of social grades (min 2 x A/B/C1, min 2 x C2/D/E) - mix of educational levels (at least 2 highest qual GCSEs or below) - mix of genders - at least 1 BME

Venue Details:

Please collect respondents’ phone numbers, including mobile phone. Ipsos MORI, 79-81 Borough Road, London SE1 1FY (Please tell them to ask for Doug Warren at reception)

Cognitive Interviews: Second round (Borough Rd)Date: 28th – 29th February Time: Daytime for adults, interview start between 9.30am and 5.30pm

Daytime/ early evening for young people, interviews to start between 9.30am and 7.00 pm Recruit 10 adults for 8 interviews and 5 young people for 4 interviews.

Quotas: Adults: - all aged 19+, of which at least 2 aged 19-39, at least 2 aged 40-64, at least 2 aged 65+ - mix of social grades (min 2 x A/B/C1, min 4 x C2/D/E) - mix of educational levels (at least 2 highest qual GCSEs or below) - mix of genders - at least 2 BME Young people: - all aged 14-18, of which at least 2 aged 14-15, at least 2 aged 16+ - mix of educational attainment levels (TBC) - mix of genders - at least 1 BME

Venue Details:


Please also note: Participants must not work in market research, public relations, the media, or have

participated in research in the past 12 months. If you have any queries or problems, please contact Doug Warren at Ipsos MORI (0207 347 3959). Thank you again for working on this study and good luck with your recruitment. Doug Warren Research Manager

Wellcome Trust Monitor wave 2 – Technical report Appendix A – Cognitive testing and pilot survey documents A.2 –Cognitive testing recruitment questionnaire

Wellcome Trust Monitor Wave 2

Cognitive Testing Recruitment Questionnaire (round 3)

Ipsos MORI 11-06157201

RESPONDENT RECRUITED FOR

INTERVIEW NUMBER: RESPONDENT

NO:

Cognitive Interviews: Third round (Borough Rd)Date: 6th – 7th March Time: Daytime for adults, interview start between 9.30am and 5.30pm

Daytime/ early evening for young people, interviews to start between 9.30am and 7.00 pm Recruit 5 adults for 4 interviews and 10 young people for 8 interviews.

Quotas: Adults: - all aged 19+, of which at least 1 aged 19-39, at least 1 aged 40-64, at least 1 aged 65+ - mix of social grades (min 2 x A/B/C1, min 2 x C2/D/E) - mix of educational levels (at least 2 highest qual GCSEs or below) - mix of genders - at least 1 BME Young people: - all aged 14-18, of which at least 6 aged 14-16, at least 3 aged 17-18 - mix of educational attainment levels (TBC) - mix of genders - at least 3 BME

Venue Details:


Cognitive Interviews: Fourth and final round (Borough Rd)Date: 8th March Time: Daytime/ early evening for young people, interviews to start between

9.30am and 7.00 pm Recruit 5 young people for 4 interviews.


Quotas: Young people: - all aged 14-18, of which at least 2 aged 14-16, at least 2 aged 17-18 - mix of educational attainment levels (TBC) - mix of genders - at least 1 BME

Venue Details:


Good morning/afternoon/evening, My name is . . . . . . . from Ipsos MORI, the independent market research company.

[ADULTS ONLY] We are inviting people to fill in a short questionnaire about their views about advances in medicine, and take part in an informal interview about how easy or difficult it was to fill in and why. I wonder if you could help me? This will take place in Borough Road, London SE1 on the 23rd, 28th and 29th of February. The interview will take around 1 hour.

To say thank you for your time and cover any expenses incurred we would like to offer £30 cash. OK?

THE INCENTIVE OFFERED REPRESENTS COMPENSATION FOR THEIR TIME, TRAVEL EXPENSES AND ANY CHILDCARE. [YOUNG PEOPLE ONLY] We are inviting people to fill in a short questionnaire about their attitudes towards science and medicine, and take part in an informal interview about how easy or difficult it was to fill in and why. I wonder if you could help me? This will take place in Borough Road, London SE1 on the 28th and 29th of February. The interview will take around 1 hour.

To say thank you for your time we would like to offer £20 of high street vouchers. We will also reimburse you for any travel expenses incurred up to £40. OK?

We are looking for particular groups of people, therefore I would like to ask you some questions about yourself. All information collected will be anonymised.

Wellcome Trust Monitor wave 2 – Technical report Appendix A – Cognitive testing and pilot survey documents Q1. Would you be interested in taking part?

SINGLE CODE ONLY Yes 1 CONTINUE No 2 THANK AND CLOSE Q2. SHOWCARD A Do you or any members of your immediate family work in any of

the following areas? A Journalism/the media 1

THANK AND CLOSE B Advertising 2 C Public relations (PR) 3 D Market Research 4 No, none of these 5 CONTINUE Don’t know 6 Q3. Have you participated in any kind of research (i.e. an interview or a focus group)

for a market research company in the last 6 months? Yes 1 THANK & CLOSE No 2 CONTINUE Q4. WRITE IN & CODE EXACT AGE

Exact Age Under 14 1 THANK & CLOSE

14-15 2 RECRUIT TO QUOTA – YOUNG PEOPLE

16-18 3 19-39 4

RECRUIT TO QUOTA - ADULTS

40-64 5 65+ 6

Q5. SHOWCARD B Which of the ethnic groups on this card do you consider you

belong to? SINGLE CODE ONLY White

A English/ Welsh/ Scottish/ Northern Irish/ British 1B Irish 2C Gypsy or Irish Traveller 3D Any other White background 4

Mixed/ multiple ethnic groupsE - White and Black Caribbean 5F White and Black African 6G White and Asian 7H Any other mixed/ multiple ethnic background 8

Asian/ Asian BritishI Indian 9J Pakistani 0

K Bangladeshi X


L Chinese YM Any other Asian background ( )

Black/ African/ Caribbean/ Black BritishN –African 1O Caribbean 2P Any other Black background 3

Other ethnic groupQ Arab 4R Any other ethnic group (Please say what) 5( )

ADULTS ONLY Q6. SHOWCARD C From this list, please tell me the highest qualification that you

have obtained. Please just tell me the letter on the lefthand side of the list. SINGLE CODE ONLY

A Post-graduate degree (e.g. Master’s or

Doctorate)1

B Honours degree 2 C A-Levels or equivalent 3 D GCSEs/ O-Levels or equivalent 4 E None of these 5

YOUNG PEOPLE ONLY Recruiter to speak to teachers to ensure range of ability ADULTS ONLY Occupation of Chief Income Earner

Position/rank/grade Industry/type of company Quals/degree/apprenticeship Number of staff responsible for PROBE FULLY CODE FROM ABOVE Class A 1

RECRUIT A MIX OF SOCIAL GRADES

B 2 C1 3 C2 4 D 5 E 6 CODE FOR ALL Q6 GENDER – DO NOT ASK Male 1 RECRUIT A MIX OF

GENDER

Female 2

END

Wellcome Trust Monitor wave 2 – Technical report Appendix A – Cognitive testing and pilot survey documents Interviewer number: ............................................. Interviewer name (CAPS): ................................... I confirm that I have conducted this interview face to face with the named person of the address attached and that I asked all the relevant questions fully and recorded the answers in conformance with the survey specification and within the MRS Code of Conduct and the Data Protection Act 1998. Interviewer Signature: .......................................... Date: ......................................................................

Wellcome Trust Monitor wave 2 – Technical report Appendix A – Cognitive testing and pilot survey documents THIS MUST BE THE LAST PAGE OF THE QUESTIONNAIRE AND MUST BE SINGLE SIDED

CONFIDENTIAL WHEN COMPLETE

Ipsos MORI 11-06157201

Cognitive Interviewing 11-06157201

RESPONDENT RECRUITED FOR

Date: Time: Recruitment Questionnaire RESPON

DENT NO:

PERSONAL IDENTIFIERS

Details

Location: Date: Time:

Name/Initial/Title: Mr/Mrs/Ms/Miss Address:

Full

Tel. Number (WRITE IN INCL. STD code)

Home/mobile 1

Work 2 Refused/Ex-directory 3

E-mail address (WRITE IN) 1

PLEASE FILL IN INTERVIEW DATE AND TIME ON THE NEXT PAGE AND HAND TO RESPONDENT

Is respondent willing to take part and available?

Yes 1 No 2

Wellcome Trust Monitor wave 2 – Technical report Appendix A – Cognitive testing and pilot survey documents Thank you for agreeing to take part in this research. The interview will take around …. hour(s). We have scheduled you in for the following date and time:

Date: ……………………………………………………..

Time: ……………………………………………………..

Venue: Ipsos MORI, 79-81 Borough Road, London SE1 1FY (please see map below). Upon arrival, please ask for Doug Warren at reception. Should you have any questions in the meantime, please do not hesitate to contact Doug Warren on Tel. 020 7347 3000 or e-mail [email protected] To say thank you for your time and cover any expenses incurred we would like to offer £……. cash.

Location map - Ipsos MORI offices

Wellcome Trust Monitor wave 2 – Technical report Appendix A – Cognitive testing and pilot survey documents A.3 Cognitive testing questionnaire

11-061572-01 Wellcome Trust Monitor Wave Two

Cognitive interview survey questions

Existing questions We’re now going to move on to test a few different questions. They cover a range of subjects, so the order may not make much sense. {ASK ALL} Research SHOWCARD S I am now going to ask you about research in a number of different areas. Using this card, please indicate which one area of research you think it is most important to be supported and encouraged, even if a lot of public money would need to be invested? And which area of research do you think is next most important? And which area of research do you think is third most important? CODE UP TO THREE

1 Environmental research 2 Historical research 3 Medical research 4 Into social problems 5 IT and communications research 6 Exploring how the universe works

97 (SPONTANEOUS: None of these) Interest SHOWCARD T AND 'MEDICAL RESEARCH' SHOWCARD GIVE RESPONDENT STANDALONE SHOWCARD. EXPLAIN THAT THIS IS THE DEFINITION OF MEDICAL RESEARCH BEING USED IN THIS STUDY. THEY CAN REFER TO THE CARD THROUGHOUT. READ OUT... Thank you. There are many different definitions of medical research. For the rest of the interview I’d like you to use the following definition ...Medical research is about how the body works, the causes of illnesses and diseases and developing and testing new treatments. How interested, if at all, would you say you are in medical research. Please pick your answer from card Q?

1 Very interested


2 Fairly interested 3 Not very interested 4 Not at all interested

GenMean What do you understand by the term human genome? INTERVIEWER: RECORD VERBATIM GMMean What do you understand by the term genetically modified? INTERVIEWER: RECORD VERBATIM {ASK ALL} MRInfo In the past year, have you tried to find out any information about medical research? This might have been about how the body works, the causes of illnesses and diseases or the testing or development of new treatments. ALTERNATE QUESTION TO TEST IN EVERY OTHER INTERVIEW In the past year, have you tried to find out any biomedical information? This might have been about how the body works, the causes of illnesses and diseases or the testing or development of new treatments.

1 Yes 2 No

{ASK ALL} PInfo Please think of the last time, before this interview, that you heard, saw or read something about medical research that you just happened to come across and had not been trying to find. Can you remember what that was? INTERVIEWER CODE:

1 Respondent remembers something they heard/saw/read

2 Respondent heard/saw/read something but cannot remember details

3 Respondent has not heard/seen/read any information about medical research

INTERVIEWER NOTE: IF RESPONDENT CANNOT REMEMBER WHETHER OR NOT THEY HAVE SEEN ANY INFORMATION ABOUT MEDICAL RESEARCH, CODE AS ‘3’.

Wellcome Trust Monitor wave 2 – Technical report Appendix A – Cognitive testing and pilot survey documents {IF PInfo=RESPONDENT REMEMBERS SOMETHING} PIWhere SHOWCARD U How did you come across this information? CODE ALL THAT APPLY. DO NOT PROBE

1 Another person told me about it 2 A newspaper 3 A magazine 4 A book 5 Television 6 Radio 7 The Internet 8 Attending a lecture or talk 9 A science blog

10 A non-science blog 11 Social media, such as Facebook or Twitter 12 A newsletter from a medical organization

(print or online)

13 At school 96 Other PLEASE SAY WHAT 95 (SPONTANEOUS: Can't remember)

{ASK ALL} AccessI SHOWCARD V Imagine that there was an area of medical research that you wanted to find out more about. How would you prefer to find out or access information on this area of medical research. Please choose your answers from the card? PROBE FULLY CODE ALL THAT APPLY

1 To be told about it in person / talk to someone 2 From a newspaper 3 From a magazine 4 From a book 5 From the television 6 From the radio 7 From websites 8 By email 9 By phoning a helpline or other information

service

10 From a lecture or talk 11 From a discussion with experts 12 From an exhibition or museum 13 From a laboratory open day 14 From a play about medical research issues 15 A science blog

A non-science blog


16 Social media, such as Facebook or Twitter 17 A newsletter from a medical organisation (print

or online)

18 From a computer game 19 At school 96 Other-PLEASE SAY WHAT

{ASK ALL} TrGovt Please tell me how much trust you have in each of the following to provide accurate and reliable information about medical research. Please pick your answer from this card...READ OUT... TrGovt SHOWCARD W First, Government departments and ministers?

1 Complete trust 2 A great deal of trust 3 Some trust 4 Very little trust 5 No trust at all

TrJourn SHOWCARD W (And how much trust do you have in) ...Journalists?

1 Complete trust 2 A great deal of trust 3 Some trust 4 Very little trust 5 No trust at all

{FOR EACH INFORMATION SOURCE FOR WHOM RESPONDENT HAS VERY LITTLE OR NO TRUST (CODES 4 or 5)} NoTrWhy SHOWCARD X Why do you not have much trust in [INFO SOURCE] to provide accurate and reliable information about medical research? CODE ALL THAT APPLY

1 They don't have training or knowledge about medical research

2 They would not be honest about the findings of medical research


3 They would not have access to all available information about medical research

4 They would try to present themselves in the most positive light

5 They would exaggerate information relating to medical research

6 They are generally corrupt, so I couldn't trust them to provide accurate information

96 Other – PLEASE SAY WHAT

Wellcome Trust Monitor wave 2 – Technical report Appendix A – Cognitive testing and pilot survey documents {ASK ALL} NEW Suppose that, with special approval, researchers could access everybody’s medical records as part of specific research studies. However, individuals would be able to opt-out of having their medical records accessed in this way. Do you think researchers should be able to use information from medical records to learn more about the population’s health? 1. Yes 2. No 3. (Don’t know) And, still supposing that researchers could access everybody’s medical records... Do you think researchers should be able to use information from medical records to identify individuals to contact to see if they would like to take part in a clinical trial or research study? 1. Yes 2. No 3. (Don’t know) {ASK ALL} SciQual SHOWCARD Y Can I just check, have you ever studied for a science-related qualification at school, college or anywhere else? IF MORE THAN ONE, CODE FIRST ITEM ON THE LIST THAT APPLIES

1 University 2 College 3 School 4 Have never studied for a qualification in

science

{ASK ALL} SCIJB Have you or any of the other adults in your household ever had a job in a scientific or medical field? INTERVIEWER: PROBE AS APPROPRIATE

1 Yes, respondent 2 Yes, both respondent and someone else in

household

3 Yes, someone else in household 4 No, no adult in household has had scientific job

Wellcome Trust Monitor wave 2 – Technical report Appendix A – Cognitive testing and pilot survey documents {ASK IF CODES 1 OR 2 AT SCIJB} NEW Why did you pursue a job in a scientific or medical field? DO NOT READ OUT, PROBE AS APPROPRIATE

1 Enjoy the subject/ field 2 Was good at science at school 3 Good career prospects 4 Parents’ encouragement 5 Had a good teacher 6 Well paid 7 Parents in science or medical role 8 Other family member in science or medical role 8 Encouraged by famous people in science 9 Encouraged by science programmes on TV

10 Chance to make a difference 11 By chance/ fell into it 12 Other reason (PLEASE SAY WHAT)

YP specific (here on in) - OUT OF SCHOOL ACTIVITIES {ASK ALL} OutSAc SHOWCARD Z Which of the places on this card, if any, have you visited in the past 12 months? CODE ALL THAT APPLY

1 Science museum 2 Art gallery 3 Science and discovery centre 4 Planetarium 5 Zoo 6 Working laboratory or similar scientific site 7 Science festival 8 Nature reserve

96 Other science-related attraction (PLEASE SAY WHAT) 97 (Has not visited any of these attractions)

This block collects information on science related activities the young person is involved in outside of school.

Wellcome Trust Monitor wave 2 – Technical report Appendix A – Cognitive testing and pilot survey documents ASK FOR EACH ANSWER AT OUTSAc NEW And on your last visit to [INSERT ATTRACTION], who, if anyone, did you go with? DO NOT PROMPT

1 Went alone 2 Mother (including step or foster) 3 Father (including step or foster) 4 Brother(s) or sister(s) (including step) 5 Friends 6 With school or college 7 Other relative 8 Other (non-relative) 97 Can’t remember

ASK ALL WHO HAVE VISITED AN ATTRACTION NEW SHOWCARD A2 How often do you go on trips to these sorts of attractions?

1 At least once a week 2 A couple of times a month 3 About once a month 4 Several times a year 5 Once a year

INTERVIEWER READ: “Now thinking about science at schools...” {ASK ALL} NEW SHOWCARD B2 To what extent do you agree or disagree with the following statement: I was able to chose all the science options (courses and exams) I want(ed) to at school/ college

1 Strongly agree 2 Agree 3 Neither agree nor disagree 4 Disagree 5 Strongly disagree

Wellcome Trust Monitor wave 2 – Technical report Appendix A – Cognitive testing and pilot survey documents {IF NO} NEW And what are the reasons for not being able to take the science options you want/ed to? DO NOT READ OUT. CODE ALL THAT APPLY

1 The subjects are not available 2 Timetable clashes 3 Not enough students wanted to take a

particular science course

4 Other subjects take priority 5 Students need to get a certain grade to take

particular science courses

6 The school doesn’t have the right teachers 96 Other reason (PLEASE SAY WHAT) 97 (None of these/Nothing encouraged me)

{ASK ALL} NEW SHOWCARD C2 Have you ever taken part in any of the following activities related to science, technology engineering or maths (STEM subjects) outside of lesson time? CODE ALL THAT APPLY

1 Attended a science or ‘STEM’ club 2 Attended a science or ‘STEM’ fair 3 Entered a science or ‘STEM’ competition or

award

96 Other reason (PLEASE SAY WHAT) 97 (None of these)

Wellcome Trust Monitor wave 2 – Technical report Appendix A – Cognitive testing and pilot survey documents {ASK ALL} SchEnc SHOWCARD D2 This card lists some of the things which might encourage people to learn science at school. Which of them, if any,[HAS] encouraged you personally to learn science? CODE ALL THAT APPLY

1 Finding the subject easy 2 The chance to learn about things relevant to

real life

3 Having friends interested in the subject 4 Having a good teacher 5 Benefits for future study/career 6 The chance to carry out experiments 7 Parents' encouragement 8 Being interested in the subject 9 Getting good marks in coursework or exams

10 It’s a logical subject area 11 It’s a challenging subject area 12 Good image/ cool 13 Famous people in science 14 Science programmes on TV 15 Complements other subjects taken 96 Other reason (PLEASE SAY WHAT) 97 (None of these/Nothing encouraged me)

{ASK ALL} SchDis SHOWCARD E2 This card lists some of the things which might put people off learning science at school. Which of them, if any, [HAS] discouraged you personally from learning science? CODE ALL THAT APPLY

1 Finding the subject too difficult 2 Not taught about things relevant to real life 3 Not a popular subject with friends/classmates 4 Having a bad teacher 5 Won't lead to a well paid career 6 Not being able to do experiments 7 Bad image, not cool 8 Finding subject boring 9 Not enough practical work

10 Having to do experiments 11 Image of scientists 12 Don’t think I’m the sort of person who studies

science

13 Parents did not want me to study science 14 It’s not creative enough 15 Already chosen a career that doesn’t need a


science qualification 96 Other reason (PLEASE SAY WHAT) 97 (None of these/Nothing discouraged me)

{ASK ALL} NEW SHOWCARD F2 I'm now going to read out a few things people have said about studying science at school. For each one please tell me how much you agree or disagree. READ OUT...

a) Science lessons teach people skills that are useful for non-scientific jobs

b) Jobs in science are very interesting

c) The science I learnt in school has not been useful in my everyday life


{ASK ALL} CarSci6 SHOWCARD F2 AGAIN (And how much do you agree or disagree that...) ...Scientists come from a wide range of social and cultural backgrounds?


{ASK ALL} CarInt SHOWCARD G2 Thinking about you personally, how interested are you in a future career in science?

1 Very interested 2 Fairly interested 3 Not very interested 4 Not at all interested

Wellcome Trust Monitor wave 2 – Technical report Appendix A – Cognitive testing and pilot survey documents New questions

Public Attitudes to Science Governance

Medical science has the potential to improve human health and well-being. However, difficult decisions have to be made about which illnesses should be targeted and which sorts of treatments should be developed. {ASK ALL} Q1. SHOWCARD A To what extent do you think each of the following groups should or should not be involved in making these decisions? Government Scientists working in universities Scientists working for the government Scientists working for pharmaceutical companies Medical charities Members of the general public Always Sometimes Never TEXT FILL BASED ON RESPONSE TO Q1. IF ALWAYS/ SOMETIMES USE “SHOULD” IF “NEVER” USE “SHOULD NOT” Q1A. For what reasons do you think the general public [should / should not] be involved in decision making? OPEN ENDED {ASK IF SAY GENERAL PUBLIC SHOULD ALWAYS/ SOMETIMES BE INVOLVED AT Q1 ASK ALL} Q2 SHOWCARD B Which of these statements, if any, comes closest to your own attitude to public consultation on medical research? Just read out the letter that applies. A. I would like to know that the public are consulted on decisions about medical

research, but I don’t want to be involved personally B. I would like to have more of a say in decisions about medical research C. I would like to become actively involved in public consultations on decisions about

medical research

Wellcome Trust Monitor wave 2 – Technical report Appendix A – Cognitive testing and pilot survey documents D. I am already actively involved in public consultations on decisions about medical

research (SP; allow DK) {IF CODE A AT 2} Q2a Why would you not be interested in contributing to decision making? {ASK ALL} Q3. SHOWCARD C To what extent do you agree or disagree with the following statement: People involved in making decisions about the future direction of medical research need to have a good understanding of the science involved.

Strongly agree Agree Neither agree nor disagree Disagree Strongly disagree (Don’t know) Vaccinations I now want to ask a few questions about illnesses and vaccinations. {ASK ALL} Q1. SHOWCARD D Thinking now about [NAME OF ILLNESS], as far as you know, for a healthy adult, is this a very serious illness, a fairly serious illness, not a very serious illness, or a not at all serious illness? Flu Meningitis Chickenpox Swine flu Measles Very serious illness Fairly serious illness Not a very serious illness Not at all serious illness (Have not heard of this illness) (Don’t know) INTERVIEWER READ OUT: “A vaccination is where you are given something by a medical practitioner or pharmacist (typically by injection or taken by mouth) that protects you against one or more illnesses.”ASK OF ALL ILLNESSES

Wellcome Trust Monitor wave 2 – Technical report Appendix A – Cognitive testing and pilot survey documents {ASK ALL} Q2 SHOWCARD E As far as you know, how much personal risk, if any, is there in having a vaccination for [ILLNESS]? Flu Meningitis Chickenpox Swine flu Measles A very high risk A fairly high risk A fairly low risk No risk at all (Don’t know) {ASK ALL} Q3 SHOWCARD F How serious a problem, if at all, do you think it is for the wider population if lots of people choose not to be vaccinated? Very serious Fairly serious Not very serious Not at all serious (Don’t know) {If SERIOUS AT Q3} Q3a. Why do you say this? {ASK ALL} Q4. SHOWCARD G Overall, from what you know or have heard, which of these statements, if any, most closely reflects your own opinion about having a vaccination for [ILLNESS]? The benefits far outweigh the risks The benefits slightly outweigh the risks The risks and benefits are about the same The risks slightly outweigh the benefits The risks far outweigh the benefits (Don’t know)

Wellcome Trust Monitor wave 2 – Technical report Appendix A – Cognitive testing and pilot survey documents {IF RISKS OUTWEIGH THE BENEFITS AT Q4} Q5 Why do you think the risks far outweigh the benefits? OPEN {ASK ALL} Q6. SHOWCARD H Thinking about vaccinations, to what extent do you agree or disagree with the following statements? a) People need to take responsibility for their own health, and not rely on the

government or the medical profession to tell them what to do. b) It is the responsibility of the government to ensure the public has information and

clear recommendations about how to keep healthy. c) Children have a right to be vaccinated against serious disease which over-rides their

parents' preference. d) It is acceptable to deny healthcare to adults who have refused vaccinations. e) People should be able to decide whether or not they wish to have vaccinations even if

they may put other people at risk by not being vaccinated. f) If health professionals recommend a vaccination, then the benefits of vaccination will

outweigh the risk of any side-effects g) I have a personal responsibility to get vaccinated to help stop the spread of disease. h) Sometimes health professionals recommend vaccinations which have a risk of serious

side-effects Strongly agree Agree Neither agree nor disagree Disagree Strongly disagree (Don’t know)

Wellcome Trust Monitor wave 2 – Technical report Appendix A – Cognitive testing and pilot survey documents Connecting Environment, Nutrition and Behaviour Change I’m now going to ask you some questions about nutrition. {ASK ALL} Q1. SHOWCARD I For each of the following items I read out, please tell me how much you believe this is a reason why people become overweight. a) Genetic factors b) The kinds of foods marketed at restaurants and shops and/ or advertised in the media c) Healthy food being more expensive than unhealthy food d) Lack of understanding or education about diet and nutrition e) Not getting enough physical exercise

A great deal A fair amount Not very much Not at all Don’t know (Don't know) {ASK ALL} Q2. SHOWCARD J Thinking about obesity, to what extent do you agree or disagree with the following statements? a) People need to take responsibility for their own health, not rely on the government or

the medical profession to tell them what to do. b) It is the responsibility of the government to ensure the public have information and

clear recommendations about how to keep healthy. c) It is acceptable to deny some healthcare treatments to an adult because he or she is

overweight. Strongly agree Agree Neither agree nor disagree Disagree Strongly disagree (Don’t know)

Wellcome Trust Monitor wave 2 – Technical report Appendix A – Cognitive testing and pilot survey documents Cognitive Enhancers Moving on, I’d now like to ask some questions about people’s focus, memory and concentration. {ASK ALL} Q1. SHOWCARD K To what extent to you think the each of the following is an effective way to improve someone's focus, memory or concentration? CODE ALL THAT APPLY 1 Doing challenging puzzles like crosswords or Sudoku, or playing certain "brain-

training" games 2 Taking vitamins or dietary supplements like fish oils 3 Taking medications that are normally used to treat medical conditions like ADHD

or dementia 4 Drinking coffee, Red Bull, or other drinks with lots of caffeine in, or taking caffeine

pills like ProPlus 5 Taking medications normally taken to relieve stress and anxiety, like betablockers. 6 Doing exercise Very effective Fairly effective Not very effective Not at all effective (Don’t know) INTERVIEWER NOTE – IF RESPONDENT IS UNFAMILIAR WITH TERMS ADHD OR DEMENTIA, PROVIDE EXPLANATIONS AS BELOW. ADHD: ‘By ADHD we mean Attention deficit/ hyperactivity disorder’ DEMENTIA: ‘Dementia is also known as Alzheimer’s disease.’ READ OUT: I am now going to hand you a paper questionnaire, and would like you to complete the next few questions on it. You will not need to show me your answers. {ASK ALL} Q2 SHOWCARD L To improve your focus, memory or concentration, have you ever tried any of the following? 1 Doing challenging puzzles like crosswords, or Sudoku or playing certain "brain-

training" games 2 Taking vitamins or dietary supplements like fish oils 3 Taking medications that are normally used to treat medical conditions like ADHD

or dementia 4 Drinking coffee, Red Bull, or other drinks with lots of caffeine in, or taking caffeine

pills like ProPlus 5 Taking medications normally taken to relieve stress and anxiety, like betablockers.

Wellcome Trust Monitor wave 2 – Technical report Appendix A – Cognitive testing and pilot survey documents None of these (Don’t know) {ASK THOSE GIVING YES TO ANSWER 3 on Q2} Q4. SHOWCARD M The medications listed on this card can all improve a person’s focus, memory or concentration for a while. They can be used to treat diseases or disorders. As far as you know, have you ever taken one of the medications listed on this card? 1. Ritalin 2. Donepezil 3. Provigil

Yes No (Prefer not to say) (Don’t know) {IF YES TO ANY AT Q4} Q5 Thinking about when you last took a medication to improve your focus, memory or concentration- Did you have a prescription for it? Yes No Don’t know {ASK IF NO OR DON’T NO AT Q5} Q6 Did you order the medication from the internet? Yes No Don’t know {ASK IF NO OR DON’T NO AT Q6} Q7. Were you given the medication by someone else who you think had a prescription for it? Yes No Don’t know

Wellcome Trust Monitor wave 2 – Technical report Appendix A – Cognitive testing and pilot survey documents {ASK ALL} Q6. SHOWCARD N Please could you tell me the extent to which you agree or disagree with the following statements. ROTATE ORDER OF READ OUT. 1 It is acceptable for anybody to use cognitive enhancing medications to try to improve

their focus, memory or concentration on a regular basis 2 It is acceptable for anybody to try to improve their exam performance through exercise

or playing brain training programmes. 3 It is acceptable for anybody to try to improve their exam performance by using cognitive

enhancing medications. 4 It is acceptable for anybody to try to improve their exam performance by using vitamins,

caffeine or dietary supplements. NOTE – IN HALF THE INTERVIEWS, CLARIFY THAT COGNITIVE ENHANCING MEDICATIONS WOULD BE TAKEN WITHOUT A PRESCRIPTION. Strongly agree Agree Neither agree nor disagree Disagree Strongly disagree (Don’t know) Health-related findings in research NB – CONTINUE TO TEST THIS SECTION, BUT MONITOR UNDERSTANDING – POTENTIAL TO DROP Imagine for a moment that you were taking part in a medical research project that looks at people’s genes. Suppose that the researcher finds an unexpected genetic finding that may affect your health. I am going to read out a number of things that could happen, and, using this card, I want you to tell me what you would want the researcher to do. Suppose the researcher finds out that, because it’s in your genes, there is a chance that you could.... {ASK ALL} SHOWCARD O Q1. develop a medical condition that is not life threatening Q2. develop a medical condition that is life threatening but curable Q3a. develop a medical condition that is life threatening but NOT curable I should definitely be told I should definitely not be told The researcher should decide whether or not to tell me (I wouldn’t take part in a medical research project) (Don’t know)

Wellcome Trust Monitor wave 2 – Technical report Appendix A – Cognitive testing and pilot survey documents {ASK ALL} Q4. SHOWCARD P In some large medical research studies, it may be very difficult to tell all participants unexpected findings about their health due to cost and other issues. To what extent do you agree or disagree with the following. 1 Research studies should only be conducted if participants who wish to know will be told

about any unexpected findings about their health. 2 Research studies can be conducted even there is no guarantee that participants who wish

to know will be told about any unexpected findings about their health. Strongly agree Agree Neither agree nor disagree Disagree Strongly disagree (Don’t know) {ASK ALL} ProgStar SHOWCARD J2 How interested, if at all, are you in watching television programmes about the universe?


ProgDram SHOWCARD J2 AGAIN And how interested, if at all, are you in watching programmes on television about medical dramas set in a hospital or doctor's surgery?


Wellcome Trust Monitor wave 2 – Technical report Appendix A – Cognitive testing and pilot survey documents Careers questions (Young people only) {ASK ALL} Q1. SHOWCARD Q How much, if anything, do you feel you know about careers in a scientific or medical field? A great deal A fair amount Not very much Nothing at all Don’t know {ASK ALL} Q2. SHOWCARD Q And how much, if anything, do you feel you know about your general career opportunities? A great deal A fair amount Not very much Nothing at all Don’t know {ASK ALL} SHOWCARD R Q3. And have you ever received any information, advice or guidance about what you may do for a living from any of the sources on this card? Q4. And which of the following have been most useful in helping you to think about what you may want to do for a living? MULTICODE FOR BOTH Friends Family Someone already working in the field A careers advisor at your school or college A teacher at your school or college Class or school presentations by school staff Class or school presentations by employers Careers library Jobcentre Plus Directgov website Other website or Internet Careers Advisory Service website Public library or learning resource centre

Wellcome Trust Monitor wave 2 – Technical report Appendix A – Cognitive testing and pilot survey documents Community, voluntary or religious organisation Seeing what people do in general TV or Radio programmes or films Other person or organisation None of these {ASK ALL} Q5. And have you ever done any work experience? Yes No (Don’t know) {IF YES} Q6. Was this in a science or health related field? Yes No (Don’t know)

Wellcome Trust Monitor wave 2 – Technical report Appendix A – Cognitive testing and pilot survey documents A.4 – Cognitive testing self completion questionnaire

SCIENCE AND HEALTH SELF-COMPLETION QUESTIONS

Q2 To improve your focus, memory or concentration, have you ever tried any of the

following?

1 Doing challenging puzzles like crosswords or Sudoku, or playing certain

"brain-training" programs

2 Taking vitamins or dietary supplements like fish oils

3 Taking medications that are normally used to treat medical conditions like

ADHD or dementia

5 Drinking coffee, Red Bull, or other drinks with lots of caffeine in, or taking

caffeine pills like ProPlus

6 Taking medications normally taken to relieve stress and anxiety, like

betablockers.

7 None of these

IF YES TO ANSWER 3 on Q2

Q4. The medications listed below can all improve a person’s focus, memory or concentration

for a while. They can be used to treat diseases or disorders.

As far as you know, have you ever taken one of the medications below?

4. Ritalin

5. Donepezil

6. Provigil

7. Other

Yes

No


IF YES TO ANY AT Q4

Q5. Thinking about when you last took a medication to improve your focus, memory or

concentration -

Did you have a prescription for it?

Yes

No

Don’t know

IF NO OR DON’T KNOW AT Q5

Q6. Did you order the medication from the internet?

Yes

No

Don’t know

IF NO OR DON’T KNOW AT Q6

Q7. Were you given the medication by someone else who you think had a prescription for

it?

Yes

No

Don’t know


ASK ALL

Q8. Please could you tell me the extent to which you agree or disagree with the following

statements.

Strongly

agree

Agree Neither agree nor

disagree

Disagree Strongly disagree

Don’t know

It is acceptable for anybody to

use cognitive enhancing

medications to try to improve

their focus, memory or

concentration on a regular basis


try to improve their exam

performance through exercise or

playing brain training

programmes.


try to improve their exam

performance by using cognitive

enhancing medications.

It is acceptable for anybody to try to improve their exam performance by using vitamins, caffeine or dietary supplements

Wellcome Trust Monitor wave 2 – Technical report Appendix A – Cognitive testing and pilot survey documents A.5 – Cognitive testing feedback on materials Wellcome Trust Monitor Materials cognitive testing - feedback Show the leaflet

What are their first reactions? Some respondents felt that they might not read the leaflet thoroughly, as it contains quite a lot of information. However, after seeing the letter, most felt that this would encourage them to read the leaflet in greater depth.

How does it make them feel? Feedback was generally positive. The leaflet is seen to be quite science-y. Most respondents felt that it was eye-catching, although some said that they feel it is a little too dark: The colour scheme could perhaps be made more vivid. One respondent felt that the wording could be a little more personal.

If they received it at home, would they read it straight away, put it down, immediately bin it?

Most respondents said that they would at least have a look at it. What do they think of the images? Do they like them? What do they think they are? What do they think they represent?

The images received mixed reviews (partly due to the print quality meaning that they did not stand out). While they were generally seen as relevant to science, and most respondents recognised at least one image, they are all very abstract. This makes it hard to engage with them. In particular, the blood/ nerve image on the inside/ middle was seen as unclear.

What about the colours? It was generally felt that the colour scheme fits in with the subject matter, but needs to be brighter. In particular, a couple of respondents found the mauve text hard to read on the black background. Some respondents felt that the dark scheme was a little sombre and dull, and should be made a little lighter.

Get them to read the leaflet in full. Is there anything they don’t understand? Would it make them interested in taking part?

Are there are terms that make them not want to take part? The leaflet was seen as easy to understand and straightforward. No one mentioned anything that they felt would make them not want to take part. However, some felt that the survey may not be relevant to them – need to emphasize that no knowledge of science is needed.

Is the amount of information given enough/ too much/ too little? One younger respondent felt there was perhaps a little too much information, and that that would put him off reading it. One respondent felt that they would like more information on what the survey would cover, and said that the leaflet should emphasize the importance of the survey more. A couple of respondents felt that the leaflet to some extent covers information already provided by the letter, and could therefore be made shorter.

Had they heard of Ipsos MORI before this interview? Some had, although few knew in what context.

Had they heard of the Wellcome Trust before this interview Again, some awareness of the Wellcome Tust, although little familiarity with their work.

Any suggestions for making the leaflet better? Could make the incentive more prominent, and therefore increase chance of people reading it who only skim through.

Wellcome Trust Monitor wave 2 – Technical report Appendix A – Cognitive testing and pilot survey documents Lighten the colour scheme, and update the data section to reflect the audio recording. Show the letter

What are their first reactions? It was generally felt that the letter was clear and to the point, and that the leaflet would primarily be a back up to provide further information should people have queries.

How does it make them feel? Very clearly laid out, and easy to read. However, it perhaps doesn’t stand out from any other letters that people receive (is this necessarily a bad thing?)

If they received it at home, would they read it straight away, put it down, immediately bin it?

Most people would at least glance over the letter, and felt that they would be more likely to read the leaflet in depth after seeing the letter.

Is there anything they don’t understand? Would it make them interested in taking part/ would they take part?

Are there are terms that make them not want to take part? There isn’t anything that people don’t understand. Perhaps not too much that would genuinely entice them to participate, however, other than the incentive. Some felt that the summary of the subject matter wouldn’t be that appealing to them, because it’s all about advances in research (and she’s ‘too old for all that’). Perhaps we need to focus more on some of the health issues we’ll be talking about?

Is the amount of information given enough/ too much/ too little? There’s enough information, with the leaflet acting to answer any further questions they may have.

How do they feel about receiving £10 in cash to take part? Would they take part for £10? If not, how much would they have to be paid?

£10 is generally seen as a fair sum of money. However, most people expected that the interview would take around half an hour. When pressed on whether the incentive would be adequate for a 40 minute interview, most people said they would still participate, but that they would be hesitant if the interview were any longer. One respondent was a little unclear as to how the incentives would work – who would get what, and would the child get a voucher in addition to the cash.

Any suggestions for making the letter better? The letter should mention that the contact number is FREEPHONE. Add text to show that the contact number is FREEPHONE. Amend the details explaining the survey to have greater focus on health issues. Principles of audio recording Explain as part of the real face-to-face interview, the interviewer will ask them for permission to be recorded for 4 of the questions. The Wellcome Trust is interested in hearing respondents’ voices and would like to store them in an archive so that other interested researchers would also be able to listen to the answers. No one outside of Ipsos MORI would be able to identify any individual though.

What are their first reactions to this? Not too many issues. Generally, people’s first instinct is to be positive about it. The text is clear, and fully explains the issue; no one experienced any difficulties in understanding the concept or any of the terms.


Would they agree to be recorded? Most respondents felt that they would agree to be recorded, although some would refuse if the questions are to be personal.

What concerns would they have about being recorded? None expressed concerns with being recorded per se, aside from a minor concern that they might say something silly.

What concerns would they have about having their answers stored in the Wellcome Trust archive?

People’s main concern is that they might say something silly, and wouldn’t want that permanently stored online. A couple of respondents had a natural aversion to a ‘record’ of theirs being stored online, but couldn’t define what their issue was exactly. In these cases, they were reassured by the fact that there’s would be one amongst c. 1,500 responses.

Would their concerns depend on the types of question that would be recorded? Read out the wording of the 4 questions – does this change their views?

Yes – many people expressed concern that questions may be too sensitive or personal, and therefore would want reassurance as to what type of questions would be asked. When probed specifically on stress-testing, no respondents felt concerned about this in any way. One respondent had strong concerns with the additional explanatory text regarding the licensing of the audio content – use of the word ‘remix’ made him think that his words could be edited/ cut such that they would be taken out of context. Likewise, other respondents felt that they would be happy with their responses being publicly accessible, but would not want them to be used in any other way (e.g. downloadable by others). Need to make more of the fact that all respondents’ responses will be stored online, so that people feel security in numbers.

Wellcome Trust Monitor wave 2 – Technical report Appendix A – Cognitive testing and pilot survey documents A.6 – Pilot survey interviewer instructions


Pilot interviewer instructions - FINAL April 2012

Thank you for agreeing to take part in the pilot stage of this study. We hope you’ll find it as interesting as we do! The key points to note for this project are:

The survey is designed for two age groups – adults (18+) and young people (14-18). The young person’s survey is quite different to the adult one although there are some overlaps, and 18 year olds can be classified as either age group, so make sure you are certain of which type of interview you intend to carry out.

The survey with adults is referred to as ‘Health and Medicine: Now and the Future’. The survey with young people is referred to as ‘Science and Medicine: Now and the

Future’. You will be audio recording answers to a few questions – please ensure that you get

permission to record respondents. We will be trialling a number of different recording devices during the pilot. You will each have a different device, so your feedback is vital.

For the pilot we’re using a quota methodology, but for the mainstage we’re using a random probability sampling method.

1. Background:

Ipsos MORI has been commissioned by the Wellcome Trust to carry out the second wave of its ‘Wellcome Trust Monitor’. The first wave was carried out by NatCen (another social research company) in 2009. The Wellcome Trust is a global charitable foundation dedicated to achieving extraordinary improvements in health by supporting the brightest minds. The Wellcome Trust Monitor is designed to measure the public’s interest and engagement in science, specifically biomedical research. It is intended to help the Wellcome Trust’s work in promoting biomedical science. In particular it is interested in gathering information on the attitudes of the future generation towards biomedical science and research – hence the young people survey, which focuses strongly on careers and attitudes towards science in school. Mainstage fieldwork will take place between May and July. A total of 1,300 adults will be interviewed across the United Kingdom (England, Scotland, Wales and Northern Ireland), plus an additional 400 interviews with 14-18 year olds.


2. Fieldwork: Overview: You will be carrying out the pilot stage of this survey. We will be piloting the survey on 30 adults and 20 young people. Fieldwork will take place between 3rd and 15th April. The survey will be carried out on CAPI in respondents’ home. We estimate that the survey will take 45 minutes for adults and 35 minutes for young people.

For the pilot we are not using random selection methods (i.e. Kish) unlike in the mainstage, so you can interview anyone who fits your quotas. However, we only want to interview a maximum of one adult and one young person per household. The adult and young person interviews are set up as parallel blocks so that you can select which interview to complete first. However, wherever possible please complete the adult interview first. The pilot is important because we need to get a better idea of the overall length of the survey, reactions to the questions, and the public response to our audio recording request. Additionally we will be testing the warm-up materials for the survey. During fieldwork, someone from the Ipsos MORI research team may choose to accompany interviewers in the field to see the interviews taking place first hand. For those working in London it is possible that the client will also want to observe some interviews. You will be informed in advance if anyone will be accompanying you.

An interviewer feedback form will be provided for you to record your experiences and comments on each interview you conduct. We would like you to discuss your comments at a telephone debriefing meeting with the client on Monday 16th April where we will discuss the questionnaire in detail, in preparation for the mainstage of this survey.

Sample:

The pilot is being conducted in four areas of the UK, which means that roughly 12 interviews will be carried out in each location. This will be confirmed in your packs. As with quota studies, you will be given an address listing and a quota sheet so you know who and where to interview.

Interviewing young people

Your first contact at an address should always be with a responsible adult. You must get verbal consent from an adult before speaking to anyone under 18. There is a separate letter for young people, so please hand it to the young person or ask an adult to pass it on to them.

The start of the young person interview collects background information on their parents’ education and occupation. Information is collected on all parents’ resident in the household. If possible, we would like to collect this information from a parent or guardian, in order to try and maximise the accuracy of the information collected. However, you should ensure that the parent/guardian then leaves the room for the remainder of the young person interview. At the start of the interview you will be prompted to ask the young person’s permission to collect some information from a parent or guardian. If the young person does not want you to speak to a parent, or if it would not be convenient for a parent/guardian to answer questions at

Wellcome Trust Monitor wave 2 – Technical report Appendix A – Cognitive testing and pilot survey documents that time, you can ask the young person to provide this information instead. The CAPI program asks you to select whether a parent/guardian or the young person themselves will be answering these background questions and then routes you to appropriately worded questions. It is very important that the adult respondent is not present in the room when the young person is being interviewed, and vice versa. This is because many of the questions are the same in both interviews and we do not want respondents to be made aware of them in advance. It is particularly important that no parent/guardian is present during the young person’s interview as this may influence the extent to which young people are willing to be honest about certain topics (their attitudes to school for example). Advance letters

For the mainstage we will sample specific addresses for you to visit and therefore send advance letters to ahead of your visit. For the pilot we have provided advance letters and leaflets to either show on the doorstep or post through the door, given the short fieldwork period.

Please note that there are two letters and two leaflets – one of each for adults and one of each for young people. Please use whichever letter and leaflet is appropriate for the situation.

Incentives:

Respondents will receive a £10 incentive for the completion of this survey. Adult respondents will receive the £10 in cash, and young people will receive £10 in high street vouchers.

As we are carrying out 50 pilot interviews it is possible that you may be carrying a fairly large amount of cash and/or vouchers – please be careful when out in field.

Please also make sure that all respondents sign your incentive receipt form so we can keep track of where the money is going. Please use a separate form for each respondent, for the purposes of confidentiality. Adults and Young People have different versions of the consent form to reflect the incentive given to them.

Audio recording:

The Wellcome Trust would like you to audio record the respondents’ answers for three questions in the survey – DNAMean, GMMean, GenMean (on page 37 of the questionnaire). They want to be able to hear the respondents’ answers, rather than just reading the verbatims or looking at the coded data.

Because of this, we have a specific consent question so that respondents are clear what the purpose of the recording is. You will need to complete a consent form prior to conducting each interview. This outlines what we are asking of respondents.

If the respondent doesn’t consent to being recorded, we would like you to type in their verbatim answers as usual. CAPI will guide you with this.

Instructions for using the microphone

The CAPI will instruct you when to turn on your microphone and when to turn it off.

The microphone plugs into the following socket on your Panasonic laptop:


On the Dell laptop the microphone connection is on the right side of the machine toward the back. Once you have the microphone in and the laptop switched on then everything should be ready to use. If you run through a practice interview and check that it has recorded your answers. If it has not recorded your answers it may be an issue with the volume of the microphone or the speaker volume is too low. Please call the CAPI Helpline and we will help resolve the problem. Important: When you are placing the microphone in the bag, please make sure the cable is disconnected from the laptop. Also make sure the arm of the microphone is packed properly and not jammed tightly against other items in the bag.

CASI

In the Cognitive Enhancer section, there are a number of sensitive questions, so we have included a self-completion section for respondents to enter their answers directly into your CAPI machine. If the respondent is not happy to use your CAPI machine, you can read the questions out as usual, but please code this accordingly in the script.

Showcards:

Please take time to review the showcards before you start interviewing. They are organised by section and therefore have a slightly different flow to normal. For the adult interviews you will need to skip a number of sections, so please familiarise yourself with them before you start.


3. Your pack

In addition to these interviewer instructions, your pack contains the following materials:

Address listings Quota sheet Consent forms (enough for each respondent interviewed) Audio recording consent forms Questionnaire showcards (normal or reverse) Medical research showcard (for use throughout the interview) Adult and young person Wellcome Trust Monitor leaflet Adult and young person advance letters Laminated copy of the advance letters Paper version of the questionnaire Incentives Microphone Standard field admin materials

Please ensure you have all these materials before starting work.


4. The questionnaire The interviews themselves are relatively straightforward and should be easy to follow. However, before starting work, it is very important that you are familiar with the questionnaire. Please make sure you have run through the questionnaire and are comfortable with how it works. Many questions are asked of both adults and young people but CAPI will guide you through it. For your information a paper copy has been provided.

The rough layout of each interview is:

Adult:

Household grid Media Introduction to medical research Knowledge of medical research Engagement with medical research Information sources Involvement in medical research Expectations and concerns Scientific literacy Public attitudes to science governance Vaccinations Connecting environment, nutrition and behaviour change Cognitive enhancers Genetics Demographics

Young person:

Parents' demographics Media Out of school activities Current and future activities Experience of school science Peer pressure Science as a career Introduction to medical research Knowledge of medical research Engagement with medical research Information sources Expectations and concerns Scientific literacy Vaccinations Connecting environment, nutrition and behaviour change Cognitive enhancers Genetics Final demographics

Wellcome Trust Monitor wave 2 – Technical report Appendix A – Cognitive testing and pilot survey documents The topics covered in the adult interview are: Media This section collects information about how respondents get their information and news. Introduction to medical research This section introduces the concept of medical research and gauges people’s general awareness and knowledge of and interest in medical research. Knowledge of medical research This section asks respondents to rate their understanding of some scientific terms that are used in news stories related to medical research. Engagement with medical research This section explores the extent to which people engage with medical research in their daily lives and in what ways. It asks about interest in particular areas of medical research, whether respondents have sought or received any information about medical research and if so, from what sources and for what reason. There are some questions about people’s engagement in cultural activities with a medical or scientific focus (watching TV programmes, reading books etc.). Information sources This section explores how people would like to receive information about medical research, which sources they would trust to provide accurate information and the reasons for this. Involvement in medical research This section asks respondents whether they have ever taken part in a medical research project, and their views on taking part. Expectations and concerns This section asks about respondents’ views on whether medical research will improve the quality of life and the media coverage of the impact of medical research. Scientific literacy This section explores people’s understanding of what it means to study something scientifically and contains a quiz to measure respondents’ scientific knowledge. There are three questions in this section that we would like you to audio record the respondents’ verbatim comments. Public attitudes to science governance This section asks respondents about how involved they think certain groups should be in making decisions about medical research. It also asks whether they would like to be involved. Vaccinations This section asks about general views of vaccinations. Connecting Environment, Nutrition and Behaviour Change This section asks about general views of the causes of obesity. Cognitive Enhancers This section measures attitudes towards improving memory, focus and concentration. It includes a self-completion CASI section.

Wellcome Trust Monitor wave 2 – Technical report Appendix A – Cognitive testing and pilot survey documents Genetics This section explores attitudes towards genetics including the issue of genetic testing. In addition to some of these topics, additional topics are included in the young person interview. These are: Out of school activities This section asks young people about what sorts of science-related activities they do and how often. Current/future study plans This section collects background information about the young person’s education so far, as well as asking about any future plans to study science. Experience of learning science at school This section asks young people how interested they are/were in science at school and about the factors which either encouraged or discouraged them from learning science. Peer pressure This section is designed to gauge how interested the young person’s friends and family are in science. This may in turn influence young people’s own attitudes. Science as a career This section asks young people for their views on the advantages and disadvantages of a scientific career and whether they would be interested in pursuing a career in science.

Please also note the following general points:

It is important that the respondents give considered responses that are as accurate as possible. We need to make sure that respondents keep up concentration levels during the interview. You can help this through speaking clearly and slowly during the read-out sections, and using a tone of voice that keeps their interest up.

As always, it is vital that you stick to the precise wording given for all questions – please do not prompt on unprompted questions, and do not rephrase any question (prompted or unprompted). This is vital for ensuring consistency between interviews, and with previous surveys that we are using as benchmarks. If the respondent does not understand a particular phrase or word, you can repeat the wording again, but you must not explain it to them in any other way. If the respondent still doesn’t understand, code “don’t know” but record on the feedback form the fact that the question was unclear and, if possible what was unclear.

A large proportion of the questions make use of showcards. The showcards for some questions are long, so please just ask respondents for the letter/letters that apply.

For any “other, specify” questions, probe fully, and code as much detail as possible. For the pilot, please write clearly and also add any recurring “others” to the interviewer feedback form, as we may then include these as pre-codes for the mainstage.

Wellcome Trust Monitor wave 2 – Technical report Appendix A – Cognitive testing and pilot survey documents As well as “other” options, we would like to know any questionnaire issues that arise

from the fieldwork. This could include common areas of confusion, incorrect or strange routing or interviewer difficulties.

5. Return of Work

Please inform your Region Coordinator of completed interviews at the end of each working day. It is very important that we have a daily record of interviews completed.

If you have problems with the quotas (or with anything else), please call your Regional Co-ordinator in the first instance or Gemma Rowden in Field on 020 7347 3378.

Many thanks for your involvement with this important project, and good luck!

Juliet Johnson, Research Director

Doug Warren, Research Manager

Jayesh Shah, Research Manager

Michael Clemence, Executive Assistant

Wellcome Trust Monitor wave 2 – Technical report Appendix A – Cognitive testing and pilot survey documents A.7 – Pilot survey report


Pilot Survey Report

Background

Ipsos MORI has been commissioned by The Wellcome Trust to carry out the second wave of the Wellcome Trust Monitor. The Wellcome Trust Monitor is a unique survey on public awareness, knowledge and attitudes towards biomedical research and science. Fieldwork for the mainstage of the survey will be undertaken between May and July 2012

This report documents the findings from the pilot of the second wave of the Wellcome Trust Monitor.

The pilot survey was undertaken in April 2012 to test the survey materials and questionnaire among both young people and adults, and to inform the development of the mainstage survey.

Pilot objectives

The pilot survey aimed to:

Test the flow and structure of the questionnaire, in light of structural changes made for the second wave;

Test the comprehensibility and unambiguity of new questions introduced for the second wave;

Obtain accurate data on the questionnaire duration among both adults and young people to;

Further test the survey materials (introductory letters and leaflets) to ensure that they hold maximum appeal to potential respondents;

Establish the most appropriate ways to introduce the survey in order to minimise refusals; and,

Test the audio-recording process for use in the mainstage of the survey, both in terms of the practical application in field and the subsequent file-transfer and transcription.

The pilot did not however, test the survey methodology due to time constraints.

Methodology

A quota sampling method was used for the pilot. This allowed us to test the survey amongst a broad spectrum of the general public. Four sample points were selected and a target was set of eight adults and five young people in each of the four areas.

The pilot survey was carried out by four interviewers between Tuesday 3 and Sunday 15 April 2012. Fifty respondents were interviewed in four areas of the UK:

Dover, Kent;

Wellcome Trust Monitor wave 2 – Technical report Appendix A – Cognitive testing and pilot survey documents Mid Suffolk;

Harrogate, North Yorkshire, and;

Hackney, London.

Of the 50 people interviewed, 32 were adults and 18 were young people.

Summary

Interviewer feedback on the survey was largely positive. All reported that they had enjoyed carrying out the interviews, and that it was a particularly interesting questionnaire – a factor which also ensured that interviewees were engaged for the duration of their interviews.

The length of the interviews in the pilot was greater than planned for in the mainstage (originally 45 minutes for adults and 39 minutes for young people) – interviewers generally agreed that 50+ and 40+ minutes for adults and young people respectively was a more accurate measure of the time required.

Audio recording responses is an experimental approach and the interviewers agreed that they were unsure about it – particularly on whether a recording had been made and on acceptable levels of background noise.

The household grid had some bugs in it, particularly in the young people script, which will be fixed now that we have access to NatCen’s Blaise script.

Interviewer feedback

Respondent engagement

All interviewers agreed that respondents were generally very interested in the survey, and remained engaged throughout, despite its length. One expressed concerns about the length of the young persons’ survey, as they felt that one of their 14 year old respondent’s attention began to wane towards the end. We need to bear in mind though that given the methodology used, interviewers were largely not constrained about who they could interview, whereas in the mainstage they will have to work harder to encourage participation at all the addresses they are issued with.

Incentives

It was felt by all interviewers that a £10 incentive worked to encourage participation. The decision to provide a cash incentive on the spot (vouchers for young people) was seen as a particularly successful way to interest people. It was strongly felt that the offer of cash on the doorstep was a key factor in respondents’ decisions to participate – as one interviewer remarked, in one instance the respondent had “run out of money” and the chance to get £10 immediately was very appealing.

The £10 voucher for young people was also seen overall as a positive encouragement; although it was felt that the voucher had more appeal to respondents in the lower years of the age bracket. An interviewer remarked that a £10 voucher had much greater value to a 14 year old than to an 18 year old – particularly as an 18 year old respondent might have received a £10 cash incentive were they selected for the adult interview instead. Interviewers requested that the full list of shops where the vouchers could be used be included in the interviewer pack for

Wellcome Trust Monitor wave 2 – Technical report Appendix A – Cognitive testing and pilot survey documents the main stage in order to alert respondents to the full range of options the voucher held for them. We will provide this in the mainstage.

One interviewer pre-empted our main stage incentive strategy by suggesting that participation could be further boosted by varying incentive amounts according to the local area – offering incentives of £15 to people in poorer neighbourhoods where participation in surveys is typically lower. This was agreed as a good idea, although interviewers expressed concerns that respondents might meet others who had earned a greater incentive. Furthermore, dropping incentives in areas where people are usually happier to participate in surveys to £5 was agreed that it would be perceived as not worth the time investment for the individual.

Interview timings

It was felt by all interviewers that the length of the interview was longer than initially stated. Looking at the pilot sample data, this hunch is borne out by the evidence. The average length of the pilot interviews was:

45.2 minutes for the young peoples’ survey; and

52.5 minutes for adults.

The young peoples’ survey was six minutes longer than the budgeted length, and in the pilot the household grid (which will take up around three or four minutes) was omitted. Interviews ranged in length, with the shortest taking 35 minutes, and the longest 55 minutes.

The adults’ survey was 7.5 minutes longer than the budgeted length, with a range of 40 to 90 minutes for completion of individual interviews.

Although in the mainstage we can expect the duration of the interview to decrease by perhaps a few minutes as interviewers become increasingly proficient in using the script, this will be insufficient to bring the survey down to its pre-agreed length. Some questions in the pilot were marked for inclusion only if there was time – we recommend that these should now be cut (or kept back for wave 3). Please see the ‘action taken’ section for more information.

Audio recording

Interviewers were positive about the use of audio recording in interviews. Audio recording is an experimental technique for this type of interview, and during the pilot we trialled four different models of microphone to determine which would be used for all interviews in the mainstage. We are currently examining the recordings and will make a decision shortly, although there is still some concern over recording quality.

The feedback we received from the interviewers suggested that we make improvements to the script and storage of interview files to assist them in ensuring that responses are recorded. Interviewers told us that they would be more comfortable if the script provided them with confirmation that a recording had been saved (beyond the pulse button in the script), and if after the interview they could check to see that the audio files had been saved on the computer. They also wanted further instructions on whether they should record themselves reading out the questions.

Wellcome Trust Monitor wave 2 – Technical report Appendix A – Cognitive testing and pilot survey documents Materials

Both the leaflet and introduction letter were viewed positively by the interviewers. The leaflet was attention-grabbing and informative, and the letter looked official and professional. There was no problem with either the incentive form or the consent form, although one interviewer forgot to use the consent form. We will emphasise this at the briefings and in the interviewer instructions. Also, that respondents were generally quite relaxed about being audio recorded.

The showcards worked well, but one interviewer noted that it was useful to tell the respondent that they won’t be using all the cards (as they are for adults and young people in a single pack).We will include this tip in the interviewer instructions.

Questionnaire feedback

Question difficulties

Demographics intro

It was felt that some of the demographic information about working status was quite intrusive.

Recommendation: We will add in additional reassurances about the purpose of the demographic information – i.e. ensuring we speak to a good cross-section of the public etc.

NewsWeb

The labelling of the codes wasn’t easy to follow on screen.

Recommendation: We will amend the lettering to make it easier to follow.

WWWHrs

The text substation of “OTHER THAN FOR WORK” for young people and those not in work. This is because the demographics have been moved from the start to the end of the interview.

Recommendation: We will not have the text substitution for young people, but leave the wording for adults to maintain consistency with Wave 1.

Carryout

This question asked respondents to name groups or organisations they were aware of which carried out medical research. This question was poorly answered because it is a closed list question – the interviewers were given a list of answers to which they had to code respondents’ answers. Few respondents were able to name any organisations. A further issue arose when respondents provided answers which were incorrect, but the interviewers did not know enough to be able to tell or to code – for example one respondent suggested ‘NICE’ (the National Institute for Health and Clinical Excellence). Although an incorrect response, as an independent government agency within the Department of Health but exclusively concerned with the NHS, it is unclear which code NICE would fall under.

Recommendation: Further instructions will be given to interviewers in the mainstage survey to ensure that they are able to code all possible responses correctly. Further, particular attention

Wellcome Trust Monitor wave 2 – Technical report Appendix A – Cognitive testing and pilot survey documents will be paid to the coding of this question post-fieldwork to ensure that any ‘other’ responses are appropriately back-coded.

Trust questions – TrPat and TrScDrg/TrPrIn

TrPat: This question asked respondents how much they trusted ‘patient groups’ to provide accurate and reliable information. A major issue was that the large majority of respondents did not know what a patient group was, and the interviewers did not know either.

TrScDrg/TrPrIN: These questions ask similar questions – trust in scientists working for pharmaceutical companies and in ‘private industry’. Respondents felt that these questions were a repetition of each other, and the interviewers felt that respondents were answering the same for both.

Recommendation: Delete TrPat and NoTrWhy for TrPat. Delete TrDrg and NoTrWhy for TrDrg.

MedRecHe

This question asked how respondents felt about researchers accessing their medical records for the purposes of research. Interviewers said that the common response was ‘yes, if it’s confidential’, and they were not in a position to provide the answer to the respondent’s question.

Recommendation: The Wellcome Trust will discuss internally the value of this question, in light of interviewer feedback and in light of the difficulty in fully conveying the legal complexities of the highlighted scenario.

GenMean, DNAMean, GMMean (audio Qs)

Interviewers also noted long pauses from some when answering these questions. The long pauses were generally from those who had answered code 4 (heard of, but little understanding) at the filter questions for each audio recording question (KnowDNA, KnowGM and KnowGen). It was felt that people who were closer to scoring 5 in the filter questions (never heard of the term) were reporting 4 in order to appear more knowledgeable, and being tripped up later in the survey.

Recommendation: Although people who coded 4 at the previous questions were not asked these questions in wave 1, it was decided to keep the routing as it is. We will however, add in a reassurance for these people specifically (using text substitution) that it is not a test etc.

SciStud

This question asked respondents to explain what it meant to describe something scientifically in their own words. Interviewers reported that this question was particularly difficult for respondents, with a lot of long pauses and hesitancy.

Recommendation: As this is a repeat question, we should leave this as it is, but provide interviewers with guidance in the interviewer instructions that they can reassure participants that it doesn’t matter if their answers are right or wrong.

Public attitudes to science governance

Wellcome Trust Monitor wave 2 – Technical report Appendix A – Cognitive testing and pilot survey documents Interviewers requested that the main question be presented at the top of each of the seven questions on decision making as they felt that by the end respondents had begun to forget what the original question was, and having it there for them to read out would serve as a useful reminder.

Recommendation: The question text will be included on each sub-question so that it can be referred back to if needed by the respondent.

Environment, nutrition and behaviour change section

Interviewers requested that the main question be presented at the top of each of the five questions on obesity as they felt that by the end respondents had begun to forget what the original question was, and having it there for them to read out would serve as a useful reminder.

Recommendation: The question text will be included on each sub-question so that it can be referred back to if needed by the respondent.

CASI questions

Interviewers raised two issues with the CASI (self completion) questions, although these were unrelated to the questions themselves.

Firstly, all reported that respondents found it difficult to use their CAPI machines. As a result the interviewers had to help some respondents select their options. The interviewers suggested that the CASI section be made much more user friendly for the mainstage.

Secondly, some interviewers also expressed some disquiet about the transmission of illnesses and dirt when handing their laptops over to respondents (and vice versa).

Recommendation: Instructions will be inserted prior to the CASI section to help respondents better understand how to operate the CAPI machine. In addition, a practice question will be included which can be run through by the interviewer in conjunction with the respondent to ensure they know how to work through the script. We will also suggest that interviewers carry antibacterial wipes with them, and wipe their keyboards/ screens before handing the machine to a respondent and then when they’ve handed it back. This will be at interviewers’ discretion.

YinAct

The text substitution did not work correctly in the pilot. This was in part due to not having the NatCen script, so it should be better now.

Recommendation: Continue to check the routing/ text substitution rules and the appropriateness of the wording.

HHIncome

Interviewers found it difficult to find the correct letter as they had been randomised on the showcards to aid confidentiality.

Recommendation: We will ensure the codes are alphabetised on screen (if the question is kept in).

Wellcome Trust Monitor wave 2 – Technical report Appendix A – Cognitive testing and pilot survey documents OptioWhy

This question asks young people about choosing science options at school. However, it was noted that not all young people in the survey will have chosen subjects yet.

Recommendation: Add in an additional code for ‘Haven’t chosen options yet’ and include an interviewer note on screen to this effect.

General actions taken:

The mainstage questionnaire is on target to be signed off by Monday 23 April) so that interviewer briefings can begin soon after.

We recommend he questions in the pilot which were only to be included in the mainstage survey if there was time be cut.

The interviewer instructions for the mainstage will be significantly strengthened around audio recording, providing explanations and troubleshooting tips for interviewers in the field as well as best practice guidance so interviewers can make the best recordings. Additionally, we are investigating options for the script to provide confirmation of saving files to interviewers, as well as providing easier interviewer access to the files, once saved.

We will also check that interviewers are given enough space to write in ‘other’ comments for all appropriate questions.

A.1 – Cognitive testing recruitment instructions Wellcome...

Documents

Transcript of A.1 – Cognitive testing recruitment instructions Wellcome...