Cognitive Testing of the National Primary Health survey 2015€¦ · This report provides an...

COGNITIVE TESTING OF THE NATIONAL PRIMARY HEALTH

SURVEY 2015

Report prepared for Richard Hamblin, Tania Simmons and Gillian Bohm Health Quality & Safety Commission June 2015

Phase 1: Part A

Background The Health Quality & Safety Commission (the Commission) is currently working with the Australian National Health Performance Authority to propose a set of measures of primary care patient experience. The measures will be aligned with international best practice and global trends and be able to be used to drive improved quality of care. The group provided a draft tool which needed to be tested and amended for the New Zealand primary care environment.

In March 2015 Point Research was commissioned to cognitively test, refine and evaluate the draft survey tool.

Cognitive testing is used to understand how patients understand and interpret questions and instructions. The aim is to find out from patients:

• If the survey instructions are easy to understand; • If the questions are easy to understand; • If the questions are relevant; • If the survey enables them to talk about what they think is important; • If any important questions are missing.

The assessment was to include cultural appropriateness and tested for suitability given patients’ experience of New Zealand’s primary healthcare context.

The proposed approach included undertaking three phases of cognitive testing with patients from the enrolled populations of six pilot Primary Health Organisations (PHOs)1 and their lead District Health Boards (DHBs) participating in the process and refinement of the tool between those rounds.

• The first phase involves cognitively testing the questions with patients in fifteen focus groups from six PHOs;

1 Primary health organisations (PHOs) are funded by District Health Boards to provide primary health care services, mostly through general practices, to people who are enrolled iwth the PHO.

1

• The second phase involves testing the draft set of questions online with a wider set of patients from PHOs, along with staff from practices, PHOs

and DHBs; • The third phase involves testing the revised survey on patients via patient interviews.

The testing was to be undertaken between April and June 2015. The final deliverable will be a survey adopted nationally as a minimum standard for the PHO measurement of patient experience.

This report provides an overview of the first phase of cognitive testing. It is noted that each phase will be reported in full and will be available on the Commission’s website www.hqsc.govt.nz . Note, the cognitive testing of this survey was not a full research project; and was subject to stringent and non-negotiable timeframes.

Method

Phase One

Point Research began by meeting with the Commission and the representatives from PHOs to outline the plan and work through the steps required to invite practices to be part of the cognitive testing phase and contact patients within the time frames.

Six PHOs agreed to participate in the cognitive testing providing a North Island, South Island, urban, and rural spread.

• Procare (Auckland); • National Hauora Coalition (national but mainly providing services in Auckland, Waikato and Tairawhiti); • Midlands Health Network (Gisborne, Taranaki, Taupo-Turangi and Waikato); • Whanganui Regional PHO (Whanganui and rural areas such as Taihape); • Compass (Wellington, Kapiti and Wairarapa); and • Pegasus (Canterbury).

It was decided that fifteen focus groups would be undertaken nationally, covering the following population groups:

• Adults aged 25-64 years;

2

http://www.hqsc.govt.nz/

• Maori (2); • Pasifika (2); • Asian (1); • Refugee new migrant (1); • Older adults aged 65 years + (2); • Younger adults aged 18-24 years (2); • Women (2); • People with disabilities (1).

The testing programme and establishment of the focus groups relied heavily on the work of PHOs and practices.

PHOs were asked to confirm which type of focus groups would be held with their PHO and practices. They were also asked to introduce Point Research to practices and provide the practices with communications about the project from the Commission.

Following the introductions of Point Research to the practices, the practices were then approached to populate the focus groups with patients from the population groups identified. Practices were asked to invite up to 10 participants to each focus group, provide the participants with the information sheet and consent forms and then provide the names and contact details of those who consented to participate to Point Research.

The criteria for selection were that the patients belonged to the population identified and were available and consented to participate on the dates set.

Some practices contacted existing groups, such as members of a consumer panel, to populate groups. Others put notices on notice boards or asked receptionist to invite patients when they came to the clinic.

Point Research sent out information sheets and consent forms to each practice to give participants. Each participant was rung the day before the focus group to remind them of the time, date and location.

The focus groups were held at venues familiar to the patients, either in a room at the practice or a local venue. Refreshments were provided and participants were incentivized with either a $40 voucher or $40 in cash depending on the recommendation of the practice.

3

Questionnaire

Prior to testing the questionnaire had been scrutinized by staff from the Commission, the Ministry of Health (MOH), Cemplicity, the six PHOs, the Royal New Zealand College of General Practitioners and Point Research. A total of 74 questions were finalised by the Commission for testing, along with introductions and conclusions.

Respondents

Focus Groups

A total of 10 focus groups with 88 participants were held in:

• Henderson, Auckland - Pacific 5 participants + 1 PHO representative • Auckland - Asian, KarangahapeRd, Auckland city, 8 participants • Taihape 11 participants • Whanganui older adults - 9 participants • Te Kuiti - rural 9 participants + 1 PHO • Tokoroa – Maori 8 participants + 4 community staff + 1 PHO • Christchurch - disability 3 participants • Woolston - 25-64 years 10 participants • Christchurch - older adults 10 participants • Auckland - young adults 8 participants

Note that 8 people with disabilities attended other focus groups in addition to people with chronic conditions who, whilst not considering themselves disabled said they found their conditions disabling. Those who considered themselves disabled said that they sight, hearing or physical disabilities.

4

Demographics of focus groups

Women were slightly over-represented in the focus groups with 49 women (56%) participating.

Gender of participants (number)

There was a spread of ethnicities. Fifty one (57%) of the focus group participants were NZ European. Eighteen Maori (21%) and twelve Pasifika (14%), along with eleven people of Asian ethnicities (13%) participated in the focus groups.

39 49

0102030405060

Men Women

5

Ethnicity of participants (number)

NB: People could select more than one ethnicity

Age group of participants (number)

51

18 12 11

3 3 0

10

20

30

40

50

60

NZEuropean

Māori Pasifika Asian MELAA Other

8

11

18 18 18

12

3

02468

101214161820

Age

16-24

25-34

35-49

50-59

60-69

70+

Not specified

6

General comments

The focus group respondents think the survey is important and many want to participate in the testing programme again to help us get the questionnaire `right’.

In general, most people found much of the questionnaire easy to understand, easy to answer and relevant, however suggested changes to the questionnaire that it would help to make it clearer and less onerous to answer. A substantial group of respondents, however, found the questionnaire wordy, and difficult to understand. Some respondents, and not only those who spoke English as a second language, had to read some introductions, questions and response options more than once and even then had to ask us what they meant.

The primary issues with the questionnaire were:

The question wording and construction

Length of the questions: Many of the questions were considered to be long and overly wordy which made them difficult to understand. This sentiment was shared by many others who felt that the questions could be simplified and shortened.

Structure: The important part of the question is often at the end of the sentence. One participant pointed out that “It feels like a lot of throat clearing before it gets to the point”. Respondents felt that the important part should at the beginning of each question;

Unfamiliar language: Some of the language in the questions was unfamiliar to many patients, in particular words such as triage, primary health care, cardiologist;

Too many answer options: Some found the number of answer options too long and off-putting. Respondents found detailed questions to be overwhelming and off-putting. Some participants noted they would abandon the survey when the questions options became too detailed. The very detailed response options were found to be confusing and expected precise recall of `insignificant events’ and the details were often vague to patients;

7

Repetitive: They felt the survey was repetitive e.g. There is no need to repeat ‘In the last 12 months’ and suggested that this be placed in the section introduction

Topic areas

Their experience: Patients were asking for questions that allowed them to talk to about their experience of care in the GP’s surgery. Some people thought that some important questions were missing, particularly in regard to how they were treated by staff, and they felt strongly they wanted to report and respond to poor treatment and they did not have opportunities to do this in the questionnaire.

In particular, they wanted to talk about:

• Wait times; • Being listened to; • Experiences with `rude’ staff, • Having their questions answered; • Not being rushed out of the door; • Clinic access for disabled patients; • Churn/ high turnover of doctors;

• The availability of the doctor you want to see; • Expense and staff not explaining costs up-front; • Lack of choice; • Receptionists/ admin staff (respectful, helpful); • Translation/ interpretation questions.

It was suggested that the section on being admitted to hospital should have questions about:

• Appointment cancelations; • Different rules to GP clinic, such as being allowed to have family with you

8

• The interpretation/ translation/ being able to communicate and understand.

It was also recommended considering asking a question about whether patients felt these issues had any impact on the quality of their safety and health. For example, patients who felt that they had been subject to `poor treatment’ from their point of view, believed this to be the case. The question could be “Do you think this had an impact on the quality or safety of your health care? (Options) Please explain”

Overall

Very few respondents (fewer than 20%) felt that they would answer the questionnaire as it was” boring”, “too wordy”, “too long”, “too detailed” and it did not ask them questions about “what mattered” to them. Were these issues rectified, most said that they would be happy to answer the questionnaire as there were things that they would like to say about their experiences, both good and bad.

In view of the early feedback, it was decided to stop testing the questionnaire after the first ten groups, modify the questions based on the feedback and restart the testing process with the five remaining groups.

Questionnaire and Testing Results Phase One Part A

Intro/ Question Clarity, Ease and Relevance Focus group feedback Our recommendations

1. Could you tell us if you are answering this survey on behalf of yourself or someone else?

• Myself

• Someone else unable to answer

Clarity: High

Ease: High

Relevance: High

Clear Easy Why unable to answer – do we need this sentence

Could you tell us if you are answering this survey on behalf of yourself or someone else?

• Myself • Someone else Please tell us why):___________________

9


this survey (Please tell us why):___________________

2. This survey has several sets of questions covering different areas of health care. Below we ask you some general questions so we know which questions will be relevant to you as the survey progresses.

Clarity: Medium

Ease:

Relevance: Low

A long slow start until relevant and meaningful questions kick in Rambling

We start with some general questions so we know which topics to ask about later in the survey. Start with GP questions and then ask about other areas of healthcare.

3. Do you take any medicines (including over-the-counter medicines) on a regular or ongoing basis?

• Yes • No • Don’t know

Clarity: Low

Ease: Easier if clarified

Relevance: High

What does ‘over the counter’ mean? What does ‘on-going basis’ mean? Does this include vitamins, pain killers, supplements? In general remove don’t know (would have to be an idiot)

Do you take any medication regularly? This includes vitamins, pain killers and supplements.

• Yes • No

Remove don’t know

10


4. In the past 12 months, were you referred for any tests such as x-rays, scans or blood tests?


Clarity: Medium

Ease: Medium

Relevance: High

What does ‘referred’ mean? Replace ‘referred’ with ‘sent’? Remove don’t know – people were starting to be very irritated by the don’t know options What’s the intent? Picker asks: Did you have any tests such as x-rays, scans or blood tests? Ie: doesn’t ask if you were referred. In the last 12 months is repetitive. Could introduce and remove the last 12 months

Did you have any tests such as x-rays, scans or blood tests?

• Yes • No

5. In the past 12 months and excluding any overnight stay in a hospital, have you seen one or more different doctors or specialists, including your GP?


Clarity: Low

Ease: Med

Relevance: Medium

Has inclusions and exclusions which makes it confusing. Question 5 and 6 seem similar – why not ask about GP question 5 and specialists questions 6? Remove don’t know

Replace “excluding …” with “not counting” or Have you seen any other health specialists (eg. nurse, physiotherapist, psychologist, social worker, counsellor, pharmacist, or dietician)?

11


6. In the past 12 months, have you visited a specialist medical doctor such as a surgeon, a cardiologist or a psychiatrist?


Clarity: Low

Ease: Med

Relevance: Medium:

Some people didn’t know what a cardiologist and psychiatrist was but ‘specialist medical doctor’ was self-explanatory. Remove don’t know

Remove the explanations Have you seen a specialist doctor?

7. In the past 12 months, have you been to a public hospital emergency department for your own health care?


Clarity: Medium

Ease: Medium

Relevance: Medium

Remove for your own health care Remove don’t know Not reading public hospital – included A and M

Have you been to the Emergency department at the public hospital?

8. In the past 12 months, have you been admitted to a hospital overnight?


Clarity: Med

Ease: Med

Relevance: High

Remove don’t know

Have you stayed in hospital overnight?

9. Has any health professional ever diagnosed you with or treated you for a chronic health condition expected to last, or which has already lasted for 6 months or more?

Clarity: Low

Ease: Low

What counts as a ‘health professional’? What does ‘diagnosed’ mean? What does ‘chronic’ mean? Remove don’t know

Do you have a health condition that will last more than 6 months?

12



Relevance: HIgh

10. Section: Regular place of care including GPs and practice nurses

This section has a few questions about the place and people that you consider your regular source of routine and ongoing health care.

Clarity: low

What do you mean by ‘regular’? Say ‘usual’ instead. Don’t understand ‘regular place of care’. Felt it was rest home. The previous section asked about ‘your GP’ which confused a few people. It made them wonder if they had completed the last section properly or if this section was about somewhere different.

Replace ‘regular’ with ‘usual’. This section has a few questions about your GP clinic or community health clinic.

11. Is there one primary care practice, health centre, or clinic you usually go to for most of your health care?

• Yes • No, I do not have a place that I

usually go • Don't know

Clarity: Low

Ease: Medium

Relevance: intro only

What is ‘primary care’? GP or something else? Low understanding of primary health care I was told that, to recently immigrated Pacific people, “primary” makes them think of schools. Not doctors clinics.

Use “community health clinic” instead of “primary care practice”? Is there one GP clinic or community health clinic you usually go to? Remove don’t know

13


“Community” is better. GP is fine. Clinic is fine. The clinics come to them – eg mobile services Not don’t know Have lots of places I normally go.

12. What sort of health care service is this?

• A doctor’s office, clinic, surgery or practice

• A walk-in clinic such as a nurse clinic

• An after-hours or an Accident and Medical Centre

• An emergency department at a public hospital

• A community health centre • A local pharmacist • Other • Don’t know

Clarity: Low

Ease: Too many options - overwhelming

Relevance: Low

“What sort of health care service …” is a bit unclear Who cares? Can they not get this another way? Way too long – too many options and words - simplify Immigrants from the Pacific Islands commonly associate “surgery” with hospitals ED is our after-hours clinic (Whanganui) After hours is unclear. Dr … is open late and weekends too, so does that make it an ‘after hours clinic’?

Get as background information Replace the question with, “Is the place you usually go to a: Allow multiple options. More than one. Remove “surgery” from the first response option

• GP or doctor’s clinic • Nursing clinic • After hours clinic • An emergency department • Hospice • Community health centre • Pharmacist • Mobile health clinic

14


Not don’t know Remove don’t know

• Other. Remove don’t know

13. Does the place you indicated you usually go to have after hour access where patients can be seen by or be contacted by a doctor or nurse when they are normally closed?

• Yes • No • Don't know

Clarity: Low – too wordy

Ease: Once understood could answer it

Relevance Low – who cares – go to A and M:

‘indicated’ wasn’t clear to one person. Too wordy This question is too long. After hours telephone numbers. Does after hours include weekends? Healthline Can’t see point

Does the place you usually go to have a phone service, doctor or nurse available during evenings, nights or weekends?


14. Is there one GP you usually see? • Yes • No, I do not have a regular GP • 3. Don't know

Clarity: High

Ease: High

Relevance: High –but lots do now have a regular GP

Remove don’t know Note most don’t have one GP but do have one clinic

Good question. Remove don’t know

15. How long have you been seeing your regular GP?

• Less than one year • One to two years • Three to four years

Clarity: Too many options

Ease: Low

Relevance: Low

Allow for regular place of care, not just GP, different doctors. The surgery open longer than the GPs been there.

How long have you been going to your usual GP?

• Less than one year • One to five years • More than five years

15


• Five to seven years • More than seven years • Don't know

See lots of GPs and would like to be included What about patients who leave for NZ for a year or two then come back? Whenever I am in NZ I see my GP

16. In the past 12 months, have you ever waited longer than you would consider acceptable to get an appointment with (please tick)

No Yes N/A Your regular GP?

Any other GP at the practice you usually go to?

A nurse at the practice you usually go to?

If you answered ‘yes’ to any of these, please tell us why [TEXT BOX]

Clarity: Low

Ease:Low

Relevance: Low but should be high

Answered this incorrectly. Some patients considered it to be wait times in the clinic.

Word ever is unnecessary

There needs to be a question on wait times immediately before this question as they are consistently misreading this question as wait times at the surgery as that is what they want to answer. In 3 focus groups all the participants misread this. In the other focus groups at least one person in the group misread it.

The next questions are about the last 12 months Have you waited longer than you consider acceptable to get an appointment? If you answered ‘yes’, please tell us why [TEXT BOX]

16


Recommend: Add a question on wait times as this is important

Recommend Add a question on receptionists

and admin staff as this is important

17. In the past 12 months, was there a time you did not visit a GP because of cost?


[ADD TEXT BOX]

Clarity: Clumsy

Ease: Med

Relevance: High

Would have to be an idiot to “not know”.

Was there a time you did not visit a GP because of cost? Remove don’t know TEXT BOX

18. In the past 12 months, was there ever a time when you thought that you needed health care but you didn’t receive it?


[ADD TEXT BOX]

Clarity: Low

Ease: Low

Relevance: Not sure as the reasons are so varied – what would it change e.g. my mum thought I should wait

What do we mean when we say ‘needed health care’? and ‘but didn’t get it’? Lots of discussion about this. Is this just GP or other providers too? Answered yes, but not sure that I was sick enough./ mum said I didn’t need to etc.

Replace ‘receive’ with ‘get’ Was there ever a time when you wanted health care but you couldn’t get it?

• Yes • No

[ADD TEXT BOX] Ie replace needed with ‘wanted’? Replace didn’t with ‘couldn’t’?

17


A participant said sometimes the reason for not getting care is unwisely choosing not to and nothing to do with health system. ‘Receive’ in the question seemed odd to one person. Get health care?

19. Now a few questions about the regular GP practice you usually go to for health care.

Clarity: Medium

GP practice – replace clinic Now a few questions about the regular GP clinic you usually go to

20. When you contact your GP practice’s office with an urgent health care question or concern during regular practice hours, how often do you get an answer that same day?

• Always • Often • Sometimes • Rarely or never • I didn’t get an answer at all • I've never tried to contact them

by phone

Clarity: Low

Ease: Medium

Relevance: Unsure

Good. Too many options – hard to read Some people said if its ‘urgent’ they would go to hospital, not their GP. Urgent has a very high threshold – if it is urgent they ring an ambulance – and are told to do this on the answerphone Is this about getting the GP others in the clinic too?

When you contact your GP clinic about something important, do you get an answer the same day?

• Yes, Always • Yes, Sometimes • No • N/A

18


21. Does your regular GP explain things in a way that is easy to understand?

• Always • Often • Sometimes • Rarely • Never • Don’t know • N/A

Clarity: High

Ease: Overwhelming options

Relevance: High – but it is also about listening and answering questions

Good.

Question good Too many options – 3 is better

• Yes, Always • Yes, Sometimes • No • N/A

22. Does your regular GP seem to know the important information about your health care history?

• Always • Often • Sometimes • Rarely • Never • Don’t know • N/A

Clarity: Wordy

Ease: High

Relevance: Med

Too many options – 3 is better Does your usual GP seem to be aware of your medical history?

• Yes, always • Yes, sometimes • No • Don’t know

23. Does your regular GP or a practice nurse help you to coordinate or arrange the care you receive from other doctors and places when you

Clarity: Reread many times

What does ‘coordinate’ actually mean? Change ‘coordinate’ to ‘organise’ or ‘arrange’?

Does your usual GP or nurse help arrange any specialist or other appointments you need?

• Yes, Always

19


need to? • Always • Often • Sometimes • Rarely • Never • Don’t know • N/A

Ease: Med

Relevance: Low

Very confusing Too many options

• Yes, Sometimes • No • • N/A

24. Does your regular GP treat you with respect and dignity?

• Yes always • yes sometimes • no

Clarity: High

Ease: High

Relevance: High

Good. What does ‘respect and dignity’ mean? The person who asked this question understood “Is your GP respectful?” Two women weren’t sure these questions were relevant because if the GP wasn’t respectful etc. they would just change GPs (but they thought having an avenue to give feedback about that was great.

Does your GP treat you with respect?


20


25. Does your regular GP treat you with kindness and understanding?


Clarity: High

Ease: High

Relevance: Moderate

Good. Two women weren’t sure these questions were relevant because if the GP wasn’t kind etc. they would just change GPs (but they thought having an avenue to give feedback about that was great.

Keep Does your GP treat you with kindness and understanding?

26. Have you been involved as much as you wanted to be in decisions about your care and treatment?

• Yes • Yes, to some extent • No

OPEN TEXT OPTION

Clarity: High

Ease: High

Relevance: High

Good. One person didn’t understand the question, especially ‘involved’ and ‘decisions’. She understood “listened to what you what you wanted to happen and not just tell you what to do”

keep

27. If there is something you would like us to know about your experience of care with your regular GP, please tell us here. TEXT BOX

Clarity: Moderate

Ease: High

Relevance: High

A person asked what ‘experience of care’ means? Tell us is their view Like us to know is ours Does this matter?

Is there something you would like to tell us about your experience with your GP?

21


28. Section 2: Medicines The next questions are about medicines, including over-the-counter medicines, you have used during the last 12 months. From now on we’ll refer to these as ‘medicine(s)’.

Clarity: Medium

Does this include vitamins, pain killers? What does ‘over-the-counter’ mean?

Keep consistent medication or medicine

29. Were you involved as much as you wanted to be in decisions about the best medicine for you?

• Yes definitely • Yes to some extent • No • I did not want to be involved

Clarity: High

Ease: High

Relevance: High

Good Keep

30. Did staff explain the purpose of the medicines you were to take in a way you could understand?

• Yes definitely • Yes to some extent

Clarity: Moderate

Ease: High

Relevance: High

Was the purpose of the medicine properly explained to you? Does this include pharmacists?

Was the purpose of the medicine properly explained to you?

22


• No

31. Did staff tell you about possible medicine side effects in a way that you could understand?

• Yes definitely • Yes to some extent • No

Clarity: Moderate

Ease: High

Relevance: High

Good Why staff –just makes it longer

Were the possible side effects of the medicine explained in a way you could understand?

32. Did the staff member tell you what could happen if you didn’t take the medicine that was prescribed to you in a way you could understand?

• Yes definitely • Yes to some extent • No

Clarity: Low - confusing

Ease: High once understood

Relevance: High

Clumsy. Needed to be read twice

Were you told what could happen if you didn’t take the medicine, in way could understand?

33. Did you take the medication as prescribed?

• Yes, definitely • Yes to some extent • No

Clarity: Moderate

Ease: Moderate – too wordy

Good. “… there were unpleasant side effects” was unclear to someone. Suggested using ‘I wasn’t happy

Instead of ‘prescribed’, Use “as you were told to?” “follow the instructions?” Did you follow the instructions

23


(if ‘yes to some extent’ or ‘no’) Can you tell us why?

• It was too expensive • There were unpleasant side

effects • There were serious side effects • I was given the wrong

medication • I was given the wrong dose • Other

Relevance: High with the side effects’ Answers should include ‘I forgot’ and ‘I felt better’ Change ‘prescribed’.

when you took the medication?

• Yes, Always • Yes, Sometimes • No

Sometimes and never response options should include ‘I forgot’ and ‘I felt better’ Simplify options

34. In the past 12 months, have you ever been given the wrong medication or wrong dose by a doctor, nurse or pharmacist?


Clarity: High

Ease: Low

Relevance: Low

Can’t answer. How would you know?

Most can’t answer this.

35. Did this wrong medication or dose cause …?

• A need for you to seek medical advice

Clarity: Low

Ease: Low – options too wordy to understand

Some people found the wording of this question confusing. Some felt we should change ‘dose’ to ‘amount’

Because of the wrong medication or dose, did you …:

Yes No

24


• A need for you to be admitted to hospital

• A reaction requiring medical care

• No health problem

Relevance: High if it happened

They would like ‘stop taking it’ added in

Stop taking it

Get medical advice

Get medical care

Get admitted to hospital

Is there anything else you’d like to tell us.

36. In the past 12 months were there times when you delayed collecting or did not collect a prescription due to the cost?


Clarity: Low


Relevance: High

Had to read twice. Clumsy

Has cost stopped you from picking up a prescription? Remove don’t know

37. If there is something you would like us to know about your experience of medication prescription and management, please tell us here. TEXT BOX

Clarity: Moderate


Relevance: High

No idea what management means in this context

Is there something you would like to tell us about your experiences with your medication?

25


Some people in one group felt there were too many questions in the medication section

38. Section 3: Diagnostic services Now a few questions about tests (such as x-rays, scans or blood tests) you may have been referred for in the past 12 months.

Clarity: Low

What are ‘diagnostic services’? Not a good section heading. Do ultrasounds count?

Section 3: X-rays, tests and scans Now a few questions about medical tests and scans (like x-rays, scans and blood tests) you may have had in the past 12 months.

39. Did a member of staff explain why you needed these test(s) in a way you could understand?

• Yes, completely • Yes, to some extent • No • I did not need an explanation

Clarity: Moderate

Ease: High

Relevance: High

Put the important point in the question first (not did a staff member). Who is a ‘member of staff’? Do you mean a ‘health professional’?

Was the need for the test(s) explained in a way you could understand?

• Yes, completely • Yes, to some extent • No • I did not need an

explanation

40. Did a member of staff tell you how you would find out the results of your test(s)?

Clarity: Moderate

Ease: Moderate

We were told we would not be told Would or could?

Were you told how you could find out the results or your tests?

• Yes

26


• Yes • No • Not sure/ Can’t remember • I did not need an explanation

Relevance: Moderate Don’t need the ‘can’t remember’ option

• No • Not sure • I did not need an

explanation

41. Did a member of staff explain the results of the tests in a way you could understand?

• Yes, completely • Yes, to some extent • No • Not sure / Can't remember • I was told I would get the

results at a later date • I was never told the results of

the tests

Clarity: Moderate

Ease: Most don’t get results

Relevance: Moderate

Not sure why this is here - I was told I would get the results at a later date?

Were the results of the tests explained in a way you could understand?

• Yes, completely • Yes, to some extent • No • Not sure • I was told I would get the

results at a later date • I was never told the

results of the tests

42. Section 4: Specialist, Allied Health and Support Services Now a few questions about health

Clarity: Very low

The section heading isn’t clear. Who are they (specialist, allied health, support….)?

Drop ‘in the past 12 months’ from the questions

27


professionals you may have seen or talked to in the past 12 months. Examples of allied health and support services are nurse, physiotherapist, psychologist, social worker, counsellor, pharmacist, and dietician.

The examples are better.

43. In the past 12 months, have you seen a doctor (other than your regular GP), specialists or allied health and support services?

• I haven't seen anyone other than my normal GP or nurse

• One person • Two different people • Three to four different people • Five to six different people • Seven to ten different people • More than ten different people • Don't know • I havent seen anyone

Clarity: Low

Ease: Low

Relevance: Low

The response option list is off-putting. 3/10 people felt they could answer it. Too specific Does the question include nurses in the GP clinic?

Delete the question How many specialists or health professional other than your normal GP or practice nurse have you seen?

• I haven't seen anyone other than my normal GP or nurse

• One • Two to five • More than five

44. In the past 12 months was there ever a time when test results, medical records, or reasons for referral were

Clarity: Moderate Don’t know. Can’t answer Drop ‘in the past 12 months’ Was there ever a time when test

28


not available at the time of your scheduled health care appointment?

• Yes • No • Don't know • Not applicable

Ease: Low

Relevance: Probably relevant but can’t answer it

results or information was not available at the time of your appointment?

• Yes • No • Don't know • N/A not applicable

45. In the past 12 months was there ever a time when doctors ordered a test (e.g. blood test, x ray etc) that you felt was unnecessary because the test had already been done?

• Yes • No • Don't know • Not applicable

Clarity: Long

Ease: High if clearer

Relevance: Low – sceptical it happens – consider the reverse more likely ie necessary tests are not done

Good _ does this happen? Has a doctor ordered a test (e.g. blood test, x ray etc) that you felt was unnecessary because the test had already been done? Drop ‘in the past 12 months’

46. In the past 12 months, have you received conflicting information from different doctors or health care professionals about the same condition?

• Yes

Clarity: Moderate

Ease: High

Relevance: High

Good. There may be a better question Drop ‘in the past 12 months’

29


• No • Don't know • Not applicable

47. In the past 12 months did you wait longer than you would consider acceptable to see a nurse, physiotherapist, psychologist, social worker, counsellor, pharmacist, dietician or any other health professional other than a doctor?

• Yes (Please tell us how long you waited)

• No • I didn’t need these services • Don't know

Clarity: Low - Moderate

Ease: High

Relevance: High

Reading while waiting in waiting room Too long – overwhelming and off putting N/A

Have you waited longer than you would consider acceptable to see a health professional other than a doctor?

• Yes (Please tell us how long you waited)

• Less than a week • 1-4 weeks • 1.3 months • 4-6 months • Longer

48. In the past 12 months did you not see or delay seeing a nurse, physiotherapist, psychologist, social worker, counsellor, pharmacist, dietician or any other health professional other than a doctor due to cost?

• Yes • No

Clarity: Low

Ease: High

Relevance: High

Clumsy question. Too long.

30


• Not applicable

49. Now a few questions about specialist medical doctors that you may have seen or talked to in the past 12 months.

Clarity: Moderate

Do we need or talked to – no one had talked to them without seeing them

Now a few questions about specialist doctors that you may have seen in the past 12 months.

50. Thinking about the most recent time you received care from a specialist medical doctor, how long did you have to wait between when you identified you needed care and when you received care?

• Less than a week • 1-4 weeks • 1.3 months • 4-6 months • Longer

Clarity: Low

Ease: Moderate

Relevance: High once understood

Question isn’t clear. Have to read and reread. Need not applicable But would like fewer options – not sure what is important

How long did you wait from the time you were first told you needed an appointment to the time you went to the specialist?

• Less than a week • 1-4 weeks • 1-3 months • 4-6 months • Longer

51. In the past 12 months did you not see or delay seeing a specialist medical doctor due to cost?


Clarity: Moderate

Ease: High

Relevance: High Clear, but clumsy

Needs rewording Has cost stopped you from seeing a specialist doctor? Remove don’t know

31


52. In the past 12 months, how many times have you been to a specialist medical doctor for your own health?

• Once • Twice • Three to four times • Five to six times • Seven to ten times • More than ten times • Don’t know

Clarity: Low

Ease: Difficult as overly precise

Relevance: Low

Is this question asking about the number of specialists or number of visits? Consistently questioned – as is one hospital visit with a surgeon, and anaesthetist, a trainee anaesthetist and an intern one specialist or four? This is hard to answer as some people have seen lots of specialists, eg people with diabetes Do the response options need to include ‘none/never’ Leave out for your own health

Delete

53. In the past 12 months when you received care or treatment from specialists or consultants, did they do the following?

Yes, always

Yes, s.times

No N/A

Tell you about

Clarity: Moderate

Ease: High

Relevance: High:

The difficulty may be the formatting issue on paper

Question c was considered most important so move it to the top.

32


treatment choices in ways you could understand? b. Involve you as much as you wanted to be involved in decisions about your treatment or care

c. Ask what matters to you

54. In the past 12 months, was there a time when a specialist medical doctor did not have basic medical information or test results from your GP about the reason for your visit?


Clarity: Low

Ease: Low – how do you know

Relevance: Low

Clumsy, unclear. Did the specialist doctor know your medical history and the reason for your visit?

• Yes, fully aware • Yes, aware in part • No • Don’t know

55. Does your regular GP seem informed and up-to-date about the care you get from specialists?

• Yes absolutely • Yes to some extent • No • Don't know

Clarity: Moderate

Ease: Can’t answer it

Relevance: Can’t answer but possibly important

Don’t usually go back to GP. Most don’t know

Does your regular GP seem informed and up-to-date about the care you get from specialists?

• Yes, always absolutely • Yes, sometimes to some

extent • No

33


• Don't know • N/A

56. If there is something you would like us to know about your experience of your GP working with specialists, allied health and support services to provide you with health care, please tell us here. TEXT BOX

Clarity: Moderate

Ease: High

Relevance: High:

Does this include eye specialists. If there is something you would like us to know about how well your GP, specialists doctors and health professionals are working together?

57. Section 5: Emergency Department The next questions are about accessing health care from a public hospital emergency department over the past 12 months.

Clarity: Low

½ the participants in one group would go to A+M and answered based on that.

A&E at the Public Hospital The next questions are about accessing health care from a public hospital emergency department.

58. In the past 12 months, how many times have you been to a public hospital emergency department for you own health?

• Once • Twice • Three to four times

Clarity: Moderate

Ease: Low

Relevance: Low

‘you’ in the question needs to be ‘your’ Cant count/ recall so specifically. Does this question need a response option for none/ never

Delete the questions

34


• Five to six times • Seven to ten times • More than ten times • Don’t know

been? Too many options – cut down Hard to answer

59. The last time you went to the public hospital emergency department, what were the reasons you chose to go there?

• It was clearly an emergency • I was taken there by ambulance • I didn’t know if my health

condition was an emergency • I was waiting to see a specialist

but my health was deteriorating • I was waiting for a test or

procedure and wanted it done sooner

• I was told to go to the emergency department (e.g. by a GP, for the purposes of a follow-up, or for an appointment with a specialist who works there)

• It was the only place to go • I go to the emergency

Clarity: Low

Ease: Low

Relevance: Low

‘chose’ in this question made people laugh. What is ‘triaged’ The response option list is very long and still doesn’t cover off all the reasons why people went to ED. Eg: GP was too busy Overwhelming – has to be shorter AND fewer options – one word

Replace with ‘The last time you went to the public hospital emergency department, why did you go there’ Maybe do the top three or four options you are trying to find out about – an

• • I was told to go to the

emergency department by a health professional

• I can’t afford to go anywhere else

• Other (please tell us why) d then other + text box

35


department whenever I need care

• I can’t afford to go anywhere else

• Other (please tell us why) • I was triaged by a nurse on the

phone and advised to go there • Don’t know •

60. Thinking about the most recent time you received care from a hospital emergency department, was it for a condition that you thought could have been treated by your regular doctor/ place of care/ GP?


Clarity: Low

Ease: Low

Relevance: Low

Had to re-read. Not clear Who cares – we needed treatment and the doctor wasn’t open. Not sure what the doctor can and can’t do

Do you think your condition could have been treated by your GP?


61. Does your regular GP seem informed and up-to-date about the care you received from the emergency department?

• Yes absolutely • Yes to some extent

Clarity: Moderate

Ease: Low

Relevance: Moderate

Don’t know. How would you know?

Delete

36


• No • I havent seen my GP • Don't know

62. If there is something you would like us to know about your experience of your GP and the Emergency Department working together to provide you with health care services, please tell us here. TEXT BOX

Clarity: Moderate

Ease: High

Relevance: High if relevant to patient

Good. If there is something you would like us to know about how well your GP and the emergency department are working together?

63. Section 6: Admitted to hospital The next questions are about staying at a hospital overnight in the past 12 months and communication and coordination between the hospital and your GP.

Clarity: Low – too wordy

The next questions are about overnight stays in hospital

64. In the past 12 months, how many times have you been admitted to a hospital overnight?

• Once • Twice

Clarity: Moderate

Ease: Low

Relevance: High

Too many options and lack precision to answer it

Needs fewer options – too hard to answer

Delete

37


• Three to four times • Five to six times • Seven to ten times • More than ten times • Don’t know • I have not been admitted to a

hospital overnight

65. In the past 12 months how many nights have you spent in hospital altogether?

• 0 • 1 night • 2 nights • 3 nights • 4 nights • 5 nights • 6 nights • 7 nights • …

Clarity: Moderate

Ease: Low

Relevance: Low

Way too many – couldn’t count – can it be grouped? Felt overwhelming. Too hard to answer

Delete - too hard to answer

66. Thinking about the most recent time you received care from a hospital; did the hospital make arrangements or make sure you had follow-up care with a doctor or other health care

Clarity: Low- moderate

Ease: Moderate

Relevance: Moderate

Had to re-read the question Does it need an N/A? Make it global introduction

The following questions are about your most recent stay in hospital. Did the hospital arrange follow-up care with a doctor or other health care professional?

38


professional? • Yes • No • Don’t know

• Yes • No • n/a

67. Thinking about the most recent time you received care from a hospital; after you left the hospital did your regular GP seem informed and up-to-date about the plan for follow-up? Select hospital you received care from <drop down box>

• Yes • No • Not applicable (e.g. you did not

see a regular doctor after you were in hospital, did not have a plan or started seeing a doctor after getting out of hospital)

Clarity: Low to moderate

Ease: Low

Relevance: Moderate

Confusing Did your usual GP seem informed and up-to-date about the plan for follow-up?

• Yes • No • Not applicable

68. After you were discharged, were you readmitted to a hospital or did you have to go to an emergency department or after-hours accident &

Clarity: Moderate

Ease: High once

Too wordy Did you have to go back to hospital or get emergency care because of complications within a month after being discharged

39


medical service within a month as a result of complications that occurred during your recovery?


understood

Relevance: High if this happened

from hospital ? • Yes • No

69. If there is something you would like us to know about your experience of your GP and the hospital working together to provide you with health care services, please tell us here. TEXT BOX

Clarity: Moderate

Ease: High

Relevance: High

Too long If there is something you would like us to know about your experience of your GP and the hospital working together?

70. Section 7: Chronic health conditions The next questions are about ‘chronic’ health conditions or in other words ‘long-term conditions’ that have been diagnosed by a health professional and are expected to last or have already lasted 6 months or more.

Clarity: Moderate

Give examples. The next questions are about ‘long-term conditions’ that are expected to last 6 months or more.

40


71. How long ago were you first diagnosed for the condition(s)?

Less than 6 mths ago

Etc. Don’t know

Arthritis

Asthma

COPD

Chronic pain

Cancer

Depression

Anxiety

Other mood disorder

Diabetes

Clarity:

Clarity: Moderate

Ease: High

Relevance: High

What is COPD?

What are ‘other mood disorders?’

Re-order answers. Alphabetically

• Alpha order

• COPD in full words & acronym

• Other mental health conditions

41


Heart disease

Stroke

High blood pressure

Other: _________

72. Please answer each of the following questions using the column headings as a guide.

Almost always

Most of the the time

Sometimes

Generally not

Almost never

N/A

a. In the past 12 months, were you given a written list of things you should do to improve your health?

Clarity: Low

Ease: Low

Relevance: Moderate

They didn’t like the option ‘almost never’. Just say ‘never’. And why ‘almost always’?

The questions are too long.

Include verbal advice?

Written information wasn’t considered as important or helpful as information that was easy to understand, which could be given verbally/ by video, etc.

Drop ‘over the past 12 months from every question. Put it once

a. Were you given information you could understand about things you should do to improve your health?

Response options should be:

Yes, Always

Yes, Sometimes

No

42


b. Over the past 12 months when you received care for your chronic condition(s), did you get help to make a treatment or care plan that would work in your daily life?

c. Over the past 12 months after you received care for your chronic condition(s), were you contacted to see how things were going?

Did you get help to make a treatment or care plan for your CC that would work in your daily life

After a treatment or care plan was made were you contacted to see how things were going?

73. You told us you have been diagnosed and/or treated for diabetes,

Clarity: Moderate This is unnecessary - You told us you have been diagnosed and/or

DELETE AS DOESN’T ASK EXPERIENCE OF CARE

43


heart disease, stroke or high blood pressure or hypertension. In the past 12 months, have you had the following tests or measurements to monitor your condition?

Yes No Don’t know

Blood pressure measurement

Cholesterol measurement

Body weight measurement

Blood sugar measurement

Ease: Moderate

Relevance: Low

treated for diabetes, heart disease, stroke or high blood pressure or hypertension. Don’t know needs to be replaced with not sure

74. If there is something you would like us to know about your chronic or long-term condition(s) health care experience, please tell us here. TEXT BOX

Clarity: Moderate

Ease: High

Relevance: High

Is there anything you would like to tell us about your experience being treated for a long term condition?

44

Cognitive Testing of the National Primary Health survey 2015€¦ · This report provides an...

Documents

Transcript of Cognitive Testing of the National Primary Health survey 2015€¦ · This report provides an...