2018 Q4 Quarterly Pulse Check Survey (QPCS) Summary Report · Other business indexes available: Q4...

2018 Q4 Quarterly Pulse Check Survey (QPCS)

Summary Report

Table of Contents

Item Page no.

1

Executive Summary- Customer Satisfaction Index – Q4 2018 QPCS- Executive Summary: Overview of Results and Background- Executive Summary – Consumer Deep Dive- Executive Summary – Business Deep Dive

3

2

Q4 2018 QPCS Customer Measures

- Customer Satisfaction Measure - Customer Satisfaction Index (CSI)- Satisfaction, Expectation and Comparison to Ideal- Customer Effort Score- Customer Effort Compared to Other Industries

8

3

Insights on Key Primary Opportunity Areas and Satisfaction Drivers

- Key Primary Opportunity Areas (KPOAs) in 2018 - Consumers- Performance of Key Primary Opportunity Areas - Consumers- Performance of Satisfaction Drivers - Consumer- Key Primary Opportunity Areas (KPOAs) in 2018 - Businesses- Performance of Key Primary Opportunity Areas – Businesses- Performance of Satisfaction Drivers - Business

15

4Channel Usage and Preference- Channel Usage- Channel Preference

30

5

Appendices

- Appendix A: Demographic Profile of Respondents- Appendix B: Background to the QPCS- Appendix C: QPCS Qualitative Research Objectives

33

Source: Office of Customer Service Commissioner, Quarterly Pulse Check Survey Q4 2018

1. Executive Summary

1.2 Executive Summary – Q4 2018 QPCS

- Consumer Deep Dive- Business Deep Dive

1.1 Customer Satisfaction Index – Q4 2018 QPCS

1.1 Customer Satisfaction Index – Q4 2018 QPCS

Consumers – CSI

76.6

78.4 78.3 77.8

79.6 79.778.2

81.3

CSMS2015

(n=1,645)

CSMS2016

(n=1,712)

CSMS2017

(n=1,638)

Q42017

(n=343)

Q12018

(n=365)

Q22018

(n=1,536)

CSMS2018

(n=1,494)

Q42018

(n=1,523)

Businesses – CSI

Consumers – Baseline Measures

Businesses – Baseline Measures

Margin of Error (QPCS)Consumer CSI: ± 1.3

Margin of Error (QPCS)Business CSI: ± 1.3

Note - Sample size for CSI and Baseline measure may differ on the basis of “Don’t Know” option selection

Note – Significance testing between CSMS and QPCS is not possible due to differing confidence intervals

76.7

78.7 79.380.2 79.9 79.6

78.979.9

CSMS2015

(n=6,549)

CSMS2016

(n=6,971)

CSMS2017

(n=6,527)

Q42017

(n=1,508)

Q12018

(n=1,610)

Q22018

(n=1,603)

CSMS2018

(n=6,701)

Q42018

(n=1,535)

7.3

7.57.7 7.7 7.7 7.7 7.6

7.87.7

7.87.9

7.87.9 7.9 7.9 8.0

6.9

7.2 7.27.3 7.2 7.3

7.27.3

6.6

7.1

7.6

8.1

8.6

CSMS2015

(n=6,593)

CSMS2016

(n=7,015)

CSMS2017

(n=6,559)

Q42017

(n=1,518)

Q12018

(n=1,628)

Q22018

(n=1,618)

CSMS2018

(n=6,733)

Q42018

(n=1,545)

Satisfaction Expectation Comparison to Ideal

7.2

7.5 7.57.5

7.6

7.8

7.4

7.97.6

7.97.7

7.67.7

8.1

7.7

8.0

7.0

7.37.1

7.07.2

7.37.2

7.4

6.6

7.1

7.6

8.1

8.6

CSMS2015

(n=1,654)

CSMS2016

(n=1,718)

CSMS2017

(n=1,646)

Q42017

(n=345)

Q12018

(n=367)

Q22018

(n=1,554)

CSMS2018

(n=1,506)

Q42018

(n=1,535)

Satisfaction Expectation Comparison to Ideal

Consumer Business

Statistically significant movement from same quarter previous year at 95% confidence level

Statistically significant movement from previous quarter at 95% confidence level

Statistically significant movement from the previous CSMS year at 99% confidence level

1.2 Executive Summary: Overview of Results and Background Q4 2018 background and context

Survey Fieldwork Period: 29 October 2018 to 5 November 2018

Preceding the survey period in Q2 2018 the following issues and initiatives were topical:

Harassment claims: Significant coverage of Opposition Leader Luke Foley’s resignation following allegations of harassment

Sydney light rail: Ongoing media coverage on cost blowouts and delays affecting a number of businesses in the CBD.

Sydney train network: An internal report detailed numerous risks and issues affecting the Sydney train network.

NDIS: reports of complaints received against the National Insurance Disability scheme (NDIS).

Foster care program: The NSW Government announced new laws designed to reduce the amount of time children spend in the foster system without a permanent home

Stadium upgrades: Continued media coverage and commentary around the Sydney stadium redevelopment plans and the upgrades to the Western Sydney Stadium.

Regional infrastructure: The NSW Government announced a plan to invest over $4 billion to improve infrastructure in rural and reginal NSW.

Invictus Games: Invictus Games held in Sydney 20th-27th

October.

Royal Visit: The Duke and Duchess of Sussex toured Sydney and Dubbo as part of their Invictus Games visit.

Lockout laws: Media commentary around this issue following the introduction of a bill to repeal the controversial Sydney lockout laws.

Music festivals: NSW Government released a plan to increase safety at large scale music festivals.

QPCS results at a glance

Consumers

Q1 2017 Q2 2017CSMS 2017

Q4 2017 Q1 2018 Q2 2018CSMS2018

Q4 2018

Satisfaction 7.6 7.6 7.7 7.7 7.7 7.7 7.6 7.8

Expectation 7.9 7.8 7.9 7.8 7.9 7.9 7.9 8.0

Ideal service

7.3 7.2 7.2 7.3 7.2 7.3 7.2 7.3

CSI 79.5 78.6 79.3 80.2 79.9 79.6 78.9 79.9

CSI Score out of 100; all other baseline measures represent scores out of 10

QPCS results at a glance

Business

Q1 2017 Q2 2017CSMS 2017

Q4 2017 Q1 2018 Q2 2018CSMS 2018

Q4 2018

Satisfaction 7.4 7.3 7.5 7.5 7.6 7.8 7.4 7.9

Expectation 8.1 7.5 7.7 7.6 7.7 8.1 7.7 8.0

Ideal service

7.4 7.1 7.1 7.0 7.2 7.3 7.2 7.4

CSI 77.6 77.6 78.3 77.8 79.6 79.7 78.2 81.3

Consumer Business




Note – Significance testing between CSMS and QPCS is not possible due to differing confidence

intervals

1.3 Executive Summary: Consumer Deep Dive – Q4 2018 QPCS

Q4 2018 QPCS Insights – Consumers

• The Customer Satisfaction Index (CSI) has increased marginally by 0.3 points: from 79.6 in Q2 2018 to 79.9 in Q4 2018 (notstatistically significant).

• Expectations remain stable at 8.0 (+0.1) since last quarter• Satisfaction remains stable at 7.8 (+0.1) since last quarter• Comparison to ideal service remains stable at 7.3 (same as last quarter)• Gap to Expectation remains narrow at -0.2 (same as last quarter)• Increases in Expectations are statistically significant from same quarter in previous year (Q4 2017)

• The marginal increase in consumer CSI corresponds with the following factors:

• Customer Effort: Statistically significant reduction in customer effort since last quarter (from 6.0 to 5.7) . Statisticallysignificant reduction from the same quarter in previous year (6.1 in Q4 2017)

• Drivers: Since Q2 2018, we have seen increases in the employee and process areas, namely:• Honesty and integrity of employees (from 7.8 in Q2 to 8.0 in Q4)• Efficiency and effectiveness of employees (from 7.5 in Q2 to 7.7 in Q4)• Simplicity and efficiency of processes (from 7.3 in Q2 to 7.6 in Q4)• Access to information (from 7.5 in Q2 to 7.7 in Q4) – also a statistically significant increase since same quarter in

previous year (Q4 2017)• All other remaining drivers have remained stable

CSI Score out of 100; all other baseline measures represent scores out of 10 Statistically significant movement from same quarter previous year at 95% confidence level


Consumer

1.3 Executive Summary: Business Deep Dive – Q4 2018 QPCSQ4 2018 QPCS Insights – Business

• The CSI has statistically significantly increased by 1.6 points: from 79.7 to 81.3 since Q2 2018:• Expectations remain stable at 8.0 (-0.1) since last quarter• Satisfaction remains stable at 7.9 (+0.1) since last quarter• Comparison to ideal service remains stable at 7.4 (+0.1) since last quarter• Gap to Expectation has decreased from -0.3 to -0.1 since last quarter• All changes are statistically significant from the same quarter in previous year (Q4 2018)

and all 3 outcome measures (expectation, satisfaction and comparison to ideal service)have been improving steadily over the last 12 months

• The overall narrowing of the satisfaction-expectation gap from 0.3 to 0.1 is leading to increases inthe CSI. To explore this further, we interrogated the data from four angles: (1) Expectation Gap forServices, (2) Business Effort and Drivers, (3) Qualitative insights, (4) Other business indexes available

1. Services: Change in expectations gap is largest for high reach services (since Q2 2018).

2. Business Effort and Drivers: In Q2, we saw Business Effort decline significantly due to improvements inprivacy, transparency, access to information, service quality and employee integrity. The decline in business effort identified in Q2 has been maintained again this quarter, driven by statistically significant improvements in clear communication, processes designed to reduce wait times and perceptions of seamless service. All other drivers have remained stable.

3. Qualitative insights from the discussion boards further support quantitative findings:• Greater communication around initiatives and ongoing projects has improved business perception of

the NSW government:• “My perception is better ever since more funds have been allocated to improving road

infrastructure down our way these last couple of years”• Increased perception of NSW Government from streamlining services to increase efficiency :

• "I have noticed a long term slow improvement”• "My perception has slightly improved. I have found it easier and quicker in recent months to

contact the Govt. Offices…and the Service NSW offices make it convenient when I need tosee someone face to face.“

Notes about business confidence indices assessed: About the Roy Morgan Business Confidence Index: The Index is based on 5 different attributes measuring Australian business’ expectations of the economic climate over the next 12 months. (n=~1,000 Australian businesses per month). Data points for the Roy Morgan Business Confidence Index are taken from the corresponding month of QPCS/CSMS fieldwork. http://www.roymorgan.com/morganpoll/consumer-confidence/roy-morgan-business-confidence.

4. Other business indexes available: Q4 QPCS business results appear to be in-line with broader Australian Business sentiment, specifically the Roy Morgan Business ConfidenceIndex.• Roy Morgan Business Index results show an increase in business confidence from September to November from 112, 113 to 113.8; our increases are in line with this

improvement• Note: the latest Sensis Business Index Report is June 2018 so unfortunately insufficient trend data to see if this publication further supports the trends we’re seeing elsewhere

Business Confidence and Business CSI

CSI Score out of 100; all other baseline measures represent scores out of 10 Statistically significant movement from same quarter previous year at 95% confidence level


Business

79.59 79.69

75.92

81.25

114

117124

120

115

119117

114114

110112

113 113.8

100

105

110

115

120

125

130

75

82

No

v-17

De

c-1

7

Jan

-18

Feb

-18

Mar

-18

Ap

r-18

May

-18

Jun

-18

Jul-

18

Au

g-18

Sep

-18

Oct

-18

No

v-18

Bu

siness C

on

fiden

ce Ind

ex

CSI

CSI

Roy Morgan Business Confidence Index*

2. Customer Satisfaction Measures

2.1 Customer Satisfaction Measures – Customer Satisfaction Index (CSI)

2.2 Satisfaction, Expectation and Comparison to Ideal

2.3 Customer Effort Score

2.4 Customer Effort Compared to Other Industries

2.1 Customer Satisfaction Measures – Customer Satisfaction Index (CSI)

Consumer – Customer Satisfaction Index Business – Customer Satisfaction Index

Cu

sto

mer

Sat

isfa

ctio

n In

dex

(o

ut

of

100)

Cu

sto

mer

Sat

isfa

ctio

n In

dex

(o

ut

of

100)

The CSI as measured by the QPCS in Q4 2018 is 79.9 for consumers. The CSI as measured by the QPCS in Q2 2018 is 81.3 for businesses.

• The Customer Satisfaction Index (CSI) has increased marginally among consumers by 0.28 points from Q2 2018 to 79.9/100 in Q4 2018 (not statisticallysignificant). The CSI for businesses has increased by 1.57 points from Q2 2018 to 81.3/100 in Q4 2018 (statistically significant).

• Compared to this time last year (Q4 2017) the CSI has decreased marginally by 0.31 points among consumers to 79.9/100 (not statistically significant) andincreased by 3.48 points among businesses to 81.3/100 (statistically significant).

Margin of Error (QPCS)Consumer CSI: ± 1.3

Margin of Error (QPCS)Business CSI: ± 1.3

Consumer Business

76.7

78.7 79.380.2 79.9 79.6

78.979.9

CSMS2015

(n=6,549)

CSMS2016

(n=6,971)

CSMS2017

(n=6,527)

Q42017

(n=1,508)

Q12018

(n=1,610)

Q22018

(n=1,603)

CSMS2018

(n=6,701)

Q42018

(n=1,535)

76.6

78.4 78.3 77.8

79.6 79.778.2

81.3

CSMS2015

(n=1,645)

CSMS2016

(n=1,712)

CSMS2017

(n=1,638)

Q42017

(n=343)

Q12018

(n=365)

Q22018

(n=1,536)

CSMS2018

(n=1,494)

Q42018

(n=1,523)




Figures in the graph are subject to rounding as some may not add up to 100%Note – Significance testing between CSMS and QPCS is not possible due to differing confidence intervals

2.2 Satisfaction, Expectation and Comparison to Ideal - Consumer

Consumer – Outcome Measures

Expectation

Average (out of 10)

Among consumers all three outcome measures have remained relatively stable since Q2 2018, with no statistically significant changes. When compared to Q4 2017, expectations have increased (statistically significant) by 0.23 points to 8.0/10.

Satisfaction Comparison to Ideal

% D

istr

ibut

ion

(acr

oss

low

, n

eutr

al a

nd

hig

h sc

ore

s)

7% 6% 7% 6% 6% 8% 8% 9% 8% 8% 11% 12% 12% 11% 13%14% 13% 12% 13% 11%

13% 14% 13% 15% 12%17% 19% 17% 20% 16%

79% 81% 81% 81% 84% 78% 78% 78% 77% 80%72% 70% 71% 69% 72%

0%

20%

40%

60%

80%

100%

Q4 2017(n=1,537)

Q1 2018(n=1,644)

Q2 2018(n=1,642)

CSMS 2018(n=6,922)

Q4 2018(n=1,579)

Q4 2017(n=1,541)

Q1 2018(n=1,651)

Q2 2018(n=1,638)

CSMS 2018(n=7,000)

Q4 2018(n=1,583)

Q4 2017(n=1,518)

Q1 2018(n=1,628)

Q2 2018(n=1,618)

CSMS 2018(n=6,733)

Q4 2018(n=1,545)

Low (1-4) Med (5-6) High (7-10)

Figures in the graph are subject to rounding as some may not add up to 100%

Consumer

7.8 7.9 7.9 7.9 8.0 7.7 7.7 7.7 7.6 7.8 7.3 7.2 7.3 7.2 7.3





CSMS results

2.2 Satisfaction, Expectation and Comparison to Ideal - Business

Business – Outcome Measures

Among businesses, expectations have decreased marginally by 0.07 points to 8.0/10 since Q2 2018 (statistically insignificant). All three outcomes have increased (statistically significant) since Q4 2017.

% D

istr

ibut

ion

(acr

oss

low

, n

eutr

al a

nd

hig

h sc

ore

s)

Average (out of 10)

Expectation Satisfaction Comparison to Ideal

6% 6% 6% 6% 5% 8% 5% 9% 7% 5% 10% 7% 12% 11% 9%

17% 16% 9% 15% 14%19% 22% 11% 18% 16%

24%20% 15% 19% 18%

77% 78% 84% 79% 82%73% 73%

80% 75% 79%66%

73% 73% 71% 73%

0%

20%

40%

60%

80%

100%

Q4 2017(n=350)

Q1 2018(n=372)

Q2 2018(n=1,555)

CSMS 2018(n=1,531)

Q4 2018(n=1,552)

Q4 2017(n=349)

Q1 2018(n=375)

Q2 2018(n=1,570)

CSMS 2018(n=1,555)

Q4 2018(n=1,553)

Q4 2017(n=345)

Q1 2018(n=367)

Q2 2018(n=1,554)

CSMS 2018(n=1,506)

Q4 2018(n=1,535)

Low (1-4) Med (5-6) High (7-10)

Business

7.6 7.7 8.1 7.7 8.0 7.5 7.6 7.8 7.4 7.9 7.0 7.2 7.3 7.2 7.4





CSMS results

2.3 Customer Effort Score

Customer Effort Score* – Service InteractionQ: Thinking about your direct dealing with [SERVICE], how much effort did you personally have to put forth? 1 is ‘Low effort’ and 10 is ‘High effort’

Service interaction Customer Effort Score.

Overall, businesses have to put forth higher effort than consumers when interacting with NSW government services.

The Customer Effort Score (CES) has seen a statistically significant decrease among consumers since Q2 2018 to 5.7/10. Both consumers and businesses have seen a statistically significant decrease since Q4 2017 in their CES scores to bring them down to 5.7/10 and 6.1/10 respectively.

5.95.7

5.96.1

5.9 6.0

6.0

5.7

6.9

6.5

4.9

6.76.9

6.0

6.6

6.1

Consumers BusinessesLowerEffort

HigherEffort

Q1 2017Consumer (n=1,529)

Business (n=382)


Business (n=366)


Business (n=345)


Business (n=373)

*The lower the average Customer Effort Score, the easier customers perceive interactions to be with different services. A decrease in CES is a positive shift.

Q2 2018Consumer (n=1,585)Business (n=1,535)

Q4 2018Consumer (n=1,526)Business (n=1,508)

Consumer Business

CSMS 2017Consumer (n=6,501)Business (n=1,650)

CSMS 2018Consumer (n=6,634)Business (n=1,496)



Statistically significant movement from the previous CSMS year at 99% confidence levelFigures in the graph are subject to rounding as some may not add up to 100%Note – Significance testing between CSMS and QPCS is not possible due to differing confidence intervals

CSMS results

2.4 Customer Effort Compared to Other Industries – Consumer

Customer Effort Score* – WHoG BenchmarkQ: Thinking about all your direct dealings with each of the following Australian industries and government services over the previous 6 months, how much effort did you personally have to put forth?

Customer Effort Score benchmarked at a Whole of Government level against other industries.

5.6 5.55.8 5.6 5.6

5.9 6.15.7 5.6 5.8 5.7 5.7

6.16.4

5.7 5.8 6.0 6.0 6.2 6.1 6.4

5.55.7 5.7 5.9 6.1 6.1

6.4

5.45.7

5.9 5.86.0 6.0

6.2

5.5 5.65.8 5.6

6.0 6.0 6.1

Airlines Banks Federal Government My local council Energy retailers NSW GovernmentServices

Telephone serviceproviders

Q1 2017 QPCS (n=722 to 954) Q2 2017 QPCS (n=669 to 955) Q4 2017 QPCS (n=763 to 948)


When benchmarked against other industries, NSW Government services (at a Whole of Government level) compare favorably to telephone service providers and has scope for improvement when compared to other industries in terms of effort required for interactions among consumers in Q4 2018. Customer effort has remained relatively stable across all industries in Q4 2018 since Q2 2018. When compared to Q4 2017, local council and telephone service providers have seen a statistically significant decrease in effort.

Note: Figures in the graph are subject to rounding*The lower the average Customer Effort Score, the easier customers perceive interactions tobe with different services. A decrease in CES is a positive shift.

Data labels in green/red indicate statistically significant movement at 95% confidence level compared to previous quarters results


Consumer

2.4 Customer Effort Compared to Other Industries - Business

Customer Effort Score* – WHoG BenchmarkQ: Thinking about all your direct dealings with each of the following Australian industries and government services over the previous 6 months, how much effort did you personally have to put forth?

Customer Effort Score benchmarked at a Whole of Government level against other industries.

6.0 6.2 6.3 6.2 6.26.6

6.46.16.3 6.5 6.3 6.2

6.7 6.7

5.76.3 6.2 6.2

6.66.3

6.86.3 6.2

6.06.2 6.4 6.4 6.7

5.6 5.8 5.8 6.0 6.0 6.1 6.45.8 5.8 5.9 5.8

6.2 6.26.4

Airlines Banks Federal Government My local council Energy retailers NSW GovernmentServices

Telephone serviceproviders



Note: Figures in the graph are subject to rounding*The lower the average Customer Effort Score, the easier customers perceive interactions tobe with different services. A decrease in CES is a positive shift.



When benchmarked against other industries, NSW Government services (at a Whole of Government level) compare favorably to telephone service providers and energy retailers and have scope for improvement when compared to other industries in terms of effort required for interactions among businesses in Q4 2018. Customer effort has remained relatively stable across all industries in Q4 2018 since Q2 2018. When compared to the same time last year (Q4 2017), the effort has seen a statistically significant decrease in banks, energy retailers and telephone service providers.

Business

3. Insights on Key Primary Opportunity Areas and Satisfaction Drivers

3.1 Key Primary Opportunity Areas (KPOAs) in 2018 - Consumers

3.2 Performance of Key Primary Opportunity Areas - Consumers

3.3 Drivers of Satisfaction and Suggested Areas of Improvement with Employee Efficiency and Effectiveness

3.4 Drivers of Satisfaction and Suggested Areas of Improvement with Employee Empathy and Communication

3.5 Drivers of Satisfaction and Suggested Areas of Improvement with Employee Accountability

3.6 Drivers of Satisfaction and Suggested Areas of Improvement with Access to Information

3.8 Key Primary Opportunity Areas (KPOAs) in 2018 - Businesses

3.9 Performance of Key Primary Opportunity Areas - Businesses

3.7 Performance of Satisfaction Drivers - Consumers

3.10 Performance of Satisfaction Drivers - Business

3.1 Key Primary Opportunity Areas (KPOAs) - Consumers

The QPCS provides a deep dive into the specific drivers of satisfaction and associated attributes as identified in the 2018 Annual CSMS. These are the key areas which will have the greatest impact on overall customer satisfaction of NSW Government services as they are of high importance yet are perceived to have lower performance.

The deep dive KPOAs for the Q4 2018 QPCS consumers are:

• Efficiency and effectiveness of employees

• Empathy and communication of employees

• Accountability

• Access to information

Consumer

LOW

HIG

H

Relative performance

LOW

HIGH

Imp

ort

ance

in d

rivi

ng

sati

sfac

tio

n

Efficiency and effectiveness of employees

Honesty and integrity of employees

Empathy and communication of employees

Employee autonomy

Simplicity and efficiency of processes

Transparency

Access toinformation

Privacy

Strengths to ‘build on’Primary opportunities

Secondary opportunities Lower importance - monitor

Legend:Employee attributes

Goals Processes

Accountability

Key Primary Opportunity Areas 2018-19

3.2 Performance of Key Primary Opportunity Areas – Consumers (1/4)

Q: Thinking now about its ‘employees’, to what extent do you agree or disagree that each of the following statements describes [SERVICE] in NSW? 1 is ‘Strongly disagree’ and 10 is ‘Strongly agree’.

Among consumers in Q4 2018, average scores for attributes relating to employee efficiency and effectiveness have increased (not statistically significant) since Q2 2018. Results are statistically stable compared with this time last year i.e. Q4 2017.

Efficiency and Effectiveness of Employees

7.2

7.6

7.3

7.0

7.5 7.47.3

7.1

7.57.5

7.8

7.6

7.4

7.67.6

7.5 7.4

7.6

6.5

7.0

7.5

8.0

Q4 2016 QPCS(n=1,484 to 1,534)

Q1 2017 QPCS(n=1,426 to 1,472)

Q2 2017 QPCS(n=1,488 to 1,520)

CSMS 2017 (n=6,058 to 6,268)

Q4 2017 QPCS(n=1,414 to 1,455)

Q1 2018 QPCS(n=1,512 to 1,556)

Q2 2018 QPCS(n=1,497 to 1,536)

CSMS 2018(n=6,266 to 6,430)

Q4 2018 QPCS(n=1,409 to 1,474)

Get things done quickly Deliver against actions promised

Consumer





CSMS results

7.6

7.9

7.7

7.4

7.7 7.87.7

7.4

7.8

7.5

7.7

7.57.3

7.6 7.67.5 7.3

7.6

7.5

7.8

7.5

7.4

7.6 7.6 7.67.4

7.7

7.4

7.7

7.4

7.3

7.6 7.67.4 7.4

7.6

7.1

7.4

7.2

7.0

7.4 7.47.3

7.0

7.4

6.5

7.0

7.5

8.0

Q4 2016 QPCS(n=1,460 to 1,558)

Q1 2017 QPCS(n=1,394 to 1,498)

Q2 2017 QPCS(n=1,467 to 1,554)

CSMS 2017(n=5,901 to 6,358)

Q4 2017 QPCS(n=1,397 to 1,481)

Q1 2018 QPCS(n=1,505 to 1,587)

Q2 2018 QPCS(n=1,504 to 1,556)

CSMS 2018(n=6,162 to 6,564)

Q4 2018 QPCS(n=1,396 to 1,495)

Communications were clear, prompt and easy to understand Focus on addressing customer needs

Provide good value service Are proactive in helping

See things from my perspective

Q: Thinking now about its ‘employees’, to what extent do you agree or disagree that each of the following statements describes [SERVICE] in NSW? 1 is ‘Strongly disagree’ and 10 is ‘Strongly agree’.


Average scores for attributes relating to empathy and communication of employees have increased since Q2 2018, with the score for proactive in helping increasing statistically significantly. Results are statistically stable compared with this time last year i.e. Q4 2017.

Empathy and Communication of employees

Consumer





CSMS results

Q: Thinking now about its ‘information surrounding your interaction’, to what extent do you agree or disagree that each of the following statements describes [SERVICE] in NSW? 1 is ‘Strongly disagree’ and 10 is ‘Strongly agree’.


Average scores for attributes relating to accountability have increased (statistically significant) since Q2 2018. Result is statistically stable compared with this time last year i.e. Q4 2017.

Accountability

Consumer

7.2

7.5

7.3

7.0

7.57.4

7.27.0

7.4

6.5

7.0

7.5

8.0

Q4 2016 QPCS (n=1,357)

Q1 2017 QPCS(n=1,298)

Q2 2017 QPCS(n=1,345)

CSMS 2017(n=5,497)

Q4 2017 QPCS(n=1,325)

Q1 2018 QPCS(n=1,437)

Q2 2018 QPCS(n=1,376)

CSMS 2018(n=5,690)

Q4 2018 QPCS(n=1,241)

Are held accountable



Statistically significant movement from the previous CSMS year at 99% confidence levelFigures in the graph are subject to rounding as some may not add up to 100%Note – Significance testing between CSMS and QPCS is not possible due to differing confidence intervals

CSMS results

7.5

7.77.6

7.1

7.6 7.6

7.5

7.1

7.6

7.3

7.7

7.5

7.0

7.4

7.5 7.4

7.0

7.6

6.5

7.0

7.5

8.0

Q4 2016 QPCS(n=1,427 to 1,499)

Q1 2017 QPCS(n=1,358 to 1,425)

Q2 2017 QPCS(n=1,401 to 1,476)

CSMS 2017(n=5,688 to 6,112)

Q4 2017 QPCS(n=1,376 to 1,440)

Q1 2018 QPCS(n=1,466 to 1,540)

Q2 2018 QPCS(n=1,422 to 1,503)

CSMS 2018(n=5,883 to 6,287)

Q4 2018 QPCS(n=1,365 to 1,455)

Is making it easier to access information Is making best use of online services

Q: Thinking now about its ‘information surrounding your interaction’, to what extent do you agree or disagree that each of the following statements describes [SERVICE] in NSW? 1 is ‘Strongly disagree’ and 10 is ‘Strongly agree’.


Average scores for attributes relating to access to information have increased (statistically insignificant) since Q2 2018. Average score for best use of online services has increased statistically significantly since this time last year i.e. Q4 2017.

Access to Information

Consumer





CSMS results

3.3 Drivers of Satisfaction and Suggested Areas of Improvement with Employee Efficiency and Effectiveness - Consumers

Q: Thinking now about its employees, to what extent do you agree or disagree with the statement “Employees acted efficiently and effectively

to reach the right outcomes”?

They acted professionally, quickly and told us what was happening along the way

They asked the right questions and acted with precision

• Reliable, well trained and empathetic staff• Staff were knowledgeable and resolved the

issue quickly• Staff listened carefully and addressed my

problem with a clear explanation

Reasons for high satisfaction with employee efficiency and effectiveness (7-10/10)

Took too long to respond and did not offer more help

The local RMS office had restricted access and the whole episode had to be explored by mail

• Too much “passing the buck”• Employees lack empathy and work at their

own pace• Dearth of knowledgeable employees to

resolve issues

Reasons for low satisfaction with employee efficiency and effectiveness (1-6/10)

Train employees to communicate well – both in listening to queries and explaining intended

actions clearly

More prompt communications and better processes to reach the final intended outcome

• Hire more staff to reduce workload andimprove their efficiency

• Regularly train the staff about the processupdates

• Reduce paperwork to do away withrepetitive tasks

Customer ideas to improve employee efficiency and effectiveness

24%

76%

Low satisfaction (1-6/10)

High satisfaction (7-10/10)

“ “

”

“

”

n=1,438

“ ” ”““

””

Consumer

3.4 Drivers of Satisfaction and Suggested Areas of Improvement with Employee Empathy and Communication- Consumers

Q: Thinking now about its employees, to what extent do you agree or disagree with the statement “Communications were clear, prompt and

easy to understand”?

It was extremely quick, yet everything was explained properly

Everything I needed to know was explained very well in a quick and friendly manner

• Patient, courteous and understanding staff• Staff listened carefully and consumers

were kept fully informed of the process• Staff showed compassion and were very

polite and accommodating

Reasons for high satisfaction with employee empathy and communication (7-10/10)

We were left homeless for two years, even though we had all the paperwork for emergency accommodation

I was transferred constantly and had to repeat my details and situation every time.

• Staff don’t seem interested inunderstanding the needs

• Staff were rude and unapproachable.• Staff provide incorrect information

Reasons for low satisfaction with employee empathy and communication (1-6/10)

Give employees the power to make decisions to reduce wait times and redundancy

Employees need to be more proactive in responding to queries online

• Staff must inform the consumers about theexact time frames

• Ensure that staff adhere to a pre-decidedcode of conduct

• Guide consumers to the right person sothat they get it right the first time

Customer ideas to improve employee empathy and communication

21%

79%



“ “

”

“

”

n=1,495

“ ” ”““

”

”

Consumer

24%

76%



Q: Thinking now about its employees, to what extent do you agree or disagree with the statement “Employees acted with empathy”?

n=1,387

3.5 Drivers of Satisfaction and Suggested Areas of Improvement with Employee Accountability - Consumers

Q: Thinking now about its employees, to what extent do you agree or disagree with the statement “Employees were held accountable for

their actions”?

Staff were proactive, prompt and had a positive demeanor in dealing with my case

They knew exactly what needed to be done and how to do it

• Helpful, professional and efficient staff• Staff were well trained and had thorough

understanding of the processes involved• Staff took responsibility from start to end

and things were done right the first time

Reasons for high satisfaction with employee accountability (7-10/10)

Employees don’t seem to be held accountable for their actions. Felt there was no implication for them after

making the decision

In most cases there is a very casual approach to work even though there is a huge backlog of people to be

processed.

• Staff were reluctant to make decisions• There was a lot of hierarchy with no senior

independent staff to talk to• Staff are not concerned about long wait

times

Reasons for low satisfaction with employee accountability (1-6/10)

All interactions must be noted with a unique reference number

Speak to the same person every time rather than a different person each time

• Constant workplace training andmonitoring

• Empower frontline staff to make decisionson the spot and increase responsibility

• Increased supervision with greateraccountability

Customer ideas to improve employee accountability

29%

71%



“ “

”

“

”

n=1,146

“ ” ”“ “

”

”

Consumer

3.6 Drivers of Satisfaction and Suggested Areas of Improvement with Access to Information - Consumers

The organisation and layout of information are well displayed and documented

Forms were online, so filled before appointment. Just needed checking and witnessing of signatures.

So quick and easy

• Information that is clear and easy to access

• Clear and sufficient website information

• Easily navigable websites

Reasons for high satisfaction with access to information (7-10/10)

It's kind of hard to navigate the Housing New South Wales website. I always need help over the phone

finding the right document to download

I had to approach several employees to get information

• Some websites are confusing and hard tonavigate for customers.

• Online information is sometimes not up todate.

Reasons for low satisfaction with access to information (1-6/10)

Less options on each page and a 'process overview' at the start

Make the websites easier to navigate, a better menu system in which I can find what I need

• Ensure websites are up-to-date and easy-to-understand

• Easy-to-navigate websites & searchfunctionalities

• Mobile apps to access information inaddition to websites

Ideas to improve access to information

25%

75%



“

”

“”

“

”

n=1,455

“”

“”

Q: Thinking now about information surrounding your interaction, to what extent do you agree or disagree with the statement “Is making it

easier to access information about their services”?

”“

Consumer

3.7 Performance of Satisfaction Drivers - Consumers

7.87.6 7.7 7.6 7.7

7.3

7.9 7.77.4 7.5 7.4

8.07.6 7.8 7.6 7.4 7.4

8.27.9

7.6 7.7 7.57.87.5 7.7 7.4 7.3 7.3

8.07.8 7.5

7.77.4

8.07.7 7.8 7.6

7.3 7.6 7.4

8.1 7.9 7.7 7.87.5

Employees wereopen and honest

during the process

Employees actedefficiently andeffectively to

reach the rightoutcomes

Communicationswere clear,

prompt and easyto understand

Employees actedwith empathy

Employees wereheld accountablefor their actions

The process wassimple and

efficient

Employees tookinitiative and

made decisions

My privacy wasupheld & personalinformation was

protected &respected

I understood thesteps involved

with the process

I had good accessto information andcould find what I

needed

I was providedwith good serviceand outcomes I

could trust

I felt there wasaccountability forservices delivered

Q4 2017 QPCS (n=1,385 to 1,541) Q1 2018 QPCS (n=1,444 to 1,601) Q2 2018 QPCS (n=1,431 to 1,588) Q4 2018 QPCS (n=1,146 to 1,521)

Statement Asked in QPCS:

Drivers of Satisfaction

Q: We would now like to ask you some questions specifically about your experience with [SERVICE]. Thinking now about [its employees; its processes; service overall; the values they uphold], to what extent do you agree or disagree that each of the following statements describes [SERVICE] in NSW? 1 is ‘Strongly disagree’ and 10 is ‘Strongly agree’.

Ave

rage

(ou

t of 1

0)

Derived Drivers (Annual CSMS): Privacy Transparency

Access to information

Honesty and Integrity of Employees

Efficiency and Effectiveness of Employees

Simplicity and Efficiency of

ProcessesEmpathy and Communication

Employee Autonomy

Service Quality

Accountability

Employees Process Goals Values

n/a

fo

r p

revi

ou

s q

uar

ters

–n

ew d

rive

r ca

tego

ry

iden

tifi

ed in

in 2

01

8 C

SMS

In Q4 2018, all drivers have remained stable, with the exception of honesty and integrity of employees, efficiency and effectiveness of employees, simplicity and efficiency of processes, and access to information, which have statistically significantly increased since Q2 2018. When compared to the same time last year (Q4 2017), all drivers have remained stable, with the exception of transparency, access to information and service quality drivers, which have experienced a statistically significant increase.



Consumer

Accountability

4.8 Key Primary Opportunity Areas (KPOAs) - Businesses

The QPCS provides a deep dive into the specific drivers of satisfaction and associated attributes as identified in the 2018 Annual CSMS. These are the key areas which will have the greatest impact on overall customer satisfaction of NSW Government services as they are of high importance yet are perceived to have lower performance.

The deep dive KPOAs for the Q4 2018 QPCS business are:

• Simplicity and efficiency of processes

• Access

Business drivers of satisfaction have been identified for the first time in 2018 for a separate piece of work and have been included in the QPCS as supplementary information. Traditionally only consumer drivers are explored in the QPCS and therefore, attributes have been mapped to business drivers where possible, however no verbatim analysis has been undertaken. The decision of inclusion of business drivers in CSMS and QPCS is to be discussed in detail for Q1 2019 and beyond.

LOW

HIG

H

Relative performance

LOW

HIGH

Imp

ort

ance

in d

rivi

ng

sati

sfac

tio

n

Are consistent

Integrity and high standards

Clear communication

Employee autonomy

Simplicity and efficiency of processes

Transparency

Access

Privacy

Strengths to ‘build on’Primary opportunities

Secondary opportunities Lower importance - monitor

Key Primary Opportunity Areas 2018-19

Legend:Employee attributes

Goals Processes

Customer focus and action oriented

Business

Q: Thinking now about its ‘information surrounding your interaction’, to what extent do you agree or disagree that each of the following statements describes [SERVICE] in NSW? 1 is ‘Strongly disagree’ and 10 is ‘Strongly agree’. (same question also asked for ‘processes’)

3.9 Performance of Key Primary Opportunity Areas – Businesses (1/2)

Among businesses in Q4 2018, average scores for attributes relating to simplicity and efficiency of processes have increased since Q2 2018, with significant increase for processes are designed to reduce wait time and service feels seamless. Results are statistically stable compared with this time last year i.e. Q4 2017.

Simplicity and Efficiency of Processes

6.8

7.2

6.7

6.3

7.1

7.0

6.9

6.7

7.2

7.0

7.6

7.2

6.9

7.4

7.57.4

7.0

7.5

6.5

7.16.9

6.3

7.17.0

6.96.7

7.1

6.7

7.3

6.9

6.4

7.0

7.1 7.0

6.7

7.2

6.0

6.5

7.0

7.5

8.0

Q4 2016 QPCS(n=323 to 323)

Q1 2017 QPCS(n=370 to 382)

Q2 2017 QPCS(n=347 to 364)

CSMS 2017(n=1,560 to 1,622)

Q4 2017 QPCS(n=326 to 341)

Q1 2018 QPCS(n=353 to 366)

Q2 2018 QPCS(n=1,391 to 1,503)

CSMS 2018(n=1,455 to 1,494)

Q4 2018 QPCS(n=1,378 to 1,470)

Processes are designed to reduce wait times Processes are easy to understand

I can get to the right person the first time Service feels seamless

*Variations in figures may appear due to rounding

Business




CSMS results


7.4

7.6

7.3

6.8

7.6

7.8

7.5

7.0

7.6

7.3

7.6

7.0

6.7

7.4

7.7

7.4

6.8

7.5

6.5

7.0

7.5

8.0

Q4 2016 QPCS(n=342 to 361)

Q1 2017 QPCS(n=363 to 369)

Q2 2017 QPCS(n=334 to 350)

CSMS 2017(n=1,536 to 1,594)

Q4 2017 QPCS(n=323 to 331)

Q1 2018 QPCS(n=347 to 362)

Q2 2018 QPCS(n=1,386 to 1,456)

CSMS 2018(n=1,435 to 1,459)

Q4 2018 QPCS(n=1,375 to 1,439)

Easier to access information Best use of online services

Q: Thinking now about its ‘information surrounding your interaction’, to what extent do you agree or disagree that each of the following statements describes [SERVICE] in NSW? 1 is ‘Strongly disagree’ and 10 is ‘Strongly agree’. (same question also asked for ‘processes’)

3.9 Performance of Key Primary Opportunity Areas – Businesses (2/2)

Average scores for attributes relating to access have increased (statistically insignificant) since Q2 2018. Results are also statistically stable compared with this time last year i.e. Q4 2017.

Access

*Variations in figures may appear due to rounding

Business




CSMS results


7.9 7.7 7.6 7.77.4 7.4

8.0 7.6 7.5 7.5 7.47.97.6

7.87.5

7.17.4

8.17.8 7.5 7.7 7.4

8.07.5

7.77.6 7.4 7.4

8.2 7.9 7.77.7 7.47.9

7.77.3

7.87.5 7.5 7.5

8.1 8.07.7 7.8 7.4

Employees wereopen and honest

during theprocess

Employees actedefficiently andeffectively to

reach the rightoutcomes

Employees wereheld accountablefor their actions

Communicationswere clear,

prompt and easyto understand

Employees actedwith empathy

The process wassimple and

efficient

Employees tookinitiative and

made decisions

My privacy wasupheld & personalinformation was

protected &respected

I understood thesteps involved

with the process

I had good accessto informationand could findwhat I needed

I was providedwith good serviceand outcomes I

could trust

I felt there wasaccountability forservices delivered

Q4 2017 QPCS (n=322 to 347) Q1 2018 QPCS (n=356 to 371) Q2 2018 QPCS (n=1,358 to 1,524) Q4 2018 QPCS (n=1,327 to 1,517)

3.10 Performance of Satisfaction Drivers - Business

Derived Drivers (Annual CSMS):

Statement Asked in QPCS:

Drivers of Satisfaction

Q: We would now like to ask you some questions specifically about your experience with [SERVICE]. Thinking now about [its employees; its processes; service overall; the values they uphold], to what extent do you agree or disagree that each of the following Statements describes [SERVICE] in NSW? 1 is ‘Strongly disagree’ and 10 is ‘Strongly agree’.

Ave

rage

(ou

t of 1

0)

In Q4 2018, all drivers for businesses have remained stable, with the exception of the clear communication, which has experienced a statistically significant increase since Q2 2018.

Business

n/a

fo

r p

revi

ou

s q

uar

ters

–n

ew

dri

ver

cate

gory

id

en

tifi

ed in

in 2

018

CSM

S

Privacy TransparencyAccess to

informationIntegrity and high standards

Simplicity and Efficiency of

Processes

Clear communica

tion

Employee Autonomy

Service Quality

Accountability

Employees Process Goals Values

Cust. focus & action

oriented



4. Channel Usage and Preference

4.1 Channel Usage

4.2 Channel Preference

For consumers, there has been little change year-on-year in channel usage with results for Q4 2018 being similar to Q2 2018. Face to face remains the most used channel among this group and the percentage of respondents using face to face channel has increased to the highest level in last one year since Q4 2017.

For businesses, face to face interactions continue to increase since Q4 2017, and is the most common channel, followed by telephone, online and email.

4.1 Channel Usage

Contact Methods Used*

Q: Which of the following contact methods have you used to carry out your direct dealings with [SERVICE] in NSW in the last 6 months? (Multi-select)

Consumer Business

Note: Contact methods used is a multi response question; Figures in the graph does not include the response for “None of the above” option*Channel usage has been rebased on total number of responses for comparison to channel preference

Shar

e o

f co

nta

ct m

eth

od

use

d (%

)

Shar

e o

f co

nta

ct m

eth

od

use

d (%

)

Business n=1,004Consumer n=1,000

56

%

32

%

29

%

20

%

11

%

7%

59

%

36

%

31

%

24

%

12

%

6%

57%

36

%

31%

20

%

12%

7%

58

%

29

% 34

%

15

%

8%

4%

60

%

31

%

29

%

20

%

12

%

5%

0%

10%

20%

30%

40%

50%

60%

70%

In person,face to face

Telephone Online Email Mail,posted

letter, fax

Thirdparties suchas Australia

Post

Q4 2017 Q1 2018 Q2 2018 CSMS 2018 Q4 2018

37

% 40

%

26

% 29

%

15

%

9%

50

%

31

%

29

%

26

%

14

%

6%

57

%

32

%

33%

23

%

14

%

5%

45

%

40

%

31

%

30%

11

%

7%

58

%

29

%

27

%

18

%

9%

5%

0%

10%

20%

30%

40%

50%

60%

70%



letter, fax

Third partiessuch as

AustraliaPost

Q4 2017 Q1 2018 Q2 2018 CSMS 2018 Q4 2018

Consumer Business

Note – Significance testing between CSMS and QPCS is not possible due to differing confidence

intervals

4.2 Channel Preference

Contact Methods Preference*

Q: Generally, which contact method do you most prefer to use when dealing directly with [SERVICE]? (Select one only for each service)

Consumer Business

There remains a clear and increasing preference for in person interactions when consumers deal with NSW Government services. Businesses also prefer in person contact methods. In Q2 2018, business’ preference for in person channel has increased by 4 percentage points, while the preference for email, telephone and online channels have decreased1.

Shar

e o

f co

nta

ct m

eth

od

pre

ferr

ed (%

)

Shar

e o

f co

nta

ct m

eth

od p

refe

rred

(%)

Business n=1,515Consumer n=1,525

38

%

18

%

19

%

15

%

3% 3%

39

%

19

% 22%

15%

2% 1%

40

%

17

%

22

%

13

%

3%

1%

33

%

18

%

25

%

16

%

4%

1%

41

%

18

% 20

%

14

%

2%

1%

0%

10%

20%

30%

40%

50%



letter, fax

Thirdparties suchas Australia

Post

Q4 2017 Q1 2018 Q2 2018 CSMS 2018 Q4 2018

33

%

24

%

19

%

11

%

7%

2%

37

%

17

%

25

%

16

%

2%

1%

37

%

17

% 22

%

18

%

3%

1%

27

%

24

%

21

%

22

%

4%

2%

41

%

16

%

21

%

14

%

3%

1%

0%

10%

20%

30%

40%

50%



letter, fax

Third partiessuch as

AustraliaPost

Q4 2017 Q1 2018 Q2 2018 CSMS 2018 Q4 2018

*Note: Contact methods preferred is a single response question; Figures in the graph does not includethe response for “Don’t know/ can’t say” and therefore, may not add up to 100%1Indicative only - does not indicate statistical significance.

Consumer Business

Appendix

Appendix A – Demographic Profile of Respondents

Appendix B – Background to the QPCS

Appendix C – Qualitative Research and Online Discussion Forum


Consumer Respondent Profile*

49% male

51% femaleGender: Age: Region:

18-3431% 35-54

33%

55-6415%

65+20%

Q4 2018 Consumers (n=1,000 respondents)

75% Metro

19% Regional

6% Rural

27% 25%

12% 10% 8%5% 5% 4% 3%

0%

10%

20%

30%

Employment Status:

17% 17%

28%

14%

5% 4%

15%

0%

10%

20%

30%

Up to $30,000 $30,001 to$50,000

$50,001 to$100,000

$100,001 to$150,000

$150,001 to$180,000

Over $180,001 Prefer not tosay/Don't

know

Annual Income:

49% male

51% femaleGender: Age:

18-3431% 35-54

34%

55-6415%

65+20%

Region: 75% Metro

19% Regional

6% Rural

Employed full time

Retired Full-time domestic

duties

Employed part time

Self-employed/ business owner

Unemployed Student Employed on a casual

basis

Other

25% 23%

11% 11%8% 8%

5% 5% 4%

0%

10%

20%

30%

Employment Status:

Employed full time

Retired Full-time domestic

duties

Employed part time

Unemployed Student Employedon a

casual basis

Self-employed /

business owner

Other

19%16%

26%

14%

5% 6%

15%

0%

10%

20%

30%

Up to $30,000 $30,001 to$50,000

$50,001 to$100,000

$100,001 to$150,000

$150,001 to$180,000

Over$180,001

Prefer not tosay/Don't

know

Annual Income:

• Data is weighted to be representative of the NSW population (ABS) based on gender, age and region• Due to rounding, figures may not sum to 100

Q2 2018 Consumers (n=1,030 respondents)

Consumer

9% 11% 12%

7%6%

7%4% 4% 3% 3% 2% 4% 4% 3% 3%

1% 1% 1%

14%

-2%

2%

6%

10%

14%

18%

Ret

ail T

rad

e

Hea

lth

car

e an

dso

cial

ass

ista

nce

Pro

fess

ion

al,

scie

nti

fic a

nd

tech

nic

al s

ervi

ces

Educ

atio

n an

dtr

ain

ing

Co

nst

ruct

ion

Art

s an

dre

cre

atio

nse

rvic

es

Ad

min

istr

ativ

ean

d s

up

port

serv

ices

Acc

omm

od

atio

nan

d f

oo

d se

rvic

es

Ren

tal,

hiri

ng a

nd

real

est

ate

serv

ices

Agr

icul

ture

,fo

rest

ry a

ndfi

shin

g

Info

rmat

ion

me

dia

and

tele

com

mu

nica

tio

ns

Fina

nci

al a

ndin

sura

nce

serv

ices

Tran

spo

rt, p

osta

lan

d w

areh

ousi

ng

Who

lesa

le T

rade

Man

ufa

ctu

rin

g

Ele

ctri

city

, gas

,w

ater

an

d w

aste

serv

ices

Pub

licad

min

istr

atio

nan

d s

afet

y

Min

ing

Oth

er


Business Respondent Profile*

39% male

61% female

Gender: Region:

Q4 2018 Business (n=1,004 respondents)

75% Metro

19% Regional

6% Rural

45% male

Gender: Region: 74% Metro

19% Regional

7% Rural

55% female

Business size:

Business size:

Industry:

14%

7% 7% 6% 5% 5% 5% 5% 3% 3% 3% 2% 2% 1% 1% 1% 0.3% 0.1%

32%

-2%

2%

6%

10%

14%

18%

Ret

ail T

rad

e

Hea

lth

car

e an

dso

cial

ass

ista

nce

Pro

fess

ion

al,

scie

nti

fic a

nd

tech

nic

alse

rvic

es

Educ

atio

n an

dtr

ain

ing

Co

nst

ruct

ion

Art

s an

dre

cre

atio

nse

rvic

es

Ad

min

istr

ativ

ean

d s

up

port

serv

ices

Acc

omm

od

atio

nan

d f

oo

dse

rvic

es

Ren

tal,

hiri

ngan

d r

eal e

stat

ese

rvic

es

Agr

icul

ture

,fo

rest

ry a

ndfi

shin

g

Info

rmat

ion

me

dia

and

tele

com

mu

nica

tio

ns

Fina

nci

al a

ndin

sura

nce

serv

ices

Tran

spo

rt, p

osta

lan

d w

areh

ousi

ng

Who

lesa

le T

rade

Man

ufa

ctu

rin

g

Ele

ctri

city

, gas

,w

ater

an

d w

aste

serv

ices

Pub

licad

min

istr

atio

nan

d s

afet

y

Min

ing

Oth

er

Industry:

Q2 2018 Business (n=1,003 respondents)

• Data is weighted to be representative of the NSW population (ABS) based on business size and region

• Due to rounding, figures may not sum to 100

20+2%

Sole Proprietor

47%2-932%

10-1918%

20+3%

Sole Proprietor

30%2-939%

10-1928%

Business

Appendix B – Background - Objectives of the QPCS across Key Outputs

Special Interest Topic

Understand respondents’ perceptions of special interest

topics relevant to Whole of Government initiatives (the SIT in

Q4 2018 Communications and Engagement)

Driver Deep Dive

Capture breadth: understand respondents’ perceptions across

the drivers of satisfaction identified in the Annual CSMS

Capture depth: deep dive into specific Key Primary Opportunity

Areas identified in the Annual CSMS

Frequency

Provide a more frequent understanding on how New South

Wales Government services are performing overall, interpreted in

the context of the Premier’s Priority 12

+ +

Lead indicator for Premier’s Priority 12 - respondents’ perceptions of

NSW Government services overall, captured by the Customer Satisfaction Index and its

composite measures (overall satisfaction,

expectation, ideal service)

Breadth: more frequent understanding of the 8 drivers of

satisfaction.

Depth: deeper understanding of Key Primary Opportunity Areas captured in the Annual CSMS

(Efficiency and effectiveness of employees and Access to

information)

+

Customer Satisfaction Index & Baseline Measures

Deep Dive into Driver Performance

To provide an opportunity to understand respondents’

perceptions of relevant key Whole of Government initiatives that are

impacting the delivery of NSW Government services (the SIT in Q4

2018 Communications and Engagement)


+

Project Objectives:

Research Outputs:

Fixed section Fixed + rotating variable section Rotating variable section

Core Component (~80% of survey) Supplement Component (~20% of survey)

Focus of this update

Appendix B – Background - Research Scope and Approach• The QPCS Methodology is aligned to the Annual Customer Satisfaction

Measurement Survey (CSMS) approach:

• Captures feedback across 23 different NSW Government services

(described in the customers language).

• Feedback received from customers about each of the individual

services is aggregated to provide a view of the performance of NSW

Government services overall.

• Each respondent provides feedback regarding one or two services (as

a result, the total number of responses received across services is

greater than the total number of customers who completed the

survey).

• The survey was completed from 29 October 2018 to 5 November 2018 and

results are therefore reflective of experiences with services over the six

months prior i.e. from May 2018 to October 2018.

• The Q4 2018 QPCS was completed with:

• N = 1,000 consumers, and

• N = 1,004 businesses

• As each respondent provides feedback regarding one or two services, the Q4

2018 QPCS number of responses:

• N = 1,592 for consumers, and

• N = 1,571 for businesses

• All scores reported in this document are out of 10, with the exception of the

Customer Satisfaction Index which is out of 100.

In scope services

Industry• Agricultural Advice and

Funding Services• Business Advisory Services• Water Supply• TAFE Services

Justice • Police• State Emergency Services• Prisons• Courts• Fire Brigades

Family & Community Services • Public Housing• Disability Services• Child Protection Services• Services for Older People

Transport• Public Transport• Car and Boat Registration• Major Roads

Finance, Services & Innovation

• Consumer Affairs (FairTrading)

Planning & Environment

• Environment and WildlifeProtection

• Art Galleries and Museums

Education

• Public Schools

Health

• Public Hospitals

• Ambulance Services

Multiple clusters

• Documentation Services (including certificates for birthsdeaths and marriages; trade licenses and certificates; anddrivers licenses)

Consumer Business

Appendix B – Background - Linkage Between the Annual CSMS and QPCSThe Annual CSMS and QPCS is incorporated into the ‘Integrated Phase’ of the CSIP Program:

• Annual CSMS – Clusters will implement an Annual review of CSIPs based on the Annual CSMS insights along with individual clusters’ customer satisfaction

measurement results.

• QPCS - the QPCS collects high level feedback on NSW Whole of Government performance more regularly. Results will be monitored and continuous

trends may trigger a diagnostic review.

Consumer Business

Key Features 2018 Annual CSMS Q4 2017 QPCS Q1 2018 QPCS Q2 2018 QPCS Q4 2018 QPCS

Qualifying Criteria and Measurement Period

• Direct dealings with NSW publicservices within last 12 months

• 2017 CSMS results are reflective of experiences with services betweenMay 2017 and May 2018

• Direct dealings with NSW public services within last 6 months

• Q4 2017 QPCS results arereflective of experiences withservices between April 2017 and October 2017.

• Q1 2018 QPCS results arereflective of experiences withservices between August 2017 and January 2018.

• Q2 2018 QPCS results arereflective of experiences withservices between November2017 and April 2018.

• Q4 2018 QPCS results arereflective of experiences withservices between May 2018 and October 2018.

Customer Satisfaction Index

• All measure expectations, satisfaction and comparison withideal service

• All measure expectations, satisfaction and comparison with ideal service

Perceptions of Services& Satisfaction Drivers

• Measures perceptions of attributes• Drivers of satisfaction are derived

from measurement at an attributelevel for all drivers

• Measures perceptions of satisfaction drivers (asked directly)• Drivers of satisfaction as identified by the CSMS

Sample Size• Consumer n=4,437;• Business n=1,098

• Consumer n=1,002;• Business n=241

• Consumer n=999;• Business n=252

• Consumer n=1,030;• Business n=1,003

• Consumer n=1,000;• Business n=1,004

Margin of Error for Customer Satisfaction Index (95% Confidence Interval)

• Consumer is ± 0.8• Business is ± 1.5





Recency of Experience

Consumers:• 0-3 months: 57%• 3-6 months: 23%• 6-12 months: 21%

Consumers:• 0-3 months: 60%• 3-6 months: 40%

Businesses:• 0-3 months: 58%• 3-6 months: 42%








N/A Citizen Engagement Staff Capability Cost of Living Communication

Appendix B – Background – Project Timelines

Ch

rist

mas

an

d N

ew Y

ear

Bre

ak

We are hereQPCS Q4 2018

QPCS Q1 2018

QPCS Q4 2018

QPCS Q2 2018

LegendRefine Survey Instrument/ SIT Selection for QPCS Survey set-up

Survey in-field

Analysis

Focus groups

Survey results

Annual CSMS Annual CSMS

RG SteerCo

The Annual CSMS will inform Key Primary Opportunity Areas to be

included in the QPCS

Longitudinal trends in consecutive QPCS will inform the inclusion of new

attributes in the Annual CSMS

Dec

2017

Jan

2018

Feb

2018

Mar

2018

Apr

2018

May

2018

Jun

2018

Jul

2018

Aug

2018

Sep

2018

Oct

2018

Nov

2018

Dec

2018

Project Activity

Reference Group Workshops

10th

April(2018 CSMS review and

plan)

27th

August(2018 CSMS

Key Findings & WhoG Rec)

Customer Service Council Meeting

22nd

March(Q1 Top-Line

Report)

31st

May(Q2 Top-Line

Report)

26th

July (CSMS

Top-Line Report)

27th

September(CSMS Key

Findings and WhoG Rec)

29th

November(Q4 Top-Line

Report)

Steering Committee

Meeting

15th

January15th

February8th

March12th

April10th

May7th

June26th

July9th

August13th

September11th

October15th

November13th

December

Appendix B – Background - Key Considerations for Interpreting QPCS Insights

General considerations:

• The QPCS results do not replace the Annual CSMS results, but rather

provide a directional indication of the shift in the results.

• Although the QPCS sample characteristics are closely representative

of the NSW population, different customers have been surveyed and

as such the results are directional indicators of shifts in the Annual

CSMS results only.

• The margin of error (MoE) for the QPCS needs to be considered

when interpreting the results.

Considerations for interpreting the QPCS data points:

• The QPCS results need to be interpreted in the context of the time of the year and in light of events in order to normalise seasonal trends in

the data. Therefore, overall caution should be taken when interpreting the QPCS findings until a minimum of a full year of results has been

collected, so that any seasonal impacts can be examined and adjusted accordingly.

• Contextual factors for the following results (Q4 2018) have been considered and include factors such as train delays and strikes, stadium

upgrades and light rail construction. Contextual factors will be further explored as part of the analysis of the summary report.

• In the following slides, the results of Q4 2018 QPCS have been compared to the results of Q4 2017 QPCS and Q1 & Q2 2018 QPCS.

Significance testing is based on the comparison to Q4 2017 and Q2 2018 results at 95% confidence level, which was the recommended and

endorsed approach for trend analysis as part of the QPCS Methodology Report (as opposed to 99% confidence level for CSMS).

• The Annual CSMS results have been provided as additional context for the QPCS data point and should not be used as a comparison to QPCS

results.

• A longitudinal dataset will need to be built over time in order to identify 'real' trends in the QPCS results and to strengthen the reliability and

validity of any conclusions drawn.

Q12017

Q22017

2017CSMS

Q42017

Q12018

Q22018

CSMS2018

Q42018

1. Set a benchmarkwith the first data point

Interpretation Plan for Tracking Study

2. Draw insights bycomparing to previous quarter

3. Form a directionaltrend

4. Develop a leadindicator

We are here

Time

Mea

sure

ILLUSTRATION ONLY - NOT REAL DATA

Appendix C – QPCS Qualitative Research Objectives

Qualitative research complements the QPCS survey results by providing further insights and context into the results of the quantitative survey. In Q4 2018, the qualitative research will provide insight into the Special Interest Topic – Communications and Engagement.

Q4 2018 QPCS Qualitative Research:

An online discussion forum was used as the qualitative research approach in Q4 2018 QPCS. There were 2 online discussion forums, a consumer group and a business group. Participants from each group were recruited from the respondents of Q4 2018 QPCS survey with a mix of demographics.

The 3-day online discussion forums ran from 19 to 21 November 2018 for the consumer group and 21 to 23 November 2018 for the business group. Participants were required to answer all the pre-designed questions and encouraged to comment on other people’s posts. Moderators monitored the two forums and follow up questions were posted which prompted to participants to make sure sufficient insights were captured.

The final responses came in both text and video formats which provided an in-depth understanding of baseline measures of Q4 2018 and the SIT topic.

Appendix C – Online Discussion Forum- Overview of Online Discussion Forum Approach

Approach:

• Participants were recruited from the respondents of Q4 QPCSsurvey

• Participants come from backgrounds with a mix of age,gender, location and experience interacting with NSWGovernment services

• The 3-day online discussion forums ran from 19 to 21November 2018 for the consumer group and 21 to 23November 2018 for the business group

• Each discussion group aimed to reach 20 respondents

• Focus areas were developed based on initial quantitativefindings and informed the discussion guide (high levelstructure on the right)

Structure of the Discussion Guide

Introduction

Topics

Perception of NSW Government[Business only]

Proactive communication preferences

Communication frequency and topic areas

Perception of NSW Governmentbrand and logo

Impact of advertising and messaging

Group Number of Respondents

Consumer 22

Business 20

Increasing awareness and perception ofNSW Government brand

Business expectations of NSW Government compared to personal expectations

Day 1

Day 3

Day 2

Appendix C – Q4 2018 QPCS Qualitative Research Online Forum Questions (1/3)

Day Topic Question

Day 1

Introduction

Before we talk about your experiences with NSW Government services, we'd like to get to know a bit about you.Q1. To start, we'd like you to record a short video (about 30 seconds in landscape mode) about you. Tell us about:

• Three things that matter most to you (i.e. your phone, your morning cup of coffee, your kids)• One thing you like most about living in NSW• Types of information you receive from NSW government services

If you cannot record a video, please respond via text and include a photo of yourself so we get to know a bit more about who you are.

Proactive communication

These questions are about NSW Government communications.Government communications can include information campaigns about Government programs, decisions, and events. Government communications can also include campaigns designed to change behaviour such tobacco control or reducing litter. Government communications can come through many channels including TV, radio, social media, newsletters, signs and banners, and websites. Please consider all of this when answering the following questions.

Q2. Which NSW government services have you seen or received proactive communication from in the last twelve months?Poll — List of services

Q3.a. How do you typically receive proactive communication from these NSW government services? Please rank the options from: 1 — Most

Likely to 10 — Least LikelyPoll — Events, Mail, Fax, Telephone, Email, Website, Social Media, TV, Print Media (Newspapers, Magazines), Other (Please specify)

b. Is this the preferred way that you would like to receive proactive communication from businesses and government organisations? [Opentext]

Q4. Thinking back to the last time you saw or received proactive communication from a NSW government service…a. Can you walk us through what happened?

(i.e. Which NSW government service it was for, how did you see or receive the communication, and what was the communication about?) [Long response] [Open text, images]

b. What do you value in communication from NSW government?c. What did you do after receiving this proactive communication? And why did you act in this way? (i.e. did you choose to act on that

information or did you choose to ignore it)

Q5. a. What are the top three topics that you feel are most important for the NSW government to communicate to citizens about?

Poll — List of most important topicsb. Why did you pick these three topics? Is there anything missing?

Q6. Imagine in the future, the communication you receive from NSW government is perfect in every single way. Based on this future experience, what do you think NSW government can do to improve their communication today? [Open text]

Day Topic Question

Day 2Impact of advertising

and messaging

Q1. When was the last time that you proactively sought out information from a NSW government service? What happened?[Open text]

Q2.a. Thinking about this experience, would you have greater trust in information from NSW government services if you sought it out yourself or

received proactive communication and/or advertising? [Open text] b. Why did you give this response? [Open text]

Q3.a. How often do you expect to receive communication from government?

Poll — Weekly, Fortnightly, Monthly, Bimonthly, Six monthly, Yearly, Rarelyb. Yesterday we asked you about the important topics that government needs to increase its proactive communication to citizens about. With

communication increasing for some topics, we want to understand where you think there can be a trade off.What communication are you currently receiving that you feel like you don’t need proactive communication for?

Q4. How much trust do you have in advertising and messages from NSW government services?Please answer based on the following rating: 1 — Extremely low trust, 2 — Low trust, 3 — Moderate trust, 4 — High trust, 5 — Extremely high trust

Q5. We'd like you to record a short video (in landscape mode) that talks about the last advertisement you saw for a NSW government service. If you cannot record a video, please respond via text. Tell us about:

• What was the advertisement about?• Where did you see the advertising?• How did this advertisement influence your opinion of this service?• How did this advertisement influence your opinion of overall NSW government?

Q6. Thinking back to the last time you told family and/or friends about a NSW government service…a. How did you find out about this service?

Poll — Events, Mail, Fax, Telephone, Email, Website, Social Media, TV, Print Media (Newspapers, Magazines), Other (Please specify)b. How did advertising influence/not influence you to tell others about this service? [Open text]

Day 3 NSW Government brand

Q1. When I think of NSW government the word that comes to mind is_________.I gave this word because_________. When NSW government services match/don’t match my opinion, I feel _________.

Q2. How strongly would you say the NSW Waratah logo influences your perception of NSW government services? Please answer based on the following rating: 1 — Extremely low influence, 2 — Low influence, 3 — Moderate influence, 4 — High influence,5 — Extremely high influencea. Why did you give this response?


Day Topic Question


Q3. We'd like you to record a short video (in landscape mode) that talks about your opinion of the NSW Waratah logo. Tell us about:• What does the NSW Waratah logo mean to you?• When you see the NSW Waratah logo, what expectations do you have of this service?

Q4.a. How does the NSW government logo and brand compare to other companies you have dealt with? (i.e. phone/internet providers, banks and

airlines) [Open text] b. Why did you give this response? [Open text]

Q5. When you receive communication and/or see advertising for services in NSW…a. Why is it important that it has NSW government branding? Can you walk us through an example? [Open text]b. Can you think of a time where there was no NSW government branding? How did this affect your perception? [Open text]

Q6. What do you think NSW government needs to improve on, to increase awareness and perception of the NSW government brand?[Open text]

Q7. [Optional] Thank you for your time. If you have any additional comments or questions we would love to hear them below.

Business - Additional Questions

Day Topic Question

Day 1Business perception of NSW

government

[Insert after Q1]

1. How does your perception with NSW government compare to 12 months ago?2. Why did your provide this response?3. Has your experience and interactions with NSW government, changed over the last 12 months based on changes in your industry and/or

outside of government?


[Insert after Q5]

When acting as a business, are your expectations in dealing with the NSW government brand, different to your expectations when dealing with NSW government as an individual?


Appendix C – Demographics of Online Forum Participants

Consumers Discussion Group Business Discussion Group

Number of Participants

Days

Day 1 21 (304 posts)



Total 22 (805 posts)

Video content uploads

9

GenderFemale 11

Male 11

Age

18-24 4

25-44 8

55+ 10

Location

Metropolitan 19

Regional 2

Rural 1

Total participants over 3 days 22

Number of Participants

Days




Total 20 (814 posts)

Video content uploads

4

GenderFemale 4

Male 16

Business size (number of employees)

1-20 10

21-200 7

200+ 3

Business stage

Startup 1

Mature 10

Growth and establishment

6

Expansion 3

Location

Metropolitan 18

Regional 2

Rural 0

Total participants over 3 days 20

2018 Q4 Quarterly Pulse Check Survey (QPCS) Summary Report · Other business indexes available: Q4...

Documents

Transcript of 2018 Q4 Quarterly Pulse Check Survey (QPCS) Summary Report · Other business indexes available: Q4...