2019 Q2 Quarterly Pulse Check Survey (QPCS) · 2020. 3. 11. · is close to meeting these...

2019 Q2 Quarterly Pulse Check Survey (QPCS)

Summary Report

Please note that this report is not web accessible.

The accessible version is coming soon.

Item Page no.

1

Executive Summary- Customer Satisfaction Index (CSI)- Key Insights 2

2

Customer Satisfaction Measures- Customer Satisfaction Index (CSI) - Consumer- Satisfaction, Expectation and Comparison to Ideal - Consumer- Findings on Key Findings - Qualitative Research for Consumer- Customer Satisfaction Index (CSI) - Business- Satisfaction, Expectation and Comparison to Ideal - Business- Findings on Key Findings- Qualitative Research for Business- Comparison to Business Confidence Indices - Customer Effort Score- Customer Effort Comparison Score

7

3

Insights on Satisfaction Drivers- Impact of Satisfaction Drivers – Consumer- Impact of Satisfaction Drivers – Business- Key Drivers of Satisfaction – Qualitative Research

20

4

Channel Usage and Preference- Channel Usage and Preference - Consumer- Channel Usage and Preference - Business- Impact of Channel Expectations – Qualitative Research

26

5

Appendix- Appendix A: Demographic Profile of Respondents- Appendix B: Background to the QPCS- Appendix C: QPCS Qualitative and Quantitative Research Approach- Appendix D: Historical Consumer and Business Baseline Measures- Appendix E: Customer Satisfaction Index Q1 2019 QPCS (CSMS at 99% Confidence Level)- Appendix F: Terminology Definitions

30

Table of Contents

1

1. Executive Summary

1.1 Customer Satisfaction Index Scores (CSI) - Consumer

1.2 Key Insights - Consumer

1.3 Customer Satisfaction Index Scores (CSI) – Business

1.4 Key Insights - Business

2

Consumer – CSI

Consumer – Baseline Measures

Margin of Error (QPCS)Consumer CSI: ± 1.3

Sample size for CSI and Baseline measure may differ on the basis of “Don’t Know” option selectionFor comparison between CSMS and QPCS results, please refer to Appendix Section for “Interpretation notes for Q2 QPCS report»For CSMS at 99% confidence level, please refer to Appendix «page 43»

76.7

78.7 79.3 79.9 79.6 78.979.9 79.4

80.9

CSMS2015

(n=6,549)

CSMS2016

(n=6,971)

CSMS2017

(n=6,527)

Q12018

(n=1,610)

Q22018

(n=1,603)

CSMS2018

(n=6,701)

Q42018

(n=1,535)

Q12019

(n=1,575)

Q22019

(n=1,555)

7.3

7.57.7 7.7 7.7

7.6

7.87.7

7.87.7

7.87.9

7.9 7.9 7.98.0

7.8

8.0

6.9

7.2 7.2 7.2 7.37.2

7.3 7.2

7.4

6.66.87.07.27.47.67.88.08.28.48.6

CSMS2015

(n=6,593)

CSMS2016

(n=7,015)

CSMS2017

(n=6,559)

Q12018

(n=1,628)

Q22018

(n=1,618)

CSMS2018

(n=6,733)

Q42018

(n=1,545)

Q12019

(n=1,590)

Q22019

(n=1,564)

Satisfaction Expectation Comparison to Ideal

1.1 Customer Satisfaction Index (CSI) - Consumer

Consumer

Statistically significant movement from same quarter previous year at 95% confidence level

Statistically significant movement from previous quarter at 95% confidence level

3

Figures in the graph are subject to rounding

• Consumer CSI has increased (statistically significant) to 80.9 from 79.4.

• The significant increase in CSI is above Premier’s Priority Target of 79.02. This is the highest recorded CSI indicating strong performance this quarter.

1.2 Key Insights - Consumer

1. The consumer CSI has increased (statistically significant). This indicates an overall uplift in experiences with NSW Government services.

4

ConsumerSatisfaction

Index

Consumer

• Of the employees related drivers, Honesty and integrity, Efficiency and effectiveness, Empathy and communication and Accountability all experienced a statistically significant increase since last quarter.

• All employee related drivers are highly correlated to overall satisfaction indicating that overall experience with employees is having a significant impact on improving customer experience.

• Of the values related drivers, Service quality and Accountability, have increased (statistically significant) since last quarter. Value related drivers reflect consumers’ perceptions of trust and outcomes delivered.

3a. Perceptions of employees have significantly improved this quarter.

Satisfaction Drivers

ConsumerSatisfaction

Index

2. All three measures of customer experience (satisfaction, expectation and comparison to ideal) experienced a statistically significant increase.

• The increase in the CSI can be attributed to increases (statistically significant) since last quarter across all three baseline measures: expectation, satisfaction and comparison to ideal.

• Comparison to ideal has experienced its first statistically significant increase since the 2016 CSMS result. This result indicates that NSW Government services are moving closer towards consumers’ ideal service experience.

• With satisfaction and expectation increasing significantly, this result indicates consumers have increasing positive views of NSW Government services. The satisfaction gap to expectation of 0.2 indicates that although consumers continue to have high expectations of NSW Government services, service performance is close to meeting these expectations.

3b. Perceptions of values have significantly improved this quarter.

76.678.4 78.3

79.6 79.778.2

81.3 81.2 80.7

CSMS2015

(n=1,645)

CSMS2016

(n=1,712)

CSMS2017

(n=1,638)

Q12018

(n=365)

Q22018

(n=1,536)

CSMS2018

(n=1,494)

Q42018

(n=1,523)

Q12019

(n=1,449)

Q22019

(n=1,372)

Business – CSI

Business – Baseline Measures

Margin of Error (QPCS)Business CSI: ± 1.2

Sample size for CSI and Baseline measure may differ on the basis of “Don’t Know” option selectionFor comparison between CSMS and QPCS results, please refer to Appendix Section for “Interpretation notes for Q2 QPCS report»For CSMS at 99% confidence level, please refer to Appendix «page 43»

7.2

7.5 7.57.6

7.8

7.4

7.9 7.8 7.87.6

7.9

7.7 7.7

8.1

7.7

8.0 8.07.9

7.0

7.37.1

7.2 7.3 7.27.4

7.6

7.4

6.6

7.1

7.6

8.1

8.6

CSMS2015

(n=1,654)

CSMS2016

(n=1,718)

CSMS2017

(n=1,646)

Q12018

(n=367)

Q22018

(n=1,554)

CSMS2018

(n=1,506)

Q42018

(n=1,535)

Q12019

(n=1,455)

Q22019

(n=1,378)


1.3 Customer Satisfaction Index (CSI) – Business

Business



5

Figures in the graph are subject to rounding

1.4 Key Insights – Business

• Business CSI has decreased (not statistically significant) to 80.7 from 81.2. This result is below the Premiers Priority Target of 81.16. This decrease in CSI is due to the stabilisation of overall satisfaction and the decrease in expectation and comparison to ideal scores (statistically significant) compared to last quarter.

• The satisfaction gap to expectation has decreased by 0.1 compared to last quarter. This result indicates that although perceptions of performance are moving closer to meeting expectations, businesses have decreasing expectations of NSW Government services.

• The Roy Morgan Business Confidence index* decreased by 3 points in the last quarter from 105.9 (January 2019) to 102.9 (April 2019). The decrease in business confidence mirrors the slight decrease in the business CSI seen this quarter.

4. The business CSI has seen a slight decrease in performance. Satisfaction and expectations have stabilised and comparison to ideal has decreased.

Business

Business Satisfaction

• All satisfaction drivers across employees, values, processes and goals have stabilised as compared to last quarter.

• Following the significant uplift in driver performance in Q4 2018, this performance has been maintained this quarter, with no significant decrease for any driver.

6. Perceptions of service quality (measured through driver performance) has stabilised this quarter.

Satisfaction Drivers

6

• The business customer effort score (CES) has decreased (statistically significant) to 6.1 this quarter from 6.4 last quarter resulting in NSW Government services having the 2nd lowest effort when compared to other industries This indicates that businesses are finding it easier to interact with NSW Government services.

• The result this quarter is consistent with historical trends observed since 2017, where business customer effort decreased from Q1 to Q2.

• This decrease in business effort mirrors a decrease in effort across all other sectors, except banks. Notably, banks experienced a statistically significant increase in effort when compared to Q2 2018.

• Although the decrease in effort is a positive change in businesses overall experience with NSW Government services, this decrease does not influence the CSI as the CSI and CES are not correlated.

5. The businesscustomer effort score has decreased (statistically significant).

CustomerEffort

Business Customer

Effort Score

*Source: Roy Morgan Business Confidence poll April 2019

2. Customer Satisfaction Measures


2.2 Satisfaction, Expectation and Comparison to Ideal - Consumer

2.3 Findings on Key Measures - Qualitative Research for Consumer

2.4 Customer Satisfaction Index (CSI) - Business

2.5 Business Confidence

2.6 Satisfaction, Expectation and Comparison to Ideal – Business

2.7 Findings on Key Measures - Qualitative Research for Business

2.8 Customer Effort Score

2.9 Customer Effort Comparison Score

7

Source: NSW Office of the Customer Service Commissioner, Customer Satisfaction Measurement Survey 2019 Source: NSW Office of the Customer Service Commissioner, Quarterly Pulse Check Survey 2019

Note: All the numbers are subject to rounding.

76.7

78.777.8

79.578.6

79.380.2 79.9 79.6

78.979.9 79.4

80.9

CSMS 2015(n=6,549)

CSMS 2016(n=6,971)

Q4 2016(n=1,612)

Q1 2017(n=1,540)

Q2 2017(n=1,594)

CSMS 2017(n=6,527)

Q4 2017(n=1,508)

Q1 2018(n=1,610)

Q2 2018(n=1,603)

CSMS 2018(n=6,701)

Q4 2018(n=1,535)

Q1 2019(n=1,575)

Q2 2019(n=1,555)



Premier’s Target -79.02

Q2 2019 QPCS Survey Details: Number of respondents: 1,003 consumers (1,600 responses)Fieldwork period: from 15 April to 23 April 2019

Consumer

Quarter-on-quarter comparison of QPCS results (Q1 2019 vs. Q2 2019)

• Customer Satisfaction Index (CSI) has increased by 1.5 points (statistically significant) to 80.9/100 from 79.4/100 (Q1 2019).

QPCS Customer Satisfaction Index (CSI) has statistically significantly increased to 80.9/100 when compared to Q1 2019 and Q2 2018, bringing it well above

the Premier’s target of 79.2 and making it the highest CSI the QPCS and CSMS has ever seen.

Year-on-year comparison of QPCS results (Q2 2018 vs. Q2 2019)

• CSI has increased by 1.3 points (statistically significant) to 80.9/100 from 79.6/100 (Q2 2018).


8

7.3

7.57.4

7.6 7.67.7 7.7 7.7 7.7 7.6

7.8 7.77.8

7.7

7.8 7.8

7.97.8

7.9

7.87.9 7.9 7.9

8.0

7.8

8.0

6.9

7.2 7.2

7.3

7.2 7.27.3

7.2 7.37.2

7.3 7.2

7.4

6.7

7.2

7.7

8.2

CSMS 2015(n=6,593)

CSMS 2016(n=7,015)

Q4 2016(n=1,621)

Q1 2017(n=1,548)

Q2 2017(n=1,605)

CSMS 2017(n=6,559)

Q4 2017(n=1,518)

Q1 2018(n=1,628)

Q2 2018(n=1,618)

CSMS 2018(n=6,733)

Q4 2018(n=1,545)

Q1 2019(n=1,590)

Q2 2019(n=1,564)

Satisfaction

Expectation

Comparison toideal

2.2 Satisfaction, Expectation and Comparison to Ideal – Consumer (1/2)

Significant changes CSMS 2015*

CSMS 2016

Q4 2016 Q1 2017 Q2 2017 CSMS 2017

Q4 2017 Q1 2018 Q2 2018 CSMS 2018

Q4 2018 Q1 2019 Q2 2019

Satisfaction NA - - - - -

Expectation NA - - - - -

Comparison to ideal NA - - - - - - - -



No significant movement from previous quarter or same quarter previous year at 95% confidence level

* Data for comparison with previous period not available

-Note: Figures in the graph are subject to rounding

Consumer

The CSI is composite of the following key measures: expectation, satisfaction and comparison to idea service. All three key measures experienced a

statistically significant increase when compared to Q1 2019 attributing to the significant uplift in the CSI. The satisfaction gap to expectation has widened

slightly by 0.1. This is due to a larger increase in expectation than satisfaction this quarter. This result indicates that consumers continue to have high

expectation of NSW Government services but service performance is close to meeting these high expectation.


• Expectation has increased by 0.2 points (statistically significant) to 8.0/10 from 7.8/10 (Q1 2019).

• Satisfaction has increased by 0.1 points (statistically significant) to 7.8/10 from 7.7/10 (Q1 2019).

• Comparison to ideal service has increased by 0.2 points (statistically significant) to 7.4/10 (Q1 2019) from 7.2/10 (Q1 2019).


• Expectation has increased by 0.1 points (not statistically significant) to 8.0/10 from 7.9/10 (Q2 2018).

• Satisfaction has increased by 0.1 points (statistically significant) to 7.8/10 from 7.7/10 (Q2 2018).

• Comparison to ideal service has increased by 0.1 points (statistically significant) to 7.4/10 from 7.3/10 (Q2 2018).

9

2.2 Satisfaction, Expectation and Comparison to Ideal – Consumer (2/2)

Consumer – Outcome Measures

Expectation

Average (out of 10)

Satisfaction Comparison to Ideal

% D

istr

ibu

tio

n (

acro

ss lo

w,

neu

tral

an

d h

igh

sco

res)

7% 6% 6% 8% 5% 9% 8% 8% 9% 8% 12% 11% 13% 13% 11%12% 13% 11% 12% 13%

13% 15% 12% 15% 13%17% 20% 16% 16% 14%

81% 81% 84% 80% 82% 78% 77% 80% 77% 80%71% 69% 72% 71% 75%

Q2 2018(n=1,642)

CSMS 2018(n=6,922)

Q4 2018(n=1,579)

Q1 2019(n=1,614)

Q2 2019(n=1,578)

Q2 2018(n=1,638)

CSMS 2018(n=7,000)

Q4 2018(n=1,583)

Q1 2019(n=1,611)

Q2 2019(n=1,577)

Q2 2018(n=1,618)

CSMS 2018(n=6,733)

Q4 2018(n=1,545)

Q1 2019(n=1,590)

Q2 2019(n=1,564)

Low (1-4) Med (5-6) High (7-10)

Consumer

7.9 7.9 8.0 8.0 7.7 7.6 7.8 7.7 7.8 7.3 7.2 7.3 7.2 7.4



CSMS results

10

7.8

Note: Figures in the graph are subject to roundingPlease refer to appendix section “Historical Consumer and Businesses Baseline Measures” for more historical data

Compared to the last quarter, the increase in expectation, satisfaction and comparison to ideal is being driven by a 2%, 3% and 4% increase in consumers who rate their experience as high (i.e. 7-10 out of 10). This increase is expectation, satisfaction and comparison to ideal is further driven by a 3%, 1% and 2% decrease in consumer’s who rate their experience as low (i.e. 1-4 out of 10).

Comparison to Ideal experienced its first statistically significant increase since Q1 2019 indicating that NSW Government is moving closer to delivering consumers’ ideal services.

I felt really close and warm when I met the personnel from the school, it gave me the feeling of trust. This can be maintained as

long as those people happy and their expectation from the government is met

I think there have been a lot of changes implemented in the last 12 months, change to the service centres, being open all weekend,

Green slip rebate are all good positive changes to happen. The new centres are so much brighter, open, friendly and staff are pleasant

and nice to you

• Overall, there has been positive sentiment supporting the increase

in CSI for NSW Government services

• Consumers positively mentioned that having the option to utilise

online channels and services provides them greater flexibility to

complete processes and receive status updates via their preferred

channel when interacting with government services

• Consumers have also expressed positivity to government services

due to recent improvements in infrastructure supporting service

delivery such as refurbished service centres. Customers mention

that these changes make it easier to interact with services due to

greater availability of staff and information, and increase time

efficiencies during dealings with government services

Consumers show positive sentiment towards changes in services

Insights from Q2 Qualitative Discussion

• Consumers have expressed feeling a sense of trust and reliability

with the service if they are met with staff that display empathy and

effective communication during the process

• Consumers are increasingly impressed by the product knowledge

and empathy of staff especially within the service categories of

emergency services and social services. Consumers have stated

that this has allowed them to build greater trust with services

Employee drivers have contributed to increasing consumer satisfaction

Shout out to the very helpful and delightful employee at Service NSW Brookvale who guided me through and sped up process of applying

for a replacement driver licence last month

“

“

“

“”

2.3 Findings on Key Measures – Qualitative

When I think about the changes I’ve seen to government services in last 12 months, I feel relief because, most of the information and

enquiries can be solved and done online. I really don’t want to waste tons of time by visiting centres

“

Consumer

”My experience has improved over the last few months by virtue of the move to making services more accessible through online self service portals. This reflects an understanding on behalf of the

government the importance of allowing users to access services they need in a flexible manner

“

”

”

11

”


Note: All the numbers are subject to rounding.

Q2 2019 QPCS Survey Details: Number of respondents: 1,009 businesses (1,394 total responses)Fieldwork period: from 15 April to 23 April 2019

2.4 Customer Satisfaction Index (CSI) – Business

76.6

78.4

76.1

77.6 77.678.3

77.8

79.6 79.7

78.2

81.3 81.280.7

CSMS 2015(n=1,645)

CSMS 2016(n=1,712)

Q4 2016(n=375)

Q1 2017(n=380)

Q2 2017(n=365)

CSMS 2017(n=1,638)

Q4 2017(n=343)

Q1 2018(n=365)

Q2 2018(n=1,536)

CSMS 2018(n=1,494)

Q4 2018(n=1,523)

Q1 2019(n=1,449)

Q2 2019(n=1,372)



Premier’s Target –81.16

Business


• Customer Satisfaction Index (CSI) has decreased by 0.5 points (not statistically significant) to 80.7/100 from 81.2/100 (Q1 2019).

For Business, the QPCS Customer Satisfaction Index (CSI) has decreased to 80.7/100 (not statistically significant) when compared to the last quarter but

increased by 1.0 points when compared to Q2 2018. The slight decrease in CSI is due to the stabilisation of the overall performance of satisfaction and

expectation at a high level.


• CSI has increased by 1.0 points (not statistically significant) to 80.7/100 from 79.7/100 (Q2 2018).

12

2.5 Business Confidence

The decrease in business CSI mirrors the results of the Roy Morgan Business Confidence Index and the Sensis Business Confidence Index. The Roy Morgan index experienced a decrease of 3.0 when compared to Q1 2019 and the Sensis Index decreased by 16 when compared to Q1 2019. The trends of the Roy Morgan Index, Sensis Index and the business CSI indicates that businesses have decreasing confidence and perceptions of their ability to effectively manage their business in the current economic environment.

Business

13Note: Figures in the graph are subject to rounding

Comparison of Business CSI and Business Confidence Measures

About the Roy Morgan Business Confidence Index: The Index is based on 5 different attributes measuring Australian business’ expectations of the economic climate over the next 12 months. (n=~1,000 Australian businesses per month). Data points for the Roy Morgan Business Confidence Index are taken from the corresponding month of QPCS/CSMS fieldwork. http://www.roymorgan.com/morganpoll/consumer-confidence/roy-morgan-business-confidence.

Business Confidence (Roy Morgan Index) and Business CSI

79.59 79.69

75.92

81.2581.20

80.70

114

117 124120

115

119117

114 114

110112

113 113.8112.2

105.9 105.6106.7

102.9

100

105

110

115

120

125

130

75

82

No

v-1

7

Dec

-17

Jan

-18

Feb

-18

Mar

-18

Ap

r-18

May

-18

Jun

-18

Jul-

18

Au

g-18

Sep

-18

Oct

-18

No

v-1

8

Dec

-18

Jan

-19

Feb

-19

Mar

-19

Ap

r-19

Bu

siness C

on

fiden

ce Ind

ex

CSI

CSI Roy Morgan Business Confidence Index*

75.92

80.70

42

49

42

50

34

30

35

40

45

50

55

60

75

82

Mar

-18

Jun

-18

Sep

-18

Dec

-18

Mar

-19

Bu

siness C

on

fiden

ce Ind

ex

CSI

CSI Sensis Business Confidence Index

Business Confidence (Sensis) and Business CSI

About the Sensis Business Confidence Index:. This index is calculated by subtracting the number of respondents who are negative about the current economic climate from the number that are positive (n=1,000 small and medium Australian businesses per quarter). Data points for the Sensis index are taken from the corresponding month of QPCS/CSMS fieldwork. https://www.sensis.com.au/about/sensis-business-index


7.2

7.5 7.2

7.4

7.37.5 7.5

7.6

7.8

7.4

7.9

7.8

7.8

7.6

7.9

7.6

8.1

7.5

7.77.6

7.7

8.1

7.7

8.0 8.07.9

7.0

7.3

7.1

7.4

7.1 7.1 7.0

7.27.3

7.2

7.4

7.6

7.4

6.7

6.9

7.1

7.3

7.5

7.7

7.9

8.1

8.3

CSMS 2015(n=1,654)

CSMS 2016(n=1,718)

Q4 2016(n=375)

Q1 2017(n=380)

Q2 2017(n=365)

CSMS 2017(n=1,646)

Q4 2017(n=345)

Q1 2018(n=367)

Q2 2018(n=1,554)

CSMS 2018(n=1,506)

Q4 2018(n=1,535)

Q1 2019(n=1,455)

Q2 2019(n=1,378)

Satisfaction

Expectation

Comparison toideal

Significant changes CSMS 2015*

CSMS 2016

Q4 2016 Q1 2017 Q2 2017 CSMS 2017

Q4 2017 Q1 2018 Q2 2018 CSMS 2018

Q4 2018 Q1 2019 Q2 2019

Satisfaction NA - - - - - -

Expectation NA -

Comparison to ideal NA - - - - - - -


• Expectation has decreased by 0.1 points (not statistically significant) to 7.9/10 from 8.0/10 (Q1 2019).

• Satisfaction has remained stable at 7.8/10.

• Comparison to ideal service has decreased by 0.2 points (statistically significant) to 7.4/10 from 7.6/10 (Q1 2019).

Note: Figures in the graph are subject to rounding

* Data for comparison with previous period not available



No significant movement from previous quarter or same quarter previous year at 95% confidence level-

Business

2.6 Satisfaction, Expectation and Comparison to Ideal – Business (1/2)

The CSI is composite of the following key measures: expectation, satisfaction and comparison to idea service. The satisfaction gap to expectation has

narrowed to 0.1 as a result of satisfaction stabilising and expectation slightly increasing.


• Expectation has decreased by 0.1 points (not statistically significant) to 7.9/10 from 8.0/10 (Q1 2019).


• Comparison to ideal service has decreased by 0.2 points (statistically significant) to 7.4/10 from 7.6/10 (Q1 2019).


• Expectation has decreased by 0.2 points (statistically significant) to 7.9/10 from 8.1/10 (Q2 2018).


• Comparison to ideal service has increased by 0.1 points (not statistically significant) to 7.4/10 from 7.3/10 (Q2 2018).

14

7.7

2.6 Satisfaction, Expectation and Comparison to Ideal – Business (2/2)

Business – Outcome Measures

% D

istr

ibu

tio

n (

acro

ss lo

w,

neu

tral

an

d h

igh

sco

res)

Average (out of 10)

Expectation Satisfaction Comparison to Ideal

6% 6% 5% 6% 5% 9% 7% 5% 9% 8% 12% 11% 9% 8% 10%9% 15% 14% 11% 14% 11% 18% 16% 12% 14%

15% 19% 18% 13%17%

84% 79% 82% 83% 81% 80% 75% 79% 79% 79% 73% 71% 73% 79% 73%

Q2 2018(n=1,555)

CSMS 2018(n=1,531)

Q4 2018(n=1,552)

Q1 2019(n=1,465)

Q2 2019(n=1,389)

Q2 2018(n=1,570)

CSMS 2018(n=1,555)

Q4 2018(n=1,553)

Q1 2019(n=1,469)

Q2 2019(n=1,387)

Q2 2018(n=1,554)

CSMS 2018(n=1,506)

Q4 2018(n=1,535)

Q1 2019(n=1,455)

Q2 2019(n=1,378)

Low (1-4) Med (5-6) High (7-10)

Business

8.0 8.0 7.9 7.8 7.4 7.9 7.8 7.8 7.3 7.2 7.4 7.6 7.4



CSMS results

15

8.1

Compared to the last quarter, the decrease in comparison to ideal for business is being driven by a 6% decrease in the businesses that rate their experience as high (i.e. 7-10 out of 10).

Note: Figures in the graph are subject to roundingPlease refer to appendix section “Historical Consumer and Businesses Baseline Measures” for more historical data

Duplicating steps is frustrating, a waste of time and makes me question whether the process I'm following is accurate or not. I don't want to be led down a rabbit hole where you need to go to various

sites, departments, organisations just to complete one process

I feel really happy as they have improved immensely by making customer service a place you can do a lot of different things…a lot

more things can be done online which is much less time consuming …which really helps for small business owners

• Businesses have expressed mixed sentiment towards the changes

within government services and overall satisfaction

• On one side, businesses have mentioned that the introduction of

online channels has improved their experience. It provides greater

convenience and status transparency during the process

• On the other hand, businesses have expressed the need for

effective communication between offline and online channels to

create a more seamless experience. This potentially removes

duplicative steps in the process as businesses need to upload

documents for approval and also prove these documents in person

Businesses show mixed sentiment towards recent changes

16Insights from Q2 Qualitative Discussion

• Businesses want to focus efforts on their business and customers.

Hence, a lack of integration between channels can cause

duplicative steps, reducing the time spent on growing the business

• Businesses mentioned that phone and online channels were

unable to provide a reliable seamless experience with a tailored

service aligned to their needs and situation. Although expectations

have not been met, offline channels are perceived to be more

approachable, as they allow for a more personal interaction

Businesses have a lower expectation towards government processes

Over the phone is always hard as they don't have the customer and their documents in front of them

“

“

“

“

”

”

2.7 Findings on Key Measures – Qualitative

When I think about the changes I’ve seen to government services in the last 12 months I feel in the last 12 months that the government

approach has become more personal and much more approachable. I like that you get an email from a genuine person

rather than just an auto response and a follow up call from the same person really makes you feel like your concerns matter

“

”

”

Business

I would say that a majority of the changes haven't improved my experience with dealing with the relevant Govt. Services.

Compliance has been become a top priority and trying to adhere to some of the changes has been challenging and also frustrating.

“

”

I would be angry I found any duplicative steps in a process as I would find it a waste of time – where I could spend more time on my

business. If there were duplicative steps within a process - it would make me question if it was reputable and trustworthy source or not.

“

”

NSW Government service interaction Customer Effort Score.

2.8 Customer Effort Score

In this section, customers rated the level of effort for individual services; this graph shows the average of those ratings

Q: Thinking about your direct dealing with [SERVICE], how much effort did you personally have to put forth? 1 is ‘Low effort’ and 10 is ‘High effort’

Overall, businesses have to put forth higher effort than consumers when interacting with NSW Government services.Since Q1 2019, the customer effort score (CES) has remained stable among consumers at 5.8/10 and decreased by 0.4 points (statistically significant) among businesses to 6.1/10 from 6.5/10 (Q1 2019). Comparing to Q2 2018, the CES has decreased by 0.2 points (not statistically significant) among consumers to 5.8/10 from 6.0/10 and has increased by 0.1 points (not statistically significant) among businesses to 6.1/10 from 6.0/10.Since 2017, a seasonal decrease in business CES from Q1 to Q2 has been observed.

5.9 5.9

5.75.9

6.15.9

6.0

6.0

5.75.8

5.8

6.3

6.9

6.5

4.9

6.76.9

6.0

6.6

6.1

6.5

6.1

Consumers BusinessesLowerEffort

HigherEffort

Q1 2017(n=1,529)(n=382)

Q2 2017(n=1,590)(n=366)

Q4 2017(n=1,496)(n=345)

Q1 2018(n=1,611)(n=373)

*The lower the average Customer Effort Score, the easier customers perceive interactions to be with different services. A decrease in CES is a positive shift.

Q2 2018(n=1,585)(n=1,535)

Q4 2018(n=1,526)(n=1,508)

Consumer Business

CSMS 2017(n=6,501)(n=1,650)

CSMS 2018(n=6,634)(n=1,496)

Statistically significant movement from same quarter previous year at 95% confidence levelStatistically significant movement from previous quarter at 95% confidence level

CSMS results

17

Q1 2019(n=1,548)(n=1,458)

Q4 2016(n=1,601)(n=378)


Q2 2019(n=1,542)(n=1,360)

ConsumerBusiness

• Businesses have mentioned the

efficiencies of having more digitally

available services

• These services allow businesses to

conduct multiple processes

simultaneously, saving time and

allowing them to be proactively

informed of their status via their

preferred channel

• Information about the process is

proactively provided online and

businesses are able to complete

processes at their own convenience

which reduces the effort during

dealings with government services

Generally being informed … via text works really well and I’m satisfied

”

I find it handy to just go online whilst travelling…also I can renew my

business licences online and get a discount for doing so

“

”“


6.26.0 5.9 6.0 5.8 5.7

5.4

6.1 6.05.8

6.05.6 5.6 5.5

6.15.9 5.8 5.9 5.7 5.5 5.5

6.2 6.0 5.9 5.9 5.8 5.6 5.6

Telephone serviceproviders

NSW Governmentservices

Federal Government Energy retailers My Local Council Banks Airlines

Q2 2018 QPCS (n=711 to 943) Q4 2018 QPCS (n=680 to 929) Q1 2019 QPCS (n=688 to 940) Q2 2019 QPCS (n=713 to 940)

2.9 Customer Effort Score: Comparison of NSW Government Services Overall to Other Industries –Consumer

For this section, customers provided a rating for overall effort with NSW Government services in comparison to other industries/sectors Q: Thinking about all your direct dealings with each of the following Australian industries and Government services over the previous 6 months, how much effort did you personally have to put forth?

CES for NSW Government overall is relatively high when compared to other industries - consumer CES increased by 0.1 points (not statistically significant) to 6.0/10 from 5.9/10 (Q1 2019) placing NSW Government services as the second highest effort behind Telephone service providers.

When compared to the same quarter, previous year (Q2 2018), NSW Government services effort score has remained stable in the relative effort required for interactions among consumers.

Customer Effort Score benchmarked at a Whole of Government level against other industries/sectors.

Consumer


*The lower the average Customer Effort Score, the easier customers perceive interactions to be with different services. A decrease in CES is a positive shift. Statistically significant movement from same quarter previous year at 95% confidence level


6.46.0 5.8

6.16.0 5.8 5.6

6.4 6.25.8

6.25.8 5.9 5.8

6.7

6.4 6.16.4 6.3 6.5

6.16.6

6.2 6.2 6.1 6.1 6.1

5.6

Telephone serviceproviders

Energy retailers Banks NSW Governmentservices

My Local Council Federal Government Airlines

Q2 2018 QPCS (n=726 to 902) Q4 2018 QPCS (n=707 to 922) Q1 2019 QPCS (n=760 to 960) Q2 2019 QPCS (n=793 to 962)

2.9 Customer Effort Score: Comparison of NSW Government Services Overall to Other Industries –Business

Compared to other industries, businesses ranked NSW Government services as the fourth highest in terms of effort, behind Telephone service providers, Energy retailers and Banks. For NSW Government services, businesses reported a decrease of 0.3 points (statistically significant) to 6.1/10 from 6.4/10 (Q1 2019). Businesses reported a decrease in effort for Airlines by 0.5 points (statistically significant) to 5.6/10 from 6.1/10 (Q1 2019) and a decrease in effort for Federal Government by 0.4 points to 6.1/10 from 6.5/10 (Q1 2019).

Business

Q: Thinking about all your direct dealings with each of the following Australian industries and Government services over the previous 6 months, how much effort did you personally have to put forth?

Customer Effort Score benchmarked at a Whole of Government level against other industries/sectors.


*The lower the average Customer Effort Score, the easier customers perceive interactions to be with different services. A decrease in CES is a positive shift. Statistically significant movement from same quarter previous year at 95% confidence level


For this section, customers provided a rating for overall effort with NSW Government services in comparison to other industries/sectors

3. Insights on Satisfaction Drivers

20

3.1 Impact of Satisfaction Drivers – Consumer

3.2 Impact of Satisfaction Drivers – Business

3.3 Key Drivers of Satisfaction – Qualitative Research

7.8

7.5 7.7

7.4 7.7

7.4

6.8

7.4 7.4

7.0 7

.3

7.1

8.0

7.7 7.8

7.6

7.3 7

.8

7.57

.8

7.5 7.7

7.4

7.2 7

.6

7.4

7.9

7.7 7.7

7.6

7.5 7

.8

7.6

Employees were open andhonest during the process

Employees acted efficientlyand effectively to reach the

right outcomes

Communications were clear,prompt and easy to

understand

Employees acted withempathy

Employees were heldaccountable for their actions

I was provided with goodservice and outcomes I could

trust

I felt there was accountabilityfor services delivered

Q2 2018 QPCS (n=1,431 to 1,575) CSMS 2018 (n=5,690 to 6,779) Q4 2018 QPCS (n=1,146 to 1,518) Q1 2019 QPCS (n=1,155 to 1,527) Q2 2019 QPCS (n=1,146 to 1,525)

3.1 Impact of Satisfaction Drivers – Consumer (1/2)D

eriv

ed D

rive

rs

(CSM

S 2

01

8)

Drivers of Satisfaction

Comparing to Q1 2019, the employee drivers honesty and integrity of employees, employees acted with empathy and employee accountability have increased (statistically significant). Looking at the values drivers, service quality and accountability have increased (statistically significant). These attributes are positively correlated to consumer CSI indicating that overall experience with employees is having a significant impact on improving customer experience. When compared to the same time last year (Q2 2018), the employee drivers efficiency and effectiveness of employees and employees acted with empathy have increased (statistically significant).

Ave

rage

(o

ut

of

10

)

Q: We would now like to ask you some questions specifically about your experience with [SERVICE]. Thinking now about [its employees; its processes; service overall; the values they uphold], to what extent do you agree or disagree that each of the following statements describes [SERVICE] in NSW? 1 is ‘Strongly disagree’ and 10 is ‘Strongly agree’.

Employees Values

21

n/a

fo

r Q

1 &

Q2

20

18

–n

ew a

ttri

bu

te

intr

od

uce

d f

rom

Q4

20

18

QP

CS

Service Quality AccountabilityAccountabilityHonesty and integrity of

employees

Consumer

Efficiency and effectiveness of employees

Empathy and communication


n/a

fo

rC

SMS

–n

ew a

ttri

bu

te

intr

od

uce

d f

rom

Q4

20

17

QP

CS



7.3

7.3

8.0

7.8

7.5

6.9 7

.5

7.57.6

7.4

8.1

7.9

7.7

7.4

7.3

8.0

7.9

7.6

7.5 7.5

8.1

7.9

7.7

The process was simple and efficient Employees took initiative and madedecisions

My privacy was upheld & personalinformation was protected & respected

I understood the steps involved with theprocess

I had good access to information andcould find what I needed


3.1 Impact of Satisfaction Drivers – Consumer (2/2)D

eriv

ed D

rive

rs

(CSM

S 2

01

8)


Process Goals

22

n/a

–N

ot

a C

SMS

attr

ibu

te

Simplicity and Efficiency of Processes Employee Autonomy Privacy Transparency Access to information

n/a

–N

ot

a C

SMS

attr

ibu

te

Consumer




For process and goal drivers in Q2 2019, employee autonomy experienced a statistically significant increase since Q1 2019. When compared to the same time last year (Q2 2018), employee autonomy and access to information experienced a statistically significant increase.


Ave

rage

(o

ut

of

10

)

8.0

7.5 7.7

7.6 7.7

7.4

6.7

7.5

7.1 7

.4

7.3

7.0

7.9

7.7

7.3 7

.8

7.5 7.8

7.48

.0

7.8

7.5 7

.8

7.7 7.9

7.67

.9

7.7

7.4 7

.8

7.6 7.7

7.5

Employees were open andhonest during the process

Employees acted efficientlyand effectively to reach the

right outcomes

Employees were heldaccountable for their actions

Communications were clear,prompt and easy to

understand

Employees acted withempathy

I was provided with goodservice and outcomes I could

trust

I felt there was accountabilityfor services delivered


3.2 Impact of Satisfaction Drivers – Business (1/2)D

eriv

ed D

rive

rs

(CSM

S 2

01

8)


Business

Employees Values

23

n/a

fo

r Q

2 2

01

8 –

new

att

rib

ute

in

tro

du

ced

fro

m Q

4 2

01

8 Q

PC

S

Service Quality AccountabilityCustomer focus &

action orientedClear communicationIntegrity and high standards

n/a

for

CSM

S –

new

att

rib

ute

in

tro

du

ced

fro

m Q

4 2

01

7 Q

PC

S




In Q2 2019, there was no statistically significant changes for the drivers since Q1 2019. Similarly, when compared to the same time last year (Q2 2018) all drivers did not experience any statistically significant movement.


Ave

rage

(o

ut

of

10

)

7.4

7.4

8.2

7.9

7.7

6.8 7

.3 7.47.5

7.5

8.1

8.0

7.7

7.6

7.6 8

.1

8.0

7.8

7.5

7.5

8.1

8.0

7.7

The process was simple and efficient Employees took initiative and madedecisions

My privacy was upheld & personalinformation was protected & respected

I understood the steps involved with theprocess

I had good access to information andcould find what I needed


3.2 Impact of Satisfaction Drivers – Business (2/2)D

eriv

ed D

rive

rs

(CSM

S 2

01

8)


Business

Process Goals

24

n/a

–N

ot

a C

SMS

attr

ibu

te

Simplicity and Efficiency of Processes

Employee Autonomy Privacy Transparency Access to information

n/a

–N

ot

a C

SMS

attr

ibu

te




In Q2 2019, there was no statistically significant changes for process and goals related drivers since Q1 2019. Similarly, when compared to the same time last year (Q2 2018) these drivers did not experience any statistically significant movement performance of these drivers have stabilised after experience significant increases over the last year.


Ave

rage

(o

ut

of

10

)

3.3 Key Drivers of Satisfaction – Qualitative Research

Employee drivers support the increase in satisfaction Quality of service and accountability drive satisfaction

We were greeted nicely and the client I was assisting was

treated with the utmost respect …everything was handled

professionally, quickly, respectfully, no double

handling and I was impressed

“


Consumer

25

”

I was emailed with a written response confirming receipt of my written complaint with a reference number which outlined when I would

receive a call…After speaking to them I received a further email outlining what steps to take and when I would be contacted again and then

received a further phone call asking if I needed anymore help or advice

“

”

• Especially important in emergency and social services, consumers feel

that empathetic employees who are open and honest helped improve

satisfaction. This is crucial as the most important* points throughout the

process as highlighted by consumers and businesses, relate to face to

face interactions rather than online channels

• Further support is seen via negative experiences with employees who

lacked product knowledge. This contributed to dissatisfaction during an

experience, with many consumers stating this as a reason for making it

difficult to build trust with the service

• However, there are differences between consumers and businesses.

Empathy and communication as well as honesty and integrity is

commonly addressed as a key driver of satisfaction whilst, efficiency and

effectiveness is a key theme for businesses. This is due to the

importance of time and need for efficient processes within businesses

Business

• Quality of service is key to improving satisfaction. For

consumers, having simple language and being guided through

the process helps ease the interaction with government

services. For businesses, the convenience of being able to

complete multiple processes helps to decrease effort

• For consumers, accountability has also contributed to

satisfaction, with customers that interacted with accountable

staff and services having greater trust and satisfaction. Being

able to provide status updates via a preferred channel has also

been important to increasing satisfaction

• Customers showed positivity towards services that provided

status updates on time, via their preferred channel and

delivered on what was promised

Staff training so that staff can find the information they need in the system…in

timely fashion would be another driver. I think if processes are designed to be time efficient, simple and require the minimum

level of updates be an essential precursor to achieving customer satisfaction and trust

“

””*Importance is commonly defined by stages within a process which require the most effort from both respondents and government services

4. Channel Usage and Preference

4.1 Channel Usage and Preference - Consumer

4.2 Channel Usage and Preference - Business

4.3 Impact of Channel Expectations – Qualitative Research

26

36

%

20

%

21

%

11

%

9%

4%

41

%

20

% 22

%

9%

5%

2%

38

%

20

%

18

%

11

%

7%

3%

34

%

19

%

21

%

12

%

7%

4%

35

%

20

% 22

%

11

%

6%

3%

41

%

19

% 23

%

9%

5%

2%

36

%

21

%

19

%

13

%

7%

4%

35

%

21

%

19

%

14

%

7%

4%

35

%

22

%

19

%

12

%

8%

4%

39

%

20

% 23

%

10

%

6%

3%

38

%

20

%

18

%

13

%

8%

3%

40

%

21

%

18

%

11

%

7%

4%

38

%

19

%

20

%

13

%

7%

3%

0%

10%

20%

30%

40%

50%

Face to face Telephone Online Email Mail, posted letter, fax Third parties such as Australia Post

CSMS 2015 CSMS 2016 Q4 2016 Q1 2017 Q2 2017 CSMS 2017 Q4 2017 Q1 2018 Q2 2018 CSMS 2018 Q4 2018 Q1 2019 Q2 2019

39

%

15

%

28

%

10

%

3%

1%

32

%

19

%

25

%

15

%

4%

1%

40

%

17

% 21

%

13

%

2%

1%

41

%

17

%

24

%

13

%

2%

1%

38

%

17

%

24

%

14

%

3%

1%

35

%

18

%

28

%

15

%

4%

1%

38

%

18

%

19

%

15

%

3%

3%

39

%

19

% 22

%

15

%

2%

1%

40

%

17

% 22

%

13

%

3%

1%

33

%

18

%

25

%

16

%

4%

1%

41

%

18

% 20

%

14

%

2%

1%

41

%

17

% 20

%

13

%

3%

2%

40

%

16

%

23

%

14

%

3%

1%

0%

10%

20%

30%

40%

50%



4.1 Channel Usage and Preference - Consumer

Contact Methods Used*

Q: Which of the following contact methods have you used to carry out your direct dealings with [SERVICE] in NSW in the last 6 months? (Multi-select)

* Contact methods used is a multi response question; Figures in the graph does not include the response for “None of the above” option. Channel usage has been rebased on total number of responses for comparison to channel preference** Contact methods preferred is a single response question; Figures in the graph does not include the response for “Don’t kno w/ can’t say” and therefore, may not add up to 100%

Shar

e o

f co

nta

ct m

eth

od

use

d (%

)Sh

are

of

pre

fere

nce

(%) n=1,600

n=1,003

Consumer

27


Q: Which of the following contact methods do you most prefer to use when dealing directly with [SERVICE] in NSW in the last 6 months? Contact Methods Preference**

For consumers, usage of all contact methods did not experience any notable movements, with all channels increasing or decreasing by 1%-2% when compared to Q1 2019. Face to face remains the most used channel among this group and the percentage of respondents interacting face to face has increased by 3% since Q2 2018. Face to face also remains the most preferred contact channel for consumers when dealing with NSW Government services since 2015. Consumers who interact face to face have the highest expectation and satisfaction with these channels. This is also a historical trend.

27

%

24

%

20

%

15

%

10

%

4%

29

%

27

%

20

%

16

%

7%

2%

24

%

26

%

22

%

15

%

10

%

3%

29

%

22

%

19

%

15

%

9%

5%

30

%

16

%

25

%

16

%

9%

4%

27

%

25

%

19

%

20

%

7%

2%

24

%

26

%

17

%

19

%

9%

6%

32

%

20

%

19

%

16

%

9%

4%

34

%

20

%

20

%

14

%

8%

3%

28

%

24

%

19

%

19

%

6%

4%

40

%

20

%

18

%

12

%

6%

4%

30

%

22

%

18

%

18

%

8%

4%

30

%

22

%

20

%

17

%

7%

4%

0%

10%

20%

30%

40%

50%



38

%

20

% 24

%

13

%

3%

1%

27

%

21

%

23

%

23

%

3%

1%

26

%

24

% 27

%

16

%

3%

1%

33

%

20

%

22

%

17

%

3%

2%

35

%

24

%

16

% 20

%

3%

1%

27

%

24

%

19

% 25

%

4%

1%

33

%

24

%

19

%

11

%

7%

2%

37

%

17

%

25

%

16

%

2%

1%

37

%

17

% 22

%

18

%

3%

1%

27

%

24

%

21

%

22

%

4%

2%

41

%

16

% 21

%

14

%

3%

1%

33

%

21

%

19

%

18

%

3%

2%

34

%

21

%

22

%

18

%

2%

1%

0%

10%

20%

30%

40%

50%



4.2 Channel Usage and Preference - Business

Contact Methods Used*

Q: Which of the following contact methods have you used to carry out your direct dealings with [SERVICE] in NSW in the last 6 months? (Multi-select)

* Contact methods used is a multi response question; Figures in the graph does not include the response for “None of the above” option. Channel usage has been rebased on total number of responses for comparison to channel preference** Contact methods preferred is a single response question; Figures in the graph does not include the response for “Don’t kno w/ can’t say” and therefore, may not add up to 100%

Shar

e o

f co

nta

ct m

eth

od

use

d (%

)Sh

are

of

pre

fere

nce

(%)

n=1,394

n=1,009

Business

28

Note: Figures in the graph are subject to rounding1Indicative only - does not indicate statistical significance.

Q: Which of the following contact methods do you most prefer to use when dealing directly with [SERVICE] in NSW in the last 6 months?Contact Methods Preference**

Face to face is the most used channel for businesses and its usage has remained stable since Q1 2019. The use of other contact methods have remained similar to Q1 2019. Businesses prefer face to face contact and this preference has increased by 1% since Q1 2019. In Q2 2019, businesses’ preference for online has increased1 by 2% compared to Q1 2019, while preference for telephone and email have remained stable. Similar to consumer results, businesses who interact face to face or online have the highest levels of satisfaction and expectation.

4.3 Impact of Channel on Expectations – Qualitative Research

• Face to face interactions help to support satisfaction as staff can

provide a highly empathetic and personal service. Comprehensive

product knowledge was identified as key to these staff interactions

• In addition, great service quality by having the right supportive

infrastructure is also important. This creates efficiencies during the

process, such as Roads where there are efficient ticketing systems,

helping to exceed expectations

29

Face to face interactions support satisfaction via employee driversThe fact that the service assistant did as she said she would made me trust the service provider and the process in that they did what

they say they would, and therefore increased my satisfaction. It meant that I did not have to chase for an answer.

I feel like the staff were friendly (and) patient…and wanted to do all they could to get a positive outcome for me. I feel it really made

me trust the process more when I was getting calls from the same person as I feel like had a genuine interest in the process

“

“ ”

”


• Customers acknowledge that NSW Government online processes

have contributed to the satisfaction and expectation of government

services due to greater convenience and time efficiencies. Having

multiple channels allows them to receive status updates and conduct

dealings via their preferred channel, reducing the need to travel to a

branch or explain complications over the phone

• Businesses commonly initiate processes via an online channel and

only interact face to face if necessary. This is creates duplications and

time inefficiencies, which reduces customer satisfaction. Hence,

online channels need to be seamlessly integrated with offline

channels through constant communication

Online channels need to be integrated seamlessly to reduce effort

Just to complete one process. It makes me question whether these departments…actually communicate with one another.

“

”

I found that the NSW Service centre has a seamless process in place which all staff and customers appear to benefit from,

everything was handled professionally, quickly, respectfully, no double handling and I was impressed with it as a whole and they

could be used as a benchmark for other NSW Govt Offices / Departments.

“

”

Transitioning to an App based notification would work...such an example would be a great customer experience as it consolidates all communications within one platform, speeds up the receipt of

communications

“

”

Consumer Business

Appendix

Appendix A: Demographic Profile of Respondents

Appendix B: Background to the QPCS

Appendix C: QPCS Qualitative and Quantitative Research

Appendix D: Historical Consumer and Business Baseline Measures

Appendix E: Customer Satisfaction Index Q1 2019 QPCS (CSMS at 99% Confidence Level)

Appendix F: Terminology Definitions

30

Appendix A: Demographic Profile of Respondents - Consumer

Consumer Respondent Profile*

49% male

51% femaleGender: Age: Region:

18-3431% 35-54

33%

55-6415%

65+20%

Q2 2019 Consumers (n=1,003 respondents)

75% Metro

19% Regional

6% Rural

27% 25%

11% 10% 9%6%

3% 3% 4%

0%

10%

20%

30%

Employment Status:

17%20%

26%

13%

5% 5%

14%

0%

10%

20%

30%

Up to$30,000

$30,001 to$50,000

$50,001 to$100,000

$100,001 to$150,000

$150,001 to$180,000

Over$180,001

Prefer not tosay/Don't

know

Annual Income:

49% male

51% femaleGender: Age:

18-3431% 35-54

33%

55-6415%

65+20%

Region: 75% Metro

19% Regional

6% Rural

Employed full time

Retired Full-time domestic

duties

Unemployed Employed part time

Student Employed on a casual

basis

Other

33%

24%

11% 9% 7% 6% 5% 3% 3%

0%

10%

20%

30%

40%

Employment Status:

Employed full time

Retired Employee part time

Full-time domestic

duties

Unemployed Student Employedon a

casual basis

Self-employed /

business owner

Other

17% 16%

26%

17%

5% 5%

14%

0%

10%

20%

30%

Up to$30,000

$30,001 to$50,000

$50,001 to$100,000

$100,001 to$150,000

$150,001 to$180,000

Over$180,001

Prefer not tosay/Don't

know

Annual Income:

• Data is weighted to be representative of the NSW population (ABS) based on gender, age and region

Q1 2019 Consumers (n=1,007 respondents)

Consumer

31

Self-employed /

business owner

Note: Figures in the graph are subject to rounding and may not sum to 100%

Appendix A: Demographic Profile of Respondents - Business

Business Respondent Profile*

47% male

53% female

Gender: Region:

Q2 2019 Business (n=1,009 respondents)

75% Metro

19% Regional

6% Rural

48% male

Gender: Region: 75% Metro

19% Regional

6% Rural

52% female

Business size:

Business size:

Industry:

12%9% 9%

7% 7% 6% 5% 4% 4% 4% 3% 3% 3% 3% 2% 1% 0.8% 0.7%

15%

-2%

2%

6%

10%

14%

18%

Pro

fess

ion

al,

scie

nti

fic a

nd

tech

nic

al s

ervi

ces

Ret

ail T

rad

e

Hea

lth

car

e an

dso

cial

ass

ista

nce

Educ

atio

n an

dtr

ain

ing

Co

nst

ruct

ion

Art

s an

dre

cre

atio

n s

ervi

ces

Acc

omm

od

atio

nan

d f

oo

d se

rvic

es

Fina

nci

al a

ndin

sura

nce

ser

vice

s

Ren

tal,

hiri

ng a

nd

real

est

ate

ser

vice

s

Agr

icul

ture

,fo

rest

ry a

ndfi

shin

g

Ad

min

istr

ativ

e a

ndsu

pp

ort

ser

vice

s

Man

ufa

ctu

rin

g

Who

lesa

le T

rade

Tran

spo

rt, p

osta

lan

d w

areh

ousi

ng

Info

rmat

ion

med

iaan

dte

leco

mm

uni

cati

on

s

Pub

licad

min

istr

atio

n a

nd

safe

ty

Ele

ctri

city

, gas

,w

ater

an

d w

aste

serv

ices M

inin

g

Oth

ers

Industry:

Q1 2019 Business (n=1,003 respondents)

• Data is weighted to be representative of the NSW population (ABS) based on business size and region

20+2%

Sole Proprietor

29%2-949%

10-1920%

20+3%

Sole Proprietor

31%2-944%

10-1922%

Business

32

9%12%

10%6% 7%

4% 5%3% 3% 3% 4% 5%

3% 2%4%

1% 0.5% 0.5%

17%

-2%

2%

6%

10%

14%

18%

Pro

fess

ion

al,

scie

nti

fic a

nd

tech

nic

al s

ervi

ces

Ret

ail T

rad

e

Hea

lth

car

e an

dso

cial

ass

ista

nce

Educ

atio

n an

dtr

ain

ing

Co

nst

ruct

ion

Art

s an

dre

cre

atio

nse

rvic

es

Acc

omm

od

atio

nan

d f

oo

d se

rvic

es

Fina

nci

al a

ndin

sura

nce

serv

ices

Ren

tal,

hiri

ng a

nd

real

est

ate

serv

ices

Agr

icul

ture

,fo

rest

ry a

ndfi

shin

g

Ad

min

istr

ativ

ean

d s

up

port

serv

ices

Man

ufa

ctu

rin

g

Who

lesa

le T

rade

Tran

spo

rt, p

osta

lan

d w

areh

ousi

ng

Info

rmat

ion

me

dia

and

tele

com

mu

nica

tio

ns

Pub

licad

min

istr

atio

nan

d s

afet

y

Ele

ctri

city

, gas

,w

ater

an

d w

aste

serv

ices M

inin

g

Oth

ers


Appendix B: Background - Research Scope and Approach

• The QPCS Methodology is aligned to the Annual Customer Satisfaction

Measurement Survey (CSMS) approach:

• Captures feedback across 23 different NSW Government services (described

in the customers language).

• Feedback received from customers about each of the individual services is

aggregated to provide a view of the performance of NSW Government

services overall.

• Each respondent provides feedback regarding one or two services (as a

result, the total number of responses received across services is greater

than the total number of customers who completed the survey).

• The survey was completed from 15th April 2019 to 23rd April 2019 and results are

therefore reflective of experiences with services over the six months prior i.e. from

October 2018 to April 2019.

• The Q2 2019 QPCS was completed with:

• N = 1,003 consumers, and

• N = 1,009 businesses

• As each respondent provides feedback regarding one or two services, the Q2 2019

QPCS number of responses:

• N = 1,600 for consumers, and

• N = 1,394 for businesses

• All scores reported in this document are out of 10, with the exception of the

Customer Satisfaction Index which is out of 100.

In scope services

Industry• Agricultural Advice and

Funding Services• Business Advisory Services • Water Supply• TAFE Services

Justice • Police • State Emergency Services • Prisons • Courts • Fire Brigades

Family & Community Services • Public Housing • Disability Services • Child Protection Services• Services for Older People

Transport• Public Transport • Car and Boat Registration • Major Roads

Finance, Services & Innovation

• Consumer Affairs (Fair Trading)

Planning & Environment

• Environment and Wildlife Protection

• Art Galleries and Museums

Education

• Public Schools

Health

• Public Hospitals

• Ambulance Services

Multiple clusters

• Documentation Services (including certificates for births deaths and marriages; trade licenses and certificates; and drivers licenses)

Consumer Business

33

General Considerations:

• The QPCS results do not replace the Annual CSMS results, but rather provide

a directional indication of the shift in the results.

• Although the QPCS sample characteristics are closely representative of the

NSW population, different customers have been surveyed and as such the

results are directional indicators of shifts in the Annual CSMS results only.

• The margin of error (MoE) for the QPCS needs to be considered when

interpreting the results.

Considerations for interpreting the QPCS data points:

• The QPCS results need to be interpreted in the context of the time of the year and in light of events in order to normalise seasonal trends in the data.

Therefore, overall caution should be taken when interpreting the QPCS findings until a minimum of a full year of results has been collected, so that any

seasonal impacts can be examined and adjusted accordingly.

• Contextual factors for the results (Q2 2019) have been considered and include factors such as train delays and strikes, stadium upgrades and light rail

construction. Contextual factors are further explored as part of the analysis of the summary report.

• In the slides, the results of Q2 2019 QPCS have been compared to the results of all QPCS starting from Q4 2016 and CSMS starting from 2015. Significance

testing is based on the comparison to Q2 2018 and Q1 2019 results at 95% confidence level. We have allowed the longitudinal trend analysis for CSMS to

QPCS having applied a 95% confidence level to both CSMS and QPCS datasets.

• The Annual CSMS results have been provided as additional context for the QPCS data point and should not be used as a comparison to QPCS results.

• A longitudinal dataset will need to be built over time in order to identify 'real' trends in the QPCS results and to strengthen the reliability and validity of any

conclusions drawn.

Q12017

Q22017

2017CSMS

Q42017

Q12018

Q22018

CSMS2018

Q42018

Q12019

Q22019

1. Set a benchmark with the first data point

Interpretation Plan for Tracking Study

2. Draw insights by comparing to previous quarter

3. Form a directional trend

4. Develop a lead indicator

We are hereTime

Mea

sure

ILLUSTRATION ONLY - NOT REAL DATA

Appendix B: Background - Key Considerations for Interpreting QPCS Insights

34

Appendix C: QPCS Qualitative and Quantitative Research - ApproachQ2 2019 QPCS Qualitative Research:

Qualitative research complements the QPCS survey results by providing further insights and context into the results of the quantitative survey. In Q2 2019, the qualitative research will provide insight into the Special Interest Topic – Process and status transparency.

An online discussion forum was used as the qualitative research approach in Q2 2019 QPCS. There were 2 online discussion forums, a consumer group and a business group. Participants from each group were recruited from the respondents of Q2 2019 QPCS survey with a mix of demographics. The 3-day online discussion forums ran from 13th to 15th May 2019 for the consumer group and 15th to 21st May 2019for the business group. Participants were required to answer all the pre-designed questions and encouraged to comment on other people’s posts. Moderators monitored the two forums and follow up questions were posted which prompted to participants to make sure sufficient insights were captured. The final responses came in both text and video formats which provided an in-depth understanding of baseline measures of Q2 2019 and the SIT topic.

35

The approach undertaken to assess changes between the CSMS results and the QPCS topline results is outlined below:

• In order to compare the CSMS topline results against the QPCS topline results, the confidence level of CSMS (typically 99%) has been adjusted to 95% confidence level (in line with that of the QPCS). This was undertaken in order to make the significance testing comparable between CSMS and QPCS results

• In doing this however, it should be noted that the survey methodology differs for CSMS versus QPCS in the following ways, and should be interpreted with caution:

• Respondents are asked about their direct dealings with a service reflecting on their last 12 months in CSMS but the last 6 months in the QPCS, therefore recall about their experience differs

• Sample size varies greatly i.e. 2018 CSMS sample size was n=6,733 versus 2018 Q4 QPCS sample size of n=1,545 for consumers (this can impact margin of error which directly impacts significance testing)

• 99% confidence interval is used to test significance of results for the CSMS results versus 95% confidence interval which is used to test significance of the QPCS results

Q2 2019 QPCS Quantitative Research:

Appendix C: QPCS Qualitative Research - Overview of Online Discussion Forum Approach

Approach:

• Participants were recruited from the respondents of Q2 2019 QPCS survey. Participants come from backgrounds with a mix of age, gender, location and experience interacting with NSW Government services

• The online discussion forums ran from 13th to 15th May 2019 for the consumer group and 15th to 21nd May 2019 for the business group.

• Participants were asked to provide insights into their experiences around process and status transparency.

Structure of the Discussion Guide

Introduction

Topics

Changes in experience over the past few months

Discussion on process transparency

Discussion on status transparency

Process Transparency Improvements

Storyboard of Process Transparency

Group Number of Respondents

Consumer 22

Business 16

Storyboard of Status Transparency

Best practice from real-world examples

Day 1

Day 3

Day 2

36

Activity Three: Drawing process transparency best practice from other real-world examples

Using examples of tools used to track the status of a process, we will seek to understand what specific features and attributes are important and useful for customers.

Customers can also reflect on how these aspects would improve their experience if applied to a recent interaction with a NSW Government service.

Activity Two: Storyboard of Process Transparency

Participants will recall their journey, pain points, and moments of delight from a NSW Government process they experienced. This can be a linear path or a storyboard.

Following this, participants will add in key points of status transparency

Activity One: Unpacking the quantitative insights

Using the quantitative questions and responses as a base, we will ask the participants questions around the following:

• What makes a process transparent and why?

• How are participants kept informed of their status and how does this impact their satisfaction with the service overall?

Appendix D: Overview of Historical Results

Results

at a glance

Consumers

CSMS

2015

CSMS

2016Q4 2016 Q1 2017 Q2 2017

CSMS

2017Q4 2017 Q1 2018 Q2 2018

CSMS

2018Q4 2018 Q1 2019 Q2 2019

Satisfaction 7.3 7.5 7.4 7.6 7.6 7.7 7.7 7.7 7.7 7.6 7.8 7.7 7.8

Expectation 7.7 7.8 7.8 7.9 7.8 7.9 7.8 7.9 7.9 7.9 8.0 7.8 8.0

Ideal

Service6.9 7.2 7.2 7.3 7.2 7.2 7.3 7.2 7.3 7.2 7.3 7.2 7.4

CSI 76.7 78.7 77.8 79.5 78.6 79.3 80.2 79.9 79.6 78.9 79.9 79.4 80.9

Results

at a glance

Businesses

CSMS

2015

CSMS

2016Q4 2016 Q1 2017 Q2 2017

CSMS

2017Q4 2017 Q1 2018 Q2 2018

CSMS

2018Q4 2018 Q1 2019 Q2 2019

Satisfaction 7.2 7.5 7.2 7.4 7.3 7.5 7.5 7.6 7.8 7.4 7.9 7.8 7.8

Expectation 7.6 7.9 7.6 8.1 7.5 7.7 7.6 7.7 8.1 7.7 8.0 8.0 7.9

Ideal

Service7.0 7.3 7.1 7.4 7.1 7.1 7.0 7.2 7.3 7.2 7.4 7.6 7.4

CSI 76.6 78.4 76.1 77.6 77.6 78.3 77.8 79.6 79.7 78.2 81.3 81.2 80.7

Consumer Business




Note: CSI is out of 100; all other measures represent scores out of 10

Appendix D: Satisfaction - Consumer


Average (out of 10)

% D

istr

ibu

tio

n (

acro

ss lo

w,

neu

tral

an

d h

igh

sco

res)

10% 9% 10% 7% 7% 8% 8% 8% 9% 8% 8% 9% 8%

19% 16% 17% 16% 17% 14% 13% 14% 13% 15% 12% 15% 13%

71% 76% 73% 76% 76% 78% 78% 78% 78% 77% 80% 77% 80%

0%

20%

40%

60%

80%

100%

CSMS 2015(n=6,790)

CSMS 2016(n=7,227)

Q4 2016(n=1,652)

Q1 2017(n=1,580)

Q2 2017(n=1,647)

CSMS 2017(n=6,789)

Q4 2017(n=1,541)

Q1 2018(n=1,651)

Q2 2018(n=1,638)

CSMS 2018(n=7,000)

Q4 2018(n=1,583)

Q1 2019(n=1,611)

Q2 2019(n=1,577)

Low (1-4) Med (5-6) High (7-10)

Consumer

7.3 7.5 7.4 7.6 7.6 7.7 7.7 7.7 7.7 7.6 7.8 7.7

CSMS results

38




7.8

Appendix D: Expectation - Consumer


Average (out of 10)

% D

istr

ibu

tio

n (

acro

ss lo

w,

neu

tral

an

d h

igh

sco

res)

7% 7% 6% 5% 6% 6% 7% 6% 7% 6% 6% 8% 5%

16% 13% 13% 13% 14% 12% 14% 13% 12% 13% 11% 12% 13%

78% 80% 81% 82% 80% 82% 79% 81% 81% 81% 84% 80% 82%

0%

20%

40%

60%

80%

100%

CSMS 2015(n=6,693)

CSMS 2016(n=7,140)

Q4 2016(n=1,636)

Q1 2017(n=1,563)

Q2 2017(n=1,624)

CSMS 2017(n=6,732)

Q4 2017(n=1,537)

Q1 2018(n=1,644)

Q2 2018(n=1,642)

CSMS 2018(n=6,922)

Q4 2018(n=1,579)

Q1 2019(n=1,614)

Q2 2019(n=1,578)

Low (1-4) Med (5-6) High (7-10)

Consumer

7.7 7.8 7.8 7.9 7.8 7.9 7.8 7.9 7.9 7.9 8.0 7.8

CSMS results

39




8.0

Appendix D: Comparison to Ideal - Consumer


Average (out of 10)

% D

istr

ibu

tio

n (

acro

ss lo

w,

neu

tral

an

d h

igh

sco

res)

13% 12% 12% 9% 12% 11% 11% 12% 12% 11% 13% 13% 11%

22%18% 19% 19% 18% 19% 17% 19% 17% 20% 16% 16% 14%

65% 70% 69% 72% 69% 70% 72% 70% 71% 69% 72% 71% 75%

0%

20%

40%

60%

80%

100%

CSMS 2015(n=6,593)

CSMS 2016(n=7,015)

Q4 2016(n=1,621)

Q1 2017(n=1,548)

Q2 2017(n=1,605)

CSMS 2017(n=6,559)

Q4 2017(n=1,518)

Q1 2018(n=1,628)

Q2 2018(n=1,618)

CSMS 2018(n=6,733)

Q4 2018(n=1,545)

Q1 2019(n=1,590)

Q2 2019(n=1,564)

Low (1-4) Med (5-6) High (7-10)

Consumer

6.9 7.2 7.2 7.3 7.2 7.2 7.3 7.2 7.3 7.2 7.3 7.2

CSMS results

40




7.4

7.5

Appendix D: Satisfaction- Business


% D

istr

ibu

tio

n (

acro

ss lo

w,

neu

tral

an

d h

igh

sco

res)

Average (out of 10)

9% 7%15%

7% 11% 9% 8% 5% 9% 7% 5% 9% 8%

18% 18%15%

24% 17% 17% 19% 22% 11% 18% 16% 12% 14%

72% 74% 70% 69% 72% 75% 73% 73%80% 75% 79% 79% 79%

0%

20%

40%

60%

80%

100%

CSMS 2015(n=1,700)

CSMS 2016(n=1,761)

Q4 2016(n=382)

Q1 2017(n=391)

Q2 2017(n=372)

CSMS 2017(n=1,705)

Q4 2017(n=349)

Q1 2018(n=375)

Q2 2018(n=1,570)

CSMS 2018(n=1,555)

Q4 2018(n=1,553)

Q1 2019(n=1,469)

Q2 2019(n=1,387)

Low (1-4) Med (5-6) High (7-10)

Business

7.2 7.4 7.3 7.5 7.5 7.6 7.8 7.4 7.9 7.8

CSMS results

41

7.2




7.8

7.9

Appendix D: Expectation - Business


% D

istr

ibu

tio

n (

acro

ss lo

w,

neu

tral

an

d h

igh

sco

res)

Average (out of 10)

8% 5% 8% 3%10% 6% 6% 6% 6% 6% 5% 6% 5%

15%14% 14%

11%

16%16% 17% 16% 9% 15% 14% 11% 14%

77% 81% 79%86%

74% 77% 77% 78%84% 79% 82% 83% 81%

0%

20%

40%

60%

80%

100%

CSMS 2015(n=1,677)

CSMS 2016(n=1,738)

Q4 2016(n=381)

Q1 2017(n=390)

Q2 2017(n=370)

CSMS 2017(n=1,682)

Q4 2017(n=350)

Q1 2018(n=372)

Q2 2018(n=1,555)

CSMS 2018(n=1,531)

Q4 2018(n=1,552)

Q1 2019(n=1,465)

Q2 2019(n=1,389)

Low (1-4) Med (5-6) High (7-10)

Business

7.6 8.1 7.5 7.7 7.6 7.7 8.1 7.7 8.0 8.0

CSMS results

42

7.6




7.9

7.3

Appendix D: Comparison to Ideal - Business


% D

istr

ibu

tio

n (

acro

ss lo

w,

neu

tral

an

d h

igh

sco

res)

Average (out of 10)

14% 9% 15% 9% 14% 11% 10% 7% 12% 11% 9% 8% 10%

18%20% 14%

17%18% 20% 24%

20% 15% 19% 18% 13%17%

67% 71% 71% 73% 68% 69% 66%73% 73% 71% 73% 79% 73%

0%

20%

40%

60%

80%

100%

CSMS 2015(n=1,654)

CSMS 2016(n=1,718

Q4 2016(n=375)

Q1 2017(n=380)

Q2 2017(n=365)

CSMS 2017(n=1,646)

Q4 2017(n=345)

Q1 2018(n=367)

Q2 2018(n=1,554)

CSMS 2018(n=1,506)

Q4 2018(n=1,535)

Q1 2019(n=1,455)

Q2 2019(n=1,378)

Low (1-4) Med (5-6) High (7-10)

Business

7.1 7.4 7.1 7.1 7.0 7.2 7.3 7.2 7.4 7.6

CSMS results

43

7.0




7.4

76.6

78.4 78.379.6 79.7

78.2

81.3 81.2 80.7

CSMS2015

(n=1,645)

CSMS2016

(n=1,712)

CSMS2017

(n=1,638)

Q12018

(n=365)

Q22018

(n=1,536)

CSMS2018

(n=1,494)

Q42018

(n=1,523)

Q12019

(n=1,449)

Q22019

(n=1,372)

Consumers – CSI

Businesses – CSI

Consumers – Baseline Measures

Businesses – Baseline Measures

Margin of Error (QPCS)Consumer CSI: ± 1.3

Margin of Error (QPCS)Business CSI: ± 1.2

Note - Sample size for CSI and Baseline measure may differ on the basis of “Don’t Know” option selection

Note – Significance testing between CSMS and QPCS is not possible due to differing confidence intervals

76.7

78.7 79.3 79.9 79.678.9

79.9 79.480.9

CSMS2015

(n=6,549)

CSMS2016

(n=6,971)

CSMS2017

(n=6,527)

Q12018

(n=1,610)

Q22018

(n=1,603)

CSMS2018

(n=6,701)

Q42018

(n=1,535)

Q12019

(n=1,575)

Q22019

(n=1,555)

7.3

7.57.7 7.7 7.7 7.6

7.87.7

7.87.7

7.87.9 7.9 7.9 7.9 8.0

7.88.0

6.9

7.2 7.2 7.2 7.37.2

7.3 7.2

7.4

6.6

7.1

7.6

8.1

8.6

CSMS2015

(n=6,593)

CSMS2016

(n=7,015)

CSMS2017

(n=6,559)

Q12018

(n=1,628)

Q22018

(n=1,618)

CSMS2018

(n=6,733)

Q42018

(n=1,545)

Q12019

(n=1,590)

Q22019

(n=1,564)


7.27.5 7.5

7.67.8

7.4

7.9 7.8 7.87.6

7.97.7 7.7

8.1

7.7

8.0 8.07.9

7.0

7.37.1

7.27.3 7.2

7.4

7.67.4

6.6

7.1

7.6

8.1

8.6

CSMS2015

(n=1,654)

CSMS2016

(n=1,718)

CSMS2017

(n=1,646)

Q12018

(n=367)

Q22018

(n=1,554)

CSMS2018

(n=1,506)

Q42018

(n=1,535)

Q12019

(n=1,455)

Q22019

(n=1,378)


Appendix E: Customer Satisfaction Index Q2 2019 QPCS (CSMS Sig-test at 99% significance level)Consumer Business



Statistically significant movement from the previous CSMS year at 99% confidence level


45

To ensure consistency in reporting significance testing, the below table of definitions has been developed and will be used for all future reporting for CSMS and QPCS.

Appendix F: Terminology Definitions

Terminology Definition Example

Remain StableRounded numerical difference between current period result and the previous period result is less than 0.1 (in line with the chart visualisations).

Satisfaction score moved from 8.07 to 8.12 (i.e. actual difference is 0.05). In this case, rounded values are both 8.1 thus the results remained stable

Increase/DecreaseRounded numerical difference between current period outcome measures and the previous period result is larger than or equal to 0.1 (in line with the chart visualisations).

Satisfaction score moved from 8.07 to 8.21 (i.e. actual difference is 0.14). In this case, the rounded value moved from 8.1 to 8.2 thus the result increased.

Statistically significantStatistically significant movement (decrease or increase) from the previous period at 95% confidence level

Not statistically significantNo statistically significant movement (decrease or increase) detected from the previous period at 95% confidence level

Visual differenceRounding numbers up to one decimal place and updating commentary based on visible difference in charts

Satisfaction score increased from 7.66 to 7.84 (i.e. actual difference is 0.18. In this case rounded values are 7.7 and 7.8, thus difference will be stated as increase of 0.1

2019 Q2 Quarterly Pulse Check Survey (QPCS) · 2020. 3. 11. · is close to meeting these...

Documents

Transcript of 2019 Q2 Quarterly Pulse Check Survey (QPCS) · 2020. 3. 11. · is close to meeting these...