Managing for organisational performance: What information do state sector managers actually rely on?...

Managing for organisational performance:

What information do state sector managers actually rely on?

Joint Seminar

School of Government, Victoria UniversityInstitute of Public Administration of New Zealand

15 & 21 May 2009

A first report on results from the Managing for Performance (M4P) project in the Chief Executives’ Emerging Issues Programme (EIP)

Derek Gill and Rob Laking

Aim of the session

Present results from a survey of state sector managers on use of information for organisational performance management:– Headline results of interest– Preliminary analysis of drivers of use of

information Supplement analysis with observations from

interview-based case studies Invite questions and comments

Historical context

We have had output based accrual budgeting and reporting for twenty years

Output budgeting is resilient but: – A lot of non–financial performance measures are

still “crap” <C&AG 2008> after nearly 20 years; – Managing for Outcomes has been a “failure”

(along with KRAs/SRAs and outcome reporting in the old Public Finance Act)

Rob Laking

What's this?

Project to date

Goals– Positive - find what information do politicians managers and

front line staff actually use to decide what to do and what, if any, use is made of the current performance information that they receive.

– Normative –develop a proposal for a requisite system that better aligns key users’ wants and experiences and make recommendations for the direction of change

Progress– Completed fieldwork and analysis for seven case studies:

two networks, five agencies; now writing up results– Completed questionnaire phase and most analysis of

results of survey of 2500 state sector managers

Survey sample

Invitations to 2,500 managers in twelve Public Service departments (covering 60% of Public Service) and five Crown Entity Agents;

75% started the survey and nearly 70% completed it;

No respondent bias evident in location, tier, non-completers or early vs. late completers;

Survey structure

Organisation parameters:Legal formStaff and budgetSSC “type”Respondent parameters:Management tier (Q1)Location (Q2)Reporting staff (Q3)Work unit tasks (Q4)Environment factors:Daily work influences (Q6)Motivation (Q8)Definition and value of “performance” (Q9, Q11)Quality of information (Q10)External climate (Q13)

General demand for information by purpose (Q14)

Demand for numerical information by purpose (Q15)

Demand for organisational information by purpose (Q17)

Dependencies

Dependent variablesExplanatory variables

Survey parameters

Went live in February 2009 Emailed invitations to work addresses from lists

supplied by HR departments Chief Executives endorsed completion Survey completed on-line Respondents assured that their identities are

confidential to core project group Results given to organisations only in aggregate

Management level of respondents

Management level No %

Tier 1 (CE) 7 0.4

Tier 2 83 4.9

Tier 3 394 23.2

Tier 4 625 36.7

Tier 5 592 34.8

TOTAL 1701 100.0

Survey: headline results

Size of respondent’s work unit

Staff reporting Number Percent

1 (self) 43 2.5

2-10 723 42.5

11-50 682 40.1

51-100 127 7.5

More 120 7.1

TOTAL 1695 99.6


Location of respondents

Location Number %

Head Office 993 58.4

Regional 278 16.3

Local 309 18.2

Other* 122 7.2

TOTAL 1702 100.1

*Mainly corporate services units separate from head office


Work unit main tasks

Task Number*%

sample

Services to Ministers 278 16.3

Services to the public 774 45.5

Enforcing the law or regulations 422 24.8

Contracts with providers 267 15.7

Managing joint projects or relationships 409 24.0

Developing or reporting on strategies, plans, etc 466 27.4

Internal services (HR, IT, Finance etc) 503 29.6

Other 556 32.7

*Managers reporting spent “a lot or nearly all” of their time (>2 tasks / mgr)


Q9.1: Clear idea from management about organisation’s objectives.

0

10

20

30

40

50

Stronglydisagree

Disagree Agree Stronglyagree

NR / NA Blank


Q9.2: “know what is expected of us”

0100200300400500600700800900

Stronglydisagree


NR / NA Blank


Q9.3: “Mostly judged against specific performance targets”

0100200300400500600700800

Stronglydisagree


NR / NA (blank)


Q.8: Key motivators

%*

The feedback and support we give each other 94

Recognition of my work from senior management or the Minister 87

The opportunity to do a job of value for the community 85

Recognition by other organisations of the quality of my work 67

The need to comply with the organisation's requirements 63

The opportunity for increased pay or promotion 62

Appreciation of my work by the public 57

*Agree / strongly agree as percent of total sample


Q.6: Professional and empowered or routinised and rule-driven?

Managers say they are empowered:– 96% of managers agree they “rely a lot on applying

professional training or knowledge” – 75% agree they “have a lot of discretion in how we

organise and prioritise our work” But:

– 80% agree they are “mostly guided by established rules or procedures”

– 76% agree “we have a work plan and we stick to it”– Only 34% agree “we have a lot of freedom in how we

allocated budget and staff”


A public service of rules and control?

Managers are not ‘muddling through’ or mainly working in horizontal informal networks …

…they are mainly managing activities to plan, following rules; so it looks like …

… the formal system based on a rational control model focussed on control and budgeting is deeply embedded in most agencies.

There is less variation in these results than we expected.


Rob Laking

This needs to be checked against agency by agency results. In other results there is a lot of variance by agency.

The dog that didn’t bark? What we expected and what we found

A former senior Minister: no-one in their right mind would rely on government performance information

We expected that managers:– Would think the information they got from the organisation was of

poor quality– Would substitute informal, unstructured information for formal,

structured information (the more so the closer they were to the front line)

We found that managers:– Have mixed views about the quality of information they get, but …– … make extensive use of formal organisational and numerical

information for performance management (the more so the closer they are to the front line)


Unstructured and local or structured and organisational?

Local: own groups and work contacts Unstructured: conversations, reading and

observation Organisational: “from your organisation” Structural: could be categorised and stored

in a database; not necessarily numerical but we settled for “numerical” as a descriptor


Use of numerical information

0

20

40

60

80

100

10 15 20 25 30 35 40 45 50

Index* of relative use (10 = almost entirely, 50 = almost never)

Nu

mb

er o

f re

spo

nd

ents

Mean = 27.9

*Validity of index: Cronbach’s Alpha = 0.917


Use of organisational information

0

20

40

60

80

100

120

140

10 15 20 25 30 35 40 45 50

Index* of relative use (10 = almost entirely, 50 = almost never)

Nu

mb

er o

f re

spo

nd

ents

Mean = 21.5

*Validity of index: Cronbach’s Alpha = 0.909


Which managers use formal, structured information the most?

We expected: the closer to the front-line, the less managers use formal organisational and numerical information and the more they use tacit information and direct experience

What we found was important was more complex: – Task*: some work unit tasks seem to require more formal,

structured information– Purpose*: similarly with some purposes for use of information– Location: the further from head office the more managers use

organisational and numerical information– Tier: Tier 1 (top) and Tier 5 (bottom) use organisational and

numerical information the most

*Discussed further later on


Regression results: what matters?

So far in analysis, three main factors identified:– Organisation: matters a lot - independent of legal form or type or

task, specific organisational factors independently influence use of formal and numerical information

– Task: the more time spent on service delivery and law enforcement and the less time spent on policy, the more use is made of numerical or organisational data

– Manager location: the further from Wellington national office, the more use is made of numerical or organisational data

But variance (particularly across organisations) is still large. Environment, political salience, institutions, leadership, culture?

Using exploratory data analysis to investigate further.


Rob Laking

We haven't analysed function in the Wilsonian sense.

Information quality

We expected managers to say: They have better information on outputs (Q10.1, Q10.5) than outcomes (Q10.2, Q10.3). And they did!

But what was surprising was the relatively high dissatisfaction with information on quality and performance after nearly 20 years of output based budgeting.


Quality of information

Work unit gets good information on … %*

Q10.1: Quality and timeliness of services 62

Q10.2: Contribution to government outcomes 49

Q10.3: Effects on public 45

Q10.4: How well we are doing 63

Q10.5: Help to improve performance 53

*Percent respondents who agree or strongly agree


“Lies damned lies and statistics…

….The reporting mechanisms are neither precise enough or encompassing enough for an accurate application to performance”

(Text response on use of information (Q16) by tier 4 manager in mid-sized policy and delivery agency)


Does external accountability drive internal performance management?

Testing central agency view that “it is critical that the same body of data that is used for internal decision-making be used for any external reporting.” (Treasury-SSC 2008).

Inferred from:– Information use by work units engaged in direct final

services (strongest direct relationship to EA)– Work units using information on outputs, outcomes,

process, inputs (basis for EA)– Use of specific performance targets (assumes meeting

organisational requirements) – Extent of reliance on organisational information (assumes

organisation enacts EA requirements internally)

Survey: external-internal coupling

Relative high uses of numerical and organisational information

20% 30% 40% 50% 60% 70% 80%

Strategies & plans

Workload & performance

Relationships

Outputs

Outcomes

Capabilities

Budget & staff time

Improve performance

Publicity, enquiries

Report upwards

Numerical

Organisational

Truncated scale


Numerical information: high use on outputs and outcomes

20% 25% 30% 35% 40% 45% 50% 55% 60%

Services to Ministers

Services to public

Enforcing law, regulations

Provider contracts

Outside relationships

Strategies, plans, processes

Internal services

Other

Outputs

Outcomes

Truncated scale


Numerical information: high use on inputs and processes

20% 30% 40% 50% 60% 70% 80%


Services to public


Provider contracts

Relationships - other organisations


Internal services

Other

Workload, performance

Budget, staff time

Truncated scale


Organisational information: high use on outputs and outcomes

30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80%


Services to public


Provider contracts



Internal services

Other

Outputs

Outcomes

Truncated scale


Organisational information: high use on inputs and processes

40% 50% 60% 70% 80% 90% 100%


Services to public


Provider contracts



Internal services

Other

Workload, performance

Budget, staff time

Truncated scale


Use of specific performance targets

50% 55% 60% 65% 70% 75% 80% 85% 90%


Services to public


Provider contracts



Internal services

Other

Truncated scale


Cautious conclusions

For direct services to the public and direct enforcement of law and regulations, numerical and organisational data is relatively important;

For direct services to Ministers and managing joint projects, non-numerical and non-organisational data is relatively important.

External accountability does have a significant impact on managers’ demand for information to manage performance, particularly for direct services to the public or direct enforcement activities.

Overall, – local and unstructured information is demanded for all tasks– there is still a large unexplained variance.


Summary of research themes

Public sector reforms have shifted the locus of control from public service wide to agency level. But control still dominates

Managers tactically manage inputs through processes to deliver activities/outputs. Little evidence of “managing for outcomes.”

But ‘managing for outcomes’ is not ‘dead in the water’ – just under the radar

Managing for organisational performance (M4P) - Next steps

Sense making of survey results, analysis of survey text responses, case studies, literature review

Finalise review of formal system Develop directions for reform (IPANZ/SOG

seminar of 15 July) IPS Book Publication (end August 2009)

Thank you and question time

Further detail at http://ips.ac.nz/events/Ongoing_research/

Or email:[email protected]@vuw.ac.nz

Managing for organisational performance: What information do state sector managers actually rely on?...

Documents

Transcript of Managing for organisational performance: What information do state sector managers actually rely on?...