This presentation refers in part to results of work generated by the author during the Projects
description
Transcript of This presentation refers in part to results of work generated by the author during the Projects
1KPI Management and Policy Consultants
Conference on Performance Evaluation of the JudiciaryAnkara, 25 March 2011
1
CoE Conference on Performance Evaluation of the Judiciary
Court Statistics, Judge Workload Analysis, Quality and
Performance Management
Dr Axel G KoetzAnkara 25 March 2011
This Document is complete only together with the oral presentation;use of isolated pages might lead to misunderstandings.
Questions: Dr Axel G. Koetz, Managing Partner, KPI Management and Policy ConsultantsUnicenter 2920, D-50939 Cologne, axel.koetz @ koetz-ag.com, Tel/Fax +49 (0)221-9411801 / 05
Dr Axel G Koetz, Managing Partner, Koetz Partner International, Unicenter 2920, D-50939 Cologne Germany, e-mail axel.koetz @ koetz-ag.comCourt Statistics, Judge Workload Analysis, Quality and Performance Management – Ankara (CoE) 25 March 2011
Dr Axel G Koetz, Managing Partner, Koetz Partner International, Unicenter 2920, D-50939 Cologne Germany, e-mail axel.koetz @ koetz-ag.comCourt Statistics, Judge Workload Analysis, Quality and Performance Management – Ankara (CoE) 25 March 2011
KPI Management and Policy Consultants
Conference on Performance Evaluation of the JudiciaryAnkara, 25 March 2011
This presentation refers in part to results of work generated by the author during
the Projects
Dr Axel G Koetz, KPI International Management and Policy Consultants, Unicenter 2920, D-50539 Cologne Germany, [email protected] Management and
Policy Consultants
ACCOUNTABILITY AND EFFECTIVENESS OF UKRAINIAN JUDICIARY FUNCIONING: Civil Service Component EuropeAid/125611/C/SER/UA
STRUKTURANALYSE DER RECHTSPFLEGEOrganisation der Amtsgerichte
Organisation der Kollegial- und InstanzgerichteOrganisation der Staatsanwaltschaften
Im Auftrag des Bundesministeriums der Justiz, Bonn(published in German by Bundesanzeiger Printing House)
and other related studies on behalf of German State Governments
Dr Axel G Koetz, Managing Partner, Koetz Partner International, Unicenter 2920, D-50939 Cologne Germany, e-mail axel.koetz @ koetz-ag.comCourt Statistics, Judge Workload Analysis, Quality and Performance Management – Ankara (CoE) 25 March 2011
KPI Management and Policy Consultants
Conference on Performance Evaluation of the JudiciaryAnkara, 25 March 2011
Use of Statisticsin Court Management
Conference MaterialPart 1 of 4
Dr Axel G Koetz, Managing Partner, Koetz Partner International, Unicenter 2920, D-50939 Cologne Germany, e-mail axel.koetz @ koetz-ag.comCourt Statistics, Judge Workload Analysis, Quality and Performance Management – Ankara (CoE) 25 March 2011
4KPI Management and Policy Consultants
Conference on Performance Evaluation of the JudiciaryAnkara, 25 March 2011
Today Problems
• Data are collected for many use(r)s – including academic exercises – but not for management
• Nobody cares for data quality as those who create do not profit from results
• Case data, HR data and financial data are collected by different departments and for different users and never integrated for management use
• An immense quantity of data is collected, stored and forgotten
• Collection of date itself creates an inappropriate resource consumption
Dr Axel G Koetz, Managing Partner, Koetz Partner International, Unicenter 2920, D-50939 Cologne Germany, e-mail axel.koetz @ koetz-ag.comCourt Statistics, Judge Workload Analysis, Quality and Performance Management – Ankara (CoE) 25 March 2011
5KPI Management and Policy Consultants
Conference on Performance Evaluation of the JudiciaryAnkara, 25 March 2011
Necessary Integration of Statistics
Case Statistics
HR andResourcesStatistics
FinancialStatistics
ExternalQuality
Statistics
CourtManagement
Data Base
Dr Axel G Koetz, Managing Partner, Koetz Partner International, Unicenter 2920, D-50939 Cologne Germany, e-mail axel.koetz @ koetz-ag.comCourt Statistics, Judge Workload Analysis, Quality and Performance Management – Ankara (CoE) 25 March 2011
6KPI Management and Policy Consultants
Conference on Performance Evaluation of the JudiciaryAnkara, 25 March 2011
What means Court Management ?
Top Down View:Optimization of the overall Judicial System from the
viewpoint of Effectiveness and Efficiency
Bottom-Up View:Ensure a proper functioning of the court according to central goals and making best use of court resources
Effectiveness, Quality and Efficiency of the Judiciary
Inst
rum
ents
Legal p
oss
ibili
ties
to „
man
ag
e“
Dr Axel G Koetz, Managing Partner, Koetz Partner International, Unicenter 2920, D-50939 Cologne Germany, e-mail axel.koetz @ koetz-ag.comCourt Statistics, Judge Workload Analysis, Quality and Performance Management – Ankara (CoE) 25 March 2011
7KPI Management and Policy Consultants
Conference on Performance Evaluation of the JudiciaryAnkara, 25 March 2011
Orient Statistics to Management Needs 1:What the Head of Court Might Like to Know
• Number of cases (incoming / completed / unfinished)• Case workload (per relevant case type and overall)• Time needed for case types• Backlogs• Percentage of cases sent back from higher court due to successful
appeals• Satisfaction of Court users with speed, friendlyness, accessibility• Differences in performance between judges – in quantity, speed
and correctness of output• De facto available staff off all types• Position of the courts performance compared with others• Changes of numbers compared with the last year/sMay be more ?
May be other ?May be none at all ?
Dr Axel G Koetz, Managing Partner, Koetz Partner International, Unicenter 2920, D-50939 Cologne Germany, e-mail axel.koetz @ koetz-ag.comCourt Statistics, Judge Workload Analysis, Quality and Performance Management – Ankara (CoE) 25 March 2011
8KPI Management and Policy Consultants
Conference on Performance Evaluation of the JudiciaryAnkara, 25 March 2011
Orient Statistics to Management Needs 2:
What the Central Level Might Like to Know
• Number of cases nationwide, regional, per court• Case workload: Overall indicator for key case types• Time needed for case types overall / per court• Backlogs per case type / per court• Percentage of cases sent back from higher court due to
successful appeals – overall and per court• Satisfaction of Court users with speed, friendlyness,
accessibility, overall and per court• Differences in performance between courts and judges – in
quantity, speed and correctness of output• De facto available staff off all types versus plan (overall /
per court)• Comparative data describing court performance• Changes of numbers compared with the last year/s
Dr Axel G Koetz, Managing Partner, Koetz Partner International, Unicenter 2920, D-50939 Cologne Germany, e-mail axel.koetz @ koetz-ag.comCourt Statistics, Judge Workload Analysis, Quality and Performance Management – Ankara (CoE) 25 March 2011
9KPI Management and Policy Consultants
Conference on Performance Evaluation of the JudiciaryAnkara, 25 March 2011
Are Big Courts More Productive Than Small Courts ?
0
50
100
150
200
250
300
350
400
450
0 5 10 15 20 25 30 350
50
100
150
200
250
300
350
400
450
0 5 10 15 20 25 30 35
666 District Courts, Unweighthed Cases, Planned Judges
Dr Axel G Koetz, Managing Partner, Koetz Partner International, Unicenter 2920, D-50939 Cologne Germany, e-mail axel.koetz @ koetz-ag.comCourt Statistics, Judge Workload Analysis, Quality and Performance Management – Ankara (CoE) 25 March 2011
10KPI Management and Policy Consultants
Conference on Performance Evaluation of the JudiciaryAnkara, 25 March 2011
Do Judges in Small Courts Need More Support Staff Than in Big Courts ?
FT
PE
Sup
port
per
FT
PE
Judg
e
FTPE Judge0
2
4
6
8
10
12
14
16
18
20
0,00 5,00 10,00 15,00 20,00 25,00 30,00
KH
OD
LV
KO
CY
DZ
Potenziell (DZ)
0
2
4
6
8
10
12
14
16
18
20
0,00 5,00 10,00 15,00 20,00 25,00 30,00
Dr Axel G Koetz, Managing Partner, Koetz Partner International, Unicenter 2920, D-50939 Cologne Germany, e-mail axel.koetz @ koetz-ag.comCourt Statistics, Judge Workload Analysis, Quality and Performance Management – Ankara (CoE) 25 March 2011
11KPI Management and Policy Consultants
Conference on Performance Evaluation of the JudiciaryAnkara, 25 March 2011
How much time does a Judge need for for an average Case ?
HoursjudgeTime per case
Jud
ge
FT
PE
pe
r C
ou
rt
9
0
5
10
15
20
25
30
0 0,5 1 1,5 2 2,5 3 3,5 4 4,5 5
OD
KH
LV
KO
CY
DZ
9
0
5
10
15
20
25
30
0 0,5 1 1,5 2 2,5 3 3,5 4 4,5 5
Dr Axel G Koetz, Managing Partner, Koetz Partner International, Unicenter 2920, D-50939 Cologne Germany, e-mail axel.koetz @ koetz-ag.comCourt Statistics, Judge Workload Analysis, Quality and Performance Management – Ankara (CoE) 25 March 2011
12KPI Management and Policy Consultants
Conference on Performance Evaluation of the JudiciaryAnkara, 25 March 2011
Why Does One Court Solve 50% of the Cases in
3,5 Months Whilst Another Takes 6 Months ?
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
1 2 3 4 5 6 7 8 9
10
11
12
13
14
15
16
17
18 24
60
12
0
mo
re
1-CUM
2-CUM
Dr Axel G Koetz, Managing Partner, Koetz Partner International, Unicenter 2920, D-50939 Cologne Germany, e-mail axel.koetz @ koetz-ag.comCourt Statistics, Judge Workload Analysis, Quality and Performance Management – Ankara (CoE) 25 March 2011
13KPI Management and Policy Consultants
Conference on Performance Evaluation of the JudiciaryAnkara, 25 March 2011
Why Differ Backlogs by the Factor 10 Between Courts of Different Regions ?
Dr Axel G Koetz, Managing Partner, Koetz Partner International, Unicenter 2920, D-50939 Cologne Germany, e-mail axel.koetz @ koetz-ag.comCourt Statistics, Judge Workload Analysis, Quality and Performance Management – Ankara (CoE) 25 March 2011
14KPI Management and Policy Consultants
Conference on Performance Evaluation of the JudiciaryAnkara, 25 March 2011
... And External Quality Evaluation ?
• Problem: External data need extra processing, this means work
• Chance: See the unbiased view of the „users“ instead of what the system produces internally
• Customer cards might provide multiple choice answers on– Accessibility of the court– Evaluation of court staff behaviour– Evaluation of felt work quality– Evaluation of processes, timing etc.
• Interesting is the time series analysis and the internal / external comparison
Dr Axel G Koetz, Managing Partner, Koetz Partner International, Unicenter 2920, D-50939 Cologne Germany, e-mail axel.koetz @ koetz-ag.comCourt Statistics, Judge Workload Analysis, Quality and Performance Management – Ankara (CoE) 25 March 2011
15KPI Management and Policy Consultants
Conference on Performance Evaluation of the JudiciaryAnkara, 25 March 2011
Questions Lead to Other Questions...
• Should we change the court network and eliminate courts with less than xx Judges ?
• In how far can we exchange court support staff to technology (and uphold small courts)
• How can we deal with non performing Judges who solve less cases / are systematically slower than the average
• Do we have under-resourced courts and are lacks in resources or regional „styles“ responsible for backlogs ?
Dr Axel G Koetz, Managing Partner, Koetz Partner International, Unicenter 2920, D-50939 Cologne Germany, e-mail axel.koetz @ koetz-ag.comCourt Statistics, Judge Workload Analysis, Quality and Performance Management – Ankara (CoE) 25 March 2011
16KPI Management and Policy Consultants
Conference on Performance Evaluation of the JudiciaryAnkara, 25 March 2011
... Other Questions Lead to Decisions
• Change the „court network“• Increase training for nonperforming judges and
judges with a high proportion of successful appeals
• Introduce better workflow software and optimize regulations to save support staff capacity
• Redistribute resources according to real workloads
• Have discussions with Court managers who fail to bring their numbers in order.
Dr Axel G Koetz, Managing Partner, Koetz Partner International, Unicenter 2920, D-50939 Cologne Germany, e-mail axel.koetz @ koetz-ag.comCourt Statistics, Judge Workload Analysis, Quality and Performance Management – Ankara (CoE) 25 March 2011
KPI Management and Policy Consultants
Conference on Performance Evaluation of the JudiciaryAnkara, 25 March 2011
Necessary Elements for a System of Collecting and Analyzing Data
Conference MaterialPart 2 of 4
Dr Axel G Koetz, Managing Partner, Koetz Partner International, Unicenter 2920, D-50939 Cologne Germany, e-mail axel.koetz @ koetz-ag.comCourt Statistics, Judge Workload Analysis, Quality and Performance Management – Ankara (CoE) 25 March 2011
18KPI Management and Policy Consultants
Conference on Performance Evaluation of the JudiciaryAnkara, 25 March 2011
From Data Cemeteries to Useful MIS
Relevant
Automatic
Motivating
„Drilling Thru“
Start from Existing Systems
Leads to the right decisions (e.g.: necessity of proper case weighting)
Data generated during the normalwork; no additional data collection
Short and easy to understand by thedecision makers and motivate them
Trace nationwide information throughall levels down do the individual
Base as far as possible on existing data and data collection structures
Dr Axel G Koetz, Managing Partner, Koetz Partner International, Unicenter 2920, D-50939 Cologne Germany, e-mail axel.koetz @ koetz-ag.comCourt Statistics, Judge Workload Analysis, Quality and Performance Management – Ankara (CoE) 25 March 2011
19KPI Management and Policy Consultants
Conference on Performance Evaluation of the JudiciaryAnkara, 25 March 2011
Relevance(Aspect 1 of 5)
• We do not need to count what we can not influence
• For all other things we need the data• In any case, a value analysis of all data collection
is necessary
• For example: Proper case weighting is extremely relevant for all management decisions and a „must have“ for the system.
Dr Axel G Koetz, Managing Partner, Koetz Partner International, Unicenter 2920, D-50939 Cologne Germany, e-mail axel.koetz @ koetz-ag.comCourt Statistics, Judge Workload Analysis, Quality and Performance Management – Ankara (CoE) 25 March 2011
20KPI Management and Policy Consultants
Conference on Performance Evaluation of the JudiciaryAnkara, 25 March 2011
Automatic Collection(Aspect 2 of 5)
• As far as possible, all data should be generated automatically within existing workflows– Case data– Human resources dData– Other resources procurement / register data– Financial data
• For all data we need integrated collection and storage systems
• For all data we need clear definitions and clearly defined interfaces
• Existing workflow software has to be modified or exchanged to software which is able to do it.
Dr Axel G Koetz, Managing Partner, Koetz Partner International, Unicenter 2920, D-50939 Cologne Germany, e-mail axel.koetz @ koetz-ag.comCourt Statistics, Judge Workload Analysis, Quality and Performance Management – Ankara (CoE) 25 March 2011
21KPI Management and Policy Consultants
Conference on Performance Evaluation of the JudiciaryAnkara, 25 March 2011
Integration of Workflow Software and Statistical Software becomes a Key
ElementWorkflow System collects inter alia
- Case Type- Case Generation Date- Judge Name- Numerous case properties
(like n of hearings, n of witnesses, lawyers use, experts use etc.)
- Last hearing date- Verdict- Appeal
Automatic transfer to the statistical system at any time
Automatic forwarding to a national Court Data Base
Other Workflows in the Court
- HR- Equipment / Maintenance- Finance
Regular reports to central / regional / court level / public
Reports on demand to all levels, according to needs
DB availability for own research to the academic world and the public
Dr Axel G Koetz, Managing Partner, Koetz Partner International, Unicenter 2920, D-50939 Cologne Germany, e-mail axel.koetz @ koetz-ag.comCourt Statistics, Judge Workload Analysis, Quality and Performance Management – Ankara (CoE) 25 March 2011
22KPI Management and Policy Consultants
Conference on Performance Evaluation of the JudiciaryAnkara, 25 March 2011
Motivation(Aspect 3 of 5)
• Data have to be understandable to decision makers
• Information has to be publicly available
• No overcomplicated indicators• Decision makers have to have the power to act
Dr Axel G Koetz, Managing Partner, Koetz Partner International, Unicenter 2920, D-50939 Cologne Germany, e-mail axel.koetz @ koetz-ag.comCourt Statistics, Judge Workload Analysis, Quality and Performance Management – Ankara (CoE) 25 March 2011
23KPI Management and Policy Consultants
Conference on Performance Evaluation of the JudiciaryAnkara, 25 March 2011
Drilling Thru(Aspect 4 of 5)
• All levels have to have the technical opportunity to compare and track the reasons of problems as well as origins of good practice
• Top-Down and cross-cutting analyses have to be possible on every level
• „Drilling thru“ capacities: Analyze individual performance on all levels ...
• This means that ALL individual case data have to be available, retrievable, connectable and analyzable
• Never „aggregate“ data and give up potential information – nothing is as cheap as data storage
Dr Axel G Koetz, Managing Partner, Koetz Partner International, Unicenter 2920, D-50939 Cologne Germany, e-mail axel.koetz @ koetz-ag.comCourt Statistics, Judge Workload Analysis, Quality and Performance Management – Ankara (CoE) 25 March 2011
24KPI Management and Policy Consultants
Conference on Performance Evaluation of the JudiciaryAnkara, 25 March 2011
Pyramid Model to Identify the Souces of Problems...
National
Regional level
Court Level
Judge level
Which informations comefrom the national Average ?
Where come Differencesin regional performance from ?
Which courts have problems,Which are fine ?
Are there problems of Judges ?
Dr Axel G Koetz, Managing Partner, Koetz Partner International, Unicenter 2920, D-50939 Cologne Germany, e-mail axel.koetz @ koetz-ag.comCourt Statistics, Judge Workload Analysis, Quality and Performance Management – Ankara (CoE) 25 March 2011
25KPI Management and Policy Consultants
Conference on Performance Evaluation of the JudiciaryAnkara, 25 March 2011
„Drilling Thru“ and Extended Analytical Capacities Needed
JUDGES
COURTS
REGIONS
„Which judges are the most (un)productive nationwide ?“„Is there a systematic performance difference between City and rural area courts ?“„Are big courts more / less productive than small courts“ ?
Dr Axel G Koetz, Managing Partner, Koetz Partner International, Unicenter 2920, D-50939 Cologne Germany, e-mail axel.koetz @ koetz-ag.comCourt Statistics, Judge Workload Analysis, Quality and Performance Management – Ankara (CoE) 25 March 2011
26KPI Management and Policy Consultants
Conference on Performance Evaluation of the JudiciaryAnkara, 25 March 2011
Start from Existing Systems(Aspect 5 of 5)
• State of the Art Management Information Systems (MIS) are challenging, costly and the development might take years
• Instead of waiting for funds for optimal solutions, in many cases small changes in existing Software might bridge the time.
• Sometimes the work with pilot systems on court and region level can give important insight before the big project is started
Dr Axel G Koetz, Managing Partner, Koetz Partner International, Unicenter 2920, D-50939 Cologne Germany, e-mail axel.koetz @ koetz-ag.comCourt Statistics, Judge Workload Analysis, Quality and Performance Management – Ankara (CoE) 25 March 2011
KPI Management and Policy Consultants
Conference on Performance Evaluation of the JudiciaryAnkara, 25 March 2011
Analysis of Judges Workload
Conference MaterialPart 3 of 4
Dr Axel G Koetz, Managing Partner, Koetz Partner International, Unicenter 2920, D-50939 Cologne Germany, e-mail axel.koetz @ koetz-ag.comCourt Statistics, Judge Workload Analysis, Quality and Performance Management – Ankara (CoE) 25 March 2011
28KPI Management and Policy Consultants
Conference on Performance Evaluation of the JudiciaryAnkara, 25 March 2011
Caseload versus Workload
Caseload:
Number of casesto be completed
- By a Judge- By a Court- By the
judges/courts in a region
- Nationwide
Workload:
The work capacity neededto complete
- a case- all cases on the judges
table- all cases in the court- all cases in the courts of
a region- all cases nationwide
Dr Axel G Koetz, Managing Partner, Koetz Partner International, Unicenter 2920, D-50939 Cologne Germany, e-mail axel.koetz @ koetz-ag.comCourt Statistics, Judge Workload Analysis, Quality and Performance Management – Ankara (CoE) 25 March 2011
29KPI Management and Policy Consultants
Conference on Performance Evaluation of the JudiciaryAnkara, 25 March 2011
The „Caseload“ concept leads to mismanagement
Caseload computes the numbers of cases irrespective of complexity.
Adding all cases and basing policies on this leads to severe problems as we are „adding pumpkins and cherries“
Many „order cases“Many „administrative offenses“
Lead to
High completion numbers„productive“ judges
Many complex criminal cases
Lead to
Low completion numbersUnproductive judges
Dr Axel G Koetz, Managing Partner, Koetz Partner International, Unicenter 2920, D-50939 Cologne Germany, e-mail axel.koetz @ koetz-ag.comCourt Statistics, Judge Workload Analysis, Quality and Performance Management – Ankara (CoE) 25 March 2011
30KPI Management and Policy Consultants
Conference on Performance Evaluation of the JudiciaryAnkara, 25 March 2011
Differences Unweighted/Weighted Cases Demonstrates Importance of Complexity
Analysis
0
0,5
1
1,5
2
2,5
3
3,5
4
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33
NW
W
Comparison of deviation from average (1) in caseload and workload per Judgein 33 Courts of a region, weighted (w) and not weighted (cases)
Dr Axel G Koetz, Managing Partner, Koetz Partner International, Unicenter 2920, D-50939 Cologne Germany, e-mail axel.koetz @ koetz-ag.comCourt Statistics, Judge Workload Analysis, Quality and Performance Management – Ankara (CoE) 25 March 2011
31KPI Management and Policy Consultants
Conference on Performance Evaluation of the JudiciaryAnkara, 25 March 2011
Determinants of Workload
CASE WORKLOAD PER CASETYPECASE WORKLOAD PER CASETYPE
Court Work Structures Workflow and IT
Case Complexity
QuantitativeAspects
QualitativeAspects
Dr Axel G Koetz, Managing Partner, Koetz Partner International, Unicenter 2920, D-50939 Cologne Germany, e-mail axel.koetz @ koetz-ag.comCourt Statistics, Judge Workload Analysis, Quality and Performance Management – Ankara (CoE) 25 March 2011
32KPI Management and Policy Consultants
Conference on Performance Evaluation of the JudiciaryAnkara, 25 March 2011
Case Complexity Determinators
Quality:Case TypeProcedural LawExisting Standards
Quality:Case TypeProcedural LawExisting Standards
Size:N of involved parties
/ defendantsN of witnessesN of needed
hearingsQuantity of
Documents
Size:N of involved parties
/ defendantsN of witnessesN of needed
hearingsQuantity of
Documents
Simplified:
Case Type plusSize Indicator
Simplified:
Case Type plusSize Indicator
Dr Axel G Koetz, Managing Partner, Koetz Partner International, Unicenter 2920, D-50939 Cologne Germany, e-mail axel.koetz @ koetz-ag.comCourt Statistics, Judge Workload Analysis, Quality and Performance Management – Ankara (CoE) 25 March 2011
33KPI Management and Policy Consultants
Conference on Performance Evaluation of the JudiciaryAnkara, 25 March 2011
Case Type Modeling
ModelAlternatives
Minimalistic Model/sConcentrates on a small number of consolidated case types
100% Model/sTries to identify (almost) all potential cases, Following the §§ of the law(s)
Key Indicator Model/sBased on a substantial number of relevant case types
Dr Axel G Koetz, Managing Partner, Koetz Partner International, Unicenter 2920, D-50939 Cologne Germany, e-mail axel.koetz @ koetz-ag.comCourt Statistics, Judge Workload Analysis, Quality and Performance Management – Ankara (CoE) 25 March 2011
34KPI Management and Policy Consultants
Conference on Performance Evaluation of the JudiciaryAnkara, 25 March 2011
Think in A-B-C Categories
Instead of killing people with megabytes of Data, follow the ABC model
A = Vital informationB = Important informationC = Unimportant information
Make sure that „A“ level information is not buried under „C“ level information
B CA
80%
90%
100%
20% 40% 100%
Information
Exp
lana
tion
Dr Axel G Koetz, Managing Partner, Koetz Partner International, Unicenter 2920, D-50939 Cologne Germany, e-mail axel.koetz @ koetz-ag.comCourt Statistics, Judge Workload Analysis, Quality and Performance Management – Ankara (CoE) 25 March 2011
35KPI Management and Policy Consultants
Conference on Performance Evaluation of the JudiciaryAnkara, 25 March 2011
Sample Key Indicator Model: PEBB§Y
• Identification of key case types, inter alia based on an ABC Analysis
• Identification of case complexity and related workload
• Detailed analysis of „A“ case types and selected others
• Correction factors and other instruments to cover the non-key case types (C, partly B)
• Please note: The following sheets show an extremely simplified picture of the methodology and the results ! PEBB§Y in reality is much more complicated when you come to the details.
Dr Axel G Koetz, Managing Partner, Koetz Partner International, Unicenter 2920, D-50939 Cologne Germany, e-mail axel.koetz @ koetz-ag.comCourt Statistics, Judge Workload Analysis, Quality and Performance Management – Ankara (CoE) 25 March 2011
36KPI Management and Policy Consultants
Conference on Performance Evaluation of the JudiciaryAnkara, 25 March 2011
PEBB§Y Goals
• Key Goal: Identify the need for Judges / prosecutors posts on state level and define the budget accordingly
• Second Goal: Ensure a just distribution of posts across the courts
• Third Goal: Create transparency and acceptance amongst stakeholders
• No Goal: Establish a legal right of judges not to work more than given by the set indicators
Dr Axel G Koetz, Managing Partner, Koetz Partner International, Unicenter 2920, D-50939 Cologne Germany, e-mail axel.koetz @ koetz-ag.comCourt Statistics, Judge Workload Analysis, Quality and Performance Management – Ankara (CoE) 25 March 2011
37KPI Management and Policy Consultants
Conference on Performance Evaluation of the JudiciaryAnkara, 25 March 2011
-
„Courts have to be equipped adequately with personnel and other resources;
details are specified by law“ (Constitution, Hamburg, Art 62 – similar in other constitutions)
Dr Axel G Koetz, Managing Partner, Koetz Partner International, Unicenter 2920, D-50939 Cologne Germany, e-mail axel.koetz @ koetz-ag.comCourt Statistics, Judge Workload Analysis, Quality and Performance Management – Ankara (CoE) 25 March 2011
38KPI Management and Policy Consultants
Conference on Performance Evaluation of the JudiciaryAnkara, 25 March 2011
Case type Selection (P1)
• 48 case types in local courts• 20 case types in lower appellate courts• 22 case types in higher appellate courts
• Other case types and administrative work taken into consideration via correction factors
• Also „Training“, „Administrative Functions“ and „other Tasks“ included, also numerous registers (German speciality) at local courts
Dr Axel G Koetz, Managing Partner, Koetz Partner International, Unicenter 2920, D-50939 Cologne Germany, e-mail axel.koetz @ koetz-ag.comCourt Statistics, Judge Workload Analysis, Quality and Performance Management – Ankara (CoE) 25 March 2011
39KPI Management and Policy Consultants
Conference on Performance Evaluation of the JudiciaryAnkara, 25 March 2011
Sample Judge Workload per Case Types
- for general civil cases 150 minutes- for civil claims from car accidents 170 minutes- for divorce cases 200 minutes- for small criminal cases 170 minutes- for major criminal cases 510 minutes- for punishment orders (comparable
to administrative offenses) 22 minutes- for economic and environmental crime 970 minutes
- Appeals to Lower Appellate Court 430/910 minutes- Cassation Cases, higher Appellate Court 660 minutes
Dr Axel G Koetz, Managing Partner, Koetz Partner International, Unicenter 2920, D-50939 Cologne Germany, e-mail axel.koetz @ koetz-ag.comCourt Statistics, Judge Workload Analysis, Quality and Performance Management – Ankara (CoE) 25 March 2011
40KPI Management and Policy Consultants
Conference on Performance Evaluation of the JudiciaryAnkara, 25 March 2011
Computing the Need for Judge Capacity
(Simplified)
• Judges work capacity per year 102.240 minutes
• Sample „car accident case“ 170 minutes• Cases per Judge per year 601 cases
N of cases x case weights
Judge work capacitycorrection factors
N of postsfor Judges
+/- =
Dr Axel G Koetz, Managing Partner, Koetz Partner International, Unicenter 2920, D-50939 Cologne Germany, e-mail axel.koetz @ koetz-ag.comCourt Statistics, Judge Workload Analysis, Quality and Performance Management – Ankara (CoE) 25 March 2011
41KPI Management and Policy Consultants
Conference on Performance Evaluation of the JudiciaryAnkara, 25 March 2011
Development of the 1st PEBB§Y 1 Model
2001 - 2002
• About 40 courts and Prosecutors Offices• 7 German States• About 1.900 Judges and Prosecutors• About 900.000 case cards analyzed• External project executed by a consulting / accounting firm
• In addition, a PEBB§Y 2 model was developed to get data for the non-judicial staff.
Dr Axel G Koetz, Managing Partner, Koetz Partner International, Unicenter 2920, D-50939 Cologne Germany, e-mail axel.koetz @ koetz-ag.comCourt Statistics, Judge Workload Analysis, Quality and Performance Management – Ankara (CoE) 25 March 2011
42KPI Management and Policy Consultants
Conference on Performance Evaluation of the JudiciaryAnkara, 25 March 2011
PEBB§Y StepsA Process of a Decade (by now)
• 2001: PEBB§Y 1 (General Courts –Judges, Prosecutors)
• 2001/2 PEBB§Y 2 (General Courts - Secretaries + Support Staff)
• 2005 PEBB§Y Fach (Judges in Specialized Courts for Labor, Tax, Social, Administrative
Cases)• 2008 PEBB$Y Update Analyses: 1,2,Fach
00 01 02 03 04 05 06 07 08 09
P 2
P 1
P Fach
P Update
Dr Axel G Koetz, Managing Partner, Koetz Partner International, Unicenter 2920, D-50939 Cologne Germany, e-mail axel.koetz @ koetz-ag.comCourt Statistics, Judge Workload Analysis, Quality and Performance Management – Ankara (CoE) 25 March 2011
43KPI Management and Policy Consultants
Conference on Performance Evaluation of the JudiciaryAnkara, 25 March 2011
Need for Participation...... and permanent updating
• States, Courts, Judges, Prosecutors intensively involved
• Numerous workshops, meetings and so on• Intensive cooperation and involvement of the
judges associations• Consideration of specialities in the different states• Check for realism and determination of the final
results in workgroups• Update of results according to chages in laws,
jurisdiction, procedures, technology after some years
Dr Axel G Koetz, Managing Partner, Koetz Partner International, Unicenter 2920, D-50939 Cologne Germany, e-mail axel.koetz @ koetz-ag.comCourt Statistics, Judge Workload Analysis, Quality and Performance Management – Ankara (CoE) 25 March 2011
44KPI Management and Policy Consultants
Conference on Performance Evaluation of the JudiciaryAnkara, 25 March 2011
Don‘t forget the civil service Structures
• PEBB§Y 1 was accompanied by PEBB§Y 2• PEBB§Y 2 covers the workforce needs in the field
of civil service– Secretaries– Typists / Note takers (if still existing)– Court guards– Other professions in the court
– Capacity needs for some professions still determined outside the PEBB§Y system
Dr Axel G Koetz, Managing Partner, Koetz Partner International, Unicenter 2920, D-50939 Cologne Germany, e-mail axel.koetz @ koetz-ag.comCourt Statistics, Judge Workload Analysis, Quality and Performance Management – Ankara (CoE) 25 March 2011
45KPI Management and Policy Consultants
Conference on Performance Evaluation of the JudiciaryAnkara, 25 March 2011
From Cases to Staff to BudgetFrom Budget to Staff
• Workload from weighted cases defines Staff need– Per single court– Per region
• Accumulated staff needs determine overall number of judges
• Overall number of Judges determine Financial Budget for Judges
• New judges have to be hired accordingly (or posts have to be made free)
• Capacity has to be distributed justly amongst courts to ensure equal workload (Courts with overcapacity lose first the posts, secondly the staff when pensioned or replaced)
Dr Axel G Koetz, Managing Partner, Koetz Partner International, Unicenter 2920, D-50939 Cologne Germany, e-mail axel.koetz @ koetz-ag.comCourt Statistics, Judge Workload Analysis, Quality and Performance Management – Ankara (CoE) 25 March 2011
46KPI Management and Policy Consultants
Conference on Performance Evaluation of the JudiciaryAnkara, 25 March 2011
Use of PEBB§Y Date in Budgeting
Weighted Case Data per Region
Staff ListHigh Courts
Min of Justice
Min of Finance
ParliamentGovernment
Does it work this way...or not ?
Most hopefullyBut not all the time
Weighted Case DataLower Courts
Staff Lists Low Courts
Dr Axel G Koetz, Managing Partner, Koetz Partner International, Unicenter 2920, D-50939 Cologne Germany, e-mail axel.koetz @ koetz-ag.comCourt Statistics, Judge Workload Analysis, Quality and Performance Management – Ankara (CoE) 25 March 2011
KPI Management and Policy Consultants
Conference on Performance Evaluation of the JudiciaryAnkara, 25 March 2011
Performance and Quality Management
Conference MaterialPart 4 of 4
Dr Axel G Koetz, Managing Partner, Koetz Partner International, Unicenter 2920, D-50939 Cologne Germany, e-mail axel.koetz @ koetz-ag.comCourt Statistics, Judge Workload Analysis, Quality and Performance Management – Ankara (CoE) 25 March 2011
48KPI Management and Policy Consultants
Conference on Performance Evaluation of the JudiciaryAnkara, 25 March 2011
What do we want ?
• Performance Management
Ensure best possible Service delivery by the couer
Multi-Dimensional Approach (finance is one dimension)
• Performance Budgeting (Funding)
Assignment of State Resources according to the performance of the courts
Orientation to the financial dimension
Dr Axel G Koetz, Managing Partner, Koetz Partner International, Unicenter 2920, D-50939 Cologne Germany, e-mail axel.koetz @ koetz-ag.comCourt Statistics, Judge Workload Analysis, Quality and Performance Management – Ankara (CoE) 25 March 2011
49KPI Management and Policy Consultants
Conference on Performance Evaluation of the JudiciaryAnkara, 25 March 2011
What do we need
1. A working system to collect / retrieve the necessary information in the necessary disaggregated form
2. A system to present the data in an understandable form with complete „drilling thru“ and analysis capacities
3. An evaluation and reaction system (legal / organizational framework)
4. At the end a complete Management Information System (MIS) for the courts
Dr Axel G Koetz, Managing Partner, Koetz Partner International, Unicenter 2920, D-50939 Cologne Germany, e-mail axel.koetz @ koetz-ag.comCourt Statistics, Judge Workload Analysis, Quality and Performance Management – Ankara (CoE) 25 March 2011
50KPI Management and Policy Consultants
Conference on Performance Evaluation of the JudiciaryAnkara, 25 March 2011
Dimensions of Performance
Dimension Key Customer
Quantitative Output (in Workload Categories)
Ministry of Finance
Quality Court User
Speed Court User
Systems (HR, IT, Structures, Workflows)
Internal use
Dr Axel G Koetz, Managing Partner, Koetz Partner International, Unicenter 2920, D-50939 Cologne Germany, e-mail axel.koetz @ koetz-ag.comCourt Statistics, Judge Workload Analysis, Quality and Performance Management – Ankara (CoE) 25 March 2011
51KPI Management and Policy Consultants
Conference on Performance Evaluation of the JudiciaryAnkara, 25 March 2011
The Basic Approach – Balanced Scorecard
Multi-Dimensionality of „Performance“ in the Court system
Interaction ofperformance dimensions
Strategic long term controlling and optimizationBased on customer / stakeholder goals
Dr Axel G Koetz, Managing Partner, Koetz Partner International, Unicenter 2920, D-50939 Cologne Germany, e-mail axel.koetz @ koetz-ag.comCourt Statistics, Judge Workload Analysis, Quality and Performance Management – Ankara (CoE) 25 March 2011
52KPI Management and Policy Consultants
Conference on Performance Evaluation of the JudiciaryAnkara, 25 March 2011
A BSC Approach takes Multi-Dimensionality and Interaction into consideration
• Take Multi-Dimensionality into consideration
• There is not one single goal, there are more than one, usually– Our financial situation today– Satisfaction of the customers to ensure our
profit of tomorrow– Staff capacity to keep satisfaction and income– Investments in the future to ensure the profit
of next year• Take Interaction into consideration
– Investment in Staff, Systems, Innovation today improves performance in the future
Dr Axel G Koetz, Managing Partner, Koetz Partner International, Unicenter 2920, D-50939 Cologne Germany, e-mail axel.koetz @ koetz-ag.comCourt Statistics, Judge Workload Analysis, Quality and Performance Management – Ankara (CoE) 25 March 2011
53KPI Management and Policy Consultants
Conference on Performance Evaluation of the JudiciaryAnkara, 25 March 2011
BSC – From Private Sector Needs to Court Needs
Company• Profit• Customer Satisfaction• Trained + Motivated Staff• New Products / Markets
Court• Court productivity• Speed / Minimized Backlogs• Trained and motivated staff• Minimized (successful) appeals
Elaborate Goals for all DimensionsSupervise goals continuosly by IndicatersDont allow the priority of one dimensionCare for a „balanced“ score
Dr Axel G Koetz, Managing Partner, Koetz Partner International, Unicenter 2920, D-50939 Cologne Germany, e-mail axel.koetz @ koetz-ag.comCourt Statistics, Judge Workload Analysis, Quality and Performance Management – Ankara (CoE) 25 March 2011
54KPI Management and Policy Consultants
Conference on Performance Evaluation of the JudiciaryAnkara, 25 March 2011
Productivity Indicator: Hours per weighted standard case in the District courts SAMPLE
HoursjudgeTime per case
Jud
ge
FT
PE
pe
r C
ou
rt
9
0
5
10
15
20
25
30
0 0,5 1 1,5 2 2,5 3 3,5 4 4,5 5
OD
KH
LV
KO
CY
DZ
9
0
5
10
15
20
25
30
0 0,5 1 1,5 2 2,5 3 3,5 4 4,5 5
Dr Axel G Koetz, Managing Partner, Koetz Partner International, Unicenter 2920, D-50939 Cologne Germany, e-mail axel.koetz @ koetz-ag.comCourt Statistics, Judge Workload Analysis, Quality and Performance Management – Ankara (CoE) 25 March 2011
55KPI Management and Policy Consultants
Conference on Performance Evaluation of the JudiciaryAnkara, 25 March 2011
Case Time Indicator:Cumulated Civil Cases Duration in
Months
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 24 36 60 120
mor
e
1-Cum
2-Cum
Dr Axel G Koetz, Managing Partner, Koetz Partner International, Unicenter 2920, D-50939 Cologne Germany, e-mail axel.koetz @ koetz-ag.comCourt Statistics, Judge Workload Analysis, Quality and Performance Management – Ankara (CoE) 25 March 2011
56KPI Management and Policy Consultants
Conference on Performance Evaluation of the JudiciaryAnkara, 25 March 2011
Case Time Indicator (2):Cumulated Admin Cases Duration in Months
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
1 2 3 4 5 6 7 8 9
10
11
12
13
14
15
16
17
18 24
60
12
0
mo
re
1-CUM
2-CUM
Dr Axel G Koetz, Managing Partner, Koetz Partner International, Unicenter 2920, D-50939 Cologne Germany, e-mail axel.koetz @ koetz-ag.comCourt Statistics, Judge Workload Analysis, Quality and Performance Management – Ankara (CoE) 25 March 2011
57KPI Management and Policy Consultants
Conference on Performance Evaluation of the JudiciaryAnkara, 25 March 2011
Quality Indicator: Appeals and District Court Quality
988,824(100%)
97,275(9.8%)
LocalCourt
Decisions
Appeals(ProceduralMistakes)
SuccessfulAppeals
max
min
Wrong Decision ?Missing Knowledge of new Jurisdiction ?Bad Knowledge of Procedures ?Bad Communication ?Greedy Lawyers ? Wrong Case
InvestigationWrong Law ApplicationFailure in ProceduresDifferent Legal Opinion
25,587(26.3%)
Dr Axel G Koetz, Managing Partner, Koetz Partner International, Unicenter 2920, D-50939 Cologne Germany, e-mail axel.koetz @ koetz-ag.comCourt Statistics, Judge Workload Analysis, Quality and Performance Management – Ankara (CoE) 25 March 2011
58KPI Management and Policy Consultants
Conference on Performance Evaluation of the JudiciaryAnkara, 25 March 2011
Systems Indicators in Human Resources and IT
Human Resources:
Staff Turnover (Retention ofqualified staff is a success indicator, so the value should be low.
Training days (More training improves staff quality, leads to more output, higher speed, less failures
ICT Systems
Different development stages exist and can be measured by a IT quality indicator
In general, the more staff has access to a server based intranet with common databases and functions and central internet access,• the higher the staff productivity,• the higher the work speed,• the lower the share of failures
Dr Axel G Koetz, Managing Partner, Koetz Partner International, Unicenter 2920, D-50939 Cologne Germany, e-mail axel.koetz @ koetz-ag.comCourt Statistics, Judge Workload Analysis, Quality and Performance Management – Ankara (CoE) 25 March 2011
59KPI Management and Policy Consultants
Conference on Performance Evaluation of the JudiciaryAnkara, 25 March 2011
BSC (Standard Form) For a Sample Court
Productivity
872
Weighted (standard) cases per judge/year
Speed
144,85 DAYS to decisive meeting
Staff Turnover
9,96 %(new + leaving/2) staff
Of total staff p.a.
Quality
3,17%Successful Appeals of
Considered cases
Goal ???City average 847
Region average 977Region best 25%
1.0724 Pilots Average 819
Dr Axel G Koetz, Managing Partner, Koetz Partner International, Unicenter 2920, D-50939 Cologne Germany, e-mail axel.koetz @ koetz-ag.comCourt Statistics, Judge Workload Analysis, Quality and Performance Management – Ankara (CoE) 25 March 2011
60KPI Management and Policy Consultants
Conference on Performance Evaluation of the JudiciaryAnkara, 25 March 2011
BSC Diagram - Model
Dr Axel G Koetz, Managing Partner, Koetz Partner International, Unicenter 2920, D-50939 Cologne Germany, e-mail axel.koetz @ koetz-ag.comCourt Statistics, Judge Workload Analysis, Quality and Performance Management – Ankara (CoE) 25 March 2011
61KPI Management and Policy Consultants
Conference on Performance Evaluation of the JudiciaryAnkara, 25 March 2011
BSC Table - Model
Introduce government goalsfor this
Dr Axel G Koetz, Managing Partner, Koetz Partner International, Unicenter 2920, D-50939 Cologne Germany, e-mail axel.koetz @ koetz-ag.comCourt Statistics, Judge Workload Analysis, Quality and Performance Management – Ankara (CoE) 25 March 2011
62KPI Management and Policy Consultants
Conference on Performance Evaluation of the JudiciaryAnkara, 25 March 2011
A View to Reality – Denmark (1)
Dr Axel G Koetz, Managing Partner, Koetz Partner International, Unicenter 2920, D-50939 Cologne Germany, e-mail axel.koetz @ koetz-ag.comCourt Statistics, Judge Workload Analysis, Quality and Performance Management – Ankara (CoE) 25 March 2011
63KPI Management and Policy Consultants
Conference on Performance Evaluation of the JudiciaryAnkara, 25 March 2011
A View to Reality – Denmark (2)
Інформація, яку можна отримати завдяки показникам. Вживання відповідних заходів.
Dr Axel G Koetz, Managing Partner, Koetz Partner International, Unicenter 2920, D-50939 Cologne Germany, e-mail axel.koetz @ koetz-ag.comCourt Statistics, Judge Workload Analysis, Quality and Performance Management – Ankara (CoE) 25 March 2011
64KPI Management and Policy Consultants
Conference on Performance Evaluation of the JudiciaryAnkara, 25 March 2011
Performance and Management
• Put instruments in place to identify the real reasons for performance problems in whatever BSC area
• Give HoC the Instruments to identify and solve problems in their own courts
• Try to create Incentives for good performance• Re-arrage the staff distribution between courts in
a region if needed• Develop plans to improve the „System“ situation
(HR and IT)
Dr Axel G Koetz, Managing Partner, Koetz Partner International, Unicenter 2920, D-50939 Cologne Germany, e-mail axel.koetz @ koetz-ag.comCourt Statistics, Judge Workload Analysis, Quality and Performance Management – Ankara (CoE) 25 March 2011
65KPI Management and Policy Consultants
Conference on Performance Evaluation of the JudiciaryAnkara, 25 March 2011
Performance and Funding
• Assign Budget and Posts according to the weighted caseload of the courts (we need the case complexity based workload determination systems)
• Have an instrument in place to give relief to courts with problems (Judge Pool – retain a percentage of open posts for this)
• Consequently withdraw posts from „inefficient“ courts and put judges to courts with work overload.
• Put a bonus system in place for very efficient courts, giving extra funding for example for IT, materials and so on.
Dr Axel G Koetz, Managing Partner, Koetz Partner International, Unicenter 2920, D-50939 Cologne Germany, e-mail axel.koetz @ koetz-ag.comCourt Statistics, Judge Workload Analysis, Quality and Performance Management – Ankara (CoE) 25 March 2011
66KPI Management and Policy Consultants
Conference on Performance Evaluation of the JudiciaryAnkara, 25 March 2011
Feedback Loop and „Continuous Improvement“
• Performance management should not end with isolated reactions on individual indicator problems.
• Follow up of results over the time and the analysis of the effects of changes is necessary
• Continous improvement has to be a key element:– You never reach an „Optimal“ result and it is in no case in one
step– If you implemented one improvement, you see what can be
done next– Actively promote re-thinking all achievements– Use BSC Indicators to measure the effect of each step
• Knowledge exchange Groups should be established to learn from each other and to improve the indicators continuously
Dr Axel G Koetz, Managing Partner, Koetz Partner International, Unicenter 2920, D-50939 Cologne Germany, e-mail axel.koetz @ koetz-ag.comCourt Statistics, Judge Workload Analysis, Quality and Performance Management – Ankara (CoE) 25 March 2011
KPI Management and Policy Consultants
Conference on Performance Evaluation of the JudiciaryAnkara, 25 March 2011
End of the PresentationThank you for your Attention
Questions:Dr Axel G Koetz
Managing Partner, KPI CologneE-mail: [email protected]