Quality Assessments of Statistical Production Processes in Eurostat Pierre Ecochard and Małgorzata...
14
Quality Assessments of Statistical Production Processes in Eurostat Pierre Ecochard and Małgorzata Szczęsna [email protected] , [email protected] a.eu
-
Upload
rudolf-fleming -
Category
Documents
-
view
220 -
download
0
Transcript of Quality Assessments of Statistical Production Processes in Eurostat Pierre Ecochard and Małgorzata...
Quality Assessments of Statistical Production Processes in Eurostat
Pierre Ecochard and Małgorzata Szczęsna
Quality assessment in the ESS: 2005-2007
6
5
3
0
14
12
4
3
10
11
16
15
0
1
7
11
quality reports?
quality indicators?
self-assessments?
quality audits?
All
Most (≈25 to 75%)
Some (< 25%)
None
“Over the last three years (2005-2007), how many of your statistical processes have been monitored using:”
Quality assessment in the ESS: 2008-2010
10
9
4
0
18
17
14
13
2
4
8
12
0
0
3
4
quality reports?
quality indicators?
self-assessments?
quality audits?
All
Most (≈25 to 75%)
Some (< 25%)
None
“Over the next three years (2008-2010), how many of your statistical processes do you plan to monitor using:”
Audits and self-assessments in the ESS
Number of NSIs conducting:
Audits SAs
5 NSIs 9 NSIs 10 NSIs
7 NSIs
The European StatisticsCode of Practice
The Eurostat Quality Assurance Framework
DocumentationMeasurement
Evaluation Conformity
•Quality reports•Quality indics•Process descriptions•Process variablesetc
Quality Assessments Labelling
Background
What is Eurostat Quality Assessment?
A systematic review and evaluation of all stages of the statistical production process with the use of the DESAP-
based Checklist
IT conditions; Management, planning and legislation; Staff, work conditions and competence
User needs
Data
collection Validation Confidentiality
Dissemination
Documentation
Follow-up
Assessment Outputs (1)
Assessment Report
Assessment Outputs (2)
Coherence General coherence
Timeliness Timeliness of final publication
Accuracy Overall accuracy
Relevance User satisfaction
Accessibility and clarityOverall quality of metadata
Comparability
Comparabilityover time
Comparabilityacross countries
0
1
2
3
4
5
Assessment Diagram
Assessment Outputs (3)
Highlight of good practice
Categories of Eurostat assessments
Self-Assessment
Supported Self-Assessment
Peer Review Rolling Review
Process 1Process 4
Process 2
Process 3
Characteristics:- Periodicity- Legal Basis- Output- ESTAT intervention
Similiarity: DESAP-based Checklist, outputsDifference: extent of external interventation in a review
Process 5
Office-wide implementation plan
Approach piloted in two domains in 2007 Most of the statistical process will benefit from an
assessment within a three-year period 2008-2010 33 reviews planned for 2008:
– 14 Supported Self-Assessments– 14 Self-Assessments– 4 Rolling Reviews– 1 Peer Review
Follow-up report to Eurostat management by the end of 2008
Benefits of quality assessment
For production teams: An opportunity for a
systematic analysis of the production process
Identify and prioritize improvement actions
Spread and benefit from the Good Practice
For Eurostat: Identify horizontal
problematic issues Foster standardisation of
statistical processes Support resource allocation,
planning and programming Show quality commitment
Feedback
The general workflow works well, the Checklist is flexible and the assessments are considered useful by domain managers
Heads of Unit should be involved earlier The diagram can be a red herring: it should be
used with caution The ownership of the results should be made very
clear More assessment should involve an external expert
Keys for success in implementing quality assessments
Top management commitment
Middle management acceptance
Sound communication
Long term perspectives
Implementation and fine-tuning in pilot projects
Standardised use of methods
Clear responsibilities and ownership
Sufficient resources allocated for the assessments
