European Statistics System Peer Review Emma Wright & Laura Clarke Survey Methodology & Statistical...

European Statistics SystemPeer Review

Emma Wright & Laura Clarke

Survey Methodology & Statistical Computing, ONS

GSS Methodology Symposium, 1 July 2015

Outline

• European Statistics Code of Practice

• ESS Peer Review Process

• UK Assessment

• Recommendations for UK Statistical System

• How does the UK compare to other EU countries?

European Statistics Code of Practice

• First adopted by the European Statistical System Committee (ESSC) in February 2005

• The Code was revised in 2011 and adopted by the ESSC in September 2011


• Aims to ensure that ESS statistics are relevant, timely, accurate, and comply with principles of professional independence, impartiality & objectivity

• Code is based on 15 principles

15 Principles of European Code of PracticeProfessional

IndependenceMandate for Data

CollectionAdequacy of Resources

Commitment to Quality

Statistical Confidentiality

Impartiality and Objectivity

Sound methodology Appropriate statistical procedures

Non-excessive burden on

respondents

Cost Effectiveness

Relevance Accuracy and Reliability

Timeliness and Punctuality

Coherence and Comparability

Accessibility and Clarity


• Aims to ensure that ESS statistics are relevant, timely, accurate, and comply with principles of professional independence, impartiality & objectivity

• Code is based on 15 principles concerning the institutional environment, statistical processes and outputs

15 Principles of European Code of PracticeProfessional

IndependenceMandate for Data

CollectionAdequacy of Resources

Commitment to Quality

Statistical Confidentiality

Impartiality and Objectivity

Sound methodology Appropriate statistical procedures

Non-excessive burden on

respondents

Cost Effectiveness

Relevance Accuracy and Reliability

Timeliness and Punctuality

Coherence and Comparability

Accessibility and Clarity


Principle Description & Indicators

Mandate for Data Collection

Statistical authorities have a clear legal mandate to collect information for European statistical purposes. Administrations, enterprises and households, and the public at large may be compelled by law to allow access or to deliver data for European statistical purposes at the request of the statistical authorities• The mandate of the statistical authorities (SAs) to collect information for the development, production and dissemination of European Statistics is specified in law.• The SAs are allowed by law to use administrative data for statistical purposes.• On the basis of a legal act, the SAs may compel response to statistical surveys.

• For each of the 15 principles, a set of indicators of good practice for provides a reference for measuring the implementation of the Code, for example:

• Quality Assurance Framework of the European Statistical System (ESS QAF) developed to assist implementation of European Code of Practice; it identifies possible activities, methods and tools that can provide guidance and evidence for the implementation of the indicators

ESS Peer Review - Background (1)

• ESS Peer Review programme has two objectivesTo assess credibility of the ESSTo reassure stakeholders about the quality and

trustworthiness of the statistics produced by the ESS

• First ESS Peer Review of UK Statistical System in 2007 only looked at 7 of the 15 principles of European Code – recommendations concerned:

• Improving the website• Standardising/modernising statistical and IT systems• Developing electronic data collection• Quality reporting

ESS Peer Reviews – Background (2)

• ESSC agreed to second round of Peer Reviews in 2012, this time assessing compliance all 15 principles of European Code

• Methodology developed by Task Force and approved by ESSC in 2013:

• Self-assessment questionnaires completed by NSIs• Follow-up visit by Peer Reviewers• Participation by ‘Other National Authorities’ (ONAs)• Reporting, Recommendations and Improvement Actions

• Peer Reviews commenced autumn 2014 and will be completed summer 2015

UK Peer Review Assessment

• Three self-assessment questionnaires (SAQs) completed by ONS/Authority (as NSI) in April 2014:

• Compliance with the Code of Practice (128 pages!)• Coordination role of the NSI within the National

Statistical System• Cooperation/level of integration achieved by the ESS

• Three ONAs (DEFRA, HMRC and Scottish Government) also completed short questionnaires

• Follow-up visit by Peer Reviewers in January 2015

• Peer Review report for UK published March 2015 and UK Improvement Actions published June 2015

Peer Review Visit to the UK

• 5th – 9th January 2015, Newport (with video link to London as required)

• Assessors:• Hallgrimur Snorrason (Iceland)• Jan Byfuglien (Norway)• Hilkka Vihavainen (Finland)

• Over the course of the week, the assessors met with:• John Pullinger, Glen Watson and Directors• Sir Andrew Dilnot and Ed Humpherson• ONS staff (14 DDs and 16 other “experts”)• ONAs (DEFRA, HMRC and Scottish Government)• 12 “junior” staff (SEO and below)• Main Users (HMT, BoE, HO, DH, SUF, BBC)• Scientific Community (ESRC, RSS, DUG, Southampton Uni)

Strengths of UK statistical system identified by Peer Reviewers

• UK Code of Practice and UKSA Assessment

• UK Statistics Authority monitoring of use of statistics by government

• Coordination and integration within GSS

• Professional independence, transparency and openness

• Good relationships with users, media and academia

• Emphasis on quality

• Training and development

Recommendations to further compliance with European Code

• 16 recommendations as to how compliance with the European Code could be improved under the following headings:

Using administrative data

Restricting pre-release access

Increasing the efficiency of the statistical system

Making UK statistics more accessible

Improving the clarity, coherence and independence of the UK statistical system

Recommendations & Actions

Using Administrative Data (1 recommendation):

• UKSA/ONS should continue to seek agreements on new legislation to authorise, encourage and facilitate the use of admin data for statistical purposes


Using Administrative Data

• UKSA/ONS should continue to seek agreements on new legislation to authorise, encourage and facilitate the use of admin data for statistical purposes

Improvement Action:

Engagement with Cabinet Office to push for policy change in next Parliament; In meantime, will utilise existing legislation to pursue new data sharing opportunities


Restricting Pre-release Access (1 recommendation):

• Current pre-release access practices should be (at a minimum) severely restricted, and ultimately abolished


Restricting Pre-release Access

• Current pre-release access practices should be (at a minimum) severely restricted, and ultimately abolished

Improvement Action:

UK Statistics Authority welcomes recommendation and will continue to seek abolition of pre-release access and to publically speak out against it as contrary to principle of equality of access


Increasing the efficiency of the statistical system (6 recommendations):

• Implement standardised software/tools across production processes, including edit & imputation

• Ensure that descriptions of divergences from existing European methodologies should cover all ES

• Compulsory use of Standards and Guidance database to assist in standardisation of methods/procedures/tools

• Resume regular consultations with business data suppliers

• Intensify efforts to modernise data collection in order to reduce costs & burden

• Review the structure/content of metadata and ensure consistency with European standards


Increasing the efficiency of the statistical system

Improvement Actions:

• Further develop standardised platforms for statistical production within ONS

• Identify where statistics feed into ES and explain divergences from European Statistics methodologies

• Create Board to manage respondent burden and re-launch small business forum

• Continue Electronic Data Programme and further explore benefits of using admin/big data to reduce data collection

• Develop metadata standards and undertake work to better understand user need for quality information


Making UK statistics more accessible (4 recommendations):

• Implement solutions at GSS level to provide an enhanced overview of and access to UK statistics in general

• Continue and intensify efforts to upgrade ONS website and offer more flexible possibilities for data extraction

• Initiate actions at GSS level to establish a common template for describing the metadata of different outputs

• Develop and implement remote access to microdata for research purposes


Making UK statistics more accessible


• Release calendar on GOV.UK already delivered; Improve user experience around looking for official statistics online

• Develop Beta version of new ONS website building on Alpha prototype

• Upgrade Virtual Microdata Laboratory (VML) to be accessible to Public Services Network (PSN) users


Improving the clarity, coherence and independence of the UK statistical system (4 recommendations):

• Continue to systematically engage with ministers and HOPs on the use of the designation of official/National statistics

• Continue efforts to sharpen the boundaries between statistical and administrative roles within the GSS

• Strengthen efforts to harmonise UK statistics to ensure coherence and comparability within UK, over time and among producers

• Review and update the UK Code of Practice and harmonise with European Code


Improving the clarity, coherence and independence of the UK statistical system


• Publish statement about National Statistics including latest thinking on boundary between official statistics and other quantitative information

• Review and confirm roles and responsibilities of statistical Heads of Profession

• Continue to progress and promote harmonisation agenda, especially business surveys & admin data

• Publish report on impact of UK Code since 2009 and view on whether should be revised

•

What’s next?

• Requirement for progress against improvement actions to be reported on an annual basis over next four years

• Peer Reviews continue into the summer, and a final report looking across all EU countries will be published by Eurostat in the autumn

Further InformationEuropean Statistics Code of Practice:

http://ec.europa.eu/eurostat/web/quality/european-statistics-code-of-practice

Peer Review Reports & Improvement Actions:http://ec.europa.eu/eurostat/web/quality/peer-reviews



http://ec.europa.eu/eurostat/web/quality/peer-reviews

http://ec.europa.eu/eurostat/web/quality/peer-reviews

How does the UK compare with other EU countries?

France:18 recommendations...

• quality reporting• access to microdata• metadata• legal independence

Germany:28 recommendations...

• standardisation• IT security• admin data• quality reporting & metadata• legal independence


Spain:14 recommendations...

• access to microdata• security of microdata• standardisation• admin data

Bulgaria:28 recommendations...

• staff skills• project management• quality management• management style• user consultation


Sweden:18 recommendations...

• user consultation• user satisfaction survey• electronic data collection• admin data

Slovenia:17 recommendations...

• quality reviews• respondent burden• access to microdata• fixed release dates• adequacy of funding

Thank you.........

???Any Questions?

???

European Statistics System Peer Review Emma Wright & Laura Clarke Survey Methodology & Statistical...

Documents

Transcript of European Statistics System Peer Review Emma Wright & Laura Clarke Survey Methodology & Statistical...