ESS Vision 2020 ADMIN: Assessing Quality · WP7. Methodological support WP4. Eurostat as user of EC...
Transcript of ESS Vision 2020 ADMIN: Assessing Quality · WP7. Methodological support WP4. Eurostat as user of EC...
![Page 1: ESS Vision 2020 ADMIN: Assessing Quality · WP7. Methodological support WP4. Eurostat as user of EC data WP1. Access to & development ... Governance technical ADMIN network Task Force](https://reader035.fdocuments.net/reader035/viewer/2022063009/5fc021b9ff4f0261fb593e3a/html5/thumbnails/1.jpg)
Eurostat Eurostat
ESS Vision 2020 ADMIN: Assessing Quality
Fabian BACH
ADMIN project manager
Eurostat Unit F.2 – Population and migration
UNECE-Eurostat Expert Meeting on Censuses
Geneva, 27 September 2018
![Page 2: ESS Vision 2020 ADMIN: Assessing Quality · WP7. Methodological support WP4. Eurostat as user of EC data WP1. Access to & development ... Governance technical ADMIN network Task Force](https://reader035.fdocuments.net/reader035/viewer/2022063009/5fc021b9ff4f0261fb593e3a/html5/thumbnails/2.jpg)
ADMIN data in multisource statistics
Lin
kag
e
Sta
tistic
al
reg
iste
r
Outputs
Po
pu
latio
n
Reg
iste
r
Tran
sfo
rm
atio
n
`
use for survey frames: directly or to supplement / update
(partly) replace data collection: e.g. business tax data survey
direct tabulation
Graph idea: ONS
Registers
Admin. data
Surveys
Full enumeration
![Page 3: ESS Vision 2020 ADMIN: Assessing Quality · WP7. Methodological support WP4. Eurostat as user of EC data WP1. Access to & development ... Governance technical ADMIN network Task Force](https://reader035.fdocuments.net/reader035/viewer/2022063009/5fc021b9ff4f0261fb593e3a/html5/thumbnails/3.jpg)
ADMIN data in multisource statistics
Lin
kag
e
Sta
tistic
al
reg
iste
r
Outputs
Po
pu
latio
n
Reg
iste
r
Tran
sfo
rm
atio
n
`
Graph idea: ONS
Registers
Admin. data
Surveys
Full enumeration
(mostly) national level! (ESS) quality??
Inputs
Key ADMIN challenge
![Page 4: ESS Vision 2020 ADMIN: Assessing Quality · WP7. Methodological support WP4. Eurostat as user of EC data WP1. Access to & development ... Governance technical ADMIN network Task Force](https://reader035.fdocuments.net/reader035/viewer/2022063009/5fc021b9ff4f0261fb593e3a/html5/thumbnails/4.jpg)
• as a single variable
• editing, imputation, validation
• indirect use
(calibration, estimation, …)
• in evaluation, including data
confrontation
(e.g. comparison of survey
estimates with related ADMIN
estimates)
Generic Statistical
Business Process Model =
Multisource data: use examples
![Page 5: ESS Vision 2020 ADMIN: Assessing Quality · WP7. Methodological support WP4. Eurostat as user of EC data WP1. Access to & development ... Governance technical ADMIN network Task Force](https://reader035.fdocuments.net/reader035/viewer/2022063009/5fc021b9ff4f0261fb593e3a/html5/thumbnails/5.jpg)
ESS Vision 2020 ADMIN project (2015-2019)
WP2. Statistical methods
WP5. Frames for social statistics
WP6. Pilot studies &
applications
WP7. Methodological
support
WP4. Eurostat as user of EC data
WP1. Access to & development
of sources
Approved by ESSC: support the ESS to make better use of available ADMIN data without compromising on quality
Work
Packages (
WP)
WP3. Quality
![Page 6: ESS Vision 2020 ADMIN: Assessing Quality · WP7. Methodological support WP4. Eurostat as user of EC data WP1. Access to & development ... Governance technical ADMIN network Task Force](https://reader035.fdocuments.net/reader035/viewer/2022063009/5fc021b9ff4f0261fb593e3a/html5/thumbnails/6.jpg)
ADMIN: key results
Multisource combination +
estimation methods
Access to ADMIN sources
Use of registers for frames
2019 Q2/2018
Quality of frames for social statistics today!
Multisource quality
Guidelines
WP6 grant summary reports social/business/agricultural stats. + quality lessons learned + knowledge transfer
Best practices
Incl. 2017, 2018 waves
![Page 7: ESS Vision 2020 ADMIN: Assessing Quality · WP7. Methodological support WP4. Eurostat as user of EC data WP1. Access to & development ... Governance technical ADMIN network Task Force](https://reader035.fdocuments.net/reader035/viewer/2022063009/5fc021b9ff4f0261fb593e3a/html5/thumbnails/7.jpg)
ADMIN: key results
Multisource combination +
estimation methods
Access to ADMIN sources
Use of registers for frames
Quality of frames for social statistics today!
Multisource quality
Guidelines
WP6 grant summary reports social/business/agricultural stats. + quality lessons learned + knowledge transfer
Best practices
Incl. 2017, 2018 waves
Stress on social/pop./ census statistics through all ADMIN!
![Page 8: ESS Vision 2020 ADMIN: Assessing Quality · WP7. Methodological support WP4. Eurostat as user of EC data WP1. Access to & development ... Governance technical ADMIN network Task Force](https://reader035.fdocuments.net/reader035/viewer/2022063009/5fc021b9ff4f0261fb593e3a/html5/thumbnails/8.jpg)
Reports to ESS Directors
(methodology, social statistics)
ADMIN: Involving ESS Members
Quality of multisource statistics (DK, IE, IT, LT, HU, NL, AT, NO)
• Stock-taking • New indicators • Methodological framework quality guidelines (x2)
ADMIN Steering Group
Governance technical
ADMIN network
Task Force Frames
ESSnet KOMUSO
Workshops
![Page 9: ESS Vision 2020 ADMIN: Assessing Quality · WP7. Methodological support WP4. Eurostat as user of EC data WP1. Access to & development ... Governance technical ADMIN network Task Force](https://reader035.fdocuments.net/reader035/viewer/2022063009/5fc021b9ff4f0261fb593e3a/html5/thumbnails/9.jpg)
ADMIN WP3: Quality (KOMUSO)
Komusō (image: Wikipedia)
ESSnet
Task 3.1 – Checklists for
evaluating the quality of input data
Task 3.2 – Definition of a
framework for the quality evaluation of statistical outputs using administrative data
Task 3.3 – Dissemination and
practical implementation of main results obtained in tasks 3.1 and 3.2 write guidelines!
https://ec.europa.eu/eurostat/cros/content/essnet-quality-multisource-statistics-komuso_en
![Page 10: ESS Vision 2020 ADMIN: Assessing Quality · WP7. Methodological support WP4. Eurostat as user of EC data WP1. Access to & development ... Governance technical ADMIN network Task Force](https://reader035.fdocuments.net/reader035/viewer/2022063009/5fc021b9ff4f0261fb593e3a/html5/thumbnails/10.jpg)
2015 until 04/2017:
Outputs published:
KOMUSO Stage 1: Preparatory Work
Take stock on output quality assessment +
reporting Review output quality measures
Review current practices on input quality assessment
• checklists for evaluating the quality of input data
• framework: 6 basis data source configurations
• quality measures and indicators (for frames)
Review existing practices on frames
quality
1st ADMIN quality workshop
(Budapest, 04/2016)
![Page 11: ESS Vision 2020 ADMIN: Assessing Quality · WP7. Methodological support WP4. Eurostat as user of EC data WP1. Access to & development ... Governance technical ADMIN network Task Force](https://reader035.fdocuments.net/reader035/viewer/2022063009/5fc021b9ff4f0261fb593e3a/html5/thumbnails/11.jpg)
6 basis data source configurations:
1) Non-overlapping microdata without coverage problems
2) Overlapping microdata without coverage problems + (2S) including sample surveys
3) Overlapping microdata with under-coverage
4) Combining microdata + macrodata sources
5) Combining only macrodata sources
6) Longitudinal data
Selected to reflect the most common/typical scenarios
KOMUSO Stage 1: Preparatory Work
![Page 12: ESS Vision 2020 ADMIN: Assessing Quality · WP7. Methodological support WP4. Eurostat as user of EC data WP1. Access to & development ... Governance technical ADMIN network Task Force](https://reader035.fdocuments.net/reader035/viewer/2022063009/5fc021b9ff4f0261fb593e3a/html5/thumbnails/12.jpg)
KOMUSO Stage 2: Guidelines
04/2017 until 09/2018:
Outputs published (continuing stage 1):
Finalise draft frames guidelines
Continue work on measures/indicators
by data config.
1st progress draft of multisource guidelines
• Quality measures and calculation methods (QMCMs)
• Quality quidelines for frames of social stats.: QGFSS v1.0
• Quality guidelines for multisource statistics: QGMSS v0.7
accuracy (18) coherence (3) timeliness (1)
![Page 13: ESS Vision 2020 ADMIN: Assessing Quality · WP7. Methodological support WP4. Eurostat as user of EC data WP1. Access to & development ... Governance technical ADMIN network Task Force](https://reader035.fdocuments.net/reader035/viewer/2022063009/5fc021b9ff4f0261fb593e3a/html5/thumbnails/13.jpg)
Structure + drafting progress of the QGMSS v0.7:
KOMUSO Stage 2: Guidelines
![Page 14: ESS Vision 2020 ADMIN: Assessing Quality · WP7. Methodological support WP4. Eurostat as user of EC data WP1. Access to & development ... Governance technical ADMIN network Task Force](https://reader035.fdocuments.net/reader035/viewer/2022063009/5fc021b9ff4f0261fb593e3a/html5/thumbnails/14.jpg)
Structure + drafting progress of the QGMSS v0.7:
KOMUSO Stage 2: Guidelines
…
…
QMCMs (annex)
Same structure for each section in Part 2:
![Page 15: ESS Vision 2020 ADMIN: Assessing Quality · WP7. Methodological support WP4. Eurostat as user of EC data WP1. Access to & development ... Governance technical ADMIN network Task Force](https://reader035.fdocuments.net/reader035/viewer/2022063009/5fc021b9ff4f0261fb593e3a/html5/thumbnails/15.jpg)
Excursion: structure of QMCMs (annex to QGMSS)
1) Name of the quality measure/indicator
2) General information
3) Quality measure & computation method in general
4) Situation & computation method in detail
5) References
+ hands-on example where needed
KOMUSO Stage 2: Guidelines
![Page 16: ESS Vision 2020 ADMIN: Assessing Quality · WP7. Methodological support WP4. Eurostat as user of EC data WP1. Access to & development ... Governance technical ADMIN network Task Force](https://reader035.fdocuments.net/reader035/viewer/2022063009/5fc021b9ff4f0261fb593e3a/html5/thumbnails/16.jpg)
E.g. QMCM_A_10 (Accuracy example 10):
KOMUSO Stage 2: Guidelines
… …
1)
2)
3)
![Page 17: ESS Vision 2020 ADMIN: Assessing Quality · WP7. Methodological support WP4. Eurostat as user of EC data WP1. Access to & development ... Governance technical ADMIN network Task Force](https://reader035.fdocuments.net/reader035/viewer/2022063009/5fc021b9ff4f0261fb593e3a/html5/thumbnails/17.jpg)
E.g. QMCM_A_10 (Accuracy example 10):
KOMUSO Stage 2: Guidelines
… …
(config. 2)
1)
2)
3)
![Page 18: ESS Vision 2020 ADMIN: Assessing Quality · WP7. Methodological support WP4. Eurostat as user of EC data WP1. Access to & development ... Governance technical ADMIN network Task Force](https://reader035.fdocuments.net/reader035/viewer/2022063009/5fc021b9ff4f0261fb593e3a/html5/thumbnails/18.jpg)
E.g. QMCM_A_10 (Accuracy example 10):
4)
5)
KOMUSO Stage 2: Guidelines
![Page 19: ESS Vision 2020 ADMIN: Assessing Quality · WP7. Methodological support WP4. Eurostat as user of EC data WP1. Access to & development ... Governance technical ADMIN network Task Force](https://reader035.fdocuments.net/reader035/viewer/2022063009/5fc021b9ff4f0261fb593e3a/html5/thumbnails/19.jpg)
E.g. QMCM_A_10 (Accuracy example 10):
4)
5)
KOMUSO Stage 2: Guidelines
Detailed calculation algorithm/recipe
![Page 20: ESS Vision 2020 ADMIN: Assessing Quality · WP7. Methodological support WP4. Eurostat as user of EC data WP1. Access to & development ... Governance technical ADMIN network Task Force](https://reader035.fdocuments.net/reader035/viewer/2022063009/5fc021b9ff4f0261fb593e3a/html5/thumbnails/20.jpg)
E.g. QMCM_A_10 + hands-on example:
KOMUSO Stage 2: Guidelines
…
![Page 21: ESS Vision 2020 ADMIN: Assessing Quality · WP7. Methodological support WP4. Eurostat as user of EC data WP1. Access to & development ... Governance technical ADMIN network Task Force](https://reader035.fdocuments.net/reader035/viewer/2022063009/5fc021b9ff4f0261fb593e3a/html5/thumbnails/21.jpg)
E.g. QMCM_A_10 + hands-on example:
KOMUSO Stage 2: Guidelines
…
11 steps: Raw input data Combinded DB Final data pool
Multisource quality propagation!
![Page 22: ESS Vision 2020 ADMIN: Assessing Quality · WP7. Methodological support WP4. Eurostat as user of EC data WP1. Access to & development ... Governance technical ADMIN network Task Force](https://reader035.fdocuments.net/reader035/viewer/2022063009/5fc021b9ff4f0261fb593e3a/html5/thumbnails/22.jpg)
KOMUSO Stage 3: Completion
10/2018 until 09/2019:
Outputs envisaged (continuing stages 1+2):
Finalise measures/indicators QMCMs by data config.
Finalise draft multisource
guidelines (QGMSS)
accuracy contd. (+6) coherence contd. reliability (1) + hands-on examples
• QMCMs contd. incl. hands-on examples
• QGMSS v1.0 tentatively June 2019
2nd ADMIN quality workshop
(winter '18 tbc.)
![Page 23: ESS Vision 2020 ADMIN: Assessing Quality · WP7. Methodological support WP4. Eurostat as user of EC data WP1. Access to & development ... Governance technical ADMIN network Task Force](https://reader035.fdocuments.net/reader035/viewer/2022063009/5fc021b9ff4f0261fb593e3a/html5/thumbnails/23.jpg)
ADMIN support to Member States
• Workshops: quality + social statistics + other
• (WP6: Grants for ADMIN pilots in Member States)
• WP7: Methodological support
ADMIN helpdesk for NSIs http://ec.europa.eu/eurostat/cros/content/ess-admin-helpdesk_en
Coaching for NSIs (on-site expert support) request through ADMIN helpdesk
ESTP training linked to ADMIN in 2017-2019
winter 2018 tbc.
spring 2019 tbd.
![Page 24: ESS Vision 2020 ADMIN: Assessing Quality · WP7. Methodological support WP4. Eurostat as user of EC data WP1. Access to & development ... Governance technical ADMIN network Task Force](https://reader035.fdocuments.net/reader035/viewer/2022063009/5fc021b9ff4f0261fb593e3a/html5/thumbnails/24.jpg)
Thank you!
CROS portal: http://www.cros-portal.eu/content/essvip-admin
![Page 25: ESS Vision 2020 ADMIN: Assessing Quality · WP7. Methodological support WP4. Eurostat as user of EC data WP1. Access to & development ... Governance technical ADMIN network Task Force](https://reader035.fdocuments.net/reader035/viewer/2022063009/5fc021b9ff4f0261fb593e3a/html5/thumbnails/25.jpg)
Annex:
6 basis data source configurations
![Page 26: ESS Vision 2020 ADMIN: Assessing Quality · WP7. Methodological support WP4. Eurostat as user of EC data WP1. Access to & development ... Governance technical ADMIN network Task Force](https://reader035.fdocuments.net/reader035/viewer/2022063009/5fc021b9ff4f0261fb593e3a/html5/thumbnails/26.jpg)
1) Non-overlapping microdata without coverage problems:
KOMUSO Stage 1: Preparatory Work
most simple
still typical errors:
classification
measurement
progressiveness
incl. single source
reference config.
![Page 27: ESS Vision 2020 ADMIN: Assessing Quality · WP7. Methodological support WP4. Eurostat as user of EC data WP1. Access to & development ... Governance technical ADMIN network Task Force](https://reader035.fdocuments.net/reader035/viewer/2022063009/5fc021b9ff4f0261fb593e3a/html5/thumbnails/27.jpg)
KOMUSO Stage 1: Preparatory Work
2) Overlapping microdata without coverage problems:
configuration (1) +
linkage errors
inconsistencies
most common configuration
![Page 28: ESS Vision 2020 ADMIN: Assessing Quality · WP7. Methodological support WP4. Eurostat as user of EC data WP1. Access to & development ... Governance technical ADMIN network Task Force](https://reader035.fdocuments.net/reader035/viewer/2022063009/5fc021b9ff4f0261fb593e3a/html5/thumbnails/28.jpg)
KOMUSO Stage 1: Preparatory Work
2S) Overlapping microdata without coverage problems:
incl. samples
configuration (2) +
sampling errors
weights handling
calibration
![Page 29: ESS Vision 2020 ADMIN: Assessing Quality · WP7. Methodological support WP4. Eurostat as user of EC data WP1. Access to & development ... Governance technical ADMIN network Task Force](https://reader035.fdocuments.net/reader035/viewer/2022063009/5fc021b9ff4f0261fb593e3a/html5/thumbnails/29.jpg)
KOMUSO Stage 1: Preparatory Work
3) Overlapping microdata with under-coverage:
configuration (2) +
under-coverage
may include samples ( 3S)
![Page 30: ESS Vision 2020 ADMIN: Assessing Quality · WP7. Methodological support WP4. Eurostat as user of EC data WP1. Access to & development ... Governance technical ADMIN network Task Force](https://reader035.fdocuments.net/reader035/viewer/2022063009/5fc021b9ff4f0261fb593e3a/html5/thumbnails/30.jpg)
KOMUSO Stage 1: Preparatory Work
4) Combining microdata + macrodata sources:
configuration (2) +
reconciliation of microdata (calibr.)
![Page 31: ESS Vision 2020 ADMIN: Assessing Quality · WP7. Methodological support WP4. Eurostat as user of EC data WP1. Access to & development ... Governance technical ADMIN network Task Force](https://reader035.fdocuments.net/reader035/viewer/2022063009/5fc021b9ff4f0261fb593e3a/html5/thumbnails/31.jpg)
KOMUSO Stage 1: Preparatory Work
5) Combining only macrodata sources:
macro-counterpart of configuration (2)
reconcile overlapping data
E.g. supply/use (SU) tables of National Accounts
![Page 32: ESS Vision 2020 ADMIN: Assessing Quality · WP7. Methodological support WP4. Eurostat as user of EC data WP1. Access to & development ... Governance technical ADMIN network Task Force](https://reader035.fdocuments.net/reader035/viewer/2022063009/5fc021b9ff4f0261fb593e3a/html5/thumbnails/32.jpg)
KOMUSO Stage 1: Preparatory Work
6) Longitudinal data:
reconcile time series
different frequencies and qualities
now-casting (time series model)
linkage errors