Activities to reduce response burden:
development of questionnaires and
electronic data submission
environment eSTAT based on
feedback from respondents and data
processing phase
Epp Karus
Statistics Estonia
Head of Department
Data Processing and Registers Department
Organisational structure
The organisational structure of SE is based on the Generic Statistical
Business Process Model (GSBPM) 4.0
Work organisation is fully centralised
In autumn 1993, marketing and dissemination, and in 2004, data
collection were centralised
A major re-engineering of the statistical production process took place
in October 2013, when data processing and data warehouse activities
were centralised
Minor changes were made in March 2015 and in October 2017.
Structure http://www.stat.ee/contacts
Statistics Estonia
Implementation of GSBPM in SE
The main reason for such a centralised work organisation is to
increase resource efficiency
Consolidation of similar functions has contributed to standardisation
Standardised tools are implemented in different parts of the statistical
information system
The use of GSBPM has raised the awareness about statistical
production and processes in SE
GSBPM is used for analysing the efficiency and similarities /
differences between statistical activities and for making other analysis
about the statistical programme
Better horizontal integration across statistical domains has been
achieved
Statistics Estonia
5.2 Classify
& code
5.3 Review,
validate & edit
5.5 Derive new
variables &
statistical units
5.4 Impute
5.6 Calculate
weights
5.7 Calculate
aggregates
5.8 Finalize
data files
5.1 Integrate data
1. Specify
Needs2. Design 3. Build 4. Collact 5. Process 6. Analyse
7. Disse-
minate8. Archive
3.1 Build data
collection
instrument
3.2 Build or
enhance process
components
3.4 Test
production system
3.3 Configure
workflows
3.5 Test statistical
business process
3.6 Finalize
production system
6.1 Prepare draft
outputs
7.1 Update
output systems
6.2 Validate
outputs
7.2 Produce
dissemination
products
6.4 Apply
disclosure control
7.4 Promote
dissemination
products
6.3 Scrutinize
& explain
7.3 Manage release
of dissemination
products
8.1 Define
archive rules
8.2. Manage
archive repository
8.4. Dispose of
data and associated
metadata
8.3. Preserve data
and associated
metadata
9.1. Gather
evaluation inputs
9.2. Conduct
evaluation
9.3. Agree
action plan
6.5 Finalize
outputs
7.5 Manage user
support
1.2 Consult &
confirm needs
1.4 Identify
concepts
1.3 Establish
output objectives
1.5 Check data
availability
2.1 Design outputs
2.2 Design variable
descriptions
2.4 Design frame
& sample
methodology
2.3 Design data
collection
methodology
2.5 Design
statiatical processing
methodology
2.6 Design
production
systems & workflow
4.1 Select sample
4.2 Set up
collection
4.4 Finalize
collection
4.3 Run
collection
1.6 Prepare
business case
9. Evaluate
Quality Management / Metadata Managemen
1.1 Determine
needs for
information
Statistics Estonia
Marketing and Dissemination Department
Statistics Departments
Data Warehouse Department
Data Processing and Registers Department
Data Processing and Registers Department
General Department
Metadata Department
Lean tools and Agile methods in SE
In 2016, SE introduced Lean tools and Agile methods to define
the value stream and make value flow better. Our objectives are:
To achieve better coherence between strategic, tactical
and operational planning
To reduce work-in-progress through a systematically
arranged management system and sharing of
competences
To improve work flow in the value stream and information
flow between employees
Statistics Estonia
Lean internal audit results, May 2017
Statistics Estonia
0
1
2
3
4
5
6
1. Visualize
2. Limit Work-in-Progress
3. Manage Flow 4. Feedback Loops
5. Continuous Improvement
Respondents’ feedback collected with
the Promoter Index (PI) questionnaire
The PI questionnaire is used to get feedback from
respondents (https://en.wikipedia.org/wiki/Net_Promoter)
PI has been measured in SE since 2014, using the web
application recommy.com (https://app.recommy.com)
Collected information is used to: receive input for the strategic goal “Reduce respondents’ perceived
burden”
get feedback from respondents about eSTAT, questionnaires, statistical
work and the reputation of SE
meet the respondent needs and develop eSTAT, questionnaires and
communications Statistics Estonia
Questions of the PI questionnaire
and coding of respondents’ comments Questions
Q1: How likely is it that you would recommend to submit the
questionnaire XXX using the electronic data submission environment
eSTAT to your friend or colleague?
Q3: How easy was the completing of this questionnaire in your opinion?
The respondent has the opportunity to comment on his/her responses
Comments are coded into three groups: eSTAT, questionnaire and
reputation
Each category is divided into several development needs and positive
feedback
Statistics Estonia
Comments to PI questionnaire
by topic, 2016
Statistics Estonia
0
100
200
300
400
500
600
eSTAT - Development need
eSTAT - Satisfaction Questionnaire - Development need
Questionnaire - Satisfaction
Reputation - Development need
Reputation - Satisfaction
Contact Centre of Respondents (CCR)
CCR was created in SE in January 2006
From the CCR, one can quickly obtain information about the
respondents’ problems and react to them
The purpose of the CCR is to assist respondents with submitting data
and to provide solutions to problems encountered by respondents of
both social and economic surveys
CCR provides consultation in Estonian, but also in English and
Russian; consultation is provided by telephone and e-mail
To help the CCR consultants to answer respondents’ questions,
guidelines for every questionnaire of SE and a list of answers to
frequently asked questions are compiled
Statistics Estonia
Topics of incoming calls
In 2017, we started to code the topics of incoming calls to
the CCR
This gives us information about the main problems of
respondents and the number of similar problems
The main problems of respondents are shared with the
survey manager, questionnaire team and main users of
data collection environments
If possible, the respondents' proposals are taken into
account in developing and maintenance of questionnaires
or eSTAT
Statistics Estonia
Topics of incoming calls, May−August 2017
Statistics Estonia
Consultation on eSTAT questionnaires 67%
Consultation on eSTAT usage
22%
Consulting social surveys'
respondents 1%
Consulting on finding information
on the website 10%
Architecture of the information system
Dissemination
Statistical
registers
Metadata
system
Data
collection Processing Statistical
analysis Persons
Administrative
registers
Users
Economic
entities
Data
Warehouse
iMETA
VVIS eGeostat
SRS
VAIS eSTAT PX-Web /.Stat
Census-HUB
KUNDE
Analyse
Statistics Estonia
Data processing system VAIS SE is implementing the template based data processing system
VAIS
VAIS was first introduced in 2012 to process the 2011
Population and Housing Census data
Today, the data of about 3/4 questionnaires are processed in
VAIS
This has created opportunities for standardising the data
processing of statistical activities and calculating standardised
quality indicators in the final microdata database
Quality indicators are calculated automatically for all statistical
activities that are processed in VAIS Statistics Estonia
Analysing of microdata edits
For questionnaire developing, we analyse microdata edits
We focus on the variables with the biggest number of edits
The need for editing indicates that the respondent has had
difficulties with collecting and submitting the data or does not
understand what data is required to be submitted
Problematic questionnaires are evaluated in collaboration between
the survey manager, questionnaire team and dataset team
At the questionnaire evaluation meeting, the participants analyse
how to make questionnaires more understandable to respondents
and more compatible with the accounting data of enterprises
Statistics Estonia
Designing and building questionnaires
Before designing and building questionnaires for the new survey
period, all available feedback information is studied by the
questionnaire team
Available sources are:
Complaints and proposals of respondents from the PI
questionnaire
Complaints and proposals of respondents to the CCR
Feedback from the questionnaire evaluation meeting
Based on respondents’ feedback to the PI questionnaire, 21
statistical questionnaires were developed for 2017
Statistics Estonia
Main developments in questionnaires wording of error messages
variable explanations
added pre-fillings, added auto-sums
decrease of variables
changed table structure
translation into Russian
changed coding list
In 2017, we shall pilot the calculation of feedback information (ratios) into
questionnaires based on the data reported with questionnaires to motivate
enterprises to submit data
In 2018, all questionnaires shall be translated into English to help non-
Estonians and e-residents to report data
Statistics Estonia
eSTAT developments
The main user of eSTAT regularly analyses the respondents’
feedback on eSTAT
Smaller improvements can be implemented quickly. As regards
bigger improvements, we are dependent on the financing of IT
investments
Currently, an eSTAT design project and two eSTAT
developments with financing from Eurostat grants are ongoing
Planned archiving of eSTAT’s older data is planned to be carried
out in October 2017 – this will speed up eSTAT and help
respondents to find the right questionnaire more easily
Statistics Estonia
Regular notifications to respondents
At the end of each year, SE informs enterprises of their obligations in
the forthcoming year
Upon first contact, we describe more precisely the principles of
statistical production and the importance of data provision, and also
provide instructions on creating the main user on eSTAT
During the year, we specify the activities of new enterprises and add
new units to data collection based on sample selection criteria
We inform new enterprises of samples of their data submission
obligation on the 12th day each month
Information on data submission is available at http://www.stat.ee/en
Statistics Estonia
Content of eSTAT notifications
In 2017, the content of the automated notifications was
changed
The most important message is included at the beginning of
the letter; the letter is easy to read and is of moderate length
At first, we are informative and later assertive. The text of the
letter is more strict after the deadline has expired
We only inform respondents about enforcement and penalties
7 days after the expiration of the deadline, as we do not want
to alarm conscientious respondents
Statistics Estonia
Timing of eSTAT automated notifications
In 2017, we changed the number of eSTAT automated
notifications: one notification sent out before the deadline was
cancelled and the last automated letter is sent out earlier than
before
New eSTAT automated notifications are sent out by e-mail
-5, +3, +7, +20 days from the data providing deadline
From October 2017, the eSTAT automated notifications are
not sent out at weekends or public holidays. Enterprises were
very dissatisfied with receiving letters from SE at weekends.
Statistics Estonia
Promoting usage of eSTAT
Although the time spent on data collection and contacting
respondents has decreased year by year, it accounts for a
considerable share of the time spent on statistical activities
In 2017, we shall pilot promoting higher eSTAT usage to
data providers for reporting data
When contacting enterprises, we first recommend the use
of eSTAT; secondly we assist with submitting data via
eSTAT and thirdly, we receive data by phone
Finally, we ask about the main barriers to not using eSTAT
Statistics Estonia
Trainings for respondents
SE organizes regular trainings for respondents
We provide trainings on the eSTAT environment and various
most complicated questionnaires
At the beginning of the year, special training to new data
providers is organised
The main user of eSTAT is responsible for eSTAT topics and
the survey manager for questionnaire topics
We inform potential enterprises interested in training about
future trainings via e-mail. Registration for trainings takes place
on the website of SE
Statistics Estonia
THANK YOU
Statistics Estonia
Statistics Estonia 21.09.2016
Top Related