European Social Survey

Documenting data processing and version control: The European Social Survey

• Introduction to the ESS

• Data processing and documentation

• Version control and documentation

Hilde Orten – ESS Data Team at NSDPresentation held at the CESSDA expert seminar – Neuchâtel 09/09 - 2004

European Social Survey

ESS

• The European Social Survey (ESS) is an academically driven multi-country survey

• Covering over 20 nations

• Fielded every second year

• Funded by the European Union, The European Science Foundation and academic funding bodies in each participating country

• Free and immediate acces to data and documents for all from the ESS Data Web Site: http://ess.nsd.uib.no

European Social Survey

Aims

Substantial:

• to chart and explain the interaction between Europe's changing institutions and the attitudes, beliefs and behaviour patterns of its diverse populations

Methodological:

• to develop and apply high methodological standards for cross-national survey research

European Social Survey

Efforts to enhance quality of data:

Face-to-face interviewingSpecialist advicory teamsData protocol,Standards

Documents

Interactive data processingArchive/National Teams

Availability of data and meta data at the www

European Social Survey

Data processing and documentation:

A transparent and interactive process:

• Data check, editing and validation is accomplished in interaction between the Archive (NSD) and the National Team of each participating country

• Standardized contact procedure between NSD and the National Teams

• Standardized data processing (work) procedure and programmes

• The data processing is documented in a two-step Data Processing Report

European Social Survey

WORK PROCESS ESS DATA TEAM NATIONAL TEAM

No deposit

No deposit, reminder to NT

Deposited

Deposit confirmed to NT

Content check

Content check, pending NT feedback

Content check complete

Content check summary produced

Data editing

Data editing, pending NT feedback

Data editing complete

Edit summary produced

Awaiting approval of draft from NT

Draft approved

File ready for integrated file

Included in integrated file

Response confirmed

Response to report

Data editing p. 6-8 Missing/error

Data editing p. 9

Validation of draft by NC

Data processing report step 2

Country draft file

Conf. agreement Response to Conf.a.

Missing/error

Draft approveCountry draft file ready for integrated file

Content check p. 3

Data deposit

Missing/error

Deposit confirmed

Content check p. 4-5

Data processing report step 1

Reminder

The figure illustrates the ESS data processing work flow at NSD, as well as the communication between the National Teams and NSD from ”no file deposit” to inclusion of country file in the international integrated file

Every participating country is assigned a primary contact person from the ESS Data Team at NSD

The contact person takes care of the processing of the country’s data, and communicates with the National Team

The data processing is accomplished in two steps:

1. A data check process and

2. A data editing process

-Each step resulting in a Data processing summary

The data processing summaries from the two steps make up the Data Processing Report

European Social Survey

Documenting the data processing:

During the data processing period:

• The Data Processing Report, all programmes used etc. are made available for the National Teams during the data processing period through the European Social Survey Archive Web Site: http://essdata.nsd.uib.no

After the data processing period:

• The Data Processing Reports, all programmes used, e-mails etc. are stored at NSD

European Social Survey

What is published about NSD’s data processing work at the ESS Data Web Site (http://ess.nsd.uib.no) and in Nesstar?

• Information of the extent of processing checks and missing values

• The handling of detected deviations

• Version notes We are handling deviations in three ways:

• Flagging

• Removing deviating variables from the integrated file, keeping the information in a country specific file

• Creating a new variable with a country-specific name in the integrated file

Three ways of accessing information of how deviations have been handled at NSD:

• The Data Processing Report

• The Fieldwork Summary and deviations link

• Online presentation of data and documentation together in Nesstar

European Social Survey

In the Data Processing Report under ”Survey documentation” you will find information about

detected deviations

European Social Survey

European Social Survey

The link ”Fieldwork Summary and deviations” leads you to information about detected

deviations, as well as to a link to further details

European Social Survey

European Social Survey

European Social Survey

Under ”On-line browsing and download” you will get access to information about detected

deviations through Nesstar

European Social Survey

European Social Survey

Version control:

Information to the users about new versions:

• All registred users are e-mailed about updates on the ESS Data Web Site

• The version notes describe all updates of the ESS Round 1 data from the first edition to the most recent one

• All versions of the ESS Round 1 data are produced at NSD

• All programmes used etc. are stored at NSD

European Social Survey

ESS1 edition 05

ESS1 edition 01

Online documentation of versions: There are three ways of accessing this documentation of versions from the ESS Data Web Site:

• The Data Processing Report under ”Survey documentation”

• The 25Version notes link

• Online documentation of versions in Nesstar under ”On-line browsing and download”