Survey Data Collection and Integration

Cristina Davino • Luigi FabbrisEditors

Survey Data Collectionand Integration

123

EditorsCristina DavinoDepartment of Studies for Economic

DevelopmentUniversity of MacerataMacerataItaly

Luigi FabbrisDepartment of Statistical ScienceUniversity of PaduaPaduaItaly

ISBN 978-3-642-21307-6 ISBN 978-3-642-21308-3 (eBook)DOI 10.1007/978-3-642-21308-3Springer Heidelberg New York Dordrecht London

Library of Congress Control Number: 2012943361

� Springer-Verlag Berlin Heidelberg 2013This work is subject to copyright. All rights are reserved by the Publisher, whether the whole or part ofthe material is concerned, specifically the rights of translation, reprinting, reuse of illustrations,recitation, broadcasting, reproduction on microfilms or in any other physical way, and transmission orinformation storage and retrieval, electronic adaptation, computer software, or by similar or dissimilarmethodology now known or hereafter developed. Exempted from this legal reservation are briefexcerpts in connection with reviews or scholarly analysis or material supplied specifically for thepurpose of being entered and executed on a computer system, for exclusive use by the purchaser of thework. Duplication of this publication or parts thereof is permitted only under the provisions ofthe Copyright Law of the Publisher’s location, in its current version, and permission for use must alwaysbe obtained from Springer. Permissions for use may be obtained through RightsLink at the CopyrightClearance Center. Violations are liable to prosecution under the respective Copyright Law.The use of general descriptive names, registered names, trademarks, service marks, etc. in thispublication does not imply, even in the absence of a specific statement, that such names are exemptfrom the relevant protective laws and regulations and therefore free for general use.While the advice and information in this book are believed to be true and accurate at the date ofpublication, neither the authors nor the editors nor the publisher can accept any legal responsibility forany errors or omissions that may be made. The publisher makes no warranty, express or implied, withrespect to the material contained herein.

Printed on acid-free paper

Springer is part of Springer Science+Business Media (www.springer.com)

Preface

Surveys are an important source of scientific knowledge and a valid decision-support tool in many fields, from social studies to economics, market research,health studies, and others. Scientists have investigated most of the methodologicalissues on statistical surveys so that the scientific literature offers excellent refer-ences concerning all the required steps for planning and realising surveys. Nev-ertheless, real problems of statistical surveys often require practical solutions thateither deviate from the consolidated methodology or do not have a specificsolution at all.

Information and communication technology and techniques are earmarking themodern world, changing people’s behavior and mentality. Innovations in remoteand wireless communication devices are being created at full steam. Purchasebehaviors, service fruition, production activities, social and natural events arerecorded in real time and stored in huge data warehouses. Warehouses are beingcreated not only for data but also for textual documents, audio files, images, andvideo files. Survey methods must evolve accordingly: surveys have to takeadvantage of all new information chances and adapt, if needed, the basic meth-odology to the ways people use to communicate, memorize, and screen theinformation. This book is aimed at focusing on new topics in today’s surveyresearchers’ agenda.

Let us provide an example of new survey needs. Complex survey designs,which were imagined as statistical means for collecting data from samples ofdwellings drawn from population registers, are going to be obsolete but for someofficial statistics and for rare, relevant, opinion surveys. In fact, the access topopulation registers has proven strict and face-to-face interviewing has becomevery expensive. Instead, there are growing possibilities for both private and publicorganisation for achieving administrative databases related to large populationsegments. Hence, sample surveys are on the verge of specialisation in collectingsubjective information through remote technological devices from samplesmechanically drawn from available databases. Our vision is purposively drasticbecause we want to stress the irreversibility of the trend.

v

Any survey researcher has to ponder practical and methodological problemswhen choosing the appropriate technique for acquiring the data to analyse. Noresearcher is allowed to ignore costs, time, and organisation constraints of datacollection. See also the introductory paper by Luigi Biggeri on this issue. Thisinduces a researcher to consider appropriate for her/his purposes:

• Integrating large datasets from various sources rather than gathering ad hoc databy means of traditional surveys;

• Collecting low-cost, real-time massive data rather than gathering data throughrefined statistical techniques at the expenses of timeliness, budget saving, andrespondent bothering;

• Estimating some parameters without sampling error, because all units can beprocessed, and other parameters based on models that hypothesize at the verylocal level relationships that are observed at a higher scale, as in the case ofsmall area estimation;

• Connecting and harmonising in a holistic approach, data collected with oranalysed at different scales, for instance analyse together data on individualgraduates, graduates’ families, professors, study programmes, educationalinstitutions, production companies, and local authorities, all aimed at describingthe complex (i.e. multidimensional and hierarchical) relationships betweengraduates’ education and work.

Of course, we go nowhere if our research aims are unclear. Supposing aims areclear, then the appropriateness of research choices is a methodological concern. Inthis volume, a paper by Tomàs Aluja-Banet, Josep Daunis-i-Estadella, and YanHong Chen, as well as another by Diego Bellisai, Stefania Fivizzani, and MarinaSorrentino concern the issue of data integration. The first paper deals with theissues to be considered when imputing, in place of a missing value, values drawnfrom related sources. It deals also with the choice of an imputation model and withthe possibility to combine parametric and non-parametric imputation models. Thesecond paper applies to the design of an official survey aimed at integrating datacollected on job vacancy and hours worked with other Istat business surveys.A third paper by Monica Pratesi, Caterina Giusti, and Stefano Marchetti discussesthe use of small area estimation methods to measure the incidence of poverty at thesub-regional level using EU-SILC sample data.

The very availability of large databases requires researchers to focus on dataquality. We mean that data quality assessment is an important issue in any survey,but it tends to the fore just because the sampling error vanishes. This, in turn,requires researchers to learn:

• How to check the likelihood of the data drawn from massive databases whoserecords could contain errors?

• How to measure the quality of, and possibly adjust, the data collected withopinion surveys that, in general, are carried out by means of high-performancetechnological tools and by specialized personnel who interview samples ofrespondents?

vi Preface

• How to guarantee sufficient methodological standards in data collection fromkey witnesses whom applied researchers turn to more and more often so tocorroborate the results of analyses on inaccessible phenomena, to elicit people’spreferences or hidden behaviors and to forecast social or economical events inthe medium or the long run?

• How to prune the redundant information that concurrent databases and repetitiverecords contain? Also, how to screen the statistically valid from the coarseinformation in excessively loaded databases created for purposes that are aliento statistics?

This is the reason why, in this volume, methodologies for measuring statisticalerrors and for designing complex questionnaires are picked out. Statistical errorsrefer to both sampling and non-sampling errors. The paper by Giovanni D’ Alessioand Giuseppe Ilardi examines methods for measuring the effects of non-responseerrors (‘‘unit non-response’’) and of some response errors (‘‘uncorrelatedmeasurement errors’’ and ‘‘biases from underreporting’’) by taking advantage ofthe experience from the Bank of Italy’s Survey on Household Income and Wealth.The Authors suggest, too, how to overcome the error effects on estimates throughvarious techniques and models and by means of auxiliary information.

Questionnaire design and question wording instructions, as strategies for pre-venting data collection errors, are dealt with extensively in the volume. CristinaDavino and Rosaria Romano discuss the case of multi-item scales that areappropriate for the measurement of subjective data, focussing on how individualpropensities in the use of scales can be interpreted, in particular within strata ofrespondents. Luigi Fabbris covers the presentation of batteries of interrelated itemsfor scoring or ranking sets of choice or preference items. Five types of itempresentations in questionnaires are crossed with potential estimation models andcomputer-assisted data collection modes. Caterina Arcidiacono and Immacolata DiNapoli present and apply the so-called Cantril scale that self-anchors the extremesof a scale and is useful in psychological research. Simona Balbi and NicoleTriunfo deal with the age-old problem of closed- and open-ended questions, in theperspective of statistical analysis of data. The Authors tackle, in particular, theproblem of transforming textual data, i.e. data that are collected in naturallanguage, into data that can be processed with multivariate statistical methods.

This volume is a consequence of the stimulating debate that animated theworkshop ‘‘Thinking about methodology and applications of surveys’’ that tookplace at the University of Macerata (Italy) in September 2010. The titles of thepapers presented in the volume have been proposed to the Authors having in mindthe relevant issues of a homogeneous field of study that is usually covered byundirected articles. In each paper, a survey of the scientific literature is discussedand remarks and innovative solutions to face today’s survey problems aresuggested. All papers aim at balancing formal rigour with simplicity of thepresentation style so as to address the book both to practitioners involved inapplied survey research and to academics interested in scientific development ofsurveys.

Preface vii

The editors of the volume like to highlight that all papers have been referred byat least two external experts in the topical field. The editors wish to thank theAuthors and also the Referees for their invaluable contribution to the quality of thepapers. The external referees involved were: Giorgio Alleva (University of Rome‘‘La Sapienza’’, Italy), Gianni Betti (University of Siena, Italy), Silvia Biffignandi(University of Bergamo, Italy), Sergio Bolasco (University of Rome ‘‘La Sapi-enza’’, Italy), Marisa Civardi (Bicocca University in Milan, Italy), Daniela Cocchi(University of Bologna, Italy), Giuseppe Giordano (University of Salerno, Italy),Michael Greenacre (Universitat Pompeu Fabra, Barcelona, Spain), FilomenaMaggino (University of Florence, Italy), Alberto Marradi (University of Florence,Italy), Giovanna Nicolini (University of Milan, Italy), Maria Francesca Romano(Scuola Superiore Sant’Anna, Italy).

The editors are open to any contribution of readers who would wish to commenton papers or to propose the Authors other ideas.

Cristina DavinoLuigi Fabbris

viii Preface

Contents

Part I Introduction to Statistical Surveys

Surveys: Critical Points, Challenges and Need for Development . . . . . 3Luigi Biggeri1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32 Frameworks to Specify and Organize the Presentation of the

Critical Issues of Statistical Surveys . . . . . . . . . . . . . . . . . . . . . . . . 42.1 The Total Quality Management Approach . . . . . . . . . . . . . . . . 42.2 The Production Process of a Survey: Different

Perspectives of the Life Cycle of a Survey . . . . . . . . . . . . . . . . 63 Critical Issues, Challenges and Need for Development

of Statistical Surveys . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 93.1 Why are There Still Critical Issues?. . . . . . . . . . . . . . . . . . . . . 93.2 Mode of Collection of Data and Construct

of a Questionnaire . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 103.3 Sampling Design and Estimation that Reply

to the Demands of the Users . . . . . . . . . . . . . . . . . . . . . . . . . . 123.4 Data Dissemination and Standardisation . . . . . . . . . . . . . . . . . . 14

4 Concluding Remarks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16

Part II Questionnaire Design

Measurement Scales for Scoring or Ranking Setsof Interrelated Items . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21Luigi Fabbris1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 212 Survey Procedures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22

2.1 Ranking . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 222.2 Picking the Best/Worst Item . . . . . . . . . . . . . . . . . . . . . . . . . . 23

ix

2.3 Fixed Total Partitioning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 232.4 Rating Individual Items . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 242.5 Paired Comparisons . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 252.6 Conjoint Attribute Measurement . . . . . . . . . . . . . . . . . . . . . . . 25

3 Statistical Properties of the Scales . . . . . . . . . . . . . . . . . . . . . . . . . . 263.1 Ranking . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 293.2 Pick the Best/Worst . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 313.3 Fixed Budget Partitioning . . . . . . . . . . . . . . . . . . . . . . . . . . . . 313.4 Rating . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 323.5 Paired Comparisons . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33

4 Practical Considerations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 344.1 Ranking . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 354.2 Picking the Best/Worst . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 364.3 Fixed Total Partitioning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 364.4 Rating . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 374.5 Paired Comparisons . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38

5 Concluding Remarks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41

Assessing Multi-Item Scales for Subjective Measurement . . . . . . . . . . 45Cristina Davino and Rosaria Romano1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 452 Subjective Measurement in the Social Sciences. . . . . . . . . . . . . . . . . 463 An Empirical Analysis: Assessing Students’

University Quality of Life . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 473.1 The Survey . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 473.2 Univariate and Bivariate Analysis . . . . . . . . . . . . . . . . . . . . . . 48

4 An Innovative Multivariate Approach . . . . . . . . . . . . . . . . . . . . . . . 514.1 The Methodology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 534.2 Main Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55

5 Concluding Remarks and Further Developments . . . . . . . . . . . . . . . . 58References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58

Statistical Tools in the Joint Analysis of Closedand Open-Ended Questions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61Simona Balbi and Nicole Triunfo1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 612 Textual Data Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 623 Pre-Processing and Choosing a Proper Coding

and Weighting System in TDA . . . . . . . . . . . . . . . . . . . . . . . . . . . . 624 Correspondence Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63

4.1 Lexical Correspondence Analysis . . . . . . . . . . . . . . . . . . . . . . 654.2 Lexical Non Symmetrical Correspondence Analysis. . . . . . . . . . 67

5 Conjoint Analysis with Textuals External Information . . . . . . . . . . . . 68

x Contents

5.1 The Watch Market . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 706 Concluding Remarks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 72

The Use of Self-Anchoring Scales in Social Research:The Cantril Scale for the Evaluation of CommunityAction Orientation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73Immacolata Di Napoli and Caterina Arcidiacono1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 732 Measuring Attitudes and Opinions . . . . . . . . . . . . . . . . . . . . . . . . . . 74

2.1 The Likert Scale . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 752.2 Reflexivity and the Self-Anchoring Scale . . . . . . . . . . . . . . . . . 75

3 The Construction of a Self–Anchoring Scalefor Community Action Orientation . . . . . . . . . . . . . . . . . . . . . . . . . 793.1 Preliminary Steps. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 793.2 Identifying Indicators of Community

Action Orientation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 793.3 The Community Action Orientation Scale. . . . . . . . . . . . . . . . . 80

4 Final Considerations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 82References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 83

Part III Sampling Design and Error Estimation

Small Area Estimation of Poverty Indicators . . . . . . . . . . . . . . . . . . . 89Monica Pratesi, Caterina Giusti and Stefano Marchetti1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 892 Poverty Indicators . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 903 Small Area Methods for the Estimation of Poverty Indicators. . . . . . . 924 Estimation of Poverty Indicators at Provincial Level in Tuscany . . . . . 955 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 100References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 100

Non-Sampling Errors in Household Surveys:The Bank of Italy’s Experience . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 103Giovanni D’Alessio and Giuseppe Ilardi1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1032 The Survey on Household Income and Wealth . . . . . . . . . . . . . . . . . 1043 Unit Non-Response and Measurement Errors in the SHIW:

Some Empirical Studies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1053.1 The Analysis of Unit Non-Response . . . . . . . . . . . . . . . . . . . . 1053.2 Measurement Errors: Uncorrelated Errors . . . . . . . . . . . . . . . . . 1083.3 Measurement Errors: Underreporting . . . . . . . . . . . . . . . . . . . . 113

4 Concluding Remarks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 116References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 117

Contents xi

Part IV Data Integration

Enriching a Large-Scale Survey from a RepresentativeSample by Data Fusion: Models and Validation . . . . . . . . . . . . . . . . . 121Tomàs Aluja-Banet, Josep Daunis-i-Estadella and Yan Hong Chen1 Introduction to Data Fusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1222 Conditions for Data Fusion. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1233 Steps for a Data Fusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 124

3.1 Selection of the Hinge Variables . . . . . . . . . . . . . . . . . . . . . . . 1243.2 Grafting the Recipient in the Space of Donors . . . . . . . . . . . . . 1263.3 Choosing the Imputation Model. . . . . . . . . . . . . . . . . . . . . . . . 1263.4 Validation Tools . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 128

4 Significance of the Validation Statistics . . . . . . . . . . . . . . . . . . . . . . 1325 Application to Official Statistics . . . . . . . . . . . . . . . . . . . . . . . . . . . 1326 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 136References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 136

A Business Survey on Job Vacancies: Integrationwith Other Sources and Calibration. . . . . . . . . . . . . . . . . . . . . . . . . . 139Diego Bellisai, Stefania Fivizzani and Marina Sorrentino1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1392 Main Characteristics of the Job Vacancy Survey . . . . . . . . . . . . . . . . 1403 Editing and Imputation. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 143

3.1 Job Vacancies: Peculiarities, Implications and Editingand Imputation Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 144

4 Calibration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1485 Concluding Remarks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 151References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 152

About the Authors. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 153

xii Contents