INFO 7470/ECON 7400 Synthetic Data Creation and Use John M. Abowd and Lars Vilhuber with a big...

INFO 7470/ECON 7400 Synthetic Data Creation and Use

John M. Abowd and Lars Vilhuberwith a big assist from Abigail Cooke, Javier

Miranda, Martha Stinson, and Kelly TrageserApril 29, 2013

© John M. Abowd and Lars Vilhuber 2013, all rights reserved

2

Outline

• SIPP Synthetic Data• LBD Synthetic Data

4/29/2013


3

SURVEY OF INCOME AND PROGRAM PARTICIPATION (SIPP) SYNTHETIC DATA

4/29/2013


4

Survey of Income and Program Participation (SIPP)

• Goal of SIPP: accurate info about income and program participation of individuals and households and its principal determinants

• Information:– Cash and noncash income on a sub-annual basis. – Taxes, assets, liabilities– Participation in government transfer programshttp://www.census.gov/sipp/intro.html

4/29/2013

http://www.census.gov/sipp/intro.html


5

Background

• In 2001, a new regulation authorized the Census Bureau and SSA to link SIPP and CPS data to SSA and IRS administrative data for research purposes

• Idea for a public use file was motivated by a desire to allow outside access to long administrative record histories of earnings and benefits linked to household demographic data

• These data allow detailed statistical and simulation study of retirement and disability programs

• Census Bureau, Social Security Administration, Internal Revenue Service, and Congressional Budget Office all participated in development

4/29/2013


6

Genesis of the SSB

• A portion of the SIPP user community was primarily interested in national retirement and disability programs

• SIPP augmented with – earnings histories from the IRS data maintained at SSA (W-2)– benefit data from SSA’s master beneficiary records.

• Feasibility assessment (confidentiality!) of adding SIPP variables to earnings/benefit data in a public-use file (PUF)– set of variables that could be added without compromising the

confidentiality protection of the existing SIPP public use files was VERY limited

• Alternative methods explored

4/29/2013


7

SSB Basic Methodology

• Experiment using “synthetic data”• In fact: partially synthetic data with multiple

imputation of missing items• Partially synthetic data:

– Some (at least one) variables are actual responses– Other variables are replaced by values sampled

from the posterior predictive distribution for that record, conditional on all of the confidential data

4/29/2013


8

History of the SSB• 2003-2005: Creation, but not release, of three versions of the “SIPP/SSA/IRS-

PUF” (SSB)• 2006: Release to limited public access of

SSB V4.2– Access to general public only at Cornell-hosted Virtual RDC (SSB server: restricted-

access setup)• With promise of evaluation of Virtual RDC-run programs on internal Gold Standard

– Ongoing SSA evaluation– Ongoing evaluation at Census (in RDC)

• 2010: Release of SSB V5 at Census and on the Virtual RDC (codebook: http://www.census.gov/sipp/SSB_Codebook.pdf )– Restructured to vastly improve analytical validity of SIPP variables

• 2013: Release of SSB V5.1 at Census and on the VirtualRDC (documentation in preparation)– First user-initiated variables

4/29/2013

http://www.census.gov/sipp/synth_data.html

http://www.census.gov/sipp/SSB_Codebook.pdf


9

Basic Structure of the SSB V4

• SIPP– Core set of 125 SIPP variables in a standardized

extract of SIPP panels 1990-1993 and 1996– All missing data items (except for structurally

missing) are marked for imputation• IRS

– Maintained at SSA, but derived from IRS records– Master summary earnings records (SER)– Master detailed earnings records (DER)

4/29/2013


10

Basic Structure of the SSB V4 (II)

• SSA– Master Beneficiary Record (MBR)

• Census– Numident: administrative birth and death dates

• All files combined using verified SSNs=> “Gold Standard”

4/29/2013


11

Basic Structure of SSB V5

• Panels: 1990, 1991, 1992, 1993, 1996, 2001, and 2004 (this variable is now in the SSB)

• Couple-level linkage: the first person to whom the SIPP respondent was married during the time period covered by the SIPP panel

• SIPP variables only appear in years appropriate for the panel indicated by the PANEL variable (biggest change from V4.2)

• Version 5.1: user-requested variables4/29/2013


12

Missing Values in the Gold Standard

• Values may be missing due to– [survey] Non-response– [survey] Question not being asked in a particular

panel– [admin] Failure to link to administrative record (non-

validated SSN)– [both] Structural missing (e.g., income of spouse if

not married)• All missing values except structural are part of

the missing data imputation phase of SSB4/29/2013


13

Scope of the Synthesis

• Never missing and not synthesized– gender– marital status– spouse’s gender– initial type of Social Security benefits– type of Social Security benefits in 2000– spouse’s benefits type variables

• All other variables in the public use file were synthesized

4/29/2013


14

Common Structure to Multiple Imputation and Synthesis

• Hierarchical tree of variable relationships (parent-child relationship, accounting for structure)

• At each node, independent SRMI is used– Statistical model is estimated for each of the variables at the same

level (one of):• Bayesian bootstrap • Logistic regression (with automatic Bayesian variable selection)• Linear regression (with automatic Bayesian variable selection)

– Statistical models are estimated separately for groups of individuals– Then, a proper posterior predictive distribution is estimated– Given a PPD, each variable is imputed /synthesized, conditional on all

values of all other variables for that record• The next node is processed

4/29/2013


15

MI and Synthesis

• Initial iterations for missing data imputation, keeping all observed values where available

• Final iteration is for data synthesis (replacing all observed values, see exceptions)

4/29/2013


16

Latest Release of SSB

• 2010: Release of limited public access of SSB V5.0

• 2013: Release of limited public access SSB V5.1

• Both versions accessed via the VirtualRDC Synthetic Data Server

4/29/2013


17

SIPP Variables

• Codebook

4/29/2013

http://www.census.gov/sipp/SSB_Codebook.pdf


18

Synthetic Data Creation• Purpose of synthetic data is to create micro-data

that can be used by researchers in the same manner as the original data while preserving the confidentiality of respondents’ identities

• Fundamental trade-off: usefulness and analytical validity of data versus protection from disclosure

• Goal: not be able to re-identify anyone in the already released SIPP public use files while still preserving regression results

4/29/2013


19

Multiple Imputation forConfidentiality Protection

4/29/2013

• Denote confidential data by and non-confidential data by

• and has no missing data• PPD: • Complete data: from

• Synthetic data: from

• Major emphasis is to find a good estimate of the PPD


20

Testing Analytical Validity• Run regressions on each synthetic implicate

– Average coefficients– Combine standard errors using formulae that take

account of average variance of estimates (within implicate variance) and differences in variance across estimates (between implicate variance)

• Run regressions on gold standard data• Compare average synthetic coefficient and standard

error to gold standard coefficient and standard error• Data are analytically valid if coefficient is unbiased

and the same inferences are drawn

4/29/2013


21

Formulae: Completed Data Only

• Notation– Script is index for missing data implicate– is total number of missing data implicates

• Estimate from one completed implicate

• Average of statistic across implicates

4/29/2013


22

Formulae: Total Variance andBetween Variance

• Total variance of average statistic

• Variance of the statistic across implicates: between variance

4/29/2013


23

Formula: Within Variance

• Variance of the statistic from each completed implicate

• Average variance of statistic: within variance

4/29/2013


24

Formulae: Synthetic and Completed

4/29/2013

• Notation– script is index for missing data implicate– script is index for synthetic data implicate – is total number of missing data implicates– is total number of synthetic implicates per missing

data implicate• Estimate from one synthetic implicate

• Average of statistic across synthetic implicates


25

Formulae: Grand Mean and Overall Variance

• Average of statistic across all implicates

• Total variance of average statistic

4/29/2013


26

Formulae: Between Variances

• Variance of the statistic across missing data implicates: between implicate variance

• Variance of the statistic across synthetic data implicates: between r implicate variance

4/29/2013


27

Formulae: Within Variances

• Variance of the statistic on each implicate

• Average variance of statistic: within variance

• Source: Reiter, Survey Methodology (2004): 235-42.

4/29/2013


28

Example: Average AIME/AMW

• Estimate average on each of synthetic implicates– AvgAIME(1,1) , AvgAIME(1,2) , AvgAIME(1,3) , AvgAIME(1,4) , – AvgAIME(2,1) , AvgAIME(2,2) , AvgAIME(2,3) , AvgAIME(2,4) , – AvgAIME(3,1) , AvgAIME(3,2) , AvgAIME(3,3) , AvgAIME(3,4) , – AvgAIME(4,1) , AvgAIME(4,2) , AvgAIME(4,3) , AvgAIME(4,4)

• Estimate mean for each set of synthetic implicates that correspond to one completed implicate– AvgAIMEAVG(1) , AvgAIMEAVG(2) , AvgAIMEAVG(3) ,

AvgAIMEAVG(4)• Estimate grand mean of all implicates

– AvgAIMEGRANDAVG

4/29/2013


29

Example (cont.)

• Between m implicate variance

• Between r implicate variance

4/29/2013


30

Example (cont.)• Variance of mean from each

implicate– VAR[AvgAIME(1,1)] , VAR[AvgAIME(1,2)] , VAR[AvgAIME(1,3)] , VAR[AvgAIME(1,4)] – VAR[AvgAIME(2,1)] , VAR[AvgAIME(2,2)] , VAR[AvgAIME(2,3)] , VAR[AvgAIME(2,4)] – VAR[AvgAIME(3,1)] , VAR[AvgAIME(3,2)] , VAR[AvgAIME(3,3)] , VAR[AvgAIME(3,4)] – VAR[AvgAIME(4,1)] , VAR[AvgAIME(4,2)] , VAR[AvgAIME(4,3)] , VAR[AvgAIME(4,4)]

• Within variance

4/29/2013


31

Example (cont.)

• Total Variance

• Use AvgAIMEGRANDAVG and Total Variance to calculate confidence intervals and compare to estimate from completed data

4/29/2013


32

SAS Programs

• Sample programs to calculate total variance and confidence intervals

4/29/2013


33

Results: Average AIME

4/29/2013

AVG STAT

Total VAR

Betw. M Var

Betw. R Var

Betw. Var

Within Var

synthetic 1094.2 91.8 59.3 13.3 21.1 1074.5 1113.9completed 1142.5 52.8 23.4 23.7 1129.3 1155.7*All individuals with TOB_2000=1

confidence interval

Average of AIME (Average Indexed Monthly Earnings)/AMW(Average Monthly Wage)


34

Public Use of the SIPP Synthetic Beta

• Full version (16 implicates) released to the Cornell VirtualRDC Synthetic Data Server (SDS)

• Any researcher may use these data• During the testing phase, all analyses must be performed

on the Virtual RDC• Census Bureau research team will run the same analysis

on the completed confidential data• Results of the comparison will be released to the

researcher, Census Bureau, SSA, and IRS (after traditional disclosure avoidance analysis of the runs on the confidential data)

4/29/2013

35

Methods for Estimating the PPD• Sequential Regression Multivariate Imputation (SRMI) is a

parametric method where PPD is defined as

• The BB is a non-parametric method of taking draws from the posterior predictive distribution of a group of variables that allows for uncertainty in the sample CDF

• We use BB for a few groups of variables with particularly complex relationships and use SRMI for all other variables

dXYpXYYpXYYp obsobsobsobsobsobs ,|,,|~

,|~

36

SRMI Method Details• Assume a joint density that defines parametric relationships

between all observed variables.• Approximate the joint density by a sequence of conditional

densities defined by generalized linear models.• Same process for completing and synthesizing data• Synthetic values of some are draws from:

where Ym, Xm are completed data, and densities pk are defined by an appropriate generalized linear model and prior

dXYpXYypXYyp mm

k

mm

kkk

mm

kk ,|,,|~,|~~

Yyk


37

SRMI Details: KDE Transforms

• The SRMI models for continuous variables assume that they are conditionally normal

• This assumption is relaxed by performing a KDE-based transform of groups of related variables

• All variables in the group are transformed to normality, then the PPD is estimated

• The sampled values from PPD are inverse transformed back to the original distribution using the inverse cumulative distribution

4/29/2013


38

SRMI Example: Synthesizing Date of Birth

• Divide individuals into homogeneous groups using stratification variables– example: male, black, age categories, education

categories, marital status– example: decile of lifetime earnings distribution,

decile of lifetime years worked distribution, worked previous year, worked current year

• For each group, estimate an independent linear regression of date of birth on other variables (not used for stratification) that are strongly related

4/29/2013


39

SRMI Example: Synthesizing Date of Birth

• Synthetic date of birth is a random variable• Before analysis, it is transformed to normal using the KDE-

based procedure• Distribution has two sources of variation:

– variation in error term in regression model– variation in estimated parameters: ’s and 2

• Synthetic values are draws from this distribution• Synthetic values are inverse transformed back to the original

distribution using the inverse cumulative distribution

4/29/2013


40

Bayesian Bootstrap Method Details

• Divide data into homogeneous groups using similar stratification variables as in SRMI

• Within groups do a Bayesian bootstrap of all variables to be synthesized at the same time.– n observations in a group, draw 1-n random variables from

uniform (0,1) distribution– let uo … ui … un define the ordering of the observations in the

group– ui – ui-1 is the probability of sampling observation i from the group

to replace missing data or synthesize data in observation j– conventional bootstrap, probability of sampling is 1/n

4/29/2013


41

Creating Synthetic Data

• Begin with base data set that contains only non-missing values

• Use BB to complete missing administrative data – i.e. find donor SSN based on non-missing SIPP variables

• Use SRMI to complete missing SIPP data • Iterate multiple times – input for iteration 2 is

completed data set from iteration 1 • On last iteration, run 4 separate processes to create

4 separate data sets or implicates4/29/2013


42

Creating Synthetic Data, Cont.

• Synthesis is like one more iteration of data completion, except all observations are treated as missing

• Each completed implicate serves as a separate input file• Run 16 separate processes to create 16 different

synthetic data sets or implicates• The separate processes to create implicates have

different stratification variables• Need enough implicates to produce enough variation to

ensure that averages across the implicates will be close to truth

4/29/2013


43

Features of Synthesizing Routines

• Parent-child relationships– foreign-born and decade arrive in US– welfare participation and welfare amount– presence of earnings, amount of earnings

• Restrictions on draws from PPD– Some draws must be within a pre-specified range from

the original value: example MBA is +/- $50 of original value.

– impose maximum and minimum values on some variables

4/29/2013


44

External Researcher Validation

• Version 4.0 – 12 projects– 1 was submitted for validation

• Version 5.0– 31 projects– 6 were submitted for validation

4/29/2013


45

Validation Details

• Henriques, Alice (2102) “How does Social Security claiming respond to incentives? Considering husbands’ and wives’ benefits separately”

• Armour, Philip (2012) “The role of information in disability insurance take-up: An analysis of the Social Security statement phase-in”

• Bertrand, Marianne, Emir Kamenica and Jessica Pan, “Gender identity and relative income within households”

4/29/2013


46

From Bertrand et al.

4/29/2013

Timeline: SDS application November 2012, gold standard results January 2013


47

SYNTHETIC LONGITUDINAL BUSINESS DATABASE

4/29/2013


48

The Synthetic Longitudinal Business Database

Based on presentations by Kinney/Reiter/Jarmin/Miranda/Reznek2/Abowd

on July 31, 2009 at the

Census-NSF-IRS Synthetic Data Workshop

[link] [link]

Kinney/Reiter/Jarmin/Miranda/Reznek/Abowd (2011) “Towards Unrestricted Public Use Microdata

: The Synthetic Longitudinal Business Database.”, CES-WP-11-04

Work on the Synthetic LBD was supported by NSF Grant ITR-0427889, and ongoing work is supported by the Census Bureau. A portion of this work was conducted by Special Sworn Status researchers of the U.S. Census Bureau at the Triangle Census Research Data Center. Research results and conclusions expressed are those of the authors and do not necessarily reflect the views of the Census Bureau. Results have been screened to ensure that no confidential data are revealed.

4/29/2013

http://www.vrdc.cornell.edu/news/nsf-census-irs-workshop2009/program/2-2-kinney-itr09/

http://www.vrdc.cornell.edu/news/nsf-census-irs-workshop2009/program/3-1-reznek/

http://www.census.gov/ces/search.php?search_what=paps&detail_key=101943




49

Overview

• LBD background

• Synthetic data generation

• Analytic validity

• Confidentiality protection

• Future plans

4/29/2013


50

Elements

4/29/2013

(Economic Surveys and Censuses)

Issue: (item) non-response

Solution: LBD

(Business Register)Issue: inexact link

recordsSolution: LBD

Match-merged and completed

complex integrated dataIssue: too much detail

leads to disclosure issueSolution: Synthetic LBD

Public-use dataWith novel detail

Novel analysis using Public-use data with novel detailIssue: are the results rightSolution: Early release/SDS


51

The Real LBD

• Economic census covering nearly all private non-farm business establishments with paid employees– Contains: Annual payroll and Mar 12 employment

(1976-2005), SIC/NAICS, Geography (down to county), Entry year, Exit year, Firm structure

• Used for looking at business dynamics, job flows, market volatility, international comparisons…

4/29/2013


52

Longitudinal Business Database (LBD)

• Detailed description in Jarmin and Miranda • Developed as a research dataset by the U.S.

Census Bureau Center for Economic Studies• Constructed by linking annual snapshot of the

Census Bureau’s Business Register (see Lecture 4)

4/29/2013


53

Longitudinal Business Database II

• CES constructed • Longitudinal linkages (using probabilistic

record linking, see Lecture 10)• Re-timed multi-unit births and • Edits and imputations for missing data

4/29/2013


54

Access to the LBD

• Different levels of access• Public use tabulations – Business Dynamics

Statistics http://www.ces.census.gov/index.php/bds

• “Gold Standard” confidential micro-data available through the Census Research Data Center (RDC) Network– Most used dataset in the RDCs

4/29/2013

http://www.ces.census.gov/index.php/bds

http://www.ces.census.gov/index.php/bds


55

Bridge between the Two

• Synthetic data set– Available outside the Census RDC– Providing as much analytical validity as possible– Reduce the number of requests for special

tabulations– Aid users requiring RDC access

• Experiment in public use business micro-data

4/29/2013


56

Why Synthetic Data?

• Concerns about confidentiality protection for census of establishments– LBD is a test case for business data

• Criteria given for public release:– No actual values of confidential values could be

released– Should provide valid inferences while protecting

confidentiality

4/29/2013


57

Generic Structure

• Gold standard: given by internal LBD (already completed)

• Partially synthetic:– Unsynthesized:

• County (but not released!) [x1]• SIC [x2]

– Synthesized• Birth [y1] and death [y2] year:• Multi-unit status [y3]• Employment (March 12) [y4]• Payroll [y5]

4/29/2013


58

Synthesis: General Approach

• Y=[y1|y2|y3|y4|y5]• X=[x1|x2]• Generate joint distribution of Y|X by sampling

from conditionals– f(y1,y2,y3|X) = f(y1|X)·f(y2|y1,X)·f(y3|y1,y2,X)

• Use SIC as “by group”

4/29/2013


59

General Approach to Synthesis

• Drawing from f(yk|X,y1,...,yk-1)– Fit model using observed data– Draw new values of parameters from posterior

distributions– Use new parameters to predict yk from X and

synthetic values of y1,...,yk-1

4/29/2013


60

The Sequential Regression Multivariate Imputation (SRMI) Approach

• Calendar:– Step1: Impute y1 | X– Step 2: Impute y2 | [y1| f(X)]

• Where f(X) uses state [x1’] instead of county [x1]

• Type of firm– Step 3: Impute y3 | [y1|y2|X]

• Characteristics– Step 4: Impute y4(t)|[y1|y2|y3|y4(t-1)|x2]– Step 5: Impute y5(t)|[y1|y2|y3|y4(t)|y5(t-1)|x2]

4/29/2013


61

First Year

• Impute y1 (Firstyear) | SIC, County using variant of Dirichlet-Multinomial– Prior information is obtained by collapsing

categories– Synthetic values obtained from sampling from

multinomial distribution

4/29/2013


62

Last Year

• Impute y2 (Last Year)| First Year, State, SIC• Simple multinomial approach

– Dirichlet-multinomial with flat prior– Sample from multinomial probabilities obtained

from matching categories in observed data

4/29/2013


63

Multi-unit Status

• Impute in two stages:– Categorical response: Always MU, sometimes MU,

never MU– Imputed using simple multinomial approach

• Given change in status occurs, impute when change occurred (future)

4/29/2013


64

Employment and Payroll

• Highly skewed longitudinal continuous variables• Imputed using a set of normal linear models with kde

transformation of response (Abowd and Woodcock, 2004)

• Impute year by year, employment and then payroll, based on groups– (3-digit SIC) – by (multiunit status) – by (continuer status)– by (top 5% status)

• If model too sparse, use 2-digit SIC as prior4/29/2013


65

Analytical Validity Tests

• Compare observed data and synthetic data for whole LBD

• Job creation and destruction• Employment volatility• Gross employment levels

4/29/2013


66

Job Destruction Rates: LBD and Implicates by Year

05

101520253035404550

19

77

19

78

19

79

19

80

19

81

19

82

19

83

19

84

19

85

19

86

19

87

19

88

19

89

19

90

19

91

19

92

19

93

19

94

19

95

19

96

19

97

19

98

19

99

20

00

Year

LBD Implicate 1 Implicate 2 Implicate (Mean)

Job Creation Rates: LBD and Implicates by Year

05

101520253035404550

19

77

19

78

19

79

19

80

19

81

19

82

19

83

19

84

19

85

19

86

19

87

19

88

19

89

19

90

19

91

19

92

19

93

19

94

19

95

19

96

19

97

19

98

19

99

20

00

Year


4/29/2013


67

Job Creation from Births: LBD and Implicates by Year

01,0002,0003,0004,0005,0006,0007,0008,0009,000

10,000

1977

1978

1979

1980

1981

1982

1983

1984

1985

1986

1987

1988

1989

1990

1991

1992

1993

1994

1995

1996

1997

1998

1999

2000

Tho

usan

ds

Year


4/29/2013


68

Job Creation from Births and Expansions: LBD and Implicates by Year

05,000

10,00015,00020,00025,00030,00035,00040,000

19

77

19

78

19

79

19

80

19

81

19

82

19

83

19

84

19

85

19

86

19

87

19

88

19

89

19

90

19

91

19

92

19

93

19

94

19

95

19

96

19

97

19

98

19

99

20

00

Th

ou

san

ds

Year


4/29/2013


69

Net Job Creation Rates: LBD v Implicates

-10

-5

0

5

10

15

1977

1978

1979

1980

1981

1982

1983

1984

1985

1986

1987

1988

1989

1990

1991

1992

1993

1994

1995

1996

1997

1998

1999

2000

Net Job Creation LBD Net Job Creation Implicate 1 Net Job Creation Implicate 2

4/29/2013


70

Employment Volatility: Establishment by Year, weighted

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

1977

1978

1979

1980

1981

1982

1983

1984

1985

1986

1987

1988

1989

1990

1991

1992

1993

1994

1995

1996

1997

1998

1999

2000

YearVolatility (LBD, Weighted) Volatility (Imp 1, Weighted)

Volatility (Imp 2, Weighted) Volatility (Imp-Mean, Weighted)

4/29/2013


71

Employment: LBD and Implicates by Year

0

100000

200000

300000

400000

500000

1977 1979 1981 1983 1985 1987 1989 1991 1993 1995 1997 1999

Year

Co

un

t

LBD Synthetic

4/29/2013


76

Confidentiality Protection

• Unavailable in SynLBD V2 (current on SDS)– Firm structure– Firm linkages (across time, across implicates)– Geography

• Basic protection– Replacing sensitive values of with draws from

probability distributions

4/29/2013


77

Disclosure Avoidance Review

• High probability that an individual establishment’s synthetic birth/death year is different from its actual birth/death year

• Synthetic maxima not necessarily near actual• High between-imputation variability at

establishment level

4/29/2013


78

Synthesizing Firstyear (Birth) and Lastyear (Death)

• Positive probability exists of producing any feasible birth year, and substantial probability exists that synthesized firstyear is not the actual firstyear

• Table on next slide shows this: prob(actual birth year=synthetic birth year l synthetic birth year) is low

• Similar results hold for deaths• Conclusions: establishment lifetimes are random,

so users can’t accurately attach establishment identifications to them

4/29/2013


794/29/2013

Example: Year of birth


81

Confidentiality Protection: Breaking Firm Links

• Firm characteristics not synthesized• Firm characteristics more skewed than

establishment characteristics• Cannot link multi-unit establishments to their

firms

4/29/2013


82

Confidentiality Protection: Breaking Links Across Implicates

• Synthetic observations with the same LBDnum across implicates are not generated from the same LBD establishment

• Can’t group (across implicates within year) observations generated from same establishment

4/29/2013


83

Confidentiality Protection: Synthesizing Employment and Payroll

• Synthesis models are essentially regressions with transformed variables

• Synthesis captures low-dimensional relationships and sacrifices higher-dimensional ones

• Synthesized employment and payroll vary substantially around regression lines

• Synthesized employment and payroll vary significantly from observed values

4/29/2013


84

Example: Correlations Among Actual and Synthetic Data

• SIC 573 - year 2000

4/29/2013

Pearson Correlation CoefficientsSIC 573Year: 2000

EmploymentSynthetic Employment Payroll

Synthetic Payroll

Employment 141000

Synthetic 0.003 1Employment 21100 41000Payroll 0.712 -0.012 1

41000 21100 41000Synthetic 0.007 0.444 0.004 1Payroll 21100 41000 21100 41000

Slide 84


854/29/2013


864/29/2013


87

Conclusions

• Analytical validity supported for broad analyses– Issues with some details– Obtain user feedback to inform future refinements

• Sufficient confidentiality protection– Basic metrics show strong protection– Differential privacy protection not yet verified

4/29/2013


88

– Include NAICS, geography, changes in multiunit status, firm age and size

– Multiple Imputations for release– Address bias in job creation/destruction– Extend time series

Ongoing Work at Census

4/29/2013


89

External Validation Exercises

• 41 approved projects (includes provisional approvals)

• 3 have submitted results for validation (one of these did two rounds of validation)

• Moscarini timeline: application approved March 2011, validation results released September 2011

4/29/2013

INFO 7470/ECON 7400 Synthetic Data Creation and Use John M. Abowd and Lars Vilhuber with a big...

Documents

Transcript of INFO 7470/ECON 7400 Synthetic Data Creation and Use John M. Abowd and Lars Vilhuber with a big...