Dueling Estimates:Closing Thoughts

Jennifer H. Madans

National Center for Health Statistics

Assumptions• Conflicting estimates are a

problem for the Federal Statistical System • They cause confusion among

policy makers, the press and the public

• They reduce confidence in the data produced by the system and each agency

• The producers may understand and accept the causes but the users do not

Assumptions• Producers and users would prefer

if estimates of what appear to be the same phenomena were similar if not the same

• Much attention has been directed toward the problem but results have been unsatisfying• We are still talking about it at

this seminar

The Challenge• Multiple estimates is a natural

result of a decentralized federal statistical system• Focus on maximizing the utility

of information for Departmental policy needs

• Focus on maximizing quality of each data collection activity

• Internal needs trump cross system needs

Session Objective

• Revisit possible solutions from a data collection life cycle point of view

• Acknowledge the benefits and costs of different approaches

• Start a discussion about what would be needed to have a more structured, proactive approach to reduce the impact of competing estimates and the associated costs

Current Situation

• Previous speakers outlined several different approaches • Front end – joint planning and

testing with coordinated implementation

• Back end -- Explanation of what is being measured; potential source of difference; joint analysis; joint publication

Front End Solutions

• Costly but hold promise• Increased cost of data collection• Reduced flexibility in collection

cycle• Potential sacrifice of survey

specific aims to achieve the system’s need for comparability

Back End Solutions

• Solutions of lowest data collection and staff cost but are probably least satisfying

• Will always be needed even if other approaches are used

System Solution• Statistical agencies produce multiple

estimates of the same thing that generate little attention or confusion

• Only a limited number are dueling• For these estimates, agree to

designate which are ‘official’ • Official estimates can be a set of

related estimates each with specific uses

System Solution• Requires more of a change in how

agencies relate to each • Limits statistical agency ability to

communicate with users, the press and the public • Could have unintended impact of

budgets

Which Way Forward

• System is too complex to embrace one solution• How do we determine what

solution works best for different types of estimates?

• Not all ‘dueling estimates’ are equally problematic• How do we identify the ones

that need most attention?

Which Way Forward

• When are the costs of minimizing the existence of dueling estimates too high?

• Are there other benefits of strengthening ‘System’ objectives in areas of data collection and dissemination of findings?