Microsimulation of Survey Collection
description
Transcript of Microsimulation of Survey Collection
Microsimulation of Survey Collection
Yves BélangerKristen Couture26 January 2010
2
Outline
Motivation Main aspects of microsimulation Overview of the system A short demo A few results Future work
3
Motivation Ultimate goal: make CATI collection more
efficient proactive collection management
Recent initiatives in the field Experimentation with time slices, cap on calls,
calling priorities, Z-groups, ... Takes time, lack of control, costly(?), results not
always easy to interpret
Need for a controlled environment, where the impact of each aspect can be isolated
4
Main Aspects of Microsimulation What is microsimulation?
A modelling technique that operates at the level of individual units, such as persons, households, vehicles, etc.
For us: a "virtual collection" system
What elements are we considering? The cases (sampled units) The servers (interviewers) The call attempts The waiting queue(s) The rules of the call scheduler (flows and priorities)
5
Main Aspects of Microsimulation (cont'd)
What do we want to simulate?1. A random component: the result of each
call attempt Use existing BTH data with appropriate
statistical models
2. A deterministic component: how the cases flow through the system Use a simulation software to replicate
Blaise: SAS Simulation Studio
6
Overview of the System
Pre-existing BTH from Survey
Model Call Outcomes Model Call Duration
Simulation
Collection Parameters
7
Overview of the System (cont'd)
Call outcome Modeled using CSGVP 2004 BTH data Five outcomes derived from BTH
outcome codes Unresolved (eg. Busy signal, wrong #) Out of Scope (eg. Cell phone, Business) Refusal Other Contact (eg. Ans. Machine, appointment) Respondent
8
Overview of the System (cont'd)
Used Multinomial Logistic Regression
7 parameters entered into model: Afternoon – 1 if call made between 12 and 5 Evening – 1 if call made between 5 and 9 Weekend - 1 if call made on weekend Resid – 1 if initial status was residential Unresolved – 1 if call history is only unresolved Refusal – 1 if history shows at least one refusal Contact – 1 if history shows at least one contact
i = 1..nj = 1..k
9
Overview of the System (cont'd)
Calculate probability for each of the five possible outcomes using estimated betas and collection parameters
10
Overview of the System (cont'd)
Call duration Modeled using existing CSGVP 2004 BTH
data Modeled distributions for each of the 5
outcomes
11
Overview of the System (cont'd)
Components of model Input
Allows user to enter parameters via SAS data sets
12
Overview of the System (cont'd)
Clock Creates Time Parameters including
Afternoon, Evening, Weekend, and Time Slice by reading the current simulation time
13
Overview of the System (cont'd)
Queuing System Cases are created and enter a queue
waiting to be interviewed
14
Overview of the System (cont'd)
Determining Call Outcome Uses probability formulas to determine call
outcome: Unresolved, Out of Scope, Other Contact, Refusal, Respondent
15
Overview of the System (cont'd)
Call Center Interview takes place Call duration is simulated Ability to control interviewer schedule
16
Overview of the System (cont'd)
Finalizing Cases Case exits system when…
Outcome code = OOS or Respondent Cap on Calls is reached
Cap of 20 for Residential Status Cap of 5 for Unknown Status
Number of Refusals=3 A BTH file is created as output in terms of a
SAS dataset
17
A Short Demo
18
A Few Results
Simulation with 10,000 cases for 30 days of collection
Interviewer Agenda Shift 1 (9am-12pm): 10 interviewers Shift 2 (12pm-5pm): 10 interviewers Shift 3 (5pm-9pm): 10 interviewers
* Note: No Time Slices in this example
19
A few results (cont'd)
Finalized Cases and Response Rate
Distribution of Outcome Codes
20
A few results (cont'd)
Impact of Changing Parameters Number of Interviewers
Length of Collection Period
21
A few results (cont'd) Changing the Time Per Unit Cap on Calls is in Effect
22
Future Work Continue improvements to system
To outcome model More explanatory variables Distinguish between hhld and person contacts
To simulation system Implement time slices Improve priorities
Presentation to JSM (incl. article) Potential cooperation with Census Other?... will depend on available budget