Post on 23-Feb-2016
description
ELLEN ZAGER HILL , MS
TB PROGRAM EPIDEMIOLOGISTIDAHO DEPARTMENT OF HEALTH &
WELFARE
Refugee and Immigrant B Notification Program Evaluation
—Idaho
Idaho Geography (1)
Geographically large western state with a low population 19.2 people per square
mile (ranked 44th in US)
Population ≈ 1.58 million people 37% of population is in
the Boise metro area (Ada & Canyon Counties)
Idaho Geography (2)
44 counties divided into 7 multi-county regional public health districts
95%+ of refugees and immigrants settle in Boise (Ada County) and Twin Falls (Twin Falls County).
Background
For a state with the size population that Idaho has, the state receives an unusually large number of refugees and immigrants
2 local public health districts receive essentially all of the refugees arriving in the state: PHD4 (90%+) and PHD5 (~10%)
Immigrants arriving in Idaho are pretty evenly distributed; in SFY 2007–2009, 83 immigrants with TB notifications arrived
SFY
Idaho Populatio
n
No. of Refugees Arriving
Per capita ratio
(per 10,000 people)
No. with TB notifications
2007 1,499,402 834 5.6 30 (3.5%)2008 1,523,816 1,080 7.1 54 (5.0%)2009 1,545,801 1,291 8.4 76 (5.8%)
Evaluation Plan
Logic Model
Refugee/ Immigrant with a TB
notification
Inputs
Private Provider
State TB Program
Staff
LPHD
Short term Long term
Outcomes
The LPHD will increase the
proportions of completion of
medical evaluations within
90 days.
Increase the proportions of refugees/immigrants who
started on LTBI treatment who complete treatment.
Reduce the number of
foreign born TB cases.
Outputs
Every LPHD has a process for contacting, evaluating, and treating refugees and
immigrants with TB notification.
Data for grant progress report.
Refugees and immigrants with a TB notification are medically evaluated for
TB.
Refugees or immigrants treated who need to be
treated.
Activities
Form turned into state TB
program
Fill out CDC TB Notification
form
Complete medical evaluation
Complete treatment for
LTBI
OEFI receives notification from CDC
Start treatment for LTBI
District receives notification from OEFI
Contact Refugee or immigrant
Initiate medical evaluation
Evaluation Goal Evaluation Team
Baseline analysis of medical screen activities of refugees and immigrants arriving in Idaho with a TB notification
Lead Evaluator Ellen Zager Hill – TB
PEN Focal Point/TB Epi
Team Members Christine Hahn – TB
Controller/State Epi Adele Smith – MPH
student from ISU
Evaluation Plan (1)
Evaluation Plan (2)
Big Picture Questions Corresponding Specific Questions
Are the medical evaluations for refugees and immigrants with TB notifications done in a timely manner?
What is the proportion of refugees or immigrants that were evaluated within 90 days of arrival?
Are the PHDs sending notifications back to the OEFI in a timely manner?
What is the proportion of TB notification forms that are received within 120 days of receiving the notification?
What is the process that the PHDs go through to contact the refugees and immigrants with TB notifications for medical evaluations?
7 out of 7 PHDs have a process to contact refugees or immigrants with TB notifications
Evaluation Plan (3)
Big Picture Questions (continued)
Corresponding Specific Questions (continued)
Do the PHDs have a different process for evaluating and treating refugees and immigrants with TB notifications?
7 out of 7 PHDs have no difference in evaluation or treatment of refugees and immigrants with TB notifications
What is the process the PHDs perform to evaluate refugees and immigrants with TB notifications?
7 out of 7 PHDs have a process to evaluate refugees or immigrants with TB notifications.
What is the process the PHDs perform to treat refugees and immigrants with TB notifications?
7 out of 7 PHDs have a process to treat the refugees or immigrants with TB notifications who have LTBI.
Data Sources Execution Needs
Idaho Refugee & Immigrant Log (MS Excel spreadsheet)
CDC’s Electronic Disease Notification System
Interviews with PHD staff who coordinate TB activities (via phone)
TimeSurvey questionsPhoneSoftware to map
processes (MS Visio) (relatively) Clean TB
notification dataData analysis tool (MS
Excel and SAS)
Evaluation Plan (3)
Carrying Out theEvaluation Plan
Carrying Out the Evaluation Plan (1)
Developed a set of questions to ask the PHD staff who carry out TB activities
Developed a script to go with the questions
Carrying out the Evaluation Plan (2)
A flow of TB notification data from the state-level perspective was developed (shown at left)
Based on survey questions, flow charts of the evaluation and treatment processes were developed for each PHD
Data Cleaning Data Analysis
Started with our Idaho Refugee & Immigrant Log Took out all records that
were not for TB notifications
Filled in what missing data we could from hard copy records and the EDN system
Initially done with MS Excel
Final analysis done with SAS Benefit of using a
program like SAS is that you have a record of exactly how the data was cleaned, what analysis was done, and how it was done.
Carrying out the Evaluation Plan (3)
Carrying out the Evaluation Plan (4)
Some lessons learned…
Logs (especially those with no data standardization programmed in) are a difficult data source to analyze
(A lesson re-learned) small denominators can be a problem
Clearly document how you collect, clean, and analyze data An evaluation report is like a lab report. Your methods section may be your most important section.
(Sometimes even more important than your results.) SAS or other statistical analysis programs are a great way to
document how you are cleaning and analyzing data
Evaluation Results
Evaluation Results (1)
Are the medical evaluations for refugees and immigrants with TB notifications done in a timely manner? What is the proportion
of refugees or immigrants that were evaluated within 90 days of arrival?
Overall (SFY2007–2009) 67% were evaluated within 90 days of arrival.
Evaluation Results (2)
Are the PHDs sending notifications back to the OEFI in a timely manner? What is the proportion
of TB notification forms that are received within 120 days of receiving the notification?
Overall 76% of TB notification forms were returned within 120 days of receiving the original notification.
Evaluation Results (3)
Process Questions Results7 out of 7 PHDs have a process to contact refugees or immigrants with TB notifications
• All 6 PHDs interviewed have a process (the 7th is known (anecdotally) to have a process.• Most have a process for contacting hard-to-reach R/Is and a defined point when they declare someone “lost-to-follow-up”
7 out of 7 PHDs have no difference in evaluation or treatment of refugees and immigrants with TB notifications
• Of the 6 PHDs interviewed only 4 have received notifications for both refugees and immigrants. Of these 4, only 1 reported a difference in protocol (a different staff member follows each group)
7 out of 7 PHDs have a process to evaluate refugees or immigrants with TB notifications
• All PHDs have a process for evaluating refugees and immigrants• There is significant variation in these processes between PHDs
7 out of 7 PHDs have a process to treat the refugees or immigrants with TB notifications who are diagnosed with LTBI.
• Of the 6 PHDs interviewed, 5 actively follow R/Is on treatment for LTBI. The remaining PHD prefers that R/Is diagnosed with LTBI are treated and followed in their medical home.
Evaluation Results (4)
Extra analysis tied to differences in processes…
*Excludes data from 1 PHD (the PHD with the most notifications) because we were unable to interview them.
PHD-level State-level
Ensure there is a system in place to track TB notifications
Review process for coordinating medical evaluation on a semi-regular schedule
Consult OEFI TB staff as needed when a problem related to medical evaluation arises
Consider adding language to TB contracts stipulating a time frame for returning TB follow-up forms
Update B Notifications chapter of Idaho TB manual
Provide at least 1 NTIP-like report a year to each PHD that contains an analysis of their TB notification statistics
Review data from calendar years 2009, 2010, and 2011 for improvement. If improvement is not seen, then consider re-evaluating this activity.
Recommendations
What has been done as a result of the evaluation
We are no longer using the Idaho Refugee and Immigrant log to monitor performance for this objective. We are using solely EDN data.
We added language to contracts stipulating when TB follow-up data needs to be submitted.
B Notifications chapter of TB manual has been reviewed an updated, but has not yet been finalized.
Completion of medical evaluation statistics as of 1/31/2012: (calendar years)
2008 2009 2010
2011
N 48 93 88 71
Mean (days) 147 132 143 70
Std Dev (days)
162 163 155 43
Low CI (days)
99 98 111 60
High CI (days)
194 166 176 80
Min (days) 15 8 15 4
Max (days) 885 795 743 1901. This data reflects only notifications for which data has been received.
2. Please note that data for all years are incomplete for various reasons.
3. The improvements in data are mainly due to process changes at the PHD that receives 90+% of refugees in the state. (This PHD was not interviewed for the evaluation.)
Questions