Making Family Outcome Data Representative of the State
-
Upload
cullen-cunningham -
Category
Documents
-
view
44 -
download
1
description
Transcript of Making Family Outcome Data Representative of the State
Making Making Family Family
Outcome Outcome DataData
RepresentatiRepresentative ve
of the Stateof the State
Monday, August 27, Monday, August 27, 20072007
Rosanne Griff-Rosanne Griff-CabelliCabelli
Sue CampbellSue [email protected]
Delaware Birth to Delaware Birth to Three Three
Early Intervention Early Intervention SystemSystem
Purpose of the Purpose of the Original Family Original Family
SurveySurveyThe survey was conducted in order to
•obtain information about families’ perceptions of and satisfaction with services they received from the Child Development Watch Program (CDW)
•determine families’ satisfaction with the CDW office facilities and staff interactions
•determine the impact of CDW and Birth to Three services on families’ quality of life
Topics Addressed Topics Addressed in the Original Family in the Original Family
SurveySurvey Did parents think their ability to parent theirchild with a disability improved?
Did parents think their child(ren)’s development improved?
Did parents feel that CDW was responsive to their needs?
Did parents feel that they had input into the services their children received?
Were families satisfied with CDW and staffinteractions?
Did CDW services have a positive impact onfamilies’ quality of life?
2006 Family Outcomes to be Assessed2006 Family Outcomes to be Assessed
Outcome 1:Families know their rights and advocateeffectively for their child
Outcome 2:Families Understand their child’s abilitiesand special needs
Outcome 3:Families help their child develop and learn
Reasons for a PilotReasons for a Pilot
OSEP Family OutcomesDevelop and integrate OSEP outcomesquestions into existing survey
Expand responses for existing survey itemsFrom a three point response to a six pointresponse
Test strategy of telephone interview for surveyingfamilies
Developing the Developing the New Family SurveyNew Family Survey
Delaware Ongoing Program Evaluation
Committee (OPEC) made recommendations:
Added questions to increase depth and range of response for the three family outcomes
Responses reported as single questions and by clusters– look at trends since 1999
Expand responses of the existing survey froma range of three to a range of sixBEFORE: Yes; Less than I’d Like; NoAFTER: Very strongly agree; Strongly agree; Agree; Disagree; Strongly disagree; Very strongly disagree
Piloting the New Family SurveyPiloting the New Family Survey
Conducted in Fall 2005
Telephone contact families randomly selected letter sent to families before calls made Calls made during day and evening
Demographics of 2005 Demographics of 2005 Telephone Pilot Family Survey Telephone Pilot Family Survey Respondents (N=49)Respondents (N=49)• Gender (child’s)
26 male(53%)22 female (45%)
• Race/Ethnicity38 Caucasian (79%)7 AfricanAmerican (15%)1 Hispanic (2%)1 Asian (2%)1 Not Reported (2%)
• Family Income11 >$100,000 (23%)19 $50-100,000 (41%)9 $20-49,999 (19%)6 <$20,000 (13%)2 Not Reported
• Location24 New Castle County (49%)25 Kent or Sussex County (51%)
2005 Federal Outcomes2005 Federal OutcomesMeasures Internal Measures Internal ConsistencyConsistency
Federal Outcome Topic
Number of
Questions
Chronbach Alpha Coefficie
nt
% of Families indicating “yes”
Families Know their Rights
4 .846 91.1%
Families Effectively Communicate their Children’s Needs
5 .715 95.1%
Families help their Children Develop and Learn
4 .808 93.4%
1.Questions from pilot found appropriate for use
2.Better to use telephone survey than mail distribution method
3.Stratified sample rather than random sample
Pilot resulted in the following:Pilot resulted in the following:
2006 Telephone Interviews 2006 Telephone Interviews with Familieswith Families
Representative sample is predetermined byinformation in data system (ISIS)•Geographic region•Race/ethnicity•Length of time in CDW (state focus)
Sample matrix currently includes 12 data cells•Each cell contains 30 families to be sampled•Total of 360 families to be sampled
Can always add specific demographics if
need to focus on specific populations
Small cell size
Sampling MatrixSampling Matrix
Geographic Region
Ethnicity Length of Time in Program
North
Caucasian < 18 months (30 families)
> 18 months (30 families)
African American
< 18 months (30 families)
> 18 months (30 families)
Other < 18 months (30 families)
> 18 months (30 families)
South
Caucasian < 18 months (30 families)
> 18 months (30 families)
African American
< 18 months (30 families)
> 18 months (30 families)
Other < 18 months (30 families)
> 18 months (30 families)
Number of Families by Region, Number of Families by Region, Ethnicity, and Ethnicity, and
Months in CDWMonths in CDW
Region
African American
Caucasian Other
Total
Less than 18Months
More than 18Months
Less than 18Months
More than 18Months
Less than 18Months
More than 18Months
North 110 57 221 147 112 37 684
South
52 26 104 60 35 15 292
Total 162 83 325 207 147 52 976
Received list of 976 families in CDW684 in New Castle County (CDW
North)292 in Kent and Sussex Counties
(CDW South)
Before the Telephone Interviews…Before the Telephone Interviews…
850 of these families were mailed:
1. a cover letter explaining the purpose of the interview
2. an information sheet describing• the usefulness of family feedback• assurances of confidentiality• examples of some information that would be asked• additional contact numbers if families had
questions about the interview
Number of Letters Mailed by Number of Letters Mailed by Region, Ethnicity, and Months Region, Ethnicity, and Months in CDW in CDW (N=850)(N=850)
EthnicityAfrican
AmericanCaucasian Other
Length of Time in Program
Less than 18
Months
Greater
than 18
Months
Less than 18
Months
Greater
than 18
Months
Less than 18
Months
Greater
than 18
Months
State
1st Mailing
112 83 120 120 147 52
2nd Mailing
6 0 6 0 0 0
3rd Mailing
44 0 100 60 0 0
Total 162 83 226 180 147 52
Letters Mailed and Return RateLetters Mailed and Return Rate
Telephone calls were made to all of the families from the original list, unless letters were returned due to incorrect addresses, until 224 families had completed the Family SurveyFamily Survey. .
The families were contacted between one and four times.
Calls would be made during the day, in the evenings, and on the weekends
Reasons families were not contacted via telephone:
•Telephone was disconnected•Number was wrong •Families were found not to meet the
criteria for completing the telephone interview
Reasons why Families were not ContactedReasons why Families were not Contacted
Reasons families were not contacted via telephone:
•97 Telephone was disconnected (35.4%)
•63 Number was wrong (23.0%)•59 Mail returned (21.5%)•28 No phone number listed (10.2%)•9 Less than six months in CDW
(3.3%)•5 Phone not in service (1.8%)•14 Other reasons (5.1%)
275 families
How did Delaware Determine How did Delaware Determine Family Outcome Data is Family Outcome Data is Representative?Representative?
1.Contract with the Center for Disabilities Studies, University of Delaware to moderate biased response
2. Sample according to demographics of Annual Child Count
Cost Data Preliminary AnalysisCost Data Preliminary Analysis
Delivery Method Mailed Survey
Telephone Interview
Year of Implementation 2004 2006
Sampling Method Random SampleStratified Random
Sample
Number Contacted 203 surveys 576 candidates
Number Completed 100 surveys 224 interviews
Return Rate 49.3% of sample 38.9% of sample
% of the Population 10.80% 22.90%
Cost/Person Contacted $3.23/family $2.34/family
Cost/Completed Survey $6.57/survey $8.91/survey
Methodology Mailed survey up to 4 times Mailed letter
Telephone call reminder
Telephone Interviews
Costs IncludedMailed letter and
survey Mailed letter
Return envelope Telephone calls
Return mailing
Survey reproduction
costs
Phone call reminders
ConclusionsConclusions
1.The expanded response possibilities allows for deeper analysis of families’ responses (move to 6 point scale)
2.The telephone survey allows for individual responses and clarification of questions
3.The telephone survey allows for the immediateidentification of the sample demographics, and allows for targeting of underrepresented categories. Data collection will continue until all categories are appropriately represented
4. The telephone survey also allows for immediatefeedback as to why families chose not to participate and the demographics of who chose not to participate
Lessons Learned for Efficient Lessons Learned for Efficient Family Surveying and Adequate Family Surveying and Adequate RepresentationRepresentation1.Contact families at various times
of the day within a shorter timeframe (1-2 weeks) rather than making multiple attempts at a larger timeframe.
2.Request updated information from CDW offices to contact families who have disconnected phone numbers, no telephone numbers or wrong telephone numbers. If phone number still not available, mail survey.
3.Send a follow-up letter to families asking families to call back to schedule an interview