Using ECI Data to Make Individual and Program Level Decisions

Judy Carta, Ph.D & Dale Walker Ph.D.

Part 2: ECI WorkshopJuniper Gardens Children’s Project

September 23, 2005

Current IGDI and ECI Development Staff

Charles Greenwood, Judith Carta, Dale Walker,Jay Buzhardt and Barbara Terry

Original development team: Judith Carta, Charles Greenwood, Dale WalkerJane Atwater, Gayle Luze, Deborah Linebarger, Carol Leitschuh, Ken Parsley, Annessa Staab, Gabe Cline, and Susan Higgins

Goals of this session

Learn how to use the ECI to make individual intervention decisions Use the Decision-Making Model for

children who may need intervention Use Exploring Solutions Model for

individual children Discuss potential interventions for

promoting communication Learn how to use ECI to make program

decisions

Focus on Outcomes-Based Program Evaluation

Increased expectations for accountability

Programs and individual staff members need to know when they are making a difference in moving children toward outcomes

EC practitioners need to know when interventions are effective

Most available approaches to program evaluation:

Conventional forms of early childhood assessment not linked to individual child progress on important outcomes

Child data not used in making ongoing program decisions Not repeatable enough Not accessible enough for program staff Not sensitive to intervention or program effects or growth

over time Difficult for parents and practitioners to understand

The ECI can be used for

Identifying children who are not progressing at expected benchmark levels

Monitoring growth over time of individual or groups of children

Assisting in intervention planning process Monitoring progress during and after

intervention

Focus so far has been…

1. Describing outcomes for children in the program

2. Identifying children who are below expected benchmarks

3. Monitoring children’s growth over time

Child slipping below benchmark.

36 Mos ExpectationChild’s

Observed Trajectory

Normative Trajectory

Monitoring Growth Over Time

We want you….

To get the most from your ECI data.

Using ECI Data for Decision-Making about Individual

Children

Just like Pediatric Height & Weight Charts

The ECI is useful both

for monitoring

AND

for examining effects of intervention

IGDIs ESAs

IGDI Decision Making Model

Monitor

Identify/Validate Need

For Intervention

Generate Intervention

Strategies

ImplementIntervention

EvaluateIntervention

Effectiveness

Monitor

©2003 Juniper Gardens Children’s Project

What is Individual Monitoring?

Doing quarterly assessments on each child Seeing if any individual falls below

benchmark Taking action if child falls below benchmark

During quarterly monitoring

If child scores… And…. Then…

Slightly below average No other concerns Continue quarterly monitoring

Slightly below average Other concerns Begin monthly monitoring

Below average No other concerns Begin monthly monitoring

Below average Other concerns Go to Exploring Solutions

On Target No other concerns Continue quarterly monitoring

During monthly monitoringIf child… And…. Then…

Continues to score slightly Below Average

No other concerns Continue monthly assessments

Continues to score Slightly Below Average

Other concerns Go to exploring solutions

Continues to score Below Average

No other concerns Go to Exploring Solutions

Returns to On Target No other concerns Return to quarterly

Using the ECI to monitor before and after intervention

Assessment Guidelines: What is needed in decision-making?

Make decisions on more than one assessment point

Use other data and information from parents and other caregivers

Check that assessment is representative of the child’s functioning

Consider that child may already be receiving Early Intervention (Part C)

Assessment Guidelines: Determining if ECI representative

Does low ECI score fit with what you already know about child’s communication?

Questions to ask: Was the child feeling well and alert? Was the assessor familiar to the child? Was the assessment conducted in a manner that

was conducive to promoting communication? If not, do another ECI to obtain a more

accurate picture.

One data point. Child care provider says child doesn’t join in to conversations very much

What’s the next step?

What do the data show?

What do you do next?

What do the data show?

What do you do?

What do the data show?

What do the data show?

What do you do next?

What do the data show?

What do you do?

What do the data show?

What do you do?

Implementing and Evaluating Interventions

Implementing and Evaluating Interventions to Promote Communication

Goals: Review General Language Intervention Strategies

and Introducing and Monitoring Use of Strategies and Activities from Tool Kit

Discuss Promoting Communication Strategies Discuss successes and challenges in

implementing intervention Ways to monitor if intervention is happening..

Exploring Solutions: Using program and child Information to plan Intervention

(Called Family Needs Assessment: Child Data in Tool Kit pages 3 - 4)

How child is growing in communication? (ECI) Family/caregiver concerns? Medical issues? (hearing, vision, premature, illness) What child knows and does? (other measures of

development) Settings, interactions, activities, routines, curriculum What is possible in terms of intervention?

Communication Intervention Strategies: How to use in practice

Introducing strategies and activities to parent(s), home visitors, and other staff

Discuss rationale for strategy May provide modeling, coaching and practice Be positive Use self-evaluation form Using ECI to monitor intervention outcome

Selecting Intervention from ECI Dropdown Menu

When ready to begin an intervention Select intervention description from menu on

Child Data Entry Form Select menu under Condition Change Select intervention that best describes the

intervention Use Other and describe in child’s file if none fit What are others that would be helpful?

Using effective interventions to promote communication

Importance of using effective, evidence-based interventions to promote communication

Importance of deciding what the possible resources are to effectively deliver intervention

Importance of monitoring the intervention delivery to make sure that it is effective in promoting communication

General Language Intervention Strategies: Tool Kit

Establishing joint attention Balancing interactions (communication turns) Matching utterances Being responsive Following child’s lead Being positive

Strategies from Promoting Communication Project, JGCP

Environmental arrangement (activities to promote attention, book-reading etc.)

Following infant/toddler’s lead and Joint Attention Commenting and labeling Expanding on child communication Increasing responsiveness (talking more) Using questions (open-ended and yes/no) Prompts for initial vocal responses if needed Praise and positive attention Time delay

Promoting Communication: Arranging the Environment

Designing the environment to promote communication Having books to read in

quiet, comfortable area

Regular routines Displaying pictures at

child’s eye level Provide opportunities for

exploring environment

Following Child’s Lead

Noticing and talking about the child’s interest, activity, or materials

Using child’s interest to provide opportunities for communication Describe child’s actions Allow child to direct

activities Give choices

Adult: “ I’ll dance with you.”

Commenting and Labeling

Describing actions, toys, activities

Children hear talk about actions and hear labels Comment on what the

infant is playing with Talk about the pictures in

a book

Teacher: Mmm. Milk

Teacher: Keysha is swinging.

Imitating and Expanding

Imitating is repeating a child’s vocalizations or words Repeat sounds child

makes Expanding is adding

something new to what the child said Add to the child’s

statement by describing in slightly more complex way

Dog!

Adult: That’s a big brown dog.

Eat! – Eat!

Adult: Eat. We’re eating carrots.

Asking open-ended questions

Asking who, what, when, where, why questions that children can answer in multiple ways Ask questions related to

child’s play or interest Ask questions related to

routines Pause after asking

question, if no answer, fill in answer yourself

Ball! – Ball!

Adult: Where are you throwing the ball?

Adult: What are we doing? We’re

changing your diaper!

Time delay/ Fill in the blank

Planning a delay during a predictable routine is promotes communication Giving child the chance to

fill in the blank in a song or common phrase they are familiar with

Children practice talking Emphasizes child’s

success with words

Ashes, ashes

They all fall ___.

Down!

Positive attention and praise

Attending to and being positive about behavior and communication Positive attention for

communication and prosocial behaviors creates opportunities for children to practice skills

Playing with children gives positive attention

Adult: You’re such a great helper.

Paying attention to good behavior

Providing Choices

Providing choices means the child will need to communicate their preferences Encourages

communication Provide choices by

rotating materials At snack time present

two options Give child options in play

and book reading

Adult: Would you like blue or green?

Adult: Which book do you want to read?

Evaluating use of language promoting interventions

Complete self evaluation form to document use of intervention

Make self evaluation practical for parents and home visitors

What to do with information from self evaluation? Determine if intervention is being implemented as

intended If not, make modifications in intervention

Evaluating and monitoring intervention effectiveness

Continue administering ECI monthly or quarterly Look at Key Skill Elements and Total

Communication rates to check that they are moving toward benchmark

Look at the slope of communication progress to determine if the slope is increasing

Is there evidence that intervention has increased communication?

If progress is not at expected rate, examine intervention and make necessary changes

Child below benchmark after intervention

Key Skill Element: Single Words

Key Skill Elements: Multiple Words

Using ECI Data for Decision-Making about Programs

For groups of children, ECI data can tell us… How children in our program are performing relative to

benchmark on communication outcome. Whether our program is moving children toward

important outcomes. When to consider modifications in program. When we institute a change in the program, whether it

makes a difference in percentage of children who are on-target.

Which children are currently off-target?

Child Name Risk Status

Randall, Donnie Slightly Below

Tarantino, Quentin Slightly Below

Pitt, Brad Below Average

Leary, Timothy Below Average

Identify children who are not on-target

Are there more/less than the previous month?

What might account for this? What strategies are in place for off-target?

More frequent monitoring? Language promoting strategies?

Histograms provide a group snapshot of Total Communication

Examine the percentage of children who are on-target

Possible questions... How does this vary by age group? What might account for differences by age? What types of changes might you make as a

result of this information?

Looking at program data by age group:

Age Group

 

On TargetSlightly Below 

AverageBelow Average Total

1-11 Months11

100% 0% 0%11

100.0%

12-23 Months

1694.1%

15.9% 0%

17100.0%

24-35 Months

1990.5% 0%

29.5%

21100.0%

36-47 Months

777.8%

111.1%

111.1%

9100.0%

Overall53

91.4%2

3.4%3

5.2%58

100%

How is the entire group performing in key skill

elements?

Group Graph for Single Words

Group Graph for Multiple Words

How does data change from one month to the next?

Looking at program data month-by month

Age GroupPercentage of ON-TARGET  

March June Sept December

1-11 Months11

100%11

100%11

100%11

100.0%

12-23 Months

1694.1%

1795.9%

1795.9%

17100.0%

24-35 Months

1990.5%

2095.5%

2095.5%

21100.0%

36-47 Months

777.8%

777.8%

777.8%

9100.0%

Overall53

91.4%57

94.5%57

94.5%67

100%

Questions to ask if program is not meeting your expectations:

Are high quality language-promoting strategies being implemented?

Is the program being delivered at the expected levels of intensity, duration?

Are children/families participating at appropriate levels?

Are additional levels of supports, resources, training necessary?

How Approach is Used to Inform Decision-Making at the Program Level

Administrative and program staff may use ECI for : Program evaluation Program planning Quality improvement Inform public and funding sources about program

progress

Importance of Sensitive Measurement

When program staff have tools that more adequately reflect

change….

They will be more effective in guiding positive change in

programs.