Challenges of Assessing Young Areas of...
-
Upload
vuongkhanh -
Category
Documents
-
view
218 -
download
2
Transcript of Challenges of Assessing Young Areas of...
1
Assessing Communication and Learning Skills in
Young Children Who are Deaf blind
Charity Rowland, Ph.D.
[email protected] Webinar, NCDB: September 24, 2010
Validation of Evidence-based Assessment Strategies to Promote Achievement
in Children who are DeafblindU. S. Department of Education Grant # H324D030001
� Dr. Deborah Chen California State University, Northridge
� Dr. Robert Stillman, University of Texas at Dallas
� Dr. Harvey Mar, Columbia University
� Dr. Charity Rowland, Oregon Health & Science University
� National Family Association for Deafblind
2
Areas of Focus
� Deafblind
� 2-8 year old children who are nonspeaking and have significant intellectual limitations
� Communication and cognitive development
3
Challenges of Assessing Young Children Who Are Deafblind
� May not use conventional forms of communication
� May use very subtle means of expression
� Professionals often inexperienced
� Appropriate assessment instruments may be hard to find
4
Activities/Studies
� Nationwide survey of parents
� Nationwide survey of professionals
� Expert query
� Family specialist query
� Poll of state projects
� Focus groups
� Collection of assessment and IEP data on 0-5 year olds
5
Parent Survey (n – 74)
6
C. Rowland, OHSU Assessing Communication and Learning Skills
Webinar, September 24, 2010
2
Parents’ Mean Ratings on Questions about the Assessment Process
7
Survey Question Mean Ratinga
1. How useful were the results of the latest evaluation report to you in terms
of identifying the specific needs of your child?
3
2. How easy was it for you to understand the results of the latest evaluation
report? That is, were the results presented in terms that you could
understand?
4
3. Do the results of the latest evaluation report match your own
observations about your child? That is, do they reveal the strengths and
weaknesses that you see in your child?
3
4. How useful were the results of the latest evaluation report in terms of
helping you participate in developing educational goals for your child?
3
5. How useful was the latest evaluation report in terms of describing your
child’s progress over time?
3
6. To what extent were you involved in the assessment of your child for the
latest evaluation?
3
Least satisfactory aspects of my child’s latest evaluation
� 40% - Tests inappropriate, underestimated child
� 20% - Evaluation didn’t provide helpful facts or suggest things to try
� 15% - Lack of progress of child
� 13% - Tests focused on what child can’t do,
rather than progress
� 10% - Evaluator didn’t know child
� 5% - Didn’t like how child was handled
8
Parents’ most frequent suggestions
� 40% - Evaluators should get to know child
� 35% - Get input from parents and classroom
� 28% - Evaluations should help parents understand how to set goals and move forward
� 20% - Use tests appropriate for child who is deafblind
� 15% - Tell parents names of assessments, what for, what doing during assessment
� 10% - Start with child’s strengths and reasonable goals
� 5% - Child’s needs should be paramount—not costs or difficulty of trying new things
9
Qualitative Data Sources
In-depth Interviews (14)
Focus Group Interview (10)
Survey of Family Specialists (12)
In-depth Interviews: Themes
N=14
� Accuracy of assessment data
� Applicability to intervention
� Modification of standardized tests
� Authentic assessment practices
Focused Interview: Themes
N = 10
� Appropriate assessment tools
� Expertise of assessment team
� Time to conduct assessment
C. Rowland, OHSU Assessing Communication and Learning Skills
Webinar, September 24, 2010
3
Focused Interview: Findings
Family concerns
� Use of test scores
� Knowledge of child
� Parent view of professionals
Professional concerns
� Assessment tools
� Relationship to intervention
� Assessment report
Focused Interview: Common Questions of Parents
� Purpose of the assessment and procedures
� Appropriateness of assessment tools;
� Qualifications of the assessment team;
� Outcomes of the assessment process;
� Accuracy of assessment results; and
� What parents can do if they disagree with assessment reports.
14
Focused Interview: Helpful Strategies
� Multiple and skilled observations
� Natural environments
� Familiar people
� Interview skills
� Team approach
� Family-centered approach
Survey of Family Specialists
Parent questions
� Purpose of assessment and procedures
� Appropriateness of tools
� Qualifications of team
� Outcomes
� Accuracy
� Disagreements
Survey of Family Specialists
Recommendations for parents
� Make list of questions
� Ask about tools and procedures
� Prepare list of own observations
� Draw on relevant professionals and resources
Specific Assessments:Criteria for “Short List”
� High ratings by 135 professionals who completed survey
� Designed for assessment of 2-8 year olds
� Include sections on cognitive and/or social-communication skills
� Readily available
� Other instruments developed specifically for population also included
18
C. Rowland, OHSU Assessing Communication and Learning Skills
Webinar, September 24, 2010
4
“Short List” of Assessments
� Callier-Azusa (Stillman) *
� Carolina Developmental Profile (Johnson-Martin, Attermeir & Hacker)
� HELP-Hawaii Early Learning Profile (Parks & Furuno)
� Infused Skills Assessment (Hagood) *
� INSITE (Morgan & Watkins) *
19
“Short List” of Assessments
� Oregon Project (Anderson, Boigon, Davis & deWaard)
� Vineland (Sparrow, Balla & Cicchetti)
� Communication Matrix (Rowland)
� Dimensions of Communication (Mar & Sall) *
� Home Talk (various authors) *
� School Inventory of Problem Solving Skills (Rowland & Schweigert)*
20
21
InstrumentDeveloped
for
deafblind?
Domains
Assessed
Age range
targeted
Focus on pre-symbolic
skills
Callier-Azusa-G Yes All domains 0-10 yr. High
Callier-Azusa-H Yes Communication 0-10 yr. High
CarolinaNo All domains 0-3 yrs (Infant &
Toddler edition)
2-5 yrs (Preschooler
edition)
Low
Communication Matrix
Yes Communication All ages: only earliest stages of
communication
High
Dimensions of Communication
Yes Communication All ages High
HELP No All domains 0-3 yr. (0-3
edition)
3-6 yr. (Preschooler ed.)
Low
22
InstrumentDeveloped
for
deafblind?
Domains
Assessed
Age range
targeted
Focus on pre-symbolic
skills
Home Talk Yes CommunicationLearning/concept
development
School-age (3+)
High
*Infused Skills Blind Social competenceOrganization
Early childhood-
secondary
Low
INSITE Yes All domains 0-2 yr. (short version)
0-6 yr. (long version)
High
*Oregon Project
Blind All domains 0-6 yr. Low
SIPSS/.HIPSS Yes Object interaction related to
cognitive and social domains
All ages High
*Vineland No Adaptive Behavior (Communication,
Daily Living Skills, Motor Skills)
All ages Low
Parent Ratings of Selected Instruments
23
Instrument
Callier-
Azusa
Communication
Matrix HELP HIPSS Home Talk
Home Talk
Section 3 only Vineland
This instrument uses clear language
4.3 4.0 4.3 4.6 3.3 4.0
This instrument is easy for me to
understand
3.9 4.0 4.1 4.5 4.0 4.0
This instrument is useful for evaluating my
child's needs 2.5 4.1 3.5 4.0 4.8 3.0 2.9
This instrument encouraged me to be
involved in my child's assessment process. 4.3 3.0 4.5 4.4 4.0 3.0
It is easy for me to understand the results
that this instrument provides. 4.0 3.4 2.5 3.4 4.1 3.7 3.4
This instrument provides results that match
my own observations of my child. 4.0 3.7 3.0 3.7 4.1 3.3 3.1
This instrument is useful for developing
educational goals for my child. 2.5 4.1 3.5 4.3 4.4 3.7 2.9
This instrument is useful for describing my
child's progress 2.5 4.1 3.5 3.9 4.0 3.0 2.8
This instrument has clear instructions (only
for instruments that you completed
yourself) 3.3 3.0 4.2 4.3 3.7
This instrument is easy for me to use (only
for instruments that you completed
yourself)
3.3 3.0 4.6 4.4 3.7
24
5 Composite Qualities RatedQ ua lity Q ue stion s
Con tent
U se f ul to a ss e s s com m unica tio n
U se f ul to a ss e s s so cia l in te ra ction
U se f ul to a ss e s s cog nit ive / lea rning
A ppr opr iaten e ss
U se f ul for 2-8 y ea r o ld s
U se f ul for ch ildre n w ith se ve re cog n it iv e im p airm e n t
U se f ul for ch ildre n w ithout lan gua ge
U se f ul for ch ildre n w ith se ve re vis ion im p airm en t
U se f ul for ch ildre n w ith se ve re h e a rin g im pa irm e n t
U se f ul for ch ild re n w ith se ve re ortho p e dic im p airm e n t
A cc ura cy
U se f ul to a ss e s s a v arie ty of ch ildre n with broa d ra n ge o f ski lls/ ne e ds
U se f ul to e va lu a te w id e ra nge of sk il ls in ea ch dom ain
U se f ul for ge n er ating a ccu ra te p ic tur e o f ch ild s ki lls
A pplica bility
U se f ul to g e ne rate instr uctio na l go als /for e du ca tion al p lann in g
U se f ul for e va luating p rogr es s
U se f ul for de s crib in g chi ld 's s tre ng th s a n d we a kne s s to pa re nts
U se f ul to e ncou rag e p are n t invo lv e m e nt
Usa bility
U se s cle a r la ng uag e
H a s c le a r ins truction s
Is e a sy for m e to und er sta nd
Is e a sy for m e to us e
Is u se r- fr ie nd ly
C. Rowland, OHSU Assessing Communication and Learning Skills
Webinar, September 24, 2010
5
Professional Ratings of Selected Instruments
25
Instrument
Callier
-Azusa
Caro-
lina
Communi-
cation
Matrix
Dimensions
of Communi-
cation HELP
Infused
Skills
Home
Talk INSITE
Oregon
Project SIPSS VIneland
n 29 16 23 8 25 7 10 16 26 9 11
Content 1: Useful
to assess
communication
4.0 4.2 4.8 4.1 3.2 5.0 4.2 4.4 3.6 NA 3.2
Content 2: Useful to
assess social
interaction
4.0 3.9 4.4 4.0 3.4 4.8 4.1 4.4 3.7 NA 3.5
Content 3: Useful to
assess cognitive,
learning skills
3.9 4.3 NA NA 3.6 4.0 4.1 4.4 3.9 4.0 NA
MEAN
Appropriateness
4.0 3.1 4.4 3.8 2.7 4.9 4.3 4.2 3.4 3.8 2.8
MEAN
Accuracy
3.4 2.9 3.9 3.6 3.2 4.6 4.4 4.1 3.7 3.8 2.9
MEAN
Applicability
4.1 3.6 4.4 4.0 3.2 4.7 4.3 4.4 4.0 4.0 3.4
MEAN
Usability
4.3 4.5 4.5 3.5 4.3 3.0 4.5 4.5 4.3 4.7 4.3
Results of State Projects Query (n=18)
26
Instrument
Am familiar with
instrument
Is used widely in
my state
I would
recommend its use
Callier-Azusa 100% 41% 94%
Carolina Developmental
Profile 62% 15%
63%
Hawaii Early Learning
Profile 67% 33%
100%
Infused Skills
Assessment 14% 7%
33%
INSITE Developmental
Checklist 75% 50%
92%
Oregon Project 87% 33% 71%
Vineland 75% 33% 60%
Communication Matrix 83% 33% 83%
Dimensions of
Communication 60% 0% 100%
Home Talk 90% 20% 100%
School Inventory of
Problem Solving Skills 56% 6% 91%
Qualitative Findings from AllInterviews and Surveys
Challenges:
� Appropriate assessment tools
� Expertise of assessment team
� Time to conduct assessment
� Accuracy of assessment report
� Relationship to program planning
27
Qualitative Findings from AllInterviews and Surveys
Recommendations:
� Family involvement
� Multiple and skilled observations
� Natural situations
� Familiarity with child
� Interview skills
� Interdisciplinary approach
28
Final Products
� Guidelines for the
Assessment
of Young Children
who are Deafblind
� Data Summary and Assessment Forms available at:
http://www.ohsu.edu/oidd/d2l/com_pro/db_assess_ab.cfm
29
Evidence Based Practices (EBP)
Combination of:
� Best available research
� Craft knowledge and community of practice
� Family wisdom and experience
C. Rowland, OHSU Assessing Communication and Learning Skills
Webinar, September 24, 2010
6
Authentic Assessment
A multidimensional, team-based approach which seeks to:
� Capture the richness and complexity of a child’s cognition and behavior
� Understand the child in the context of his/her natural social and physical environment
� Integrate the family’s perspective with those of professionals
� Apply the information to enhance the child’s development and acquisition of skills.
Steps of an Authentic Assessment Process
� Identify assessment team
� Arrange for family involvement
� Select appropriate instruments
� Conduct observations in multiple situations
� Arrange structured observations
� Request evaluations by specialists
� Emphasize strengths and interests
� Provide recommendations
The Planning Phase
� Assembling the Team
� Getting the Family Involved
� Selecting Appropriate Assessment Instruments
� Scheduling Observations in Natural and Staged Contexts
� Involving Specialists
33 34
Assembling the Team
� Define the questions and the scope of the assessment
� Multidisciplinary
� Include individuals with deepest knowledge of child
35
Getting the Family Involved
What the assessment team can do:
� Meet with the family in advance
� Address their questions and concerns
� Identify any obstacles to participation
36
C. Rowland, OHSU Assessing Communication and Learning Skills
Webinar, September 24, 2010
7
How Families May Prepare for an Assessment
� Prepare your own questions and identify your goals and concerns in advance.
� Prepare a list of your child’s special skills and interests.
� Discuss what instruments will be used to assess your child and who will administer them.
� Provide the evaluator with possible times to observe your child at home or in other familiar settings.
� Prior to the assessment, complete any parent questionnaires or assessments so that the evaluator can better understand your child’s abilities.
37
Selecting Appropriate Assessment Instruments
� Helps structure observations:
� What to look for
� How to interpret
� Compare across observers
� Accuracy, Applicability, Convenience
� May use parts of several instruments
� See reviews in our assessment guide
� What about standardized tests? 38
Considerations for Selecting an Assessment Instrument
� Do the items describe behaviors a child with vision and hearing losses or motor impairments could be expected to show?
� Are there enough items at the early developmental levels to clearly identify a child’s current skills and measure progress in small steps?
� Are the items appropriate to the child’s chronological age or do they describe the behavior one would only expect to see in infants and toddlers?
� Are there a sufficient number of examples to clarify the items and to determine how a particular skill might be observed in a child who is deafblind?
39
Considerations for Selecting an Assessment Instrument
� Does the assessment include information derived from observations in natural settings?
� Do the results provide applicable information for program planning or are they primarily numerical scores? Does the instrument give you ideas about the “next step” for the child?
� Are the results in a format that can be easily communicated to and understood by families?
� Who can use the instrument?. Does it require specialized training or professional credentials?
� Is there a parent version or a way for parents to provide their input and perspectives?
40
Observing in Natural Contexts and Staged Situations
Natural
� Gather information about the child in everyday contexts and routines
� Identify typical and optimal skills
� Multiple perspectives
Staged
� Manipulate the natural context to encourage specific behaviors
� Fill in gaps in knowledge and validate impressions
41
Involving Specialists
� Vision specialists
� Hearing specialists
� Physical therapist
� Occupational therapist
� Speech-language pathologist
� Health professionals
42
C. Rowland, OHSU Assessing Communication and Learning Skills
Webinar, September 24, 2010
8
What Questions Should an Authentic Assessment Address?
43 44
45
Expressive Communication
What existing communication behaviors should be promoted and what new skills should be targeted?
46
47
Expressive Communication
What behaviors does this child use to make his/ her needs and wants known (body movements, gestures, facial expressions, vocalizations, words, sign language, picture
symbols, object symbols, etc.)?
Does this child’s expressive behavior appear to be intentional? Is it directed toward a goal? Does it appear that the child anticipates a response to the communication?
How frequently does this child communicate?
What specific messages or communicative functions does this child express (protests, requests, greetings, etc.)?
Under what circumstances is this child most communicative (with whom? when? where?)?
Does the child need prompting or support to communicate clearly or consistently? What type of support?
Receptive Communication
What forms of communication does the child understand? How can we best communicate with this child?
48
C. Rowland, OHSU Assessing Communication and Learning Skills
Webinar, September 24, 2010
9
49
Receptive Communication
What types of communicative behavior do you think this child understands (spoken words, manual signs, gestures, facial expressions, vocal intonation, picture symbols, object symbols, etc.)?
What messages or communicative functions does this child appear to understand(directives, greetings, requests, etc.)? Is prompting and support needed for the childto respond to a communication?
Who communicates effectively with this child? Are there particular activities in whichthe child seems most likely to respond?
Social Interaction
What opportunities exist to enhance the child’s social interactions, play, and communication with others?
50
51
Social Interaction
Does this child enjoy interacting with adults? If yes, under what
circumstances?
Does this child enjoy interacting with peers? If yes, under what
circumstances?
Hearing/Vision
What adaptations and accommodations are needed to support the child’s use of hearing and vision for communication?
52
53
Hearing
Is this child able to perceive speech, vocalizations or other auditory input?
Would this child benefit from hearing aids or amplification devices (such as an FM system)?
Vision
Is this child able to use visually-based symbols for expressive communication (can s/he perceive, discriminate between, and understand visually-based symbols)? If
not, is a tactile system indicated (e.g., object symbols, Braille)?
Is this child able to perceive visually-based symbols for receptive communication (e.g., picture symbols, sign language, print)? If not, is tactile input required (e.g.,
tactile signs, object symbols)?
What accommodations are needed to help this child perceive visually-based communication (positioning of child and partner, illumination, position and spacing
of stimuli or symbols, size and color of symbols, etc.)?
Motor/Fine Motor Skills
What adaptations are needed to support the motor behaviors and tactual experiences needed for communication?
54
C. Rowland, OHSU Assessing Communication and Learning Skills
Webinar, September 24, 2010
10
55
Motor/Fine Motor Skills
Does this child have the fine motor skills needed to use gestures or manual
signs, to indicate choices or to activate communication devices?
What accommodations need to be made to allow physical access to
expressive and receptive communication systems?
Preferences
What activities, materials, people, and contexts motivate the child to communicate and learn?
56
57
Preferences
What people, things, and activities does this child prefer?
What people, things, and activities does this child dislike?
Is this child mostly engaged by objects or by people?
Where and when is this child at his/her best?
To what extent does this child tolerate direct assistance, such as hand-
over-hand or hand-under-hand support?
What is this child’s primary mode of exploring new things (tactile, visual,
oral)?
Temperament
How does the child’s temperament affect learning and communication?
How can we use this information to develop effective interventions?
58
59
Temperament
Is this child easily excited or passive and quiet?
Does this child show sustained attention to tasks or quickly lose interest?
Is this child calm or irritable?
Is this child sociable or shy?
Does this child accept or reject changes in routines, materials, people,
and situations?
60
C. Rowland, OHSU Assessing Communication and Learning Skills
Webinar, September 24, 2010
11
61
Generic Expressive Communication Outcomes
� Communicate more frequently using existing communication behaviors
� Communicate more independently (with less assistance)
� Use new (more conventional or sophisticated) forms of communication
� Express new messages/communicative functions (e.g., request “more”, ask questions)
� Communicate about a broader range of topics
� Communicate with more people and across settings and activities
� Initiate communication62
Generic Receptive Communication Outcomes
� Respond to communication from others with less cueing/assistance
� Understand new (more conventional or sophisticated) forms of communication
� Understand new messages/communicative functions (e.g., requests for information, greetings)
63
Generic Social Interaction Outcomes
� Interact comfortably with adults
� Interact comfortably with peers
� Initiate social interactions
64
Sample Expressive Communication Outcomes & Goals
� Outcome 1: Communicate more frequently and independently using
current behaviors.
� Goal 1: Maria will increase independent use of gestures.
� Suggestions: Provide frequent opportunities to make choices in snack, circle, play; promote the use of gestures to make choices/requests; fade
physical assistance; record frequency of independent responses.
� Outcome 2: Use new symbolic forms of communication.
� Goal 2: Maria will use object symbols to make simple choices.
� Suggestions: Use familiar objects associated with preferred items and
activities. Probe ability to choose correct one (out of two) associated with preferred snacks, toys.
65
Sample Receptive Communication Outcomes & Goals
� Outcome 1: Increase responsiveness to current communication forms.
� Goal 1: The consistency of Maria’s awareness of and reactions to
auditory input will increase.
� Suggestions: Provide FM system; track responsiveness to spoken cues.
� Outcome 2: Understand new forms of communication.
� Goal 2: Maria will demonstrate understanding of object symbols for major activities.
� Suggestions: Provide daily schedule system with one familiar object to
represent each major activity in school schedule.
66
C. Rowland, OHSU Assessing Communication and Learning Skills
Webinar, September 24, 2010
12
Sample Social Interaction Outcomes & Goals
� Outcome 1: Increase ability to interact comfortably with peers.
� Goal 1: Maria will have more opportunities to interact with peers during regular classroom routines.
� Suggestions: Include Maria in all group activities at school; provide consistent structure for simple interactions with peers; sit Maria next to favorite peers; where appropriate, involve peers instead of (or in addition to) one-to-one assistant.
67
Qualitative Indicators of Progress
� Increased consistency of response
� Increased independence of response/decreased assistance required
� Increased frequency of response
� Increased duration of response
� Increased clarity of response
68
Qualitative Indicators of Progress, contd.
� Faster response time
� Partial participation in one step of a routine
� Ability to participate in an activity for longer periods of time
� Sustained attention to a task
� Increased stamina
� Improved strength or accuracy of a motor response
69 70
C. Rowland, OHSU Assessing Communication and Learning Skills
Webinar, September 24, 2010