A standardization and validity study of a speech and language screening tool A look at a Canadian...
-
Upload
anthony-gregory -
Category
Documents
-
view
217 -
download
2
Transcript of A standardization and validity study of a speech and language screening tool A look at a Canadian...
A standardization and validity study of a speech and language screening toolA look at a Canadian linguistic minority outside Québec
Michèle Minor-Corriveau, M.Sc.S. Speech-Language Pathologist
Associate ProfessorLaurentian University
Sudbury, Ontario Canada
Abstract : The purpose of this longitudinal and cross-sectional study is to standardize and validate the Speech and Language Profile, a speech and language screening tool developed by and for educational Speech-Language Pathologists. Reliability and validity measures obtained will draw upon data taken from administering the screening tool, which will be measured against each individual child’s academic performance through teacher report and standardized testing, including numeracy and literacy scores obtained by the Education Quality and Accountability Office’s (EQAO) provincial mandatory standardized testing. This study hopes to demonstrate predictive, concurrent and criterion-referenced validity of this tool, as well as interrater agreement and test-retest reliability. Favorable results could support the use of this tool by educational Speech-Language Pathologists to help them determine, at the earliest possible moment in the child’s academic trajectory, which services would be required in order to maximize his/her learning potential. A sufficiently high level of reliability and validity could also support the importance for standardizing this tool with other populations in minority and majority language settings at an international level. Statistical analysis will speak to the measures of internal consistency, content validity and face validity.
Context Methods
Data and results
Conclusions
References
2001 : SLPs from CSCNO school board design the Profile de la langue, du langage et de la parole (PLLP) a tool that would help them identify which children should receive community-based speech and language services upon entry into Junior Kindergarten (from 46 to 58 months)
2007 : PhD research aimed at validating and standardizing the PLLP begins
2008 : an internet poll aimed at French-speaking S-LPs confirms : no gold standard is used to identify speech or language delays in francophone children attending French-language schools
Of 74 respondants (49% response rate) :• 39% = no protocol in place for a universal screening in JK in their school boards• 61% = a protocol is in place for screening in JK their school boards
IS THERE A UNIVERSAL SYSTEMATIC PROTOCOL IN PLACE USED TO HELP DETECT SPEECH AND LANGUAGE DELAY IN YOUR SCHOOL BOARD ? IF SO, WHAT ARE S-LPs USING ?
• Results from ongoing validity and reliability measures will be published as part of the author’s doctoral thesis.• Development of an iPad application currently in the works to enable S-LPs to administer the PLLP electronically and enter data online.
Normative sample and Item analysisChi-square analysis (p<0,05) indicate that on the whole, test items were consistent with existing theory.
in the results for 2004, 2005, 2009 and 2010 ≠ significant difference linked to sex
linked to language (French ; English). Test-retest reliability results show universal agreement among 19/48 items ≠ significant difference on 25/48 items significant difference for 4/48 items (English /l/, s, cup, why).
Interrater reliability results show high interrater agreement, ranging from 0.90 to 0.98.
Criterion-referenced validityAlongside the PLLP, the following tests were administered. Preliminary results yield high correlation between the PLLP vocabulary subtest and the Leiter-R:
• Carrow-Woolfolk (1999)• Concepts and directions (CELF-CDN-FR)• Échelle de vocabulaire en images – Peabody (ÉVIP)• Edmonton Narrative Norms Instrument (ENNI)• Leiter International Performance Scale• Non-word repetition test (Courcy, 2000)• Number repetition (CELF-CDN-FR)• Rapid automatic naming• Sentence repetition (CELF-P)
Predictive validity (preliminary results)The results of 2251 PLLP scores were measured against their report card grades and IEPs from grade 1 to grade 6 as well as on EQAO provincial testing. Preliminary predictive validity results indicate a weak to moderate correlation, depending on the area. Correlation was also weak when measuring academic achievement against EQAO provincial testing, even within the same school year. Correlation of PLLP scores and academic achievement is comparable to that between academic achievement and EQAO provincial testing.
Internal consistency will be measured using the split-half reliability method and Cronbach’s Alpha. Data analysis is ongoing. Preliminary results yield promising findings.
There is sufficient evidence to suggest that the PLLP should be used by S-LPs to help identify which JK students would benefit from further speech and
language assessment.
• Validity measures demonstrate the PLLP’s stability across time (test-retest reliability), and across examiners (interrater agreement).• Preliminary results indicate a high correlation between PLLP and some subtests belonging to the Leiter-R, weak to moderate correlation between the PLLP test results and academic achievement. (grades obtained on report cards in grades 1 to 4, and IEPs up to grade 6). By the same token, data obtained for the same sample indicate a weak correlation between academic achievement in grades 1 to 4 and EQAO provincial test results. Correlation to IEP is pending.
This is an observational and cross-sectional study aimed at demonstrating reliability and validity of the PLLP .
OBSERVATIONAL STUDY
Normative sample (n=2251) : Data obtained through administration of the PLLP in 2004, 2005, 2008, 2009 and 2010. Analysis of data will determine if there is a difference in results obtained during these years, and whether this difference can be attributed to sex or language of administration (French or English).
Test-retest reliability (n=24) is used to measure the correlation between thescores obtained by the same persons on 2 administrations of the test. The PLLP was administered once in October 2008 and again in January 2009. The goal is to establish test stability over time.
Interrater reliability (n=75) refers to the degree to which the same individual would receive the same score, even if the test was administered by different examiners. The PLLP was administered as part of the interrater reliability study. All sessions were videotaped. 5 SLPs who were blind to the original test results rated the children’s test scores using the same test form as usedby the original examiner, while watching the videotaped session. The goal is to establish the transparency of the test.
Criterion-related validity (n=26) refers to the effectiveness with which thetest is able to correlate to an individual’s performance in specified activities or on other tests administered at the same time or within a short interval. The PLLP was administered alongside other assessment batteries normed inQuébec. This data collection is part of another doctoral study (see Mayer-Crittenden, PhD candidate, Human Studies, Laurentian University).
CROSS-SECTIONAL STUDY
Predictive validity refers to the effectiveness with which a test is able to predict future performance in specified activities or on like tests administeredafter a longer interval : Test scores obtained on the PLLP will be measured against the students’ academic achievements (report card results and IEPs) ingrades 1 – 6, as well as provincial test results (EQAO) in reading, writing andmathematics.
TEST ANALYSIS
Item analysis will measure success and failure rates of each item to determine degree of difficulty of items and whether items should be added, substituted or deleted. Internal consistency will measure whether test items belonging to the same subset of scores are correlated and whether they belong to the same construct.Face validity will determine whether the test items are appropriate and to what degree they reliably represent the targeted items. Content validity will link the theory in support of the test items used, those added, substituted or deleted.
• American Educational Research Association, American Psychological Association & National Council on Measurement in Education. (1999). Standards for Educational and Psychological Testing. USA : American Educational Research Association.
• Anastasi A., & Urbina, S. (I1997). Psychological testing (7th ed). New Jersey : Prentice Hall.• Brown, L., & Bryant, b. (1984). The why and how of special norms. Remedial and Special Education, 5(4), 52-61.
11% - observational checklist14% - parent questionnaire4% - non-word repetition test4% - Fluharty Preschool Screening Test*19% OEW (French equivalent of WBTT)7% PLLP4% TPLS*4% TSRI** unstandardized, unadapted measures
NO33% YES
67%
Wh-Q
uestions
Spati
cal C
oncepts
Vocabular
y
2-Step Dire
ctions
Body Part
s
Articu
lation
8486889092949698
100
% in
terr
ater
agr
eem
ent
Future directions
INTERRATER RELIABILITY
This initiative has been made possible thanks to funding fromLaurentian University/Université Laurentienne and Health Canada (CNFS).
X5
Oct 2008 Jan 2009
IEPA+
EQAO
To S-LPs and School Boards • Lists compiled help prioritize
service delivery and help manage wait lists efficiently.
BENEFITSTo Parents
• Goals are targeted early onto help remediate minor speech
or language issues in theshort term.
To Children• All children entering JK at CSCNO between 2001 and
2010 have been seen and/or flaged as requiring follow-up.
• PLLP Available in French and English• Average administration time = 12 minutes
• Excellent platform for S-LPs to identify students in need of service or follow-up
• See Direct Benefits (below)
Carrow-Woolfolk
Baccalauréat (B.Sc.S.) et maîtrise (M.Sc.S.) ès sciences de la santé (orthophonie)
Michèle Minor-Corriveau, M.Sc.S.Speech-Language PathologistAssociate ProfessorLaurentian UniversitySudbury, OntarioCanada
• PLLP data entered online• Accompanying report provides
recommendations based on results (cut-off scores)
ATTRIBUTES