KMcGrew Austin2 WJIV handouts...

6/3/2014

1

The New WJ IV Battery: Introduction and Overview

Dr. Kevin McGrewInstitute for Applied Psychometrics (IAP)

WJ IV Presentation Menu: Main

Introduction and Overview

New Tests (COG/OL)

Cognitive Battery (GIA & CHC clusters)

Oral Language Battery

Achievement Battery

Variations and Comparisons

General Technical Information

Tech info.

Scholastic Aptitude clustersNew Ga

Tech info.

Tech info.

WJ IV Presentation Menu: Sub

Design principle: Cognitive complexity

Ga: No longer the Rodney Dangerfield of CHC abilities

Tech: Norm sample

Tech: Norms and score construction

Beyond CHC: CHC + Info. Proc. + AC

Hot off the press: Analyses in process

6/3/2014

2

Contemporary CHC broad and narrow ability content coverage by WJ‐IV Cognitive, Oral Language, and

Achievement batteries.General

Intelligence (g)

Mathematical achievement

(A3)

Reading decoding (RD)

Reading comprehension

(RC)

Reading speed (RS)

Spelling ability (SG)

English usage (EU)

Writing ability (WA)

Writing speed (WS)

General verbal information (K0)

Lexical knowledge (VL)

Listening ability (LS)

Induction (I)

General sequential

reasoning (RG)

Quantitative reasoning (RQ)

Memory span (MS)

Working memory capacity (WM)

Associative memory (MA)

Meaningful memory (MM)

Speed of Lexical access (LA)

Naming facility (NA)

Visualization

(Vz)

Visual Memory (MV)

Spatial Scanning

(SS)

Phonetic coding (PC)

Quantitative Knowledge

(Gq)

Reading & Writing (Grw)

Comp ‐Knowledge

(Gc)

Fluid Reasoning (Gf)

Short‐Term Wk Mem. (Gsm)

Long‐Term Retrieval (Glr)

Visual Processing (Gv)

Auditory Processing

(Ga)

Processing

Speed (Gs)

Perceptual speed (P)

Number Facility (N)

General science info. (K1)

Knowledge of culture (K2)

Domain‐Specific Knw.

(Gkn)

Geography ach. (A5)

Shading designates proposed changes in CHC model based on analysis of WJ IV COG, OL. ACH norm data (see Chapter 1 and Appendix A in WJ IV Technical

Manual)

Language development

(LD)

Attentional Control (AC)

Memory for Sound Patterns

(UM)

Word Fluency (FW)

Verbal (print) lang. comp. (V)

Gsm has been renamed Gwm at the round table of cognitive CHC abilities

During the past two decades, and the last decade in particular, cognitive neuroscience has indicated that the more narrow Gsmdefinition was outdated and incorrect (Dehn, 2008).

Working memory refers to a dynamic, temporary storage system that allows information to be held immediate awareness and be manipulated.

Working memory refers to individual differences in both the capacity(size) of primary memory and to the efficiency of attentional control mechanisms that manipulate information within primary memory.

Short‐term memory refers to tasks that involve significant storage but only minimal processing or manipulation.

Attentional Control (Gwm‐AC). The ability to focus on task‐relevant stimuli and ignore task‐irrelevant stimuli. The ability to regulate intentionality and direct cognitive processing. Sometimes referred to as spotlight or focal attention, focus, control of attention, executive controlled attention or executive attention.

Memory for Sound Patterns (Ga‐UM). Ability to retain (on a short‐term basis) auditory codes such as tones, tonal patterns, or speech sounds.

Word Fluency (Glr‐FW). Ability to rapidly produce words that share a phonological (e.g., fluency of retrieval of words via a phonological cue) or semantic feature (e.g., fluency of retrieval of words via a meaning‐based representation). Also includes the ability to rapidly produce words that share non‐semantic features (e.g., fluency of retrieval of words starting with the letter “T”).

Speed of Lexical Access (Glr‐LA). Ability to rapidly and fluently retrieve words from an individual's lexicon; verbal efficiency or automaticity of lexical access.

Proposed changes/additions to CHC narrow ability taxonomy

6/3/2014

3

OM

PC US UM U8 URU1U9

UP UL

Vz SR MV CS SS CF IM PI LE IL PN

Sensory‐M

otor Domain‐

Specific Abilities

Sensory

The CHC Periodic Table of Human

Abilities

Adapted from Schneider & McGrew (2012) and McGrew, LaForte and Schrank (2014)

I RG RQ

WM MS AC

R3 PT MT

P N R9

R1 R2 R4 R7 IT

MA MM M6 FI FA FE SP F0 NA FW LA FF FX

Ideas Words Figures

Domain‐Independent

Cap

acities

Glr‐Learning efficiency

Glr‐Retrieval fluency

Broad ability

Narrow ability

© Institute for Applied Psychometrics (IAP)Dr. Kevin McGrew 4‐17‐14

KM A3

LD VL K0 LS CM MY

KL K1 A5 MK KF LP BC

V RD RC RS WA SG EU WS

Acquired Knowledge

Systems

K2

PI P2 P3 P4 P6 P7 P8

U1U9

UP UL

Motor

A1

KM A3

LD VL K0 LS

K1 A5

V RD RC RS WA SG EU WS

PC UM

Vz MV SS

Acquired

Knowledge

System

sSensory‐M

otor Domain‐

Specific Abilities

I RG RQ

WM MS AC

P N

MA MM NA FW LA

Ideas Words Figures

Domain‐Indep

enden

t Cap

acities

K2

Broad and Narrow Abilities Measured by the WJ IV COG, OL and ACH

FI

© Institute for Applied Psychometrics (IAP) Dr. Kevin McGrew 4‐11‐14

6/3/2014

4

Gc Gwm Gs Ga Glr GvGf

Vocab.(2)

(These two clusters require one test each from Cognitive and Oral Language easels)

Aud. Mm. Sp.(2)

Organization of WJ IV Cognitive (COG) and Oral Language (OL) batteries

General Intellectual Ability – g (7)

Brief Intellectual Ability (3)

Gf+Gc Composite (4)

Clusters available from Standard Cognitive

easel(10 tests)

(#) = # tests

Clusters available from Oral Language

easel(12 tests)

(#) = # tests

ST Wk. Mem.(2)

Cmp. Know.(2)

Fld. Reas.(2)

Cognitive Efficiency (2)

Other clusters available by

combining Standard Cognitive tests with select tests from

Extended Cognitive easel (8 tests)

(#) = # tests

Cog. Pr. Spd.(2)

LT Retrieval(2)

Visual. Proc.(2)

Aud. Proc.(2)

ST Wk. Mem.(3)

Cmp. Know.(3)

Fld. Reas.(3)

Qnt. Reas.(2)

Perc. Spd.(2)

Number Facility (2)

Cognitive Efficiency (4)

Reading, Math and Writing Scholastic Aptitudes (each comprised of a mix of 4 different CHC abilities)

Broad Oral Lang (3)*

Oral Expression (2)

Oral Lang. (2)*

Listening Comp. (2)* (* English or Spanish)

Phon. Cod.(2)

Spd. Lx. Acc.(2)

6/3/2014

5

WJ IV COG and OL tests by CHC factor domains

Oral Vocabulary

(VL)

Number Series

(RQ)

General Information

(K0)

Object‐Num.

Sequencing

(WM)

Pair Cancellation

(P/WM‐AC)

Memory for Words

(MS)

Analysis‐Synthesis

(RG)

Concept Formation

(I)

Numbers Reversed

(WM)

Picture Recognition

(MV)

Visual‐Aud.

Learning

(MA)

Story Recall

(MM)

Number‐Pat.

Matching

(P)

COG

OL

New or sign. change

Verbal Attention

(WM/AC)

Letter‐Pat. Matching

(P)

Nonword Repetition

(PC/UM‐MS)

Phonological Processing

(PC/Glr‐LA)

Visualization

(Vz)

Picture Vocabulary

(VL/LD)

Oral Comp.

(LS)

Under.

Directions

(WM/Gc‐LS)

Sentence Repetition

(MS)

Retrieval Fluency

(FI/LA)

Rapid Picture Naming

(NA/LA)

Sound Awareness

(PC)

Sound Blending

(PC)

Segmentation

(PC)

CHC Factors

Comp ‐Knowledge

(Gc)




Auditory Processing

(Ga)

Processing

Speed (Gs)

Short‐Term

Wrk Mem(Gwm)

WJ IV ACH tests by CHC factor domains

Grw ‐Reading

CHC Factors

New or sign. change

Grw ‐Writing

Gq Gc/Gkm

Letter‐Word

Identification

(RD)

Passage Comp.

(RC)

Editing

(EU)

Social Studies

(K0/K2)

Sentence Wr. Fluency

(WA/WS)

Reading Vocabulary

(RC/Gc‐VL)

Writing Samples

(WA)

Calculation

(A3)

Humanities

(K0/K2)

Spelling

(SG)

Science

(K0/K1)

Applied Problems

(A3/RQ)

Word Attack

(RD/Ga‐PC)

Sentence Rdg. Fluency

(RC/Gc‐RS)

Spelling of Sounds

(SG/Ga‐PC)

Math Facts Fluency

(A3/Gs‐N)

Reading Recall

(RC/Glr‐MM)

Word Rdg.

Fluency

(RC/Gs‐RS)

Oral Reading

(RD/V)

Number Matrices

(Gf‐RQ)

Grw‐W

Grw‐R Gq Gc/Gkn

Phon.‐Graph. Know. (2)

Rdg. Cmp.(2)

Other clusters available by combining Standard ACH tests with select tests from Extended ACH easel (9

tests)

(#) = # tests

Rdg. Cmp.(3)

Rdg. Rate(2)

Bas. Wr. Sk.(2)

Math Pr.Slv.(2)

Acd. Knw.(3)

Clusters available from Standard Acheasel (11 tests)

(#) = # tests

See Table 1‐3 for COG test‐cluster interpretive

cluster composition

Organization of WJ IV

Achievement (ACH) battery

Bas. Rdg. Sk.(2)

Brd. Rdg.(3)

Rdg. Flu.(2)

Rdg.(2)

Wr. Exp.(2)

Brd. Wr. Lg.(3)

Wr. Lg.(2)

Math Cal.Sk.(2)

Brd. Math.(3)

Math.(2)

Academic Skills (3)

Academic Applications (3)

Academic Fluency (3)

Broad Achievement (9)

Brief Achievement (3)

6/3/2014

6

Organizational overview of the WJ IV variations and comparisons

Dr. Kevin McGrewInstitute for Applied Psychometrics (IAP)

Comparison Options

• GIA/Achievement

• Scholastic Aptitude/Achievement

• Gf‐Gc/Achievement/other abilities

• Broad Oral Language/Achievement

• Academic Knowledge/Achievement

Five ability/achievement difference score procedures to help compare ability to current

levels of achievement.

[Procedures account for regression‐to‐the mean (and how it varies by age)]

Variation Options

• Intra‐cognitive based on COG Tests 1—7

• Intra‐achievement

– Based on ACH Tests 1—6

– Based on Academic Skills, Academic Fluency, and Academic Applications clusters

• Intra‐oral language based on OL Tests 1—4

Four variation procedures to help document an individual’s pattern of strengths and weaknesses.

6/3/2014

7

Note: All score distributions represent real scores for all 9‐13 year old norming subjects from

WJ IV co‐normed sample. Actual

prediction models vary by age or grade (developmentally shifting prediction models).

Note: The SD of predicted and difference score distributions are not 15. They would only be 15 if GIA/Brd. Rdg. correlation

was perfect (1.0).

Illustration of procedures used to develop ALL WJ IV variation/comparison difference (standard score) norms (GIA‐Broad Reading ACH example)

-E

(minus)

B Actual Broad Reading scores

in WJ IV norm data

40 80 120 160100 14060

‐1 to +1 SD

SD = 15

AGeneral Intellectual

Ability (GIA) scores in WJ IV norm data (predictor score)

40

80

120

160

100

140

60

‐1 to +1

SDSD

= 15

=(equals)

FDistribution of Actual‐Predicted GIA/Broad Reading Difference Scores in WJ IV norm data

(SD of difference score

distribution allows for

specification & evaluation of significant S/W with SD

and PRmetrics)

‐40 ‐30 ‐20 ‐10 0 10 20 30 40

‐1 to +1SD

SD = 10.8

D Predicted (criterion)

Broad Reading scores in WJ IV norm data

40 80 120 160100 14060

‐1 to +1 SD

SD = 11.6

General Intellectual Ability (GIA) cluster (Predictor score)

General Intellectual Ability (GIA) /Achievement comparison procedure

Gwm Gs Ga Glr GvGcGf

CHC Cognitive Abilities and WJ IV COG and OL Clusters

Oral Language

Grw Gq

Brief Achievement

Academic Skills

Academic Fluency

Academic Applications

Broad Achievement

Reading

Brd. Rdg.

Rdg. Flu.

Bas. Rdg. Sk.

Rdg. Cmp.(& Ext)

Rdg. Rate

Wr. Lng.

Brd. Wr. Lg.

Bas. Wr. Sk.

Wr. Exp.

Phn.‐Grp.Kn.

Math.

Brd. Math

Math Cal.Sk.

Math Pr. Slv.

(Possible criterion or predicted target scores)Broad Oral Language

Oral Expression

Listening Comprehension

CHC Achievement Abilities and WJ IV

clusters

Regression‐based prediction models that account for regression‐to‐the‐mean (and how it varies by age)

Predicted Target

Cluster Score

Actual Target Cluster Score(minus) (equals)

Difference Score

SD and PR for specifiedSD cut‐off score

(Compare to distribution of difference scores in WJ IV norm sample to determine

significant strength or weakness)

CHC Cognitive Abilities and WJ IV COG and OL Clusters


Oral Language/Achievement comparison procedure

SD and PR for specified

SD cut‐off score

(Compare to distribution of difference scores in WJ IV norm sample to determine

significant strength or weakness)

Broad Oral Language

(Predictor score)

Sp. Lx. Acc.Phon. Cod.

(Possible criterion or predicted target scores)

Regression‐based prediction models that account for regression‐to‐the‐mean (and how it varies by age)

Predicted Target Cluster

Score


Difference Score

Academic Skills

Academic Fluency


Brd. Rdg.

Rdg. Flu.

Bas. Rdg. Sk.

Rdg. Cmp.(& Ext)

Rdg. Rate

Wr. Lng.

Brd. Wr. Lg.

Bas. Wr. Sk.

Wr. Exp.

Phn.‐Grp.Kn.

Brd. Math

Math Cal.Sk.

Math Pr. Slv.

Reading Math.

Grw Gq

CHC Achievement Abilities and WJ IV ACH clusters


© Institute for Applied Psychometrics; Kevin McGrew 01-23-14

6/3/2014

8

Phon. Cod. Sp. Lx. Acc.

ST Wk. Mem.(&Ext)

Cog .Pr.Spd. LT Retrieval Visual Proc.Aud. Proc.

Perc. Spd.

Cognitive Efficiency (& Ext)

Aud. Mm. Sp.

Other CHC broad/narrow cognitive and orallanguage abilities and WJ IV clusters

Gwm Gs Ga Glr Gv Grw Gq

Brief Achievement

Academic Skills

Academic Fluency


Broad Achievement

Reading

Brd. Rdg.

Rdg. Flu.

Bas. Rdg. Sk.

Rdg. Cmp.(& Ext)

Rdg. Rate

Wr. Lng.

Brd. Wr. Lg.

Bas. Wr. Sk.

Wr. Exp.

Phn.‐Grp.Kn.

Math.

Brd. Math

Math Cal.Sk.

Math Pr. Slv.

CHC achievement abilities and WJ IV clusters

Gf‐Gc Composite(Predictor score)

GcGf

WJ IV Gf+Gc cognitive Ability cluster

Gf‐Gc “hybrid” variation/comparison procedure

Num. Fac.


Score



Difference Score


(Compare to distribution of difference scores in WJ IV norm sample to determine significant

strength or weakness)

Regression‐based prediction models that account for regression‐to‐the‐

mean

(See Table 1‐12for the specific tests and cluster information)

Phon Cod Sp Lex Acc

ST Wrk Mem(&Ext)

Cog Prc Spd LT Retrieval Visual ProcAud Proc

Perc Spd

Cognitive Efficiency (& Ext)

Aud Mem Sp

Other broad/narrow cognitive abilities and processing abilities and WJ IV clusters

Possible S/W predicted targets available for

evaluation

Gwm Gs Ga Glr Gv

Gf‐Gc Composite(Predictor)

GcGf

Gf+Gc cognitive abilityand WJ IV cluster

WJ IV Gf+Gc strength and weakness comparison procedure and options

Grw Gq

Brief Achievement

Academic Skills

Academic Fluency


Broad Achievement

Reading

Brd Rdg

Rdg Flu

Bas Rdg Sk

Rdg Cmp(& Ext)

Rdg Rate

Wr Lng

Brd Wr Lng

Bas Wrt Sk

Wr Exp

Phn‐Grp Kn

Math

Brd Math

Math Calc Sk

Math Pr Solv

Achievement domains and WJ IV clusters

Possible S/W

predicted targets

available for evaluation

Regression‐based predictions that account for regression‐to‐the‐

mean and produce “real discrepancy norms” to identify possible patterns of significant strengths and weaknesses

Grw Gq

Brief Achievement

Academic Skills

Academic Fluency


Broad Achievement

Reading

Brd. Rdg.

Rdg. Flu.

Bas. Rdg. Sk.

Rdg. Cmp.(& Ext)

Rdg. Rate

Wr. Lng.

Brd. Wr. Lg.

Bas. Wr. Sk.

Wr. Exp.

Math.

Brd. Math

Math Cal.Sk.

Math Pr. Slv.

CHC achievement abilities and WJ IV clusters

Academic Knowledge(Predictor score)

Academic Knowledge/Achievement comparison procedure


Score



Difference Score


(Compare to distribution of difference scores in WJ IV norm sample to determine significant

strength or weakness)


CHC cognitive abilities and WJ IV cognitive and oral language clusters

Sp. Lx. Acc.Phon. Cod.

(See Table 1‐8 for the specific tests and cluster information)

Regression‐based prediction models that account for regression‐to‐the‐mean

6/3/2014

9

Gc Gwm Gs Ga Glr GvGf

WJ IV Scholastic Aptitude Cluster Organization

Visualization(Vz)

NumbersReversed(WM)

OralVocabulary(LD/VL)

Analysis‐Synthesis(RG/RQ)

PairCancellation

(P/EF)

Visualization(Vz)

Oral Vocabulary(LD/VL)

NumberSeries (RQ)

Num PatternMatching

(P)

PhonologicalProcessing

(PC)

OralVocabulary(LV/VL)

VerbalAttention(WM)

Num PatternMatching

(P)


(PC)


VerbalAttention(WM)

Num PatternMatching

(P)


(PC)

OralVocabulary(LV/VL)

Story Recall(MM)

Num PatternMatching

(P)


(PC)


Concept Formation

(I)

ReadingBroad ReadingReading CompReading Comp‐ExtReading FluencyReading Rate

Basic Rdg Skills

WritingBroad WritingWritten Expression

Basic Writing Skills

MathBroad MathMath Calc Skills

Math Prob Solving

WJ IV Ach Clusters

Grw/Gq domain general

Grw domain specific

Gq domain specific

Reading Brd. Rdg.

Rdg. Flu.

Bas. Rdg. Sk.

Rdg. Rate

Wr. Lng. Brd. Wr. Lg.

Bas. Wr. Sk.

Wr. Exp.

Math. Brd. Math

Math Cal.Sk.

Math Pr. Slv.

GrwGwm Gs Ga Glr GvGcGf Gq

Rdg. Cmp.

Rdg Cmp. Ex

Rdg. Apt. A(Predictor Score)

Rdg. Apt. B(Predictor Score)

WL. Apt. A(Predictor Score)

WL. Apt. B(Predictor Score)

Math Apt. A(Predictor Score)

Math Apt. B(Predictor Score)

Regression‐based prediction models that account for regression‐to‐the‐mean

CHC cognitive abilities and WJ IV Scholastic Aptitude cognitive clusters CHC achievement abilities and WJ IV clusters


Score



Difference Score

(Compare to distribution of difference scores in WJ IV norm sample to determine significant strength or

weakness)Scholastic Aptitude/Achievement comparison procedures

(See Table 1‐10 for the specific tests and cluster information)

Copyright, Institute for Applied Psychometrics, K.McGrew; 01‐23‐14

Scholastic Aptitude Clusters: Potential Uses

• Designed to predict near term academic performance

• Time efficient referral‐focused selective testing

• Time efficient academic domain‐specific screening

• Time efficient annual review evaluations

• Gifted and talented screening – domain‐specific talents?

• Potentially useful in estimating quickness of response to intervention

• Provide information regarding the concept of “expected underachievement”

• Formulation of differential academic domain expectations

• ?

6/3/2014

10

One of the WJ IV guiding design objectives was for the WJ IV to retain a focus on psychometric quality that has been associated with the previous editions of Woodcock‐Johnson batteries:

•By providing a new, large, and nationally‐representative standardization sample of the US population;

•By updating items and simplifying test administration and interpretation procedures;

•By augmenting the underlying scaling of speeded tests; and

•By utilizing state‐of‐the art data collection, test development, and data analytic methods as models to facilitate progress in the field of applied test development.

WJ IV Gs tests based on new Rasch “rate‐based” item calibration method: WJ III Visual Matching / WJ IV

Letter‐Pattern Matching example

WJ III Visual Matching (3 minutes)• 60 items• 22 Possible “artificial” bonus points• 82 Possible raw score points (27 % of total scale was “artificial” bonus points)

WJ IV Letter‐Pattern Matching (3 minutes)• 84 items (24 more “real” items to eliminated need for bonus points)• Test scores based on count of number of correct responses in each of thethree minute intervals. Thus, three different rate‐based “item” scores

• Rasch rating scale model (Andrich, 1978) use to obtained difficulty estimatefor each minute of testing

• Need for artificial bonus points eliminated• Method rewards examinees who work quickly and accurately• Method penalizes examinees who respond quickly and not cautiously

(Note: IRT scaling of speed tests has historically been problematic due to violations of IRT assumptions. IRT methods stilltypically used for practical purposes, recognizing the assumption violations. See McGrew, Werder & Woodcock, 1991)

National Norm Sample

• 7,416 participants

• Preschool (664)

• K‐12 (3,891)

• College/University (775)

• Adult (2,086)

• Ages 2‐90+ years, Grades K.0‐18.0

• 100 geographically diverse communities from 46 states and the District of Columbia

• The Standards for Educational and Psychological Testing (aka., the Joint Standards) (American Psychological Association, American Educational Research Association, & National Council on Measurement in Education, 1999; in press) guided the norming and technical analyses

6/3/2014

11

Median reliabilities (across

all technical age groups) for WJ IV COG

clusters

General Intellectual Ability 0.97Brief Intellectual Ability 0.94Gf-Gc Composite 0.95Comprehension-Knowledge 0.93Comprehension-Knowledge- Extended 0.94Fluid Reasoning 0.94Fluid Reasoning- Extended 0.96Short-Term Working Memory 0.91Short-Term Working Memory- Extended 0.93Cognitive Processing Speed 0.94Auditory Processing 0.92Long-Term Retrieval 0.97Visual Processing 0.86Quantitative Reasoning 0.94Auditory Memory Span 0.90Number Facility 0.90Perceptual Speed 0.93Cognitive Efficiency 0.95Cognitive Efficiency-Extended 0.95Reading Aptitude A 0.93Reading Aptitude B 0.93Math Aptitude A 0.94Math Aptitude B 0.95Writing Aptitude A 0.92Writing Aptitude B 0.93


all technical age groups) for WJ IV OL clusters

Oral Language 0.90Broad Oral Language 0.92Oral Expression 0.89Listening Comprehension 0.90Phonetic Coding 0.95Speed of Lexical Access 0.89Vocabulary 0.93

Copyright; Institute for Applied Psychometrics; K.McGrew 02‐5‐14

Reading 0.95Broad Reading 0.97Basic Reading Skills 0.95Reading Comprehension 0.93Reading Comprehension-Extended

0.96

Reading Fluency 0.96Reading Rate 0.96Mathematics 0.96Broad Mathematics 0.97Math Calculation Skills 0.97Math Problem Solving 0.95Written Language 0.94Broad Written Language 0.95Basic Writing Skills 0.95Written Expression 0.92Academic Skills 0.97Academic Applications 0.96Academic Fluency 0.97Academic Knowledge 0.95Phoneme-Grapheme Knowledge 0.94Brief Achievement 0.97Broad Achievement 0.99


all technical age groups) for WJ IV ACH

clusters

6/3/2014

12

Sources of validity evidence presented

•Representativeness of the WJ IV Test Content, Process, and Construct Coverage

•Developmental Patterns of WJ IV Ability Clusters

•Internal Structure and Relations within the WJ IV

•Relationship of WJ IV Scores to Other Measures of Cognitive Abilities, Oral Language, and Achievement

•Performance of Clinical Samples on WJ IV Measures

Stage 3: Confirmatory structural

model cross‐validation phase

Stage 2: Exploratory structural model

generation (MG) and evaluation

Ages 3‐5 A(n = 209)

Ages 3‐5 B (n = 208)

Ages 6‐8 A(n =412)

Ages 6‐8 B(n = 411)

Ages 9‐13 A(n = 785)

Ages 9‐13 B(n = 787)

Ages 14‐19 A(n = 842)

Ages 14‐19 B(n = 843)

Ages 20‐39 A(n = 625)

Ages 20‐39 B(n = 626)

Ages 40‐90+ A(n = 571)

Ages 40‐90+ B(n = 575 )

Model Development

(MD) samples (A)

Model Cross‐Validation

(MCV) samples (B)

Stage 1: Split‐sample

random sample generation

Three‐stage internal/ structural validity procedures for WJ IV battery

+

+

Cluster analysis(CA)

Multidimensional scaling analysis

(MDS)

Principal component analysis(PCA)

Stage 2AExploratory structuralanalysis in Sample A

Review of prior WJ,

WJ‐R, WJ III structural validity research

Specify initial confirmatory factor analysis model (CFA)

Review of contemporary

CHC and cognitive

neuroscience research

CFA of MG model result from MD (A) sample in MCV (B)

sample(s) Yes

Most plausible and best fitting odel?

Model adjustment(s)

No

Stage 2BSpecify initial CFA

models

Stage 2CEvaluate and adjust

CFAmodel

Wards cluster analysis of ages 9-13 year sample (model development sample A; n=785)

NUMREVOBJNUMVRBATN

MEMWRDSENREP

NWDREPUNDDIR

MEMNAMWRDATKSPLSND

SNDAWRSEGMNTPHNPROSNDBLN

VALVISUAL

ANLSYNCONFRM

VISCLOPICREC

RPCNAMRETFLU

ORLCMPVRBANL

HUMPICVOC

ORLVOCGENINF

SOCSCI

STYRECWRTSMPRDGRECNUMSENAPPROB

CALCNUMSERNUMMAT

SPELLLWIDNT

PSGCMPRDGVOC

EDITORLRDGMTHFLUWRDFLUSNRDFLSNWRFLNUMPATLETPATPAIRCN

Gc

Glr-MM/Grw

Gq/Gf-RQ

Grw

Gs

Gs-ACH

Gs-COG

Acquired knowledge systems

Gv

GfGf+Gv

Glr-LA

Gwm-WM

Gwm-MS Gwm

Ga-PC

Ga-PC/Grw

Ga

Ga+Gwm (sequential/serial processing)

Cognitive Operations: Controlled/deliberate/system 2

General intelligence (g)

Bold font = Broad or general CHC abilities

Underlined font= Possible intermediate level stratum dimensions

Groupings to the left of the vertical dashed lines represent the groupings interpreted as representing valid CHC abilities. Higher‐order groupings to the left of the dashed line are potentially interesting hypothesized intermediate ability dimensions.

Glr-NA

Glr (Retrieval Fluency)

Cognitive Operations: Automatic/system 1

6/3/2014

13

-2 -1 0 1 2

-2

-1

0

1

2

NUMSER

VRBATN LETPAT

PHNPRO

VAL VISUAL

GENINF

NUMREV

NUMPAT

NWDREP

STYREC

PICREC

ANLSYN

OBJNUMPAIRCN

MEMWRD

SEGMNT

RPCNAM

SENREP

UNDDIR

SNDBLN

RETFLU

SNDAWR

LWIDNT

SPELL

PSGCMP

CALC

WRTSMP

ORLRDG

MTHFLU

SNWRFL

RDGREC

NUMMAT

EDIT

WRDFLU

SPLSND

RDGVOC

SCI

MEMNAM

VRBANL

VISCLO

Gs‐Cog

Gs‐Ach

Glr‐LA

Ga

Grw

Gv

Glr‐MM

Gc

Gq/Gf‐RQ

Gwm‐WM

Gwm‐MS

WRDATK

MDS (Guttman Radex) of ages 9-13

year sample (model

development sample A; n=785 )

GfCONFRM

Test EFAOther exp. methods CFA Cross‐validation

WJ III No No Yes No

SB5 No No Yes No

DAS‐II No No Yes No

KABC‐II No No Yes (in “exploratory manner”)

No

WAIS‐IV No No Yes No

WISC‐IV Yes No Yes Yes: Factor score (7) congruence in random CV

sample of 440

WJ IV Yes Yes: MDS and cluster analysis

Yes: Exploratory model generating CFA (MD samples) and CV CFA (CV

samples)

Yes: Randomly split samples at six age groups. CV of final MD models in

CV sample

Structural validity method comparisons across 7 intelligence batteries

15 independent WJ IV concurrent validity

studies differentiated by age/grade

WJIV COG compared to:

• WISC‐IV, WAIS‐IV, WPPSI‐III, KABC‐II, DAS‐II

WJIV OL compared to:

• CELF‐F, PPVT‐4, CASL, OWLS

WJIV ACH compared to:

• KTEA‐II, WAIT‐III, OWLS‐WE

6/3/2014

14

Clinical Validity Group Inclusion Criteria

Learning Disability (LD):

6–17 years Documented learning disability in reading Reading LD must be the primary diagnosis or eligibility category Currently receiving special education services under SLD category

Learning Disability (LD): Math

6–17 years Documented learning disability in math Math LD must be the primary diagnosis or eligibility category Currently receiving special education services under SLD category

Learning Disability (LD): Writing

6–17 years Documented learning disability in writing Writing LD must be the primary diagnosis or eligibility category Currently receiving special education services under SLD category

Head Injury 7–17 years Documented traumatic brain injury TBI must be the primary diagnosis Currently receiving special education services

Language Delay 7–13 years Documented language delay Language delay must be the primary diagnosis Currently receiving special education services

Autism Spectrum Disorder (ASD)

6–17 years Documented Autism Spectrum Disorder1

ASD is the primary diagnosis Currently receiving special education services

Attention Deficit Hyperactivity

Disorder (ADHD)7–12 years

Documented ADHD2

ADHD is the primary diagnosis Currently receiving special education services and/or 504 plan

accommodations

Gifted3 4–9 years Currently participating in high ability/gifted and talented school curriculum Currently receiving gifted services

Mental Retardation 6–12 years Documented mental retardation Mental retardation is the primary diagnosis Currently receiving special education services

WJ IV clinical validity evidence reported for 9 clinical groups

What is cognitive complexity?

Cognitive complexity

CHC factor breadth

Degree of g‐loading

Complicated

(Does not necessarily equal)

Factorial complexity

Content/stimulus dimension (Extension of BIS and Brunswick Symmetry Model)

Cognitive knowledge domains/systems

Cognitive operations

Cognitive control

Cognitive efficiency

Sensory functions

Motor functions

Abilty domain dim

ension

Type I

Processing

Type II

Processing

Note: CHC taxonomy is embedded in the ability domain dimension

McGrew (2010) – First formal introduction of cognitive complexity in “Beyond CHC” test development /evaluation framework(Neuropsych. Conference in Fremantle, Australia)

Cognitve complexity dimension

High

Medium

Low


This is a heuristic framework and is not intended to represent a theoretical model of intelligence

6/3/2014

15

• Larger number of cognitive component processes(factorial complexity)

• Accumulation of speed component differences

• Increased demands of attentional control (AC) & working memory

•More demands on adaptive functions (assembly, control, and monitoring – executive functions)

• More important component processes (e.g., inference; eduction of relations)

Parameters of cognitive efficiency in info. proc. models

Cognitively complex design characteristics (Lohman & Lakin, 2011)

Approach 1. Increasing the cognitive complexity of a test is often accomplished by making the test a mixed measure of more than one narrow CHC ability (factorially complex mixed CHC measures)

Approach 2. A second approach is to increase the complexity of information processing demands of the tests within a specific narrow CHC domain (Lohman & Larkin, 2011; McGrew, 2012). This second form of cognitive complexity, not to be confused with factorial complexity, places greater demands on cognitive information processing (cognitive load), requires greater allocation of key cognitive resources (working memory or attentional control), and invokes the involvement of more cognitive control or executive functions (Arend, Colom, Botella, Contreras, Rubio, & Santacreu, 2003; Jensen, 2011; Lohman & Larkin, 2011; Marshalek, Lohman, & Snow; 1983). This second approach to increasing test cognitive complexity was a primary design principle for the WJ IV.

One design objective in the WJ IV was to increase the cognitive complexity requirements for selected tests and clusters to provide greater ecological validity and

interpretive relevance of the measures.

Most contemporary CHC designed individual tests have focused on developing relatively pure measures of each cognitive ability

(mental pulley)

Gf Gwm Ga Gv

Gc Gs Glr Gs

Analogy: Think of general intelligence (g) as a system of relatively independent cognitive abilities (relatively construct

“pure” pulleys) working together to deal with a specific cognitive task load

6/3/2014

16

In this approach a test is designed to be a mixed measure of two (or more) cognitive abilities (mental pulleys; Gf + Gv)

Gf + Gv

Gf + Gv Gwm Gs Ga Glr Gv Gs

Approach 1 to developing cognitively complex tests:Construct factorially complex measures (a system of pulleys from 2 or more domain functions working in

combination).

Wechsler Arithmetic test

Gf + Gv

Gf + Gwm + Gc + Gq Ga Glr Gv Gs

Approach 1 example

Ga Gv Gwm Gs Gf Glr Gv Gs

Approach 2 is to increase the complexity of information processing demands of the tests within a specific CHC cognitive functional domain. Tasks are still as relatively pure a measure of the CHC domain as possible but there is a deliberate increase in the number of “mini‐pulleys” (cognitive information component complexity) that work together within the CHC domain. This was the primary approach used for certain WJ IV tests.

6/3/2014

17

Working Memory

Long‐Term Memory

Storage Retrieval

Central Executive (Executive functions or control?

• Inhibit•Shift

•Update

PerceptionSensation

DecisionsOutput

????

Conceptual explanation of parameters of cognitive efficiency

Focus of Attention

Recently activated concepts (Blue‐new; Red‐old)

0.4 0.5 0.6 0.7 0.8g-loading

-0.10

0.45

1.00

1.55

2.10

MD

Sre

lativ

e C

C (

rCC

)

ORLVOC‐Gc

SNDAWR‐Ga

ORLCMP‐Gc

VRBATN‐Gwm

PHNPRO‐Ga

OBJNUM‐Gwm

NUMSER‐GfCONFRM‐GfUNDDIR‐Gwm

NUMREV‐Gwm

PICVOC‐Gc

STYREC‐Glr

SENREP‐Gwm

SEGMNT‐Ga

ANLSYN‐GF

MEMWRD‐Gwm

SNDBLN‐Ga

VISUAL‐Gf

RETFLU‐GlrLETPAT‐Gs

NWDREP‐GaNUMPAT‐Gs

PAIRCN‐Gs

RPCNAM‐Glr

PICREC‐Gv

Relationship between g‐loadings and MDS‐based relative cognitive complexity (rCC) for WJ IV COG and OL tests

Correlation = .93; but correspondencediverges as test become higher in g and

cognitive complexity

• COG: Verbal Attention

• COG Number Series

• COG Letter‐Pattern Matching

• COG Visualization

• COG Phonological Processing

• COG Nonword Repetition

• OL Segmentation

New COG OL Tests

6/3/2014

18

COG Test 2: Number Series

• Was in WJ III Diagnostic Supplement• Gf‐RQ (Quantitative Reasoning)• Not a “controlled learning” test as are ConceptFormation (Gf‐I) and Analysis‐Synthesis (Gf‐RG)

• More Gf “in the wild” – without examiner provided scaffolding

• Extensive history as a premier Gf test in thepsychometric measurement of intelligence

• High in cognitive complexity and g. Best single testpredictor of achievement. Best indicator of Gf factor.

• In GIA, BIA, Gf‐Gc Composite, Gf, Gf‐Ext, QuantitativeReasoning (RQ), and one Math Aptitude clusters.

COG Test 7: Visualization

• Measure of Gv‐Visualization (Vz)

•Visualization consists of two subtests that each measure Gv‐Vz (visualization) via tasks that vary on task complexity and degree of “minds eye” (mental rotation) manipulations

• Within Gv, highest on cognitive complexity, g, Gv factor, and prediction of achievement

• In GIA, Gv and both Math Aptitude clusters

COG Test 3: Verbal Attention

• Measure of Gwm (working memory‐WM; attentionalcontrol‐AC)

• More ecological “real world” valid measure ofworking memory

• High in cognitive complexity and g. Within Gwm, themost cognitively complex, one of bestindicators of Gwm factor, and best predictorof achievement

• In GIA, BIA, Gwm, Gwm3, Cognitive Efficiency, and oneReading and 1 Written Language Aptitude clusters.

6/3/2014

19

COG Test 4: Letter‐Pattern Matching

• Measure of Gs (perceptual speed) and orthographicprocessing

• This speeded test (all WJ IV speeded tests) is based on a newrate‐based method of scaling the scores that eliminates theneed for bonus points

• Within Gs, it matches Number Pattern Matching in g, Gs factor loading, and prediction of achievement. Is more cognitively complex than Number Pattern Matching

• In GIA, Gs, Perceptual Speed (P), Cog. Eff. and Cog. clusters

COG Test 5: Phonological Processing

• Ga (PC) / Glr (LA/FW)•3 subtests (Word Access; Word Fluency; Substitution• Measures three aspects of speech sound processingthat requires the efficiency construction of sound‐based lexical representations

• High in cognitive complexity and g. Best single Gatest predictor of achievement. High loading on Gaand secondary low loading on Gc (accessing thelexicon). Also loaded on narrow LA factor inbroard+narrow bottom‐up CFA models.

• In GIA, Ga, and all reading and writing scholasticaptitude clusters

Nonword repetition tasks

Examinee listens to a nonsense word and then mustrepeat the word exactly.

Requires temporary storage of phonological segments in immediate awareness.

Significant body of research has found such tasks tobe significantly related to (and be possible “markers”of) reading disabilities, dyslexia and SLI (specific language impairment)

6/3/2014

20

OL Test 3: Segmentation

• Ga (PC)

• Examinee listens to words and identifies word parts

• In OL Phonetic Coding (PC) cluster

• Highest loading test on Ga factor across all ages

• A moderate measure of g and predictor of ach. across all ages; much more so (and more cognitively complex) than Sound Blending.

• Such tasks have been reported to be strong predictors of early reading (Bouwmeester et al, 2011; Geuden & Sandra, 2003)

Contemporary CHC broad and narrow ability content coverage by WJ IV Cognitive (as per WJ IV authors)

General Intelligence (g)


(A3)



(RC)

Reading speed (RS)


English usage (EU)


Writing speed (WS)




Induction (I)

General sequential

reasoning (RG)


Memory span (MS)






Visualization

(Vz)

Visual Memory (MV)

Spatial Scanning

(SS)



(Gq)


Comp ‐Knowledge

(Gc)






Auditory Processing

(Ga)

Processing

Speed (Gs)


Number Facility (N)




(Gkn)

Geography ach. (A5)

Shading designates abilities measured by WJ IV COG battery


(LD)



(UM)

Word Fluency (FW)



The WJ IV COG is not your fathers intelligence test!

The WJ IV COG GIA is a much more cognitively complex(and high g) measure of intelligence

How did we do this?

What evidence do we have to support this conclusion?

This image cannot currently be displayed.This image cannot currently be displayed.


6/3/2014

21

WISC‐IV WAIS‐IV WPPSI‐III KABC‐II SB‐5 DAS‐II

FS IQ FS IQ FS IQ FCI FS IQ GCA

(n=174) (n=177) (n=99) (n=50) (n = 50) (n = 49)

WJ IV g‐measures

General Intellectual Ability (GIA)

0.86 0.84 0.72 0.77 0.80 0.83

Brief Intellectual Ability (BIA) 0.83 0.74 0.76 0.79

Gf‐Gc Composite 0.83 0.78 0.71 0.82

Note. Correlations in italic represent correlations with a pseudo‐WJ IV GIA score computed from 7

WJ IV tests (WJ IV COG does not provide an GIA‐Edv cluster)

Select concurrent validity evidence: Correlations of WJ IV primary COG g‐scores with external measures

Conclusion: The WJ IV GIA, BIA and Gf‐Gc composite clusters demonstrate strong validity evidence as measures of general intelligence when the criterion are the global composite/total

scores from other major IQ batteries in the field

WJ III GIA other IQ score correlations

were from .67 to .76

GIA ‐Standard

GIAOral Vocabulary

General Information

Number Series

Concept Formation

Verbal

Attention

Numbers Reversed

Story Recall

Visual‐AudLearning

Visualization

Picture Recognition

Letter‐Pat. Match.

Pair

Cancellation

Phonological

Processing

Nonword

Repetition

Primary WJ III and WJ IV COG tests and Clusters

Tests in WJ IV COG clusters

Tests in WJ III COG Clusters

WJ IV GIA

WJ III GIA‐Standard

* Visual Matching is renamed Number‐Pattern Matching in WJ IV

Comparison of composition of primary WJ III and WJ IV COG CHC and GIA clusters

(WJ III Spatial Relations is half

of WJ IV Visualization)

(WJ IV Oral Vocab. was part

of WJ III Verbal Comp.)

Auditory Attention

Decision

Speed

Analysis‐Synthesis

Retrieval

Fluency

Spatial

RelationsVerbal Comp.

Memory for Words

Sound

Blending

Visual

Matching *


Comp ‐Knowledge

(Gc)




Auditory Processing

(Ga)

Processing

Speed (Gs)

Short‐Term

Wrk Mem(Gwm)

© Institute for Applied Psychometrics; Kevin McGrew 01‐23‐14

Four primary design principles have resulted in the WJ IV COG GIA being significantly different in composition than the WJ III COG GIA‐Std.

The WJ IV COG GIA cluster tests should:

1. Be the best factor indicators of each CHC broad domain

2. Be the best predictors of achievement from each CHC broad domain

3. Be the most cognitively complex indicators from each CHC broad domain

4. Be the best measures of g (general intelligence) from each CHC broad domain

5. Collectively should have a relatively equal balance of type of stimulus characteristics (verbal, numeric, figural)

(Note: Trade‐offs between the four design objectives were sometimes necessary)

6/3/2014

22


This image cannot currently be displayed.

The Cognitive tests were evaluated on the basis of four (of five total) quantifiable COG design criteria

Data augmented by Siskel and Ebert informal rating system

Average CHC factor loadings

Average achievement correlation across domains

Average degree of g‐loadings

Average degree of relative cognitive complexity

(Average across ages 6‐90+)

Comprehension – Knowledge(Gc)

Lexical Knowledge(VL)

General Information(K0)

Language Development (LD)

ORLVOC* GENINFPICVOC

* Test in GIA

Author/expertCHC narrow

factor classifications

(Gc cluster)(Voc. Cluster‐

LD/VL)

Induction(I)

General SequentialReasoning (RG)

QuantitativeReasoning (RQ)

ANLSYNCONFRMNUMSER* (Gf cluster)(Gf3

cluster)

(Quantitative Reasoning Cluster‐RQ)

* Test in GIA

Fluid Reasoning(Gf)



6/3/2014

23

Short‐Term WorkingMemory (Gwm)

Working Memory(WM)

Memory Span(MS)

Attentional Control(AC)

MEMWRD

SENREP

(Auditory MemorySpan Cluster‐MS)

OBJNUMVRBATN* NUMREV

* Test in GIA

(Gwm cluster)

(Gwm3 cluster)




Cognitive Processing Speed(Gs)

Attentional Control

PerceptualSpeed (P)

PAIRCNNUMPATLETPAT*

* Test in GIA

Short‐Term Working Memory (Gwm)

(Gs cluster)

(Perceptual Speed cluster‐P)



Long‐Term Retrieval(Glr)

AssociativeMemory (MA)

MeaningfulMemory (MM)

* Test in GIA

VALSTREC* (Glr cluster)

Speed of LexicalAccess (LA)

RETFLURPCNAM (Speed of Lexical Access

cluster‐LA)



6/3/2014

24

AuditoryProcessing (Gs)

PhoneticCoding (PC)

NWDREPPHNPRO*

* Test in GIA

Short‐Term Working Memory (Gwm)

Memory for Sound Patterns (UM)

SEGMNT SNDBLN

(Ga cluster)

(Phonetic Coding cluster‐PC)




This image cannot currently be displayed.

The WJ IV Auditory Processing (Ga) cluster is not your fathers Ga measure.

WJ IV still has the Oldsmobile Ga (Phonetic Coding) in OL: COG now has more cognitively

complex Gameasures

The WJ IV has taken a broader contemporary view of the domain of Ga


Auditory Processing (Ga) abilities, when properly measured, should have a

prominent chair at the round table of cognitive CHC abilities


6/3/2014

25

Visual Processing(Gv)

Visual Memory(MV)

Visualization(Vz)

* Test in GIA

PICRECVISUAL* (Gv cluster)



COG/OL Mdn.Clusters r r2General Intellectual Ability 0.76 0.58Brief Intellectual Ability 0.73 0.53Gf-Gc Composite 0.68 0.46

Comprehension-Knowledge (Gc) 0.58 0.34Comprehension-Knowledge-Ext (Gc3) 0.58 0.34Fluid Reasoning (Gf) 0.64 0.41Fluid Reasoning-Ext (Gf3) 0.59 0.35Short-Term Working Memory (Gwm) 0.53 0.28Short-Term Working Memory-Ext (Gwm3) 0.52 0.27Cognitive Processing Speed (Gs) 0.49 0.24Auditory Processing (Ga) 0.51 0.26Long-Term Retrieval (Glr) 0.43 0.18Visual Processing (Gv) 0.38 0.14Quantitative Reasoning (Gf-RQ) 0.59 0.35Auditory Memory Span (Gsm-MS) 0.47 0.22Number Facility (Gs-N) 0.62 0.38Perceptual Speed (Gs-P) 0.58 0.34Cognitive Efficiency (Gsm+Gs) 0.55 0.30Cognitive Efficiency-Ext (Gsm+Gs) 0.66 0.44

Oral Language 0.55 0.30Broad Oral Language 0.57 0.32Oral Expression 0.54 0.29Listening Comprehension 0.55 0.30Phonetic Coding (Ga-PC) 0.44 0.19Speed of Lexical Access (Glr-LA) 0.37 0.14Vocabulary (Gc-LD/VL) 0.60 0.36

WJ IV COG and OL cluster correlations

with WJ IV ACH clusters:

Correlations across 15 ACH clusters (ages

6‐90+)

WISC‐IV WAIS‐IV WPPSI‐III KABC‐II SB‐5 DAS‐II

FS IQ FS IQ FS IQ FCI FS IQ GCA

(n=174) (n=177) (n=99) (n=50) (n = 50) (n = 49)

WJ IV g‐measures

General Intellectual Ability (GIA)

0.86 0.84 0.72 0.77 0.80 0.83

Brief Intellectual Ability (BIA) 0.83 0.74 0.76 0.79

Gf‐Gc Composite 0.83 0.78 0.71 0.82

Note. Correlations in italic represent correlations with a pseudo‐WJ IV GIA score computed from 7

WJ IV tests (WJ IV COG does not provide an GIA‐Edv cluster)

Select concurrent validity evidence: Correlations of WJ IV primary COG g‐scores with external measures

Conclusion: The WJ IV GIA, BIA and Gf‐Gc composite clusters demonstrate strong validity evidence as measures of general

intelligence when the criterion are the global composite/total scores from other major IQ batteries in the field

WJ III GIA other IQ score correlations

were from .67 to .76

6/3/2014

26

WISC‐IV WISC‐IV WISC‐IV WISC‐IV

VCI PRI WMI PSI

WJ IV Measures (Gc) (Gf/Gv) (Gwm) (Gs)

CHC Factor Clusters

Comprehension‐Knowledge (Gc) 0.79Fluid Reasoning (Gf) 0.70

Short‐Term Working Memory (Gwm) 0.72

Processing Speed (Gs) 0.55

Auditory Processing (Ga)Long‐Term Retrieval (Glr)

Visual Processing (Gv) 0.55

Narrow Ability & Clinical ClustersQuantitative Reasoning (RQ) 0.65

Auditory Memory Span (MS) 0.52

Number Facility (N) 0.57Perceptual Speed (P) 0.56

Note: Bold font values represent correlations between best comparable CHC broad composites.Red bold font are interesting correlations given the Wechsler composites composition.

WISC‐IV/WAIS‐IV PRI and WMI composites are not comparable‐‐different mixtures of CHC abilities. (see next slide)

WJ IV COG /WISC‐IV CHC composite select score correlations ( n = 174)

Block Design‐Gv‐VzPic Cocnepts – Gf‐IMatrix Reasoning – Gf‐I

Digit Span – Gwm‐MS/MWLetter‐Num Seq – Gwm‐MW

WAIS‐IV WAIS‐IV WAIS‐IV WAIS‐IVVCI PRI WMI PSI

WJ IV Measures (Gc) (Gv/Gf) (Gwm/Gq) (Gs)

CHC Factor ClustersComprehension‐Knowledge (Gc) 0.74Fluid Reasoning (Gf) 0.57Short‐Term Working Memory (Gwm) 0.67Processing Speed (Gs) 0.44Auditory Processing (Ga)Long‐Term Retrieval (Glr)Visual Processing (Gv) 0.57

Narrow Ability & Clinical ClustersQuantitative Reasoning (RQ) 0.54 0.53Auditory Memory Span (MS)Number Facility (N) 0.65 0.52Perceptual Speed (P) 0.61

Note: Bold font values represent correlations between best comparable CHC broad composites.Red bold font are interesting correlations given the Wechsler composites composition.WISC‐IV/WAIS‐IV PRI and WMI composites are not comparable‐‐different mixtures of CHC abilities. Noted Wechsler correlations with tests classifications based on Flanagan et al., 2013.

WJ IV COG / WAIS‐IV CHC composite select score correlations ( n = 177)

Block Design‐Gv‐VzVisual Puzzles – Gv‐VzMatrix Reasoning – Gf‐I

Digit Span – Gwm‐MS/MWArithmetic – Gwm‐WM/Gf‐RQ (Gq?)

KABC‐II KABC‐II KABC‐II KABC‐II KABC‐IIKnow./ Plan./ Sim./ Seq./ Lrng./

WJ IV Measures GcIndex

GfIndex

Gv Index

GsmIndex

GlrIndex

CHC Factor ClustersComprehension‐Knowledge (Gc) 0.82Fluid Reasoning (Gf) 0.46Short‐Term Working Memory (Gwm)

0.42

Processing Speed (Gs)Auditory Processing (Ga)Long‐Term Retrieval (Glr) 0.64Visual Processing (Gv) 0.37

Note: Bold font values represent correlations between best comparable CHC broad composites.Low to moderate corresponding Gf, Gv and Gwm correlations most likely reflect narrow ability content differences in composites (classifications based on Reynolds et al., 2007 and Flanagan et al., 2013).

WJ IV COG / KABC‐II composite select score correlations (n = 50)

Pattern Reasoning ‐ Gf‐I/Gv‐Vz Story Completion – Gf‐RG

Number Recall – Gsm‐MSWord Recall – Gsm‐MS/WM?

Rover – Gv‐SSTriangles – Gv‐Vz

6/3/2014

27

SB‐5 SB‐5 SB‐5 SB‐5 SB‐5Fluid Quant. Vis.‐Spatial Working

Know. Reas. Reas. Proc. MemoryWJ IV Measures (Gc) (Gf) (Gf‐RQ) (Gv) (Gwm)

CHC Factor ClustersComprehension‐Knowledge (Gc) 0.68 0.75 0.72 0.72Fluid Reasoning (Gf) 0.67 0.56 0.66 0.66Short‐Term Working Memory (Gwm) 0.62 0.69Cognitive Processing Speed (Gs)Auditory Processing (Ga) 0.68 0.73 0.72Long‐Term Retrieval (Glr)Visual Processing (Gv) 0.40

Note: Bold font values represent correlations between best comparable CHC broad composites.Red bold font are interesting correlations given the SB‐5 composites composition.The convergent/divergent validity of the SB‐5 CHC composite scores has been seriously questioned (Canivez, 2008; DiStefano & Dombrowski, 2006; Keith & Reynolds, 2010) as well as very high composite score intercorrelations in the SB‐5 technical manual (.65 to .75)

WJ IV COG / SB‐5 CHC composite select score correlations (n = 50)

• 7 new tests• Oral Reading, Reading Recall, • Word Reading Fluency• Number Matrices• Science, Social Studies, Humanities

• 8 new clusters• Reading, Reading Comprehension‐Extended, Reading Fluency, Reading Rate

• Written Language• Mathematics• Brief Achievement, Broad Achievement

Achievement: What’s New?

• Conormed with WJ IV Tests of Cognitive Abilities andWJ IV Tests of Oral Language

• Qualitative Observation Checklists for Tests 1‐11– Located in Test Record– Helps document important information about how examinee performed on the task

– Includes data on percentage of age mates at each rating

Achievement: What’s New?

6/3/2014

28

Test 1: Letter‐Word Identification

Test 2: Applied Problems

Test 3: Spelling

Test 4: Passage Comprehension

Test 5: Calculation

Test 6: Writing Samples

Test 7: Word Attack

Test 8: Oral Reading‐‐NEW

Test 9: Sentence Reading Fluency (Renamed)

Test 10: Math Facts Fluency (Renamed)

Test 11: Sentence Writing Fluency (Renamed)

Tests of AchievementStandardBattery

WJ IV Tests of Achievement‐‐‐Extended Battery

Test 12: Reading Recall‐‐NEW

Test 13: Number Matrices‐‐EXPANDED

Test 14: Editing

Test 15: Word Reading Fluency‐‐NEW

Test 16: Spelling of Sounds

Test 17: Reading Vocabulary‐‐REVISED

Test 18: Science‐‐EXPANDED

Test 19: Social Studies‐‐EXPANDED

Test 20: Humanities‐‐EXPANDED

Provides clusters for:•ReadingComprehension

•Reading Comprehension‐Extended

•Reading Rate•Math ProblemSolving

•Basic Writing Skills•Academic Knowledge

• Includes 8 tests for evaluating different aspects of reading– Letter‐Word Identification, Passage Comprehension, Word Attack, Sentence Reading Fluency, Reading Vocabulary

– Oral Reading, Reading Recall, Word Reading Fluency

• Provides 7 clusters for a comprehensive evaluation of reading performance• Reading, Broad Reading, Basic Reading Skills, Reading Comprehension

• Reading Fluency, Reading Rate, Reading Comprehension‐Extended

Summary: Reading

3 new tests

3 new clusters

6/3/2014

29

• Includes 4 tests measuring various aspects of math achievement:– Applied Problems, Calculation, Math Facts Fluency, and Number Matrices

• Provides 4 clusters for a comprehensive evaluation of math performance:– Mathematics, Broad Mathematics, Basic Math Skills, Math Problem Solving

Mathematics: Summary

• Includes 5 tests measuring various aspects of written language:– Spelling, Writing Samples, Sentence Writing Fluency, Editing, and Spelling of Sounds

• Provides 4 clusters for a comprehensive evaluation of writing performance:– Written Language, Broad Written Language, Basic Writing Skills, Written Expression

• Contributes to Phoneme‐Grapheme Cluster (Spelling of Sounds + Word Attack)

Written Language: Summary

• Includes 3 tests measuring various aspects of academic knowledge (comprehension‐knowledge [Gc])

• Test 18: Science• Test 19: Social Studies• Test 20: Humanities

• Provides 1 cluster: Academic Knowledge– Serves as the ability score in an ability/achievement comparison procedure

Academic Knowledge

6/3/2014

30

• A cluster that includes tests from different academic domains (reading, writing, math)

• 7 cross‐domain clusters are available– Brief Achievement and Broad Achievement

– Academic Knowledge

– Academic Skills

– Academic Fluency

– Academic Applications

– Phoneme‐Grapheme Knowledge

Cross‐Domain Clusters

Two options

Intra‐Achievement using Tests 1‐6•Test 1. Letter Word Identification• Test 2. Applied Problems•Test 3. Spelling• Test 4. Passage Comprehension• Test 5. Calculation• Test 6. Writing Samples

Intra‐Achievement using 3 clusters: •Academic Skills•Academic Fluency•Academic Applications

Intra‐Achievement Variation Procedures

• Additional tests can be added to procedures

• Compares performance on one test or cluster to average performance on other tests or clusters

• Provides information about examinee’s strengths and weaknesses within academic areas

• Helps pinpoint a pattern of strengths and weaknesses

Intra‐Achievement Variation Procedures

6/3/2014

31

Academic Skills/Academic Fluency/Academic Applications

When ACH Tests 1 through 6 and 9 through 11 are administered, a cross‐domain comparison is made.

• Academic Skills cluster is compared to a predictor score from the Academic Applications and Academic Fluency clusters;

• Academic Applications cluster is compared to Academic Skills and Academic Fluency clusters;

• Academic Fluency cluster is compared to a predictor score from the Academic Skills and Academic Applications clusters.


all technical age groups) for WJ IV

ACH clusters

Reading 0.95Broad Reading 0.97Basic Reading Skills 0.95Reading Comprehension 0.93Reading Comprehension-Extended 0.96Reading Fluency 0.96Reading Rate 0.96Mathematics 0.96Broad Mathematics 0.97Math Calculation Skills 0.97Math Problem Solving 0.95Written Language 0.94Broad Written Language 0.95Basic Writing Skills 0.95Written Expression 0.92Academic Skills 0.97Academic Applications 0.96Academic Fluency 0.97Academic Knowledge 0.95Phoneme-Grapheme Knowledge 0.94Brief Achievement 0.97Broad Achievement 0.99

KTEA‐II KTEA‐II KTEA‐II KTEA‐II KTEA‐II

WJ IV Measures Oral. Lang. Rdg. Math Wr. Lang Comp. Ach.Oral Language ClustersOral Language 0.64Oral Comprehension 0.74Oral Expression 0.64Listening Comprehension 0.74Reading Achievement ClustersReading 0.94Broad Reading 0.92Basic Reading Skills 0.93Reading Comprehension 0.85Reading Fluency 0.86Reading Speed 0.78Math Achievement ClustersMathematics 0.94Broad Mathematics 0.91Math Calculation 0.83Math Problem Solving 0.82Written Language ACH ClustersWritten Language 0.81Broad Writing 0.84Basic Writing Skills 0.91Written Expression 0.65Cross Domain ACH ClustersBrief Achievement 0.88Broad Achievement 0.91

WJ IV ACH OL / KTEA‐II composite select score correlationsAges 8‐12 (n = 49)

6/3/2014

32

KTEA‐II KTEA‐II KTEA‐II KTEA‐II KTEA‐IIOral Wr. Comp.

WJ IV Measures Lang. Rdg. Math Lang Ach.Oral Language ClustersOral Language 0.74Oral Expression 0.64Listening Comprehension 0.58Reading Achievement ClustersReading 0.83Broad Reading 0.78Basic Reading Skills 0.81Reading Comprehension 0.68Reading Fluency 0.79Reading Speed 0.65Math Achievement ClustersMath 0.87Broad Math 0.86Math Calculation 0.82Math Problem Solving 0.69Written Language ACH ClustersWritten Language 0.87

Broad Writing 0.85

Basic Writing Skills 0.89

Written Expression 0.66

Cross Domain ACH ClustersBrief Achievement 0.91Broad Achievement 0.88

WJ IV ACH OL / KTEA‐II composite select score correlationsAges 13‐18 (n = 50)

WJ IV ACH OL /WIAT‐III composite select score correlations– Grades 1‐8 (n=51)

WIAT‐III WIAT‐III WIAT‐III WIAT‐III WIAT‐III WIAT‐III WIAT‐III WIAT‐IIIOral Total Basic Rdg. Cmp. Math Wr. Total

WJ IV Measures Lang. Rdg. Rdg. & Flu. Math. Flu. Expr. Ach.Oral Language ClustersOral Language 0.82Oral Expression 0.57Listening Comprehension 0.60Reading Achievement ClustersReading 0.93 0.92 0.82Broad Reading 0.89 0.87 0.82Basic Reading Skills 0.93 0.94 0.78Reading Comprehension 0.88 0.85 0.81Reading Fluency 0.87 0.84 0.80Reading Speed 0.67 0.62 0.64Math Achievement ClustersMath 0.92 0.57Broad Math 0.90 0.79Math Calculation 0.82 0.85Math Problem Solving 0.83 0.53Written Language ACH ClustersWritten Language 0.78Broad Writing 0.77Basic Writing Skills 0.79Written Expression 0.62Cross Domain ACH ClustersBrief Achievement 0.93Broad Achievement 0.90Bold font designates correlations >= .70

WJ IV ACH OL /WIAT‐III composite select score correlations—Grades 9‐12 (n=49)

WIAT‐III WIAT‐III WIAT‐III WIAT‐III WIAT‐III WIAT‐III WIAT‐III WIAT‐III

Oral Total Basic Rdg. Cmp. Math Wr. Total

WJ IV Measures Lang. Rdg. Rdg. & Flu. Math. Flu. Expr. Ach.Oral Language ClustersOral Language 0.79Oral Expression 0.68Listening Comprehension 0.75Reading Achievement ClustersReading 0.78 0.75 0.74Broad Reading 0.79 0.72 0.80Basic Reading Skills 0.86 0.86 0.74Reading Comprehension 0.54 0.50 0.60Reading Fluency 0.78 0.72 0.77Reading Speed 0.66 0.61 0.72Math Achievement ClustersMath 0.84 0.42Broad Math 0.76 0.68Math Calculation 0.58 0.72Math Problem Solving 0.79 0.45Written Language ACH ClustersWritten Language 0.73Broad Writing 0.76Basic Writing Skills 0.77Written Expression 0.52Cross Domain ACH ClustersBrief Achievement 0.85Broad Achievement 0.85Note. font designates correlations >= .70

6/3/2014

33

Contemporary CHC (Schneider & McGrew, 2012) broad and narrow ability content coverage by WJ‐IV Oral Language



(A3)



(RC)

Reading speed (RS)


English usage (EU)


Writing speed (WS)




Induction (I)

General sequential

reasoning (RG)


Memory span (MS)






Visualization

(Vz)

Visual Memory (MV)

Spatial Scanning

(SS)



(Gq)


Comp ‐Knowledge

(Gc)




Auditory Processing

(Ga)

Processing

Speed (Gs)


Number Facility (N)




(Gkn)

Geography ach. (A5)

Shading designates abilities measured by WJ IV OL battery


(LD)



(UM)

Word Fluency (FW)


ST Working

Mem (Gwm)


Median reliabilities (acrossall technical age groups) for

WJ IV OL clusters

Oral Language 0.90Broad Oral Language 0.92Oral Expression 0.89Listening Comprehension 0.90Phonetic Coding 0.95Speed of Lexical Access 0.89Vocabulary 0.93

CELF‐4CELF‐4CELF‐4 CELF‐4 CELF‐4 CELF‐4 PPVT‐4WJ IV Measures Core Rec. Expr. Lang. Lang. Work.

Lang. Lang Lang. Con. Struc. Mem.Oral Language Clusters

Oral Language 0.63 0.59 0.64 0.67 0.67 0.50 0.74Oral Expression 0.74 0.64 0.72 0.67 0.73 0.58 0.70Listening Comprehension 0.64 0.62 0.63 0.63 0.68 0.59 0.69Speed of Lexical Access 0.31 0.37 0.32 0.35 0.38 0.57 0.43Bold font correlations > = .60

WJ IV OL / CELF‐4 and PPVT‐4 composite select score

correlations—Ages 5‐8 (n = 50)

6/3/2014

34

CELF‐4 CELF‐4CELF‐4CELF‐4 CELF‐4 CELF‐4 PPVT‐4WJ IV Measures Core Rec. Expr. Lang. Lang. Work.

Lang. Lang Lang. Con. Struc. Mem.Oral Language ClustersOral Language 0.75 0.46 0.69 0.49 0.65 0.17 0.76Oral Expression 0.83 0.56 0.79 0.49 0.74 0.33 0.62Listening Comprehension 0.76 0.50 0.68 0.42 0.69 0.30 0.55Speed of Lexical Access 0.42 0.19 0.36 0.18 0.39 0.23 0.14. Bold font correlations > = .60

WJ IV OL / CELF‐4 and PPVT‐4 composite select score correlations—Ages 10‐18 (n = 56)

CASL OWLS

Core Basic Sent. Syntax Parag. Prag. Oral List. OralWJ IV Measures Comp. Conc. Comp Constr Comp Judge Comp Comp Exp

Oral Language ClustersOral Language 0.60 0.56 0.57 0.47 0.41 0.49 0.60 0.42 0.61Oral Expression 0.48 0.43 0.51 0.42 0.41 0.31 0.46 0.31 0.49Listening Comprehension

0.58 0.60 0.53 0.49 0.44 0.46 0.57 0.44 0.55

Speed of Lexical Access

0.48 0.41 0.36 0.43 0.36 0.40 0.24 0.07 0.33

Note. n = 50 for all measures except WJ IV Phonological Processing (n = 48). All correlations with WJ IV Phonological Processing have n = 48 sample size.

WJ IV OL / CASL & OWLS composite score correlations in ages 3‐6 sample

WJ IV Oral Language ClustersOral Oral Oral Listening Speed of

CASL/OWLS Measures Language Comp. Expression Comp. Lex. Acc.CASL MeasuresCore Composite 0.85 0.82 0.72 0.76 0.57Antonyms 0.82 0.78 0.64 0.75 0.39Synonyms 0.71 0.73 0.55 0.70 0.32Sentence Completion 0.76 0.73 0.66 0.68 0.51Idiomatic Language 0.82 0.80 0.75 0.72 0.42Syntax Construction 0.71 0.68 0.57 0.64 0.47Paragraph Comprehension 0.66 0.70 0.54 0.75 0.56Grammitcal Morphemes 0.63 0.60 0.53 0.56 0.47Sentence Comprehension 0.59 0.60 0.58 0.57 0.47Grammaticality Judgement 0.67 0.65 0.70 0.58 0.57Nonliteral Language 0.67 0.63 0.64 0.54 0.40Meaning from Context 0.77 0.74 0.63 0.67 0.56Inference 0.70 0.70 0.62 0.67 0.42Amibgious Sentences 0.72 0.73 0.58 0.70 0.45Pragmatic Judgement 0.52 0.47 0.34 0.42 0.48

OWLS MeasuresOral Composite 0.68 0.68 0.62 0.64 0.41Listening Comprehension 0.53 0.57 0.50 0.56 0.25Oral Expression 0.67 0.64 0.60 0.59 0.45

Bold font are correlations >= .70

WJ IV OL / CASL & OWLS score correlations in ages 7‐17 sample (n = 50)

KMcGrew Austin2 WJIV handouts...

Documents

Transcript of KMcGrew Austin2 WJIV handouts...