Kevin McGrew, Ph.D.Kevin McGrew, Ph.D.

Director, IAPDirector, IAPInstitute for Applied PsychometricsInstitute for Applied Psychometrics

Visiting Professor, Ed. Psych.Visiting Professor, Ed. Psych.University of MinnesotaUniversity of Minnesota

Beyond the CHC Beyond the CHC “Tipping Point”“Tipping Point” – – Back to the FutureBack to the Future

Cattell-Horn-Carroll (CHC) Theory of Cognitive Abilities:

Historical Context and Development Within “Waves Of Intelligence Test Interpretation”

In sociology, a tipping point or angle of repose is the event of a previously rare phenomenon becoming rapidly and dramatically more common

Tipping Point

There is only one proven “law” in psychology

The law of individual differences

On Intelligence...(Neisser et al., 1996)

• Individual’s differ in their ability to understand complex ideas, to adapt effectively to the environment, to learn from experience, to engage in various forms of reasoning, etc.

• Concepts of intelligence are attempts to clarify this complex set of phenomena (p. 77).

The Need For A Scientific Foundation For The Development And Interpretation Of

Cognitive Ability Tests

“Useful technology is based on scientificunderstanding; the better the science, themore effective the technology can be. Soit is with psychological tests. They shouldbe based on the most dependable andcurrent evidence of science” (Horn, 1991)

Three Primary Research Traditions

•Psychometric

•Information Processing

•Dynamic

On Intelligence...(Neisser et al., 1996)

• The psychometric approach is the oldest and best established, but others also have much to contribute.

• We should be open to the possibility that our understanding of intelligence in the future will be rather different from what it is today (p.80).

Importance Of Classification Taxonomies In All Sciences

Classification is arguably one of the most central and generic of all our conceptual exercises…without classification, there could be no advanced conceptualization, reasoning, language, data analysis, or for that matter, social science research (K.D. Bailey, 1994).

A specialized science of classification of empirical entities known as taxonomy (Bailey, 1994; Prentky, 1994) is ubiquitous in all fields of study because it guides our search for information or truth.

Importance Of Classification Taxonomies: Two Examples

Linnean hierarchical taxonomy

In psychology, we have have been searching for decades for an empirically and theoretically grounded taxonomy of cognitive abilities for the interpretation of

the reliable variance of tests

(Rectangle represents the total variance of a single cognitive ability test)

Unique abilities (variance) not shared in common with other tests (specificity)

Error variance

Reliable variance(reliability)

Error variance-individual/situational variables (e.g., distractibility)-item variables (e.g., item sampling and item gradients; test floor and ceiling)-examiner variables (e.g., rapport, scoring and administration errors)-testing environment variables (e.g., noise, comfort)

Waves Of Intelligence Test Waves Of Intelligence Test InterpretationInterpretation

(Kamphaus et al., 1997, 2005)(Kamphaus et al., 1997, 2005)

• Wave 1 – Quantification of a General Level (g)

• Wave 2 – Clinical Profile Analysis

• Wave 3 - Psychometric Profile Analysis

• Wave 4 – Applying Theory to Test Interpretation

• Wave 5 – Greater emphasis on content validity

McGrew’s Wave 5 Speculation

• Wave 1 – Quantification of a General Level (g)

• Wave 2 – Clinical Profile Analysis

• Wave 3 - Psychometric Profile Analysis

• Wave 4 – Applying Theory to Intelligence Test Interpretation

• Wave 5 – Application of computer technology & new statistical & measurement methods/software to theory-based test methods (Wave 4) + integration of

psychometric and information processing research and theories may now allow us to realize the promise(s) of prior waves

Classification of individuals into groups based on IQ (g) scores

• Classification schemes based on global IQ scores

Wave Wave 1 - Quantification Of 1 - Quantification Of A General Level (A General Level (gg))

Spearman’s g- theory was major theoretical influence

Wave Wave 1 - Quantification Of A 1 - Quantification Of A General Level (General Level (gg))

Wave Wave 1 - Quantification Of A 1 - Quantification Of A General Level (General Level (gg))

Binet’s intelligence test was primary measurement breakthrough (provided

global g-type score)

Reliable variance (reliability)

s

g

Wave Wave 1 - Quantification Of A 1 - Quantification Of A General Level (General Level (gg))

Primary test interpretation was focused onvariance associated with g (general intelligence)

Wave 2Wave 2 - Clinical Profile - Clinical Profile AnalysesAnalyses

Rapaport, D., Gill, M. M., & Schafer, R. (1945–1946). Diagnostic psychological testing. (2 vols.). Chicago: Year Book

Introduced Clinical Subtest Profile and Test Item Level Analyses

•Was found to be seriously wanting based on empirical research

Wave 2 to 3Wave 2 to 3 – Advances in Structural – Advances in Structural Theories of Intelligence Had Theories of Intelligence Had Little Little InfluenceInfluence on Test Interpretation on Test Interpretation

•Factor-analytic research resulted in significant advances in understanding the structure of intelligence

•These studies did not immediately “cross-over” to influence the applied practice of intelligence test development and interpretation

•The most influential structural advances during this time included the following:

Cattell’s Fluid (Gf) and Crystallized(Gc) Theory of Intelligence (circa 1941)

Did not immediately result in the development of a clinical/applied measure of intelligence

•The ETS Factor Reference Group Made Significant Gains in Delineating the Structure of Human Cognitive Abilities

•WERCOF Abilities (Well Replicated Common Factors)

•Kit of Factor-Referenced Tests

•Goal was to identify/catalogue human cognitive abilities and to provide ability/factor “reference” or “marker” tests

•Cattell/Carroll APA 1957 communication (click here)

Thurstone’s Primary Mental Abilitiesmodel was influential (circa 1955)

Shifted focus away from g to primary (WERCOF) mental abilities

Structure-Of-Intellect(SOI Model)

• The SOI model lacks solid empirical evidence (Carroll, 1993; Cronbach & Snow, 1977; Gustafsson & Undheim, 1996; Messick, 1992; Vernon, 1961).

•The SOI model “is fundamentally defective” (Carroll, 1993; p. 59) and should be “marked down as a somewhat eccentric aberration in the history of intelligence models; that so much attention has been paid to it is disturbing, to the extent that textbooks and other treatments of it have given the impression that the model is valid and widely accepted, when clearly it is not” (Carroll, 1993; p. 60).

Wave 2 to 3Wave 2 to 3 – Advances in Applied – Advances in Applied Measures of IntelligenceMeasures of Intelligence

•David Wechsler developed and presented the first practical dichotomous practical/clinical intelligence test battery (1939)

An Important Point Regarding Wechsler’s Original Dichotomous Battery Structure

• The Wechsler scales were designed to assess two different ways intelligence can be expressed (i.e., two different languages). Wechsler did not consider the Verbal and Performance scales to represent two different types of intelligence

– (Kamphaus; 1993; Reynolds & Kamphaus, 1990; Zachary, 1990)

• There is no such thing as nonverbal abilities, and no empirically supported verbal/nonverbal theories of intelligence (Kamphaus, 1993)

• Although making a huge practical contribution, the Wechsler battery structure had the indirect and unfortunate effect of misdirecting our understanding and interpretation of intelligence test performance

Wave 3Wave 3 – Psychometric Profile – Psychometric Profile AnalysesAnalyses

Cohen, J. (1959). The factorial structure of the WISC at ages 7-6, 10-6, and 13-6, Journal of Consulting Psychology, 23, 285-299.

Kaufman, A. S. (1979). Intelligent testing with the WISC-R. New York: Wiley-Interscience.

Kaufman’s Wechsler-based psychometric (“Intelligent Testing”) approach contributed to the movement (in test

interpretation) to beyond g

Wave 3Wave 3 – Psychometric Profile Analyses – Psychometric Profile Analyses

Unique abilities not shared in common with other CHC factor indicators (specificity)

Reliable variance (reliability)

g

POVC

Secondary ability

Primary ability

Error variance

VC = Verbal ComprehensionPO = Perceptual Organization

VCI/POI/FFD/PS

Evaluating Existing Tests

Test Interpretation Individual DifferencesResearch (e.g., ATI)

Designing New Tests

Communication(standard nomenclature)

Outcomes Research

An unfortunate impediment to the narrowing the intelligence theory/measurement gap was the development and influence of a

Wechsler-centric philosophy of test interpretation

Ipsative interpretation

………the process of generating strength and weakness hypotheses about cognitive abilities based on an analysis of an individual’s subtest scores that deviate significantly (either in a positive or negative direction) from the average (mean) of all the subtest scores on the intelligence battery

Wave 2 & 3 both focused on Wave 2 & 3 both focused on Ipsative Profile AnalysisIpsative Profile Analysis

Ipsative Profile AnalysisIpsative Profile Analysis

Average(mean)

+ 1 SD

- 1 SD

Strength

Weaknesses

Generate hypotheses based on patterns of strengths and weaknesses among subtests

Despite its popularity and intuitive appeal,

ipsative analysis has significant limitations

which have led to considerable criticism of the

approach

““Just Say No To Subtest Just Say No To Subtest Analysis”Analysis”

According to academicschool psychology,you should feel guiltyand ashamed if yougenerate hypothesesbased on test profiles

Wave 2 & 3 Contributed to Wave 2 & 3 Contributed to Attempts to Measure “Differential Attempts to Measure “Differential

Aptitudes”Aptitudes”

1

Apt 1

Apt 2

Apt 3

IQ(g)

1

IQ(g)

Criterion

After g was accounted for In the prediction……..

Apt 3

….measures of specialized or differential abilities contributed little or nothing to the prediction

But, research found that……….

McNemar, Q. (1964). Lost: Our intelligence: Why? American Psychologist, 19, 871-882

The search to develop differential predictors of academic performance was

deemed a failure

Therefore…long live full-scale g-scores !!

“Just Say Maybe” (McGrew et al., 1997)

Improved Theory

• Cattell-Horn-Carroll (aka. Gf-Gc) Theory

Improved Theory-Based Operational Measurement

• WJ-R and WJ-III• CHC Cross-Battery

““Just say no & maybe” research differs in Just say no & maybe” research differs in the the construct validityconstruct validity of the measures used of the measures used

CHC focus on clusters instead of individual tests:

–Increased reliability

–Increased construct validity

• Reduction of construct irrelevant variance

• Better construct representation

“Just say no” focused on individual tests

“Just say maybe” focuseson combinations (clusters) of tests

The failure to find that specific abilities add anything to the prediction of achievement beyond that already provided by a g score may be a correct interpretation for the set of constructs measured by the Wechsler batteries, but may be a premature generalization to apply to all intelligence batteries

(McGrew, Flanagan, Keith, Vanderwood, 1997)

“ “ Just say maybe”Just say maybe”

Wave 4+5Wave 4+5

So…..get ready, buckle up and………

Wave 4 – Applying Contemporary Intelligence

Theories to Test Interpretation (and research & development)

Wave 4 – The placement of empirically-grounded theories of intelligence in the center of test development and interpretation – a

Theory-centric philosophy and approach to test interpretation

VCI/POI/FFD/PS

Evaluating Existing Tests

Test Interpretation Individual DifferencesResearch (e.g., ATI)

Designing New Tests

Communication(standard nomenclature)

Outcomes Research

Wave 4 – Applying Contemporary Intelligence Theories to Test Interpretation: Two Primary Theories

Luria(Sim-Succ)

Cattell(Gf-Gc)

Primary Theorists & Researchers

Horn(Expanded Gf-Gc)

Carroll (Gf-GcThree-Stratum)

Das(PASS)

CHCTheory

Luria-DasTheory

Resulting AssessmentBatteries and Approaches

• K-ABC• CAS• K-ABC II

• WJ-R• WJ-III• KAIT• CHC Cross- Battery• SB5• K-ABC II• DAS-II

In sociology, a tipping point or angle of repose is the event of a previously rare phenomenon becoming rapidly and dramatically more common

Tipping Point

“It is this author’s personal opinion, that this moment, a moment where the interests and wisdom of a leading applied test developer (Woodcock), the leading proponent of Cattell-Horn Gf-Gc theory (Horn), and one of the preeminent educational psychologists and scholars of the factor analysis of human abilities (Carroll) intersected (see section C in Table 1), was the flash point that resulted in all subsequent theory-to--practice bridging events that led to today’s CHC theory and related assessment developments.  A fortuitous set of events had resulted in the psychometric stars aligning themselves in perfect position to lead the way for most all subsequent CHC assessment related developments.” (McGrew, 2004, 2005)

The fortuitous March, 1986 “meeting of the minds” – the CHCIntelligence-to-Theory “flash point”

1998 1999 2000 2001 2002 2003 2004 2005 2006 2007

Year

0

100

200

300

400

500

600

700

Fre

q. o

f te

rms

in N

AS

P li

stse

rv m

essa

ges

WJ

CHC

Gf-Gc

CHC designed WJ III battery published

Carroll/Horn informally agree to CHC umbrella term

2002 2003 2004 2005 2006 2007Year

400

500

600

700

800

900C

HC

list

serv

cum

mul

ati v

e n

Founded July 8, 2001

Current n (11-5-07) = 956

The CHC “Theory-to-Practice” Tipping Point has occurred – sometime between

2001 and 2003

CHC Tipping Point

So……what!!!!!

Why is thisrelevant to me???

Three effects of intelligence batteries being based on confluence of research and theory – Wave 4

(Kamphaus et al. 2005)

• Test-specific training is of less value

•After a psychologist knows the theories….he or she can interpret most modern intelligence tests with confidence

• “CHC-as-a-second-language”

• It is now more important for a clinician to understand the constructs of intelligence, as opposed to receiving specific “Wechsler” or “Binet” training

• Pre- and post-professional training priority shifts to sufficient knowledge of theories of intelligence that inform modern test construction and interpretation

• As intelligence tests seek to measure similar core constructs, they increasingly resemble commodities

Three effects of intelligence batteries being based on confluence of research and theory – Wave 4

(Kamphaus et al. 2005)

Among contemporary intelligence tests, the Woodcock-Johnson III…is the instrument most closely aligned with the Cattell-Horn…and Carroll theories of intelligence.

CHC theory … served as the blueprint for the WJ III

(Kamphaus et al., 2005)

Wave 5 – (stay tuned –Part II of today’s presentation)