How much power and speed is measured in this test?

31/10/2012

Introduction

• Speed and power are two basic concepts to be considered for cognitive tests.

• Speed can be defined in various ways– Speed tests of a rather high complexity, tests with

easy tasks, etc.

• From an analysis of correlations and from factor analysis results, it was concluded that speed is different from power.

• The topic of speed versus power has seen a revival in more recent research on the nature of intelligence.– whether speed of information processing (SIP)

contributes to what is called psychometric intelligence

• The correlation between speed and intelligence is in the range of 0.30 - 0.40.

• Speeded intelligence tests correlate more highly with SIP than do non-speeded tests.

Approaches for speed

• Response time data are more difficult to model.– Exclusively– Concurrently but separately from responses and

accuracy– Concurrently but jointly

• To define speed– Response time or latency

• Alternative for response time

The notions of speed and power

• Power and speed cannot be directly observed and need to be derived from observables, and they are therefore latent variables.– Power is the power to overcome something,

speed is the time to cover a distance, and so a latent counterpart from the item side is required.

• Speed and power can easily vary for the same task and for the same person.– Stress, motivation, & strategy

A pragmatic approach

• Speed and power are understood as the level of power and speed as used in the test under consideration, which may be influenced by other factors.

• Coded responses instead of response times as data– Speed: whether or not a person responds within a

stringent posterior time limit– Power: whether or not the response is correct given a

lenient posterior time limit

• Posterior time limits– Various time cut-off

• Only responses preceding the cut-off are counted as successes (if correct), and slower responses are considered either as failures or as missing.

• Advantages and disadvantages of using posterior time limits

Kinds of data

• Binary time data (ti)• Binary time-accuracy data (ta)• Incomplete binary accuracy data (ac)• Anchors for speed and power

Research Issues

• The main research issue is how much speed and power is measured by a test.– If speed and power are highly related…– If speed and power are not highly related…

• A second research issue concerns the reliability of the measurements obtained through the posterior time limits for the various kinds of data.

Methods

• A CAT developed by Hornke (verbal analogy)• First test form: 24 items– 180s per item

• Posterior time limits: 90%, 80%, … , 30%• Anchors:– original response data (= ac100 = ta100) for power– 1/RT and ti30 for speed

• 2PLM

• Reliability: Equation 2• Correlation and PCA• A speededness coefficient

• Analysis of item parameter estimates– Mixed model (multilevel analysis)

Figure 1

Figure 2

Figure 3

Discussion

• From a fine-grained view on the issue of power and speed, it was found that speed and power seem to be more or less uncorrelated (in a verbal analogy test).

• Review Figure 2 & 4• In sum, should test time be included in a test

or not?