1995 EDO-CG-95-26

Computer-Assisted Testing in Counseling and TherapyJames P. Sampson, Jr.

Test interpretation may be enhanced by providingthe counselor with an expanded and consistent knowl-edge base to assist in the interpretation of test data. Com-puter-based test interpretation (CBTI) is typically basedon research data and clinical experience. Roid andGorsuch (1984) described four approaches to CBTI: 1)descriptive interpretations; 2) clinician-modeled interpre-tations (renowned clinician type); 3) clinician-modeledinterpretations (statistical model type); and 4) clinical ac-tuarial interpretations. Counselors can use CBTI to sup-port or challenge their judgments about the nature of cli-ent problems and potentially effective intervention strat-egies.

Test integration may be enhanced by including com-puter-assisted instruction as part of CAT. Clients can bebetter prepared to use their test results by being moreaware of basic concepts and the general nature of theirscores. Relieved of presenting repetitive test interpreta-tion information, counselors have more time to exploreclients perceptions of their test data and the implicationsof the test data for behavior change. The computer canbe used to deliver both text-based and interactive video-based instruction (Sampson, 1990a).

Potential Limitations of Computer-Assisted Testing

Computer-assisted testing can limit, as well as en-hance, test administration and interpretation. Althoughpaper-and-pencil and computer administration of testsoften produce equivalent results, variations in results havesometimes been found to exist. French (1986) recom-mended that the equivalency of results from differenttypes of administration modes needs to be establishedfor each instrument. Establishing equivalency will reducethe likelihood that computer administration is influenc-ing the nature of test results. Scoring errors represent an-other potential limitation for computer-assisted test ad-ministration. Most (1987) noted that, The computer it-self does not contribute error, but the complex nature ofcomputer programming and the difficulty involved inreading computer programs or code makes it easy to makeprogram errors which are difficult to find (p. 377).

Concerns have been raised about the validity of com-puter-based test interpretation. Eyde and Kowal (1987)found differences in CBTI reports generated from a singleset of scores from one instrument. Differences also werenoted in their study between the CBTI reports and thejudgments of a clinician. Eyde and Kowal (1987) stated,Buyers should be aware of the limitations of computerproducts and remind themselves that computer outputis only as good as the data behind the decision rules usedto produce the interpretation (p. 407). Ethical concernsalso exist about counselor misuse of CBTI. Unqualifiedcounselors may be more likely to use CBTI reports to com-pensate for a lack of training and experience. By using

Computer-assisted testing (CAT) in counseling andtherapy is becoming increasingly common due to dra-matic improvements in cost-effectiveness and increasedcounselor familiarity with computer applications. Theassumption underlying the use of CAT is that the effec-tiveness of counseling is improved by allocating repeti-tive computational and instructional tasks to the com-puter, thus allowing counselors to more fully focus oninterpersonal tasks, such as helping clients understandthe role of testing in counseling and helping clients inte-grate the self-knowledge obtained in testing into a con-crete plan for behavior change. The potential benefits ofCAT, however, need to be evaluated against the potentiallimitations of this technology.

Computer-Assisted Testing Options

The following options exist for using computer-as-sisted testing in counseling and therapy:

1) Test administration via: a) keyboard input by theclient from items presented on the computer display, withalternative input options available for physically disabledclients; or b) client completion of a specially prepared testanswer sheet that is then optically scanned for computerinput; or c) client completion of a traditional test answersheet with keyboard input by a clerical staff person.

2) Test scoring via the computer (local or remote).

3) Test score profile generation via the computer.

4) Narrative interpretive report generation via thecomputer with reports available for both the client andpractitioner if appropriate (the narrative report may alsoinclude the test profile).

5) Videodisc-based generalized test interpretationprovided to the client immediately following test admin-istration (Sampson, 1990a, p 452-453).

Potential Benefits of Computer-Assisted Testing

Computer-assisted testing can enhance test admin-istration, scoring, interpretation, and integration. Test ad-ministration and scoring may be enhanced due to the stan-dardization inherent in computer functioning. Each testtaker receives an identical presentation of test items andresponse sets (with the exception of adaptive testingwhere each test taker receives a unique minimum selec-tion of items necessary to achieve a valid result). Greaterstandardization of item presentation eliminates errorscaused when a test taker gets out of sync between theanswer sheet and a printed test item (Byers, 1981). Theavailability of adaptive devices allows persons with a dis-ability to complete tests with minimal staff assistance(Sampson, 1990b). Test results can be more valid sincestaff members have less of an opportunity to influenceclient responses. Test scoring is enhanced due to reducedcomputational errors.

ERIC Digest

For information on other ERIC/CASS products and services, please call toll-free (800) 414-9769 or (910) 334-4114 or fax (910)334-4116 or write ERIC/CASS, School of Education, University of North Carolina at Greensboro, Greensboro, NC 27412.

CBTI to replace rather than supplement counselor judgement,counselors become more dependent on the potentially dubi-ous validity of some CBTI software and are less likely to inte-grate data from valid CBTI reports effectively with othersources of client data due to their lack of background knowl-edge.

Recommendations

Counselors should become familiar with existing CATapplications (see Krug, 1993; Walz, Bleuer, & Maze, 1989) andthe various professional standards that relate to CAT. Coun-selors then should carefully select and effectively implementvalid software that is subsequently evaluated in terms of ser-vice delivery impact.

Conclusion

The use of CAT can either enhance or limit the effective-ness of testing in counseling and therapy. Having an openmind about the potential of this technology and a willing-ness to change needs to be matched with good critical think-ing skills and a healthy skepticism for any innovation prom-ising substantial benefits from minimal efforts. It is the re-sponsibility of counselors to guide the design and use of thistechnology.

References

Byers, A. P. (1981). Psychological evaluation by means of anon-line computer. Behavior Research Methods & Instrumen-tation, 13, 585-587.

Eyde, L. D., & Kowal, D. M. (1987). Computerised test inter-pretation services: Ethical and professional concerns re-garding U.S. producers and users. Applied Psychology: AnInternational Review, 36(3/4), 401-417.

French, C. F. (1986). Microcomputers and psychometric as-sessment. British Journal of Guidance and counselling, 14,33-45.

Krug, S. E. (1993). PSYCHWARE SOURCEBOOK: A resourceguide to computer based products for assessment in psychol-ogy, education, and business (4th Ed.). Champaign, IL:Metritech.

Most, R. (1987). Levels of error in computerized psychologi-cal inventories. Applied Psychology: An International Re-view, 36(3/4), 375-383.

Roid, G. H., & Gorsuch, R. L. (1984). Development and clini-cal use of test-interpretive programs on microcomput-ers. In M. D. Schwartz (Ed.), Using computers in clinicalpractice: Psychotherapy and mental health applications (pp.141-149). New York: Haworth Press.

Sampson, J. P., Jr. (1990a). Computer applications and issuesin using tests in counseling. In C. E. Watkins, Jr. & V. L.Campbell (Eds.), Testing in counseling practice (pp. 451-474). Hillsdale, NJ: Lawrence Erlbaum Associates, Inc.

Sampson, J. P., Jr. (1990b). Computer-assisted testing and thegoals of counseling psychology. The Counseling Psycholo-gist, 18, 227-239.

Walz, G. R., Bleuer, J. C., & Maze, M. (Eds). (1989). Counselingsoftware guide: A resource for the guidance and human devel-opment professions. Alexandria, VA: American Associationfor Counseling and Development.

James P. Sampson, Jr. is a professor in the Department of HumanResources and Studies and Co-Director of the Center for the Studyof Technology in Counseling and Career Development at FloridaState University in Tallahassee, Florida.