1801

Interobserver Reliability of a ClinicalClassification of Acute Cerebral Infarction

Richard I. Lindley, MRCP (UK); Charles P. Warlow, MD; Joanna M. Wardlaw, FRCR;Martin S. Dennis, MD; Jim Slattery, MSc; Peter A. G. Sandercock, DM

Background and Purpose: The Oxfordshire Community Stroke Project (OCSP) clinical classification ofsubtypes of cerebral infarction (total and partial anterior circulation infarction, lacunar infarction, andposterior circulation infarction) can be used to predict early mortality, functional outcome, and whetherthe infarct was likely due to large- or small-vessel occlusion. The OCSP classification was originallydeveloped and tested by neurologists as part of a community-based study of first-ever stroke, in whichsome cases were seen after the acute phase. We examined the interobserver reliability of the classificationwhen used in everyday clinical practice in patients seen during the acute phase of stroke shortly afteradmission to the hospital.

Methods: Two clinicians independently assessed consecutive patients admitted to the hospital with an

acute stroke and recorded both the neurological features and their opinion of the subtype of infarct.Results: Eighty-five patients were assessed. Interobserver agreement for the classification was moderate

to good (c=0.54; 95% confidence interval, 0.39 to 0.68). Differences in the assessment of the commonlyelicited neurological signs explained many of the disagreements: interobserver agreement was good forsome signs (hemiparesis [K=0.77], dysphasia [Kc=0.70]), moderate for some (hemianopia [JC=0.39]), andpoor for others (sensory loss [Kc=0.15]).

Conclusions: The classification was simple and practicable (and could be widely used in routine clinicalpractice, randomized controlled trials, and audit), and interobserver reliability was satisfactory. (Stroke.1993;24:1801-1804.)KEY WoRDs * cerebral infarction * interobserver agreement * prognosis * stroke classification

C erebral infarction is heterogeneous. Physicianswho treat patients with stroke are increasinglyseeking a relatively straightforward classifica-

tion of clinical subtypes of infarction that can be appliedin everyday clinical and research practice. The utility ofany classification is all the greater if it can identifypatients with good and with poor prognosis and patientslikely to have infarcts of a particular underlying patho-physiology. Two classifications that focus mainly on thepresumed underlying mechanism and require detailedinvestigation have been reported recently.1'2 The firstapproach relies heavily on diagnostic technology: com-puted tomographic (CT) scanning, magnetic resonanceimaging (MRI), transcranial Doppler, angiography, andechocardiography. This approach is also expensive andtime consuming and can be difficult to apply uniformlyacross patients in different hospitals with differing diag-nostic facilities. Moreover, these etiology-based classi-fications do not give an estimate of prognosis and, even

Received March 24, 1993; final revision received September 2,1993; accepted September 4, 1993.From the Department of Clinical Neurosciences, University of

Edinburgh, Western General Hospital (R.I.L., C.P.W., M.S.D.,J.S., P.A.G.S.), Edinburgh, and the Department of Clinical Neu-roradiology, Institute of Neurological Sciences, Southern GeneralHospital, Glasgow (J.M.W.), UK.

Correspondence to Dr Peter A.G. Sandercock, Department ofClinical Neurosciences, University of Edinburgh, NeurosciencesTrials Office, Bramwell Dott Building, Western General Hospital,Crewe Road, Edinburgh EH4 2XU, UK.

with meticulous attention to detail, still leave approxi-mately 20% to 40% of infarcts classified as of "unknowncause."'1,2A second approach based on the clinical assessment

of the patients (in whom CT has excluded hemorrhage)has been described by Bamford et al.3 This classificationwas developed during the Oxfordshire CommunityStroke Project (OCSP) and defines four subtypes ofcerebral infarction. This approach has the advantagethat it can immediately classify patients at a time whenboth CT and even MRI are still normal. Virtually allpatients can be classified, there is no need to standard-ize the imaging technology used, and each subtype hasa relatively predictable prognosis.3 The classificationwas developed during a community project in whichsome patients were not admitted to the hospital andsome patients were not seen until after the acute phaseof their stroke. If the classification is to be useful innormal hospital clinical practice, it is particularly impor-tant to establish its reliability for patients seen in theacute phase of stroke shortly after admission. There-fore, we set out to measure the interobserver reliability(and, in passing, its practicability) of the classification inthis setting.

Subjects and MethodsAll patients admitted to our hospital with a clinical

diagnosis of suspected acute stroke (World HealthOrganization definition)4 who could be assessed inde-pendently by two of us within 10 days of the onset of

by guest on June 11, 2018http://stroke.ahajournals.org/

Dow

nloaded from

1802 Stroke Vol 24, No 12 December 1993

their symptoms were eligible. We excluded patients whohad a clinical syndrome suggestive of subarachnoidhemorrhage and had no focal neurological deficit. Pa-tients with previous stroke were eligible if they hadmade a complete recovery before the most recent event.At the start of the study, observer 1 had 3 years ofpostregistration general medical training (including 6months of neurology) and observer 2 had 10 years ofgeneral medical training (including 3 years of neurolo-gy). We did not routinely use the hospital case records,but we sometimes referred to these to confirm details ofthe patient's history if no witness was available forinterview at the time of the assessment. During thisstudy, the medical staff responsible for the clinical careof patients did not use the OCSP classification. For eachpatient, we recorded the name, date of birth, time of theexamination, and the OCSP classification. If uncertainof the classification, we recorded our uncertainty to-gether with the "best guess" classification (eg, "uncer-tain, possibly lacunar syndrome"). We recorded thepresence or absence of the key neurological signs toenable us to explore possible reasons for any subsequentdisagreement. As far as possible, we classified patientswithout reference to the results of imaging investiga-tions; occasionally, through inadvertent disclosure byother clinicians unaware of the study, we were told theresults of investigations, and we recorded the investiga-tion and result known if this occurred. We placedcompleted study forms in a sealed container, which wasonly opened when recruitment was complete. Thechoice of investigations was at the discretion of theattending physician and was not influenced by our studyprotocol. A neuroradiologist (J.W.) reviewed all theavailable CT scans at the end of the study "blind" to theclinical data, recording the presence or absence ofintracerebral or subarachnoid hemorrhage.The interobserver reliability of the classification and

the recorded neurological signs were described usingunweighted K statistics for each pair of observations.5 K

statistics represent the degree of agreement over andabove that expected by chance. A K of 0 represents thedegree of agreement expected by chance alone; a valueof 1 indicates perfect agreement. It has been suggestedthat a K of <0.2 represents poor agreement; 0.21 to 0.4,fair agreement; 0.41 to 0.60, moderate agreement; 0.61to 0.80, good agreement; and 0.81 to 1.00, excellentagreement.5

All data were entered onto a dBase IV computer database (Borland International), and contingency tableswere generated using sPsspc (SPSS Inc).The definitions of the four OCSP subtypes are de-

scribed in detail elsewhere and are available from theauthors on request.3 The subtypes are as follows: totalanterior circulation infarct, partial anterior circulationinfarct, lacunar infarct, and posterior circulation infarct.When intracerebral hemorrhage has not been excluded(eg, by early CT scanning), the subtypes are referred toas syndromes: total anterior circulation syndrome, par-tial anterior circulation syndrome, lacunar syndrome,and posterior circulation syndrome.

ResultsWe assessed 90 patients between December 1990 and

April 1991. Two patients were excluded because theirfinal diagnosis was not stroke, and three patients were

TABLE 1. Overall Interobserver Agreement (Includingan "Uncertain" Category) of the OxfordshireCommunity Stroke Project Classification of Stroke

Observer 2Classifi-cation TACS PACS LACS POCS Uncertain Total

Observer 1

TACS 11 4 0 0 2 17PACS 3 19 1 1 3 27

LACS 1 3 7 0 2 13POCS 0 1 1 5 2 9Uncertain 6 4 2 1 6 19

Total 21 31 1 1 7 15 85

TACS indicates total anterior circulation syndrome; PACS,partial anterior circulation syndrome; LACS, lacunar syndrome;and POCS, posterior circulation syndrome.The results indicate the overall interobserver agreement of the

Oxfordshire Community Stroke Project classification for all eligi-ble patients when an "uncertain" category was included (un-weighted K=0.43; 95% confidence interval, 0.30 to 0.57).

discharged before the second clinician could assessthem, leaving a total of 85 patients. Forty-nine (58%) ofthe patients were examined first by observer 2, and 36(42%) of the patients were examined first by observer 1.The mean time from onset of the patients' symptoms ofstroke to examination by the first observer was 2 days(median, 1 day). The mean time between assessmentswas 1.5 days (median, same day).The observers agreed on the classification in 48

(56%) of the 85 patients when the "uncertain" categorywas included in the analysis (Table 1), corresponding toa K of 0.43 (95% confidence interval [CI], 0.30 to 0.57).Removing either the patients with hemorrhagic stroke(K=0.41) or patients in whom the assessments weremore than 2 days apart (K=0.44) did not significantlyalter the agreement. The observers agreed on theclassification in 42 (74%) of the 57 patients in whomboth observers were "certain" of the classification,corresponding to a K of 0.62 (95% CI, 0.45 to 0.78).A "best guess" classification for all those initially

coded as "uncertain" was available for 27 pairs ofobservations. When these were included with the "cer-tain" group, the observers agreed in 57 (68%) of the 84patients, corresponding to a K of 0.54 (95% CI, 0.39 to0.68) (Table 2). The results were similar when weexcluded the 14 with known hemorrhagic stroke (70patients; K=0.59; 95% CI, 0.43 to 0.74), 10 seen >2 daysapart (74 patients; K=0.54; 95% CI, 0.39 to 0.69), and 18in whom one or both observers knew the results of anyimaging such as CT scanning or Doppler ultrasound (66patients; K=0.59; 95% CI, 0.43 to 0.74).The interobserver reliability for each of the neuro-

logical signs is shown in Table 3. In a few categories(sensory loss in face and leg), a K value could not becalculated because of missing values by one (or both)observers.

Reasons for DisagreementThe observers disagreed on the classification in 27

(32%) of the 84 patients (using the "best guess" codingfor those initially coded as "uncertain"). Disagreement

by guest on June 11, 2018http://stroke.ahajournals.org/

Dow

nloaded from

Lindley et al Interobserver Reliability of Stroke Classification 1803

TABLE 2. Overall Interobserver Agreement of theOxfordshire Community Stroke Project Classification ofStroke Recording "Uncertain" Cases According to"Best Guess" of Subtype

Observer 2

Classification TACS PACS LACS POCS Total

Observer 1

TACS 16 4 0 2 22

PACS 5 26 5 2 38

LACS 1 4 8 0 13

POCS 2 1 1 7 11

Total 24 35 14 1 1 84

TACS indicates total anterior circulation syndrome; PACS,partial anterior circulation syndrome; LACS, lacunar syndrome;and POCS, posterior circulation syndrome.The results indicate the overall interobserver agreement of the

Oxfordshire Community Stroke Project classification for all eligiblepatients. The "best guess" category was used for all those initiallyclassified as "uncertain" (unweighted K=0.54; 95% confidenceinterval, 0.39 to 0.68). (Note that there is a total of 84 patientsbecause one observer still coded one stroke as "uncertain.")

arose because of differences in the neurological signselicited by the two observers in 10 patients (the signcausing the disagreement being hemianopia in three,dysphasia in four, visuospatial dysfunction in two, andcerebellar signs in one), the assessment of the presenceor absence of confusion in four, the presence of deepcoma in three, and an inadequate clinical history in five.In five patients, both observers agreed on the neurolog-ical signs but disagreed on the classification.

DiscussionIn this group of patients admitted to the hospital

because of acute stroke, the interobserver reliability forthe OCSP subclassification of cerebral infarction was

moderate to good. The observers agreed in the majorityof patients, and the level of agreement did not changewhen patients with known hemorrhagic stroke or pa-

tients with assessments >2 days apart were excluded.The differences in the neurological signs elicited were

an important cause of interobserver disagreement. In a

previous study by Shinar et al,6 the interobserver agree-

ment between staff neurologists with a special interestin stroke was assessed for commonly elicited neurolog-ical signs in 17 patients with stroke. Their results were

remarkably similar to our own (see Table 3). Part of thedisagreement in the present study may have been due todifferences in experience of the examiners, but otherdisagreements are likely to have been due to the tem-poral variation in clinical signs and the practical diffi-culties of examining acutely ill stroke patients who maybe confused, drowsy, or dysphasic. A study assessingonly alert, cooperative, and communicative patientsmight improve the interobserver agreement but wouldhardly be relevant to routine clinical practice.

Five disagreements arose because of differences inthe clinical history. A very important practical point isthat the classification of the ischemic stroke must bebased on the maximum deficit. One of our patients wascoded as a lacunar stroke by one observer and a totalanterior circulation syndrome by the other. The secondobserver was not aware of the patient's status at thetime of maximum deficit (hemiparesis, hemianopia, and

TABLE 3. Interobserver Agreement for Neurological Signs Recorded for Patients WithAcute Stroke in the Present Study and Comparison With Equivalent Scores Obtained inthe Study by Shinar et a16

Present Study (n=85) Shinar et al6 (n=17)

Signs Examined Kc Signs Examined pc

Weak arm 0.77 Not stated ...

Dysphasia 0.70 Language 0.54

Weak hand 0.68 Weakness scale, 0.58, 0.49hand (right, left)

Weak leg 0.64 Not stated

Weak face 0.63 Weakness scale, face 0.51, 0.66(right, left)

Conscious level 0.60 Degree of alertness 0.38

Dysarthria 0.51 Dysarthria 0.53

Cerebellar signs 0.46 Ataxia 0.45

Visuospatial dysfunction 0.44 Not stated

Hemianopia 0.39 Visual fields 0.40

Cranial nerve palsies 0.34 Not stated ...

Extraocular eye movement disorder 0.30 Extraocular 0.77movements

Confusion 0.21 Examiner believes 0.34patient is demented

Sensory loss, hand 0.19 Sensory scale, hand 0.50, 0.32(right, left)

Sensory loss, arm 0.15 Not stated ...

by guest on June 11, 2018http://stroke.ahajournals.org/

Dow

nloaded from

1804 Stroke Vol 24, No 12 December 1993

dysphasia). The patient then improved rapidly withresolution of the hemianopia and dysphasia but was leftwith a residual hemiparesis (mimicking a pure motorstroke) at the time of the assessment by the secondobserver. A witnessed account of the stroke onset wouldhave avoided this disagreement.An important advantage of the OCSP classification is

the ability to classify the patients in whom the CT scanfails to reveal an appropriate ischemic lesion with eitherearly or late scanning. Furthermore, classifications thatare dependent on imaging will depend on the sensitivityof the CT or MRI scanning technology. For example,centers with MRI scanners may be able to reliablyidentify cerebellar infarcts > 1.5 cm in diameter andthus classify the stroke subtype as "large-artery athero-sclerosis" according to the TOAST2 classification. An-other center with an early generation CT scanner maynot be able to reliably detect such a lesion and mayclassify the stroke as a "small-artery occlusion(lacune)."

It is possible that certain treatment strategies foracute ischemic stroke will have important differences inthe balance of risks and benefits for the treatment ofdifferent subtypes of cerebral infarction. For example, itis plausible that thrombolytic therapy may have a muchgreater benefit in patients with large-vessel occlusivestroke compared with those patients with small-vesseldisease (lacunar stroke). Thus, very early identificationof distinct subtypes of cerebral infarction may be animportant part of the management of patients withacute stroke. The OCSP classification is ideal, as thesubtype can be derived in minutes by using only simpleclinical data from a bedside examination. The disadvan-tage of the more technological classifications is thatpatients cannot be classified until the results of detailedand complex investigations are available, which may bemany hours, if not days, after admission; this is clearlynot practicable if therapy needs to be guided by theclassification and then started within 1 to 2 hours ofadmission.We found that the OCSP classification could be used

reliably to categorize most patients with cerebral infarc-tion in routine (as opposed to research) clinical prac-tice. The present study suggests that interobserverreliability may be further improved if the clinical historyincludes a record of the maximum deficit. Examination

needs to focus on just a few key neurological signs. Theclassification, which is anatomically based, can be usedto guide subsequent investigation (as appropriate) toexplore the underlying pathophysiology and thus pro-vides a logical way of managing patients with acutestroke. Indeed, we now routinely use the classification inour own clinical practice. The classification is also beingused to characterize patients entered in the Interna-tional Stroke Trial, a large multicenter randomizedcontrolled trial (personal communication, P. Sander-cock). The classification of important subtypes of cere-bral infarction recognizes the heterogeneous nature ofcerebral ischemia. The early recognition of these sub-types may well be useful in deciding the urgency andneed for subsequent investigation, the identification ofpatients with strokes of a broadly similar pathophysio-logical mechanism, predicting prognosis, audit, and theselection of patients for acute treatment trials. It seemsthat the reliability of the OCSP criteria is comparable tothat of the TOAST classification, but the OCSP criteriahave the advantages of greater speed, lower cost, andeasier standardization between institutions with differ-ing imaging technology.

AcknowledgmentsThis study was supported by grants from The Stroke Asso-

ciation, Lilly Industries, and the Clinical Trial Service Unit,Oxford (Dr Lindley); by The Stroke Association (Dr Dennis);and by the Medical Research Council (Mr Slattery and DrsWardlaw and Sandercock).

References1. Sacco RL, Ellenberg JH, Mohr JP, Tatemichi TK, Hier DB, Price

TR, Wolf PA. Infarets of undetermined cause: the NINCDS strokedata bank. Ann Neurol. 1989;25:382-390.

2. Adams HP, Bendixen BH, Kappelle LI, Biller J, Love BB, GordonDL, Marsh EE, TOAST Investigators. Classification of subtype ofacute ischemic stroke: definitions for use in a multicenter clinicaltrial. Stroke. 1993;24:35-41.

3. Bamford J, Sandercock P, Dennis M, Burn J, Warlow C. Classifi-cation and natural history of clinically identifiable subtypes ofcerebral infarction. Lancet. 1991;337:1521-1526.

4. Hatano S. Experience from a multicentre stroke register: a pre-liminary report. Bull World Health Organ. 1976;54:541-553.

5. Brennan P, Silman A. Statistical methods for assessing observervariability in clinical measures. BMJ. 1992;304:1491-1494.

6. Shinar D, Gross CR, Mohr JP, Caplan LR, Price TR, Wolf PA, HierDB, Kase CS, Fishman IG, Wolf CL, Kunitz SC. Interobservervariability in the assessment of neurologic history and examinationin the stroke data bank. Arch Neurol. 1985;42:557-565.

by guest on June 11, 2018http://stroke.ahajournals.org/

Dow

nloaded from

R I Lindley, C P Warlow, J M Wardlaw, M S Dennis, J Slattery and P A SandercockInterobserver reliability of a clinical classification of acute cerebral infarction.

Print ISSN: 0039-2499. Online ISSN: 1524-4628 Copyright © 1993 American Heart Association, Inc. All rights reserved.

is published by the American Heart Association, 7272 Greenville Avenue, Dallas, TX 75231Stroke doi: 10.1161/01.STR.24.12.1801

1993;24:1801-1804Stroke. 

http://stroke.ahajournals.org/content/24/12/1801World Wide Web at:

The online version of this article, along with updated information and services, is located on the

  http://stroke.ahajournals.org//subscriptions/

is online at: Stroke Information about subscribing to Subscriptions: 

http://www.lww.com/reprints Information about reprints can be found online at: Reprints:

  document. Permissions and Rights Question and Answer process is available in the

Request Permissions in the middle column of the Web page under Services. Further information about thisOnce the online version of the published article for which permission is being requested is located, click

can be obtained via RightsLink, a service of the Copyright Clearance Center, not the Editorial Office.Strokein Requests for permissions to reproduce figures, tables, or portions of articles originally publishedPermissions:

by guest on June 11, 2018http://stroke.ahajournals.org/

Dow

nloaded from