Introduction

The severity of acute myocardial infarction (AMI) andits associated mortality have been evaluated using spe-cific indices for this condition [1]. However, there arereports that general severity indices may be equally use-ful [2–4]. The main advantage of general indices in mul-tidisciplinary intensive care units (ICUs), where pa-tients with AMI are also treated, is that all patients canbe evaluated with just one index.

Treatment of AMI in Spain normally takes place inICUs. Most Spanish ICUs are in public hospitals, aremedical-surgical units, and include patients with coro-nary heart disease. There is a specialist in intensive med-icine in continuous attendance 24 h/day, the bed/nurseratio is 3 : 1, and the mean number of beds/unit is 12

(range 5–40). The ICUs in hospitals without facilitiesfor angiographic studies or cardiac surgery transfer pa-tients to reference centers, normally hospitals with> 600 beds. Fibrinolysis is possible in all the ICUs but isonly exceptionally performed in an extrahospital set-ting. AMI and unstable angina cases constitute about35% of total ICU admissions, and around 75% of thisgroup suffer from AMI.

The most acceptable general indices are the AcutePhysiology, Age and Chronic Health Evaluation (APA-CHE) III [5] and the Simplified Acute Physiology Score(SAPS) II [6]. Both are valid for any kind of critical pa-tient. Although patients with AMI were not included inthe development of the SAPS system, a study compar-ing previous versions of these indices in 77 patientswith AMI found that both indices discriminated in-hos-

A. ReinaG.VazquezE.AguayoI.BravoM. ColmeneroM. BravoPAEEC Group

Mortality discriminationin acute myocardial infarction:comparison between APACHE IIIand SAPS II prognosis systems

Received: 4 July 1996Accepted: 27 November 1996

Funding: Spanish Government ResearchGrant FIS 92/34

A.Reina ⋅ G.Vazquez ()) ⋅ E.Aguayo ⋅I.Bravo ⋅ M. ColmeneroServicio de Medicina Crıtica y Urgencias,Hospital Virgen de las Nieves, Avda.Fuerzas Armadas 2, E-18014 Granada,SpainFAX: + 34 (58) 241282

M. BravoStatistician, University of Granada, Spain

PAEEC GroupProject for the Epidemiological Analysisof Critical Care Patients, Spain

Abstract Objective: To comparethe Acute Physiology, Age andChronic Health Evaluation (APA-CHE) III with the Simplified AcutePhysiology Score (SAPS II) in dis-criminating in-hospital mortality forintensive care unit (ICU) patientswith acute myocardial infarction(AMI).Design: Prospective, observational,multicenter study.Setting: 70 Spanish ICUs.Patients and participants: 1711 pa-tients with AMI and representativeof Spanish ICUs.Measurements and results: APA-CHE III score, APACHE III systemprobability of death (APACHE IIIprobability), SAPS II score and in-hospital mortality were noted foreach patient. Two hundred and

twenty three (13.0%) patients diedin the hospital. The sensitivity(± SE), specificity (± SE), and accu-racy (± SE) for the APACHE IIIscore were, respectively, 75.8 ± 2.9,75.9 ± 1.1, and 75.9 ± 1.0. The corre-sponding figures for APACHE IIIprobability were 75.3 ± 2.9, 79.2 ±1.1, and 78.7 ± 1.0, and for SAPS II72.2 ± 3.0, 75.9 ± 1.1, and 75.4 ± 1.0.Conclusions: The results indicategood discrimination by the threetests. APACHE III probabilityshows a statistically significant im-provement in accuracy and specific-ity when compared with the twoscores.

Key words Acute myocardialinfarction ⋅ Intensive medicine ⋅Prognosis

Intensive Care Med (1997) 23: 326–330 Springer-Verlag 1997 ORIGINAL

pital mortality well, with no significant differences be-tween them [2]. The objective of our study was to com-pare APACHE III with SAPS II, the current versionsof the indices, in their ability to discriminate the in-hos-pital mortality of patients admitted to ICUs for AMI.

Material and methodsThe results presented here form part of the Project for the Epide-miological Analysis of Critical Care Patients (PAEEC), a prospec-tive observational multicenter study of Spanish ICUs, analyzingthe mortality, length of stay, unit workload, and quality of life ofcritical care patients [7]. The study covers two periods, one corre-sponding to the hospital stay and carried out in 1992 and 1993,and the other corresponding to the 12- and 24-month follow-up ofa cohort of survivors. The results analyzed here correspond to thehospital stay phase.

In 1992, all the ICUs recognized by the Spanish Society of In-tensive Medicine and Coronary Units (n = 136) were invited tojoin the study. Forty ICUs refused participation on the grounds ofwork overload or lack of interest. Ten ICUs abandoned the studyafter their initial acceptance. Each ICU was asked to provide dataon approximately 20 % of admissions in a 6-month consecutive pe-riod, with a minimum number of 50 patients from any one ICU.The study was performed in 1992 or 1993, depending on the hospi-tal in question. Patients were selected by a systematic samplingtechnique (every 4th or 5th patient). The hospital phase of thestudy was completed by 86 ICUs (72 in public hospitals, 14 pri-vate). Patients with coronary heart disease were not attended in16 of these ICUs, either because the hospital had an independentcoronary unit or because they were single purpose, i. e., trauma orpostoperative units. The remaining 70 ICUs participated in thispresent study of patients with AMI.

Patients aged under 16 years were excluded from the globalsample. Of the 7883 critical patients reported by the 70 ICUs,1791 (22.7%) patients had a diagnosis of AMI, with 80 dying with-in the first 6 h of ICU admission. These 80 patients were excludedbecause the APACHE system requires a longer period of hospital-ization. The remaining 1711 patients stayed in the ICU for > 6 h.The mean number of patients with AMI per ICU was 24.4(SD = 13.5).

AMI was diagnosed when at least two of the following threecriteria were present: typical severe chest pain, electrocardio-graphic changes (new Q waves), and increased cardiac enzymeconcentrations (at least twice the normal value of creatine kinase).

From the extensive data for the PAEEC protocol, we collecteddata on age, sex, prior location (emergency area, operating theatre,etc.), mean length of stay in the ICU, use of thrombolysis, pace-maker, mechanical ventilation, Swan-Ganz catheter, cardiopulmo-nary resuscitation, and the activities making up the Therapeutic In-tervention Scoring System [8]. We also calculated, using the worstvalues for the first 24 h of ICU admission, the SAPS II score [6],the APACHE III score [5], and the APACHE III system probabil-ity of death (APACHE III probability) [5], which incorporates theAPACHE III score, the diagnosis, and the prior location of the pa-tient. The probability of death predicted by the SAPS system wasnot used, as it is based only on the SAPS score. The b coefficientfor calculating mortality prediction from APACHE III probabilitywas provided by Dr. W. Knaus, as it has not been published. Nei-ther albumin nor bilirubin was collected for PAEEC patients withAMI, because their concentrations do not alter in this type of pa-tient, and a normal value was assigned for the calculation of the in-dices. Finally, hospital mortality was recorded.

Statistical analysis strategy was designed to compare the dis-criminating abilities of the three tests. When comparing survivorsversus non-survivors, Student’s t -test for unpaired data and thechi-square test were used for interval and categorical descriptivevariables, respectively. Sensitivity (Se), specificity (Sp), and accu-racy (Ac) were calculated for each severity index at the cut-off pointgiving the best (highest) Youden index, i. e., Se + Sp − 1 [9]. Se, Sp,and Ac obtained with each severity index were compared by meansof McNemar’s test [10]. A receiver operating characteristic (ROC)curve was built for each severity index, with areas and their stan-dard errors calculated by means of the Wilcoxon statistic [11]. Com-parisons between these areas were based on the calculation of a crit-ical ratio, “z”, by using Pearson’s correlation coefficient [12]. Afterlooking at the severity index distributions, they were converted intointervals of growing values. Observed mortality proportions withineach interval were contrasted against the mean test score using theMantel-Haenszel test for linear association, based on a chi-squaredistribution with 1 degree of freedom (df) [10].

Results

Of the 1711 patients in the study, 223 died in hospital(13.03 %). The patients who died were older, a higherproportion were females, and they consumed more re-sources, measured either by length of stay or by TISS(Table 1). Thrombolysis was used more in surviving pa-tients, while pacemakers, cardiopulmonary resuscita-tion, Swan-Ganz catheters, and IPPV were used morein nonsurvivors (Table 1).

The scores of all three indices were higher in nonsur-vivors (Table 2). Observed mortality increased progres-sively with rising scores in all three indices (Table 3).

The area under the ROC curve was 0.842 (SE =0.013) for APACHE III score, 0.846 (SE = 0.013) forAPACHE III probability of hospital death, and 0.828(SE = 0.014) for the SAPS II score. When comparingthe areas for APACHE III score and APACHE IIIprobability statistically nonsignificant differences werefound (z = 0.02, p = 0.98). The corresponding figures

327

Table 1 Characteristics of patients with AMI according to hospitaldeath (mean ± SD) (CPR cardiopulmonary resuscitation, IPPVventilation with intermittent positive pressure, TISS therapeuticintervention scoring system)

Variable Survivors(n = 1488)

Nonsurvivors(n = 223)

pvaluea

Age (years) 63.1 ± 11.4 71.5 ± 9.0 < 0.001Male/female ratio 1191/295 145/80 < 0.001Hospitalization length (days) 5.1 ± 5.7 6.3 ± 7.3 < 0.050Thrombolysis (yes/no) 610/878 52/171 < 0.001Pacemaker (yes/no) 81/1407 42/181 < 0.001CPR (yes/no) 44/1444 50/173 < 0.001IPPV (yes/no) 54/1434 83/140 < 0.001Swan-Ganz (yes/no) 55/1433 47/176 < 0.001TISS 15.6 ± 6.0 22.0 ± 10.2 < 0.001a Student’s t -test for age, length of hospitalization, and TISS, andchi-square test for the remaining variables

for the APACHE III score-SAPS II score and APA-CHE III probability-SAPS II score comparisons were,respectively, z = 1.59 (p = 0.112) and z = 1.98 (p = 0.05).For the chosen cut-off points, Se, Sp, and Ac were above70% in the three indices. The APACHE III probabilityhad a higher statistically significant Sp and Ac than didAPACHE III score and SAPS II score (Table 4).

Discussion

In interpreting our results, it should be noted that wewere not attempting to use the formulas provided bythe APACHE or SAPS systems to calculate the proba-bility of death, and the calibration of these systems was

not our purpose [13]. Furthermore, we only comparedtheir discriminating ability, so the APACHE III proba-bility is considered here as a growing severity score(from 0 to 1), and the SAPS II probability was not calcu-lated because it does not incorporate new patient sever-ity information to the SAPS II score. The mortality ofour series is similar to that reported by other authors[3, 4]. Considered separately, the APACHE III score,APACHE III probability, and the SAPS II score pro-vide acceptable values, i. e., good Se, Sp, and Ac, for in-hospital mortality, and their respective ROC curves arein a high position, indicating that all three systems accu-rately discriminate the severity of the patient with AMI.

There were no significant differences in the discrimi-nating ability of the APACHE III score and the SAPS II

328

Variable Survivors(n = 1488)

Nonsurvivors(n = 223)

Total(n = 1711)

p valuea

APACHE III score 37.5 ± 15.7 62.5 ± 22.1 40.8 ± 18.7 < 0.001APACHE III probability 0.09 ± 0.10 0.28 ± 0.22 0.12 ± 0.14 < 0.001SAPS II score 33.3 ± 10.0 49.2 ± 15.9 35.3 ± 12.2 < 0.001a Student’s t -test

Table 2 APACHE III score,APACHE III probability ofdeath, and SAPS II score ofpatients with AMI accordingto hospital death (mean ± SD)

Test Test interval Mean test scorein interval

No. of patientsin interval

Observed mortalityproportions

APACHE IIIscoreb

4–19 15.1 152 0.00720–25 22.7 161 0.01226–29 27.6 155 0.00630–33 31.5 195 0.01034–37 35.6 183 0.04938–41 39.4 174 0.12642–45 43.5 164 0.10446–51 48.4 172 0.17452–63 57.3 178 0.25864–139 81.2 177 0.525

APACHE IIIprobabilityc

0.007–0.021 0.017 166 0.0060.022–0.031 0.027 172 0.0170.032–0.041 0.037 171 0.0000.042–0.052 0.047 173 0.0170.053–0.066 0.059 166 0.0420.067–0.081 0.074 177 0.1240.082–0.105 0.093 171 0.0990.106–0.154 0.125 171 0.1990.155–0.262 0.199 172 0.2620.263–0.910 0.469 172 0.529

SAPS II scored 11–22 19.7 112 0.00923–25 23.6 149 0.00726–27 26.2 119 0.00828–30 28.7 289 0.03531–32 31.5 159 0.05733–34 33.4 172 0.11035–37 36.0 191 0.11038–41 39.3 165 0.18242–49 45.1 168 0.19650–106 62.3 187 0.524

Table 3 Relationshipa ofAPACHE III score,APACHE III system probabil-ity of death, and SAPS II withobserved hospital mortality inpatients with AMI (n = 1711)

ac

2 (Mantel-Haenszel test forlinear associationbc

2 = 342.24, 1 df, p < 0.001cc

2 = 359.73, 1 df, p < 0.001dc

2 = 329.75, 1 df, p < 0.001

score, confirming results with earlier versions of theseindices [2], despite SAPS not being validated for pa-tients with coronary heart disease. In our series, the Acof 75.9 % using the APACHE III score rose to 78.7 %with the APACHE III probability (Table 4). This slightimprovement is due solely to the incorporation of thepatients’ prior location, as AMI constitutes one singlediagnostic category in the APACHE III probabilityand adds no few information to the score.

One consideration for the internal validity of thestudy is assigning a zero score for the bilirubin compo-nent of SAPS II. This is the normal strategy for uncol-lected data in the construction of the APACHE III sys-tem but is not accepted by the authors of the SAPS sys-tem. Bilirubin data are not collected for patients withAMI in Spanish ICUs, as it is rarely abnormal, at leastnot in the first 24 h. Nevertheless, in order to confirmthis assumption, we collected bilirubin data from 71 pa-tients with AMI in one of the participating ICUs, andin no case was the value above 2 mm/dl.

In the APACHE III system, physiologic and labora-tory abnormalities, alongside comorbid conditions andage, explained 83.3 % of the mortality, while the diagno-

sis only explained 13.6 % [14]. This means that whengeneral indices like APACHE and SAPS are applied toconcrete diseases, such as AMI, they work well, and in-deed, other authors have suggested that they can be asgood as specific indices in the same conditions [2], oreven better [1, 15]. Specific indices for AMI normally in-clude anatomic, enzymatic or electrocardiographic as-pects that are indirectly captured by general indicesthrough their clinical repercussions. In multidisciplinaryICUs, the use of these general indices to evaluate the se-verity of the patient’s illness would be more useful,avoiding the simultaneous use of several indices accord-ing to the patient’s disease. On the other hand, specificindices could be more useful in certain studies of AMI,and the combination of both types of index could offera more complete picture of these patients. However, itmust not be forgotten that there are other end-pointswhich may also be used in this type of patient, includingventricular function, long-term mortality, and quality oflife.

Based on the results of this present study, all threegeneral indices give good discriminating values for pa-tients with AMI. However, while the APACHE IIIprobability is statistically better, the improvement overSAPS II and APACHE III score in Sp and Ac is rela-tively low, and there are other factors to take into ac-count. These would include the performance of the in-dex with regard to other diseases, the number of vari-ables required for its calculation (larger in the case ofAPACHE III), the need for specific software (as forAPACHE III probability), and the absence of a value-assignment strategy for missing data (as for SAPS II), acommon problem in the clinical setting. These consider-ations must determine which index is most useful inevaluating the totality of patients in a multidisciplinaryICU.

Acknowledgements The authors wish to thank Dr. Knaus for pro-viding the coefficients for calculating mortality prediction fromAPACHE III.

329

Table 4 Sensitivity (Se), specificity (Sp), and accuracy (Ac) calcu-lated for APACHE III score, APACHE III system probability ofdeath, and SAPS II score in patients with AMI (mean ± SE)

APACHE IIIscore

APACHE IIIprobability

SAPS IIscore

Cut-off pointa 46 0.113 38Seb 75.8 ± 2.9 75.3 ± 2.9 72.2 ± 3.0Spc 75.9 ± 1.1 79.2 ± 1.1 75.9 ± 1.1Acd 75.9 ± 1.0 78.7 ± 1.0 75.4 ± 1.0a Given the best Youden indexb McNemar’s test: comparing by pairs non-significant differencesare foundc McNemar’s test: APACHE III system > APACHE III score(p < 0.001), APACHE III system > SAPS II score (p < 0.001)d McNemar’s test: APACHE III system > APACHE III score(p < 0.001), APACHE III system > SAPS II score (p < 0.001)

References

1. Alemi F, Rice J, Hankins R (1990) Pre-dicting in-hospital survival of myocar-dial infarction. A comparative study ofvarious severity measures. Med Care28: 762–775

2. Moreau R, Soupison T, Vauquelin P,Derrida S, Beaucour H, Sicot C (1989)Comparison of two simplified severityscores (SAPS and APACHE II) for pa-tients with acute myocardial infarction.Crit Care Med 17: 409–413

3. Ludwigs U, Hulting J (1995) Acutephysiology and chronic health evalua-tion II score system in acute myocardialinfarction: a prospective validationstudy. Crit Care Med 23: 854–859

4. Abizanda R, Jorda R, Marse P, ValleFX, Velasco J, Rotger A (1996) Valorpronostico de APACHE II en pacientescoronarios agudos. Med Intensiva 13:159–162

5. Knaus WA, Wagner DP, Draper EA(1991) The APACHE III prognosticsystem. Risk prediction of hospital mor-tality for critically ill hospitalized adults.Chest 100: 1619–1636

6. Le Gall JR, Lemeshow S, Saulnier F(1993) A new simplified acute physiol-ogy score (SAPS II) based on a Euro-pean/North American multicenterstudy. JAMA 270: 2957–2963

330

7. Vazquez G, Rosado JL, Cerda M (1993)Estudio multicentrico “proyecto deanalisis epidemiologico de los enfermoscrıticos” (PAEEC). Med Intensiva 17:47–70

8. Keene AR, Cullen DJ (1983) Thera-peutic Intervention Scoring System: up-date 1983. Crit Care Med 11: 1–3

9. Youden WJ (1950) Index for rating di-agnostic tests. Cancer 3: 32–35

10. Domenech Massons JM (1991) Meto-dos estadısticos en ciencias de la salud.Pruebas de chi cuadrado. Editorial Gra-ficas Signo, Barcelona

11. Hanley JA, McNeil BJ (1982) Themeaning and use of the area under a re-ceiver operating characteristic (ROC)curve. Radiology 143: 29–36

12. Hanley JA, McNeil BJ (1983) A meth-od of comparing the areas under re-ceiver operating characteristic curvesderived from the same cases. Radiology148: 839–843

13. Schuster DP (1992) Predicting outcomeafter ICU admission. The art and scienceof assessing risk. Chest 102: 1861–1870

14. Knaus WA, Wagner DP, ZimmermanJE, Draper EA (1993) Variations inmortality and length of stay in intensivecare units. Ann Intern Med 118: 753–761

15. Blumberg MS (1991) Biased estimatesof expected acute myocardial infarctionmortality using MedisGroups Admis-sion Severity Groups. JAMA 265:2965–2970