There are barriers to proper implementation of risk stratification scores in patients with acute coronary syndromes (ACS), including their complexity. Our objective was to develop a simple score for risk stratification of all-cause in-hospital mortality in a population of patients with ACS.Methods
The score was developed from a nationwide ACS registry. The development and internal validation cohorts were obtained from the first 31829 patients, randomly separated (60% and 40%, respectively). The external validation cohort consisted of the last 8586 patients included in the registry. This cohort is significantly different from the other cohorts in terms of baseline characteristics, treatment and mortality. Multivariate logistic regression analysis was used to select four variables with the highest predictive potential. A score was allocated to each parameter based on the regression coefficient of each variable in the logistic regression model: 1 point for systolic blood pressure ≤116 mmHg, Killip class 2 or 3, and ST-segment elevation; 2 points for age ≥72 years; and 3 points for Killip class 4.Results
The new score had good discriminative ability in the development cohort (area under the curve [AUC] 0.796), and it was similar in the internal validation cohort (AUC 0.785, p=0.333). In the external validation cohort, there was also excellent discriminative ability (AUC 0.815), with an adequate fit.Conclusions
The ProACS risk score enables easy and simple risk stratification of patients with ACS for in-hospital mortality that can be used at the first medical contact, with excellent predictive ability in a contemporary population.
Existem algumas barreiras à implementação adequada dos scores de estratificação de risco em doentes com síndrome coronária aguda (SCA), tais como a sua complexidade. O nosso objetivo foi desenvolver um score simples para estratificação de risco de mortalidade hospitalar de todas as causas numa população de doentes com SCA.Métodos
O score foi desenvolvido a partir de um registo nacional de SCA. A coorte de desenvolvimento e de validação interna foi obtida a partir dos primeiros 31829 doentes, aleatoriamente separados (60 e 40%, respetivamente). A coorte de validação externa é composta pelos últimos 8586 doentes incluídos no registo. Esta coorte é significativamente diferente das restantes (características basais, tratamento e mortalidade). Foi utilizada análise de regressão logística multivariada para selecionar as quatro variáveis com maior potencial preditivo e foi atribuída uma pontuação baseada no coeficiente de regressão de cada variável no modelo de regressão logística: um ponto para TAS ≤ 116 mmHg, classe Killip 2 ou 3, e elevação segmento ST, dois pontos para idade ≥ 72 anos e três pontos para classe Killip 4.Resultados
O novo score tem uma boa capacidade preditiva na coorte de desenvolvimento (area under curve [AUC] 0,796), semelhante à coorte de validação interna (AUC 0,785, p=0,333). Na coorte de validação externa também apresentou uma excelente capacidade discriminativa (AUC 0,815), com calibração adequada.Conclusões
O score de risco ProACS permite uma estratificação de risco precoce e simples em doentes com SCA para mortalidade hospitalar, que pode ser utilizada no primeiro contacto médico, com excelente capacidade preditiva numa população contemporânea.
The management of acute coronary syndromes (ACS) has changed dramatically in the past 20 years following demonstration of the benefits associated with invasive strategies, particularly in high-risk patients.1–5 Selection for these strategies is an important task in every patient with ACS. Several risk scores have been developed for this purpose, from both clinical trials and registries. They differ in predictive accuracy as well as in the number and type of variables included. The first to be developed was the TIMI risk score,6,7 but its predictive accuracy is usually lower than more recent scores. The most recent and most widely used is the Global Registry of Acute Coronary Events (GRACE) risk score, developed from the GRACE registry.8,9 This has very high predictive accuracy, but includes many variables with significant complexity, which may explain why it is often underused.10,11
Previously, our group and others have demonstrated that these risk scores can be simplified, with a slight reduction in predictive accuracy compared to the GRACE score, but that can be considered acceptable.12,13 These simplified scores show similar accuracy to the TIMI risk score.6,7,14
Our objective was to develop a simple score for risk stratification of in-hospital mortality in patients with ACS, to be used very early in patient management, including at pre-hospital level.Methods
The Portuguese Registry on Acute Coronary Syndromes (ProACS) is a multicenter nationwide registry of ACS. It is a prospective, continuous observational registry, with 33 participating cardiology departments from Portugal (mainland and islands).15 Patient inclusion in the registry began on January 1, 2002, and all consecutive adult patients (≥18 years) registered until October 31, 2014 were included in the present study. Criteria for inclusion in the registry were a history of chest pain at rest or other symptoms suggestive of an ACS (with the most recent episode occurring within 48 hours of admission) with or without new or presumed new significant ST-segment or T-wave changes, new left bundle branch block or elevated biomarkers of myocardial damage with a rise and/or fall in levels. Acute myocardial infarction (MI) was defined according to the universal definition of type 1 myocardial infarction.16 A diagnosis of ST-elevation myocardial infarction (STEMI) was made in the presence of persistent (>30 min) ST-segment elevation. All other cases with elevated biomarkers of myocardial damage were considered non-STEMI (NSTEMI).
Data were collected in a dedicated computer database, and included demographic, clinical and patient management-related characteristics, as well as clinical outcome. Hypertension, diabetes and hyperlipidemia were defined as either previously known or on specific therapy. If the patients had smoked during the previous 30 days they were classified as smokers and were self-reported. Decisions on patient management strategy, including referral for coronary angiography and mode of myocardial revascularization, if any, were left to the attending physician and site-specific protocols. The primary endpoint was all-cause mortality during the index hospitalization.
From the main study population (40415 records), we obtained three cohorts of patients (Figure 1). From January 1, 2002 to June 30, 2011, 31829 patients were included in the registry and were randomly divided into a cohort for risk score development (60% of the patients) and the cohort for internal validation (the remaining 40%). All patients included between July 2011 and October 2014 (8586 patients) were included in the external validation cohort. Patients with missing values for the main variables were excluded (2955 patients – 7.3% of the initial patient sample).
The study complies with the 1975 Declaration of Helsinki. The national data protection committee for clinical research approved the study protocol and informed consent was obtained from each patient. The registry is registered at ClinicalTrials.gov with the identification number NCT01642329.
This manuscript was written in accordance with the Transparent Reporting of a Multivariable Predictor Model for Individual Prognosis or Diagnosis (TRIPOD) statement on appropriate description of studies for score validation.17Statistical analysis
Categorical variables are reported as percentages and differences between groups were tested with the chi-square test or Fisher's exact test, as appropriate. Continuous variables are reported as means and standard deviation. Normality was tested with the Kolmogorov-Smirnov test. One-way ANOVA was used to compare continuous variables with normal distribution. Continuous variables without normal distribution are reported as medians and interquartile range and were compared with the Kruskal-Wallis test.
Multivariable stepwise forward logistic regression models were used to select the four variables with the highest predictive potential for score development. Variables were removed from the model when the p-value exceeded 0.10 and were kept in the final model when less than 0.05. The variables included were age, admission heart rate, systolic blood pressure and Killip class, creatinine, ST-segment deviation, gender, diabetes, smoking status, previous myocardial infarction, percutaneous coronary intervention (PCI), coronary artery bypass grafting, stroke or peripheral arterial disease. The selected continuous variables were categorized (dichotomized) by receiver operating characteristic (ROC) curve analysis. Cut-offs were obtained by maximizing the sum of sensitivity and specificity. For each variable, appropriate weights were determined based on the regression coefficient of the final logistic model (to the closest whole number). Points were summed to obtain the final risk score. ROC curve analysis and the area under the curve (AUC) were used to study the predictive value of the new risk score in each cohort. Bootstrap techniques were used for ROC curve comparison and for confidence intervals.
IBM SPSS Statistics software (version 22.214.171.124) was used for all statistical analyses. All statistical tests were two-sided with a critical value of 0.05 for statistical significance.Results
Patients were included in each cohort according to the flowchart presented in Figure 1. Patients in the development cohort had fewer risk factors and less past history of coronary artery disease compared to the internal and external validation cohorts (Table 1). On admission they had higher heart rate and systolic blood pressure and worse Killip class and STEMI was more frequent, particularly compared to the external validation cohort. They less often underwent coronary angiography and revascularization procedures. In-hospital mortality was lower in the validation cohorts, particularly in the external validation cohort.
Clinical characteristics of the patients in each cohort.
|Development cohort (n=17380)||Internal validation cohort (n=11548)||External validation cohort (n=8532)||p|
|Age, years||66 (13)||66 (13)||66 (13)||0.155|
|BMI (kg/m2)||27 (4)||27 (4)||27 (4)||0.785|
|Previous history (%)|
|Heart rate (bpm)||79 (20)||79 (20)||77 (19)||<0.001|
|SBP (mmHg)||141 (30)||141 (30)||138 (29)||<0.001|
|Killip class ≥2 (%)||18.9||19.3||15.5||<0.001|
|Killip class 4 (%)||2.0||2.0||1.9||0.801|
|Creatinine (mg/dl)||0.9 (0.8-1.2)||0.9 (0.8-1.2)||0.9 (0.8-1.2)||0.130|
|Blood glucose (mg/dl)||130 (108-170)||130 (107-175)||129 (106-174)||0.739|
|Hemoglobin (g/dl)||13.7 (1.9)||13.8 (1.9)||13.7 (1.9)||0.618|
|Coronary angiography (%)||68.9||69.1||86.8||<0.001|
|Multivessel diseasea (%)||50.6||50.6||50.7||0.994|
|LVEF <50% (%)||39.9||39.9||36.7||<0.001|
BMI: body mass index; CABG: coronary artery bypass grafting; COPD: chronic obstructive pulmonary disease; LVEF: left ventricular ejection fraction; PAD: peripheral arterial disease; PCI: percutaneous coronary intervention; SBP: systolic blood pressure; STEMI: ST-elevation myocardial infarction; TIA: transient ischemic attack.
In the multivariate model, the variables with the highest predictive ability were age, systolic blood pressure on admission, Killip class on admission and ST-segment elevation. The cut-off obtained for age was 72 years and for systolic blood pressure it was 116 mmHg. The ProACS score was developed according to these cut-offs with a simple scoring system (Table 2).
The ProACS score was then tested in the development cohort as well as in the internal and external validation cohorts. In all cohorts the discriminative ability was also very good (AUC >0.75) and similar between all cohorts, although slightly higher in the external validation cohort (Table 3). The fit of the developed logistic model was also good in the development cohort (Figure 2). Performance was similar comparing patients with STEMI and non-ST-elevation acute coronary syndrome (NSTACS). However, compared to the GRACE risk score, the predictive ability was slightly lower (Figure 3).
Area under the curve for the ProACS risk score in the different cohorts.
|Cohort||C-statistic (95% CI)|
|Internal validation||0.785 (0.767-0.803)|
|External validation||0.815 (0.793-0.837)|
CI: confidence interval; NSTACS: non-ST-elevation acute coronary syndrome; STEMI: ST-elevation myocardial infarction.
On the ProACS risk score, patients with a score 0 have a low risk of in-hospital death (<1%) and those with a score ≥3 have a high risk (>4%). Patients with score 1-2 have intermediate risk (1-4%) (Figure 4).Discussion
In the past 20 years, invasive strategies for the treatment of ACS have had a significant impact on prognosis.1–5 It is thus important to identify and select the patients who can derive the most benefit from these strategies. In STEMI primary PCI is now generally recommended1,3 and so risk stratification for these patients is less important, but some cases require urgent pre-hospital referral to a more specialized center. NSTACS patients are a heterogeneous group and invasive strategies are recommended in high- and intermediate-risk patients.2,4 For this reason, several risk scores have been developed in the past 15 years. The TIMI risk scores for STEMI and unstable angina/NSTEMI were developed from large clinical trials.6,7 These scores were simple and intuitive with simple variables, and their use spread rapidly worldwide. Other risk scores were also developed but the TIMI score remained the most used until 2003, when the GRACE registry group published a risk score for in-hospital mortality in patients with ACS and later for six-month mortality.8,9 This score was developed from a registry that represented real-world patients instead of the highly controlled and selected populations of clinical trials, from which elderly patients and those with severe renal dysfunction were usually excluded. The GRACE risk score showed high predictive accuracy compared to previous risk scores; it soon became the most used score, and was subsequently updated.18 However, the GRACE score's nomogram includes several variables with a wide range of categories, some of which can only be obtained after initial diagnostic tests. This adds complexity to risk scoring and means it cannot be used at an early stage. This might explain its underuse in clinical practice in spite of the development of several tools.10,11
Our group demonstrated in a single-center study that it is possible to simplify these risk scores with simple and fewer variables, with similar predictive accuracy, although we did not set out to propose a new risk score due to the inherent limitations of our study.12 A Canadian group has developed another score, the Canada Acute Coronary Syndrome risk score (C-ACS), based on large ACS registries in Canada.13 The selected variables were age, Killip class, systolic blood pressure and heart rate, with a simple dichotomization of continuous variables. This score enables rapid stratification of patients with ACS, even by health care professionals without advanced medical training, in pre-hospital settings or emergency departments. It showed a good predictive value for both short- and long-term mortality and in both STEMI and NSTACS. The c-statistic of this new score is adequate (≥0.75) but slightly lower than classical risk scores for the prediction of short- and long-term mortality.13
Since North American populations have different characteristics from other populations, including European and particularly Mediterranean (low-risk) countries, we thought it would be valuable to develop a similar risk score for early use in ACS that would be simple and easy to memorize.19,20 The ProACS risk score was developed from a nationwide registry of ACS in Portugal. The sample is large and is representative of real-world patients. In our score, we used simple variables (age, systolic blood pressure, ST-segment elevation and Killip class on admission), similar to the C-ACS score, and confirmed the validity of the selected variables (those with the highest predictive value). It also showed good predictive accuracy, which although lower than the GRACE risk score, is still acceptable. It is also easy to calculate and can be applied in pre-hospital settings or emergency departments by non-medical healthcare professionals. It improves referral of high- and intermediate-risk patients to hospitals with catheterization facilities, particularly in NSTACS. External validation was performed in a more recent cohort of patients, which may explain some of the differences observed, compared with the development and internal validation cohorts, particularly the lower proportion of STEMI and higher rate of coronary angiography and revascularization. It is also important to mention that predictive accuracy was higher in the external validation cohort, which highlights its utility in contemporary cohorts. Predictive accuracy was similar in patients with STEMI and NSTACS, suggesting that it can also be used in STEMI patients, although in this group the recommended strategy is primary PCI.Limitations
This score was developed and validated in a Portuguese population. It requires further validation in other populations, preferably in contemporary cohorts.
We chose to exclude from the analysis all patients with missing values of the score variables. Overall, only 7.3% of the patients were excluded, which is acceptable because of our large sample size. Data from the external validation cohort were more complete and fewer patients were excluded from this cohort.
We could not test goodness of fit with the new risk score because this is a categorical score and is thus not suitable for this type of analysis.Conclusion
The ProACS risk score enables easy and simple risk stratification for in-hospital mortality at the first medical contact of patients with ACS. It also has excellent predictive ability in a contemporary population of patients with ACS, although slightly lower than the GRACE risk score. Its simplicity can improve implementation of scores for risk stratification in clinical practice, although further validation in other populations and countries is warranted.Ethical disclosuresProtection of human and animal subjects
The authors declare that no experiments were performed on humans or animals for this study.Confidentiality of data
The authors declare that they have followed the protocols of their work center on the publication of patient data.Right to privacy and informed consent
The authors declare that no patient data appear in this article.Conflicts of interest
The authors have no conflicts of interest to declare.