Elsevier

The Lancet

Volume 397, Issue 10270, 16–22 January 2021, Pages 199-207
The Lancet

Articles
Machine learning-based prediction of adverse events following an acute coronary syndrome (PRAISE): a modelling study of pooled datasets

https://doi.org/10.1016/S0140-6736(20)32519-8Get rights and content

Summary

Background

The accuracy of current prediction tools for ischaemic and bleeding events after an acute coronary syndrome (ACS) remains insufficient for individualised patient management strategies. We developed a machine learning-based risk stratification model to predict all-cause death, recurrent acute myocardial infarction, and major bleeding after ACS.

Methods

Different machine learning models for the prediction of 1-year post-discharge all-cause death, myocardial infarction, and major bleeding (defined as Bleeding Academic Research Consortium type 3 or 5) were trained on a cohort of 19 826 adult patients with ACS (split into a training cohort [80%] and internal validation cohort [20%]) from the BleeMACS and RENAMI registries, which included patients across several continents. 25 clinical features routinely assessed at discharge were used to inform the models. The best-performing model for each study outcome (the PRAISE score) was tested in an external validation cohort of 3444 patients with ACS pooled from a randomised controlled trial and three prospective registries. Model performance was assessed according to a range of learning metrics including area under the receiver operating characteristic curve (AUC).

Findings

The PRAISE score showed an AUC of 0·82 (95% CI 0·78–0·85) in the internal validation cohort and 0·92 (0·90–0·93) in the external validation cohort for 1-year all-cause death; an AUC of 0·74 (0·70–0·78) in the internal validation cohort and 0·81 (0·76–0·85) in the external validation cohort for 1-year myocardial infarction; and an AUC of 0·70 (0·66–0·75) in the internal validation cohort and 0·86 (0·82–0·89) in the external validation cohort for 1-year major bleeding.

Interpretation

A machine learning-based approach for the identification of predictors of events after an ACS is feasible and effective. The PRAISE score showed accurate discriminative capabilities for the prediction of all-cause death, myocardial infarction, and major bleeding, and might be useful to guide clinical decision making.

Funding

None.

Introduction

Patients with acute coronary syndrome (ACS) are at high risk for ischaemic and bleeding events, with both being drivers of adverse prognosis.1 Careful evaluation of these risks plays a fundamental role in the clinical management of each patient, with important implications regarding the choice of optimal medical therapy for secondary prevention.2, 3, 4, 5, 6

To this aim, several predictive tools have been developed to estimate ischaemic and bleeding risks following an ACS, some of which have potential to support clinical decision making around the optimal duration of dual antiplatelet therapy (DAPT).7, 8, 9, 10, 11 However, the overall accuracy of these scores, along with their generalisability to external cohorts, remains modest, representing an unmet need for individualised patient management strategies.12, 13

From a clinical standpoint, the poor performance of existing risk scores among patients with ACS might be related to their derivation from unselected percutaneous coronary intervention populations encompassing patients with stable presentation. Moreover, machine learning methods might be able to overcome some of the limitations of current analytical approaches to risk prediction by applying computer algorithms to large datasets with numerous, multidimensional variables, capturing high-dimensional, non-linear relationships among clinical features to make data-driven outcome predictions.14 The effectiveness of this approach has been shown in several cardiovascular applications, where machine learning was superior to validated traditional risk stratification tools, including prediction of death among patients with suspected coronary artery disease or of heart failure in candidates for cardiac resynchronisation therapy.15, 16 Thus, we sought to develop a machine learning-based risk stratification model integrating clinical, anatomical, and procedural features to predict ischaemic and bleeding events after an ACS, by pooling several large international cohorts of patients to inform model development and validation.

Research in context

Evidence before this study

Prediction of all-cause death in patients with acute coronary syndrome (ACS) has relevant implication on post-discharge follow-up and treatment approaches. Similarly, ischaemic and bleeding events after an ACS can also severely affect patient prognosis. We searched PubMed on July 30, 2020, without language or date restrictions for publications about risk scores validated to predict mortality along with bleeding and thrombotic risks in patients with ACS. We used the search terms “acute coronary syndrome”, “percutaneous coronary intervention”, “risk score”, “mortality risk score”, “bleeding risk score”, “thrombotic risk score”, and “antiplatelet therapy”. We excluded articles reporting only multivariate predictors without prediction score, papers referring only to antithrombotic management for patients with a concomitant indication to oral anticoagulation, risk prediction models for only in-hospital outcomes, and those without a validation cohort. Currently available scores conceived and validated to predict patients' mortality after all types of ACS (ie, the GRACE and TIMI risk scores) were mostly derived on cohorts of patients not treated with contemporary standards of care. Current guidelines support the use of dedicated scores (ie, PRECISE-DAPT and DAPT scores) to tailor the intensity and duration of dual antiplatelet therapy (DAPT) strategy according to the offsetting risk of ischaemia and bleeding. Nonetheless, such scores were derived from unselected cohorts of patients mostly treated with clopidogrel after percutaneous revascularisation both for stable coronary artery disease and ACS, thus limiting their accuracy in external cohorts and their applicability in clinical practice. Furthermore, all of these scores are based on linear models applied to variables frequently based on a-priori assumption. The feasibility and the effectiveness of a machine learning-based prognostic risk assessment in this setting has never been explored before.

Added value of this study

We developed and externally tested the PRAISE score, a machine learning-based model to predict the risk of 1-year post-ACS all-cause death, recurrent myocardial infarction, and major bleeding. The model was derived from a contemporary cohort of patients treated with percutaneous coronary intervention for ACS and treated according to current guidelines recommendation on DAPT. The PRAISE score showed excellent predictive abilities for all the explored endpoints both in the derivation and external validation cohort. To highlight the clinical implication of the risk estimated by the model, we suggested a stratification into three classes (low, intermediate, and high) entailing a clinically significant increase in the risk of event occurrence. According to such classification, the PRAISE scores would classify almost 10% of patients as being at high risk of 1-year post-discharge ischaemic and bleeding events, thus being candidates for a tighter follow-up. Furthermore, patients deemed at high ischaemic risk according to the PRAISE score showed a consistent prevailing ischaemic risk, regardless of their bleeding risk class, which could potentially be used to identify patients who would benefit most from an extended DAPT strategy. By contrast, patients with intermediate-to-high bleeding risk and a low ischaemic risk would benefit from a shortened DAPT strategy.

Implications of all the available evidence

In the setting of risk assessment, the machine learning approach offers a way to overcome the shortcomings of existing methods by applying computer algorithms to large datasets and capturing non-linear relationships between clinical variables. The PRAISE score is a bedside risk assessment tool that could be easily implemented in everyday clinical practice to predict patients' prognosis after ACS, along with their risk of ischaemic and bleeding events. This instrument has the potential to support clinical decision making with respect to antithrombotic treatments and follow-up planning, thus addressing the unmet need for tailored care after percutaneous coronary intervention.

Section snippets

Datasets

To develop the machine learning models, we used a derivation cohort of 19 826 adult patients (≥18 years) with ACS with 1 year of follow-up. These patients were obtained from two registries: the BleeMACS registry (NCT02466854) and the RENAMI registry.17 The BleeMACS registry included 15 401 consecutive patients with ACS who were admitted between Jan 1, 2003, and Dec 31, 2014, at 15 tertiary hospitals in North and South America, Europe, and Asia and treated with either clopidogrel, ticagrelor, or

Results

Clinical and therapeutic characteristics of the study population are shown in table 1, and are shown stratified by each outcome occurrence in the appendix (pp 17–19).

In the derivation cohort, death occurred in 662 (3·3%) patients, myocardial infarction in 609 (3·1%) patients, and major bleeding in 562 (2·8%) patients, within 1-year follow-up (table 2). In the external validation cohort, death occurred in 58 (1·7%) patients within 1 year and 301 (8·7%) within 2 years, myocardial infarction in 58

Discussion

In this study, we used data on 23 270 patients who were discharged after an ACS to develop and test machine learning-based risk scores to predict the risk for all-cause death, myocardial infarction, and major bleeding 1 year after discharge. We found that the PRAISE scores presented excellent discriminative abilities for the prediction of 1-year all-cause death, myocardial infarction, and major bleeding following an ACS, also when externally validated. Clinically meaningful risk cutoffs for

Data sharing

Due to the different data-sharing policies of the various datasets included in this study, not all of which provided free access to data, data included in this study will not be made available. Requests for the data from each included dataset should be made to the corresponding author of each single registry and trial.

References (31)

  • F D'Ascenzo et al.

    TIMI, GRACE and alternative risk scores in acute coronary syndromes: a meta-analysis of 40 derivation studies on 216,552 patients and of 42 validation studies on 31,625 patients

    Contemp Clin Trials

    (2012)
  • GD Dangas et al.

    Development and validation of a stent thrombosis risk score in patients with acute coronary syndromes

    JACC Cardiovasc Interv

    (2012)
  • F D'Ascenzo et al.

    Long versus short dual antiplatelet therapy in acute coronary syndrome patients treated with prasugrel or ticagrelor and coronary revascularization: insights from the RENAMI registry

    Eur J Prev Cardiol

    (2020)
  • F D'Ascenzo et al.

    Impact of design of coronary stents and length of dual antiplatelet therapies on ischaemic and bleeding events: a network meta-analysis of 64 randomized controlled trials and 102 735 patients

    Eur Heart J

    (2017)
  • M Valgimigli et al.

    2017 ESC focused update on dual antiplatelet therapy in coronary artery disease developed in collaboration with EACTS: The Task Force for dual antiplatelet therapy in coronary artery disease of the European Society of Cardiology (ESC) and of the European Association for Cardio-Thoracic Surgery (EACTS)

    Eur Heart J

    (2018)
  • Cited by (158)

    View all citing articles on Scopus
    View full text