Article Text
PDF
Abstract
Background Emergency Medical Services (EMS) studies have shown that prehospital risk stratification and triage decisions in patients with suspected non-ST-elevation acute coronary syndrome (NSTE-ACS) can be improved using clinical risk scores with point-of-care (POC) troponin. In current EMS studies, three different clinical risk scores are used in patients suspected of NSTE-ACS: the prehospital History, ECG, Age, Risk and Troponin (preHEART) score, History, ECG, Age, Risk and Troponin (HEART) score and Troponin-only Manchester Acute Coronary Syndromes (T-MACS). The preHEART score lacks external validation and there exists no prospective comparative analysis of the different risk scores within the prehospital setting. The aim of this analysis is to externally validate the preHEART score and compare the diagnostic performance of the these three clinical risk scores and POC-troponin.
Methods Prespecified analysis from a prospective, multicentre, cohort study in patients with suspected NSTE-ACS who were transported to an ED between April 2021 and December 2022 in the Netherlands. Risk stratification is performed by EMS personnel using preHEART, HEART, T-MACS and POC-troponin. The primary end point was the hospital diagnosis of NSTE-ACS. The diagnostic performance was expressed as area under the receiver operating characteristic (AUROC), sensitivity, specificity, negative predictive value (NPV) and positive predictive value (PPV).
Results A total of 823 patients were included for external validation of the preHEART score, final hospital diagnosis of NSTE-ACS was made in 29% (n=235). The preHEART score classified 27% as low risk, with a sensitivity of 92.8% (95% CI 88.7 to 95.7) and NPV of 92.3% (95% CI 88.3 to 95.1). The preHEART classified 9% of the patients as high risk, with a specificity of 98.5% (95% CI 97.1 to 99.3) and PPV of 87.7% (95% CI 78.3 to 93.4). Data for comparing clinical risk scores and POC-troponin were available in 316 patients. No difference was found between the preHEART score and HEART score (AUROC 0.83 (95% CI 0.78 to 0.87) vs AUROC 0.80 (95% CI 0.74 to 0.85), p=0.19), and both were superior compared with T-MACS (AUROC 0.72 (95% CI 0.66 to 0.79), p≤0.001 and p=0.03, respectively) and POC-troponin measurement alone (AUROC 0.71 (95% CI 0.64 to 0.78), p<0.001 and p=0.01, respectively).
Conclusion On external validation, the preHEART demonstrates good overall diagnostic performance as a prehospital risk stratification tool. Both the preHEART and HEART scores have better overall diagnostic performance compared with T-MACS and sole POC-troponin measurement. These data support the implementation of clinical risk scores in prehospital clinical pathways.
Trial registration number NCT05243485.
- acute coronary syndrome
- pre-hospital care
- emergency ambulance systems
- risk management
- point-of-care testing
Data availability statement
Data are available on reasonable request. The data that support the findings of this study will be made available 1 year after publication on reasonable request from the corresponding author (PJV).
Statistics from Altmetric.com
- acute coronary syndrome
- pre-hospital care
- emergency ambulance systems
- risk management
- point-of-care testing
WHAT IS ALREADY KNOWN ON THIS TOPIC
Emergency Medical Services studies have shown that prehospital risk stratification and triage decisions in patients with suspected non-ST-elevation acute coronary syndrome (NSTE-ACS) can be improved using clinical risk scores with point-of-care (POC)-troponin.
These clinical risk scores, including POC-troponin, lack external validation in the prehospital setting and there exists no prospective comparative analysis of the different risk scores within this setting.
WHAT THIS STUDY ADDS
This prespecified analysis of a prehospital, prospective, multicentre study conducted in the Netherlands between 2021 and 2022 showed that in the prehospital setting, the preHEART and HEART scores demonstrate good overall diagnostic performance in the determination of NSTE-ACS among patients who are transported to a hospital.
Both scores were superior to Troponin-only Manchester Acute Coronary Syndromes or POC-troponin-only methods.
HOW THIS STUDY MIGHT AFFECT RESEARCH, PRACTICE OR POLICY
Many countries are considering, or already implementing, clinical risk scores for patients with suspected NSTE-ACS in the prehospital setting.
This information will contribute to making an evidence-based choice regarding which tool to use.
Introduction
Chest pain suspicious for an acute coronary syndrome is one of the most common reasons for calling the Emergency Medical Services (EMS).1 2 At present, the prehospital ECG is used for identifying patients with ST-elevation acute coronary syndrome (STE-ACS). In many healthcare systems, based on the ECG, pathways are activated to facilitate primary percutaneous coronary intervention (PCI).3 However, in the majority of the patients suspicious for an ACS the prehospital ECG is non-diagnostic and these patients are suspected of having an acute coronary syndrome without ST-elevation (NSTE-ACS).4 5 Depending on the healthcare system, no further prehospital risk stratification is performed by the EMS and such patients are transferred to the nearest ED, with or without PCI facilities, for further diagnostic evaluation.6
In many EDs, clinical risk scores (clinical variables combined with a troponin) are used for risk stratification and can rapidly rule out a sizeable proportion of patients at low risk for NSTE-ACS.7–9 Although several clinical risk scores are available, the History, ECG, Age, Risk and Troponin score (HEART) and Troponin-only Manchester Acute Coronary Syndromes (T-MACS) are commonly used.7 10 In recent years, these clinical risk scores have been evaluated in the EMS setting including a point-of-care (POC)-troponin.11 12 Recently published prospective EMS studies using clinical risk scores to decide whether low-risk patients should not be transferred to the ED or whether patients at high risk for NSTE-ACS required immediate transfer to a PCI centre have shown that implementation of such scores for prehospital risk stratification are safe and feasible, can help to reduce ED crowding, improve logistics for patients diagnosed with NSTE-ACS and lower healthcare costs.13–18
The progression from using these scores in the prehospital setting led to adaptation of a new, specific prehospital score derived from the original HEART score: the prehospital History, ECG, Age, Risk and Troponin (preHEART) score. The results from the initial study were very promising but need to be examined in other populations.12 The primary aim of this study was to externally validate the diagnostic performance of the preHEART score against the ‘gold standard’ of NSTE-ACS diagnosis at hospital admission. The secondary aim of the study was to compare the diagnostic performance of the preHEART score with the HEART score, T-MACS and POC-troponin measurement alone for NSTE-ACS diagnosis.
Methods
Study design and setting
This was a prespecified analysis from the TRIAGE-ACS study (2021−2022). The design and results of the original study have been described previously.15 19 In short, the TRIAGE-ACS study was a multicentre, prospective, two-cohort study conducted in the EMS setting in the Netherlands. The aim of the study was to reduce the time from first medical contact to final invasive coronary diagnostics or culprit revascularisation by transferring patients classified as high risk for having NSTE-ACS directly to the ED of a PCI centre for further diagnostic workup. In patients suspected of NSTE-ACS, and the EMS paramedic decided to transfer the patient to the ED for diagnostic evaluation, further prehospital risk stratification was performed. For the first (observational) cohort (n=400), paramedics collected variables for the preHEART score but the ED destination was not chosen according to this. In the second (interventional) cohort (n=423), EMS paramedics prospectively analysed and scored all clinical variables of the three currently investigated clinical risk scores (preHEART, HEART and T-MACS) including on-site POC-troponin measurement and patients were transferred to an ED according to the results of the preHEART score. Therefore for the current analysis, assessment of the preHEART score was based on both cohorts, while the comparison of all three scores were based on the subset which had data collected only in the second cohort. Patients whom the paramedic decided did not need transfer to an ED did not receive prehospital risk stratification in either phase.
The comparison between the three clinical risk scores was performed only in patients with data available on all three risk scores. Follow-up was performed by reviewing medical records after 30 days to gain insight into hospital data and diagnoses.
Study population
All patients ≥18 years suspected of NSTE-ACS, presenting with EMS were eligible for inclusion. As noted above, patients were excluded if the EMS paramedic did not intend to transfer the patient to the ED (eg, decided to leave the patient at home or transfer to primary care for further evaluation) or if there was evident suspicion of a ST-elevation myocardial infarction (STEMI) on the prehospital ECG. The complete list of inclusion and exclusion criteria is provided in the online supplemental eTable 1.
Supplemental material
POC-troponin and clinical risk scores
For measuring troponin I, blood was collected on-site via intravenous access by an EMS paramedic and measured using a POC-analyzer ‘I-stat’ (Abbott Industries, Minneapolis, Minnesota, USA). The reportable range of the POC-analyzer is 0.00–50.00 ng/mL, and 17 µL of blood was required to fill the cardiac troponin-I cartridge. The assay had a limit of detection (LoD) of 0.02 ng/mL and a 99th percentile upper reference limit (URL) of 0.08 ng/mL. Analysing the blood sample took approximately 10 min.20 Patients with a POC-troponin measurement equal or below the LoD were labelled as low risk for NSTE-ACS. If the measurement was above the URL, patients were labelled as high risk for NSTE-ACS.
The clinical risk scores were applied as previously reported in the literature.12 21 22 The preHEART score and HEART score both included history, ECG, age, risk factors and POC-troponin. In the preHEART score, the cut-off values for age and POC-troponin differ from the HEART score and the risk factor scoring was simplified in the preHEART score to fit the prehospital setting (table 1). The total score for both the preHEART and HEART scores ranged between 0 and 10 points. Patients with a preHEART or HEART score ≤3 were labelled as low risk, 4–7 as intermediate risk and ≥8 as high risk for having NSTE-ACS.
Variables of the different clinical risk scores and points awarded
The T-MACS was calculated as a probability as follows: T-MACS=1/(1+e-(-4.65+1.828a+1.54b+0.849c+1.783d+1.878e+1.412f+0.084g)). Where, a represented acute ECG ischaemia (ST-depression or T-wave inversion), b represented worsening or crescendo angina, c represented pain radiating to the right arm or right shoulder, d represented pain associated with vomiting, e represented visible diaphoresis in the EMS setting, f represented hypotension (systolic BP <100 mm Hg) and g represented POC-troponin concentration. For the variables a to f, a value of ‘1’ was entered if the variable was present and a value ‘0’ if it was not present. If the T-MACS value is <0.02, the patient was classified as very low risk. If T-MACS is ≥0.02 and <0.05, the patient was classified as low risk. For T-MACS values between ≥0.05 and <0.95, the patient was classified as intermediate risk. If T-MACS is ≥0.95, the patient was classified as high risk for having NSTE-ACS.
Study end point
The primary endpoint of the study was the hospital diagnosis of NSTE-ACS. Adjudication of the diagnosis of NSTE-ACS was performed by applying current ESC guidelines and the fourth universal definition of myocardial infarction.23 24 NSTE-ACS was defined as non-ST-segment elevation myocardial infarction (NSTEMI) or unstable angina pectoris. The diagnosis was checked by two independent medical doctors. If no consensus was reached, a third medical doctor was involved for adjudication of the primary endpoint. The medical doctors adjudicating the diagnosis were blinded to the clinical risk scores calculated by the EMS paramedic. Secondary endpoints were major adverse cardiac events (MACE) within 3 days and 30 days after inclusion. MACE was defined as all-cause death, unplanned revascularisation (PCI or coronary artery bypass grafting) or ACS diagnosis.
Statistical analysis and sample size
As this was a prespecified analysis of a multicentre, cohort study, no specific power calculation was performed. For external validation of predictive models, at least 100 events and 100 non-events were required.25 The diagnostic performance was expressed as area under the receiver operating characteristic (AUROC) (overall diagnostic performance) including 95% CI, sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV), positive likelihood ratio (LR+) and negative likelihood ratio (LR−). Data were considered normally distributed if skewness and kurtosis were between −1 and 1. Continuous variables were expressed as means±SD or medians (IQR). Categorical variables were reported as frequencies and percentages. Differences in continuous variables between groups were assessed using an unpaired t-test or Mann-Whitney U test, while differences in categorical data were assessed with the χ2 test or Fisher’s exact test. The AUROCs are compared according to the method by DeLong et al.26 Two-by-two tables were used to calculate the sensitivity, NPV and LR− for identification of the lowest risk group for each clinical risk score, and specificity, PPV and LR+ for the identification of the highest risk group of each clinical risk score and were compared using McNemar’s test. Statistical significance was defined as a two-sided p value <0.05. Analyses were performed by using SPSS V.29.0 and R V.4.3.1 (R Foundation for Statistical Computing, Vienna, Austria).
Patient and public involvement
The TRIAGE-ACS was reviewed and supported by the Dutch national patient advisory group (Harteraad).
Results
Between April 2021 and December 2022, 1579 patients were screened for eligibility by the EMS. After application of exclusion criteria, 988 patients were eligible for inclusion. In 823 patients, data were available for the validation of the preHEART score (figure 1).
Flow chart for external validation of the prehospital History, ECG, Age, Risk and Troponin (preHEART) score. STEMI, ST-elevation myocardial infarction. HD, hemodynamic
Study population
The mean age of the study population was 66±13 years and 45% were female. The median time from symptom onset to POC-troponin measurement by the EMS was 135 min (IQR 65–360) and the median preHEART score was 5 (IQR 3–6). A final clinical diagnosis of NSTE-ACS was made in 29% (n=235). Of these 235 patients, 12 (2%) patients developed STEMI after inclusion by the EMS during diagnostic workup at the ED. The preHEART score was significantly higher in patients with NSTE-ACS than in patients with no NSTE-ACS (preHEART score 6 (IQR 5–8) vs 4 (IQR 3–5), p<0.001) (table 2 and online supplemental eTable 2).
Baseline characteristics
External validation of the preHEART score
The preHEART score classified 27% as low risk, with a sensitivity of 92.8% (95% CI 88.7 to 95.7) and NPV of 92.3% (95% CI 88.3 to 95.1). The preHEART classified 9% of the patients as high risk, with a specificity of 98.5% (95% CI 97.1 to 99.3) and PPV of 87.7% (95% CI 78.3 to 93.4) (table 3).
External validation of preHEART score
Direct comparison of clinical risk scores
Data on the HEART and T-MACS were only collected during the interventional cohort of the original study and available in 316/423 (74.7%) of the patients included in this phase (online supplemental eTables 3–6). In these patients, there was no evidence for a difference in overall diagnostic accuracy between the preHEART score and HEART score (AUROC 0.83 (95% CI 0.78 to 0.87) vs AUROC 0.80 (95% CI 0.74 to 0.85), p=0.19), and both scores were superior to T-MACS (AUROC 0.72 (95% CI 0.66 to 0.79), p≤0.001 and p=0.03, respectively) and POC-troponin alone (AUROC 0.71 (95% CI 0.64 to 0.78), p<0.001 and p=0.01, respectively) in the prehospital setting. No statistically significant difference was found between the overall diagnostic accuracy of the T-MACS and POC-troponin alone (p=0.66) (figure 2).
Receiver operating characteristic curves for clinical risk scores and POC-troponin and corresponding area under the curve. HEART, History, ECG, Age, Risk and Troponin; POC, point-of-care; preHEART, prehospital History, ECG, Age, Risk and Troponin; T-MACS, Troponin-only Manchester Acute Coronary Syndromes.
The preHEART score identified 67 patients (21%) as low risk with an NPV of 95.5% (95% CI 87.5 to 99.1). No difference was found compared with the HEART score (45 patients (14%) at low risk, NPV 91.8% (95% CI 80.4 to 96.8), p=1.00). Both were superior to the T-MACS (97 patients (31%) at low risk, NPV 83.5% (95% CI 75.8 to 89.1), respectively, p<0.001 and p=0.01) and POC-troponin (265 patients (84%) at low risk, NPV 77.0% (95% CI 74.0 to 79.7%), respectively, p<0.001 and p<0.001) (table 4).
Direct comparison of clinical risk scores
A total of 26 patients (8%) were identified as high risk with the preHEART score (PPV 92.3% (95% CI 74.9 to 99.1)). No differences were found compared with the T-MACS (26 patients (8%) at high risk, PPV 96.2% (95% CI 77.5 to 99.5), p=1.00) and POC-troponin (20 patients (6%) at high risk, PPV 100.0 (95% CI 83.2 to 100.0), p=0.21). All were superior compared with the HEART score (38 patients (12%) at high risk, PPV 81.8% (95% CI 68.5 to 90.3)) (table 4). The diagnostic performances of the original validation studies are mentioned in online supplemental eTable 7.
No differences were found in the AUROC of the different clinical risk scores between the incidence of index ACS and MACE within 3 days and 30 days (online supplemental eTable 8).
Discussion
We have externally validated the preHEART score as a tool for prehospital risk stratification in patients suspected of NSTE-ACS. Although the diagnostic performance in our study is less conclusive than in the original derivation study, our findings support the use of the preHEART score as a robust tool for prehospital risk stratification in patients suspected of NSTE-ACS. Among the prehospital clinical risk score we investigated, both the preHEART and HEART scores outperformed the T-MACS score and sole POC-troponin measurement in overall diagnostic accuracy of predicting the clinical diagnosis of ACS in the prehospital setting.
Our results show that the preHEART and HEART scores have good diagnostic performance to be used as prehospital risk stratification tools.27 Compared with other validation studies, the sensitivity and NPV are lower in this study.12 16 28 Although the performance of risk scores is known to be lower in external validation studies, we observed a few differences in populations that could have influenced the sensitivity and NPV in our study.25 First of all, the higher incidence of ACS in our study population will play an important role in the lower NPV and AUROC.29 The higher incidence was most likely a result of the inclusion and exclusion criteria (only patients in whom the EMS paramedic had the intention to transfer the patient to the ED were included). Second, there were baseline differences between the study populations (such as a shorter time from symptom onset to POC-troponin measurement in this study), which could have led to more false negative patients and misclassification into a low-risk group.
The T-MACS has been validated and is widely used in the ED setting.21 Recently, the T-MACS was validated for the first time in the prehospital setting. This study showed excellent sensitivity and NPV for the T-MACS and superiority to the HEART score.30 However, the primary endpoint (acute myocardial infarction type 1) of the study was very specific and prevalence was low. Nonetheless, T-MACS also outperformed the HEART score in the secondary endpoint, MACE within 30 days. Notably, the MACE definition did not encompass unstable angina pectoris. The present analysis shows that the overall diagnostic performance of the T-MACS in the prehospital setting is moderate and not of added value to POC-troponin measurement alone. The NPV and sensitivity are significantly lower as compared with its own validation study and to the other clinical risk scores in this study. A possible explanation for the low NPV and sensitivity could be that we used a different primary endpoint (including unstable angina pectoris) and the prevalence was higher in this study. Furthermore, the history and physical examination variables of the T-MACS are more focused on severe signs and symptoms of myocardial ischaemia (vomiting, sweating, hypotension) and the risk score does not include age or other cardiovascular risk factors. The T-MACS may therefore be not sensitive enough for the complete range of patients with NSTE-ACS (including patients with short, transient ischaemia and unstable angina pectoris). Nevertheless, the T-MACS has an excellent specificity and PPV to identify patients at high risk for NSTE-ACS.31
Several clinical risk scores are currently under investigation in the prehospital setting. In this study, we have compared the three most commonly used scores. Our findings indicate that the preHEART and HEART demonstrate the highest overall diagnostic accuracy. However, the choice of which score is best to use depends on the setting, population and triage decisions. To identify low-risk patients, the preHEART and HEART have a superior sensitivity and NPV compared with the T-MACS and sole POC-troponin measurement. However, in our study, the clinical risk scores did not meet the required sensitivity and NPV to be used as a rule-out tool.4 32 Notably, the original TRIAGE-ACS study was conducted in a higher risk patient population; before using one of the clinical risk scores for prehospital risk stratification, the EMS paramedic had already determined that the patient needed to be transported to the ED for further diagnostic evaluation. Therefore, the diagnostic performance of the clinical risk scores as a rule-out tool may be underestimated in our study. For triage decisions in patients at high risk for having NSTE-ACS, the preHEART, T-MACS and POC-troponin have a statistically significant better PPV and specificity compared with the HEART score.
While the preHEART and HEART scores share certain similarities and no difference was found in overall diagnostic performance, they diverge in their approach to assessing risk factors. Patients often lack precise awareness of their complete medical history and medication usage, and EMS paramedics lack access to the patients’ medical records or recent laboratory results to corroborate the presence of specific risk factors. Moreover, in emergency situations, patients may not be able to provide detailed information about their medical history or risk factors. In light of these challenges, the preHEART score, designed specifically for the prehospital setting, incorporates specific adjustments and might be simpler to use in the prehospital setting.
In its original validation study, the threshold for the HEART score’s high-risk group was 7.8 Since the preHEART score is derived from the HEART score, we have chosen to use the same threshold for the HEART score as for the preHEART score (≥8). Using a lower cut-off value for the HEART score compared with the preHEART score would disadvantage the HEART score, as the specificity and PPV of the high-risk group would be reduced. Nonetheless, if a cut-off value of 7 was used for the HEART score, it would not alter the conclusion of our study.
Future perspective
The overall diagnostic performance of current clinical risk scores is still lower in the prehospital setting as compared with the ED. This might be caused by differences in troponin assays (current EMS studies used conventional POC-troponin assays), shorter time between symptom onset and troponin measurements and differences in patient populations.7–9 28 Recently, high-sensitive POC-troponin assays became commercially available and are currently investigated (URGENT 2.0, NCT04904107). These assays might further improve the overall diagnostic performance to identify patients with NSTE-ACS. Furthermore, desired safety and accuracy levels are well defined for rule-out and rule-in algorithms at the ED, but not well determined for the prehospital setting. As patients with suspected NSTE-ACS may present to a GP, EMS or ED, they comprise a major burden on healthcare resources. Therefore, balancing safety with the clinical and health economic impact of these new prehospital strategies is needed, especially when good safety-netting is available for the patients (primary care, peer support).
Limitations
This prespecified analysis from a prospective cohort study suffers from limitations inherent to the study design. Although there were sufficient events and non-events in each population of the clinical risk scores for validation, population sizes were relatively small. In the second cohort of the original study, variables were scored to analyse the different clinical risk scores. These data were only available in 316/423 (74.7%) of the patients. It is unclear why these variables were not collected in all patients by the EMS paramedics and could therefore influence our results. Current validation studies all used different end points and follow-up duration, which make external validation and comparison of the performance of each risk score difficult. To overcome these differences, we performed analyses for each different endpoint and follow-up duration used in the validation studies (eg, ACS diagnosis at hospital admission, MACE within 3 days and 30 days), but found no major differences in performance of each risk score.
Conclusion
On external validation, the preHEART demonstrates good overall diagnostic performance as a prehospital risk stratification tool. Both the preHEART and HEART scores have better overall diagnostic performance compared with T-MACS and sole POC-troponin measurement. These data support the implementation of clinical risk scores in prehospital clinical pathways.
Data availability statement
Data are available on reasonable request. The data that support the findings of this study will be made available 1 year after publication on reasonable request from the corresponding author (PJV).
Ethics statements
Patient consent for publication
Ethics approval
The study was reviewed and approved by the regional and local Medical Ethics Committees of the participating medical centers (MEC-U AW22.043/W19.190).
References
Supplementary materials
Supplementary Data
This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.
Footnotes
Handling editor Edward Carlton
Correction notice Since this paper first published, figure 1 has been replaced with the correct artwork.
Contributors Study planning and design: JPAD, AK, DS, RH, EH, HT, PvdH, MvV, LD, PT, PJV. Analysis and interpretation of results: JPAD, AK, MeF, RE, MvV, LD, PT, PJV. Draft manuscript preparation: JPAD, AK, DS, RH, EH, HT, MeF, RE, PvdH, MvV, LD, PT, PJV. Guarantor for overall content: JPAD, PJV.
Funding The study is funded by ZonMw, the Dutch Organisation for Health Research and Development, specifically through the grant program 'Topspecialistische Zorg en Onderzoek' (grant number 10070012010001).
Competing interests None declared.
Patient and public involvement Patients and/or the public were involved in the design, or conduct, or reporting, or dissemination plans of this research. Refer to the 'Methods' section for further details.
Provenance and peer review Not commissioned; externally peer reviewed.
Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.