Article Text
PDF
Abstract
Background The Canadian Syncope Risk Score (CSRS) has been proposed for syncope risk stratification in the emergency department (ED). The aim of this study is to perform an external multicenter validation of the CSRS and to compare it with clinical judgement.
Methods Using patients previously included in the SyMoNE database, we enrolled subjects older than 18 years who presented reporting syncope at the ED. For each patient, we estimated the CSRS and recorded the physician judgement on the patients’ risk of adverse events. We performed a 30-day follow-up.
Results From 1 September 2015 to 28 February 2017, we enrolled 345 patients; the median age was 71 years (IQR 51–81), 174 (50%) were men and 29% were hospitalised. Serious adverse events occurred in 43 (12%) of the patients within 30 days. The area under the curve of the CSRS and clinical judgement was 0.75 (95% CI 0.68 to 0.81) and 0.68 (95% CI 0.61 to 0.74), respectively. The risk of adverse events of patients at low risk according to the CSRS and clinical judgement was 6.7% and 2%, with a sensitivity of 70% (95% CI 54% to 83%) and 95% (95% CI 84% to 99%), respectively.
Conclusion This study represents the first validation analysis of CSRS outside Canada. The overall predictive accuracy of the CSRS is similar to the clinical judgement. However, patients at low risk according to clinical judgement had a lower incidence of adverse events as compared with patients at low risk according to the CSRS. Further studies showing that the adoption of the CSRS improve patients’ outcomes is warranted before its widespread implementation.
- syncope
- diagnosis
- arrythmia
Data availability statement
Data are available upon reasonable request. Data are available upon reasonable request to giorgic2@gmail.com.
Statistics from Altmetric.com
Key messages
What is already known on this subject
Several risk stratification tools have been developed to stratify the risk of adverse events of patients with syncope in the emergency department (ED), but none has shown to perform better than clinical judgement at predicting short-term serious outcomes.
The Canadian Syncope Risk Score (CSRS) was recently published as a new decision rule, showing high sensitivity and accuracy in the Canadian setting.
What this study adds
In the present multicenter validation study, outside the Canadian setting, the predictive accuracy of the CSRS is similar to the clinical judgement and the latter allows discharging from the ED fewer patients who will have adverse events at follow-up.
Further studies showing that the adoption of the CSRS improve patients’ outcomes is warranted before widespread implementation of the CSRS in clinical practice.
INTRODUCTION
Syncope is a common reason for presenting to the emergency department (ED) and it may be the presenting symptom of a variety of clinical conditions spanning from benign to life-threatening diseases.1–3 Nevertheless, the aetiology of a syncopal episode is not always easy to be determined. Indeed, only in a small percentage of cases, ED physicians are able to identify a precise cause and a considerable proportion of patients ends up being hospitalised to exclude an underlying cardiac disease.3 Recent data show, however, that hospital admission has an overall low diagnostic yield and high costs.4 Therefore, early risk stratification is crucial to determine which patients might benefit from observation or immediate further investigations. Several risk-stratification tools have been developed to stratify the risk of adverse events of patients with syncope in the ED.5–9 These tools rely on medical history, examination, ECG findings and cardiac biomarkers, but none has shown to perform better than clinical judgement at predicting short-term serious outcomes.10 11 Therefore, the use of risk-stratification tools is no more recommended in the assessment of ED patients with syncope.12 13
The Canadian Syncope Risk Score (CSRS) was recently proposed as a new risk-stratification tool. It was developed by prospectively enrolling, in a multicenter study, a large cohort of patients presenting to Canadian EDs for syncope with the aim of identifying 30-day serious adverse events.14 In this particular setting, where the hospital admission rate for syncope is one of the lowest recorded in the Western world,15 the CSRS has shown to have high sensitivity and accuracy.14 16
To our knowledge, the CSRS has not been validated outside Canada yet, and it is not known if its high diagnostic yield is better than clinical judgement.
The aim of this study is to perform an external validation of the CSRS to predict serious outcomes in patients with syncope and to compare it with clinical judgement.
Methods
Population
We performed an external validation of the CSRS on the population enrolled in the SyMoNE (Syncope Monitoring and Natriuretic peptides in the ED) study. Briefly, the SyMoNE study was a prospective multicentre investigation conducted in six hospitals (four teaching hospitals and two community hospitals) in northern Italy, designed to assess the roles of brain natriuretic peptides and ECG monitoring in the ED management of patients with syncope.17
We included subjects older than 18 years of age who presented reporting syncope at the EDs of the participating hospitals from 1 September 2015 to 28 February 2017. Exclusion criteria were (1) loss of consciousness (LOC) following head trauma, (2) nonspontaneous recovery of consciousness, (3) episodes of falling, dizziness or lightheadedness without LOC, (4) LOC associated with alcohol or drug abuse, (5) pregnancy or breastfeeding status, (6) inability to provide informed consent to study participation or to complete follow-up, (7) syncope as an underlying symptom of an acute condition diagnosed in the ED or requiring therapeutic intervention irrespective of syncope (ie, acute myocardial infarction, pulmonary embolism, aortic dissection, cerebral haemorrhage, carotid sinus syndrome or arrhythmia diagnosed before ECG monitoring in the ED), (8) nonsyncopal LOC (ie, history of epilepsy), (9) poor prognosis in the next 30 days for a pre-existing condition besides syncope (eg, neoplasms).
This study complied with the Declaration of Helsinki and received approval from the Ethics Committee of the coordinating centre (Ospedale Sacco, Milano, Italy, approval number 608/2015). All participants provided written consent, oral consent to telephone interviews, as applicable.
Outcomes
We prospectively followed patients by telephone at 30 days to assess the occurrence of any of the following adverse events: (1) all-cause and syncope-related death, (2) ventricular fibrillation, (3) sustained and symptomatic nonsustained ventricular tachycardia, (4) sinus arrest with cardiac pause >3 s, (5) sick sinus syndrome with alternating bradycardia and tachycardia, (6) second-degree type 2 or third-degree atrioventricular block, (7) permanent pacemaker (PM) or implantable cardioverter defibrillator (ICD) malfunction with cardiac pauses, (8) aortic stenosis with valve area ≤1 cm2, (9) hypertrophic cardiomyopathy with outflow tract obstruction, (10) left atrial myxoma or thrombus with outflow tract obstruction, (11) myocardial infarction, (12) pulmonary embolism, (13) aortic dissection, (14) occult haemorrhage or anaemia requiring transfusion, (15) syncope or fall resulting in major traumatic injury (requiring admission or procedural/surgical intervention), (16) PM or ICD implantation, (17) cardiopulmonary resuscitation, (18) syncope recurrence with hospital admission, (19) cerebrovascular events. These sets of criteria were identified by an international panel of syncope researchers and experts13 18 and are being adopted in international prospective studies on syncope risk stratification in the ED, including the derivation of the CSRS.14 Therefore, although this study has a retrospective design, all the data to calculate the CSRS and on the occurrence of adverse events were available and collected prospectively in the SyMoNe database.
Patient and public involvement
Outcome measures were informed by previous consensus papers that involved patient representatives and evaluation of patient experience.18 19 Patients were not involved in the development of the research question, recruitment to and conduct of the study. We have no plans to disseminate the findings to study participants.
Canadian syncope risk score
The CSRS includes nine predictors from clinical evaluation (1) predisposition to vasovagal syncope, (2) history of heart disease, (3) any systolic pressure reading in the ED (<90 or >180 mm Hg) investigations, (4) troponin level >99th percentile for normal population ECG, (5) abnormal QRS axis, (6) prolonged QRS interval, (7) prolonged corrected QT interval and ED diagnosis, (8) vasovagal or (9) cardiac syncope. Each predictor is a binary variable with a score that ranges from −2 to +2 and the score is calculated as the sum of the points assigned to each predictor (online supplemental appendix table 1). The score ranges from −3 to 11, with a risk of a serious adverse event within 30 days ranging from 0.4% for a score of −3% to 83.6% for a score of 11. The authors suggest to consider patients to be at very low risk (estimated risk of 30-day adverse events <1%) if the score is −3 or −2; at low risk, if the score is −1 or 0; at medium risk, if the score is 1, 2 or 3; at high risk, if the score is 4 or 5; at very high risk if the score is >6.
Supplemental material
Score calculation
For each enrolled patient, we assessed or extrapolated the predictors of the CSRS. As the ED physicians managed patients regardless of their participation in the study, for patients who did not have troponin measured during the ED evaluation, we assumed that all missing values were within the normal range, as done by the authors in the score derivation and validation studies.14 16 Moreover, in the SyMoNe database, we had recorded the ED physician judgement on the patients’ risk of short-term adverse events, classified as low, high and intermediate (ie, neither high nor low) according to their clinical gestalt. Since the aim of a risk prediction tool is to stratify the risk of patients before the diagnosis is established, we assigned −2 points (final diagnosis of vasovagal syncope) and 2 points (final diagnosis of cardiac syncope) if the ED physician categorised the patients’ risk of adverse events as low and high, respectively. We assigned 0 points to patients deemed at intermediated risk.
We also performed a sensitivity analysis defining as ‘clinical judgement’ the emergency clinician’s decision to either admit patients to the hospital (high risk) or discharge them from the ED (low risk), in a similar fashion to what we did in a previous study on the comparison between risk stratification tools and clinical judgement.10 We assigned +2 points (final diagnosis of cardiac syncope) to patients admitted to the hospital and –2 points (final diagnosis of vasovagal syncope) to patients discharged from the ED.
We classified patients who left the ED against medical advice to be at high risk of adverse events.
We assessed the presence of all the other predictors as done in the CSRS derivation study. Indeed, all the variables that allow calculating the CSRS were collected prospectively in the SyMoNe database, as they had been previously defined in consensus statements.13 18
Statistical analysis
We expressed continuous variables as median and IQR and categorical variables as frequencies and percentage. We assessed the prognostic accuracy of the CSRS and clinical judgement by calculating the area under the receiver operating characteristic curve (AUC), sensitivity, specificity, positive predictive value (PPV) and negative predictive value (NPV) with 95% CI. For each risk category, we also assessed the risk of adverse events and their 95% CI. We considered a p value <0.05, two-tailed, as statistically significant. We performed the analyses using the SAS statistical software (release V.9.4; SAS Institute, Cary North Carolina, USA).
Results
During the study period, we screened 414 patients for inclusion. After excluding 69 patients due to lack of follow-up data (43 patients) or because the aetiological diagnosis of syncope was identified in the ED (26 patients), we enrolled 345 patients. The median age of the study population was 71 years (IQR 51–81) and 174 (50%) were men. A total of 102 patients (29%) were hospitalised. Table 1 shows the sample characteristics. Troponin was measured in 178 patients and it was positive in 39. The characteristics of the 43 patients lost to follow-up are comparable to the included ones (online supplemental appendix table 2).
Patients’ characteristics
At the 30-day follow-up, 43 patients (12%) experienced at least one adverse event and 5 patients died. Online supplemental appendix table 3 describes the prevalence of the CSRS predictors in patients with and without events at follow-up. Among patients with adverse events, the prevalence of heart disease, high or low blood pressure, elevated troponin, abnormal QRS axis, prolonged QT interval and ED diagnosis of cardiac syncope were significantly higher than among patients without adverse events.
The AUC of the CSRS and clinical judgement in identifying serious adverse events was 0.75 (95% CI 0.68 to 0.81) (figure 1) and 0.68 (95% CI 0.61 to 0.74), respectively.
CSRS ROC curve with AUC. CSRS, Canadian Syncope Risk Score; AUC, area under the curve; ROC, receiver operating characteristic.
Table 2 reports the risk of adverse events of patients in the different risk categories according to the CSRS. None of the 83 patients in the very low CSRS risk category and 6.7% (95% CI 3.6% to 11%) of the 196 patients in the very low and low risk category (ie, CSRS ≤0 as suggested as a cut-off to discharge patients from the ED by the CSRS authors) had 30-day serious outcomes. The risk of adverse events in patients at low risk according to clinical judgement was 2% (95% CI 0.3% to 7%).
Risk of adverse events according to the CSRS risk categories
The proportion of adverse events in patients at intermediate and high risk was 14% and 23%, respectively (table 3).
Risk of adverse events according to clinical judgement
Table 4 reports the sensitivity, specificity, PPV and NPV with 95% CI of the CSRS and clinical judgement according to the different risk categories.
Sensitivity, specificity, positive predictive value and negative predictive value of the CSRS risk categories and clinical judgement
The AUC of clinical judgement in the sensitivity analysis performed assigning 2 points to the emergency clinician’s decision to admit the patients to the hospital and −2 points to the decision to discharge them from the ED was 0.74 (95% CI 0.67 to 0.81). The sensitivity, specificity, PPV and NPV were 72% (95% CI 56% to 84%), 76% (95% CI 71% to 81%), 30% (95% CI 22% to 40%) and 95% (95% CI 91% to 97%), respectively.
Discussion
In this study, we performed an external validation of the CSRS and we compared it to clinical judgement in predicting 30-day adverse events in ED patients with syncope. The results showed that the accuracy of the CSRS is similar to clinical judgement. Moreover, the assessment of CSRS diagnostic accuracy at different risk cut-offs shows that the sensitivity is high in the very low risk, but it is fair in the low-risk category. Indeed, considering at low-risk patients with a CSRS≤0, which, as suggested by Thiruganasambandamoorthy et al,14 16 would identify patients who could be discharged from the ED, the sensitivity of the score is only 70%, with up to 6.7% of patients experiencing adverse events. We do believe that such a rate is too high to safely discharge patients from the ED. On the contrary, adverse events occurred in only 2% of patients deemed at low risk by the ED physician. Our data show that the incidence of adverse events is similar if we compare patients at low risk according to the clinical judgement with patients at very low risk according to the CSRS. However, in patients with CSRS ≤0 (cut-off used to discharge patients from the ED, as suggested by the rule derivation authors), we observed a higher incidence of adverse events when compared with patients at ‘low risk’ according to clinical judgement.
The authors of the CSRS derived the score with the aim of overcoming the limitations of the previous risk-stratification tools.10 20 21 Thiruganasambandamoorthy et al enrolled 4030 patients.14 In this study, serious adverse events were found in 3.6% of patients, and 9.5% were hospitalised. The incidence of adverse events was less than 2% for a score of ≤0, considered as a low-risk cut-off. Sensitivity and specificity were 99.2% and 25.4%, respectively, with a cut-off ≥ −2, and 97.7% and 45.1% with a cut-off ≥ −1. In a recently published validation conducted at nine EDs across Canada, the CSRS showed a very good predictive accuracy, with an AUC of 0.91.16 Compared with other risk-stratification tools, the CSRS has several advantages. First, the authors derived it from one of the largest datasets currently available. Second, the choice to provide the probability of adverse events based on the score, instead of establishing an arbitrary cut-off, could increase its clinical utility. Indeed, a continuous risk estimate allows greater flexibility, as the clinician could use the score with different cut-offs according to the risk he/she is willing to tolerate for the single patient. Furthermore, the CSRS explores the risk of serious adverse events over 30 days, differently from other risk-stratification tools. This is pivotal to improve the clinical relevance of the score, because about 10% of patients have adverse events in the first 7–30 days after the occurrence of syncope.19 22 23 On the other side, the application of the CSRS might have some limitations. Indeed, after excluding an acute and potentially life-threatening condition, such as aortic dissection, pulmonary embolism or occult bleeding, the ED physician should assess the risk of adverse events.12 This happens after the patient’s first assessment with history, physical examination, blood tests, imaging and ECG, but before the ECG monitoring or a presumptive diagnosis and disposition is made. In the CSRS derivation study, although the authors excluded patients who had a serious adverse event identified during the index ED visit, it is unclear whether the detection of an arrhythmia at ECG monitoring was considered an adverse event or rather part of the final diagnosis. In addition, one of the items of the score is related to the clinical diagnosis of syncope (cardiac or vasovagal syncope), and this makes the application and reproducibility of the CSRS difficult when it would be more needed: before the clinical picture is enough to formulate a diagnosis.
All the above considerations might explain why in our study the CSRS performed worse than in the Canadian cohorts. Also, it must be pointed out that the score was derived in Canada, where hospitalisation rates after syncope are the lowest in the western world.15 This could be due to several factors, including access to the ED of patients with low-risk syncope. Indeed, in the derivation study, hospital admission occurred in only 9.5% of patients and 30-day adverse events in 3.6%. In contrast, in our study, both the adverse event rate and the percentage of patients admitted were higher (12.5% and 29%, respectively). The two cohorts differed also for the baseline characteristics. Our population turned out to be older and with a higher prevalence of comorbidities. This highlights the importance of carefully evaluate the broad and generalised applicability of the score in different clinical settings.
The current analysis confirms the results of a recently published multicentre study that assessed different prediction tools in syncope and showed that none of the scores brings a relevant improvement to the early judgement of the clinician.24 Such results warrant caution in the application of the CSRS. Indeed, to be able to improve the current practice, every tool should prove to have a better accuracy than clinical judgement and to improve patients’ outcomes when compared with the standard evaluation in a randomised controlled trial.25
Limitations
We should acknowledge some limitations of this study. First, the retrospective nature of this external validation analysis represents a limitation on the interpretation of the results. However, we validated CSRS on a population that we had enrolled prospectively and had most of the clinical and electrocardiographic data, including score predictors, reproducing the derivation study design and setting. Second, our work-up may not precisely reflect what the authors did in the derivation study, as many patients in our population were monitored for several hours, and this may have increased the number of events recorded. Third, instead of the ED diagnosis of cardiac or vasovagal syncope, we assigned 2 and −2 points according to the ED physician judgement on the patients’ risk of short-term adverse events. We decided to use clinical judgement as an estimate of ED diagnosis because about 40% of patients are discharged from the ED or admitted to hospital with a diagnosis of ‘unexplained syncope’.14 In addition, including the diagnosis in a risk stratification tool that should help the clinician estimate the risk of patients in whom the diagnosis is unclear would make the score less useful in clinical practice. Fourth, the definition of clinical judgement as the decision to admit or discharge the patients after the diagnostic work-up might be criticised, as hospitalisation might not be related to syncope itself but to other conditions such as social problems, trauma and so on. Finally, similar to the derivation study, troponin values were not available for some patients. Based on what was done in the original CSRS development and validation study, the missing values were interpreted as normal, also in view of the potential reasons why these dosages were not performed (eg, younger patients, with few comorbidities, or in patients with frankly noncardiac syncopal episodes).
Conclusion
This study represents the first validation analysis of CSRS outside Canada. In our cohort, the overall predictive accuracy of the CSRS is similar to clinical judgement. However, patients at low risk according to clinical judgement had a lower incidence of adverse events as compared with patients at low risk according to the CSRS. Further studies showing that the adoption of the CSRS improve patients’ outcomes is warranted before widespread implementation of the CSRS in clinical practice.
Data availability statement
Data are available upon reasonable request. Data are available upon reasonable request to giorgic2@gmail.com.
Ethics statements
Acknowledgments
The authors would like to acknowledge Elena Callisto, Sandra Cambursano, Gianluigi Citterio, Elisa Fiorini, Diego La Greca, Giada Mammoliti, Roberto Menè, Maura Minonzio, Barbara Parodi, Paola Pasquino, Lorenzo Porta, Marina Rivetti, Sonia Seghezzi, Claudia Silva, Antonio Voza for their valuable help in patients’ recruitment and data collection.
References
Supplementary materials
Supplementary Data
This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.
Supplementary Data
This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.
Footnotes
Handling editor Richard Body
Correction notice Since this article was first published online the institution for Dr Furlan and Dr Dipaola has been updated.
Contributors MS, GT, GCa and GCo designed the study; PV, FD, LF, EMF, FR, IC, KC, SR, MR and AV collected the data; MS, GT, GCa, GCo analysed and interpreted the data; MS, GT, GCa and GCo drafted the manuscript; all the authors critically revised the manuscript for important intellectual content; GCa provided statistical expertise.
Funding The authors have not declared a specific grant for this research from any funding agency in the public, commercial or not-for-profit sectors.
Competing interests None declared.
Provenance and peer review Not commissioned; externally peer reviewed.
Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.