Objectives To establish the accuracy of emergency department (ED) nursing staff risk assessment using an established chest pain risk score alone and when incorporated with presentation high-sensitivity troponin testing as part of an accelerated diagnostic protocol (ADP).
Design Prospective observational study comparing nursing and physician risk assessment using the modified Goldman (m-Goldman) score and a predefined ADP, incorporating presentation high-sensitivity troponin.
Setting A UK District ED.
Patients Consecutive patients, aged ≥18, with suspected cardiac chest pain and non-ischaemic ECG, for whom the treating physician determined serial troponin testing was required.
Outcome measures 30-day major adverse cardiac events (MACE).
Results 960 participants were recruited. 912/960 (95.0%) had m-Goldman scores recorded by physicians and 745/960 (77.6%) by nursing staff. The area under the curve of the m-Goldman score in predicting 30-day MACE was 0.647 (95% CI 0.594 to 0.700) for physicians and 0.572 (95% CI 0.510 to 0.634) for nursing staff (p=0.09). When incorporated into an ADP, sensitivity for the rule-out of MACE was 99.2% (95% CI 94.8% to 100%) and 96.7% (90.3% to 99.2%) for physicians and nurses, respectively. One patient in the physician group (0.3%) and three patients (1.1%) in the nursing group were classified as low risk yet had MACE. There was fair agreement in the identification of low-risk patients (kappa 0.31, 95% CI 0.24 to 0.38).
Conclusions The diagnostic accuracy of ED nursing staff risk assessment is similar to that of ED physicians and interobserver reliability between assessor groups is fair. When incorporating high-sensitivity troponin testing, a nurse-led ADP has a miss rate of 1.1% for MACE at 30 days.
Trial registration number Controlled Trials Database (ISRCTN no. 21109279).
- cardiac care, diagnosis
- cardiac care, acute coronary syndrome
- nursing, emergency departments
Statistics from Altmetric.com
What is already known on this subject?
Few studies have investigated the role of nursing staff in the assessment of low-risk patients with chest pain. Advanced nursing interventions during initial patient assessment have been proven to reduce time to treatment and diagnosis, improve patient flow through the emergency department and reduce length of stay across a wide variety of emergency presentations. Nursing staff may therefore be an underused resource in mitigating crowding.
What might this study add?
This prospective, single-centre observational study demonstrates that emergency department nursing staff risk assessment using an established chest pain risk score is similar to that of emergency department physicians. When combining nursing risk stratification with presentation high-sensitivity troponin testing, a nurse-led accelerated discharge protocol would have a miss rate of 1.1% for major adverse cardiac events at 30 days. This finding, together with fair interobserver reliability of nursing and physician assessments in the identification of low-risk patients, suggests the future role of nursing staff in rapid rule-out pathways holds promise.
Chest pain is one of the most common complaints of patients presenting to the emergency department (ED), with approximately one million visits per year in the UK. The majority of patients require prolonged assessment prior to safe discharge despite the fact that only 15–25% of these patients have a final diagnosis of acute coronary syndrome (ACS).1 Recently, accelerated diagnostic protocols (ADPs) have successfully incorporated chest pain risk scores with early biomarker testing to identify those patients at low risk of major adverse cardiac events (MACE) who may be suitable for early discharge.2–6
Advanced nursing interventions during initial patient assessment have been proven to reduce time to treatment and diagnosis, improve patient flow through the ED and reduce length of stay across a wide variety of emergency presentations.7 It is also evident that chest pain-specific risk scores, such as a modified Thrombolysis in Myocardial Infarction (TIMI) score,8 can improve the accuracy of nursing assessments.9 Yet, the ability of ED nursing staff to safely risk-stratify low-risk patients with suspected ACS who may be suitable for early rule-out biomarker testing and therefore early discharge has never been investigated. Consequently, ED nursing staff remain a potentially underused resource in the assessment of chest pain.
This study aimed to establish the diagnostic accuracy of ED nursing staff risk assessment using an established chest pain risk score (modified Goldman (m-Goldman)) alone and when incorporated with presentation high-sensitivity troponin testing as part of a nurse-led ADP. The secondary aim was to evaluate the interobserver reliability of nursing and physician assessments within the m-Goldman risk score.
This prospective observational clinical trial was designed to assess the predefined Triage Rule-out Using high-Sensitivity Troponin (TRUST) ADP. This ADP incorporates a non-ischaemic ECG, an m-Goldman risk score10 and a single-presentation high-sensitivity troponin (hs-cTnT) result (table 1). The study protocol was designed to be truly pragmatic in order to enhance the widespread applicability of the study results:11 treating ED physicians and nursing staff performed m-Goldman risk scores, real-time sample processing and 24/7 recruitment. Results from physician assessment using the TRUST ADP have been published previously.6 The study was designed using the Standards for Reporting Diagnostic Accuracy (STARD).12
Study setting, recruitment and data collection
Poole NHS Foundation Trust is a UK District General; the ED has approximately 62 000 new patient attendances per year. Patients with suspected ACS are managed according to the local hospital protocol, which involves risk assessment by ED physician staff using the m-Goldman risk score and blood drawn for hs-cTnT at 6 h after presentation. As part of the study protocol, blood was also taken at presentation for hs-cTnT analysis. While historical clinical protocols, at the time of this study, did not include troponin measurement at presentation, this had the benefit of ensuring that treating physicians were blinded to the initial hs-cTnT result to avoid selection bias and observation bias.13
The fifth-generation Roche ELECSYS hs-cTnT assay (Roche, Switzerland), which has a 99th percentile of 14 ng/L and 10% coefficient of variation of <10% at 9 ng/L, was used for both presentation and reference (6 h) samples. During initial assessment, clinical staff drew blood for routine admission samples and an additional 3.5 mL of whole blood in a pre-labelled study-specific serum settling tube for hs-cTnT analysis. All serum samples were tested in real time.
Consecutive patients attending the ED with suspected ACS were prospectively screened from July 2012 to August 2013. Patients were included if they were ≥18 years of age and had at least 5 min of chest pain suggestive of ACS, and for whom the treating physician determined inpatient evaluation was required. No patient was observed within the ED to await 6 h blood draws due to the UK national healthcare target that patients must be seen, treated, admitted or discharged within 4 h. Possible cardiac symptoms included acute chest, epigastric, neck, jaw or arm pain, or discomfort or pressure without an apparent non-cardiac source, in accordance with the American Heart Association case definitions.14 Patients were excluded if any of the following were present: ST segment elevation myocardial infarction or left bundle branch block not known to be old; ECG changes diagnostic of ischaemia (ST segment depression ≥1 mm or T-wave inversion consistent with the presence of ischaemia); arrhythmias (new-onset atrial fibrillation, atrial flutter, sustained supraventricular tachycardia, second-degree or complete heart block or sustained or recurrent ventricular arrhythmias); hs-cTnT not suitable for analysis (eg, haemolysis); age ≥80 years; atypical symptoms in the absence of chest discomfort; a clear non-ACS cause for chest pain was found at presentation (eg, pulmonary embolism, pneumonia, aortic dissection); another medical condition requiring hospital admission, refusal or inability to give informed consent; non-English speaking; pregnancy; and renal failure requiring dialysis or inability to be contacted after discharge.
ED physician staff undertook initial ECG evaluation as part of clinical care, later confirmed by two local cardiologists (nursing staff were not expected to undertake ECG evaluation). Patients with ECG evidence of acute myocardial infarction (AMI) or acute ischaemia were immediately defined as high risk in accordance with Goldman's original rule and therefore not recruited.
Data were collected prospectively using a published data dictionary.15 ED nursing staff undertaking initial assessment were asked to record the m-Goldman risk score on a case report form, at the time of patient presentation to the ED. Nursing staff were experienced in the primary assessment and triage of ED patients with chest pain but had no formal training in the use of the m-Goldman score. Consequently, they were provided with written explanatory notes on how to complete the risk score. The nursing risk score was kept separate from the clinical notes in a coloured envelope and removed by a member of the research team at the earliest opportunity. Attending ED physicians completed an identical m-Goldman risk score as part of routine clinical assessment on a separate clinical assessment form.
Follow-up was undertaken by independent review of hospital electronic patient records, summary of health records from the patient's general practitioner (GP) obtained at least 6 months after attendance and a national clinical records search (which identifies death). Where a participant had not attended hospital follow-up and/or a GP had failed to provide a health record/not GP-registered, the patient was regarded as lost to follow-up.
The index test was the m-Goldman score evaluated by both physicians and ED nursing staff. In order to establish the potential diagnostic accuracy of the m-Goldman score within a nurse-led ADP, the secondary index test of the TRUST ADP was used. This defined a patient as ‘low risk’ if all of the following conditions were satisfied at presentation: an m-Goldman score of ≤1 (table 1), a non-ischaemic ECG and a single central laboratory hs-cTnT of <14 ng/L.
The endpoint was the presence of MACE occurring within 30 days of hospital attendance (including the index visit). MACE included death due to ischaemic heart disease, cardiac arrest, symptom-induced revascularisation, cardiogenic shock, ventricular arrhythmia, high-degree atrioventricular block needing intervention and AMI. This definition is consistent with previous large-scale research analysing the diagnostic accuracy of ADPs.3
The presence of AMI was defined according to the Third Universal Definition of MI, which states that a rise and/or fall in troponin, with at least one value above the 99th centile value in the context of a patient with ischaemic symptoms or signs (ECG changes or imaging evidence), would satisfy the diagnosis.16 Based on current consensus guidance for high-sensitivity troponin assays, a rise or fall of 20% (delta) was considered statistically significant and consistent with a diagnosis of AMI.17 Adjudication of the primary endpoint was carried out by two local cardiologists blinded to the nursing m-Goldman score but who had access to the clinical record, ECG and serial hs-cTnT results.
χ2 analyses were used to generate 2×2 tables for the calculation of sensitivity, specificity and positive and negative likelihood ratios. Receiver-operating characteristic curves were generated from sensitivity and specificity to give an overall summary of diagnostic accuracy. Significance was calculated using the Fisher's exact test for contingency tables and Mann–Whitney U test for non-parametric data; all reported p values are two-tailed. Interobserver reliability was assessed using Cohen's kappa. Statistical analysis was carried out using SPSS V.20.
Of 1096 eligible patients, 964 were recruited; 4 patients were lost to follow-up, meaning that 99.6% were successfully monitored for 30 days. In total, 132 patients were eligible but not recruited due to missing the consent process; these were similar in age, gender and risk factors (p>0.05 for all). Of the patients, 124/960 (12.5%) had the outcome event MACE within 30 days. Figure 1 is a STARD diagram depicting a participant recruitment flow chart according to physician and nursing assessments. In total, 912/960 (95.0%) had m-Goldman scores recorded by ED physicians and 745/960 (77.6%) by nursing staff.
There were no significant differences between physician and nursing patient groups in age, gender, risk factors for coronary artery disease, prior cardiovascular history and hospital length of stay (p>0.05 for all) (table 2).
Diagnostic accuracy of nursing staff risk assessment using the m-Goldman score
Contingency tables showing the occurrence of MACE according to index tests are available as online supplementary data. The primary aim of the m-Goldman score is to identify low-risk patients who may be suitable for discharge; therefore, the test metric of interest is sensitivity (rule-out). Table 3 demonstrates a sensitivity for the diagnosis of 30-day MACE of 73.9% (95% CI 65.5% to 81.1%) and 63.0% (95% CI 53.0% to 72.3%) for physicians and nursing staff, respectively, when using the rule-out m-Goldman cut-off of ≤1.
Using the area under the curve (figure 2) as an estimate of the overall diagnostic accuracy of the m-Goldman score in predicting 30-day MACE, there was no significant difference between assessor groups: 0.647 (95% CI 0.594 to 0.700) for physicians and 0.572 (95% CI 0.510 to 0.634) for nursing staff assessments (p=0.09).
Diagnostic accuracy of the TRUST ADP
Table 3 also presents the statistical analysis of the TRUST ADP for predicting MACE at 30 days according to assessor groups. One patient in the physician group (0.3%) and three patients (1.1%) in the nursing group were classified as low risk by the ADP yet had MACE at 30 days. Sensitivity of the ADP for the rule-out of MACE was 99.2% (95% CI 94.8% to 100%) and 96.7% (90.3% to 99.2%) for physician and nursing groups, respectively.
Table 4 summarises interobserver reliability of individual components of the m-Goldman score and those patients identified as low risk (m-Goldman ≤1). The degree of reliability varied with four components showing fair agreement, three showing moderate agreement and only one showing substantial agreement (though the finding of pain within 6 weeks of an AMI or revascularisation was only present in 1.1% of the population). Using the m-Goldman score, there was fair agreement in the identification of low-risk patients between physicians and nursing staff (kappa 0.31, 95% CI 0.24 to 0.38).18
This study demonstrates that the diagnostic accuracy of ED nursing staff risk assessment using an established chest pain risk score is similar to that of ED physicians. When combining nursing risk stratification with presentation high-sensitivity troponin testing, a nurse-led ADP would have a miss rate of 1.1% for MACE at 30 days. This finding, together with fair interobserver reliability of nursing and physician assessments in the identification of low-risk patients, suggests the future role of nursing staff in rapid rule-out pathways holds promise.
Few studies have investigated the role of nursing staff in the assessment of low-risk patients with suspected ACS, and this is the first to compare the assessments of physician and nursing staff using an ADP designed specifically to identify patients suitable for early discharge. Chest pain in the acute setting is traditionally triaged as a high-risk presentation;19 consequently, this cohort of patients are nursed in high-acuity areas, despite the fact that few patients (only 12.5% of our study population) have major adverse outcomes. It has been established that the interventions of nursing staff are important in improving the care of high-risk patients with chest pain.20 However, our results also suggest that risk assessment by nursing staff with a focus upon low-risk patients may be a viable strategy with the potential to improve ED efficiency, through early biomarker testing and the use of low-acuity clinical areas.
This study is also important in highlighting interobserver reliability of chest pain assessment, which remains under-reported in the literature. The m-Goldman risk score uses elements of chest pain history to identify those patients without unstable features; it, therefore, requires some clinical judgement and subjectivity in interpretation. All nursing participants were experienced in the primary assessment of ED patients with chest pain. Therefore, the only fair agreement between assessors may be seen as unexpected. This finding will not be limited to the m-Goldman score, as other commonly used risk scores also incorporate elements that require clinical judgement. Examples include the History, ECG, Age, Risk factors and Troponin (HEART) score,2 Manchester Acute Coronary Syndromes (MACS) rule4 and Vancouver chest pain rule.5
The ED nursing staff who took part in our analysis were all experienced in the primary assessment of ED patients with chest pain; however, none were trained as advanced practitioners and they had no formal training in the use of the m-Goldman score, other than written instructions provided. Therefore, we suggest that with tailored educational interventions diagnostic accuracy may be improved. Studies investigating simple training interventions, such as workshops, in non-specialist ED nursing staff have consistently demonstrated improved correlation between physician and nurse ordering, as well as more accurate test interpretation.21–23 As such, further research is required that incorporates formal training in chest pain assessment for nursing staff and focuses on the identification of low-risk patients who may be suitable for early discharge.
An important limitation to this study is that we included in this analysis only patients with suspected ACS, as decided by the treating physician. Therefore, we can make no conclusions on the ability of nursing staff to identify those patients with suspected ACS from undifferentiated chest pain. Current evidence here is limited, with one small study suggesting that nursing staff have a sensitivity approaching 90% in identifying cardiac chest pain.19 In order to avoid over-selection of patients for rapid rule-out protocols and consequent adverse effects of resource use through unnecessary biomarker testing, this issue requires clarification.
A further limitation to this analysis is the difference in the proportion of patients who underwent assessment by physicians (95.0%) compared with those assessed by nursing staff (77.6%). This finding may be explained by the ethical necessity for nursing staff to provide written informed consent prior to study participation and the transient nature of the staff body during the recruitment period. However, this finding may cause unseen bias in the clinical characteristics of patient groups and may mean that the study had insufficient power to detect a diagnostic difference that did in fact exist (a type II statistical error).
The diagnostic accuracy of ED nursing staff risk assessment using an established chest pain risk score is similar to that of ED physicians, and interobserver reliability between assessor groups is fair. When combining nursing risk stratification with presentation high-sensitivity troponin testing, a nurse-led ADP would have a miss rate of 1.1% for MACE at 30 days.
We thank Dr John Beavis (Bournemouth University) for statistical support. We also thank Georgina Gemmell, Dr Elena Cowan and staff at Poole Hospital emergency department and Biochemistry departments for their assistance and support. Thanks also to Dr Nick Jenkins (Emergency Department, Wexham Park Hospital) for assistance with concept development.
This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.
- Data supplement 1 - Online supplement
Twitter Follow Edward Carlton at @eddcarlton
Contributors EWC: conception and design, analysis and interpretation of the data, drafting manuscript or critical revisions for intellectual content and final approval of the manuscript. AK and KG: conception and design, drafting manuscript or critical revisions for intellectual content and final approval of the manuscript.
Funding The TRUST study was supported by a research grant from the College of Emergency Medicine of the United Kingdom and research fellowship funding from Bournemouth University, UK.
Competing interests EWC has received funding from Abbott in support for related research. KG has received funding from AstraZeneca and Takeda UK for related research.
Patient consent All patient participants and nursing staff provided written informed consent.
Ethics approval UK National Research Ethics Service.
Provenance and peer review Not commissioned; externally peer reviewed.
Data sharing statement All requests for further data from this study should be addressed to the corresponding author.
If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.