You are here

  • Home
  • Archive
  • Volume 33, Issue 2
  • Beyond triage: the diagnostic accuracy of emergency department nursing staff risk assessment in patients with suspected acute coronary syndromes

Article Text

Article menu
Original article
Beyond triage: the diagnostic accuracy of emergency department nursing staff risk assessment in patients with suspected acute coronary syndromes
  1. Edward Watts Carlton1,2,
  2. Ahmed Khattab2,
  3. Kim Greaves3
  1. 1Emergency Department, Southmead Hospital, Bristol, UK
  2. 2School of Health and Social Care, Bournemouth University, Bournemouth, Dorset, UK
  3. 3Department of Cardiology, Sunshine Coast Hospital and Health Services, University of the Sunshine Coast, Sunshine Coast, Queensland, Australia
  1. Correspondence to Dr Edward Watts Carlton, Department of Emergency Medicine, Southmead Hospital, North Bristol NHS Trust, Westbury-on-Trym, Bristol BS10 5NB, UK; eddcarlton{at}gmail.com

Abstract

Objectives To establish the accuracy of emergency department (ED) nursing staff risk assessment using an established chest pain risk score alone and when incorporated with presentation high-sensitivity troponin testing as part of an accelerated diagnostic protocol (ADP).

Design Prospective observational study comparing nursing and physician risk assessment using the modified Goldman (m-Goldman) score and a predefined ADP, incorporating presentation high-sensitivity troponin.

Setting A UK District ED.

Patients Consecutive patients, aged ≥18, with suspected cardiac chest pain and non-ischaemic ECG, for whom the treating physician determined serial troponin testing was required.

Outcome measures 30-day major adverse cardiac events (MACE).

Results 960 participants were recruited. 912/960 (95.0%) had m-Goldman scores recorded by physicians and 745/960 (77.6%) by nursing staff. The area under the curve of the m-Goldman score in predicting 30-day MACE was 0.647 (95% CI 0.594 to 0.700) for physicians and 0.572 (95% CI 0.510 to 0.634) for nursing staff (p=0.09). When incorporated into an ADP, sensitivity for the rule-out of MACE was 99.2% (95% CI 94.8% to 100%) and 96.7% (90.3% to 99.2%) for physicians and nurses, respectively. One patient in the physician group (0.3%) and three patients (1.1%) in the nursing group were classified as low risk yet had MACE. There was fair agreement in the identification of low-risk patients (kappa 0.31, 95% CI 0.24 to 0.38).

Conclusions The diagnostic accuracy of ED nursing staff risk assessment is similar to that of ED physicians and interobserver reliability between assessor groups is fair. When incorporating high-sensitivity troponin testing, a nurse-led ADP has a miss rate of 1.1% for MACE at 30 days.

Trial registration number Controlled Trials Database (ISRCTN no. 21109279).

  • cardiac care, diagnosis
  • cardiac care, acute coronary syndrome
  • nursing, emergency departments

https://doi.org/10.1136/emermed-2015-204780

Statistics from Altmetric.com

    Request Permissions

    If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

    Key messages

    What is already known on this subject?

    • Few studies have investigated the role of nursing staff in the assessment of low-risk patients with chest pain. Advanced nursing interventions during initial patient assessment have been proven to reduce time to treatment and diagnosis, improve patient flow through the emergency department and reduce length of stay across a wide variety of emergency presentations. Nursing staff may therefore be an underused resource in mitigating crowding.

    What might this study add?

    • This prospective, single-centre observational study demonstrates that emergency department nursing staff risk assessment using an established chest pain risk score is similar to that of emergency department physicians. When combining nursing risk stratification with presentation high-sensitivity troponin testing, a nurse-led accelerated discharge protocol would have a miss rate of 1.1% for major adverse cardiac events at 30 days. This finding, together with fair interobserver reliability of nursing and physician assessments in the identification of low-risk patients, suggests the future role of nursing staff in rapid rule-out pathways holds promise.

    Introduction

    Chest pain is one of the most common complaints of patients presenting to the emergency department (ED), with approximately one million visits per year in the UK. The majority of patients require prolonged assessment prior to safe discharge despite the fact that only 15–25% of these patients have a final diagnosis of acute coronary syndrome (ACS).1 Recently, accelerated diagnostic protocols (ADPs) have successfully incorporated chest pain risk scores with early biomarker testing to identify those patients at low risk of major adverse cardiac events (MACE) who may be suitable for early discharge.2–6

    Advanced nursing interventions during initial patient assessment have been proven to reduce time to treatment and diagnosis, improve patient flow through the ED and reduce length of stay across a wide variety of emergency presentations.7 It is also evident that chest pain-specific risk scores, such as a modified Thrombolysis in Myocardial Infarction (TIMI) score,8 can improve the accuracy of nursing assessments.9 Yet, the ability of ED nursing staff to safely risk-stratify low-risk patients with suspected ACS who may be suitable for early rule-out biomarker testing and therefore early discharge has never been investigated. Consequently, ED nursing staff remain a potentially underused resource in the assessment of chest pain.

    This study aimed to establish the diagnostic accuracy of ED nursing staff risk assessment using an established chest pain risk score (modified Goldman (m-Goldman)) alone and when incorporated with presentation high-sensitivity troponin testing as part of a nurse-led ADP. The secondary aim was to evaluate the interobserver reliability of nursing and physician assessments within the m-Goldman risk score.

    Methods

    This prospective observational clinical trial was designed to assess the predefined Triage Rule-out Using high-Sensitivity Troponin (TRUST) ADP. This ADP incorporates a non-ischaemic ECG, an m-Goldman risk score10 and a single-presentation high-sensitivity troponin (hs-cTnT) result (table 1). The study protocol was designed to be truly pragmatic in order to enhance the widespread applicability of the study results:11 treating ED physicians and nursing staff performed m-Goldman risk scores, real-time sample processing and 24/7 recruitment. Results from physician assessment using the TRUST ADP have been published previously.6 The study was designed using the Standards for Reporting Diagnostic Accuracy (STARD).12

    View this table:
    Table 1

    The modified Goldman score and the Triage Rule-out Using high-Sensitivity Troponin accelerated diagnostic protocol (TRUST ADP)

    Study setting, recruitment and data collection

    Poole NHS Foundation Trust is a UK District General; the ED has approximately 62 000 new patient attendances per year. Patients with suspected ACS are managed according to the local hospital protocol, which involves risk assessment by ED physician staff using the m-Goldman risk score and blood drawn for hs-cTnT at 6 h after presentation. As part of the study protocol, blood was also taken at presentation for hs-cTnT analysis. While historical clinical protocols, at the time of this study, did not include troponin measurement at presentation, this had the benefit of ensuring that treating physicians were blinded to the initial hs-cTnT result to avoid selection bias and observation bias.13

    The fifth-generation Roche ELECSYS hs-cTnT assay (Roche, Switzerland), which has a 99th percentile of 14 ng/L and 10% coefficient of variation of <10% at 9 ng/L, was used for both presentation and reference (6 h) samples. During initial assessment, clinical staff drew blood for routine admission samples and an additional 3.5 mL of whole blood in a pre-labelled study-specific serum settling tube for hs-cTnT analysis. All serum samples were tested in real time.

    Consecutive patients attending the ED with suspected ACS were prospectively screened from July 2012 to August 2013. Patients were included if they were ≥18 years of age and had at least 5 min of chest pain suggestive of ACS, and for whom the treating physician determined inpatient evaluation was required. No patient was observed within the ED to await 6 h blood draws due to the UK national healthcare target that patients must be seen, treated, admitted or discharged within 4 h. Possible cardiac symptoms included acute chest, epigastric, neck, jaw or arm pain, or discomfort or pressure without an apparent non-cardiac source, in accordance with the American Heart Association case definitions.14 Patients were excluded if any of the following were present: ST segment elevation myocardial infarction or left bundle branch block not known to be old; ECG changes diagnostic of ischaemia (ST segment depression ≥1 mm or T-wave inversion consistent with the presence of ischaemia); arrhythmias (new-onset atrial fibrillation, atrial flutter, sustained supraventricular tachycardia, second-degree or complete heart block or sustained or recurrent ventricular arrhythmias); hs-cTnT not suitable for analysis (eg, haemolysis); age ≥80 years; atypical symptoms in the absence of chest discomfort; a clear non-ACS cause for chest pain was found at presentation (eg, pulmonary embolism, pneumonia, aortic dissection); another medical condition requiring hospital admission, refusal or inability to give informed consent; non-English speaking; pregnancy; and renal failure requiring dialysis or inability to be contacted after discharge.

    ED physician staff undertook initial ECG evaluation as part of clinical care, later confirmed by two local cardiologists (nursing staff were not expected to undertake ECG evaluation). Patients with ECG evidence of acute myocardial infarction (AMI) or acute ischaemia were immediately defined as high risk in accordance with Goldman's original rule and therefore not recruited.

    Data were collected prospectively using a published data dictionary.15 ED nursing staff undertaking initial assessment were asked to record the m-Goldman risk score on a case report form, at the time of patient presentation to the ED. Nursing staff were experienced in the primary assessment and triage of ED patients with chest pain but had no formal training in the use of the m-Goldman score. Consequently, they were provided with written explanatory notes on how to complete the risk score. The nursing risk score was kept separate from the clinical notes in a coloured envelope and removed by a member of the research team at the earliest opportunity. Attending ED physicians completed an identical m-Goldman risk score as part of routine clinical assessment on a separate clinical assessment form.

    Follow-up was undertaken by independent review of hospital electronic patient records, summary of health records from the patient's general practitioner (GP) obtained at least 6 months after attendance and a national clinical records search (which identifies death). Where a participant had not attended hospital follow-up and/or a GP had failed to provide a health record/not GP-registered, the patient was regarded as lost to follow-up.

    Index tests

    The index test was the m-Goldman score evaluated by both physicians and ED nursing staff. In order to establish the potential diagnostic accuracy of the m-Goldman score within a nurse-led ADP, the secondary index test of the TRUST ADP was used. This defined a patient as ‘low risk’ if all of the following conditions were satisfied at presentation: an m-Goldman score of ≤1 (table 1), a non-ischaemic ECG and a single central laboratory hs-cTnT of <14 ng/L.

    Outcome measures

    The endpoint was the presence of MACE occurring within 30 days of hospital attendance (including the index visit). MACE included death due to ischaemic heart disease, cardiac arrest, symptom-induced revascularisation, cardiogenic shock, ventricular arrhythmia, high-degree atrioventricular block needing intervention and AMI. This definition is consistent with previous large-scale research analysing the diagnostic accuracy of ADPs.3

    The presence of AMI was defined according to the Third Universal Definition of MI, which states that a rise and/or fall in troponin, with at least one value above the 99th centile value in the context of a patient with ischaemic symptoms or signs (ECG changes or imaging evidence), would satisfy the diagnosis.16 Based on current consensus guidance for high-sensitivity troponin assays, a rise or fall of 20% (delta) was considered statistically significant and consistent with a diagnosis of AMI.17 Adjudication of the primary endpoint was carried out by two local cardiologists blinded to the nursing m-Goldman score but who had access to the clinical record, ECG and serial hs-cTnT results.

    Statistical analysis

    χ2 analyses were used to generate 2×2 tables for the calculation of sensitivity, specificity and positive and negative likelihood ratios. Receiver-operating characteristic curves were generated from sensitivity and specificity to give an overall summary of diagnostic accuracy. Significance was calculated using the Fisher's exact test for contingency tables and Mann–Whitney U test for non-parametric data; all reported p values are two-tailed. Interobserver reliability was assessed using Cohen's kappa. Statistical analysis was carried out using SPSS V.20.

    Results

    Of 1096 eligible patients, 964 were recruited; 4 patients were lost to follow-up, meaning that 99.6% were successfully monitored for 30 days. In total, 132 patients were eligible but not recruited due to missing the consent process; these were similar in age, gender and risk factors (p>0.05 for all). Of the patients, 124/960 (12.5%) had the outcome event MACE within 30 days. Figure 1 is a STARD diagram depicting a participant recruitment flow chart according to physician and nursing assessments. In total, 912/960 (95.0%) had m-Goldman scores recorded by ED physicians and 745/960 (77.6%) by nursing staff.

    Figure 1
    Figure 1

    Participant recruitment flow chart. MACE, major adverse cardiac event; TRUST ADP, Triage Rule-out Using high-Sensitivity Troponin accelerated diagnostic protocol.

    There were no significant differences between physician and nursing patient groups in age, gender, risk factors for coronary artery disease, prior cardiovascular history and hospital length of stay (p>0.05 for all) (table 2).

    View this table:
    Table 2

    Patient demographics

    Diagnostic accuracy of nursing staff risk assessment using the m-Goldman score

    Contingency tables showing the occurrence of MACE according to index tests are available as online supplementary data. The primary aim of the m-Goldman score is to identify low-risk patients who may be suitable for discharge; therefore, the test metric of interest is sensitivity (rule-out). Table 3 demonstrates a sensitivity for the diagnosis of 30-day MACE of 73.9% (95% CI 65.5% to 81.1%) and 63.0% (95% CI 53.0% to 72.3%) for physicians and nursing staff, respectively, when using the rule-out m-Goldman cut-off of ≤1.

    View this table:
    Table 3

    Diagnostic accuracy of the modified-Goldman (m-Goldman) score and the Triage Rule-out Using high-Sensitivity Troponin accelerated diagnostic protocol (TRUST ADP) for predicting major adverse cardiac events (MACE) at 30 days according to assessor groups

    Using the area under the curve (figure 2) as an estimate of the overall diagnostic accuracy of the m-Goldman score in predicting 30-day MACE, there was no significant difference between assessor groups: 0.647 (95% CI 0.594 to 0.700) for physicians and 0.572 (95% CI 0.510 to 0.634) for nursing staff assessments (p=0.09).

    Figure 2
    Figure 2

    Receiver-operating characteristic curves of the modified-Goldman (m-Goldman) score according to assessor groups. AUC, area under the curve.

    Diagnostic accuracy of the TRUST ADP

    Table 3 also presents the statistical analysis of the TRUST ADP for predicting MACE at 30 days according to assessor groups. One patient in the physician group (0.3%) and three patients (1.1%) in the nursing group were classified as low risk by the ADP yet had MACE at 30 days. Sensitivity of the ADP for the rule-out of MACE was 99.2% (95% CI 94.8% to 100%) and 96.7% (90.3% to 99.2%) for physician and nursing groups, respectively.

    Interobserver reliability

    Table 4 summarises interobserver reliability of individual components of the m-Goldman score and those patients identified as low risk (m-Goldman ≤1). The degree of reliability varied with four components showing fair agreement, three showing moderate agreement and only one showing substantial agreement (though the finding of pain within 6 weeks of an AMI or revascularisation was only present in 1.1% of the population). Using the m-Goldman score, there was fair agreement in the identification of low-risk patients between physicians and nursing staff (kappa 0.31, 95% CI 0.24 to 0.38).18

    View this table:
    Table 4

    Interobserver reliability of the modified-Goldman (m-Goldman) score

    Discussion

    This study demonstrates that the diagnostic accuracy of ED nursing staff risk assessment using an established chest pain risk score is similar to that of ED physicians. When combining nursing risk stratification with presentation high-sensitivity troponin testing, a nurse-led ADP would have a miss rate of 1.1% for MACE at 30 days. This finding, together with fair interobserver reliability of nursing and physician assessments in the identification of low-risk patients, suggests the future role of nursing staff in rapid rule-out pathways holds promise.

    Few studies have investigated the role of nursing staff in the assessment of low-risk patients with suspected ACS, and this is the first to compare the assessments of physician and nursing staff using an ADP designed specifically to identify patients suitable for early discharge. Chest pain in the acute setting is traditionally triaged as a high-risk presentation;19 consequently, this cohort of patients are nursed in high-acuity areas, despite the fact that few patients (only 12.5% of our study population) have major adverse outcomes. It has been established that the interventions of nursing staff are important in improving the care of high-risk patients with chest pain.20 However, our results also suggest that risk assessment by nursing staff with a focus upon low-risk patients may be a viable strategy with the potential to improve ED efficiency, through early biomarker testing and the use of low-acuity clinical areas.

    This study is also important in highlighting interobserver reliability of chest pain assessment, which remains under-reported in the literature. The m-Goldman risk score uses elements of chest pain history to identify those patients without unstable features; it, therefore, requires some clinical judgement and subjectivity in interpretation. All nursing participants were experienced in the primary assessment of ED patients with chest pain. Therefore, the only fair agreement between assessors may be seen as unexpected. This finding will not be limited to the m-Goldman score, as other commonly used risk scores also incorporate elements that require clinical judgement. Examples include the History, ECG, Age, Risk factors and Troponin (HEART) score,2 Manchester Acute Coronary Syndromes (MACS) rule4 and Vancouver chest pain rule.5

    The ED nursing staff who took part in our analysis were all experienced in the primary assessment of ED patients with chest pain; however, none were trained as advanced practitioners and they had no formal training in the use of the m-Goldman score, other than written instructions provided. Therefore, we suggest that with tailored educational interventions diagnostic accuracy may be improved. Studies investigating simple training interventions, such as workshops, in non-specialist ED nursing staff have consistently demonstrated improved correlation between physician and nurse ordering, as well as more accurate test interpretation.21–23 As such, further research is required that incorporates formal training in chest pain assessment for nursing staff and focuses on the identification of low-risk patients who may be suitable for early discharge.

    An important limitation to this study is that we included in this analysis only patients with suspected ACS, as decided by the treating physician. Therefore, we can make no conclusions on the ability of nursing staff to identify those patients with suspected ACS from undifferentiated chest pain. Current evidence here is limited, with one small study suggesting that nursing staff have a sensitivity approaching 90% in identifying cardiac chest pain.19 In order to avoid over-selection of patients for rapid rule-out protocols and consequent adverse effects of resource use through unnecessary biomarker testing, this issue requires clarification.

    A further limitation to this analysis is the difference in the proportion of patients who underwent assessment by physicians (95.0%) compared with those assessed by nursing staff (77.6%). This finding may be explained by the ethical necessity for nursing staff to provide written informed consent prior to study participation and the transient nature of the staff body during the recruitment period. However, this finding may cause unseen bias in the clinical characteristics of patient groups and may mean that the study had insufficient power to detect a diagnostic difference that did in fact exist (a type II statistical error).

    Conclusion

    The diagnostic accuracy of ED nursing staff risk assessment using an established chest pain risk score is similar to that of ED physicians, and interobserver reliability between assessor groups is fair. When combining nursing risk stratification with presentation high-sensitivity troponin testing, a nurse-led ADP would have a miss rate of 1.1% for MACE at 30 days.

    Acknowledgments

    We thank Dr John Beavis (Bournemouth University) for statistical support. We also thank Georgina Gemmell, Dr Elena Cowan and staff at Poole Hospital emergency department and Biochemistry departments for their assistance and support. Thanks also to Dr Nick Jenkins (Emergency Department, Wexham Park Hospital) for assistance with concept development.

    References

    1. ↵
      1. Goodacre S,
      2. Cross E,
      3. Arnold J, et al
      . The health care burden of acute chest pain. Heart 2005;91:229–30. doi:10.1136/hrt.2003.027599
      OpenUrlFREE Full Text
    2. ↵
      1. Six A,
      2. Cullen L,
      3. Backus B, et al
      . The HEART score for the assessment of patients with chest pain in the emergency department: a multinational validation study. Crit Pathw Cardiol 2013;12:121–6. doi:10.1097/HPC.0b013e31828b327e
      OpenUrlCrossRefPubMed
    3. ↵
      1. Cullen L,
      2. Mueller C,
      3. Parsonage W, et al
      . Validation of high-sensitivity troponin I in a 2-hour diagnostic strategy to assess 30-day outcomes in emergency department patients with possible acute coronary syndrome. J Am Coll Cardiol 2013;62:1242–9. doi:10.1016/j.jacc.2013.02.078
      OpenUrlCrossRefPubMedWeb of Science
    4. ↵
      1. Body R,
      2. Carley S,
      3. Mcdowell G, et al
      . The Manchester Acute Coronary Syndromes (MACS) decision rule for suspected cardiac chest pain: derivation and external validation. Heart 2014;100:1462–8. doi:10.1136/heartjnl-2014-305564
      OpenUrlAbstract/FREE Full Text
    5. ↵
      1. Cullen L,
      2. Greenslade J,
      3. Than M, et al
      . The new Vancouver Chest Pain Rule using troponin as the only biomarker: an external validation study. Am J Emerg Med 2014;32:129–34. doi:10.1016/j.ajem.2013.10.021
      OpenUrlCrossRefPubMed
    6. ↵
      1. Carlton E,
      2. Cullen L,
      3. Than M, et al
      . A novel diagnostic protocol to identify patients suitable for discharge after a single high-sensitivity troponin. Heart 2015;101:1041–6. doi:10.1136/heartjnl-2014-307288
      OpenUrlAbstract/FREE Full Text
    7. ↵
      1. Rowe B,
      2. Villa-Roel C,
      3. Guo X, et al
      . The role of triage nurse ordering on mitigating overcrowding in emergency departments: a systematic review. Acad Emerg Med 2011;18:1349–57. doi:10.1111/j.1553-2712.2011.01081.x
      OpenUrlCrossRefPubMedWeb of Science
    8. ↵
      1. Antman E,
      2. Cohen M,
      3. Bernink P, et al
      . The TIMI risk score for unstable angina/non-st elevation MI; A method for prognostication and therapeutic decision making. JAMA 2000;284:835–42. doi:10.1001/jama.284.7.835
      OpenUrlCrossRefPubMedWeb of Science
    9. ↵
      1. Ho J,
      2. Suen L
      . Effectiveness of using the front door score to enhance the chest pain triage accuracy of emergency nurse triage decisions. J Cardiovasc Nurs 2013;28:E55–64. doi:10.1097/JCN.0b013e318277c5ed
      OpenUrlCrossRefPubMed
    10. ↵
      1. Goldman L,
      2. Cook E,
      3. Johnson P, et al
      . Prediction of the need for intensive care in patients who come to the emergency departments with acute chest pain. N Eng J Med 1996;334:1498–504. doi:10.1056/NEJM199606063342303
      OpenUrlCrossRefPubMedWeb of Science
    11. ↵
      1. Tunis SR,
      2. Stryer DB,
      3. Clancy CM
      . Practical clinical trials: increasing the value of clinical research for decision making in clinical and health policy. JAMA 2003;290:1624–32. doi:10.1001/jama.290.12.1624
      OpenUrlCrossRefPubMedWeb of Science
    12. ↵
      1. Bossuyt PM,
      2. Reitsma JB,
      3. Bruns DE, et al
      ., Standards for Reporting of Diagnostic Accuracy. The STARD statement for reporting studies of diagnostic accuracy: explanation and elaboration. Ann Intern Med 2003;138:W1–12. doi:10.7326/0003-4819-138-1-200301070-00012-w1
      OpenUrlCrossRefPubMed
    13. ↵
      1. Stiell I,
      2. Wells G
      . Methodologic standards for the development of clinical decision rules in emergency medicine. Ann Emerg Med 1999;33:437–47. doi:10.1016/S0196-0644(99)70309-4
      OpenUrlCrossRefPubMedWeb of Science
    14. ↵
      1. Luepker RV,
      2. Apple FS,
      3. Christenson RH, et al
      . Case definitions for acute coronary heart disease in epidemiology and clinical research studies: a statement from the AHA Council on Epidemiology and Prevention; AHA Statistics Committee; World Heart Federation Council on Epidemiology and Prevention; Centers for Disease Control and Prevention; and the National Heart, Lung, and Blood Institute. Circulation 2003;108:2543–9. doi:10.1161/01.CIR.0000100560.46946.EA
      OpenUrlFREE Full Text
    15. ↵
      1. Cullen L,
      2. Than M,
      3. Brown A, et al
      . Comprehensive standardized data definitions for acute coronary syndrome research in emergency departments in Australasia. Emerg Med Australas 2010;22:35–55. doi:10.1111/j.1742-6723.2010.01256.x
      OpenUrlCrossRefPubMed
    16. ↵
      1. Thygesen K,
      2. Alpert J,
      3. Jaffe A, et al
      . Third universal definition of myocardial infarction. Circulation 2012;126:2020–35. doi:10.1161/CIR.0b013e31826e1058
      OpenUrlFREE Full Text
    17. ↵
      1. Thygesen K,
      2. Mair J,
      3. Giannitsis E, et al
      ., Study Group on Biomarkers in Cardiology of ESC Working Group on Acute Cardiac Care. How to use high-sensitivity cardiac troponins in acute cardiac care. Eur Heart J 2012;33:2252–7. doi:10.1093/eurheartj/ehs154
      OpenUrlFREE Full Text
    18. ↵
      1. Landis J,
      2. Koch G
      . The measurement of observer agreement for categorical data. Biometrics 1977;33:159–74. doi:10.2307/2529310
      OpenUrlCrossRefPubMedWeb of Science
    19. ↵
      1. Speake D,
      2. Teece S,
      3. Mackway-Jones K
      . Detecting high-risk patients with chest pain. Emerg Nurse 2003;11:19–21. doi:10.7748/en2003.09.11.5.19.c1131
      OpenUrlPubMed
    20. ↵
      1. Tierney S,
      2. Cook G,
      3. Mamas M, et al
      . Nurses’ role in the acute management of patients with non-ST-segment elevation acute coronary syndromes: an integrative review. Eur J Cardiovasc Nurs 2014;12:293–301. doi:10.1177/1474515112451555
      OpenUrl
    21. ↵
      1. Seaberg D,
      2. MacLeod B
      . Correlation between triage nurse and physician ordering of ED tests. Am J Emerg Med 1998;16:8–11. doi:10.1016/S0735-6757(98)90057-2
      OpenUrlCrossRefPubMedWeb of Science
    22. ↵
      1. Zhang H,
      2. Hsu L
      . The effectiveness of an education program on nurses’ knowledge of electrocardiogram interpretation. Int Emerg Nurs 2013;21: 247–51. doi:10.1016/j.ienj.2012.11.001
      OpenUrlCrossRefPubMed
    23. ↵
      1. Varvaroussis D,
      2. Kalafati M,
      3. Pliatsika P, et al
      . Comparison of two teaching methods for cardiac arrhythmia interpretation among nursing students. Resuscitation 2014;85:163–4. doi:10.1016/j.resuscitation.2013.09.023
      OpenUrlCrossRefPubMed

    Supplementary materials

    • Supplementary Data

      This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.

    Footnotes

    • Twitter Follow Edward Carlton at @eddcarlton

    • Contributors EWC: conception and design, analysis and interpretation of the data, drafting manuscript or critical revisions for intellectual content and final approval of the manuscript. AK and KG: conception and design, drafting manuscript or critical revisions for intellectual content and final approval of the manuscript.

    • Funding The TRUST study was supported by a research grant from the College of Emergency Medicine of the United Kingdom and research fellowship funding from Bournemouth University, UK.

    • Competing interests EWC has received funding from Abbott in support for related research. KG has received funding from AstraZeneca and Takeda UK for related research.

    • Patient consent All patient participants and nursing staff provided written informed consent.

    • Ethics approval UK National Research Ethics Service.

    • Provenance and peer review Not commissioned; externally peer reviewed.

    • Data sharing statement All requests for further data from this study should be addressed to the corresponding author.