Article Text
PDF
Abstract
Objective To determine the diagnostic accuracy of emergency physician gestalt in emergency department (ED) patients with suspected cardiac chest pain, both alone and in combination with initial troponin level and ECG findings.
Methods We prospectively included patients presenting to the ED with suspected cardiac chest pain. Clinicians recorded their ‘gestalt’ at the time of presentation using a five-point Likert scale, blinded to outcome. Troponin T and high-sensitivity troponin T (hs-cTnT; both Roche Diagnostics Elecsys) levels were measured in admission blood samples. All patients underwent troponin testing at least 12 h after peak symptoms. The primary outcome was acute myocardial infarction (AMI).
Results 458 patients were included in this study, 81 (17.7%) of whom had AMI. Clinician gestalt alone had an area under the receiver operating characteristic curve of 0.76 (95% CI 0.70 to 0.82) for AMI. Immediately discharging patients with normal initial troponin and ECG in whom the clinician felt the diagnosis was ‘probably not’ or ‘definitely not’ acute coronary syndrome (ACS) would have avoided admission for 23.1% (95% CI 19% to 28%) patients with 100% sensitivity (95% CI 95.6% to 100%). With hs-cTnT, 100% sensitivity could have been achieved even if only patients with ‘probable’ or ‘definite’ ACS were investigated further, which would have allowed 41.7% patients to be discharged immediately.
Conclusions Gestalt alone cannot be used to ‘rule in’ or ‘rule out’ ACS. By combining clinician gestalt with the admission ECG and troponin level, we found 100% sensitivity without the need for serial troponin testing. These findings have the potential to reduce unnecessary hospital admissions for suspected ACS but must be prospectively validated before considering clinical implementation.
- acute coronary syndrome
- diagnosis
Statistics from Altmetric.com
Key messages
-
What is already known about this subject?
-
Previous work has robustly shown that, in patients presenting to the Emergency Department (ED) with chest pain, the typicality of symptoms is not useful for differentiating patients with acute coronary syndromes from those with non-cardiac diagnoses. The role of overall clinical judgement (gestalt) has not been extensively studied, however.
-
What does this study add?
-
This study demonstrates that the unstructured clinical judgement of emergency physicians alone has moderate overall diagnostic accuracy but is insufficient to enable the confirmation (‘rule in’) or exclusion (‘rule out’) of acute coronary syndromes in the ED. However, in patients with a normal ECG and initial troponin level our findings suggest that 1 in 4 patients could avoid unnecessary hospital admission using a standard troponin assay, or 2 in 5 patients if a high sensitivity troponin assay is used.
-
How might this impact on clinical practice?
-
If these findings can be confirmed by future work, this work may provide a simple solution to substantially reduce unnecessary hospital admissions for chest pain, which is currently the most common reason for emergency hospital admission. As the strategy requires only an ECG, a single troponin test on arrival and the clinical judgement of the treating physician it could be readily translated into clinical practice.
Background
For many years, there has been great interest in developing strategies that would permit early exclusion of acute coronary syndromes (ACSs) in the emergency department (ED), thus avoiding unnecessary hospital admissions. Recent research has focused on strategies such as serial biomarker testing with and without risk stratification,1 ,2 and on the development of clinical decision rules that combine clinical and biochemical variables to facilitate more accurate early decision making.3 ,4
The high prevalence of atypical symptoms in patients with ACSs,5 ,6 coupled with the fact that as many as 6% of patients with chest pain who are discharged from the ED have unrecognised acute myocardial infarction (AMI),7 leads many to believe that unstructured clinical judgement or ‘gestalt’ is likely to have limited diagnostic value in this context. Indeed, previous research has shown that general practitioners have only moderate diagnostic accuracy when estimating the likelihood of coronary artery disease in their patients8 and the gestalt of cardiologists is unreliable for predicting findings at coronary angiography.9 However, the judgement of the treating physician has been shown to have independent diagnostic value in patients with suspected deep vein thrombosis and pulmonary embolism and is an important component of widely used clinical decision rules for those diagnoses.10 ,11 Further, recent research has demonstrated that if emergency physicians estimate that the probability of an ACS is less than 2% they have a sensitivity of 96.1% for that diagnosis.12
We aimed to determine the diagnostic value of the clinical judgement of emergency physicians, both alone and in combination with ECG findings and troponin levels at the time of initial presentation to the ED.
Methods
We undertook a prospective diagnostic cohort study at Stockport NHS Foundation Trust, a District General Hospital in Greater Manchester, UK. Ethical approval was obtained from the Research Ethics Committee (reference 09/H1014/74), and all participants provided written informed consent. From this cohort, we have recently reported external validation of the Manchester Acute Coronary Syndromes (MACS) decision rule.4 We included consecutive adult patients presenting to the ED with suspected cardiac chest pain occurring in the previous 24 h, excluding those with another medical condition necessitating hospital admission; those with renal failure requiring dialysis; patients with chest trauma and suspected myocardial contusion; those who did not speak English; prisoners and patients for whom all means of follow-up would be impossible.
Clinical features were recorded by the treating physician using a custom-designed case report form. The treating physician was asked to record their unstructured ‘gestalt’ or clinical judgement about the probability of an ACS using a five-point Likert scale as follows: ‘definitely not’, ‘probably not’, ‘not sure’, ‘probably’ and ‘definitely’. As these data were recorded at the time of initial presentation and before the results of investigations were available, the treating clinicians were blinded to the final outcome of the patient. As we recorded clinician gestalt following a genuine consultation with the patient, clinicians were not blinded to the ECG or other clinical information. However, we took a number of measures to ensure that treating clinicians were blinded to the initial troponin level. First, clinical protocols in place at the time of this study did not include troponin testing at the time of presentation unless >12 h had already elapsed since peak symptoms. Second, when troponin levels had been evaluated, research staff would verify that clinicians remained unaware of the initial troponin results before collecting case report forms. All patients were followed up by telephone, home visit or in clinic after 30 days.
Laboratory analyses
Patients underwent venepuncture at the time of presentation in the ED and at least 12 h after symptom onset. All 12 h samples were immediately tested for cardiac troponin T (Roche Diagnostics Elecsys, 4th generation, 99th centile 10 ng/L, coefficient of variation <10% at 35 ng/L). At the time of this study, samples drawn at the time of arrival were not routinely tested for cardiac troponin T except in the case of patients who presented >12 h after their peak symptoms. Serum samples drawn at the time of presentation and 12 h after symptom onset were therefore stored at −70°C and later tested for cardiac troponin T (if not already) and also for high-sensitivity troponin T (hs-cTnT, Roche Diagnostics 5th generation Elecsys, 99th centile 14 ng/L, coefficient of variation <10% at 13 ng/L). After our initial analyses, we became aware that the batch of reagent supplied by the manufacturer for hs-cTnT testing had been affected by a calibration shift. Aliquots of all samples were therefore retested using an unaffected batch of reagent and statistical analyses were repeated.
Outcomes
The primary outcome was a diagnosis of AMI. This was defined in accordance with the Third Universal Definition of Myocardial Infarction as a rise and/or fall of cardiac troponin with at least one level above the 99th centile for a healthy reference population.13 Based on the analytical characteristics of the troponin assay, a change of 20 ng/L on serial sampling was considered to represent a rise and/or fall. The diagnosis of AMI was adjudicated by two independent investigators blinded to clinician gestalt.
We also analysed data for the secondary outcome of major adverse cardiac events (MACE) within 30 days. This was defined as death (all cause), prevalent or incident AMI or the need for coronary revascularisation within 30 days. New angiographic stenoses >50% the diameter of a major epicardial vessel (as reported by the responsible cardiologist) were also prospectively considered to constitute MACE.
Statistical analyses
We calculated sensitivity, specificity, positive and negative predictive values using MedCalc V.12.4.0.0 (Mariakerke, Belgium). The area under the receiver operating characteristic (ROC) curve was calculated in IBM SPSS Statistics V.20.0.0. We also evaluated whether gestalt gave independent predictive value once initial troponin level and the presence or absence of ECG ischaemia had been taken into account by entering these data into a logistic regression analysis using IBM SPSS Statistics V.20.0.0
Sample size
Sample size for this study was determined by calculating the required number of patients to ensure that, for a diagnostic strategy with 100% sensitivity, the lower bound of the 95% CI was no lower than 95%. Assuming a 20% incidence of the primary outcome, this would require a minimum sample size of 450 patients.
Results
In total, 458 patients who presented to the ED between April and July 2010 were included in the final analysis (figure 1). Of these patients, 81 (17.7%) were given an adjudicated diagnosis of AMI. After 30 days, an additional 19 patients had developed MACE. These included three deaths (all cardiac or presumed cardiac, including one death due to aortic dissection); a total of seven AMIs (excluding those prevalent at the time of inclusion); and 14 patients who had a new coronary stenosis identified (9 of whom underwent urgent revascularisation within the follow-up period). Thus, in total, 100 (21.8%) patients had developed one or more MACE. Baseline characteristics are shown in table 1.
Baseline characteristics of included patients
Patient flow diagram.
Clinician gestalt had an area under the ROC curve of 0.76 (95% CI 0.70 to 0.82) for AMI. The overall prevalence of AMI and incidence of MACE stratified by clinician gestalt are shown in table 2. Interestingly, in 11 (2.4%) cases, clinicians recorded that the diagnosis was ‘definitely not ACS’ despite the fact that they were investigating the patients for that diagnosis. Of those 11 patients, 10 were given final diagnoses of non-cardiac chest pain and 1 was diagnosed with heart failure and an exacerbation of chronic obstructive pulmonary disease. Two of the eleven patients underwent further investigation for stress-inducible myocardial ischaemia (one exercise tolerance test; one myocardial perfusion scan), both of which demonstrated no inducible ischaemia. None of these 11 patients had AMI on the initial attendance, but one (9.1%) developed MACE within 30 days. In that patient, AMI was initially excluded by serial troponins but the patient developed AMI within 30 days.
Prevalence of AMI and incidence of MACE within 30 days stratified by physician’s gestalt
Conversely, in 53 (11.6%) cases, clinicians recorded that the diagnosis was ‘definitely ACS’ yet only around half of these patients had AMI (n=27, 50.9%) or MACE within 30 days (n=28, 52.8%). The relationship of clinician gestalt to the prevalence of AMI stratified by seniority of treating clinician is shown in table 3. With increasing seniority, doctors were marginally more likely to be correct when they believed that the diagnosis was ‘definitely ACS’. Interestingly, the prevalence of AMI was actually higher when more senior doctors recorded that the diagnosis was ‘probably not ACS’.
Number (%) with AMI stratified by physician’s gestalt and grade of doctor
The overall diagnostic accuracy of physician gestalt as a dichotomous tool that could potentially be used to guide decisions to admit or discharge patients is shown in table 4. We report the diagnostic accuracy of gestalt alone and in combination with the initial troponin level and ECG findings. High-sensitivity troponin assays have been implemented since the time of this study. We have therefore presented the diagnostic accuracy of gestalt in combination with both standard (non-high sensitivity) and high-sensitivity troponin assays. The use of a high-sensitivity troponin assay would have achieved a sensitivity of 100% (95% CI 95.4% to 100%) when combined with gestalt and the ECG, even if only those with ‘probable’ or ‘definite’ ACS were admitted to hospital for further investigation. In this cohort, that strategy would have enabled 191 (41.7%) patients to be discharged immediately from the ED, with no missed AMIs. Three (1.6%) of these patients developed MACE within 30 days.
Diagnostic accuracy of strategies incorporating physician gestalt
To achieve a sensitivity of 100% with a standard troponin assay in combination with the ECG and clinician gestalt, only those in whom the clinician felt the diagnosis was either ‘definitely not’ or ‘probably not’ ACS could be discharged. This strategy would have enabled 106 (23.1%) patients to be immediately discharged with no missed AMIs. One (0.9%) of these patients developed MACE within 30 days.
On multivariate analysis adjusting for initial ECG ischaemia and the troponin level on arrival, gestalt remained independently predictive of both AMI (OR 2.37, 95% CI 1.55 to 3.62, p<0.0001) and MACE (OR 1.96, 95% CI 1.40 to 2.74, p<0.0001).
Discussion
These findings demonstrate an important, simple and yet novel concept. They confirm that it would be unsafe for emergency physicians to use their clinical judgement alone to discharge patients with suspected cardiac chest pain from the ED without further investigation. However, if the treating emergency physician deems that the diagnosis is ‘probably not’ or ‘definitely not’ ACS, then a normal initial troponin level and ECG would potentially enable almost one quarter of patients to be discharged, even using a non-high-sensitivity troponin assay. In this cohort, that strategy missed no AMIs and the incidence of MACE at follow-up was low (0.9%).
Our findings go further by demonstrating that the use of a high-sensitivity troponin assay would enable clinicians to also consider immediate discharge for patients with normal initial ECG and hs-cTnT unless they deemed that the diagnosis was ‘probably’ or ‘definitely’ ACS. In this cohort, unnecessary hospital admission could have been avoided for over 40% of patients through use of this strategy with no missed AMIs and a low incidence of MACE (1.7%).
It is important to recognise that we included patients with suspected cardiac chest pain, that is, the treating clinician used their clinical judgement to identify patients in whom the diagnosis of an ACS was suspected. Patients in whom there was no suspicion of an ACS based on the clinician's clinical judgement are not investigated for an ACS in clinical practice and were not included in this study. Therefore, it is important to point out that this study evaluates the clinical judgement of emergency physicians once they have considered that a patient's symptoms may be cardiac in nature and therefore warrant investigation.
There has been surprisingly little previous work in this area. Prior research has demonstrated that atypical symptoms5 ,6 and a lack of risk factors for coronary artery disease14 do not preclude a diagnosis of ACS. There is also evidence to suggest that approximately 2% of AMIs are missed in the ED,15 which has prognostic implications for patients and medicolegal implications for doctors. Such findings have undoubtedly helped to encourage a culture in which reliance on clinical judgement is discouraged.
Kline et al evaluated the diagnostic accuracy of clinician gestalt in a cohort of 840 patients who presented to the ED with undifferentiated chest pain or shortness of breath, 23 of whom had ACS.16 They found that unstructured clinical judgement, which was defined as the clinician's estimate for the pre-test probability of ACS using a visual analogue scale, had an area under the ROC curve of 0.64. Using gestalt alone at a threshold of 2.5% estimated probability of ACS, clinicians had a sensitivity of 91% for the diagnosis, which would be insufficient to enable safe immediate discharge from the ED.
Such research makes it clear that the clinical judgement of a clinician is an imperfect diagnostic tool and should not be relied upon alone to make decisions to admit or discharge patients with suspected ACS. Combination with other factors is necessary if it is to be useful in this regard. The Wells score for pulmonary embolism is an example of a similar approach. This score includes clinician gestalt as an important factor. In combination with D-dimer levels, the Wells score facilitates safe, immediate exclusion of pulmonary embolism in the ED11 and is widely used in practice. We have demonstrated that the simple combination of gestalt, ECG and troponin could potentially be used to achieve the same thing for patients with suspected cardiac chest pain, although it is essential that our findings are prospectively validated before considering clinical implementation.
Strengths and limitations
Our findings do break new ground by identifying a strategy that could potentially enable over 40% of patients to be immediately discharged from the ED, avoiding unnecessary hospital admission. This strategy only uses resources that are already routinely used in current practice meaning that, if successfully implemented, it has a high probability of being cost-effective. However, it is important to recognise that there are limitations to this work. The first reports of novel diagnostic technology tend to overestimate diagnostic performance compared with subsequent (validation) studies. Given the potential prognostic implications for patients and medicolegal implications for doctors, it is therefore imperative to recognise that although we report a 100% sensitivity for AMI, the 95% CIs extend down to 95.4%. These findings must therefore be prospectively validated in independent populations prior to clinical implementation. It is also important to consider that the incidence of MACE was up to 1.7% in patients who could be considered for early discharge using this strategy. This may be unacceptable to doctors.17 Further, it will be important to consider the indications for provocative testing even in low-risk patients. In particular, we must evaluate how clinicians should safely and judiciously identify patients for further investigation such as CT coronary angiography, functional imaging or invasive coronary angiography.
Lastly, it will be important to evaluate the performance of the strategy when used in practice as the behaviour of clinicians may change when data collection is not undertaken for research that is entirely observational in nature. For example, clinicians may be more risk-averse if asked to use the strategy in practice and, as such, the overall impact of the diagnostic strategy may not be as great as anticipated. Despite these limitations, this work has identified an important area for further research. While it is important for further work to prospectively validate the diagnostic strategies defined here, our work also suggests that there may be potential to refine and improve other emerging diagnostic algorithms by taking account of clinician gestalt.1–3
Conclusions
Taken in isolation, the clinical judgement or gestalt of emergency physicians has only moderate diagnostic accuracy for AMI in patients with suspected cardiac chest pain. However, by combining this gestalt with initial troponin level and ECG findings, hospital admission could potentially be avoided for a substantial proportion of patients (approximately one quarter when a standard troponin assay is used; over 40% with a high-sensitivity troponin assay). In this cohort, such a strategy would miss no AMIs. While these findings cannot be translated into clinical practice until they have been prospectively validated, this novel and simple strategy has the potential to substantially reduce resource utilisation. Performance may also be enhanced by refining emerging alternative early rule out strategies to take account of clinician gestalt.
Acknowledgments
We acknowledge the support of the National Institute for Health Research Greater Manchester Comprehensive Local Research Network, Manchester Academic Health Science Centre and all of the staff in the Emergency Department and Biochemistry Department at Stepping Hill Hospital who assisted with this work.
References
Supplementary materials
Supplementary Data
This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.
Files in this Data Supplement:
- Data supplement 1 - Online supplement
Footnotes
-
Collaborators Joanna Jarvis; Bethany Haves.
-
Contributors All authors meet the criteria for authorship as follows: RB and PL: conception and design, analysis and interpretation of the data, drafting manuscript or critical revisions for intellectual content and final approval of the manuscript. GC, GB and SC: analysis and interpretation of the data, drafting manuscript or critical revisions for intellectual content and final approval of the manuscript.
-
Funding The study was funded by a grant from the United Kingdom College of Emergency Medicine, was supported by fellowship funding (an Academic Clinical Lectureship for the first author) from the United Kingdom National Institute for Health Research (NIHR) and by the NIHR Clinical Research Network (UK CRN 8376), and used reagents donated for the purposes of the research by Roche Diagnostics.
-
Competing interests RB has undertaken research under collaborative agreements with Roche Diagnostics, Siemens Diagnostics, Alere Diagnostics and Randox Laboratories. RB has accepted travel and accommodation for conferences from Roche Diagnostics and Randox Laboratories.PSL has accepted travel and accommodation to lecture for Randox Laboratories.
-
Patient consent Obtained.
-
Ethics approval Research Ethics Committee Greater Manchester West.
-
Provenance and peer review Not commissioned; externally peer reviewed.
-
Data sharing statement Please contact RB for any questions about access to unpublished data from this study.
Linked Articles
- Commentary