Article Text

PDF

The Manchester Acute Coronary Syndromes (MACS) decision rule: validation with a new automated assay for heart-type fatty acid binding protein
  1. Richard Body1,2,
  2. Gillian Burrows3,
  3. Simon Carley2,4,
  4. Philip S Lewis3
    1. 1The University of Manchester, Manchester, UK
    2. 2Central Manchester University Hospitals NHS Foundation Trust, Manchester, UK
    3. 3Stockport NHS Foundation Trust, Manchester, UK
    4. 4Manchester Metropolitan University, Manchester, UK
    1. Correspondence to Dr Richard Body, Emergency Department, Manchester Royal Infirmary, Oxford Road, Manchester M13 9WL, UK; richard.body{at}manchester.ac.uk

    Abstract

    Objective The Manchester Acute Coronary Syndromes (MACS) decision rule may enable acute coronary syndromes to be immediately ‘ruled in’ or ‘ruled out’ in the emergency department. The rule incorporates heart-type fatty acid binding protein (h-FABP) and high sensitivity troponin T levels. The rule was previously validated using a semiautomated h-FABP assay that was not practical for clinical implementation. We aimed to validate the rule with an automated h-FABP assay that could be used clinically.

    Methods In this prospective diagnostic cohort study we included patients presenting to the emergency department with suspected cardiac chest pain. Serum drawn on arrival was tested for h-FABP using an automated immunoturbidimetric assay (Randox) and high sensitivity troponin T (Roche). The primary outcome, a diagnosis of acute myocardial infarction (AMI), was adjudicated based on 12 h troponin testing. A secondary outcome, major adverse cardiac events (MACE; death, AMI, revascularisation or new coronary stenosis), was determined at 30 days.

    Results Of the 456 patients included, 78 (17.1%) had AMI and 97 (21.3%) developed MACE. Using the automated h-FABP assay, the MACS rule had the same C-statistic for MACE as the original rule (0.91; 95% CI 0.88 to 0.92). 18.9% of patients were identified as ‘very low risk’ and thus eligible for immediate discharge with no missed AMIs and a 2.3% incidence of MACE (n=2, both coronary stenoses). 11.1% of patients were classed as ‘high-risk’ and had a 92.0% incidence of MACE.

    Conclusions Our findings validate the performance of a refined MACS rule incorporating an automated h-FABP assay, facilitating use in clinical settings. The effectiveness of this refined rule should be verified in an interventional trial prior to implementation.

    Trial registration number UK CRN 8376.

    • acute coronary syndrome
    • cardiac care, diagnosis
    • diagnosis

    Statistics from Altmetric.com

    Key messages

    What is already known on this subject?

    • The Manchester Acute Coronary Syndromes (MACS) decision rule has been shown to effectively risk stratify patients with suspected cardiac chest pain in the emergency department, and to safely identify approximately a quarter of patients who could be discharged following a single blood test. The original MACS rule incorporated heart-type fatty acid binding protein (h-FABP) but was calibrated using an assay that could not be easily run with a sufficiently low turnaround time for use in the Emergency Department.

    What this study adds?

    • In this work, we recalibrated the MACS rule to incorporate a new, automated, commercially available h-FABP assay that could easily be used in clinical practice with a similar turnaround time to routine tests. We validated the refined rule and demonstrated that overall diagnostic performance is maintained.

    How might this impact on clinical practice?

    • The incorporation of a h-FABP assay in the MACS rule presented a significant barrier to clinical implementation. Our findings remove that barrier. Prior to widespread clinical implementation, we recommend that the impact of using the rule in practice should be evaluated in interventional trials.

    Background

    Chest pain is the most common reason for emergency hospital admission accounting for approximately 250 000 admissions each year in England and Wales alone.1 Because it is so often impossible to differentiate chest pain caused by an acute coronary syndrome from less serious causes on the basis of a patient's symptoms,2 ,3 patients with previous history4 or ECG,5 are routinely admitted for serial troponin testing. This is limited by the need to repeat troponin testing hours after admission due to the lack of sensitivity of troponin at the time of first presentation to the emergency department (ED), even with a high sensitivity assay.6 The lack of specificity of high sensitivity troponin assays also poses challenges to emergency physicians who must accurately risk stratify patients on arrival, especially at lower levels of elevation and with a wide variety of comorbidities.7 ,8

    We recently reported the derivation and prospective external validation of the Manchester Acute Coronary Syndromes (MACS) decision rule, which combines biomarker levels (high sensitivity troponin T, hs-cTnT; and heart-type fatty acid binding protein, h-FABP) measured at the time of admission with clinical features (sweating observed; systolic BP <100 mm Hg on arrival; pain associated with vomiting; worsening angina; pain radiation to the right arm or shoulder) and ECG findings compatible with acute myocardial ischaemia to stratify patients into four risk groups. Patients in the very low risk group (27.0% of the validation cohort) could have been immediately discharged with no missed acute myocardial infarctions (AMIs) and a low rate (1.6%; both isolated coronary stenoses) of major adverse cardiac events (MACE) within 30 days. Of the 9.9% of patients in the high-risk group, 95.7% developed MACE within 30 days, suggesting that the MACS rule will help to guide clinicians to make judicious use of specialist inpatient resources.

    The MACS rule incorporates levels of h-FABP and was derived and validated with a semiautomated ELISA assay (Randox Laboratories Evidence Investigator Cardiac Array). It would be impractical to use this assay in the ED setting as it requires a large amount of manual input, currently relies on testing in batches (making testing single samples impossible) and would have an unacceptable turnaround time for use in emergency settings. Therefore, to facilitate clinical implementation Randox Laboratories has developed a Conformité Européenne (CE) marked, fully automated immunoturbidimetric assay that is compatible with commercially available modular analysers and can provide results in 15–20 min.9 This automated assay has different analytical characteristics to the ELISA and the results given by the two assays will not be perfectly correlated. Potentially this could lead to clinically important differences in diagnostic performance. It is imperative, therefore, that the MACS rule should be recalibrated to optimise performance with the new assay and that the diagnostic accuracy of the recalibrated rule should be verified.

    We aimed to validate the performance of the MACS rule when this alternative, fully automated assay is used.

    Methods

    This work is a secondary analysis from a prospective diagnostic cohort study (registered on the National Institute for Health Research Portfolio, UKCRN registration number 8376). We included consenting adults aged >25 years who presented to the ED at Stepping Hill Hospital, Stockport within 24 h of experiencing chest pain suspected to be cardiac in origin by the initial treating physician. We excluded patients with another medical condition requiring hospital admission, renal failure needing dialysis, significant chest trauma with suspicion of myocardial contusion, pregnancy, those who did not speak English, prisoners and those for whom all means of follow-up would be impossible. We have already published the external validation of the MACS rule using a semiautomated ELISA assay for h-FABP in this cohort10 and an evaluation of the diagnostic value of clinical judgement for acute coronary syndromes.11 When an automated assay became available (facilitating use in an urgent care setting), aliquots of stored serum samples were tested for the purposes of this analysis.

    Laboratory analyses

    Blood was drawn at the time of presentation for biomarker evaluation. Serum samples were stored at −70°C pending later analysis. As part of their routine care, all patients underwent cardiac troponin T testing (cTnT, Roche Diagnostics 4th generation Elecsys, 99th centile 10 ng/L, coefficient of variation <10% at 35 ng/L) on arrival and at least 12 h after symptom onset. In accordance with the third universal definition, this formed part of the reference standard for AMI diagnosis.12 Aliquots of samples were later tested for hs-cTnT (Roche Diagnostics Elecsys, 99th centile 14 ng/L, coefficient of variation <10% at 12 ng/L, limit of blank 3 ng/L) and h-FABP (using the Randox Evidence Investigator Cardiac Array and the automated immunoturbidimetric assay from Randox Laboratories, County Antrim, Northern Ireland). After our initial analysis we became aware that the batch of hs-cTnT reagent used had been affected by a calibration shift.13 Samples were therefore retested using an unaffected batch. We present here the results of the retest using an unaffected batch.

    Re-calibration of the MACS rule

    The original MACS rule enables the computerised calculation of the probability (p) that MACE will occur within 30 days as follows:Embedded Image

    Where a is the level of hs-cTnT (ng/L); b is the h-FABP level (ng/mL); c is the presence of ECG findings consistent with acute myocardial ischaemia; d is the presence of sweating observed by the clinician; e is pain associated with vomiting; f is an initial systolic BP <100 mm Hg; g is the presence of worsening angina; and h is the presence of pain radiating to the right arm and/or shoulder. For each categorical variable, a value of 1 is assigned if the characteristic is present and 0 if absent.10

    The manufacturer provided the following correlation data for the two h-FABP assays, based on 355 samples: Y=1.3520x+0.2844, where Y is the observed level with the automated immunoturbidimetric assay and x is the observed level using the previous assay (r=0.971).14 Thus, x=((Y−0.2844)/1.3520). Applying this correction, the amended MACS rule formula returns the probability of MACE within 30 days as follows:Embedded Image

    Using this refined formula for the MACS rule, we calculated the probability of MACE within 30 days. As in the original decision rule, patients were then assigned to one of four risk groups, each of which is associated with a suggested area for disposition from the ED, as follows: (1) Very low risk (suitable for immediate discharge from the ED); (2) Low risk (suitable for serial troponin testing in a low dependency environment such as an ED observation ward); (3) Moderate risk (suitable for serial troponin testing in an acute environment such as an acute medical ward); (4) High-risk (patients in whom the diagnosis of an acute coronary syndrome may be considered ‘ruled in’ and who may therefore be suitable for early referral to a cardiologist).

    Follow-up

    Patients were followed up after 30 days by telephone, email, letter, home visit or in clinic. If we were unable to complete follow-up by any of these means we contacted the patient's general practitioner.

    Outcomes

    The primary outcome was MACE within 30 days. This was a composite of prevalent or incident AMI, death (all cause) or the need for coronary revascularisation (primary percutaneous coronary intervention or coronary artery bypass grafting). In the event that a patient was found to have a significant (>50%) angiographic coronary stenosis (as reported by the treating interventional cardiologist) that was not known to be old, they were also deemed to have met the primary outcome. As a secondary outcome we also analysed the prevalence of AMI as a lone outcome.

    The diagnosis of AMI was adjudicated by two independent investigators who had all clinical, laboratory and imaging evidence available for review, but who were blinded to investigational biomarker levels. In accordance with the universal definition of AMI,12 patients were required to have a troponin rise and/or fall (≥20 ng/L, based on the analytical characteristics of the assay) with at least one troponin measurement above the 99th centile (10 ng/L) in the appropriate clinical context.

    Sensitivity analyses

    We also ran a sensitivity analysis to evaluate the impact of using a reference standard for AMI based on a hs-cTnT. We measured hs-cTnT levels in all available samples (presentation and 12 h) and readjudicated the diagnosis of AMI based on those levels, blinded to cTnT levels and other investigational biomarker levels. A rise and/or fall of >9.2 ng/L (ie, ≥10 ng/L as levels were reported in integers) was considered to be significant.15

    Statistical analysis

    To evaluate diagnostic performance we calculated the area under the receiver operating characteristic curve (AUC; also known as the C-statistic), sensitivity, specificity, positive predictive value (PPV) and negative predictive value (NPV) together with 95% CIs (using SPSS V.20.0, SPSS, Chicago, Illinois; and/or MedCalc V.12.4.0.0, Mariakerke, Belgium). Paired proportions were compared using McNemar's test (SPSS V.20.0, Chicago, Illionois, USA). CIs for single proportions were calculated by the modified Wald method.16

    Sample size

    We powered the study such that, if the clinical decision rule (CDR) was shown to have a sensitivity of 100%, the 95% CI would extend no lower than 95%. Assuming a 20% incidence of the primary outcome, a sample size of 450 participants would ensure this degree of precision.

    Results

    In total 456 patients, who participated between April 2010 and July 2010, were included in this analysis. All of these patients completed follow-up (figure 1). Of those patients, 78 (17.1%) had AMI and 97 (21.3%) developed MACE within 30 days. Baseline characteristics are shown in table 1.

    Table 1

    Baseline characteristics of included patients

    Figure 1

    Patient flow diagram. h-FABP, heart-type fatty acid binding protein; hs-cTnT, high sensitivity troponin T.

    Using the automated immunoturbidimetric (IT) h-FABP assay the MACS rule had an AUC of 0.91 (95% CI 0.88 to 0.95) for predicting MACE (figure 2) and an AUC of 0.96 (95% CI 0.94 to 0.98) for diagnosing AMI, implying near perfect overall diagnostic accuracy. The rates of AMI and MACE stratified by the refined MACS rule risk group (incorporating the IT assay), are shown in table 2. Assuming that only patients in the very low risk group would be discharged from hospital, as a dichotomous tool to guide hospital admission for predicting MACE the refined MACS rule had a sensitivity of 97.9% (95% CI 92.8% to 99.8%), a specificity of 23.4% (19.1% to 28.1%), a PPV of 25.7% (21.3% to 30.5%) and a NPV of 97.7% (91.9% to 99.7%). For the diagnosis of AMI, the refined MACS rule had a sensitivity of 100.0% (95% CI 95.4% to 100.0%), specificity 22.8% (18.6% to 27.3%), PPV 21.1% (17.0% to 25.6%) and NPV 100.0% (95.8% to 100.0%).

    Table 2

    Rates of AMI and MACE stratified by the refined MACS rule risk groups

    Figure 2

    Receiver operating characteristic curves showing the overall accuracy of the refined Manchester Acute Coronary Syndromes (MACS) rule for predicting major adverse cardiac events (MACE) at 30 days.

    In comparison with the original MACS rule (incorporating the ELISA assay for h-FABP), the recalibrated MACS rule incorporating the automated IT assay identified fewer patients as being at very low risk and therefore eligible for immediate discharge (18.9% vs 27.0%, p<0.0001). The same two patients in the very low risk group developed MACE. Both had coronary stenoses identified at angiography that were not deemed to warrant intervention. No AMIs were missed using either version of the MACS rule. Interestingly, using the IT assay, the incidence of MACE at 30 days was 0.0% in the ‘low risk’ group, suggesting that this group of patients may also be safely discharged from hospital. This strategy would enable a total of 40.5% of patients to be eligible for immediate discharge with no missed AMIs and a 1.1% incidence of MACE (isolated coronary stenoses only).

    Sensitivity analyses

    In our first sensitivity analysis we evaluated the impact of using hs-cTnT as the reference standard for AMI. With the outcome readjudicated, 76 (16.7%) patients had AMI meaning that a total of 96 (21.1%) patients developed MACE within 30 days. The AUC for MACE was unchanged (0.91, 95% CI 0.88 to 0.95), as was the AUC for AMI (0.95, 95% CI 0.93 to 0.97). The prevalence of AMI and incidence of MACE stratified by MACS rule risk group using this refined reference standard are shown in the online supplementary table 1. Diagnostic performance was essentially unchanged.

    For the second sensitivity analysis we examined the impact of incorporating the h-FABP IT assay without making any adjustments to the original formula. Under those conditions the AUC was 0.92 (95% CI 0.88 to 0.95) for MACE and 0.96 (95% CI 0.94 to 0.98) for AMI. As shown in the online supplementary table 2, performance of the rule was unchanged using this formula, although 31.6% of patients could have been discharged immediately with no missed AMIs and a 1.4% (n=2) incidence of MACE.

    Discussion

    In this report we have demonstrated the performance of the MACS rule when recalibrated to incorporate the use of a fully automated immunoturbidimetric assay to determine h-FABP levels. This assay can be run using commercially available modular laboratory analysers and has a similar turnaround time to other widely used diagnostic tests, making it possible to affect clinical decisions within the required timescale for patients attending the ED. For our primary analysis we applied a formula to recalibrate the MACS rule for this specific assay, based on correlation data provided by the manufacturer. Our findings demonstrate that the overall diagnostic performance of the refined MACS rule is similar to the original version. We observed no increase in the prevalence of missed AMI (0.0% in this study) or the incidence of MACE at 30 days (n=2, 2.3%; both isolated coronary stenoses managed on an outpatient basis).

    The proportion of patients that would be eligible for immediate discharge from the ED is lower using this refined version of the MACS rule (18.9% vs 27.0% with the original rule). Interestingly, however, the incidence of MACE in the ‘low risk’ group is 0.0% suggesting that it may also be safe to immediately discharge this group of patients. Such a strategy would enable a total of 40.5% of patients to be immediately discharged from the ED, which is likely to substantially reduce healthcare resource utilisation and economic costs.

    One feature of the MACS rule that is different to some contemporary accelerated diagnostic protocols is that it aims to provide overall risk stratification for undifferentiated patients with suspected cardiac chest pain. As such, our studies have included patients with high-risk features such as ECG evidence of myocardial ischaemia, rather than specifically selecting the low risk patients. On this note, the refined MACS rule presented here identified 11% of patients as being at ‘high-risk’. These patients had a 92% incidence of MACE, which could be considered to effectively ‘rule in’ the diagnosis of an acute coronary syndrome. If ‘very low risk’ and ‘low risk’ patients were considered to have had the diagnosis ‘ruled out’, a minority (48.5%) of patients would remain undiagnosed pending serial troponin testing. It is likely that the additional serial sampling after 60–180 min would enable an even greater proportion of patients to have the diagnosis ‘ruled in’ or ‘ruled out’. Such a ‘staggered rule out protocol’ warrants further evaluation.17–20

    Our study has some limitations. Prior to any statistical analysis (and thus a priori) we made the decision to recalibrate the MACS rule based on correlation data provided by the manufacturer. While our findings suggest that this refined version of the MACS rule would be safe to use in practice and would effectively risk stratify patients, the proportion of patients eligible for immediate discharge would fall. As demonstrated in a sensitivity analysis, if we had used the original formula for the MACS rule then the performance of the decision rule would actually have been enhanced, making a greater proportion of patients eligible for safe immediate discharge. Given that this finding was made only as part of a sensitivity analysis, it still requires prospective confirmation.

    Another potential limitation of this work is the incorporation of coronary stenosis in the primary outcome (MACE). The rationale for this decision was that, if only patients who actually underwent coronary revascularisation were considered to have MACE, then patients who had significant coronary disease that was not amenable to revascularisation may not have been accounted for. However, with hindsight the inclusion of this criterion may have led to inappropriate underestimation of the sensitivity and NPV of the MACS rule. The two patients in the very low risk group who were found to have coronary stenoses had undergone outpatient angiography and were not deemed to warrant coronary revascularisation. Arguably, such events would not preclude early discharge from the ED.

    It is also worth noting that cardiac troponin forms an important part of the reference standard for diagnosing AMI, which requires the detection of rise and/or fall of cardiac troponin.12 Cardiac troponin levels are also included in the MACS rule. Strictly speaking, this introduces a form of incorporation bias. Research in this field is typically subject to similar incorporation bias when investigators aim to compare early troponin measurement with late sampling. This is likely to have little effect on the primary analyses we have reported but it is important to note the incorporation bias as it may affect alternative interpretations of the data presented.

    This work opens up three important avenues for further research. First, our findings make it possible to evaluate the MACS decision rule in practice using a h-FABP assay that can be readily used in the clinical setting. Prior to clinical implementation of the MACS rule we recommend that its use should be evaluated in practice, for example in the context of a randomised controlled trial. This will enable the performance of the MACS rule to be verified when clinicians and patients (who are likely to have imperfect compliance with the rule) are asked to use the rule to guide decisions in real time. It will also enable robust evaluation of cost-effectiveness, which is especially important given the findings of previous work. The Randomised Assessment of Treatment using Panel Assay of Cardiac Markers trial evaluated an early ‘rule out’ strategy for acute coronary syndromes, which consisted of serial testing for troponin, creatine kinase-MB and myoglobin at the time of presentation and 90 min later. Use of this protocol was found to successfully increase the proportion of patients that was successfully discharged from the ED.21 However, because of an increase in the use of specialist coronary care resources in the intervention group, overall healthcare resource use increased and the strategy was not cost-effective.22

    Second, it is important to evaluate whether the MACS rule can be combined with other promising early rule out strategies that involve serial high sensitivity troponin testing over 60–180 min. This may further reduce the number of unnecessary hospital admissions and reduce the number of patients who remain in the ‘observational zone’ (who have had the diagnosis of an acute coronary syndrome neither ‘ruled in’ nor ‘ruled out’) upon leaving the ED still further. Third, it is important to prospectively compare the MACS rule to other emerging alternatives that could potentially lead to safe, immediate hospital discharge, including the History, ECG, Age, Risk factors, Troponin (HEART) score,23 ,24 use of copeptin25 and strategies to ‘rule out’ acute coronary syndromes among patients who have initial troponin levels below the limit of blank or limit of detection of a high sensitivity assay.26 ,27 It will also be important to compare the use of the MACS rule with tools that could enable an acute coronary syndrome to be excluded within 1 h to 2 h of arrival, for example the ADAPT protocol18 or the recently described method to exclude AMI within 1 h using hs-cTnT.19

    Conclusions

    We report validation of the performance of the MACS rule incorporating an automated assay for h-FABP. We found that diagnostic performance is maintained. This version of the MACS rule could enable 18.9% of patients presenting to the ED with suspected cardiac chest pain to be immediately discharged while identifying 11.1% of ‘high-risk’ patients who may benefit from early specialist opinion and use of high dependency resources. Validation of the MACS rule with an automated h-FABP assay that has an acceptable turnaround time for use in emergency settings will facilitate clinical implementation. To verify the findings from this observational study, we recommend use of the refined MACS rule is evaluated in a trial setting prior to clinical implementation.

    Acknowledgments

    The authors acknowledge the support of the Manchester Academic Health Science Centre, the Manchester Biomedical Research Centre and the National Institute for Health Research (UK) Clinical Research Network. The authors also acknowledge the support of all the staff in the Emergency Department and Biochemistry Department at Stepping Hill Hospital and the Department of Laboratory Medicine at Manchester Royal Infirmary who assisted with this work, in particular Sister Joanna Jarvis.

    References

    View Abstract
    • Supplementary Data

      This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.

      Files in this Data Supplement:

    Footnotes

    • Collaborators Joanna Jarvis.

    • Contributors RB: Conception and design; Analysis and interpretation of the data; Drafting the manuscript or critical revisions for intellectual content; Final approval of the manuscript. GB, SC, PSL: Analysis and interpretation of the data; Drafting the manuscript or critical revisions for intellectual content; Final approval of the manuscript.

    • Funding This study was funded by a grant from the UK College of Emergency Medicine, was supported by: (A) fellowship funding (an Academic Clinical Lectureship for the first author) from the UK National Institute for Health Research (NIHR); (B) by the NIHR Clinical Research Network Clinical Research Network (UK CRN 8376); and (C) reagents donated for the purposes of the research by Roche Diagnostics.

    • Competing interests RB has undertaken research under collaborative agreements with Roche Diagnostics, Siemens Diagnostics, Alere Diagnostics and Randox Laboratories. RB has accepted travel and accommodation for conferences from Roche Diagnostics and Randox Laboratories. PSL has accepted travel and accommodation to lecture for Randox Laboratories.

    • Patient consent Obtained.

    • Ethics approval Research Ethics Committee, North West Cheshire (reference 09/H1014/74).

    • Provenance and peer review Not commissioned; externally peer reviewed.

    • Data sharing statement Please contact the corresponding author to request additional data or analyses.

    Request permissions

    If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.