Article Text
PDF
Abstract
Background The RATPAC trial showed that using a point-of-care panel of CK-MB(mass), myoglobin and troponin at baseline and 90 min increased the proportion of patients successfully discharged home, leading to reduced median length of initial hospital stay. However, it did not change mean hospital stay and may have increased mean costs per patient. The aim of this study was to explore variation in outcome and costs between participating hospitals.
Methods RATPAC was a pragmatic multicentre randomised controlled trial (N=2243) and economic analysis comparing diagnostic assessment using the panel to standard care for patients with acute chest pain due to suspected myocardial infarction at six hospitals. The difference in the proportion of patients successfully discharged (primary outcome) and mean costs per patient between the participating hospitals was compared.
Results Point-of-care assessment led to a higher proportion of successful discharges in four hospitals, a lower proportion in one and was equivocal in another. The OR (95% CI) for the primary outcome varied from 0.12 (0.01 to 1.03) to 11.07 (6.23 to 19.66) with significant heterogeneity between the centres (p<0.001). The mean cost per patient for the intervention group ranged from being £214.49 less than the control group (−132.56 to 657.10) to £646.57 more expensive (73.12 to 1612.71), with weak evidence of heterogeneity between the centres (p=0.0803).
Conclusion The effect of point-of-care panel assessment on successful discharge and costs per patient varied markedly between hospitals and may depend on local protocols, staff practices and available facilities.
- Myocardial infarction
- randomised trial
- cardiac markers
- diagnosis
- cost-effectiveness
- cardiac care
- acute myocardial infract
Statistics from Altmetric.com
- Myocardial infarction
- randomised trial
- cardiac markers
- diagnosis
- cost-effectiveness
- cardiac care
- acute myocardial infract
Introduction
The RATPAC (Randomised Assessment of Treatment using Panel Assay of Cardiac markers) trial was a pragmatic unblinded randomised controlled trial comparing point-of-care panel assessment with a combination of creatine kinase MB enzyme (CK-MB), myoglobin and cardiac troponin I (cTnI) at baseline and 90 min, to standard care for patients with chest pain in whom myocardial infarction needed to be excluded. It showed that point-of-care panel assessment resulted in more patients being successfully discharged after emergency department assessment, reduced median length of hospital stay but not mean length of stay, and increased use of coronary care.1 An economic analysis showed that the point-of-care panel assessment did not save health and social care costs, may have increased costs and was unlikely to be considered cost-effective.2
The effects and cost-effectiveness of diagnostic technologies depend on the setting in which they are used.3 Factors such as the available facilities (clinical decision units or observation areas), the decision-making protocol in use, existing guidelines and standards, and the attitudes of staff to new technologies may determine whether diagnostic tests influence practice in a particular setting. In particular, when the RATPAC trial was undertaken, the UK National Health Service was subject to a national standard requiring 98% of patients to be admitted or discharged from the emergency department by 4 h after arrival. The approach taken by hospitals to achieving the target could have a significant influence on the impact of the point-of-care panel.
RATPAC was a multicentre trial involving six diverse hospitals that each contributed substantially towards patient recruitment. This provided an opportunity to explore variation between sites in terms of the facilities available to manage acute chest pain, the standard care protocols in existence and the outcomes of the trial. The aim of this study was to compare the outcomes of the RATPAC trial in the different settings and determine whether there was significant variation.
Methods
The methods of the RATPAC trial1 and economic evaluation2 are described fully elsewhere but are briefly outlined below. The trial took place in six hospitals, as described in table 1. The hospitals varied in size and facilities, with three having access to a Clinical Decision Unit.
Characteristics of the participating centres
Adults (age >25 years) were eligible for recruitment if they had acute chest pain due to possible myocardial infarction (ie, no diagnostic ECG changes or alternative pathology) within the previous 12 h and could be discharged home if diagnostic assessment for myocardial infarction was negative. Informed consent was sought from all participants, who were then randomised to point-of-care panel assessment or standard care using a web-based randomisation system.
Patients randomised to point-of-care panel assessment received diagnostic testing with CK-MB (as a mass assay), myoglobin and cTnI at baseline and 90 min using the Seimens Stratus CS analyser. The analytical characteristics of the assays were as follows: cTnI detection limit 0.03 μg/l, analytical range 0.03–50 μg/l, interassay CV 4.0–8.2% (0.067–0.344 μg/l). The 99th centile of the assay is 0.07 μg/l. Myoglobin: detection limit 1 μg/l; analytical range 1–900 μg/l; interassay CV 1.9–12.7% (56–308 μg/l); 95% reference interval, males 21–98 μg/l, females 19–56 μg/l, combined 20–82 μg/l. CK-MB: detection limit of 0.3 μg/l; analytical range 0.3–150 μg/l; interassay CV 0.15–1.27% (3.7–39.3 μg/l); 95% reference interval 0.6–3.5 μg/l. Hospital admission or discharge was ultimately at the discretion of the physician, but guidelines provided for the trial recommended admission if there was a troponin rise above 0.03 μg/l (amended during the trial to above 0.07 μg/l or a rise from <0.03 μg/l to ≥0.03 μg/l), a myoglobin rise of >25% from baseline, a CK-MB elevation above 5 μg/l at both time points or a CK-MB gradient >1.6 μg/l. The troponin threshold was amended as emerging experience with the assay suggested that the initial choice of threshold was too conservative and resulted in low initial ‘positive’ results that were not subsequently shown to be the start of a rise above the 99th centile.
Standard care was provided according to the existing guidelines at each hospital. These are outlined in table 2. Patients were referred to the cardiology team if biomarkers were positive and discharged (with or without exercise treadmill testing or referral to an outpatient clinic) if negative. No attempt was made to influence standard care at any hospital.
Existing management strategies for low-risk chest pain
Patient management in both arms of the trial was at the discretion of the physician and followed normal hospital procedures, with the exception of use of the point-of-care panel assessment in the intervention arm. All patients were followed up by case note review and self-complete postal questionnaire at 1 and 3 months.
The primary outcome was the proportion of patients successfully discharged home after emergency department assessment. To be considered successfully discharged the patient had to 1. have either left the hospital or be awaiting transport home with a discharge decision having been made at 4 h after initial presentation and 2. suffer no major adverse event (as defined below) during the following 3 months. Secondary outcomes included health utility (measured using the EQ-5D self-complete questionnaire at 1 and 3 months after attendance), length of initial hospital stay and total inpatient days over 3 months, use of coronary care, intensive care and cardiac interventions and major adverse events (death, non-fatal acute myocardial infarction, life-threatening arrhythmia, emergency revascularisation or hospitalisation for myocardial ischaemia). Diagnosis of myocardial infarction was based on the decision of the most senior clinician recorded in the case notes. Emergency revascularisation was defined as revascularisation performed within 24 h of the decision to revascularise.
Case note review was used to identify hospital resource use (length and location of hospital stay, use of cardiac interventions, outpatient reviews, emergency department attendances and subsequent hospital admissions), whereas the postal questionnaire was used to identify community and social care resource use. Resources were valued using national unit costs4 5 to estimate total health and social care costs up to 3 months after initial attendance.
The proportion of patients successfully discharged was analysed through logistic regression, fitting concurrently with intervention group the effect of centre and appropriate baseline measures (including age, gender and past history of coronary heart disease), to present adjusted odds ratios along with their corresponding 95% CIs. The test for heterogeneity between centres was tested by fitting an interaction between outcome and centre, applying a likelihood ratio test. Cost analysis compared bootstrap estimates of the mean cost per patient of the two groups. It was anticipated that some of the resource use data would be incomplete (missing). Thus, in order to maximise the information collected from the trial, missing values were imputed using multiple imputation.6 All analyses were undertaken using the Stata statistical package (Release 10).
Results
Patients were recruited between 30 January 2008 and 2 June 2009. Table 3 summarises the process of screening and recruitment at each centre. All hospitals stopped recruitment on 2 June 2009 but the staggered start meant that some recruited for more days than others. Overall, 2263 patients were recruited over a total of 2658 hospital-days (0.9 patients per day). The total at each site ranged from 327 to 469, and the recruitment rate ranged from 0.7 to 1.1 per day.
Screening and recruitment
Some 20 patients (seven in the point-of-care arm and 13 in the standard care arm) did not complete initial follow-up to recording of the primary outcome, so 2243 cases were available for analysis. Mean age was 54.5 years, 1307/2243 (58%) were male and 269 (12%) had known coronary heart disease. The median time from worst symptoms to presentation was 129 min (IQR 80–246). The flow of patients through the trial is shown in supplementary figure 1. Table 3 also shows the basic characteristics of the participants at each centre.
Table 4 shows the proportion of patients successfully discharged across all centres and at each centre. Point-of-care panel assessment was associated with substantial increases in successful discharge rates at Barnsley and Edinburgh, modest increases at Derriford and Frenchay, and no increase at Leeds or Leicester. The heterogeneity in outcomes was highly statistically significant (χ2=75.5, degrees of freedom=5, p<0.001), indicating that the effect of point-of-care testing varied significantly between the participating hospitals.
Proportion of patients successfully discharged by study centre
Figure 1 shows the proportion of patients in hospital as a function of time from arrival. Point-of-care panel assessment was associated with markedly fewer patients being in hospital up to 24 h at Barnsley and Edinburgh. At Derriford the difference in proportion in hospital was only apparent between 4 and 8 h. At Frenchay the difference was marked up to 12 h, but after 12 h the proportion of patients in hospital was greater in the point-of-care group. At Leeds the difference between the groups did not emerge until 6 h after attendance, but between 6 and 24 h point-of-care panel assessment was associated with markedly fewer patients being in hospital. Finally, at Leicester there was no difference in the proportions in hospital up to 12 h and only slightly fewer patients in hospital in the point-of-care group from 12 to 36 h.
Duration from arrival to discharge from hospital (individual centres). Solid line, point-of-care group; dashed line, standard care group.
Table 5 shows the mean costs per patient in the two treatment groups, across all centres and at each individual centre. The difference between the mean costs at each centre are reported with a 95% CI. This difference ranged from £214.49 less in the point-of-care group at Leeds to £646.57 more at Edinburgh, but only the difference at Edinburgh was statistically significant (p=0.025). An ANOVA test across centres (based on permuted test summed across the imputed data sets) yielded a p value of 0.0803, indicating weak evidence of heterogeneity between centres. The statistical power of this test to detect variation between centres is limited by the large variances in cost data, but it suggests that the effect of point-of-care panel assessment on means costs per patient varied between hospitals.
Mean total costs per patient (£) by treatment group and site
Discussion
The RATPAC trial showed that cardiac marker point-of-care panel assessment increased the proportion of patients successfully discharged after emergency department assessment, but the effect on the proportion of patients in hospital was limited to the first 24 h after attendance.1 So, although median length of stay was reduced, mean length of stay was unchanged. Costs associated with point-of-care testing, additional coronary care admissions and cardiac interventions meant that point-of-care panel assessment did not save costs and may have increased costs.2
This study has shown that the effect of point-of-care panel assessment varied markedly between hospitals, suggesting that the effect of point-of-care panel assessment may depend on the setting and that the general findings of the RATPAC trial may not apply at all hospitals. It is likely that differences in the facilities available, local protocols, existing guidelines for chest pain, existing troponin assays or staff using the point-of-care tests explain the variation in outcomes and costs. However, caution should be taken about attempting to identify explanations for outlying results in specific characteristics of the hospital concerned. The estimates of proportion of patients successfully discharged or mean costs per patient are subject to substantial random error when analysed at individual hospital level. For example, the reversed trend observed in the proportion successfully discharged at Leeds was based on eight cases in the control group versus one in the intervention group being successfully discharged within 4 h after assessment. Furthermore, a statistically significant result for a specific hospital (such as the comparison of mean costs per patient at Edinburgh) is one of many hypothesis tests and thus carries a risk of being a spurious false-positive finding.
Nevertheless, some evidence of the effect of local practice on outcome seems to be apparent in figure 1, which shows the differences between the hospitals in terms of the proportion of patients in hospital as a function of time from initial attendance. For example, standard care at Derriford was based on a troponin level measured 6 h after arrival in hospital, which may explain why the effect of point-of-care panel assessment at Derriford was limited to the 4–8 h window. By contrast, at Leeds all patients with chest pain are admitted to a Clinical Decision Unit where diagnosis and management is undertaken without the pressure of the 4 h target. This may explain why the effect of point-of-care panel assessment was delayed at Leeds until 6 h after arrival. Finally, at Leicester the point-of-care tests did not seem to alter decision-making with regards to hospital admission. This may be because the point-of-care tests were performed by research nurses at Leicester, leading to less ‘buy-in’ by the decision-making medical staff, whereas at other hospitals they were performed by the physician. A number of other factors may have influenced outcomes at different hospitals, such as differences in the sensitivity or threshold used for the troponin assay. Overall, the present analysis suggests that the intervention would be more likely to have an impact at hospitals where it is more distinct from standard care, where it helps to address specific service targets and where it is used by decision-making clinicians. However, these observations are difficult to generalise between settings.
RATPAC is the only randomised trial to date to have examined the effect of point-of-care testing in a rapid rule-out protocol. However, other trials3 7 have compared the use of point-of-care and laboratory troponin assays. The DISPO-ACS trial3 was a multicentre trial that evaluated point-of-care troponin testing in four emergency departments and found that the effect varied between settings, with length of stay in the emergency department being increased in one hospital and decreased in another. Meanwhile, the ESCAPE multicentre trial of chest pain units showed that variation in the structure, processes and outcomes of chest pain units was associated with variation in unit activity, which was in turn associated with differences in outcome between centres.8 9
These findings from multicentre trials suggest that there is a pattern in the evaluation of chest pain diagnostic testing regimes, whereby promising findings from earlier single-centre studies are variably reproduced across multiple centres with only a modest, negative or no overall effect. The implications of this observation for interpreting research findings are twofold. First, new diagnostic testing regimes should not be introduced on the basis of positive experience at a single centre, which may be atypical, but should be evaluated in multicentre trials before widespread adoption. Second, negative findings from a multicentre trial, although suggesting that widespread adoption of a testing regime is inappropriate, should not be considered proof that the intervention cannot be effective in specific circumstances. It may be relatively easy within a single centre for the medical staff to engage with a new intervention, whereas in a multicentre trial physicians may be less willing to change their behaviour. Future studies will need to collect more data about system and physician factors that might influence the outcome in order to better define the circumstance in which the intervention may be of value.
The limitations of the main RATPAC analysis have previously been outlined.1 The most relevant of these to this analysis is that clinician behaviour (especially discharge decisions) may have been influenced by participation in the trial. If this influence varied between hospitals then it could have produced some of the variation observed in the results. Additional limitations that relate to this particular study include 1. the trial was powered on the basis of recruitment across all centres rather than at each individual centre so it was underpowered to detect potentially important differences at each centre, 2. analysis at six different hospitals involves multiple hypothesis testing with the associated risk of spurious false-positive findings and 3. the six sites, although selected to represent a diverse range of hospitals, were also selected on the basis of ability and willingness to support research, so they may not be typical hospitals.
In conclusion, the effect of point-of-care panel assessment on the proportion of patients successfully discharged and mean costs per patient in the RATPAC trial varied between participating hospitals. This suggests that the impact of point-of-care panel assessment depends on the setting in which it is used. New diagnostic technologies should not be implemented until they have been evaluated in diverse settings, and ideally in multicentre studies that explore the interaction between the setting and the effect of the intervention.
Acknowledgments
We would like to thank Margaret Jane for her invaluable clerical assistance; David Gaze at St George's Hospital, London, for his helpful advice and assistance with all things blood tube related; staff involved at the NHS sites for all their hard work and participants for their involvement; Francis Morris for advice on emergency department data and Siemens for providing the Stratus CS machines and technical support and assistance.
Appendix I The RATPAC Research Team
Charlotte Arrowsmith (RATPAC Research Nurse, Derriford Hospital, Plymouth); Julian Barth (Consultant in Chemical Pathology, Leeds General Infirmary/Co-applicant); Jonathan Benger (Professor of Emergency Care, University of the West of England/Co-applicant); Mike Bradburn (Senior Medical Statistician, Clinical Trials Research Unit, University of Sheffield); Simon Capewell (Professor of Epidemiology, University of Liverpool/Co-applicant); Tim Chater (Database Manager, Clinical Trials Research Unit, University of Sheffield); Tim Coats (Professor of Emergency Medicine/Co-applicant); Paul Collinson (Consultant in Chemical Pathology, St George's Hospital, London/Co-applicant); Cindy Cooper (Director, Clinical Trials Research Unit, University of Sheffield); Mandy Cooper (RATPAC Research Nurse, Leicester Royal Infirmary); Judy Coyle (RATPAC Research Nurse, Edinburgh Royal Infirmary); Liz Cross (Trial Manager, Health Services Research, University of Sheffield); Simon Dixon (Professor of Health Economics, Health Economics & Decision Science/Co-applicant); Patrick Fitzgerald (Research Fellow, Health Economics & Decision Science, University of Sheffield); Emma Gendall (RATPAC Research Nurse, Frenchay Hospital, Bristol); Steve Goodacre (Professor of Emergency Medicine, Health Services Research, University of Sheffield/Chief Investigator); Emma Goodwin (RATPAC Research Nurse, Barnsley Hospital); Alasdair Gray (Consultant in Emergency Medicine, Royal Infirmary of Edinburgh/Co-applicant); Alistair Hall (Professor of Clinical Cardiology, University of Leeds/Co-applicant); Kevin Hall (RATPAC Research Nurse, Barnsley Hospital); Taj Hassan (Consultant in Emergency Medicine, Leeds General Infirmary/Co-applicant); Julian Humphrey (Consultant in Emergency Medicine, Barnsley Hospital); Steven Julious (Senior Lecturer in Medical Statistics, Medical Statistics Group, Health Services Research, University of Sheffield/Co-applicant); Jason Kendall (Consultant in Emergency Medicine, Frenchay Hospital, Bristol); Vanessa Lawlor (RATPAC Research Nurse, Frenchay Hospital, Bristol); Sue Mackness (RATPAC Research Nurse, Leicester Royal Infirmary); Yvonne Meades (RATPAC Research Nurse, Leeds General Infirmary); David Newby (Professor of Cardiology, University of Edinburgh/Co-applicant); Dawn Newell (RATPAC Research Nurse, Leicester Royal Infirmary); Doris Quartey (RATPAC Research Nurse, Leeds General Infirmary); Karen Robinson (RATPAC Research Nurse, Leicester Royal Infirmary); Glen Sibbick (RATPAC Research Nurse, Leicester Royal Infirmary); Jason Smith (Consultant in Emergency Medicine, Derriford Hospital, Plymouth); Roz Squire (RATPAC Research Nurse, Derriford Hospital, Plymouth).
Trial Steering Committee
Simon Carley, Consultant in Emergency Medicine, Manchester Royal Infirmary (Independent Member); Paul Collinson, Consultant in Chemical Pathology, St George's Hospital, London (Co-applicant); Liz Cross, Research Associate, Health Services Research, University of Sheffield (Trial Manager); Sue Dodd, CHD Emergency & Acute Care Manager (Vascular Programme) Department of Health (Independent Member); Marcus Flather, Director Clinical Trials & Evaluation Unit, Royal Brompton Hospital, London (Chair); Steve Goodacre, Professor of Emergency Medicine, Health Services Research, University of Sheffield (Chief Investigator); Sara Hilditch, Statistician, Director of Statistical Services Unit, Sheffield University (Independent Member); Enid Hirst (Lay representative); Richard Hudson, Quality Assurance Manager, Research Office, Academic Division, University of Sheffield (Sponsor Representative); Jason Kendall, Consultant in Emergency Medicine, Frenchay Hospital, Bristol (Investigator Representative); Rebecca Whitlock-Moss, Programme Manager, NIHR HTA (Funder Representative).
Data Monitoring Committee
John Greenwood, Senior Lecturer/Consultant Cardiologist, Academic Unit of Cardiovascular Medicine, Leeds General Infirmary (Independent Clinician); Jon Nicholl, Professor of Health Services Research, University of Sheffield (Acting Chair); Helen Thorpe, Principal Statistician, Clinical Trials Research Unit, University of Leeds (Chair & Independent Statistician); William Townend, Consultant in Emergency Medicine, Hull Royal Infirmary (Independent Clinician).
Footnotes
Disclaimer The views and opinions expressed therein are those of the authors and do not necessarily reflect those of the NIHR HTA.
↵* The RATPAC Research Team, Trial Steering Committee and Data Monitoring Committee are listed in appendix 1.
Funding National Institute for Health Research, Health Technology Assessment Programme
Competing interest None.
Ethics approval This study was conducted with the approval of the Leeds East REC.
Provenance and peer review Not commissioned; externally peer reviewed.
Linked Articles
- Primary survey