Article Text
PDF
Abstract
Aim We compared the abilities of two established clinical scores to predict emergency department (ED) disposition: the Glasgow Admission Prediction Score (GAPS) and the Ambulatory Score (Ambs).
Methods The scores were compared in a prospective, multicentre cohort study. We recruited consecutive patients attending ED triage at two UK sites: Northern General Hospital in Sheffield and Glasgow Royal Infirmary, between February and May 2016. Each had a GAPS and Ambs calculated at the time of triage, with the triage nurses and treating clinicians blinded to the scores. Patients were followed up to hospital discharge. The ability of the scores to discriminate discharge from ED and from hospital at 12 and 48 hours after arrival was compared using the area under the curve (AUC) of their receiving-operator characteristics (ROC).
Results 1424 triage attendances were suitable for analysis during the study period, of which 567 (39.8%) were admitted. The AUC for predicting admission was significantly higher for GAPS at 0.807 (95% CI 0.785 to 0.830), compared with 0.743 (95% CI 0.717 to 0.769) for Ambs, P<0.00001. Similar results were seen when comparing ability to predict hospital stay of >12 hour and >48 hour. GAPS was also more accurate as a binary test, correctly predicting 1057 outcomes compared with 1004 for Ambs (74.2vs70.5%, P=0.012).
Conclusion The GAPS is a significantly better predictor of need for hospital admission than Ambs in an unselected ED population.
- emergency care systems, emergency departments
- effectiveness
- admission avoidance
- research, operational
- triage
Statistics from Altmetric.com
- emergency care systems, emergency departments
- effectiveness
- admission avoidance
- research, operational
- triage
Key messages
What is already known on this subject
The Ambulatory Score (Ambs) is recommended by the Royal College of Physicians as a tool to determine which patients may be suitable for ambulatory emergency care. The Glasgow Admission Prediction Score (GAPS) is known to be an accurate predictor of emergency department (ED) disposition from the point of triage, outperforming experienced triage nurses in most cases. It is not known which is the best predictor of disposition in an unselected ED population.
What this study adds
In this prospective cohort study at two geographically distinct EDs, a direct comparison of Ambs and GAPS shows that in the general ED population, GAPS is a better predictor of ED disposition than the Ambs.
Introduction
Crowding in the emergency department (ED) threatens clinical safety, reduces patient satisfaction, worsens the working environment and increases costs.1–3 Although ED clinicians have little to say on the number of patients presenting to their department, making efficient and appropriate disposition decisions can help prevent crowding by improving patient flow.4–7
Predicting the outcome of an emergency visit, before a patient has had a full medical assessment, could help staff to direct patients to the medical service that best meets their needs; be it ED majors, a general practitioner (GP) surgery, an ambulatory care facility, a short stay unit, a minor injuries unit, an admissions ward or a specialty hospital bed.8 9 Diverting patients who do not require a full medical review in the emergency department to a more appropriate service could allow a more rational use of healthcare resources.
A number of disease-specific tools such as HEART, PESI and Blatchford are already used to identify patients who have a low probability of adverse outcomes and can therefore be considered for discharge or outpatient management.10–12 However, these tools can only be applied to the small proportion of all ED patients who have these well-defined presentations.
Triage is usually the first clinical assessment that a patient has after arrival to the ED, but several studies conclude that triage personnel are unable to accurately predict admission using clinical judgement alone.13–16 A variety of scoring systems have been applied at triage or in the prehospital setting to estimate the probability of admission using variables such as age, triage category and physiological early warning score.13–21 Some of these methods are more accurate than clinical judgement alone, but none has been widely adopted, perhaps because the simpler tools are not accurate enough to be clinically useful, and others are too complex for routine use.
The Glasgow Admission Prediction Score (GAPS) is an example of one such tool, which uses the data routinely collected in ED triage to predict a patient’s final disposition.13 It was derived from 322 000 unselected adult attendances to ED triage in Glasgow and has proved more accurate than nursing judgement in predicting ED discharge (table 1).14
The Glasgow Admission Prediction Score
Another example is the Amb Score, which the Royal College of Physicians recommends as part of its Ambulatory Emergency Care (AEC) toolkit.15 The Amb Score was developed by identifying the factors that differentiate the patients most likely to be discharged in under 12 hours from those who require a hospital stay of more than 48 hours.16 It was created from a cohort of 625 GP-referred emergency attendances with medical complaints in a mostly rural setting (table 2).
The Amb Score
Since both of these scores identify a clinical group with a high probability of discharge from the front door, either could in theory be used to highlight such patients early on in the patient journey. Ambs is currently recommended by the Royal College of Physicians and AEC Network for this purpose and is used by several UK sites.15 GAPS has also been used for this purpose at several UK sites, including Nottingham, the Royal Free Hospital, Torbay Hospital, Sheffield and Glasgow.
Since both GAPS and Ambs are predictors of disposition for unscheduled hospital attendances and both are in routine clinical use, we sought to determine which would better at discriminating between patients who require an inpatient stay and those who will be discharged by carrying out a prospective, multicentre observational study.
Methods
This was a prospective cohort study comparing the ability of two clinical scores to predict disposition decisions and was carried out at two large EDs in different regions of the UK.
Setting and participants
We collected data on all adult patients attending for ED triage at two large urban teaching hospitals in the UK: Glasgow Royal Infirmary and Northern General Hospital, Sheffield which have approximately 95 000 and 150 000 attendances per year, respectively.
Patients who were taken directly to the resuscitation room or to minor injuries without formal nurse triage were not included in the study. All children aged below 16 years of age were excluded. Any patient who left the ED before treatment was complete was also excluded from the main analysis.
The data were collected on consecutive patients at each site in 21 scheduled 8 hour blocks; these were arranged in such a way that every hour of each day of the week was represented once for each site in the sample. The data were collected between 8 and 17 February 2016 in Sheffield and between 5 and 23 May 2016 in Glasgow.
Sample size
Our sample size was estimated using a database of past ED triage attendances at the Glasgow site with known GAPS and approximate Amb Scores. We knew from a previous study that GAPS’ receiving operator characteristic (ROC) would have an area under the curve (AUC) of about 0.85.14 Our database suggested that the correlation coefficient of GAPS and Ambs was approximately −0.4, with the ratio of discharges to admissions in our sample being close to 1.5. From these parameters, we calculated that to have a 95% power to detect a clinically important difference of 0.05 between the AUC of GAPS and Ambs, with statistical significance at 95%, we required a sample of 1428 attendances across the two sites.17 We anticipated that we would slightly exceed this sample size and eliminate diurnal and weekday variation by collecting one full weeks’ worth of data at each site.
Data collection
Data collection was designed to equally sample all time periods of the week totalling 168 hours. Collection periods were arranged in shifts, with a single researcher collecting data on consecutive patients at triage during each shift. The GAPS was calculated using data collected from the standard triage process at both centres.
In addition, to help generate the Amb Score, each patient was asked by the researcher whether they had access to private or public transport, and the triage nurse was asked by the researcher if they felt that intravenous therapy was likely to be required during this presentation. In cases where the nurse was unsure about the need for intravenous therapy, the patient was followed up within the ED to determine whether intravenous therapy was felt necessary by the examining doctor. However, since the aim was to predict admission from triage, if the triage nurse was relatively certain of the need (or lack of need) for intravenous therapy, this was not followed up. The other variables comprising each score were objective and available for each patient at the point of triage.
After triage, each patient saw an assessing clinician as normal. The clinicians, who subsequently made the disposition decisions, were blinded to both scores. Patients who were admitted from the ED were followed up to hospital discharge to determine their length of hospital stay (table 3). Any patients who died in hospital or transferred to another hospital were considered not to have been discharged for the purposes of this study.
Demographics of Glasgow and Sheffield patients
Statistical analysis
All statistical analysis was carried out using R V.3.2.2.18. We constructed two ROC curves to show the ability of GAPS and Ambs to discriminate ED disposition, categorised as hospital admission or ED discharge. The areas under the ROC curves were compared using Delong’s method.19
To make the fairest comparison, we also tested the criteria that were used in the development of Ambs, by comparing the two scores’ ability to predict hospital lengths of stay less than 12 hours and those greater than 48 hours.
The three sets of binary outcomes against which the scores were tested were therefore:
Admission from ED versus discharge from ED.
Discharge from hospital more than 48 hours after presentation versus less than 48 hours.
Discharge from hospital more than 48 hours after presentation versus those discharged in less than 12 hours, retrospectively excluding those who stayed 12–48 hours (as per original Ambs paper).
Results
One thousand four hundred and eighty seven adult patients attended for triage during the study; the sample population comprising 686 patients in Sheffield and 801 in Glasgow. There were 63 patients who left before treatment was complete and who were therefore excluded from analysis, leaving a sample of 1424 attendances. Of these, 567 (39.8%) were admitted. Four patients who were admitted were subsequently lost to follow-up, so that their hospital length of stay was unknown. (figure 1).
Flow chart for inclusion and exclusion, with binary results included at optimum cut-offs. Ambs, Ambulatory Score; GAPS, Glasgow Admission Prediction Score.
The AUC for predicting admission was 0.807 (95% CI 0.785 to 0.830) for GAPS, compared with 0.743 (95% CI 0.717 to 0.769) for Ambs, P<0.00001 (figure 2). The cut-offs that maximised sensitivity+specificity were GAPS <17 and Ambs >5.
Comparison of receiver-operating characteristic curves for Glasgow Admission Prediction Score (GAPS) and Ambulatory Score (Ambs) in predicting hospital admission.
There was no significant difference between the two sites with regard to the AUC of GAPS (0.800 in Glasgow vs 0.817 in Sheffield, P=0.47) or the AUC of Ambs (0.724 in Glasgow vs 0.764 in Sheffield, P=0.135). Conversely, within each site, there was a significant difference between the two scores, both in Glasgow (0.800 for GAPS vs 0.724 for Ambs, P=0.0003) and in Sheffield (0.817 for GAPS vs 0.764 for Ambs, P=0.008).
A similar picture was seen in ability to predict discharge from hospital within 48 hours. Discounting the four patients whose length of stay was unknown, GAPS had an AUC of 0.813 (95% CI 0.789 to 0.837) and Ambs an AUC of 0.738 (95% CI 0.709 to 0.767), P<0.00001.
The pattern remained after removing the difficult middle-ground patients with a length of stay between 12 and 48 hours, as was done in the original Ambs study. In this case, GAPS had an AUC of 0.841 (95% CI 0.818 to 0.864) and Ambs’ AUC was 0.769 (95% CI 0.737 to 0.795), P<0.00001 (figure 3). The cut-offs that maximised sensitivity+specificity for this prediction were GAPS <18 and Ambs >5.
Comparison of receiver-operating characteristic curves for Glasgow Admission Prediction Score (GAPS) and Ambulatory Score (Ambs) in the ability to discriminate those with length of stay below 12 hours and above 48 hours.
As a binary test, using the thresholds which gave the highest percentage of correct classifications for each score, the overall accuracy for GAPS at the optimum threshold of <20 was 1057/1424 (74.2%, 95% CI 71.9% to 76.5%), whereas for Ambs the optimum cut-off of >5 correctly predicted 1004/1424 (70.5%, 95% CI 68.1% to 72.9%), P=0.012 by McNemar’s test.20
Discussion
These results should be put in context. Although the figure of 39.9% seems like a high admission rate for the general ED population, this figure is only for the subset of patients who attended formal triage. Many patients were streamed directly to minor injuries by clerical staff without being included, though a much smaller number would also have been taken directly to the resuscitation room without triage. This fact is also very likely to have lowered the AUC of both scores substantially, as it removes the most easily categorised presentations, leaving only those in whom decisions are more difficult (and hence a predictive tool is more useful).
In the original development of these scores, there were substantial differences in the sample populations. GAPS was derived from data collected from more than 215 000 unselected emergency attendances in an urban setting and validated in 107 000 separate attendances.13 The sample sizes were a lot smaller for the Amb Score: data from 282 patients were used to derive the score and the sample size for validation of the score was 343.16 The setting was mostly rural. Neither score was developed specifically for the general ED population, since Ambs was derived from data on GP-referred medical patients and GAPS also included these patients along with ED and minor injury unit data.
On comparison of the two scores, it was found that GAPS is a better discriminator of ED disposition and need for inpatient care than Ambs. Since Ambs also requires additional questions to be asked that are not part of routine triage, we would argue that GAPS is the more useful of the two tools in the general ED population.
Limitations
A potential weakness in the derivation of both GAPS and Amb Score was that they were each carried out in a single geographical region. A subsequent validation study of the Amb Score in a different area found it to have a lower sensitivity (88%) and positive predictive value (39%) than in the original paper.21 While this study was conducted in two centres; they are both tertiary units in the UK with similar resources. Data collection only occurred during a single episode in each centre and does not account for seasonal variation in presentations or admission rates. The simplicity of GAPS when compared with machine learning or Artificial Intelligence solutions may limit its accuracy but does aid its implementation.22 While accuracy could be increased by focusing of specific disease groups, this would limit ease of use and integration with current systems.23 24
Conclusion
Our multicentre comparison has demonstrated the accuracy of GAPS as a predictor of patient admission in a different area of the UK with different patient and hospital resource characteristics. This is crucial when considering widespread adoption and its utility to clinicians in other centres. Since the Manchester Triaging System is in use in sites outside the UK and the National Early Warning Score comprises basic physiological measurements, GAPS may still be applicable in countries with different healthcare models.
This study demonstrates that GAPS is a better predictor of disposition than the Amb Score in the general ED population. While current AEC guidelines suggest the Amb Score as a tool to aid streaming of patients to AEC, we believe that GAPS is a more accurate tool and can be applied to both acute medical and ED presentations. Integration of GAPS at triage does not require any extra time or effort on the part of the triage nurse and may enhance flow by predicting early demand for beds.
As with any such clinical tool, however, its purpose is to support rather than supplant human judgement; previous work has indicated that combining GAPS with a veto from triage nurses would provide an accurate strategy for bed management.14 We suggest that future work should focus on implementation of this approach and impact on ‘front door’ performance.
References
Footnotes
Contributors All authors contributed to the design of the study. EL and DJL collected and recorded the data. DJL and AC wrote the manuscript with significant input from all authors during each revision.
Funding DJ, CAOK and SMM were supported by NIHR CLAHRC YH Grant number IS-CLA-0113-10020.
Competing interests None declared.
Ethics approval The advice of the West of Scotland Research Ethics committee was sought and the chairman advised that this study should be considered part of service evaluation. Local Caldicott guardian approval was sought and given.
Provenance and peer review Not commissioned; externally peer reviewed.