Article Text
PDF
Abstract
Objective This study aims to validate previously reported triage tool titled Sydney Triage to Admission Risk Tool (START+) and investigate whether an extended version of the tool could be used to identify and stream appropriate short stay admissions to ED observation units or specialised short stay inpatient wards.
Methods This was a prospective study at two metropolitan EDs in Sydney, Australia. Consecutive triage encounters were observed by a trained researcher and START scores calculated. The primary outcome was length of stay <48 hours. Multivariable logistic regression was used to estimate area under curve of receiver operator characteristic (AUROC) for START scores. The original START tool was then extended to include frailty and multiple or major comorbidities as additional variables to assess for further predictive accuracy.
Results There were 894 patients analysed during the study period. Of the 894 patients, there were 732 patients who were either discharged from ED or admitted for <2 days. The AUROC for the original START+ tool was 0.80 (95% CI 0.77 to 0.83). The presence of frailty was found to add a further five points and multiple comorbidities added another four points on top of the START score, and the AUROC for the extended START score 0.84 (95% CI 0.81 to 0.88).
Conclusion The overall performance of the extended ED disposition prediction tool that included frailty and multiple medical comorbidities significantly improved the ability of the START tool to identify patients likely to be discharged from ED or require short stay admission <2 days.
Trial registration number ACTRN12618000426280
- triage
- emergency departments
- emergency care systems
- efficiency
- assessment
Statistics from Altmetric.com
Key messages
What is already known on this subject
Evidence for the use of ED prediction tools is lacking despite the growing pressure to improve ED performance and maintain state-wide targets that facilitate high standards of patient care.
A few studies have previously investigated the use of admission risk prediction tools in the context of ED care with varying levels of accuracy.
What this study adds
Our study builds on this research, showing that the final disposition of ED patients can be accurately predicted from point of ED arrival without the use of complex algorithms or additional staff resources.
More specifically, this triage prediction tool could be used to accurately predict short stay admissions and therefore improve patient flow and more appropriate use of emergency short stay units.
Introduction
Hospitals have experienced a disproportionate increase in ED presentations over the past decades1 2 leading to overcrowding, treatment delays, and adverse patient outcomes.3 Evidence suggests that ED performance and patient flow can be improved through rapid assessment and early decision making by senior ED clinicians.4–6 Such models of care include streaming of patients to appropriate clinical locations based on complexity and likely disposition.7 Short stay observation units in ED are models of care for assessing and managing patients with conditions that are likely to require a short period of observation prior to discharge, usually between 24 and 48 hours.8–10 ED short stay observation units across New South Wales (NSW) generally require patients to be admitted to an inpatient unit or discharged within 48 hours of ED arrival.9 A tool that differentiates patients based on likely disposition and length of stay would assist in patient flow by allowing such patients to be streamed directly to an observation unit, medical assessment unit or fast track area for clinical assessment and management. This may help free-up ED beds for patients that require more extensive workup and minimise the effects of overcrowding and treatment delays in ED.
Clinical prediction tools represent a potentially novel way to assist early senior decision making and streaming of patients to these units.11 Previous studies have investigated predicting admission to medical short stay units with promising results.8 10 12 13 Powter et al 10 used a model that predicted length of stay <72 hours with an area under the curve (AUC) of 0.68. Variables included age >80 years, cognitive impairment and multiple medications on admission.10 In another single-centre study of 704 patients, admission disposition was correctly predicted by ED clinicians 71%–85% of the time while the accuracy of predicting inpatient length of stay >3 days varied between 50% and 56%.12
Using NSW state-wide linked ED data, we have previously derived and internally validated a prediction tool for ED disposition called the Sydney Triage Admission Risk Tool (START).11 The triage tool used age, mode of arrival, triage category, presenting problem type and time of ED arrival to derive a risk score that correlated with a patient’s likelihood of admission or discharge. The final multivariable model had an AUC of 0.82 (95% CI 0.81 to 0.82).
One aspect of the original START report that required clarification was the classification of observation unit patients as inpatient admissions. While administratively this made sense, from a clinical and patient flow perspective, such patients are important to differentiate from inpatient admissions because patients admitted to ED observation units are generally earmarked for expedited discharge. Longer stay admissions on the other hand generally need further assessments and management in a more traditional inpatient ward setting. Identifying such patients earlier may also identify opportunities for earlier intervention and resource management to reduce overall length of stay.
The purpose of the present study was to prospectively validate the original START model and investigate a extended version of the START tool to identify patients who could be discharged from ED or require a short stay admission of 2 days or less. This would allow the tool to be used at triage to facilitate streaming to ambulatory care or short stay observation units for patients requiring simple treatment or a short period of observation in ED, potentially enabling more efficient use of inpatient ward beds for more complex admissions.
Methods
Setting
This study was conducted between November 2016 and June 2017 at two urban EDs in Sydney. One was a tertiary level six ED with an annual census of 72 000 presentations. The second ED was a district level three ED with an annual census of 45 000 presentations.
Design
This was a prospective study using a convenience sample of ED patients. Validation was performed on the original START tool to predict admission versus discharge and an extended START tool derived using the same prospectively collected data to predict length of stay <2 days and discharge from ED versus longer stay admission.
Patient population
All patients over the age of 16 years who presented to the ED during allocated research days were included. Exclusion criteria included transfers from other hospitals, planned admissions, those brought in by police or immediately life-threatening presentations such as cardiac arrests, trauma or stroke calls. These were apparent in advance of the triage assessment.
START tool
The START prediction tool was developed using NSW state-wide ED data from 2010 to 2014. Risk scores were derived using logistic regression and internally validated using bootstrap cross validation. Risk scores derived using these methods are calculated by adding risk scores shown in table 1 (range −6 to +46), which correlates with a probability of requiring inpatient admission.
Original START tool calculation
Data collection
Baseline data were collected prospectively at the point of triage by the research nurse observing the triage encounter and noting variables including triage category, presenting problem, patient frailty, decreased mobility, multiple or major comorbidities, general practitioner (GP) referral and mode of arrival. Patient demographics such as age and gender were also included in the START score, however, these are routinely collected by admin staff prior to triage. Frailty was defined as a clinical state of age-related vulnerability, severe weakness, reduced mobility and decreased physical function that impacts on daily living.14 The additional variables were all assessed, calculated cumulatively and recorded by the research nurse using a standard data collection form (table 1). A patient who presented to ED as either being bed-bound or unable to mobilise independently, were scored as having reduced mobility. Acute single limb injuries were not included in this category as such injuries were considered to not impact a patient’s mobility greatly. The additional variable of GP referral was included if the patient had a referral letter with them at triage, or had been told to present to ED by their primary healthcare physician.
Actual patient disposition, admitting medical officer and hospital length of stay was then followed up prospectively using the electronic medical record (FirstNet, Cerner Millennium) on the day after triage encounter. Presenting problem categories were defined using the original derivation data classification.2 Data were collected on consecutive patients during allocated research hours (Monday to Friday between 10:00 and 21:00) by a trained research nurse who observed each triage and who was unaware of START score calculations or the eventual disposition of the patient. An independent investigator, blinded to the disposition, calculated START scores, using a previously described algorithm.11 The research nurse took less than a minute to calculate the score and occurred simultaneously to the triage encounter, so that the START score was available for immediate implementation and use.
Outcomes
The primary outcome was patient disposition, which was a binary outcome comprising short stay admission (length of stay of 2 days or less including discharge from ED), versus longer stay inpatient admission (length of stay >2 days).
The START score was the arithmetic sum of the predictive variables as summarised in table 1.
Receiver operator characteristic curve for original START model (model 1 dotted red line) and extended START model (blue line).
Statistical analysis
Validation of original START score
The relationship between the extended START score and the primary outcome (short stay admission or discharge) was analysed using logistic regression with model discrimination assessed using area under receiver operator characteristic curve (AUROC) and Hosmer-Lemeshow test statistic for model calibration and positive predictive values used to assess accuracy.
Derivation of extended START score
Chi-squared tests were used to compare the group of patients with and without the primary outcome (table 2). The extended model was assessed using the calculated START score as the primary predictor together with markers of complexity (frailty, multiple comorbidities and GP referral), and risk scores for each variable derived by dividing the variable coefficient with the base coefficient of the START score derived from regression analysis. The extended model was then compared with the original START score-only model as a predictor of any inpatient admission as originally reported and a sensitivity analysis performed using the range of coefficients derived from logistic regression. The analysis was performed on SAS V.9.4 (SAS Institute, Cary, North Carolina, USA). As this was a derivation study, no sample size calculation was performed.
Comparison of baseline characteristics of patients who were discharged from ED or short stay admission vs those requiring longer stay admission (>2 days)
Results
There were 894 patients analysed during the study period. The mean age was 49.0 (SD 20.6) years (median age 46 years, IQR 31–66), and 460 (51.5%) were female. Of the 894 patients, there were 732 patients who were either discharged from ED or admitted for <2 days. Baseline characteristics of those who had short length of stay (including ED discharge) were compared with those in the long length of stay group in table 2.
Validation of original START score
The START score when used to predict any inpatient admission had AUROC of 0.80 (95% CI 0.77 to 0.83) with a Hosmer and Lemeshow goodness-of-fit test of 12.35 and a p value of 0.09. The multivariable logistic regression results for predicting short stay admission including ED discharge using the extended START is shown in table 3. The AUROC for this model was 0.84 (95% CI of 0.81 to 0.88).
Multivariable logistic regression model for extended START score to predict short stay admission or ED discharge vs long stay admission
Extended START score
Table three indicates that frailty and multiple or major comorbidities were also significant risks for admission to a short stay area. The presence of frailty was found to add a further five points and multiple or major comorbidities added another four points on top of the START score. With these variables included in the risk score, the positive predictive values of the extended model are shown in table 4. With low risk scores (<10) there were only nine patients (2.5%) who were misclassified as long stay admissions (table 4). When the original START and extended START models were compared, the extended START model for predicting short stay admission had a significantly higher AUC ROC (p0.035). Figure 1 graphically compares ROC curves between the extended START score to predict short stay admission or discharge compared with the original START score to predict any inpatient admission.
Prediction of short stay admission at various extended START score ranges
Sensitivity analysis
Using the range of risk scores for frailty the AUROC did not change (risk score=2: 0.84 (95% CI 0.80 to 0.89) and risk score=8: 0.84 (95% CI 0.81 to 0.87)). For risk score ranges of multiple medical comorbidities, the AUROC were (risk score=1: 0.83 (95% CI 0.80 to 0.86) and risk score=6: 0.84 (95%CI 0.81 to 0.87)).
Discussion
The present study was undertaken to prospectively validate an ED disposition support tool based on triage information and to refine this tool to account for patients being streamed to short stay observation units in ED. This was made possible by prospective data collection including length of stay in hospital and the clinical location of the patient throughout their hospital journey. In doing so, we aimed to investigate whether an extended version of the prediction tool could be used at triage to stream appropriate patients to observation in ED followed by discharge, or admitted to a specialised short stay inpatient ward such as a medical assessment unit (MAU). The results of the study suggest that the performance of the original START score was similar to the derivation study, which validates the tool in this clinical context and provides a basis for which it can be further implemented and studied. The results also indicate that the tool may be used for identifying and streaming patients who are likely to be discharged or require only a short hospital stay.
Implicit in the performance of short stay and ED observation units are appropriate selection of patients and ability to accurately predict likely length of stay.12 A study by McNeill et al 8 surveyed 50 hospitals in Australia to determine the design and operational characteristics of acute MAUs, and found that MAUs were a promising model of care to help reduce ED overcrowding and access block. However, the authors also found that there were issues with MAUs, including delays in transferring patients from ED to MAUs and difficulties in making accurate disposition decisions early on in the patient journey.8 As such, when based on clinician prediction alone, evidence suggests that predicting short-term admissions is often inaccurate.12 Another study conducted a retrospective audit of 6703 patients admitted to an ED Short Stay Unit (EDSSU) to determine predictive factors for ED observation unit patients who eventually convert to a formal inpatient admission.6 This study found that age >70 years was associated with an increased risk of these discharge ‘failures’; however, time of admission to the EDSSU showed no significant change in failure rates.6 This was an important finding that was taken into account in our extended START. As seen in figure 1, increasing age was a positive indicator of likelihood of longer length of admission stays with age >60 and 85 providing an additional 6 and 9 points, respectively.
Our study validated the original START prediction model and also demonstrated that the inclusion of frailty and multiple medical comorbidities, improved the ability of the START score to predict short stay admission or discharge. When these two variables were included, the ability of an extended START tool to predict very likely or likely short stay or ED discharge (score of <10) had an accuracy of 97.5%. This meant that just 9 out of 304 patients who scored <10 on their START score were misclassified and ended up requiring inpatient care for >2 days. In the context of this study, the results suggest that the extended START score may be potentially used to stream patients reliably to short stay observation units or other clinical areas in ED that can facilitate early and safe discharge planning.
From a practical point of view, the extended START tool could be incorporated into the triage process to identify patients who could be quickly streamed either to an ambulatory care unit within ED to treat minor conditions, or to a short stay observation unit. Although these are two distinct clinical units within the ED and cater for different types of patients, these can be easily distinguished at the point of triage.6 The possible benefits of this approach, as opposed to simply predicting admission alone would be improved utilisation of ED observation units and reduced risk of observation unit ‘failures’.6 15 As the START score variables are all collected on registration or during triage, the aim would be for the score to be automated and incorporated into electronic medical records (EMR) so that additional resources or staff are not required. Patient who are likely to require admission based on the START score and have evidence of frailty or multiple comorbidities who do not score a sufficiently lower START score could then be worked up for longer stay inpatient admission units.
Limitations
This study had several acknowledged limitations. The main limitations were the size of the study and that the data came from just two hospitals within the same local health district. Larger studies are needed to validate this extended START tool across multiple different ED settings. Due to resource constraints, another limitation was that our study data were mostly collected during business hours. However, the START variable of ‘time of arrival’ was found to only change the scores by one point in the original derivation study so we believe that this would not have impacted the overall performance of the model. Another limitation was that the inclusion of frailty and multiple comorbidities into the extended START score may prolong assessment times during ED triage and may not be feasible. Anecdotally, there was no disruption to normal triage processing times or triage time performance, but again this requires formal evaluation. Finally, we have not compared the tool against senior clinician decision making and this would be an interesting comparative effectiveness study. A previous study14 reported comparable disposition accuracies when using senior ED clinicians.
In conclusion, we found that an extended ED disposition prediction tool that included frailty and multiple medical comorbidities identified at triage significantly improved the ability of the START tool to identify patients likely to be discharged from ED or require short stay or observation unit admission. Further studies are planned to validate the extended START tool and test the effectiveness of the prediction tool in the clinical context.
Acknowledgments
The authors acknowledge Claire Hutchinson and Glen Wiseman from The Canterbury Hospital as well as triage nurses from both hospital Emergency Departments for their assistance and support of this study.
Footnotes
Contributors All authors cited contributed to the development of the START+ idea and writing of the manuscript. AE-W performed most of the data collection while KJB and MMD did most of the data analytics.
Funding This project was funded by the NSW Agency for Clinical Innovation and the Emergency Care Institute (reference number ACI/D14/2288).
Competing interests None declared.
Patient consent Not required.
Ethics approval Sydney Local Health District Research Ethics Committee (RPAH Zone).
Provenance and peer review Not commissioned; externally peer reviewed.