Forecasting Emergency Department Crowding: A Discrete Event Simulation

doi:10.1016/j.annemergmed.2007.12.011

Annals of Emergency Medicine

Volume 52, Issue 2, August 2008, Pages 116-125

https://doi.org/10.1016/j.annemergmed.2007.12.011 Get rights and content

Study objective

To develop a discrete event simulation of emergency department (ED) patient flow for the purpose of forecasting near-future operating conditions and to validate the forecasts with several measures of ED crowding.

Methods

We developed a discrete event simulation of patient flow with evidence from the literature. Development was purely theoretical, whereas validation involved patient data from an academic ED. The model inputs and outputs, respectively, are 6-variable descriptions of every present and future patient in the ED. We validated the model by using a sliding-window design, ensuring separation of fitting and validation data in time series. We sampled consecutive 10-minute observations during 2006 (n=52,560). The outcome measures—all forecast 2, 4, 6, and 8 hours into the future from each observation—were the waiting count, waiting time, occupancy level, length of stay, boarding count, boarding time, and ambulance diversion. Forecasting performance was assessed with Pearson's correlation, residual summary statistics, and area under the receiver operating characteristic curve.

Results

The correlations between crowding forecasts and actual outcomes started high and decreased gradually up to 8 hours into the future (lowest Pearson's r for waiting count=0.56; waiting time=0.49; occupancy level=0.78; length of stay=0.86; boarding count=0.79; boarding time=0.80). The residual means were unbiased for all outcomes except the boarding time. The discriminatory power for ambulance diversion remained consistently high up to 8 hours into the future (lowest area under the receiver operating characteristic curve=0.86).

Conclusion

By modeling patient flow, rather than operational summary variables, our simulation forecasts several measures of near-future ED crowding, with various degrees of good performance.

Introduction

The Institute of Medicine recently noted that emergency department (ED) crowding represents an obstacle to the safe and timely delivery of health care.1, 2 Previous research has linked ED crowding with adverse patient outcomes,3, 4, 5, 6, 7 impaired access to care,8, 9, 10, 11, 12, 13, 14, 15 and decreased profitability.16, 17, 18

A substantial body of literature has focused on techniques for measuring the phenomenon of ED crowding, with the intent of allowing care providers, administrators, and policymakers to better manage the problem.19, 20, 21, 22, 23 At least 2 major problems are associated with measuring ED crowding. First, the lack of a standard crowding definition makes it difficult for unified progress to be made because different interpretations exist for what the term “crowding” should imply. A recent editorial emphasized the need for measuring ED patient flow, rather than measuring crowding itself.²⁴

Second, proposed measures of ED crowding have tended to focus on the present crowding state, and reports of forecasting the future crowding state have been relatively recent.25, 26, 27, 28, 29, 30 Predictions of the near-future status of the ED would arguably have substantial value because they could trigger early interventions designed to lessen the burden of crowding situations before they arise.31, 32, 33, 34, 35 A focus on forecasting the future, in addition to monitoring the present, may represent the difference between being reactive and being proactive in managing ED crowding.

Both of the above research gaps may be addressed by using a novel application of computer simulation in the ED. We attempt to demonstrate that, with a sufficiently detailed simulation of ED patient flow, near-future forecasts of almost any crowding measurement of interest may be obtained from a single model. The feasibility of developing an ED simulation model has already been well established.36, 37, 38, 39, 40, 41, 42, 43, 44 Previous studies have focused on using simulation to evaluate the effect of hypothetical changes in ED operations. However, to the best of our knowledge, no previous studies have explored the ability of an ED simulation to serve as a generalized, real-time forecasting model.

The first goal of this study was to develop a computer simulation for the specific purpose of real-time forecasting of ED operating conditions. The second goal was to validate the ability of the simulation to forecast several measures of ED crowding.

Section snippets

Theoretical Model of the Problem

Our study was based on the following premise: With a simulation model that expresses ED crowding in terms of individual patients and their characteristics, perhaps we could forecast any outcome measure of interest. The conceptual process of obtaining forecasts from an ED simulation is outlined in Figure 1. The model would consist of a set of theoretical distributions governing patient flow, with parameters calculated from historical patient data. The model would be initialized with a detailed

Results

A total of 57,995 patients visited the adult ED during the study period, of which 4,776 patients were excluded (8.2%). The medians and interquartile ranges for each continuous outcome measure during the 2006 calendar year were the following: waiting count=2 patients (1 to 6), waiting time=13 minutes (0 to 50), occupancy level=83% (63 to 95), length of stay=6.4 hours (4.4 to 9.1), boarding count=10 patients (5 to 16), and boarding time=6.8 hours (3.1 to 10.5). A total of 188 ambulance diversion

Limitations

One potential limitation of this study is the narrow purpose for which the ForecastED simulation was intended. We developed and validated it for the sole purpose of forecasting near-future operational measures in the ED. We intentionally kept the purpose narrow because an effort to create an all-purpose simulation of ED patient flow might have compromised one or more of our design goals. Its use for other common applications of simulation, such as evaluating long-term effects of proposed

Discussion

We have designed and implemented ForecastED, a discrete event simulation that uses patient flow to predict near-future ED operational measures. The findings indicate that the distributions used to represent the model's random processes closely fit the observed data. The simulation forecasts correlated well with the actual operational measures at 2, 4, 6, and 8 hours in the future. This correlation equaled or exceeded the inherent autocorrelation of the data across all outcome measures and

References (61)

M. Eckstein et al.
The effect of emergency department crowding on paramedic ambulance availability
Ann Emerg Med
(2004)
M.J. Schull et al.
Emergency department crowding and thrombolysis delays in acute myocardial infarction
Ann Emerg Med
(2004)
P. Krochmal et al.
Increased health care costs associated with ED overcrowding
Am J Emerg Med
(1994)
M.D. Bayley et al.
The financial burden of emergency department congestion and hospital crowding for chest pain patients awaiting admission
Ann Emerg Med
(2005)
N.R. Hoot et al.
Measuring and forecasting emergency department crowding in real time
Ann Emerg Med
(2007)
A.J. Forster
An agenda for reducing emergency department crowding
Ann Emerg Med
(2005)
C.E. Saunders et al.
Modeling emergency department operations using advanced computer simulation systems
Ann Emerg Med
(1989)
J. Huddy et al.
Computer simulation: making better operational and architectural ED design decisions
J Emerg Nurs
(1999)
M.I. Edmonds et al.
The use of computer simulation as a strategic decision-making tool: a case study of an emergency department application
Healthc Manage Forum
(1999)
B.R. Asplin et al.
A conceptual model of emergency department crowding
Ann Emerg Med
(2003)

The overcrowded emergency department: a comparison of staff perceptions

Acad Emerg Med

(2003)

Cited by (184)

An explainable machine learning approach for hospital emergency department visits forecasting using continuous training and multi-model regression
2024, Computer Methods and Programs in Biomedicine
In the last years, the Emergency Department (ED) has become an important source of admissions for hospitals. Since late 90s, the number of ED visits has been steadily increasing, and since Covid19 pandemic this trend has been much stronger. Accurate prediction of ED visits, even for moderate forecasting time-horizons, can definitively improve operational efficiency, quality of care, and patient outcomes in hospitals.
In this paper we propose two different interpretable approaches, based on Machine Learning algorithms, to accurately forecast hospital emergency visits. The proposed approaches involve a first step of data segmentation based on two different criteria, depending on the approach considered: first, a threshold-based strategy is adopted, where data is divided depending on the value of specific predictor variables. In a second approach, a cluster-based ensemble learning is proposed, in such a way that a clustering algorithm is applied to the training dataset, and ML models are then trained for each cluster.
The two proposed methodologies have been evaluated in real data from two hospital ED visits datasets in Spain. We have shown that the proposed approaches are able to obtain accurate ED visits forecasting, in short-term and also long-term prediction time-horizons up to one week, improving the efficiency of alternative prediction methods for this problem.
The proposed forecasting approaches have a strong emphasis on providing explainability to the problem. An analysis on which variables govern the problem and are pivotal for obtaining accurate predictions is finally carried out and included in the discussion of the paper.
An integrated approach of discrete event simulation and a non-radial super efficiency data envelopment analysis for performance evaluation of an emergency department
2023, Expert Systems with Applications
An emergency department (ED) has been considered as one of the most congested department in a hospital. The congestion is typically contributed by long waiting time to get medical service, primarily caused by insufficient resource allocation or improper resource configurations. However, how various resource allocation configurations affect ED performance and how their efficiency can correctly be evaluated, using an integrated approach is rarely discussed. This paper thus integrates discrete event simulation (DES) and data envelopment analysis (DEA) to measure ED performance and evaluate the efficiency of potential resource allocation configurations for future performance improvement. For this, a DES model for an ED is first designed and developed. Its performance improvement is then tested using 35 potential resource allocation configurations, and their impacts on the performance are measured. To evaluate their efficiency and identify the optimal configuration, a mixed integer super efficiency of slacks-based measure data envelopment analysis (SE-SBM-DEA) approach dealing with undesirable outputs is proposed. The model utilizes resource allocation as inputs and simulation performance measures as outputs. All inputs were considered as integer, while the outputs were classified into desirable and undesirable in mixed integer-valued data. The desirable real outputs are the utilization of receptionists, nurses and doctors. The undesirable real outputs are the average of patient cycle time and time spent in queues, while the undesirable integer output is average number of patients in queues. The desirable integer output is average number of patients received treatment. The results obtained from the DEA approach show that 21 efficient resource configurations have the capability to increase their inputs, undesirable outputs and/or decrease desirable outputs simultaneously without affecting their efficiency status. The integrated approach helps decision makers manage their healthcare facilities by identifying the sources of inefficiency and (the maximum levels of inputs-undesirable outputs and minimum levels of desirable outputs) to improve and (retain) efficiency.
Dynamic scheduling of patients in emergency departments
2023, European Journal of Operational Research
Emergency department overcrowding is a global issue that poses a great threat to patient health and safety. The timeliness of medical services provided to patients is crucial to emergency departments as it directly impacts the mortality and morbidity of urgent patients. However, critical resources (e.g., doctors and nurses) are typically constrained due to the limited financial budget. Thus, hospital administrators may need to investigate solutions to improve the efficiency of the emergency department. In this work, we study the dynamic problem of scheduling patients to doctors, aiming at minimizing the total weighted tardiness. We propose a simple reoptimization heuristic based on multiple queues of patients in accordance with their urgency levels, and then combine it with an effective variable neighborhood search. We also propose a scenario-based planning approach that uses sampled scenarios to anticipate future events and the variable neighborhood search to schedule patients. The methods are adapted to handle a problem variant where information on arrival time and urgency level of some patients can be received in advance by the emergency department. With a comprehensive computational study on two sets of realistic instances from Hong Kong SAR of China and Italy, we validate the performance of the proposed methods, evaluating the benefits of having more doctors and receiving early information.
A framework for developing generalisable discrete event simulation models of hospital emergency departments
2022, European Journal of Operational Research
Discrete event simulation (DES) is routinely used to model hospital emergency departments (EDs), primarily due to its ability to represent complex patient flow processes and investigate improvement strategies. Despite this, it is clear from published studies that many DES models are not subsequently implemented in hospitals or reused for other sites. This research addresses a gap in the literature by presenting a new data-driven modelling framework ‘GE-DES’, which outlines an approach to the design and development of generalisable ED models. The nature of the framework means that it is sufficiently flexible (i) for use across multiple EDs, and (ii) for investigating hospital-specific problems through data-driven customisation. The primary aim of GE-DES is to support model reuse and implementation. The framework is demonstrated through application to a case study ED in Australia.
Discrete event simulation for emergency department modelling: A systematic review of validation methods
2022, Operations Research for Health Care
Discrete event simulation has been widely used for decades to model the Emergency Department (ED) patient flow, understand system bottlenecks, and analyse resource capacity planning constraints. Given the complexity and constraints related to modelling the ED setting, such as data availability, there is often a gap in prescribing the correct validation approach.
The purpose of this review study is to come up with practical guidelines for employing specific validation techniques by comparing the available “best practice” simulation validation approaches against the approaches commonly found in published ED Discrete Event Simulation (DES) studies.
We conducted a systematic review of the peer-reviewed literature to identify DES studies of patient flow within hospital EDs across the globe. Our search strategy focused on two main domains of knowledge associated with the current study: DES and ED. In total, we selected 90 studies a basis for the analysis. Additionally, we have identified a total of 7 papers focused on best practice approaches in validation.
A plurality of studies only discuss a single type of validation (data-led) at 30%, closely followed by none (23%), and data-led paired with qualitative validation, but no verification (at 22%). LOS is the most common validation metric, with 47% of studies that used data-led validation selecting length of stay (LOS) as the key validation metric. The next most frequently validated metric is throughput (9%), followed by time to triage (TTT) (8%). “Confidence Interval” and “% Difference” are by far the most common, with 32% of studies employing the former, and 22% employing the latter. Remaining techniques tend to be used more sporadically, with hypothesis testing, correlation coefficient, Student t-test, and Welch’s two-sample t-test being the most frequent (5% to 9% of studies).
Based on the reviewed studies, we propose guidelines for the validation procedure given five “levels” of available data quality. The guideline incorporates both the analysis of best practice literature, as well as the trends for validation based on the review of 90 generic and specific simulation studies.
Analyzing the Variations in Emergency Department Boarding and Testing the Transferability of Forecasting Models across COVID-19 Pandemic Waves in Hong Kong: Hybrid CNN-LSTM approach to quantifying building-level socioecological risk
2024, arXiv

View all citing articles on Scopus

Supervising editor: David J. Magid, MD, MPH

Author contributions: NH and DA conceived the study. All authors contributed substantially to the study design. IJ and DA obtained research funding. NH implemented the software and collected the data. NH and CZ performed the statistical analysis. NH drafted the article, and all authors contributed substantially to its revision. NH takes responsibility for the paper as a whole.

Funding and support: By Annals policy, all authors are required to disclose any and all commercial, financial, and other relationships in any way related to the subject of this article, that may create any potential conflict of interest. See the Manuscript Submission Agreement in this issue for examples of specific conflicts covered by this statement. Dr. Hoot was supported by the National Library of Medicine grant LM07450-02 and National Institute of General Medical Studies grant T32 GM07347. The research was also supported by the National Library of Medicine grant R21 LM009002-01. This project is an academic endeavor. It was supported by federal funding, as noted above, and there was no corporate funding. There are no current plans to commercialize this research that would cause any conflicts of interest.

Publication dates: Available online April 30, 2008.

Reprints not available from the authors.

View full text

Health policy and clinical practice/original researchForecasting Emergency Department Crowding: A Discrete Event Simulation

Study objective

Methods

Results

Conclusion

Introduction

Section snippets

Theoretical Model of the Problem

Results

Limitations

Discussion

Ann Emerg Med

Ann Emerg Med

Am J Emerg Med

Ann Emerg Med

Ann Emerg Med

Ann Emerg Med

Ann Emerg Med

J Emerg Nurs

Healthc Manage Forum

Ann Emerg Med

Ann Emerg Med

Ann Emerg Med

Ann Emerg Med

Ann Emerg Med

Hospital-based Emergency Care: At the Breaking Point

Crisis in the emergency department

N Engl J Med

Decreased health care quality associated with emergency department overcrowding

Eur J Emerg Med

Emergency department diversion and trauma mortality: evidence from Houston, Texas

J Trauma

The effect of emergency department crowding on the management of pain in older adults with hip fracture

J Am Geriatr Soc

The association between hospital overcrowding and mortality among patients admitted via Western Australian emergency departments

Med J Aust

Increase in patient mortality at 10 days associated with emergency department overcrowding

Med J Aust

Patients who leave a public hospital emergency department without being seen by a physicianCauses and consequences

JAMA

Emergency department overcrowding and ambulance transport delays for patients with chest pain

CMAJ

Impact of critical bed status on emergency department patient flow and overcrowding

Acad Emerg Med

Emergency department gridlock and out-of-hospital delays for cardiac patients

Acad Emerg Med

Factors associated with patients who leave without being seen

Acad Emerg Med

Characteristics of patients who leave emergency departments without being seen

Acad Emerg Med

The financial impact of ambulance diversions and patient elopements

Acad Emerg Med

Development and validation of a new index to measure emergency department crowding

Acad Emerg Med

The overcrowded emergency department: a comparison of staff perceptions

Acad Emerg Med

Health policy and clinical practice/original research
Forecasting Emergency Department Crowding: A Discrete Event Simulation