Article Text
Abstract
Many computer simulation models of emergency departments have been developed to aid clinicians and managers to maintain and improve the performance of their departments. A model is presented that can also be used to understand changes in performance that may occur as a result of the 4-hour target regime in the English NHS. The model simulates the performance resulting from normal activity, and the differences between this and actual performance are revealing. The results from two departments are presented to demonstrate this mode of model use. These reveal the extent of special action taken in some emergency departments as patients approach the target time, and also show the true underlying performance of the departments.
Statistics from Altmetric.com
Since 2002, emergency departments in English NHS hospitals have been required to meet a 4-hour target for the length of time taken to treat patients in the department. Under this target, 98% of patients arriving at an emergency department must either be discharged on completion of treatment or admitted as an inpatient for further treatment within 4 h. The target was introduced as part of the government’s drive to reduce unacceptable waiting times for hospital care. Evidence suggests that waiting times in emergency departments have reduced since the target was introduced, and most departments claim to process patients within 4 h.1 There remains, however, a suspicion that meeting the 4-hour target may have forced clinicians to cut corners at times of high demand, or may have encouraged managers to adopt some of the undesirable behavioural responses discussed by Smith2 and Bevan and Hood.3 This has been investigated at a macro level by Friedman and Kelman4 5 who concluded that no such gaming or corner cutting is evident.
Here we describe a generic simulation model of an emergency department designed to be used in one of two modes. First, it can be used to experiment with alternative configurations and staffing to see how this affects patient waiting times. Second, it can be used to understand historical behaviour and thereby to spot special action taken as a result of waiting time targets, which is the mode of use discussed here. We present the results from simulations of two English emergency departments to show how the model can be used to understand observed performance, even when a department apparently meets the 4-hour target. Goodhart’s law6 is a neat summary of the potentially distorting effect of using a performance indicator as a target. With this in mind, the model could be used by commissioners to understand service quality and by regulators to observe the effect of waiting time targets.
Simulation models of emergency departments are not particularly difficult to construct using modern software and several are presented in the literature.7 8 9 10 11 12 The one discussed here was developed as part of the DGHPSim project in which the operating processes of whole hospitals are modelled to assess the effects on waiting times of capacities and processing rates within the hospital. A more detailed description of the emergency department model has been given elsewhere,13 and more information about the DGHPSim project can be found at www.hospitalsimulation.info.
Overview of the simulation model
Figure 1 shows the broad conceptual model on which the simulation is based. Like all models it is a simplification, which is what makes the model useful.14 In this case, the simplifications aim to focus on tasks and processes that affect performance as measured by times spent by patients in the department. The model is implemented in the Micro Saint Sharp software (http://www.maad.com/index.pl/micro_saint), which is well suited to the simulation of systems that involve human processes.
The model is configurable—that is, its structure represents a typical emergency department and, using data appropriate to a particular department, can be used to simulate the activities and performance of that department. It does not attempt to capture actions taken in special circumstances such as a major road traffic accident, but represents the normal activities of the department. Likewise, it does not represent special interventions that are made when patients look likely to breach the 4-hour target. Hence, it represents the normal activities of an emergency department and could be use to help improve such a department. However, here we discuss its use to detect altered performance.
Processes represented in the model
Figure 1 shows that simulated patients are assumed to arrive at the department either by ambulance or as walk-in cases. On arrival, the latter will be registered and may be triaged, (we assume five-colour Manchester Triage15) and will then wait for treatment. In developing the conceptual model we observed that there are typically two treatment streams, even when a five-colour triage system is in use, and we label these as major and minor. Patients arriving by ambulance are assumed to be urgent and may be registered en route, although they too may have to wait for treatment. Once called from the waiting area, patients are modelled as occupying a cubicle and will participate in a process that may have three stages: initial treatment, tests and reassessment/treatment after tests. The cubicle is freed at the end of each of these stages for use by another patient. Doctors and nurses are required during initial treatment and reassessment/treatment, but not during the tests. Following reassessment/treatment, patients are either discharged or admitted as inpatients.
Time-dependent demand
The principal inputs to the simulation model are demand data, details of emergency department staffing and process times. It is well known that demand for the emergency department varies by time of day and by day of week (and, in some departments, by time of year), which is represented in the model by non-homogeneous Poisson processes. The model therefore displays the type of dynamic variability that is all too familiar in real life. To simulate historical behaviour the Poisson processes are constructed from records of actual arrivals at the emergency department being simulated. If the simulation were to be used to simulate changes in demand from current levels, the probability distributions can be modified appropriately. Thus, the demand side of the simulation is a representation of individual patients as they arrive for treatment at the department and this demand varies through the day and the week.
Staffing workloads and task switching
The simulation assumes continuous operations on a 24/7 basis and, to represent a particular department, must be parameterised with staffing levels that specify the number of experienced doctors, trainee doctors, nurses and clerical staff using appropriate shift patterns. A failing of most emergency department simulators described in the literature is their inability to represent task switching; that is, the well-known fact that each doctor and nurse is likely to be simultaneously responsible for more than one patient during busy periods.
The number of patients simultaneously treated has been studied empirically,16 17 and the latter includes a time and motion study in the USA in which clinicians were shadowed for a month, defining eight possible “tasks” including patient care and viewing diagnostic test results, and defining an “interruption” as any event that briefly required the attention of the subject but did not result in switching to a new task. If a subject switched from one task to another, the latter was defined as a “break-in task”. The data show that the mean (SD) number of patients managed simultaneously by experienced physicians is 5.1 (2.1), and that the number of break-in tasks increases during busy periods.
Since this task switching is a feature of emergency departments and is crucial to their operation, it is important that it be represented in a simulation model if the model is to give reasonable estimates of performance. The obvious way to do this is to find some way to record how long each interrupted task and break-in task takes. This is appealing but rather difficult in practice, especially during busy periods. An alternative representation, which was adopted in our model, is to include multiple representations of each clinician (eg, each doctor may be represented by several “mini docs”). In this way, the same clinician can be simulated as attending to several patients concurrently. Based on the findings of Chisholm et al,17 we fragment clinicians into 6 for experienced doctors, 4 for trainee doctors and 2 for nurses.
Two very different emergency departments
Locker and Mason18 analysed data from 83 English emergency departments and found that about one in eight admitted patients spent between 220 and 239 min in the department, which is a clear indication of the effect of the 4-hour target. However, such an analysis cannot reveal the underlying performance of a department (ie, how it would perform if no special actions were taken when a breach is imminent). It is important to know this, since it reflects the true underlying performance of a department.
To illustrate the use of this simulation model in understanding this true underlying performance of emergency departments, we present the results from two such departments. The data used are no longer current and so the performance of both departments will have changed. Figure 2 shows the performance of department A. The horizontal axis shows the total time spent by patients in the department from their arrival to their discharge or admission. The vertical axis shows the proportion of patients treated within those times. The solid line is the actual performance of the emergency department during this period of operation and the dotted line is the results of the simulation of the department during that same period.
The figure shows that, in this case, the simulated performance is very similar to the actual performance of the department. The main—albeit small—difference occurs around 240 min when the line of the actual performance briefly rises. Since 4 h is the emergency department target and the simulation takes no account of the target, it is reasonable to suppose that the looming breach point caused the department to find ways to quickly complete the processing of a small proportion of their patients. As a consequence, the solid line drops below the dotted line after this point. In this case, the model reveals that the existence of the target may have caused staff quickly to complete the processing of a few patients; however, the proportion is small, which suggests that department A is not indulging in any serious gaming to meet the 4-hour target.
Figure 3 shows the performance of department B; the two lines are very different from those for department A in fig 2. As before, the dotted line shows the simulated performance (ie, the expected performance if the department is run normally with no special actions taken as the 4-hour deadline approaches). The solid line, which shows the actual reported performance of department B is very different. There is a substantial sharp peak as the 4-hour deadline approaches, which suggests that special action is being taken in many cases as the deadline approaches, and the difference between the dotted line and the solid line indicates how many patients may be affected by this special action. This suggests that department B may not be under control, and that some serious interventions may have occurred to meet the 4-hour target as it approached.
Figure 4 helps us to understand what is happening in department B. As before, the dotted line is the simulated performance, but now the stacked histogram shows the proportion of patients who are admitted or discharged. As the deadline approaches, the proportion of patients admitted starts to increase and peaks at about 4 h. Why should this happen? Some of these patients will have been in the emergency department for a long time because there is uncertainty about their diagnosis and treatment or because longer treatment is needed. However, it seems unlikely that this is true of all the patients, and it could be that some are admitted as inpatients simply to meet the target—placing inpatient resources under unnecessary strain. This conclusion accords with the findings of Locker and Mason18 and of Cooke et al19 that bed occupancy affects total time in an emergency department and complements the simpler approach of Bagust et al.20 Such use of assessment units may not always be inappropriate since many such patients will quickly be discharged after further assessment; however, its use in meeting or beating the 4-hour target seems questionable.
Conclusions
Computer simulation is a widely used and relatively straightforward tool that can help people to understand the performance of emergency departments. It is possible to build a generic model of such departments that, in the English NHS, provides a sufficiently accurate representation of the performance of these departments. The generic model is populated by demand, staffing and resource data that are specific to a particular emergency department in order to simulate that department. Since task switching by staff is an important feature of emergency departments, the model described here represents this by fragmenting staff into “mini staff” which reduces data demands and is sufficiently accurate.
Most simulations of emergency departments are intended to enable clinicians and managers to try out process configurations and shift patterns to maintain or improve performance, and our model can be used in this way. In addition, however, our model is useful for studying altered performance in an emergency department, as shown by the two examples in this paper. The model does not explain the causes of this changed performance, but identifies it and encourages its further investigation with a view to ensuring that patients are appropriately treated in future.
The use of the model in emergency department B suggests that macro economic studies4 5 may be over-optimistic in arguing that the 4-hour target is being met without any gaming or altered behaviour. The emergency department simulator can help commissioners and regulators to understand the true underlying performance of departments, and demonstrates how the targets might in some circumstances be distorting the behaviour of healthcare staff even though targets are being met.
Acknowledgments
The authors acknowledge research collaborators Gwyn Bevan and Alec Morton (London School of Economics) and Peter C Smith (University of York). For obvious reasons the two emergency departments are not named, but the authors are grateful for their collaboration.
REFERENCES
Footnotes
-
Funding The model was developed as part of the EPSRC-funded DGHPSim project (ref EP/C010752/1).
-
Competing interests None.
-
Provenance and peer review Not commissioned; externally peer reviewed.