We describe a typology for the classification of critical incidents in emergency medicine in an accompanying paper.¹ We now present a study in which this typology was used.

In order that risk levels in emergency medicine can be controlled, it is important that those risks that occur commonly, and particularly those which have a potentially serious outcome, are identified. A small proportion of the incidents occurring in large adverse event studies, such as the Harvard medical practice study² and the quality in Australian health care study³ occurred in emergency departments (EDs), but little useful information regarding ED risks can be extracted from these. Several studies have looked at specific areas of failure in EDs, such as recognition of ECG or radiological abnormalities,^4–8 errors of diagnosis as a whole,⁹ or failures in the management of minor injuries by junior doctors when compared with nurse practitioners.¹⁰ Stella et al¹¹ published the only previous study of emergency medicine critical incidents, occurring in an Australian ED. However, the results that they reported did not give an indication of the types of incidents that occurred, but instead were concerned with possible aetiological factors. Additionally, they did not include any measure of risk stratification. The classification of risks occurring specifically in EDs has not been previously described in any study involving a wide range of risks.

The first aim of this study, then, was to classify a large number of situations involving risk, or critical incidents, according to the typology previously described. The second aim was to compare the types and the levels of risk between different EDs. The null hypothesis was that the risks are the same in different EDs.

METHODS

Demography and data collection

The demography and collection of data for the main database of critical incidents has been described previously.¹ Two of the EDs involved were in teaching hospitals (departments A and D), the other two (departments B and C) being situated in district general hospitals (DGHs).

As part of the data collection, each department was studied by MT for six, one week periods spread over the duration of the study. Throughout each of these weeks, critical incidents were identified in an identical, uniform manner by:

• daily review of the ED record of each patient visiting the department;

• direct observation of functioning of the departments; and

• direct reporting of critical incidents by ED medical and nursing staff to MT.

The radiological reports and pathology reports were not checked individually, but all recognised failures were included via the direct reports.

Application of typology

Each critical incident in the database was classified according to the typology. The number of critical incidents at each level in each category was calculated, along with percentage figures.

Comparison of results of application of typology between EDs

For purposes of comparison, only the subgroup critical incidents identified by MT in a consistent manner across all the departments were included. These critical incidents were selected from the database of all collected critical incidents. Statistical comparison was made between the categories of critical incidents occurring in each department using a χ² test. Further comparison was made of the levels of severity of the selected critical incidents using the Kruskal-Wallis test. All of these statistical calculations were performed using SPSS 9.0 for Windows. For these analyses, significance was set at p<0.05.

RESULTS

Results of applying typology to database

Table 1 shows the results of classifying all 816 identified critical incidents by using the typology. It can be seen from this table that failures relating to omissions in patient assessment are the commonest, accounting for 291 (35.6%) of the 816 critical incidents collected. These are followed in frequency by omission failures of external disposal (156, 19.1%), and of treatment (105, 12.9%). Equipment failures, commission failures of treatment, and general failures account for 59 (7.2%), 57 (7.0%), and 49 (6.0%) of the collected critical incidents respectively. Patient actions (32, 3.9%), documentation failures (26, 3.2%), internal disposal failures (22, 2.7%), and commission failures of patient assessment (14, 1.7%) follow, with commission failures of external disposal being the least frequent category of failure, accounting for only five (0.6%) of the critical incidents.

When the 49 critical incidents classified as “General” were re-assessed collectively, they were not found to have any common characteristics that would have suggested the need for an additional category.

The commonest level of severity in the critical incidents collected was level 3, n = 289 (35.4%). These are generally incidents where there is a risk of prolonged pain, disability, or illness. The next commonest level was level 2, which accounted for 239 (29.3%) of the critical incidents identified. These are broadly defined as incidents that are potentially life threatening, sight threatening, or limb threatening. Level 4 incidents, those with the potential to produce short term pain or illness, accounted for 192 (23.5%) of the critical incidents identified. Incidents that resulted in no harm, other than delays without significant effect, level 5 incidents, numbered 81 (9.9%). Fifteen level 1 (life threatening) critical incidents were identified, representing 1.8% of the total.

The commonest single category, by far, was the level 3 patient assessment omission failure category, which accounted for 149 (18.3%) of the critical incidents identified. This category includes incidents that relate to failures or delays in diagnosing conditions associated with severe pain, or with prolonged pain, disability, or illness. It also includes failures in diagnosing conditions where such a diagnosis significantly changes management of the patient, such as a missed ligament tear that requires surgical repair, and failure to perform investigations to exclude a significant diagnosis where such investigations are indicated, such as failure to exclude a pulmonary embolism in a patient presenting with pleuritic pain and shortness of breath. The next commonest single category was also in omission failures of patient assessment, at level 2, which accounted for 86 (10.5%) of the critical incidents. This category includes failures or delays in diagnosing potentially life threatening conditions, and would thus include failing to diagnose an acute myocardial infarction on an ECG. Level 3 and level 2 external disposal failures were also relatively common, accounting for 77 (9.4%) and 55 (6.7%) of all incidents respectively. Level 3 external disposal failures include failures to admit, or arrange appropriate follow up in patients with conditions associated with long term pain, disability, or illness. It also includes situations where no beds are available on wards, such that patients have to remain within the ED for prolonged periods. Level 2 external disposal failures include failures to admit patients with potentially life threatening conditions, such as unstable angina.

Of the 15 level 1 failures, equipment failures were the most common, accounting for five (33.3% of all level 1 failures). These included a situation in which the functioning of a whole department was brought to a halt by a power failure, and other situations where patients’ lives were threatened by malfunction of equipment, such as failures of syringe drivers giving essential treatments during transfers to other units.

Comparison between departments

During the 24 weeks when data regarding critical incidents were collected in a consistent manner (six separate weeks in each of the four departments), a total of 443 critical incidents were identified. Departments A–D saw 7292, 7300, 8116, and 9308 new patients during the six weeks of data collection respectively. Adverse incident rates varied from 11.1 (8.8–13.8) to 15.9 (13.5–18.7) per 1000 new patient attendances.

Table 2 shows the results of categorising these 443 critical incidents for each ED.

Statistical analysis performed on these data by χ² test gave a χ² result of 39.517, 21 degrees of freedom, p = 0.009. There was therefore a significant difference between the departments with respect to the types of critical incidents collected there. Examination of calculated standardised residuals showed that this difference lay in three areas. Firstly, the number of critical incidents classified as “Patient assessment” at department C was significantly higher than elsewhere. Secondly, the number of critical incidents of the “Internal disposal” type was significantly higher at department D than elsewhere. Finally, the number of critical incidents classified as being of the “Patient action” type was higher at department A than elsewhere, though it should be noted that the total number of patients of this type was very small.

Table 3 shows the results of classification by level of severity of the 443 critical incidents collected during the weeks when each department was visited by MT.

Statistical analysis performed on these data using the Kruskal-Wallis test gave a χ² result of 3.384, 3 degrees of freedom, p = 0.336. Thus, there was not found to be any significant difference in the levels of severity of critical incidents identified in different departments.

DISCUSSION

Principal findings

Classification of all collected critical incidents

After development of the typology, each of the 816 critical incidents was classified. Omission failures of patient assessment were the commonest type of critical incidents, accounting for a total of 291 (35.6%). Level 3 failures of this type were the single category that occurred most frequently, numbering 149 (18.3%). Training improvements would be likely to have the effect of lowering these high numbers, as many critical incidents in these categories occurred as a result of failures of interpretation of ECGs and radiographs, and failures to recognise the need to exclude significant medical conditions, all of which can be improved by training. The likelihood of treatment and external disposal failures, which together accounted for 323 (39.6%) of all critical incidents, would also probably be reduced by improvements in training.

High numbers of critical incidents were classified into levels 1 and 2, the combination of these accounting for 254 (31.1%) of all critical incidents collected. These critical incidents generally consisted of life threatening, or potentially life threatening, limb threatening, or sight threatening, situations. High numbers of these types of critical incidents are therefore particularly worrying. It is not possible to estimate how many of these critical incidents actually led to patient deaths, but clearly the more level 1 or 2 incidents occur the higher the number of unnecessary deaths is likely to be. It may be that the proportion of incidents at these levels is actually lower than the study showed, as it may be more likely that such incidents would be reported, or detected on review of patient records, than less significant failures. However, the absolute numbers of these incidents is high, and it is likely that many other incidents of these levels occurred but were either not detected, or were not reported. Further work will be necessary to look at ways of reducing the number of level 1 and 2 incidents. Approaches such as changes in skill mix and closer supervision of staff in training may be appropriate.

Comparison of incident classifications between departments

There were found to be significant differences between the departments in the types of the 443 critical incidents identified by MT during the weeks of individual departmental study. These differences lay in the higher numbers of “Patient assessment” failures in department C, the higher number of “Internal disposal” failures detected in department D, and the higher number of critical incidents attributed to patient actions in department A. Further examination of the data suggests that department C’s relatively high number of “Patient assessment” failures may be at least partly explained by delays in patient assessment caused by different triage priorities used there. Department D’s high number of “Internal disposal” failures arises largely as a result of filling of the department, which in turn is a result of delays in sending patients to wards. Because of low total numbers of critical incidents classified in the “Patient action” category, the finding of higher numbers at department A should probably be disregarded. There were no significant differences found when the levels of severity of critical incidents detected during the weeks of individual department investigation were compared.

Strengths and weaknesses of the study

Ideally, the critical incidents described in this study would have been prospectively collected and classified, rather than using the same critical incidents that were used to derive and validate the typology. However, it is difficult to see how this could have influenced the findings. As mentioned previously, the typology should be further validated against a new set of emergency medicine critical incidents.¹

It could be argued that a single investigator is prone to bias in that particular departments or workers might be favoured. However, we believe that the results of the comparison between different departments are significantly strengthened by the single investigator method. This permitted a uniform method of incident identification to be put in place and a single point of contact to be recognised. Although there is no guarantee that every single critical incident that occurred was identified, there is no reason to suppose that different numbers, or different types, of critical incidents were missed in different departments. Therefore, valid comparison can be made between departments using the data collected.

Comparisons with other studies

To our knowledge, no studies have previously been published that gave an indication of the risks to patients occurring in emergency medicine. Additionally, comparison of types of risk between departments has not previously been reported. There are therefore no studies with which to compare our results.

Implications

The high numbers of level 1 and 2 critical incidents occurring across all the departments should provide each of them with additional awareness of the risks that are being run on a continuous basis. Their medical and nursing management may be encouraged to examine the systems in which people work, and try to identify where improvements may be made. The results of comparison between the EDs of the types of critical incidents occurring in each have identified particular problem areas. This may act as an impetus to the departments and to others in their hospitals to act on these areas to attempt to lessen their impact.

Future research

Further research is indicated to establish whether the profile and interdepartmental variations of risks shown in this study are found in all types of departments and emergency systems. The typology could also be used to establish whether the risk profile of individual departments varies with time. Furthermore, the recognition that different risks occur in different departments will allow workers to investigate the relation of those risks with underlying factors such as departmental staffing, workload, and culture.

Conclusions

We have shown that large numbers of incidents occur in EDs involving high levels of risk to patients. Strategies to combat these risks are urgently required. Such strategies should be tailored to individual EDs, as risks differ significantly between EDs.