METHODS

This study took place in a teaching hospital over six months from August 1998 to February 1999, corresponding to the period of employment of six SHOs in emergency medicine. Three consultants, four specialist registrars, one staff grade doctor, and two nurse practitioners also staffed the ED. All radiographs (including chest radiographs) taken during this period were prospectively and independently reviewed in both the emergency and radiology departments to detect cases where a radiograph that was considered normal by ED staff was then reported as abnormal by the reporting radiologist. These were termed “discrepancies”. One of four consultants or two specialist registrars (years two and three) in radiology issued the final reports, though it was common practice for the specialist registrars to discuss abnormal films with a consultant before reporting. Aside from the authors, ED and radiology staff were unaware that the study was taking place.

Where a discrepancy was detected the patient was reviewed by a senior doctor (either a middle grade or consultant) in emergency medicine, with further radiological examination and follow up where indicated. This allowed a “gold standard” diagnosis to be established in all cases.

RESULTS

During the study period, 19 468 new patient attendances generated 11 749 radiographic examinations, with a radiographic examination representing a discrete radiographic assessment of a single anatomical area. Many patients underwent more than one radiographic examination (for example, foot and ankle), so the actual proportion of patients undergoing radiography was somewhat lower than 50%. Discrepancies were detected in 175 patients (representing 0.9% of new attendances and 1.5% of radiographic examinations). The radiographic examinations giving rise to four or more discrepancies are listed in table 1 . Of these 175 patients, 136 were subsequently shown to have been incorrectly interpreted in the ED (ED false negatives), with 40 patients undergoing a change in management as a result. These 40 significant ED false negatives are detailed further in table 2 . In the remaining 39 patients the ED interpretation was judged to be correct (radiology false positives), with 16 patients undergoing further investigations or visits to the ED to confirm this.

Table 3 shows the grade of ED doctor involved in each discrepant case, and also in those 40 patients requiring a change in management as a result of an error of interpretation in the ED. Where a more junior ED doctor consulted a senior for advice the error is attributed to the senior.

DISCUSSION

These results demonstrate that:

• About 1.5% of the plain radiographic examinations requested by an ED will give rise to a reporting discrepancy of this type.

• The formal reporting of ED radiographs detects significant missed abnormalities (that is, ED false negatives requiring a change in management) at the rate of 3 per 1000 radiographs reported.

• For every five significant missed abnormalities that are detected two patients will require further unnecessary radiographs and/or visits to the ED as a result of a false positive radiology report.

The Royal College of Radiologists recommends that three minutes be spent reporting each investigation,¹⁵ and this means that in theory 587 hours were required to report all of the films generated in our ED during the study period. It also means that 16.7 hours of radiological time (plus all the attendant overhead costs) were required to detect one significant missed abnormality. Given that the cost of a significant missed abnormality is often high, both in terms of patient suffering and in subsequent litigation costs,⁷ this investment of radiological time seems worthwhile, and also acts to justify a policy of universal radiological reporting. In such circumstances the occasional unnecessary recall of a patient on the basis of an incorrect radiology report is perhaps inevitable, and justifiable where it is an infrequent event.

A potential compromise would be to adopt a policy of selective reporting that reduces the demand on radiology services without allowing significant abnormalities to be overlooked. It is interesting to note from table 1 that the radiological investigations that most commonly give rise to a discrepancy (as a percentage of the number of radiographs of that type requested) are finger and toe films, but that these are rarely significant (table 2 ). On the other hand, a radiologist should always report wrist, chest, ankle, foot, and elbow films as these commonly give rise to ED false negatives.

Table 3 shows that in the 2481 patients seen or reviewed by an ED consultant no significant abnormalities were missed. Assuming that the radiograph rate for consultants is the same as the overall mean this suggests that 1489 investigations interpreted by an ED consultant did not need to be further reported by a radiologist, reducing the radiological workload by 13%. In contrast middle grade staff, SHOs and nurse practitioners missed a similar proportion of significant abnormalities. This confirms previous reports regarding the satisfactory performance of nurse practitioners in the ED,¹⁶ but is perhaps surprising for the middle grade doctors. One possible explanation is that they were asked to review a large number of the most challenging radiographs by the junior staff, thereby increasing their apparent error rate.

Several previous studies have reported the accuracy with which ED practitioners interpret the radiological investigations that they request. For example, Gratton et al, working in Missouri, found clinically significant errors in 2.8% of 12 395 investigations interpreted by ED residents,¹⁷ while Overton quotes a figure of 0.59% from 6740 investigations in Michigan.¹⁸ Mucci, in 1983, reported a UK “significant miss” rate of 2.5% for “casualty officers”, and suggested that selective reporting would be accompanied by a corresponding increase in overlooked abnormalities.¹¹ A more recent report demonstrated 351 missed or strongly suspected fractures from approximately 34 000 trauma radiographs (1%), with 286 cases prompting further action from ED staff (0.8%).¹⁹ An alternative approach to assessing error rates is that described by Vincent at al.⁶ This group specifically studied junior doctors interpreting only abnormal radiographs, finding an error rate of 35% overall, and 39% for abnormalities with clinically significant consequences. Initially this rate seems high, and at odds with the other studies quoted, but is consistent with the concept that abnormalities are missed in only a small percentage of the total number of radiographs requested, as most radiographs are normal. The same mechanism will also act to reduce the apparent frequency with which an abnormality is missed when inexperienced doctors request large numbers of unnecessary, and normal, radiographs. However, less than 50% of our patients underwent radiography, in keeping with national practice.³

In response, a range of approaches has been used to reduce the frequency of missed radiographic abnormalities. Radiographers can play a vital part through the “red dot” system, by which they indicate radiographs that appear to be abnormal before review by an ED clinician,²⁰ and specific training programmes can also permit radiographers to report skeletal radiographs with a high degree of accuracy.^21,22 An alternative approach is the implementation of a picture archiving and communications system (PACS), which in one recent study reduced the overall rate of misdiagnosis from 1.5% to 0.7%, and the rate of misdiagnosis requiring patient recall from 0.6% to 0.4%, though the last change did not achieve statistical significance.²³

No standard has yet been set for the interpretation of radiographs by emergency department staff, and such a measure would need to tackle the significant interobserver variability that has previously been identified. The level of major disagreement between even experienced pairs of observers has been estimated at 5%–9%,²⁴ and this has important implications when determining a gold standard. Nevertheless, we believe that simple initiatives, coupled with increasing numbers of senior medical staff and awareness of the problem, continue to drive down the rate of significant error and improve the quality of care in UK emergency departments. Our data demonstrate an overall “significant miss” rate of just 0.3%, and identify a basis on which selective reporting may now prove successful.

In conclusion, radiological reporting of all ED radiographs detected three clinically significant abnormalities that would otherwise have been overlooked, for every 1000 radiographs reported, but also generated a smaller proportion of incorrect radiology interpretations, leading to further unnecessary investigation. The minor inconvenience and cost created by such unnecessary investigations seems to be a price worth paying in order to reduce the number of significant abnormalities that are overlooked in the ED.

The formal reporting process is therefore highly worthwhile, but the adoption of a more selective policy may reduce radiology workload without compromising patient care. It is possible that radiographs of the digits and those interpreted by an ED consultant may be successfully removed from the radiological reporting process, but this requires confirmation in a further prospective study designed to assess whether such an approach would be safe and effective in actual clinical practice.