METHODS

Patients were selected at random (using computer generated random numbers). No patients were excluded a priori from analysis. Because a shift system is in operation for junior doctors on the MAU, we did not select consecutive patients. A total of 108 patients were chosen at random, nine from each of 12 consecutive months. Case notes were then reviewed by RDH to identify any drugs started while on the MAU (or by MAU staff while the patient awaited admission in A&E) and the indications for each of these.

A list of treatment-problem pairs was generated and electronic database (Medline, Embase, Cochrane, Clinical Evidence) searching was carried out by all authors, using the strategy described by Sackett et al.⁷ The quality of individual papers found by the search was not assessed as part of this work (the aim of which was to establish whether or not there was evidence for an intervention rather than establish which intervention was supported by the best evidence).

RESULTS

Notes were available for 83 patients. Altogether 132 treatment-problem pairs were identified. For 65 of these there was only one patient with that particular problem; for eight treatment-problem pairs there were two patients; for three pairs there were three patients; for eight pairs there were four patients, and for one pair there were 10 patients. Many of these occurred in more than one patient; 85 different treatment-problem pairs were found. An evidence base was found in support of 78 of the treatments started (59.1%, 95% CI 50.7% to 67.5%).

There were 41 additional problems for which no research evidence could be found to justify the treatment given, but which we would argue are reasonable practice (in some cases supported by guidelines). Examples of such interventions include the use of hypertonic dextrose to treat hypoglycaemia or the use of dexamethasone to treat headache in a patient thought to have a space occupying lesion. Along such lines, 90% (95% CI 85.1% to 95.2%) of drug interventions on the MAU have either a research base or are accepted practice.

The most striking example of a treatment we feel does not represent good practice was the use of flumazenil to treat a patient with a (self) overdose of benzodiazepines. In such patients co-ingestion of a tricyclic antidepressive is not uncommon; antagonising the benzodiazepine may lead to seizures. Most of the other treatments that were felt to deviate from standard practice were antibiotics (for example, the use of ampicillin alone in the treatment of cellulitis).

DISCUSSION

The key finding of this audit—that an evidence base exists for 59% treatments started—is comparable with other studies in internal medicine. Although the paper by Ellis et al¹ reported data from urgently admitted patients, the data presented here are the first such data from a MAU (which receives acute patients constantly, not just when scheduled to be “on take”). Another difference between this and previous publications is that while they considered treatments only for the main problem in this study treatments were considered for all problems that had been identified. In some cases picking the most important problem in patients with multiple problems may be arbitrary—the methods used here avoid the possibility of introducing bias by choosing as the main problem the one for which most evidence exists.

The methods used did not entail appraisal of the quality of the evidence that was identified. This was because the emphasis in this work was to identify those treatments started for which evidence could not be found (followed by consideration of whether or not that treatment was reasonable). The methods used also do not assess the evidence base behind non-drug interventions, nor do they establish the proportion of patients who received the “best” treatment. The second would be impossible to carry out in a retrospective way, as in many instances there will be effective alternatives, and what constitutes the best will be determined by patient preferences. Drug treatments started were audited rather than all interventions for three reasons: to permit comparison with previously published work; because there is comparatively little evidence for other types of treatments; because this audit was carried out by medical and pharmacy staff who did not feel competent to judge the appropriateness of nursing or therapy interventions.

There are implications for research and for practice. If problems are identified for which there is no evidence base for their treatment, then either primary research to address that deficiency or the adoption or development of interim local policies or “expert” guidelines could follow on.

Regular audit along these lines (on treatments given to randomly chosen patients) is feasible in units admitting acute medical patients. In this group of patients, a figure of 50%–60% treatments having an evidence base has been consistently found; units with a figure lower than this should be prompted to look at their search strategy (ideally in collaboration with a medical librarian), their patient mix (which is unlikely to be hugely different for “undifferentiated” acute medicine), and also to review whether any of the treatments (particularly frequently used ones) are unusual. To be effective at improving practice, this type of audit should lead to prompt feedback about any deficiencies found, coupled with training and education to suggest how similar problems should be subsequently managed. Regular re-audit and education (closing the audit “loop”) is likely to be the most effective way of using these methods to improve care.

Frequent monitoring of a small number of patients (perhaps as few as 10 per month) is likely to yield useful information after a few months (the patients should be randomly chosen to try to ensure the full range of patients is included in the audit). Someone familiar with electronic searching methods could probably complete a literature search in an afternoon for about 10 patients; this does not include time for critical appraisal of the papers found. The workload will probably fall after the first few months because of increased speed with searching and because fewer new problem-treatment pairs are likely to be found. Workload could be further reduced by collaboration between units—this would permit comparison between units but, important from a practical perspective, reduce the number of searches each unit would have to perform. In an environment where it is becoming increasingly necessary to demonstrate that the “right” care is being provided, this type of audit is likely to be helpful as it permits comparison with other published data, and also highlights areas where changes in practice may be needed. CHI clinical governance reviews are looking for clinical audit from which learning is derived that leads to sustained improvement in patient care and that leads to the development of research questions.⁵ While we would not suggest that the type of audit we describe should be the only sort carried out, we do believe that it is likely to meet these requirements. Generic audits such as this one, as well as those that are limited to a particular condition, are likely to be needed in areas, like MAUs, that take unselected patients.