Objectives: To audit the proportion of drug treatments started on a medical admissions unit that is justified by published evidence, and the proportion for which no justification could be found.
Methods: Retrospective review of randomly selected case notes to identify drug treatments started and the problem(s) for which they were prescribed, followed by literature searches.
Results: A total of 132 treatment-problem pairs were found, comprising 85 unique treatment-problem pairs. An evidence base was found in support of 78 of the treatments started (59.1%). A further 41 treatment-problem pairs could be argued to be reasonable practice (sometimes included in guidelines), even though no published trial data support them. Ninety per cent of drug treatments started on the medical admissions unit have either an evidence base or are accepted practice.
Conclusions: Regular audit of this nature could be carried out on units admitting acute medical patients. Similar audits in internal medicine have delivered consistent results (50%–60%); there is a baseline level against which units can compare themselves. Clinical audit is an integral feature of clinical governance; all wards admitting acute medial patients could conduct similar audits on a random sample of patients.
- medical audit
- drug therapy
- acute disease
- evidence based medicine
- retrospective studies
Statistics from Altmetric.com
The aim of this work was to estimate, through retrospective audit, firstly the proportion of drug treatments that have a basis in valid research and, secondly, the proportion of drug treatments for which neither a research evidence base nor consensus exists. It excluded organisational interventions (for example, referral to a stroke unit), ”surgical” interventions (for example, drainage of a pleural effusion), nursing treatment, and interventions from professions allied to medicine.
Though there may be some who believe that little treatment in internal medicine has a basis in sound evidence, others suggest otherwise. A study from Oxford found that most management of the “primary problem” was based on valid research (53%) or was felt to be standard treatment on the basis of non-experimental evidence (29%).1 Similar studies from Canada and Sweden found that 64.8% and 50% treatments were supported by published experimental evidence.2,3
At the General Infirmary in Leeds, most patients with acute medical problems are admitted from accident and emergency (A&E) via the medical admissions unit (MAU): 6000–7000 per year. It is important that treatment started in this area is the best possible. An important element of treatment provided on an MAU is new drug treatment.
Neither the effectiveness of MAUs (or medical assessment units) nor that of the care provided there has been subject to formal evaluation: “it has not been possible so far to evaluate these facilities or to make informed comparisons between them”.4 To audit outcomes an adequate database to allow adjusting for casemix and severity is needed to produce credible comparisons between units or between individual units and a national average. Audit of process could, reasonably, be viewed as a poor relation to audit of outcomes, but it is, we believe feasible at present. “Research and effectiveness” is one of the technical components of clinical governance assessed in Commission for Health Improvement (CHI) clinical governance reviews.5 Specifically, as part of this, CHI reviewers assess:
The importance placed on implementing and monitoring evidence based practice.
Mechanisms to make operational effective practices—for example, evidence based guidelines for disease management.
Dissemination of the findings of research.
The need to link medicines management to clinical governance has also been highlighted recently.6
Patients were selected at random (using computer generated random numbers). No patients were excluded a priori from analysis. Because a shift system is in operation for junior doctors on the MAU, we did not select consecutive patients. A total of 108 patients were chosen at random, nine from each of 12 consecutive months. Case notes were then reviewed by RDH to identify any drugs started while on the MAU (or by MAU staff while the patient awaited admission in A&E) and the indications for each of these.
A list of treatment-problem pairs was generated and electronic database (Medline, Embase, Cochrane, Clinical Evidence) searching was carried out by all authors, using the strategy described by Sackett et al.7 The quality of individual papers found by the search was not assessed as part of this work (the aim of which was to establish whether or not there was evidence for an intervention rather than establish which intervention was supported by the best evidence).
Notes were available for 83 patients. Altogether 132 treatment-problem pairs were identified. For 65 of these there was only one patient with that particular problem; for eight treatment-problem pairs there were two patients; for three pairs there were three patients; for eight pairs there were four patients, and for one pair there were 10 patients. Many of these occurred in more than one patient; 85 different treatment-problem pairs were found. An evidence base was found in support of 78 of the treatments started (59.1%, 95% CI 50.7% to 67.5%).
There were 41 additional problems for which no research evidence could be found to justify the treatment given, but which we would argue are reasonable practice (in some cases supported by guidelines). Examples of such interventions include the use of hypertonic dextrose to treat hypoglycaemia or the use of dexamethasone to treat headache in a patient thought to have a space occupying lesion. Along such lines, 90% (95% CI 85.1% to 95.2%) of drug interventions on the MAU have either a research base or are accepted practice.
The most striking example of a treatment we feel does not represent good practice was the use of flumazenil to treat a patient with a (self) overdose of benzodiazepines. In such patients co-ingestion of a tricyclic antidepressive is not uncommon; antagonising the benzodiazepine may lead to seizures. Most of the other treatments that were felt to deviate from standard practice were antibiotics (for example, the use of ampicillin alone in the treatment of cellulitis).
The key finding of this audit—that an evidence base exists for 59% treatments started—is comparable with other studies in internal medicine. Although the paper by Ellis et al1 reported data from urgently admitted patients, the data presented here are the first such data from a MAU (which receives acute patients constantly, not just when scheduled to be “on take”). Another difference between this and previous publications is that while they considered treatments only for the main problem in this study treatments were considered for all problems that had been identified. In some cases picking the most important problem in patients with multiple problems may be arbitrary—the methods used here avoid the possibility of introducing bias by choosing as the main problem the one for which most evidence exists.
The methods used did not entail appraisal of the quality of the evidence that was identified. This was because the emphasis in this work was to identify those treatments started for which evidence could not be found (followed by consideration of whether or not that treatment was reasonable). The methods used also do not assess the evidence base behind non-drug interventions, nor do they establish the proportion of patients who received the “best” treatment. The second would be impossible to carry out in a retrospective way, as in many instances there will be effective alternatives, and what constitutes the best will be determined by patient preferences. Drug treatments started were audited rather than all interventions for three reasons: to permit comparison with previously published work; because there is comparatively little evidence for other types of treatments; because this audit was carried out by medical and pharmacy staff who did not feel competent to judge the appropriateness of nursing or therapy interventions.
There are implications for research and for practice. If problems are identified for which there is no evidence base for their treatment, then either primary research to address that deficiency or the adoption or development of interim local policies or “expert” guidelines could follow on.
Regular audit along these lines (on treatments given to randomly chosen patients) is feasible in units admitting acute medical patients. In this group of patients, a figure of 50%–60% treatments having an evidence base has been consistently found; units with a figure lower than this should be prompted to look at their search strategy (ideally in collaboration with a medical librarian), their patient mix (which is unlikely to be hugely different for “undifferentiated” acute medicine), and also to review whether any of the treatments (particularly frequently used ones) are unusual. To be effective at improving practice, this type of audit should lead to prompt feedback about any deficiencies found, coupled with training and education to suggest how similar problems should be subsequently managed. Regular re-audit and education (closing the audit “loop”) is likely to be the most effective way of using these methods to improve care.
Frequent monitoring of a small number of patients (perhaps as few as 10 per month) is likely to yield useful information after a few months (the patients should be randomly chosen to try to ensure the full range of patients is included in the audit). Someone familiar with electronic searching methods could probably complete a literature search in an afternoon for about 10 patients; this does not include time for critical appraisal of the papers found. The workload will probably fall after the first few months because of increased speed with searching and because fewer new problem-treatment pairs are likely to be found. Workload could be further reduced by collaboration between units—this would permit comparison between units but, important from a practical perspective, reduce the number of searches each unit would have to perform. In an environment where it is becoming increasingly necessary to demonstrate that the “right” care is being provided, this type of audit is likely to be helpful as it permits comparison with other published data, and also highlights areas where changes in practice may be needed. CHI clinical governance reviews are looking for clinical audit from which learning is derived that leads to sustained improvement in patient care and that leads to the development of research questions.5 While we would not suggest that the type of audit we describe should be the only sort carried out, we do believe that it is likely to meet these requirements. Generic audits such as this one, as well as those that are limited to a particular condition, are likely to be needed in areas, like MAUs, that take unselected patients.
Conflicts of interest: none declared.
If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.