Article Text

PDF

The NICE guidelines in the real world: a practical perspective
  1. J Dunning,
  2. F Lecky
  1. Department of Emergency Medicine, Manchester Royal Infirmary, Manchester, UK
  1. Correspondence to:
 Mr J Dunning
 Department of Emergency Medicine, Manchester Royal Infirmary, Oxford Road, Manchester M13 3BW, UK; joeldunningdoctors.org.uk

Abstract

This review examines the derivation of the NICE guidelines and discusses some of the problems of putting research into practice.

  • craniocerebral trauma
  • evidence based medicine
  • guidelines

Statistics from Altmetric.com

Head injury is a common problem. However while emergency physicians see one million head injuries per year, only 2000 of these will require neurosurgery and only half of these patients will present as a minor head injury. Therefore while emergency physicians will see a large number of head injuries they may only see one patient every year who presents as a minor head injury and goes on to require neurosurgery. In addition we should remember that the large majority of minor head injuries are seen by physicians with a year or less experience in the specialty.1–3

It is therefore clear that clinical experience cannot be relied upon to pick up these uncommon patients who present as a minor head injury but who will shortly require emergency surgery for an intracranial haematoma. Therefore guidelines must be exclusively relied upon for the triage of minor head injuries.

The goal of institutions that derive guidelines is therefore to provide guidance that safely allows low risk patients to be discharged, while identifying all patients at high risk as rapidly as possible. This has been done in many ways since the Harrogate guidelines were first published in 19844 but all use a series of diagnostic tests including the skull radiograph, admission, and computed tomography to perform this triage.

So why do we need new guidelines in this area and what is the problem with the way we used to perform this triage?

ADMISSION AS A DIAGNOSTIC TEST

The major decision in guidelines such as Scottish Intercollegiate Guidelines Network (SIGN) and the Royal College of Surgeons (RCS) guidelines is the selection of patients for admission.5,6 One in 10 children with head injury are selected for admission by RCS guidelines and the figure is slightly lower for adults. However, up to 97% of these patients will go home the next day requiring no intervention. Thus as a diagnostic tool, admission performs poorly with a positive predictive rate of 3% or less. In addition those who do deteriorate on the ward are often picked up late and thus suffer a less than optimal outcome because of the delay in evacuating the haematoma. In addition these patients have been historically cared for on non-specialist units, with staff unaccustomed to performing neurological observations and patients invariably require hospital transfer after deterioration has been detected. Thus admission for observation is an expensive and unreliable diagnostic test for the detection of an intracranial haematoma.1,6–9

SKULL RADIOGRAPHY AS A DIAGNOSTIC TEST

The RCS and SIGN guidelines also use the skull radiograph as an intermediate diagnostic test to select patients for further testing either by computed tomography (CT) or by admission. This test too performs very poorly. Between 25% and 50% of all patients attending with a minor head injury will have a skull radiograph, but only 2% of these will show a fracture. Of these fractures less than 1 in 30 will actually have an underlying intracranial haematoma requiring neurosurgical intervention. However, far more worrying is the large percentage of patients who have an intracranial haematoma with no detectable fracture, and thus the sensitivity of the skull radiograph is as low as 75% in these cases10–12 and unpublished data.

In addition it has been consistently shown that between 10% and 20% of all fractures are missed by competent emergency physicians thus rendering this test highly unreliable as a diagnostic tool.12,14,15 (It is interesting to note that there is much resistance to emergency physicians reporting head computed tomograms as is done very competently in the USA and Canada, while it is expected that they should still report skull radiographs when they routinely miss up to 20% of all fractures!)

CT AS A DIAGNOSTIC TEST

CT is the only reliable way to rule out an intracranial haematoma. Its sensitivity approaches 100% and if performed more than an hour after the initial injury there are virtually no false negative scans. This performance is in direct contrast with the poor performance of both admission and the skull radiograph. In addition most of the developed world uses CT as the primary diagnostic tool in head injuries and only admits patients for observation after a positive scan.16,17

EVIDENCE BASED GUIDELINES IN THE 21ST CENTURY

The most common criticism of the NICE guidelines (National Institute of Clinical Excellence) is the question of why we do not just continue in the way we have been treating patients for the past 20 years. It is important to realise that the RCS and SIGN guidelines are all consensus documents and provide no data as to the performance of these guidelines and indeed it is unknown what their true performance is as a diagnostic tool. Although physicians may state that they have been using them for years and they have not had any problems with them, this is not a good enough reason to continue with their use. In our own cohort study of 23 000 children suffering head injury we have identified 12 patients who have been sent home, but came back and had an abnormal CT scan, six of whom also required neurosurgery. Of note two of these patients went to a different hospital on the second occasion. In addition we have also identified 27 patients who had been observed on a ward without CT and deteriorated there. These patients then required intubated transfer or neurosurgery which, it could be argued, could have been performed earlier had CT been performed on admission (unpublished data).

These patients invariably do not come to the attention of the original physician who applied the RCS or SIGN guidelines and either sent them home or referred them for admission, and thus these physicians will not be aware of their less than optimal outcome.

DERIVATION OF THE NICE GUIDELINES

It is important to note at this stage that the guideline development group for the NICE guidelines consisted of 15 practising clinicians who had been elected to represent their own stakeholder associations, including The British Association of Accident and Emergency Medicine (BAEM), the British Paediatric A&E group, neurosurgeons, and the Royal College of Radiologists. There were five full time staff, four of whom were non-clinical, who were given the task of performing the systematic review and writing the document on behalf of the guideline development group. In addition there were two patient representatives, representing head injury charities. If the list of those involved is browsed it can be seen that at least half the members of the guideline development group, including the chairman, have seen head injury patients on a daily basis for most of their careers.8 It is also important to note that this group is entirely independent of any government agency or influence and the original remit of the group was only to produce guidelines on the basis of the best evidence, not on the basis of cost, resources, or convenience.

In the derivation of these guidelines, the guideline development group started from the position that the guidelines had to have well documented and well studied performance statistics. They felt that it was unacceptable to advocate guidelines that did not tell the physician using them that they would be highly sensitive (that is, missed virtually no patients with a haematoma) and have the optimal specificity in their hands. Thus further refinement of current consensus guidelines would be unacceptable until such time that studies documented their performance and to date no such studies exist.

THE CANADIAN HEAD CT RULE

In direct contrast with the RCS or SIGN guidelines, the Canadian head CT rule has been derived in 3121 patients and validated in a second cohort of a similar number.18,19 In addition they have been shown to have a 100% sensitivity for identifying patients requiring neurosurgery and near 100% sensitivity in identifying all abnormal CT scans. The only other study of comparable quality was by the New Orleans study group who also show a high sensitivity but with a much lower specificity.17 It was for this reason that we strongly believe that only these guidelines can be advocated as best practice in the management of patients with head injury.

PUTTING RESEARCH INTO PRACTICE

Once it is established that the Canadian head CT rule is the optimal evidence based guideline, and skull radiograph and admission based guidelines are unacceptable because of the multiple problems illustrated above, it was left to the guideline development committee to decide how to turn a research based guideline into practical guidelines that would apply to all patients. Thus it was necessary to investigate the problem of those patients that had not been included in the study—that is, patients with a coagulopathy, patients who had not had a period of loss of consciousness, etc. These issues were dealt with individually and these guidelines are notable for the fact that where this has been done the strength of evidence for the recommendation had been downgraded to optimise the transparency of the guidelines.

HEAD INJURY GUIDELINES IN CHILDREN

This was perhaps the most difficult issue for the guideline development committee. A full systematic review was conducted for children and no papers were found that came close to the methodological rigour of the Canadian head CT rule in adults. Therefore there was a dilemma. Do we advocate using the RCS or SIGN guidelines until such studies are performed? This would create two very different guidelines for a 15 year old and a 17 year old patient and much confusion. The alternative was to extrapolate the Canadian head CT rule to children with the assumption that it would “probably” not miss any intracranial haematomas and await further research.

The second alternative was chosen and it is important to note that the NICE guidelines are due to be revised in two years rather than the usual five years directly because of this problem in children. Also of note there are several very large studies that will have reported in children by 2005, including the CATCH trial from Canada, the CHALICE study from the UK (unpublished data), the NEXUS study from the USA and the ISHIP study, an international study currently led from Italy.20–22

Many of the issues raised against the NICE guidelines are to do with the issue of how the guidelines relate to children, and it is hoped that these will be addressed in 2005.

PROBLEMS WITH THE NICE GUIDELINES

Implementation of guidelines

A major barrier to implementation of the NICE guidelines is the logistics of setting up the infrastructure to perform CT scans on a larger number of patients. Radiologists and radiographers are currently in very high demand and these guidelines will only increase the demands on their time particularly out of hours. It was clearly realised by the guideline development group that financial and organisational problems would certainly pose great problems to centres proposing to implement the NICE guidelines. Indeed many of the members of the guideline development group would have to implement these guidelines in their own hospitals. Thus in section 5.2.1 it was stated that “Skull X-rays in conjunction with high quality in-patient observation also have a role where CT scanning resources are unavailable.8 This recognises that where resources do not permit full implementation of the NICE guidelines, using previous guidelines such as the RCS guidelines6 is an acceptable alternative. This should not detract from the fact that CT based guidelines clearly represent the optimal way to treat patients with head injury in well resourced hospitals.

Children who vomit

Some of the strongest criticism comes from the guidance that children who vomit twice should have a CT scan. This rule accounts for up to 90% of the CT scans now demanded by the NICE guidelines. It is certainly true that children vomit far more easily than adults but it should be remembered that currently all these children are submitted to the diagnostic test of a night of admission, which as well as being expensive and causing possible delays in evacuating evolving haematomas, also causes a great deal of anxiety among parents.

What is the alternative? If we adhere to our principle of advocating only the best evidence guidelines, then we cannot change this criterion to three or four vomits as we no longer know the performance of the decision rule. If we advocate only observing them then we are back to the problems of admission as a diagnostic test, which is both expensive and harmful. Thus until 2005 it is important to recognise that the extrapolation of the Canadian head CT rule to children is a recommendation based on grade D evidence and thus perhaps this gives clinicians the ability to interpret this criteria according to the clinical situation faced by them in the emergency department.

Alcohol and head injury

A second criticism is the fact that the NICE guidelines make no allowance for alcohol consumption. This is in fact an incorrectly held belief by physicians. In the Canadian head CT study, alcohol consumption was recorded and investigated and serum alcohol concentrations were also routinely recorded and it was found that if the concentration of alcohol was sufficient to cause a decrease in the Glasgow coma score (GCS), persisting after two hours, then this was a significant risk factor for developing an intracranial haematoma, but if the person was still GCS 15 this was not a significant risk factor. In addition they found that 12% of all patients had an unreliable GCS score secondary to alcohol, which may be similar to the levels found in UK hospitals. Thus there was no need to provide specific guidance to those under the influence of alcohol, and it was found to be an unreliable predictor in itself.

Resolution of symptoms

Many physicians comment that if a person is waiting four hours for a CT scan but the hour before their scan all their symptoms have resolved, it is impractical to continue to insist on a CT scan. This is particularly relevant in the vomiting child. This is a legitimate concern however we again would argue that we should continue to do what the best evidence suggests. There are no data that exist to suggest that if a child vomits for a prolonged period and then stops, that it is safe to assume that they therefore do not have intracranial pathology. The only evidence that exists is that they are in a high risk group and thus should be submitted to the definitive diagnostic test. As soon as the decision is taken not to scan that child, the clinician can have no idea as to the likelihood that there may be occult pathology. Guidelines will never be perfect and it is important to note that we are not aiming to diagnose positive patients but to safely discharge negative patients. Without a scan that patient is not yet at low risk.

Discharge after CT

Over 90% of the CT scans by the NICE guidelines in children are for vomiting. It is suggested therefore that to send that patient home after the scan is impractical. It is certainly true that these patients may need to continue to stay in hospital and that this may in fact mean that the NICE guidelines are not cost neutral.

There are two points here. The first is that after the scan, the patient is low risk and therefore neurological observations become unnecessary, and once symptoms resolve, the patient may immediately go home. On the issue of cost effectiveness, the NICE guidelines were deemed to be cost neutral on the basis that CT replaced a large percentage of those previously admitted.

It is very difficult to fully assess the cost implications of the NICE guidelines and a detailed cost analysis does actually appear in the long version of the guidelines that suggests cost neutrality. However, initial set up of a CT based service in the place of an admission based service is not assessed. Data from UK TARN (Trauma Audit Research Network) suggest that if set up costs are not considered, the NICE guidelines will be less expensive then current guidelines as the reduction in admission and skull radiography costs will more than offset increased CT costs.

SUMMARY

No guidelines are perfect, however the NICE guidelines for the management of patients with a head injury are a great step in changing from consensus based guidelines based on flawed diagnostic tests such as skull radiography and admission, to transparent guidelines where the sensitivity and specificity is well known and early diagnosis of serious pathology or safe discharge is put at the forefront of decision making. There is no doubt that they will result in better treatment for patients suffering a head injury, and it is only the logistics of access to CT and acceptance of the guidelines that stand in the way of full implementation.

All guidelines should be critically assessed and indeed these guidelines are particularly strengthened by the fact that they are destined to evolve with regular updating, starting in 2005. Therefore it is important that physicians feed back their concerns to those involved in the process of the modification of the guidelines. This series of articles is an important part of that process as we all learn how to apply research studies to everyday practice.

REFERENCES

View Abstract

Footnotes

  • Funding: none.

  • Competing interests: Joel Dunning was employed for six months full time as a systematic reviewer at the National Collaborative Centre for acute care, performing the systematic review for the NICE guidelines for head injury: assessment, investigation and early management of head injury in children and adults, and assisted the guideline development group in the writing of these guidelines.

Request permissions

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.