Article Text
PDF
Abstract
Aim To evaluate the efficacy of the UK swine flu algorithm as a screening tool in unwell children, and to compare the management advice given with the National Institute for Health and Clinical Excellence (NICE) feverish illness guideline advice.
Method All paediatric medical admissions to the unit, with a fever and over the age of 1 year, during 2 weeks in November were analysed, and their histories were put through both the swine flu algorithm and the NICE fever guidance for the under 5s.
Results Of 72 patients, 71 would have had a diagnosis of swine flu had their symptoms been put through the algorithm. Two patients had confirmed swine flu on testing, and 32 patients definitely did not have swine flu. The positive predictive value of the algorithm is between 2.8% and 56.3% in this population. 39% would have been advised to have a face-to-face consultation by the NICE guidance, but would not have been advised to have an urgent consultation by the swine flu guidance. At least 79% of patients had treatments only available in hospitals.
Conclusions The swine flu algorithm is of little use in differentiating unwell children, and advice given does not correlate well with that of the NICE guidance. There is a significant risk of harm with false-positive diagnoses and potential delays in appropriate treatment. The authors were unable to obtain the data and rationale behind the algorithm, and believe that this should be published. Face-to-face consultations may be the only way to ensure patient safety.
- Clinical assessment
- major incident/planning
- paediatrics
- prehospital care
Statistics from Altmetric.com
Fever is a common presenting problem in young children.1 The majority of cases are caused by self-limiting viral infections; however, infections remain a leading cause of mortality in this age group and the recently published UK National Institute for Health and Clinical Excellence (NICE) feverish illness guidelines for preschool children provide an algorithm to help differentiate those with serious illness. In an influenza pandemic a large number of children will present with fever caused by influenza but there remains a background risk of serious bacterial infection.
In the UK, a National Pandemic Flu Service was created, to be contacted by telephone directly by patients or parents. The national pandemic influenza algorithm aimed to identify patients with pandemic influenza. There have been reports of children with false-positive diagnoses of swine flu who in retrospect had more serious alternative diagnoses.2–4 Paediatricians are concerned that the pandemic influenza algorithm does not correctly differentiate young children with serious febrile illness from those with self-limiting viral illness. A delay could put children at risk of significant deterioration in their condition while they are not receiving appropriate treatment.
We hypothesised that children who were unwell with serious infection or illness during an influenza pandemic would be incorrectly identified as having pandemic influenza by the National Pandemic Flu Service swine flu clinical assessment algorithm5 resulting in delayed assessment by a secondary paediatrician when compared with the NICE feverish illness guideline.
We aimed to evaluate the efficacy of the swine flu algorithm as a screening tool in a population of unwell children and to compare the management advice given by the swine flu algorithm with the advice from the NICE feverish illness in children clinical guideline.6
Methods
We used two algorithms: the NICE guideline ‘Feverish illness in children—assessment and initial management in children younger than 5 years’ and the swine flu algorithm designed by the National Pandemic Flu Service.
We analysed all non-elective paediatric medical admissions with a fever, over the age of 1 year, to our unit during two calendar weeks in November. Children under the age of 1 year were excluded as the swine flu algorithm does not apply to these patients. For each child, pre-admission signs and symptoms were evaluated and those histories including confirmed fever, or a history of fever were put through both the algorithm and the NICE guideline if under 5 years of age. This work was undertaken by retrospective notes review and had no impact on patient care. The initial differential diagnosis, outcome of investigations and the final clinical diagnosis were recorded. These results were then used to calculate the sensitivity and specificity for the swine flu algorithm.
Results
There were 115 fever-related medical admissions to the paediatric unit during the 2-week study period, of whom 77 were over the age of 1 year. Complete data were available for 72 of these cases. On application of the swine flu algorithm to their presenting symptoms, 71 (99%) resulted in a diagnosis of swine flu (figure 1). There were 32 patients who had a negative test or had an alternative diagnosis confirmed by investigation. Two patients had confirmed H1N1 (pandemic influenza) on testing. The remaining 38 patients did not have testing to isolate the H1N1 pathogen and neither were any other tests positive.
Flow chart of 77 patients over the age of 1 year admitted with fever.
Each patient under the age of 5 years was also put through the NICE fever guideline, and consequent advice compared with that given by the swine flu algorithm (table 1). The NICE guideline classifies patients into a traffic light system, from ‘green’ for low risk of serious illness, through ‘amber’ for moderate risk to ‘red’ for the highest risk. For ‘green’ and ‘red’ outcomes, there was a reasonable correlation towards less urgent and more urgent care. However, for ‘amber’ outcomes, there was little correlation seen with the swine flu algorithm.
Advice given by the swine flu algorithm and the NICE fever guideline for the 57 children under the age of 5 years
We analysed the advice regarding further assessment given by the two different algorithms (table 2). Eighty-eight per cent of children admitted would have required to have a face-to-face assessment by a health professional under the NICE guideline. The swine flu guideline does not specify face-to-face assessments, but to ‘call GP urgently’; only 49% would have been asked to do so. Twenty-two (39%) children would have been advised to have a face-to-face assessment by the NICE guideline, when they would not have been advised to have an urgent consultation by the swine flu algorithm. Seven (12%) patients would not have been advised to have a face-to-face or urgent assessment by either system.
Rates of face-to-face and urgent assessment advised in the 57 admitted patients under the age of 5 years
The data were analysed for sensitivity, specificity, positive predictive value and positive and negative likelihood ratios. We performed a ‘what if?’ analysis, once with all the indeterminate diagnoses classified as not swine flu (‘worst case’), and once with all the indeterminate diagnoses classified as swine flu (‘best case’) (table 3).
Sensitivities, specificities, positive predictive value and likelihood ratios positive and negative for the swine flu algorithm.
Discussion
Our data show that the swine flu algorithm has significant limitations as a screening tool in children with a serious febrile illness. While it has a high sensitivity the very low specificity means it is unable to differentiate children with other illnesses. The management advice provided by the swine flu guidance is frequently at odds with that outlined in the NICE feverish illness guideline. The target populations of the two algorithms are very similar: unwell children in the community.
Those responsible for planning in the event of public health emergencies face an extremely difficult situation during a pandemic. Their aim must be to ensure the greatest health for the largest number of people. The institution of a population health measure, in this case the mass screening and treatment with respect to swine flu, must be with the aim of avoiding serious illnesses. Within populations, outliers on the illness severity spectrum can become buried within a large number of milder cases. In this case a screening test has been instituted, which in the important target population (seriously unwell children) has a positive likelihood ratio of between 0.975 and 1.01. As a screening tool, it falls well short of the accepted criteria in this population.7
We considered whether this algorithm is a good tool for identifying children in whom hospital treatment is necessary. We found that 28 out of 57 patients (49%) would have been asked to have an urgent assessment by the swine flu algorithm, compared with 88% who would have been advised to have a face-to-face consultation with the NICE fever guidance. Worryingly, this was in a population of children who were all unwell enough to be admitted, so the rates of recommended face-to-face or urgent consultation should be 100%. The 22 patients in whom urgent assessment was not advised by the swine flu algorithm, but were advised to have a face-to-face assessment by the NICE fever guidance, are most at risk. Could we in a future influenza pandemic use the same algorithm over and above the NICE fever guideline, when we have shown their advice to be so divergent in this unwell population?
There is the possibility that patients with severe pandemic flu or other viral illness have been treated in the community and have therefore not reached hospital.8 However, available evidence is that the illness attenuation of oseltamivir is mild.9 We cannot compare rates of such illnesses in the community, as we did not have access to community population data. While our retrospective data appear compelling, a prospective cohort study of children starting in the community would be a more rigorous methodology. Robust evaluation of any algorithms must be an integral part of future pandemic planning.
Of the 77 patients admitted, 57 (79%) had treatments that they could not have accessed at home, for instance intravenous antibiotics or oxygen therapy. False-positive diagnoses of pandemic flu may cause a delay in instituting the correct treatment. Although we cannot demonstrate it from our data, it is likely that such delays would have resulted in deterioration for many of these patients.
The NICE fever guidance has 289 citations, with a list of 14 named authors and the level of supporting evidence is clearly stated throughout the guideline. Despite our best efforts (communication with the National Pandemic Flu Service, the local Primary Care Trust, the Strategic Health Authority and the Department of Health), we were unable to identify either the authors of the swine flu algorithm or the level of evidence on which the algorithm is based, including important areas of controversy.
Conclusion
We have found that the pandemic swine flu guideline had a sensitivity of between 97.5 and 100%, but a specificity of between 0 and 1.4% for identifying swine flu in a population of children admitted to hospital during the swine flu epidemic. The positive likelihood ratio of the test is between 0.975 and 1.01. We estimate that at least 79% of these patients may have come to harm if treatment had been delayed due to misdiagnosis. It is concerning that when using the swine flu algorithm, the signs and symptoms of this unwell population resulted in only 49% being advised to have an urgent consultation, compared with 88% with the NICE fever guideline.
Population algorithms should consider the outcome that the intervention is intended to avoid. This population measure aims to avoid serious illness through swine flu. However, the very high false-positive rate means that it is very poor at differentiating causation in seriously unwell children. The algorithm should not have been geared to the diagnosis of any illness in the population, but to the diagnosis of serious illness, as there are significant risks of giving false-positive diagnoses. Publication of the data and rationale behind such measures, and subsequent rigorous evaluation, should be routine. When the construction of a safe algorithm proves impossible, then although inconvenient, face-to-face consultations may be the only way to ensure the safety of our patients reliably.
Footnotes
Competing interests None.
Provenance and peer review Not commissioned; externally peer reviewed.