Article Text

Download PDFPDF

Analysis of emergency department prediction tools in evaluating febrile young infants at risk for serious infections
  1. Sarah Hui Wen Yao1,2,
  2. Gene Yong-Kwang Ong1,3,
  3. Ian K Maconochie4,
  4. Khai Pin Lee1,3,
  5. Shu-Ling Chong1,3
  1. 1 Department of Emergency Medicine, KK Women’s and Children’s Hospital, Singapore, Singapore
  2. 2 Faculty of Medicine, Nursing and Health Sciences, Monash University, Clayton, Victoria, Australia
  3. 3 Duke-NUS Medical School, Singapore, Singapore
  4. 4 Paediatric Emergency Department, Imperial College Hospital NHS Healthcare Trust, London, UK
  1. Correspondence to Dr Shu-Ling Chong, Department of Emergency Medicine, KK Women's and Children’s Hospital, Singapore 229899, Singapore; Chong.Shu-Ling{at}kkh.com.sg

Abstract

Objective Febrile infants≤3 months old constitute a vulnerable group at risk of serious infections (SI). We aimed to (1) study the test performance of two clinical assessment tools—the National Institute for Health and Care Excellence (NICE) Traffic Light System and Severity Index Score (SIS) in predicting SI among all febrile young infants and (2) evaluate the performance of three low-risk criteria—the Rochester Criteria (RC), Philadelphia Criteria (PC) and Boston Criteria (BC) among well-looking febrile infants.

Methods A retrospective validation study was conducted. Serious illness included both bacterial and serious viral illness such as meningitis and encephalitis. We included febrile infants≤3 months old presenting to a paediatric emergency department in Singapore between March 2015 and February 2016. Infants were assigned to high-risk and low-risk groups for SI according to each of the five tools. We compared the performance of the NICE guideline and SIS at initial clinical assessment for all infants and the low-risk criteria—RC, PC and BC—among well-looking infants. We presented their performance using sensitivity, specificity, positive, negative predictive values and likelihood ratios.

Results Of 1057 infants analysed, 326 (30.8%) were diagnosed with SI. The NICE guideline had an overall sensitivity of 93.3% (95% CI 90.0 to 95.7), while the SIS had a sensitivity of 79.1% (95% CI 74.3 to 83.4). The incidence of SI was similar among infants who were well-looking and those who were not. Among the low-risk criteria, the RC performed with the highest sensitivity in infants aged 0–28 days (98.2%, 95% CI 90.3% to 100.0%) and 29–60 days (92.4%, 95% CI 86.0% to 96.5%), while the PC performed best in infants aged 61–90 days (100.0%, 95% CI 95.4% to 100.0%).

Conclusions The NICE guideline achieved high sensitivity in our study population, and the RC had the highest sensitivity in predicting for SI among well-appearing febrile infants. Prospective validation is required.

  • paediatrics, paediatric emergency medicine
  • triage
  • clinical assessment

Statistics from Altmetric.com

Request Permissions

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

Key messages

What is already known on this subject

  • Numerous observational studies have suggested the need to accurately identify febrile young infants ≤3 months old with serious infections (SI). Current low-risk criteria have been specifically set up to assess ‘well-looking’ infants. The National Institute for Health and Care Excellence (NICE) guideline and the Severity Index Score are two tools used in the general paediatric population; however, there is a paucity of literature specifically examining and comparing the effectiveness of these tools in young infants.

What this study adds

  • In this retrospective validation study of febrile infants ≤3 months old presenting to a paediatric emergency department in Singapore, the NICE guideline outperformed the Severity Index Score and performed satisfactorily even in the youngest febrile infants. In a subset of initially well-appearing infants at triage, the Rochester Criteria had the highest sensitivity compared with other low-risk criteria in predicting for SI in our population. The NICE guideline appears promising for the initial clinical assessment of febrile infants; however, prospective validation is required before these findings can be recommended for clinical application.

Introduction

The febrile infant ≤3 months old constitutes a common presentation to the paediatric emergency department (ED). While most recover uneventfully with a self-limiting viral course of illness, 7%–12.8% will have a serious bacterial infection (SBI).1 2 Of these, the prevalence of invasive bacterial infections is reportedly highest among infants≤28 days old.2 While there has been extensive research on young infants with SBI,3–6 viral infections, such as encephalitis, meningitis and viraemia, also require prompt recognition and management because they may result in death, cognitive deficits and hearing loss.7 It is imperative that ED systems accurately identify these infants at risk for serious infections (SI) of both viral and bacterial aetiologies to ensure timely intervention.

There is an undermet need for effective triage of febrile infants presenting to the ED. A triage system with high sensitivity will allow resources to be focused first on unwell and high-risk infants.8 Common paediatric systems include the National Institute of Health and Care Excellence (NICE) Traffic Light System of Clinical Risk Factors and the Severity Index Score (SIS).9–11 While they have been recommended for use in children under 5 years old,12 they have not been specifically studied in the youngest infants.

While high-risk infants who appear ill at triage should be prioritised and managed on an emergent basis, well-appearing young febrile infants comprise another group that deserve further study. After triage, these well-appearing febrile infants undergo a detailed physician assessment and often have initial laboratory investigations performed on them. Contention surrounds the need for invasive investigations and empirical management. Multiple studies have proposed the use of comprehensive low-risk criteria that use clinical history, physical examination findings and laboratory tests to enable the clinician to confidently identify the febrile infant at low risk of a SI.13 Among these, the Rochester Criteria (RC), Philadelphia Criteria (PC) and the Boston Criteria (BC) are most well-known.14 Literature on the performance of these criteria have emerged mainly out of North America and Europe.5 15–17

The aims of this study are (1) to evaluate the test performance of two clinical assessment tools (NICE guideline and SIS) as screening tools in the ED in predicting SI among all febrile infants≤3 months old and (2) to evaluate the test performance of three low-risk criteria (RC, PC and BC) in the subset of febrile infants who appeared well at triage.

Materials and methods

Study design and setting

A secondary analysis was performed on the data obtained from our primary study. The primary study was a retrospective cohort study evaluating the performance of heart rate guidelines in predicting SI, conducted from 1 March 2015 to 29 February 2016 at the ED of KK Women’s and Children’s Hospital (KKH) in Singapore.7 Singapore is a highly urbanised high income country in Asia with 4.8% of its population under 5 years’ old.18 KKH is the largest paediatric institution in Singapore, with an annual ED attendance of approximately 175 000 children.7

Participants

In our institution (KKH, Singapore), all febrile infants≤3 months old with an axillary temperature of ≥37.5°C at triage have their temperature rechecked during consultation. Infants with an axillary temperature of ≥38.0°C at the recheck were included in the study. All febrile infants≤3 months old are hospitalised for observation and investigations and hence forms a natural patient pool to study the outcomes of febrile young infants (figure 1). Neonates (≤28 days old) undergo blood, urine and cerebrospinal cultures and receive intravenous antibiotics, while older infants may only receive limited investigations depending on clinical assessment of risk. Investigations are done in the ED only for ill infants clinically assessed to require urgent laboratory tests and intravenous antibiotics, while majority of the infants have investigations performed after they are hospitalised. In most cases, the decision on the type and extent of investigations are made by physicians in the ward. All subjects in this study were enrolled in the primary study.7

Figure 1

Flow diagram of study population.

Variables

Abstracted data from the primary study reviewed in this study included pertinent clinical history, examination findings, investigations, outcome and management.

Assignment of risk stratification status for all febrile infants according to clinical assessment tools

For the study, all eligible infants were risk-stratified at triage according to the NICE guideline (figure 2) and SIS (figure 3). The SIS is the triage tool currently used in our institution. Both clinical assessment tools rely on subjective criteria such as colour and respiratory effort and do not include laboratory parameters as part of their assessment. The NICE guideline additionally uses physiological criteria—heart rate, respiratory rate and oxygen saturation to further risk stratify infants who do not meet the clinical criteria of green (low-risk) features of colour and respiratory effort (figure 2) to separate these patients into ‘red’ or ‘amber’. SIS is interpreted as: SIS 10 (not very sick), SIS 8 or 9 (moderately sick) and SIS ≤7 (very sick) (figure 3).

Figure 2

NICE traffic light system of clinical risk factors in infants younger than 3 months.11 NICE, National Institute of Health and Care Excellence.

Figure 3

Severity Index Score.9

For the purposes of analysis, infants were categorised as ‘low-risk’ or ‘high-risk’. Infants were categorised as ‘low-risk’ in the absence of red and amber features for the NICE guideline (figure 2).7 9 11 Infants with SIS<10 were identified as ‘high-risk’.

Assignment of risk stratification status among initially well-looking infants according to low-risk criteria

The determination of a well-looking infant was made by investigators retrospectively, based on the data recorded by the triage nurse at the time of attendance. We did not base the identification of well-looking infants on the NICE guideline nor the SIS, since these have not yet been validated in our population to identify well-looking infants and this study purported to assess their performance. Instead, we developed a set of criteria based on an extensive literature search:1 3 5 6 14 16 19–21 normal activity, colour, respiratory effort and vital signs, as documented by the triage nurse during triage. We defined the well-looking infant as an infant with normal activity and tone (no poor feeding, not lethargic, irritable or decreased response to stimuli, inconsolable), normal colour (not mottled, cyanotic or pale), normal respiratory effort and who had vital signs within normal limits.

The trained investigators then assigned ‘well-looking’ infants to ‘low-risk’ and ‘high-risk’ groups for SI based on the RC, PC and BC (online supplementary table 1), using a structured form in Microsoft Excel. As application of all three low-risk criteria required laboratory test results, infants with incomplete laboratory values were excluded from this analysis.5 14 15 Equivalent items within the datasheet were matched to the items comprising the criteria. As the RC contained some clinical parameters that had not been collected, approximations were used (online supplementary table 2). Information on faecal leucocytes in infants with diarrhoea, as required by the RC, was determined from stool culture results.22 The band-neutrophil count (in RC) and band-neutrophil ratio (in PC) are not routinely measured at our institution. Instead, the absolute neutrophil count (ANC) was used. This is substantiated by previous studies, which have suggested that the ANC may be more sensitive that the band-neutrophil count in predicting SI.4 23 Low-risk infants were required to have an ANC of 1–10.0×109/L.4 13 All other domains for the PC and BC were otherwise available in the dataset. Infants who scored ‘not low risk’ according to each criterion were considered high-risk for SI.

Supplemental material

Outcome measures

SI was defined as: urinary tract infections (UTI) (where positive urine cultures were defined as growth of a single organism >108 CFU/L from clean catch or in-out catheters), sepsis (defined as infection causing haemodynamic instability and/or end-organ dysfunction), bacteraemia (defined as bacterial growth in blood cultures—excluding common bacterial contaminants such as coagulase-negative staphylococcus, not requiring additional treatment), osteomyelitis, abscess, lobar pneumonia (reported by a radiologist), enterocolitis, encephalitis and meningitis (viral and bacterial).1 4 7 These diagnoses were identified using discharge ICD-10 diagnostic codes and subsequently confirmed by manual review of all the electronic medical records by trained investigators, in the initial data collection of the primary study.7 A trained independent investigator then performed a random audit of 5% of cases to confirm that the data entered was accurate and complete.

Statistics

All data were entered into a Microsoft Excel database. Statistical analysis was performed using IBM SPSS Statistics V.23.0. The study sample was described with descriptive statistics. Categorical data were summarised by percentages, while continuous variables were expressed as mean values±SD or median with IQR, depending on normality. The performance of each tool was presented using sensitivity, specificity, positive predictive value, negative predictive value (NPV), positive and negative likelihood ratios and compared for each age group—infants 0–28 days old, 29–60 days old and aged 61–90 days old. By pragmatic design, we included all infants who met the inclusion criteria during the year of study, instead of using a precalculated sample size.

Results

Characteristics of study subjects

Of the 2093 screened infants, 1057 infants were analysed for the outcome of SI in the primary study, all of whom were eligible for this study.7 The majority of the infants included in this analysis were 61–90 days old, while neonates 0–28 days old comprised the smallest proportion (table 1). Of the 1057 study infants, 326 (30.8%, 95% CI 28.1% to 33.7%) were diagnosed with SI.7 The prevalence of SI was 23.9% (95% CI 19.0 to 29.6) in infants≤28 days old, 37.3% (95% CI 32.6% to 42.3%) in infants 29–60 days old and 29.0% (95% CI 24.9 to 33.5) in infants 61–90 days old. UTI was the most common SI (54.9%), with E. coli and Klebsiella Pneumoniae as the most likely pathogens.7 This was followed by meningitis (38.9%), with Enterovirus and E. coli as the most common organisms and septicaemia (6.4%) from Group B Streptococcus (GBS), E. coli and K. Pneumoniae as causative organisms.7 The prevalence of each SI in each age group is outlined in table 2.

Table 1

Baseline characteristics and demographics of all infants7

Table 2

Diagnosis of SI in each age group

Main results

Evaluation of clinical assessment tools in all eligible infants

All domains for both clinical assessment tools were available in the dataset, and hence all subjects were assigned into ‘high-risk’ and ‘low-risk’ groups. The NICE guideline performed significantly better than the SIS for sensitivity (93.3%, 95% CI 90.0% to 95.7% and 79.1%, 95% CI 74.3% to 83.4%) and NPV (82.4, 95% CI 75.1% to 87.9% and 76.5%, 95% CI 71.9% to 80.5%), but had lower specificity (table 3). In the age-based analyses, the NICE guideline scored higher sensitivities across all ages (online supplementary table 3). The NICE guideline was better able to discriminate between high-risk and low-risk infants (prevalence of SI among high-risk infants was 32.6%, while low risk infants was 17.6%) compared with SIS (prevalence of SI among high-risk infants was 33.6%, while low risk infants was 23.5%) (table 3).When we analysed individual outcomes of SI, the NICE guideline misclassified 22 patients (including 11 with UTI, 6 meningitis and 2 bacteraemia), while the SIS misclassified 68 patients (including 37 with UTI, 22 meningitis and 3 bacteraemia). However, both tools correctly identified all septic infants.

Table 3

Performance parameters of clinical assessment tools for SI

Evaluation of low-risk criteria in well-looking infants

Nine hundred and twenty-nine (87.9%) infants were identified as well-looking infants, by study investigators and hence eligible for analyses with low-risk criteria. Two hundred and eighty-three (283/929 or 30.4%) had SI. Among the 929 infants, 798 (85.6%) had complete documentation using the RC, 603 (64.9%) for the PC and 604 (65.0%) for the BC (figure 1). Overall, the RC performed the best with a sensitivity for detecting SI (96.0%, 95% CI 93.0% to 98.0%) and NPV (88.2%, 95% CI 80.2% to 93.2%) (table 4).

Table 4

Performance parameters of low-risk criteria for detecting SI among well-appearing infants

The performance of each ‘low risk’ tool was analysed by age (online supplementary table 3). In infants aged ≤28 days, the RC performed with the highest sensitivity (98.2%, 95% CI 90.3% to 100.0%) and NPV (98.4%, 95% CI 89.7% to 99.8%). In infants aged 29–60 days, the RC again fared with the highest sensitivity (92.4%, 95% CI 86.0% to 95.6%). In infants aged 61–90 days, the PC performed with the highest sensitivity (100.0%, 95% CI 95.4% to 100.0%) followed closely by the RC (99.0%, 95% CI 95.4% to 100.0%).

The RC missed 10 infants with viral meningitis (1 aged ≤28 days, 9 aged 29–60 days). Although both PC and BC identified all cases of bacterial meningitis, the PC missed 11 cases of viral meningitis, while the BC missed 33. Additionally, the BC missed 19 infants with UTI and 1 with bacteraemia. However, all criteria successfully identified the septic infants.

Discussion

We retrospectively assessed the performance parameters of various ED tools in a large Asian cohort of febrile infants≤3 months presenting to the paediatric ED. SI occurred in 30.8% of our study sample. As a clinical assessment tool, the NICE guideline performed significantly better than the SIS. Of the low-risk criteria applied to well-looking infants, the RC performed best with the highest sensitivity and NPV and accurately identified all infants with septicaemia. When examining the age-based generalisability of the low-risk criteria, the RC, designed for infants≤60 days old,5 14 15 had excellent performance across all age groups in this study population. The PC, although designed for infants aged 29–56 days, was least sensitive in that age group and instead most sensitive in infants aged 61–90 days.

Our study demonstrated a higher prevalence of SI than previously reported.1 3 4 7 While most studies traditionally include only SBI, we believe that a definition of SI that also includes viral aetiologies is more pragmatic because conditions such as viral meningoencephalitis may have long-term sequelae of cognitive and neurological deficits. We found that the prevalence of SI was highest in infants aged 29–60 days, despite 28 days being the conventional high-risk cut-off,13 emphasising the need to exercise caution in assessment beyond the neonatal period. The higher prevalence of SI in the 29–60 day age group differs from other large studies.2 This could be due to a high follow-up rate for booked pregnancies in Singapore with robust prophylaxis for maternal GBS. Additionally, our definition of SI included both bacterial and viral meningitis, the latter being more common among infants 29 days and older (table 2).

Notably, we found the rates of SI to be comparable among clinically ‘well-looking’ infants (30.4%) and those who were not ‘well-looking’ (33.6%). Our findings complement that of the Paediatric Emergency Care Applied Research Network,3 which found that clinical assessment alone was insufficiently sensitive in identifying high-risk infants. Repeated clinical assessments with careful monitoring of vital signs must be performed to detect deterioration even among infants who may initially appear ‘well’.

Clinical scores are used at triage with the aim of screening these febrile infants and directing resources first to the sickest patients.3 Examples of commonly used clinical assessment scores include the SIS, Yale Observation Score and Paediatric Assessment Triangle (PAT).24 Of note, the PAT is widely used for the paediatric population. The PAT consists of three parameters—appearance, work of breathing and circulation, sharing common domains with the SIS, which although used in our ED, is less commonly used internationally.9 The SIS did not perform satisfactorily as a triage tool in our study. We note that tools that consist of broad combinations of ‘red-flag’ features, such as poor reactivity and abnormal skin colour, will naturally have limited utility as these classic findings only appear late in the course of disease. Young infants are also limited to the basic functional activities of feeding, sleeping, crying and social smiling, resulting in significant limitation in the objective, accurate application of ‘play’ to this young age group. This could account for the poorer performance of the SIS in our study.

On the other hand, systems such as the NICE guideline are more thorough with the inclusion of physiological criteria such as heart rate and oxygen saturation and clinical findings such as respiratory effort and reduced urine output.3 7 This likely accounts for the NICE guideline’s good performance in our study.3 7 Importantly, the utility of clinical assessment could differ based on the level of training and experience of each physician or nurse involved. Further research must be done to ascertain its implications on the clinical assessment of risk.

The NICE guideline missed two infants with bacteraemia and two infants with bacterial meningitis. The NICE guideline also missed a substantial proportion of UTI in our study, as found in previous evaluations of the tool.10 This stresses the importance of screening the urine even in well-appearing febrile infants.

While ill infants are attended to immediately, well-looking infants form another diagnostic dilemma. A survey by the American Academy of Paediatrics found that the RC, PC and BC are the most common strategies for managing the well-looking febrile infant without focal infection.5 13 The RC and PC have been evaluated to have high NPV, which enables clinicians to confidently identify low-risk infants.5 Both the RC and PC performed with similar high sensitivities for all age groups. When applied to the youngest infants aged ≤28 days old, the RC performed with much higher specificity compared with the PC (41.5%, 95% CI 33.4% to 49.9% vs 8.4%, 95% CI 3.7% to 15.9%) and a lower negative likelihood ratio (-LR 0.04, 95% CI 0.0% to 0.5% vs 0.68, 95% CI 0.5% to 1.0%). The RC performed well across all age groups in our study population, even beyond its intended age limit for infants<60 days. The PC includes more comprehensive laboratory parameters, including a mandatory lumbar puncture. This could suggest that the RC would be more practical to implement in routine clinical settings and may potentially reduce the number of unnecessary and invasive investigations.

Limitations

The primary limitation of this study is its retrospective design. We recognise that retrospective test validation is less ideal than prospective test validation, as clinical variables may be less accurate or missing. Data analysis was performed based on information abstracted from the electronic health record. Study investigators were not blinded to the purpose of the study, the clinical presentation, or the SI status when assessing the risk status for each tool. However, investigators were trained to perform structured data entry using a standardised template. Second, we excluded infants with incomplete laboratory data who could not be scored with the low-risk criteria. This may have introduced bias, because infants who did not receive complete laboratory workup would potentially be those who had remained very well clinically while in the ward. Exclusion of these presumably well infants could have inflated the sensitivities of the low-risk criteria in our study as well as the overall prevalence of SI in our study, hence limiting the accuracy of our findings. Finally, we did not perform a true comparison to the RC, PC and BC as certain laboratory elements were unavailable locally although we defined approximations to our best ability.

Conclusion

Our study has important implications on assessing young febrile infants in the paediatric ED. In our study, the NICE guideline outperformed the SIS in the triage of febrile infants≤3 months old. We emphasise the need to exercise caution beyond the neonatal period, where rates of certain SI such as viral meningitis may be higher. For well-appearing infants at triage, the RC had the highest sensitivity among low-risk criteria in predicting for SI in our population. Prospective validation is required before these findings can be recommended for clinical application.

Acknowledgments

We would like to thank Ms Lau Yew Ping for assisting us in obtaining the data from the Electronic Health Records. We would also like to thank Dr Wendy Yi Wen Chin, Dr John Mingzhou Chua, Dr Praseetha Nair, Dr Alicia Shu Zhen Ong and Associate Professor Kee Chong Ng, the original authors of our primary paper for their assistance with the data collection on the study.

References

Footnotes

  • Contributors Conceptualisation: SHWY, S-LC. Data curation: SHWY, GY-KO, S-LC. Formal analysis: SHWY, S-LC. Methodology: SHWY, S-LC. Project administration: S-LC. Resources: GY-KO, KPL, S-LC. Supervision: GY-KO, IKM, KPL, S-LC. Validation: GY-KO. Writing—original draft: SHWY. Writing—review and editing: SHWY, GY-KO, IKM, KPL, S-LC.

  • Funding The authors have not declared a specific grant for this research from any funding agency in the public, commercial or not-for-profit sectors.

  • Competing interests None declared.

  • Patient consent for publication Not required.

  • Ethics approval Ethics approval was granted by the Singapore SingHealth Institutional Review Board. Informed consent was waived as this was a retrospective analysis.

  • Provenance and peer review Not commissioned; externally peer reviewed.

  • Data availability statement Data are available on reasonable request.

Linked Articles