Article Text

PDF

Comparing CATCH, CHALICE and PECARN clinical decision rules for paediatric head injuries
  1. Mark D Lyttle1,
  2. Louise Crowe2,
  3. Ed Oakley2,3,
  4. Joel Dunning4,
  5. Franz E Babl1,2,5
  1. 1Emergency Department, Royal Children's Hospital, Melbourne, Australia
  2. 2Murdoch Children's Research Institute, Melbourne, Australia
  3. 3Emergency Department, Monash Medical Centre, Clayton, Australia
  4. 4James Cook University Hospital, Middlesbrough, UK
  5. 5The University of Melbourne, Melbourne, Australia
  1. Correspondence to Dr M D Lyttle, Emergency Department, Bristol Royal Hospital for Children, Bristol, UK; mark.lyttle{at}uhbristol.nhs.uk

Abstract

Many children present to emergency departments following head injury (HI), with a small number at risk of avoidable poor outcome. Difficulty identifying such children, coupled with increased availability of cranial CT, has led to variation in practice and increased CT rates. Clinical decision rules (CDRs) have been derived for paediatric HI but there is no published comparison to assist in deciding which to implement. The content of the three of highest quality and accuracy are described and compared. Systematic reviews of paediatric HI CDRs were published in 2009 and 2011. To identify CDRs published since the most recent review, key databases were searched, selecting studies which included CDRs involving children aged 0–18 years with a history of HI. Quality was assessed using the Quality Assessment of Diagnostic Accuracy Studies Tool, and performance evaluated by reported accuracy. Three high quality CDRs were identified: CATCH (Canadian Assessment of Tomography for Childhood Head Injury) CHALICE (Children's Head Injury Algorithm for the Prediction of Important Clinical Events) and PECARN (Paediatric Emergency Care Applied Research Network). All were derived with high methodological standards but differed in key areas, including study population, outcomes and severity of HI. Each stated different predictor variables and only PECARN provided a separate algorithm for young children. CATCH and CHALICE identify children requiring CT and PECARN those who do not. All perform with high sensitivity and low specificity. PECARN is the only validated CDR, and none has undergone impact analysis. These three CDRs should undergo validation and comparison in a single population, with analysis of their impact on practice and financial implications, to aid relevant bodies in deciding which to implement.

View Full Text

Statistics from Altmetric.com

Introduction

In developed countries, injury is the leading cause of death and neurological disability among children,1 with head injury (HI) being the most common injury type.2 The annual incidence of HI is estimated at 1850/100 000 for 0–4 year olds, 1100/100 000 for 5–9 year olds and 1170/100 000 for 10–14 year olds.3 HI is therefore a very common reason for presenting to emergency departments (EDs), although the vast majority (80–90%) are mild in severity.4–7 However, among these children there are a number at risk of avoidable poor outcome.2 ,6 ,8–11 Assessment in the ED focuses on identifying injuries which are potentially life threatening or require neurosurgical intervention.

Cranial CT is currently the reference standard investigation for intracranial injury. It provides rapid identification and guides management, with early diagnostic imaging linked to improved outcomes and reduction in admission rates.12 However, it does not detect all brain injuries and may detect coincidental abnormalities.2 ,8 ,13 Cranial CT also poses a degree of risk to the patient. The lifetime cancer mortality risk attributable to the ionising radiation dose from a single cranial CT is about 1 in 1500 in a 1 year old and about 1 in 5000 in a 10 year old.14–16 Exposure of the infant brain to ionising radiation may influence cognitive abilities in adulthood.17 Children with HI may be non-compliant either due to fear or agitation, and may require sedation, which carries risks of airway and haemodynamic compromise.18 ,19

A lack of evidence to aid in the identification of children with significant injury, and clinician concern regarding missing such an injury, has given rise to uncertainty as to which patients require investigation. This, coupled with increased availability and reduction in the time it takes to perform cranial CT, has led to substantial variation in practice and an increase in cranial CT rates. The number performed in North America has more than doubled since the early 1990s,10 ,20–23 despite the fact that a large proportion in this setting are normal.10

To optimise the balance between identifying significant injury and minimising exposure to radiation, several clinical decision rules (CDRs) have been derived. CDRs are decision making tools derived from original research (as opposed to consensus based clinical practice guidelines) which incorporate three or more variables from history, physical examination or simple tests. They help clinicians cope with the uncertainty of medical decision making and improve their efficiency.24–26 CDRs for HI have been derived and widely incorporated into adult practice. Several specific to paediatric HI have been derived but have been less widely incorporated into practice due to variable quality and heterogeneity.

Two recent systematic reviews of CDRs for paediatric HI have been performed.10 ,12 The first, in 2009,10 reviewed CDRs for HI of all severities, and identified two of high quality and performance: CHALICE (Children's Head Injury Algorithm for the Prediction of Important Clinical Events),9 and the CDR derived by Palchak27; the second, published by Pickering in 2010,12 reviewed CDRs for minor HI, and identified the CDR derived by the Paediatric Emergency Care Applied Research Network (PECARN)2 as being the best validated CDR. Pickering planned a meta-analysis of rules that had been independently validated in multiple homogenous cohorts but found that no rule had been evaluated to this extent and therefore this meta-analysis could not be undertaken.12

At present, there is no published direct comparison of CDRs to aid EDs, institutions, or regional or national bodies, in deciding which to implement. In this article, we therefore describe, compare and contrast the content and quality of the three CDRs for paediatric HI which have been derived using the highest methodological standards in order to provide a more complete understanding of their details.

Methods

CDR selection methods

To identify CDRs derived with high methodological quality published since the most recent reviews,10 ,12 we searched MEDLINE, EMBASE, CINAHL, CENTRAL and the EBM Review to February 2011. In addition, the references of identified CDRs were searched manually. The search method allowed for articles written in any language. Two reviewers (ML and FEB) independently assessed potentially relevant articles using a two step process. The title and abstract from each article identified were assessed for inclusion; publications identified as potentially relevant were reviewed manually. When discrepancy existed, studies were discussed and included by consensus. Blinding of journal, institution and author was not performed.

Only studies that derived, validated or assessed the impact of a CDR in children aged 0–18 years with a history of HI were included. Studies that involved both adults and children were included if a separate data analysis was performed for children. A CDR was defined as a clinical decision making tool that included three or more variables obtained from the history, physical examination or simple diagnostic tests, providing the probability of an outcome or suggesting a course of action for an individual patient, that was not a decision analysis, decision support tool or practice guideline.

The methodological quality of included studies was assessed using the Quality Assessment of Diagnostic Accuracy Studies Tool.28 Methodological quality assessment and data extraction were performed by one reviewer (MDL) and checked by another (FEB). The opinion of a third reviewer (EO) was sought in cases of discrepancy.

Using this method, we identified an additional CDR of high quality and performance (CATCH, Canadian Assessment of Tomography for Childhood Head Injury),8 resulting in a total of four CDRs for analysis.

CATCH8 was derived by the Paediatric Emergency Research Canada group (box 1). CHALICE9 was derived by the UK Emergency Medicine Research Group (box 2). The remaining two CDRs were derived in the USA.2 ,27 The decision rule for identifying children at low risk for brain injuries after blunt head trauma was published in 2003 and was a single centre pilot study authored by Palchak et al.27 The prediction rule for identification of children at very low risk of clinically important traumatic brain injury (ciTBI)2 was chosen for inclusion in preference to the pilot study as it: (1) was conducted by the same group of authors, (2) is more recent chronologically, (3) is larger in sample size, (4) has an age differentiation, (5) has multicentre involvement and (6) has been validated. It was reported by PECARN, and is referred to as the PECARN2 CDR in this article (figures 1 and 2).

Box 1

Canadian Assessment of Tomography for Childhood Head Injury (CATCH)

CT of the head is required only for children with minor head injury* and any one of the following findings:

High risk (need for neurological intervention)

  • 1. Glasgow Coma Scale score <15 at 2 h after injury

  • 2. Suspected open or depressed skull fracture

  • 3. History of worsening headache

  • 4. Irritability on examination

Medium risk (brain injury on CT scan)

  • 5. Any sign of basal skull fracture (eg, haemotympanum, ‘raccoon’ eyes, otorrhoea or rhinorrhoea of the CSF, Battle's sign)

  • 6. Large, boggy haematoma of the scalp

  • 7. Dangerous mechanism of injury (eg, motor vehicle crash, fall from elevation ≥3 ft (≥91 cm) or 5 stairs, fall from bicycle with no helmet)

*Minor head injury is defined as injury within the past 24 h associated with witnessed loss of consciousness, definite amnesia, witnessed disorientation, persistent vomiting (more than one episode) or persistent irritability (in a child <2 years of age) in a patient with a Glasgow Coma Scale score of 13–15.

©Access Copyright, The Canadian Copyright Licensing Agency. Osmond MH, et al.8 This work is protected by copyright and the making of this copy was with the permission of Access Copyright. Any alteration of its content or further copying in any form whatsoever is strictly prohibited unless otherwise permitted by law.

Box 2

Children's Head Injury Algorithm for the Prediction of Important Clinical Events Rule (CHALICE)

A CT scan is required if any of the following criteria are present

  • History

    • Witnessed loss of consciousness of >5 min duration

    • History of amnesia (either antegrade or retrograde) of >5 min duration

    • Abnormal drowsiness (defined as drowsiness in excess of that expected by the examining doctor)

    • ≥ 3 vomits after head injury (a vomit is defined as a single discrete episode of vomiting)

    • Suspicion of non-accidental injury (NAI, defined as any suspicion of NAI by the examining doctor)

    • Seizure after head injury in a patient who has no history of epilepsy

  • Examination

    • Glasgow Coma Score (GCS) <14, or GCS <15 if <1 year old

    • Suspicion of penetrating or depressed skull injury or tense fontanelle

    • Sign of a basal skull fracture (defined as evidence of blood or CSF from ear or nose, panda eyes, Battle's sign, haemotympanum, facial crepitus or serious facial injury)

    • Positive focal neurology (defined as any focal neurology, including motor, sensory, coordination or reflex abnormality)

    • Presence of bruise, swelling or laceration >5 cm if <1 year old

  • Mechanism

    • High speed road traffic accident either as pedestrian, cyclist or occupant (defined as accident with speed >40 m/h)

    • Fall of >3 m in height

    • High speed injury from a projectile or an object

If none of the above variables are present, the patient is at low risk of intracranial pathology.

Reproduced from Dunning J, et al.9 Copyright 2006, with permission from BMJ Publishing Group Ltd.

Figure 1

Paediatric Emergency Care Applied Research Network (PECARN) rule for children <2 years. ciTBI, clinically important traumatic brain injury. *This box indicates children at very low risk of ciTBI in whom CT scans could be obviated. Reprinted from The Lancet, Vol 374, Kuppermann N, Holmes JF, Dayan PS et al, Identification of children at very low risk of clinically-important brain injuries after heads trauma: a prospective cohort study, pages 1160–1170, Copyright 2011, with permission from Elsevier.

Figure 2

Paediatric Emergency Care Applied Research Network (PECARN) rule for children ≥2 years. ciTBI, clinically important traumatic brain injury. *This box indicates children at very low risk of ciTBI in whom CT scans could be obviated. Reprinted from The Lancet, Vol 374, Kuppermann N, Holmes JF, Dayan PS et al, Identification of children at very low risk of clinically-important brain injuries after heads trauma: a prospective cohort study, pages 1160–1170, Copyright 2011, with permission from Elsevier.

CDR comparison methods

Methodological standards for the development of CDRs have been described previously.24–26 ,29 Stiell et al25 provide a comprehensive report outlining six key stages to be undertaken in the development and testing of a CDR for use in emergency medicine. We have used this as a framework to compare and contrast the CATCH,8 CHALICE9 and PECARN2 CDRs.

Evaluation and comparison of the three clinical decision rules

Stage 1: Is there a need for the clinical decision rule?

This is demonstrated by reviewing the prevalence of the clinical condition, current use of the diagnostic test, variation in practice, and attitudes and clinical accuracy of physicians.25 All CDRs highlighted the absence of a well derived, evidence based and widely accepted CDR for management of paediatric HI, and clearly described need in terms of their own population.

CATCH8 focused on children with ‘minor HI’ (a history of loss of consciousness, amnesia or disorientation in a patient who is conscious and responsive in the ED (Glasgow Coma Score (GCS) 13–15)), quoting 650 000 such ED attendances annually in North America. They highlighted increasing cranial CT rates, and discussed risks of exposing more children to radiation, consequent increased healthcare costs, persistent low yield from cranial CT in this population and the potential to miss intracranial injuries despite performing cranial CT.

CHALICE9 quoted 500 000 ED attendances annually in the UK for children with all severity of HI in whom there is a low prevalence of life threatening complications. Prior to CHALICE9 derivation there were no good quality studies on the management of paediatric HI.

PECARN2 quoted over 600 000 ED attendances, 60 000 hospital admissions and 7400 deaths annually due to paediatric HI in the USA. They focused on children with ‘minor HI’ (GCS 14–15) and indicated cranial CT as the reference standard investigation. They highlighted increasing cranial CT rates despite associated risks, low yield and the possibility of false positives, and described variation in practice and the need for separate CDRs for different age groups as younger children are more difficult to assess and more sensitive to radiation.

Stage 2: Was the clinical decision rule derived according to methodological standards?

This is assessed by reviewing definition of outcome, definition and reliability of predictor variables, selection of subjects, sample size, mathematical techniques, and sensibility and accuracy of the CDR.25

Definition of outcome

The outcome should be both clinically important and clearly defined.25 All groups clearly defined outcomes (table 1) and explained how these were decided. There are important similarities and differences in the definitions and terminology which each employed.

Table 1

Comparison of outcomes defined in derivation of the clinical decision rules

Each defined a different primary outcome. CATCH8 used ‘need for neurological intervention’, defined as death or specified procedures secondary to the HI within 7 days. CHALICE9 used ‘clinically significant intracranial injury’, a composite of death due to HI, neurosurgical intervention or marked abnormalities on cranial CT. PECARN2 used ciTBI, a composite of death from TBI, neurosurgery, intubation for >24 h for TBI or hospital admission of ≥2 nights associated with TBI on cranial CT.

All included defined traumatic changes on cranial CT. In CATCH,8 this was the secondary outcome, while in CHALICE9 and PECARN2 they were included in the composite primary outcome. All included depressed skull fractures although CHALICE9 was alone in including non-depressed fractures in the secondary outcome.

CHALICE9 and PECARN2 included hospital admission (secondary outcome in CHALICE9 and component of primary outcome in PECARN2). Stiell states “as this outcome is often dependent on local factors, it may be difficult to replicate reliably, and therefore may not be of high clinical relevance”.25 PECARN,2 in order to decrease the effects of local factors, only included patients admitted ‘for ≥2 nights for TBI in association with TBI on cranial CT’. They stated that admission had to be for ‘persistent neurological symptoms or signs such as persistent alteration of mental status, recurrent emesis due to HI, persistent severe headache or ongoing seizure management’.

Definition of predictor variables

Potential predictor variables should be clearly defined and collected in a standardised prospective manner, and physicians must be adequately trained to evaluate patients and collect data using a proforma developed especially for the task. To avoid observational bias this proforma should be completed before the outcome is known.25

CATCH8 delivered a 1 h training session to enable prospective collection of data on 26 standardised predictor variables. These were selected by a team of investigators using current literature and pilot study results. Data collection sheets were completed prior to cranial CT. Radiologists reviewing cranial CTs were unaware of their content.

CHALICE9 delivered a 1 h training session and collected information on ‘around 40’ predictor variables on a specially designed proforma. These were selected by performing systematic reviews of both adult and paediatric HI (personal communication, J Dunning, September 2011). Clinicians were not always blinded to the outcome of neuroimaging before completing the proforma, and radiologists were not blinded to clinical information.

PECARN2 collected data on 34 predictor variables which had previously been found to be important or had biological plausibility. These were identified from previous work performed by the group27 and a comprehensive literature review of CDRs for TBI (personal communication, N Kuppermann, September 2011). Data were collected by trained site investigators and other (trained) ED physicians prior to cranial CT using a specially designed proforma. All clinicians and research coordinators undertook a standardised training session, reinforced through monthly conference calls, a detailed study instruction manual and regular site monitoring visits. The study design stated that the radiologist interpreting the initial cranial CT should be blinded to clinical data, and site radiologists did not have access to patient case report forms. However, it was impossible to monitor whether radiologists ever had any patient clinical information at the time of cranial CT interpretation (personal communication, N Kuppermann, September 2011). A dedicated study radiologist blinded to clinical data made definitive interpretations of inconclusive CTs.

Reliability of predictor variables

For a CDR to be dependable, the predictor variables must display good interobserver reliability. Only those with agreement beyond that of chance alone should be considered for inclusion.25

CATCH8 performed independent assessment by a second emergency physician where feasible (a total of 333 patients, 8.6%), including for further analysis predictor variables with a κ value >0.5. Kappa values for variables created by cut-point (eg, amnesia ≥30 min before injury) or those collected from the medical record (eg, age) were not calculated.

In CHALICE,9 100 patients had an independent assessment by another emergency physician within 30 min of the first assessment (personal communication, J Dunning, September 2011). Potential predictor variables with a κ value >0.6 were included for further analysis.

In PECARN,2 4% of patients had a separate independent assessment by another emergency physician within 60 min of the first assessment. Potential predictor variables with a κ value of ≥0.5 and lower bound of the one sided 95% CI of ≥0.4 were included for further analysis.

Selection of subjects

When applying a CDR, it is important that this occurs in a patient group and location similar to that in which it was derived.25 There were important differences between the CDR derivation populations (table 2).

Table 2

Comparison of patient selection criteria in the derivation of the clinical decision rules

All were derived using data from a large number of centres. CATCH8 was derived using data solely from Canadian tertiary paediatric EDs. CHALICE9 and PECARN2 were derived from data collected in both tertiary paediatric and other EDs in the UK and USA, respectively. The upper age limit differed, with CATCH8 including 0–16 years, CHALICE9 <16 years and two age categories in PECARN,2 which were <2 years and ≥2 years to <18 years.

Each detailed inclusion and exclusion criteria based on previous medical history, mechanism of injury, presenting history and clinical examination findings. These varied and reflected the different focus of the groups (table 2). CHALICE9 derivation was performed on a broad range of patients with few inclusion or exclusion criteria as they aimed to produce a CDR for all children presenting after HI. CATCH8 and PECARN2 limited their populations, although with different criteria, as their aim was to derive a CDR to assist clinicians managing children in whom the decision of whether to perform cranial CT is not clear cut. They used their inclusion and exclusion criteria to focus on those with ‘minor HI’, excluding both those with more severe HI and those in whom HI is so mild that no physician would consider a cranial CT necessary.

While most of the criteria were clear, CATCH8 excluded children with chronic developmental delay, and PECARN2 excluded children with a pre-existing neurological disorder complicating assessment. This categorisation was at the discretion of the treating clinician in both studies (personal communication, M Osmond, N Kuppermann, September 2011).

All reported the number of eligible patients who were missed, and compared their characteristics to the enrolled patients.

Sample size

The number of subjects enrolled must be appropriate for the type of multivariate analysis used, large enough to avoid problems with overfitting of data and will determine the degree of precision in the CI.25

CATCH8 enrolled 3866 patients based on a desired precision of 99% sensitivity to detect brain injury with a 95% CI between 97% and 100% (personal communication, M Osmond, September 2011). In the assessment of the derived CDR, they employed the bootstrapping method to evaluate classification performance and assess overfitting of the model. This method generates many alternative versions of a given statistic that would ordinarily be calculated, thus providing an estimate of the distribution of the statistic. It is an appropriate way to control and check the stability of results, and is of particular use when a sample size is insufficient for straightforward statistical inference or when performing power calculations from a small sample. They described the small number (n=277) of children <2 years of age enrolled in their study as a limitation.

CHALICE9 enrolled 22 772 patients, stating the calculation of the original projected required sample size (7500 patients over 1 year) and adjustments made at interim analysis to reach final sample size and study duration.

PECARN2 enrolled 33 875 patients in their derivation, split into age subgroups of <2 years (n=8502) and ≥2 years (n=25 283). The sample size was determined by balancing risk in order to create a CDR where the low risk population would have a risk of ciTBI less than the risk of radiation induced malignancy (personal communication, N Kuppermann, September 2011). They stated that ‘the current study is very large, allowing sufficient statistical power to generate robust and generalisable rules’.

Mathematical techniques

The three groups stated that their primary aim was to derive a CDR which performed with high sensitivity. The ideal approach in this case is to perform multivariate analysis using recursive partitioning (with screening of variables by assessment of reproducibility and association with outcome).25

While there were some differences in the univariate analysis, all performed multivariate analysis using recursive partitioning. PECARN2 also described the methods used to create their prediction trees, with assignation of a relative cost of 500 to 1 for failure to identify a patient with ciTBI versus incorrect classification of a patient without ciTBI.

Sensibility of the decision rule

In determining sensibility, one must decide whether the CDR is clinically reasonable, easy to use and provides a course of action. This is a subjective judgement, rather than one based on the methods employed in the derivation.25

The predictor variables from each CDR are presented for comparison (table 3). These would be acceptable to most clinicians and are therefore clinically reasonable. Many appeared in more than one CDR, although often with differences in definition, or cut-point for time related or episodic variables. The inclusion criteria in CATCH8 contained some predictor variables of the other CDRs.

Table 3

Comparison of the predictor variables in the three clinical decision rules

All predictor variables are easily obtained, and are mostly clear and well defined. ‘Irritability on examination’ in CATCH8 has been criticised as being ambiguous.30 ‘Abnormal drowsiness’ in CHALICE9 and ‘acting normally as per parent’ in PECARN2 could be similarly criticised. However, all predictor variables were analysed for interobserver reliability. These showed good agreement, hence their inclusion.

The three CDRs are easy to use and provide clear courses of action. CATCH8 required clear criteria indicative of minor HI to be present before application of seven predictor variables. It provided a clear course of action, stating that cranial CT is required only if the inclusion criteria were met and any predictor variables were present.

CHALICE9 provided 14 predictor variables which could be applied to any severity of HI at any time after HI, and stated that cranial CT is required if any predictor variables were present.

PECARN2 is easy to use although its purpose differs significantly from the others. Its aim was to identify children at very low risk of ciTBI after HI who did not require cranial CT. Six predictor variables for age subgroups were presented in prediction tree format, reporting risk of ciTBI in each branch. Outlined in the final branch were the children in whom cranial CT is unnecessary, therefore providing a clear course of action. For children not in this low risk group, clinicians can combine the data presented and their clinical judgement to decide whether or not to perform cranial CT.

Derivation accuracy

The groups reported to varying degrees the sensitivity, specificity, negative predictive value and positive predictive value of their CDRs in the derivation populations (table 4).

Table 4

Accuracy of the clinical decision rules in the derivation groups

To fully assess accuracy, all patients would ideally undergo the reference standard investigation (cranial CT).25 Clinicians would find this approach unreasonable, and a proxy measure of outcome can therefore be employed through rigorous follow-up as patient centred outcomes overcome the imperfect sensitivity and specificity of cranial CT.12 ,25 In the three derivations, follow-up was performed, but with different methodology. CATCH8 performed telephone follow-up on those who had no imaging at 14 days post injury (by a nurse who was unaware of the initial clinical data) and excluded from further analysis the 245 patients it was not possible to contact. CHALICE9 did not complete telephone follow-up but stated that they reinforced their composite end point with a prospective, thorough follow-up strategy, designed to ensure that no children who died as a result of HI or who had late neurosurgical intervention were missed. PECARN2 performed telephone follow-up of patients discharged from the ED between 7 and 90 days after the ED visit, and classified the outcome accordingly. If they were unable to establish contact they accepted review of medical record, ED process improvement records and county morgue records as a robust method of ensuring that no discharged patient was subsequently diagnosed with ciTBI.

As each CDR was derived in a different population using different outcomes, age limits, and inclusion and exclusion criteria, it is not possible to directly compare them in all areas as these variables inherently affect the reported accuracy of the CDR.31 However, some key areas are amenable to comparison.

Each group derived their CDR to perform with high sensitivity, a reasonable strategy as clinicians would not accept into practice a CDR which failed to detect all children with a significant injury. All performed with very high sensitivity although CATCH8 is the only one which performed with 100% sensitivity for its primary outcome. This approach means that a lower specificity must be accepted, which may influence cranial CT rates. Projected CT rates differed, ranging from 14% (CHALICE9) to 50% (CATCH8), but the CDRs targeted different HI severity and populations, making direct comparison of these rates impossible (table 5).

Table 5

Projected rate of cranial CT if clinical decision rules are applied

CHALICE9 and PECARN2 reported accuracy in the form ‘no predictors present versus any predictor present’, while CATCH8 reported accuracy in two formats: accuracy of four high risk predictor variables when applied to the primary outcome and accuracy of all seven predictor variables when applied to the secondary outcome.

CHALICE9 reported accuracy when their CDR was applied to the patient subgroup with GCS 13–15. However, this cannot be directly compared with either CATCH8 or PECARN2 due to differing inclusion and exclusion criteria.

CATCH8 (<2 years) and CHALICE9 (<1 year) reported age dependent predictor variables for younger children but did not report age specific accuracy. PECARN2 developed separate algorithms and reported accuracy separately for children <2 years and ≥2 years.

Stage 3: Has the rule been prospectively validated and refined?

Validation is crucial as many statistically well derived CDRs fail to perform well when tested in a new population due to statistical problems from the derived model, differences in disease prevalence or difference in the CDR application. To validate a CDR it should be applied prospectively to a completely new patient population, representative of a wide spectrum of ages and severity of the condition, selected in an unbiased fashion. Investigators performing the validation must ensure that the CDR is appropriately understood and applied.25 As in the derivation, all patients cannot be expected to undergo cranial CT, and the proxy measure of outcome through rigorous follow-up is acceptable. Similar statistical analysis should be undertaken, including analysis of interobserver agreement, and accuracy should be reported as in the derivation. Validation permits CDR refinement and assessment of potential impact on the target population.25 ,26

The only CDR which has undergone validation is PECARN.2 This was a multicentre validation in a population which was diverse but identical to the derivation population. Following completion of enrolment to the derivation stage, enrolment of patients continued in participating centres, and these data were used to perform the validation. The CDR displays high sensitivity in both age groups in validation (table 6). There has not been refinement of the CDR following validation but the authors stated that if it were applied, cranial CT rates in patients with minor HI would decrease by 24% in those <2 years and by 20% in those aged ≥2 years.

Table 6

Accuracy of the PECARN clinical decision rule in the validation group

CATCH8 and CHALICE9 stated the need for their CDRs to be prospectively validated and refined but to date this has not been reported for either CDR. However, validation of the CATCH study has been completed with a report expected soon (personal communication, M Osmond, September 2011). No CDR has been validated outside its derivation setting.

CHALICE9 has been retrospectively applied in two populations different to its derivation.32 ,33 The first applied CHALICE9 retrospectively to children presenting with HI to a tertiary paediatric ED in Australia and reported an actual CT rate of 19% and a projected cranial CT rate of 46% when CHALICE9 was applied. Of 303 patients who fitted CHALICE9 criteria and did not have cranial CT, 198 were discharged from the ED; five re-presented with persisting symptoms and received cranial CT; four were abnormal but none required neurosurgery. The authors stated that CHALICE9 would increase cranial CT rate with apparent small gain in earlier identification of abnormalities. They did not comment on markers of accuracy.32 The second retrospectively applied the 2007 guideline for management of HI produced by the National Institute for Health and Clinical Excellence (NICE)34 (based on CHALICE9) to children presenting to a district general (mixed) ED in the UK, and reported a cranial CT rate of 10.5% had the guideline been applied. Markers of accuracy of the CDR in this population were not reported.33 These cranial CT rates can be compared with the projected rate reported by CHALICE9 of 13.3%.

Stage 4: Has the rule been successfully implemented into clinical practice?

CHALICE9 has been widely implemented into clinical practice in its derivation setting (the UK) in a national guideline for management of HI provided by NICE.34 CATCH8 and PECARN2 have not yet been widely implemented.

Implementation studies have not been performed for any of the CDRs to determine the ability to change physician decision making, or assess their effects, accuracy or acceptability.

Stage 5. Would use of the rule be cost effective?

Formal economic evaluations were not reported by any group to demonstrate either healthcare savings or increased financial burden that would be created by their application.

Stage 6. How will the clinical decision rule be disseminated?

None discussed their strategy for dissemination of the derived CDRs.

Discussion

CDRs have proven beneficial to clinicians in assessing a range of conditions in the ED.10 ,25 ,35 ,36 Several have been derived to aid in the management of paediatric HI although none has been widely incorporated into clinical practice. We have therefore provided a comparison of the three most promising in this area: CATCH,8 CHALICE9 and PECARN.2 All were derived with high methodological standards in large populations in multicentre settings using good quality statistical methods and modelling techniques. Any could be implemented into practice as they are easy to use but the question of which is the best to implement remains as they differ in many key areas, and the shift from identifying any lesion on CT to focusing on clinically significant lesions has made results more difficult to compare for researchers and clinicians.12

They were derived using different outcomes, age limits, and inclusion and exclusion criteria, and focused on different severities of HI. Although CATCH8 and PECARN2 both focused on minor HI, their definition was very different. Despite large cohorts, the prevalence of neurosurgical injury varies, ranging from 0.11% to 3.4%, highlighting their heterogeneity.12 While all assessed similar potential predictor variables, those which were significant (and which were therefore included in the final CDR) differ, with GCS being the only variable present in all, although with different cut-points. ‘Signs of basilar skull fracture’ is the next most common, present in CATCH,8 CHALICE9 and PECARN2 for children aged ≥2 years. Others were present in more than one CDR but often with a different format, description or cut-point. For example, vomiting is included as either any vomiting, >1 episode, ≥3 episodes or not included.

A fundamental difference is in their suggested course of action, as CATCH8 and CHALICE9 were derived to identify children who require cranial CT, while PECARN2 was derived to identify those who do not. However, one is not simply the inverse of the other. CATCH8 and CHALICE9 suggest a dichotomous course of action; perform cranial CT if any predictor variable is present, do not if none is present. The course of action suggested by PECARN2 is more complex. It stated that if no predictor variables were present then cranial CT is unnecessary. However, it is not the case that if any predictor variable is present then cranial CT should be performed. Rather, they provided an expression of risk of intracranial injury correlated with each predictor variable. PECARN2 designed their CDR to identify the group of patients in whom the risk of ciTBI was lower than the risk of radiation induced lethal malignancy. They deliberately left much to clinician discretion for those not in this low risk group in order to reflect practice as different clinicians, patients and parents have different risk thresholds and preferences. In their paper, they provided a suggested decision making algorithm in which the outcomes are ‘CT recommended’, ‘observation versus CT on the basis of other clinical factors’ and ‘CT not recommended’. However, this algorithm should not be read as being the PECARN2 CDR.

Some of the difficulties in deriving paediatric HI CDRs are the variability in clinical signs and symptoms from birth to adolescence, difficulty in confidently assessing younger children and increased sensitivity of the developing brain to ionising radiation.15 ,16 ,37 Some authors have therefore suggested that different CDRs should exist for preverbal (<2 years) and verbal (≥2 years) children. CATCH8 and CHALICE9 included some age dependent predictor variables, but PECARN2 was the only CDR to outline separate pathways for these age groups.

All CDRs were derived to perform with very high sensitivity, creating lower specificity with a consequent impact on projected cranial CT rates. Because the CDRs have been derived to manage different severities of HI, it is impossible to directly compare these projected rates. The resource implications and population at risk of radiation can therefore only be established by directly comparing the CDRs in the same patient population.

The potential impact of a CDR can be estimated by assessing its predictive validity and clinical sensibility and by measuring its potential to improve decision making. However, the actual impact of a CDR will depend on how its predictions are translated into decisions and how clinician input is effectively incorporated before, during and after testing in actual practice.26

Current practice for some children in whom there is clinical uncertainty includes a period of observation to assess the evolution of their condition. This is a powerful tool in deciding whether a child has a significant injury, and due to the balance of risk and benefit, clinicians may prefer this course of action rather than perform early cranial CT, even in the presence of one or more predictor variables. CHALICE9 and PECARN2 both highlighted the role of observation, with PECARN2 recommending practitioner discretion in deciding whether to observe patients or perform cranial CT for those at moderate risk of ciTBI. A recent publication by the PECARN group analysing their population concluded that clinical observation may be an effective strategy to reduce cranial CT rates.38 The practice of observation may therefore have an impact on the introduction of any of these CDRs.

There is a clear need for the widespread implementation of a CDR for paediatric HI given the large number of presentations, potential for avoidable poor outcome, difference in practice and concerns regarding increased cranial CT rates.20–23 While the three CDRs compared in this paper present potentially powerful tools to improve clinical decision making, their heterogeneity, coupled with their comparable accuracy, create an air of uncertainty as to which would be the most appropriate to implement. While this uncertainty exists, it is unlikely that any will be definitively incorporated into practice.

These three CDRs should therefore undergo a process of prospective validation and comparison in a single population. This should incorporate analysis of the impact that each would have on practice, and the financial implications of implementing one rule over another. This will make it possible to decide which, if any, of these CDRs is the best fit in a given healthcare system.

Conclusions

We have provided a detailed analysis of the methodological standards and content of three high quality CDRs for paediatric HI. While these CDRs are superficially similar, they were derived in and targeted towards different populations, have considerable differences in their suggested action and employ different predictor variables. These three CDRs should now undergo validation and comparison in a single setting.

References

View Abstract

Footnotes

  • Funding The study was supported in part by grant funding from the Murdoch Children's Research Institute (MCRI), Melbourne, Australia. MCRI is a non-governmental paediatric research institute. There was no influence on the part of the funder on the design or conduct of the study.

  • Competing interests None.

  • Provenance and peer review Not commissioned; externally peer reviewed.

Request Permissions

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.