Article Text
Abstract
Objective Misdiagnosis of abusive head trauma (AHT) has serious consequences for children and families. This systematic review identifies and compares clinical prediction rules (CPredRs) assisting clinicians in assessing suspected AHT.
Design We searched MEDLINE, Embase, PubMed and Cochrane databases (January 1996 to August 2016). Externally validated CPredRs focusing on the detection of AHT in the clinical setting were included.
Results Of 110 potential articles identified, three studies met the inclusion criteria: the Pediatric Brain Injury Research Network (PediBIRN) 4-Variable AHT CPredR, the Predicting Abusive Head Trauma (PredAHT) tool and the Pittsburgh Infant Brain Injury Score (PIBIS). The CPredRs were designed for different populations and purposes: PediBIRN: intensive care unit admissions (<3 years) with head injury, to inform early decisions to launch or forego an evaluation for abuse (sensitivity 0.96); PredAHT: hospital admissions (<3 years) with intracranial injury, to assist clinicians in discussions with child abuse specialists (sensitivity 0.72); and PIBIS: well-appearing children (<1 year) in the emergency department with no history of trauma, temperature <38.3°C, and ≥1 symptom associated with high risk of AHT, to determine the need for a head CT scan (sensitivity 0.93). There was little overlap between the predictive variables.
Conclusion Three CPredRs for AHT were relevant at different stages in the diagnostic process. None of the CPredRs aimed to diagnose AHT but to act as aids/prompts to clinicians to seek further clinical, social or forensic information. None were widely validated in multiple settings. To assess safety and effectiveness in clinical practice, impact analyses are required and recommended.
- child abuse
Statistics from Altmetric.com
What is already known on this topic?
Abusive head trauma (AHT) has high morbidity and mortality; if AHT is missed, patients are at increased risk of further injury and death.
Clinical prediction rules (CPredRs) may assist in deciding which head injured children might have sustained injury as a result of AHT.
What this study adds?
Three recently validated CPredRs for AHT investigate very different populations, focus on different stages of the diagnostic process and use differing predictive and outcome variables.
The aim of the CPredRs was not to diagnose AHT but to act as aids or prompts to seek further information, investigation and assessment.
Introduction
Abusive head trauma (AHT) is a leading cause of traumatic death in children less than 1 year of age and the most common cause of fatal child abuse.1 Children with AHT have an estimated mortality rate of 26%2 and long-term disability ranging from 44%3 to 92%4 among survivors.5 6 Patients are at increased risk of further injury and death if AHT is missed.7 8 However, deciding which children should undergo a full evaluation for AHT is difficult, as histories provided by the caregiver might be absent or fabricated, and the clinical findings may be similar to those seen in accidental trauma.9 Clinicians might hesitate to raise suspicion of AHT, as a wrongful accusation means emotional strain for the families, endangers patient–doctor relationships and leads to unnecessary investigation-related costs and risks for the child.10
Clinical prediction rules (CPredRs) are evidence-based tools, which incorporate three or more variables from clinical findings including history, physical examination and results of investigations, to predict aetiology or outcome.11 They are especially important for conditions where decision-making is difficult, clinical stakes are high and clinical experience and intuition are limited.12 There are three main phases in the development of CPredRs: derivation, external validation and impact analysis.13 Each requires a different and rigorous methodological approach. In the initial phase, predicted probabilities are derived from the statistical analysis of patients with known outcomes, typically using multivariable regression techniques or classification and regression tree analysis.13 Predictor variables should be clinically sensible and clearly defined,14 and the data set used to derive the rule should be representative of the target population. Reilly and Evans11 distinguish between assistive prediction rules that simply provide clinicians with predicted probabilities without recommending a specific clinical course of action, and directive decision rules that explicitly suggest additional diagnostic tests or treatment in line with the obtained score.11 CPredRs must be validated with a data set external to the one in which it was derived, preferably in multiple settings, and tested in clinical practice to determine their impact on patient care.15
There have been few systematic reviews in this field that explore the quality and effectiveness of CPredRs in child abuse. While Louwers et al compared several screening techniques for child abuse in emergency departments (ED)16 we set out to find and critically appraise CPredRs that aim to detect AHT across various medical settings and compare them in terms of their quality and performance.
Methods
We conducted a systematic review to identify CPredRs for AHT and to compare them in terms of derivation, population, definition of AHT, variables used, external validation and performance. For this purpose, we followed a protocol based on the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) statement (figure 1; for PRISMA checklist see online supplementary table 1).17 Our review was registered with the PROSPERO international prospective register of systematic reviews (http://www.crd.york.ac.uk/; record number CRD42017058141).
Supplementary file 1
Inclusion/exclusion criteria
We included externally validated CPredRs with a focus on the detection of AHT in children from 0 to 18 years of age in the clinical setting. Papers that reported associations between one or two variables or markers and AHT were excluded, as well as clinical assessment tools trying to detect child abuse in general.
Search strategy
We searched the electronic databases MEDLINE, the Cochrane Database of Systematic Reviews, Cochrane Controlled Trials (1996 to August 2016) and Embase (1974 to August 2016) using the Ovid and PubMed platforms. No limitations were applied for languages. The search strategy including search terms is presented in the online supplementary appendix.
Study selection
Duplicates were removed and relevant abstracts were reviewed. When eligibility could not be determined from the abstract, a full-text review was performed. When there were uncertainties about inclusion, other lead authors were consulted (LC, FEB).
Assessment of quality
Maguire et al 18 proposed 17 quality items for the development and validation of CPredRs for children in their systematic review. We chose this approach to compare the methodological quality of the CPredRs, calculating a score (table 2) according to the number of quality standards achieved.
Data analysis
It became apparent early on that the included CPredRs were heterogeneous and a meta-analysis was unlikely to be possible, therefore a descriptive analysis is provided.
Results
The search in Medline resulted in 74 articles and in Embase 54 articles were found. The search in PubMed and the Cochrane Database of Systematic Reviews and Cochrane Controlled Trials did not add any additional relevant studies. After removing duplicates, reviewing abstracts and excluding irrelevant articles, three recently published, externally validated CPredRs for AHT met the inclusion criteria: the Pediatric Brain Injury Research Network (PediBIRN) 4-Variable AHT CPredR,19 the Predicting Abusive Head Trauma (PredAHT) tool20 and the Pittsburgh Infant Brain Injury Score (PIBIS) for AHT21 (see figure 1 and table 1).
Pediatric Brain Injury Research Network clinical prediction rule
The PediBIRN CPredR19 was designed to ‘inform [pediatric intensivists’] early decisions to launch (or forego) an evaluation for abuse.’10 It was derived in a prospective study including 209 children published in 2013 (95 AHT cases)10 and validated in a further prospective study comprising 291 children (125 AHT cases), published in 2014.19 The validation study took place in 14 hospitals within the USA, of which 10 sites had been part of the derivation study. The population of interest was that of acutely head-injured children aged <3 years admitted to the paediatric intensive care unit (PICU). Following a bivariate analysis of 45 potential factors for discrimination and reliability the authors applied a classification tree to their data using binary recursive partitioning to derive a 4-variable CPredR with maximum sensitivity to determine the risk of AHT. The four variables were respiratory compromise, bruising involving the ears, neck and torso, bilateral/interhemispheric subdural haemorrhages and skull fractures (other than an isolated, unilateral, non-diastatic, linear, parietal skull fracture). If one or more of the variables were present, the child should be thoroughly evaluated for abuse. In the validation study, the CPredR achieved a sensitivity of 0.96 and a specificity of 0.46 to detect AHT cases.
Predicting Abusive Head Trauma tool
The PredAHT CPredR provides an estimated probability of AHT to assist clinicians in discussions with child abuse specialists.20 It was derived from a pooled analysis of individual patient data from six previously published studies, which included prospectively collected data on 133 children (58 AHT cases) and retrospective data on 920 children (290 AHT cases) with head injury.22 PredAHT identified the positive predictive values and ORs for AHT given any combination of six possible variables using multilevel logistic regression. The validation study comprised a retrospective data set of 60 children (23 AHT cases) in Cardiff, UK, and prospective data on 138 children (42 AHT cases) from Lille, France.23 The Welsh data set included children <3 years of age with intracranial injuries confirmed on neuroimaging, who were admitted to hospital, whereas the French cases comprised patients <2 years old with craniocerebral traumatic lesions diagnosed on CT and referred alive to the ED, PICU or neurosurgical department. Missing data were accounted for using multiple imputation by chained equations. PredAHT gave probabilities of AHT that ranged from 4% if none of the variables were present to close to 100% when all six variables were present.22 Following validation, the probability of AHT was always greater than 81.5% if any three or more of the predictive variables—head/neck bruising, seizures, apnoea, rib fractures, long-bone fractures or retinal haemorrhages—were present. In the validation study the CPredR performed with a sensitivity of 0.72 and a specificity of 0.86 in detecting AHT using a cut-off probability of AHT of 50%.
Pittsburgh Infant Brain Injury Score
The PIBIS for AHT21 assists in determining which high-risk infants in the ED should undergo head CT to rule out abnormalities including AHT. It was derived based on retrospective data on 187 children (150 without brain injury and 37 with mild AHT), which were not published, and validated using logistic regression in a prospective study carried out in the USA with 1040 infants. Missing data were handled with listwise deletion. The included sample was of well-appearing children between 30 and 364 days of age with a temperature <38.3°C, no history of trauma and at least one symptom associated with high risk of AHT (apparent life-threatening event/apnoea, vomiting without diarrhoea, seizures/seizure-like activity, soft tissue swelling of the scalp, bruising or other non-specific neurological symptoms such as lethargy, fussiness or poor feeding). If children received a score of 2 points or more when adding: abnormality such as bruises observed on dermatologic examination (2 points), age equal to or greater than 3 months (1 point), head circumference >85th percentile (1 point) and haemoglobin <11.3 g/dL (1 point), further neuroimaging should be performed. In the validation study, the CPredR performed with a sensitivity of 0.93 and a specificity of 0.53 to detect abnormal neuroimaging. It is important to stress that the outcome case definition did not exclusively comprise AHT, but included other clinically significant traumatic and non-traumatic abnormalities.
Comparing the rules
All three CPredRs defined their outcome and predictive variables clearly, reported their results adequately and used 95% CIs on rule properties. Using a standardised approach18 the PediBIRN CPredR19 received the highest score for methodological quality (table 2). It was the only CPredR that described an independent blinded assessment of predictive and outcome variables and an evaluation of inter-rater reliability in the derivation study, comparing the assessment of blinded duplicate data on 20% of included patients by different investigators.10 The PIBIS study conducted the only follow-up of cases (6 months after enrolment or up to 1 year of age) to identify further abnormal neuroimaging and assess the progress of symptoms at presentation.10 This approach was an attempt to verify the true negatives as only 61% of controls had neuroimaging. The PediBIRN and the PIBIS CPredRs both proposed a clear course of action (PediBIRN—thorough evaluations for abuse, PIBIS—CT scan), whereas PredAHT provided a probability of AHT in order to ‘assist clinicians in their discussions with child abuse specialists, in addition to facilitating discussions between child abuse specialists and social welfare, law enforcement, or other professionals involved in the child protection process.’20
Discussion
We identified three validated CPredRs that met inclusion criteria. These prediction rules are aimed at very different populations and different time points within the clinical assessment (figure 2): PIBIS21 is targeted at a specific population of well-appearing infants in the ED who might benefit from a head CT scan. PredAHT20 applies to children <3 years of age admitted to hospital with intracranial injury, where children have been examined and may have had some ophthalmological and radiological investigations, and PediBIRN19 applies to a narrower population of <3-year-olds admitted to PICU with a cranial or intracranial injury excluding head trauma resulting from motor vehicle collisions. It is notable that PredAHT20 does not apply to children with cranial injury only, as defined within the Centers for Disease Control and Prevention definition of AHT.
Just as the populations are different, so are the predictor variables prima facie. As apparent in table 3, the only overlap among the items used by the different CPredRs are cutaneous injuries and respiratory compromise/apnoea. However, some of the PIBIS high-risk AHT symptoms used for the inclusion of patients (apnoea, seizures, bruising) appeared among the predictor variables of PediBIRN and PredAHT. Beyond that, both of these CPredRs had tested some of the same predictive variables in their derivation studies (eg, PediBIRN: seizures and PredAHT: skull fractures), yet they had not significantly improved their CPredR’s performance.10 22
In terms of published accuracy of the rules, the PediBIRN CPredR19 performed best with a sensitivity of 0.96, closely followed by PIBIS21 with 0.93, whereas the PredAHT CPredR detected 0.72 of AHT cases applying a 50% cut-off; however, the PredAHT tool provides a sliding scale of probability from 4% to nearly 100% depending on the presence or absence of each of the six features.20 In addition, the sensitivity and specificity of PredAHT depends on the probability cut-off applied. The pretest prevalence in the PediBIRN population19 was 0.43, which raises the question whether all children in this high-risk group should be screened. In the PredAHT and PIBIS studies the pretest probabilities were lower at 0.33 and 0.26, respectively.20 21
The PIBIS21 variables are all available from physical examination, simple blood test and history, whereas neuroimaging is required for PediBIRN,19 and further investigations such as ophthalmological examination and rib and long-bone X-rays are required for PredAHT20 (table 3). These tests would usually only be performed if physical abuse or serious trauma had already been suspected placing PredAHT as a potentially useful tool for assessing the significance of the results of these investigations at a specific stage in the diagnostic process.
Due to the lack of gold standard diagnostic criteria for AHT, different approaches were chosen to minimise circular reasoning where AHT may have been decided based on the presence of the predictor variables within the CPredRs. The PediBIRN CPredR aimed to avoid definitional criteria that involved intracranial injuries, injury severity, any of the predictor variables and child protection team assessment.19 When challenged on the issue that bruising was included in their definitional criteria and as a predictor variable24 the authors stated that of 73 patients with bruising, 61 met other definitional criteria based on the history and the 12 remaining patients were subsequently diagnosed with definite/probable AHT by the treating physicians25 The PredAHT group applied the outcome of the child protection process including only cases where multidisciplinary or court proceedings had confirmed AHT.20 Arguably this decision will include a consideration of clinical features, as is the case in any clinical diagnosis. However, within the child protection social care and legal process, there are multiple additional forensic, clinical, social and historical factors that are included in decisions about the balance of probability of abuse and future risks for the child. Similar reasoning applies to PIBIS where the child protection team assessment decision was also used.21 Regarding PIBIS, the true negatives could not be identified as only 61% of controls had neuroimaging. Case follow-up was undertaken for 6–12 months to determine if any neurological imaging was performed at a later stage and cases potentially missed.
None of the CPredRs have yet been widely validated in multiple settings or undergone an impact analysis to determine their safety and effectiveness in clinical practice. Hymel et al have undertaken a theoretical impact analysis of PediBIRN in the combined population of the derivation and validation study.26 Of note, because the PIBIS rule as originally derived was updated in the validation study, ideally this CPredR should be validated in another external data set before it can be applied to new patients.27
As the three CPredRs apply at different time points in the diagnostic process, in different populations and with a different degree of investigations completed, this explains and allows for differences in sensitivity and specificity. At the outset, it is paramount to ensure that cases are not missed and undergo sufficient investigation, therefore high sensitivity is the focus, with specificity of lower importance. When more investigations have been undertaken, a higher specificity would be desirable to ensure that a diagnosis of AHT is not made incorrectly. This suggests that the three CPredRs might complement each other in clinical practice. For instance, if a well-appearing infant between 30 and 364 days of age met the inclusion criteria and predictive values of PIBIS underwent neuroimaging and had an intracranial injury, further investigations should include skeletal survey and ophthalmological exam, providing more of the items required for PredAHT. In the critically ill children admitted to PICU, the PediBIRN rule would apply and would be useful to inform decision-making at a time when the child may be too sick to undergo a skeletal survey.
Limitations
The number of included studies was small and a meta-analysis was not possible. A further CPredR by Wells et al on the radiological differentiation of intentional and non-intentional intracranial haemorrhages was excluded, as it had not been externally validated.28
Conclusion
The three CPredRs for AHT focus on different populations with different inclusion criteria. They use different predictive variables available at different stages in the diagnostic process, and different outcome variables. PediBIRN aims to rule out AHT in the PICU. PredAHT calculates the probability of AHT for hospitalised children. PIBIS aims to detect abnormal neuroimaging in the ED. None of the CPredRs aimed to diagnose AHT but to act as aids or prompts to clinicians to seek further clinical, social or forensic information and move towards a multidisciplinary child protection assessment should more information in support of AHT arise. Wider validation in multiple settings is recommended for each CPredR, in addition to impact analyses to assess their safety and effectiveness in clinical practice.
Acknowledgments
We thank Ms Poh Chua, librarian, Library Service, Royal Children’s Hospital, Melbourne, Australia, for her assistance in the literature search.
References
Footnotes
Contributors HP contributed to the design of the study, conducted the systematic review, and drafted and revised the article. LC, AMK and LEC contributed to the design of the study, reviewed the search, made substantial contributions to the interpretation and discussion of findings and critically revised the manuscript for important intellectual content. JASS contributed to the design of the study and critically revised the manuscript for important intellectual content. FEB had the initial study idea, contributed to the design of the study and critically revised the manuscript for important intellectual content. FEB takes responsibility for the paper as a whole.
Funding The study was in part funded by grants from the National Health and Medical Research Council (Centre of Research Excellence for Paediatric Emergency Medicine GNT1058560), Canberra, Australia; the Murdoch Children’s Research Institute, Melbourne, Australia; and supported by the Victorian Government’s Infrastructure Support Program, Melbourne, Australia. FEB’s time was in part funded by a grant from the Royal Children’s Hospital Foundation and a Melbourne Campus Clinician Scientist Fellowship, Melbourne, Australia and an NHMRC Practitioner Fellowship.
Competing interests AMK and LEC were involved with the development of one of the clinical prediction rules described in this paper.
Provenance and peer review Not commissioned; externally peer reviewed.
Data sharing statement The search strategy as well as the PRISMA checklists are available as supplementary files.