Objective The validated Predicting Abusive Head Trauma (PredAHT) clinical prediction tool calculates the probability of abusive head trauma (AHT) in children <3 years of age who have sustained intracranial injuries (ICIs) identified on neuroimaging, based on combinations of six clinical features: head/neck bruising, seizures, apnoea, rib fracture, long bone fracture and retinal haemorrhages. PredAHT version 2 enables a probability calculation when information regarding any of the six features is absent. We aimed to externally validate PredAHT-2 in an Australian/New Zealand population.
Methods This is a secondary analysis of a prospective multicentre study of paediatric head injuries conducted between April 2011 and November 2014. We extracted data on patients with possible AHT at five tertiary paediatric centres and included all children <3 years of age admitted to hospital who had sustained ICI identified on neuroimaging. We assigned cases as positive for AHT, negative for AHT or having indeterminate outcome following multidisciplinary review. The estimated probability of AHT for each case was calculated using PredAHT-2, blinded to outcome. Tool performance measures were calculated, with 95% CIs.
Results Of 87 ICI cases, 27 (31%) were positive for AHT; 45 (52%) were negative for AHT and 15 (17%) had indeterminate outcome. Using a probability cut-off of 50%, excluding indeterminate cases, PredAHT-2 had a sensitivity of 74% (95% CI 54% t o89%) and a specificity of 87% (95% CI 73% to 95%) for AHT. Positive predictive value was 77% (95% CI 56% to 91%), negative predictive value was 85% (95% CI 71% to 94%) and the area under the curve was 0.80 (95% CI 0.68 to 0.92).
Conclusion PredAHT-2 demonstrated reasonably high point sensitivity and specificity when externally validated in an Australian/New Zealand population. Performance was similar to that in the original validation study.
Trial registration number ACTRN12614000463673.
- trauma, head
- non accidental injury
Statistics from Altmetric.com
What is already known on this subject
Predicting Abusive Head Trauma version 2 (PredAHT-2) is one of only two published prediction rules determining the likelihood of abusive head trauma in children <3 years old with intracranial injury.
PredAHT-2 has not been validated in a broader setting outside Europe.
What this study adds
We externally validated the prediction tool in an Australian/New Zealand dataset.
The tool performed with reasonably high sensitivity (74% (95% CI 54% to 89%)) and specificity (87% (95% CI 73% to 95%)) using a 50% probability cut-off.
Performance was similar to that in the original validation study.
Abusive head trauma (AHT) continues to be a major cause of traumatic deaths and long-term morbidity in infants due to child abuse.1–3 Ascribing AHT as the cause of an intracranial injury (ICI) is challenging for clinicians as the differential diagnosis may include abuse, accidental trauma or other childhood diseases.4 Perpetrators of AHT may deny abuse or offer alternative explanations as to what happened, and the presenting history is frequently inaccurate or incomplete.4 The consequences of a missed diagnosis of AHT can put the child in increased danger and risk the child’s life and future well-being5 6; equally, a false accusation of abuse can have devastating consequences for the child and family. The validity of AHT as a medical diagnosis is constantly questioned and any evidence regarding which combinations of clinical features are associated with a diagnosis of AHT or accidental trauma can support decision making and lend weight to the diagnostic process.
Clinical prediction rules (CPRs) are evidence-based tools that combine clinical features, history or results of investigations to predict diagnosis, prognosis or response to therapy.7 They may assist clinicians in making complex decisions, improving their accuracy and decreasing variability between clinicians.7 In 2011 and 2015, a Welsh team of experts in child protection research derived and validated the Predicting Abusive Head Trauma (PredAHT) tool, which calculates the probability of AHT in children <3 years of age with ICI based on different combinations of six clinical features (head or neck bruising, seizure, apnoea, rib fracture, long bone fracture and retinal haemorrhage, detailed in figure 1).8 9
The PredAHT derivation study provided predicted probabilities of AHT for all 64 possible combinations of the presence or absence of these features.9 In the validation study, using a 50% probability cut-off, PredAHT performed with a sensitivity of 72% and a specificity of 86% in identifying AHT.8 In order to address one or more missing elements of clinical features, the authors used their derivation dataset9 to create PredAHT version 2 (PredAHT-2).10–12 The probability of AHT was estimated when one or more of the six clinical features were unknown, using multiple imputation by chained equations.11 PredAHT-2 thus provides predicted probabilities of AHT for all 729 possible permutations of the six clinical features, depending on whether each is present, absent or unknown (see online supplementary appendix 1 for all possible permutations). In the clinical setting, PredAHT-2 can therefore provide a probability calculation when certain investigations such as ophthalmology or skeletal survey have not yet been undertaken. This is important in clinical practice as clinicians might hesitate to expose the child to extensive tests without an already high suspicion of AHT; skeletal surveys are associated with radiation exposure, and an ophthalmology examination can be difficult in young children. When applied to the validation dataset,8 using a 50% probability cut-off, the sensitivity and specificity of PredAHT-2 were 72% and 86%, respectively (unpublished data). However, despite validation at two of the original derivation sites, PredAHT-2 requires validation in multiple locations and by independent investigators.13 14
We describe an external validation of PredAHT-2 on an Australian/New Zealand population, of children <3 years of age admitted to hospital who have sustained ICI confirmed on neuroimaging.8 9
Study design, setting and patients
This study was a planned secondary subgroup analysis of children enrolled into a prospective multicentre observational study of children (0–<18 years) with head injuries in 10 Australian and New Zealand paediatric EDs between April 2011 and November 2014.15 16 We obtained ethics approval from five participating sites (Australia: Royal Children’s Hospital Melbourne, Monash Children’s Hospital, Clayton, and Perth Children's Hospital, Perth; and New Zealand: Starship Children’s Hospitals and KidzFirst Children’s Hospitals, Auckland) for additional medical record review of possible cases of abusive head trauma.
Full details of the primary study protocol are described elsewhere.16 In short, the parent study aimed to externally validate and compare three clinical decision rules, which assist in determining the need for CT in head-injured children. Children and young people with head injury were enrolled by the treating ED clinician who collected clinical data and a research assistant recorded ED and hospital management data following the visit. We collected injury and clinical variables, demographic and epidemiological information as well as information about neuroimaging, admission and neurosurgery. In this study, we analysed data from a subset of children <3 years of age admitted to hospital with ICI identified on neuroimaging (regardless of whether AHT was considered in the radiology request or subsequent radiology report or not), as this represented a high-risk group in which the differential diagnosis of AHT should be considered.
Patients who did not undergo neuroimaging or who had normal neuroimaging results were excluded. Patients with skull fracture with no accompanying ICI and those with an underlying structural abnormality or pre-existing disease (hydrocephalus, cystic lesion or tumour, metabolic cause, malformation or abnormal brain development), injuries caused by neglect or birth injuries were excluded as in the original PredAHT validation study.8
Strategy to identify possible AHT cases
In order to identify all possible AHT cases from the parent study,17 we extracted records of all children aged <3 years of age admitted to the hospital with head injury and abnormal neuroimaging results, excluding injuries due to motor vehicle accidents. In addition, at The Royal Children’s Hospital, Melbourne (5372 (40.2%) of 13 371 patients enrolled at the five sites), we accessed the database of the Victorian Forensic Paediatric Medical Service, the hospital child protection team, which we searched for all children aged <3 years of age admitted to the hospital with head injury and abnormal neuroimaging results.
We then accessed the medical records of all possible AHT cases at five sites and abstracted relevant data including predictor variables, outcomes and eligibility criteria for PredAHT.8
We used senior radiologists’ reports to determine the results of neuroimaging. ICI was defined as any combination of: any extra-axial haemorrhage, diffuse or focal parenchymal injury, cerebral oedema, cerebral contusion, hypoxic ischaemic injury or diffuse axonal injury visible on head CT or MRI.8
AHT was defined as the diagnosis of ICI (confirmed on neuroimaging), which was due to physical child abuse by parents or caregivers, rather than neglect, according to the decision of a multidisciplinary child protection team at the conclusion of their investigation. This decision was based on the consideration of the relevant social, forensic and clinical features in the context of the presenting history, in accordance with the Australian and New Zealand standard child protection assessment processes. Non-AHT was defined as ICI following a witnessed accidental injury or an accidental injury confirmed by the decision of a multidisciplinary child protection team. Cases were categorised as AHT positive or AHT negative (non-AHT) by the study investigators on retrospective review of the multidisciplinary team records. Cases in which this categorisation was not clear were deemed indeterminate.18 Any uncertainty in terms of category assignment on review of the records was arbitrated by the director of the Victorian Forensic Paediatric Medical Service (AS) on the basis of the forensic reports and medical records. In Australia and New Zealand, skeletal survey and retinal examination are routinely used as part of the work up for suspected non-accidental injuries
Application of PredAHT-2 to the dataset
We applied PredAHT-2 to each child <3 years of age with ICI confirmed on neuroimaging8 blinded to the forensic outcome categorisation. We calculated the specific probability of AHT for each individual patient based on whether the six clinical features were present, absent or unknown (online supplementary appendix 1). As a primary analysis, we used a 50% probability cut-off to categorise all patients with a probability of ≥50% as higher risk for AHT and those with a probability of <50% as lower risk for AHT. Individual clinician’s interpretation and application of probability thresholds to risk and decision making differs, and therefore in a secondary analysis, we explored the implications of using different probability cut-off points to categorise cases as AHT.12 19 20 To do this, we used a 20% probability cut-off and an 80% probability cut-off, respectively.
Data were entered into Epidata (The Epidata Association, Odense, Denmark) and were later entered into REDCap.21 Data were analysed using Stata V.13. Summary statistics were derived to describe total and subgroup characteristics, proportions and frequencies for categorical variables and the median (IQR) for continuous variables. Using the Stata command diagt we calculated the sensitivity, specificity, positive predictive value, negative predictive value, positive likelihood ratio and negative likelihood ratio of PredAHT-2 using the three different probability cut-offs (20%, 50% and 80%), with 95% CIs, excluding AHT-indeterminate cases. To assess model discrimination, we produced a receiver operating characteristic (ROC) curve and calculated the area under the curve (AUC). Separate ROC curves were also produced for those cases where all six clinical features were known and those where one or more features were unknown. Model calibration was assessed using a series of calibration plots.
This study is reported in accordance with the Transparent Reporting of a multivariable prediction model for Individual Prognosis or Diagnosis guidelines.22
Patient and public involvement
No patients were involved.
Of the 20 137 patients at 10 centres in the parent study, 13 371 (66%) patients presented at the five centres included in this secondary analysis. Of these patients, 5264 were <3 years old (39%), of which 3038 (58%) were male. The medical records of 142 cases of children <3 years old admitted with possible physical abuse-related head injuries on neuroimaging were reviewed, and 87 children with ICI were identified (figure 2).
Sixty-one (70%) were aged <1 year, 51 (59%) were male, 13 (15%) were admitted to paediatric intensive care unit (PICU), 26 (30%) underwent neurosurgery and 6 (7%) died (table 1).
Patients were categorised as AHT positive in 27 (31%), AHT negative in 45 (52%) and AHT indeterminate in 15 cases (17%). Head or neck bruising was more strongly associated with AHT-negative cases than AHT-positive cases, while seizures, apnoea, rib fractures, long bone fractures and retinal haemorrhages were more strongly associated with AHT-positive cases than AHT-negative cases. Many AHT-negative cases did not have an ophthalmology examination or skeletal radiology (table 2).
Figure 3 shows the distribution of the PredAHT-2 predicted probabilities against the outcome (AHT positive, AHT negative and indeterminate). Using a probability cut-off of 50%, PredAHT-2 correctly identified 20/27 AHT-positive cases (sensitivity=74% (95% CI 54% to 89%)) and correctly identified 39/45 AHT-negative cases (specificity=87% (95% CI 73% to 95%)) (table 3).
Applying PredAHT-2 using a 20% probability cut-off increased the sensitivity to 81% (95% CI 62% to 94%) at the expense of a much lower specificity (33% (95% CI 20% to 49%)). Conversely, applying PredAHT-2 using an 80% probability cut-off increased the specificity to 91% (95% CI 79% to 98%) at the expense of a much lower sensitivity (56% (95% CI 35% to 75%)).
The ROC curve depicting overall model discrimination is shown in figure 4. The AUC was 0.80 (95% CI 0.68 to 0.92). Model calibration, that is, the agreement between the predicted probabilities and the observed outcomes, was reasonable as shown in online supplementary figures 1-3.
A total of 7/15 (47%) AHT-indeterminate cases were classified as higher risk by PredAHT-2 at a cut-off of 50%, while 8/15 (53%) were classified as lower risk. This distribution did not change significantly using different cut-offs, as seven cases had a probability of <20%, seven cases had a probability of >80% and one case had a probability of 47%.
The presence or absence of all six clinical features were recorded in all but one AHT-positive cases and all indeterminate cases, while only 36% (16/45) AHT-negative cases had complete data recorded. For the AHT-negative cases, the probability of AHT was more likely to be <20% for cases where all six features were known (14/16 (86%) compared with when one or more features were unknown (23/45 (51%)). ROC curves for those cases in which all six clinical features were known and for those cases in which one or more clinical features were unknown are shown in online supplementary figures 4 and 5, respectively.
In this external validation of PredAHT-2 in an Australian and New Zealand dataset, the performance of the tool was very similar to the performance of PredAHT in the original validation study (sensitivity 74% vs 72% and specificity 87% vs 86%).8 With its added capacity to give a probability of AHT for an individual case where one or more of the six features are unknown, PredAHT-2 has the potential to contribute to decision making at multiple points along the assessment and referral pathway.10
Exploring the implications for clinical practice
There were seven AHT-positive cases that were assigned a probability of AHT of <50% by PredAHT-2 (figure 3). In four of these cases, the perpetrator confessed or was accused by the other parent, or the child’s injuries were severe, and included complex skull fractures and widespread bruising. These were additional factors that strongly increased the likelihood of physical abuse, highlighting the importance of interpreting probability estimates given by PredAHT-2 in combination with all other available information.
The extent of unknown features was considerable for AHT-negative cases (figure 3) and likely to be related to the clinicians’ decisions not to undertake a skeletal survey or ophthalmology examination, based on their level of confidence that the injury was accidental and consistent with the mechanism of injury described. Independently witnessed mechanisms of injury included falls from a parent’s arms, a fall down the stairs or being hit by a falling heavy object, which contrasted with the lack of history or inadequate explanation of trauma given by parents in AHT-positive cases. PredAHT-2 allows an assessment of the probability of AHT, even if not all six features are available due to a low index of suspicion or while a child abuse work-up is in progress. Where AHT-negative cases were fully investigated, the predicted probability of AHT was low, while missing information resulted in less definitive results.
Six AHT-negative cases were assigned a probability of >50% (figure 3). Five of these cases did not have an ophthalmology examination or skeletal survey. Completing the investigation would identify whether retinal haemorrhages, rib or long bone fractures were present and refine the probability estimate. For example, in children with ICI and head/neck bruising but no information about retinal haemorrhages or fractures, the calculated probability of AHT is 44.2% (see online supplementary appendix 1). If skeletal survey and ophthalmology were normal, this would decrease to 14.7%. Conversely, if either long bone fracture, retinal haemorrhage or rib fracture were identified, the probability would increase to 70.2%, 85.3% and 88.5%, respectively. This highlights the importance of considering an ophthalmology examination and skeletal survey for those children presenting with ICI in the absence of an independently witnessed accident.
All of the 15 indeterminate cases were fully investigated and PredAHT-2 predicted 7/15 to be high risk (>80%) and 7/15 to be low risk (<20%) for AHT. This finding suggests that the uncertainty in these cases did not arise from a consideration of the clinical features alone and is consistent with a study conducted by Chaiyachati et al,23 who found that there was no single component of the injury, incident or history associated with the uncertainty around clinicians’ perceived likelihood of physical abuse. Of seven indeterminate cases with a probability of abuse of >80%, two died and AHT was deemed ‘likely’ in four cases. Among the seven with a probability of AHT of <20%, AHT was deemed ‘likely’ in one case; however, in each of these cases, AHT could not be definitively confirmed, partly due to differing opinions between members of the multidisciplinary child protection team, most notably between medical clinicians and child protection social workers.
The study findings reinforce those from the original derivation and validation studies8 9 that no set of clinical features is specific for AHT. It is therefore unlikely that any CPR based on clinical features alone could perfectly predict AHT and emphasises that PredAHT-2 should be used in combination with a full multidisciplinary assessment and consideration of all of the other clinical, social, historical and forensic elements of each individual case. PredAHT-2 is designed to provide a specific probability estimate for each individual case based on six key features that should be identified during an assessment of a young child with ICI to inform further investigations or decisions.
This validation strengthens the utility of PredAHT-2 and raises its level of evidence.20 24 Roll-out of a computerised version would enable simple application of PredAHT-2 at the bedside, as new information is collected. PredAHT-2 should now be tested in an impact analysis study to determine its impact on clinician behaviour and relevant patient outcomes.25
Comparison with existing literature
The Pediatric Brain Injury Research Network (PediBIRN) four-variable CPR is the only other CPR for paediatric AHT intended for use in an inpatient setting and was designed to assist in deciding which children with cranial or ICI admitted to the PICU should be evaluated further for abuse.26–29 In a recent external validation of PediBIRN by our group, its sensitivity was 96% and its specificity was 43% when applied to children <3 years old with ICI or cranial injury, admitted to all inpatient settings.17 Taking the arbitrary probability cut-off of 50%, PredAHT-2 had a lower sensitivity than PediBIRN, but a much higher specificity, and will categorise fewer AHT-negative cases as higher risk for AHT than PediBIRN.
The study has a number of limitations. The majority of the predictive and outcome variables were collected prospectively; the predictive variable ‘apnoea’, however, was extracted from medical records. Future validation studies should ensure that the six variables are collected prospectively and should consider assessing their inter-rater reliability. Since case selection in our dataset was mostly based on ED identification, cases of possible AHT identified in a hospital ward or PICU would have been missed. We excluded patients with underlying structural abnormalities or pre-existing disease of the brain as well as birth injuries; we acknowledge, however, that these patients might have also suffered AHT and in fact be at increased risk for AHT because of mental and physical disabilities. While based on a large dataset, our sample size of 87 was relatively small, with inherent lower statistical power. Although the parent study had been conducted at 10 sites, this secondary analysis was limited to five sites where additional ethics approval could be obtained and coinvestigators were available.
One of the concerns in all studies of AHT is the possibility of creating a circular argument by defining the condition based on the features included in the tool. However, in this study, AHT diagnosis was assigned by local multidisciplinary teams independent of the study. A cautious approach was taken, and if there was any doubt, a category of indeterminate was assigned. In addition, the data for the study were extracted blinded to the case outcomes.
PredAHT-2 performed with reasonably high sensitivity and specificity when externally validated. The inclusion of probability estimates in incompletely investigated cases offers an opportunity for clinicians to consider the probability of AHT at different stages of the clinical assessment and whether further investigations are indicated.
Tables of contents summary
This study externally validates a tool to predict the probability of abusive head trauma in young children.
Contributors HP: contributed to the design of the study, conducted the review of medical records, carried out the initial analyses, drafted the initial manuscript and revised the article. LEC: contributed to the design of the study, made substantial contributions to the interpretation and discussion of findings and the drafting of the manuscript, produced figure 2 and critically revised the manuscript for important intellectual content. AMK, JAC, SRD, MLB, SO, MB, JN, EO, LMC, MDL and SB contributed to the design of the study, made substantial contributions to the interpretation and discussion of findings and critically revised the manuscript for important intellectual content. AS: contributed to the design of the study, supervised the categorisation of cases and critically revised the manuscript for important intellectual content. SH: contributed to the design of the study, carried out the initial analyses, drafted the tables and critically revised the manuscript for important intellectual content. FEB: had the initial study idea, contributed to the design of the study and critically revised the manuscript for important intellectual content. He takes responsibility for the paper as a whole.
Funding The study was funded by grants from the National Health and Medical Research Council (project grant GNT1046727, Centre of Research Excellence for Paediatric Emergency Medicine GNT1058560), Canberra, Australia; the Murdoch Children’s Research Institute, Melbourne, Australia; the Emergency Medicine Foundation (EMPJ-11162), Brisbane, Australia; Perpetual Philanthropic Services (2012/1140), Australia; Auckland Medical Research Foundation (No. 3112011) and the A + Trust (Auckland District Health Board), Auckland, New Zealand; WA Health Targeted Research Funds 2013, Perth, Australia; the Townsville Hospital and Health Service Private Practice Research and Education Trust Fund, Townsville, Australia; and supported by the Victorian Government’s Infrastructure Support Program, Melbourne, Australia. FEB’s time was part funded by a grant from the Royal Children’s Hospital Foundation and the Melbourne Campus Clinician Scientist Fellowship, Melbourne, Australia, and an NHMRC Practitioner Fellowship, Canberra, Australia. SD’s time was part funded by the Health Research Council of New Zealand (HRC13/556).
Competing interests AMK and LEC are part of the team that derived and validated the Predicting Abusive Head Trauma tool. However, all data collection and analyses were undertaken independently of either of these authors.
Patient consent for publication Not required.
Provenance and peer review Not commissioned; externally peer reviewed.
Data availability statement No data are available.
If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.