Article Text
Abstract
Objective The National Emergency X-Radiography Utilisation Study II (NEXUS II) clinical decision rule (CDR) can be used to optimise the use of CT in children with head trauma. We set out to externally validate this CDR in a large cohort.
Methods We performed a prospective observational study of patients aged <18 years presenting with head trauma of any severity to 10 Australian/New Zealand EDs. In a planned secondary analysis, we assessed the accuracy of the NEXUS II CDR (with 95% CI) to detect clinically important intracranial injury (ICI). We also assessed clinician accuracy without the rule.
Results Of 20 137 total patients, we excluded 28 with suspected penetrating injury. Median age was 4.2 years. CTs were obtained in ED for 1962 (9.8%), of whom 377 (19.2%) had ICI as defined by NEXUS II. 74 (19.6% of ICI) patients underwent neurosurgery.
Sensitivity for ICI based on the NEXUS II CDR was 379/383 (99.0 (95% CI 97.3% to 99.7%)) and specificity was 9320/19 726 (47.2% (95% CI 46.5% to 47.9%)) for the total cohort. Sensitivity in the CT-only cohort was similar. Of the 18 022 children without CT in ED, 49.4% had at least one NEXUS II risk criterion. Sensitivity for ICI by the clinicians without the rule was 377/377 (100.0% (95% CI 99.0% to 100.0%)) and specificity was 18 147/19 732 (92.0% (95% CI 91.6% to 92.3%)).
Conclusions NEXUS II had high sensitivity, similar to the derivation study. However, approximately half of unimaged patients were positive for NEXUS II risk criteria; this may result in an increased CT rate in a setting with high clinician accuracy.
- trauma, head
- emergency departments
- imaging, ct/mri
- paediatrics, paediatric emergency medicine
Statistics from Altmetric.com
Key messages
What is already known on this subject
CDRs can assist emergency staff in determining which children should receive a CT scan after a head injury.
One of the few CDRs developed for both children and adults is the NEXUS head CT decision tool.
The NEXUS CDR has not been externally validated in a paediatric-specific cohort.
What this study adds
We conducted an external validation of the National Emergency X-Radiography Utilisation Study (NEXUS) clinical decision rule (CDR) in a paediatric multicentre setting with a low baseline CT rate.
NEXUS had high sensitivity in detecting clinically important intracranial injury.
Half of all unimaged head injured children were positive for NEXUS criteria in a setting where clinicians did not miss any clinically important intracranial injury.
Introduction
Background
Over the last 20 years, a number of head injury clinical decision rules (CDRs) have been developed. They are generally designed to determine which patients should or should not undergo CT of the head by stratifying patients into those at increased or decreased risk of intracranial injury (ICI). In children with head injuries, concerns about diagnostic speed, accuracy, resource use and variation in practice are amplified by concerns about radiation exposure and subsequent iatrogenic cancer risk1 2 and the need for sedation in young and uncooperative patients.3
Several child-specific head injury CDRs have been developed using large data sets.4–8 This necessitates that clinicians use a CDR for paediatric patients and a separate CDR for adult patients. In contrast, the National Emergency X-Radiography Utilisation Study II (NEXUS II) collaboration aimed to provide a low-risk decision instrument for head injured patients of all ages.9 10 This CDR, prospectively derived in a multicentre data set of 13 728 patients who underwent CT for their blunt head injury includes eight predictor variables (age over 65 years, evidence of skull fracture, scalp haematoma, neurological deficit, abnormal alertness, abnormal behaviour, coagulopathy or persistent vomiting). A 7-criteria NEXUS II instrument, excluding the age predictor variable, was then tested in the paediatric subgroup of 1666 children included in the original derivation data set, showing a very high sensitivity (98.6%, 95% CI 94.9% to 99.8%) and negative predictive value (NPV) (99.1%, 95% CI 96.9% to 99.9%). However, the NEXUS II CDR has not been externally validated in a child-specific cohort of undifferentiated head injuries.
Only CDRs that have been derived according to rigorous methodological standards and are externally validated should be implemented in routine clinical practice.4 As >80% of paediatric emergencies are managed in EDs that care for both adult and paediatric patients CDRs developed to be used for patients of all ages have the potential to be ideal decision instruments for emergency physicians in mixed departments.
We undertook an external validation of the NEXUS II CDR in head injured children using prospectively collected multicentre data outside the derivation setting.11 12 We used the NEXUS II predictor variables and the rule-specific outcome of clinically important ICI to determine the diagnostic accuracy of the rule in an undifferentiated cohort of children with head injuries of any severity. We also analysed the accuracy in the cohort that underwent head CT, consistent with the original derivation cohort inclusion criteria.
Methods
Study design and setting
This was a planned secondary analysis of a prospective multicentre observational study, which enrolled children aged <18 years presenting with head injury of any severity to 10 paediatric EDs in Australia and New Zealand between April 2011 and November 2014. All EDs are members of the Paediatric Research in Emergency Departments International Collaborative research network.
While the focus of the parent study11 was on the validation of three other paediatric-specific neuroimaging rules ((i) the prediction rule for the identification of children at very low risk of clinically important traumatic brain injury developed by the Paediatric Emergency Care Applied Research Network (PECARN),6 (ii) the Canadian Assessment of Tomography for Childhood Head Injury (CATCH) rule7 and (iii) the Children’s Head Injury Algorithm for the Prediction of Important Clinical Events (CHALICE))8 published elsewhere,11 we also collected the predictor and outcome variables of the NEXUS II rule.9 10 Using the published predictor variables and ICI as outcome variable as defined by NEXUS, we assessed the accuracy (sensitivity, specificity, negative predictive value (NPV), and positive predictive value (PPV)) of NEXUS II in patients in the whole cohort of patients, using systematic telephone follow-up data as a surrogate for negative primary outcome of no ICI if no CT scan had been performed. In a secondary analysis, we assessed the accuracy of NEXUS II only in the cohort who underwent a CT scan, similar to the original derivation papers.9 10 As predictor and outcome variables were not collected verbatim as published,9 10 we adapted them based on available data (see online supplementary tables 1 and 2). We also assessed clinician accuracy for predicting the outcome of ICI by whether they conducted a CT in the ED and compared the results with the accuracy of NEXUS II.
Supplementary file 1
We obtained informed verbal consent from parents/guardians apart from instances of significant life-threatening or fatal injuries where participating ethics committees granted a waiver of consent.
The trial protocol was published in detail elsewhere.12 The study was registered with the Australian New Zealand Clinical Trials Registry ACTRN12614000463673 and followed the Standards for Reporting Diagnostic accuracy studies guidelines.13
Selection of participants
Patients were enrolled by the treating ED clinician who collected demographic, epidemiological and clinical data on a standardised case report form prior to any neuroimaging. ED clinicians decided to obtain head CTs at the initial presentation in ED based on their clinical judgement and their own criteria, the study exerted no influence on this process. A research assistant (RA) recorded ED and hospital management data after the visit and conducted a telephone follow-up for patients who had not undergone neuroimaging. Up to six follow-up call attempts were made up to 90 days after injury. In addition, data of any patients who had representations to the study hospitals leading to a CT scan within the follow-up period prior to the phone call were used to assess outcomes. Any patients who had representations to other hospitals based on the telephone follow-up had neuroimaging and neurosurgery reports requested where applicable.
Patients were excluded if they had trivial facial injuries, refused participation, had neuroimaging prior to arrival in ED, did not wait to be seen or were referred for care outside the ED and if there were social issues preventing an approach of the patient or family.
RAs were not blinded to the purpose of the study. Site investigators, RAs and participating ED clinicians received formal training prior to and during the study.
Methods and measurements
The NEXUS II decision instrument9 10 was used for this study. It includes eight risk criteria: evidence of significant skull fracture, altered level of alertness, neurological deficit, persistent vomiting, presence of scalp haematoma, abnormal behaviour and coagulopathy and age >65 years. Patients fulfilling one or more of these criteria are considered at high risk for ICI and are recommended to undergo a head CT. Similar to Oman et al,10 we removed the age criterion as not relevant; online supplementary table 1 presents the definition of the seven risk criteria used10 and how closely they matched the predictor variables available from PEACRN, CATCH and CHALICE.
Outcomes
The definition of ICI applied in this study was based on the original definition proposed by Mower et al,9 according to the presence of one or more of a number of CT findings shown in online supplementary table 2. These were derived from previous clinical work,14 which showed that patients with these CT findings may require neurosurgical intervention (craniotomy, intracranial pressure monitoring or mechanical ventilation), or are likely to suffer significant long-term neurological impairment.10 14
In this study, we used senior radiologist reports to determine the results of CT scans. RAs and site investigators abstracted these data from CT reports for agreement with the outcome measures and consulted local site radiologists if there was uncertainty in the interpretation of individual scans. De-identified copies of CT findings were provided to the central site. If there was a question as to the classification of the CT, a central site investigator would review the reports and if needed use a third site investigator to resolve disagreements.
A comparison between the NEXUS II CT criteria and the corresponding information used in our study is provided in online supplementary table 2.
For the descriptive analyses of neurosurgical interventions, we used operative reports. RAs and site investigators abstracted the information from operative reports. If there was a question as to the classification of the operative reports, a central site investigator would review the reports and if needed use a third site investigator to resolve disagreements.
Analysis
Data were entered into Epidata (The Epidata Association, Odense, Denmark), and later REDCap,15 and analysed using Stata 13 (StataCorp, College Station, Texas, USA). Descriptive statistics were calculated for key variables with 95% CIs where relevant.
Primary analysis was the diagnostic accuracy (sensitivity, specificity, NPV and PPV) of NEXUS II for ICI in patients in the cohort of all presenting patients. In this analysis, patients who did not undergo a CT scan, but were followed up by telephone up to 90 days after the injury were coded as not having ICI. In a secondary analysis, the diagnostic accuracy (sensitivity, specificity, NPV and PPV) of NEXUS II for ICI was tested in the cohort who underwent a CT scan. Finally, we assessed the diagnostic accuracy of clinician practice to detect ICI if a CT scan had been conducted.
Missing predictor variables were treated as missing presumed negative. A sample size calculation had been conducted for the parent study11 12; no additional sample size calculation was conducted for this substudy and all available patients who fulfilled inclusion criteria were used.
We conducted a secondary analysis where patients with missing predictor variables were excluded.
As in the study by Oman et al,10 we also performed a subgroup analysis in children aged ≤3 years. This subgroup analysis was performed in consideration of the fact that assessment of criteria pertaining to behavioural and neurological functioning domains, as well as to other domains, may be more challenging in children of this very young age10 and thus may affect the diagnostic accuracy of the CDR instrument.
Results
Characteristics of study subjects
A total of 29 433 patients presented to the ED with injury of any severity, of which 5203 were missed. Of the remaining 24 230 patients assessed for eligibility, 1706 were excluded. Of the 22 524 eligible patients, 2240 were lost to follow-up, 147 had records that were not evaluable and 28 were suspected by the ED clinician to have sustained a penetrating head injury; this led to a total number of 20 109 patients evaluable for analyses. Patient flow is described in figure 1.
Of 1962 (9.6%) patients who received a CT scan in the ED, 377/1962 (19.2%) had ICI as defined by NEXUS II (table 1). Mean age of patients was 6.6 (SD 5.2) years for patients with ICI and 8.8 (SD 5.2) years for those without ICI on CT. Overall, 26.3% of patients with a CT scan were <3 years of age. Main signs and symptoms were loss of consciousness and vomiting, main mechanism of injury was fall related. Ten children died from their head injuries. The most frequent types of ICI identified on CT were skull fractures, and subdural/extradural haemorrhages and contusions, with the most frequent neurosurgical interventions (74, 19.6%) being monitoring of intracranial pressure and craniotomy (table 2).
Main results
The most frequent positive NEXUS II risk criteria in children with ICI (table 3) were scalp haematomas (217/377; 57.6%) and altered level of alertness (235/377; 62.3%). Four patients with CT confirmed ICI had no NEXUS II risk criteria. These four children were aged between 4 and 15 years, presented with a GCS of 15 and fell from height, fell off scooter or were struck by another person at school. All were admitted and none required neurosurgery (see online supplementary table 3). Most children with ICI had two or three positive risk criteria (table 3). When analysing risk criteria among 18 022 children who had no initial or subsequent CT scan, 8909 (49.4%) had at least one risk criterion and 2476 (13.7%) had at least two risk criteria (see online supplementary table 4).
When analysing the diagnostic accuracy of the rule based on all presenting patients, with the assumption that patients not undergoing a CT scan were negative for ICI based on telephone follow-up, it showed a sensitivity of 379/383 (99.0% (95% CI 97.3% to 99.7%)) and specificity of 9320/19 726 (47.2% (95% CI 46.5% to 47.9%)) (table 4). When assessing the accuracy of NEXUS II based on initial CT scan in ED, the sensitivity for ICI based on the NEXUS II CDR was 373/377 (98.9%; 95% CI 97.3% to 99.7%) and specificity 156/1585 (9.8%; 95% CI 8.4% to 11.4%) (table 4). Results were similar when all CTs performed at any time point were included and in children aged <3 and ≥3 years (table 5). The sensitivity of NEXUS II increased with the number of risk criteria (table 6). We also conducted a sensitivity analysis where patients who had a CT scan with missing risk criteria were excluded. Accuracy results were unchanged (see online supplementary table 5). Clinician accuracy in detecting ICI without the rule was 377/377 (100.0 (95% CI 99.0% to 100.0%)) and specificity was 18 147/19 732 (92.0% (95% CI 91.6% to 92.3%)) (table 4).
Discussion
In this multicentre validation study, we found NEXUS II to have high sensitivity in detecting the NEXUS-specific outcome of ICI in the overall cohort and in the cohort who underwent CT scanning. Four patients were missed by the rule, that is, had no NEXUS II risk criteria but were positive for ICI; none of them required neurosurgery. Overall accuracy results did not change when patients were excluded who had missing information for NEXUS criteria. Both high sensitivity (98.9%; 95% CI 97.3% to 99.7%) and low specificity (9.8%; 95% CI 8.4% to 11.4%) in the CT-only cohort were similar to the analysis of the original NEXUS data overall9 and for the paediatric cohort in particular (sensitivity 98.6% (95% CI 94.9 to 99.8); specificity 15.1% (95% CI 13.3 to 16.9)).10 Our data set differed from the NEXUS II cohort in that more young children were enrolled compared with the ‘J’-shaped distribution with an increased number of teenagers in NEXUS.
External validation for NEXUS II in children is limited. In three paediatric studies from Finland,16 Italy17 and the USA18 outside the derivation setting, NEXUS II was assessed in terms of accuracy and compared with other head injury rules. All were limited by being conducted in single-centre settings and by the retrospective nature of data extraction16 18 or the use of a prospective data set collected before the NEXUS II derivation data had been published.17 Neither paper lists in detail how the NEXUS definitions were modified to conform with pre-existing data sets. NEXUS sensitivities were reported at 96% (95% CI 90 to 99),16 88.9% (95% CI 63.9 to 95.6)17 and 78.3% (95% CI 69.9 to 86.7),18 respectively. It is not clear if a multicentre prospective study of NEXUS II from Korea includes any children.19 Recently, the original team which derived the NEXUS II CDR conducted a validation study in a new data set including 11 750 blunt head injuries20; in an analysis of the paediatric cohort of 1018 children sensitivity was 98.0% (95% CI 89.1% to 99.9%) and specificity of 34.0% (95% CI 31.0% to 37.0%) for the assessment of high-risk status in patients with ICI.21 However, similar to the derivation cohort only patient who had received a CT scan were included in the study.
While we also followed the original adult and paediatric papers for NEXUS II9 10 by analysing rule accuracy (external validation) solely based on patients who actually underwent CT scanning, in particular as CT imaging was required for outcome assessment, this does not reflect how rules such as NEXUS II may actually be used in the clinical setting. Therefore, we primarily analysed the accuracy using all patients, including those without CT scans; while sensitivity was similar, specificity was higher in the overall cohort compared with the CT-only cohort. Of 18 022 head injured children in our study who did not undergo CT imaging, 49.4% of these patients fulfilled at least one of the NEXUS predictor variables and therefore, based on the rule, were deemed not at very low risk. If this rule was applied and followed in an undifferentiated cohort of paediatric patients, it would have the potential to increase the head CT rate. When adding predictor positive patients who underwent any CT scan (n=1497) and those who did not (n=8909), 51.7% (10 406 of 20 109) of all head injured children would be required to undergo CT imaging as compared with 10.4% (2087 of 20 109) who actually did, an increase of about 400%. This is particularly striking considering the high sensitivity—where clinicians did not miss a single patient with ICI—and specificity of clinician practice at the participating sites without using the NEXUS rule.
NEXUS II used ICI as primary outcome. Although the outcome was consensus based and aimed to capture CT changes that may require neurosurgical intervention, ‘or lead to rapid deterioration or significant long-term neurological impairment’, the actual criteria used to define ICI were solely based on recorded CT changes.9 22 Other paediatric CDRs, such as PECARN, use definitions of clinically important traumatic brain injury that explicitly require, in addition to specified CT changes, relevant clinical changes, such as admission >2 days or intubation >1 day (in addition to death or neurosurgery).6 Such tightening of the outcome definition would likely change sensitivity and specificity calculations of NEXUS II in both the derivation data set and in our data.
This study has a number of limitations. The predictor and outcome variables for the NEXUS II CDR would have ideally been collected verbatim. Any changes in definitions and wording of head injury rules have the potential to alter the final results.23 The focus of the parent study11 used for this paper, however, was on the paediatric-specific CATCH, CHALICE and PECARN rules.6–8 We felt that by collecting the predictor and outcome variables of three rules in detail, the modified definitions could be closely approximated to the actual NEXUS II criteria used for this study. A further limitation of this study was a 10% loss to follow-up in the parent study, which may have affected the analysis including patients who did not undergo CT scanning. In the NEXUS derivation study as well as in the validations study by the same group, patients who were not imaged were generally not followed up.9 In our study, participating sites were usually also the only paediatric neurosurgical centres in the local areas. A further limitation is that CT scans were not reviewed centrally; rather CT reports were used to assess ICI. Finally, this study reflects practice at tertiary Australian and New Zealand centres, where imaging rates are generally lower than in North America6 7 11 24 and no specific rule predominates.25 The developers of NEXUS presumed that patients in the derivation study perceived by clinicians to be at high risk (independent of the NEXUS criteria per se) would be imaged by treating clinicians anyway. Therefore, no comprehensive follow-up system was put in place and only a small fraction of the original derivation cohort (9.2%) were followed up by interview9; it is unclear how many head injured children were included in this cohort. Although no denominator of patients presenting to participating EDs or the actual imaging rate at NEXUS sites was provided in the NEXUS derivation or validation papers,9 10 20 21 in settings with a presumably high CT rate this may accurately capture all or most relevant patients. In settings with a much lower scan rate, such as in our setting, this may not be true and patients may be missed unless a comprehensive follow-up system is in place.
In this multicentre prospective external validation study, the NEXUS II CDR had high sensitivity and low specificity similar to the derivation study. However, as approximately half of patients who did not receive a CT scan were positive for NEXUS II risk criteria, this CDR has the potential to lead to increased scanning rates in a setting with high clinician accuracy.
Acknowledgments
The authors would like to thank the participating families, emergency department staff and research staff at participating sites.
References
Footnotes
Contributors FEB: conceived the study, obtained grant funding, designed the study, provided overall supervision, interpreted the data, wrote the initial draft of the paper, gave final approval to be published and agreed to be accountable for all aspects of the work. MLB, NP, AK, SD, JAC, YG, JF, JN, MDL, MA, SH, SB, LMC, EO, SRD: designed the study, obtained the data, provided supervision, interpreted the data, drafted or revised it critically, gave final approval to be published and agreed to be accountable for all aspects of the work. SD: designed the study, supervised the analysis of the data, contributed to the interpretation of the data, revised the paper critically, gave final approval to be published and agreed to be accountable for all aspects of the work.
Funding The study was funded by grants from the National Health and Medical Research Council (project grant GNT1046727, Centre of Research Excellence for Paediatric Emergency Medicine GNT1058560), Canberra, Australia; the Murdoch Childrens Research Institute, Melbourne, Australia; the Emergency Medicine Foundation (EMPJ-11162), Brisbane, Australia; Perpetual Philanthropic Services (2012/1140), Australia; Auckland Medical Research Foundation (No. 3112011) and the A + Trust (Auckland District Health Board), Auckland, New Zealand; WA Health Targeted Research Funds 2013, Perth, Australia; the Townsville Hospital and Health Service Private Practice Research and Education Trust Fund, Townsville, Australia and supported by the Victorian Government’s Infrastructure Support Program, Melbourne, Australia. FEB’s time was part funded by a grant from the Royal Children’s Hospital Foundation, Melbourne, Australia, an NHMRC Practitioner Fellowship GNT1124466 and a Melbourne Children’s Clinician Scientist Fellowship. SRD’s time was part funded by the Health Research Council of New Zealand (HRC13/556).
Competing interests None declared.
Patient consent Not required.
Ethics approval The study was approved by the Royal Children’s Hospital, Melbourne, Australia.
Provenance and peer review Not commissioned; externally peer reviewed.
Data sharing statement There are no additional data available.
Press Release Yes.