Article Text
PDF
Abstract
Objective Children with mental health crises require access to specialised resources and services which are not yet standard in general and paediatric EDs. In 2010, we published a systematic review that provided some evidence to support the use of specialised care models to reduce hospitalisation, return ED visits and length of ED stay. We perform a systematic review to update the evidence base and inform current policy statements.
Methods Twelve databases and the grey literature were searched up to January 2015. Seven studies were included in the review (four newly identified studies). These studies compared ED-based strategies designed to assess, treat and/or therapeutically support or manage a mental health presentation. The methodological quality of six studies was assessed using the Cochrane Effective Practice and Organization of Care Risk of Bias tool (one interrupted time series study) and a modified Newcastle-Ottawa Scale (three retrospective cohort and two before–after studies). The Grading of Recommendations Assessment, Development and Evaluation (GRADE) system was applied to rate overall evidence quality (high, moderate, low or very low) for individual outcomes from these six studies. An additional study evaluated the psychometric properties of a clinical instrument and was assessed using criteria developed by the Society of Pediatric Psychology Assessment Task Force (well-established, approaching well-established or promising assessment).
Results There is low to very low overall evidence quality that: (1) use of screening laboratory tests to medically clear mental health patients increases length of ED stay and costs, but does not increase the risk of clinical management or disposition change if not conducted; and (2) specialised models of ED care reduce lengths of ED stay, security man-hours and restraint orders. One mental health assessment tool of promising quality, the home, education, activities and peers, drugs and alcohol, suicidality, emotions and behaviour, discharge resources (HEADS-ED), has had good accuracy in predicting admission to inpatient psychiatry.
Conclusions Lower-quality data suggest benefits to the use of specialised resources and services for paediatric mental health care in general and paediatric EDs. Experimental evaluation of strategies and the inclusion of patient-reported outcomes will improve confidence in these findings. Additional psychometric studies are needed for the HEADS-ED tool to be considered well established.
- systematic review
- mental health care
- pediatrics
- emergency department
Statistics from Altmetric.com
Key messages
What is already known on this subject?
Children with mental health crises require access to specialised resources and services which are not yet standard in general and pediatric emergency departments.
A previously published review provided some evidence to support the use of specialised care models to reduce hospitalisation, return ED visits, and lengthof ED stay.
In this systematic review we evaluated whether specialized mental health resources and services are more effective than standard emergency care in improving practice delivery, patient-reported outcomes, and/or reducing ED costs.
What this study adds?
Our updated systematic review suggests benefits to the elimination of laboratory tests to medically clear mental health patients, the use of a specialised tool to predict admission to inpatient psychiatry, and specialised care models to reduce lengths of stay, security man-hours, and restraint orders.
However, the studies reviewed had important methodological shortcomings, which emphasize the need for more robust evaluations of these approaches to improve confidence in the current findings.
Introduction
Visits to general and paediatric EDs for children’s mental health have increased significantly over the past 10 years in North America and Europe.1–4 These visits are made by children and youth who may or may not have been previously diagnosed with a mental disorder or problem, and who are ill, injured or experiencing a psychiatric crisis.5 As such, ED clinicians require a range of resources, tools and skills to identify and manage a broad mental health patient population.
In 2011, the American Academy of Pediatrics (AAP) and the Committee on Pediatric Emergency Medicine issued a joint policy statement that stated children with acute mental health crises require multidisciplinary care, including the use of specialised screening tools, paediatric-trained mental health consultants and a broader availability of treatment options.5 It is unknown whether these recommended resources are standard in general and paediatric EDs, although past research suggests that they are not.6 7 One reason for the lack of resources may be that there is a greater degree of clinical practice direction available for patients with serious medical and/or patient safety implications (eg, deliberate self-harm),8 but there is less guidance when the presenting mental health complaint is non-urgent and non-specific.9
In this systematic review we evaluated whether ED-based management strategies for children’s mental health presentations are more effective than standard ED care in improving practice delivery, patient-reported outcomes and/or reducing ED costs. This review is an update from a previously published review, which provided some evidence to support the use of specialised care models to reduce hospitalisation, return ED visits and length of ED stay.10 This updated review aims to provide renewed recommendations for paediatric care for general and paediatric EDs.
Methods
We systematically reviewed the literature on the effectiveness of ED-based management strategies for paediatric mental health presentations. We used a protocol that was developed a priori and followed the PRISMA statement checklist for reporting.
Literature search and selection
A research librarian used the search strategy developed for the previous review10 to search 12 databases for literature published up to January 2015: Ovid MEDLINE (online supplementary file 1), Ovid MEDLINE In-Process and other non-indexed citations, namely Ovid Embase, Cochrane Central Register of Controlled Trials, Cochrane Database of Systematic Reviews, Health Technology Assessment Database, Database of Abstracts of Reviews of Effects, ACP Journal Club, Ovid PsycINFO, CINAHL via EBSCOhost, SocIndex via EBSCOhost and ProQuest Dissertations and Theses. The previous review included literature up to March 2009; therefore, the search was restricted to the years 2008–2015. The search was also restricted by language (English and French) and age (0–18 years). Grey literature was searched via online archive of conference proceedings for the Canadian Association of Emergency Physicians and Society for Academic Emergency Medicine. Reference lists of included studies and relevant systematic reviews were checked for eligible articles. Authors of original studies also provided additional study publications for review.
Supplemental material
Two independent reviewers (ASN, AS) screened titles and abstracts in the search results for eligibility. The full text of each study assessed as ‘relevant’ or ‘unclear’ was evaluated by two reviewers (ASN, AS) using a standardised inclusion/exclusion form. Discrepancies were resolved through consensus, and agreement was quantified with the kappa statistic.11
Studies examining an ED-based management strategy (any intervention designed to assess, treat and/or therapeutically support or manage a mental health presentation, irrespective of diagnosis) for general mental health care in children that reported ED-based outcomes were eligible for inclusion in the review. Authors of studies with unknown populations were contacted to confirm eligibility. Studies that included paediatric and adult patients with a mean participant age >18 years were not included in the review as results were based on an adult patient population. Studies needed to be experimental, quasi-experimental, psychometric, diagnostic or observational in design. Studies evaluating the efficacy or effectiveness of pharmacological interventions or for specific patient populations (ie, self-harm) were excluded from the review. Study outcomes of interest included ED-based, patient-reported outcomes (ie, changes in symptoms or behaviour during the ED visit) or changes in ED practice delivery and costs. Other outcome data reported in studies that were not ED specific were not included in the review.
Data extraction
Study-reported data were extracted on study characteristics (eg, language of publication, country), characteristics of the study population and description of the evaluation including comparisons, outcome measures and results. In the case of unclear or unreported information, primary authors were contacted. Data were extracted into standardised forms by one reviewer (AS) and were verified for accuracy and completeness by another (SK). Discrepancies were resolved by consensus.
Quality assessment
Studies included in this review were designed as interrupted time series, retrospective cohort, before–after and psychometric investigations. The methodological quality of an interrupted time series study included in the review was assessed using the Cochrane Effective Practice and Organization of Care Risk of Bias tool, which assesses intervention independence and effect on data collection, the shape of intervention effect, knowledge of allocation and outcome reporting.12 The methodological quality of retrospective cohort studies included in the review were rated using the modified Newcastle-Ottawa Scale for non-randomised studies.13 Studies were assessed on three domains (cohort selection, comparability and outcome) to evaluate their overall risk of bias. Before–after studies included in the review were assessed using an assessment tool developed by Rowe et al and based on the Newcastle-Ottawa Scale.14 The tool rates the selection of preintervention and postintervention groups, the comparability of groups, the assessment of outcome and the comparability of time frames of the preintervention and postintervention periods. We used criteria developed by the Society of Pediatric Psychology Assessment Task Force to assess the research and clinical utility of a study with psychometric objectives. The criteria rate the availability of instrument reliability and validity data, availability of the instrument with instructions on its use and scoring, and use of the instrument by multiple investigative teams with findings published in peer-reviewed journals.15 The resultant assessment is a rating of a clinical instrument/tool to be a well-established assessment (highest rating), approaching a well-established assessment (middle range rating), or a promising assessment (lowest rating).15 The quality assessments of all studies involved two reviewers (SK, AS) independently assessing each study and resolving disagreements through discussion and third-party involvement (ASN).
Two reviewers (SK, AS) also graded the quality of evidence for each outcome across the interrupted time series, retrospective cohort and before–after studies through consensus and third-party involvement (ASN) using four levels (high, moderate, low or very low) and summarised as an evidence profile table according to Grading of Recommendations Assessment, Development and Evaluation (GRADE).16 Four domains were assessed: risk for bias, consistency, directness, precision. Publication bias was not assessed because we could not combine study data for meta-analysis. As the evidence to which GRADE was applied was from studies that were not randomised controlled trials, the quality of evidence started at moderate/low and was downgraded to low/very low if there were concerns for any of the four domains. We decided a priori to exclude psychometric studies from the GRADE evidence profile as the study design can differ significantly from other quantitative designs and the domains do not readily apply.
Data analysis
A lack of available studies, in addition to the high heterogeneity in management strategies and outcomes, precluded the use of meta-analysis to pool and interpret study results. To provide some commonality to outcome reporting, we calculated relative risks (the ratio of the risk of an event in the intervention group compared with that of the control group) and risk differences (comparing the intervention and control groups in terms of their absolute difference) with 95% CIs for dichotomous outcomes. Number needed to treat (the average number of patients who need to be treated with the ED-management strategy for one to benefit compared with patients who received treatment as usual) was also calculated for significant effects. We calculated the difference in means (MD; a measure of the absolute difference between the mean values of the intervention and control groups) with 95% CIs for continuous outcomes for studies with known sample sizes; we calculated absolute differences for MDs without 95% CIs for studies with unknown sample sizes as CIs are dependent on sample size. Where means and standard deviations were given, but sample sizes were not, we computed the minimum sample size that would result in a statistically significant difference by setting the p-value of the difference to 0.05 and solving for n in the p-value computation equation. Study-reported data were extracted when independent calculations were not possible.
Results
Literature search and selection
The search identified 6893 articles through database searching (figure 1). An additional 87 articles were identified through a review of reference lists, primary author contact and inclusion of the studies from the original review. During article screening, the agreement of study selection between the two reviewers was substantial with a kappa of 0.79. After exclusions based on title or abstract, 475 studies remained and underwent full-text review. No additional articles were identified through hand-search of conference proceedings. Of the 463 studies identified from the updated search, four studies were included. Of the 12 studies from the previous review, three were included; eight studies were excluded due to age restrictions17–24 and one study was excluded after we were unable to confirm the study population with the corresponding author.25 Thus, a total of seven studies were included in this review. Of the seven studies, one was an designed as an interrupted time series,26 three were retrospective cohort studies,27–29 two were before–after studies,30 31 and one was a psychometric investigation.32
Literature flow diagram.
Description of included studies
Three studies evaluated a clinical assessment/screening approach27 28 32 and four studies evaluated specialised models of care.26 29–31 Study characteristics are presented in table 1. While the strategies were intended for general mental health care, only one study identified the spectrum of mental health presentations in the study population.27 One study’s patient population was described as predominantly characterised by suicide-related behaviours,26 while the remaining studies did not describe the range of psychiatric conditions of treated patients. We were unable to confirm the average age of patients in two studies that included patients under the ages of 18 and 19 years.17 26 Authors of four included studies provided additional study information relating to missing data for the review.27 29–31
Summary of included studies
Methodological quality of studies
Using the Cochrane Effective Practice and Organization of Care Risk of Bias tool, the interrupted time series study was assessed as having a high risk of bias due to a lack of information as to whether the intervention was independent of other changes in time and lack of clarity regarding confounding (seasonality).26 Overall, the cohort studies were deemed to be of good quality using the modified Newcastle-Ottawa Scale with the only potential source of bias stemming from the lack of information on factors that could have affected the comparability of the cohorts (ie, age, ethnicity, gender, triage score, hold type).27–29 Risks of bias identified in the before–after studies included a lack of available information regarding selection, comparability, outcome assessment and time of assessment,31 and a lack of information regarding outcome assessment and comparability.30 Criteria developed by the Society of Pediatric Psychology Assessment Task Force were used to assess a study of the home, education, activities and peers, drugs and alcohol, suicidality, emotions and behaviour, discharge resources (HEADS-ED) tool,32 which was described in a psychometric study. Applying these criteria, the tool was evaluated as a ‘promising assessment’. Although the HEADS-ED tool has been measured in only one peer-reviewed article, the publication included detailed statistical analysis indicating good validity and reliability, and the authors provided detailed information to allow for sufficient evaluation and replication.
Effects of ED management strategies
The effects of ED management strategies are presented in tables 2 and 3 and organised by approach below.
Assessment and screening strategies
ED management and disposition decisions
There was no statistically significant difference in the risk of management and disposition changes for patients with screening laboratory tests conducted for medical clearance compared with those patients who did not have testing conducted (table 2).27 A mental health screening tool, the HEADS-ED, was found to predict admission to inpatient psychiatry with good accuracy (sensitivity 82%, specificity 87%, area under the receiver operator characteristic curve 0.817).32
Effect of ED-based assessment and screening strategies for mental health presentations on practice delivery and costs
Effect of ED-based specialised models of care on practice delivery and costs
Length of ED stay
A longer length of ED stay was identified among mental health patients with screening laboratory tests compared with those patients who did not have testing conducted for medical clearance during the ED visit.27 28
ED-based specialised models of care
ED management and disposition decisions
Reduced security and restraint use were reported following a redesign of the ED environment to include an ED-housed unit designed to accommodate psychiatric patient management (table 3).29 Two specialised models of care, the rapid response model and emergency room follow-up team, were found to reduce monthly admissions to inpatient psychiatry26 and hospitalisation.31
Length of ED stay
A decrease in length of stay was reported following the redesign of the ED environment (ED-housed unit)29 and use of a specialised team, the child guidance model, for disposition decision making.30
Cost
Use of the child guidance model was associated with cost savings of US$18.98 per patient and total savings of US$10 651.57 over a 7-month period.30
Return visits to the ED
An emergency room follow-up team did not reduce the risk of ED return among children relative to those children who received treatment as usual.31
Overall quality of evidence
The overall quality of evidence for each outcome from the interrupted time series, retrospective cohort and before–after studies was summarised in a GRADE evidence profile table, and is presented in table 4. Overall, the quality of evidence for reported outcomes was low or very low, primarily due to the design of studies as well the limited number of studies for any given outcome. The risk of bias was considered not serious for restraint and security use, changes in patient management or disposition, or ED length of stay. The risk of bias was considered serious for hospital admissions and return ED visits. For outcomes measured by more than one study, we found no inconsistency present. All outcomes were considered to be directly measured. Imprecision was identified for monthly admissions to inpatient psychiatry.
GRADE evidence profile
Discussion
Our previously published review identified promising evidence from three paediatric studies in favour of the use of specialised models for paediatric mental health care.10 These models were shown to reduce hospitalisation, return ED visits and length of ED stay among children with ED mental health visits; however, the findings were limited by the moderate methodological quality of the research studies and the absence of patient-reported outcomes. Our updated review reported herein includes results from four newly identified studies. These new studies, published in the last 5 years, showed that new assessment and care approaches can positively impact ED care. In two studies, screening laboratory tests for medical clearance were associated with a longer length of ED stay and higher costs, but not a risk of clinical management or disposition change if not conducted. A mental health screening tool used by ED staff in one study predicted a paediatric patient’s admission to inpatient psychiatry, while a new specialised care model was associated with reduced lengths of ED stay, as well as security man-hours and restraint orders per 100 patients. Despite the promising findings, further research is required to address methodological limitations in the evidence base in this review and increase the generalisability of findings.
The administrative drive towards more cost-effective and time-efficient approaches in the ED should be informed by the impact of such approaches on clinical management and decision making as well as patient outcomes. Preliminary evidence from studies included in this review highlight the potential advantages of introducing new tools (the HEADS-ED)32 and re-examining traditional approaches (laboratory screening tests for medical clearance)27 28 to ED screening. In a clinical landscape with few ED-tested options for clinicians for mental health assessment and clinical decision making, additional evaluation of these strategies (and others) should be a priority. Prospective evaluation across multiple ED settings and with a general paediatric mental health population (in the case of laboratory screening tests for medical clearance) would increase the generalisability and transportability of screening and assessment approaches.
That initial evaluations of specialised models of paediatric mental health care have demonstrated a beneficial impact on ED care necessitates a new generation of clinical evaluations. This new approach will help general and paediatric EDs meet policy recommendations for ED-based paediatric mental health care. Moving forward, because studies to date are largely constrained by setting (having been conducted in paediatric EDs and in single centres), design (retrospective vs prospective) and outcome (inconsistency and lack of comprehensiveness across studies), future studies should: (1) be prospective, multi-site endeavours that involve both paediatric and general EDs; (2) measure patient-reported outcomes (eg, patient functioning); and (3) include economic evaluations. Such studies would address the concerns highlighted in this review regarding study quality as well as provide evidence of model generalisability and allow for assessment of model feasibility. In addition, the evaluation of different resource-based models—with a particular focus on varying multi-disciplinary components and the function of paediatric-trained mental health consultants vis-à-vis ED staff (ie, consultation–liaison services33 or telehealth/telemedicine services)34—would provide clinically translatable information that could enhance existing ED services in general and paediatric EDs, while comparative effectiveness studies of different specialised models of care could help ED administrators and clinicians select among models of potentially different effects.
On a final note, the recommendation from the AAP and Committee on Pediatric Emergency Medicine for access to a broader availability of treatment options gives direction for research not identified in this review. Study of ED referral and follow-up protocols for schools and primary care settings would further delineate clinical pathways for families, health care providers and other professionals and provide an opportunity to examine decision making, outcomes and communication across the continuum of children’s mental health care. Also absent from this review were studies evaluating strategies for children and adolescents who make repeat visits to the ED for mental health care. This population has been shown to comprise 30%–37% of annual ED mental health visits in Canadian studies.3 4 The current evidence base is grounded in observational study of characteristics and factors associated with return visits to the ED after an initial mental health visit.35 36 As concurrent use of specialised mental health services has been identified among children and adolescents with multiple ED mental health visits,35 37–39 evaluation of interventions for this population may be best situated in studies of clinical care pathways. Frequent use of the ED for mental health care has been perceived as a potentially preventable health care pattern,40 but among children, little is known about the context of repeated ED use. As a start, investigations to explore reasons for repeated ED use and factors (eg, patient-level, family-level, community-level) that could be addressed in ED and community-based care are recommended.40
Limitations
While considering our findings, it is important to note that the search strategy was restricted to studies published in English. It is possible that studies published in other languages were excluded from this review and could add to an understanding of the effect of ED-based mental health care approaches. While we followed rigorous methodological standards for conducting systematic reviews, there are also limitations stemming from the primary studies. It is difficult to synthesise the literature on ED-based paediatric mental health care due to the small number of studies and the large variability across studies in interventions and outcomes. Providing overall generalisations and conclusions in this context is difficult. The literature included in this review also has inadequacies. There was a lack of high-quality evidence to support ED-based strategies for paediatric mental health care. Little description was provided for standard ED care to which interventions were compared. Issues of potential confounding and a lack of reporting of key methodological features (eg, patient selection, outcome assessment) were evident in the current knowledge base and should be addressed in future studies.
Conclusions
Despite evidence that the demand for mental health services in the ED continues to grow, appropriate and commensurate level of mental health care to meet the needs of these children and youth remains inadequate. Since, the previous publication in 2010, the results of this study demonstrate that while there has been an increase in research in this area, most of the evidence is limited by methodological shortcomings. In addition, our findings highlight the need for high quality evidence to guide mental health screening, early and effective interventions, and ongoing follow-up care after the ED visit.
Acknowledgments
The authors thank Ms Robin Featherstone (University of Alberta) for conducting the study search update and Mr. Ben Vandermeer (University of Alberta) for assistance with statistical analysis. The authors would also like to thank the study authors for responding to their requests for additional information.
References
Footnotes
The findings from this review were presented at the Emergency Strategic Clinical Network (SCN) Core Committee Meeting on 3 November 2015 in Edmonton, Alberta.
Contributors ASN conceptualised and designed the study, obtained funding for the study and supervised the study, analysed and interpreted the data, drafted the initial manuscript, critically revised the manuscript for important intellectual content, and approved the final manuscript as submitted. LH contributed to the design of the study, provided statistical expertise, analysed and interpreted the data, drafted the initial manuscript, critically revised the manuscript for important intellectual content, and approved the final manuscript as submitted. AS, SK contributed to the design of the study, acquired the data, analysed and interpreted the data, drafted the initial manuscript, critically revised the manuscript for important intellectual content, and approved the final manuscript as submitted. MPD, MC contributed to the design of the study, analysed and interpreted the data, drafted the initial manuscript, critically revised the manuscript for important intellectual content, and approved the final manuscript as submitted.
Funding Funding for this review was provided by the Emergency Strategic Clinical Network (Alberta). In-kind support of the research librarian services was provided by the Alberta SPOR Support Unit, Health Systems Research, Implementation Research & Knowledge translation platform. ASN and LH hold New Investigator Awards from the Canadian Institutes of Health Research.
Competing interests MC is the lead author of one of the studies included in this review. He was not involved in the quality assessment of this study. ASN, LH, MPD and MC are authors of the previous systematic review published in 2010. LH is the co-director of the Alberta SPOR Support Unit, Health Systems Research, Implementation Research & Knowledge translation platform. Her involvement in the study was academic and not under this role.
Provenance and peer review Not commissioned; externally peer reviewed.