Article Text
PDF
Abstract
Objectives (1) Validate an existing clinical tool for assessing risk after deliberate self-harm (DSH), Manchester Self-Harm Rule, in a new setting and new population, (2) develop a clinical decision rule based on factors associated with repeated self-harm in a Swedish population and (3) compare these rules.
Design A consecutive series of 1524 patients attending one of Scandinavia's largest emergency departments (ED) due to DSH during a 3-year period were included. Explanatory factors were collected from hospital charts and national databases. A nationwide register-based follow-up of new DSH episode or death by suicide within 6 months was used. We used logistic regression, area under the curve and classification trees to identify factors associated with repetition. To evaluate the ability of different decision rules to identify patients who will repeat DSH, we calculated the sensitivity and specificity.
Main outcome measure Repeated DSH or suicide within 6 months.
Results The cumulative incidence for patients repeating within 6 months was 20.3% (95% CI 18.0% to 22.0%). Application of Manchester Self-Harm Rule to our material yielded a sensitivity of 89% and a specificity of 21%. The clinical decision rule based on four factors associated with repetition in the Swedish population yielded a sensitivity of 90% and a specificity of 18%.
Conclusions Application of either rules, with high sensitivity, may facilitate assessment in the ED and help focus right resources on patients at a higher risk. Irrespective of the choice of decision rule, it is difficult to separate those who will repeat from those who will not due to low specificity.
- deliberate self
- mental health, self harm
- suicide
- self harm
- emergency department
Statistics from Altmetric.com
Introduction
Deliberate self-harm (DSH) is a major cause of suffering and a heavy burden on the healthcare system. The risk of repetition is high within the first year after DSH, but studies have shown that some patients repeat very quickly, within days.1–3 The risk of suicide among DSH patients is much higher than in the general population and among those who commit suicide, 20%–25% have had an episode of DSH within the year before their death.4 ,5
The high risk of suicide after DSH means that DSH patients, especially those with characteristics indicating a higher risk, such as repeated DSH, should be targeted in prevention programmes according to previous studies.6
To identify the patients at high risk of repeating their deliberate acts of self-harm is a key objective of the assessment to be able to focus psychiatric resources on these patients.7
Tools and scales have been developed to identify suicide risk and to predict subsequent suicide among DSH patients, but there is also a need for clinical tools to identify DSH patients at a high risk of repetition in order to offer these patients the right resources and care.8–12
There are several studies regarding risk factors associated with DSH and also studies regarding risk factors associated with repetition. However, only a few use these factors to predict repetition and to develop a clinical tool to facilitate assessment in the emergency department (ED).1 ,13–17 Cooper et al have developed a clinical decision rule for assessing risk after self-harm, the Manchester Self-Harm Rule, for use by ED clinical staff to identify patients at a high risk of repeating DSH or subsequent suicide within 6 months. The Manchester Self-Harm Rule uses four clinical correlates: any history of self-harm, previous psychiatric treatment, benzodiazepine use in the attempt and any current psychiatric treatment. The rule had a high sensitivity (97%), but a low specificity (26%) when validated on an English population.13
The aim of this study was to: (1) validate an existing clinical tool for assessing risk after self-harm (Manchester Self-Harm Rule) in a new setting and new population, (2) develop a clinical tool based on factors associated with DSH repetition in a Swedish population and (3) compare the ability of the two different clinical tools to predict repetition of DSH.
Materials and methods
We followed the methodological standards for development of clinical decision rules in emergency medicine.18 We used data from a consecutive series of DSH patients (n=1524) attending Södersjukhuset with one of Scandinavia's largest EDs, which serve a catchment population of approximately 600 000 inhabitants during a 3-year period (1 January 2003 to 31 December 2005). We defined the index episode as the first DSH attendance that led to a psychiatric assessment during the 3-year period or, if a psychiatric assessment was never made, the first DSH attendance at Södersjukhuset for an individual during the period.
All DSH methods were included: self-poisoning, self-injury and other methods (attempted hanging, strangulation, suffocation, etc). Self-poisoning is defined as the intentional self-administration of more than a prescribed dose of any drug, whether or not there is evidence that the act was intended to result in death. Self-injury is defined as any injury which has been deliberately self-inflicted. The motives that may be involved in DSH vary and the definition includes all DSH acts regardless of intention.19 ,20
All residents of Sweden are included in the National Registry under a unique personal identification number allocated at birth or on receiving a residence permit. Each visit to an ED, admission to hospital or open care contact is recorded with the patient's personal identification number, thus enabling automatic and accurate record linkages. The registers of Statistics Sweden and The National Board of Health and Welfare are also linked by the personal identification number.
Use of the International Classification of Diseases (ICD-10) by both somatic and psychiatric care units made it possible to identify DSH patients, to identify previous DSH episodes (2002–2005) and new acts of DSH/suicide nationwide (2003–2006) using the ICD-10 codes X60–X84.21
To collect explanatory factors in the literature previously shown to be associated with repeated DSH and variables considered in the Manchester Self-Harm Rule, we used a structured protocol to collect data manually from the hospital charts and from Statistics Sweden to collect data concerning sociodemographic factors and we used the National Board of Health and Welfare records to collect data on previous DSH episodes registered at any healthcare unit in Sweden.
Main outcome
To be able to compare the Manchester Self-Harm Rule with our results, we chose the same outcome measure: patients who deliberately self-harm again or commit suicide within 6 months after the index episode. We identified the cases from the records of the National Board of Health and Welfare, thus providing for our material a nationwide follow-up.
Explanatory factors
We included explanatory factors including sociodemographic factors, treatment and previous self-harm, method of self-harm, precipitants and circumstances of the index episode, mental and behavioural disorders at the time of the index episode (ICD-10), and factors regarding discharge after index episodes. These factors have previously been shown to be of importance for assessing the risk of repetition and are also well described in our previous study.3
Among the explanatory factors, we selected the variables that could be used in the ED by clinical staff without psychiatric expertise in order to develop a useful clinical tool for any ED with or without psychiatric consultants. This implied that we, for example, excluded the factors that were based on the psychiatric assessment: mental and behavioural disorders classified by ICD codes F01–09, F10–19, F20–29, F30–39, F40–49 and F60–69.
Statistical analysis
If data on the precipitants and circumstance of the index episode and on previous/current psychiatric and antidepressant treatment were not reported in any of the hospital records (somatic or psychiatric), we were not able to investigate this further, and so these factors were classified as ‘no’ in the statistical analysis.
We used logistic regression models to identify factors associated with the risk of repetition within 6 months. To evaluate the performance of individual factors and different classification models and to identify patients who would repeat within 6 months, that is, to discriminate between repeaters and non-repeaters, we calculated the area under the receiver operating curve (ROC: trapezoid rule). An area under the curve (AUC) equal to 0.5 suggests no discrimination, 0.7 to <0.8 is acceptable, 0.8–0.9 excellent and >0.9 outstanding discrimination.22 In order to describe the ability of the classifications model to classify patients according to risk of repetition correctly, we calculated the sensitivity (true repeater, ie, the proportion of patients predicted to repeat within 6 months according to the decision rule among those who did repeat) and specificity (true non-repeaters, ie, the proportion of patients predicted not to repeat among those who did not repeat), positive and negative predicted values, and corresponding 95% CIs. In an ideal model, both sensitivity and specificity should be high.
Our model strategy to develop a clinical decision rule to identify patients with a high risk of repetition was as follows: first, we studied the unadjusted associations between repetitions in univariable models and calculated the AUC to identify the predictive ability for each individual factor (table 1); second, a multivariable model was developed by including all factors that showed a significant unadjusted association (p<0.05). This resulted in nine possible factors that either showed a significant adjusted association and/or had a significant discrimination (AUC>0.5). We report ORs with their corresponding 95% CIs. All variables were tested by the two-sided Wald χ2 test. Finally, two-way interactions between the factors in the final model were tested. Since no interaction was detected, the logistic models only consist of main effects.
Factors associated with repeated deliberate self-harm within 6 months after the index episode and area under the curve to identify the predictive ability
In addition, we also used classification trees to identify factors associated with repetition and to stratify groups of patients according to risk of repetition. We used the CHAID algorithm to build the tree.23 A CHi-squared Automatic Interaction Detection (CHAID) analysis starts with all data in one group. Each possible split on each factor is considered to find the split that leads to the strongest association with the outcome: repetition within 6 months (yes/no). The possible nine independent factors were those that showed an adjusted significant (p<0.05) association and/or had a significant discrimination (AUC>0.5). The resulting groups were split until one of the following stop criteria was reached: tree depth was limited to three levels, no groups with less than 30 patients was formed and no split with a Bonferroni adjustment of less than 0.05 was executed. A summary of the models used to predict repetition is presented in table 2.
Summary of models used as clinical decision rules to predict patients at high risk of repeated self-harm
The statistical analysis was done using SPSS V.18.
The study was approved by the Regional Ethics Committee at Karolinska Institutet, Sweden.
Results
Baseline demographics
After the exclusion of 155 patients, 1524 patients were included in the study population.
Criteria for exclusion were: death at the index episode, patient leaving the ED before a somatic or psychiatric assessment, hospital chart not available, incorrect classification in the register (not a DSH according to the hospital chart), and personal identification number missing. The demographic data for all patients are presented in table 1. The psychiatric status indicated that 55% had an ongoing psychiatric contact, 30% had a previous psychiatric contact and 35% had a previous self-harm event. In all, 57% of all patients were treated with antidepressants.
The cumulative incidence for patients repeating their DSH actions within 6 months was 20.3% (95% CI 18.0% to 22.0%). Of the 309 patients who repeated within 6 months, 3.8% (12/309) repeated by committing suicide. Among the 309 patients who repeated, 70% were women, the mean age was 39.5 years (18–91 years) and 26% were under 25 years of age. The most common method of DSH among the repeating patients was self-poisoning (83%) and the most common drug used was benzodiazepines (42%). Self-injury, as a method for self-harm, was reported by 15% of the patients who repeated their act of self-harm within 6 months. Of the 309 patients who repeated, 64 (20.7%) were detected in another hospital than at the index time.
Validation of the Manchester Self-Harm Rule
The Manchester Self-Harm Rule uses four clinical correlates: history of self-harm, previous psychiatric treatment, benzodiazepine used in the attempt and current psychiatric treatment. The patients who have any of the above clinical correlates are classified as belonging to a high risk group and only the patients without any of the four clinical correlates are classified as belonging to a low/median risk group. Application of the Manchester Self-Harm Rule to our material gave a sensitivity of 89% and a specificity of 21%, which is comparable with, but not as good as, the results when validated on the data from the Manchester and Salford Self-Harm Project (sensitivity 97%, specificity 26%) (table 3).
Manchester Self-Harm Rule* as applied to the Swedish dataset
Possible factors in predicting repetition
The unadjusted factors we found in our Swedish material to be associated with repetition within 6 months using logistic regression were gender, parent with young children, unemployed, disability pension, current psychiatric treatment, previous psychiatric treatment, previous self-harm, antidepressant treatment, self-injury requiring a surgical procedure and a precipitant event (table 1). In the adjusted model, the following factors were significant (p<0.05): gender, parent with young children, current psychiatric treatment, previous psychiatric treatment, previous self-harm, antidepressant treatment and self-injury requiring a surgical procedure.
The classification tree analysis indicated that the four factors associated with repetition within 6 months were gender, antidepressant treatment, previous self-harm and current psychiatric treatment (figure 1). For example, the risk of repetition was 32% for patients with a previous psychiatric contact (not admitted), female gender and previous self-harm. The three different models used to classify patients at high risk for repetition are summarised in table 2.
Classification tree showing the factors which, at each step, had the strongest association with repeated DSH among patients attending the emergency department due to DSH (n=1524) within the whole study period (2003–2006). The percentage repeated DSH is calculated using the total number of patients in each node (n). The CHAID alogorithm was used to build the tree. All Bonferroni-adjusted valies were <0.05 for all nodes.
Evaluation of the ability to predict repetition
In order to evaluate the ability of the different models in table 2 to predict repetition within 6 months, we used ROC and compared AUC for the different models. Using all nine factors (Södersjukhuset Self-Harm Rule: SoS-9) in the logistic regression yielded an AUC of 0.67, while using the four factors (SoS-4) from the classification tree analysis gave an AUC of 0.64. In comparison, the Manchester Self-Harm Rule gave an AUC of 0.55 when applied to our material (figure 2).
Evalution of the ability to predict repeated deliberate (DSH) within six months in 1524 patients attending the emergency due to DSH for different models. Results expressed by the ROC.
The ROC shows the possible combination of sensitivity and specificity for the estimated risk of repetition of DSH at different cut-off points for the different models (figure 2). For example, a cut-off point of 0.14 implies that patients with an estimated risk above this point were predicted to repeat their DSH acts within 6 months, while those with a risk equal to or below 0.14 were predicted not to repeat.
Figure 2 shows that when a low cut-off point (0.14 or lower) is chosen, the sensitivity for repetition is similar for all models while the Manchester Self-Harm Rule has a lower sensitivity and specificity when higher cut cut-off points are chosen. Figure 2 also shows that the models with nine and four factors are similar with respect to AUC and sensitivity and specificity. The model with four factors gave a sensitivity of 90% (86.3 to 93.3%; 95% CI) and a specificity of 18% (16.2% to 20.1%) when a low cut-off is chosen (table 4) compared with a sensitivity of 17% (12.6% to 21.2%) and a specificity of 93% (89.9% to 92.8%) when a high cut-off point is chosen.
Södersjukhuset Self-Harm Rule (SoS-4)* as applied to the Swedish dataset according to different cut-off points
Discussion
Principal findings
This large retrospective study shows that (1) the Manchester Self-Harm Rule performed similarly in a new setting and in a new population and (2) the factors found to be associated with repetition within 6 months in our study were similar to the factors used in the Manchester Self-Harm Rule, previous self-harm and current psychiatric treatment being used in both rules (table 2). The Manchester Self-Harm Rule also used the variables benzodiazepines taken as an overdose and previous psychiatric treatment while we found antidepressant treatment and gender to be associated with repetition. Finally, (3) irrespective of whether our new clinical rule or the Manchester Self-Harm Rule is used, the ability to identify patients who will repeat was similar; it is possible to obtain a high sensitivity (ie, to identify those who will repeat), but the price for this is that most individuals are predicted to have a high risk of repetition and therefore many of those who are classified as a potential repeater will not actually repeat.
It is not without controversy to use clinical decision rules to identify patients at high risk of repetition and, for example, the NICE guidelines do not recommend the use of risk assessment tools and scales to predict future suicide or repetition of self-harm.24 Of course, the ideal would be to offer all DSH patients psychological intervention that is specifically structured for people who self-harm, with the aim of reducing self-harm, but if that not is possible due to lack of resources, availability or other reasons, decision rules is a simple way to identify patients who are at a higher risk of repetition and focus special resources to these patients.
Comparison with other studies, particularly considering any differences in results
Compared with many other studies regarding factors associated with repetition, we have taken the analysis one step further and evaluated the predictive capabilities of the associated factors to predict repetition.1 ,14 Although a factor is highly associated with repetition (eg, has an OR significantly above or below 1), this is not equivalent to being a strong predictor of repetition. For example, although patients with a self-injury requiring a surgical procedure had a fivefold higher odds to repeat DSH, this factor was not useful in predicting because the absolute number of patients in this group was low.
Methodological considerations (strength and limitations)
One of the main important points in developing and modifying a clinical decision rule for repeated DSH was that the instrument could be used in the ED by clinical staff with or without psychiatric competence. This was why such variables as the psychiatric diagnosis and the authority to which the patients were discharged, although being significantly associated with repetition, were not included in the decision rule.
In contrast to the Manchester Self-Harm Rule, which we have now validated in a new population and setting, our clinical decision rule has not been evaluated in any new individuals. Further prospective studies are therefore needed to validate these rules in new settings and other populations. Thus, it is likely that our rules will yield a lower sensitivity in a new population in a similar way that the Manchester Self-Harm Rule did in the Swedish population compared with when it was evaluated in an English population.
One reason why we had a higher proportion of individuals who repeated within 6 months (20.3%) compared with other studies is that we used a nationwide follow-up via the register of the National Board of Health and Welfare.13 ,16
Clinical implication
Both the Manchester Self-Harm Rule and the Södersjukhuset Self-Harm Rule can be of use in the clinical assessment of DSH patients by identifying those at a high risk of repeating their acts of DSH so that the right resources can be provided for this group of patients. Both rules focus on high sensitivity because the identification of patients at high risk must be of greater importance than correctly ruling out those at the lowest risk.
Predicting which patients run the risk of repeating their deliberate acts of self-harm is a difficult task with or without clinical decision rules, but the combination of clinical assessment and a clinical decision rule might improve the accuracy.
References
Footnotes
Contributors All authors conceived and designed the study. KB, SP and CO conducted the data collection. KB and HP managed the data and analysed the data. HP provided statistical advice on study design and analysing the data. KB and HP drafted the manuscript, and all authors contributed substantially to its revision. KB takes responsibility for the paper as a whole.
Funding Research Fund of Stockholm County Council, Karolinska Institutet, KID funding.
Competing interests None.
Ethics approval Ethics approval provided by the Regional Ethics Committee at Karolinska Institutet, Sweden.
Provenance and peer review Not commissioned; externally peer reviewed.