Trauma survival prediction in Asian population: a modification of TRISS to improve accuracy
- 1Department of Surgery, Queen Elizabeth Hospital, Kowloon, Hong Kong SAR
- 2Department of Management Sciences, City University of Hong Kong, Kowloon, Hong Kong SAR
- Correspondence to Dr Canon King OnChan, Department of Surgery, Queen Elizabeth Hospital, Block H, 10/F, Kowloon, Hong Kong SAR;
- Received 7 August 2012
- Revised 14 December 2012
- Accepted 18 December 2012
- Published Online First 12 January 2013
The probability of survival (PS) in blunt trauma as calculated by Trauma and Injury Severity Score (TRISS) has been an indispensable tool in trauma audit. The aim of this study is to explore the predictive performance of the latest updated TRISS model by investigating the Age variable recategorisations and application of local Injury Severity Score (ISS) and Revised Trauma Score (RTS) coefficients in a logistic model using a level I trauma centre database involving Asian population.
Methods Prospectively and consecutively collected 5684 trauma patients’ data over a 10-year period at a regional level I trauma centre were reviewed. Four modified TRISS (mTRISS) models using Age coefficient from reclassifications of the Age variable according to their correlation with survival by logistic regression on the local dataset were acquired. RTS and ISS coefficients were derived from the local dataset and then applied to the mTRISS models. mTRISS models were compared with the existing Major Trauma Outcome Study (MTOS)-derived TRISS (eTRISS) model. Model 1=Age effect taken as linear; Model 2=Age classified into two groups (0–54, 55+); Model 3=Age classified into four groups (0–15, 16–54, 55–79, 80+) and Model 4=Age classified into two groups (0–69, 70+). Performance measures including sensitivity, specificity, accuracy and area under the Receiver Operating Characteristic (ROC) curve were used to assess the various models. The cross-validation procedure consisted of comparing the PS obtained from mTRISS Models 1 and 2 with the PS obtained from the MTOS derived from eTRISS.
Results A 5147 blunt trauma patients’ dataset was reviewed. Model 1, where Age was taken as a scale variable, demonstrated a substantial improvement in the survival prediction with 91.6% accuracy in blunt injuries as compared with 89.2% in the MTOS-derived TRISS. The 95% CI for ROC derived from mTRISS Model 1 was (0.923, 0.940), when compared with the hypothesised ROC value 0.886 obtained from eTRISS, it clearly indicated a significant improvement in predicting survival at 5% level. Furthermore, ROCs have shown clearly the superiority of Model 1 over Model 2, and of Model 2 over MTOS-derived TRISS. The recategorisation of the Age variable (Models 3 and 4) also demonstrated improved performance, but their strength was not as intense as in Model 1. Overall, the results point to the adoption of Model 1 as the best model for PS. Cross-validation analysis has further assured the validity of these findings.
Conclusions The present study has demonstrated that (1) having the Age variable being dichotomised (cut-off at 55 years) as in the eTRISS, but with the application of a local dataset-derived coefficients give better TRISS survival prediction in Asian blunt trauma patients; (2) improved performance are found with certain recategorisation of the Age variable and (3) the accuracy can further be enhanced if the Age effect is taken to be linear, with the application of local dataset-derived coefficients.
The probability of survival (PS) as calculated by Trauma and Injury Severity Score (TRISS) has been an indispensable tool in trauma audit, and since its introduction in 1987 by Boyd and colleagues,1 it continues to be the most commonly used tool for benchmarking trauma outcome. However, TRISS had been faced with criticisms in terms of its accuracy and inappropriate use of coefficients2–6 since its revision by the American College of Surgeons Committee on Trauma coordinated Major Trauma Outcome Study (MTOS).7 ,8 One important argument was against the dichotomised Age variable at 55 years in the TRISS methodology, which was not clearly addressed in the original publication.1 ,7 As traumatology is evolving into a geriatric discipline, and more reports have indicated that hospitalisation rates for trauma increases with age,9–11 age has represented an independent predictor of survival in the trauma population.12 ,13 In addition, external validation of MTOS PS on a population with very different population characteristics provides evidence that data-derived PS are robust to case-mix variation, which has been proven by studies in the UK and Canada.14 ,15 It is possible that age misclassification may have led to an underestimation of injury severity in these patients.16 Many studies, thus, have emerged to claim that the predictive performance of TRISS can be improved by reclassifying the variables and even treating the categorical variables nominally,16 ,17 although evidence in this area remained lacking. There is also literature suggesting that local database-derived coefficients might further enhance the predictive power of TRISS.18 ,19 Nonetheless, whether or not this argument still stands after the latest updated TRISS coefficients published in 20108 remained to be elucidated. So far, a large amount of effort has been devoted to improving TRISS based on comparing western databases with the old version of MTOS-derived TRISS,7 but research work on comparing Asian populations with the latest TRISS and its updated coefficients8 have been scarce. This study is, therefore, aimed to verify whether the latest updated MTOS-derived TRISS model can be improved by Age-variable recategorisations and application of local Injury Severity Score (ISS) and Revised Trauma Score (RTS) coefficients in a logistic model using a level I trauma centre database involving Asian blunt trauma patients.
The Queen Elizabeth Hospital (QEH) trauma database was comprised of more than 5600 trauma patients’ data collected over a 10-year period (2000–2010) at a regional level I trauma centre in Hong Kong. Patient data included Emergency Department admission vital parameters, type of trauma, Glasgow Coma Scale (GCS), ISS, RTS and age. All records were individually reviewed by a single trauma specialty dedicated nurse, and the ISS and RTA were subsequently calculated. ISS is computed as the sum of squares of the highest Abbreviated Injury Scale in three body regions.20 ,21 RTS is given by weighed combination of respiratory rate, systolic blood pressure and GCS.22 Trauma outcome including survival or mortality was based on assessment at discharge, or 90 days, whichever was earlier. Survival of the patients were computerised and entered into a hospital database, which was also linked up to the regional death registry, hence, there was high precision of survival status imputation. As only retrospective anonymous data are stored, ethics committee approval was not required for this study.
The TRISS model
The original TRISS model has two separate specifications on coefficients for adults above or equal to 15 years of age, namely injury sustained from blunt or penetrating mechanisms. TRISS coefficients will give the PS as a result:where b=b0+b1(RTS)+b2(ISS)+b3(Age)
This study focused on the evaluation of TRISS survival prediction in blunt trauma patients and, hence, only the ‘blunt injury’ specifications of the TRISS were included for comparison.
The initial part of the study investigated the relationship of Age-variable modification and trauma survival. The dependent variable was the survival of patient at discharge, or 90 days from admission. The independent variables were different categorisations of Age. The justification to use of Age as a scale variable was tested by the Box–Tidwell Test. Also, logit residual (after fitting RTS and ISS as covariates in logistic regression) was plotted against Age to assess the linear functional form of the Age effect. Subsequent to this initial analysis, four new models with modification of the Age variable in the original TRISS formula would be derived from logistic regression and would be prefixed as mTRISS Models 1–4 in this study. New coefficients for b0 (constant), b1 (ISS) and b2 (RTS) developed from the QEH dataset were employed to calculate the probability of survival in the four new mTRISS models. The performance of the mTRISS models was then compared with the existing MTOS TRISS model, abbreviated as eTRISS, after imputation of QEH patients’ survival status. The eTRISS utilised the latest updated coefficients published by Schluter and coworkers8 to calculate the probability of survival of blunt trauma patients in the QEH dataset. mTRISS Model 1 treated Age as a scale variable. Model 2 represented the MTOS formula except with local coefficients applied. Model 3 recategorised the Age variable into four groups (0–15, 16–54, 55–79, 80+). Model 4 recategorised the Age variable into two groups (0–69, 70+). These models allowed evaluation of the effect of different recategorisation of the Age variable and the application of local dataset-derived coefficients on the survival prediction performance relative to the latest updated MTOS TRISS methodology.
Internal validity was evaluated by discrimination and calibration. Discrimination of the predictive ability of each mTRISS model was assessed on the eTRISS model using the Receiver Operating Characteristic curves (ROC). The Area under the ROC (AUC) would represent the sensitivity and specificity of the model and shows the ability of the model to distinguish between positive and negative outcome, its value varies between 0.5 (no accuracy) and 1.0 (perfect accuracy). AUC 95% CI were calculated, and significant difference was detected if the 95% CI did not cover the hypothesised value (AUC derived from MTOS). Hosmer–Lemeshow Goodness-of-Fit Test also applied to assess the adequacy of the fitted mTRISS models.
To assess the validity of the results, Kappa-fold cross-validation procedure was conducted to validate the PS derived from the representative mTRISS models (Models 1 and 2). The advantage of K-fold cross-validation was that all the samples in the dataset were eventually used for both training and validating. K=3 (ie, threefold) was chosen after taking into consideration the size of data, percentage mortality and avoidance of overlapping use of subset data to estimate the parameters. In threefold cross-validation, the data were first partitioned into three nearly equal-sized folds. Subsequently, three iterations of training and validation were performed such that one fold was used for validation while the remaining two folds were used as training data. Model prediction performance was likewise evaluated with the ROC and AUC 95% CI. In general, if average coefficients derived from the training subsets and the corresponding average prediction performance measures agree closely with those derived from the entire QEH dataset, the use of the coefficients derived from the QEH dataset can further be assured due to this cross-validation analysis. Comparisons between categorical variables were made using the χ2 test. Statistical analysis was performed using IBM SPSS 20.0 for Windows.
Over a period of 10 years, there were a total of 5684 anonymous trauma cases in the QEH dataset and 5147 (90.5%) cases were secondary to blunt injuries (table 1). Patients of Chinese ethnicity contributed to the majority of the dataset (92.5%), while the remaining was composed of other Asian ethnicities (7%), and Caucasian accounted for only 0.5% of the study population. Missing data accounted for 0.02% (1 out of 5147) of the database, and such missing data were excluded from the corresponding analysis. All the component TRISS variables, as well as the survival status were available in the hospital dataset. Intubated patients were given the best score on the GCS before their intubation, and the lowest score of 3 represented unrecordable GCS. Similarly, GCS was assessed before the patient became sedated, or else the lowest score of 3 was given if GCS could not be recorded after sedation. Overall mortality in our dataset was 11.9%. The percentages of overall and blunt trauma patients over age 55 years were 38.3% and 40.6%, respectively. The majority of trauma patients were male accounting for 69% of cases, and the most common mechanism of injury was road traffic accidents (35.6%) (table 2), followed by domestic injuries (31.6%), and then assault cases (14.3%). The mean ISS, which represented the anatomical description of the severity of injury, was 12.62.
As an initial exploration, the relationship between different Age categorisation with the survival of blunt trauma patients is presented in table 3. Models 2–4 have shown that survival was negatively associated with increasing age, as illustrated by the negative sign in the Spearman rank correlation. In particular, such correlation changed from −0.271 (Model 2) to −0.300 and −0.308 (in Models 3 and 4, respectively). These findings suggest that revised Age categorisations are potentially useful in improving the discriminative power of the Age variable when applying the TRISS model. In addition, taking the Age as a scale variable (Model 1) is considered as a viable alternative that can demonstrate competitiveness with those Age-categorised models.
To assess the linear functional form of the Age effect, figure 1 gives the plot of Logit Residual versus Age, after running the logistic regression model with ISS and RTS as covariates. The fitted linear regression line was found: Logit Residual=2.593–0.047 ×Age, with significant linear Age effect (p value<0.001). In addition, the Box–Tidwell test did not give a significant Age*ln(Age) effect (p value=0.408) when having the extra term Age*ln(Age) in addition to Age in the logistic regression analysis. Summing up, no departure from linearity was detected for the Age variable and, hence, the linear Age functional form in the logistic regression was justified.
The b coefficients on blunt trauma for the TRISS models obtained from QEH dataset are presented in table 4. These new coefficients were acquired, using logistic regression analysis on the ISS and RTS scores in the QEH dataset. Together with the new Age coefficients developed from the logistic regression analysis, the Survival Prediction (PS) can be calculated from each mTRISS models. The accuracy and internal validity of the mTRISS models were assessed using a ROC analysis and their corresponding 95% CI.
Latest updated MTOS8 b coefficients were derived directly from the covariates: (RR, SBP, GCS, ISS, Age), where Age is in two groups (0–54, 55+). The recommended logit prediction equation (eTRISS model)8 for adult blunt mechanism injuries was: 1.6494+0.0095×RR+0.4260×SBP+0.6307×GCS−0.0795× ISS−1.6216×Age.
Analysis of sensitivity, specificity and accuracy in the four mTRISS models and eTRISS model showed that Model 1, where Age effect was assumed linear, demonstrated a substantial improvement in the survival prediction when QEH dataset was imputed with 91.6% accuracy (table 5). This result was further supported by the AUC (figure 2). The eTRISS ROC value derived from MTOS was 0.886 for blunt injuries, whereas the ROC 95% CI generated from mTRISS Model 1 was 0.923–0.940 for blunt injuries. Clearly, the ROC 95% CI did not cover the hypothesised value in both cases, implying that the proposed mTRISS Model 1 is significantly better than the eTRISS in predicting survival at 5% level. The accuracy of Model 1 consistently showed superiority over the other models in blunt trauma survival prediction (table 5). In addition, the best calibration was obtained from Model 1 as it was found to fit the data adequately in predicting survival according to the Hosmer–Lemeshow goodness-of-fit test.
Information about the cross-validation analysis using the k-fold cross-validation procedures is provided in tables 6⇓–8. Three non-overlapping subsets of data with specified sample size were randomly generated with the aid of the data function in SPSS. Table 6 shows the similar baseline characteristics of the three subsets of data obtained from the QEH dataset for training and validation purposes. Among the three data subsets, no significant differences were found for variables Age, Sex, ISS and Mortality. Only Models 1 and 2 were taken in this cross-validation analysis. Model 2 is essentially the MTOS TRISS model, but applying locally derived coefficients. Model 1 has shown its superiority over the recategorised Age Models 3 and 4. Table 7 shows the average coefficients estimated from the training data subsets as compared with those estimated from the full dataset. In general, both sets of results agreed closely with each other. Furthermore, the assessment according to prediction accuracy measures in table 8 showed there was effectively no difference between the AUC of Models 1 and 2 in the threefold (average) and in the full dataset (0.931 vs 0.932 in Model 1 and 0.919 vs 0.920 in Model 2).
TRISS has been a widely used method in day-to-day trauma management, auditing, as well as research.15 ,23–27 Its validity has been proven by publications worldwide following its introduction in the 1990s.24 ,28–32 Despite this, many have tried to improve on this model by introducing new and more complex methodologies. Some added factors were associated with survival, such as vital parameters on admission6 ,19 ,33–37 or comorbidities,12 ,13 ,38–41 while others applied modifications to the ISS and RTS.42–46 Nonetheless, none appear to achieve the same level of acceptance and popularity as the original TRISS, therefore, it may be more feasible to improve on the TRISS model instead. This is the aim of the present study, which has made an attempt to improve the existing TRISS model based on an Asian dataset involving mainly adult Asian trauma patients.
Our study has shown that the TRISS model in predicting PS can be improved and can be tailored to the local population by adopting local b coefficients and treating Age as a scale variable in the formula. From the results of the ROCs, Model 1 is the optimal choice for blunt type of injury prediction as it has the largest AUC. As verified by the comparison using ROC 95% CIs, for Model 1 the ROC 95% CI were 0.923 and 0.940, in which the hypothesised value 0.886 does not fall in. In other words, Model 1 is significantly better than the existing MTOS-derived TRISS in predicting trauma survival at 5% level of significance. As represented on the graphical representation of the Logit Residual versus Age in our results, the relationship was linear, and hence, it was justified to be treated as a scale variable in Model 1, instead of a binary indicator (which was originally proposed in the TRISS).20 This finding concurs with that recently published by Schluter et al who illustrated the improvement of TRISS by reclassifying the component variables and treating the variable categories nominally.17 The reason for this superiority could relate to a better resemblance of the clinical situation when Age was represented as scale variable, because setting the linear Age effect allows the gradual decrement impact of Age toward trauma survival probability. In the original TRISS, the two-tier Age categorisation at age 55 years may have led to over/underprediction of survival,16 ,47 ,48 which raised the important argument against the use of dichotomised Age variable.1 ,7 In our initial analysis on the effect of Age recategorisations on blunt trauma survival, we found that the Spearman rank correlation was weakest when Age was divided at 55 years in our dataset (table 3). Elevated correlation strength and prediction accuracy was found in Models 3 and 4 in which a recategorisation of the Age was done. When Age effect was taken as linear, Model 1 clearly demonstrated its superiority over the other three models. The performance of the various models was clearly seen by ROC analysis in table 5. In Models 3 and 4, we chose to stratify Age into the respective categories, as patients within each age category shared similar survival probability in our sample, and this is also a commonly utilised stratification method by previous authors.39 ,49
Revisiting the MTOS-derived TRISS model based on local trauma dataset is not a novel idea, previous studies with similar design have shown superiority in PS estimates based on local datasets.27 ,50–54 Studies based on this idea also gave inspirations as to the factors that may affect the performance of survival estimation by the MTOS TRISS. Based on a previous study, the predicted mortality was lower than the observed mortality by up to 15% using MTOS TRISS54 and, hence, the importance of a regional database to develop local coefficients has been put forward. Both institution-bound factors and specific limitations in the TRISS methodology were responsible for the difference between predicted and observed mortality. Institution-bound factors may include regional trauma demographics, interobserver difference in assessing ISS and RTS, trauma triage facilities and the availability of a multidisciplinary team for trauma management. Another limitation in the TRISS is the failure to provide accurate estimates for injuries other than blunt or penetrating as well as for subgroup trauma patients, such as the paediatric populations. In view of this, various authors have introduced new or modified methods based on the original TRISS, and aimed to improve the PS for subgroup trauma patients.43 ,55
The limitations on the adoption of MTOS-derived TRISS model coefficients were also discussed in the literature. First of all, since the TRISS derived its formula coefficients from the MTOS dataset in 1995, the level of care currently expected has made the TRISS unexpected survivors a statistical phenomenon only, as the trauma centre performance improved over the years. This is the reason for our study utilising the latest updated TRISS coefficients published in 20108 for use as comparison with coefficients derived from our local dataset in the hope of ameliorating this chronological bias in trauma care.56 Second, the TRISS model may have used the Age variable in a mathematically inappropriate way that impairs the calibration, and some authors have attempted to modify this accordingly.57 It was subsequently found that by improving the stratification for Age, the predictive accuracy of the TRISS model would significantly increase,39 a finding which was consistent in our study. Third, considering that TRISS was a logistic regression model with standardised coefficients based on the MTOS data, it would seem logical that revision of these coefficients would further improve the ability of TRISS to predict survival. This was another reason in support for generating and applying a new set of coefficients from our hospital dataset to test against the MTOS-derived coefficients. Our study has clearly demonstrated the improvement of PS by this modification based on the comparison between Model 2 and TRISS. The above reasoning together with our findings support the use of locally derived coefficients instead of those derived from the MTOS for regional trauma auditing.
There were not many studies of TRISS based on Asian datasets. Shortly after the publication of MTOS TRISS, Yang et al58 described his work in 1999 based on 1297 Chinese trauma patients, and in an attempt to obtain new weighted coefficients through conducting a logistic regression analysis between the outcome and the injury severity. The study found a slightly superior performance based on the coefficients derived from local database when compared with the MTOS, and suggested that a revision of TRISS's weighted coefficients was necessary. The improvement on TRISS after revising the weight coefficients developed from the QEH dataset was likewise demonstrated in our study. In addition, the coefficients derived from our hospital dataset (based on more than 5000 trauma patients) projected a higher power PS prediction in our mTRISS models than in the previous studies on Asian patients.58–61
TRISS is an equation with weighted coefficients, and the improvement in prediction by the revision of the b coefficients may reflect the influence of trauma demographic differences between trauma institutions in the prediction of survival.2 ,47 ,62 This is one limitation of the current study, which relates to its external validity. The study population is based on a regional trauma centre in Hong Kong, and the ratio of blunt-injured to penetrating-injured patients was around 9 : 1, whereas in the MTOS it was 4 : 1. The overall mortality rate from our dataset was 11.9%, and that observed in the MTOS was 9.0%. Motor vehicle-related injuries were prevalent in the MTOS and accounted for almost half the total injured (49.1%).7 However, in the QEH dataset, there was only 35.6% due to such injury. Dissimilarity between trauma populations appeared to be a common limitation among studies on trauma-scoring systems. This study population does, however, represent the Hong Kong trauma experience,63 and comes close to that of many other Asian countries, such as Japan19 and Thailand, where trauma patient demographic are similar.64 ,65 Our study made an attempt to utilise the cross-validation procedures to assess the validity of the results obtained from our models. Our models used coefficients derived using the training data subset after its prediction performance was tested on the validation subset. The results indicated that the models were satisfactory and the main conclusions of this study are likely to hold in trauma patients with similar baseline characteristics, and in centres with a similar level of trauma care.
Both the MTOS study and ours calculated PS excluding patients with comorbidities, but this along with other factors may come to play or even exert a larger role in survival prediction than the ISS, RTS and Age. It has been shown that the TRISS-derived model with the additional term for the presence of comorbidity was associated with a higher accuracy for PS.39 ,66 It is therefore possible that lack of information on comorbidities could have affected the validity of study results,66 but whether or not this has any clinical significance remains to be elucidated.41 Lastly, penetrating injuries were not analysed because their prevalence in our centre was less than 10%.
Our study has shown that the TRISS model in predicting PS can be improved by simply replacing the existing coefficients with ones developed from the local dataset, and treating the Age as scale variable. This allows a higher level of survival prediction that tailors to the local trauma care and compromises systemic bias due to interinstitutional-bound factors. Our model with Age considered as scale variable should be replicated in different datasets for further validation. Those relying on the original TRISS for benchmarking trauma performance should consider the continuing change of trauma demographics and interinstitutional practice when applying the model for trauma auditing.
The present study has shown that the application of local dataset-derived TRISS coefficients, and the modification of Age variable to scale variable resulted in a significantly better survival prediction in blunt trauma patients. The results of our study should shed light on the possibility of altering the Age variable to a linear scale in the MTOS TRISS as well as applying local b coefficients to improve trauma survival prediction in any regional trauma database.
Special thanks goes to Miss Annice Chang, a dedicated trauma nurse specialist, who had provided most effortful support to the trauma databank of our hospital, without which this study could not have been possible.
Collaborator Annice Chang.
Contributors CKOC has contributed to the drafting and revising this paper critically for important intellectual content. KKWY has contributed to the data analysis and interpretation of results in this study. MT C has contributed to the conception, design, as well as the final approval of the version to be published. CKOC is responsible for the overall content as guarantor.
Source(s) of support The present study is not supported by any grant or institutional funding.
Competing interests None.
Provenance and peer review Not commissioned; externally peer reviewed.