External validation of prognostic models among cancer patients undergoing emergency colorectal surgery

doi:10.1016/j.amjsurg.2007.03.012

The American Journal of Surgery

Volume 195, Issue 4, April 2008, Pages 439-441

https://doi.org/10.1016/j.amjsurg.2007.03.012 Get rights and content

Abstract

Background

The aim of this study was to evaluate the predictive accuracy of different scoring systems on patients undergoing emergency colorectal surgery.

Methods

The Acute Physiology and Chronic Health Evaluation II or III, the Simplified Acute Physiology Score II, the Mortality Probability Model II, and the Colo-rectal POSSUM scoring systems were applied to 102 patients who underwent colorectal resection for cancer. Validation of scoring systems was tested by assessing calibration and discrimination. Calibration was assessed using Hosmer-Lemeshow goodness-of-fit test and the corresponding calibration curves. Evaluation of the discriminative capability of both models was performed using receiver-operating characteristic curve analysis.

Results

Overall, 17 deaths occurred. The Simplified Acute Physiology Score II showed good calibration (x² = 1.079, P = .982) and discrimination (areas under the receiver-operating characteristic curve .83).

Conclusions

These data suggest that the SAPS II scoring system was accurate in predicting outcome for patients undergoing emergency colorectal surgery.

Section snippets

Methods and Materials

A total of 102 patients who underwent emergency colorectal surgery for colorectal cancer in Ankara Numune Education and Research Hospital between January 1998 and July 2004 were included the study. Emergency indications for surgery were defined, including (1) patients with history and examination consistent with peritonitis, (2) patients with intra-abdominal abscess with systemic signs of sepsis (fever, increased white blood cell count, and/or signs of hemodynamic instability, and (3) patients

Results

Postsurgical mortality was 17 % (17 of 102). Mean age of the 102 (52 male and 50 female) study patients was 61 years (range 18 to 97). Demographic distribution of the patient group is listed in Table 1.

Comments

Intensive care is a costly area for which resources are limited. Decisions related to resource use and treatment plans must be based on objective data, which are generally provided by disease severity indices and mortality predictions. Performances of the commonly used disease severity scoring and mortality prediction systems may be affected by advances in technology and therapeutic strategies because the system have been described by differences in case mix of specific clinic from an original

Acknowledgments

The authors thank Mahmut Mutlu Dasdag (Department of Biostatistics, Dicle University School of Medicine) for statistical review.

References (15)

J.A. Alvarez et al.
Presentation, treatment, and multivariate analysis of risk factors for obstructive and perforative colorectal carcinoma
Am J Surg
(2005)
M. Soares et al.
Performance of six severity-of-illness scores in cancer patients requiring admission to the intensive care unit: a prospective observational study
Crit Care
(2004)
S. den Boer et al.
Performance of prognostic models in critically ill cancer patients–a review
Crit Care
(2005)
P.P. Tekkis et al.
Development of a dedicated risk-adjusment scoring system for colorectal surgery (colorectal POSSUM)
Br J Surg
(2004)
Available and open access online at http://www.icu.hacettepe.edu.tr/micis.html. Accessed January 9,...
J.A. Hanley et al.
The meaning and the use of the area under a receiver operating characteristic (ROC) curve
Radiology
(1982)
Lemeshow S, Hosmer Jr DW. A review of goodness of fit statistics for use in the development of logistic regression...

There are more references available in the full text version of this article.

Cited by (13)

Risk assessment tools validated for patients undergoing emergency laparotomy: A systematic review
2015, British Journal of Anaesthesia
Citation Excerpt :
Of six tools specific to colorectal surgery, CR-POSSUM alone was assessed in multiple studies. Discrimination of 30-day outcome after unplanned surgery for colorectal cancer by CR-POSSUM was moderate or poor (AUC 0.65, 0.72).48 52 Of four identified studies of tools specific to peritonitis, the Mannheim Peritonitis Index (MPI) and Peritonitis Index of Altona II (PIA II) underwent multiple assessments of external validity.
Emergency laparotomies are performed commonly throughout the world, but one in six patients die within a month of surgery. Current international initiatives to reduce the considerable associated morbidity and mortality are founded upon delivering individualised perioperative care. However, while the identification of high-risk patients requires the routine assessment of individual risk, no method of doing so has been demonstrated to be practical and reliable across the commonly encountered spectrum of presentations, co-morbidities and operative procedures. A systematic review of Embase and Medline identified 20 validation studies assessing 25 risk assessment tools in patients undergoing emergency laparotomy. The most frequently studied general tools were APACHE II, ASA-PS and P-POSSUM. Comparative, quantitative analysis of tool performance was not feasible due to the heterogeneity of study design, poor reporting and infrequent within-study statistical comparison of tool performance. Reporting of calibration was notably absent in many prognostic tool validation studies. APACHE II demonstrated the most consistent discrimination of individual outcome across a variety of patient groups undergoing emergency laparotomy when used either preoperatively or postoperatively (area under the curve 0.76–0.98). While APACHE systems were designed for use in critical care, the ability of APACHE II to generate individual risk estimates from objective, exclusively preoperative data items may lead to better-informed shared decisions, triage and perioperative management of patients undergoing emergency laparotomy. Future endeavours should include the recalibration of APACHE II and P-POSSUM in contemporary cohorts, modifications to enable prediction of morbidity and assessment of the impact of adoption of these tools on clinical practice and patient outcomes.
Predicting smoking relapse with a multidimensional versus a single-item tobacco craving measure
2013, Drug and Alcohol Dependence
Citation Excerpt :
Results were considered significant if different from AUC = 0.50 at p ≤ 0.05. In many studies (Swets, 1988; Andresen, 2000; Salter et al., 2007; Ertan et al., 2008; Sivan et al., 2011), discrimination was considered excellent if AUC ≥ 0.90, adequate if AUC = 0.70–0.89, and poor if AUC < 0.70. ROC analyses were performed using MedRoc, Version 1.1 (StenStat, Michigan City, IN).
Research suggests that craving is a predictor of smoking relapse. Craving can be assessed by multiple item or multifactorial scales or by single items. However, no systematic comparisons of their prognostic validity or accuracy have been published.
The French versions of the 12-item Tobacco Craving Questionnaire (FTCQ-12) and the single craving item on the Minnesota Nicotine Withdrawal Scale (MNWS) are brief, valid, and reliable self-report measures of tobacco craving. In this secondary study, we analyzed data from French smokers with health-related problems enrolled in the Adjustment of DOses of NIcotine in Smoking (ADONIS) cessation trial. We estimated prediction models for each measure and compared their ability to distinguish correctly participants who relapsed from those who did not at 1–8 weeks after their quit date.
Adjusted for all potential confounders FTCQ-12 risk score (RS; Factor 2, Expectancy plus Factor 4, Purposefulness) and MNWS craving were valid predictors of smoking relapse at endpoints measured 1–7 weeks apart. Prognostic accuracy of FTCQ-12 RS was greatest at 1–2 weeks follow-up compared to only 1 week for MNWS craving. Sensitivity for FTCQ-12 RS and MNWS craving was 85% and 53%, respectively.
FTCQ-12 RS suggests a relapse process involving urges and desires in anticipation of the positive benefits of smoking linked with intent and planning to smoke. Findings also suggest that FTCQ-12 RS may be a better predictor instrument for smoking relapse than MNWS craving.
When will we actually conduct a risk-adjusted surgical audit worldwide?
2009, American Journal of Surgery
Evaluating and improving current risk prediction tools in emergency laparotomy
2020, Journal of Trauma and Acute Care Surgery
Performance of the Surgical Outcomes Monitoring and Improvement Program in comparison to Portsmouth Physiological and Operative Severity Score for enUmeration of Mortality and Morbidity and National Surgery Quality Improvement Program calculators in predicting mortality for emergency colectomy: A retrospective case series study
2019, Surgical Practice
Survival in solid cancer patients following intensive care unit admission
2014, Intensive Care Medicine

View all citing articles on Scopus

View full text

Clinical surgery—InternationalExternal validation of prognostic models among cancer patients undergoing emergency colorectal surgery

Abstract

Background

Methods

Results

Conclusions

Section snippets

Methods and Materials

Results

Comments

Acknowledgments

Am J Surg

Performance of six severity-of-illness scores in cancer patients requiring admission to the intensive care unit: a prospective observational study

Crit Care

Performance of prognostic models in critically ill cancer patients–a review

Crit Care

Development of a dedicated risk-adjusment scoring system for colorectal surgery (colorectal POSSUM)

Br J Surg

The meaning and the use of the area under a receiver operating characteristic (ROC) curve

Radiology

Clinical surgery—International
External validation of prognostic models among cancer patients undergoing emergency colorectal surgery