Abstract
OBJECTIVE: To evaluate the predictive validity and calibration of the pneumonia severity-of-illness index (PSI) in patients with community-acquired pneumonia (CAP).
PATIENTS: Randomly selected patients (n=1,024) admitted with CAP to 22 community hospitals.
MEASUREMENTS AND MAIN RESULTS: Medical records were abstracted to obtain prognostic information used in the PSI. The discriminatory ability of the PSI to identify patients who died and the calibration of the PSI across deciles of risk were determined. The PSI discriminates well between patients with high risk of death and those with a lower risk. In contrast, calibration of the PSI was poor, and the PSI predicted about 2.4 times more deaths than actually occurred in our population of patients with CAP.
CONCLUSIONS: We found that the PSI had good discriminatory ability. The original PSI overestimated absolute risk of death in our population. We describe a simple approach to recalibration, which corrected the overestimation in our population. Recalibration may be needed when transporting this prediction rule across populations.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Medicare Hospital Mortality Information. Baltimore, Md: U.S. Department of Health and Human Services, Health Care Financing Administration; 1992.
Fine MJ, Smith MA, Carson CA, et al. Prognosis and outcomes of patients with community-acquired pneumonia: a meta-analysis. JAMA. 1996;275:134–41.
Gilbert K, Fine MJ. Assessing prognosis and predicting patient outcomes in community-acquired pneumonia. Semin Respir Infect. 1994;9:140–52.
Iezzoni LI, Shwartz M, Ash AS, Mackiernan YD. Using severity measures to predict the likelihood of death for pneumonia inpatients. J Gen Intern Med. 1996;11:23–31.
American Thoracic Society. Guidelines for the initial management of adults with community-acquired pneumonia: diagnosis, assessment of severity, and initial antimicrobial therapy. Am Rev Respir Dis. 1993;148:1418–26.
Farr BM, Sloman AJ, Fisch MJ. Predicting death in patients hospitalized for community-acquired pneumonia. Ann Intern Med. 1991;115:428–36.
Brewster AC, Karlin BG, Hyde LA, et al. MEDISGRPS: a clinically based approach to classifying hospital patients at admission. Inquiry. 1985;12:377–87.
Fine MJ, Hanusa BH, Lave JR, et al. Comparison of a disease-specific and a generic severity of illness measure for patients with community-acquired pneumonia. J Gen Intern Med. 1995;10:359–68.
Iezzoni LI. The risks of risk adjustment. JAMA. 1997;278:1600–7.
Fine MJ, Singer DE, Hanusa BH, Lave JR, Kapoor WN. Validation of a pneumonia prognostic index using the MedisGroups comparative hospital database. Am J Med. 1993;94:153–9.
Lemeshow S, Le Gall JR. Modeling the severity of illness of ICU patients. JAMA. 1994;272:1049–55.
Iezzoni LI, Shwartz M, Ash AS, et al. Severity measurement methods and judging hospital death rates for pneumonia. Med Care. 1996;34:11–28.
Wasson JH, Sox HC, Neff RK, Goldman L. Clinical prediction rules: application and methodologic standards. N Engl J Med. 1985;313:793–9.
Hanley JA, McNeil BJ. The meaning and use of the area under a receiver operating characteristic (ROC) curve. Radiology. 1982;143:29–36.
Iezzoni LI. Risk Adjustment for Measuring Health Care Outcomes. Ann Arbor, Mich: Health Administration Press; 1994.
Hosmer DW, Lemeshow S. Applied Logistic Regression. New York, NY: J. Wiley & Sons; 1989.
Zeger SL, Liang KY. Longitudinal data analysis for discrete and continuous outcomes. Biometrics. 1986;42:121–30.
Iezzoni LI, Ash AS, Schwartz M, Daley J, Hughes JS, Mackierman YD. Predicting who dies depends on how severity is measured: implications for evaluating patient outcomes. Ann Intern Med. 1995;123:763–70.
Blumberg MS. Biased estimates of expected acute myocardial infarction mortality using MedisGroups admission severity groups. JAMA. 1991;265:2965–70.
Daley J, Jencks S, Draper D, Lenhart G, Thomas N, Walker J. Predicting hospital-associated mortality for Medicare patients: a method for patients with stroke, pneumonia, acute myocardial infarction, and congestive heart failure. JAMA. 1998;260:3617–24.
Houston MS, Silverstein MD, Suman VJ. Risk factors for 30-day mortality in elderly patients with lower respiratory tract infection: community based study.Arch Intern Med. 1997;157:2190–5.
Kleinbaum DG, Kupper LL, Cassel JC, Tyroler HA. Multivariate analysis of risk of coronary heart disease in Evans County, Georgia. Arch Intern Med. 1971;128:943–8.
Keys A, Aravanis C, Blackburn H, et al. Probability of middle-aged men developing coronary heart disease in five years. Circulation. 1971;45:815–28.
Laurier D, Chau PNP, Cazelles B, Segond P, and the PCV-METRA Group. Estimation of CHD risk in a French working population using a modified Framingham model. J Clin Epidemiol. 1994;47: 1353–64.
Brand RJ, Rosenman RH, Sholtz RI, Fridman M. Multivariate prediction of coronary heart disease in the Western Collaborative Group Study compared to findings of the Framingham Study. Circulation. 1976;53:348–55.
Leaverton PE, Sorlie PD, Kleinmen JC, et al. Representativeness of the Framingham risk model for coronary heart disease mortality: a comparison with a national cohort study. J Chron Dis. 1987;40:775–84.
Carmines EG, Zeller RA. Reliability and validity assessment. In: Quantitative Applications in the Social Sciences, Number 17. Newbury Park, Calif: Sage Publications. 1979;9–11.
Poses RM, Cebul RD, Collins M, Fager SS. The importance of disease prevalence in transporting clinical prediction rules: the case of streptococcal pharyngitis. Ann Intern Med. 1986;105:586–91.
Gustafson DH, Fryback DG, Rose JH, et al. A decision theoretic methodology for severity index development. Med Decis Making. 1986;6:27–36.
Fine MJ, Aubile TE, Yealy DM, et al. A prediction rule to identify low-risk patients with community acquired pneumonia. N Engl J Med. 1997;336:243–50.
National Hospital Discharge Survey. Atlanta, Ga: Centers for Disease Control and Prevention. National Center for Health Statistics. 1998–1990.
Author information
Authors and Affiliations
Additional information
The conclusions and opinions expressed and methods used herein are those of the authors. They do not necessarily reflect the policy of the Health Care Financing Administration, which sponsored the work.
The analyses upon which this publication is based were performed under contract 500-96-P704, entitled, “Operation Utilization and Quality Control Peer Review Organization (PRO) for the State of Georgia,” sponsored by the Health Care Financing Administration, Department of Health and Human Services. The authors assume full responsibility for the accuracy and completeness of the ideas presented. This article is a direct result of the Health Care Quality Improvement Program initiated by the Health Care Financing Administration, which has encouraged identification of quality improvement projects derived from analysis of patterns of care, and therefore required no special funding on the part of this Contractor, Ideas and contributions to the authors concerning experience in engaging with issues presented are welcomed.
Rights and permissions
About this article
Cite this article
Flanders, W.D., Tucker, G., Krishnadasan, A. et al. Validation of the pneumonia severity index. J GEN INTERN MED 14, 333–340 (1999). https://doi.org/10.1046/j.1525-1497.1999.00351.x
Issue Date:
DOI: https://doi.org/10.1046/j.1525-1497.1999.00351.x