We used administrative (Part A Medicare) data to identify a representative sample of 1126 patients with congestive heart failure and 1150 with acute myocardial infarction in hospitals with significant unexpectedly high inpatient, age-sex-race-disease-specific death rates ("targeted") vs all other ("untargeted") hospitals in four states. Although death rates in targeted hospitals were 5.0 to 10.9 higher per 100 admissions than in untargeted hospitals, 56% to 82% of the excess could result from purely random variation. Differences in the quality of the process of care (based on a medical record review) could not explain the remaining statistically significant differences in mortality. Comparing targeted hospitals with subsets of untargeted ones, eg, those with lower than expected death rates, did not affect this conclusion. Severity of illness explained up to 2.8 excess deaths per 100 admissions for patients with myocardial infarction. Identifying hospitals that provide poor-quality care based on administrative data and single-year death rates is unlikely; targeting based on time periods greater than 1 year may be better.