Article Text
Abstract
Background The acute heart failure index (AHFI) is a previously derived prediction rule to identify patients presenting to emergency departments (ED) with decompensated heart failure (DHF) at low risk of early life-threatening events.
Study objectives To validate the AHFI prospectively.
Methods Using a prospective cohort study, adult patients presenting to an urban university hospital ED with DHF were included. Data on 21 variables were gathered to calculate the AHFI. Primary endpoints included inpatient death and non-fatal serious outcomes (myocardial infarction, ventricular fibrillation, cardiogenic shock, cardiac arrest, intubation, or cardiac reperfusion). Secondary endpoints included death from any cause or readmission for heart failure within 30 days. Primary and secondary endpoint rates were calculated with 95% CI for the low and higher-risk subgroups.
Results 259 patients were enrolled. 245/259 (95%) were admitted. 60/259 (23%) met low-risk criteria, of whom 1/60 (1.7%, CI 0.04 to 8.9) was discharged after sustaining pulseless electrical activity arrest. The comparable primary outcome rate in the derivation study was 1.4% (CI 1.1 to 1.7). 17/199 (8.5%, CI 5.1 to 13.3) higher-risk patients experienced an endpoint, compared with 13.3% (CI 12.9 to 13.7) in the derivation cohort. One low-risk patient (1.7%, CI 0.04 to 8.9) died within 30 days, and five (8.3%, CI 2.8 to 18.4) were readmitted. Corresponding rates in the derivation study were 2% and 5%, respectively.
Conclusion The results are consistent with those previously reported for the low-risk subgroup of the AHFI. Further research is needed to determine the impact, safety and full range of generalisability of the AHFI as an adjunct to decision making.
- Acute heart failure index
- clinical prediction rule
- heart failure
Statistics from Altmetric.com
Heart failure affects 5.7 million people in the USA, and 670 000 new cases are diagnosed each year.1 Heart failure leads to one million hospital admissions each year.2 Care of heart failure patients in the USA costs approximately US$33.2 billion a year, with hospitalisation accounting for US$17.8 billion of that expense.3 More than 75% of patients admitted to hospital with heart failure arrive through the emergency department (ED).4 There is currently wide variability in practice regarding hospital admission for patients with decompensated heart failure (DHF).5 ,6 Hospital admission rates vary widely across geographical regions, with evidence suggesting that emergency physicians overestimate the probability of short-term death or severe complications for patients with heart failure.7
Several clinical prediction rules have been developed to help assess disease severity in heart failure patients, and may ultimately help physicians make appropriate decisions regarding hospital admission.8–12 None of these rules have to date been prospectively validated. Prospective validation of a prediction rule in at least one small trial is considered to be a minimum criterion justifying clinical use.13 The acute heart failure index (AHFI) is a clinical prediction rule derived by Auble et al 8 at the University of Pittsburgh. The AHFI uses 21 prognostic factors to determine if patients presenting to the ED with DHF are at low risk of short-term fatal and inpatient non-fatal outcomes. We sought to validate the performance of AHFI prospectively in predicting a low incidence of short-term clinical outcomes in patients presenting with DHF to a high volume urban ED.
Methods
We used a prospective cohort design. Our study site was the ED of an urban university hospital, with an approximate annual adult ED volume of 73 000. The hospital serves a predominately Dominican, non-English-speaking community, with many of the patients being uninsured, and without regular medical care. Approximately 60% of patients are Hispanic, with the remaining portion comprising Caucasians, African-Americans, Asians and other nationalities. Survey data of our patient population has shown that approximately 1% of ED patients present with DHF.
Research assistants were available between the hours of 08:00 and 00:00 and enrolled study participants in cooperation with emergency medicine attending physicians. To be included in the study, patients were required to have a primary diagnosis of heart failure. Diagnosis of heart failure was made by the emergency medicine attending physician responsible for patient care and was based on several factors, including reported symptoms, clinical findings, chest x-ray findings, brain natriuretic peptide level, and data from previous admissions, including discharge summaries and echocardiogram results, which are available round the clock on all patients in this ED. A fixed set of criteria was not used in making the diagnosis of heart failure, nor were laboratory studies such as brain natriuretic peptide level used to rule in or rule out the possibility of heart failure; instead, attending diagnostic impressions were used to make the diagnosis. As a diagnosis of heart failure was required for enrolment, patients were enrolled after evaluation and diagnostic assessment by the attending physician.
Patients with a previous history of heart failure and those presenting with a first episode of heart failure were included. We included patients who were admitted to the hospital and those discharged from the ED. Decisions regarding hospital admission versus outpatient management were made by the emergency medicine physician attending, based on factors such as the severity of illness, likelihood of compliance and reliability of outpatient follow-up. Decisions to admit or discharge a patient were not influenced by the AHFI or by a patient's enrolment in the study. Patient management and site-of-care decisions were made at the ED attending and in-house physicians' discretion, without knowledge, use or application of the AHFI or any specifications stemming from the study protocol.
Enrolled patients were at least 18 years of age. There were no exclusions based on gender or ethnicity. Research assistants identified patients who were given a primary diagnosis of heart failure, and obtained written consent from patients for study participation. The study was approved by the Institutional Review Board of Columbia University Medical Center.
For enrolled patients, we collected data on 21 prognostic factors, which included: gender; medical history; vital signs; laboratory values; electrocardiographic findings; and radiographic findings8 (see table 1). Data on the 21 prognostic factors were prospectively collected by research assistants through chart review and patient interviews using a predesigned data sheet. Data on vital signs and laboratory values represented the first values available to physicians after patient presentation.
A software program is available that allows for calculation of the AHFI. The program allows users to input the 21 variables, and then determines if the patient is at low risk or higher risk of short-term life-threatening events. Figure 1 illustrates the branch points involved in calculation of the AHFI. All branches identify low-risk patients; all other patients are at higher risk. For each patient, we used the software program to determine if the patient fell into the low-risk or higher-risk subgroup. Although 21 variables are required for calculation of the AHFI, the software program allows for ease and expeditious use of the AHFI.
Inpatient medical charts were reviewed by study investigators to assess primary outcomes (inpatient death or serious medical complication during hospitalisation). Following discharge, study investigators made telephone interviews within 45 days of the ED visit to determine if secondary outcomes were met (death from any cause or hospital readmission for heart failure within 30 days). For patients who had died, family members available at the contact number confirmed the date of death. Investigators assessing outcomes were blinded to the AHFI.
We defined our primary outcomes as inpatient death from any cause or serious medical complications. A patient was considered to have a serious medical complication if he or she experienced a life-threatening condition attributable to their underlying cardiac disease or received a life-saving inpatient treatment, as listed in table 2. We defined secondary outcomes as death from any cause within 30 days of admission and hospital readmission within 30 days with a primary diagnosis of heart failure (table 2).
We used the AHFI to determine the percentage of patients who were at low risk of serious outcomes. We calculated the rates of primary and secondary outcomes and their 95% CI for patients in the low-risk and higher-risk subgroups. Investigators determining event rates were blinded to the AHFI.
Results
We enrolled 259 patients from January 2007 to July 2009. All enrolled patients had a primary diagnosis of heart failure, as made by the emergency medicine attending physician. A cross-sectional survey performed in our ED estimated that approximately 1% of ED patients present with DHF (730 per year). Patients were recruited into the study over a 2.5 year enrolment period. We discontinued enrolment when, due to logistical failures at a proposed second site, it became clear that our original target was not attainable. Over the 2.5-year enrolment period, we enrolled approximately 14% of the estimated volume of DHF patients at our primary study site.
Of the 259 enrolled patients, 245 (95%) were admitted. The mean age was 68 years; 131 (51%) were men and 128 (49%) were women; 144 (56%) had coronary artery disease; 60/259 (23%) were classified as low risk. Table 3 compares characteristics between low-risk and higher-risk risk subgroups.
No patients in the low-risk subgroup died during hospitalisation. One of 60 patients (1.7%, CI 0.04 to 8.9) sustained pulseless electrical activity arrest requiring intubation and cardiopulmonary resuscitation. The combined primary outcome rate in the low-risk subgroup was 1.7% (CI 0.04 to 8.9).
One of 60 (1.7%, CI 0.04 to 8.9) patients in the low-risk subgroup died within 30 days of hospital admission, and five of 60 (8.3%, CI 2.8 to 18.4) were readmitted to the hospital within 30 days. The combined secondary outcome rate was 10.0% (CI 3.8 to 20.5).
Of the higher-risk patients, six of 199 (3.0%, CI 1.1 to 6.5) died during their hospitalisation, and 11/199 (5.5%, CI 2.8 to 9.7) sustained a serious medical complication. The combined primary outcome rate was 8.5% (CI 5.1 to 13.3). Five of 199 (2.5%, CI 0.8 to 5.8) higher-risk patients died within 30 days of hospitalisation, and 55/199 (27.6%, CI 21.6 to 34.4) were readmitted within 30 days of hospitalisation. The combined secondary outcome rate was 30.2% (CI 23.9 to 37.0). Table 4 summarises primary and secondary outcome rates in the low-risk and higher-risk subgroups.
Discussion
A retrospective cohort study by Auble et al 8 at the University of Pittsburgh was used to derive a clinical prediction rule to identify ED patients presenting with DHF who are at low risk of inpatient death or serious medical complications.
To derive the rule, the authors analysed data for 33 533 patients with a primary hospital discharge diagnosis of heart failure in 1999 who were admitted from ED in Pennsylvania. The authors selected candidate predictors, including demographic and medical history variables and diagnostic test values measured in the ED. The authors constructed classification trees to identify a subgroup of patients with an observed rate of death or serious medical complications before discharge of less than 2%. The tree that identified the subgroup with the lowest rate of this outcome and an inpatient mortality rate of less than 1% was chosen. Patients in this subgroup were considered low risk; all others were considered higher risk. Of the 33 533 patients were enrolled, 5758 (17.2%) were classified as low risk.8
A retrospective cohort study was performed by Hsieh et al 12 at the University of Pittsburgh to validate the AHFI. Of the 8384 adult patients enrolled, 1609 (19.2%) were classified as low risk.12
Table 5 compares primary and secondary outcome rates and 95% CI for low-risk and higher-risk subgroups among the derivation and validation studies. Event rates in the low-risk subgroup were found to be comparable among derivation and validation studies.
The purpose of the derivation study was to identify a subgroup of DHF patients (based on candidate predictor values) with a low rate of short-term life-threatening events. Our prospective study sought to validate the AFHI13 clinically by applying it to DHF patients in our clinical setting to determine whether it could identify a subgroup of patients with a similarly low rate of short-term life-threatening events.
Given the small numbers of patients in our validation study, we avoided any attempt to simplify or otherwise modify the rule, which generates a dichotomous rather than a numerical risk projection. Rather, our objective was to contribute to validation of the rule as originally derived through a prospective application to a different population. The complexity of the AHFI potentially mandates an electronic adjunct, such as a calculator built into an electronic medical record, for the instrument to be used routinely in practice. We do not see this as ultimately limiting its adoption. Simplicity of prediction rules has not been shown to guarantee translation into practice, as experience with the straightforward Ottawa ankle rules has demonstrated.14 Rather, routine adoption and the use of almost all prediction rules is likely to depend upon the use of information technology, and particularly incorporation into the workflow via the electronic health record.
Ours is the first prospective validation study on the previously derived AHFI. According to a published hierarchy of evidence for clinical decision rules, in most instances a prospective validation should be regarded as the minimum requirement for consideration for clinical use.13 Previous studies of the AHFI have all been conducted retrospectively. Our small study was performed on a limited population in a single centre. Our prospective design was calculated to control for uncertainty in the primary diagnosis of DHF among the emergency patients enrolled. Previous studies have relied on retrospective diagnoses based on insurance coding. Our design also served to maximise the accuracy of data collection relative to exclusive reliance on retrospectively collected data from insurance and medical records. On the other hand, the AHFI was not actually calculated or made available to the treating physicians during the care of the patients. Our study therefore, at best, moves the AHFI as a level III rule using a published hierarchy, rendering it potentially suitable for limited use in that or in very similar populations.13 Several steps are involved in the development and testing of a clinical prediction rule. Before a rule can be fully recommended for implementation in clinical practice, it must be validated, and its impact on clinical behaviour assessed through prospective use by clinicians. In the case of the AHFI, wider prospective validation studies are needed, followed by impact studies demonstrating clinical effectiveness.
The AHFI, no matter how well validated, does not constitute a direct decision rule in the same way as do guidelines such as the Ottawa ankle rule. Rather it is akin to the well-known and highly validated PORT score for pneumonia (pneumonia severity index), which was derived using very similar methods to those later used to derive the AHFI. The PORT score does not directly determine whether a patient should be admitted.15 Rather, a severity score such as PORT or the AHFI contributes objective information to a practitioner's clinical assessment and evaluation of factors such as the reliability of follow-up in making a recommendation regarding hospital admission. Following prospective validation of the performance of a severity score in predicting clinical outcomes across different patient populations and settings, cluster randomised trials or other comparative studies may be performed to determine whether systematic availability of severity scores modifies practitioner behaviour in such a fashion as to increase the efficient use of healthcare resources with no adverse effect on patient outcomes.16
The calculation of the AHFI in our study, although based on prospective data collection, was not based on assessments made exclusively by the practitioners. However, the prospective design increased the directness of both patient identification and of data collection relative to previous studies, which relied on the International Classification of Disease, version 9 coding to classify and enrol patients.12
Limitations
Our own study, like the previous derivation and validation studies of the AHFI, was largely limited to admitted patients. Had all low-risk patients been discharged from the ED, event rates might have been higher. The hospital admission rate of patients with DHF in our centre may be explained partly by a variety of factors including the high prevalence of patients lacking adequate primary care sources and support systems. This underscores the need to regard information from a severity score as a contributing factor, but not a sole determinant of site-of-care decision making.
We did not attempt to confirm the diagnosis of CHF beyond the criteria used by the clinicians who treated them. However, we did not become aware of diagnostic misclassifications in the course of following up any of the included patients. We furthermore believe that the interests of clinical validation of the AHFI are best served if patients in a validation study are selected using the same criteria as would be used by practitioners in the course of using the instrument for decision making.
Our study enrolled only 14% of our projected eligible population over the study period. Although research assistants were available most hours of the day, the limited enrolment rate may be explained by factors such as lack of coordination or communication between attending physicians and research assistants. Future studies can employ strategies to maximise patient enrolment, such as by having research assistants review the admitting/discharge diagnosis of all ED patients to identify potential study subjects. If enrolled patients were on average lower risk than those in our target population, the event rate in the low-risk group may have been underestimated. Furthermore, our study was small and cannot in itself serve as a definitive prospective performance validation of the AHRI.
The availability of research assistants, who were responsible for patient recruitment, largely determined the composition of our convenience sample. The research assistants were available from 08:00 to 00:00 hours during their on-call months, which included both summer and winter time periods. Therefore, the limitations in capture largely expose the study to seasonal rather than diurnal variation. We consider a systematic error in sampling to be unlikely under the circumstances.
Our population is largely made up of Hispanic patients of Dominican origin, and may not be representative of other heart failure populations. Future studies should incorporate larger study populations and multiple sites for the AHFI to attain a high level of external validity.
While DHF patients discharged home after ED stabilisation were eligible for our study, we were only able to enrol 14 such patients (14/259, 5%). Socioeconomic and access to care issues may be responsible for the high admission rate at our study site. Practice patterns may also be responsible, and can be positively impacted by the use of a rule such as the AHFI. Future validation studies should address outcomes of DHF patients discharged from the ED.
Ninety-two per cent (55/60) of low-risk patients in our study were admitted. The effect of the hospital admissions themselves on potential target clinical outcomes in these patients cannot be determined. Therefore, our results do not directly predict the safety of an outpatient management alternative for these patients. Future validation studies need to explore the impact of the clinical use of the AHFI to guide site-of-care decisions.
In our study, the AHFI was not used to influence or guide clinical decision making. The results of application of the AHFI were tabulated only after patients left the ED. Therefore, decisions on whether to admit a DHF patient were made at the physician's discretion, without the use or application of the AHFI. This study does not address the direct potential impact of using the rule on site-of-care decisions.
Conclusion
The results from our prospective cohort study suggest that the clinical prediction rule derived by Auble et al 8 may be able to identify patients presenting with heart failure to the ED who are at low risk of serious outcomes and who might be considered for outpatient management. Event rates in the low-risk subgroup were found to be comparable within our prospective study and the retrospective validation study by Hsieh et al 12 and the derivation study by Auble et al.8 Our study utilised prospective data collection, which has certain advantages given the limitations in accuracy and reliability that retrospective studies may be subjected to. Despite the small sample size and limited population of our study, the strong agreement of our results with those of the retrospective studies contributes strength to the validity of the rule and suggests that it merits additional, larger prospective validation studies. Further prospective validation studies utilising larger and more diverse populations may demonstrate the utility of the rule in guiding hospital admission decision making and its potential of reducing the number of hospitalisations for heart failure.
References
Footnotes
-
Funding This study was made possible by grant number UL1 RR024156 from the National Center for Research Resources (NCRR), a component of the National Institutes of Health (NIH) and NIH Roadmap for Medical Research. Its contents are solely the responsibility of the authors and do not necessarily represent the official view of NCRR or NIH.
-
Competing interests None.
-
Patient consent Obtained.
-
Ethics approval The study was approved by the Institutional Review Board of Columbia University Medical Center.
-
Provenance and peer review Not commissioned; externally peer reviewed.