Study objective Sepsis quality improvement programmes typically focus on severe sepsis (ie, with acute organ failure). However, quality of ED care might be improved if these programmes included patients whose progression to severe sepsis could still be prevented (ie, infection without acute organ failure). We compared the impact on mortality of implementing a quality improvement programme among ED patients with a suspected infection with or without acute organ failure.
Methods This prospective observational study among ED patients hospitalised with suspected infection was conducted in two hospitals in the Netherlands. After stratification by sepsis category (with or without organ failure), in-hospital mortality was compared between a full compliance (all quality performance measures achieved) and an incomplete compliance group. Multivariable logistic regression analysis was used to quantify the impact of full compliance on in-hospital mortality, adjusting for disease severity, disposition and hospital.
Results There were 1732 ED patients and 130 deaths. Full compliance was independently associated with approximately two-thirds reduction in the odds of hospital mortality (adjusted OR of 0.30 (95% CI 0.19 to 0.47), which was similar in patients with and without organ failure. Among the 1379 patients with suspected infection without acute organ failure, there were 64 deaths, 15 (1.1%) in the full compliance group and 49 (3.6%) in the incomplete compliance group (mortality difference 2.5% (95% CI 1.6% to 3.3%)). Among 353 patients with organ failure, there were 66 deaths, 12 (3.4%) in the full compliance compared with 54 (15.3%) in the incomplete compliance group (mortality difference 11.9% (95% CI 8.5% to 15.3%)). Thus, there was a difference of 76 deaths between full and incomplete compliance groups, and 34 (45%) who benefited were those without acute organ failure.
Conclusions Sepsis quality improvement programmes should incorporate ED patients in earlier stages of sepsis given the potential to reduce in-hospital mortality among this population.
- infectious diseases
- quality improvement programmes
- quality performance measures
- emergency medical services
Statistics from Altmetric.com
- infectious diseases
- quality improvement programmes
- quality performance measures
- emergency medical services
What is already known on this subject?
Sepsis quality improvement programmes typically focus on severe sepsis. However, the impact of these programmes among ED patients with a suspected infection without acute organ failure is not yet known.
What this study adds?
Full compliance instead of incomplete compliance with sepsis quality performance measures was associated with a greater reduction in in-hospital mortality in ED patients with sepsis (without acute organ failure) as well as patients with severe sepsis/septic shock. Quality improvement programmes should incorporate recommendations for ED patients with earlier stages of sepsis given the improved survival that can be achieved.
Quality improvement programmes such as the ‘Surviving Sepsis Campaign’ primarily focus on the quality of care administered to intensive care unit (ICU) patients with severe sepsis and septic shock.1–3 Fulfilment of quality performance measures among these patients in the relatively late stages of sepsis is associated with lower mortality.1 However, no universal guidelines exist for the care of ED patients in earlier stages of sepsis (ie, suspected infection without acute-onset organ failure). Including patients without acute organ failure in quality improvement initiatives might help prevent their progression to severe sepsis, which occurs in approximately 22% of these patients.4 Because patients with earlier stages of sepsis are a larger ED population than those with later stages of sepsis, focusing on this group of patients in ED management may have a large impact on overall in-hospital mortality, as long as the complete chain of care is also addressed. Otherwise, a beneficial effect of one aspect of ED management, for example early administration of appropriate antibiotics, might be offset by failure to achieve another aspect, such as disposition to an appropriate level of care.5–7
In 2011, our hospitals introduced a quality improvement programme that specifically included ED patients in earlier stages of sepsis, in addition to those in later stages of sepsis. This protocol contained a standard screening procedure designed to optimise sepsis recognition and facilitate early ED resuscitation and disposition to an appropriate level of care.
The aim of this study was therefore to assess the impact on in-hospital mortality of instituting this quality improvement programme among ED patients with earlier stages of sepsis, as compared with those with later stages of sepsis. If fulfilment of these quality performance measures reduces mortality, then future guidelines should incorporate recommendations for ED patients with earlier stages of sepsis.
Study design and setting and patient population
This prospective observational study was conducted in two EDs in the Netherlands, each with approximately 30 000 patient visits per year: the Leiden University Medical Centre and Rijnstate Hospital. The study included data collected from 1 June 2011 to 1 June 2014 at Leiden University Medical Centre, and from 1 March 2012 to 1 April 2013 at Rijnstate Hospital. All ED patients >16 years old with a suspected infection and Manchester triage category yellow, orange or red (constituting those with urgent medical needs)8 who received intravenous antibiotics in the ED and were subsequently hospitalised were included in the study population. Patients were stratified as having early sepsis (sepsis without acute organ failure) or severe sepsis/septic shock (sepsis with acute organ failure), as defined previously.9 The study was approved by the medical ethics committee of the Leiden University Medical Centre.
Sepsis quality improvement programme
The sepsis quality improvement programme included a standard screening procedure aimed at facilitating sepsis recognition (including clinical and biochemical signs of acute organ failure), early ED resuscitation and disposition to an appropriate level of care (see online supplementary file 1). ED patients with suspected sepsis without acute organ failure were specifically included in the programme and screened for clinical signs of inflammation and organ failure. This screening procedure was developed by an expert group of ED physicians, intensivists, surgeons and infectious disease specialists from Leiden University Medical Centre and was based on the Surviving Sepsis Campaign sepsis programme of the National Patient Safety Agency (Safety Management System (VMS)), which had been strongly promoted at both hospitals by means of workshops, presentations and posters (for details see http://links.lww.com/CCM/A923). The identical quality improvement programme was voluntarily implemented in both hospitals at different time points. The programme was not funded by the hospitals or an external group as a specific initiative. Two emergency physicians (BDG, AA) informed all ED nurses, physicians and residents, and surgery, internal medicine and neurology staff about the programme, the data that would be collected as part of the programme, and the assessment of the programme through oral presentations, posters and flyers in the ED. The same two emergency physicians instructed new medical personnel who missed the official launch of the programme.
ED patient entry into the sepsis quality improvement programme began at ED triage or at any time during their ED evaluation. Patients with symptoms or signs of suspected infection (eg, fever, coughing or erythema) triggered the initiation of sepsis screening by the triage nurses or the nurses or physicians caring for the patients during their ED stay. The triage or treating nurse placed a patient identification sticker on the registration form of patients with a suspected infection and whose triage category was yellow, orange or red. Following this entry into this clinical care pathway, the ED nurses and doctors had to follow the protocol (see online supplementary file 1), and commensurate data collection assessing compliance with the pathway began.
Data collection and measurements
As described in detail previously,10 the ED visit data collected included patient demographic characteristics and comorbidities, time points for start of oxygen and fluid therapy and antibiotic administration, laboratory values, triage categories and vital signs, treatment administered (including antibiotics, intravenous fluids and oxygen), and disposition from the ED. In addition to these data, we reviewed charts for do not resuscitate (DNR) status as indicated by their previous medical files or decided during their ED stay or hospital admission, and determined the presence or absence of acute-onset organ failure as described by Dellinger et al. 9 We also calculated illness severity using the patient’s initial Predisposition, Infection, Response and Organ failure (PIRO) score. The PIRO score reflects disease severity and consists of 16 variables, including demographic data, type of infection, vital signs and biochemical signs of organ failure.11 12 The PIRO score as validated for use in the Netherlands ED setting was used because this measurement enables separation of the non-modifiable predisposition and infection aspects (the PI components) of disease severity from the potentially reversible response and acute organ failure aspects of disease severity (RO components). Missing values used to calculate the PIRO score were imputed as normal values, as was done previously in the Acute Physiology and Chronic Health Evaluation score calculations.13
Data were prospectively registered in the digital information system (Chipsoft, Amsterdam, the Netherlands) of each participating hospital. A medical student or registrar in emergency medicine subsequently transferred data from the electronic hospital information system to a web-based data collection file (Promise, Leiden, the Netherlands), which automatically calculated the illness severity scores. In addition, to limit typing errors, Promise warned if entered variables had extreme or illogical values (For detailed information about Promise see https://www.msbi.nl/promise/promise.aspx.). Each month BDG and AA checked whether the medical students or residents complied with the definitions (see online supplementary file 2 and reference 10 for details) of the following collected variables: appropriateness of antibiotics, accuracy of suspected source of infection, ICU consultation and unanticipated transfer from ward to ICU. Data were downloaded into SPSS V.20.0 and were analysed as described below.
Quality performance measure assessments
We assessed whether all or part of the following quality performance measures were achieved for each patient. Full compliance was considered to be present if all measures were achieved, and incomplete compliance if one or more, but not all, were achieved.
1. Time to antibiotics within 3 hours after ED registration, measured by subtraction of ED desk registration time from the time of antibiotic administration by the nurse.10 14 15
2. Appropriateness of the initial dose of antibiotics administered in the ED, assessed as is summarised in the online supplementary file 2.16 For culture-positive patients (blood or other cultures), initial antibiotics were considered to be appropriate if the cultured micro-organism could be a causative pathogen in relation to the clinical findings, and showed in vitro sensitivity to the initial dose of the antimicrobial agent. Staphylococcus epidermidis, S. hominis and other coagulase-negative staphylococci were considered to be contaminants (and therefore analysed as if culture-negative) except in cases with an endocarditis or infections caused by foreign bodies such as prostheses. For culture-negative patients, the initial antibiotics prescribed in the ED were considered to have been appropriate when patients recovered without any subsequent change to other antibiotics due to clinical deterioration. If patients had died during hospital admission, antibiotics were considered to have been effective if the initial antibiotics had been in accordance with the institutional protocol for antibiotic therapy in specific infections. An infectious disease specialist was consulted to help with interpretation of the microbiological data.
3. Whether blood cultures obtained before administration of antibiotics.8
4. Accuracy of the suspected source of infection in the ED was assessed by comparison with the suspected source of infection in the final hospital discharge letter, which was considered to be the ‘gold standard’ with regard to accuracy of diagnosis. If the initial suspected source of infection differed from the final hospital discharge letter, the initial ED working diagnosis was considered to be incorrect.
5. Lactate measured <6 hours after ED registration.8
6. Mean arterial pressure (MAP) >65 mm Hg <6 hours after ED registration.8
7. ICU consultation for severe sepsis or septic shock.
8. At least 1.5 L of intravenous fluids administered in the case of shock, in accordance with the surviving sepsis campaign (SSC) recommendations.8 Any amount of intravenous fluids was considered ‘sufficient’ as long as an ED patient did not have signs of shock, that is, lactate >4 mmoL/L and/or systolic blood pressure <90 mm Hg or a decrease of the systolic blood pressure of >40 mm Hg compared with the baseline blood pressure of the ED patient.
9. No unanticipated transfer from ward to ICU occurred. A patient was considered to have an unanticipated transfer from the ward to the ICU if he/she was first admitted to a normal ward and had a subsequent transfer to the ICU <48 hours after hospital admission because of sepsis progression.
Some of the quality measure assessments require additional explanation. Culture-negative patients were included because the ED physician cannot know at the time of patient presentation if cultures will become positive. This inclusion provides a more realistic evaluation of the appropriateness of antibiotics and in-hospital mortality. Although the appropriateness of antibiotics is only known in retrospect, the ED physician should be aware of strategies that can be used in the ED to increase the number of patients receiving appropriate antibiotics.17 Second, in the Netherlands, treatment requiring central venous and arterial catheters is usually performed in the ICU and not in the ED. Optimal haemodynamic resuscitation therefore requires ICU consultation. Early transport to an optimal level of care has been shown to improve outcome.18 Third, in previous studies, it has been shown that unanticipated transfers from a ward to the ICU have a negative impact on outcome.5 6 18 Although the result of the disposition decision is only known in retrospect, it is a reflection of the recognition of the severity of the illness and the response to ED treatment in the ED.
The primary outcome measured was in-hospital mortality.
Sample size estimation
We based the sample size needed for this investigation on expected difference in mortality among patients with sepsis without acute organ failure who received care that was fully compliant with the quality measures, as compared with those whose care was incompletely compliant. Mortality in the incomplete compliance group was expected to be similar to the 10% mortality observed in a study by Houck et al 19 of patients hospitalised with pneumonia. We estimated that to detect a mortality difference of 10%–5% with a power of 80% and α=0.05, we needed to include at least 474 patients. This suggested that we needed to include 3 years of data to achieve this study sample size.
Screening and enrolment results were summarised. Patient characteristics were described as follows: normally distributed data were presented as mean (SD) and skewed data as median (IQR). Categorical data were presented as percentage of total. Unpaired Student’s t-tests, Mann-Whitney U and Pearson χ2 tests (as appropriate) were used to compare patients by mortality status.
Main statistical analyses
In-hospital mortality first was compared according to compliance with sepsis quality performance measures (full vs incomplete compliance) using Pearson χ2 testing after stratification by sepsis category (sepsis with or without acute organ failure). We subsequently assessed the impact on in-hospital mortality of the achievement of all performance measures using multivariable binary logistic regression analysis, adjusting for the PIRO score (categorised as 0–8, 9–17 and >17), disposition location (to ward or ICU) and hospital (Leiden University Medical Centre or Rijnstate Hospital). Disposition was forced into the model because this variable partially reflects response to ED treatment (which reflects a different aspect of disease severity) and partially reflects the quality of management on the ward or ICU.10 12 We also tested whether the association between compliance with quality performance measures and in-hospital mortality differed between ED patients with sepsis without and with acute organ failure by the addition of an interaction term. Next, we repeated the analysis with the nine individual performance measures instead of whether implementation was complete or incomplete. Hosmer-Lemeshow’s goodness-of-fit testing was used to assess model fitness.
Variance inflation factors (VIFs) were used to investigate multicollinearity. If the VIF was below 3, multicollinearity was not considered to be problematic. ORs with 95% CIs were reported. For all analyses, an α of 0.05 was used to assess statistically significant differences.
We performed three sensitivity analyses to evaluate the consistency of our observed results by varying the model components. In the first sensitivity analysis, we investigated whether confounding by indication could explain the impact of full compliance on in-hospital mortality. Such confounding might be present if achieving the quality performance measures of MAP <65 mm Hg <6 hours, no unanticipated transfer and ICU consultation could have been a reflection of response to ED treatment (and thus illness severity) rather than a quality improvement intervention. This possibility was tested in the following manner. Although we corrected for ICU admission as a measure of response to ED treatment (as a measure of disease severity besides PIRO score), we excluded patients in whom the aforementioned three quality performance measures of MAP <65 mm Hg <6 hours, no unanticipated transfer and ICU consultation were not achieved. In a different analysis, all patients were included, but the definition of full compliance with quality performance measures was changed: the aforementioned three quality performance measures were not considered as a quality performance measure anymore. As a result, the impact of full compliance with six instead of nine performance measures on the adjusted ORs of the multivariable regression analysis could be evaluated.
In the second sensitivity analysis, the possibility of bias arises because the apparent difference in outcome between two groups may depend on characteristics that affect whether or not a group received a given treatment instead of the effect of the treatment per se. In a propensity score analysis, within the estimation of the treatment effect, the covariates that predict receiving the treatment are taken into account. We used logistic regression to calibrate a propensity model for the probability of attaining quality of care by regressing the observed quality of care status on triage status, admission to ICU or medium care unit (MCU), treating physician (medical, surgical, ED physician) and PIRO score. The effect of quality of care on mortality outcome was then assessed from a logistic regression model using the quality of care status as predictor after adjusting for the propensity to achieve quality of care.
In the third sensitivity analysis, we evaluated the impact of including DNR status in our models on the adjusted ORs of the main multivariable regression analysis, because DNR status may reflect illness severity, which is not specifically captured in the PIRO classification. In addition, DNR status may influence the decision to admit a patient to the ward or ICU.
Patient inclusion and characteristics
Figure 1 shows the stratification of study patients by sepsis category (sepsis with or without acute organ failure) and by compliance status with the quality performance measures (full or incomplete compliance). As shown, there were a total of 1732 patients in the study. Of these, 1379 ED patients had sepsis without acute organ failure, of whom 43% received fully compliant care. Of the 353 with severe sepsis/septic shock, 37% received fully compliant care. Table 1 stratifies patients by compliance. The characteristics of patients with full and incomplete compliance were similar, with the exception of the quality performance measures and in-hospital mortality.
Impact of achieving quality performance measures on in-hospital mortality
Among the entire study cohort of 1732 ED patients with sepsis (with or without acute organ failure), in-hospital mortality was 7.5%. Among the 719 patients in the full compliance with quality performance measure group, in-hospital mortality was 3.8%, which was significantly lower than the 10.2% in-hospital mortality among the 1013 patients in the incomplete compliance group (<0.001).
Of the 130 patients who died, 50.8% had severe sepsis or septic shock (with acute organ failure), and 49.2% of patients had sepsis (no acute organ failure) at ED presentation. Among all 1732 patients, all nine quality performance measures were fulfilled in 41.5% of them.
There were 1732 patients and 130 deaths. Full compliance was independently associated with approximately two-thirds reduction in the odds of hospital mortality (adjusted OR of 0.30 (95% CI 0.19 to 0.47), which was similar in patients with and without organ failure. Among the 1379 ED patients with suspected infection without acute organ failure, there were 64 deaths, 15 (1.1%) in the group of patients receiving fully compliant care and 49 (3.6%) in the incomplete compliance group (mortality difference 2.5% (95% CI 1.6 to 3.3). Among 353 patients with organ failure, there were 66 deaths, 12 (3.4%) in the full compliance compared with 54 (15.3%) in the incomplete compliance group (mortality difference 11.9% (95% CI 8.5 to 15.3)). Thus, there was a difference of 76 deaths between full and incomplete compliance group, and 34 (45%) who benefited were those without acute organ failure.
The association between full compliance with quality performance measures and in-hospital mortality was similar in ED patients with sepsis (without acute organ failure) and severe sepsis/septic shock (with acute organ failure), as shown in tables 2 and 3. Full compliance was independently associated with approximately two-thirds reduction in the odds of hospital mortality.
In the analysis aimed at identifying which quality performance measures were specifically associated with a reduction in in-hospital mortality, appropriateness of initial antibiotics, accuracy of ED diagnosis and no unanticipated transfer from ward to ICU had the largest impact (tables 4 and 5).
The sensitivity analyses assessing the potential confounding effect of the quality performance measures of MAP <65 mm Hg <6 hours, no unanticipated transfer and ICU consultation yielded results similar to the original regression analyses, suggesting that confounding by indication did not explain the lower mortality in the full compliance groups (see online supplementary file 3). Likewise, the sensitivity analysis assessing the impact of adding DNR status did not affect the adjusted ORs of PIRO categories and did not affect the adjusted OR of the effect of full compliance.
We observed two major findings from this investigation regarding the implementation of quality performance measures for ED patients with suspected sepsis with or without acute organ failure. First, we found that a quality improvement programme that specifically included ED patients with sepsis without acute organ failure yielded improved survival among this patient population.
Second, full compliance with all sepsis quality performance measures was associated with a lower mortality than was incomplete compliance, both for patients with and without acute organ failure.
Compared with the incomplete compliance group, unadjusted in-hospital mortality was lower in the sepsis categories with or without acute organ failure who received fully compliant sepsis care. After correction for confounding, the impact of full compliance on in-hospital mortality was still similar in sepsis with or without acute organ failure. The observed large impact of fully complying with all sepsis quality performance measures on in-hospital mortality warrants a greater emphasis on quality improvement programmes among ED patients who are in earlier stages of sepsis. The relatively low number of patients among whom all performance measures were achieved (41.5%) suggests that mortality could have been reduced even more if full compliance had been achieved for all patients.
Although we believe that all aspects of ED sepsis management are important, it seems likely that quality of care is improved cost-effectively20 if we focus on the quality performance measures with the largest impact, as Meyer et al 21 suggested. More specifically, our study suggests that improvement of the correct diagnosis of the suspected source of infection, administration of appropriate antibiotics and disposition from the ED to an appropriate level of care will have a larger effect on mortality than early administration of antibiotics. However, the impact of individual performance measures on mortality may differ across EDs and healthcare systems. For example, it was recently shown in the participating EDs of the present study that a further reduction of time to antibiotic infusion did not improve the survival of ED patients with sepsis,10 probably because antibiotics had already been administered very early. If we would like to improve the survival in these EDs, we should focus on the aforementioned other aspects of ED sepsis management. However, in EDs where time to antibiotic infusion is still relatively long, there may still be an association between time to antibiotics and outcome. In these EDs, it would still be worthwhile decreasing time to antibiotics. Monitoring the associations between individual quality performance measures and mortality would help to decide which performance measures have the highest priority for improvement at the local ED level.
Our observation that compliance with several individual performance measures was associated with reduced mortality is in line with the findings of the Surviving Sepsis Campaign in ICU patients with severe sepsis or septic shock. This suggests that the impact of full compliance can indeed be attributed to specific individual performance measures.1 2 5 6 14–19 In contrast, in a study by van Zanten et al,1 the individual treatment bundle elements were not significantly associated with mortality (except glucose control), suggesting that the bundle is more effective than the sum of its parts,22 or the Hawthorne effect plays a role.
Numerous studies have reported a reduction in mortality as a consequence of compliance with a quality improvement programme among patients with severe sepsis and septic shock,1–3 23 but none included patients with sepsis without acute organ failure, as we did in this study. Although there are several studies evaluating quality improvement initiatives in pneumonia,19 24 direct comparison of these studies with ours is complicated by the many differences in methodology and patient population. In addition, none of these studies considered accuracy of suspected source of infection and disposition to an appropriate level of care as quality performance measures, which appear to have a large impact on mortality as observed in our investigation.
Our study has several limitations. First, ethical concerns prevented the performance of a randomised controlled trial. Therefore, our question had to be addressed in a prospective observational study, in which causality cannot be proven and which is more prone to selection bias and unmeasured confounding. We believe that the most important confounders, including patient demographic characteristics, comorbidities, sources of infection, illness severity and aspects of ED management, were accounted for in the analyses, which should mitigate the impact of confounding on our study findings. In addition, the sensitivity analyses also suggested that confounding by indication did not explain the lower mortality in the full compliance groups.
Nevertheless, confounding by indication might have resulted in the large unadjusted mortality difference between the groups with and without full compliance, because the performance measures MAP >65 (used in the Surviving Sepsis Campaign) and no unanticipated transfer may partially reflect response to ED treatment (and thus illness severity) as well as compliance with quality performance measures. It is important to note, however, that this type of confounding would only be relevant for patients with severe sepsis/septic shock, because in the group with sepsis without acute organ failure these performance measures were all achieved. Therefore, the effect of full compliance with quality performance measures on mortality would only have been overestimated in the group with severe sepsis/septic shock, and not in the group with sepsis without acute organ failure. Accordingly, the possible overestimation would have been much larger in the group with severe sepsis/septic shock than in the sepsis group, and therefore the ~45% reduction in unadjusted mortality that can be attributed to full compliance in the patients with sepsis is an absolute minimum estimate. Finally, the external validity of our study is limited to EDs with comparable patient populations such as those included in our study.
In summary, full compliance with all elements of a sepsis quality improvement programme that also included ED patients with earlier stages of sepsis was associated with lower in-hospital mortality than was incomplete compliance. Optimisation of diagnostic accuracy, administration of appropriate antibiotics and disposition to an appropriate level of care appeared to have the largest impact on mortality reduction. Because these performance measures were achieved in less than half of the patients, future studies should focus on strategies to improve these aspects of ED sepsis management.
We are grateful to all the nurses, staff members, senior house officers and residents who were involved in patient inclusion. We are specifically grateful to Professor Dr Jolanda van der Velden and Dr Christian Heringhaus for their valuable comments on the manuscript. In addition, we are grateful to Dr Geert Groeneveld, an infectious disease specialist, for help with the interpretation of the microbiological data. We would also like to thank Lisa van de Lint for editing the text of the manuscript. Finally, we thank the doctors and nurses who were involved in the development of the sepsis protocol.
Contributor BDG devised and designed the study, collected data, contributed to the analyses and edited the manuscript. BS and RN collected data, did the analyses and wrote the manuscript. NH, AA, DR, LP and DV collected data. BM did the propensity score analysis. BDG, BS and RN contributed equally to the manuscript. BDG takes full responsibility for the study as a whole.
All authors read and approved the manuscript.
Competing interests None declared.
Ethics approval The study was approved by the medical ethics committee of the Leiden University Medical Centre who waived the need for individual consent because of the purely observational character of the study.
Provenance and peer review Not commissioned; externally peer reviewed.
If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.