Article Text
PDF
Abstract
Introduction Patients admitted to hospital in an emergency at weekends have been found to experience higher mortality rates than those admitted during the week. The National Health Service (NHS) in England has introduced four priority clinical standards for emergency hospital care with the objective of reducing deaths associated with this ‘weekend effect’. This study aimed to determine whether adoption of these clinical standards is associated with the extent to which weekend mortality is elevated.
Methods We used publicly available data on performance against the four priority clinical standards in 2015 and estimates of Trusts’ weekend effects between 2013/2014 and 2015/2016 for 123 NHS Trusts in England. We examined whether adoption of the priority clinical standards was associated with the extent to which weekend mortality was elevated, and changes over a 3 year period in the extent to which mortality was elevated.
Results Levels of achievement of two of the four clinical standards (ongoing review and access to diagnostic services) had small positive associations with the magnitude of the weekend effect in 2015/2016. Levels of achievement of the remaining two standards (time to first consultant review and access to consultant directed interventions) had small negative associations with the magnitude of the weekend effect in 2015/2016. No association was statistically significant. The same pattern was observed in the associations between achievement of the standards and changes in the magnitudes of the weekend effect between 2013/2014 and 2015/2016.
Discussion We found no association between Trusts’ performance against any of the four standards and the current magnitude of their weekend effects, or the change in their weekend effects over the past 3 years. These findings cast doubt on whether adoption of seven day clinical standards in the delivery of emergency hospital services will be successful in reducing the weekend effect.
- death/mortality
- quality
This is an Open Access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/
Statistics from Altmetric.com
Key messages
What is already known on this subject
Patients admitted to hospital in an emergency at weekends have been found to experience higher mortality rates than those admitted during the week.
The NHS in England has introduced four priority clinical standards for emergency hospital care to reduce this weekend effect.
There is no evidence that adoption of these standards will reduce mortality.
What this study adds
In this retrospective study of 123 Trusts in England, Trusts' performances on the clinical standards was not associated with the current magnitude of, or changes in, their weekend effects over the past 3 years.
Achievement of seven day clinical standards for emergency hospital care may not reduce weekend mortality.
Introduction
Patients admitted to hospital in an emergency at weekends have been found to experience higher mortality rates than those admitted during the week.1–4 This finding has been noted in many high income countries, including the UK, USA and Canada.5 6 However, the reasons for this ‘weekend effect’ are contested.7–11 Numerous potential explanations have been put forward, including case mix and severity differences,12 13 artefactual differences in coding,14 and differing levels of staffing and service provision.3 Focusing on the last of these potential explanations, the National Health Service (NHS) in England has introduced four priority clinical standards for emergency hospital care with the stated aim being "to reduce the deaths associated with the ‘weekend effect’ ".15 Compliance with these standards is mandatory for all NHS hospitals by 2020 as part of the seven day services policy.16 Commissioners have been encouraged to use the threat of financial sanctions to ensure that progress is made against the clinical standards under the Commissioning for Quality and Innovation framework.3
The clinical standards were developed by expert consensus based on existing recommendations of best practice from national and professional bodies.3 However, the accompanying review presented no evidence that the adoption of these standards would reduce mortality.17 18 The four standards require consultant presence, yet a recent point prevalence survey of consultant presence across 127 NHS hospital Trusts in England found no association between consultant presence and ratios of weekend to weekday mortality.19
Focus on achievement of these four priority standards may divert effort away from other quality improvement initiatives, or mean that limited resources become diverted to low risk patients who will experience little or no benefit. It is therefore important to establish whether adoption of the clinical standards is associated with an improvement in mortality.
We examined the association between adoption of the clinical standards and the extent to which mortality was elevated among weekend admissions across all hospital Trusts in England. We used the most recent data available. We considered whether achievement of the priority clinical standards was associated with the extent to which weekend mortality was elevated, and changes over a 3 year period in the extent to which mortality was elevated.
Methods
Data
We obtained publicly available data on Trusts’ performances against the four priority clinical standards from the My NHS website.20 These four standards are: time to first consultant review; access to diagnostics; access to consultant directed interventions; and ongoing consultant review (table 1). The data were downloaded on 7 February 2017 and are described as relating to the summer of 2015.
Priority clinical standards for seven day services
The data record, for each Trust, how many specialties achieved each standard out of a total number of eligible specialties. In all Trusts, there are 14 specialties eligible for the access to diagnostics standard and nine interventions eligible for the access to consultant directed interventions standard. The number of eligible specialties for the two remaining standards varied across Trusts, with maximums of 10 specialties for the time to consultant review standard and 13 specialties for the ongoing consultant review standard. For the Trusts that had less than the maximum number of eligible specialties for these two standards, we re-scaled the measure of achievement to be comparable across Trusts (see table 2 note). For the 12% of Trusts that reported achievement at multiple sites, we selected the records with the maximum number of eligible specialties.
Distribution of rusts’ achievements of clinical standards, and levels and changes in weekend effect
We obtained figures on mortality within 30 days of admission from the NHS Digital website, which have been published to aid research into 7 day NHS provision.21 We focussed on the indicator comparing the odds of mortality for patients admitted at the weekend (Saturday and Sunday) to the odds of mortality for patients admitted midweek (Tuesday, Wednesday and Thursday). We present results for the indicator relating to emergency admissions only, and for the indicator relating to all admissions. However, we believe the indicator relating only to emergency admissions to be the most appropriate indicator in relation to the emergency care clinical standards, and so present these as our primary results.
These mortality figures are publicly reported annually at the Trust level, and are available for three financial years 2013/2014–2015/2016. They are presented as ORs, and are risk adjusted for primary and secondary diagnoses, age, gender, deprivation, ethnicity, seasonality, admission source (usual place of residence vs other), admission method (emergency vs elective) and prior admission history over the past year,22 using a methodology similar to that employed by Freemantle et al.1 Deprivation is measured as relative deprivation in 32 844 small areas in England called lower layer super output areas, with a mean population of 1500.23 The lower layer super output area of residence is originally derived from an individual’s postcode. The data used to produce the mortality indicator were originally sourced from Hospital Episode Statistics data linked to the Office for National Statistics death registrations data.
Analysis
We related Trusts’ weekend effect ORs to their achievement of each priority clinical standard using multivariable linear regression models.
We began by relating the weekend effect in the latest financial year (2015/2016) to achievement of the standards in the summer of that financial year. However, it is possible that this correlation could be confounded if the Trusts who made the most progress towards achieving the standards are those who previously had larger weekend effects prior to the measurement of the standards. To investigate this possibility, we also examine the association between the change in the weekend effect between 2013/2014 and 2015/2016 and levels of achievement of the standards in summer 2015.
As the published weekend effect ORs are non-linear transformations of the underlying mortality rates, we also repeated the analysis using logged values of the OR values. We related the logged weekend effect OR in the latest financial year (2015/2016) to achievement of the standards in the summer of that financial year. Finally, we examined the association between the change in the logged weekend effect OR between 2013/2014 and 2015/2016 and the levels of achievement of the standards in summer 2015.
This study utilised publicly available data and did not require ethics approval.
Results
The weekend effect statistics are produced for 136 Trusts. Information on achievement of the clinical standards is not available for 13 of these Trustsi. Our analysis therefore contains data on 123 Trusts. The weekend effect for emergency admissions in 2015/2016 was not significantly different among the 13 Trusts that did not report their achievement of the standards compared with the 123 Trusts that did (1.139 vs 1.119, difference=0.019, 95% CI −0.025 to 0.064). The change in the weekend effect between 2013/2014 and 2015/2016 was not significantly different between the 13 Trusts that did not report their achievement of the standards compared with the 123 Trusts that did (0.029 vs 0.004, difference=0.025, 95% CI −0.037 to 0.088).
Descriptive statistics summarising Trusts’ performances against the four priority clinical standards in the summer of 2015 are shown in table 2. There was large variation across Trusts for each standard and in average (mean) national performance across the four standards. Trust performance was highest against the access to consultant directed interventions standard, with an average of 8.1 of 9 (90%) eligible specialties meeting this target within each Trust. Trust performance was lowest against the time to first consultant review standard, with on average 4.9 of 10 (49%) specialties meeting this target. An average of 7.0 of 13 (54%) specialties within each Trust met the ongoing review standard, with 10.7 of 14 (76%) achieving the target for access to diagnostic services. Trust performance ranged from 0% to 100% of specialties meeting the standards for ongoing review, access to consultant directed interventions and time to first consultant review. However, the lowest achievement observed against the access to diagnostic services standard was 3 of 14 areas, with the maximum again 100%.
The weekend effect was expressed as an OR, meaning that a value of 1.000 represents no risk adjusted difference in mortality rates between weekday and weekend admissions. The average Trust weekend effect in 2015/2016 for emergency admissions was 1.119. The magnitudes of Trusts’ weekend effects varied in the range 0.920 to 1.360 (SD=0.081). The average change over time between 2015/2016 and 2013/2014 in Trusts’ weekend effects was 0.004. There was substantial variation in these changes (SD=0.118), with the largest decrease for any Trust equal to −0.340 and the largest increase equal to 0.380. The average change was not significantly different from 0 (mean change 0.004, 95% CI −0.017 to 0.025).
The correlations between the levels of achievement of each of the four standards are shown in table 3. These correlations were all positive but the vast majority were weak. There was a moderate (r=0.348, 95% CI 0.182 to 0.494) correlation between achievement of the time to first consultant review standard and the ongoing review standard.
Correlations between numbers of specialties achieving clinical standards across Trusts, summer 2015
The levels of achievement of two of the four clinical standards (ongoing review and access to diagnostic services) had small positive associations with the level of the weekend effect for emergency admissions in 2015/2016 (table 4). Levels of achievement of the remaining two standards (time to first consultant review and access to consultant directed interventions) had small negative associations with the level of the weekend effect for emergency admissions in 2015/2016. No association was statistically significant. The results using the logged values of the weekend effect ORs for emergency admissions in 2015/2016 were very similar, and again no association was statistically significant.
Estimated regression associations between weekend mortality effects for emergency admissions and numbers of specialties achieving clinical standards
The relationships between levels of achievement of the standards and the magnitudes of the weekend effect ORs for emergency admissions in 2015/2016 are also presented graphically in figure 1. These graphs illustrate that performance against the standards for time to first consultant review and ongoing review were fairly evenly spread across the full range of performance. Conversely, performance on the standards for access to diagnostics and consultant directed interventions were clustered to the right hand side, with the majority of Trusts achieving compliance in at least 50% of eligible specialties. Sixty-seven (54%) of the 123 Trusts reported providing access to consultant directed interventions in all nine eligible specialties. These graphs confirm the lack of consistent relationship between achievement of the standards and the magnitudes of Trusts’ weekend effects.
Scatterplots and lines of best fit between magnitudes of weekend effects for emergency admissions in 2015/2016 and levels of achievement of each of the four clinical priority standards.
The same pattern was observed in the associations between achievement of the standards and the changes in the weekend effect for emergency admissions within Trusts between 2013/2014 and 2015/2016 (table 4). The levels of achievement of two of the four clinical standards (ongoing review and access to diagnostic services) had positive associations with the changes in the weekend effects. Levels of achievement of the remaining two standards (time to first consultant review and access to consultant directed interventions) had negative associations with the changes in the weekend effects. No association was statistically significant. The results using the changes in the logged weekend effect ORs for emergency admissions between 2013/2014 and 2015/2016 were very similar, and again no association was statistically significant.
The results using the weekend effects published for all admissions were very similar, and again no association was statistically significant (table 5).
Estimated regression associations between weekend mortality effects for all admissions and numbers of specialties achieving clinical standards
Discussion
Four priority clinical standards for emergency care have been introduced in England with the aim of reducing the number of deaths associated with the weekend effect. However, we found that adoption of these clinical standards for seven day services was not associated with the extent to which mortality was elevated for patients admitted at the weekend. We found no association between Trusts’ performances against any of the four standards and the current magnitude of their weekend effects, or the change in their weekend effects over the past 3 years.
The High intensity Specialist Led Acute Care (HiSLAC) study has previously shown that elevated weekend mortality in England was not associated with variations across hospitals in the extent to which senior doctors were present on Sundays compared with Wednesdays.19 While this was the first study to report differences between weekday and weekend medical staffing levels, it focused on whether senior doctors were present rather than the activities they undertook or the processes of care provided. We used information reported by organisations on their compliance with the priority clinical standards, which encompass the speed and frequency at which patients are seen by consultants, and the availability of wider diagnostic and interventional services. Nonetheless, our results are complementary to the HiSLAC findings, that the extent to which weekend mortality was elevated was not associated with available measures of care provision. In addition, we also showed that levels of achievement were not associated with changes in the weekend effect over time.
There are at least three possible explanations for the lack of relationship between achievement of the standards and elevated weekend mortality. First, this could be a result of poor quality data on Trusts’ compliance. We are unable to check the veracity of these data. Nonetheless, these are the figures on which Trusts are assessed and on which NHS England will track progress towards meeting the commitment to seven day services, and so finding that the reported levels of achievement are not associated with mortality is important.
Second, it is possible that the data on compliance against the standards are accurate, but the standards themselves are not effective in reducing mortality. Given the absence of evidence supporting a causal link between the standards and mortality, this would not be unexpected. While it is unlikely that offering this standard of care to all patients would result in any direct harm, doing so diverts resources and attention away from other potential care provision and therefore has an opportunity cost.
Finally, the weekend effect in mortality may be the wrong metric by which to judge the benefits of the seven day services policy. We have previously demonstrated that the weekend effect is driven by a reduction in admission volumes at the weekend rather than an increase in the number of deaths.7 If the weekend effect is indeed a statistical artefact, then it will not be reduced by changing care provision.
There is a need for the publication of statistics tracking the progress of the seven day services policy and the impact this has on patient outcomes for those admitted both at the weekend and during the week. These should link outcomes to care provision, and track the volumes and composition of admissions throughout the week in addition to mortality rates. Implementing seven day services has been estimated to cost £1.07–1.43bn,18 most of which relates to recruitment of additional medical staff,24 yet there is currently no evidence that this initiative has resulted in any patient benefit.
Limitations
Information on achievement of the clinical standards was not available for 13 of the Trusts for which NHS Digital produces weekend effect figures. However, we found no significant difference between the magnitudes of the weekend effects reported for these 13 Trusts and the 123 Trusts for which performance data were available. We examined the number of specialties that achieved the standards, not which specialties these were. The achievement of the standards may be more important in some specialties than others, but our analytical approach is consistent with the measurement and reporting of the standards which accords each specialty equal importance. Our ability to detect statistically significant results is unavoidably restricted by the sample size of 123 Trusts. Nonetheless, the pattern of two positive and two negative associations between achievement of the four standards and mortality is further evidence of no systematic relationship.
Trusts’ performances against the clinical standards is currently only available at one point in time. The absence of information on historical figures means that we are unable to examine whether Trusts have responded to the introduction of the priority standards, and cannot investigate whether increasing achievement reduces mortality. Nonetheless, we examined whether current levels of achievement were associated with levels of, and reductions in, mortality and found no statistically significant associations.
Conclusions
This is the first study to examine the association between Trusts’ performances against the four priority clinical standards and the mortality figures produced by the national information body in order to monitor the progress of the seven day services policy. The results presented add to the increasing body of evidence questioning the link between levels of service provision and weekend death rates.7 19 While implementation of these standards is unlikely to result in any direct harm to patients, the requirements may divert care away from the most high risk patients, limiting clinicians’ abilities to prioritise patients based on their professional judgement. Compulsory compliance across all patients and specialties may therefore not be the best way to allocate scarce NHS resources or increase the overall quality of care provided. The lack of association between the stated aims of the seven day services policy and the clinical standards being introduced to meet these aims suggests that the four priority clinical standards should be reviewed before compliance for all NHS hospitals is mandated in 2020.
References
Footnotes
↵ i Information on achievement of the clinical standards is not available for the following 13 NHS foundation Trusts: University Hospitals Bristol; Luton and Dunstable University Hospital; York Teaching Hospital; South Tyneside; The Rotherham; Cambridge University Hospitals; South Warwickshire; Norfolk And Norwich University Hospitals; Doncaster and Bassetlaw Hospitals; University Hospitals Birmingham; Gloucestershire Hospitals; Blackpool Teaching Hospitals; and Western Sussex Hospitals.
Contributors RM and MS designed the paper. MS extracted the data, designed and performed the analysis, and edited the manuscript. RM wrote the manuscript and is the guarantor of this article.
Competing interests None declared.
Provenance and peer review Not commissioned; externally peer reviewed.