Article Text
PDF
Abstract
Background The National Early Warning Scores (NEWS) is used in various healthcare settings to augment clinical decision making, and there is growing interest in its application in primary care. This research aimed to determine the distribution of NEWS among patients in UK out-of-hours (OOH) general practice and explore the relationship between NEWS and referral of patients to hospital.
Methods A historical cohort study using routinely collected data from the Birmingham Out-of-hours general practice Research Database. This includes patients who attended a large out-of-hours general practice provider in the West Midlands, UK, between July 2013 and July 2018. All adults who were seen face to face who had a full set of physiological observations recorded were included. NEWS was calculated post hoc, and subsequent hospital referral was the outcome of interest.
Results A NEWS was calculated for 74 914 consultations. 46.9% of patients had a NEWS of 0, while 30.6% had a NEWS of 1. Patients were referred to hospital in 8.5% of all encounters. Only 6.9% (95% CI 6.3% to 7.5%) of the 6878 patients referred to hospital had a NEWS of ≥5. Of the 1509 patients with a NEWS ≥5, 68.6% (95%CI 66.2% to 70.9%) were not referred to hospital. When considering how NEWS was related to hospital referral, the area under the receiver operating characteristic (AUROC) for patients seen in their own home was 0.731 (95%CI 0.681 to 0.787). For patient seen in treatment centres, the AUROC was 0.589 (95% CI 0.582 to 0.596).
Conclusions Patients seen in out-of-hours general practice have low physiological acuity. Those referred to hospital have a slightly higher NEWS overall. NEWS is poorly associated with hospital referral, although the association is stronger for patients seen in at home compared with patients seen in treatment centres. Implementing NEWS-based referral from OOH general practice is likely to increase hospital admissions.
- clinical assessment
- emergency care systems, primary care
- primary care
Statistics from Altmetric.com
Key messages
What is already known on this subject
National Early Warning Scores has been validated as a way of identifying hospital in-patients at risk of Intensive Care Unit (ICU) admission and death and has been used in this way for a number of years. More recently, NEWS has been shown to be associated with adverse outcomes for patients arriving at EDs and in prehospital settings. It has been recommended that NEWS scores be assigned to patients seen in the community. However, the NEWS of patients in the community who are not referred to hospital is not known, and it is unclear how the adoption of NEWS may impact on referral decisions from this setting.
What this study adds
In this retrospective cohort study using routinely collected data, patients seen in out-of-hours general practice generally have low NEWS.This is true of patients who are managed in the community as well as those who are referred to hospital. There is an inconsistent relationship between NEWS and referral decisions, although the two are more closely aligned for patients seen in their own home. If all patients with a NEWS ≥5 were referred to hospital, the referral rate would increase by 16.4%. However, introducing a referral cut-off may decrease the referral of low acuity patients.
Background
The National Early Warning Score (NEWS) was developed in 2012 to standardise the assessment of acute illness severity in the NHS.1 NEWS uses standard physiological parameters to give an overall assessment of the acutely unwell patient in order to aid communication and set a threshold for actions (such as escalating care). In general emergency medical patients, NEWS identifies individuals at risk of the combined outcome of cardiac arrest, unanticipated Intensive Care Unit (ICU) admission or death better than comparative scoring systems, using a threshold score of 5.2 In 2015, a refinement (NEWS2) was published which, among other things, took greater consideration of patients on long-term oxygen therapy.3
Although there has been widespread adoption of NEWS in secondary care in the UK, its use outside secondary care is not established. There have been calls to standardise the use of early warning scores throughout the NHS, including primary care, to aid early identification of acutely unwell patients and facilitate communication across care settings.4 There are examples of out-of-hours general practice services using scoring systems and demonstrating that is feasible to calculate NEWS routinely,5 and clinicians report that using NEWS may aid decision making and communication.6 Currently, NHS England ‘encourage’, but do not mandate, the use of NEWS in the community.7 However, the impact of using NEWS in general practice has not been studied.8 Despite this, there are moves towards widespread adoption in both mainstream and out-of-hours general practice.9
The value of a test is dependent on the population to which it is applied.10 Community settings have a different prevalence of acute illness and a lower prevalence of patients who are at risk of physiological deterioration than EDs and inpatient settings; therefore, there is uncertainty about the utility of NEWS in primary care settings, compared with its established role in acute care. If patients are referred to hospital from urgent primary care settings only because they are at risk of deterioration, then higher NEWS values should be associated with the decision to refer. There is a need, therefore, to establish how NEWS relates to referrals in the primary care setting:, to confirm that higher NEWS values are associated with referrals and to explore how using a NEWS referral threshold in this setting could impact on referrals.
The objective of this research was to use out-of-hours general practice data to investigate the distribution of NEWS in a primary care population seeking urgent care and to assess the relationship between NEWS and the current decisions made by general practitioners (GPs) to escalate care to hospital.
Methods
This was a historical cohort study using electronic patient record data contained within the Birmingham Out-of-hours general practice Research Database (BORD).
Setting
BORD is a database of all consultations with a large out-of-hours general practice provider (Badger Group) from July 2013 to July 2018 (inclusive). The consultations encompass the whole range of out-of-hours general practice care provided to a population of around 1.5 million in the Birmingham area. The area covered includes a wide variation in ethnicities, cultures and socioeconomic status. All clinical encounters are recorded on the Adastra clinical patient management system,11 which contains coded and free-text information including patient demographic details, clinical observations, diagnostic classifications and outcome codes for each case but with no linkage to core hours general practice or secondary care systems. Clinicians have unique identifiers, and patient identifiers are consistent across multiple contacts. Face-to-face clinical encounters take place in a number of treatment centres across the area as well as in patients’ homes. Patients attending treatment centres are usually seen on arrival by healthcare support workers who take basic details and record clinical observations prior to the consultation with a GP or advanced nurse practitioner. Observations are entered into the electronic record using a template, but this does not generate a NEWS value. Patients seen in their own homes are seen directly by the GP who records the consultation on a laptop computer with access to the Adastra system, usually at the end of the visit.
Participants
We included all adults (aged ≥16 years) who had a face-to-face consultation with a clinician (GP, GP registrar or nurse practitioner) either in a treatment centre or in their home between 1 July 2013 and 31 July 2018 (inc). There was no prespecified study size. All clinical presentations were included. Only patients with a full set of clinical observations, entered using a clinical template, were used in the analysis. For encounters where NEWS could not be calculated, observations were still recorded, but they may not have been a complete set and/or were not recorded using the template. An individual patient may be included in the cohort more than once as each individual encounter was considered as a separate episode of care.
Variables
The variables included were: patient demographics (age, gender, ethnicity and national deprivation quintile (first signifying least deprived)12), location (treatment centre or home) and their examination findings. Documented physiological observations were extracted from encounters where the observations were recorded using the specific template within the clinical system.
Patients with physiological observations that were at the biological extremes (respiratory rate <7 or >40 breaths/minute, temperature <34°C or >41°C, oxygen saturations <90%, systolic blood pressure <80 mm Hg or >220 mm Hg and pulse <40 or >150 beats/minute or >150) were identified, and these encounters were checked for biological and clinical plausibility through comparison with other physiological parameters, the clinical setting, treatments and consultation outcome. Clinical plausibility was agreed between two authors (SF and DL), both with clinical experience in this setting. Where the extreme values were clearly implausible, these were regarded to be as a result of a data entry error, and these encounters were removed.
The outcome of interest was the plan for follow on care after the consultation. This variable contained one or more statements that described the outcome of the consultation. Clinicians could select from a number of set statements. Consultation outcome strings of ‘999 Arranged By Badger’, ‘Referral To Hospital’, ‘Advised To Attend A&E’ and ‘Advised To Ring 999’ were considered as a referral to hospital. For some encounters, these codes were entered along with other codes indicating that the patient had not been referred to hospital, but the clinician had advised attending under certain circumstances (a ‘safety net’ code.) These patients were not interpreted to have been referred to hospital (see online supplementary appendix 1 for codes).
Supplemental material
The NEWS was calculated post hoc using the extracted physiological data. All patients were assumed to be conscious and breathing air (not oxygen) and so the variables used to calculate NEWS were the respiratory rate, oxygen saturations, systolic blood pressure, pulse and temperature.
Statistical analysis
Data were manipulated and analysed using STATA V.14. The demographics of the cohort were described using summary statistics. The encounters were identified as either home visits or treatment centre consultation, and those with a full set of observations (allowing for a post hoc NEWS calculation) were treated as a separate group. The differences between the populations with and without a calculated NEWS were tested using t-tests and χ2 test.
Once biologically implausible variables were removed, the distribution of the NEWS was described for the whole cohort, according to whether patients were referred (to hospital) and by age categories (10 year age brackets). NEWS for patients who were referred, and those who were not, were compared using the Wilcoxon rank-sum statistic.
The alignment of NEWS with referral to hospital was established using the area under the receiver operating characteristic (AUROC) curve. A subgroup analysis was undertaken to establish whether the performance of NEWS differed between consultations in the treatment centre and home visits. Logistic regression modelling was undertaken to establish the influence of NEWS on risk of admission compared with age, gender and Townsend quintile (as a categorical variable). AUROCs were calculated using the dataset without excluding the biologically implausible physiological observations as a sensitivity analysis and to establish ‘real world’ performance characteristics.
Patient and public involvement (PPI)
The database used (BORD) was developed with PPI. The patient group involved with the creation of BORD recognised the need to use these data to investigate care in the OOH setting and felt that this was sufficient justification to use anonymised patient records without individual consent.
Results
From the 537 663 encounters in the database, 242 451 were face-to-face encounters with adults. Of these, 187 256 (95% CI 77.2%, 77.1% to 77.4%) were encounters in treatment centres, and 55 195 (95% CI 22.8%, 22.6% to 22.9%) were home visits. After excluding observations that were biologically implausible, a NEWS was calculated for 74 914 of these face-to-face encounters (figure 1).
CONSORT diagram. CONSORT, Consolidated Standards of Reporting Trials; NEWS, National Early Warning Score.
Table 1 shows the characteristics of the patients involved in these encounters and statistical differences in characteristics between encounters where the clinical observation template was used (from which a NEWS could be calculated) and those where it was not. Overall, 8.5% (95% CI 8.4% to 8.6%) were referred. The referral rates were higher for patients with a full set of observations (where a NEWS could be calculated) compared with those without (9.2% vs 8.2%, respectively, p<0.001).
Demographic characteristics*
The vast majority of encounters where a NEWS could be calculated took place in treatment centres (99.1%, 95% CI 99.0% to 99.1%). The NEWS of all patients in these encounters are shown in table 2. A percentage of 46.9% (95% CI 46.6% to 47.3%) of all patients had a NEWS of 0 with a further 30.6% (95% CI 30.2% to 30.9%) having a NEWS of 1. Figure 2 shows the distribution of NEWS overall, in each age category and in different locations (home visit and treatment centres). The minimum NEWS was 0, and the maximum was 12 (recorded for one patient).
Distribution of NEWS in each age category and location (scores of 5 and above are grouped together due to low frequencies). HV, home visit; NEWS, National Early Warning Scores; TC, treatment centre.
Distribution of NEWS for patients referred and not referred to hospital
Figure 3 shows the distribution of NEWS for patients that were referred to hospital and those that were not; these distributions are statistically different (Wilcoxon rank-sum z=−26.9, p<0.001). Of the 68 036 patients not referred to hospital, 1035 (1.52%, 95% CI 1.43% to 1.62%) had a NEWS of ≥5. Of the 6878 patients referred to hospital, 474 (6.89%, 95% CI 6.32% to 7.51%) had a NEWS of ≥5.
Frequency distribution of NEWS for patient referred and not referred and proportion referred for each NEWS value. NEWS, National Early Warning Scores.
The AUROC for NEWS in predicting referral to hospital for all encounters was 0.591 (95% CI 0.584 to 0.598). For patients seen in their own homes, the AUROC was 0.731 (95% CI 0.681 to 0.787), whereas in the treatment centres, the AUROC was 0.589 (95% CI 0.582 to 0.596). These analyses were performed without excluding biologically implausible values, and the overall AUROC was 0.592 (95% CI 0.585 to 0.599); treatment centre consultations and home visits were 0.590 (95% CI 0.583 to 0.597) and 0.732 (95% CI 0.680 to 0.785), respectively.
Logistic regression modelling showed that increasing age in years and increasing NEWS both significantly increase the risk of being referred to hospital from the OOH encounter table 3.
Odds of referral to hospital by demographic variable and NEWS
Discussion
The vast majority of patients seen in out-of-hours general practice, and those who are referred to hospital from this setting, have a low NEWS (0 or 1) indicating that these patients do not have significantly abnormal physiology
In this situation, where NEWS has not been calculated but observations are recorded, most patients with a NEWS of ≥5 are not referred to hospital. If all patients with a NEWS of ≥5 were referred to hospital, there would have been an extra 1116 referrals in addition to the 6878 patients already referred. This 16.2% increase in referrals would equate to around 26 500 extra hospital attendances in England every year (see appendix 2 for assumptions). It is possible that using a NEWS cut-off would reduce referrals for patients with a NEWS of <5, although modelling this potential shift in behaviour is complex. Acute medical illnesses have a spectrum of severity and may not cause a raised NEWS for patients presenting in an urgent care setting in the community. In addition, referral to hospital can also be made by GPs as a response to a temporary functional decline with a lack of social care during an acute illness.
In the setting of primary care, NEWS does not align consistently with referral decisions. There are likely to be a variety of other factors influencing referral decisions that are particularly relevant to this setting including limited (or no) access to patient records and investigations, a lack of alternative care pathways and poor social and healthcare support mechanisms out-of-hours. At the same time, a decision not to refer could be because the patient’s problem can be managed in the community, with an appropriate safety net and follow-up advice.
For patients seen in their own homes, there is a clearer relationship between NEWS and referrals than those seen in treatment centres. This is likely to be due to the fact that the populations are quite different, with home visit patients being considerably older (by 37 years), with less physiological reserve and more complex medical needs. However, a much smaller proportion of patients seen at home had observations recorded using the template allowing a NEWS to be calculated. This is probably due to the usual practice of a small number of clinicians but may indicate that these patients were systematically different from the patients without a NEWS (eg, they could be the more unwell patients)
Strengths
This study adds to existing knowledge on the distribution of NEWS in a community setting and is the first to include undifferentiated primary care patients not referred to hospital. The large sample size and long time frame increase the reliability of the results. The literature suggests that recording bias can exist for clinical observations when there are pre-existing thresholds known to the clinician.13 14 By calculating NEWS post hoc, the impact of recording bias is minimised. Additionally, in the majority of encounters in treatment centres (not home visits), the clinical observations are recorded by staff prior to their clinical consultation and therefore independently of the decision makers, again reducing chances of recording bias. Finally, as the NEWS was not known to the clinicians when making referral decisions, this study avoids any bias or threshold effect that may be introduced when using a scoring system.
Limitations
This study examines whether NEWS is related to the decision to refer. However, it is not known whether the decision to refer resulted in the patient subsequently attending hospital, whether the referral was deemed appropriate by the secondary care team, or indeed whether patients not referred were successfully managed in the community. This could be overcome through linkage with secondary care data, but this was not possible in this study.
Although the population covered by this out-of-hours provider is large and diverse in ethnicity and socioeconomic status, the generalisability to other areas may be affected by different demographics and out-of-hours arrangements. Additionally, there were differences in the demographics of patients in encounters where a NEWS could be calculated and patients where a NEWS could not be calculated, especially among the encounters in patients’ homes (where healthcare support workers were not present and the observation recording templates were used infrequently and only by particular clinicians). This is likely to be due to differences in practice between clinicians and systematic differences across the area with some treatment centres having better access to healthcare support workers (who record the observations in a systematic manner) than others. Additionally, for patients seen in their own homes, the observations may have been recorded after decision making, which introduces the possibility of recording bias that could make NEWS appear more consistent with referral decisions. In the treatment centres, whether a patient had observations entered via the clinical template was independent of their presentation but would have been influenced by the availability of healthcare support workers
Finally, some assumptions had to be made when the NEWS was calculated. First, all patients were assumed to be conscious. Given the nature of the clinical pathway of care (general practice rather than emergency services), this assumption is not contentious. Second, it was assumed that the patients were not assessed while receiving supplementary oxygen, reflecting the community healthcare delivery setting. It is probable that a few patients seen may have been receiving long term oxygen therapy although the numbers are likely to be small. These patients may have inaccurate NEWS estimates. The removal of presumed data entry errors (biologically and clinically implausible observation values) did not appear to impact on the main findings significantly (appendix 3). It is also assumed that clinicians are not using the recorded observations to calculate NEWS during their consultations to aid their clinical decision making. Although it is possible that some clinicians may use this tool, it is considered unlikely as this is not standard practice across the organisation and is not a practice that has been observed by the clinical teams within the organisation.
Comparisons
The referral rates in this cohort are similar to those published elsewhere in comparable populations. In the southwest of the UK, referrals to hospital from out-of-hours general practice were 9.2% between 2001 and 2004,15 and 9.5% in 2005/2006.16 In the Oxford area, 8.3% of patients seen in out-of-hours care were referred directly to secondary care.17 These are similar to the 8.5% overall referral rate in this cohort.
In an area of the UK where there has been widespread adoption of NEWS across healthcare settings, there has been an analysis of NEWS distributions that found that 72% of patients seen by a community health team (mainly seeing patients at risk of admission or recently discharged from hospital) had a NEWS of 2 or lower, considerably less than the 90% in our cohort.18 Additionally, when the population referred to hospital by GPs was considered, 46% had a NEWS of 2 or lower and 3% scored ≥9, whereas the comparable figures in our cohort were 79.6% and 0.4% indicating again that the cohort seen in out-of-hours care has a higher proportion of patients with a low NEWS than has been observed in other settings. These data also suggest that the threshold for referral to hospital may be lower in out-of-hours general practice when compared with care during standard working hours. However, the community health team patient population studied was not representative of mainstream general practice. Differential referral thresholds between mainstream and out-of-hours care may be due to different capacities to deal with risk, which has been found to influence referrals from out-of-hours care.19
Further research and implications
This study highlights the need to understand how clinicians may respond to an early warning system in a primary care setting. Given the low acuity of patients seen in treatment centres, and the weak association between NEWS and referral decisions, the impact of adopting a scoring system in this setting is difficult to predict. If NEWS were to be used to guide referrals, this may result in the additional referral of 1.5% of patients currently managed in community settings with a NEWS of 5 or more. Additionally, there may be underreferral of patients with a lower NEWS currently referred based on clinical judgement alone. The clinical impact of this strategy should be measured if wider adoption of NEWS is considered in community settings. Further research is needed into how clinicians use scoring systems alongside clinical judgement when making decisions about referring patients to hospital and the place of NEWS in aiding communication of risks and benefits of referral with patients.
Conclusions
This study establishes the NEWS and physiological acuity of patients referred from out-of-hours general practice to hospital from one primary care organisation and, importantly, those patients who are seen but not referred. The decision to refer a patient to hospital is an important one given the potential benefits and harms of the decision and the economic costs. Further research should provide evidence of the clinical impact of the introduction of early warning score-based decision making in out-of-hours general practice in order to inform recommendations on its use.
Acknowledgments
We would like to thank the management team and the data analysts at Badger Group for their support and generosity in allowing and facilitating the creation of Birmingham Out-of-hours general practice Research Database (BORD). Thank also to Professor Tom Marshall for his comments on the manuscript.
References
Supplementary materials
Supplementary Data
This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.
Supplementary Data
This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.
Footnotes
Twitter @sfinnikin
Correction notice This paper has been updated since first published with an update in figure 1.
Contributors SF and DL designed the study. Data collection and analysis was performed by SF. All authors contributed to data interpretation and writing of the manuscript. The lead author (SF) affirms that the manuscript is an honest, accurate and transparent account of the study being reported and that no important aspects of the study have been omitted.
Funding This research was unfunded, but the authors would like to acknowledge the West Midlands Clinical Research Network whose funding allowed the creation of BORD. GH and DSL are funded by the NIHR Community Healthcare Medtech and IVD Cooperative.
Disclaimer The views expressed are those of the authors and not necessarily those of the NHS, the National Institute for Health Research or the Department of Health and Social Care.
Competing interests SF is a fellow in Evidence and Values at the RCGP and a senior clinical tutor at the University of Birmingham. He receives income from freelance writing, lecturing and book authorship and does consultancy work for EBSCO industries. FW is the medical director of Badger Group.
Patient consent for publication Not required.
Ethics approval This research was carried out under the ethical approval for the BORD, which was granted by the West Midlands – South Birmingham Research Ethics Committee on the 22 January 2019 (reference 19/WM/0010).
Provenance and peer review Not commissioned; externally peer reviewed.
Data availability statement Data are available on reasonable request.