Article Text

Ability of prehospital NEWS to predict 1-day and 7-day mortality is reduced in older adult patients
  1. Jussi Pirneskoski1,
  2. Mitja Lääperi1,
  3. Markku Kuisma1,
  4. Klaus T Olkkola2,
  5. Jouni Nurmi1
  1. 1 Department of Emergency Medicine and Services, University of Helsinki and Helsinki University Hospital, Helsinki, Uusimaa, Finland
  2. 2 Department of Anesthesiology, Intensive Care and Pain Medicine, University of Helsinki and Helsinki University Hospital, Helsinki, Uusimaa, Finland
  1. Correspondence to Jussi Pirneskoski, Department of Emergency Medicine and Services, University of Helsinki, Helsinki, Uusimaa, Finland; jussi.pirneskoski{at}helsinki.fi

Abstract

Background National Early Warning Score (NEWS) does not include age as a parameter despite age is a significant independent risk factor of death. The aim of this study was to examine whether age has an effect on predictive performance of short-term mortality of NEWS in a prehospital setting. We also evaluated whether adding age as an additional parameter to NEWS improved its short-term mortality prediction.

Methods We calculated NEWS scores from retrospective prehospital electronic patient record data for patients 18 years or older with sufficient prehospital data to calculate NEWS. We used area under receiver operating characteristic (AUROC) to analyse the predictive performance of NEWS for 1 and 7 day mortalities with increasing age in three different age groups: <65 years, 65–79 years and ≥80 years. We also explored the ORs for mortality of different NEWS parameters in these age groups. We added age to NEWS as an additional parameter and evaluated its effect on predictive performance.

Results We analysed data from 35 800 ambulance calls. Predictive performance for 7-day mortality of NEWS decreased with increasing age: AUROC (95% CI) for 1-day mortality was 0.876 (0.848 to 0.904), 0.824 (0.794 to 0.854) and 0.820 (0.788 to 0.852) for first, second and third age groups, respectively. AUROC for 7-day mortality had a similar trend. Addition of age as an additional parameter to NEWS improved its ability to predict short-term mortality when assessed with continuous Net Reclassification Improvement.

Conclusions Age should be considered as an additional parameter to NEWS, as it improved its performance in predicting short-term mortality in this prehospital cohort.

  • prehospital care
  • clinical management
  • clinical assessment
  • emergency ambulance systems

Data availability statement

All data relevant to the study are included in the article or uploaded as supplementary information.

Statistics from Altmetric.com

Key messages

What is already known on this subject

  • Increasing National Early Warning Score (NEWS) has been shown to be a good predictor of early mortality in both in-hospital and prehospital environments. Age is not a parameter in NEWS, despite studies showing improved performance of early warning scores incorporating age as a parameter.

What this study adds

  • In this retrospective prehospital cohort study of 35 800 patients, accuracy of NEWS in predicting death within 1 and 7 days decreased as patients’ age increased. Adding age as a three-category parameter to NEWS improved its predictive performance for 1 and 7 day mortalities when assessed with continuous Net Reclassification Improvement.

Introduction

National Early Warning Score (NEWS) was developed in the UK in 2012 by the Royal College of Physicians to standardise the physiological evaluation of patients.1 It has been shown to predict short-term mortality with good precision in a number of studies in hospital wards,2 EDs3 and recently also in prehospital environments.4–6

Normal physiological responses to different acute disturbances change dramatically as people age. Older adult patients also often present with decreased general condition rather than specific symptoms.7 These characteristics make appropriate detection of the critically ill older adults during emergency care challenging8 and it is known that the recognition of worsening physiological state of older adult patients often goes unnoticed in the EDs leading to undertriage both in the general ED and trauma populations.9 10 Frailty is a known risk factor in older adults, but current frailty scores are ill-suited to emergency use due to complexity and lack of established gold standard method.11

Despite these physiological differences between younger and older patients, age was not included in the original NEWS1 or its revised edition,12 even though earlier research suggested that this might improve mortality prediction.13 Age has also been shown to be an independent risk factor for mortality in, for example, ED patients.14

We used a previously collected large prehospital data base to analyse the effect of age on the predictive performance of NEWS for early mortality as 1-day mortality as a primary outcome and 7-day mortality as a secondary outcome. We also analysed the potential improvement of predictive performance by adding age as a new component to NEWS. Our hypothesis was that the predictive performance would decrease in the older age groups and addition of age would improve the predictive performance of NEWS.

Methods

Study setting

The retrospective cohort study was conducted in the Hospital District of Helsinki and Uusimaa, Finland. The study protocol was approved by the Department of Emergency Medicine and Services, Helsinki University Hospital. No informed consent or Ethics Committee approval was required by Finnish legislation for this retrospective registry study.

Study was conducted in a physician-led emergency medical services (EMS) system described in detail in a previous study4 in a hospital district with a population of 1.6 million people. The EMS system is three-tiered, composing of basic life support units manned by emergency medical technicians, ALS units manned by paramedics and two mobile intensive care units manned by physicians. Dispatching is handled in a national Emergency Response Centre by non-medical personnel.

Data collection

We included all patients over 18 years of age from 17 August 2008 to 18 December 2015. Patients younger than 18 years, patients with incomplete NEWS parameters and patients with no national personal identity number were excluded. Physiological data were collected using a monitor-defibrillator for oxygen saturation, systolic BP and HR and automatically. Other physiological parameters were entered manually. Mortality data were acquired from Statistics Finland based on the national personal identity number.

Patients with ‘Do Not Attempt to Resuscitate’ (DNAR) orders were not excluded from the study, as these data are not entered in the prehospital electronic patient records in a structured format and was thus unavailable.

Study groups

Based on previous research, we divided the patients in the following three age groups: <65, 65–79 and ≥80 years. Even though there is no medical consensus on the definition of an older adult patient, the upper limit of 64 years in the first group was selected as the term older adult is commonly associated with patients aged ≥65 years in previous research regarding critically ill patients.15 More recently, patients aged over 80 have been considered to create a specific group of patients, ‘very old patients’,16 and thus this was selected as the cut-off for the last group. For the purposes of the model improvement analyses, we assigned scores 0, 1 and 2 for the first, second and third age groups, respectively, based on the logistic regression model estimates.

Statistical methods

We analysed the effect of increasing age on the ability of NEWS to predict early mortality, with the primary outcome of 1-day mortality and the secondary outcome of 7-day mortality. We also analysed if adding age as an additional parameter increased the predictive performance of NEWS.

Effect of age on predictive performance of NEWS

We fitted logistic regression models for the NEWS parameters, NEWS parameter scores and the NEWS score. The models were also fitted with the addition of the age parameter. We also fitted the models for each age subgroup to investigate potential age-related differences in the NEWS parameters. The 95% CIs for the coefficients were calculated using the profile likelihood method. We investigated the model discrimination using area under receiver operating characteristic (AUROC). The results for all AUROC analyses are presented with 95% CIs in parentheses.

Model performance with age as an additional parameter

We used loess regression to investigate the predicted risk for the NEWS score. We compared the models using Likelihood-ratio tests (LRT) and assessed reclassification performance using continuous Net Reclassification Improvement (cNRI). cNRI evaluates how well the new model classifies single subjects appropriately to true and false positives and negatives compared with the old model. The results are presented as a sum of proportion of events and non-events correctly reclassified with the new model when comparing to the old one. The ranges of the cNRI of survivors and non-survivors are −100% to 100% with the sum of these ranging from −2 to 2.17

Statistical analysis was performed using Prism 6.0 hour (GraphPad Software, San Diego, CA, USA) and R software V.3.5.1 (The R Foundation for Statistical computing, Wien, Austria) using packages ggplot2, plotROC.

Results

There were 750 964 ambulance calls during the study period. 36 343 of these had sufficient data to calculate NEWS values. 35 800 records were deemed appropriate for the final analysis. Flow chart of the study population selection is shown in figure 1. Demographic data of the study population are shown in table 1 and available demographic data of the excluded patients are shown in online supplemental table 1. One-day mortality was 1.1% and 7-day mortality was 2.4%. The relative risks for 1 and 7 day mortalities as a function of age are presented in figure 2.

Supplemental material

Figure 1

Flow chart of study patient selection.

Figure 2

Loess plot of the risk of death at 1 and 7 days as a function of age of the patient. Risk increases slightly up to around 65 years, then plateaus and again starts to sharply increase at around 80–85 years of age.

Table 1

Study population characteristics

Effect of age on predictive performance of NEWS

AUROC of NEWS for 1-day mortality in age groups<65, 65–79 and ≥80 years were 0.876 (0.848 to 0.904), 0.824 (0.794 to 0.854) and 0.820 (0.788 to 0.852), respectively. The corresponding values for AUROC of NEWS for 7-day mortality in the same groups were 0.841 (0.812 to 0.869), 0.802 (0.778 to 0.826) and 0.787 (0.765 to 0.808), respectively.

Data regarding risk of death at different NEWS scores in the different age groups are shown in figure 3. Older age groups are in higher risk of death at lower NEWS values than the youngest age group.

Figure 3

Loess plot of the risk of death as a function of the value of NEWS in the different age groups at 1 and 7 days. Risk of death at equal NEWS values is increased in the older age groups. NEWS, National Early Warning Score

Figure 4 shows a Forest plot of ORs for 1-day and 7-day mortality for each individual NEWS parameter from the logistic regression models in the three age groups. The NEWS score was fitted as an univariate model, while the NEWS parameters were fitted to a multivariate model. For both 1-day and 7-day mortality, the different parameters had a varying effect on risk of mortality. RR and body temperature had a higher OR in the youngest age group, whereas the level of consciousness had a lower OR. HR did not affect the risk of mortality in any of the age groups.

Figure 4

OR for the risk of death at 1 and 7 days for NEWS score univariate model and NEWS parameters multivariable model. The ORs are per one point increase for each parameter. RR and body temperature had a higher OR in the youngest age group, whereas the level of consciousness had a lower OR. HR did not affect the risk of mortality in any of the age groups. NEWS, National Early Warning Score.

Model performance with age as an additional parameter

When age was included as an additional NEWS parameter, the values of AUROC in regard to 1-day or 7-day mortality did not change in a statistically significant degree. The values of AUROC for 1-day mortality were 0.840 (0.823 to 0.858) and 0.845 (0.829 to 0.862) for NEWS without and with the three-category age parameter, respectively. For 7-day mortality, the corresponding values of AUROC were 0.809 (0.795 to 0.823) and 0.821 (0.808 to 0.834), respectively.

Table 2 presents the data regarding reclassification tests for 1 and 7-day mortalities. For both 1 and 7 day mortalities, cNRI showed improved overall classification with the new score. The age parameter improved the sensitivity of NEWS by correctly reclassifying non-survivors to have higher scores, but on the other hand worsened the specificity by also increasing the points for older survivors. LRT showed that addition of age resulted in statistically significantly different models favouring the new model (p<0.001).

Table 2

Reclassification test results

Discussion

We found that the accuracy of NEWS in predicting death within 1 and 7 days decreased as patients’ age increased. We also found that the risk of death for patients over 65 years of age was statistically significantly higher when the value of NEWS exceeded seven for 1-day mortality and four for 7-day mortality compared with patients under 65 years of age. Adding age as a three-category parameter to NEWS improved its predictive performance for 1 and 7 day mortalities when assessed with cNRI. In the multivariate logistic regression model, single NEWS parameters had an uneven effect on the prediction of mortality.

Omission of age as a parameter from the original NEWS seems somewhat confusing. It is mentioned as a parameter considered to be included in both the first and second editions based on the knowledge that older age increases risk of death,1 12 but the only in-hospital study reference given as an explanation13 actually promotes age as a component of an EWS. To the best of our knowledge, there are no prehospital studies regarding the effect of age on NEWS and only one recent study evaluates the addition of age as a parameter to NEWS to improve its predictive performance.18 This study demonstrated an improved prediction of in-hospital mortality.

De Groot et al found out that a number of early warning scores (EWSs) including NEWS performed worse in patients aged 70 years or over than in younger patients in predicting in-hospital mortality of ED sepsis patients.19 Jarvis et al have also explored the comparability of mortality prediction in different EWSs.20 In this study, removing age as a parameter decreased the predictive performance of the EWSs incorporating age, although it made them more comparable to EWSs using solely vital sign data. Two studies21 22 have compared the predictive performance of NEWS when recorded in a prehospital environment and on arrival to ED. Of these, the study by Mitsunaga et al focused specifically on older adult patients. In their study, the endpoints were admission to hospital and in-hospital mortality. For both, the AUC was significantly lower than what we present here for 1 and 7 day mortalities. In addition to these, we were able to find only two studies23 24 evaluating the predictive performance of EWSs in older adult patients, neither of which are from prehospital cohorts.

Frailty is considered as a superior outcome predictor compared with chronological age in older adults, but as Aguayo et al point out, there is currently no golden standard for measuring frailty. The different frailty scales cannot be compared due to differing concepts and diagnostic accuracy.11 They suggest preferring a multimodal scale, which would be cumbersome to use in an EWS context, especially in emergency care. Thus, even though chronologic age does not strictly correlate with medical fitness of a specific patient, on a population level, older age still is strongly associated with increased disease burden, use of medications possibly altering physiological measurements (eg, beta blockers) and lower physiological reserves for recovering from critical illness. Decisions limiting the older patients outside the most invasive and intensive interventions might also partly explain the results of improved performance in predicting 1- and 7-day mortality.

Even though the cut-offs for age groups used in our study might not be optimal or fully generalisable to all populations, as their risk stratification weight could differ in different societies and clinical environments, our data suggest that age is a significant determinant of outcome and can be used to improve a physiological scoring system. As demonstrated in figure 2, the age groups based on previous studies seemed to fit well with the changes in risk of death observed for this prehospital cohort: the risk of death increases up to around 65 years, stays constant for up to around 80–85 years and then starts to increase rapidly.

Our data support de Groot et al, 19 who showed that predictive performance of NEWS is reduced in older in-hospital patients. Our results are also in agreement with recent study by Lee et al,18 which shows that age as an additional parameter in NEWS significantly improved the in-hospital mortality prediction based on NEWS calculated at admission. Even though in-hospital mortality is a suboptimal outcome measure for performance of NEWS, as it is best suited to detect short timescale mortality, the study still supports the rationale of using age as a parameter to improve the predictive ability of an EWS. Optimally, this would also improve the ability of NEWS to identify patients who are likely to benefit from treatment from those who are likely to die despite active efforts.

We used cNRI and LRT to analyse the improvement in diagnostic performance when adding age as an additional parameter to NEWS, as AUROC is suboptimal for detecting small improvements when modifying already well-performing models. Both cNRI and LRT also have their issues. cNRI improvement may not be easy to determine, while LRT p-values should be interpreted with regard to the large sample size. As shown in table 2, the overall performance of the model improved, and the new model had a better ability to predict death when the score value increased. According to the groupwise cNRI data, the new model decreased the ability to correctly detect true negatives and thus reduced specificity; this was overcome by an even stronger performance in correctly detecting mortalities thus increasing sensitivity. The reduced specificity is caused by surviving older patients receiving higher total scores but compensated in the overall model performance as older patients are also more likely to die. Increased maximum score due to an additional parameter should be taken into account and if the risk strata are to be used, they need to be modified accordingly.

The data regarding varying ORs for mortality for different parameters in the three age groups are intriguing. For example, for the youngest age group, a disturbance in level of consciousness seemed to be less hazardous. Reasons for lowered level of consciousness are likely to differ between the age groups, but this could also reflect the reduced capacity of older adults to overcome such significant medical issues that result in lowered consciousness. Interestingly, HR did not seem to predict 1-day or 7-day mortality to any degree in any of the age groups in our study population.

There were limitations in this study. This is a retrospective analysis of a prehospital patient cohort in just one developed country in one geographical area. Thus, it likely does not fully represent the global perspective in regard of, for example, the age limits used for risk stratification. There is also a high risk of selection bias as at the time of data collection there was no EMS protocol requiring input of all required data to the electronic patient record system and a significant proportion of the patients encountered were excluded from the study based on missing vital signs and due to this, a large majority of the patients encountered by EMS were excluded from the study. We were also unable to exclude patients with DNAR orders, which may have altered these patients’ treatment priorities and outcomes, as these data were not available in a structured form in the EMS patient records. Specifically, DNAR orders may have affected a larger proportion of the older age groups and thus increased mortality in these subgroups. The limitations and possible sources of bias are further discussed in our previous publication from the same patient cohort.4 As this was a post-hoc analysis of existing data, no power calculations were performed for this specific research question. Thus, despite the large study cohort, the study might not have been powered to achieve statistical significance in the performed analyses.

On the other hand, the strengths of the study include a large study cohort, data accuracy through electronic patient records and linking to mortality via patient specific national identity number. Performing the study in a prehospital environment also gives access to data from patients that have received no treatments altering their vital signs. These data were also collected from a single EMS system, which is likely to further improve data quality.

Future studies should include confirming these results in another cohort. Further work into determining the optimal age groups for risk stratification should also be conducted. Age should be reconsidered as an additional parameter for NEWS, as it improves the predictive performance without significantly increasing its complexity, as age of the patient can be easily automatically extracted from electronic patient record systems. It is still unsolved whether current NEWS parameter scoring thresholds are optimal and what are the optimal parameters. As shown in the current study, HR did not seem to contribute to mortality prediction.

In conclusion, these results demonstrate that NEWS does not work similarly in mortality prediction of the prehospital patients older than 65 years, as their risk of death is higher than the younger patients at similar NEWS values. We suggest confirming these results in other patient populations and to consider age as an additional parameter in possible future revisions of NEWS. Meanwhile, prehospital clinicians should take into account the increased risk of death at similar scores in the older patients when assessing NEWS values of these patients.

Data availability statement

All data relevant to the study are included in the article or uploaded as supplementary information.

Ethics statements

Patient consent for publication

Ethics approval

This study did not involve human participants. The study protocol was approved by the Department of Emergency Medicine and Services, Helsinki University Hospital. No informed consent or Ethics Committee approval was required by Finnish legislation for this retrospective registry study.

References

Supplementary materials

  • Supplementary Data

    This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.

Footnotes

  • Handling editor Kirsty Challen

  • Contributors All authors have had substantial contributions to the conception and design of the work. JP, MK and JN were responsible for the data acquisition and all authors contributed to the analysis and interpretation of data. JP and ML were responsible for drafting the first version of the manuscript and all authors contributed significantly on revising it critically and approved the final version. All authors agreed to be accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved.

  • Funding The authors have not declared a specific grant for this research from any funding agency in the public, commercial or not-for-profit sectors.

  • Competing interests None declared.

  • Provenance and peer review Not commissioned; externally peer reviewed.

  • Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.

Request Permissions

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.