Article Text
PDF
Abstract
Early Warning Scores (EWS) have become increasingly used by hospitals throughout the world to prevent unexpected admission to intensive care or even death in their inpatient population. It is well known that signs of deterioration are present well before collapse and by a combination of systems, EWS enable healthcare professionals to intervene at an appropriate time. A number of national bodies and regulators in the UK have required the use of Early Warning Scores in locations where children are inpatients. This article attempts to describe the background to their development, identify common problems and provide information for units interested in introducing an EWS into their department.
Statistics from Altmetric.com
Introduction
The reliable identification of the critically ill or deteriorating child has been both the Holy Grail and Achilles' heel of paediatric practice for some time. It is well recognised that paediatric inpatients who die or subsequently require intensive care often show signs of physiological and behavioural disturbance prior to arrest or collapse.1 As a result, one of the recommendations in the Confidential Enquiry into Maternal and Childhood report ‘Why Children Die’ stated:
‘For paediatric care in hospital we recommend a standardised and rational monitoring system with imbedded early identification systems for children developing critical illness – an Early Warning Score (EWS)’.2
The National Patient Safety Agency (NPSA) ‘Review of patient safety for children and young people’ in 2009 supported this view3 as also the National Institute for Clinical Excellence.4 In the UK, it is therefore incumbent on providers of inpatient paediatric services to review their monitoring systems. The importance of this is highlighted by the National Health Service (NHS) outcomes framework which contains a specific indicator incidence of harm to children due to ‘failure to monitor’.5 A cross-sectional survey of UK hospitals in 2005 found only a small proportion of units were using them. The questionnaire determined: ‘that 21.5% of the 144 NHS Trusts that care for children as inpatients have an early warning system, but there is no consistency of approach’.6
Anecdotal reports that EWS increase workload rather than improve patient safety combined with variable definitions of their purpose have often created mistrust in their use reciprocally, the increasing use of short-stay units and pressures on out-of-hours services have prompted their introduction in acute assessment areas, although in this context, they may be considered serious illness identification aids rather than true ‘Early Warning’ Scores.
It is important for patient safety, clinical governance and for healthcare professional education that all units have established mechanisms by which children who require enhanced care can be easily recognised. One such, but not the only, mechanism is an EWS. Although relevant literature will be discussed, this article is not intended to be a systematic review of EWS as this has been recently performed.7 The aim is to describe the background to their development and identify common issues to provide information for units interested in introducing an EWS into their department. The term EWS in this article refer to paediatric systems unless stated in the text.
Background information
The First scoring systems
It may be considered that triage was the first real use of a ‘score’ based system to identify illness. However, in its original context it was used to identify those individuals, out of a large amount of casualties, who needed the most urgent intervention. Adult and paediatric EWS represent a means of identifying deterioration in those already deemed to have some medical need and therefore are different from triage (table 1). Whether they are relevant to illness identification at initial point of contact is not something that has been clearly delineated. If we are expansive in our inclusion of such systems, one of the earliest scores to be published was the Yale system developed by McCarthy et al. In the early eighties, he created an observation scale to identify serious illness in febrile children and identified six observational items (table 2) that were independent predictors of serious illness.8 The latter is defined as the culture of a bacterial pathogen in normally sterile tissue culture or any abnormalities of electrolytes, chest radiographs or blood gases. In the USA, primary care cohort 11.9% (37/312) patients aged less than 24 months were subsequently found to have serious illness. The initial Yale score had a sensitivity of 77%, specificity of 88% and a positive predictive value of 56%, that is, just over 20% of children had a serious illness when the Yale Score predicted there should be none. Of note, no child who smiled normally in this study had a serious illness.
Different types of scoring system
The initial Yale Score
The use of sensitivity and specificity to describe EWS and other measures of illness identification is often performed. However, to meaningfully compare studies, the outcomes must be identical and the overall incidence of disease measured. In Chapman et al's systematic review,7 values of false-negative rates were between 10% and 22% and it was noted a number of studies failed to or incorrectly calculated the figures.
The Yale Study's wide inclusion criteria for serious illness, and the fact that it was not designed as an EWS in the conventional sense, means it is not validated for inhospital use today. It does however form the back bone for the National Institute for Health and Clinical Excellence (NICE) Feverish illness in Children Guideline. The high-incidence rate of serious illness in the Yale Study was in a primary care group who you would have expected to have had a lower pretest probability of serious illness than a hospital cohort. The European research network on recognising serious infection investigators highlighted the importance of understanding the effect pretest probability has on the function of illness identification.9 Although they were not specifically looking at EWS, you can imagine that a tool identifying patients based on a postoperative recovery ward following grommet surgery may be at a lesser risk and type of deterioration than those following emergency surgery. The utility of the score therefore being very dependent on the characteristics of the population it is being used on.
A decade after the Yale score, Morley et al developed the Baby Check score.10 This system was designed to detect serious illness in infants below 6 months. It has been validated in settings outside its initial location of use including countries beyond the UK. However, those who have attempted to develop EWS soon discover that controlling for variation in physiology as you grow older makes the creation of an age-unifying chart problematic. This may explain the reason following the Yale and Baby Check publications, there seemed to be a nadir in published scoring systems for children.
The adult experience
It was not until in adult practice there was increasing realisation that patients were dying in hospital wards with potential warning signs being present nearly 24 h before their deaths,11 that national guidance began to appear. In 2003, the NHS modernisation agency12 coined the term ‘Track and Trigger system’ to bring together all the possible types of EWS that existed at that time. This was followed by NICE guidance4 on managing the acutely ill adult in hospital which contained some definitions regarding the different types of track and trigger systems available (table 3). An RCP report clearly found in favour of EWS and recommended an NHS warning score be used nationally in adult practice13 although this has been challenged due to the large heterogeneity of the population it is used on.14
The National Institute for Health and Clinical Excellence group divided known Track and Trigger systems into four categories
Development of children's systems
One of the first UK children's studies aiming to improve early medical staff review in the potentially deteriorating patient was by Monaghan whose ‘Brighton Score’ was a combination system15 (table 4).
The Brighton score
The evidence validating its use was not particularly strong (the original audit contained only 30 patients who had triggered the system and there is no clear data presented on re-audit). However, it is testament to the simplicity of the system that a version of the Brighton score is used in trusts outside of Brighton16 with the score continuing to undergo validation. Monaghan notes in the paper, ‘Some staff could not see why we needed a score as they felt they were quite capable of recognising patients at risk’. Implementation was also exacerbated by the additional paperwork that the score required on top of the normal observation chart (this was solved by incorporating the score into it). Engagement with staff is vital to ensure compliance, the introduction of an EWS at the Royal Free Hospital (London, UK) required over 16 iterations of a chart to be produced before a final version was agreed on (Personal Communication Sebastian Yuen, 2011) (figure 1).
NHS Institute PEWS Chart 1-5. PEWS, Paediatric Early Warning Score.
At a similar time, researchers in Australia17 showed (without definitive statistical significance) the benefits of a medical emergency team (MET) to prevent cardiac arrest and death in an inpatient paediatric population. This study used nurse and doctor ‘worry’ and range of physiological parameters as triggers.
Problems with EWS
It will always be difficult to quantify the effect the MET has over and above the education obtained from knowing that there is a team and what the criteria to call them are. Obviously controlling for this effect is extremely difficult. The Australian paper17 demonstrated the low incidence of mortality in children (0.19/1000) and hence the problem in using this as a relevant outcome measure. Despite this research on a MET being activated by an EWS, it has been replicated positively in other studies.18 Tibballs et al19 have recently looked specifically at systems to prevent inhospital arrest which shows a trend (both significant and non significant) towards a reduction in cardiopulmonary arrests in the studies reviewed. He notes that: ‘The key feature of a system is empowerment of any staff, including junior nurses and doctors and parents of children, to summon help without deferring to senior colleagues or to medical staff.’
Unfortunately, there is also the confounder of generally increasing standards of care as time goes on. A recent publication demonstrated a natural improvement in mortality without any system in place at all.20 Joffe et al compared mortality data in a hospital without a MET with data from studies during a similar time period and concluded there was little difference. The research team hypothesised that a number of links are needed to effectively capture the deteriorating child (table 5):
Chain of events needed to demonstrate an improved response to deterioration in inpatient children
From 2005 onwards, there was an explosion of abstracts and papers on Paediatric Early Warning systems. The Plymouth21 and subsequent Bristol22 systems were designed to be used by all staff to identify children needing paediatric intensive care unit (PICU) intervention as audits had demonstrated key critical signs were being ignored. Their charts followed an ABCDE approach (figure 2).
The initial Bristol PEWS chart (reproduced with the permission of Caroline Haines, Bristol Royal Hospital for Children).
Centres in Birmingham23 and Yorkshire24 used the Delphi method to design EWS which were subsequently validated. The Cardiff system has undergone a prospective evaluation and also applied the Australian MET system to its own unit.25 ,26 In both the Cardiff studies, the systems had reasonable sensitivities but at the cost of low specificity and positive predictive value, that is, few critical children were missed but a number of children activated unnecessarily. In these studies, a number of children with completely normal observations (table 6) had adverse outcomes, meaning, it is likely that no EWS will be 100% sensitive (ie, no children are falsely negative).
Number of observation for each Cardiff and Vale PEWS score
As well as systems which have been peer reviewed (but not necessarily validated) there are a variety of non-published scoring systems already in use across the UK. This profligation may demonstrate their use becoming more widely welcomed. However, there is no one system that is universally favoured and the thorny issue of proving benefit continues to be an issue.27 The lack of standardisation in charts may indicate that departments or trusts were not keen on using systems borrowed or derived from others. The need to adjust, refine and test any system in its place of practice (regardless of its validation elsewhere) is supported by implementation theory.28 It would be rational to suppose the introduction of an outside system into an unfamiliar environment is going to be fraught with difficulty. These problems with implementation and engagement had led the NHS Modernisation Agency to stipulate the positives and negatives of Track and Trigger systems (table 7).
Track and Trigger scoring systems (modernisation agency 2003)
There is also the particular problem of which actual measures should be used in the score. In the last survey of EWS 36, different physiological and observational parameters were used.6 There is already wealth of information on individual parameters prediction of serious illness. In a large cohort of febrile children presenting to a children's emergency department in Sydney, Australia, there was evidence that ‘… for all serious bacterial infections, appearing generally unwell was the strongest diagnostic marker, with raised temperature, no fluid intake in the previous 24 hours, increased capillary refill time, and chronic disease also predictive’.29 A small study in the UK, also based in an Emergency Department but prior to the introduction of the pneumococcal vaccine demonstrated developmental delay, risk factors for infection, alertness, level of temperature, capillary refill time, hydration status, respiratory rate and hypoxia to be relevant in the construction of a predictive model.30 These studies and others indicate certain physiological, behavioural and background parameters may be more beneficial than others in identifying illness. The studies producing this data tend to be aimed at identifying unwell children. An EWS, because it is being used in an already unwell child, needs to identify relevant changes in parameters as well as the relevance of the individual parameter in its own right. A study looking at change in parameter against deterioration has not yet been formally performed.
Finally, there is the difficulty in defining serious illness in the context of a deteriorating child. For example, a positive culture of an invasive microbiological organism is a reliable outcome measure. Asthma, renal or heart disease may also result in serious illness but grading the extent of severity is more difficult. Admission to paediatric high-dependency or critical care area is often used in these cases as a surrogate measure of a child's illness. However, increased frequency of medical review, medications, fluids or other interventions may also be a marker of quality of care and safety.
Being able to predict which child will go on to develop illness of sufficient severity to be admitted to intensive care areas may enable treatment to be commenced earlier and therefore prevent admission and is the raison d’etre of EWS. However, developing such a model based on individual parameters is problematic across different disease processes and types of hospital (for example, if you do not have an on-site intensive care unit). The heterogeneity of previously published studies has limited the comparison of smaller versus larger hospitals. It would be reasonable to hypothesise that larger units may support response teams with greater skills, thus improving outcomes. However, this is also dependant on the action instigated by activation of the EWS being timely enough to prevent deterioration. The time point of intervention is rarely recorded but may significantly imbalance the reported performance of a score.23
Systematic review
The collation of available evidence on EWS has been difficult due to the very heterogenous nature of the research and the problems described above. A systematic review of the field was carried out in 2010 by Chapman et al.7 This body of work identified only 11 papers meeting the inclusion criteria: research describing the development, testing or use of an EWS or activation of a MET in children hospital's outside of a critical care setting. The study confirmed the diversity of criteria used and commented on the low rate of studies examining inter-rater reliability in using the scores (which was only performed in one study). It is known the examination of key clinical signs differs according to the sign itself31 and the experience of the examiner.32 Chapman et al concluded that, ‘The potential of Paediatric Alert Criteria (EWS and METs) to improve the care of hospitalised children by aiding earlier identification of those at risk of critical deterioration and thereby improved outcome has not, as yet, been demonstrated.’ An alternative reflection could be that no harm has been demonstrated by these studies and given the biological plausibility that EWS should identify at least some patients prior to collapse, their use should be recommended. A previous literature review on Paediatric Rapid Response systems highlighted this point. Although the authors felt the downside of unvalidated systems may be to increase admissions to intensive care, they believed there maybe a total reduction in actual bed days.33 Not included in the systematic review was a large multinational study looking at EWS which has been published recently.34 This study found that a beside Paediatric Early Warning Score (PEWS; which had previous been published) was able to distinguish ‘sick’ (those patients admitted unexpectedly to an intensive care unit) from ‘well’ inpatients with at least 1 h notice. This is the first study in which a score has been validated across sites (four university hospitals in this case). Although the study was prospective, the authors noted the data collected was routine observations and the staff involved were not aware of the Bedside PEWS system and its component items (heart rate, systolic blood pressure, capillary refill time, respiratory rate, respiratory effort, transcutaneous oxygen saturation and oxygen therapy). This meant that the data was not identical for the case and control patients. Interestingly in the 23 288 h studies, only 5.1% of patients had all seven items recorded.
The future of EWS
EWS are increasing in use, are recommended by national reports and appear to predict deterioration in a subset of patients. However, the difficulties in validation have made a robust evidence base difficult to demonstrate. For those groups currently using EWS, a reflection on the role it is playing and how effective it is maybe necessary. For those interested in introducing a system, it may be pragmatic to consider the following (table 8).
Considerations prior to introduction of an EWS
The NHS Institute for Innovation and Improvement has produced an information portal regarding EWS which provides information on implementing and assessing change (http://www.institute.nhs.uk/safer_care/paediatric_safer_care/).
It will be important for researchers to clarify the difference between illness identification systems at the ‘front door’ to healthcare services and those used on inpatients, although it may be possible, and in fact desirable, to combine systems to aid communication during the patient journey. A more fundamental question, given the difficulty in validating in different healthcare locations, and noting the potential educational benefits it provides, is whether the use of the system is more important than its outcome?
Given the financial pressures on healthcare services in the immediate future, the costs of implementing EWS must be defined. This however is difficult as little of the development, implementation and outcomes are fixed quantities. Education and training of staff, for example, occurs over time due to the fluid healthcare workforce. Maintaining engagement and uptake requires staff with named EWS responsibilities but this varies on the size of department. Savings, such as reduced stays on PICU, may be easier to classify but determining an absolute reduction in days will to some extent be connected to whether the unit is on or off site. Litigation costs vary hugely and although protecting against one child with long-term disability as a result of failed recognition of septicaemia is a persuasive emotional argument to commissioners, it is harder to judge a true financial burden. Calls for national initiatives to examine these issues including the need for collaboration between the major organisations involved with child health such as the Royal Colleges of Paediatrics and Child Health (RCPCH) and Care Quality Commission have already been made.35
Conclusions
A number of EWS are available, some published and some not. Strict methodological assessment would indicate there is a lot of work still to be done in proving their benefit. However, it is important, for the safety and health of children, that mechanisms exist to be able to recognise and respond to the potentially deteriorating child in any healthcare setting. National guidance suggests that these mechanisms must be established. There may be no one type of EWS that can effectively and efficiently do this given the range of different healthcare settings and patient acuities that exist. An ideal EWS may in fact be one that is locally derived but based on previous positive (and preferably validated research), which has engaged staff and delivers on designated outcomes.
References
Footnotes
Funding This report is independent research arising from a Doctoral Research Fellowship supported by the National Institute for Health Research. The views expressed in this publication are those of the author and not necessarily those of the NHS, the National Institute for Health Research or the Department of Health.
Competing interests Dr Roland is in receipt of an East Midlands HIEC grant to develop POPS (Paediatric Observation Priority Score) for three children's emergency departments.
Provenance and peer review Commissioned; externally peer reviewed.