Study objective Previous studies on the construct validity of the Emergency Severity Index (ESI) were focused on outcome measures which could not be obtained directly at triage. A study was conducted to the construct validity of the ESI by measuring the association between the ESI triage categories and patients' vital signs at triage.
Methods A prospective observational study was conducted at an emergency department (ED) in the Netherlands. All patients who entered the ED between 20 July 2009 and 21 August 2009 were eligible for inclusion in the study. Patients' vital signs, triage category, age, gender, referrer and main complaint were registered. Vital signs were scored according to the Worthing Physiological Scoring System (WPSS) and the numerical pain rating scale. The data were analysed using ordinal logistic regression analyses.
Results An association was found between ESI triage categories and patients' vital signs at triage. Patients in WPSS categories ‘urgent’ and ‘alert’ were more likely triaged into the urgent triage categories (ESI triage categories 1 and 2) than patients with normal WPSS scores. However, no associations were found between pain scores and ESI triage categories.
Conclusion This study supports the validity of the ESI as it showed that patients' vital signs are associated with the ESI triage categories. However, a revision of the ESI guidelines concerning pain assessments is necessary.
- reproducibility of results
- health status indicators
- emergency care systems
- emergency care systems
- emergency departments
Statistics from Altmetric.com
- reproducibility of results
- health status indicators
- emergency care systems
- emergency care systems
- emergency departments
To sort the increasing number of patients presenting to emergency departments (EDs) on the urgency of their complaints, several triage systems have been developed and implemented.1–8 Frequently mentioned triage systems in the literature are: the Australasian Triage Scale, the Manchester Triage System, the Canadian Triage and Acuity Scale and the Emergency Severity Index (ESI).7 9 10
Compared to other triage systems, the ESI is different in that, as well as the level of urgency, it estimates the number of resources that patients need. ESI resources are defined as laboratory tests, radiology, intravenous fluids, specialty consultation, a simple or complex procedure and intravenous, intramuscular or nebulised medications. Patients can be allocated into five urgency categories. ESI categories 1 and 2 represent patients who need immediate life saving interventions (eg, defibrillation), or patients at an increased risk for deterioration (eg, patients with severe pain). When ESI categories 1 or 2 criteria are not met, the triage nurse estimates the number of ESI resources that patients need. In case more than one resource is needed, or when vital signs are in a predefined danger zone, patients are triaged into ESI category 3. Moreover, the triage nurse can decide to triage a patient in ESI category 2 on the basis of disturbed vital signs, even though initially the criteria for ESI category 2 were not met. Patients are triaged in ESI categories 4 and 5 when one or no resources are required respectively.11
For reasons of patient safety, it is important that ED triage systems are reliable and valid. Previous studies of the reliability of the ESI reported κ scores12 representing moderate to almost perfect reliability.3 4 8 10 13–16 The reliability places an upper limit on the validity of triage systems, that is, a triage system which produces different urgency scores when used in the same patient cannot be valid.17 The validity of the ESI has mainly been studied by means of construct validity because a gold standard to measure criterion validity is absent. The construct validity has been studied by measuring associations between the ESI triage categories and factors related to urgency. Previous studies have reported such associations of the ESI with actual ED resource usage, ED and hospital length of stay, hospital admission, mortality, survival after an ED visit and physiological measurements.1 2 4–8 10 14–16 18 19 However, a limitation of these studies is the focus on outcomes of care because of the time lag between the moment of triage and the reported outcomes. As a result, other factors could have influenced the associations since the patient's condition could have been changed between arrival in, and departure from the ED. To reduce this bias in triage validity studies, one could focus on measures that can be obtained directly at triage. Furthermore, because of the focus on construct validity in the validation of ED triage systems, it is important to keep studying different constructs, as construct validity is a process of making and testing inferences.17 Although vital signs play an important role in the ESI, no studies have assessed if vital signs are associated with urgency categories. Therefore, a study was conducted to the construct validity of the ESI by measuring the associations between the ESI triage categories and patients' vital signs at triage. We hypothesised that patients triaged in the urgent triage categories of the ESI were more likely to have disturbed vital signs, and therefore at increased risk for dying in the ED, than patients triaged in less urgent triage categories.
The study was conducted in a prospective observational design. The protocol was reviewed and approved by the medical ethics committee of the University Medical Center Utrecht. Informed consent was obtained from patients before inclusion in the study.
Study setting and population
The study was conducted at the ED of the University Medical Center Utrecht, which is a designated level one trauma centre in the Netherlands. Severely injured patients in the designated trauma region are transported to this ED. The ED is staffed 24 h a day by six full time attending physicians and has an annual census of 21 000 patients. The ESI was implemented in 2008 and nurses received a 1-day training course on how to triage with the ESI before implementation. All patients over 16 years of age who entered the ED of the University Medical Center Utrecht on weekdays between 20 July 2009 and 21 August 2009 were included in the study.
The data were prospectively collected by one researcher (HR) from Monday to Friday, 09:00 to 17:00 (4 days a week) and from 12:00 to 20:00 (1 day a week). A sample size calculation for regression analysis20 estimated a minimum required sample size of 445 patients. A drop out rate of 10% due to unforeseen circumstances was taken into account in this calculation. Each patient was triaged by the triage nurse on duty and assigned a triage category using standard procedures. Immediately after the triage nurse finished the triage assessment and reported the ESI triage category, the researcher registered patients' gender, referrer, main complaint, age and measured vital signs that were not measured by the triage nurse. The following vital signs were registered: blood pressure, pulse rate, respiratory rate, oxygen saturation, temperature, the Alert, Voice, Pain and Unconsciousness score, and pain. They were measured using an automated vital signs monitor, a tympanic thermometer and the numerical pain rating scale. The numerical pain rating scale was scored by asking patients to allocate a score between 0 and 10, with 0 indicating no pain and 10 the worst pain imaginable. The researcher was trained in the use of these instruments by a triage nurse during a 1 day training in the ED prior to data collection. In case a patient needed to be seen by a doctor immediately, the patient's data were collected by the triage nurse. All the data were registered on a form.
Interpretation of vital signs
The ESI guidelines state that the interpretation of vital signs for allocating a patient to ESI categories 2 or 3 is up to the triage nurse, for example, a patient with disturbed vital signs does not automatically meet ESI level 2 criteria.11 For example, a patient who has a history of COPD and presents with an oxygen saturation of 89% might not meet ESI level 2 criteria. However, another patient with the same complaints but no such history does require ESI level 2 criteria when presenting with such an oxygen saturation level. Interpreting vital signs separately is not useful in this study as disturbed vital signs are not necessarily related to higher urgency or life threatening situations. Therefore, a prognostic scoring system in which vital signs were interpreted in relation to short term mortality was applied in this study. The Worthing Physiological Scoring System (WPSS) is such a prognostic scoring system (table 1). The WPSS is based upon identifying physiological markers for mortality at an early stage to undertake timely action. The system has been derived from and prospectively validated in ED patients and is therefore suitable for use in this study.21 Except for pain, the system consists of the vital signs used in the ESI as well as systolic blood pressure.
The data were analysed by means of ordinal logistic regression analyses.22 Ordinal logistic regression analysis is an extension of binary logistic regression analysis and used in case the dependent variable is ordinal. The effects of the independent variables are interpreted by assuming that they are constant over the categories of the dependent variable, that is the assumption of parallel lines. This is an important assumption and was checked in the analysis. Because of this assumption the analyses were performed using the complementary log-log link function. This link function is used when the higher categories of the dependent variable (eg, the lower urgency categories) are more common. As a result, the associations should be interpreted as rate ratios instead of ORs.23
In these analyses, the ESI category was the dependent variable. ESI categories 4 and 5 were merged because of the small number of patients triaged in ESI category 5 and used as a reference category (n=237). Besides WPSS and pain scores, other independent variables were gender, referrer, main complaint and age. The latter four variables were used in the analyses because in a previous study these appeared to be related to the ESI triage categories.24 The variable referrer consisted of patients referred by ambulance, a specialist, a general practitioner or patients who referred themselves to the ED. The variable main complaint was coded in accordance with the chapters of the International Classification of Diseases 10th edition (ICD 10), immediately after triage.25 It was decided to code main complaints this way to decrease the variability in ICD 10 codes, which was needed for the analyses. Univariate ordinal logistic regression analyses were performed and all variables which significantly (p≤0.05) predicted urgency were selected for multivariate analysis. All analyses were performed using SPSS for Mac V.17.0.
In total, 929 patients presented at the ED during the study hours. Of these, 584 patients (62.9%) consented for participation and were included in the study. Of these, 40 patients had missing data (WPSS score (6.0%), pain score (1.0%) or age (0.3%)) and were excluded from the study, leaving 544 patients for further analyses. No significant differences (p=0.53) were found between included and excluded patients, based on age and gender. However, there was a significant difference (p=0.045) in ESI triage category. Patients in ESI categories 1 and 2 more often had incomplete data compared to patients in other triage categories. Table 2 presents the characteristics of the patients included in the study. Compared to the characteristics of patient presenting to this ED in April 2009, patients in the present study were significantly older (+11.79 years on average; p<0.01) and more often referred by a general practitioner (p<0.01). Furthermore, slightly more patients were triaged in ESI categories 2 (+3.1%) and 3 (+6.4%) but these differences were not statistically significant (p=0.63). Also no differences were found on gender (p=0.33).
In univariate ordinal logistic regression analyses, WPSS score, referrer, main complaint and age were significant (p≤0.05) predictors for ESI triage category. The results of the multivariate ordinal logistic regression analysis are presented in table 3. In this analysis the variable ‘main complaint’ was merged into two categories because of the assumption of parallel lines. Patients with WPSS scores ‘urgent’ and ‘alert’ were more likely triaged into the more urgent triage categories of the ESI compared to patients with a normal WPSS score. Remarkably, no associations were found between urgency categories and patients' pain scores (p>0.05), while pain is a discriminator of the ESI.
To our knowledge, this is the first study that measured the construct validity of the ESI by using vital signs, which were directly obtained at triage. Strong associations were found between the ESI triage categories and WPSS scores. A perfect association was not to be expected since the interpretation of vital signs was used as a construct for urgency. These results are comparable to the findings of Kim et al16 who studied associations between ESI triage categories and APACHE II scores. The highest mean APACHE II scores were found in the highest triage categories.
A remarkable finding of this study is that pain, a discriminator of the system (ESI category 2), was not associated with urgency. The ESI guidelines state that it is up to the discretion of the triage nurse to indicate whether a patient's pain score is supported by his/her clinical condition and warrants triage in higher ESI categories. However, the guidelines have not specified how to make this decision. This could be an explanation for the lack of association between pain scores and urgency. To increase this association, and consequently the validity of the ESI, a revision of the guidelines is necessary. More specific, the guidelines should describe symptoms or patient behaviour related to severe pain. Further study will be needed to measure the effects of such a revision on the reliability and validity of the ESI.
The results of this study should be interpreted carefully because of several limitations. First, the study was conducted at one ED during day and evening shifts. Therefore in the analyses we could not account for hospital influences or influences caused by the collection of data in shifts. Also, 37.1% of the total number of patients who presented to the ED during the study period were not included in the study because no consent was obtained. As a result, patients were significantly older and more often referred by their general practitioner. This could have influenced the results of the study in that patients referred by the ambulance service were more often triaged in the more urgent ESI triage categories. It is therefore possible that the reported associations are slightly underestimated.
Second, the variable ‘main complaint’ was merged into two categories which negatively affected the proportion of variance explained by the regression model. However, we were now able to conduct an ordinal logistic regression analysis which is preferred over the alternative, the multinomial logistic regression analysis. The latter would have ignored the ordinal nature of the dependent variable which therefore would have biased the effects of the associations under study. Finally, the nurses were aware of the study objective and the main researcher was present in the triage area to collect data. Blinding in this study was not a possibility as it would have resulted in other, possibly more serious, biases. To avoid biases due to the awareness of nurses in the study, data could have been abstracted retrospectively. However, this probably results in more missing data especially in the urgent triage categories. Moreover, biases could occur due to the variability in registrations. Finally, in retrospective designs there is no control over the data. Therefore, a prospective design was preferred. As a result, nurses were possibly more likely to account for vital signs earlier. This could have influenced their triage decisions and consequently have overestimated the reported associations in this study. To reduce the impact of this flaw the researcher who was present in the ED did not interrupt the triage conversation and was not involved in making a triage decision. Furthermore, after the nurse finished the triage conversation and registered his or her finding, the researcher measured additional vital signs and registered other information required for the study.
In conclusion, the findings of this study support the validity of the ESI as it showed that patients' vital signs are associated with the ESI triage categories. There were strong associations between the ESI triage categories and patients' WPSS scores at triage. However, no associations were found between pain scores and ESI triage categories, which indicates that the ESI guidelines need to be revised.
Competing interests None.
Ethics approval This study was conducted with the approval of the medical ethics committee of the University Medical Center Utrecht.
Provenance and peer review Not commissioned; externally peer reviewed.
If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.