Limitations of current measures

In high-income nations, the accepted and most commonly used tool for analysing emergency care systems is the Trauma Injury Severity Score (TRISS).7 TRISS is a validated score that can be retrospectively used to measure the effectiveness of trauma care.8 ,9 It calculates a patient's probability of survival through the equation: Ps=1/(1+e^-b), where Ps is the probability of survival and b=b0+b1(RTS)+b2(ISS)+b3(A).8 RTS, the Revised Trauma Score, is calculated by the summation of fixed codes, based on physiological parameters, multiplied by previously published weight coefficients. ISS, the Injury Severity Score, is calculated by assigning injuries an Abbreviated Injury Scale code of 1–6, depending on the type and location of the injuries, and summing the squares of the highest codes of the three highest scoring anatomical regions.8 ‘A’ represents the patient's age code (0 if ≤54 years, and 1 if >54), and b0, b1, b2 and b3 are prepublished coefficients based on regression analysis of thousands of patients from high-income nations, with a different set of coefficients for blunt and piercing injuries.8 Based on a population's calculated probabilities of survival, researchers can compare an emergency care system's expected outcomes with its actual outcomes to see if the system or an intervention is increasing or decreasing the patients' chances of survival and by how much.

However, although TRISS and its components are popular ways of measuring the success of emergency care in high-income nations, they present limitations that prevent developing countries from implementing their use.10 ,11 TRISS and the scores it is based on involve complex algorithms that rely on a plethora of variables and codes, many of which require extensive training to assign.12 For hospitals and healthcare systems in developing countries, many of these codes and variables are extremely difficult to reliably collect, due to the typically poor record-keeping infrastructure of developing countries.6 ,13 ,14 In such areas, healthcare workers often have limited training and supervision in data collection and management, which leads to frequently incomplete records and makes the implementation of complex measurements that rely on a high number of variables difficult.14 Even if healthcare workers consistently collected the necessary variables, healthcare facilities often lack the technology or administrative support to organise a high volume of patient records in a way that all records are traceable and can quickly be collected6 ,13; this greatly multiplies any extra time and effort required to do such research. Furthermore, healthcare facilities in developing countries are typically overwhelmed,6 making the chances for healthcare workers to take the time to record and calculate complex measurements more unlikely. In addition, since the calculation of TRISS and its components involve assessments from various personnel, such complicated measurements require good cooperation and accuracy of multiple medical parties,15 which developing healthcare systems often lack.

Even if developing countries had the infrastructure to collect the data required for such scores, their implementation would still be inappropriate. TRISS and the scores it is based on rely on coefficients based on logistic regressions performed on populations from high-income nations. Between high-income nations, population demographics and emergency care may be similar enough to allow the use of such methodology. However, populations and emergency care can be dramatically different in developing countries, rendering the coefficients invalid.10 ,11 Additionally, determining a separate set of coefficients for developing countries would be moot, as such countries do not have the resources to perform such endeavours.13

How these limitations can be overcome

Because most developing countries are unable to determine the effectiveness of emergency care systems within their own domains, it is common for clinicians and health officials from these areas to adopt the policies of high-income nations without testing them at home.16 Many health workers assume that if a high-income nation uses a certain strategy then it must be sufficient. This approach can be counterproductive, as Western models are often too expensive for developing countries to handle and may be ineffective in the conditions that exist in the developing world.16 To curb this, a methodology to assess the efficacy of emergency care interventions in developing countries must be established so that developing countries can determine what works best in their own respective areas and quickly improve their emergency care systems in response.

To develop an effective emergency care assessment for developing countries, the measurement must be, first and foremost, usable in a low-resource setting with severely limited infrastructure. This means that the assessment can only rely on variables that clinicians can reasonably collect in a developing country. In addition, such an assessment must employ only the most basic calculations and protocol. Since medical record keeping, research and coordination in developing countries can be limited,6 ,13 ,14 these low-resource areas cannot handle overly complex calculations or procedures. In effect, an emergency care assessment for developing countries will require complexity, coordination, resource-use and effort to be kept to an absolute minimum. This is especially critical because if these criteria are not met, clinicians and health officials in the developing world will have no other choice but to resort to the convenience of adopting high-income nations' strategies that have not been verified for the developing countries.

A second necessity for an emergency care assessment in developing countries is that it must be applicable to a variety of developing contexts. As it is already difficult for one low-resource nation to develop a proper assessment, expecting each developing country to develop its own personalised assessment is implausible. Therefore, an effective measurement must rely on objective parameters universal to any location, rather than depend on population-specific coefficients or other variables that can change between the diverse settings of developing countries.

TEWS: a suitable system measure?

The Triage Early Warning Score (TEWS) is a validated composite triage score and is a component of the larger South African Triage Scale (SATS).17–19 It is based on the Modified Early Warning Score (MEWS), a validated score that triages acutely ill emergency patients.20 The MEWS was adapted to include mobility, trauma and AVPU components. AVPU reliably assesses the central nervous system as to whether a patient is alert (‘A’), only responding to voice (‘V’), only responding to pain (‘P’), unresponsive (‘U’) or confused. The adapted MEWS was then validated against the need for hospital admission of the South African population with its unique burden of disease and renamed the TEWS.12 ,17–19

To calculate TEWS for an individual emergency case, each vital sign is assigned a subscore from 0 to 3 according to established parameters (see figure 1). Afterwards, the subscores are added together to yield a total score; a higher total score indicates more physiological derangement and is used as a proxy for more severe illness or injury. The TEWS is very user friendly,12 can be taught quickly to inexperienced staff19 and uses simple clinical parameters, making it useful at all levels of emergency service delivery in a developing setting.21 Unlike TRISS, TEWS can be much more readily handled in developing countries and is therefore a potential candidate to assist such countries in measuring the success of an emergency care system or intervention.

• Download figure
• Open in new tab
• Download powerpoint

Figure 1

The Triage Early Warning Score. RR, respiratory rate; HR, heart rate; SBP, systolic blood pressure; AVPU = alert, voice, pain, unresponsive. Reprinted with permission. Infant and paediatric versions of the scorecard, along with any updated versions (if developed), can be found at http://emssa.org.za/sats.

Although TEWS was originally designed as a triage score, we propose using it as part of a TEWS methodology to assess emergency care in developing countries. In the TEWS methodology system that we propose, assessing both emergency centre (EC) and prehospital care (PHC) effectiveness focuses on the use of the TEWS taken upon the patient's arrival to the EC, henceforth referred to as intake-TEWS.

Using TEWS to assess EC effectiveness

To assess the effectiveness of an EC, the outcomes of patients (discharged, admitted to ward, deceased) are compared with a parameter of intake-TEWS. For example, the outcome statistics of all patients with an intake-TEWS of 6–7 can be compared with the outcome statistics of patients with the same intake-TEWS at a different EC, or at the same EC but at different points in time (see table 1 for an illustration).

If one EC's outcome statistics for a TEWS parameter is better (eg, higher proportion of patients are discharged and a lower proportion dies), then that EC is more effective. If the outcome statistics for a single EC gets better over time, then the EC is becoming more effective.

Using TEWS to assess PHC effectiveness

To use TEWS to assess the impact of a PHC system or intervention, the intake-TEWS of all patients affected by a system or intervention are averaged and plotted over time. If a PHC system or intervention has a positive effect on a population, the affected patients will be physiologically more stable upon arrival at the hospital, which results in a lower average intake-TEWS. To eliminate confounding factors that could also reduce TEWS, the average TEWS over time for the target population can be compared with a control population (see figure 2 for an illustration).

• Download figure
• Open in new tab
• Download powerpoint

Figure 2

Illustrative average intake-TEWS between two populations. In this example, the PHC system or intervention has a positive effect on burn patients from the intervention population, compared with those of the control population without the PHC system or intervention. PHC, prehospital care; TEWS, Triage Early Warning Score.

In addition to the average intake-TEWS, information on prehospital mortality or the outcome statistics (eg, expedited alive to EC, deceased en route) based on a parameter of TEWS at the time of injury (much like the assessment procedure for EC effectiveness) can further justify or validate findings. However, many developing areas in need of the TEWS methodology will not have reliable access to such data.

TEWS should decrease with more effective PHC

Although the general consensus in the emergency medical community is that effective care stabilises vital signs, or at least alleviates their deterioration, some studies still argue whether PHC has a significant effect on patient physiology.22 However, almost all of these studies were done in high-income nations that can be drastically different from developing ones. In developing countries, which are typically more rural and have much longer prehospital transportation times, the effect of PHC can be significantly different.23

Previous studies have confirmed that PHC can stabilise patients' vital signs in developing countries. After implementing PHC systems in northern Iraq and Cambodia, two research teams tracked their personnel's effects on patient physiology using the vital signs based Physiologic Severity Score (PSS).24 Both teams found that as the PHC systems matured and improved, as indicated by reduced prehospital mortality, quicker transportation times and anecdotal evidence, the average improvement from the initial on-scene PSS to the intake-PSS at the hospital became greater, indicating that more effective PHC in developing areas correlates with an increasingly positive effect on vital signs.4 ,5

This evidence suggests that improvements in PHC will correlate with a reduction in average intake-TEWS, as both TEWS and PSS rely on vital signs.19 ,24 However, confirmation of this result is still needed.

Benefits of the TEWS methodology

The TEWS methodology is a very simple protocol that can be performed using very basic data that are not region specific and can be reasonably collected in a developing country. This allows researchers to easily measure the efficacy of PHC, an EC or emergency system intervention in any developing country. Additionally, the TEWS methodology requires minimal calculation and resources to perform, and generates more mileage out of a single intake-TEWS data set that can be collected in one project. This allows healthcare workers from developing countries to monitor their own emergency care systems, as it requires minimal training and does not require excessive resources or attention, and helps them to keep track of multiple components of the emergency care system simultaneously using just one set of intake-TEWS data.

There is also the benefit in the TEWS being a part of the larger SATS. As SATS is a validated triage scale designed for usability in developing countries,19 its adoption by developing hospitals and emergency care systems is convenient and may be rapid in the future. SATS is already being implemented in South Africa,12 Médecins Sans Frontières—Operational Centre Brussels (Sebastian Spencer 2011), Botswana, Malawi, Brazil, Poland and New Zealand, and is in the planning stages for implementation in Sudan, Ghana, Rwanda, Tanzania, Saudi Arabia and for an adapted version with TEWS in Sweden (South African Triage Group 2011). Any region that adopts SATS will already be collecting and calculating the data for the TEWS methodology.

Additionally, the TEWS methodology can be versatile for locations that have not adopted SATS and do not have the infrastructure to collect all of the basic vital signs expected for TEWS. Under these circumstances, researchers and local health workers can still monitor their emergency care systems using a modified TEWS that is the composite of TEWS subscores based on the vital signs that are available. However, this alternative is not optimal, as the ability of the TEWS to predict the severity of an emergency is greater with the inclusion of more vital signs,17 and because ECs, PHC systems and interventions that are monitored using a modified TEWS can only be compared with other ECs, PHC systems and interventions that are monitored using the same modified TEWS. Though, it is understandable that these alternative methods may be the only option in certain developing areas.

Finally, although the TEWS methodology is purposefully simple, simplicity does not imply invalidity. The TEWS methodology is simple and non-cumbersome out of necessity, and even complex measurement scores and calculations can have their complications.25

There is a clear research gap in emergency care for developing areas; we propose the TEWS methodology as a potential solution. By building a methodology around TEWS, we sought to find a simple, non-cumbersome measurement that a single hospital's EC in a developing country can easily use to objectively analyse the effectiveness of an emergency care system or intervention. This factor could become essential in the development of new emergency care systems worldwide.

Limitations of the TEWS methodology

As with any analytical method, even ones used in high-income nations, the TEWS methodology has some limitations. The TEWS methodology does not take into account patients who never reach the hospital. This is because many developing areas will not have access to this information, but if it is available then such data can be presented along with the results from the TEWS methodology.

Also, the TEWS methodology shows promise and evidence of reliability, but has yet to be fully validated. Obstacles to complete validation involve the lack of robust data in areas where this would be implemented.13 However, if these obstacles can be overcome, one way to validate the TEWS methodology for assessing PHC would be to verify that more effective PHC in developing areas does have a more positive effect on TEWS scores. This would involve the observation of patients' TEWS decreasing (or not increasing as much) after being treated by PHC providers, such as first responders or paramedics. Also, fluctuations in average TEWS improvements over time can be compared with concurrent fluctuations in other measurements of PHC, such as transport times and mortality rates. As these effects have already been shown for the similar PSS,4 ,5 finding them for TEWS is more likely. As for assessing EC efficacy, validation is not needed since TEWS is used in the methodology as an input variable that categorises patients according to emergency severity, and TEWS has already been validated as a triage score.17–19

Finally, as an example we have only included the adult version of TEWS (figure 1); paediatric and infant versions, along with any updated versions (if developed), are available (http://emssa.org.za/sats) and should be used to assess the efficacy of PHC or ECs for these populations.