Article Text


Evaluation of emergency medical services systems: a classification to assist in determination of indicators
  1. C MacFarlane1,
  2. C A Benn2
  1. 1Emergency Medical Services Training, Gauteng Provincial Government, South Africa, Johannesburg Hospital Trauma Unit, and Department of Surgery, University of the Witwatersrand, Johannesburg, South Africa
  2. 2Chris Hani Baragwanath Hospital, Johannesburg and Department of Surgery, University of the Witwatersrand, Johannesburg, South Africa
  1. Correspondence to:
 Dr C MacFarlane, 29 Athlone Road, Parkview, Johannesburg, 2193, South Africa; 


Emergency medical services (EMS) systems, and prehospital care are difficult to evaluate. Accordingly, the true efficacy and value of such systems are difficult to determine. The multitude of variations and combinations of involved factors makes standardisation and comparison difficult, and universal indicators are hard to develop. Various attempts have been made to determine valid indicators of effectiveness, but there has been little success. Prehospital care has been seen by some as a single entity. As a result, experience from well resourced first world trauma centres has been taken, by many, to be applicable to all prehospital situations. This article attempts to assist in the development of valid EMS indicators of performance and effectiveness by categorising prehospital scenarios into a classification reflecting the reality of their conditions of practice.

Statistics from

The search for indicators and parameters to assess the efficacy of prehospital care continues. This has always been a difficult area to monitor and evaluate because of the multitude of variables present. To tackle this, various means of assessment have been attempted. The most widely used are ambulance response times and on scene times, indeed, many emergency medical services (EMS) are evaluated mainly on these parameters. Treatment and transport protocols have been developed to reduce these times.1 A Canadian study found that in the case of severely injured patients, a total prehospital time over 60 minutes was associated with a statistically significant increased risk of dying.2 A model for evaluating prehospital EMS was developed in Arizona, again related to time intervals.3 The use of on site physicians for major injuries was assessed in Canada, and was again related to prehospital times, the conclusion being that such activities merely increased the risk of death, because of prolonged prehospital times.4 In 1994, EMS dataset development took place in the USA, resulting in an 81 item list.5 An assessment was used in the USA in 1995 to try to determine if, in patients with serious injury, there was a prehospital time threshold that, when exceeded, significantly increased mortality in patients transferred to a trauma centre, rather than the closest hospital. No threshold beyond which time patient transport to the closest hospital would have decreased mortality was identifiable, because no prehospital time less than 90 minutes exhibited a significant adverse effect on survival.6 Even the administration of on scene analgesia has been related to on scene times.7

A review of the “scanty science” of prehospital emergency care was undertaken in the USA in 1997. Of 5842 publications reviewed, only 54 were randomised controlled trials. Because there was little scientific support for EMS interventions and because monitoring of outcomes and adverse effects was so poor, a re-examination of the whole of EMS practice was advocated.8 EMS motorcycle response has been advocated to reduce response times in congested centres.9 To confuse matters further, a 1999 study in the USA determined that different agencies used different time points as the start and end points of response times,10 and it is possible that this is widespread. In 1999, the International Association of Firefighters bemoaned the fact that there were few validated indications of effectiveness and quality in the EMS systems, that those in existence had not been studied for use in evacuation and, as a result, there are no universally accepted methods of measurement.11 Work started in California, trying to develop EMS system evaluation indications, but a report in 200112 indicated serious data collection problems. Work is slowly progressing, but with difficulty.

To put things in perspective, endotracheal intubation in a head injured patient in a short transit urban area may indeed not be necessary, but can the same necessarily be said for the same patient during lengthy transit in a rural area?


Several papers have considered the value of prehospital EMS, with variable messages emanating from these studies. In many of these, particularly from North America, the value of prehospital Advanced Life Support (ALS) has been questioned. As an example, a paper from Canada, in 2000, failed to demonstrate benefit of on site ALS in trauma.13 In a classic paper in 1996, Demetriades et al showed that severe trauma patients transported by family and colleagues to a trauma unit did better than those in whom ALS responded to scene.14 The same group, in 2000, indicated that prehospital intubation of severe head injury patients did not improve survival.15 A paper from Canada, in 1995, similarly determined that paramedic prehospital presence had no benefit in myocardial infarction patients.16 Another Canadian paper, in 1998, found no benefit in the institution of prehospital Advanced Life Support.17 There are many other examples,18–20 mostly with regard to trauma, but also relating to other conditions, such as patients with altered levels of consciousness,21 all casting doubt on the value of prehospital ALS. Even the value of helicopter based response has been seriously questioned.22–24

The other side of the coin has been somewhat less vociferously expressed, but the case remains. For example, back in 1994, a Boston study concluded that ALS resuscitation resulted in more favourable outcomes in major trauma.25 An Australian study, in 1988, suggested improved 24 hour survival with prehospital resuscitation, in similar circumstances.26 A Canadian study, in 1994, concluded that “life threatening” patients had “improved” when prehospital ALS was present.27 A study from Taiwan, in 1996, suggested that ALS care “may” be valuable for severe trauma patients.28 A review from Germany, in 1997, contended that on scene treatment had lead to a significant reduction of early post-traumatic death. The authors acknowledged that on scene infusion treatment is not always necessary and can even aggravate bleeding, this being supported by a Canadian study,29 but valued intubation, ventilation, and chest drains.30 A literature review from the USA31 maintained that the predominant finding in recent research was that ALS demonstrated improved effectiveness over Basic Life Support (BLS), with certain disorders. This was supported by a paper from the USA, in the same year, which showed improved airway management with ALS prehospital presence.32 With regard to acute myocardial infarction, a recent US paper suggested decreased mortality and morbidity with appropriate EMS prehospital care.33 Paramedic skills were applauded in a 2002 USA paper, by demonstrating low occurrence of unrecognised oesophageal placements in paediatric patients dealt with by paramedics in the prehospital environment.34

The value of prehospital thrombolysis for acute myocardial infarctions was initially controversial, but has been proved to be both safe and effective, because of shortened needle time from incident to thrombolysis, as a result of prehospital EMS thrombolytic intervention. The complication rate has also been low. It is interesting to note that prehospital EMS thrombolysis has been mainly used, to great effect, in Europe.35–38

There has been some support in Canada,39,40 but considerable reluctance in the USA. There has been some tentative support,41,42 but, even as recently as 1999, the world famous Maryland Emergency Medical Service stated that there were currently no data in the USA supporting the prehospital administration of thrombolytics.43 It has been recognised that rapid diagnosis, reduced transit time and EMS intervention has improved outcomes for acute myocardial infarction.44,45


It has been pointed out that there are two main prehospital emergency care systems that have dominated the world, the Anglo-American system, which uses predominantly emergency medical technicians, and the Franco-German model, which is physician driven.46 Comparisons between the two are difficult. However, other systems are developing, particularly in the developing countries, and these are often resource constrained. Marked differences between systems exist in Western Europe47 and new services are developing in Eastern Europe. In Latin America, circumstances commonly mandate a prehospital transport time unacceptable in the USA.48 In Taiwan, the prevailing circumstances have defined a need for a different system from the USA in the rural areas.49 Similarly, study of prehospital emergency care requirements in Malaysia has indicated a need to consider alternatives to the North American model.50 A comparative study of systems in Ghana, Mexico, and USA concluded that most deaths occur in the prehospital setting and that improvements in EMS and emergency departments are vital in low and middle income countries.51 A US study of developing countries advocated the use of community based services and available health providers to enable a developing system to function within a primary health network.52 In the Ukraine, the emergency physician trainee spends 18 months of his training programme in prehospital care at an EMS base station.53 In Madagascar, rural EMS is comparatively under-resourced, but population shift from rural to urban areas has an impact on decisions in EMS development.55 Prolonged response time to scene and transport time to hospital are the norm in Iran.56

As can be seen, there are enormous differences in prehospital emergency care in different countries, both developed and developing. Resources, education, planning, control, geography, population density, distances, and cultural diversity all need to be taken into consideration when planning, assessing, and monitoring prehospital care.


It has not been possible to assess, adequately, the efficacy of prehospital care, due to the difficulty of developing appropriate indicators. Some work has been done in this area and some is ongoing. Opinions are divided, and often based on anecdotal or emotional factors. Not infrequently prescriptive papers emanate from mainly North American urban teaching trauma centres. These are well resourced institutions, served by EMS able to attain short prehospital times. Such papers, because of the well deserved excellent reputations of the individuals and institutions concerned, tend to be considered “the law” by many. The principles they deduce are probably entirely appropriate for the circumstances in which they are fortunate enough to find themselves, but a very great deal of prehospital emergency care is not practised under such favourable circumstances and there must be caution in applying their recommendations universally and unreservedly.

There is a tendency to talk about “prehospital care” as if it were a universally uniform entity, and, therefore, one set of rules applies to all. This “one size fits all” philosophy is not necessarily appropriate.

In searching for rules and guidelines, appropriate indicators are necessary. This requires much greater audit, research, and analysis in all of the prehospital settings. Accurate data capture is vital. In designing indicators and guidelines it is suggested that the following be borne in mind;

  1. The geography, resources, medical and social culture of the country involved.

  2. The type of prehospital system in place, (physician led, paramedic led, etc)

  3. Prehospital EMS must not be considered in isolation. The EMS and emergency departments must be considered two limbs of one system. These must be integrated and coordinated, as the one cannot work effectively without the other.

  4. In making predictions regarding effectiveness, factors such as patient survival, morbidity, and subsequent hospital discharge, must be part of the assessment. Many EMS personnel see their goal as bringing patients alive to hospital and have little knowledge of what the patients’ progress was and how their activities could have influenced this.

  5. Much of the literature emanating from North American centres concerns itself mainly with trauma, particularly penetrating trauma. This is quite different in other countries, and must be taken into account. In medical, compared with trauma cases, a greater on scene time may be necessary for the best patient management.

In considering “prehospital care”, particularly from an EMS point of view, it may be useful to consider services as belonging to one of the following categories;

  1. Services in which there is a comparatively short prehospital time and the receiving institution is a sophisticated trauma and emergency centre. This is the situation in many Western urban centres.

  2. Services in which there is a comparatively long prehospital time to a sophisticated centre. In this situation, rural services may have to make long journeys.

  3. Services in which there is a comparatively short prehospital time, but to a small unsophisticated receiving centre. This may occur in rural areas.

  4. Services in which there is a comparatively long prehospital time to a small unsophisticated receiving centre. ALS prehospital staff may have a role in augmenting emergency department resuscitation teams under these circumstances.

These four scenarios have vastly different considerations and implications with regard to EMS activity, practice, and supervision. They could form the basis for a classification appropriate for the more accurate and realistic assessment of EMS systems. It would assist in the development of more appropriate indicators, relevant to the different types of services. Aeromedical evacuation may be lifesaving in some situations, bringing ALS to distant areas where there is none, and dramatically shortening prehospital times. In the urban environment a similar response may be an expensive waste of resources.

Communications, telemetry, online medical direction play an important part also.

A paper from Emergency Medical Services (2001) states “we are reminded not to approach people and places with belligerence, implying that we have all the answers”.57


Adequate means of assessment of prehospital care has not yet evolved, the complexity and number of variables making indicator development difficult.

Prehospital care embraces a variety of scenarios, and prescriptive publications from sophisticated urban trauma and emergency centres may not be appropriate for all of them.

It is vital, now, for this matter to be grasped seriously and as scientifically as possible. Standardised data capture must be embarked upon, in the search for the elusive indicators that are desperately sought after. In the search for such means of assessment of activity, a classification of prehospital services has been suggested, along with other factors relevant to this area. It is hoped this will assist in the assessment and utilisation of services and the development of protocols relevant to the various categories.



Dr C MacFarlane is guarantor for the paper. Dr C MacFarlane and Dr CA Benn jointly developed the paper, which was written by Dr C MacFarlane.


View Abstract


  • Funding: none.

  • Conflicts of interest: none.

Request permissions

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.