Article Text


Delphi type methodology to develop consensus on the future design of EMS systems in the United Kingdom
  1. T B Hassan1,
  2. D B Barnett2
  1. 1Accident and Emergency Department, Leeds General Infirmary, Leeds, UK
  2. 2Clinical Pharmacology and Therapeutics, Leicester Royal Infirmary, Leicester, UK
  1. Correspondence to:
 Dr T B Hassan, Accident and Emergency Department, Leeds General Infirmary, Great George Street, Leeds LS1 3EX, UK;


Objective: To develop consensus opinion on future design characteristics of Emergency Medical Services (EMS) systems in the UK with particular regard to advanced life support skills (ALS).

Design: A Delphi questionnaire design with two rounds to gain a consensus of opinion. Investigation of four aspects of EMS design is reported—type of response to a priority based dispatch category, transportation options, enhancement of paramedic skills, and structure of a first responder system.

Subjects: Chief executives, directors of operations, and medical directors of Ambulance Trusts in the United Kingdom.

Outcome measures: Likert scales (0–9) to score opinion on a series of statements with achievement of inter-round consistency. A median score of 0–4 was classified as disagreement and 6–9 as agreement.

Results: A 65% response to the first questionnaire and with iteration, 52% response to the second questionnaire was attained. A tiered response (paramedics, technicians, and basic life support first responders) with technicians responding to selected category A and B calls and all category C calls (median score (MS) 7.5, interquartile range (IQR) 4), was recommended. Inter-unit handover of selected calls to maintain paramedic availablity ( MS 7.5, IQR 3.75) and enhancement of paramedic skills (MS 7.0, IQR 4.0) was also proposed. Finally, the development of a first responder system fully integrated into the EMS (MS 8.0, IQR 2.75) involving other agencies including the police force, fire service, and trained members of the local community was agreed.

Conclusions: Senior expert staff from Ambulance Trusts in the UK achieved consensus on certain design characteristics of EMS systems. These are significantly different from the present EMS model.

Statistics from

Despite increasing sophistication in the care given by prehospital care providers in the UK, Emergency Medical Service (EMS) systems continue to under perform. Response times, quality assurance, and cost effectiveness related to desired outcome are below an acceptable standard (NHS Exec, Audit Commission). The design of a system is considered critical to each of these markers of performance.1

Unfortunately, there are no validated tools for the overall evaluation of EMS systems. Rather, performance and outcome for selected conditions are used as indicative markers. Measurement of the outcome from prehospital cardiac arrest (CA) is regarded as a robust tool or “tracer” for the evaluation of a system's performance and overall quality.2 A tracer is a condition that is important, measurable, has a relatively high frequency and great potential to be affected by medical care.3 Major trauma is another commonly used tracer in EMS systems.4

Critical to success for both of these conditions is prioritisation of emergency calls, dispatch of an appropriate unit or units, minimisation of response time, provision of appropriate treatment at the scene, and timely transfer to definitive care. Measurement and optimisation of the tracer is hopefully then indirectly reflected in similar high quality care being given for a number of other life threatening disorders.2 However, considerable debate continues as to how the outcome of prehospital CA, for example, can be improved in the most cost efficient manner.

A number of changes in the processes of care within a system might make it more efficient and effective. For example, the implementation of priority dispatch systems (PDS) should improve response times, especially for life threatening conditions.5 This has yet to be proved. To date, the ability to respond rapidly has been compromised by the continuing annual rise in the number of emergency calls.6

Another component in the process of care is the type of response to send to an emergency call. The single tiered Advanced Life Support (ALS) system currently in place in the UK offers simplicity and safety because every emergency call receives the benefit of a paramedic attendance. This prevents the likelihood of under-triage by the PDS. However, a number of UK studies have shown that apart from defibrillation, which can be provided by technicians with automated external defibrillators (AEDs), there is no beneficial effect on outcome using other ALS skills in prehospital CA.7, 8 In addition, analysis of the workload of EMS staff has shown that less than one tenth of emergency calls require paramedic skills.9

In the United States (US) paramedic training is on average four to five times longer. The single tiered ALS configuration in some systems it has been argued, is costly and uses highly trained personnel unnecessarily.10 Others have disagreed. Multi-tiered systems with mature PDS are perceived to use resources more efficiently. In contrast, in the UK, further redesign to the single tiered ALS response has been thought necessary. Recommendations have included a maintenance of the single tier, with expansion of the number of single paramedic “fast responders”, and development of first responder schemes.5 There are no studies to support these changes in system design or evidence to suggest that they are representative of the views of experts in the UK.

The main objective of this study was to identify what a group of UK experts in the field of EMS believed to be the optimal design of systems for the future. An emphasis was made on ALS skills. They were asked to take into account, resource constraints, the continuing rise in demand for EMS, and the need to improve the outcome of certain tracer conditions such as CA.


Study design

To gain maximal input and a consensus of opinion among a group of experts in EMS systems, a Delphi type study design was used. Because of the nature of the study, ethical approval for the study was not considered to be necessary.

Delphi methodology

The Delphi method is a means to determine the extent to which consensus exists among a group of people.11 Consensus takes place in a series of “rounds”. The first round entails obtaining the opinions of selected experts about a particular issue. In subsequent rounds the same experts are asked to rate the extent of their agreement or disagreement with a series of statements describing the opinions expressed in the first round. Responses are analysed for the degree of consensus achieved. Failure to reach consensus can result in further rounds.

To reduce the number of rounds and hence maximise the return rate of the questionnaires, in the first round, the topics most relevant to future EMS system design were derived from a review of the literature.12 A pilot study using only senior directors from the Leicestershire Ambulance and Paramedic Service (LAPS) was carried out. This same group were also interviewed individually to further refine the questionnaire. Appropriate changes were made to the questionnaire design, but the results were not included in the main study.


To gain expert consensus opinion and gauge the national view, all chief executives, directors of operations, and medical directors of Ambulance Trusts in the UK were chosen as “Delphi experts”. A total of 91 people were invited to enter into the study from the 42 existing Ambulance Trusts at the time.

Questionnaire design

The “Delphi experts” were each sent a questionnaire that was divided into two main sections. In addition they were sent a copy of a paper outlining the Delphi technique and methodology for their interest and to enhance the quality and rigour of the study.13 This questionnaire constituted the first round in the study. The first section asked respondents to answer questions that described the existing components of their EMS system. The second section aimed to gauge their opinion on how systems should be developed in the future. Respondents were asked to indicate their level of agreement with each statement on a scale from 0 to 9 ( 0 indicating total disagreement and 9 indicating total agreement).

Results were collated and a second questionnaire sent to those who had responded to the first. This second questionnaire was similar to the first but it also showed the initial responses of all the other participants. In addition, the person's rating for each of the previous round was shown. The purpose of this was to offer the respondents an opportunity to amend their ratings in view of the expressed attitude of their colleagues.

Definition of agreement/disagreement

Clear prior categorisation of median scores was developed to define disagreement (MS 0–4) and agreement (MS 6–9) with statements in the questionnaire.

Statistical analysis

Agreement between the first and second scores was measured by calculating the difference in opinion obtained after subtracting the second score from the first for each statement of each respondent, the median change of opinion score (MCOS). The MCOS, interquartile range (IQR), and the minimum and maximum differences (range) were calculated for the difference of opinion.

After good agreement between the first and second questionnaires, median scores (MS) and IQR for each of the statements to the second questionnaire alone were calculated.


After the first round, replies were received from 56 of the 91 directors and chief executives. Five respondents sent joint responses from their Trusts and one medical director replied but refused to enter into the study. A response rate of 56 of 86 (65%) was therefore achieved. This represented 37 of the 42 Ambulance Trusts in the UK in operation at the time.

Apart from the small systems serving populations of less than 200 000, the number of emergency calls per thousand population per available unit at peak times was reported to vary considerably from 0.23 to 3.0. Twenty one systems (57%) reported that they operated some form of additional paramedic “fast responder” service. Only six systems (16%) reported an integrated “first responder” scheme.

A second questionnaire was sent to the 56 who responded to the first one. From this second round 29 replies (52%) were received. Only minimal changes occurred between the two rounds of the Delphi study as shown by the mean change in outcome Score (table 1).

Table 1

Description of the change in opinion score from the first to the second questionnaire for each statement

The results of the second questionnaire are shown (table 2). A tiered response (paramedics, technicians, and basic life support first responders) with technicians responding to selected category A and B calls and all category C calls (MS 8.0, IQR 3.0) was proposed by the panel. The need for an automatic dual unit response for patients assumed to have a CA was also suggested (MS 7.0, IQR 3.5). Inter-unit handover of selected calls to maintain paramedic availablity (MS 8.0, IQR 2.0) and enhancement of paramedic skills was proposed for certain circumstances. The development of a first responder system fully integrated into the EMS involving other agencies including the police force, fire service, and trained members of the local community was agreed (MS 9.0 IQR 2.0).

Table 2

Final opinions of respondents to the second questionnaire


Principal findings

This study has shown that senior expert staff from Ambulance Trusts in the UK achieved consensus on certain aspects of EMS system design for the future. A tiered response, with paramedics and technicians being dispatched selectively to category A, B and C calls is recommended with a dual response for patients with assumed CA. In addition, they concluded that an active first responder scheme should be developed that is fully integrated into the EMS system. They also suggested that the composition of staff could include members of the police and fire services as well as trained members of the local community. Finally, the expansion of advanced skills for paramedics in certain circumstances was supported.

However, these proposals differ from the present single tiered all ALS model.14 In addition, they also vary from recommendations made by the NHS Executive to attain performance standards of eight minutes in 90% of calls for all immediately life threatening conditions.5

Evidence to support the consensus of opinion

This is the first study to investigate the potential design of EMS systems in the future using the Delphi technique. The need for structural change is obvious. There has been an ever increasing demand on EMS systems in the UK in the past decade. The number of emergency 999 calls having increased annually by between 4.8% and 9.4% since 1992.15 This has resulted in a need to be more effective, efficient and appropriate for the response provided to an emergency call.12, 16

PDS systems have been formally introduced into UK practice permitting prioritisation of calls and more appropriate care to be targeted to the needs of the patient.5 However, debate continues around the most appropriate and cost effective response type given that a substantial proportion of calls do not require paramedic ALS skills.17–19 There is evidence to suggest that a tiered service will increase flexibility within a system and produce better average response times. A review of the EMS systems of 25 mid-sized cities in the United States found that the average response time for two tiered systems was 5.9 minutes compared with 7.0 minutes for one tiered systems (p<0.05).17 In addition in a meta-analysis of 41 EMS systems, a rate of discharge for patients suffering pre-hospital CA was 5.2% in one tiered systems and 10.5% in two tiered systems.20 Evidence also suggests that systems able to provide a dual response for CA will have a much greater percentage discharge from hospital.21

The role of other agencies as first responders in certain groups of patients, especially with possible CA, is still very much in its infancy in the UK. In 1998, six fire services (11%) and four police forces (14%) were involved to some extent as “first responders” in responding to patients with presumed CA.22 This role was predominantly in rural areas but also included central London. In contrast, a survey of EMS systems in 200 American cities, the fire service provided cross trained/dual role personnel for emergency care in 67 systems (33%). Non-transport ALS support was also provided by fire department based EMS in another 35 cities (17%).23 An integrated first response with other emergency services has been shown to improve the outcome of patients suffering CA from 3% to 26%.24 This study has identified that only 16% of systems have some form of integrated “first responder” system.

Outcome of patients suffering prehospital CA is critically related to the defibrillation response interval (DRI). The DRI is classified as “call received by dispatch” to “arrival at scene by responder with defibrillator”. The aim of EMS systems in US thus far has been to achieve and maintain a target in eight minutes or less for at least 90% of cases.25 In the UK this target is regarded as a long term performance standard to be attained.5 However, evidence from a prospective cohort study in Canada of 9267 adult victims of prehospital CA suggests that there is a steep drop off in the first four minutes of the survival curve and the slope gradually tails off after six minutes.26 They suggested that EMS systems will need to develop novel strategies to further optimise their 90th centile DRI significantly below eight minutes.

The role of ALS skills may change in the future from being targeted at management of CA to other issues such as prehospital diagnosis of acute myocardial infarction and thrombolysis as well as definitive airway management in the head injured patient.12 The experts in this study supported the concept of possible expansion of skills for paramedic staff in certain circumstances in the future.


Alongside the implementation of PDS, existing proposals recommend an expansion of single paramedic “fast responders” and some development of “first responder” schemes to meet the 90th fractile eight minute target. These proposals are at variance with the views of senior experienced personnel as shown in this study. They suggest more cost effective alternatives that would be likely to produce a similar or improved impact on outcome in certain groups of patients.

The exact benefits of PDS, tiered services, and the additive role of paramedic ALS skills in managing these patients remains to be properly defined in a prospective study.


The Delphi technique was chosen to develop a consensus of opinion. The process is known to have a number of strengths that include anonymity, controlled feedback, and a statistical group response.27 Others have argued that the method forces consensus and is weakened by the inability of the “experts” to discuss the issues.11

Anonymity is important as a means of avoiding dominance, and was secured in the study by using a questionnaire. Good agreement between rounds is adjudged to be representative of consensus. Statistical group response ensures that each opinion is represented in those who contribute to the final questionnaire. Although the responses of only 29 of the original 85 entered into the study were taken into account in the final analysis, the high level of agreement between the two questionnaires suggests that this was a representative group. However, it is possible that this may not have been an entirely representative group.

Another weakness in the Delphi technique is its failure to stipulate at what level consensus is deemed to have been reached. Investigators therefore arbitrarily define it. Median scores were chosen for each statement having developed clear prior categorisation of the scores to define agreement and disagreement.11

In addition, the results of such a study can be sensitive to the number and selection of the participants. Selecting a particular group of Delphi “experts” presents a number of conceptual problems in defining who has an interest in the issue under investigation and who can provide a relevant and informed opinion on the matter. We chose to use senior executives and directors from Ambulance Trusts because they were a clearly identifiable group, had the greatest experience and expertise of EMS system design in the UK and were likely to be instrumental in instigating change within their respective systems. However, it must be accepted that this group may be biased towards a particular strategy. The notable lack of dedicated medical directors within EMS systems in the UK at present prevented any representative opinion by them.28


This study has quantified the opinion of senior directors of EMS systems in the UK. They suggest that the optimal system would be for a tiered service, better integrated into other emergency services, and with a dual response to certain category A emergency calls. Certain circumstances would require the development of additional ALS skills. The results from this study can be used to stimulate further debate among policymakers.

Evidence suggests the need for innovative and progressive strategies to significantly reduce the EMS response time for immediately life threatening conditions to improve their outcome. Failure to adopt such strategies will result in the continuing dismal outcomes for time critical conditions like prehospital CA.


We would like to thank senior staff from the Leicestershire Ambulance and Paramedic Service for their helpful advice in the design of the questionnaires. Thanks also to the individuals who were kind enough to participate in the study.

 Dr Hassan initiated the design of the study, executed it and wrote the manuscript, which contributed to his MD thesis. Professor Barnett commented on the design and reviewed the manuscript. Dr Hassan is the guarantor for the paper.

Funding: Dr Hassan was funded by the Trent Institute for Health Services Research on a Training Fellowship.

Conflicts of interest: none.


View Abstract

Request permissions

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

Linked Articles

  • Primary Survey
    Peter Driscoll Jim Wardrope