The number of emergency (999) calls received by ambulance services in the UK has been rising persistently¹ although not all calls require emergency department (ED) care.^2–6 Up to 30% of patients for whom a 999 call has been made are left at the scene,¹ with onward referral to a primary or other health or social care provider being rare. Emergency medical technicians (EMTs) and paramedics (ambulance crews) have usually had little, if any, formal training and do not generally have protocols to support decisions to leave patients at the scene.⁷

With an increasing emphasis on integrated emergency care systems in the UK,⁸ the need to consider alternatives to the standard emergency ambulance response has been highlighted (box 1). Recent national policy changes have allowed exploration of alternative responses to less serious 999 calls—termed “category C” calls in the 1996 NHS Executive Steering Group Review of Ambulance Performance Standards⁹—and the option not to convey to the nearest ED has been confirmed.¹⁰ Similar pressures to provide appropriate and safe care exist internationally and have led to several preliminary descriptive studies.^11–17 Issues explored have included the ability of crews to select cases for alternative management as well as optimal crew configuration, with the inclusion of nurses in the field response tested in Sweden.

Implementing alternative responses to transportation to the ED has been advocated many times.^2–6 Avoiding unnecessary journeys to the ED may bring benefits for patients by providing appropriate care with less inconvenience; for the ambulance service by freeing up ambulances more quickly to attend other more serious cases; for crews by improved job satisfaction and reduced stress; and for the wider health service through more efficient and effective use of resources. However, the development of alternatives to transportation to the ED depends on the ability of emergency crews to make decisions to leave patients at the scene. Several studies exploring this issue have highlighted the difficulties of pre-hospital identification of patients’ clinical needs.^18–20 These studies have been shown in a recent review to be methodologically weak, with varying methods used to assess appropriateness of care and safety, and no randomised controlled trials have been completed in this field.²¹ However, preliminary results have raised concern, consistently showing a substantial risk for a small minority of patients^22,23 and, indeed, one US trial was recently abandoned due to concerns about safety.²⁴

Against this background, this study was set up to assess the effectiveness of implementing a new alternative to emergency transportation to the ED for non-serious 999 callers. The aim of the study was to develop and evaluate “Treat and Refer” (T&R) protocols for ambulance crews which allowed them to leave patients at the scene with referral to community based services or self-care advice. Indicators of effectiveness were set up to measure the operational effects on the emergency care system, as well as the quality and safety of care delivered to patients, from the perspective of patients as well as using clinical measures.

METHODS

Study design overview

The study included both the development and testing of protocols. During phase 1 the treatment protocols were drafted and agreed by a clinical panel. During phase 2 the crews from one ambulance station were trained to use the protocols and the new service was implemented for a trial period (intervention group). During this period processes and outcomes of care were assessed for patients meeting the study inclusion criteria attended by trained crews with similar patients attended and treated by crews working according to standard practice at a neighbouring station (control group).

Study setting

Both ambulance stations were in West London. The protocols were implemented between May and August 2000. The site was selected after extensive external and internal consultation as its crews were reported by local ambulance service managers as being receptive to participation in the study. It was recognised from the outset that this new service would require partners to work together across the traditional boundaries of primary, secondary and social care, so a project steering group was set up with a wide membership in order to bring the perspectives of different service providers and commissioners, professional and patient groups to the study. This group met regularly throughout the study and provided advice and support at all stages.

Development of protocols

A multidisciplinary clinical panel made up of representatives from the London Ambulance Service (LAS) and local ED and primary care services was formed to oversee the development of the protocols. A subgroup of ED consultants, the LAS Medical Director, a specialist registrar (JR), and a representative from the LAS training department met more frequently to draft and revise the detail of the protocols. The protocols were based on findings from:

• a review of previous research literature;

• a national survey of ambulance services to identify other initiatives concerning alternatives for patients with primary care needs (published separately²⁵);

• baseline local data collection at the two participating stations between June and September 1999: individual cases falling within illness codes with a high rate of non-conveyance were reviewed against the protocols developed to check coverage;

• routine LAS management information data concerning conveyance rates across the service.

Crew training

The intervention station crews (five paramedics and five EMTs) participated in a 2 day training course in October 1999 led by LAS trainers. The first day included a brief overview of the T&R protocols and role play exercises based on the clinical conditions. The second day gave an overview of the new organisational arrangements being introduced (primary care trusts) to provide an improved community service. Representatives of several local agencies talked to the crews about their service and referral and assessment criteria. Senior managers from the LAS put the project into the context of service developments taking place and the need for research in this area. Research sessions were delivered by members of the research team and covered fundamentals of research methods, including the importance of recruiting a control group and the need for accurate documentation of patient care.

At the outset of the study no further training was envisaged but, due to identified needs, two additional half day workshops were put on in March 2000. This training was of a clinical nature and was led by the Medical Director of the LAS and the specialist registrar (JR) who had led the drafting of protocols (fig 1).

Clinical scenarios were used in the workshops to review the crews’ assessment and decision making skills about conveyance. Emphasis was put on covering all the assessment criteria outlined in protocols and the taking of a consistent and systematic history including the measurement and documentation of clinical observations.

Each crew member underwent a competency assessment, with all crew members deemed competent to use the protocols for the trial.

Study outcome measures

In order to meet the first study objective—to assess the effects of the intervention on the emergency care system—the primary outcome of the evaluation was set as the difference between the proportion of patients left at the scene by crews from the intervention station and those attended by crews from a neighbouring control station. As a secondary outcome concerned with operational impact, the interval between the 999 call and the ambulance reporting the case finished (job cycle time) was also compared between groups.

Further secondary outcomes aimed to assess the quality and safety of care delivered to patients included (1) frequency and patterns of protocol usage; and (2) completeness of clinical documentation. For the patients of most concern—that is, those who were not taken to hospital—comparisons were made between the groups of (3) the number of patients admitted to hospital within 14 days of their emergency ambulance attendance who were judged to have been inappropriately left at the scene by their attending crew; and (4) patient satisfaction.

Identification of intervention and control group cases

As it would have been inefficient to include all 999 patients in the study, a method was needed for identifying similar patients in each group who would be appropriate to leave at the scene that did not rely on crew selection. Locally collected baseline data concerning patterns of conveyance in the study area were used to identify illness codes (routinely assigned by ambulance crews on their patient report forms, completed for each individual patient they attend) that were most frequently given to patients who were left at the scene. These illness codes were used to identify patients for inclusion in each arm of the study.

Cases for which crews reported that they had used their new protocols but which fell outside of these illness codes are described but not included in the comparative analysis as they were not matched in the control group by any similar cases.

Sample size calculation

Based on activity levels before the trial, it was estimated that each week there would be approximately 50 cases meeting the study inclusion criteria. A sample size of 530 patients was aimed for in each arm of the study to give an 85% chance of detecting an increase in non-conveyance rate of 9% (from 35% to 44%) at the 5% level.

Data collection

Patient report forms completed by ambulance crews were collected for all cases included in the study. Cases were reviewed by JR to assess whether protocols were used appropriately. A second clinician (ED consultant/Medical Director of LAS) independently assessed protocol usage for 15% of the intervention patients to assess inter-rater reliability. ED records were collected for all patients taken there, and these were reviewed by JR for all patients discharged home with only basic or no care.

Follow up data were collected on patients who were left at the scene from ED and GP records and by postal questionnaire. The patient questionnaire (modified version of the “Quality of Care Monitor”²⁶) measured patient satisfaction and health service usage in the 14 days following the incident.

All non-conveyed cases identified as having been admitted to hospital in the 14 days after the 999 call were reviewed by two local ED consultants who were asked to judge whether non-conveyance was appropriate at the time of the 999 call and which (if any) protocols should have been used for intervention group patients.

Figure 2 illustrates the data collection processes.

Use of routine management information

In order to help us interpret our findings, once the illness codes had been identified for the trial, routinely collected management information was used to compare the proportions of patients not conveyed to hospital and job cycle times for cases meeting the identified codes for the two study stations and service wide for the same periods in the 2 years before the trial.

Data analysis and research ethics approval

The study illness codes were grouped into condition categories for the purposes of analysis. Binomial data were analysed by Fisher’s exact test, other categorical data by a χ² test with correction for continuity. Numerical data were analysed using the Mann-Whitney U test. Inter-rater reliability was assessed with Cohen’s kappa.

Approval for the study was granted by the local research ethics committees for the area.

RESULTS

Baseline operational indicators

Pre-study routine management information from the same months in 1998 and 1999 showed that non-conveyance rates for cases meeting the study inclusion criteria were significantly higher at the study stations than service wide (1998: intervention station 34%, control station 37%, service wide 23%, p<0.001; 1999: intervention station 32%, control station 32%, service wide 24%, p<0.001). However, there was no significant difference between the intervention and control stations (1998: 34% v 37%, p = 0.34; 1999: 32% v 32%, p = 1.0). Both stations had relatively short mean job cycle times for non-conveyed cases (1998: intervention station 44 minutes, control station 46 minutes, service wide 51 minutes; 1999: intervention station 46 minutes, control station 46 minutes, service wide 55 minutes), again with no difference between the sites.

Protocol development

Twenty three protocols were developed (box 2), intended for active use during the patient care process and including decision points for transportation to the ED, referral to other agencies, or self-care (fig 3). For any individual patient, all protocols appropriate to the presented problem were required to be used, in conjunction with one another if necessary. Together, the protocols covered the problems documented as presented for 75% of the cases that had not been conveyed to hospital in the locally collected baseline data (table 1).

Main trial: sample details

From the baseline data collected at the study sites, 24 illness codes were selected to be used to identify patients to be included in the study. The primary and secondary outcomes depended on data collected from within the ambulance service and, to our knowledge, complete retrieval was achieved. In all, 797 patients were included in the study: 251 in the intervention group (attended by a crew member trained to use the new protocols with a condition that fell within the study illness codes); 537 in the control group (attended by a crew from the control station, with a designated study illness code and treated according to standard practice); and nine attended by trained crews but with a condition outside the designated illness codes and therefore excluded from the comparative analysis.

There were no statistically significant differences between the intervention and control groups in age (54 v 47 years, p = 0.08, 95% CI of difference 0 to 7), sex (52% v 51% male, p = 0.69, 95% CI of difference −5 to 8), or condition category of patients (χ² = 15.23, p = 0.06), with the most common problems in each group being falls (29.6% v 22.9%), generally unwell/medical problems (17.3% v 18.2%), and minor injuries (14.6% v 12.1%). There were differences in the temporal distribution of calls, with intervention group patients somewhat more likely to have been attended during daytime hours than at night (61% v 52%, p = 0.02, 95% CI of difference 2 to 15).

Main trial: primary outcome

There was no significant difference in the proportion of patients left at the scene in the intervention and control groups (93/251 (37.1%) v 195/537 (36.3%), p = 0.90, 95% CI of difference −6 to 7).

Main trial: secondary outcomes

Protocol usage

Protocols were recorded as having been used for 101 of the 251 patients in the intervention group (40.2%) and for nine other patients who fell outside the study inclusion illness codes. All eight crew members who were trained to use the protocols recorded having used them, although the proportion of patients seen during the study period for whom they did this varied widely (mean 42%, range 11–100). Seventeen of the 23 protocols were reported as having been used during the study, with the falls protocol used far more than any other (57 times). The next most frequently used was soft tissue injury (9 times).

On retrospective review of crew patient report forms, JR judged a further 74 cases (29.5%) to have been appropriate for protocol usage, and three (1.2%) for which a protocol was documented as having been used to have been inappropriate for protocol usage. Hence, of the 181 patients treated by trained crews and deemed suitable for assessment with a protocol, crews recorded having done so for 107 (59.1%) of them. Independent assessment of a sample of cases by a second clinician showed moderate but significant agreement with the first assessment (seven discordant pairs and 30 concordant pairs, kappa = 0.55, p = 0.001).

In 93/156 (59.6%) conveyed patients in the intervention group, A&E department records could be located who did not wait, self-discharged, or were discharged home; 36 (38.7%) of these patients received no investigations and no or only minor treatment (Steristrips, written/verbal guidance, oral analgesia, other oral drugs). Crews had used their protocols for 17 of the 36 cases and a review by JR indicated that six more patients may have been appropriate for protocol use.

Job cycle times

The median job cycle time was 4 minutes longer for intervention than control group patients (51 v 47 minutes; p<0.001, 95% CI of difference 2 to 9). As shown in table 2, in both groups the median job cycle time was considerably longer for patients who were conveyed than for those not conveyed. However, the median job cycle time was 8 minutes longer for non-conveyed patients in the intervention group than in the control group (35 minutes v 27 minutes, p<0.0001, 95% CI of differences 5 to 11).

Clinical documentation

Overall, as shown in table 3, clinical observations were more comprehensively documented in the intervention group. However, a similar proportion of patients in both groups had no clinical assessments recorded at all.

Patients who were not conveyed to ED: further analysis

Response rates and missing data

As summarised in table 4, after excluding repeat callers, patients who lived outside the study area and those without sufficient detail for contact, 117/215 (54.4%) of those invited to participate completed a follow up questionnaire. Data on subsequent care were retrieved from GPs and local ED clinical records for a further 36 patients (16.7%), but there remained 62 (28.9%) for whom no follow up information could be retrieved.

Patient safety

Five of the 93 patients (5.4%) in the intervention group and 12 of the 195 (6.2%) in the control group were identified who had been left at home and were admitted to hospital in the 14 days after the 999 call. After excluding patients who refused to travel, the clinical reviewers judged that three patients in each group should have been taken to the ED at the time of the 999 call. Crews had reported using protocols for two of those in the intervention group, but the ED reviewers judged from the details of the condition of the patient on scene and the ED records that additional protocols should have been used and that use of these protocols would have led to a decision to convey to an ED (see case reviews 1 and 2, box 3).

Only two of the nine patients assessed with protocols who were outside the study inclusion illness codes were left at home and neither of these was identified as having been subsequently admitted to hospital.

Patient satisfaction

A higher percentage of non-conveyed patients in the intervention group strongly agreed that their ambulance crew gave them the right amount of advice (69% v 46%, p<0.05); that they were reassured by the advice (72% v 45%, p<0.05); that they were given clear advice about when to get more help (71% v 47%, p<0.05); and that they were generally satisfied with the ambulance crew (81% v 58%, p<0.05; table 5).

DISCUSSION

Summary of key findings

Operational impact

This study failed to demonstrate a change in the primary outcome measure (the proportion of patients not conveyed to hospital) associated with the introduction of “Treat and Refer” protocols. However, the intervention was found to have an operational impact through increased job cycle times.

Quality and safety of care

Although the protocols were used to some extent by all the crew members trained in their use, retrospective review of cases suggested that there was potential both for further use of existing protocols and for development of new protocols. Overall, documentation of clinical assessments was higher in the intervention group than in the control group.

Concerns were identified with the safety of current usual practice and of practice using the protocols. High rates of satisfaction were reported in both groups, although satisfaction with some aspects of care—particularly those related to communication of advice—showed higher levels in the intervention group than those attended by crews from the control station acting according to their usual practice.

Study limitations

Study design

There were a number of constraints on the design and implementation of the study. Because of the nature of the intervention which necessitated training of crews to use new protocols and the operational requirements of an ambulance service, it was impractical to set up a study in which patients could be randomly allocated to study groups. Hence, the study design was to track two cohorts of patients that were attended by crews over the same period. Adjacent stations were selected for pragmatic reasons to increase the comparability of the two groups, with the two stations serving similar populations in terms of socioeconomics and geography. However, overlap of station patient catchment areas and receiving hospitals meant possible contamination caused by crew interaction.

Site selection

As the study site was selected due to the anticipated compliance of crews, lower levels of compliance might be seen if protocols were introduced elsewhere. On the other hand, historical data showed that the study sites had relatively high non-conveyance rates and short job cycle times compared with the ambulance service as a whole. The study might therefore have been less likely to find an impact on these measures than at other sites.

Reliability of patient satisfaction follow up

Patient satisfaction measures tend to overestimate satisfaction.²⁷ In this study this effect should have been found in both groups and therefore it is unlikely to have led to any bias in the comparisons presented.

Case recruitment

The power of the study was reduced because of lower recruitment to study groups than anticipated. This occurred due to unexpectedly high rates of absence of trained crews during the study period and non-participation in the study when trained crew members were “driving” rather than “attending” appropriate patients. We had aimed to recruit 530 patients to each arm of the study but, in the event, this was only reached in the control group with half that number recruited to the intervention arm. With this number we would have been 75% certain of detecting a difference of 9% in non-conveyance at the 5% level. The observed difference was only 1%, which was not statistically significant and was not considered by the research team to be of any clinical or operational significance.

Impact of introduction of protocols

No impact on conveyance rates was identified. However, there was some evidence to suggest changes in practice that may have been masked by the overall result—most noticeably, the increased time on scene spent with patients in the intervention group. The increased time may indicate improved quality of care, supported by more thorough documentation of observations for patients in the intervention group and increased levels of satisfaction of intervention group patients.

If the increased time spent on scene noted in this study proved to be a persistent effect, a considerable impact would be made on operational performance of an ambulance service. With no journeys saved and an increased job cycle time, ambulances would be tied up for longer and availability for primary response to new emergency calls might be affected. In addition, in this study protocols were not used to their full potential, with usage variable between crew members. Crews’ views and attitudes towards the new intervention have been explored in a qualitative study that was undertaken alongside the quantitative study reported here.²⁸ If protocols were used to their full potential, the operational impact of increased job cycle times would be increased. Although care may be more appropriate, the operational implications of this need to be considered by ambulance services.

Conclusions

The necessity to develop the 999 service to improve the quality and appropriateness of service provided is widely recognised, but this study confirms the complexities involved in doing this. Although the trial showed that protocols for this purpose were used by crews and were acceptable to patients, the anticipated operational benefits were not demonstrated and safety concerns were identified. The lessons learned about the design and implementation of “Treat and Refer” protocols are valuable for taking the work forward, but to fully realise the potential benefits of this innovative change in service delivery these issues will need to be addressed in further research and/or service development work.