Article Text
Abstract
Background The emergency care practitioner (ECP) role in the UK health service involves paramedic and nurse practitioners with advanced training to assess and treat minor illness and injury. Available evidence suggests that the introduction of this role has been advantageous in terms of managing an increased demand for emergency care, but there is little evidence regarding the quality and safety implications of ECP schemes.
Objectives The objectives were to compare the quality and safety of care provided by ECPs with non-ECP (eg, paramedic, nurse practitioner) care across three different types of emergency care settings: static services (emergency department, walk-in-centre, minor injury unit); ambulance/care home services (mobile); primary care out of hours services.
Methods A retrospective patient case note review was conducted to compare the quality and safety of care provided by ECPs and non-ECPs across matched sites in three types of emergency care settings. Retrospective assessment of care provided was conducted by experienced clinicians. The study was part of a larger trial evaluating ECP schemes (http://www.controlled-trials.com/ISRCTN22085282).
Results Care provided by ECPs was rated significantly higher than that of non-ECPs across some aspects of care. The differences detected, although statistically significant, are small and may not reflect clinical significance. On other aspects of care, ECPs were rated as equal to their non-ECP counterparts.
Conclusions As a minimum, care provided should meet the standards of existing service models and the findings from the study suggest that this is true of ECPs regardless of the service they are operational in.
- Emergency care practitioners (ECPs)
- safety
- quality
- emergency ambulance systems
- effectiveness
- emergency care systems
- major incidents
- clinical care
- management
- quality assurance
- prehospital care
- advanced practitioner
Statistics from Altmetric.com
- Emergency care practitioners (ECPs)
- safety
- quality
- emergency ambulance systems
- effectiveness
- emergency care systems
- major incidents
- clinical care
- management
- quality assurance
- prehospital care
- advanced practitioner
Introduction
The UK health service has experienced an increased demand for emergency care over the last decade, including calls to ambulance services and attendances at emergency departments (ED). The emergency care practitioner (ECP) role was introduced to address this challenge by extending the skills of paramedics and nurses operating in a range of pre-hospital and emergency care settings. There is no criterion regarding length of previous experience required to be an ECP, though in practice they have to be experienced practitioners in their field. The only stipulation for an ECP is that they have to be registered with one of three registering bodies which permit working as autonomous practitioners (Health Professions Council; Nursing and Midwifery Council; General Medical Council).1 The first ECPs started working in 2003 following a 12 month core competency programme. ECPs are trained to take a patient history, conduct a physical examination, order further investigations such as x-rays, as well as administer and supply certain medications. They may assess and treat minor illness and injury in pre-hospital, primary care or secondary care settings and make decisions on the most appropriate care pathways.1 Available evidence suggests that the introduction of ECPs has been advantageous in terms of managing the increased demand for emergency care.2–5 Although there is some evidence for the safety of extended paramedic skills in assessing and treating patients in the community,6 more rigorous evaluation is needed to assess the clinical effectiveness and safety of ECP services.7 As a minimum, the standard of care provided by staff in these extended roles should be equivalent to that in existing roles.
There is no clear consensus on methods to assess quality of care in service delivery, specifically whether to focus on care outcomes or the process of care. Lilford et al recommend the use of process measures as more suitable for judging quality in service delivery.8 A well-established method for process measurement is record (case notes) review,9 which has been used in a variety of healthcare settings, including emergency care.10–13 This approach is reliant on the information recorded in case notes, which may not always reflect the care delivered, though completeness of recording is itself a quality and safety issue.
This paper presents findings from a study to compare the quality and safety of care provided by ECPs and non-ECPs (eg, paramedic, nurse practitioner) in different emergency care settings. The study was part of a larger trial (http://www.ISRCTN22085282%20Controlled%20trials.com) evaluating ECP schemes.
Methods
Study design
A retrospective patient case note review was conducted to compare the quality and safety of care provided by ECPs and non-ECPs in three different types of emergency care setting: static centres (ED, walk-in-centre, minor injury unit); ambulance/care home services (mobile); primary care out of hours services.
Study setting and population
Stratified random sampling was used to select clinical records for the case note review. The units of stratification were service settings across UK sites. Patient case notes were selected from six services that represented the three main settings in which ECPs were operational (four static; one mobile; one primary care). Each service had a corresponding control service in which ECPs were not operational.
A random sample of 40 patient episodes was selected from all recruited patients presenting to these 12 services between 22 June 2006 and 22 August 2007, providing a total of 480 notes (240 from ECP services and 240 from non-ECP services). All patient, staff and organisational identifiers were removed. Figure 1 shows the distribution of notes reviewed across settings. A comparison of patient case mix for ECPs and non-ECPs during this period was conducted as part of the main trial study and did not show any significant differences. All cases were minor illness and minor injury in equivalent settings.
Study protocol
Given the variety of presenting conditions addressed by ECPs, an implicit approach, based on expert professional judgement, was used to retrospectively assess care as documented in case notes.14 This approach to assessing quality and safety of care is by its very nature subjective and reliant on the individual reviewers' interpretation. Therefore, in order to ensure a degree of consistency and reduce the potential for bias, a semi structured review form was used, supported by piloting and reviewer training. This form was based on the version used for implicit reviews in a UK study of review methods.14 In addition, 14 records were assessed by all reviewers in order to measure inter-rater reliability.
A panel of seven specialist registrars in emergency medicine from three UK locations were recruited to conduct the reviews. The use of experienced clinicians is consistent with the approach employed by large-scale quality and safety review programmes in the USA.15 16
Data collection
Reviewers rated quality and safety using numerical scales to assess five criteria: assessment of clinical problem; investigations performed; patient management; overall care (1=unsatisfactory, 6=very best care), and overall quality of the clinical record (1=inadequate, 6=excellent). Reviewers were provided with written guidance to aid consistency in assessments.14 Reviewers were also asked to provide textual comments regarding overall care received by the patient. Piloting of the review form was conducted with three of the reviewers. Each reviewer was provided with one record from each of the six different services and representing the range of different record types (paper and electronic).
Reviewer training
A 1-day reviewer training workshop was conducted to provide guidance on how to rank care and the type of textual comments required, and to allow reviewers to practice conducting reviews. They were informed that it was a national study evaluating the safety of care provided for minor injury and illnesses but the focus on staff roles was not identified.
At the end of the training, each reviewer was provided with a set of records and data collection software to enable the completion and return of assessments electronically. In all, 480 records were divided between the seven reviewers with each receiving a unique allocation of 67–69 records, proportionally matched from the 12 services and from both ECP and non-ECP records. Reviewers also received two extra records from those allocated to other reviewers so that 14 records were common across all reviewers, permitting the assessment of inter-rater reliability.
Data analysis
Quantitative data were analysed using SPSS V.14.17 The extent of reliability between reviewers was assessed by examining inter-rater reliability and intra-rater consistency. Intraclass correlation coefficients were calculated to assess inter-rater reliability between scores for the 14 records assessed by all seven reviewers. To assess intra-rater consistency (consistency of individual reviewers), a mean quality of care score was calculated from the combined ratings for three aspects of care (assessment, investigations and management). Pearson correlation coefficients were calculated to assess whether mean ratings for the combined aspects of care were consistent with overall ratings for quality of care. The textual comments regarding overall care were examined for consistency across reviewers.
Descriptive statistics were calculated to examine any differences, first between the ECPs and non-ECPs and then between the three different types of emergency care setting. Two-tailed t tests were used to establish the statistical significance of any differences between quality of care scores. A simple content analysis of textual comments regarding quality of care was conducted by one researcher. This made explicit the implicit assessment criteria used by reviewers and to identify any obvious differences in care for ECP and non-ECP care across the three emergency care settings.
Results
Reliability and consistency of raters
Table 1 provides the results of intraclass correlations calculated to assess inter-rater reliability for individual raters. The average measures (or sum) of the scores of the seven raters all correlate above 0.8 indicating a good level of agreement across the different reviewers.
The single measure intraclass correlations are also provided (table 1) as the comparison of care provided by ECPs and non-ECPs will rely on ratings from single reviewers rather than averaging multiple ratings for each record. In general, these will be lower than the reliability that might be expected from using the average of several raters. The correlations range from 0.52 to 0.64 and represent a reasonable level of agreement.14
The inter-rater reliability scores are higher than previous studies using the implicit review approach.14 18 Although an intraclass correlation of greater than 0.8 is regarded as indicative of good agreement, this generally relates to data having a clear right and wrong answer and where 100% agreement is possible. However, in the case of the implicit review approach based on subjective judgements, it is optimistic to expect such a high level of agreement.
Table 2 provides the results of the Pearsons correlations calculated to assess consistency in rating by individual reviewers. All seven reviewers achieved correlations between the mean scores for the three individual aspects of care and overall quality of care of over 0.7. Five of the seven reviewers achieved correlations of 0.8 or above.
Table 3 presents examples from a synthesis of reviewers' comments regarding the quality of care received by the patient overall to give an indication of the implicit assessment criteria being used to assess care, and the extent of consistency across the same 14 records. The mean overall care score for each record is also provided for comparison purposes. Though some minor variations were observed in the comments noted for the same records, there was notable consistency in the strengths and weaknesses identified.
The high level of agreement on subjective judgements suggests that the training process was successful in ensuring consistency across reviewers in distinguishing between different levels of care. It may also reflect the shared view of experienced ED clinicians in relation to quality and safety of care criteria across the range of conditions encountered by emergency care providers.
Quality and safety of care scores
The mean scores for ECPs and non-ECPs across all settings are presented in table 4. Mean scores for overall care, assessment and quality of records were significantly higher for ECPs compared with non-ECPs (p<0.01).
Table 5 presents the mean scores for ECPs and controls across the three different emergency care settings. For the static centre based settings, the mean scores for ECPs were significantly higher than controls on four of the five rating categories: assessment; management; overall care, and quality of records (p<0.01). For the mobile services, there were no significant differences observed between the scores for the ECPs and controls. For the out of hours service the mean scores for ECPs were significantly higher than controls on only one of the five rating categories—assessment (p<0.01).
Comments on overall quality and safety of care
Examination of the reviewers' text comments indicated that the implicit assessment criteria appeared to be broadly consistent across the ECP and control groups in each of the three emergency care settings. The quality of the patient record was identified as a key factor in assessing the care delivered. For example, where limited detail was provided regarding assessment and/or investigations, it was difficult to ascertain whether management was appropriate. Reviewers generally provided a greater level of detail where they gave lower scores for overall care, highlighting the perceived shortcomings in care as detailed in the patient notes. Where care was not as good as it could have been, reviewers identified deficiencies and how care could have been improved.
Where records were scored 1 or 2 for overall care, the care was generally regarded as poor or very poor and common issues included: a limited history; important omissions in history, examination and management; and major omissions in recording/documenting information, including badly written notes and difficult to read handwriting. Where overall care was rated 3, the care was often described as poor, whereas overall care rated 4 was regarded as satisfactory or adequate. Common issues identified in reviewers' comments included: limited detail and a number of omissions in assessment, investigations or management, poor description and documentation of care. For records given scores of 5 or 6, overall care was regarded as good, very good or excellent. The individual aspects of care were generally regarded as appropriate and consistent with relevant guidelines and there were no unnecessary investigations or referrals. Minor omissions tended to result in scores of 5 rather than 6. Common omissions included: tetanus status not established/recorded, analgesia not offered/given; respiration rate not recorded, urine dip test not conducted/recorded; and scope for more specific advice/information.
Discussion
ECPs scored significantly higher than non-ECPs for overall quality and safety of care provided. ECPs also scored significantly higher than non-ECPs for assessment (of the clinical problem) and for quality of the clinical record. However the differences between the ECP and non-ECP mean scores in these aspects of care were small and may represent limited clinical significance. Nevertheless, the findings from this notes review suggest that ECPs are providing a standard of clinical care which is slightly better than non-ECP providers. ECPs also scored higher in the other two aspects of care compared (investigation and management of the clinical problem) although the differences were not statistically significant.
The mean scores for ECPs and non-ECPs were compared between three different models of service delivery. ECPs working as a static resource in urgent healthcare centres had significantly higher mean scores for overall care than non-ECPs in these same centres. ECPs working in these urgent healthcare centres also had significantly higher mean scores for assessment, management and overall quality of care. Nevertheless, although statistically significant, the actual differences between the mean scores of ECPs and non-ECPs were small and were unlikely to point to a real clinical difference in the care provided. In the other two categories of service (ambulance/care home and out of hours) ECPs did not score significantly higher than non-ECPs in any aspect of care except for assessment of the clinical problem in out of hours.
ECPs currently work in a variety of settings and it is essential that an evaluation of quality and safety of patient care is carried out in each model of service provision. The selection of records for this review allowed a comparison of the quality and safety of care provided by ECPs working in different healthcare settings and indicates that care provided is as safe as existing service models in each setting. US studies evaluating pre-hospital triage decisions have mainly involved assessing the utilisation of protocols or guidelines. Silvestri et al found that subsequent ED attendance and hospital admission rates were unacceptably high in those patients triaged to be left at home, and recommended additional training for paramedics.19 Other US studies found that even with brief training, paramedics and emergency medical technicians, were not able to triage patients adequately.20 21 Difficulties have also been noted in relation to the identification of cases eligible for community treatment.22 23
In the UK, safety concerns have been raised about decisions not to convey to hospital in the case of older patients who had experienced a fall.24 However, an assessment of the care provided by extended role paramedic practitioners attending older people with minor conditions, found no difference between the paramedic practitioners and control physicians in relation to mortality and unplanned ED attendance.25 The results of the current study confirm earlier findings that in the UK, the overall care provided by ECPs is at least equally as safe as the care provided by non-ECPs in similar settings.
Limitations
This study did not measure patient outcomes in determining quality and safety of care but focused on the process of care as recommended for judging quality of care.8 9 A limitation of record review is that reviewers are reliant on information recorded in case notes to make their judgements but the level of detail in clinical records may not reflect the care delivered. Reviewers did identify quality of the patient record as a factor in assessing the care delivered, specifically noting that limited detail made it difficult to ascertain whether management was appropriate. However, completeness of recording is itself a quality and safety issue, and ECP scores were significantly higher than those of non-ECPs. An alternative approach to assessing quality and safety of care is direct observation of the actual care provided, however, this would have been more time consuming and costly.
A specific limitation of the implicit review method is its reliance on subjective judgements, rather than an explicit standard of care as with explicit (criterion based) review.9 Though a UK study comparing the two review methods found that they generated similar quality ratings. The implicit approach was employed in the current study because the wide variety of patient presentations meant it was not feasible to give reviewers a standard to refer to. The inter-rater reliability scores indicate a good level of agreement across all reviewers. To avoid any potential bias arising from reviewers' perceptions of emergency care providers and ECPs in particular, reviewers were not informed of the study focus on different roles and identifiers were removed from records.
The differences identified between ECP and non-ECP care in this study were small, however, it may be unreasonable to expect large differences in assessments of care as this would rely on ECPs providing exceptional care and non-ECPs delivering very poor care. The reliability of the content analysis of textual comments may have been improved with two independent analyses.
Conclusions
The results have shown that an extended skill role, such as these ECP services, scored significantly higher than non-ECPs across three aspects of care (assessment, overall care and quality of records). The differences detected, although statistically significant, are small and may not reflect clinical significance. However, the finding is important. It indicates that ECPs are providing a slightly better level of care than their control counterparts for the pairs of services involved in this study. As a minimum, care should be as safe as existing service models and the findings from the study suggest that this is true of ECPs regardless of the service in which they are operational.
Acknowledgments
The authors wish to thank the NHS staff who assisted with this study, in particular the reviewers (Jeremy Reynard, Steven How, Susan Croft, Ahmed Aziz, Janine Vermeulen, Anna Shekhdar and Alistair Maclean), Stephen Walters for statistical assistance and Karen Beck for developing data collection software.
References
Footnotes
Funding NETSCC, SDO Alpha House Enterprise Road University of Southampton Science Park Southampton SO16 7NS. The study was funded by the UK National Institute for Health Research (NIHR) program on Service Delivery and Organisation (SDO). However, the views expressed are those of the authors alone.
Competing interests None.
Ethics approval This study was conducted with the approval of the Scottish Multi-centre Research Ethics Committee (06/MRE00/20).
Provenance and peer review Not commissioned; externally peer reviewed.