Literally translated, telemedicine means “medicine at a distance”. This definition includes telephone use,¹ although the term telemedicine is more commonly used to describe a consultation involving the transmission of visual images. In truly remote locations, such as Antarctica,² the benefits of telemedicine are obvious. Extrapolation to less remote settings offers the potential to provide urgent medical advice without the need for prolonged patient or staff travel.

Recent years have seen considerable pressure to rationalise emergency services in the United Kingdom. The Audit Commission’s report of 1996 highlighted a shortage of medical staff, and proposed the closure of up to 30 emergency departments nationally.³ On the other hand, rural communities seek to keep care as local as possible.⁴ One approach has been to develop the emergency nurse practitioner.⁵ Such people reduce pressure on medical staff and are well suited to the management of minor injuries.⁶ An increasing number of units are employing emergency nurse practitioners, who have been shown to be as effective as junior doctors in ordering and interpreting radiographs.⁷

Telemedicine can support emergency nurse practitioners working in remote locations, continuing to provide a local service with reference to central expertise.⁸ This approach has been endorsed by the British Association for Accident and Emergency Medicine.⁹ While a handful of studies have suggested that telemedicine may be clinically effective,^10–13 no large or scientifically rigorous trials have been undertaken in the United Kingdom.¹⁴

This paper reports the results of a study conducted to determine the safety of minor injuries telemedicine in comparison with the current practice of general practitioner care, on-site specialist management, and a gold standard. The study also aimed to assess the clinical effectiveness of this new technique through the medium of a prospective, randomised, and blinded clinical trial. An assessment of the relative efficiency of the different options will be reported separately (unpublished data).

METHODS

Setting

The peripheral unit is a typical community hospital located in a Cotswold town. Four experienced nurses, supported by healthcare assistants, staff the emergency department, which sees about 10 000 new patients per annum. Sixteen local general practitioners (GPs) from four practices provide medical input. On-site radiography is available from 9 am to 4 pm on Monday and Tuesday, and 9 am to 1 pm on Wednesday to Friday. Under normal circumstances about 35% of patients presenting with minor injuries are treated exclusively by nursing staff, 62% are referred to a GP (often to arrange and interpret radiography), and about 3% are referred directly by nursing staff to the main hospital. The main hospital has an emergency department typical of a district general hospital, seeing about 35 000 new patients per annum. None of the staff in either hospital had any prior experience of telemedicine.

A PictureTel Swiftsite videoconferencing system was installed in the emergency department of each hospital. At the peripheral site there were two additional devices: a PictureTel 550 document camera for radiological transmission, and a Panasonic VX54B camera to provide close up views. The equipment was installed and optimised four weeks before the start of the study. Staff training ensured competency in equipment operation, with telemedicine procedures based on a previously published protocol.¹⁵

Participants

Participants were recruited from patients attending the peripheral site at times when the onsite specialist (JRB, a year three specialist registrar in emergency medicine), was available. The onsite specialist played no routine part in the peripheral unit, and was present only for the purposes of the trial. The onsite specialist attended the peripheral unit from 8 am to 8 pm on a total of 156 days spread throughout a 12 month period. Recruitment was not conducted overnight as patient attendances were very infrequent, and there was no senior doctor immediately available for telemedicine at the main hospital. An experienced nurse, who was also able to request radiographs of the peripheral skeleton, assessed each patient on arrival according to normal practice. Patients fulfilling the inclusion and exclusion criteria (box 1) were referred to the onsite specialist for recruitment, with the nurse prospectively recording what would have happened to the patient had they not entered the trial.

Written consent was obtained from all patients, or from a parent or legal guardian for children. Approval for the study was gained from the East Gloucestershire Ethics Committee.

Study design

To evaluate safety the study directly compared three independent medical assessments with a gold standard. In addition, a prospective, randomised, and blinded trial was conducted to measure clinical effectiveness. Process measures relating to consultation duration, radiograph requests, and follow up were also collected prospectively. Figure 1 is a summary trial flow diagram.

Patients were independently assessed by each of: the onsite emergency medicine specialist, a second emergency medicine specialist (either a consultant, staff grade, or specialist registrar) from the main hospital using telemedicine, and an onsite GP (if available). GPs visited the peripheral unit on a rota basis from their individual surgeries, and were therefore not always immediately available to assess recruited patients. Participants were generally unwilling to wait more than one or two hours to see a GP if two emergency medicine specialists had already seen them, and GP assessment was therefore omitted if a GP was not readily available in the peripheral unit.

Assessment by telemedicine entailed an initial telephone call to the main emergency department to indicate that a teleconsultation was required. When a senior doctor became available they started the telemedicine link and undertook a remote consultation with the patient, facilitated by the nurse.

Radiographs requested by the nurse before medical consultation were made available to all doctors during their assessment. If any doctor felt that further radiographs were required they requested these as part of their treatment plan, but these radiographs were not made available to the other doctors unless they had also independently requested them.

Each doctor made an independent written record of their diagnosis, radiological interpretation, and itemised treatment plan with follow up arrangements. The doctor did not discuss these with the patient. When all treatment plans had been recorded patients were randomised to follow one of these plans using one-to-one-to-one block randomisation with a variable block size. A second nurse opened a sealed, opaque envelope corresponding to the patient’s study number. This indicated which plan should be followed. The treatment instructions from this plan were transcribed onto a loose sheet, which was returned to the first nurse to enact. This ensured that neither the treating nurse nor patient knew which plan was being followed. Where the GP treatment plan was indicated a second randomised option was also stated for use if the GP assessment had been omitted.

All treatments were carried out by nursing staff, with the patient referred to a GP or transferred to the main hospital if further medical intervention was indicated on the treatment plan.

Participants were reviewed seven days after their initial assessment, as substantial improvement is common by this time. All participating doctors were equally aware of the trial protocol and arrangements for review. Patients remained blinded to which treatment plan they had followed. A standardised questionnaire was completed by each participant (or consenting adult in the case of a younger child). A gold standard diagnosis was then established by direct clinical assessment of each patient by the principal investigator, the report of a consultant radiologist blinded to the study and referral to a consultant specialist or further follow up until complete resolution.

Outcome measures

Outcomes were assessed in terms of safety, process, and clinical effectiveness. Safety was assessed by comparing each arm of the trial to the gold standard. All discrepancies in diagnosis, radiological interpretation, and treatment plan were submitted to an expert panel comprising 10 consultants in emergency medicine who were not otherwise involved in the study. Panel members did not know which trial arm had given rise to each discrepancy, and were asked to score these according to the scale shown in box 2, which was developed specifically for this research. The scores allocated by each of the 10 panel members were used to calculate both a mean and median discrepancy score. The mean score was selected for further analysis because it served as a better discriminator between the 44 discrepancies that had a median score of +1 or −1. Process outcomes were the duration of each consultation and the frequency with which radiographs and follow up appointments were requested. Measures of clinical effectiveness were derived from the review questionnaires, and are listed in box 3.

Sample size

The sample size was calculated for the primary outcome measure of safety, rather than the secondary outcome measures of clinical effectiveness that were addressed in the randomised trial. We selected a sample size of 600 because the absence of a significant discrepancy in this number would allow us to state, within 95% confidence limits, that the risk of significant discrepancy in actual clinical practice would lie somewhere between zero and 1 in 200.¹⁶ This is comparable to existing and accepted risks in emergency medicine.¹⁷ With regards to the secondary outcomes of clinical effectiveness, this sample size is sufficient to detect a 12% difference between telemedicine and the onsite specialist (α = 0.05, two sided, and power 0.8).

Data handling and statistical methods

Data management and analysis were performed using a computerised Stata database (Stata Corporation, Texas, USA). In the randomised trial data were analysed on an intention to treat basis incorporating the two stage randomisation procedure. For process measures, statistical testing took into account the non-independent nature of the data. The numbers available for each pairwise comparison are noted, because not all patients received all three alternatives. For categorical variables the Cochran Q test and McNemar’s χ² test were used.¹⁸ Paired t tests were used for normally distributed continuous variables. To take account of the multiple significance testing, a Bonferroni correction was applied.¹⁹ This meant that to maintain a 5% significance level p values from the three pairwise tests between the three groups were only considered significant if they were below 0.017.

For effectiveness data χ² testing and the Kruskal-Wallis equality of populations rank test were used. A two sided 5% significance level was used throughout, and 95% confidence intervals (CI) are presented where appropriate.

RESULTS

Recruitment and patient progression

Recruitment of patients took place between December 1999 and November 2000. Figure 2 summarises the flow of patients through the clinical effectiveness trial. A total of 713 patients met the inclusion criteria, of whom 103 were excluded and 10 declined to participate. The “other” exclusion was an 8 year old girl with a penetrating injury to the perineum, who was not considered an appropriate candidate for telemedicine. In 21 recruited patients (11 of whom were subsequently randomised to follow the telemedicine treatment plan) no telemedicine consultation took place because the main hospital was too busy with seriously ill patients to respond within two hours: this meant that for the primary outcome of safety, telemedicine could be compared with onsite specialist assessment in 579 patients, and compared with onsite GP assessment in 165 patients.

In the effectiveness trial, of the 600 patients recruited, 200 were initially randomised to receive the telemedicine treatment plan, 201 the onsite specialist plan and 199 the onsite GP plan. However, 135 patients randomised to the GP plan had not seen a GP, because no GP was immediately available to assess the patient. Therefore secondary randomisation assigned 74 of these patients to telemedicine and 61 to the onsite specialist plan. This meant that 274 patients were allocated to telemedicine, 262 to the onsite specialist plan, and 64 to the GP treatment plan.

The mean age of recruited patients was 34 years with a range of 9 months to 92 years. A total of 475 patients (79.2%) were registered with a local GP. Table 1 shows the characteristics of all recruited patients by the treatment plan to which they were randomised.

Safety

There were 73 cases where a study diagnosis or radiological interpretation, or both, differed from the gold standard. Expert panel assessment of these gave:

• 12 discrepancies with a mean score greater than +1 (“over-treatments”)

• 35 discrepancies with a mean score between +1 and 0

• 15 discrepancies with a mean score between 0 and −1

• 11 discrepancies with a mean score less than −1 (“under treatments”)

The 50 discrepancies with a mean score between +1 and −1 were excluded from further analysis as these were minor, and attributed to variations between clinicians rather than errors likely to have a significant effect on patient outcome.

The 12 “over-treated” and 11 “under-treated” cases are detailed further in tables 2 and 3. The significant under-treatment rate ranged from 0% (95% CI 0% to 1.7%) for general practitioners to 0.5% (95% CI 0.1% to 1.5%) for the onsite specialist, and 1.4% (95% CI 0.6% to 2.7%) for telemedicine. Telemedicine appears to have the highest rate of significant under-treatment, but this is not statistically significant on χ² testing. As none of the 579 telemedicine consultations received a score worse than −2 it is possible to state that for minor injuries telemedicine the risk of error leading to severe consequences has a 95% confidence interval between 0% and 0.52%.

Process measures

The mean delay between injury and presentation at the peripheral hospital was 1.3 days (95% CI 1.2 to 1.5 days). The mean wait to see a nurse at the peripheral hospital was 4.2 minutes (95% CI 3.4 to 5.0 minutes).

Table 4 shows the action that would have been taken by the assessing nurse, if the patient had not been recruited into the trial.

For all telemedicine consultations, the mean delay between a request for telemedicine being made and actual start of the teleconsultation was 7.1 minutes (95% CI 6.3 to 7.9 minutes). Technical failure occurred in two cases (0.3%).

The mean duration of all consultation types was as follows: telemedicine 6.0 minutes (95% CI 5.7 to 6.2 minutes), onsite specialist 3.1 minutes (95% CI 2.9 to 3.3 minutes), and onsite general practitioner 3.4 minutes (95% CI 3.2 to 3.6 minutes). Paired t testing showed no difference between the onsite specialist and GP consultations (p = 0.64, n = 177), but a significant difference between the time taken to perform a telemedicine consultation and either an onsite specialist (n = 578) or GP consultation (n = 171) (p<0.0001 in both cases). The average time taken to conduct a telemedicine consultation reduced as the study progressed, with a mean of 7.3 minutes (95% CI 6.2 to 8.4 minutes) for the first 100 patients and 5.0 minutes (95% CI 4.7 to 5.4 minutes) for the last 100. However, telemedicine consultations remained significantly (p<0.0001) longer than either the onsite specialist or the GP for the last 100 patients.

For each consultation type the number of patients who had radiographs and asked to attend for a further follow up appointment is shown in table 5. Overall there were no significant differences in the proportion who had radiographs by the three alternatives (p = 0.11, n = 171, Cochran Q test). There were, however, significant overall differences between the proportion followed up by the three alternatives (p<0.0001, n = 170, Cochran Q test). GPs asked a significantly greater proportion of patients to attend for follow up than either telemedicine (p<0.0001, n = 170, McNemar’s χ² test), or the onsite specialist (p<0.0001, n = 177 McNemar’s χ² test). A significantly greater proportion of patients were asked to attend for follow up after a telemedicine consultation than after an onsite specialist consultation (p<0.0001, n = 578, McNemar’s χ² test). The proportion of patients asked to attend for follow up after a telemedicine consultation did not reduce during the study, being identical at 40% for both the first 100 and last 100 patients.

Clinical effectiveness

Eight patients (1.3%) were lost from the study. The mean time between recruitment and review was 7.0 days (SD 1.4 days, range 3 to 14 days). There was no significant difference in the time to review between the three randomisation groups (p = 0.82, Kruskal-Wallis test).

Table 6 summarises the main measures of clinical effectiveness. No significant differences were detected for any of these outcomes

DISCUSSION

These results show that the safety of minor injuries telemedicine is similar to conventional practice. It is inevitable that errors will occur during any large series of consultations, and this research quantified and compared these errors for the three alternatives studied, finding no significant differences between them.

Only one randomised trial of telemedicine in a similar context has previously been published.¹³ This was a much smaller study, conducted in North America with just 104 participants, but showed similar results.

Over-treatments, radiographs, and follow up

We studied the over-treatment rate because this might be increased for telemedicine, particularly if medical staff were unsure of the diagnosis, but found no evidence that this was the case.

While the radiography rate was nearly identical for telemedicine and onsite specialist consultation (59.2% compared with 60.5%), a follow up appointment was arranged for a significantly greater proportion of telemedicine patients (35.8% compared with 27.5%: p<0.0001 using McNemar’s χ² test), perhaps reflecting a lack of confidence in telemedicine. This did not reduce during the course of the study. The highest rate of planned follow up was seen in GP consultations (65%). This may reflect the way in which GPs normally practice, as we observed that follow up appointments arranged by a GP were frequently an option that the patient could pursue only if problems persisted. This is supported by the finding that, at the seven day study review, there were no significant differences between the randomisation groups in planned or unplanned use of healthcare services.

Under-treatments

Under-treatments are of greater importance as they may lead to long term effects and possible litigation. Several studies have attempted to quantify the incidence of error in emergency medicine, often concentrating on radiological interpretation. In the United Kingdom, a recent study quoted an error rate of 1.5%, falling to the more acceptable figure of 0.7% when a picture archiving and communications system (PACS) was introduced.²⁰ In emergency care it has been necessary to establish a risk below which a patient can be reasonably discharged, and this threshold is commonly set at 1%.²¹ In this study only one consultation, by the onsite specialist, received a mean under-treatment score worse than −2, and for telemedicine the risk of error leading to severe consequences was less than 1% (95% CI 0% to 0.52%).¹⁶

Process measures

Patients attending the peripheral hospital were usually seen by a nurse very promptly, and had to wait a mean of only 7.1 minutes for a telemedicine consultation, excepting those cases where the main hospital was too busy to respond. This suggests that patients attending the peripheral site were likely to be assessed by a senior doctor much more rapidly than if they had attended the main hospital in person. Indeed, medical staff commented that there was a possibility that they would leave a patient with more urgent problems to respond to a request for minor injuries telemedicine.

If telemedicine acted to preferentially bypass the queue in the main emergency department, patients might choose to travel to the peripheral hospital for treatment. One approach to this problem would be to “triage” telemedicine cases as if they had come to the main site in person. This would cause practical difficulties at the peripheral site, and might also prove unacceptable to patients who are used to the prompt service that the peripheral hospital usually provides.

Only 17 (2.8%) of patients would have been referred directly to the main hospital if they had not been recruited into the trial, so the potential for telemedicine to reduce unnecessary patient transfers in this context is very limited. The nurse’s assessment of what would have happened had the patient not been recruited could have been biased by the knowledge that the patient was about to enter the trial, but this transfer rate of 2.8% is very similar to the 3% which existed before the start of the study. The low transfer rate is attributable to the effective service already provided, and the fact that most of the injuries were minor. If the telemedicine plan had been followed for these 17 patients then eight would still have been transferred, with the remaining nine treated locally.

Telemedicine weaknesses

In 16 cases (2.8%) the doctor providing a telemedical opinion felt that telemedicine was inadequate in some way. This related either to problems in visualising and interpreting a radiograph (particularly chest radiographs) or to difficulties in assessing a wound (for example, burn depth). Nevertheless, no significant discrepancy resulted in any of these cases, suggesting that the problem was recognised and appropriately resolved.

Problems with telemedicine are often attributed to technical deficiencies that can be overcome by improving the performance of equipment.²² In this study, however, we were conforming (with the exception of using a television rather than a computer monitor) to a technical standard that achieves satisfactory results in a laboratory setting.²³ This suggests that further improvements in technology may not provide complete solutions, and that some difficulties relate to the medium itself.

Study weaknesses

We assumed that treatment discrepancies were attributable to telemedicine rather than variations in clinician performance. As most participants had comparatively straightforward injuries, and the doctors involved were of comparable experience, this assumption appears reasonable. Furthermore, the use of a large expert panel and removal of discrepancies scoring between +1 and −1 should have prevented individual treatment variation from influencing the results.

It was not possible to achieve consecutive recruitment because of the requirement for the onsite specialist to be available. Nevertheless, minor injuries rarely presented overnight, when recruitment would have been inefficient, and there is no reason to suspect that our sample is not representative of the wider population of minor injuries patients.

The methodology of this study was unusual in that rather than being randomised to receive only one type of consultation, we elected to have each recruited patient undergo more than one consultation type, with strict blinding between the consultations. This allowed us to collect a large sample in which telemedicine and conventional consultation could be directly compared with high sensitivity. A disadvantage was that the sample size was not specifically calculated for the randomised trial of clinical effectiveness. Therefore the clinical outcomes, although supportive of our safety data, should be viewed with caution, particularly as only a comparatively small number of patients followed a GP treatment plan.

Each patient underwent two or three consecutive consultations, and the onsite assessment always preceded telemedicine. This could lead to an order effect, with refinement and shortening of the history given by the patient. If such an effect existed, however, it would tend to reduce the length of telemedicine consultations, in contrast with the results obtained. The GP consultations might also have been shortened by this effect, although some were performed before and some after telemedicine. The doctors involved were also careful to avoid giving information to the patient during each consultation, minimising the effect of consecutive consultations.

Application to practice

We have shown that telemedicine is safe and effective in minor injuries practice. However, there is very little evidence regarding the effectiveness of telemedicine in other aspects of emergency medicine, such as acute illness or more severe injury.

If telemedicine is used to support nursing staff a high consultation volume can be expected, particularly if it replaces an existing medical service. This is time consuming for the hospital providing the consultations, and also runs the risk that the transfer rate will be increased because of system failures or diagnostic doubt. Direct referral to the main hospital was a compatively rare event (2.8%), but where it occurred transfer was avoided in 53% of cases. Therefore, telemedicine used to support doctors or as a supplement to an existing service may approximately halve the patient transfer rate, and is in keeping with the one previous paper that studied telemedicine to support remote general practitioners, in which transfer was avoided in 58% of cases.¹¹

While a comparatively low cost system is capable of delivering satisfactory clinical results in terms of safety and patient outcome, there are a number of potential problems that may be overlooked by enthusiasts. Many of these problems will not be influenced by further technological advances, and must be tackled in other ways: some are inherent to telemedicine itself, but many are related to service delivery and organisation. It is important that decision makers are not overly distracted by technical concerns but focus on the managerial and economic issues that underpin telemedicine as a change in the way that care is delivered.

Conclusion

In the management of minor injuries, telemedicine is capable of providing a satisfactory standard of care. Significant errors are rare and similar in frequency to conventional practice and accepted risks. There is no evidence that telemedicine is superior to the routine practice of GP consultation.

Telemedicine requires about twice as long as an onsite consultation, in addition to the time entailed in responding to a request and breaking away from other activities. While telemedicine may be effective for minor injuries work, emergency medicine encompasses a variety of other conditions that may be less amenable to this technology, and where research is still required.