Methods

The direct observation procedure is outlined below.

• Observation of SHO-patient consultation

• Feedback elicited from SHO on their performance

• Observer feedback given to SHO (written and verbal)

• Negotiation of educational contracts

• Follow up of educational contracts

At the outset of the study each SHO was given a written outline of the teaching format and an opportunity to clarify any issues with an observer. Patients were selected to provide a stimulating case mix. Consent was not requested. After each consultation feedback was elicited away from the clinical area. In particular the observers encouraged reflection, explored the SHOs' thoughts and feelings and sought alternative behaviour in consultation skills, as well as investigating clinical reason. The observer then gave verbal and written feedback using the chart illustrated in figure 1. This chart, an adaptation of the validated assessment tool used in the membership examination of the Royal College of General Practitioners focuses on the four key areas of history, examination, assessment and disposal. Subsequently each SHO was encouraged to identify clinical learning needs, and make and later complete an educational contract with the observer. Finally a copy of the feedback chart was given to the SHO so as to reinforce comments made on communication skills.

This approach to teaching is a synthesis of problem-based interviewing devised at McMaster University,²⁴ and key elements from other models.^25–27 All three observers were senior ED trainees who instructed in one or more advanced life support courses. Each had trained to give feedback in this fashion by piloting their skills on the preceding set of SHOs. Practice sessions were peer reviewed with comments invited from both SHO and fellow observer.

Common weaknesses in SHO consultation skills were identified through review of the feedback charts by all three observers. Any change in SHO behaviour was evaluated qualitatively through the process of triangulation (a combination of research methods designed to improve validity).²⁹ Firstly, an independent skilled researcher from outside the department interviewed each SHO in a semi-structured format. The question content of the interview is outlined below.

• What are your views about observed sessions?

• What do you believe the benefits of the method to be?

• Have you changed your consultation skills as a result? If so how?

• Did you make any educational contracts? How did you feel?

• Does observation lead to or improve self directed learning?

• Was there anything you disliked about the process?

• Any other comments?

Individual interviews were tape recorded and transcribed systematically by the lead author and an independent reviewer. Common responses were further agreed with the interviewer.

Secondly, a focus group facilitated by the same researcher allowed the SHOs the opportunity to comment further and reach consensus on the questions posed. Finally, an observer focus group elicited any perceptions of change in SHO behaviour.

The value of the teaching tool as perceived by both SHOs and observers was evaluated by the same methods. In addition, attitudes towards the option of videoed consultations with group feedback were explored in the SHO focus group session.

Results

Forty SHO-patient consultations were observed between March and April 1998. All nine SHOs had been in post for a month before the onset of the study. Five had just completed their house officer post, one had six months SHO experience and three were third year SHOs, two of whom had completed six month ED posts previously. Only one of the junior doctors had had communication skills training as an undergraduate and none had as postgraduates.

COMMON WEAKNESSES

The common weaknesses observed in the SHO-patient consultations are listed below.

• Use of closed questions

• The house officer clerking style

• Use of medical jargon

• Poor negotiation and explanation of treatment plan and follow up care

• Limited advice

• Limited exploration of patients thoughts and concerns

• Poor approach to paediatric consultations

Most introductions by the SHOs were appropriate in establishing the basis of good doctor-patient rapport. This was improved by eye level consultations in the majority of cases. However, there was frequent use of closed questions, particularly as the opening question of any consultation. While no cases of domestic violence or non-accidental injury were apparently missed, the potential danger of a closed question strategy was explained in the SHO feedback. Furthermore, the misleading potential of a triage note in SHO data gathering was illustrated in some of the consultations—that is, the doctor would take the presenting complaint as read and explore no further.

The five SHOs who had recently completed their pre-registration training seemed to, and declared themselves more comfortable with medical rather than trauma cases. Each was observed to run a consultation along a house officer clerking style, rather than focusing on the patient's presenting complaint. These consultations were often inappropriately lengthy.

The weakest part of most observed consultations involved the explanation and negotiation of any treatment plan and follow up. Medical jargon was often used and little use was made of any supplementary written instructions. Advice with regard to return to work/school/sport was sporadic. Patients thoughts and concerns were rarely explored— “do you have any questions?” or a similar enquiry was frequently absent.

Though only a few of the observed consultations involved children, it was apparent that establishing a rapport with them proved difficult. Failure to consult at eye level and to make use of the designated children's room were observed in some cases. The strategy for exploring the potential of a non-accidental injury could have been improved by the SHOs in all cases, for example, by gaining the child's confidence with straight forward closed questions (name, age, etc) before directing an open enquiry as to the nature of the accident to the child and not to the accompanying adult. In addition when examining children there was little instruction in the use of distraction techniques for either a parent or a named nurse.

CHANGE IN SHO BEHAVIOUR

The SHOs perceived that they had changed their practice as a result of the study. Having been encouraged to reflect on their observed consultations the SHOs had begun to analyse their consultation skills routinely. Specific examples of changed practice included regularly offering analgesia, asking the patient if there were any questions, explaining treatment plans and clear exploration and discussion of patient follow up care. Improvements in communication skills were observed as the study progressed. Many of the SHOs began to identify their own learning needs, setting educational “contracts” for themselves. None of these reached completion on inquiry.

SHO AND OBSERVER PERCEPTIONS OF THE TEACHING TOOL

Each of the SHOs felt that they had benefited from the teaching experience and they welcomed the feedback they received. In particular they valued the emphasis on the good points of their communication skills, which they felt built confidence and raised self esteem. A common comment made was that feedback was rare outside the context of the study unless an error had been made. On the observers' part each consultation generated several points for discussion, not only in terms of communication skills but also in diagnostic reasoning. In addition the SHOs were poor at identifying many of their errors in consultations.

The SHOs were generally supportive of the idea of educational contracts. “It helps to have directed learning”, was one of the comments (though the observers' aim had been to promote self directed learning). However, completion of those contracts negotiated between SHO and observer was varied, being incompatible with a busy ED schedule according to some SHOs. The observers' perception was that those SHOs keen to learn had completed their educational contracts regardless.

There was initial concern by both SHOs and observers that direct observation would prove intimidating and indeed some SHOs found the experience uncomfortable. Most felt that their performance was changed by being observed by a senior doctor, some suggesting that they were more thorough than usual, others that the senior doctor made them feel nervous. Similarly the observers' perception was that the study was not well received by some of the SHOs and that they treated it like an examination. Junior doctor anxiety however was perceived to diminish as the study progressed. All observers felt comfortable giving feedback and were able to adapt life support instructor skills to the study. “It is easy to forget what you were like as an SHO” was a salutary comment.

Other points raised by the SHOs included the suggestion that this type of teaching should be part of medical student training, and that it is particularly useful in fostering a change from the regimented clerking style used as house officers to the consultation skills required as an ED SHO.

The observers felt that the study did not make efficient use of teaching time. Each consultation and feedback episode would take up to 40 minutes, making this type of teaching impractical at times when the department was busy. As most of the weaknesses in communication skills were common to all SHOs it was proposed that videoed consultations with feedback to the group would make much better use of valuable time and allow each SHO a better opportunity to critically review their performance. The SHOs themselves felt comfortable with the concept of videoed consultation with feedback in a group setting.

Finally, should other parties use the written feedback chart, the observers recommended a modification using prompts for each quadrant. Figure 1 represents a modified chart with suggested comments for the observer, rather than the original used in the study.

Discussion

Direct observation represents a valid method of identifying weaknesses in consultation skills, as the ED SHOs are observed in their natural setting. It overcomes the discrepancy between what the SHO says (perhaps in presenting their findings to a senior colleague) or writes (as elicited in the auditing of patient records) and what they actually do. While many senior ED doctors may routinely observe or overhear in part junior doctor-patient consultations, direct observation offers a structured format for analysing and giving feedback on SHO performance. Allied to this is an approach to encouraging reflection and the setting and completion of educational contracts. Though SHO behaviour may be modified by observation—the so called Hawthorn effect²⁹—it is difficult to argue that video recording might not do the same.

The frequency of consultation errors has not been measured as the study was geared towards the provision of a stimulating non-randomised case mix. The list of common weaknesses therefore represents generalisations, and indeed each observer's subjective perceptions may have been influenced by experiences from outside the study.

The teaching seems to be effective in changing clinical behaviour. A scientifically robust methodology for measuring any change was not however selected, because such a strategy would have been complex, costly and might importantly have been considered as threatening. Though this educational tool seems to have encouraged reflection, its ability to stimulate self directed learning (as measured by the number of educational contracts set by and completed by the SHOs) seems limited.

The SHOs clearly valued the teaching, though the potential for bias in their responses is acknowledged. The potential benefits of video recording consultations with group feedback over direct observation may include increased self awareness, peer review and improved efficiency in terms of teaching time. It may however be limited by cost, technical problems (especially with regards to sound recording), vulnerability of the equipment in an ED setting and ethical considerations, particularly where patient examinations or aggressive or violent patients are concerned. Training needs for the facilitators of group feedback may also be required. In addition, SHOs may be threatened by group review¹² though this study suggested otherwise.

Alternative methods in teaching communication skills include the use of role play and tape recording of consultations as described by Dale et al.¹² In addition, standardised patients and standardised videos may be used, neither of which has been explored in the ED setting. This study would suggest a particular need for the latter in teaching communication skills for doctor-child consultations.

By aiming to increase reflection and self directed learning, direct observation and feedback has much in common with the service-based learning model for ED SHO training.³⁰ Integration within any existing teaching and appraisal programme would improve many of the goals of this teaching tool.

A further application may include the observation of ED trainee-patient consultations by a consultant or peer. A careful selection of challenging patients may maximise the benefit of direct observation in this format.

Conclusion

This study identifies common weaknesses in the consultation skills of ED SHOs and confirms the need for training in this area. Direct observation is effective in changing SHO behaviour to this end though self directed learning is not necessarily stimulated. Video recorded consultations with group feedback may be a more effective teaching tool.