Article Text

other Versions

Download PDFPDF
A multicentre observational study to evaluate a new tool to assess emergency physicians' non-technical skills
  1. Lynsey Flowerdew1,
  2. Arran Gaunt2,
  3. Jessica Spedding3,
  4. Ajay Bhargava2,
  5. Ruth Brown4,
  6. Charles Vincent5,
  7. Maria Woloshynowych5
  1. 1Centre for Patient Safety and Service Quality, Imperial College London, London, UK
  2. 2Conquest Emergency Department, East Sussex Healthcare NHS Trust, Hastings, UK
  3. 3The Royal London Hospital Emergency Department, Barts and the London NHS Trust, London, UK
  4. 4St Mary's Emergency Department, Imperial College Healthcare NHS Trust, London, UK
  5. 5Centre for Patient Safety and Service Quality, Department of Surgery and Cancer, Imperial College, London, UK
  1. Correspondence to Dr Lynsey Flowerdew, Imperial College London, 5th Floor, Wright Fleming Building, St Mary's Campus, Norfolk Place, London W2 1PG, UK; l.flowerdew{at}


Objective To evaluate a new tool to assess emergency physicians' non-technical skills.

Methods This was a multicentre observational study using data collected at four emergency departments in England. A proportion of observations used paired observers to obtain data for inter-rater reliability. Data were also collected for test-retest reliability, observability of skills, mean ratings and dispersion of ratings for each skill, as well as a comparison of skill level between hospitals. Qualitative data described the range of non-technical skills exhibited by trainees and identified sources of rater error.

Results 96 assessments of 43 senior trainees were completed. At a scale level, intra-class coefficients were 0.575, 0.532 and 0.419 and using mean scores were 0.824, 0.702 and 0.519. Spearman's ρ for calculating test-retest reliability was 0.70 using mean scores. All skills were observed more than 60% of the time. The skill Maintenance of Standards received the lowest mean rating (4.8 on a nine-point scale) and the highest mean was calculated for Team Building (6.0). Two skills, Supervision & Feedback and Situational Awareness-Gathering Information, had significantly different distributions of ratings across the four hospitals (p<0.04 and 0.007, respectively), and this appeared to be related to the leadership roles of trainees.

Conclusion This study shows the performance of the assessment tool is acceptable and provides valuable information to structure the assessment and training of non-technical skills, especially in relation to leadership. The framework of skills may be used to identify areas for development in individual trainees, as well as guide other patient safety interventions.

  • Emergency medicine
  • assessment
  • leadership
  • non-technical skills
  • safety
  • training

Statistics from

Request Permissions

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.


  • Funding This study was funded by the National Institute of Health Research. They had no role in the conduct of the study or the decision to submit the paper for publication. The Clinical Safety Research Unit is affiliated with the Centre for Patient Safety and Service Quality at Imperial College Healthcare NHS Trust, which is funded by the National Institute of Health Research.

  • Competing interests None.

  • Ethics approval Ethics approval was provided by North West London REC1.

  • Provenance and peer review Not commissioned; externally peer reviewed.

  • Data sharing statement Further details of behaviours observed during the course of the study are available from the corresponding author at l.flowerdew{at}