Background: Non-technical skills are critical for good anaesthetic practice but are not addressed explicitly in normal training. Realization of the need to train and assess these skills is growing, but these activities must be based on properly developed skills frameworks and validated measurement tools. A prototype behavioural marker system was developed using human factors research techniques. The aim of this study was to conduct an experimental evaluation to establish its basic psychometric properties and usability.
Method: The Anaesthetists' Non-Technical Skills (ANTS) system prototype comprises four skill categories (task management, team working, situation awareness, and decision making) divided into 15 elements, each with example behaviours. To investigate its experimental validity, reliably and usability, 50 consultant anaesthetists were trained to use the ANTS system. They were asked to rate the behaviour of a target anaesthetist using the prototype system in eight videos of simulated anaesthetic scenarios. Data were collected from the ratings forms and an evaluation questionnaire.
Results: The results showed that the system is complete, and that the skills are observable and can be rated with acceptable levels of agreement and accuracy. The internal consistency of the system appeared sound, and responses regarding usability were very positive.
Conclusions: The findings of the evaluation indicated that the ANTS system has a satisfactory level of validity, reliability and usability in an experimental setting, provided users receive adequate training. It is now ready to be tested in real training environments, so that full guidelines can be developed for its integration into the anaesthetic curriculum.