Objective: The Emergency Triage Education Kit was designed to optimize consistency of triage using the Australasian Triage Scale. The present study was conducted to determine the interrater reliability of a set of scenarios for inclusion in the programme.
Methods: A postal survey of 237 paper-based triage scenarios was utilized. A quota sample of triage nurses (n = 42) rated each scenario using the Australasian Triage Scale. The scenarios were analysed for concordance and agreement. The criterion for inclusion of the scenarios in the programme was kappa >or= 0.6.
Results: Data were collected during 2 April to 14 May 2007. Agreement for the set was kappa = 0.412 (95% CI 0.410-0.415). Of the initial set: 92/237 (38.8%, 95% CI 32.6-45.3) showed concordance >or=70% to the modal triage category (kappa = 0.632, 95% CI 0.629-0.636) and 155/237 (65.4%, 95% CI 59.3-71.5) showed concordance >or=60% to the modal triage category (kappa = 0.507, 95% CI 0.504-0.510). Scenarios involving mental health and pregnancy presentations showed lower levels of agreement (kappa = 0.243, 95% CI 0.237-0.249; kappa = 0.319, 95% CI 0.310-0.328).
Conclusion: All scenarios that showed good levels of agreement have been included in the Emergency Triage Education Kit and are recommended for testing purposes; those that showed moderate agreement have been incorporated for teaching purposes. Both scenario sets are accompanied by explanatory notes that link the decision outcome to the Australasian College for Emergency Medicine Guidelines on the Implementation of the Australasian Triage Scale. Future analysis of the scenarios is required to identify how task-related factors influence consistency of triage.