Study objective: To measure interrater and intrarater agreement for an emergency department triage system.
Methods: A 2-phase experimental study was conducted using previously described in-person scripted encounters with emergency nurses who perform patient triage and attending emergency physicians at a tertiary referral center. Standardized patient scenarios were presented twice over 6 weeks. Participants rated severity for each patient using a 5-tier triage system (nurses only) and estimated the probability of hospital admission, the most appropriate time frame to physician evaluation (5 choices, from "Immediate" to "More than 24 hours"), the need for a monitored ED bed, and the need for diagnostic services. Interrater agreement was measured by a coefficient of agreement for multiple raters and multiple categories.
Results: Of the 37 participants (fewer than 90% of those eligible), 19 (51%) completed both phases (12 nurses, 7 physicians). Four (33%) of the nurses assigned the same severity ratings for the 5 cases in phase 2 as they did in phase 1. Intrarater agreement among the 12 nurses rating triage severity was.757. Interrater agreement of nurses and physicians was substantial regarding need for ED monitoring, and moderate to substantial for other triage assessments.
Conclusion: There was general agreement in interrater assessment of triage classification. Continued work is necessary to more fully delineate areas of variation.