Introduction: The study objectives were to compare reliability and validity of a 3-level (3L) triage system with a new 5-level (5L) triage system and determine the effect of nursing experience on triage reliability.
Methods: The study was conducted in a southeastern tertiary emergency department. With a stratified random sample, reliability of 3L triage ratings was measured with weighted kappa (time 1). The 5L system was then implemented, and weighted kappa was remeasured (time 2). Validity was assessed by comparing case mix, sensitivity, and specificity at times 1 and 2, and comparing 5L ratings with physician billing (Evaluation and Management) codes and nursing resource intensity at time 2.
Results: Time 1 case mix (15,324 patients) was: level 1, 6%; level 2, 36%; level 3, 59%, and time 2 (16,024 patients) was: level 1, 1%; level 2, 8%; level 3, 38%; level 4, 41%; level 5, 13%. Three hundred-five triage ratings were evaluated from time 1, and 303 were evaluated from time 2. Weighted kappa was 0.53 for time 1 and 0.68 for time 2. Spearman correlations were: 5L and nursing resource intensity, 0.55 (P <.0001); and 5L and Em, 0.57 (P <.0001). Sensitivity was 58% for the 3L and 68% for the 5L. Specificity was 83% for the 3L and 91% for the 5L. Under-triage rates were 28% for the 3L and 12% for the 5L, and less-experienced nurses were more likely to under-triage using the 3L system.
Discussion: The 5L triage system is safer and provides greater discrimination, better reliability, and improved sensitivity and specificity than the 3L triage system.