Objectives To validate and assess reliability of a novel triage system, one-two-triage (OTT), that can be applied by inexperienced providers in low-resource settings.

Methods This study was a two-phase prospective, comparative study conducted at three hospitals. Phase I assessed criterion validity of OTT on all patients arriving at an American university hospital by comparing agreement among three methods of triage: OTT, Emergency Severity Index (ESI) and physician-defined acuity (the gold standard). Agreement was reported in normalised and raw-weighted Cohen κ using two different scales for weighting, Expert-weighted and triage-weighted κ. Phase II tested reliability, reported in Fleiss κ, of OTT using standardised cases among three groups of providers at an urban and rural Cambodian hospital and the American university hospital.

Results Normalised for prevalence of patients in each category, OTT and ESI performed similarly well for expert-weighted κ (OTT κ=0.58, 95% CI 0.52 to 0.65; ESI κ=0.47, 95% CI 0.40 to 0.53) and triage-weighted κ (κ=0.54, 95% CI 0.48 to 0.61; ESI κ=0.57, 95% CI 0.51 to 0.64). Without normalising, agreement with gold standard was less for both systems but performance of OTT and ESI remained similar, expert-weighted (OTT κ=0.57, 95% CI 0.52 to 0.62; ESI κ=0.6, 95% CI 0.58 to 0.66) and triage-weighted (OTT κ=0.31, 95% CI 0.25 to 0.38; ESI κ=0.41, 95% CI 0.35 to 0.4). In the reliability phase, all triagers showed fair inter-rater agreement, Fleiss κ (κ=0.308).

Conclusions OTT can be reliably applied and performs as well as ESI compared with gold standard, but requires fewer resources and less experience.