Objectives To measure the reliability and predictive validity of a four-level triage system (I-4L).

Methods This observational study was conducted in an urban hospital. Five nurses were randomly selected to assign a triage level to 246 paper scenarios, using the I-4L model. The I-4L model is a four-level triage system: urgency category (UC) 1 requires immediate response; UCs 2, 3 and 4 require assessment within 20, 60 and 120 min, respectively. Weighted κ statistics were used to measure the inter-rater and intrarater reliability of the triage tool and the validity of the model was assessed based on the accuracy in predicting admission and in predicting a reference standard's triage code.

Results The I-4L model's inter-rater reliability was κ=0.73 (95% CI 0.67 to 0.79), and the intrarater reliability was κ=0.82 (95% CI 0.67 to 0.96). Its accuracy of triage rating for admission and for prediction of a reference standard's triage code was good: 79% (95% CI 73% to 86%) and 93% (95% CI 89% to 96%), respectively. The percentages of patients admitted per triage level using the I-4L model was: 100% UC 1; 42% UC 2; 6% UC 3; and 2% UC 4.

Conclusions The I-4L triage model shows a good inter-rater and intrarater reliability for rating triage acuity and for accuracy in patient admission and prediction of a reference standard's triage code.