Summary of SA expert's agreement with the Delphi expert's rating of vignettes, % (CIs)
| Vignettes (n) | Triage ratings (n) | 2 SA experts combined for 42 vignettes (a total of 84 ratings) | Performance relative to Delphi experts as the objective reference standard | |||||||
|---|---|---|---|---|---|---|---|---|---|---|
| Emergency | Very urgent | Urgent | Routine | Sensitivity | Specificity | Over-triage | Under-triage | |||
| Delphi panel triage category | ||||||||||
| Emergency | 9 | 18 | 44.4* | 44.4† | 11.1 | 0.0 | 44.4 (CI 24 to 66) | 86.8 (CI 77 to 93) | n/a | 55.6 (CI 34 to 76) |
| Very urgent | 17 | 34 | 0.0† | 94.1* | 5.9† | 0.0 | 94.1 (CI 83 to 99) | 94.7 (CI 85 to 98) | 0.0 | 5.9 (CI 1 to 17) |
| Urgent | 10 | 20 | 0.0 | 30.0† | 70.0* | 0.0† | 70.0 (CI 49 to 86) | 90.3 (CI 82 to 96) | 30.0 (CI 14 to 51) | 0.0 |
| Routine | 6 | 12 | 0.0 | 0.0 | 33.3† | 66.7* | 66.7 (CI 40 to 87) | 94.7 (CI 88 to 98) | 33.3 (CI 13 to 60) | n/a |
| Mean | 73.8 (CI 64 to 82) | 91.7 (CI 87 to 94) | 11.9 (CI 6 to 20) | 14.3 (CI 8 to 23) | ||||||
*These values correspond to exact agreement with Delphi expert categories for vignettes.
†These values correspond to agreement within one level of the Delphi expert category.
n/a, not applicable.