κ Coefficients (p values) for comparing each candidate’s assessments with gold standard for the 17 cases
| Candidate | Old | New |
|---|---|---|
| 1 | 0.39 (0.04) | 0.87 (<0.001) |
| 2 | −0.12 (0.58) | 0.56 (0.01) |
| 3 | 0.0 (1.0) | 0.72 (0.002) |
| 4 | 0.48 (0.05) | 0.56 (0.01) |
| 5 | 0.39 (0.04) | 0.56 (0.01) |
| 6 | 0.44 (0.06) | 0.56 (0.01) |
| 7 | 0.06 (0.79) | 0.44 (0.06) |
| 8 | 0.16 (0.48) | 0.72 (0.002) |
| 9 | 0.27 (0.21) | 0.72 (0.002) |
| 10 | 0.44 (0.06) | 0.72 (0.002) |