Table 4

Detailed metrics of different machine learning (random forest) models in the training and validation sets

Machine learning modelMetricsTraining setValidation set
Model 1
  • Age

  • Gender

  • Variables retrieved by the operator-based interview

AUC0.85 (0.84 to 0.87)0.76 (0.75 to 0.77)
SENS0.96 (0.96 to 0.97)0.92 (0.91 to 0.92)
SPEC0.18 (0.17 to 0.19)0.17 (0.17 to 0.18)
ACC0.58 (0.57 to 0.59)0.75 (0.74 to 0.75)
PPV0.35 (0.34 to 0.36)0.09 (0.09 to 0.09)
NPV0.91 (0.90 to 0.93)0.96 (0.96 to 0.96)
Model 2
  • Model 1 variables

  • Clinical parameters retrieved on-scene ambulance

AUC0.92 (0.91 to 0.94)0.80 (0.79 to 0.81)
SENS0.90 (0.89 to 0.91)0.92 (0.92 to 0.93)
SPEC0.73 (0.72 to 0.74)0.23 (0.23 to 0.23)
ACC0.94 (0.93 to 0.95)0.77 (0.77 to 0.78)
PPV0.71 (0.70 to 0.72)0.1 (0.1 to 0.1)
NPV0.62 (0.60 to 0.63)0.97 (0.97 to 0.97)
Model 3
  • Model 1 variables

  • Current local SARS-CoV-2 epidemiology, and geographical distribution of EMS calls for respiratory and infectious diseases in the previous 7 days

AUC0.92 (0.91 to 0.93)0.82 (0.81 to 0.82)
SENS0.90 (0.89 to 0.91)0.91 (0.90 to 0.91)
SPEC0.70 (0.69 to 0.71)0.41 (0.41 to 0.41)
ACC0.71 (0.70 to 0.72)0.85 (0.85 to 0.85)
PPV0.57 (0.56 to 0.58)0.12 (0.12 to 0.12)
NPV0.94 (0.94 to 0.95)0.98 (0.98 to 0.98)
Model 4
All variables included in models 1–3
AUC0.94 (0.93 to 0.95)0.85 (0.84 to 0.86)
SENS0.90 (0.89 to 0.91)0.91 (0.91 to 0.92)
SPEC0.81 (0.80 to 0.82)0.44 (0.44 to 0.45)
ACC0.72 (0.71 to 0.73)0.85 (0.85 to 0.85)
PPV0.69 (0.68 to 0.71)0.13 (0.13 to 0.14)
NPV0.95 (0.94 to 0.95)0.98 (0.98 to 0.98)
  • Detailed metrics (with 95% CIs) of the random forest algorithm trained to predict the positivity to the rtPCR test. Metrics are the AUC of the ROC curve, the SENS, SPEC, ACC, PPV and NPV of a custom working point (chosen as the first point with SENS ≥90%).

  • ACC, accuracy; AUC, area under the curve; NPV, negative predictive value; PPV, positive predictive value; ROC, receiving operating characteristics; SENS, sensitivity; SPEC, specificity.