You are here

Article Text

Article menu
Download PDFPDF
RCEM Annual Scientific Conference 2024 Abstracts
RCEM Lightning Papers
2772 AI assisted reader evaluation in CT head interpretation (AI-REACT): results from a multi-case multi-reader study
  1. Alex Novak1,
  2. Abdala Espinosa2,
  3. Kanika Bhatia3,
  4. Andrea Romsauerova3,
  5. Tilak Das4,
  6. Mariapaola Narbone5,
  7. Rahul Dharmadhikari6,
  8. David Lowe7,
  9. Haris Shuaib5,
  10. Sarim Ather3
  1. 1Emergency Medicine Research Oxford (EMROx), Oxford University Hospitals NHS Foundation Trust, Oxford
  2. 2Emergency Medicine Research (EMROx), Oxford University Hospitals NHS Foundation Trust
  3. 3Oxford University Hospitals NHS Foundation Trust
  4. 4Department of Clinical Radiology, Cambridge University Hospitals NHS Foundation Trust, Cambridge
  5. 5Guy’s and St Thomas’ Hospitals NHS Trust, London
  6. 6Northumbria Healthcare NHS Foundation Trust, Northumberland
  7. 7NHS Greater Glasgow and Clyde, Glasgow

Abstract

Aims and Objectives A non-contrast CT head scan (NCCTH) is the most common cross-sectional imaging investigation requested in the Emergency Department (ED), and several Artificial Intelligence (AI) tools have been developed to detect abnormalities on NCCTH, however there is currently little real-world evidence to support adoption. The AI-REACT study (IRAS 310995, NCT05427838) evaluated the impact of AI algorithm on the diagnostic performance of ED clinicians, radiologists and radiographers.

Method and Design A retrospective dataset of 150 NCCTH was compiled, including 63 normal control cases and 97 abnormal cases containing intracranial haemorrhage (ICH), infarct, midline shift, mass effect, or skull fracture. 30 readers of varying experience were recruited across four NHS trusts including 10 general radiologists, 15 Emergency Medicine clinicians, and five CT radiographers. Readers interpreted each scan first without, then with, the assistance of the qER EU 2.0 AI tool, with an intervening 2-week washout period. Using an arbitrated consensus opinion of 2 neuroradiologists as ground truth, the stand-alone performance of qER was assessed, and its impact on the readers’ diagnostic performance analysed.

Results and Conclusion Pooled analyses demonstrated a significant increase in reader sensitivity for abnormal scans (0.828 to 0.897, +0.069, 95%CI +0.106 to +0.0136, p >0.001) and ICH (0.846 to 0.916, +0.07 95%CI 0.108 to 0.0321 p = >0.001). ED clinicians with AI assistance demonstrated a sensitivity of 0.879 (abnormality) and 0.948 (ICH) compared to unaided radiologist sensitivity 0.890 (abnormality) and 0.939 (ICH), with no statistically significant changes in specificity.

Use of AI-assisted image interpretation led to a significant increase in the ability of ED clinicians to accurately identify abnormality and ICH on CT Head scans, to a level comparable to that of radiologists. These important findings should be fully explored prospectively. Further analysis of the effects on pathology and reader subgroups will help to identify potential strengths and use cases for this application.

https://doi.org/10.1136/emermed-2024-RCEM.26

Statistics from Altmetric.com

    Request Permissions

    If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.