Statistics from Altmetric.com
Graber’s article raises several valid points about the provision of diagnostic decision support in the Emergency Department (ED).1 The ED is one setting where reaching the correct diagnosis (for simple clinical problems as well as unusual ones) may reduce the burden of diagnostic error and its costly adverse consequences.2
In Graber’s study, QMR and ILIAD were tested for their diagnostic accuracy with the limited amount of data available at initial clinical presentation; quite rightly, the authors used the final diagnosis at discharge from ED as the gold standard. However, this testing was not performed by the lay user, and the systems were provided detailed clinical information derived from multiple physicians’ assessment - conditions that may not be satisfied in real life usage. Despite this, the systems do not appear to be very useful: in an individual case, their accuracy (compared to an ED physician) is less than 50%; even if it were 100%, would a user be able to select the correct diagnosis from the 20–30 diagnoses offered; and even if they could, do ED physicians have the time to spend 20–40 minutes with these systems for each patient? The authors rightly conclude that a diagnostic ‘reminder’ system, rather than a diagnostic oracle, might serve ED physicians better, a conclusion that has been confirmed in previous studies of diagnostic decision support.3
We have been involved in the development and validation of a diagnostic reminder system, called ISABEL (http://www.isabel.org.uk). It was developed by a UK medical charity after a 3 year old child suffered a mis-diagnosis in ED.4,5 We have circumvented many of the criticisms that Graber et al raise about “expert systems” by utilising 4 standard, widely accepted textbooks as the knowledge base, which are searched by a powerful software (Autonomy) that uses advanced textual pattern recognition techniques to identify candidate diagnoses based on clinical features entered by users in free text. Only 10–12 diagnoses are offered, arranged in broad headings of causation (Toxicology, Cardiology etc.) rather than in order of likelihood. Further information on each diagnosis is available as text from the textbook.
Demanding, time-pressured ED physicians will be interested by the fact that ISABEL displayed the final ED diagnosis >85% of the time, when tested against a sample of 100 children;6 all the diagnoses considered to be important in the diagnostic workup of these patients were displayed by ISABEL in 73% cases; and it took less than 2 minutes for lay users to enter clinical features in free text and generate meaningful results. Testing the impact of such a system with real clinicians in a laboratory setting suggested that in 1 out of every 7 consultations, they were reminded a ‘significant’ diagnosis that would otherwise have been missed. Similar results have been replicated in real life in a recent multi-centre study in 4 UK paediatric EDs (awaiting publication). The ISABEL model is also currently covers adult as well as paediatric comditions, as well as many specialties.
It seems reasonable to conclude that in the context of an ED, systems that deliver rapid, practical and easy-to-use diagnostic reminders might prove more useful than “expert problem-solver” systems that may provide accurate results, but following lengthy interaction.