A Bayesian model for triage decision support

doi:10.1016/j.ijmedinf.2005.07.028

International Journal of Medical Informatics

Volume 75, Issue 5, May 2006, Pages 403-411

https://doi.org/10.1016/j.ijmedinf.2005.07.028 Get rights and content

Summary

Objective

To compare triage decisions of an automated emergency department triage system with decisions made by an emergency specialist.

Methods

In a retrospective setting, data extracted from charts of 90 patients with chief complaint of non-traumatic abdominal pain were used as input for triage system and emergency medicine specialist. The final disposition and diagnoses of the physicians who visited the patient in Emergency Department (ED) as reflected in the medical records were considered as control. Results were compared by χ²-test and a binary logistic regression model.

Results

Compared to emergency medicine specialist, triage system had higher sensitivity (90% versus 64%) and lower specificity (25% versus 48%) for patients who required hospitalization. The triage system successfully predicted the Admit decisions made in the ED whereas the emergency medicine specialist decisions could not predict the ED disposition. Both triage system and emergency medicine specialist properly disposed 56% of cases, however, the emergency medicine specialist in this study under-disposed more patients than the triage system considering Admit disposition (p = 0.004) while he appropriately discharged more patients compared to the triage system (p = 0.017).

Conclusion

The triage system studied here shows promise as a triage decision support tool to be used for telephone triage and triage in the emergency departments. This technology may also be useful to the patients as a self-triage tool. However, the efficiency of this particular application of this technology is unclear.

Introduction

Triage is the task of prioritizing patients based on their medical condition with the goal of optimizing the usage of resources [1]. With increasing healthcare costs, increasing number of uninsured patients who may use Emergency Departments (ED) as a routine source of healthcare [2], [3], [4] and increasing number of overall emergency department visits, current triage systems fail to keep unnecessary emergency visits at a reasonable level. One study showed that as many as 50% of ED visits are unnecessary [5]. Although this overcrowding is attributed to a number of factors [6], [7], triaging the patients out of EDs seems one way to deal with the problem without significant adverse effects [8].

Consequently, telephone triage has become part of a pre-hospital cost-reduction demand management strategy [9], [10], [11], [12] employed by physician practice groups [13], insurance companies, MCOs, and HMOs to keep these patients out of the ED [14], [15], [16], [17], [18]. Such telephone triage hotlines are used by some HMOs and insurance companies as gatekeepers and pre-approval by a nurse is mandatory before referring to ED. Although there is some indication that such practices may effectively reduce unnecessary ED visits [14], [16], [19], [20], some studies show that this particular application is considered inappropriate by physicians and staff [21].

There is a significant difference in the triage performance of experienced versus inexperienced nurses [22]. Additionally, it is suggested that the presentation and use of decision rules used by experienced triage nurses can enhance the development of skills in novice triage nurses. Computer applications designed to assist triage personnel not only provide a convenient way of documenting the triage encounter with the patients [23], but also can assist nurses and doctors in their decision-making [24], [25], [26]. Such automated systems for documentation and decision support seem to have a positive impact on the performance of telephone triage centers [23]. Additionally, there is evidence that patients could help in documenting their medical problem [27] which makes a self-administered triage tool an intriguing idea. It has been suggested that the Internet increases self-care, manages demand for health services, lowers direct medical costs, and if integrated with telephone-based triage lines holds the promise of reducing unnecessary medical services [28], [29].

We used Bayesian network technology to develop a decision support system for triage that could be operated by nurses for in-person triage or telephone triage. In this paper, triage decisions suggested by one of the algorithms developed for this system, namely abdominal pain, is compared to the triage decisions of an emergency medicine specialist retrospectively based on patient charts pulled from the Emergency Department of a large teaching hospital.

Section snippets

Study design

This is a retrospective study to compare triage decisions of an automated emergency department triage system with decisions made by an emergency specialist in patients referred to the emergency department with the chief complaint of non-traumatic abdominal pain. Study protocol was approved by the Institutional Review Board (IRB) at the University of Texas Health Science Center at Houston. Outcome of the study was the final disposition made by either the automated triage system or the emergency

Results

Ninety patients were included in final analysis; mean age of patients was 40.3 ± 18.9 years (range: 16–90 years). Seventy three per cent of patients were female (66 of 90). Descriptive statistics of patients’ dispositions are shown in Fig. 2, Fig. 3. Appropriateness of disposition as described above and in Table 1, is depicted in Fig. 4. As for the Admit disposition, triage system (TS) had higher sensitivity and lower specificity compared to emergency medicine specialist (ES) — sensitivity of 90%

Discussion

Based on this study the triage system performs better than the emergency medicine specialist. Additionally, the triage system has a higher sensitivity than the physician (90% versus 64%) and a lower level of specificity compared with the physician (25% versus 48%). Considering the goals of triage which include not missing a patient that is in need of urgent medical care, even if the physician was successful in predicting the ED Admit disposition, the sensitivity and specificity of the triage

Conclusion

Acknowledgement

Authors wish to thank Dr. James H “Red” Duke, MD and Dr. S. Ward Casscells, MD of University of Texas Health Science Center at Houston, School of Medicine for their original idea of self-triage using artificial intelligence. Authors also thank Dr. Craig W. Johnson, PhD of University of Texas Health Science Center at Houston, School of Health Information Sciences, for his contributions to the statistical analysis of the data.

References (58)

J. Cioffi
Triage decision-making: educational strategies
Accid. Emerg. Nurs.
(1999)
G.A. Pane et al.
Health care access problems of medically indigent emergency department walk-in patients
Ann. Emerg. Med.
(1991)
J. Koziol-McLain
Seeking care for nonurgent medical conditions in the emergency department: through the eyes of the patient
J. Emerg. Nurs.
(2000)
R.W. Derlet
Triage of patients out of the emergency department: three-year experience
Am. J. Emerg. Med.
(1992)
J. Cioffi
Decision-making by emergency nurses in triage assessments
Accid. Emerg. Nurs.
(1998)
A. Qamar
The Goldman algorithm revisited: prospective evaluation of a computer-derived algorithm versus unaided physician judgment in suspected acute myocardial infarction
Am. Heart J.
(1999)
W.C. Kinney
Web-based clinical decision support system for triage of vestibular patients
Otolaryngol. Head Neck Surg.
(2003)
W.W. Cheung et al.
An advance triage system
Accid. Emerg. Nurs.
(2002)
J. Nakagawa
Interobserver agreement in emergency department triage
Ann. Emerg. Med.
(2003)
J.M. Gill et al.
Disagreement among health care professionals about the urgent care needs of emergency department patients
Ann. Emerg. Med.
(1996)

D.A. Wachter

Pediatric telephone triage protocols: standardized decision-making or a false sense of security?

Ann. Emerg. Med.

(1999)

J.C. Brillman

Triage: limitations in predicting need for emergent care and hospital admission

Ann. Emerg. Med.

(1996)

M.K. Marks et al.

Inappropriate use of hospital emergency departments

Med. J. Aust.

(2003)

S.M. Schappert

National Hospital Ambulatory Medical Care Survey: 1992 Emergency Department summary

Vital Health Stat.

(1997)

S. Schneider

Rochester, New York a decade of emergency department overcrowding

Acad. Emerg. Med.

(2001)

R.B. Baer et al.

Recently discharged inpatients as a source of emergency department overcrowding

Acad. Emerg. Med.

(2001)

M.E. Greenberg

Telephone nursing: evidence of client and organizational benefits

Nurs. Econ.

(2000)

E. Rosenblatt

Telephone triage. A common sense approach

RN

(2001)

M. Sabin

Telephone triage improves demand management effectiveness

Healthc. Financ. Manage.

(1998)

W.M. Post

FreeCall, a system for emergency-call-handling support

Meth. Inf. Med.

(1996)

J.P. O’Connor

Nurse phone care — a new way of thinking (Part 2)

Med. Group Manage. J.

(1996)

E. McGraw et al.

A model for demand management in a managed care environment

Mil. Med.

(2000)

J.M. Kastens

Integrated care management: aligning medical call centers and nurse triage services

Nurs. Econ.

(1998)

DM nurse line creates ‘win–win’ scenario for point-of-service, product

Healthc. Demand Dis. Manag.

(1997)

J.A. Lester et al.

Nurse triage telephone centers: key to demand-management strategy

NAHAM Manage. J.

(1996)

B.W. Wolcott

Managed care's driving force: demand management

Infocare

(1996)

S.M. Franco et al.

Primary care physician access and gatekeeping: a key to reducing emergency department use

Clin. Pediatr. (Phila)

(1997)

J.W. Barber

Evaluation of emergency department referrals by telephone triage

Pediatrics

(2000)

D. Glotzer

Prior approval in the pediatric emergency room

Pediatrics

(1991)

Cited by (49)

Current review and next steps for artificial intelligence in multiple sclerosis risk research
2021, Computers in Biology and Medicine
Citation Excerpt :
It is increasingly widely recognized within the AI community that graphical causal models [3] provide a powerful way of understanding the limitations of observational data and enhancing this data with causal knowledge. A particularly powerful graphical causal model – the Bayesian network –which combines expert causal knowledge with Bayesian probability theory has been used to improve risk assessment in a range of medical conditions [4–8]. We propose a causal Bayesian network approach to supplement classical data-driven epidemiological methods.
In the last few decades, the prevalence of multiple sclerosis (MS), a chronic inflammatory disease of the nervous system, has increased, particularly in Northern European countries, the United States, and United Kingdom. The promise of artificial intelligence (AI) and machine learning (ML) as tools to address problems in MS research has attracted increasing interest in these methods. Bayesian networks offer a clear advantage since they can integrate data and causal knowledge allowing for visualizing interactions between dependent variables and potential confounding factors. A review of AI/ML research methods applied to MS found 216 papers using terms “Multiple Sclerosis”, “machine learning”, “artificial intelligence”, “Bayes”, and “Bayesian”, of which 90 were relevant and recently published. More than half of these involve the detection and segmentation of MS lesions for quantitative analysis; however clinical and lifestyle risk factor assessment and prediction have largely been ignored. Of those that address risk factors, most provide only association studies for some factors and often fail to include the potential impact of confounding factors and bias (especially where these have causal explanations) that could affect data interpretation, such as reporting quality and medical care access in various countries. To address these gaps in the literature, we propose a causal Bayesian network approach to assessing risk factors for MS, which can address deficiencies in current epidemiological methods of producing risk measurements and makes better use of observational data.
A Bayesian Model to Predict Survival After Left Ventricular Assist Device Implantation
2018, JACC: Heart Failure
This study investigates the use of a Bayesian statistical models to predict survival at various time points in patients undergoing left ventricular assist device (LVAD) implantation.
LVADs are being increasingly used in patients with end-stage heart failure. Appropriate patient selection continues to be key in optimizing post-LVAD outcomes.
Data used for this study were derived from 10,277 adult patients from the INTERMACS (Inter-Agency Registry for Mechanically Assisted Circulatory Support) who had a primary LVAD implanted between January 2012 and December 2015. Risk for mortality was calculated retrospectively for various time points (1, 3, and 12 months) after LVAD implantation, using multiple pre-implantation variables. For each of these endpoints, a separate tree-augmented naïve Bayes model was constructed using the most predictive variables.
A set of 29, 26, and 31 pre-LVAD variables were found to be predictive at 1, 3, and 12 months, respectively. Predictors of 1-month mortality included low Inter-Agency Registry for Mechanically Assisted Circulatory Support profile, number of acute events in the 48 h before surgery, temporary mechanical circulatory support, and renal and hepatic dysfunction. Variables predicting 12-month mortality included advanced age, frailty, device strategy, and chronic renal disease. The accuracy of all Bayesian models was between 76% and 87%, with an area under the receiver operative characteristics curve of between 0.70 and 0.71.
A Bayesian prognostic model for predicting survival based on the comprehensive INTERMACS registry provided highly accurate predictions of mortality based on pre-operative variables. These models may facilitate clinical decision-making while screening candidates for LVAD therapy.
Automatic learning of mortality in a CPN model of the systemic inflammatory response syndrome
2017, Mathematical Biosciences
Citation Excerpt :
The Sepsis CPN assesses the severity of illness based on the degree of the inflammatory response indicated by a set of “infection variables”. CPNs can be used to construct diagnostic models for diseases [10–13], and are ideal for this purpose due to their ability to combine knowledge, represented by patient databases, expert opinion and reports in the literature, with strict reasoning adhering to the axioms of probability theory. Construction of a CPN can be manual, or automatic where the latter refers to the use of machine learning techniques.
The aim of this paper is to apply machine learning as a method to refine a manually constructed CPN for the assessment of the severity of the systemic inflammatory response syndrome (SIRS).The goal of tuning the CPN is to create a scoring system that uses only objective data, compares favourably with other severity-scoring systems and differentiates between sepsis and non-infectious SIRS. The resulting model, the Learned-Age (L_A) -Sepsis CPN has good discriminatory ability for the prediction of 30-day mortality with an area under the ROC curve of 0.79. This result compares well to existing scoring systems. The L_A-Sepsis CPN also has a modest ability to discriminate between sepsis and non-infectious SIRS.
Predicting hospital admissions to reduce emergency department boarding
2016, International Journal of Production Economics
Citation Excerpt :
Existing studies vary in target groups of patients, objectives, and methods. Some studies focused on a particular group of ED patients (Sadeghi et al., 2006; Considine et al., 2011), while others considered all ED encounters. Study objectives included identifying important factors in admission (Considine et al., 2011), identifying high-risk patients for admission (Ruger et al., 2007), developing hospital admission prediction models (Leegon et al., 2006; Li et al., 2009), and estimating the total number of ED-to-inpatient-unit admissions (Peck et al., 2012).
Recent research has established that Emergency Department (ED) congestion is often caused by the inability to transition patients into inpatient units within the hospital in a timely fashion. This problem, in which the ED boards inpatients, is common across the U.S. Predicting ED patient admission using demographic and clinical information with only a few admission predictor factors investigated so far. We have developed a prediction model that can be used as a decision support tool and help reduce ED boarding. Using secondary data from the ED of a local hospital, we have examined the importance of eight demographic and clinical determinant factors of ED patients' admission to the hospital. We have employed Logistic Regression (LR) and Neural Network (NN) modeling techniques and based on our statistical analysis, we have identified encounter reason, age, and radiology exam type as the most significant factors. We have studied patterns between input variables (i.e. age) and output variables (i.e. admitted or not) and have developed a set of rules of thumb for predicting admissions. These unique rules can be used without any modeling or further investigation during operations, therefore providing important information regarding the ultimate status of a patient after ED operations without any time or cost. The study proves that an admission prediction model based on demographic and clinical determinant factors can accurately estimate the likelihood of patient admission, thus decreasing ED boarding and congestion, both significant problems in hospital operations.
Design and evaluation of a decision support system for pain management based on data imputation and statistical models
2016, Measurement: Journal of the International Measurement Confederation
The self-reporting of pain complaints is considered the most accurate pain assessment method and represents a valuable source of data to computerised clinical decision support systems (CCDSS) for pain management. However, the subjectivity and variability of pain conditions, combined with missing data, are constraints on the usefulness and accuracy of CCDSS. Based on data imputation principles, together with several mathematical models, this paper presents a CCDSS, the Patient Oriented Method of Pain Evaluation System (POMPES), that produces tailored alarms, reports, and clinical guidance based on collected patient-reported data. This system was tested using clinical data collected during a six-week randomised controlled trial involving thirty-two volunteers recruited from an ambulatory surgery department. The decisions resulting from the POMPES were fully accurate when compared with clinical advice, which proves the ability of the system to cope with missing data and detect either stability or changes in the self-reporting of pain.
A Bayesian approach to model-development: Automatic learning for tuning predictive performance
2015, IFAC-PapersOnLine
The value of manually constructed and tuned Bayesian networks has been demonstrated empirically, however this informal process is limited in terms of what can be reasonably achieved. This paper presents the application of a formal machine learning process, EM learning, to a manually constructed CPN for the assessment of the severity of sepsis. Through learning, the model is tuned to predict 30-day mortality, and displays a significant improvement in discriminatory ability assessed by area under the ROC curve (previous model AUC = 0.647, new model AUC = 0.739, p<0.001).

View all citing articles on Scopus

: Parts of this paper – descriptive statistics and not analytical statistics – were presented at “Medicine Meets Virtual Reality 2002 Meeting (MMVR02)” in Newport Beach, CA in January 2002.

View full text

A Bayesian model for triage decision support

Summary

Objective

Methods

Results

Conclusion

Introduction

Section snippets

Study design

Results

Discussion

Conclusion

Acknowledgement

Accid. Emerg. Nurs.

Ann. Emerg. Med.

J. Emerg. Nurs.

Am. J. Emerg. Med.

Accid. Emerg. Nurs.

Am. Heart J.

Otolaryngol. Head Neck Surg.

Accid. Emerg. Nurs.

Ann. Emerg. Med.

Ann. Emerg. Med.

Ann. Emerg. Med.

Ann. Emerg. Med.

Inappropriate use of hospital emergency departments

Med. J. Aust.

National Hospital Ambulatory Medical Care Survey: 1992 Emergency Department summary

Vital Health Stat.

Rochester, New York a decade of emergency department overcrowding

Acad. Emerg. Med.

Recently discharged inpatients as a source of emergency department overcrowding

Acad. Emerg. Med.

Telephone nursing: evidence of client and organizational benefits

Nurs. Econ.

Telephone triage. A common sense approach

RN

Telephone triage improves demand management effectiveness

Healthc. Financ. Manage.

FreeCall, a system for emergency-call-handling support

Meth. Inf. Med.

Nurse phone care — a new way of thinking (Part 2)

Med. Group Manage. J.

A model for demand management in a managed care environment

Mil. Med.

Integrated care management: aligning medical call centers and nurse triage services

Nurs. Econ.

DM nurse line creates ‘win–win’ scenario for point-of-service, product

Healthc. Demand Dis. Manag.

Nurse triage telephone centers: key to demand-management strategy

NAHAM Manage. J.

Managed care's driving force: demand management

Infocare

Primary care physician access and gatekeeping: a key to reducing emergency department use

Clin. Pediatr. (Phila)

Evaluation of emergency department referrals by telephone triage

Pediatrics

Prior approval in the pediatric emergency room

Pediatrics