Derivation and Validation of a Bayesian Network to Predict Pretest Probability of Venous Thromboembolism

doi:10.1016/j.annemergmed.2004.08.036

Annals of Emergency Medicine

Volume 45, Issue 3, March 2005, Pages 282-290

https://doi.org/10.1016/j.annemergmed.2004.08.036 Get rights and content

Study objective

A Bayesian network can estimate a numeric pretest probability of venous thromboembolism on the basis of values of clinical variables. We determine the accuracy with which a Bayesian network can identify patients with a low pretest probability of venous thromboembolism, defined as less than or equal to 2%.

Methods

Using commercial software, we derived a population of Bayesian networks from 25 input variables collected on 3,145 emergency department (ED) patients with suspected venous thromboembolism who underwent standardized testing, including pulmonary vascular imaging, and 90-day follow-up (11.0% of patients were venous thromboembolism positive). The best-fit Bayesian network was selected using a genetic algorithm. The selected Bayesian network was tested in a validation population of 1,423 ED patients prospectively evaluated for venous thromboembolism, including 90-day follow-up (8.0% were venous thromboembolism positive). The Bayesian network probability estimate was normalized to a score of 0% to 100%.

Results

Of 1,423 patients in the validation cohort, 711 (50%; 95% confidence interval [CI] 47% to 52%) had a score less than or equal to 2% that predicted a low pretest probability. Of these 711 patients, 700 (98.5%; 95% CI 97.2% to 99.2%) had no venous thromboembolism at follow-up.

Conclusion

A Bayesian network, derived and independently validated in ED populations, identified half of the validation cohort as having a low pretest probability (≤2%); 98.5% of these patients were correctly classified by the network.

Introduction

Prior probability plays a central role in decisionmaking for the diagnostic evaluation of venous thromboembolism.1, 2 Published studies of pretest probability for venous thromboembolism have focused mainly on scoring systems derived from logistic regression analysis or a point estimate from the solution to the logistic regression equation.3, 4, 5 Logistic regression has several methodologic shortcomings, including collinearity and overestimation of importance of uncommon covariates.⁶ Moreover, scoring systems cannot provide a percentage point estimate of pretest probability, which is necessary to compute the percentage point estimate of posttest probability after the results of diagnostic testing are revealed.⁷ Although the antilog transformation of a logistic regression can be solved for a point estimate of probability, most solved logistic equations rarely output a probability in the 0% to 5% range.⁸ Within this 0% to 5% pretest probability range, most physicians formulate the belief that testing for venous thromboembolism is warranted. A useful decision-support system must accurately report a point estimate of pretest probability in the 0% to 2% range, because in this range physicians can justify not testing for venous thromboembolism.⁹

The theoretical utility of Bayesian network analysis has been described for the diagnosis of venous thromboembolism.¹⁰ Briefly, Bayes' theorem brings quantitative reasoning to the bedside. For the condition of venous thromboembolism, the clinician approaches the patient and formulates a nebulous, unstructured belief about the possibility of venous thromboembolism. However, at the same time, multiple alternative hypotheses compete for that belief. The clinician then gathers information from the medical history and physical examination that either strengthens or weakens the belief about the diagnosis of venous thromboembolism. Bayes' theorem quantifies the degree of belief adjustment resulting from these observations. Figure 1 illustrates how 1 observation (the pulse oximetry reading) can modify the previous probability of venous thromboembolism.

We hypothesized that when the Bayesian network outputs a normalized score less than or equal to 2, the actual pretest probability (or prevalence) of venous thromboembolism would be 2% or less (≤2%). Thus, a score of less than or equal to 2 represents the equivalence of a “test negative” result—a pretest probability that could support the decision not to order additional diagnostic testing for venous thromboembolism.⁸

Section snippets

Materials and methods

In this report, we use a custom data-mining tool that uses a genetic algorithm to search all possible Bayesian networks that can be developed from 25 clinical variables collected from the history and physical examinations of 3,145 emergency department (ED) patients who were evaluated for venous thromboembolism. In accordance with methodology used by other experts, the term “venous thromboembolism” refers to a patient evaluated for clinically suspected pulmonary embolism and who was then treated

Results

The clinical characteristics of the derivation and validation population are shown in Table 1. This table shows the mean values for continuous variables (age and vital signs) and the frequencies of dichotomous nodes in both populations.

From the derivation population, the variables in Table 1 served as the input nodes used to construct a large number of potential Bayesian networks. Each candidate network was then tested for fitness using the genetic algorithm. Specifically, the genetic algorithm

Limitations

On the basis of conventional grading of decision-rule validity, this report provides only level III evidence.¹⁸ The data show that the derived Bayesian network can support the decision to rule out venous thromboembolism only in a patient population with a low disease prevalence. In the derivation population (prevalence 11%), 2.0% of patients with a score less than or equal to 2% had a venous thromboembolism. This outcome rate overlapped the computed test threshold for venous thromboembolism of

Discussion

Pretest probability assessment refers to a complex process of predicting the probability of an unknown feature—a disease—according to known features, such as signs and symptoms. In this report, we derive and test a Bayesian network to predict the pretest probability of venous thromboembolism using data obtained with a standard medical history and physical examination. This report represents the first step toward developing a Bayesian network tool to transform clinical data at the bedside into a

Acknowledgements

We thank Francis Fesmire, MD, for his contribution to this work.

The following institutions participated in the derivation study: Elizabeth G. Israel, MD, Barnes Hospital, St. Louis, MO; Christopher Kabrhel, MD, Carolinas Medical Center, Charlotte, NC; Jeffrey A. Kline, MD, Detroit Receiving Hospital, Detroit, MI; Brian J. O'Neil, MD, Henry Ford Hospital, Detroit, MI; David C. Portelli, MD, Mayo Clinic, Phoenix, AZ; Peter B. Richman, MD, Northwestern Memorial Hospital, Feinberg School of

References (23)

J. Kelly et al.
The utility of pretest probability assessment in patients with clinically suspected venous thromboembolism
J Thromb Haemost
(2003)
M. Miniati et al.
A structured clinical model for predicting the probability of pulmonary embolism
Am J Med
(2003)
E.J. Gallagher
Clinical utility of likelihood ratios
Ann Emerg Med
(1998)
J.A. Kline et al.
Clinical criteria to prevent unnecessary diagnostic testing in emergency department patients with suspected pulmonary embolism
J Thromb Haemost
(2004)
D. Luciani et al.
The role of Bayesian networks in the diagnosis of pulmonary embolism
J Thromb Haemost
(2003)
P.B. Richman et al.
Contribution of indirect computed tomography venography to computed tomography angiography of the chest for the diagnosis of thromboembolic disease in two United States emergency departments
J Thromb Haemost
(2003)
J.A. Kline et al.
Impact of a rapid rule-out protocol for pulmonary embolism on the rate of screening, missed cases, and pulmonary vascular imaging in an urban US emergency department
Ann Emerg Med
(2004)
J.A. Kline et al.
Criteria for the safe use of d-dimer testing in emergency department patients with suspected pulmonary embolism: a multicenter United States study
Ann Emerg Med
(2002)
S.D. Chunilal et al.
Does this patient have pulmonary embolism?
JAMA
(2003)
J. Wicki et al.
Assessing clinical probability of pulmonary embolism in the emergency ward: a simple score
Arch Intern Med
(2001)

P.S. Wells et al.

Derivation of a simple clinical model to categorize patients' probability of pulmonary embolism: increasing the models utility with the SimpliRED d-dimer

Thromb Haemost

(2000)

Cited by (32)

Medical idioms for clinical Bayesian network development
2020, Journal of Biomedical Informatics
Citation Excerpt :
Building a BN involves two main tasks [6]: (1) determine the BN structure; and (2) specify the parameters. Both the BN structure and parameters can be built: (a) by automated learning from data if sufficient data are available [7,8,9]; (b) “by-hand” using knowledge elicitation methods to capture domain expert knowledge and extract necessary information from the literature [1,10,11,12]; or, (c) through a combination of both [13,14]. In many medical problems it is not feasible or appropriate to use automated techniques and the structure and/or parameters of the BN must be elicited from experts.
Bayesian Networks (BNs) are graphical probabilistic models that have proven popular in medical applications. While numerous medical BNs have been published, most are presented fait accompli without explanation of how the network structure was developed or justification of why it represents the correct structure for the given medical application. This means that the process of building medical BNs from experts is typically ad hoc and offers little opportunity for methodological improvement. This paper proposes generally applicable and reusable medical reasoning patterns to aid those developing medical BNs. The proposed method complements and extends the idiom-based approach introduced by Neil, Fenton, and Nielsen in 2000. We propose instances of their generic idioms that are specific to medical BNs. We refer to the proposed medical reasoning patterns as medical idioms. In addition, we extend the use of idioms to represent interventional and counterfactual reasoning. We believe that the proposed medical idioms are logical reasoning patterns that can be combined, reused and applied generically to help develop medical BNs. All proposed medical idioms have been illustrated using medical examples on coronary artery disease. The method has also been applied to other ongoing BNs being developed with medical experts. Finally, we show that applying the proposed medical idioms to published BN models results in models with a clearer structure.
A review on evolutionary algorithms in Bayesian network learning and inference tasks
2013, Information Sciences
Citation Excerpt :
Zhu et al. [140] proposed a MB-embedded GA for gene selection in microarray datasets and showed that using GA to search for the MB of the class variable results in higher classification accuracy. Kline et al. [73] showed the use of GAs to search for the most accurate BN structure to predict venous thromboembolism according to gathered data. Also in the field of BN classification, Peña [110] applied UMDA to search for the optimal dependency structure between predictor variables in unsupervised learning using a specific representation of BNs.
Thanks to their inherent properties, probabilistic graphical models are one of the prime candidates for machine learning and decision making tasks especially in uncertain domains. Their capabilities, like representation, inference and learning, if used effectively, can greatly help to build intelligent systems that are able to act accordingly in different problem domains. Bayesian networks are one of the most widely used class of these models. Some of the inference and learning tasks in Bayesian networks involve complex optimization problems that require the use of meta-heuristic algorithms. Evolutionary algorithms, as successful problem solvers, are promising candidates for this purpose. This paper reviews the application of evolutionary algorithms for solving some NP-hard optimization tasks in Bayesian network inference and learning.
Comparing risks of alternative medical diagnosis using Bayesian arguments
2010, Journal of Biomedical Informatics
Citation Excerpt :
diagnosis of specific diseases [2,13,23,27,28,40,43,47,57]; predicting risk of specific diseases [8,32,53,54]; predicting specific medical outcomes [19,25,31,52,56];
This paper explains the role of Bayes Theorem and Bayesian networks arising in a medical negligence case brought by a patient who suffered a stroke as a result of an invasive diagnostic test. The claim of negligence was based on the premise that an alternative (non-invasive) test should have been used because it carried a lower risk. The case raises a number of general and widely applicable concerns about the decision-making process within the medical profession, including the ethics of informed consent, patient care liabilities when errors are made, and the research problem of focusing on ‘true positives’ while ignoring ‘false positives’. An immediate concern is how best to present Bayesian arguments in such a way that they can be understood by people who would normally balk at mathematical equations. We feel it is possible to present purely visual representations of a non-trivial Bayesian argument in such a way that no mathematical knowledge or understanding is needed. The approach supports a wide range of alternative scenarios, makes all assumptions easily understandable and offers significant potential benefits to many areas of medical decision-making.
Future Developments in Chest Pain Diagnosis and Management
2010, Medical Clinics of North America
Citation Excerpt :
Various new techniques are being used to provide an actual point-estimate of PTP of PE, over and above the physician's usual unstructured Gestalt estimate, assisted by validated scoring systems such as the Canadian (Wells) or Geneva scores. These methods include a back-transformed logistic regression equation, and nonlinear models such as artificial intelligence or Bayesian Network analysis.105–107 PREtest ConsultPE (PREtest Consult Inc, Charlotte, NC) is a novel computerized method of pretest probability assessment derived from a process called attribute matching.
CT pulmonary angiography is the first-line imaging test for acute pulmonary embolism: A survey of US clinicians
2006, Academic Radiology
Our aim is to document current imaging practices for diagnosing acute pulmonary embolism (PE) among physicians practicing in the United States and explore factors associated with these practices.
Between September 2004 and February 2005, we surveyed by mail 855 physicians selected at random from membership lists of three professional organizations. Physicians reported their imaging practices and experiences in managing patients with suspected acute PE during the preceding 12 months.
Completed questionnaires were received from 240 of 806 eligible participants (29.8%) practicing in 44 states: 86.7% of respondents believed that computed tomographic pulmonary angiography (CTPA) was the most useful imaging procedure for patients with acute PE compared with 8.3% for ventilation-perfusion (V-P) scintigraphy and 2.5% for conventional pulminary angiography (PA). After chest radiography, CTPA was the first imaging test requested 71.4% of the time compared with V-P scintigraphy (19.7%) and lower-limb venous ultrasound (5.8%). Participants received indeterminate or inconclusive results 46.4% of the time for V-P scintigraphy, 10.6% of the time for CTPA, and 2.2% of the time for PA. CTPA was available around the clock to 88.3% of participants compared with 53.8% for V-P scintigraphy and 42.5% for PA. A total of 68.6% of respondents received CTPA results in 2 hours or less (vs 37.5% for V-P scintigraphy and 22.9% for PA). CTPA also provided an alternative diagnosis to PE or showed other significant abnormalities 28.5% of the time, and these findings frequently altered management.
US clinicians unequivocally prefer CTPA in patients with suspected acute PE. Reasons for this preference include availability and timely reporting, a lower rate of inconclusive results, and the additional diagnostic capabilities that CTPA can provide.
Evaluation of a Bayesian decision network for diagnosing pyloric stenosis
2006, Journal of Pediatric Surgery
Most infants undergoing an ultrasound to rule out pyloric stenosis will have a negative study, suggesting the low accuracy of clinical assessment. The purpose of this study was to evaluate the feasibility of using a Bayesian network to improve the accuracy of diagnosing pyloric stenosis.
Records of 118 infants undergoing an ultrasound to rule out pyloric stenosis were reviewed. Data from 88 (75%) infants were used to train a Bayesian decision network that predicted the probability of pyloric stenosis using risk factors, signs, and symptoms of the disease. The emergency department records of the remaining 28 (25%) infants were used to test the network. Two groups of pediatric surgeons and pediatric emergency medicine physicians were asked to predict the probability of pyloric stenosis in the testing set: (1) physicians using the network and (2) physicians using only emergency department records. Accuracy was evaluated using area under the ROC curve (discrimination) and Hosmer-Lemeshow (H-L) c-statistic (calibration).
Physicians using the Bayesian decision network better predicted the probability of pyloric stenosis among infants in the testing set than those not using the network (ROC 0.973 vs 0.882; H-L c-statistic 3.9 [P > .05] vs 24.3 [P < .05]). Physicians using the network would have ordered 22% fewer ultrasounds and missed no cases of pyloric stenosis.
The use of a Bayesian decision network may improve the accuracy of physicians diagnosing infants with possible pyloric stenosis. Use of this decision tool may safely reduce the need for imaging among infants with suspected pyloric stenosis.

View all citing articles on Scopus

Author contributions: JAK and AJN conceived the study. JAK, PBR, CK, and DMC obtained study funding and supervised data collection, including quality control. JAK, PBR, CK, and DMC contributed substantially to data collection, data entry, and data analysis. AJN constructed and tested the Bayesian network. JAK performed other statistical analysis. JAK drafted the manuscript, and all authors contributed substantially to its revision. JAK takes responsibility for the paper as a whole.

Funding and support: The authors report this study did not receive any outside funding or support.

Reprints not available from the authors.

View full text

Original ResearchDerivation and Validation of a Bayesian Network to Predict Pretest Probability of Venous Thromboembolism

Study objective

Methods

Results

Conclusion

Introduction

Section snippets

Materials and methods

Results

Limitations

Discussion

Acknowledgements

J Thromb Haemost

Am J Med

Ann Emerg Med

J Thromb Haemost

J Thromb Haemost

J Thromb Haemost

Ann Emerg Med

Ann Emerg Med

Does this patient have pulmonary embolism?

JAMA

Assessing clinical probability of pulmonary embolism in the emergency ward: a simple score

Arch Intern Med

Derivation of a simple clinical model to categorize patients' probability of pulmonary embolism: increasing the models utility with the SimpliRED d-dimer

Thromb Haemost

Original Research
Derivation and Validation of a Bayesian Network to Predict Pretest Probability of Venous Thromboembolism