Bayesian Network Literature and Software

Literature

• Judea Pearl
  Probabilistic Reasoning in Intelligent Systems
  M. Kaufmann, San Mateo, CA, 1988/1992 (1st/2nd edition)
  The classic reference for probabilistic networks and their use for reasoning under uncertainty, which focuses on the expert system view. It contains some remarks about learning probabilistic networks from data, but is very limited in this respect, since it discusses only learning trees and polytrees.
• Finn V. Jensen
  An Introduction to Bayesian Networks
  UCL Press, London, England 1996
  A very readable textbook on Bayesian networks, which focuses on the construction of Bayesian networks for applications, starting from a causal model of the domain under consideration. It discusses evidence propagation in considerable detail, but does not treat learning probabilistic networks from data.
• E. Castillo, J.M. Gutierrez, and A.S. Hadi
  Expert Systems and Probabilistic Network Models
  Springer, New York, USA 1997
  This book provides a lot of material about probabilistic networks and is very detailed in graph algorithms. It contains a chapter on learning Bayesian networks from data, which is, however, somewhat limited in scope, since it focuses on Bayesian and minimum description length approaches.
  
• Brendan J. Frey
  Graphical Models for Machine Learning and Digital Communication
  Bradford Book/MIT Press, Cambridge, MA, USA 1998
  Due to its strong focus on digital communication problems (data compression and coding) it is somewhat limited in scope. The explanations are often very condensed. Not recommendable as an initial textbook, but a good reference for advanced students.
• Joe Whittaker
  Graphical Models in Applied Multivariate Statistics
  Wiley & Sons, Chichester, England 1990
  This is a mathematical book with a strong statistical focus. It contains a brief chapter on model selection, which discusses only a very restricted set of approaches. Due to its strong mathematical orientation it is not very accessible for students/researchers with a computer science background. An excellent book for statisticians.
• S.L. Lauritzen
  Graphical Models
  Clarendeon Press, Oxford, England 1996
  A very mathematical book, which contains a lot of results that cannot be found elsewhere. It focuses on mathematical and statistical aspects and is thus not very accessible for students/researchers with a computer science background. Learning probabilistic graphical models from data is treated in detail, but with a very strong statistical focus. Highly recommendable for advanced students.
• U.M. Fayyad, G. Piatetsky-Shapiro, P. Smyth, and R. Uthurusamy (eds.)
  Advances in Knowledge Discovery and Data Mining
  AAAI Press/MIT Press, Menlo Park/Cambridge, CA/MA, USA 1996
  Contains a chapter ``Bayesian Networks for Knowledge Discovery'' by D. Heckerman, which focuses on Bayesian approaches to learn a Bayesian network from data. However, this chapter presupposes substantial statistical background and is not recommendable as an introduction.
  
• M. Jordan (ed.)
  Learning in Graphical Models
  MIT Press, Cambridge, MA, USA 1999
  A collection of papers by a large number of authors, all of which deal with some aspect of learning probabilistic graphical models from data. It is very detailed in some respects. As an edited book, however, it lacks the coherence of a monograph, which makes it unsuited as a textbook.
• P. Spirtes, C. Glymour, and R. Scheines.
  Causation, Prediction, and Search (Lecture Notes in Statistics 81)
  Springer, New York, NY, USA 1993
  This book focuses on the relation of conditional independence and causation and develops learning algorithms for Bayesian networks that are based on conditional independence tests. Its orientation is philosophical.

Software

• Hugin
  Hugin Expert A/S
  Niels Jernes Vej 10, 9220 Aalborg, Denmark
  Hugin is one of the oldest and best-known tools for Bayesian network construction and inference. It comes with an easy to use graphical user interface, but also has an API, so that the inference engine can be used in other programs. A drawback of this program is that it does not incorporate learning from data yet. (However, Hugin Expert A/S also offers Pronel, which learns the structure of a Bayesian network from fully observed data (no missing values).) Hugin is a commercial tool, but a free demo version may be retrieved.
• Netica
  Norsys Software Corp.
  2315 Dunbar Street, Vancouver, BC, Canada V6R 3N1
  Like Hugin, Netica is a commercial tool with a very advanced graphical user interface. It supports Bayesian network construction and inference and also comprises an API, so that the inference engine may be used in other programs. Netica incorporates quantitative network learning (known structure, parameter estimation), but not structural learning. A version of Netica with restricted capabilities may be retrieved free of charge.
• Bayesian Knowledge Discoverer
  Knowledge Media Institute
  The Open University, Great Britain
  The Bayesian Knowledge Discovery is free software that can learn Bayesian networks from data (structure as well as parameters). The dataset to learn from may contain missing values. The Bayesian Knowledge Discoverer is free software, but is being succeeded by a commercial version, Bayesware Discoverer, which is available from Bayesware Ltd..
• Tetrad
  Tetrad Project
  Dept. of Philosophy, Carnegie Mellon University, USA
  Tetrad is based on the algorithms developed in the book Causation, Prediction, and Search by Spirtes, Glymour, and Scheines mentioned above and, of course, subsequent research in this direction. It can discover the structure as well as the parameters of a Bayesian network from a dataset of sample cases. Currently the program is being ported to Java (Tetrad IV). Tetrad is commercial, but available at a moderate fee.
• Belief Network Power Constructor
  Jie Cheng
  Dept. of Computer Science, University of Alberta, USA
  Like Tetrad, the Bayesian Power Constructor uses conditional independence tests to learn the structure of a Bayesian network from data. It comes with a graphical user interface and has the big advantage that it is free software.

More extensive lists of Bayesian network tools:

• Software list by Kevin Patrick Murphy
• Software list by Russel Almond