Bayesian network

A Bayesian network or Bayesian net (named after Thomas Bayes) is a directed acyclic graph (DAG) in which the nodes describe random variables and the edges describe conditional dependencies between the variables. Each node of the network is given a conditional probability distribution of the random variable it represents, which assigns random variables to parent nodes. They are described by probability tables. This distribution can be arbitrary, but discrete or normal distributions are often used. Parents of a node v are those nodes from which an edge leads to v.

A Bayesian network is used to represent the joint probability distribution of all variables involved as compactly as possible by exploiting known conditional independents. In doing so, the conditional (in)dependence of subsets of the variables is combined with the a priori knowledge.

If X1, ..., Xn are some of the random variables occurring in the graph (which are closed with the addition of parent variables), then their joint distribution is calculated as

If a node has no parents, the associated probability distribution is an unconditional distribution.

Example

Closure in Bayesian networks

If the value of some of the variables, such as E1, ..., _Em, is known, i.e. evidence is available, the conditional probability distribution of X1, ..., Xn with given E1, ..., _{Em can} also be calculated with the help of various algorithms and thus inference can be performed.

The inference problem, both exact and approximate, in Bayesian networks is NP-hard. In larger networks, however, approximative methods lend themselves well. Exact methods are somewhat more accurate than approximate ones, but this often plays a negligible role in practice, since Bayesian networks are used for decision making where the exact probabilities are not needed.

It should be noted that software implementations of exact inference methods often use only double-precision floating-point numbers. This limits the accuracy of these calculations.

Exact inference

Suitable algorithms for exact inference in Bayesian networks include the following:

• Variable elimination
• Clustering Algorithms

Approximate inference

• rejection sampling
• likelihood weighting
• Self-Importance
• Adaptive Importance
• Markov chains
• Monte Carlo algorithm, e.g. Gibbs sampling

Inference types

• Diagnostic: From effects to causes
• Causal: From causes to effects
• Intercausal: Between causes of a common effect
• Mixed: Combination of the previous