A Bayesian network (BN, belief network) is a directed acyclic graph whose nodes are random variables and whose edges encode direct probabilistic influence. Each node stores a conditional probability distribution given its parents, and together they define a full joint distribution compactly. BNs let you combine expert knowledge with data, reason in any direction (from causes to effects for prediction, from effects to causes for diagnosis), and handle missing evidence naturally. They are used in medical and fault diagnosis, risk assessment, reliability engineering and causal analysis.
Part of Knowledge & Reasoning. Related: Naive Bayes is the simplest BN, and Logic & Inference is the certainty-only counterpart.
A BN over variables X1..Xn factorizes the joint distribution as P(X1..Xn) = ∏i P(Xi | Parents(Xi)). With binary variables and at most k parents per node, that needs O(n·2k) numbers instead of 2n − 1 for the full joint table. The graph is the qualitative part (who influences whom); the conditional probability tables (CPTs) are the quantitative part. Continuous variables use conditional densities, for example linear-Gaussian CPDs.
When edges point from causes to effects, the network tends to be sparse and its numbers are easy for experts to elicit. A BN by itself encodes associations, though; reading it as a causal model (and answering interventional "what if we do X" queries) requires the additional assumptions of causal Bayesian networks.
The graph encodes conditional independencies. The local rule: each variable is independent of its non-descendants given its parents. The global rule, d-separation, says sets X and Y are independent given evidence Z if every path between them is blocked: a chain or fork is blocked when its middle node is observed; a collider is blocked unless the collider or one of its descendants is observed. The collider case produces explaining away: Sprinkler and Rain are independent given Cloudy, but once you see wet grass, learning that it rained makes the sprinkler less likely.
Inference computes posteriors such as P(Rain | WetGrass = true) by summing the factorized joint over hidden variables and normalizing. Variable elimination pushes sums inside products and eliminates one variable at a time; its cost is exponential in the size of the largest intermediate factor, governed by the graph's treewidth. The junction tree algorithm compiles the graph once to answer many queries. Polytrees (singly connected graphs) admit linear-time belief propagation. Exact inference is NP-hard in general, so dense networks use sampling (likelihood weighting, Gibbs sampling) or variational methods. MAP queries ask for the most probable joint assignment rather than marginals.
Parameter learning with complete data is counting: maximum likelihood CPT entries are conditional frequencies, and Bayesian estimation with Dirichlet priors (adding pseudo-counts) avoids zero probabilities. With missing data, use expectation-maximization. Structure learning searches for a DAG: score-based methods (hill climbing with BIC or BDeu) or constraint-based methods (PC algorithm using conditional independence tests). Data alone generally identifies a graph only up to its Markov equivalence class, so domain knowledge about edge direction is valuable. In practice many deployed networks use an expert-drawn structure with data-learned parameters.
# pip install pgmpy (older releases name the class BayesianNetwork)
from pgmpy.models import DiscreteBayesianNetwork
from pgmpy.factors.discrete import TabularCPD
from pgmpy.inference import VariableElimination
m = DiscreteBayesianNetwork([("C", "S"), ("C", "R"), ("S", "W"), ("R", "W")])
# state 0 = False, 1 = True; columns iterate over parent states
cpd_c = TabularCPD("C", 2, [[0.5], [0.5]])
cpd_s = TabularCPD("S", 2, [[0.5, 0.9], # P(S=F | C=F), P(S=F | C=T)
[0.5, 0.1]], evidence=["C"], evidence_card=[2])
cpd_r = TabularCPD("R", 2, [[0.8, 0.2],
[0.2, 0.8]], evidence=["C"], evidence_card=[2])
cpd_w = TabularCPD("W", 2, [[1.0, 0.1, 0.1, 0.01], # S,R = FF, FT, TF, TT
[0.0, 0.9, 0.9, 0.99]],
evidence=["S", "R"], evidence_card=[2, 2])
m.add_cpds(cpd_c, cpd_s, cpd_r, cpd_w)
assert m.check_model()
infer = VariableElimination(m)
print(infer.query(["R"], evidence={"W": 1})) # diagnosis
print(infer.query(["S"], evidence={"W": 1}))
print(infer.query(["S"], evidence={"W": 1, "R": 1})) # explaining away: S drops
Other options: pomegranate, bnlearn (R and Python), and probabilistic programming languages such as PyMC or NumPyro when variables are continuous or the model is easier to write as code.
Using the CPTs in the first diagram, the evidence WetGrass = true has probability P(W=T) ≈ 0.647, obtained by summing P(C) P(S|C) P(R|C) P(W=T|S,R) over all eight combinations of C, S and R. Restricting the same sum to rows with R = T gives about 0.458, so P(R=T | W=T) ≈ 0.458 / 0.647 ≈ 0.708. Doing the same for the sprinkler gives P(S=T | W=T) ≈ 0.430. Before seeing anything, the priors were 0.5 for rain and 0.3 for the sprinkler, so wet grass raised belief in both causes.
Now add the evidence Rain = true. The posterior for the sprinkler falls to P(S=T | W=T, R=T) ≈ 0.194. Rain already explains the wet grass, so the sprinkler is less needed as an explanation, and its probability drops even below the 0.3 prior because rain also implies cloudy weather, which makes the sprinkler less likely to be on. These are the numbers the pgmpy code above prints, and doing the sum by hand once is the best way to see that inference is just bookkeeping over the factorized joint.
A dynamic Bayesian network (DBN) copies one slice of variables per time step and adds edges from slice t-1 to slice t. Because the same transition and observation CPTs are shared across slices, the model stays compact no matter how long the sequence is. A hidden Markov model is a DBN with one discrete hidden state and one observation per slice; a Kalman filter is the linear-Gaussian version. Filtering (current state given past evidence), smoothing (past state given all evidence) and prediction are the standard queries, and particle filters are the usual approximate method when the state is continuous and nonlinear.
Where BNs are used in practice:
| Domain | Typical query | Why a BN fits |
|---|---|---|
| Medical diagnosis | P(disease | symptoms, tests) | Experts can supply CPTs; partial evidence is the norm |
| Fault diagnosis and troubleshooting | Most probable failed component | Explaining away ranks competing causes |
| Risk and reliability analysis | P(failure) under scenarios | Transparent model that auditors can inspect |
| Tracking and sensor fusion | Filtered state estimate | DBNs combine noisy sensors over time |