Bayes' theorem: updating probabilities and Bayesian inference

Statement and simple derivation

The theorem can be written compactly as P(A|B) = P(B|A) P(A) / P(B), where P(A|B) denotes the conditional probability of A given B. It follows from the symmetry of the joint probability P(A and B) = P(A)P(B|A) = P(B)P(A|B). Rearranging these expressions yields the Bayes formula, which holds for discrete events and generalizes to densities for continuous variables.

Key components and interpretation

• Prior P(A): the probability assigned to the hypothesis before seeing the new data.
• Likelihood P(B|A): how probable the observed evidence is if the hypothesis A is true.
• Evidence P(B): the overall probability of observing the evidence under all considered hypotheses (also called marginal likelihood).
• Posterior P(A|B): the updated probability of the hypothesis after accounting for the evidence.

These pieces frame how information is combined: the likelihood re-weights the prior, and the evidence normalizes the result so posteriors sum to one across mutually exclusive hypotheses.

Example and common pitfalls

A typical illustrative example involves diagnostic testing. Suppose a condition is rare in a population and a test is fairly accurate. Even with high sensitivity and specificity, a low prevalence can make most positive tests false positives. Concretely, if roughly 1% of people have a condition and a test correctly identifies cases 99% of the time but also yields a few percent false positives, then the probability that a person who tests positive actually has the condition can still be quite low. This is an instance of the base-rate fallacy, where ignoring the prior prevalence leads to misleading conclusions.

History, applications and distinctions

The theorem is named after Thomas Bayes, an 18th-century statistician and minister, though subsequent mathematicians developed and popularized Bayesian ideas. Bayes' theorem underpins Bayesian inference: a framework for statistical reasoning that treats unknowns as random variables and updates beliefs with data. It is widely used in machine learning, medical diagnosis, natural language processing, spam filtering, and decision making under uncertainty. Practical use often requires computational methods (for example, sampling or approximation techniques) when exact calculation of the evidence is difficult.

Bayesian methods differ philosophically from frequentist approaches primarily in the role of probability: Bayesians permit probability statements about hypotheses via priors, while frequentists interpret probability as long-run frequencies. Choices of priors, computational cost, and interpretational differences are common topics in applied work and debates.

For further reading on probability foundations see probability theory and for the idea of assessing hypotheses see hypothesis.