Law of large numbers

Formal statement and variants

There are several precise formulations. The two most common are the weak law and the strong law. The weak law states that the sample mean converges in probability to the expected value: for any margin of error, the probability the average lies within that margin tends to one as sample size grows. The strong law gives a stronger conclusion: the sample mean converges almost surely (with probability one) to the expected value. Although both capture the same intuitive idea, they differ in the type of convergence and in the assumptions needed for proof.

Typical conditions and related results

Classical versions of the law assume independent, identically distributed (i.i.d.) random variables with a finite expectation. Common technical conditions include:

• Independence or weak forms of dependence that permit similar conclusions.
• Identical distribution for the simplest statements; there are also generalizations allowing changing distributions.
• Existence of a finite expected value; some stronger forms require finite variance.

Related theorems include Chebyshev's inequality, which gives a simple proof of the weak law under finite variance, and the central limit theorem, which describes the distribution and rate of fluctuation of the sample mean around its limit. Ergodic theorems generalize the idea to dependent processes encountered in physics and dynamical systems.

Examples and applications

A familiar illustration is repeated coin tosses: the proportion of heads tends to the true probability of heads as the number of tosses grows. Likewise, when rolling a fair six-sided die, the average of outcomes approaches the population mean (3.5) as rolls accumulate. More general uses include survey sampling, quality control, insurance risk pooling, and experimental science, where averaging many measurements reduces random fluctuations and reveals underlying quantities.

In practical terms, the law justifies estimating unknown quantities by sample averages and supports the design of sampling schemes in statistics (statistics). It also helps explain why long-run frequencies observed in large data sets are meaningful and reproducible.

History, limitations, and notable facts

The first rigorous statement is often attributed to Jacob Bernoulli (late 17th to early 18th century), who proved an early form known as Bernoulli's theorem. Later contributors such as Chebyshev and Kolmogorov extended and clarified the result, producing modern proofs and generalizations. The law does not specify how quickly convergence occurs; the average may wander substantially in small samples or in short runs, and long deviations can still occur by chance. The law of large numbers should not be confused with the central limit theorem or used to justify the gambler's fallacy (the mistaken belief that past random outcomes change future probabilities).

Distinctions and practical advice

The law guarantees stability of averages in the long run but does not prevent variability in finite samples. When applying the law, practitioners combine it with error bounds, variance estimates, and sample-size calculations to determine how large a sample is needed for a desired accuracy. For dependent data or heavy-tailed distributions, specialized versions or additional assumptions are required.

For further introductory material and formal proofs, consult standard texts in probability and statistics or online references that cover the various forms and hypotheses of the law of large numbers (see theorem, general statistics resources, and expository pages on expected value and averages via the provided links).