Bayes’ Rule

Synonyms

Bayes’ Theorem

Definition

Bayes’ rule provides a decomposition of a conditional probability that is frequently used in a family of learning techniques collectively called Bayesian learning. Bayes’ rule is the equality

$$\displaystyle{ P(z\vert w) = \frac{P(z)P(w\vert z)} {P(w)} }$$

(1)

P($w)$ is called the prior probability, P($w\vert z)$ is called the posterior probability, and P($z\vert w)$ is called the likelihood.

Discussion

Bayes’ rule is used for two purposes. The first is Bayesian update. In this context $z$ represents some new information that has become available since an estimate P($w)$ was formed of some hypothesis $w$. The application of Bayes’ rule enables a new estimate of the probability of $w$ (the posterior probability) to be calculated from estimates of the prior probability, the likelihood, and P($z)$.

The second common application of Bayes’ rule is for estimating posterior probabilities in probabilistic learning, where it is the core of Bayes...