Bayesian statistics provides probability estimates of the true state of the world. An unremarkable statement, you might think -what else would statistics be for? But classical frequentist statistics, strictly speaking, only provide estimates of the state of a hothouse world, estimates that must be translated into judgements about the real world. For example, suppose you have been monitoring a new hypertension drug that has just come onto the market (Phase IV surveillance) to watch for side effects. Out of 5000 patients being tracked, 16 have newly-developed glaucoma one year into the monitoring period, double what you would expect.
Classical statistics will say “IF the drug has no glaucoma-producing tendencies, what is the probability of seeing such a high number of glaucoma cases?” From the answer to that question you make further inference judgements about the drug. But what you really want to know is “What is the probability that the drug is associated with increased glaucoma?” Classical statistical methods do not directly answer that question, though they do provide p-values and confidence intervals that are often misinterpreted as doing so.
Bayesian statistics seeks to answer the latter question by combining prior information and beliefs (in the form of probability distributions) with the current information in the study to arrive at an estimate of the true state of affairs (again, in the form of a probability distribution).
Bayesian theory has been around for a long time, but it was not until the computer revolution of the last quarter century that the necessary computational power arrived to actually calculate Bayesian models for a wide class of problems.
We offer a series of courses in Bayesian Statistics – see more here.