My notes on Bayesian Inference and Probabilistic Programming.
In simple terms, Bayesian inference is simply updating your beliefs after considering new evidence.
In Bayesian world probability is measure of believability in an event, that is, how confident we are in an event occurring, whereas in, Frequentist world (classical version of statistics), assume that probability is the long-run frequency of events.
For example, the probability of plane accidents under a frequentist philosophy is interpreted as the long-term frequency of plane accidents. This makes logical sense for many probabilities of events, but becomes more difficult to understand when events have no long-term frequency of occurrences.
Bayesians, on the other hand, have a more intuitive approach. Bayesians interpret a probability as measure of belief, or confidence, of an event occurring. An individual who assigns a belief of 0 to an event has no confidence that the event will occur; conversely, assigning a belief of 1 implies that the individual is absolutely certain of an event occurring. Beliefs between 0 and 1 allow for weightings of other outcomes.
For eg: