Probability and Likelihood

In this article I am going to discuss about the difference between Probability and likelihood. These are two closely related concepts that are very easy to get confused.

Probability:

Let’s look at a Probability w.r.t to Normal distribution. Imagine you have data set of Mouse weights which follows a normal distribution with mean 32 and Standard deviation of 2.5

The Probability of a weighing a random Mouse that will be between 32 & 34g is the area under the curve between 32 & 34g.

The area under the curve is 0.29. Which means there is 29% chance a randomly selected mouse will weigh between 32 & 34g. Mathematically we say with the following notation.

If we want to know the probability of mouse weighing more than 34g, then

Mathematically we can represent this using the below equation.

Here we are changing the Highlighted part of equation and the non highlighted part of the equation remains constant which determines the shape and location of the distribution.

So when we are talking about the Probability, the distribution remains same and we can get the different probabilities by changing the highlighted part of the equation.

Now we have worked out about probability, let’s talk about Likelihood.

Likelihood:

You assume that you already weighed a mouse and it is weight is 34g. The likelihood of weighing a 34g mouse is 0.12.

Mathematically, this will be described by the following equation.

If we shift the distribution over and the new mean is 34 then the likelihood will be 0.16.Mathematically, this will be described by the following equation.

So, in likelihood, the distribution keep changes which is given by the highlighted part in the above equation and other part remains same.

Likelihood is just an instantaneous point on the curve. 0.16 does not equal “16%”. It is at the moment just a number for one data point. You would compare it to another datapoint which would have its own likelihood given the assumed model, e.g. the likelihood of getting a 34g mouse is 0.16, or, it is more likely given the model that a randomly selected mouse is 34g instead of 32g

Summary:

Probabilities are the area under a fixed distribution.

Likelihoods are the Y-axis values for a fixed data points, for which distribution can be moved

Interpreting Probability & Likelihood in Machine Learning:

Probability is the quantity which deals with predicting the data given a known model.

Likelihood deals with fitting the models given some known data.