Skip to main content
Loading Explore guides…
Probability Distributions · Acedra
Home
Explore
Mathematics
Probability Distributions
Sign in to save
Chat
If YouTube asks you to sign in, open the video on YouTube.
Watch on YouTube
Reload player
Transcript
65 segments
Copy
Hide video
0:00
Probability distributions are a core concept in science. They underlie everything from simple tests to complicated models.
0:08
In this video I will explain three very commonly used probability distributions for continuous variables, counts and ratios. I will explain why they look the way they do, and why they can be good approximations to the processes that we use them for. I'll not explain much of the mathematics involved, but it will be displayed on the screen here and there for completeness.
0:31
Suppose you pick a random adult male in your country of origin, what do you expect his height to be? For instance, in The Netherlands a quick search will show you recent estimates put the average at a little over 1.83 meters, or 6 foot. Let's for the moment ignore the fact that this is only an estimate and pretend that this is the true mean of Dutch adult males in The Netherlands as of 2020. What does this value tell us?
0:57
If you're actually going to pick someone, they're not going to be exactly this number. The mean is just the value that has the lowest expected difference from what you will actually measure in a random sample. So rather than looking at only the mean, it is more interesting to look at the whole probability distribution. This is what that looks like. On the x-axis we see the possible values for someone's height and on the y-axis we see whether this is more likely or less likely.
1:26
For human height, most observations are close to the average, so the probability density here is high. Some are further away but this is less common and further deviations are exceedingly rare. Also, deviations in either direction are equally common. That means there are as many short people as there are tall people, so the distribution is symmetric. This particular type of probability distribution is called the normal
1:52
distribution, also known as the Gaussian distribution, and it arises when an outcome like human height is the sum of a very large number of independent effects.
2:04
Height depends on environmental factors like nutrition, hormones, activity levels and there are many hundreds of genes contributing to your eventual height as an adult.
2:14
There are of course large systematic differences, like the difference between countries, the gradual increase of average height over time, or the average difference between males and females. But if we narrow it down to some very specific subgroup, like adult males in The Netherlands as of 2020, then this normal distribution is actually a very good approximation.
2:37
A normal distribution has two parameters. The first one is the mean, represented by the Greek letter mu. The mean tells us where observations are expected to be on average. Tt can be any positive or negative number or zero. If we increase the mean the distribution shifts to the right and if we decrease the mean the distribution shifts to the left. In case of adult male height in The Netherlands, the mean is about 1.838 meters.
3:06
The second parameter of the normal distribution is its variance, represented by the symbol sigma squared. Variance tells us the extent to which observations deviate from one another. This is much easier to understand if you don't think of a single random individual, but instead imagine we pick say 10 random males. Variance then tells us how much we can expect these men to differ from each other in their height. If the variance is large then individual differences are
3:35
large and if the variance is small individual differences are small.
3:40
Hypothetically, if the variance were to be zero, then all men would have identical height, which of course is unrealistic.
3:49
Variance is calculated as the average squared distance from the mean. That's why the unit of measurement is also squared and we represent it by sigma squared. But where do we draw squared meters in a figure that shows meters? To be honest, I have no idea. But if we take the square root of the variance, we now have something in the same unit of measurement as the mean, in meters, which is a lot easier to interpret.
4:15
This measure is called the standard deviation represented by the Greek letter sigma.
4:20
There is a very useful property of the normal distribution that you may have heard before. Namely, you expect about 68% of all observations to fall within 1 standard deviation of the mean, and you expect around 95% within 2 standard deviations of the mean. It doesn't matter if we change the mean or the variance of the distribution, this relationship always holds. But how did I come up with these numbers? And what if I don't care about 68%, but I want to know for example
4:51
what proportion of men should I expect to be taller than 2 meters? The way that works is as follows: The total area under any probability distribution is 1. So asking the chance of a particular range of values is really just asking what proportion of the distribution is covered by that range. How to calculate that is the same as you would calculate any surface area under a function: by taking the integral. We're not actually going to do that by hand
5:20
and in statistical software there are predefined functions to calculate this for you. Using these, it turns out that the chance of being 2 meters or taller is approximately 1%. So if you're 2 meters or more that is pretty rare.
5:35
For women, since the average is lower, the chance of being that tall is also much lower, namely about 0.001%.
5:45
You can use these functions in statistical software and related functions to answer all kinds of question about probability, which I will demonstrate in another video. But here's a question that doesn't require any functions: What is the chance that if you pick a random male, their height is exactly equal to the mean of 1.838?
6:07
You might think the answer is 6? Percent? It clearly can't be 6, probabilities are between 0 and 1. What would it even mean for something to happen with a 600% chance? But it also can't be 6%, because then my next question would be: What about the chance of being ever so slightly taller, like 1.838001 or 1.838001? In fact there is an uncountably infinite number of possible values between 1.838 and 1.839 alone so that would never sum up to a total of 100%,
6:46
which brings us to the answer, that the chance of any particular outcome is 0. Any exact outcome of a continuous variable occurs with probability 0.
6:57
It might take some time to wrap your head around that. If you're more easily convinced by math, then just compute the integral again. The integral from a certain value to the same value is twice the same term, which is just going to cancel out.
7:12
The y-axis shows probability density, not probability. These values are meaningless on their own unless we use them to calculate the surface area of a particular range. So instead of asking the chance of being exactly equal to the mean, you might ask for the chance of being within 0.5 cm of the mean, or within 1 mm, or whatever range you want to know.
7:35
One final important property of the normal distribution is that it goes on forever, which is useful to allow for the occasional, rare outlier. For example, the tallest person to have ever lived. The probability of an observation that large or larger is extremely small, but it is possible. At the same time this also shows us that the normal distribution is just an approximation and not the actual probability distribution of male adults in a given country.
8:04
After all, height cannot be negative, nor is it possible to reach any height up to infinity. But since the probability in the tails of the distribution diminishes so quickly, it really doesn't matter for practical purposes and the normal distribution is actually an excellent approximation.
8:22
So what have we learned so far? The normal distribution has 2 parameters: The mean (mu) which is the location of the distribution, and the variance (sigma squared) which is the scale of the distribution, or how diffuse or wide it is so to speak. The normal distribution is a good approximation when most observations are close to the mean, deviations in either direction are equally likely, further deviations are exceedingly rare,
8:48
and extreme values are possible but occur with very low probability. And whatever measuring is continuous, so it can be 2 but also 1.9 or 2.1 or 2.001, etc. Examples include temperature, height BMI and blood pressure, within a specific subgroup, like healthy adult males in a given country.
9:12
Let's compare this with another common probability distribution and see how it differs. Suppose you have an apple tree and you want to predict the number of apples that it will produce in a year. For simplicity let's assume the tree blooms once a year. Think of questions like: What is the chance of getting 20 apples in one year, or what is the chance of getting at least 30 apples in a year? And what is the largest number of apples that I can realistically expect?
9:38
These types of questions can be answered using a Poisson distribution. All we need for this is the expected number of apples. In the absence of other data, and since there are 25 apples in the tree this year, let's set the expected number to 25.
9:53
Here is what that looks like. As an exercise try to answer these 3 questions just by looking at the graph. You don't have to do it precisely. You'll notice a few differences with the normal distribution. First, this is a discrete probability distribution. A tree can grow 0, 1, 2, 3, or more apples in a year, but not 1.5 or 2.1 apple. Because of this there is only a limited number of possible values, that means that rather than having to evaluate an integral over a range,
10:26
we can just read the probability of a specific value from the y-axis. So with an expected number of apples of 25, the chance of getting 20 apples is about 5% and you can read any probability like this.
10:42
The chance of getting at least 30 apples is then the sum of the probability of all values from 30 onwards. You could add all the probabilities that you can still see, which will get you fairly close to the actual answer.
10:56
The largest number of apples you can realistically expect depends on what you find realistic.
11:03
Now let's go over some important properties of the Poisson distribution. The first property I already mentioned: The Poisson distribution is not continuous but discrete. It only takes on integer values. Also, you cannot count a negative number. There is no such thing as -1 birds and a tree cannot produce -1 apple, so at 0 the probability distribution stops.
11:27
One property that you can't really tell from the figure alone is that the Poisson distribution only has a single parameter lambda. This value represents both the mean and the variance of the Poisson distribution. This is a lot easier to see if I increase lambda gradually: Watch how both the location changes but also the diffuseness, or wideness of the distribution. The first time you hear this you're probably thinking... okay?
11:53
But this is actually very intuitive if you think about it: A tree that yields on average 25 apples can be logically expected yield somewhere between, maybe a couple apples more, or a couple apples less. But a tree that has an expected number of apples of 250 would still not surprise us much if it yielded as much as 30 apples more or less than the average. So the larger the expected number, the larger the variance. This is very typical for count data.
12:21
One final important property of the Poisson distribution is that the closer the expected number gets to zero, the less symmetric the distribution becomes. The distribution is as we call it "right skewed", because the right tail of the distribution is much longer than the left tail. The reason this happens is because you cannot count a negative number, so at 0 the probability distribution stops. When the mean gets close to 0, the remaining probability gets pressed up into the side,
12:49
so to speak, whereas on the right end we may expect the occasional outlier. The larger the mean, the less skewed the distribution becomes because we're moving away from zero, so the problem I just mentioned becomes less obvious.
13:03
Let's summarize again what we have learned so far. The Poisson distribution has one parameter, lambda, which represents both its mean and its variance. So it's both the location of the distribution and the scale, or diffuseness. The Poisson distribution is a good approximation when observations are discrete, non-negative numbers, observations are right skewed, especially if the expected number is small,
13:27
and examples include any independent count in a fixed amount of time or space, like the number of apples a tree produces in a given year or the number of birds in a park on a given day etc. That brings us to the last of the three probability distributions I'll explain in this video. The binomial distribution describes the number of successes out of the total number of trials. For example, if I throw a coin 10 times, how many times will I get heads?
13:54
Of course with a fair coin you expect it to be half of the time, so that would be 5 out of 10 times. But if it happens 4 times, or 6 times, that isn't really surprising either. The binomial distribution describes the chance of any particular number of successes. It has two parameters: The number of trials, so in this case 10, and the probability of success, which in this case is 0.5. Here's what that looks like with a different probability of success,
14:23
for example, what if we make one side of the coin out of wood and the other side of lead? Now it has a larger probability of landing on one side than the other. Or here's a different example: What if I randomly guess the answers to a multiple choice exam where there are 10 questions, and every question has 4 possible answers to choose from. If you choose purely at random, you would expect to answer about 25% correctly. But you could get more right, you could get fewer right.
14:51
This binomial distribution then shows the chance of any particular number of correct answers. It can help you answer questions like: What is the chance of getting 3 questions right? Or what's the chance of getting a passing grade? And what is the highest grade I can realistically expect from random guessing?
15:11
Again as an exercise, try to answer these questions just by looking at the graph. It doesn't have to be precise, just test your own understanding.
15:21
Like the Poisson distribution the binomial is a discrete probability distribution, because you cannot guess one and a half, or 1.1 question correctly. Only 0, 1, 2, etc. That means that we can answer these questions in the same manner as we did with the Poisson distribution. The chance of getting 3 questions right can be read directly from the graph: about 25%.
15:45
The chance of getting a passing grade of course depends on what the threshold for passing is. If you could pass with 6 correct questions out of 10, then any number of questions equal to, or greater than 6 would suffice. That means we sum up all the possibilities (equal to or) greater than 6 which adds up to about 2%.
16:03
Unsurprisingly, the chance of passing by randomly guessing is not that great. The highest grade you can realistically expect, is again a question that requires you to give an interpretation. You may think this is vague, but it's not too different from actual research. You start out, often with some vague idea of what you want to demonstrate, and then you have to convert it to something very precise in order to give an answer.
16:27
For example, if you interpret realistic to mean a chance greater than 5%, the answer would be 5 correct answers, as everything past that sums up to less than 5%.
16:38
But in a class of 100 students, if all were randomly guessing, you would still expect an event with at least 1% chance to occur on average once, which means that the highest realistic grades would be a 6.
16:51
Again, how to perform these kinds of calculations easily in statistical software is something I'll demonstrate in another video.
16:59
Unlike the Poisson distribution, the binomial distribution is bounded in two directions: There's both a minimum, 0, and a maximum, which is the number of trials. That is to say, you cannot guess -1 correct answer, but of course you also cannot guess more correct answers than there are questions.
17:20
Since it is bounded in two directions, the binomial distribution can be right skewed, or left skewed, depending on the probability of success.
17:30
Only near a 50% chance of success, like when flipping a fair coin, is the distribution symmetric.
17:37
Now let's go back to randomly guessing answers on an exam. What do you think will happen to the answers if we increase the number of trials? So instead of 10, suppose the exam had 100 questions, and we still have a 25% chance of guessing the right answer on any given question. Do you think the probabilities will go up, down, or stay the same?
18:01
Here's what a binomial distribution looks like with 100 trials. I left the probability of success unchanged at 25%.
18:10
The chance of getting precisely 3 questions right out of 100 by random guessing is very small. The chance of getting a passing rate by guessing has also diminished completely by increasing the number of questions to 100. For example, to get at least 60% right, now has a probability of... well, whatever this is. Because of that, the highest grade you can expect has also decreased. If you take realistic to mean at least a 5% chance,
18:36
you can now expect to answer at most 32 questions correctly. And even if you go with a 1% chance, you can still expect to answer no more than 35 questions correctly.
18:48
More generally the larger the number of trials, the closer you can expect to be to the actual probability of success. Another difference you'll notice with a larger number of trials is that the skew is much less apparent. This too makes sense intuitively for if we have a larger number of trials, you have to get a lot closer to the boundaries of the distribution before the probability gets pressed up into the side.
19:12
Finally, there is a special case of the binomial distribution when there is only one trial, like the outcome of a single coin flip, or the chance of rolling 6 when you only roll once. This is called a Bernoulli distribution and it is used for binary outcomes. For example, in a case control study we can call cases "1" and controls "0", and that allows us to estimate the effects of various contributing factors on developing cancer, contracting a disease etc.
19:42
Let's summarize once more what we've learned so far before we wrap up. The binomial distribution has two parameters: The probability of success, p, and the number of trials, n, which together with p determines the scale, so how diffuse it is. The binomial distribution is used when observations are discrete numbers between 0 and n. Observations are skewed, especially with a low number of trials, or the probability of success is close to the boundaries.
20:09
And examples include binary data and ratios.
20:13
Probability distributions are a core concept in science. So far we've covered 3 commonly used ones: The normal distribution, the Poisson distribution and the binomial distribution. These are used to model continuous outcomes with the central tendency, independent counts, and ratios or binary outcomes. Keep in mind though, that there are many other probability distributions that we might use. For example, just because something is continuous and somewhat follows a bell shape
20:43
does not guarantee that the normal distribution is a good approximation. Likewise, not all counts follow an exact Poisson distribution, and neither do ratios always follow an exact binomial distribution. The good news is though, that you don't have to memorize a large number of probability distributions and all their properties to further your understanding of statistics. The three I mentioned here really are very common and I too often use them as a starting point in analyses.
21:10
What is important is that you know how to assess whether a distribution is a good approximation or not, which is something you'll learn in later lectures. Lastly, I admitted a lot of phenomena for simplicity, like the structural differences in height between men and women and between different countries. Or the effect of growth, disease and rainfall in a given year on the number of apples. And for the binomial distribution... well,
21:35
most students actually learn for exams and aren't randomly guessing everything, so how do we factor in something like time spent studying, or previous grades. This is actually a major part of statistics called regression analysis, which I talk about at length in my other videos. I hope that this video can make the others a bit easier to understand. Thank you for watching!
Here to help you learn
Sign in to keep a private conversation with your course tutor.
Ask me anything about the material…
Sign in to chat