Skip to main content
Loading Explore guides…
Probability Axioms & Set Theory · Acedra
Home
Explore
Mathematics
Probability Axioms & Set Theory
Sign in to save
Chat
If YouTube asks you to sign in, open the video on YouTube.
Watch on YouTube
Reload player
Transcript
37 segments
Copy
Hide video
0:00
PROFESSOR: Welcome back. So we're introducing this notion of probability, or the chances of some event occurring out of a larger set of possible things that can happen. We've looked at things like dice rolls, poker hands, coin flips, things like that. And we've seen very quickly that one of the issues is how to enumerate or count all the possible outcomes. And before I go much further, I am going to need to introduce some basic set theory, some formalism
0:36
that allows us to say things about these different events and probability spaces and sample spaces and things like that. This is going to seem a little bit technical and dry, but it's really, really important to set up the basic language of how we think about probabilities in a way that generalizes and gives us shorthand, so we can communicate much more rapidly together. So again, we know this basic idea that a probability is measuring the chances or likelihoods of some event happening.
1:08
So probability measures the likelihood or chance of some event A happening, some event A occurring or happening. Good. And I'm going to start defining some sets, like we've learned about set theory before. There's the set of integers and the set of the real numbers and things like that. So we know that probability is measuring the likelihood. I'm going to start relating this to set theory. So we're going to define a sample space. A sample space omega is the set of all possible things that
2:04
can happen in my experiment. So basically, I'm going to run either thought experiments or real experiments. We're going to talk about flipping 10 coins and what is the chance that at least five of them are heads. So the sample space omega is the set of all possible things that could happen in that experiment. So the sample space omega is the set of all possible outcomes of a random experiment of-- some random experiment I'm doing-- of a random-- let's say random experiment.
2:53
Good. And this could be me flipping a coin 10 times. This could be dealing a hand of poker. It could be playing rolling the dice in backgammon. It be any number of experiments, either real or thought experiments. So this is the sample space. And it's going to be a set of all possible outcomes. And we're going to say that a specific element. Let's put this in a different color, a specific element omega-- we say that this is in the set big omega, a specific element.
3:30
Little omega is basically one instance or one actualization, one realization of this experiment is one. We're going to say that this is one outcome or realization of our experiment. So lots and lots and lots and lots of examples of what this sample space could be. This could be the number of possible ways that I could flip five coins. So I have five coin flips. Omega could be the set of all possible ways I could flip five coins. Omega could be the number of heads in five coin flips.
4:19
Omega could be the number of emails I get in an hour. That's a large number. Omega could be a distribution of heights sampled from 10 people. There's all kinds of things omega could be. It could be today's high temperature. It could be the number of hands of poker, things like that. And a specific element could be I rolled five heads in a roll. Heads, heads heads heads, heads. That would be a specific element of the space of five coin flips. Good. And we're going to start to get into this notion
5:01
of a random variable soon. But for now, let's just talk about the space of all possible things that can happen. So let's make this concrete. Let's say that I have-- I'm going to go to green. Keep it colorful. Let's say as an example I have three coin flips. So three coin flips. And for this, I can actually enumerate this set omega. So omega equals heads heads heads, heads heads tails, heads tails heads, heads tails tails, tails heads heads,
5:46
tails heads tails, tails tails heads, tails, tails, tails. So there are eight possible elements of this set omega. And we're going to define these events A we're going to say them in words. But then they're also going to define subsets of my sample space. So if I have this event A, let's say, the first flip is a head. Let's say first flip is a head that defines an event A, where is a subset of omega. So if the first flip is a head, then I
6:36
can take all of the elements of omega, where the first coin flip was a head. And you'll notice that the first four of these are all of the instances, all of the specific elements where the first flip was ahead. So A is the subset HHH, HHT, HTH, and HTT. And again, I can compute the probability of A by just counting how many elements are in A divided by how many elements are in omega. That's a really simple way, because these are all equally likely to happen. If these were different probabilities
7:13
of these happening, I'd have to compute my probability slightly differently. I can do other things. I can say my second flip. Second flip is a tail. So let's call that event B. And let's just-- we're going to do this the slow way. Let's enumerate all of the possible ways. My second coin flip can be a tail. So I have H, heads tails heads, heads tails tails, tails tails heads, and tails tails, tails. So that's my set B. And these are two different events.
7:53
There's one event where my first flip is a head, and there's another event where my second flip is a tail. Turns out these are completely independent. The first flip and the second flip are independent. We'll talk about what that means later in terms of probabilities. But these are just two outcomes that could happen. And we can compute probabilities of the first flip being a head, the second flip being a tail and so on and so forth. We can start to think about the sample
8:19
space and the probabilities in terms of pictures like Venn diagrams. So I'm trying to keep my sample space blue here. So let's say that I have my sample space omega. And I'm just going to draw abstractly what this means. And let's say I have event A and event B. Technically here these events are independent. So it's a little different. I'll show you what that means in a minute. But I'm just going to define roughly these different sets. A set A, set B and a set omega.
8:59
And there are things we can define in terms set theory that are going to be really useful in probability theory. So we can define this notion of a union. So A or B is any element with everything in A or B. So either A happens or B happens. So either the first flip is a head or the second flip is a tail. And the way we think about that is everything in the set A and in B. So we basically take everything in set A and everything in set B, and we combine them together.
9:45
And I really should be careful to not use the word and because that is the exact opposite of what I mean. Everything in A or in B, anything that is in either A or B. And that is going to equal-- I could literally just enumerate everything in A or B is HHH, HHT, HTH. It's in this one and this one. So I'm not going to double count it. HTT that one's also in this one and this one. So I'm not going to double count it. And then TTH and TTT. So this set is bigger than A and B. So this one
10:30
has six elements because it's anything in either of these two. So that's what A or B means. So I'm going to just going to draw that here like A or B is this set here of all of anything in A or in B, very common sense. I also have this set A and B. So A and B is-- A and B is everything that is in both A and B. This is called the intersection. And it means that for an element to be in the intersection, it has to be in both A and in B. So it would be this intersection region here, this region here
11:27
that's A and B. And we can compute A and B just as easily. So we have to go through and look at which elements are in both A and B. Be and very quickly we see that heads tails heads, and heads tails tails.
11:45
And that makes sense. These are the elements that the first flip is a head and the second flip is a tails. So this is a smaller set. The intersection generally is more restrictive. The union is more of a big umbrella more permissive, more inclusive. So union is everything that's either in A or B, intersection is everything that is in both A and B. And then there's another one which is the complement. That's useful too. Maybe I'll just pick orange. So a complement is everything not in A.
12:24
And I don't want to enumerate it. But it's all of the things not in A. So since A is the first flip is a head, then everything not in A is everything where the first flip is a tail. And pictorially, we would draw that as the set of everything outside of it. It's literally everything that's not in this set A. And these are the three most useful of concepts that we're going to use all the time. So we're going to compute the probability of A and B
12:57
or the probability of A or B or the probability of A not happening everything but A. And we're going to use enumerations. We're going to count how many elements are in the union, in the intersection, in the complement to compute those probabilities Very, very useful. There are some basic properties here. Like there's other things like commutativity A or B is the same as B or A, A and B is the same as B and A. A or B or C it doesn't matter what order you do the or.
13:34
A and B and C doesn't matter what order you do that. It's the same answer. So that's commutativity and associativity. These are commutative and associative operations on these sets is the mathematical way of saying that. But it's very common sense in terms of these pictures.
13:57
If I really wanted to be clever about this or if I wanted to be really, really accurate, what I'd actually draw, maybe forget three coin flips. Let's just say there's two coin flips make my life a lot easier. So I have heads heads, heads tails, tails heads and tails tails. What I could do is I could say that the first flip is a head would be all of these. So say this is the first flip being heads. This is the first flip being a tail. And let's say that the second flip being
14:33
a heads is this half and the second flip being a tail is this half. That would be element B. So this would be B. The second flip is a tail. This set would be A. This is A. This is B. And it's really easy now to see that the probability of A and B is 1/4. The probability of A or B is 3/4. The complement of A is 1/2. You can do this really, really easily. You can actually make this diagram make sense with the problem at hand here. Good. OK. Cool. So the last thing I want to say is, again, we
15:21
have this probability measures the likelihood of some event happening. And we have these sets which determine the sample space of possible outcomes, and the space of how many of those outcomes are consistent with some event that I'd like to keep track of that I'd like to compute the probability of getting a head on the first coin flip. Now, here we have in all of our examples, we've dealt with events that have equal probability. It's equally likely to get a head or a tail.
15:58
But I could have a biased coin. So before I shaved today, I was thinking about bringing a biased coin where it would have had more likely chance of getting heads than tails, and then it would be a little bit different than just counting the number of outcomes of A divided by the number of outcomes of omega. I'd have to weigh those by their likelihood, by their probability. So a probability is actually a measure. So this probability p is specifically a measure on these subsets.
16:36
It says how likely is a given subset. So it is a map from subsets of omega to the real numbers, to the reals and specifically to a number between 0 and 1 in the reals. So this is mathematical language that says probability for every single set that is a subset of omega. For every subset A, p is going to assign a value to that subset, which is the probability between 0 and 1 of that subset-- of a random element being in that subset, and this sounds abstract, but it allows
17:27
us a lot more flexibility. Now, I can have the probability of heads and tails be different. I can have the probability of my second coin. I can have one fair coin and one unfair coin, and I can define the probabilities of each of those events with this probability measure p. So this is abstract. And if this is taking a minute to sink in that's totally fine. We're going to come back to this and we're going to develop this more. But this probability measure has some properties
17:57
that I want to tell you. The first most important property is that p of omega always equals 1. This is essentially a statement that the probability of something happening is 1. If I flip two coins or three coins, one of these things is going to happen. The probability of omega is equal to 1. So this is a property 1. Property 2 is that if A is a subset of omega, this is A and it's a subset of omega, then the probability of A is greater than or equal to 0. There is some chance that A will happen.
18:45
There is some chance it could be 0. But it's going to be-- if is a subset then it has a probability. And three, the third property if A and B are disjoint-- disjoint means that they are completely separate. That would mean A and B have no overlap. So this means A and B equals the empty set. They have no elements in common. Then the probability of A or B is equal to the probability of A plus the probability of B. These are the basic axioms of probability. And from this, you can build almost all
19:41
of the properties of probability. Now, this is pretty abstract. We don't really think about sets and measures every day on simple problems. But for complicated problems and for theorems, you're absolutely going to need to understand what a set is, how to think about these things, these basic axioms, and how to use them. And I'll point out this idea here of-- I drew this picture of these probabilities. I literally am thinking that the area of A divided
20:12
by the area of omega is the probability of A. And that's a cartoon that helps me think about this. Like, if I'm throwing darts at this board and I'm uniformly-- I've got equal chance of hitting anywhere in omega, the chances of me hitting A is the area of A divided by the area of omega. So that's how I think about this pictorially for these problems. And there's a lot of follow ons from these axioms, you can derive some other things. The probability of A complement equals
20:45
1 minus the probability of A. That means the probability of A not happening is 1 minus the probability of A, very common sense. And you can think about it in terms of these notions. Another one, the probability of nothing happening of the empty set is 0. If A is a subset of B, if it's inside of B or equal to B, then the probability of A is going to be less than or equal to the probability of B. And then the last one and then I'm going to stop. This is already getting long. The last property.
21:25
This is actually a pretty important property. And I want to show. I'm just going to do it over here. So I don't get into too much of a space issue is that if I have the probability of A or B, this is A or B here the probability of A or B is equal to the probability of A plus the probability of B minus the probability of A and B. And again, you can think of this pictorially almost as the area of A or B equals the area of A plus the area of B minus the area of the intersection of A
22:09
and B. Because this piece gets double counted if I just add those up. Let's zoom out for one minute and we'll come back to this occasionally. But I just want you know to this actually goes deeper. There are pretty big fields of calculus and probability theory called measure theory about how do you measure exotic sets. Here these are discrete sets. How do I measure the size of natural images in image space? That's a weird thing to measure. How do I measure the Lorenz attractor in three space?
22:50
How I measure fractals? How I measure Cantor sets. You can go really deep down the rabbit hole, and you can assign probabilities to things that are on weird sets that I'm not talking about here. This is like the page one of a book on measure theory. But the basic idea is that formally, we define a space of all the things that can happen, and we define events to be subsets of that sample space. And for every possible set, we can define a probability of that set of that event happening.
23:27
And as long as these three things are true, that is a well-defined probability mapping. As long as these things are true for all sets A and B, then it is a well-defined probability. And so you're going to be building these probabilities for different sets for different random experiments, coin flips. How long you're going to wait at the DMV before getting helped, probability of failure of a part. All kinds of things. We are going to define a sample space and a probability measure.
24:01
And as long as these are true, then it's a well-defined probability. And I can do these common sense things with that probability measure. OK. Thank you.
Here to help you learn
Sign in to keep a private conversation with your course tutor.
Ask me anything about the material…
Sign in to chat