Skip to main content
Loading Explore guides…
Binomial Distribution · Acedra
Home
Explore
Computer Science
Binomial Distribution
Sign in to save
Chat
If YouTube asks you to sign in, open the video on YouTube.
Watch on YouTube
Reload player
Transcript
163 segments
Copy
Hide video
0:05
I have a couple quick announcements before we get started today um the first quick announcement is we use this thing in cs19 called high resolution course feedback basically goes like this have you ever been a class at a mid-corder evaluation it's too late to change by them but we're probability people maybe we don't need to sample everybody at one point in the quarter so in cs109 we do the slightly more elegant thing where we're constantly set randomly sampling students to get some feedback
0:32
so we can make the course better and that means we can adjust immediately we don't have to wait till like week six or week seven to make changes so you might get an email that says something like please submit Anonymous feedback for your class uh at the end of the course we see you complete the survey of course we see responses but we don't see your individual responses not connected to uh your participation you guys have said some wonderful things thanks for all the positive feedback on
0:57
how cs109 has gone there's also three things that we want to adjust to give you motivation to actually respond to these because we care about what you say the first one can we have more Virtual Office hours I brought this up with the Tas and they're like yeah we in fact added four brand new office hours and now there's going to be a total of eight hours that are going to be Zoom only because there's some people for whom coming to a physical app store is really
1:20
difficult we have so many office hours in cs1 I thank you guys to the Tas you guys are amazing use them the Tas are there so that you guys can get help if you need it some people ask I wish that the secret code thing was put into the recordings aha I can do that every day I can say what is the secret code and today it is binomial bunny all one word all lower case y binomial because we have such an exciting celebratory lecture today and it's kind of about binomial distributions uh and then people
1:50
mentioned that the internet in Nvidia is not that good I couldn't just immediately fix that one but I did start the process of seeing if we can do the deeper fix so that more people can connect to internet at the same time thank you guys all for your wonderful feedback as always you can follow along from the problem set there's a a place where you can write in your answers to some questions that we'll talk about in today's class uh and also you can mark your presence
2:15
okay oh we have such a fun this is such a celebratory lesson you guys have worked so hard and now we can start the process of putting your hard work into nice packages so that you can solve harder and harder problems you have a couple learning goals for today when you walk out of today's class you should be able to recognize and a binomial random variable when it shows up in the wild and be able to use it to solve probability problems and similarly for this thing called a Bernoulli random
2:40
variable and then finally we're going to be talking about variance and you should know how to calculate variance for a random variable so three learning goals you guys in okay let's do this I have a hook for you guys um and it's the hook is is peer grading accurate enough you know one of the things that people have thought about for scaling online education is to have peers grade each other you can imagine it'd be a lot easier for us to grade if we just had you guys grade each other's
3:06
work but does that work and is it accurate enough to be a feasible uh mechanism for a course we'll talk about that by the end of today's class and you'll probably really have to understand variants to be able to understand that discussion okay we're gonna get there by the end but we have to go through a few different steps to get to that point a little bit of review in last Friday's class we introduced this idea of a random variable and a random variable is the equivalent of a
3:34
variable but brought to the world of probability it's a value which is going to take on a number but it will do so with probability we don't know exactly what number it's going to take on yet and so we have to think about the distribution of different values that it can take on so certainly one of the most important things about a random variable is that relationship between the values it can take on and probability and we call that the probability Mass function
4:02
so if we declare something to be a random variable we'll often use a capital letter to distinguish it from like a non-random variable so why help welcome to the world why you are a random variable if that's a random variable it's not an event you can't ask what's the probability of Y you need to ask the probability questions about events but if you ask any you know any less than equal to not equal to question on why you will end up with an event so this is the same as
4:31
saying the event that y takes on the value to so why it's going to take on value probabilistically we now have an event which means we can now uh ask a probability question similarly y less than three also an event if you ask the probability of an event you will get a number between zero and one and that all leads to this interesting but sometimes confusing at first conclusion that this is a function this is a function where you're putting in different values and it's returning
4:58
you the probability that your random variable takes on that value and this is the be all end-all of random variables if somebody gives you this thing called the probability Mass function which relates the different values you random variable can take on to its different probabilities thank you person you've just made our life very simple now any probability question involving this random variable will be straightforward a little bit of Ruby random variables why are they such a big
5:23
deal it just feels like no notation and the reason there's such a big deal is because it gives other people a mechanism to provide you with this probability Mass function and if somebody gives you a probability Mass one they've done a whole bunch of work for you thank you person who did a lot of work for me we can now use all that work in our Downstream probability tasks a probability Mass function that relationship between the value random variable can take on and its probability
5:47
can be expressed as an equation it can be expressed as a graph it can be expressed as code lots of ways that somebody can communicate this relationship between values a random variable can take on and probabilities okay so Random variables there's a whole bunch of things that we're going to want to know about random variables today we're going to cover this spectrum every random has a semantic meaning the most important thing is the probability Mass function and then we already know about
6:14
this thing called expectation that's a summary statistic of your random variable where you boil down all of its interestingness into one number hopefully a number that best captures the random variable expectation we defined as this is kind of like the weighted average over all the values the random variable can take on so for every value of the random variable can take on you multiply the value by its probability and that kind of gives you a weighted Center of mass of your random variable
6:40
this is a summing loop as you guys already know a lot of review three things that made expectations so interesting was linearity expectations sum a sum of expectations and this rule of unconscious statistician we stated these two in a little bit of depth and we said we're going to hold off on that one until further in the class which is going to be today today we're going to go deeper in unconscious past decision but not quite yet it's worth talking about expectation to
7:07
give you the idea of exploitation of a sum if you have a bunch of data and you want to calculate the expectation of this data how would you do that there's no wrong answer to this well there is a right answer to this but I just mean this as an exploratory question I haven't told you how to do this already but you guys want to take a moment and just like get those cobwebs out and get thinking probability talk to the person next to you think about how would I calculate expectation if
7:29
somebody just gave me a data set what we have the definition for expectation right there wouldn't you oh how'd you do that think about it with the person next to you for just a second and we'll talk about it all together
7:47
just a warm-up we haven't told you how to do it but anyone have a guess I love guessing I feel like guessing is one way of exploring what you don't know as well yes an idea over here yeah you could just take the average it turns out that's exactly right but why why is that the right answer if you were to calculate like accent p of X for each thing is like the probability of each thing happening is one over the total number of things deterioration of exactly that idea
8:18
um so that is right it turns out you just take the average of all your numbers that is what the expectation is that's why some people think about it as the weighted average um and here's why that's legitimate thing to do for every probably x equals X using the original definition of probability we have that limit definition of probability that's about equal to how many times does X take on that particular value divided by how many experiments were there so if Little
8:45
X is 3 or 10 you're going to say how many times did 10 show up divided by n but that 1 over n you know it just gets pulled out here and this sum over x x times the count of how many times x shows up is the same as just saying let's sum over each of the values and count them once and so long story short if you want to know expectation from data set it's just your average of the values like you could just sum them all up and divide by n and that would give you expectation
9:18
and this uses a little bit of interesting notation certainly if that didn't just pop out uh something worth checking out a little bit more in depth that does give me a nice way to talk expectation of some sum of expectations like let's say this is X and this is y and we have a bunch of data on X and Y for we could just create a third column which is the sum of X and Y and it turns out that no matter how X and Y is related the expectation of x if you calculate that number take that average
9:47
if you take the average of this column if you add them together it's the same as taking the average of this last column and it doesn't matter how X and Y are related no matter what this law always holds linearity of expectation you know if you take a random variable and multiply it you kind of like elongate it's probably Mass function if you add four you kind of shift it's probably Mass function and no matter how you manipulate a random variable if it's a linear manipulation
10:12
expectation Still Remains easy to calculate okay but is expectation enough no expectations not enough it's not complete information you know expectation is just taking a whole random variable bearing it summarizing in one number like the average of your values is not complete information it's just the average of your values the probability Mass function that's your complete information okay that's the end of review it's been a weekend I hope you guys are caught up questions
10:48
information to reconstruct the pmf yeah so we're going to get into more moments today moments is a fancy word for like higher order summary statistics uh if you had I mean there's infinitely large moments so we can talk about that after class it's an interesting question yes when we take the average do we just assume that everything has the same probability yes so you're kind of assuming that every single thing was generated well not everything has the same probability but when you talk about
11:18
like the probability that x equals x equals like five I'm assuming that that's about equal to how many times five shows up in the so count x equals five tells me how many times does X actually equal 5 in my data set uh divided by the number of things in the data set that's what I'm assuming yeah expectation some people call it the weighted average yeah exactly it is that's very correct thing to do okay let's jump in you guys are here we've made so much profit progress we talked
11:57
about all the counting Theory you need we've talked about the core probability there's like you know seven or eight really really important ideas that we've talked about a lot of total probability Bayes theorem you know a lot for you guys to understand and now we get to relax into the nice comfortable happy world of random variables welcome to this third section of class and when we talk about random variables I'm going to say it's nice and happy because we get to just celebrate what
12:21
other mathematicians have had and use their great ideas and bring it into our tool sets and I'm going to bring you through some of the classics we're gonna really natural phenomena that every person in this world should know about and these natural phenomena that lead to very very helpful random variables are you guys ready whether or not you're ready we're going to jump in we're going to start with the most classic random variable the number one if I could say everybody in the
12:45
world should know this random variable it would be this one there's lots of stories that involve a very common piece of probability in fact we've seen this story before you have n independent Trials of the same experiment so at every single trial has equal probability B of being what we call a success and you want to know what's the probability of exactly K successes this shows up all the time we saw it very concretely in the world of flipping a coin so imagine you're
13:18
going to flip a coin n times and you want to know what's the probability of getting K successes there's very few moments in the world where I actually care about that with coins but there's so many things in the world that fit this narrative you know you have n voters and they each have a probability P of voting for you and you want to know what's the probability of getting exactly K votes and you know if you can know exactly K votes you could do K votes K plus one
13:42
votes add up those probabilities so many narratives fit this exact uh formula here's a couple examples how many heads do you get in N coin flips it's like n independent trials and you want to know how many successes you get in those if you randomly generate a bit string of length n and you want to know how many ones are generated if you have a thousand computer cluster and you know that the computers crash independently with some probability and you want to know what's the probability
14:14
of different numbers of computers crashing you can calculate that and in fact when Computer Sciences like first made Google and they started to First make like large social networks they would have their servers hosted on thousands of computers and as the people who really understood this natural phenomena who are able to predict okay when are our servers going to be crashing and they were able to identify uh how much extra load that they would need to know you know voting for a candidate we
14:39
mentioned uh quickly and we'll talk about this later but you know if you want to make sure that a jury is not unfairly selected you could use this Theory to figure out uh if things are probabilistically out of distribution all these stories really involve the same key metaphor this metaphor of I have a coin and I'm flipping it n times it's got a p probability being heads I want to know how many what's the probability exactly K heads and we've seen that before in class and
15:11
in fact it's so important I spent a lot of time trying to make a really cute thing in the course reader so you can go through it one by one and you can see exactly why if you have 10 heads n equals 10 heads and has a probability of 0.7 that the probability of getting exactly six heads is going to be this beautiful beautiful equation down here
15:38
the probably exactly K heads in N coin flips is going to be n choose k P to the power of K times 1 minus P to the power of n minus K and you don't have to have memorized the derivation that's kind of the point here but if you're curious about the derivation you should know all these steps because it's a beautiful combination of all the core probability we've learned so far we talked about this earlier in class the important takeaway today though is what if you didn't want to rederive this
16:09
every single time you saw that metaphor what if you just want to package all of that work we did when we did the end coin flips into a beautiful equation that you could use in all other different contexts somebody beat you to it and in fact back in the day somebody was like this story of n coin flips where the pr with the probability of coming up heads shows up so much in different probabilistic real world examples I want to make a random variable around it and
16:37
I'm going to call it the binomial random variable huzzah okay anyways a long story short this person who invented this binomial random variable that we're going to learn about today is called Jacob Bernoulli also known as James uh Swift mathematician uh he's one of the many mathematicians of the Bernoulli family uh the Bernoulli random variables named after him even though he is most famous for identifying the binomial random variable this is kind of cute I I when I was in
17:06
PhD I had an advisor that's like my advisor and then the advisor of that advisor Daphne color is like my great grandmother or my grandmother and then if you keep going there's like more and more advisors that are in my Heritage and he's my academic great to the 12th grandfather which is kind of cute not that important and I think I went with celebrity look-alike Ice Cube but I don't know okay the great thing here is if you can recognize anything that fits this story
17:33
end coin flips probability P of success we care about how many successes you get you get to just write this line in your problem set and you inherit all the mathematics related to this story you can say I have a random variable I'm calling it X it is distributed as a binomial we write bin for binomial we don't have time to write the whole binomial no one's got time for that bin is enough uh and this binomial always takes in two parameters your story is allowed to have two pieces that you can
18:04
Define how many coin flips are there in and what was the probability of a success on any trial P anytime you recognize a story you write that line and you inherit all the mathematics because everyone knows the mathematics around X and if they don't know it they should learn it because it's such a beautiful natural phenomena most importantly if you declare X to be a binomial you automatically inherit this beautiful equation and this beautiful equation says you've told me X is a binomial
18:34
I will tell you the probability that X takes on the value K you don't have to rederive it this probability Mass function for binomial is already known the derivations in that multiple coin flips question uh it has some terms is called the binomial term because it's the multiple coefficient uh and it's just going to be this binomial term P to the power of K one minus P to the power of n minus K you've already told us what p is you've already told us what n is and when you put in K you've got
19:05
all those three values and it can give you the probably that x equals any value now if you go to our course reader under random variables you'll find the binomial distribution and here it is and you can try different values of N and different values of P and as you try different values of n in different values of P it will change the probability Mass function that's given for you each of these columns can be calculated using the pmf equation um and but they each have their own
19:38
value this is saying if n is 23 and P is 15 the problem is getting exactly five successes is 0.137 you can use this tool go for it have a good time get some intuition okay probability Mass function as a graph I'm going to show you a couple this is when n is 20 20 experiments probably 0.6 of getting heads and this is the chance of getting seven heads and eight heads and nine heads isn't that just a beautiful little curve like let's just take a moment and appreciate the Aesthetics
20:08
ah good looking curve okay now it's not always the same you know if you took exactly that same n but change P to 0.1 then we get a new curve we get here's the probability of getting zero successes that's much more likely not the probability of success is point one and people can give you stories with different ends and different P's and you will automatically have access to this graph okay I got a little extreme there let's calm down okay now I want to revisit this question we
20:38
talked about before we said let's say you flip three coins every coin comes up heads with probably p x is the number of heads now I'm going to Define X to be a binomial and as soon as I Define X to be a binomial I know these things automatically I know that the probability of x equals zero is just plugging into that equation you know n was three here K is one and P is equal to 0.5 and if you just put those three numbers together you get question answers for all these we've derived these before
21:11
but you could have solved this problem without deriving it if you just recognize the coin flips as a binomial expression you're done you don't have to drive anything you just have all your answers okay let's try another example here's what a binomial question could look like in the while it's hiding it's lurking it's not telling you it's a binomial distribution but if you can recognize the biology distribution in the world then you can solve these problems really easily in fact the
21:39
solution just becomes for quick lines a real world problem thousand ads are served every ad is clicked with a probability of 0.01 otherwise with a probably a 0.99 people are ignoring the ads most of us ignore the ads come on let's be real and then you can ask questions what's the probability of exactly 10 clicks and if you want to answer this question before you knew about binomial random variables you'd really have to go back to doing all of this work you would have an
22:13
answer that would be this long and you go through all these steps but now you know about a binomial random variable you can answer it in four lines you say I know H it's going to be the number of Clips it's a random variable and it's not just any random variable because of the story this is the same as a thousand coin flips each with a point zero one probability being heads and the question is just asking what's the chance of 10 heads and because of that again declare H to be a binomial
22:42
this is a powerful line I'm saying I've seen the binomial metaphor in this story and once you've written this line you get this equation for free you know the probability that H takes on the value of K and you get that it's going to be a thousand choose K .01 to the power of K and 0.99 to the power of a thousand minus k then if you plug 10 in for K what's the chance of getting exactly 10 coin flips you know exactly what that is amazing you guys don't have to drive
23:11
things but now you have a new challenge you have to recognize this natural pattern in the wild yeah questions
23:21
zeros clicks plus probability of 10 plus oh that's a deep question okay so far with binomials we can just say what's probably exactly 10 but let's expand our abilities what if you wanted to know the probability of exactly zero so the probability that H equals zero or H equals 10. a challenge and an important challenge that you can solve with what you've got so far and in fact so important I'm just going to give you guys a second think about it talk about what the person next
23:55
to you either solve this or come up with a question regarding what we've talked about so far that you'd like to see clarified anything confusing talk about with a person next to you or try and solve this one because it'd be nice to be able to do more than just exactly 10 clicks eh okay go for it
24:37
okay before we go into answers anyone come up with a clarifying question that they think is relevant you're like why here's a good question why did no one tell me about the binomial before it's so present in so many natural phenomena maybe you've heard about before if you hadn't what a beautiful day in our lives okay how are we going to solve this problem H equals zero or H equals ten you can solve it with the tools I've given you what's the idea here what makes probably or easy
25:11
but why can you do that you can't always just take an or and turn it into plus something has to be true it needs to be expensive but are they I challenge you on them no no sorry sorry that was too much they are you're absolutely right they are mutually exclusive because it's not possible for H to both be zero and ten those are different numbers H is either 0 or 10 the outcome spaces are mutually exclusive you can't have both things happen at the same time and because of that the probability of
25:40
or becomes easy it becomes the probability of this times the probability of H equals 10.
25:51
I'm going to tell you we can can answer other questions what if you care about the probability of less than 10 clicks you think this is like a failed campaign we haven't talked about less than 10 . how are you going to do that no no think about it think about no don't talk to your partner just think how would you do that less than 10. be inspired feel the inspiration
26:17
okay hopefully you're inspired to come up with this right you've got probably that H equals zero or H equals one or H equals two or H equals nine what about ten not less than ten man those those off by ones are killer and you're like each of these are mutually exclusive you can't be zero and one you can't be zero and two you've got to be only one of those numbers and so this is going to be equal to the sum looping from x equals zero up to and including nine the probability that H equals X
26:59
wild okay anyways we've gotten ahead of ourselves but I'll just leave that up there this was the probability that h H is less than 10. which you might care about you oh my gosh there's so much we can do once you know it's a binomial random variable Yeah question form for B of h less than 10 no no no this is it oh and what would you do for H equals x I'm like why can't you just stop there Chris
27:30
we already have the equation for that because we declare it as a binomial you automatically inherit this beautiful probably Mass function which is the probability that H takes on a particular value and you would just plug that in nine times question is is there a shorter way of doing it no you must do it nine times hopefully you have a computer if you have a computer nine times not a problem in fact speaking about having a computer I love that you guys are computer scientists it makes teaching
27:52
this so much more reasonable because I can tell you haha stats just gives you a way to do this stats is a python Library you can say from scipy import stats this is a lot to write but you'll be very happy because after you write stats.binomial.pmf it gives you this equation in Python you have to say what is n and you have to say what is p those are the two parameters your binomial and then you have to also say what is K so if you have a thousand ads with a .01 probability of anyone clicking the
28:21
probability that exactly 10 ads are clicked on are it's a global thing we're not the only ones who know about this somebody told python uh and you could ask any question you care about any value of n p and K and you can get an answer so certainly if I wanted to calculate this I would write a simple for Loop run it 10 times uh and I would do stats.binomial.pmf I comma a thousand comma 0.01 and I get the probability that is less than 10. oh amazing
28:55
so I meant to just give you guys a chance this is super important that you can do this now it's your turn to do I've done this a lot but this is like the 10 out of 10 you must must be able to do this walking out today's class 1000 ads serve probably 0.01 of getting somebody click on one was a probability of exactly five clicks you have to be able to solve this you have to be able to answer a question like this as of now and so if you don't good time for a question do do
29:27
okay what you have to do four steps declare a random variable I could call it h for the number of people who click on an ad declare H to be a binomial I would say it has n a thousand and P 0.01 and then I want to know what's the probability that H equals five and you could just go into let's say your favorite Pisa app
29:52
now we find out if my Wi-Fi is working um okay nice ad Chris and you can do n equals a thousand P equals 0.01 whoa whoa whoa no one can see that can you guys see that now I could say um the probability of 5 equals stats dot binom dot pmf for probably Mass function and 5 comma n comma p and then in theory if I print this we'll find out if I remembered all of those commands off top of my head oh because it's not loading scipy okay [Music] do you know Python's also on your computers python
30:46
let's run all of you guys stats is not defined I don't know why I didn't copy the Top Line oh has no module called python okay
31:02
python 3.9 I recently got python 10 and my python 10 doesn't have stats that's not important right now what is important is now I've run this on my computer and it told me that zero point three seven is the probability that my random variable equals five so this is what you should feel comfortable doing beauty of recognizing the binomial random variable you see a question like this and you don't have to rederive everything you inherit the probably Mass function it's even in scipy
31:36
okay do you guys feel like you guys could do that in the Wild on your own I see a bunch of nodding heads that makes me very happy okay
31:47
um you know you could put this into the problem set app to see this graph and just look it up on the graph that's one way to do it but obviously I'll use scipy let's try this again you guys kind of know where I'm going but I just want to give you this idea that it shows up in lots of places seven runs of a program each run crashes with probably a 0.3 what's the chance of exactly three crashes out of your seven so maybe you have that program that has some intermittent bug and you're trying to
32:12
figure out will it pass the auto graders when I submit it to cs11 seven runs a program each are independent each crash independently with probably 0.3 what's the chance of exactly three crashes how would you solve this talk about with the person next to you while I get the board ready okay
32:33
who here there's like a bunch of random variables there's a binomial there's a poisson sometimes something's not a classic random variable you just have to drive it from the beginning who here thinks this is a poissa you're like Chris you haven't taught us that not me who serious thinks that it's like no classic random variable we have to drive everything from first principle I love it who here thinks is a binomial rocking okay so if you want to solve this problem you just say x is defined
32:54
as a number of crashes and the next line in your solution is X is distributed as a binomial so this is you declaring it's a binomial and you have to give me two parameters n and p can you guys just shout out what's in wait how come it's not three here like because that's not what it is and is how many experiments there are not how many successes you're looking for what's the probability of success in every experiment now this is kind of interesting we've defined success as a crash
33:27
and in what way is crashing a program a success yeah the event of a crush happening yeah it's the event of a crash happening is what was the thing we're trying to count uh but anyways just it's like the heads the head's equivalent we're thinking of crashes being like the equivalent of getting a heads on our coin flip and then you define what's the question asking the questions asking what's the probability that x equals three and at this point you could just plug in the binomial distribution you
33:57
know it's seven choose three P which is 0.3 to the power three and 0.7 to the power of 7 minus 3 which is four and I know people over there might have lost this but it's exactly what you expect it would be cool and you're done you could put this into a calculator you could use the Sci-Fi function we just showed you which does all this in one go you now have the ability to calculate binomials you don't have to like if you had to redefine derive the binomial when you saw a
34:27
question like this your midterm would be brutal okay oh here's a fun one binomials they're everywhere so here's a Gallatin board if you guys haven't seen a Galaxy board I actually have one over here look really closely now you can't see that we're going to use the DOT cam in just a second people who do the doc cam can you help me get set up but just in a second don't switch over yet but just know it's coming a gelatin board looks like this there's all these pins and we drop these balls
34:55
on the pins you're like what does this have to do with probability wait for me wait for me when the balls hit the pins they either go left or right and every single time they hit a pin we think they have an equal probability of going left and right oh marbles whatever every single pin is independent so what happens on this pin doesn't change what happens on the next pin and eventually your ball ends up in a bucket at the bottom and we're not going to just drop one
35:20
marble we're going to drop a whole bunch of marbles in fact we're going to be dropping millions of these ones wait they never fall out if they go all the way to the right they end up in this bucket ah so I'm going to Define R to be how many uh sorry sorry
35:44
I'm going to define B to be how many times did you go right and how many times can you go right there's one two three four five chances where you could go right or left and so I'm going to say B is going to be the number of times when I chose to go right if I choose to go right zero times I'll go left left left left I'll end up here if I chose to go right five times so in the first level I go right and the second level I go right and the third level I go right fourth level right I go
36:16
fifth level or I go right I'll end up here if the number of times I choose to go right is 2 no matter where I choose to go right twice I'll end up here so if I go left left right left left left right right I'll end up here the number of times you choose to go right determines which bucket you end up in and the Beautiful Thing is the number of times that you choose to go right is going to be binomial there's five experiments and each experiment has a 0.5 chance of giving you a write
36:48
and you can figure out the probability of a marble Landing in each of these buckets using the binomial probability Mass function the probability that a marble goes into this bucket is the same as a binomial with N5 P equals 0.5 being 0. and you can calculate what that is a 0.03 you can calculate the probability of it going to the one bucket that's going to be 0.16 so you can do the same thing for all the different buckets you know exactly what this probability Mass
37:14
function is can I go to the DOT cam is that a reasonable thing to do right now
37:21
oh yeah okay let's see if we can figure this out though there's this thing called a doc cam wow what a time I'm gonna need this to be balanced though
37:34
oh no did I press the off button I just got it where I wanted it
37:43
oh man doc cam did I break you no you're back you're alive you are alive amazing oh oh it's upside down that's hilarious imagine if you flip this upside down so this is just kind of like the binomial distributions prediction of where how many marbles would or the probably of a marble ending up in each bucket and look how good a prediction is oh how beautiful okay let's do it one more time wow that's it upside down going into I can't flip it upside down can I
38:14
no no okay you guys we just have to get used to looking at the world upside down wow amazing the probability predicts it so well yes thank you technicians we really appreciate it all right back to the slides
38:33
okay
38:37
here is a real one not a small one but a really big one and they built this somewhere and they made this huge construction and there's this music playing goes yeah yeah yeah yeah yeah all your marbles are going up to the top of your Belton Belton board all of them are going where are they going to land which bucket oh my gosh they're hitting pins and the number of Rights they take binomial distribution all of them going go go go and through all of this chaos such predictable order
39:09
it comes is it going to follow the distribution that the binomial predicts yes again is it going to do it this time I don't know does probability lie sometimes people lie with probability but this time the probability is right it goes It goes It goes you get the bridal distribution oh my gosh from chaos you get such order wait I'll that's actually the next example uh it is such a beautiful thing in this world that from all of this uncertainty that you get such incredible order and it's a
39:44
natural phenomena you now know what to call it is the binomial distribution and it shows up in lots of other places for example what's binomial distribution doing in my basketball game let me tell you if you don't know this when people get to the playoffs in the basket in basketball they play seven game series and in those seven game series if you win four of those games you have just won the series and so often people care not about the probability of just winning one game but what's the
40:13
probability of winning a seven game series so out of those seven games you play what's the probability of winning four or more my claim for you is that if you wanted to drive this from first principle it would be very very hard you would have to kind of go back to the whole coin flipping derivation but if you can see the binomial here your life is a lot easier so take a moment look at this question can you see the binomial so the Warriors are playing the Bucks and the best of seven series
40:39
sorry this is before I knew who they actually play uh they won how crazy is that wow it's our local basketball team yay okay um what's the probability of winning at least four games and see the binomial
41:01
it's seven coin flips and a head is a win so X my random variable I'm going to be finding to the be the number of wins in my seven point flips and as soon as I do that my next step is to let the whole world that X is binomial the number of coin flips is seven the probability of a heads on each coin flip is a 0.55
41:26
but that just tells me the whole distribution for what's the chance that they win zero of those games what's the chance that they win one of those games what's the chance that they win two or three four games means that they win the series five games also means to win the series six games win the series and seven games also means that they win the series somebody's like wait don't they stop playing if they win four like they don't bother with the next three like if they win win win win win
41:52
no one plays those other three in my world they do play those other three uh and it doesn't change the outcome of this probability experiment thank you that'd be kind of fun but tiring for the players and maybe they get injured and then we'd be sad okay I'm glad that they do it the way they do it uh and this will allow us to solve the problem of the probability that they win at least four the question is asking what's the probability X greater than or equal to four
42:21
and we don't have an equation for that we only have the probability that X is equal to four but for those of you guys who've been paying attention to Mutual exclusivity you know that this is the same as saying what's the probability that x equals 4 plus the probability that x equals five plus the probability that x equals six plus the probability that x equals 7. every single one of these equations is given to you by the probability Mass function you can ask what's the probability that
42:52
binomial with N7 T 0.55 equals four and you can ask it for five and six and seven since they're mutually exclusive we can add them up and this would be the probability that they win the series so just to put that into the slides what I'm saying is Define a binomial with seven games and 0.55 and the probability Mass function will tell you what's the probability of zero wins out of seven not very likely they're quite good one two and three and what I'm saying is we're going to add the
43:19
probability mass of four with a probably mass of five with a probably a mass of six and the probability mass of seven that's what adding up each of these things is doing it's looking up each of the columns and adding them together if you did that you could write it as a for Loop and a for Loop in math looks just like this and everything in the for Loop you know we're plugging in the binomial distribution probably Mass function I is the index of my for Loop and it shows up in this probably Mass
43:44
function where K would normally show up in the pmf and if you put this into uh into something like PSI pi and since I uh what we could do is we could say for I in range four to seven or four to eight I think if you do this you get four five six seven yeah for Iron Range four to eight we're going to just oh I need to get a sum
44:18
for I in range zero to eight some probability plus equals stats dot binomial dot probably Mass function I because that's the number of wins we're looking for there are seven games and each one has a 0.55 probability so this is going to Loop four times and I will take on the value 4 5 6 7 and each time we'll say what's probably exactly I wins add those all together and we have some probability and the chance of them winning at least four games is 61. so even though it's pretty even chance
44:57
for any one game the chance that they win the whole series is pretty large I'll just make that quite big because going from math understanding to code understanding I think is something that you probably wouldn't have seen in other classes and I think it's very important if you want to start making Tools around this cool theory that we're learning this is like Theory to practice in one in one screen question yeah yes uh is it possible for a random variable to take on multiple different values
45:29
no it's always mutually exclusive because we think of it if it takes on One value it just can't be another one if they won four games they can't have one six games because we think equals two as being just that value such a good question so always mutually exclusive can always use the sum trick yes if they had to stop playing afterwards well they do and this is still the right answer even though they stop I'm so glad you asked that because it's such a common problem when people
45:59
do probability to get tripped up by the fact that they stop at four games in the real world that I want to take a moment and talk about this I've seen so many students make this bug can we just talk about it together here's how people make this bug you want to know what's the probability of at least K successes in N independent trials X is going to be the number of successes in end trials with probability of K so X is our binomial random variable some people say what's the problem that
46:27
X is greater than or equal to K they don't use the fact that it's a binomial random variable or they just think of it like this I want to choose case slots for my successes and then I'm going to say in those case successes you know each one has a probability P of being a success right and there's K of my successes so you choose your case successes you choose where they go and for the other ones they could be successes are not successes if they're not successes in the other slots then
47:01
you still have exactly at least K successes because we've chosen K of them to be successes you know if they end up being success is great we had more than K successes that also fits this definition does this seem like a pretty reasonable thing to do you know K successes each have probably P you can choose where they go turns out this is not the way to solve this you still need to use this sum from k equal I equals K to n and the binomial probability of your random variable equally exactly I
47:36
and a good way to show that this is a bug is to First think about what happens if the probability is one
47:46
so if your probability is one let's go back to this equation you have one times itself K times that's going to be one right but n choose K is going to be something greater than 1. and this is going to say that the probability that X is greater than K is going to be a number greater than one and it turns out that doesn't work you can't have a probability that's greater than one so if it turns if this case fails then there must be something wrong uh with this outcome
48:18
the reason we have this n choose K in this binomial distribution probably Mass function is because each of those outcomes of heads and tails were mutually exclusive you know if you had tails heads Heads Tails let's say there's four trials and you're looking for two heads heads Tails heads Tails each of these outcomes were mutually exclusive there is n choose K of them but because they're Mutual exclusive we could add the probably this row with the probably of this row and every single
48:50
row ended up having P to the K times 1 minus P to the N minus K so that's why we could if you add it up P to the K times P to the 1 minus K and choose K times the same as multiplying entries K by the probability of n one row so it relied on the fact that these rows were mutually exclusive in this derivation you have rows where you have like the heads are defined and you have blanks where there's no heads and you're saying I'm going to add up all the rows the probability of every
49:23
row is just the probability of the heads and I'm going to not care about what shows up in these other slots but it turns out you could end up with let's say we put a head here and a tail here with some choices for these other slots you end up with non-mutually exclusive rows which means you can't use this n choose K multiplier that we used in our binomial distribution okay um I think that answers one bug but I believe your question was slightly different your question was what happens if they
49:57
stop after four games so let's think about the case where the Warriors have won four games the first four games you have heads heads heads heads and I've what I've done is there's three more games and I've got every combination of like heads Tails heads here you go see heads heads heads Heads Tails Tails Tails heads heads heads every two to the three ways shows up here my claim for you is that if you added up all of these probabilities that that is the same as the probability
50:41
of just heads heads heads
50:47
on its own and there's a bit of a law of total probability thing going on here you know like these background contexts don't matter and I'm looping over every single possibility and adding them together and so by a lot of probability argument these things end up being equivalent to the probability of heads heads heads Heads not worrying about the other four that is a very complicated argument but um it you know if we thought about this not mathematically but intuitively I'm
51:16
saying you know all these cases are going to exist but when these ones exist what you'll experience in the real world is just they stop playing and even though I've thought about the futures of the what would happen if they did play in the real world You observe them stop playing but it doesn't change the fact that this derivation uh to do that I have over here is the correct one so thinking about seven trials and the probability of getting I successes out of those seven trials yeah
51:49
which means our n 20.7 right yeah but in my thing I treat it as seven and it just made this case happily work out if you wanted to just think about this case on its own I guess you could say okay if you wanted to drive this with n of four you would say in four experiments what's the probability of getting exactly four successes and then in five experiments what's it probably getting exactly four successes no that's problematic because they're not mutually exclusive
52:28
um but um so my derivation I always play out all seven games I always set n equals seven and P equals 0.055 which means I'm always making the Warriors play even though these are ghost games and in reality they wouldn't play them and the reason that this doesn't affect it like the fact that they're ghost games versus not Ghost Games doesn't affect the probability that this will give you the chance that they win
52:59
okay good questions okay big picture now if you can recognize a binomial distribution in the real world you get so much for free you get all of these things that you get in the course reader for free um you get the PMS equation you get that the expectation is just equal to n times p and you get a little bit more we're going to talk about that a little bit more after our pedagogical pause so I'll give people two minutes to just take a break solidify and crystallize
53:28
what we've learned so far before we walk into our next piece of today's class okay fantastic that was such a good question about the hey they don't actually play seven games they sometimes stop after four why does that not matter and I feel like that's best explained by just kind of enumerating over a whole bunch of possibilities so we're going to write that up for people curious about that detail and we'll make an End post for you guys okay as we last left off we learned about the
53:56
binomial distribution one of the most beautiful natural distributions in the world you can use it anytime you recognize it you'll start recognizing a lot in the world but let's go back to Jacob Bernoulli he's like I invented one of the most useful random variables in the world and they didn't call the newly random variable he's like sad face Bernoulli uh and he's like can I have a variable named after myself and people in fact were like okay you can have one and they kind of gave him the most
54:18
boring random variable to be honest uh he gave him the special case of the binomial where n is equal to one if n is equal to 1 what are the different values that your random variable could take on if x is the number of successes in one coin flip it can be a zero or a one which is kind of cool he kind of gets the probabilistic equivalent of a bit um but anyways yeah he gets his the Bernoulli random variable is named after him special case of the binomial if you declare X to be a Bernoulli random
54:46
variable it will take on the value one with probability P you have to say what's the probability of heads and it's going to be a zero with probably of one minus p if you find something that is basically a single coin flip and you're measuring number of heads on a single coin flip which doesn't happen all that often but you know it's very important to give you this terminology because it does show up again in machine learning literature a lot you define X to be a Bernoulli
55:08
random variable and you say x is a Bernoulli uh with probability P probably of success the expectation of this random variable it turns out is equal to exactly is p and here's a couple examples you flip a single coin and you want to know number of heads you get a random binary digit and you want to know whether or not it's a one and you want to know the outcome of it whether a single dish dive crashes and whether somebody likes a single Netflix movie it's a special case of a
55:35
nearly random variable I'm not gonna go too much into it because I think a lot of this is pretty self-evident um you can just Define if something says like I serve a single ad and people click with a probably p and they ignore it with probably one mysp C is going to be a one if the ad is clicked you can declare C to be a Bernoulli and it doesn't buy you that much the binomial buys you so much and the Bernoulli really just doesn't buy you that much uh and you can think of the though as if a
56:01
Bernoulli is a zero or one you can think about the binomial as being a sum of bernoullis and that's actually quite helpful so you can imagine if this is this Bernoulli is going to be like there's four coin flips and this Bernoulli represents whether the first coin flip was ahead it'll be a zero one this particularly represents where the second coin flips the heads it'll be a zero one this third Bernoulli represents how many coin flips our heads in that third coin flip it will be a zero one is
56:29
how many coins are heads in the fourth coin flip it'll be zero one you can think of the binomial of four coin flips as the sum of each of those bernoullis it's gonna be the sum of those zero Z ones zero if it's Tails one if it's a heads you add all those up and you get your number of successes okay and I'm going to give you a reason to have learned the Bernoulli we're going to use the fact that we now have a Bernoulli to come up with a simple derivation for the expectation of
56:55
binomial I told you the expectation of binomial random variable is just n times P now I'm going to drive it for you so X is going to be our binomial but we know that X is just going to be the sum of a bunch of underlying Bernoulli's so x i is going to be a 1 if the ith coin flip is a heads and it'll be a zero if the ith coin flip is a tailed and if you sum up all those zeros and ones and there's n of them you'll get the number of coin flips this is the very first time we've really
57:26
seen a random variable defined as being the sum of other random variables you add up all these zeros and ones and you get X which is the number of successes and that's quite interesting because if you want to do expectation of X you can now replace x with that sum you can say this is really the expectation of the sum one of the properties that I reminded you guys of at the beginning of this class is the expectation of the sum is a sum of expectations another way of
57:53
thinking about that is expectation always has the keys to the house like look at this beautiful house expectations like hey house I just want to get in there and like as soon as expectations on the outside of the house is like hi guys I'm inside because the expectation of sum is the sum of expectations what's the expectation of a single coin flip a single coin flip which has a probability of P of being heads well let's think about it let's say the expectation of y i
58:22
why I can take on two values we're going to take each of those values and multiply them by the probabilities one value that y I can take on is zero you can get zero coin flips in your I coin flip zero heads in your ice coin flip what's the probability of getting zero coin flips doesn't matter it's multiplied by zero what's the other outcome of the ice coin flip yeah one these bernoullis separate from binomials special case they can only take on value zero and one and what's the probability
58:51
of getting one heads in a single coin flip P yeah p is the probability of getting a heads so this whole expectation of y i is just equal to P there's zero times whatever one minus p goes away because explanation of y i is equal to P we can just substitute it in there and this is going to say the sum from 1 to n of p is going to be the expectation of your binomial so the expectation of binomial is n times p very hard to derive if you didn't use the fact that the binomial is actually
59:24
the sum of these bernoullis so even though the Bernoulli is just really not that interesting of a random variable sometimes it shows up in proofs like this and also you just kind of need to know it if you want to be like a a conversational probabilist in the world so anyway we have this binomial what a beautiful curve and then you have the Bernoulli it's like I'm zero or one and I'm one with p and I'm zero with one minus P deal with it but anyways it's a random variable too and it's a class of
59:50
random variables you can declare things to be bernoullis okay so here we are we will continue to grow our understanding of natural phenomena and classical ideas of random variables but before we go any further we need to talk about other things we can do with random variables one thing we know we can do with random variables you could summarize it into a single number expectation but expectation is a very lacking number what if we want to add a second number so we could summarize the random
1:00:21
variable a little bit more holistically expectation leaves so much to be desired in fact a lot of people who lie with Statistics are lying with expectations and my question for you is I want you guys to invent another summary statistic to go beyond expectation and we're going to start with a story of peer grading so we're going to think about somebody who is being peer graded by other people and they want to get the grade that they deserve imagine three different graders you have
1:00:52
greater a if this is the true grade that somebody deserves this is the distribution of grades that greater a gives sometimes they give the grade that this deserved but sometimes they give a grade that's way lower and sometimes give the greatest way higher here's greater B and here's greater C I want to find out if we can tell a story of who we want to be grading us and figure out what we would like to measure in order to talk about greater accuracy now think about this for a second do you
1:01:27
want greater a b or c grading U
1:01:34
the red lines the grade you deserve and this distribution is the grade that they're giving you so C is kind of a jerk if you're like I deserved a 70 it's like 50. I'm like but I deserved a 70. so I think a lot of us would be like okay see if it was the other way around they were giving us grades higher than we deserve we'd take them but because they're giving us lower grades we don't want it A and B have exactly the same expectation in expectation they give you the exact same grade
1:02:05
but who wants to be graded by person a and who wants to be graded by person B there's not a wrong answer here you're like actually b means you're getting the grade you deserve and a is like they're throwing a dart at the dartboard and adding it to the grade you deserve and like maybe you're getting a 40 or maybe you're getting a hundred and some people just like live on the edge like that uh but what if we had to put in words what's the difference between a and b
1:02:28
it's not expectation in expectation they're exactly the same now I don't want that and worried I want another term for it bread spread is a nice natural term there's a lot of spread here and there's a little spread here and it'd be nice if we could come up with a quantification of spread I'm going to give you a slightly simpler example where you can go into exact numbers here are three distributions with the same exact expectation they all have expectation of three
1:02:58
but the difference between these three distribution is how much spread that they have you know this has a ton of spread and this has very little spread I want you guys to invent what would be a good definition for spread so
1:03:16
how can we Define spread and I'm going to give you guys a second to think about it talk about with the person next to you I'm going to give you three whole minutes to think about how would you quantify spread and to make it easy you could think about this as being like 10 000 unique numbers and this is just the histogram of those 10 000 unique numbers that might make it easier to think about quantifying spread
1:03:43
okay let's get ideas let's throw them on the board how can we do it and I don't know if it's helpful but I imagine you know we have a whole distribution and I want us to think about two pieces of this distribution here's a chunk of the probability that's a certain distance from below the expectation and here's a chunk of probably that's a certain distance above the expectation how can we quantify this spread
1:04:10
looking for maybe somebody who hasn't said something today
1:04:16
yes well you can start by taking the distance from each data point to the expectation okay let's talk about the distance to the expectation
1:04:32
like that because you know when there's High spread you have these things that are really far away from the expectation when there's low spread you have things that are really close so distance seems like a very good idea there's lots of ways you can measure distance let me give you a bad one you could take the value and you could just subtract off the expectation and we can say we just want the average of all those you know give me the average of the value itself minus the
1:05:01
expectation so like over here this would be like a positive 5 and this would be a minus five why is that a bad way to measure distance
1:05:12
yeah it's like these positive ones will lead to a large value and these negative ones will lead to a negative value but if you take the average of these positive and negative they'll cancel out so what's a better way of calculating distance than just taking the value and subtracting off the expectation
1:05:29
oh okay I've heard two two good ideas you can take the absolute value so you take the distance you subtract off see how far it is away from the expectation and if it's negative you make it positive and then if you take the average of that over all of your distribution that would work and somebody else said don't do that I want to do the average of the squared distance so take the difference between X and its expectation Square it and I kind of want to average that over my whole distribution
1:06:06
and expectation is my function for averaging over the whole distribution which one is right
1:06:16
both right they're both right if we had gone back in time and we were inventing variance we could have made either of these the definition of variance and they both would have been reasonable people happen to have chosen this one so I think this one would have made absolutely good sense to get the Pun It's absolute okay question um different skills that use different definitions of like just kind of so like sometimes things prefer absolute values and something for first wearing there's
1:06:46
times when people when you're thinking about distances in lots of fields like when you get into machine learning there's lots of times people think about distances uh and people call this the L1 Norm people call this the L2 norm and there's times when you might care about these different distances there's even other ones Beyond these two and people have argued about them in lots of different fields and in probability they just happen to have chosen this one and I think every
1:07:08
time people talk about variants I've seen them talk about it in terms of this distance measure yes it punishes stuff that's further or further away which seems worse yeah you're right it does punish things so like if you get really really big differences between the expectation and your value when you square it it's going to look really really bad whereas if you get small numbers and you square it's not so bad so it does have that property whereas the absolute value one doesn't punish
1:07:35
being too far away as harshly so they do have this qualitative difference and that's a really nice maybe that's a good reason for choosing the L2 Norm anyways long story short people derive Define this thing this measure of spread that people call variance and they Define it to be over your whole distribution on average what is the distance of values to the expectation I'm going to use this shorthand form for the expectation of X mu because otherwise you have too many expectations of x's there
1:08:05
and just to give you an idea of what's going on here if you had a data set and you had to calculate this let's say x here is 25 x minus the expectation so 25 minus 60 will be a negative number why don't you square it it will be a large positive number and you could keep doing this for every single data point in your data set if either you Loop over your whole data set or you take every data point and weight it by its probability you would get the average of the distances of
1:08:34
every point in this distribution to its mean and you could calculate for this you could say it's 52 points squared that's the units what's a point squared I don't know that's just what the units are um and well it turns out a lot of times people are like I don't like these points squared so there's this other definition of spread standard deviation which is just the square root of the expectation is just taking this number Square rooting it that's the standard deviation um so
1:09:01
variance it's defined to be the expectation of the distance from points in your distribution with its mean of the distribution and distance is measured by taking the difference squared it's a formal definition of spread it has other names somebody asked earlier in class what if you have other moments well it turns out variance is the second moment expectations the First Central moment variance is the second Central moment you can imagine that there's moments Beyond this we're going to stop
1:09:28
at variance in cs109 okay um this is an idea normalized histograms are approximations are probably Mass function we'll talk about that more another day I did want to talk about how you compute this in the real world that is the definition we really cared about we really cared about having the average of the distance from the mean now earlier in class I told you there's this law of unconscious statistician and the law of unconscious statistician says if you apply any function to an expectation
1:10:03
like if you want to know the expectation of any function of your random variable
1:10:12
that this is equal to the sum of all the values just apply the function to each of those values and then multiply it by its probability that was the law of unconscious statistician and this is the very first time we're going to use it I want to come up with an easier way to calculate variance our function here is take X subtract off its mean and square it that's a function and so we can use the law of unconscious statistician and it's going to enroll expectation like this it's just going to
1:10:42
apply that exact same function to every single value so every single value you're going to subtract off the mean and square it and you're going to do that before you multiply that probably that x equals X if you started to apply this law of unconscious statistician you can take this expression and just do a little bit of algebra which I'll leave for you guys to do at home if you're curious but you can just expand this term kind of regroup it into different parts
1:11:07
take those groupings and simplify them again and you end up with well at this point you enter this expectation of x squared which we'll talk about in a second um leave that in there and if you just simplify this like this is the mean times expectation well expectation is what we defined the mean to be and this thing just simplifies till eventually you get that if you want to calculate variance you could calculate this equation that's semantically what we care about but we
1:11:38
can derive that there's this equivalent way of calculating variance and this is the one that's so much easier to do and this calculations looks crazy it looks like you have the same thing multiplied or subtracted from itself you have expectation of x squared and you're subtracting that from expectation wait for it x squared and that's the equivalent of that and that feels like it should be zero but it's not zero this is the expectation of x and this is the expectation of applying
1:12:12
the squared function to X if we use the law of unconscious statistician we can know what the expectation of x squared is the expectation of x squared well is going to be equal to using the law of unconscious statistician expectation of x squared is going to be the sum over all the values that X can take on Square that value before you multiply by the probably that x equals x that's what the expectation of x squared is it's different from taking expectation of X and then squaring it
1:12:55
here yeah yeah so I guess you're just wearing the efficient and the X in this case yeah here you're taking each of the values and squaring it when you're weighting it into the expect into the sum and in the other term you're calculating the normal expectation and then squaring it afterwards and it turns out it really does matter whether or not you square each value versus whether if you take the expectation squared and this thing is called the second moment it's just a
1:13:21
funny name for something that we use in our calculation of variance and it really comes down to this if somebody asks you to calculate variance you're going to use that equation okay we are at the end of today's day and you know the big takeaway is we've defined variance we now Define it to be this thing and you can now standard deviations defined to be the square root of variance we'll pick up this conversation on Wednesday thank you guys so much for such a fun Class come back on Wednesday
1:13:45
we'll continue this conversation of random variables you'll learn about the poisson one of my second favorite random variables of all time have a great day come back on Wednesday see you later cs109
Here to help you learn
Sign in to keep a private conversation with your course tutor.
Ask me anything about the material…
Sign in to chat