Skip to main content
Loading Explore guides…
Multi-Output Linear Prediction · Acedra
Home
Explore
Computer Science
Multi-Output Linear Prediction
Sign in to save
Chat
If YouTube asks you to sign in, open the video on YouTube.
Watch on YouTube
Reload player
Transcript
33 segments
Copy
Hide video
0:04
hi this is Ikhlaq Sidhu uh welcome this is the second part of the module on the topic of prediction within the data x course in this module we're going to extend the topic of linear prediction from the case of two variables which was basically one input variable and one output variable to multiple inputs and multiple outputs okay in the last section we talked about having one input and one output so you have basically a system or a formula inside it you put in an x and with a formula you
0:46
get a predicted value y so this time let's take a look at a couple different examples two cases one is suppose you have one input like x but instead of just one output we want to have multiple outputs like one like y1 y2 y3 so this case it's actually not a problem at all what you can do is you can have three different formulas inside this system so you put in x and you use say g1 to give you a y1 you use a second formula to give you y2 and you use a third formula to give you y3 so
1:26
there you can have multiple outputs that way the second case it's a little more complicated and this is what if you put in multiple inputs not just 1x but x1 x2 x3 all as inputs and now you want one or more outputs and what we're going to do is we're going to talk about what can you do you know what can be done to handle this case so to do this first of all what we're talking about is we're saying that we're going to have an x as an input we're going into a
2:03
predictor formula and we're going to have y the x is is the way to think about this is that the x this time is in a structure that's a table or more specifically in a matrix and what what we're looking at is that you've got a row of that table like say one three four six two that row is being provided as an input into this predictor formula and you're going to have one or more outputs and that will correspond to one row in a matrix which is y and
2:45
if you have one output then there's only one column in that matrix of y it's a vector if you have two outputs or let's say three outputs like in this picture then there will be three different columns and that row the corresponding row from the x will give you that corresponding output row here with three different values three columns in y now this is the basic machine learning framework that we're using in this course and that we want to explain
3:22
so we're going to start with the idea that there's a table of data and in this case every row is like a different person in your in your table or in your database and some of this data you also know something about what it is you want to predict so you have a column of outputs which are labeled or marked or already known and then you have other data for people where you don't actually know the answer and the basic ideas that you want to use
3:58
the top of this table in order to learn something so that you can make predictions on on rows that are below that is on the data that is out of sample data so what happens is that you take this data the input part and we're going to basically put that into the matrix of x so whatever was a text value for example like married or their job occupation either we're gonna convert that into numbers uh like categories or you know something like that
4:38
or we're gonna throw those columns out we're not going to use them and so that part becomes x again it's n rows deep and d is the number of columns the number of features or dimensions that there are and in terms of the output in this case we're talking about prediction so there's some predicted value that we'd like to have so it might be like a credit score in which case we would have a column of all the different credit scores that we already know and
5:14
the basic idea here is that x is chosen from some distribution we don't actually know this distribution but if we had this data that's in this form x and it was chosen from that distribution that there would be a function we don't know this function but there is a function that if you put x into that function you would get that value of y so now let me show you the same thing uh except we'll do it with us with a simple example and will
5:52
illustrate the idea of prediction or specifically in this case linear regression and what we have here is a table it has four rows in it for example the names of the each person is a different row and also on each row are three characteristics their age how many years they've worked for a certain employer and their current income and then there's also a column y here which is the credit score that they currently have and so what we're going to do is put
6:30
that into the format that we talked about before where x is the input and it has exactly these same numbers from the matrix in it there's a w which we don't know but we need to calculate and there's y which we already do know because it was in sample data and we do know what the credit scores for these particular people are and the basic idea is that if you if you know or if you can calculate what the best w is you know calculate w to give us the best predictor then we should be able to take
7:09
any future row in x multiply it by that w and get a predicted value for y the one other thing which is worth understanding on this example is that we've added an extra column of ones and this extra column is here it was not in the original data the reason that we did this is that we are adding something so that we can have the concept of bias or the concept of the y-intercept that we had before in our predictor so if you remember before the way it worked with two variables
7:53
was that you would take an x you would multiply it by a slope and add a constant value like a y-intercept and then you would actually get the predicted value in this case you're taking x as a matrix you're going to be multiplying it by w that's like the slope for the most part and you're going to be getting the predicted values of y but this column this set of ones always gets multiplied by this first element w0 for example 1 times w0 plus 25 times w1 plus 3
8:34
times w2 plus 50 times w3 that's how you would do a vector multiplication or even a matrix multiplication and so the 1 is always being multiplied by w 0 and what you'll see is that w 0 then turns out to be this idea of the y-intercept or the idea of the bias that the model has to start with for for creating the prediction okay so what we need to do now is basically be able to calculate the best possible w after that as i said we'll be able to
9:16
take any row multiply it by the same w and be able to get a predicted value of y so we'd like to just continue on with this example now that we've got x w and y in the right format x is a matrix w is another matrix and y is the output matrix our goal is to be able to to have a predictor or an estimate for y for new values of x that we've never seen in order to do this we are going to need to be able to calculate a w based on the values of x and values of y that we already have when we do this
9:54
when we calculate that w you could say that we are basically training or learning from the data that we already have in order to make this predictor now um just as an example so that it's clear we could take any row once we know w we could take any row for example the second one 1 23 2 60 we should be able to multiply it by w and so 1 times w0 plus 23 times w1 2 times w2 and 60 times w3 and when we add when we basically take that dot product
10:34
we should get a number that's close to 580. if we have a good predictor of w it would give us a value that's close to 580. so in that way w1 through w3 are like the coefficients that got multiplied by each of these feature variables that are in these columns that are in the future columns of x okay so now the question is only how is it that you calculate the best w and the place that we're going to start with this is by looking at the error that it would would get created
11:15
so you can think of it as well what would the error be if we took x and multiply it by w we have our predicted value of y and we can subtract away the actual value of y and try to see what is the expected difference now this is not as good because sometimes you have a positive value of y and sometimes you have a positive error and sometimes you have a negative error and they might cancel each other out so a better measure of error would be the square
11:46
to take the error and square it and then take the average so in this kind of summation version of the same formula but this time we're going to take w times x for the predicted value subtract away y the the actual value and take the square before we take the average and so we're basically looking at the mean square error as a measure of error now that we have the error and we can do this in matrix form which is what this top formula is what we're going to do now is
12:26
we're going to and this is important to understand what we're trying to do is find that w which would give us the minimum error and so this is really error as a function of w and we're trying to minimize it so if you were trying to find the minimum or the maximum of any other function something that you might consider doing is taking the derivative in the matrix form we're basically going to do the same thing of taking the derivative and setting it
13:03
equal to zero so we're going to take the gradient with respect to w and get something that looks like this as the gradient and set it equal to 0. by setting it equal to 0 we can basically solve for the value w and when you solve for the value of w so you'll see when you do this it looks a little bit messy but it's really not and you just do a little bit of matrix multiplication and you can get this w by itself move the x transport rows and x times x to the other side and so here you have
13:42
a result for calculating you know effectively calculating the optimal value of w now it's possible that when you look at this it might look messy but it's actually an amazing result and it's a very tractable and one thing i'd like you to do is consider the dimensions of this because it's not as messy as it looks so if you look at x transpose times x uh let's look at these one at a time x transpose has this shape which is um it's very wide you know n is a large number and d
14:24
the number of columns that's a small number so that has a shape that looks like that and x has a shape as you know it's a long it's a deep table and is very long and the dimensions is relatively narrow when you multiply those two together what you actually get even though this is a very large matrix with another very large matrix you get something that's pretty compact it's actually just d by d meaning if you had five features in the x matrix then this would be a matrix it's only
15:00
five by five and then when you take the inverse of it which you need to do in the original formula then you would see that's also the same thing d by d or five by five in the example if you then take that and multiply it by x transpose and y what you'll see is that you're multiplying it by a d by n matrix and an n by m matrix and these also basically collapse the whole size down to only something that's d by m d is remember the number of features in the x matrix
15:37
m is the number of outputs that you are expecting to have so again if you had five features in your x matrix like age and height and income level and you have three outputs like credit score and um you know frequency of visits to the to the business and you know one more then five by three would you know then your your matrix would literally be only five rows deep and three columns wide and so once you have that d by n matrix that's what w actually is and then you can take
16:26
any set of data that you've never seen before even in matrix format multiply it by that w and you'll get another matrix of y so let me show you the same thing now in code format this is an example in numpy uh note that we took the exact data of the Xs 25 3 50 those are the same data that was in the table we also still kept the ones that were the first column as in the example and we have y a vector with the different credit scores and calculating w
17:07
is simply just executing the formula so the formula we just showed you in terms of matrix multiplications and what you get is w with three coefficients and one w0 which is like the bias term so think of it this way which is that everyone's credit score turns out to have a bias or a base of 426 points then for every year you are older then you will lose negative 60 then you'll lose 16 points on your credit score and for every year that you've worked with the same employer
17:54
with the current employer you gain 149 credit points and for uh every extra thousand dollars that you make you gain 3.7 extra points on the credit score now is this a this is not a great formula for a credit score it's just what comes out of this particular value of x and y but it does illustrate how w is calculated and you know from a code point of view how it can be done um with with actually very little code in in numpy now i want to bring you back to this slide which we looked at in part one
18:42
so we've seen this example before what it shows is that we can use the same library function with x x can be you know one variable or x can be a vector like we did before with one input and one alpha but x can also be a matrix and it can have multiple columns of different features y can be a matrix that can have multiple columns of different outputs and the code to use this library with multiple inputs and multiple outputs is otherwise the same it's identical
19:24
and so it's a good example that all of the details they've been done within the library function uh two three lines of code all of that has been taken care of for you but in addition what this module explains is what is underneath this code when those predictions are are being done for you so um uh this concludes the linear regression module it also gives you an example of the machine learning framework that we're learning and that we're using so in this framework
20:06
x is our input it can be a single variable it can be a vector it can be a matrix we generally put it into a function we'll just call this function g of x and we will get an output Y that would be our predicted value we usually use some data to train so that we can learn g(x) and and then later we can put in values of x that we haven't seen to get new predicted values there are cases where we can just calculate the best of parameters for g(x) like we did with linear regression and
20:48
there's also cases where we can't calculate it but we can take a good guess and then we can see how close we are to a good predicted value of y and then we can through iterations try other versions of g kind of over and over after we look at the error and then try another version of g and then look at the error until g is getting closer and closer and basically becomes good enough to be a good classifier we would say that g of x has become close enough to that ideal or perfect
21:30
function f x which would actually be the perfect predictor um and then once we have a version of g of x that is good enough that has a small enough error when you use it then of course we can use that in predicting any x that we haven't seen up till now so we're going to see more of these types of predictors and classifiers in future sections of the course
Here to help you learn
Sign in to keep a private conversation with your course tutor.
Ask me anything about the material…
Sign in to chat