Skip to main content
Loading Explore guides…
Machine Learning vs Deep Learning · Acedra
Home
Explore
Computer Science
Machine Learning vs Deep Learning
Sign in to save
Chat
If YouTube asks you to sign in, open the video on YouTube.
Watch on YouTube
Reload player
Transcript
79 segments
Copy
Hide video
0:00
in this beginner tutorial we'll cover what machine learning and deep learning are how you can apply them and when you should use one technique over the other
0:15
to start with machine learning and deep learning are both forms of artificial intelligence and believe it or not AI arose in the 1950s and 60s in order to simulate human decisions and judgment here's a screenshot from a very early AI called Eliza and Eliza tried to simulate a therapist so it would have a chat with you where you could ask it questions and it could ask you questions
0:43
and AI is all about building programs that can make decisions like a human you'll hear a lot of hype about AI but it's really just code and data there's no magic involved and today I'll show you under the hood how some AI algorithms work so I wrote some code here that loads a pre-trained deep learning model by an organization called big science and then it asks the AI to explain what AI is and here's the output of this code it's an explanation of AI and the explanation is that AI
1:21
is just a computer program that processes data and makes decisions so let's learn a little bit more about how humans make decisions and about how we teach machines to do the same thing
1:36
let's say we wanted to predict tomorrow's temperature we might look outside and make a very simple rule we might decide that tomorrow's temperature is going to be the exact same as today's temperature it's not a very complicated rule but it actually works decently well and let's see how well it works we're going to write a little bit of code so we'll import a python Library called pandas that helps us work with data and then we'll load in a data set of
2:04
weather observations so these observations are from November 20th 2022 to November 26th so each row here is one day's worth of data the T-Max column is the maximum temperature that day and the prediction column is our prediction for tomorrow's temperature and as you can see it's the exact same as the T-Max column because we're just predicting that tomorrow's temperature will be the same as today's temperature all right once we make predictions we want to be able to see how good
2:39
they are and to do that we can calculate the error of our predictions and we're going to calculate error by just subtracting the actual value so tomorrow's actual temperature from our predictions
2:55
so what we'll do is I'll read in another set of data that has tomorrow's temperature so I didn't explain this earlier but I'm using the pandas read CSV function which reads in a CSV file which is basically just like a sheet in a spreadsheet and it lets us use that data in Python
3:16
then I'm going to calculate a column called error and this column we get by subtracting our predicted temperature from tomorrow's temperature so to tomorrow's temperature is in this column T Max tomorrow and we take the absolute value which just removes any negative signs so it gives us the difference but just the absolute difference
3:40
and then we can take a look at our observations data including this error column okay so we now have four columns the same TMax and prediction column we had before but we also have tomorrow's temperature in a column and here's the error between our predictions and tomorrow's temperature so in some cases the error is pretty small we're just off by one degree Fahrenheit but some of our predictions are pretty far off all right it's hard to just look at a bunch of different error
4:12
numbers like in the table we just saw and actually figure out how good the algorithm is so usually what we do is we turn that into a single error metric in this case we'll just take the average of all of the errors and that's called mean absolute error so we take the error column from our observations data and we just take the mean of that column which is just the average of that column which gives us 3.28 so that's our average error across all of our data
4:45
and it just makes it easy to evaluate how good or bad our technique for prediction was that's what this error metric is doing telling us how good or bad our prediction was so on average our prediction is 3.28 degrees away from the actual temperature and usually we want this error to be as low as possible right we want to make a really good prediction so we want to try to shrink this error and to do that we can use a technique
5:14
called expert systems and expert systems were very very popular in the 1980s they were part of a big AI wave then and in expert systems computers run human generated rules to make predictions so for example we might we might make two rules so we might say tomorrow's temperature is going to be the average of the temperature over the past five days and we might say if today is more than five
5:40
degrees warmer than yesterday add 2 to tomorrow's temperature so these two rules help us make a prediction and then we can tell a computer to actually evaluate these rules to make predictions
5:52
so I'll write a little bit of code that will do this so I'm going to read in into a pandas data frame this CSV file expert error.csv where I've already applied these rules and calculated the error and then just like before we're going to take the mean of the error column so first let's see what this data looks like take this off so we can see that we have same table as before with the same error column but some of the error values are now different
6:26
so let's add back in our code to calculate the error and we can see our error with this technique is 3.05 so on average we're within 3.05 degrees of the actual temperature
6:40
all right so that's expert systems now we can move on to actually thinking about machine learning so that's the next evolution of AI from expert systems to machine learning in the 1990s and 2000s
6:55
so the idea behind machine learning is that instead of making the rules ourselves instead of having a human make the rules an algorithm automatically generates the rules so this makes it a lot easier to use than expert systems right because in expert systems you may have to come up with hundreds or thousands of rules but machine learning automates all of that and this is called supervised machine learning and supervised machine learning is when you give the algorithm some data
7:26
and something to predict and it learns the rules for how to predict it there's also unsupervised machine learning which I'm not going to cover in this video but unsupervised machine learning is where you give the algorithm some data and it finds patterns in your data but we're going to focus on supervised machine learning and here's an example of a supervised machine learning algorithm
7:49
called a decision tree so let's say we work at a theater and we're deciding whether or not to let someone in to see an R-rated movie and an R-rated movie is a movie that you have to be 17 years old to see in the US you also need the money for a ticket so you can imagine a pretty straightforward decision tree here so we check if the person we want to let in is under 17 if they are we move to the left and we check if they're with an adult if they are with an adult we let them in if
8:22
they're not we don't let them watch the movie meanwhile if they're over 17 we then check if they have enough money to get into the movie the movie costs 20 to watch if they don't have twenty dollars we don't let them see the movie if they do have twenty dollars we let them see the movie
8:39
so a decision tree algorithm which is a form of a machine learning algorithm will automatically learn all of these rules and the whole tree which is very cool I'm going to focus on a different machine learning algorithm called linear regression and in linear regression we learn a linear relationship between predictors X and targets y usually one target but you can also have multiple targets so the equation for linear regression is y equals MX plus b so Y is what
9:11
you're trying to predict so tomorrow's temperature X is the data you're passing in so in this case today's temperature and M is what you multiply X by and then you add B so m is typically called a slope and B is typically called a y-intercept if you're familiar with linear regression the model will learn m and b automatically using an equation or gradient descent
9:37
so let's take a quick look at what it means for two variables to have a linear relationship so I'm going to load in a weather data set and I'll show you this data set this has data from 1970 all the way through to 2022. so this model is able to learn over a very large data set
9:57
and then what we can do is we can actually make a quick plot of this data you'll notice there's a few columns here so there's the T-Max which is the maximum temperature in a certain day there's team Min the minimum temperature in that day there's rain so the amount of rain that fell and then there's tomorrow's temperature so let's plot two of these columns against each other we're going to
10:20
make a scatter plot of the maximum temperature today against the maximum temperature tomorrow so let's go ahead and plot that I'll actually make this a little smaller so you can see it
10:33
and you can see here that when we make this scatter plot the shape of the data kind of looks like a line right you can imagine drawing a line through the center of that data that indicates a linear relationship between today's temperature and tomorrows so that relationship might be hey if you multiply today's temperature by 1.01 and then add some constant like two or three that gives you tomorrow's temperature
11:01
but the model will automatically learn what you multiply today's temperature by and what you add
11:09
all right so let's go see how that works so we're going to actually use a python Library called scikit-learn to train a linear regression model so we first import the linear regression class and then we create our linear regression instance of the class then what we're going to do is split up our weather data to get a training set so we're going to train our algorithm on any weather observations from before November 20th 2022. and then we're going to go ahead and fit the model
11:46
so we're basically saying use today's temperature the T-Max column to predict tomorrow's temperature T-Max tomorrow so we can run that and the model has actually been fit
12:00
so what we can do once the model has been fit is take a look at some of our parameters
12:06
so let's take a look here so what we can say is take our linear regression instance and say lr.of and this actually shows us the value of M so you'll remember from the previous slide y equals MX plus b so m is what we multiply today's temperature by to get tomorrow's temperature so we're actually multiplying by 0.81 and then we can check out The Intercept also and The Intercept is what we add to the multiplied temperature to get tomorrow's temperature so we're basically
12:42
saying take today's temperature so if today's temperature is 80 we multiply by the coefficient and then we add the intercept and this gives us tomorrow's temperature
12:54
or the predicted temperature from this model for tomorrow
12:58
all right so now that we have the coefficients we can actually evaluate how well this model works
13:05
so we really want to evaluate a model on different data from what we train it on this ensures that the model is building general rules that apply to all data so imagine that you're taking a test and it's an open book test and you haven't studied at all you don't know anything you still might pass the test just because it's open book you can read exactly the thing in your book that tells you the answer but if you had got a closed
13:32
book test then you wouldn't know the answer because you didn't actually learn anything so that's exactly why we evaluate the model on different data from what we train it on we don't want it to have learned all the answers just because it's already seen the data
13:47
all right so let's make some predictions and then from those predictions we can calculate our error so we first create a test set which is what we're going to make predictions on and use to evaluate our model and this is all of the data in our weather data set after November 20th 2022. then we can use the linear regression predict method and pass in our test temperatures to get predictions and let's take a look at these predictions
14:19
so these are predicted tomorrow temperatures for each day so November 20th 21st 22nd Etc and once we have these we can actually calculate our mean absolute error so we see we've reduced it to 2.92 from the expert model it was 3.05 so by automatically learning some rules we have improved our error all right now machine learning models can also use multiple predictors so for linear regression that equation looks like this we just have multiple M values and still a single B value
15:02
so let's see what that actually looks like so if you'll remember before we had that rain column
15:09
and we can use that to add in some information for our algorithm so we can fit our linear regression model not just on the maximum temperature but on the maximum temperature and the amount of rain that fell and we'll use both of those predictors to predict tomorrow's temperature
15:29
so let's go ahead and fit that and once we fit it we'll take a look at this coeff so you now see we have two coefficients so this is our coefficient for T Max so we multiply today's maximum temperature by 0.81 to get tomorrow's maximum temperature and you can see the coefficient for rain is negative 1.53 so that means basically what this means is if it rained today our model thinks that it's going to be cooler tomorrow
16:00
which makes sense right if it rains one day usually it's cooler the next day so that's what this negative coefficient means we're going to multiply the amount of rain that fell by this amount and add it into our equation and we can also take a look at our intercept which is B that's going to be 12. okay so we can use this linear regression model to create predictions so similarly to what we did before we'll call the predict method but this time
16:30
we'll pass in T Max and rain and then we can calculate our mean absolute error
16:38
all right 2.89 so let's go back to the previous screen 2.92 so we can see we did a little bit better here by adding in an extra predictor rain
16:51
all right so generally the best way to reduce error in a machine learning model is to either give the model more data so more rows of data more observations or give the model better features to make predictions with so in this case we can actually add some extra columns that tell the model more about how the data is organized and this is called feature engineering we're not we're not going out and Gathering more data but we're actually giving the algorithm a better view
17:20
into the data we have so for example we could add a column that shows the average temperature in the last seven days so not just today's temperature but the average over the past week we can also give the algorithm the ratio between today's temperature and the average temperature and the model can use these two columns to make better predictions now eventually all algorithms will hit a point where error can't be reduced anymore so you can't just keep feature engineering
17:50
and keep getting better and better performance there is a point where you have to either add more data switch to a better model or just accept that you can't actually model it any better
18:03
all right so let's see how this is going to look if we do a little bit of feature engineering so we'll create our average temperature column so this is going to be the maximum temperature the rolling average of the maximum temperature over the past seven days
18:21
then we'll add in a temp ratio column which is the ratio between today's temperature and the average temperature over the last seven days and we're going to remove any rows that are missing okay let's take a look at what this looks like
18:38
all right this table is a little hard to see down there but you can see that we added in these two columns so average temp is the average temperature over the last week and then temp ratio is the ratio between today's temperature and the average I did this drop n a just to remove some missing rows at the beginning if you're familiar with pandas you'll you'll know what that did okay now what we can do is we can take a look at our training data set again so our training
19:06
data set is all of our data up until November 20th 2022. and then we can go ahead and fit our model again but this time we're adding in these two extra columns these extra predictors okay so we'll go ahead and fit that okay and then let's continue once we fit the model we can again create our test set which is everything from November 20th 2022 onwards we can generate predictions similar to what we did before we're passing in these two extra columns
19:41
and we can again check out our mean absolute error and we can see it's down to 2.69 which is a pretty big improvement from our last Model so feature engineering has helped us actually really reduce our error all right now let's move on to deep learning so machine learning automatically makes rules for predictions using features so unlike an expert system machine learning automates creating the rules now if we want to reduce error with machine learning we usually
20:15
have to do feature engineering to add better and better features the cool thing about deep learning is it automatically makes the features and the rules so you don't need to do feature engineering because the Deep learning algorithm will automatically do the feature engineering for you
20:33
and how it works is you input some data and the neural network has hidden layers deep learning models are also called neural networks so those two are interchangeable terms the neural network has what are called hidden layers and your data flows through these hidden layers and in the hidden layers the model generates new features and these new features look a lot like the feature engineering we just did so they can be ratios between columns rolling averages all kinds of
21:04
all kinds of different things you can create out of these hidden layers okay so that is at a high level how a neural network works let's dive a little bit deeper and take a closer look
21:18
deep learning repeatedly applies linear and non-linear functions to generate a prediction
21:26
so here's an example of a network y equals r-e-l-u w1b plus w1x plus B1 so you might notice that this looks a lot like our earlier linear regression model and it does actually have a lot to do with linear regression this is basically applying a linear transformation to our X values our input data and it's adding again something similar to a y-intercept but this is called a bias in deep learning and this is called a weight or a set of weights
22:03
and re-l-u relu is a non-linear activation function that we apply on top of our linear transformation and neural networks are really just a series of non-linear Transformations and linear transformation so here's an example of a two hidden layer Network so this is hidden layer one and that's the inside and then outside we again do another linear transformation and another non-linear transformation so that's the two layer Network right there
22:38
so let's take a deeper look at how this works you can see at the very bottom here that's an example of a neuron in in deep learning and you can see that data flows into the neuron and the neuron applies an activation function and then spits out a value which if it happens at the end of the network is a prediction if it happens in the inside of the network is a value for the next hidden layer all right and we use an algorithm called gradient descent to automatically find
23:08
the correct values for all of the W's and B's all the weights and biases okay so let's actually do a little code example of a deep neural network so this this code I'm not going to get into a ton of depth but all this is doing is loading in the same training and test data we saw before and splitting it up so train X will be our predictors and train Y is what we're trying to predict
23:36
same with test test X and Test y so let's go ahead and run this and this is the initial configuration for our neural network and neural networks take a decent amount of configuration to actually get them to run properly they're a little bit finicky so where you have to configure a learning rate and a learning rate essentially defines how quickly the network gets the correct weights and biases
24:03
if you set it too high the network will change the weights and biases too fast and never actually get the correct ones because it actually changes so fast that it misses the correct values if you set it too low the network will move towards the correct values very very slowly and it'll just take forever to run so it does take some trial and error to set the correct learning rate
24:27
this layer configuration configures the layers of our neural network so our neural network as this input data which is just the number of columns in our train X data the next layer will actually generate 20 different features for each of our rows in our X data then the next layer will generate 10 features so it'll actually turn those 20 features into 10 by combining them and summarizing them and then it'll summarize those 10 features into one feature
24:59
which is our prediction so that's that's kind of the general configuration of our Network and then this line just sets up the layers correctly and we can actually take a look at our first layer of our Network here and we can see it's just a bunch of numbers so these correspond to our W and our B values our weights and our biases which is what the network is learning
25:25
okay now we can actually use this network to make predictions so in deep learning you usually want to feed the same piece of data into the network multiple times so it has multiple chances to actually learn from the data so what this outer loop does is it's just going to Loop five times so we're going to feed each piece of data into our Network five times this Loop Loops over every piece of our data and feeds it into the network
25:54
all right and then the call here this call is a little bit complicated
26:00
take that up what this does is it basically goes through each row in our data it sends that row of data into our neural network the neural network makes predictions then it figures out if those predictions were correct or incorrect so it makes predictions using this that's our train X then it figures out if they're correct or not using train y and if they are very incorrect then it goes back and it changes its weights a lot if they're a
26:33
little bit incorrect it goes back and it changes its weights a little and over time its weights and its biases will be tuned to actually make correct predictions so let's go ahead and run this code okay so our network has been trained and once we train it we actually want to make predictions so to make predictions our network will will basically follow these steps it'll take in our our data it'll multiply it by W1 so that first set of Weights it'll add the first
27:06
set of biases then it'll apply our non-linear activation function then it'll do the same thing with our second set of weights and biases then it'll it'll apply the non-linear function again then it'll multiply by our third set of weights and add in our third set of biases and it will not apply the non-linear activation because this last layer is what spits out our prediction so this describes this is layer one of the network layer 2 layer 3.
27:38
all right and let's see how good the predictions our Network makes are
27:43
so the first thing we'll do is we will run our net in forward mode which does exactly these things to generate predictions across our test set and then we will measure our error
27:59
so our error here is 2.88 it is about the same a little bit worse than our latest machine learning algorithm and I'll talk a little bit later about why that happens and when to use machine learning and when to use deep learning
28:17
okay so in summary machine learning and deep learning both automatically learn prediction rules machine learning requires a lot of feature engineering at times but deep learning automatically Engineers features that doesn't mean you don't need to clean your data make sure you have the right columns it just means you may not need to actually create new features just for the algorithm
28:42
machine learning is usually better for tabular data which is the type of data we just worked with so it's data that's like an Excel sheet like a table machine learning is usually better there because that data is already structured so the algorithm doesn't have to learn the structure of the data deep learning is really good for times when you have to learn the structure of the data like if the data is just free text or images like you've seen with algorithms like stable diffusion
29:10
or GPT deep learning can automatically learn the structure of that data and generate a good response in general deep learning needs a lot more data time and parameter optimization than machine learning it's a lot more finicky to get it to work but when it does work the results are incredible and it's usually a little bit harder to understand why a model is making a prediction with deep learning deep learning models are sometimes referred to as Black Box models
29:40
because data goes in it's not easy to tell what happened inside to cause a certain prediction and a prediction comes out whereas with something like linear regression we saw that we can easily look at the coefficients and understand what they mean so there this is an active area of research and there is a lot of work trying to make deep learning models more interpretable and explainable but it's not always easy to tell how or why a deep learning model made a certain decision
30:10
so at a high level this is the difference and when to use one or when to use the other there's a lot of special cases the field is moving fast things may change sometimes the best way to tell the right technique is honestly to try both and see what works given your error metric as long as you have a strong error metric and you're able to measure it effectively you can actually tell which one is better for your specific data all right well I hope you enjoyed this walkthrough
30:41
of deep learning and machine learning and if you want to learn more please check out our dataquest courses on both we just launched a lot of new machine learning and deep learning courses
Here to help you learn
Sign in to keep a private conversation with your course tutor.
Ask me anything about the material…
Sign in to chat