Skip to main content
Loading Explore guides…
ML · Acedra
Home
Explore
Computer Science
ML
Sign in to save
Chat
If YouTube asks you to sign in, open the video on YouTube.
Watch on YouTube
Reload player
Transcript
45 segments
Copy
Hide video
0:04
LABEEB KHAN: Hello, everyone. Thank you for attending the SAS Global Conference presentation. My name is Labeeb, and today, we'll be talking about an introduction and fundamental of machine learning. So we have 20 minutes, so we'll go ahead and get started. So what is the machine learning? Machine learning is a subset of artificial intelligence that focuses on allowing computer systems to perform tasks without actually having to explicitly program them.
0:30
Traditionally, you might expect that a task requires a hard set of coded rules, like If or Else conditions, to accomplish a very complex task. But the goal of machine learning is to avoid that and do it without the explicit instructions. So how is it done? Well, you'll take a set of training data, whether it be labeled or unlabeled, and you would train a machine-learning algorithm to predict the labels in the context if it's labeled. And hopefully, once the machine-learning algorithm
0:58
learns from the label training set-- when new data comes in to the space that it's never seen before, hopefully the algorithm can make a prediction on it. So a very simple example is determining if an email is spam and not spam. You can imagine this is an age-old problem, where probably for the last 40 to 50 years, even before the concept of machine-learning was popularized, email engines like Google or Yahoo Mail were trying to face this idea to figure out if an email is spam or not spam.
1:33
And originally, they probably used very hard-coded set of rules or NLP techniques to do that. But now, we can take it even a step further and use more modern, state of the art methods, like deep learning, to see if we can label it spam or not spam. So in the context of spam or not spam, you would be like a binary classifier, where there's two classes. And in this case, so we'd want to predict spam or not spam. And the inputs to the system would probably be the email message itself.
2:03
So the text, or maybe a tokenized form of the text going into the system. So what is our traditional approach? Well, traditionally, what we would do is we would first study the problem, and write some set of hard-coded rules, and see if our problem solution is good or bad. If it's bad, we can go back to the drawing board and revisit which areas of our model are performing poorly. And if it's good, we can deploy that. But that comes with a couple of challenges, right?
2:32
So the first challenge is-- you will have a system that has a long set of complex rules. Secondly, they might not be that flexible or dynamic, and become obsolete over time. You can imagine someone who's an advertiser starts noticing that he's getting less and less responses. So he starts modifying the messages spamming out to everybody, he or she is spamming out. So your spam detector now is becoming obsolete because the advertiser is changing some words up.
2:58
And they'll also be difficult and expensive to maintain. Because coming into a code base, if you see thousands and thousands of lines of code, that might be much harder to maintain. Especially if we're not familiarized with it. And also, the rules might not be adaptable to all instances, right? So as we step into the future, you might have emails that are very, very different. Some sending images, some sending different paragraphs. The hard-coded rules won't really
3:25
be able to capture all of those different cases. And it could also introduce many false positives. So you have an email that's classified as a spam one, when it actually wasn't a spam. So you probably want to introduce-- try and avoid that, actually. And certain problems outside of the world of language modeling-- certain problems are unsolvable, like speech recognition. Like it would be hard to write hard-coded rules to convert some of the frequency we hear from speech into a classifier.
3:57
So that's where machine learning comes in. We do a similar process. We will study the problem. In this case, we'll take a training set and train a machine-learning algorithm to do the predictions for us. And from there, we can check the accuracy of the machine-learning algorithm. If it's bad, then we can go and identify is it the data set's problem? Is it the algorithm that's an issue? And we can revisit or retrain the algorithm. And if it's good, we can provide it to launch.
4:24
So the benefits of the machine-learning approach-- so it's a lot bit shorter to write. Like if you think about TensorFlow or Keras, you can get a deep-learning architecture in six or seven lines of code. And if you're using scikitlearn, you could load the linear model or a decision tree in two, three lines of code. So it's pretty quick. Also easier to maintain. Because you can identify what's wrong with it and improve it over time. And it's seriously more accurate.
4:51
Like if you look at some of the cool accomplishments machine learning and deep learning have had over the last 10 years, it's been a really, really interesting to see how powerful they are. And also, it can learn to new instances. So it could also generalize a lot better, especially in the case of a language model, like spam filters. So let's talk about a concept, called online-- offline learning or batch learning. So this is probably what you're, for everyone
5:19
in this presentation, you're probably really familiar with this. Where you take a training set, and offline, you will train a machine-learning algorithm. Now, once you train that machine-learning algorithm and you check the accuracy, you can launch it. And let's say six months go by, and you're starting to realize, well, we're collecting a lot more data now. Let's promote retraining the algorithm. So you could actually take that same model you had and update the performance of the model
5:46
with a whole new set of data. So you could retrain all new data. This is called batch learning or offline learning, because you have a model that's sitting in production. And you're going to train a new model offline, and then eventually replace that model in production. And usually, it takes a longer time period, like six to eight months, or even a year to two years when you're collecting more data, right? Now, why would you need to retrain the model?
6:09
Well, you can have issues like concept drift or data drift, where if any distributions of your features change, then your model starts deteriorating. And that's called that's called data drift. So if any of your features start changing in their distribution, so you're starting to see something different in your data, then all of a sudden your model is not really trained to perform well in that instance. And then all of a sudden, your model's performance starts going down.
6:37
So you'll have to kind of take that new data set that you've been collecting over time, and see which areas your model's started introducing a gap in performance. And you want to close that. Concept drift is a bit different. Concept drift is when your relationship between your targets and features are no longer predictive anymore. So before, you had this assumption that your target could be predicted by your set of features. And all of a sudden, that might not be the case.
7:03
So for example, if we look at, let's say, at Aviva Insurance, what we do is we predict car accidents, right? And let's say driver age is a really good indicative of being in a car accident. And usually, it's because younger drivers are more riskier than the older drivers. So an example of concept drift would be like, let's say autonomous vehicles start going all over the road. All of a sudden, car accidents are independent of the driver.
7:30
So all of a sudden driver age is not a relevant feature anymore. So our relationship between our target and our features are different. And data drift would be an example where, let's say we start collecting a lot more older drivers, and our model just doesn't know how to perform on the older drivers. So that's not the case here, but that would be an example of a data drift if that was to occur. So a approach you're probably unfamiliar with, or you've come across this as well if you're
7:55
working at tech companies or doing a lot of big data engineering is the concept of online learning. So online learning is similar to offline learning. But your model that you train-- once it goes into production, you will be retraining it every night or as frequent as possible. And it's called online learning, because you're learning as new instances come along. And typically, that could also be make your model very vulnerable to degradation. Because what if the new data is just bad, right?
8:27
So you're starting to get worse and worse. So sometimes, people might introduce a fail switch to say, like, don't replace the model if the training session was doing worse on a hard-coded set in comparison to the current existing model in production. So you could have these threshold measures to not make your model worse. And if you ever see-- a funny examples of this is if you ever see a model that's built on Twitter data, you'll see there's a Twitter bot that tweets things, right?
9:00
And all of a sudden, people start seeing that, over time, the Twitter bot starts speaking nonsense, right? And it's probably because the data that it was being fed in was really bad. And so over time, the model just becomes worse. And then the Twitter bot seems like it's either being racist or crazy or something like that.
9:17
So these are examples of machine-learning systems. So if you have a labelled training set, that's called supervised learning. If you have a training set there's no labels, that's unsupervised learning. if you're training offline and in batches, that's called offline learning. And if you're training over time and it's happening on the fly, then that's called online learning. Another example is when you have a predictive model. So if your, for example, regression
9:44
will predict a numerical value, that's called predictive learning. And if you have an instance-based model, so you're classifying new data points into existing instances-- an example of KNN, where you have a specific set of regions and new data points come into one of those regions, that would be an instance-based model. Or a classifier would be an example of an instance-based model, where you're classifying new points into existing categories. So you can have a mixture of these.
10:11
You can have-- the spam filter can be an instance-based supervised learning and it's trained on the fly. That'd be a cool way to build a spam filter. But it's very problem-specific. So let's talk more about the stages of a machine-learning project. So if you're in the industry, or if you're going-- if you're planning on being new into the industry. Like I notice a lot of data scientists, they focus a lot on the modeling and the theory.
10:35
Which is great, but they miss an entire business aspect of it. Like if you're in banking, financial services, tech, you're hired to solve these use case business problems using machine learning and deep learning as a tool to help solve that. So a lot of times, they come in, and they're speaking RMSE, R-squared, all these technical concepts. But they're totally missing the fact that you have a business use case that you're trying to solve. So the first stage of your machine-learning project,
11:05
you want to really take the time to understand the business problem. So define the problem with your stakeholders. Understand as much as you can. Ask all of your questions to have a full scope of what you're doing. And create a plan of how you're going to solve that problem. You will get a lot of insight and guidance from your stakeholders. Because they've been in the problem a lot longer than you have, right? So for example, in our case, we build models and we work with actuaries, right?
11:29
And they are insurance experts. So they have been in the business for so long. And they understand our problems so well. So they can provide a lot of guidance and technical expertise in that area. The next step, once you really understand the business problem, is to do exploratory data analysis. So you want to-- and this is a huge step. You want to actually start collecting some of that data that hopefully, data does support the problem that you're trying to solve.
11:56
Or else they wouldn't be calling on a data scientist or a machine-learning engineer to do that. But you would start doing some initial exploratory data analysis. And sometimes, funny enough, the problem stops here. Because you build some nice dashboards. And there's a lot of information here to work off of. And the business can use that, right? So you don't always need to go and build the fanciest XGBoost model, you can just start delivering some good insights in the beginning.
12:21
And that might sometimes be enough. So touch base with those stakeholders once you have your initial insights. You know, it'll be a guidance, too. Are you on the right track? Is this making sense to them? Is this stuff that they're used to seeing, right? Like show them some of the plots that you found. And give them some of that tangible understanding of the same problem that you're solving. And start forming a hypothesis. So plot your target against your set of features.
12:44
And start saying, OK, this does look like it would be a predictive feature to put into a model. Obviously that's a one-way approach. But you could start forming some initial hypotheses. And also look for correlations as well. Correlation between the target, correlation between features. So that can help you really give you insights. And don't just start drawing features into a model. Like truly understand the features that you're looking at. And understand the feature you're working with.
13:08
And how does that impact the business, too? Also, define architecture for your ETL. ETL stands for Extract, Transform, and Load. So you want to take your raw data sources and clean them up, probably merge them. Get them ready for a nice modeling project. So you want to work with your data and figure out an architecture that can help funnel that process. Especially if, in the future, they're asking the request again, or you're retraining your model.
13:35
You don't want to do everything from scratch again. You want to run it through a pipeline. So prioritize your data understanding and data collection over building a really fancy model. Because sometimes, if you have the bad data, you're just going to have a bad model. And the power of good data always will outperform the power of a good model. Because if you look in this example, this is a plot of the accuracy based on the number of million words trained on an NLP model.
14:05
Obviously, it's good to explore the differences between the different models in terms of the application that you're solving. But if you just have more data, your problem's just going to be-- your model's just going to perform a lot better. Because then you can segment across different things really well in the model. You can learn across a lot of different abstractions. So after you've collected your data and you've done some EDA, you want to start doing the data prep and cleaning.
14:31
So look into any missing values. Hopefully your EDA's given you guidance on data that needs to be modified. So like imputations that need to be made, missing values that need to be adjusted for. Does the data makes sense? Like if you have an age variable, are you starting to see negative values? You might want to do some minimum or maximum capping and an imputation on missing data, right? And you can also do some initial feature engineering. So if you feel like age and gender are a good feature,
14:57
you can combine those two and see what they look like. So you can do some initial feature engineering as well. And sometimes, the feature engineering process can be building a model separately. And then finally, create a data pipeline. So whatever processes you're doing to transform your data, you want to pipeline them into a process. Because the next time data comes along, you're going to follow that same process. So you can funnel it down the pipeline. In the example I showed on the right,
15:22
this is a tool called Luigi. And Luigi is a really cool pipelining framework that can also do modeling for you-- sorry, it can support the modeling process as well because it works off of dependencies. And it's been used by a lot of different tech companies. I think Shopify or Spotify, I forget which one, they tend to use this one as well. I think they built it, actually. Cool. And finally you're at your modeling stage. So this is where you'll train your machine-learning model.
15:49
And you could do a number of different approaches, like split your data into training and testing. You can split it, do cross-validation on the modeling. And there's a whole wide range of modeling tools you can kind of look at, right? Classification, regression clustering. This does give you the freedom to try many different ones. So depending on how much time you have, definitely explore and try to find the best one. You probably should have an inkling beforehand of like, oh,
16:15
I think this model will probably perform better. Like if I'm doing like a model that assesses geographical problems, then I would see something, like KNN would do really better than, let's say, a linear model. Because KNN has a spatial smoothing and more geographical constraints on it, right? Just because it's more of like a region-based model. And if you're looking at a graph model, maybe like a decision tree or a neural network, because it's more inherently graphical type of structure
16:43
on it, right? Our time series model, great for time series data, obviously. Like on a recurring neural network or a moving average model. And finally, you can evaluate the success of your model. So do tell your stakeholders how you're building your model and the evaluation metric. Because then you'll have an understanding of what you're trying to optimize, right? So if that's like Gini index, for example, you're optimizing Gini. Let your stakeholders know what Gini means.
17:10
And then show them this is what you're optimizing. And there can be a lot of different metrics. I'll skip over this because of time. But essentially, for classification models, you have precision and recall. For, obviously, not classification models, you have different metrics like, RMSC, or Gini, and stuff like that. In terms of evaluation, you can assess your model if it's overfitting, underfitting, or if there's a good balance. Overfitting means when you're training accuracy is very high
17:37
and your testing accuracy is very low. It means that your model is started to learn some noise in the training set. And if the training sets differ from each other, that means you're very high variance, your model's very high variance. Because if it's overfit and the data sets are very different, then all of a sudden, another data set's going to totally throw away your model. Because the noise in the data that you've learned from is not going to be the same in the other data
18:02
sets that you end up testing on. So that's a high variance model. High bias is when you have a lot more opinion than trusting the data. So in the second example, you truly believe that your model is flat, even though there's clearly a signal of a curve there, right? So bias means how much of a personal opinion are you putting into your model. So too much is bad, because then you're totally avoiding the information from the data. But too little, like on the left case,
18:30
you're saying I have no information about how the relationship between my target and my feature is. So I'm just going to depend on my model completely. So that's also a really bad thing. Because you want to have a little bit of bias so you can give some preconceived opinions on your data if you have those. On your relationships, right? So want to have a balance between variance and bias. Like you don't want too high or too low on either of them. And they're always usually a trade-off.
19:00
And you'll notice that if you do a parameter search-- for more complex models, your XGBoost or neural networks, your training accuracy will start going up. But your testing accuracy will start going down if you make a model that's really complex. And then your error will start going up for your testing set, and your training error will start going down. So you're kind of heading towards the overfit state of a model. And if it's too simple, then you're
19:23
underfit, where you just have high training and testing accuracy. It means you haven't really thought about the relationships as much as you should have, right? Just for time, I'm going to kind of skip over some of this stuff. So finally, we end off at the deployment aspect of it, where this is the chance where you speak to your engineers to understand what the best approach to deploying your model is. Sometimes your model's one component of a very big process.
19:45
So you'll want to speak with those engineers. Don't try to do it yourself. You want to really understand what the best environment for your model to be in a production is. Whether you're putting it on an AWS EC2 instance, whether it's going to be something local that will be used, right? And then communicate your findings to your stakeholders. And kind of deliver that model. Thank you guys so much. So this is the stages of the machine-learning project. And I hope you guys have a great day.
Here to help you learn
Sign in to keep a private conversation with your course tutor.
Ask me anything about the material…
Sign in to chat