Skip to main content
Loading Explore guides…
Regression Models & Hypothesis Testing · Acedra
Home
Explore
Engineering
Regression Models & Hypothesis Testing
Sign in to save
Chat
If YouTube asks you to sign in, open the video on YouTube.
Watch on YouTube
Reload player
Transcript
428 segments
Copy
Hide video
0:01
- Hi, welcome back to the next steps of your journey as a data professional. I'm so excited to join you. We'll be discussing regression models and more hypothesis testing so that you can explore the relationships in your data. The tools we learn about together will allow you to uncover powerful insights about your data, tell compelling stories, and influence decisions and strategy-making. The modeling fundamentals we explore in this course will give you an even stronger foundation
0:30
to pursue entry-level jobs in the data career space and to tackle more advanced topics in the future, like machine learning. I'm eager for us to connect all the pieces you've learned so far. You've come a long way since the start of your data journey, from the foundations of data science and Python fundamentals to exploratory data analysis and statistics, you've learned so much about the data landscape. Together we'll be applying your current skills to your first model, regressions.
0:59
My name is Tiffany. I'm a marketing science lead, and I work at the intersection of data science and marketing here at Google. As a child, I gravitated towards math, probably because it was my favorite subject. When I got to college, I was not exactly sure what I wanted to major in, but I was drawn towards quantitative fields. As many of you may have also experienced, I didn't love my first decision and had the opportunity to try out a few majors, exposing me to the variety
1:26
of analytical career opportunities. Fast forward to my career, I've held positions in finance, statistical research, and data science consulting before joining Google. I've worked with a variety of data including patent, fraud, hardware sales and now marketing data. The great thing about being a data professional is that it's in high demand in most industries, which means that you can find interesting problems to solve across a huge variety of companies, products and countries.
1:56
I love the flexibility to find the right fit for me while also knowing that I will be able to deliver value to whatever team I'm on. I've learned so much in my career, and I'm excited to share my experience with you. In this course, we will talk about modeling relationships between variables. To do this, we will focus on regression analysis. Regression analysis or regression models are a group of statistical techniques that use existing data to estimate the relationship
2:24
between a single dependent variable and one or more independent variables. We will unpack this definition together later in this program. The techniques we learn in this course will help you answer any number of questions with actionable steps to achieve your organization's goals. For example, regression models can help you understand what variables impact sales. They can also help you understand the factors that lead a customer to subscribe to a newsletter.
2:52
Regression models can even help you understand why a user keeps scrolling on a company's website. As we work through the course together, we'll continue discussing the role data professionals play in making responsible decisions, from data handling to modeling. No matter how good a potential data story is, how we arrive at our story is the priority in being honest and effective storytellers. Python programming will be critical to running and testing complex models,
3:21
visualizing data and communicating results. Rigorous exploratory data analysis, or EDA, will inform which models we choose and how we approach the modeling process. Statistics will play a major role in helping us understand how our models work and will allow us to present actionable results to stakeholders. Our statistics toolkit will allow us to build the models we talk about. Regression analysis is a relevant and marketable skill. Regression models are flexible,
3:53
so you can design them based on the data that you have. Additionally, regression model results offer opportunities for interpretation and communication. Data professionals can provide insights from these models that align clearly with actionable steps. For example, we're able to build regression models to identify what actions on a website are indicative of high value customers. For instance, there is a high likelihood that customers who visit the sale page, watch a video or sign up for emails
4:24
are customers who will make multiple purchases in a year. Some of these indicators may seem obvious, but having a regression analysis that identifies and quantifies the relationship is powerful. We're able to use the information in marketing campaigns to attract new customers. Together, we'll be exploring regression analysis as the basis of a solid foundation for building more complex machine learning models. I'm so excited to show you each of the tools in your regression toolkit.
4:53
You'll get a lot of practice in Python. We'll explain how to determine if a model is appropriate for the data, how to run the model, and how to understand the computer's results. We're also going to discuss some math and review the concept step-by-step the whole time. You can also review course resources whenever you want. I'm so excited to get started on regressions. So let's begin.
5:18
I can't wait to explore regression analysis with you. Together, we'll learn how data professionals go from a problem to actionable insights using regression analysis techniques. We'll begin by introducing regression analysis. The focus for regression analysis is about understanding relationships between variables. We'll use the PACE framework, plan, analyze, construct and execute, to guide us through the rest of the course. Next, you'll have the chance to apply PACE to simple linear regression,
5:50
which will be our first model we explore comprehensively. We'll go over the entire process from beginning to end using different scenarios and data. Then we'll examine multiple linear regression closely. Multiple linear regression expands upon a lot of the concepts from simple linear regression but will allow us to problem solve a larger variety of questions. We will focus on some more nuanced topics, like variable selection and model interpretation. We'll also consider a few hypothesis tests,
6:22
such as chi-squared test and ANOVA. These tests will help us explore different groups in the data, allowing us to glean interesting insights. Finally, we will review the fundamentals of logistic regression. This is the last and most complex model, which will set you up well to approach the rest of the program on machine learning. Whenever you're ready, I'm excited to get started on the next video.
6:49
You may be familiar with terms like machine learning models or regression analysis, and if not, you're in the right program. Next up, we'll get started on our learning journey. To begin, we're going to discuss a few key terms and definitions. Regression models are based on a statistical foundation. Models used in the data career space are a family of techniques that rely on existing information or data points to inform what we might think other data points will look like.
7:19
The goal is always to tell a story about the relationships between the variables and the data. The story will help stakeholders adjust their business strategy and decisions. Modeling follows an iterative process. You may be familiar with other frameworks such as the data lifecycle or the six steps of exploratory data analysis. In this course, we will cover each step of the modeling process using PACE. Outlined earlier in this program, PACE, plan, analyze, construct and execute,
7:50
provides us with the foundation for conducting regression analysis. Let's take some time to preview how PACE works in the context of regression analysis. In regression modeling, the plan stage is about understanding your data in the problem context. Knowledge you bring, whether from industry or others, can be instrumental in the plan stage. By considering what data you have access to, how the data was collected and what the business needs are, you'll be able to strategically analyze,
8:20
construct and execute the rest of your work. The plan stage will guide the other three stages of PACE. After you plan, you have to analyze. In this stage, you examine your data more closely so you can choose a model or a couple of models you think might be appropriate. When working with regression analysis, this is where you use Python to perform EDA and check the model assumptions as needed. Model assumptions are statements about the data that must be true
8:49
to justify the use of particular data techniques. As a data professional, you'll use statistics to check whether model assumptions are met. A good understanding of statistics gives data professionals the power to construct meaningful models. After you analyze, you must construct. For regression analysis, this is where you actually build the model in Python or your coding language of choice. This step involves selecting variables, transforming the data as needed and writing code.
9:18
Even though you checked model assumptions before you built the model, many model assumptions need to be rechecked after the model is built. So you'll do that in the construct phase as needed. The last part of construct phase is evaluating the model results. At this point, you are answering the questions, how good is my model? You'll choose metrics, compare models, and get preliminary results. Then based on your evaluation, you can use EDA to refine your model accordingly.
9:48
Of course, as a data professional, you must first and foremost be an honest storyteller. Studying the results produced by the regression will uncover relationships within your data and help you discover insights to tell the full story. This leads to the last part of PACE, execute. You'll interpret everything you learned from analyzing and constructing to share the story, you'll prepare formal results and visualizations and share them with stakeholders. To do this, you'll convert model statistics
10:20
into statements describing the relationship between the variables and the data. These descriptions must consider the context and initial questions from the plan phase. At the center of everything is data, and the PACE framework helps data professionals stay organized. The insides data professionals produce must be data-driven and accurate, and they have to make sense given the business or community context. We'll go over these steps using examples later in the course, but PACE is iterative.
10:50
As you grow as a data professional, your experience will help you decide when to pivot between stages of PACE. You might switch the order or repeat steps depending on the situation. Now that we've laid out the pieces of the modeling puzzle, we can talk about how correlation and regression are related. Then we will explore two foundational regression models, linear and logistic regression. These models will be covered in more depth in later videos. This overview will provide you
11:20
with a solid basis of understanding. Time to start putting together our puzzle. Don't forget your statistical grammar tools. You'll need them.
11:30
Previously, we learned that regression analysis is about estimating relationships between a single dependent variable and one or more independent variables. In this video, we will discuss our first modeling technique, linear regression. Many patterns that you've observed in daily life can be expressed using linear regression models, which is pretty cool. For example, as a version of computer software gets older, online searches for that software version may decrease.
12:00
As a social media personality gains followers, their book sales increase. These relationships can be modeled using a linear regression. The linear in linear regression indicates the kind of relationship we can visualize on a graph, a line. A line is a collection of an infinite number of points extending in two opposite directions. In a graph, the individual points show up as a line, and we only see a portion of the line. Linear regression is a technique that estimates the linear relationship
12:33
between a continuous dependent variable Y and one or more independent variables X. For example, we could model the relationship between the prices of a product and the number of sales. Our Y variable would be the number of sales, and our X variable would be the prices. In an earlier course, you learned the difference between continuous and categorical variables. As a reminder, continuous variables are variables that can take on any real value between its minimum and maximum value.
13:05
For example, product sales, vehicle speed and time spent on a webpage are all continuous variables. Whereas types of products and educational level are not. These are categorical. Categorical variables have a finite number of possible values. While linear regression allows us as data analytics professionals to estimate continuous dependent variables, there are other regression models that let us estimate categorical variables. We'll learn more about those in a later course.
13:39
Throughout this course, we'll talk about dependent and independent variables. The dependent variable is the variable a given model estimates. Sometimes the dependent variable is also called a response or outcome variable and is commonly represented with the letter Y. We assume that the dependent variable tends to vary based on the values of independent variables, typically represented by an X. Independent variables are also referred to as explanatory variables or predictor variables.
14:12
For example, let's say you're working at a cake shop, and you're trying to understand the factors that contribute to cake sales. The dependent or Y variable would be the number of cake slices sold on any given day. An independent or X variable could be how many cups of coffee are sold that day. Maybe as more coffee is purchased, more cake is also purchased. In linear regression, you might encounter two more terms, slope and intercept. The slope refers to the amount we expect Y,
14:47
the dependent variable, to increase or decrease per one unit increase of X, the independent variable. The intercept is the value of Y, the dependent variable, when X, the independent variable, equals zero. Going back to cake and coffee, the slope would be how many slices of cake are purchased per cup of coffee purchase. The intercept would be the number of cake slices that are sold when zero cups of coffee are sold. When two variables X and Y are related in a linear way,
15:24
we say they're correlated. Using statistics, we can actually calculate how strong the linear relationship between X and Y is. Pretty cool. There are two kinds of correlation, positive and negative. Positive correlation is a relationship between two variables that tend to increase or decrease together. For example, as more cups of coffee are purchased, more cake slices are purchased. Negative correlation, on the other hand, is an inverse relationship between two variables.
15:59
When one variable increases, the other variable tends to decrease, and the reverse is true too. For example, let's say you're still working at the cake shop and you're estimating how often to refill the iced coffee dispenser. You can model the relationship between iced coffee and hot coffee sales. As hot coffee sales increase, you might notice iced coffee sales tend to decrease. Or perhaps you're working at a media company, and you're analyzing readership.
16:30
As the length of new articles increases, the number of people that finish reading the article might decrease. This is also an example of negative correlation. Identifying these kinds of relationships can be incredibly useful in the workplace and in everyday life. Determining linear relationships helps us answer questions such as which factors are associated with an increase or decrease in product sales? Which factors make a social service providers increase resources in a given region?
17:05
Which factors lead to more or less demand for public transportation? In the cases mentioned, how big the slope of the regression is tells us how much sales resource allocation and public transportation increase or decrease. Using linear regression, you can help answer similar questions in any industry. However, it is important to note that correlation is not causation. For example, in your cake shop, people buying coffee does not cause cake sales to increase.
17:38
When modeling variable relationships, a data scientist must be mindful of the extent of their claims. Causation describes a cause-and-effect relationship where one variable directly causes the other to change in a particular way. Proving causation statistically requires much more rigorous methods and data collection than correlation. For a data professional, the distinction between correlation and causation is especially important when presenting results.
18:11
For example, although we can say that as someone gets older, the number of places they have visited tends to rise. We cannot necessarily say that someone's age causes the number of places they have visited to increase. There could be other factors causing you to travel more that coincide with your aging, such as visiting family or traveling more for work. Any of these factors could be correlated to aging, but it's hard to say whether age or other factors are causing the traveling.
18:42
Articulating that correlation is not causation is part of a data professional's best practices and ethical toolbox. Both correlational and causal relationships provide useful insights. Regression analysis helps data analytics professionals tell nuanced stories without needing to prove causation. That concludes a high-level overview of your first modeling technique. To recap, linear regression is a way to model linear relationships. Dependent variables vary according to independent variables.
19:18
The slope identifies how much the dependent variable changes per one unit change in the independent variable. Positive and negative correlation describe linear relationships between variables. Always be mindful when interpreting regression results. Correlation is not causation. There are lots of linear relationships in the world and in industry. In this video, we've provided just a few examples, but there are plenty more.
19:52
That's all for now. Next, we'll address the math behind linear regression. Your ever-growing statistical foundations will help you explain regression results in the clearest way possible.
20:04
In an ideal world, you would want every data point relevant to the question you are trying to answer. In a previous course, you learned that data professionals describe this as population. Let's imagine you're working for a publishing company. You want to understand the relationship between an author's social media following and their book sales. You would need data on every book author's social media follower account and every book sale of all time. This is an impossible task, but luckily,
20:34
you don't actually need an entire population to run meaningful regression analysis. You can get a reasonable estimate with a representative sample. A sample is a part of the population, which is just the statistical way of saying a sample is some of the data you could possibly have. If you have a set of sample data, each data point can be represented with its own set of attributes or X and Y values. In that case, the sample does not contain all possible values from the population of data.
21:09
The observed values or actual values are the existing sample of data. Each data point in this sample is represented by an observed value of the dependent variable and an observed value of the independent variable. In the publishing company example, the dependent variable is book sales, and the independent variable is how many followers the author has on social media. An observed value you might have in your dataset would be X or number of followers equals 10,000,
21:41
and Y or number of book sales equals 500. The goal of the regression analysis is to define a relationship mathematically between the sample Ys and Xs to understand how the two variables interact. You can imagine that at every X value, there are many possible values that Y could take on. To simplify this understanding, linear regression analysis focuses on the mean of Y given a particular value of X. This mean of Y is the value on the line in a linear regression and is denoted with a Greek letter
22:18
that looks like a lowercase M. You can remember M is for mean. The Greek symbol is mu.
22:27
As described previously, in order to define a linear relationship, we need a slope and an intercept. In statistics, we write the intercept as beta-zero, which we sometimes call beta-naught, and the slope is written as beta-one. Mu of Y and the betas are sometimes called parameters. Parameters are properties of populations and not samples. So we can never know their true value since we can't observe the whole population, but we can calculate estimates of the parameters using our sample data.
23:05
To differentiate between the population parameters and the estimates of the parameters, we denote the estimates with a hat. The calculation is beta-zero-hat, beta-one-hat and mu-hat are all parameter estimates. Although it's valuable to recognize the mu notation, for the remainder of this course, we'll use a simplified notation. Y equals beta-zero plus beta-one times X. For example, let's say that we estimate beta-zero as negative one and beta-one as five. Now let's input some values for X.
23:44
If X equals zero, we get negative one, which is the Y-intercept or beta-zero. If X equals one, we get Y equals four. If X equals two, we get Y equals nine. If X equals three, we get Y equals 14. From just four data points, a pattern emerges. For every one unit increase in X, we get a five unit increase in Y. Remembering our equation, five is our slope or beta-one. The slope tells us how much Y increases for every one unit increase in X. These estimated betas
24:23
are also called regression coefficients. So now whenever you see the hat symbol, you'll know that you are estimating betas, also known as regression coefficients. In the prior formula, the regression coefficients where the slope and intercept. They described the linear relationship found in the sample data. In order to enter values of X and get estimated values for Y, we assumed that beta-zero was negative one and that beta-one was five, but how did we arrive at those regression coefficients?
24:58
One of the most common ways to calculate linear regression coefficients is ordinary least squares estimation or OLS for short. We will go over the math behind OLS later in the course. For now though, let's discuss an overview of how OLS works. In linear regression analysis, we as data professionals are trying to minimize something called a loss function. The loss function is a function that measures the distance between the observed values and the model's estimated values.
25:30
Theoretically, we could draw an infinite number of lines that model the data we have, but we don't want to find just any line. We wanna find the best-fit line, so we want to minimize the loss function. With this foundational understanding of how linear regression works, we will be able to talk about PACE in linear regression analysis, which readdressed earlier. Later in the course, we'll build on the concepts we covered in the video to learn more about how regression works,
26:00
when to use linear regression, its variance and OLS and Python.
26:07
I still remember being on a sports team needing to determine which team first played offense or defense using a coin toss. What's the chance the coin will come up heads? What's the chance the coin will come up tails? The probability of either event occurring is 0.5 or 50%. This is a classic probability problem, but in the field of data analytics, we get to use a technique called logistic regression to model much more complex probability problems.
26:39
For example, what factors lead to someone subscribing or not subscribing to a newsletter? Under what circumstances does someone comment on an online video or social media post? Given certain factors, how likely is it that someone renews their membership to an organization? All of these questions are tackling discreet events or categorical data. There are a specific set of possible outcomes, subscribe or don't subscribe, comment or don't comment, renew or don't renew.
27:15
To answer these kinds of questions, data professionals use a model called logistic regression. Logistic regression is a technique that models a categorical variable based on one or more independent variables. The dependent variable can have two or more possible discreet values. Let's say that your company has a newsletter and is interested in increasing readership. On the company website, users have the opportunity to subscribe to the newsletter. One factor related to the newsletter subscription
27:50
could be how many minutes the user spends on the webpage before leaving. Our dependent variable Y has two possibilities. Users don't subscribe, which we'll represent with a zero, or users do subscribe, which we'll represent with a one. Our independent variable X is continuous and measures how many minutes the users spend on the webpage before leaving. Let's graph the hypothetical data on a scatter plot like we would when doing EDA or exploratory data analysis. We can observe that the data points
28:29
are roughly in two horizontal lines. The higher one indicates the user subscribed. This is when Y equals one. The lower one indicates that the user did not subscribe. This is when Y equals zero. The X-axis indicates how long the user was on the site in minutes. Since the relationship between X and Y is not just a straight line, we need a new mathematical way of expressing the relationship between X and Y. Logistic progression will allow us to model
29:02
the probability a user will subscribe to the newsletter. The key concept is that the mean of Y given X is equal to the probability of Y equals one given X. Let's explore this idea. At this point, our observed Y values are just a bunch of zeros and ones. To find the mean, we would sum all of our observations and then divide the total by the number of observations. Because some of the observed data are zeros, when summing the observations, the zeros all add up to zero.
29:39
Therefore, the sum of all the observations is equal to the total number of ones. Then we divide by the total number of observations. This is equal to the probability of Y equaling one or someone becoming a subscriber. In this case, we know that the mean of Y given X is equal to the probability of Y equaling a certain outcome given X. Sometimes the probability of Y given X is written as P to reinforce the idea of probability. To help you remember, you can think P is for probability.
30:18
Now we want to understand what variables help explain those probabilities. Mathematically, we need a way to relate the X variables to the probability that Y equals one. Imagine we only have one independent variable. So then we need to relate the probability of Y equals one to beta-zero plus beta-one times X. In logistic regression, we use a link function to express the relationship between the Xs and the probability that Y equals some outcome. A link function is a non-linear function
30:57
that connects or links the dependent variable to the independent variables, mathematically. We will discuss how logistic regression and link functions work in more detail later on, but for now, just know that this is an example of the differences between linear and logistic regression models. Let's review a few more similarities and differences between modeling approaches. First, linear regression involves a Y that is continuous like book sales.
31:29
While logistic regression involves a Y that is categorical like newsletter subscription. Choosing one model over the other is about the kind of data you have. You can answer similar questions about what factors impact an outcome of interest. Secondly, since linear regression models a continuous variable, we're estimating the mean of Y, but logistic regression models a categorical variable. So we're modeling the probability of an outcome. For example, if Y equals one or if Y equals zero.
32:07
Lastly, for linear regression, we can express Y directly as a function of X, but for logistic regression, we need a link function to connect the probability of Y with X. We'll continue focusing on logistic regression with just two categories like the subscription problem, but there are more complex version of logistic regression that can model multiple outcomes or categories, such as types of skincare products people buy or types of services that people receive.
32:43
In this video, we covered the basics of logistic regression. We also compared and contrasted linear and logistic regression models. Later in this course, we'll cover an estimation technique for logistic regression called maximum likelihood estimation, and we'll uncover a bit more of the math. For now, know that computers are incredibly powerful and enable us as data practitioners to focus less on the math and more on the storytelling. The rest of the course will provide you
33:15
with a solid understanding so you know what's going on in that powerful machine of yours.
33:22
Together, we've covered a lot of high-level regression concepts already. You should be proud of yourself for learning all of these new concepts. It's been incredible starting you off on your regression journey. In prior courses, you learned about the data career space, the importance of communication in data-driven work, considerations for how data is cared for, the value of Python programming as a data tool, EDA and statistics. In this course, we've started connecting these concepts together
33:55
to prepare you to build your first models. So far we've talked about PACE in regression analysis. Planning allows you to consider how the data was collected and what the business needs are in a particular instance. In the analyze phase, you perform EDA, which helps you determine whether one model is more suitable than another. When constructing your model, you'll be amazed by the power of your computer and the Python programming language. Through model construction, you'll apply your creativity
34:27
as you visualize your data and regression models as well. You'll then rely again on statistics and math to evaluate any model you encounter. Finally, in the execute phase, you have to focus on communication as you interpret the results of your model. By combining the four steps of PACE, you'll start creating data-driven stories soon.
34:50
We covered two models briefly, linear regression and logistic regression. These models are able to estimate common relationships between variables that we observe in our personal and professional lives. Regression models help us answer questions about what factors are associated with a variable of interest and by how much. The data always leads us to the questions we can ask and answer. Linear regression needs a continuous dependent variable
35:20
so it can help model anything measurable and quantifiable. Logistic regression needs a categorical dependent variable so it can determine the probability of something occurring. Linear regression focuses on positive correlation, variables that increase or decrease together, or negative correlation. One variable goes up, and the other goes down. Logistic regression models use a link function to relate the Xs and the categorical Y. Later we will learn how to use estimation techniques
35:54
in Python with real data, and we'll go through more use-cases of regression. Although you'll encounter many formulas along the way, keep in mind that the end goal is to figure out the story the data is trying to tell. You have many resources available to you in this course. Good luck with the rest of this section and be patient with yourself. I'm thrilled to join you again next time.
36:22
Welcome back to Google's course on regression models. It's great to be with you again. Previously, we provided a high level overview of regression analysis and two core foundational regression models, linear and logistic regression. In this section of the course, we'll go over how to set up, build, evaluate and interpret our first regression model. We'll also review model assumptions, construction, evaluation and interpretation as we've previously defined. There are a lot of new concepts.
36:53
So if you ever need to get reorganized, remember the PACE framework, plan, analyze, construct and execute. Each part of PACE aligns with a part of the process of regression analysis. Sometimes we have to repeat some steps, but these are the phases to keep in mind, and we'll go through the process concretely together using examples to help guide you. To recap, linear regression is a technique that estimates the linear relationship between a continuous dependent variable
37:24
and one or more independent variables. Recall that an independent variable is the variable whose trends are associated with the dependent variable. Independent variables are commonly represented by the letter X. Meanwhile, the pining variable is the variable that a given model estimates, also known as the outcome variable. The dependent variable is commonly represented by the letter Y. We'll learn more about simple linear regression. Simple linear regression is a technique
37:54
that estimates the linear relationship between one independent variable X and one continuous dependent variable Y. The model assumptions, code, evaluation metrics and interpretation skills we review here will extend directly into more complex models like multiple linear regression. By solidifying your foundation through simple linear regression, you'll be prepared to tackle advanced models that can answer more complex questions in different industries and business contexts.
38:28
As we go through simple linear regression together, you'll combine many of your prior skill sets, including Python programming, exploratory data analysis, or EDA, and statistics. These tools will allow you to construct a simple linear regression model that can help you influence strategy and decision-making in any company or organization. The key terms, activities and learning resources that we explore together in these videos will also be important in the rest of this course,
38:59
which will refer back to our discussions of simple linear regression. Together, we'll go through all the stages of PACE while learning regression modeling. We'll first review the linear regression equation and then learn about estimating parameters using ordinary least squares. Then will define each of the four key assumptions of simple linear regression. As for the analyze stage, we'll use Python and EDA to verify if our data meets these assumptions. In the construct stage,
39:30
we'll build your first regression model in Python together. You'll also have opportunities to practice both Python and EDA on your own as well. We'll also learn several evaluation metrics and a technique to help you quantify how good your model is. Lastly, aligned with the execute stage, we'll use our metrics to practice interpreting our results for stakeholders and non-technical audiences. I'm so excited to get started on simple linear regression modeling, so let's begin.
40:01
In prior videos, we've mentioned simple linear regression, which is a regression technique that estimates the linear relationship between one independent variable X and one continuous dependent variable Y. Recall that the linear in linear regression indicates what the data looks like when plotted on an X-Y coordinate plane, a line. In simple linear regression, we're only interested in two variables, one X and one Y variable. So the equation for a regression line
40:33
is Y equals intercept plus slope times X, which is represented as beta-zero plus beta-one times X. Since we'll have a number of data points for any given problem, there are many different lines we could draw that might fit the data. However, we're looking for the best-fit line, the line that fits the data best by minimizing a loss function or error. In order to find the best-fit line, we need to measure error. We can consider error as some difference
41:08
between the data we have, the observed values, and the predicted values generated by a given model. The predicted values are the estimated Y values for each X calculated by a model. The difference between observed or actual values and the predicted values of the regression line are what's known as a residual. The equation for residual value is residual equals observed value minus predicted value. Each data point has one residual. Using mathematical notation,
41:45
the equation for the residual of an individual data point is epsilon-I equals Y-I minus Y-I-hat.
41:55
Epsilon is a Greek letter that resembles the letter E as in E for error. We can calculate individual residuals for each data point, but an important thing to note is that the sum of the residuals is always equal to zero for OLS estimators. For some estimators, the sum of the residuals is not always equal to zero. In order to capture a summary of total error in the model, we square each residual and then add up the residuals for every data point. This is called the sum of squared residuals.
42:29
The sum of squared residuals, or SSR, is the sum of the squared differences between each observed value and the associated predicted value. For linear regression, we'll be using a technique called ordinary least squares to get our best-fit line. Ordinary least squares, also known as OLS, is a method that minimizes the sum of squared residuals to estimate parameters in a linear regression model. Using OLS, we can calculate beta-zero-hat and beta-one-hat using properties of the sample data.
43:04
Recall that the hat symbol means it is an estimate of a parameter. We will never know the exact parameter. Remember, parameters, or betas, are a characteristics of a population. Since we will only ever have sample data, our goal is to get a reasonable estimate of the parameter. Let's say that we have a certain sample of data, and we now want to determine a line that fits the data well. For our first attempt at a best-fit line, the slope is one and the intercept is 2.5.
43:40
To calculate the sum of squared residuals, first we calculate the predicted values for each X. Next, we can find the residual for every observed value of X. On the graph, we've plotted the residuals. The residuals are the difference between each observed value and what the line predicted, so this line is okay, but let's try to get a little bit closer to the points. Let's try another line. This time we'll set the slope equal to 1.25 and the intercept equal to three.
44:13
Again, we have to plot the residuals and calculate the sum of squared residuals. From the graph, it seems we're getting closer, but it's hard to tell if we've got the best line. We could just keep trying different lines, and through trial and error, pick a line that we think is closest to the data points, but that's very time-consuming. The good news is that, in Python, the computer will use OLS, or the ordinary least squares estimation technique, to test out many lines
44:42
and identify which one is the best-fit line. With OLS estimation, we find that beta-zero-hat equals 1.5, and beta-one-hat equals 3.2. The lines we're estimating represent the best fit of the model to the data. Later, we'll talk about uncertainty using P-values and confidence intervals to aid in the interpretation of results. Now that we've covered what simple linear regression is, we'll go back to PACE in our regression analysis framework. We'll examine the model assumptions
45:16
that the data needs to follow for us to use this cool new tool in our regression toolbox.
45:25
Alright, let's begin with the analyze stage of the PACE framework. The first task in simple linear regression analysis is checking the assumptions of the model. In addition to the technical needs of the model, you'll need to consider the business context of the problem you're working on. This will come in the plan stage. Previously, I spoke about model assumptions as statements about the data that must be true in order to justify the use of a particular modeling technique.
45:53
Ensuring that we're using the right model given the data that we have allows us to be confident in the results those models produce. Think of model assumptions as the bridge between the analyze and construct phase of the PACE framework. In other words, examine the assumptions before the construct phase when possible. Certain assumptions can only be checked after model construction. So make sure you check those assumptions after you apply the model to confirm if the model is valid or not.
46:23
Data visualizations can be used as a tool to determine if model assumptions are met. Thankfully, Python will help you tremendously with generating these, and I'll be here to guide you through it all too. There are four key assumptions of a simple linear regression, linearity, normality, independent observations, and homoscedasticity. For now, we'll focus on understanding what each of these assumptions mean and how they are checked using data visualizations.
46:54
The first assumption of a linear regression just so happens to be the simplest to check for, the linearity assumption. You already know that the linear in linear regression comes from the way the data looks when plotted on an X-Y coordinate plane, a line. To detect if this assumption is met, you just have to make sure that the points on the plot appear to fall along a straight line. If the visualization looks like a random cloud or resembles a curve rather than a line,
47:26
then the assumption is considered invalidated, meaning that this model does not fit the data well. You might need a different or a more complicated model for this dataset. In contrast, a scatter plot that shows the data points clustering around a line indicates the linear regression would be an appropriate model to represent the relationship between X and Y.
47:49
Next up on the checklist is the normality assumption. This assumes that the residual values or errors are normally distributed. Since this assumption is about residuals, you cannot check the assumption until after the model is built, but once the model is built, you can create a specific plot called a quantile-quantile or Q-Q plot of the residuals. If the points on the plot appear to form a straight diagonal line, then you can assume normality and check this assumption off the list.
48:21
Next is the independent observation assumption, which states that each observation in the dataset is independent. Here, it is helpful to use contextual information about data collection and the variables used to determine if this is true. If the assumption is met, we would expect a scatter plot of the fitted values versus residuals to resemble a random cloud of data points. If there are any patterns, then we might need to re-examine the data. Last but not least,
48:52
the homoscedasticity assumption is fourth on the list. This one sounds complicated, but knowing the literal meaning of the term helps. Homoscedasticity means having the same scatter. Scatter plots come to the rescue once again when checking for homoscedasticity. Returning to the scatter plot of fitted values versus residuals, there should be constant variance along the values of the dependent variable. This assumption is true if you notice no clear pattern in the scatter plot.
49:24
Sometimes you'll hear this described as a random cloud of data points, but for example, if you observe a cone shaped pattern, then the assumption is invalid. Linearity, normality, independent observations and homoscedasticity are the four assumptions of a simple linear regression. Before moving ahead, don't feel like you need to memorize all of this material right now. Remember, data analysis is an iterative process. You can go back to these concepts,
49:53
check to see how the assumptions align with your data, and then move forward with the regression process. You'll have plenty of opportunities to practice and hone your skills throughout this course. Now let's apply everything you've learned here and try it out on a dataset using Python.
50:11
Are you ready to start practicing your computer programming skills now? In this video, we'll apply some of the concepts around simple linear regression to data. The move from theory to application is a big milestone, but remember that you have worked hard to get to this point. We'll go through the code in some depth today. You will have access to the code for you to review in detail. Let's begin by exploring a problem that's been affecting a local zoo
50:38
where you were recently hired as part of the analytics team. The caretakers who manage the penguin habitats are having trouble keeping their population adequately fed. They're hoping to find out if certain features of the birds are related to body mass to better manage their feeding routines. The dataset includes structural measurements of different penguins, such as bill length and body mass. Now that you have some context, you can use exploratory data analysis to start analyzing the data.
51:07
First, import some packages, Pandas and Seaborn, both will be especially useful today. So the amazing thing about Seaborn and many other libraries in Python and other programming languages is that there are built-in datasets that you can work with. You don't even need to download any files. Now load the penguin dataset and save it as a variable called penguins. The load dataset function returns a data frame. So penguins is a data frame. Now that you can access the data,
51:39
use the head function to examine the first couple of rows. There are a few continuous variables, bill length, bill depth and flipper length, all measured in millimeters. Body mass is measured in grams. There are also a few discreet variables, species, island and sex. Since you're working with simple linear regression, you'll focus on the continuous variables. If you would like to, you can access the code to see how I clean the data. I subset the data to include only two species of penguin
52:14
and dropped a couple of rows with missing data. Now you can start creating plots to identify some linear relationships between the continuous variables. The clean data is saved in a data frame called penguins_final.
52:28
Now you can input the data frame into Seaborn's pair plot function. Seaborn's pair plot function creates a scatter plot matrix. A scatter plot matrix is a series of scatter plots that show the relationship between pairs of variables. By using the pair plot function, you'll observe a few linear relationships in the scatter plots. The diagonal displays the distribution of the continuous variables. This assures you that the data has met the linearity assumption
52:57
for building a simple linear regression. First, bill length and body mass seem to be positively correlated. Next, flipper length and bill length also seem to be positively correlated. Lastly, body mass and flipper length also seem to be correlated. Let's explore the relationship between bill length and body mass further in terms of linear regression assumptions. We know we have met the first linear regression assumption of linearity. Luckily, the diagonal of the pair plots
53:29
also shows us the distribution of each variable. We can observe that both bill length and body mass are close to being normally distributed. This suggests that we'll probably have normally distributed residuals. The third assumption is of independent observations. Since each row has data on a different penguin, we have no reason to believe that one penguin's bill length or body mass is related to any other penguins. We can confirm the last assumption, homoscedasticity,
53:56
after we build our model when we graph the residuals. Now let's subset the data once more to isolate bill length and body mass. Note the use of double squared brackets, which tells Python which columns you want to choose. Write out the regression formula in terms the computer can understand, and you'll save it as a variable called formula. As you do this, pay careful attention to the column names. You need to specify the column names exactly so the computer knows how to run the regression model.
54:30
First, type the Y variable column name, which is body_mass_g, a space, and then a tilde, another space, and the X variable column name, which is bill_length_mm.
54:48
The tilde lets the computer know that whatever comes after is our X variable. The spaces are not necessary but can be helpful for clarity. Now that you have the data and the formula, you can create an OLS object using the OLS function from the Statsmodel module. Save the object as a variable called OLS.
55:13
Input the OLS formula variable as the formula argument of the OLS function. Then input OLS data variable as the data argument.
55:24
Next, you'll use the OLS objects fit method to actually fit your linear regression model to the data. Save the results as a variable called model. Finally, print out the result of the ordinary least squared's estimation, which is the technique the OLS function used to build the linear regression model. Then use the model's summary method, which will print out a table of many different statistics. This table contains lots of information. We'll go through some different sections of the table
55:58
later in this course
56:00
and leave the rest for you to explore on your own. For now, we'll focus on the bottom section which details the coefficients the model determined would generate the best-fit line. Since we're using a simple linear regression model, we have two coefficients, an intercept, or beta-zero, and a slope, or beta-one. You can find the Y-intercept of the best-fit line in the intercept row of the coefficient column, which is abbreviated to COEF in the table. In this case, it's negative 1,707.2919.
56:34
You can find the slope in the bill length row of the coefficient column. The slope of the best-fit line is 141.1904.
56:46
Let's rewrite this as a linear equation, which will help us interpret the results later.
56:52
Begin by plugging the variables into the linear equation. Y equals intercept plus slope times X. Y is the penguin's body mass in grams. X is the penguin's bill length in millimeters. Next, the model will just calculate the intercept and slope. Round both to the nearest hundredth or two decimal places. This means that penguin with one millimeter longer bill length has 141.19 grams higher body mass on average. Remember, you still need to examine some assumptions
57:27
about the residuals to double check the conclusion. Great, you have fit a linear regression model to the data. To finish checking the model's assumptions, calculate some fitted values using your model's predict method. Then you can access the residuals using the model variable. By residuals, we mean the difference between the actual and fitted values using the model's resid attribute.
57:55
Finally, create a couple of plots to confirm your findings. Use the Seaborn's reg plot function to plot the data with the best-fit regression line. You can observe a linear relationship between the variables, the best-fit line and a small shaded region around the line indicating the uncertainty around the model estimates. Returning to the linear regression assumptions, create a scatter plot of the fitted values against the residuals. This is a very common plot you'll encounter
58:28
when working with linear regression to check various assumptions.
58:33
From this plot, you can observe that residuals seem randomly spaced, which means you can assume homoscedasticity. A random-looking scatter plot is indicative that the independence assumption is not violated, but it's not the sole reason that we believe it to be true. You could examine inputs and other more advanced statistical tests to confirm this.
58:56
Lastly, create a histogram of our residuals to determine if the residuals are normally distributed. If the residuals are normally distributed following that classic bell curve shape, then you can confirm the normality assumption has been met as well. The residuals are a little bit skewed in the histogram, so you can create a Q-Q plot to verify normality.
59:20
You can use the Statsmodel Q-Q plot function to create the graph. There is a straight diagonal line trending upward with some slight curvature on the extremes. You may want to explore this further, but for now, this is pretty good confirmation of the normality assumption. Now that you've confirmed all the assumptions are met, you can say that the results from the regression model are likely reasonable. Wow, we went through a lot of code and many plots together. You should be so proud
59:52
of what you've accomplished in this video. Keep in mind that you can always review what we've covered along with the code and documentation. Great work.
1:00:02
So far, we've used PACE to think like a data professional as we address the penguin problem at our local zoo. We planned by thinking through the problem and sub-setting the available data appropriately. How can we better understand the relationship between penguin anatomy and their body mass? In the analyze stage, we performed EDA and checked model assumptions. Then moving on to the construct stage, as a reminder, there are two parts. We built our model
1:00:33
and were able to pull out some parameter estimates. Now we're going to focus on the next step of the construct phase, model evaluation. Model evaluation is an important practice in data analysis. Careful evaluation and interpretation of your regression model helps you understand its performance and accuracy. To start, let's revisit the results from our regression model. Based on the summary of results, you know that OLS has determined that the best-fit line has an intercept of negative 1,707.29
1:01:07
and a slope of 141.19, but randomness and unpredictability are characteristics of every regression model that make it difficult to predict outcomes with 100% certainty. After all, there is still a difference between our observed and predicted values. You've just found the model that you are most certain about. To explore the notion of uncertainty further, let's turn our attention to the rest of the OLS summary table rows about the intercept and bill length. There is a column labeled
1:01:42
P greater than the absolute value of T. This indicates the P-value associated with the coefficient estimates. The two columns to the right of the P-value column indicate a 95% confidence interval around the coefficient estimates. When evaluating simple linear regression results, you'll focus less on the intercept row and more on the row involving your independent variables of interest. In this case, that's bill length. So you can say that the coefficient estimates for bill length is 141.19
1:02:20
with a P-value of 0.000 and a confidence interval from 131.788 to 150.592.
1:02:32
Previously, when you learn about hypothesis tests, confidence intervals were defined as a range of values that describe the uncertainty surrounding an estimate. In the case of linear regression, we are estimating parameters, so a 95% confidence interval means that interval has a 95% chance of containing the true parameter value of the slope. What if the slope and intercept were slightly different? Well, let's draw out a few different lines on our plot
1:03:02
with slightly different slopes and intercepts, all within our confidence intervals. We get a region around the regression line that is tight around the center and fans out a bit towards either end of the line. This shape may appear familiar. You plotted it previously using Seaborn's reg plot function. These lines make up the shaded region that was around the regression line. Essentially, the confidence interval around the parameter estimates reveal what we call a confidence band.
1:03:32
A confidence band is the area surrounding the line that describes the uncertainty around the predicted outcomes at every value of X, typically expressed as a shaded region around the best-fit line on a scatter plot, a confidence band reveals the confidence interval for each point on a regression line. Confidence bands are simply another way to report your findings responsibly. Simple linear regression is a powerful addition to any data professional's toolbox.
1:04:02
Whether you are analyzing the financial impact of price increases to a streaming service or you're forecasting sales for a fashion boutique, regression analysis can help you make discoveries and understand the relationship behind the data. But we must remember data is noisy and results can be uncertain. When using regression models like simple linear regression, even the best data doesn't tell a complete story. As a data analytics professional,
1:04:32
you should always aim not only to evaluate the performance and accuracy of your models, but also to report uncertainty. Communicating about confidence intervals and confidence bands is part of being a responsible data professional. These metrics will also help you understand how well the models can tell the story behind the data.
1:04:55
Up to this point, we've progressed through the plan and analyze phases of the PACE framework for regression modeling. We've even started the construct phase by actually building a linear regression model. I'm excited to continue guiding you through model evaluation. We're getting so close to the execute phase where you share the stories behind the data you're studying. Using a variety of evaluation metrics supports data professionals' confidence and insights produced by their analysis.
1:05:26
These metrics are key to responsible communication of results. If models are inaccurate or imprecise, decisions made based on those insights may also be inaccurate.
1:05:38
Three metrics you might encounter are R-squared, mean squared error, also called MSE, and mean absolute error, or MAE. The main metric that academics, researchers and data professionals use when evaluating regression models is called the coefficient of determination, or R-squared. So that's what we'll focus on. You may have noticed that in the output from OLS model you previously built, there was a part of the output labeled R-squared. This is what we're talking about.
1:06:13
R-squared or the coefficient of determination measures the proportion of variation in the dependent variable Y explained by the independent variables X. To explain this metric more thoroughly, let's think through the example about penguins and linear regression again. You identified a linear relationship between the penguin's bill length in millimeters and the penguin's body mass in grams. Based on your regression analysis, you found the best-fit line, body mass in grams equals negative 1,707.30
1:06:50
plus 141.19 times bill length. But the data points just cluster around this best-fit line. Many of the data points are actually not on the line. This means that bill length only accounts for some of the changes in body mass. R-squared helps data professionals determine how much of the variation in the X variable explains the variation in the Y variable. At most, R-squared can equal one, which would mean that X explains 100% of the variance in Y. If R-squared equals zero,
1:07:28
then that would mean X explains 0% of the variance in Y. The OLS summary table shows the model has an R-squared of 0.769. This means that bill length explains about 77% of the variance in body mass. There is still about 23% of the variance of body mass that is unexplained by the model. This variance might be due to other factors or natural, unexplained differences from penguin to penguin. There is no benchmark value that R-squared has to equal, but in general, the higher R-squared, the better,
1:08:05
because it adds validity to any recommendation you make based on your analysis. R-squared is a useful metric that can help you evaluate your model, but there are also processes that help strengthen the evaluation of a model. Typically, when we have a dataset, we use at least part of the dataset to build and test the regression model. The computer uses the data to calculate a measure of difference between the actual and predicted values, such as sum of squared residuals.
1:08:39
Then based on the computer's calculations, it can find the best line, but sometimes, we want our model to be good at generating predictions for data we haven't collected or that doesn't exist yet. For example, let's return to the penguins at the local zoo. They welcomed a new group of penguins to their flightless bird habitat. It would be helpful to have a sense of how the new penguins structural measurements relates to their body mass. So in those cases, we want to know
1:09:10
how the model we built performs on the data it learned from and how the model performs on data it hasn't experienced yet. In this case, we'll need to save a holdout sample before we build the model. A holdout sample is a random sample of observed data that is not used to fit the model. Then you can evaluate how well the model fits the data used to build the model, and you can evaluate how well the model fits the holdout sample. We've covered a lot of different ways
1:09:42
you might evaluate linear regression model. R-squared and holdout samples will serve you in many cases, allowing you to confidently share the insights you've discovered. I'm excited to discuss the execute phase and how to communicate model findings soon.
1:10:00
Earlier in the course, you built a regression model that passed all four of the assumptions for simple linear regression. Residual plots confirmed a linear relationship, and you were able to successfully evaluate the performance of the model using a few common metrics. Next, it's onto the execute phase of the PACE framework. How exciting. This is the point of the PACE framework when your ability to communicate is crucial. Together we're going to review
1:10:28
interpreting the results of our regression model, exploring ways in which those insights can be translated into formal visualizations and communicating a meaningful narrative with stakeholders. In one of my previous positions, I was brought on to help a newly launched mobile phone service provider reduce the amount of fraudulent orders they were receiving. After building out a model, it was time to present the model insights and recommendations to my stakeholders.
1:10:57
These folks were non-technical business partners. So I knew that data specific terminology wouldn't interest them. As a matter of fact, it would be too technical, and they would probably lose interest very quickly, which would cause lack of buy-in. What was important to my business partners was stopping fraudulent orders, so telling the story in a way where I succinctly articulated how many fraudulent orders the model would detect and how that translated to the bottom line
1:11:24
resulted in the model being implemented immediately. I remember feeling so accomplished in that moment because after all the hard work of building the model, I was able to get the model into production. Once the model was in production, fraudulent orders dropped drastically. Telling the right story to the right people takes time and practice. This is a skill that we'll continue to work on throughout our careers. It's super important to know your audience and tailor your delivery and story
1:11:53
in order to have the most impact. Enough about me. Let's return to the example that we've been exploring where you were working at a local zoo. You were trying to understand the relationship between penguins' structural measurements and their body mass. In the regression model you created, you observed a positive correlation between bill length and body mass of a group of penguins. Let's use the results of that model to translate the statistical findings for stakeholders.
1:12:21
Regression model interpretation depends on coefficients and P-values. Coefficients will determine how changes in the independent variable are associated with changes in the dependent variable. P-value demonstrate whether coefficients are statistically significant. Recall that the slope is used to determine the amount you expect Y to increase or decrease per one unit increase of X. Let's go through the numbers yielded in the OLS summary table to gain a better understanding.
1:12:52
Body mass equals intercept plus slope times bill length. The slope is positive 141.19, and the intercept is negative 1,707.29. Next, let's review the P-value. The coefficient for bill length has a P-value of 0.000.
1:13:13
This P-value test the null hypothesis that the coefficient is zero. If we observe a small P-value, the null hypothesis is likely false. So we reject the null hypothesis and conclude that the coefficient is not zero, meaning that there is a strong relationship between X and Y. It's important to understand that the positive correlation between bill length and body mass you've observed does not necessarily reflect causation. Remember, causation describes a cause-and-effect relationship
1:13:46
where one variable directly causes the other to change in a particular way. The positive correlation between bill size and mass could be the result of a variety of factors. So even though we cannot speak to causation, we can still provide valuable insights about the penguins. Based on our regression analysis, we can say that for penguins with one millimeter more in bill length, their body mass is 141.19 grams more. These results are statistically significant with a P-value of 0.000,
1:14:21
a confidence interval of 131.79 to 150.59
1:14:27
and an R-squared of 0.77. By providing measures of uncertainty around our estimates, we're responsibly reporting our results. Based on the interpretation of the numbers, you can contribute to the zoo community around you. Each person on the team at Untrue Zoo brings a variety of experiences and knowledge. The input from your colleagues adds richer detail to the story the model tells. Without the informed perspectives of the other bird caretakers, the avian department would not be able
1:14:59
to make a sound argument for an increase in food to maintain an adequate inventory that supports the penguins and other birds at the zoo.
1:15:08
When sharing insights as a data analytics professional, you need to make sure that your findings can be quickly understood and correctly interpreted. Communicating the context of your data is one way you can report it responsibly. For example, if your results only apply to a specific penguin population, make it clear that people should be cautious about extrapolating to larger or different groups of penguins. Data visualizations are excellent ways of making statistics relatable for others.
1:15:39
Use caution when presenting terms like coefficients and P-values in your visualizations. Consider that not everyone will fully grasp the significance of these terms immediately.
1:15:51
There are many useful libraries like Matplotlib and Seaborn that can help create visualizations. Programs like Tableau, PowerPoint or Google Slides allow for creation of high-quality presentations to help you provide context that's relevant to the business problem. Regardless of your visualization tool of choice, communicating with those around you is critical to your success at all points of the PACE process. As a data analytics professional,
1:16:20
clear communication allows you and your work to have impact throughout your team and organization.
1:16:29
Well done. You've come a long way, and you built your very first regression model. I'm so thrilled to continue your regression journey with you. Before we wrap up, let's recap what you've added to your data toolkit. Linear regression analysis is a foundational data science technique. By now, you're familiar with PACE, plan, analyze, construct and execute, in linear regression analysis. You had an opportunity to use ordinary least squares estimation in Python.
1:17:01
You used OLS to get the best-fit line that minimizes the error between predicted and actual values. Next, you learn the four main model assumptions of simple linear regression, linearity, normality of residuals, independent observations and homoscedasticity. You had some practice applying EDA and Python to check whether linear regression is appropriate based on meeting these assumptions. You built a model in Python, and you learned how to evaluate model fit using R-squared,
1:17:33
holdout samples and measures of uncertainty like confidence intervals and P-values. Lastly, you turned numbers and statistics into a story. You explored a penguin dataset to show how a data analytics professional can present findings to others in clear, simple terms. You learn the value of producing visualizations in Python to communicate these results of your simple linear regression model to stakeholders, a valuable skill that you'll be able to use throughout your data analysis journey.
1:18:04
I'm proud of how far you've come along, and you should be too. So far, we've gone through all of PACE and simple linear regression. While we completed the circuit once, it doesn't stop there. Next, we'll extend our knowledge of simple linear regression to multiple linear regression models. Simple linear regression is great for tackling problems with a single independent variable. However, the more complex the problem, the more variables can influence what's going on.
1:18:34
That's where multiple linear regression becomes useful. You're making amazing progress and are on your way as a future data analytics professional.
1:18:45
Hi there. It's great to join you again on your journey towards becoming a data analytics professional. You've come a long way since the start of this course. Together, we covered PACE in regression modeling and an overview of two foundational regression models, linear and logistic regression. We used your statistical knowledge to discuss how simple linear regression works and when to use it. You learned about model evaluation. You also learned how to interpret and communicate regression results
1:19:16
effectively and accurately. All of these skills will translate to other regression and machine learning models. As you grow as a data professional, you'll be able to use these skills to discover many data-driven insights. Simple linear regression is a great foundational technique, but it can feel limiting as a technique because it only allows for one independent variable. There are many cases where you might be interested in two, three, or four independent variables.
1:19:48
For example, there could be many factors that are associated with product sales in a given month, such as holidays, new product launches, changes to the retail website or changes to marketing campaigns. We need a new technique to figure out how each of these variables is correlated with product sales. This is where multiple linear regression comes in. In the upcoming videos, we'll be exploring the world of multiple linear regression, often referred to as multiple regression.
1:20:19
Just how we went through PACE, plan, analyze, construct and execute, with simple linear regression, we'll do the same with multiple regression. To start, we'll discuss multiple regression. While simple linear regression only allows one independent variable X, multiple regression allows us to have many independent variables that are associated with changes in the one continuous dependent variable Y. Adding more independent variables into the equation complicates the math,
1:20:51
but everything we covered is just an extension of simple linear regression concepts. As a result, we'll go back to our statistical basis and revisit PACE. Since this isn't your first model, we're going to start with A, analyze. We'll review model assumptions from multiple linear regression. EDA will continue helping us determine if our assumptions hold true. Then we'll focus on construct phase where we'll learn how to build the model, and you'll have a bit more practice with Python.
1:21:26
Next, in the execute phase, we'll focus on interpretation and telling a story from multiple linear regression. With more independent variables, we have to think more carefully about the insights we derive and how we communicate our results. Then we'll iterate back to the construct phase and learn a bit more about the nuances of multiple linear regression. As data analytics professionals, one of the most important skills in figuring out the best model for each use-case,
1:21:55
which goes back to the plan and PACE. In pursuit of finding the best model and as models become more complex, we need new ways to evaluate them effectively. As a part of finding the best model, we'll learn about variable selection and regularization. In your career as a data analytics professional, you'll likely encounter some very big datasets, and it may be challenging to figure out which variables are statistically important just by looking at the data,
1:22:25
but the tools you learn here will allow your computer to calculate a possible set of variables for a given model, output and evaluation metric. Although the computer is performing incredible mathematical operations, you as the data professional are reading the output so that you can understand and communicate the relationships the computer has uncovered. Now it's time for us to learn more about multiple regression and how we can use different independent variables
1:22:55
to estimate a dependent variable. Let's keep putting together the regression puzzle.
1:23:02
Previously, we learned about simple linear regression. For example, the more advertisements a company uses, the more clicks their website receives. This is an example of a relationship between variables that can be modeled with simple linear regression because there is only one X variable and one Y variable. The number of website clicks is a continuous dependent variable. The number of advertisements is the independent variable. But the company might be interested
1:23:32
in exploring the characteristics of each advertisement to determine what kinds of advertisements are related to more website clicks. Perhaps shorter advertisements are correlated with more clicks, or maybe advertisements containing a call to action, such as donate or subscribe, are associated with more clicks. Multiple linear regression can help us answer these kinds of questions. Multiple linear regression, also known as multiple regression, is a technique that estimates the linear relationship
1:24:03
between one continuous dependent variable and two or more independent variables. Let's re-examine the equation for a simple linear regression. Y equals beta-zero plus beta-one times X. Recall that beta-zero, also called the Y-intercept, and that beta-one is also called the slope.
1:24:26
Let's say that Y represents the website clicks, and X represents the number of people that are in the advertisement. Now let's add in the length of the advertisement into the equation. Y equals beta-zero plus beta-one times X-one plus beta-two times X-two. The X-one represents the number of people that are in the advertisement, and X-two represents the length of advertisement. Note that because we have two X variables now, we've added subscripts to differentiate between them.
1:25:01
You'll encounter this notation often in multivariate analysis. Multiple regression allows us, at a basic level, to add any number of independent variables that we're considering. So a full multiple regression equation might be Y equals beta-zero plus beta-one times X-one plus beta-two times X-two plus as many variables you have up to beta-N times X-N where you're interested in N independent variables. Even though we are just adding more beta coefficients and independent variables,
1:25:39
we can still reap the benefits of linear regression. Just like simple linear regression, multiple linear regression can yield highly interpretable and communicable results, but because the underlying math is a bit more complex, we have to be mindful of how we convey our results and what the coefficients mean. To ensure we're ethically communicating our results as clearly as possible, we'll go over two concepts, one hot encoding and interaction terms. One hot encoding allows us
1:26:13
to use categorical independent variables in our multiple linear regression. For example, we might have print advertisements and digital advertisements. This is a categorical variable, and one hot encoding will allow us to incorporate it into our regression model. If we want to account for how two independent variables affect the Y variable together, we can use something called an interaction term.
1:26:41
Both of these topics are important because they will change how we interpret the coefficients of our model. We will learn how to do this together in the upcoming videos.
1:26:52
As a data analytics professional, you might encounter scenarios in which you're interested in variables that are not continuous. They could be categorical. For example, in the case of website clicks and advertisements, some ads are black and white and some ads are all in color, or maybe some ads have a call to action while other ads don't, or perhaps some product ads are on different streaming services. These are all categorical variables that could be related
1:27:22
to how many clicks the website receives. In the case where we have a categorical independent variable, we have to represent the categories as numbers for the computer to understand the data, there are two main ways to handle categorical data, one hot encoding and label encoding. In this video, we will learn about one hot encoding. One hot encoding is a data transformation technique that turns one categorical variable into several binary variables. Let's take the example
1:27:54
of an advertisement having a call to action or not. We would create a new variable in the dataset. Let's call it action, and mathematically we'll denote it as X-action. If an advertisement has a call to action, then X-action equals one. If an advertisement commercial does not have a call to action, then X-action equals zero. Our equation would become Y equals beta-zero plus beta-one times X-one plus beta-two times X-two plus beta-action times X-action.
1:28:31
X-one measures the number of people in the advertisement, and X-two measures the length of the advertisement. Now let's assume that we have two advertisements where X-one and X-two are the same, but one has a call to action and one doesn't. We can then assume that the advertisement with the call to action had beta-action more website clicks than the advertisement without a call to action. Now let's remove the call to action variable. What if we are interested
1:29:02
in which streaming platform the ad is on? Let's say the company is running ads on three services, A, B and C. Let's also assume that ads can only run on one platform at a time. So if an ad is on service A, it's not on service B or C. Now we have a categorical X variable that has three possibilities. Let's figure out how to represent it mathematically. To represent two possibilities, has a call to action versus does not have a call to action, we use one binary variable.
1:29:36
In order to represent three possibilities, we need two binary variables. Let's examine why this works. Let's imagine we have a binary variable X-service A. If X-service A equals one, we know that the advertisement is playing on service A, but we also know two other pieces of information. If the ad plays on service A, then it is not playing on service B and the ad is also not playing on service C. If X-service A equals zero, we only know that advertisement is not on streaming service A.
1:30:11
That advertisement could play on either service B or C. Since we have missing information, we need another binary variable to help us and the computer figure it out. Let's add a variable X-service B. If X-service A equals one, we already have all of our information. We know that the ad plays on service A. So X-service B must equal zero. The ad does not play on service B, and we also know it does not play on service C, but if X-service A equals zero,
1:30:48
we can learn more information from X-service B. If X-service B equals one, then we know that the ad plays on service B. In turn, we know that the ad does not play on service C. Lastly, if X-service A equals zero and X-service B also equals zero, then we know the ad plays on neither service A nor service B. So the ad must play on service C. Now we have all the information we need with just two variables. So let's revise the equation one more time.
1:31:27
We have Y equals beta-zero plus beta-one times X-one plus beta-two times X-two plus beta-service A times X-service A plus beta-service B times X-service B. Now that the equation is written out, you'll notice there is no variable X-service C because it would not provide us with more information, but the interpretation is a little bit different. We can think of service C as the default streaming service, so beta-service A is the difference in website clicks
1:32:04
for two advertisements that are the exact same except one ad is played on service C and one is played on service A. Similarly, beta-service B is the difference in website clicks for two advertisements that are the exact same except one ad is played on service C and one is played on service B. In this video, you learned how one hot encoding allows us to turn one categorical variable into several binary variables. Now we can start estimating how many clicks the website will get
1:32:38
based on variables about the ad. Soon we'll go over how to use Python to one hot encode categorical variables. We will revisit PACE and regression modeling through model assumptions, model construction, model evaluation and model interpretation together. Great work so far. I can't wait to join you in the next video.
1:33:03
In previous videos, you learned about the model assumptions of simple linear regression. In this video, we'll review the assumptions you're already familiar with and introduce a new assumption specific to multiple linear regression. Simple linear regression and multiple regression share four assumptions, linearity, independent observations, normality and homoscedasticity. The main diagnostic tools are the same, scatter plots and plotting the residuals after model construction.
1:33:34
To recap, the linearity assumptions states that each predictor variable X-I is linearly related to the outcome variable Y. Plotting scatter plots of each X variable against the Y variable will inform us which variables likely have a linear relationship with Y. The independent observation assumption states that each observation in the dataset is independent. We can only examine this assumption by checking the data collection process. For example, if we are collecting data about income,
1:34:07
people from the same household might not be independent from each other. The next assumption is the normality assumption, which states that the residuals are normally distributed. The last assumption we're reviewing is the homoscedasticity assumption, which states that the variation of the residuals, errors, is constant or similar across the model. There is a completely new assumption when we are working with multiple regression. The data cannot be multicollinear.
1:34:37
The no multicollinearity assumption states that no two independent variables, X-I and X-J, can be highly correlated with each other. This means that X-I and X-J cannot be linearly related to each other. For example, if you're working at a concert venue, you might want to predict concert ticket sales. There are many factors involved, number of social media followers, number of streams on the music platforms, year the artist debuted, cost of the tickets, how many days until the concert and more.
1:35:13
Although the cost of the ticket and how many days left until the concert might both be strong predictors of concert sales, it is likely that the cost of the ticket is correlated with how many days are left until the concert. Maybe there's a sale the week before the concert. So sales spike up. If you keep both variables in the regression model, it will be unclear which factor has what kind of effect. Additionally, you might not be explaining significantly more of the variance in ticket sales.
1:35:42
Let's do some exploratory data analysis to confirm our initial thoughts.
1:35:47
We can create a scatter plot matrix using Python code to show the relationship between pairs of variables. The scatter plot matrix creates a scatter plot for every pair of variables. If we're observing linear relationships between an independent variable and the dependent variable, we should consider including it in our multiple regression model. If we observe linear relationships between two independent variables, and we include both variables in the model,
1:36:15
we'll likely violate the no multicollinearity assumption. In the concert ticket example, you have six variables in the dataset, one dependent variable, concert ticket sales, and five independent variables, number of social media followers, number of streams on music platforms, year the artist debuted, cost of the ticket, how many days until the concert. You might expect the number of social media followers and the number of streams to be highly correlated with one another based on our context,
1:36:50
but you can't confirm this hunch until you create scatter plots. That being said, EDA and visualizations are powerful tools, but they can't detect every relationship, so we turn back to the math. Conveniently, our computer can calculate VIF. The VIF quantifies how correlated each independent variable is with all the other independent variables. The minimum value of VIF is one, and it can get very large. The larger the VIF, the more multicollinearity there is in the model.
1:37:24
Once we have identified multicollinearity, one solution is to drop one or more variables with multicollinearity. Another possible solution is creating new variables using existing data. Then you can calculate the VIF again to check the multicollinearity assumption again. Remember, when in doubt, explore your data. EDA and visualizations are critical to understanding underlying trends and telling a compelling story. By taking these steps, you can ensure that that your model fits the data
1:37:57
and that your results are reasonable. Great work so far, keep it up.
1:38:04
We've already interpreted the results of simple linear regression before. For example, if we plot some data about temperature and ice coffee sales, the scatter plot will show us a series of points in a diagonal line trending upward. We are able to interpret the results relatively simply. In this video, we'll go step-by-step through the process of building a multiple regression model and then interpreting the results. As an example, let's say the data follows this equation,
1:38:33
sales equals negative 44 plus 2.2 times temperature. We can say that a one degree increase in temperature is associated with 2.2 more ice coffee sales. This is great and probably explains a large amount of why ice coffee sales vary on any given day, but there are other factors involved, and if you work for a large coffee company, you might be interested in exploring factors under your control. You can't change the temperature, but you can be strategic about where you build your store
1:39:09
or whether or not you have an ad on a nearby building. Thankfully, we're learning about multiple linear regression now. So as a data analytics professional, you can help your coffee company answer these questions. So let's add one more variable to the equation. Is there an ad near the store? Let's revise the equation accordingly. Sales equals beta-zero plus beta-temperature times X-temperature plus beta A times X-ad. The ad variable is a binary variable. There are two possible scenarios,
1:39:47
when there is an ad nearby and when there isn't an ad nearby. If there is an ad posted nearby, then X-ad equals one, and the temperature will take on some value. Let's say it's 75 degrees Fahrenheit. So the equation becomes sales equals beta-zero plus beta-temperature times 75 plus beta-ad times one.
1:40:13
We estimate that the presence of the ad is associated with some increase in sales. Now let's take the same temperature and say that there isn't an ad nearby. The equation then becomes sales equals beta-zero plus beta-temperature times 75 plus beta-ad times zero.
1:40:35
Simplifying this equation, we get beta-zero plus beta-temperature times 75. The beta-ad terms drops out because it's multiplied by zero. When there isn't an ad, then the sales are only a function of the temperature. Now let's remove the ad variable and introduce the question of proximity to public transportation. The variable transportation will measure how many kilometers away a given coffee shop is to a bus, train, or subway stop. The equation becomes sales equals beta-zero
1:41:09
plus beta-temperature times X-temperature plus beta-transportation times X-transportation, where temperature and distance to transportation are continuous variables. For every one degree increase in the temperature, while holding distance to public transportation constant, we expect ice coffee sales to increase by beta-temperature. For every one kilometer further store is from public transportation, while holding temperature constant, we expect ice coffee sales
1:41:43
to decrease by beta-transportation. Note that we have to hold the other variable constant when interpreting the results. The math will explain why. Now we've gone over what to do when we have several standalone variables, but there are cases when we might expect two variables to interact. For example, in the case of temperature and distance of transportation, we might expect that if it's cooler, distance of transportation might have a different effect. If we want to account
1:42:16
for how two variables' values affect each other, we include an interaction term. An interaction term represents how the relationship between two independent variables is associated with changes in the mean of the dependent variable. Typically, we represent an interaction term as the product of the two independent variables in question. Going back to the example of coffee shop sales, if we suspect that distance from transportation might be associated with different changes
1:42:47
in coffee shop sales based on the temperature, we can include the interaction term temperature times transportation. Originally we said that sales equals beta-zero plus beta-temperature times X-temperature plus beta-transportation times X-transportation. If we want to include the interaction between temperature and transportation, we can revise the equation to be sales equals beta-zero plus beta-temperature times X-temperature plus beta-transportation times X-transportation
1:43:23
plus beta-interaction times the interaction between temperature and transportation. The interaction is represented with a multiplication symbol. In this example, we took into consideration the interaction between independent variables using the interaction term. We'll continue to learn how to interpret the nuances of regression coefficients in this course. We've covered a lot so far and practice will help you build confidence in these concepts.
1:43:52
Spend time connecting the resources available in this course to help you use multiple regression, interpret the results and tell the data story. Next, we'll keep digging into multiple regression.
1:44:06
We explored some interesting and more complex questions that multiple regression can help us answer. We went through an overview of how to interpret multiple regression models. Now let's turn to Python. Just like with simple linear regression, we'll find the best-fit line by minimizing the sum of squared residuals, which is a measure of error. While we could spend a lot of time performing ordinary least squares estimation one step at a time, Python has some built-in functions
1:44:35
that help us build models. This way, we can spend our time as data analytics professionals exploring the data and communicating insights from the calculations. We're going to revisit the penguin's dataset from earlier to see if we can learn more about the penguin's body mass. We've already done some exploratory data analysis and data cleaning and saved data as a variable called penguins. Get a quick summary of the data using the Pandas head function.
1:45:03
There are four different variables in the dataset, body mass in grams, bill length in millimeters, gender and species. The data is relatively clean. For this problem, you'll try to predict body mass based on the other variables. Next, divide the dataset into the independent variables, or Xs, and the dependent variable Y. This step helps prepare the data for the function you'll use to create training and holdout datasets. Use the train test split function
1:45:35
from the scikit-learn library to create a training dataset and a holdout or testing dataset. First, import the function. Now use the prepared data to divide the data into training and holdout datasets. Note that the test size variable is the proportion of data you're randomly assigning to the holdout dataset. In this case, you're holding back 30% of the data to test the model. Depending on the context, it may be appropriate to hold back more or less of the data.
1:46:08
The random state variable does not have to be set, but you are assigning it the value of 42 so that you can replicate our results. You'll encounter the number 42 frequently in code documentation. The number 42 is a significant number in a popular science fiction novel and has been adopted by the computer science community. If you put in a different value for the random state variable, you'll get different results. It does not matter what number you put, but setting the random state
1:46:39
allows someone else to replicate your work.
1:46:43
Next, we need to think about our multiple regression formula. Our independent variables are bill length, gender and species. We'll use the Statsmodel's module to run the regression like you did for simple linear regression. The Statsmodel's ordinary least squared function, OLS, needs to know your regression formula. Save it as a variable. OLS_formula equals body mass tilde bill length
1:47:13
plus gender plus species. Note that we added capital C and parenthesis around gender and species. The notation lets Statsmodel's OLS function know that gender and species are categorical variables. The function will then encode the variables. If you haven't already imported the OLS function, do so with the following line of code.
1:47:38
The data was saved already as a data frame called OLS data. You can input your formula and the data frame into the OLS function. Lastly, use the OLS objects fit method to actually fit the model to the data. One of the benefits of using Statsmodel's OLS function is that it provides us with a compact summary table of relevant statistics. We can easily find the coefficients. standard errors, T-statistics P-value and confidence intervals for each independent variable.
1:48:11
These values let us interpret the regression results quickly and easily. Access the OLS summary table using the summary function. Let's explore the male variable. Under the column for the coefficient, there is a row labeled C parenthesis male. The way the variable was encoded was male is one and female is zero. This means that the baseline or reference point is female penguins. So the coefficient indicates how much body mass would differ between two penguins that only differ in gender.
1:48:45
Assuming the male and female penguins are the same species and have the same bill length, we expect the male penguin's body mass to be about 528.95 grams more than the female penguin. The P-value is very small, so this coefficient is statistically significant. Now let's consider the row for bill length. Assuming that two penguins are the same gender and species, if the bill length increases by one millimeter, we would expect the penguin with the longer bill
1:49:18
to be about 35.55 grams larger in body mass. The P-value is very small. So the estimate is statistically significant. The OLS summary table also gives you model evaluation metrics like R-squared. The R-squared is 0.85 indicating that your model explains about 85% of the variance in body mass. This seems reasonable, but we will discuss the importance of metrics other than R-squared when working with multiple linear regression later. You know how to fit the model to the data
1:49:52
and get a summary table of statistics, and you explored the coefficients and P-values. There's a lot more in the table so you can investigate any of the metrics we have not covered yet in this course. I encourage you to peruse the readings in this lesson and try out the code on your own. All of these statistics form the basis for interpreting results and communicating with stakeholders. Great work so far. I'll join you again next time.
1:50:20
So far, we've extended simple linear regression to multiple regression, which is a powerful tool, because it allows us to answer a wide variety of questions. Whenever we're estimating a continuous variable by using several different independent variables, multiple regression is a good first step, but every tool has its limitations. When we first discussed simple linear regression, we focused on R-squared as the most common metric for evaluating linear regression.
1:50:48
To recap, R-squared is the proportion of variance of the dependent variable Y explained by the independent variable or variables X.
1:51:00
This seems like a reasonable and highly interpretable metric for determining how good of a linear regression model you have. If you recall, when we examine the output of multiple regression model, R-squared was still one of the metrics listed, but when we started adding more independent variables into the equation, R-squared gets more complicated. Whenever you add another independent variable to a multiple regression model, the R-squared increases without fail,
1:51:29
but not all variables added to a model equally contribute to understanding changes in Y. This is a problem because the high R-squared can be misleading. If we are just trying to get a high R-squared without considering what each variable contributes, the model becomes very specific to the data it was built on. Therefore, the model is no longer applicable to a larger population. As data analytics professionals, we call this overfitting. Now we're going to focus on what to do
1:52:01
now that we know about overfitting. Overfitting in the data space is when a model fits the observed or training data too specifically and is unable to generate suitable estimates for the general population. The conclusion from the regression model no longer applies to the population, which we are trying to draw conclusions for, and only applies to the data used to build the model. Overfitting tends to occur when a model is too complex or incorporates too many variables.
1:52:32
Preventing overfitting is one of the reasons that we use model evaluation techniques like holdout sampling, which is when we set aside a part of the data we already had but did not use to fit the model. By using a holdout sample, we can observe if the model performs just as well on data it has not experienced yet. Aside from holdout sampling, we can also use another metric called adjusted R-squared to evaluate multiple regression models.
1:53:02
Adjusted R-squared is a variation of the R-squared regression evaluation metric that penalizes unnecessary explanatory variables. Just like R-squared, adjusted R-squared varies from zero to one. Adjusted R-squared is used to compare multiple models of varying complexity. R-squared is more useful when interpreting the results of a regression model as you can determine how much variation in the dependent variable is explained by the model. Now that we have reviewed the problem of overfitting
1:53:36
and some ways to better evaluate multiple regression models, we can turn to variable selection. After all, we need a method to determine which variables to include and exclude in our models. I'm excited to explore variable selection and other techniques later in this lesson.
1:53:56
In previous videos, we define overfitting as a problem when we modeled the observed or training data too specifically and are unable to generate suitable estimates for the general population. One way to better evaluate how good a model is while factoring in overfitting is using adjusted R-squared, which penalizes unnecessary variables in a model. We learn that adjusted R-squared is most effective when we can compare multiple models that are using different subsets of independent variables.
1:54:27
In this video, we'll review a couple of variable selection techniques. Thankfully, Python and our computer will take care of a lot of the work very efficiently, but it's important as a data analytics professional to have a high-level understanding of what's going on in your machine. Variable selection, also known as feature selection, is the process of determining which variables or features to include in a given model. As with many of the processes discussed in this program,
1:54:56
variable selection is iterative. As you grow as a data analytics professional, you will develop a stronger intuition for how to go about variable selection. In this video, we'll cover forward selection and backward elimination, which are based on extra-sum-of-squares F-test. These simple techniques will allow you to continue exploring the world of multiple regression and prepare you for more advanced techniques covered later. Forward selection and backward elimination
1:55:27
essentially work from opposite directions of the problem. We know that a model with zero independent variables is probably not the best choice. We know that a model with all of the possible independent variables is also probably not the best choice. Forward selection is a stepwise variable selection process. It begins with the null model, with zero independent variables, and considers all possible variables to add. It incorporates the independent variable
1:55:56
that contributes the most explanatory power to the model based on the chosen metric and threshold. The process continues one variable at a time until no more variables can be added to the model. Backwards elimination is a stepwise variable selection process that begins with the full model, with all possible independent variables, and removes the independent variable that adds the least explanatory power to the model based on the chosen metric and threshold.
1:56:24
The process continues one variable at a time until no more variables can be removed from the model. Both forward selection and backward elimination require more cutoff points or threshold to determine when to add or remove variables respectively. One common test is the extra-sum-of-squares F-test. The extra-sum-of-squares F-test quantifies the difference between the amount of variance that is left unexplained by a reduced model that is explained by the full model.
1:56:56
The reduced model can be any model that is less complex than the full model. For the F-test, like other hypotheses tests, data professionals usually evaluate it based on a P-value. Based on the P-value, we can be fairly confident that important variance is being explained by a given model. We'll revisit F-tests later in this course when we talk more about hypothesis testing and estimating categorical variables. But great work so far. We've covered forward selection, backward elimination,
1:57:28
and extra-sum-of-squares F-test. This provides a great start to variable selection and making intentional decisions when constructing a multiple regression model. Coming up, we'll go over how to perform variable selection using Python and continue exploring ways to control for overfitting.
1:57:50
So far, we've covered a lot about the versatility of linear regression models in impacting business decisions and strategy, but no tool is perfect. Previously, we introduced the problem of overfitting when a regression model is fit too closely to the training data and therefore has trouble properly estimating the population data. The problem of overfitting is related to the bias-variance tradeoff, a concept at the heart of statistics and machine learning.
1:58:18
The bias-variance tradeoff balances two model qualities, bias and variance, to minimize overall error for unobserved data. The ideal model has some bias and some variance. Bias simplifies the model predictions by making assumptions about the variable relationships. A highly biased model may oversimplify the relationship underfitting to the observed data and generating inaccurate estimates. For example, given some data, we could assume that Y equals two. This is a highly biased model.
1:58:52
Variance in a model allows for model flexibility and complexity. So the model can learn from existing data, but a model with high variance can overfit to the observed data and generate inaccurate estimates for unseen data. Note, this variance is not to be confused with the variance of a distribution. We can think of bias and variance as two ends of a scale. We don't want there to be too much bias or too much variance. So we have to ask ourselves as data professionals
1:59:23
how to balance some bias and some variance to minimize error and get the best model possible. Speaking about balancing, as data analytics professionals, we have to balance knowing that even as we gain experience, there's always going to be more to learn. I think it's super important to always keep learning and avoid complacency, like you all are currently doing. Personally, I try to attend at least two conferences a year to learn what is going on in the industry
1:59:50
and to network with other data professionals. I'm also really fond of collaborating and asking questions and have found that I learn so much from my colleagues this way. Know that at this point, you have a really solid foundation, and we want to continue working with you to expand your vocabulary and toolkit even further. Now it's time to learn about regularized regression. Regularization is a set of regression techniques that shrinks regression coefficient estimates towards zero,
2:00:19
adding in bias to reduce variance. By shrinking the estimates, regularization helps avoid the risk of overfitting the model. There are three common regularization techniques, lasso regression, ridge regression, and elastic-net regression. We won't go into all of the mathematical underpinnings of the techniques, but there are lots of resources in this course and online for you to explore further if you're interested. For all three kinds of regularized regression,
2:00:48
some bias is introduced to lower variance in the model. Lasso regression completely removes variables that are not important to predicting the Y variable of interest. In ridge regression, the impact of less relevant variables is minimized, but none of the variables will drop out of the equation. Ridge is a great option if you want to include all of the variables. When working with large datasets, we can't always know if we want variables to drop out of the model or not.
2:01:19
So we can use something called elastic-net regression to test the benefit of lasso, ridge and a hybrid of lasso-ridge regression all at once. Each regularized regression technique is trying to help us better fit our model, but keep in mind that the estimated parameters are much harder to interpret than with simple linear regression or multiple regression. That concludes our brief exploration into regularization and the bias-variance tradeoff. Now that you know the basics of regularization
2:01:49
and the bias-variance tradeoff, you can continue learning about how to find the best regression model. Keep up the great work, and I'm excited for us to continue on your journey to regression analysis.
2:02:02
Wow, we extended a lot of what we learned about simple linear regression to multiple linear regression. While simple linear regression models the linear relationship between one independent variable X and one continuous dependent variable Y, multiple regression models the linear relationship between two or more independent variables and one dependent variable. The allowance for multiple independent variables provides data analytics professionals
2:02:31
with the ability to ask a wider variety of questions. We reviewed the model assumptions for simple linear regression and extended them to multiple regression. The model assumptions that apply to multiple linear regression are linearity, independent observations, normality, homoscedasticity and no multicollinearity. We showed how to test each of these assumptions using either math, built in Python functions, or through visualizations during EDA. We highlighted multicollinearity
2:03:03
as that assumption is specific to multiple regression. We then provided some examples of how to code a multiple regression in Python. There are similarities with coding a simple linear regression, but there are some key differences. Next, we focused our time together on how to interpret and evaluate multiple regression results, now that we have so many independent variables. We also discussed understanding the computer output, which is critical to providing accurate and nuanced storytelling.
2:03:35
Lastly, as part of model interpretation and evaluation, we reviewed the problem of overfitting and ways to counteract it. Overfitting occurs when a model too closely matches a training dataset or observe data and is unable to predict unseen data or generalize to the population either. To combat overfitting, we discussed two variable selection techniques, forward selection and backward elimination. Both use the extra-sum-of-squares F-test to determine whether to add or remove a variable.
2:04:10
To wrap variable selection, we provided an overview of regularization, which helps prevent overfitting. To understand regularization, we defined the bias-variance tradeoff, which is core to many data science and machine learning model decisions. A model with high bias might underfit the data, oversimplifying the model. A model with high variance might overfit the data, over complicating the model. We introduced three regularization techniques, lasso, ridge and elastic-net regression.
2:04:44
They all add bias to reduce variance. Regularization is particularly helpful when working with large datasets as it can be difficult to predetermine which variables are or are not important. Wonderful work so far. We've gone over many concepts. Everything we've worked on together is here for you to review on your time. Good luck, and I'll join you again next time.
2:05:09
Hey, welcome back to modeling variable relationships. We've covered a lot of material about regression so far. We worked through the assumptions, construction, evaluation and interpretation of simple linear regression model and then built a more complex model using multiple regression. These tools gave us powerful ways to ask and explore questions about continuous variables. For example, if we were interested in understanding product sales,
2:05:38
multiple regression would allow us to consider the impact of both digital video and print advertising on sales. Now we'll start focusing more on categorical variables. By expanding our toolkit to include more hypothesis testing, we can start asking and answering a different range of questions. For example, is the difference among three or more groups statistically significant? Is the distribution of observed data different from what we expected? Perhaps we're conducting user research,
2:06:10
and you want to make a change to the website you're analyzing. You can use hypothesis testing to determine if user engagement is different between a couple groups where each group is using distinct website layout. Recall that hypothesis testing is a statistical procedure that uses sample data to evaluate an assumption about a population parameter. You are testing hypotheses about a population to see if there are any significant differences. For example, in the case of user research on a website,
2:06:40
you could test the hypothesis that changing the color of the subscribe button or the placement of images might change how much time users spend on the website. You can utilize different models and tests from your data toolbox to answer various questions. You can also use some techniques together. For example, sometimes, you might want to combine a regression model with hypothesis tests or a series of hypothesis tests. The core principles of data analysis continue to apply to each technique.
2:07:13
We want to explore and better understand the relationships between different variables. We can use what we have learned to tell compelling stories about the data. Based on the kind of data we have available, we can determine which test or model will be most appropriate, but sometimes we have to use several tests and models before we can determine the best approach to answer our questions. In this video, we'll start with chi-squared distribution and chi-squared test.
2:07:41
Chi is the 22nd letter of the Greek alphabet and looks a lot like a fancy capital X in English. Our discussion will expand on prior learnings about T-test and hypothesis testing. Chi-squared test will help us determine if two categorical variables are associated with one another and whether a categorical variable follows an expected distribution. For example, if we flip a two-sided coin many, many times, we would expect about half of the time one particular side comes up,
2:08:12
about the other half of the time, the other side comes up. Next, we will examine analysis of variance, or ANOVA, and its variance, analysis of covariance ANCOVA and multivariate analysis of variance and covariate MANOVA and MANCOVA, respectively. By unpacking variable relationships with a focus on categorical data, we'll be able to expand the types of data we can effectively use to draw conclusions about various decisions, strategies and practices in industry.
2:08:48
Previously, we encountered hypothesis testing in the form of one-tail and two-tail T-tests. T-tests help us answer questions about whether the mean of two different groups is significantly different. In this lesson, we'll explore if our data is what we expected it to be. We'll introduce two hypothesis tests, the chi-squared goodness-of-fit test and the chi-squared test of independence. These tests will help us compare our expected and observed data. We'll revisit null and alternative hypotheses
2:09:22
when we define chi-squared tests, just like you might have done with T-tests. When using linear regression, we were primarily focused on continuous variables, but what if our variables aren't continuous? Chi-squared tests can address questions involving categorical variables. For example, let's say you're working at a movie theater which sells small, medium, large, and extra large portions of popcorn. You have projections or some expected counts of how each size will sell.
2:09:52
In a bar chart, it can be difficult to determine if the expected counts are statistically the same as what you observed, but you can run a chi-squared goodness-of-fit test to answer the question. The chi-squared goodness-of-fit test determines whether an observed categorical variable follows an expected distribution. As a data practitioner for the movie theater we mentioned, you're working on the popcorn sale problem. An employee claims that 25% of people order each size on any given day.
2:10:24
Now you can create a table of counts based on the number of popcorn buyers on a given day. So let's say a hundred people bought popcorn yesterday. Then you can can multiply the percentage by the total to figure out the expected counts for each cell in the table. 25% of a hundred is 25. So you expect that 25 people bought each size of popcorn. For the chi-squared goodness-of-fit tests, the null hypothesis states that 25 people buy each size of popcorn on any given day.
2:10:53
Basically, the null hypothesis states that the variable follows the expected distribution. If we accept the null hypothesis, then we can say that the observed distribution of popcorn sells is the same as what the employee claims. The alternative hypothesis states that the variable doesn't follow the expected distribution. Basically, the distribution of the observed data is significantly different from what we expected it to be. This means that different number of people
2:11:20
buy each size of popcorn on any given day. In this case, the chi-squared statistic equals the sum of the observed number minus the expected number squared divided by the expected number. Once you gather the observations about popcorn sells, you can then use the chi-squared statistic to calculate the P-value and determine if you can reject the null hypothesis at the given confidence level. The next test is called the chi-squared test for independence, sometimes called a test of homogeneity.
2:11:53
The chi-squared test for independence determines whether or not two categorical variables are associated with each other. For example, let's say you are wondering if weather is associated with popcorn sales. Maybe when it rains, people are more likely to buy hot buttery popcorn. To state the hypothesis, you first need to determine your variables. In the popcorn case, the first variable could be whether there's rain or not. The second variable could be if more than a hundred people buy popcorn
2:12:22
or if a hundred people or fewer buy popcorn. Now we're ready to state our hypotheses. The null hypothesis in the test for independence is that the variables are independent and are not associated with each other. The alternative hypothesis states that the variables are not independent and are associated with each other. For the chi-squared test, it's important to construct a two-by-two table of counts to see how many observations fall under each category. Let's say we have data
2:12:52
from the spring, summer and fall popcorn sales. There were 275 days of data collected, including 83 days where it rained and 192 days where it did not. On 135 days, more than a hundred people bought popcorn, and on 140 days, fewer than a hundred people bought popcorn. You can then fill in each cell with the count of days for each category. You can then calculate the expected value for each cell in the two-by-two table using the following formula. The expected value for the cell
2:13:24
in the I'th row and J'th column is equal to the total of row I times the total of column J divided by the total count in the two-by-two table. The formula comes from the definition of independence where two events are independent if the probability of both occurring, probability of A and B is equal to the probability of A times the probability of B. The observed value is just the count for the cell in the I'th row and J'th column. You can then use the same formula as the goodness-of-fit test
2:14:00
to calculate the chi-squared statistic. As with the goodness-of-fit test, you can use the chi-squared statistic to calculate the P-value and determine if the two categorical variables are independent or not. Great work. We'll cover a couple of the assumptions of chi-squared tests and how to perform chi-squared tests in more depth in upcoming readings. The key here is connecting concepts you have learned about hypothesis testing to these chi-squared tests as well. You'll answer more questions
2:14:32
about categorical variables soon.
2:14:38
Knowing our data well helps us determine what we can do with the data. It helps us understand which tests we can run, which tests we can't run, and which tests we might be able to run if we transform the data a bit more. In prior videos, we showed the difference between continuous and categorical variables. We've spent most of this course so far talking about linear regression, which can estimate continuous variables of interest, but there is so much categorical data out there.
2:15:08
So far, we've learned about the chi-squared goodness-of-fit test and test of independence. Both of these tests examine the relationship between categorical variables. Now we'll focus on analysis of variance, or ANOVA, which helps examine the relationship between categorical variables and continuous variables. Analysis of variance, commonly called ANOVA, is a group of statistical techniques that test the difference of means between three or more groups.
2:15:39
If this sounds familiar, you might recall T-tests, which are a common statistical test. ANOVA is an extension of T-tests. While T-tests examine the difference of means between two groups, ANOVA can test means between several groups. Let's say you work at a botanical garden, and you're wondering if different species of butterflies have different lifespans. ANOVA testing can help in this situation. There are two main types of ANOVA tests, one-way and two-way.
2:16:11
One-way ANOVA testing compares the means of one continuous dependent variable in three or more groups. We'll use a categorical variable to represent the groupings. When using ANOVA, we have a null hypothesis and an alternative hypothesis. Let's say you're measuring the lifespans of three different butterfly species, monarch, morning cloak and swallow tail butterflies. Your colleagues suggest that the lifespans may be the same regardless of butterfly species.
2:16:39
Since the null hypothesis is that the lifespans are equal, a one-way ANOVA test is appropriate. This means that the lifespan of monarch butterflies is equal to the lifespan of morning cloak butterflies, which is also equal to the lifespan of swallow tail butterflies. We can write H-naught as mu-monarch equals mu-morning cloak equals mu-swallow tail. To generalize, we can use one-way ANOVA when the null hypothesis states that the means of each group are equal.
2:17:10
The alternative hypothesis would then be the lifespan of these three butterfly species are not all the same. This is a little difficult to express using math symbols. So you can just write not before the null hypothesis. So not mu-monarchs equals mu-morning cloak equals mu-swallow tail. Only one of the mean lifespan has to be different to reject the null hypothesis. We represent the alternative hypothesis as H-sub-one or H-one because you may run more complex tests in the future
2:17:43
where you are testing more than two hypotheses at once. One-way ANOVA is a great tool, but sometimes you might encounter situations in which there are two factors that are associated with the continuous dependent variable. At the botanical garden, let's say you wanna study if butterfly lifespan is related to the species of the butterfly and the size of the butterfly. Imagine the butterflies can be small, medium or large. Now the data is varying according to two factors, species and size.
2:18:16
Two-way ANOVA testing compares the means of one continuous dependent variable based on two categorical variables. You are now testing three null hypotheses and alternative hypotheses statements at once. The first null and alternative hypothesis pair is the same as before. They focus on the relationship between species and lifespan. The null hypothesis states that there is no difference in lifespans between the three butterfly species. The alternative hypothesis states that there is a difference
2:18:47
in lifespans between the three butterfly species. The next pair of hypotheses focuses on our second new categorical variable, size. The null hypothesis is that there is no difference in lifespans based on butterfly size. The alternative hypothesis is that there is a difference in lifespans based on butterfly size. The last pair of hypotheses test the interaction between the two variables. This concept of interaction may be familiar from our work on multiple regression. The null hypothesis states
2:19:15
that the effect of species on lifespan is independent of the effect of butterfly size and vice versa. The alternative hypothesis states that there is interaction effect between butterfly size and species on lifespan. Regression lets you understand how independent variables impact dependent variables. ANOVA allows you to zoom in on some of those relationships to tell a complete story by unpacking relationships in a pairwise fashion. Wonderful job. In this video, we reviewed the differences
2:19:47
between one-way and two-way ANOVA and were able to state the null and alternative hypotheses for each test in our butterfly scenario. Next, we'll learn how our computers and Python can help us run ANOVA tests.
2:20:03
At the core of all of our regression analyses and statistics is storytelling. We want to understand how the different variables are related. Although regression and ANOVA can help answer similar questions, one or the other might be more useful in specific cases. A regression analysis will help provide a holistic picture of if and by how much a number of different variables impact an outcome variable. On the other hand, ANOVA helps unpack pair-wise comparisons among subsets of those variables
2:20:37
to better understand the nuances among the elements that fueled the regression analysis. ANOVA can work like a magnifying glass, zooming in on specific parts of the regression story. Now let's use a subset of a dataset about diamonds to explore ANOVA in Python. You can load in the original dataset through the Seaborn library. We've already cleaned the data and transformed some variables. We have two variables, logarithm of the price and the color grade of each diamond.
2:21:11
Using Pandas, load in the CSV file. Now that the dataset is loaded in, examine the data. If you haven't already, import the Seaborn package as SNS. Use a box plot to determine how the price varies based on the color grade. There are a few different color grades represented, D, E, F, H and I. There seems to be some variation in the log of the price, but it's not clear if there is a difference based on color grade. So let's test it out using a one-way ANOVA.
2:21:47
You'll be using the Statsmodel's module. Start by creating a regression model using the OLS function and then fitting the model to the data using the fit method. This will allow you to use the ANOVA function to see if there is a statistically significant difference in price between the groups. Remember that you have to add the capital C and parenthesis around color in the OLS formula to indicate color is a categorical variable. Based on the results from the model, you can observe
2:22:17
that there is a statistically significant relationship between diamond color and price, but it's unclear if the price differs between the various colors. To learn more, you can run a one-way ANOVA test. You can create an ANOVA table using the Statsmodel's anova_lm function. The table will provide you with statistics about the color variable. Recall that a one-way ANOVA test compares three or more groups of one categorical independent variable based on one continuous dependent variable.
2:22:51
Let's state the null and alternative hypotheses. The null hypothesis is that there is no difference in price of diamond based on color grade. The alternative hypothesis is that there is a difference in price of diamond based on color grade. Note, there are different ANOVA types, one, two, and three, which you can read about in the documentation. The ANOVA table gives you the sum of squares and degrees of freedom for the color variables and its residuals, as well as the F-statistic
2:23:22
and P-value for the color variable. From the results, the P-value is very small, which means that you can reject the null hypothesis that the mean of the price is the same for all diamond color grades. Now let's add another categorical variable into the mix. Let's add in the cut of the diamond. There are three kinds of cuts to include, ideal, premium and very good. You can load in the dataset that has been cleaned. You'll do the same as before and first fit a regression model,
2:23:53
but the equation will address the possible interaction effects between cut and color. The colon indicates interaction between the color and cut categorical variables. Before you run the two-way ANOVA test, let's review the three hypothesis pairs you'll be testing. First up are the null and alternative hypotheses about diamond price based on color. Next are the hypotheses about diamond price based on cut. Lastly are the hypotheses about the interaction between color and cut on diamond price.
2:24:25
Now get the results of a two-way ANOVA test using the same anova_lm function. The table includes a row for each of the two categorical variables and a row for the interaction between cut and color. Since the P-value is very small for all three, you can reject all three null hypotheses. In conclusion, the logarithm of the price is not the same for different colors. Additionally, the logarithm of the price is not the same for different diamond cuts. Finally, there is an interaction effect
2:24:59
between the color and cut that impacts the price of the diamond.
2:25:03
Wonderful job. In this video, we reviewed the difference between one-way and two-way ANOVA, the null and alternative hypotheses, as well as how to code and interpret the results from Python. We've covered a lot so far, and in the following videos and readings, we'll continue exploring the power of ANOVA testing.
2:25:26
Previously, we talked about the effectiveness of ANOVA testing to understand continuous variables a bit more in relation to categorical variable of interest. To recap, ANOVA testing can be applied to the results we get from a linear regression. If we have a categorical independent variable, such as diamond color, and we're trying to estimate the price of a diamond, we can use an ANOVA test to determine if there is a statistically significant difference in price based on diamond color.
2:25:54
But as we stated before, the null hypothesis just states that the means are different. If we have three or more groups, we don't know which one is different or how many are different from each other. There might be cases where it's really important to know if one category in particular is different. For example, if you're building a rollercoaster, you'll be choosing among a couple of different material types based on their strengths. You really want to know
2:26:20
if and how much the materials differ in strength. In this case, an ANOVA post hoc test can be helpful. An ANOVA post hoc test performs a pairwise comparison between all available groups while controlling for the error rate. If you recall from learning about statistics, we have confidence intervals and P-values to quantify our uncertainty. There is always a small chance that we falsely reject the null hypothesis purely based on probability. Falsely rejecting the null hypothesis
2:26:49
is sometimes referred to as type I error. Typically, there's a 5% chance we've rejected the null hypothesis when it was actually true, but if we run a bunch of tests, all with a 5% chance that we're incorrectly rejecting the null hypothesis, the chance that we've made a mistake multiplies. So the odds that we've made at least one mistake increases very rapidly the more tests we perform. Post hoc ANOVA tests control for that increasing probability. One of the most common ANOVA post hoc tests
2:27:21
is the Tukey's HSD, honestly significantly different. After performing ANOVA test where you get statistically significant results, all you know is that at least one of the groups' means are different. Tukey's HSD tests will then compare all the pairs of groups and determine which pairs are different from one another, while controlling for the fact that you're running multiple hypothesis tests all at once. Now let's return to the one-way ANOVA diamond example.
2:27:50
Let's say you already have your packages imported from last time, and you've saved the dataset as a data frame called diamonds. Create the linear model again. You already know that there is a statistically significant relationship between color and price. Now run the one-way ANOVA just like before. Here, you can observe that the P-values are much smaller than 0.05, so the results are statistically significant, and therefore you can reject the null hypothesis
2:28:21
that the mean price is the same across all diamond colors. Now that you have significant results from an ANOVA test, you can run the Tukey's HSD post hoc test. First, import the Python function pairwise_tukeyshsd from SM. Next run the test. The endog variable indicates which variable is being compared across groups. The group's variable tells the function which variable holds the groups you're interested in, in this case color. The alpha variable tells the function the significance
2:28:54
or confidence level you are testing for. You set alpha to 0.05 as you are aiming for a 95% confidence level. You can access the results of Tukey's test in two ways. One way is to use the summary function. You can also use the print function. In both cases, you can observe all of the pairwise comparisons between each pair of diamond color grades. There is an adjusted P-value column, a column with the lower and upper bound of the confidence intervals, as well as a column letting you know
2:29:26
whether or not you can reject the null hypothesis. If the reject column reads false, then you cannot reject the null hypothesis. If the reject column reads true, then you can reject the null hypothesis. For example, the first row compares the mean diamond price between color grades D and E. The null hypothesis is that the diamond prices between the two color grades are the same, but the Tukey's HSD test informs us that you can reject the null hypothesis with the P-value of 0.001.
2:29:58
In conclusion, the diamond price is not the same for H and I-grade diamonds. Awesome job today. To recap, combining tests may seem like a complex task, but it's so powerful to have a wide set of tools to run analysis. We'll continue unpacking ANOVA and post hoc tests in the course materials. As always, revisit any prior videos and readings as needed. You can do this. Good luck. And I look forward to joining you again next time.
2:30:30
So far, we've learned some exciting ways to uncover stories about categorical variables. One of these tests is ANOVA or analysis of variance. ANOVA helps us learn more about the differences in continuous Y variable based on different groupings. Using ANOVA, we can determine whether groups are actually different from one another. You may recall that when we learned about simple linear regression, we could only take into account one independent variable
2:30:58
to explain the variance of one dependent variable. Then we learned about multiple regression, which allowed us to incorporate lots of different factors into our analysis. By working with simple linear regression, you build a strong foundation for understanding the mechanics of regression modeling. By working with multiple regression, you were able to vastly expand the kinds of questions you could explore. For example, using simple linear regression, you could explore the relationship
2:31:28
between the weather and ice coffee sales, but with multiple regression, you could then include temperature and distance to public transportation or other variables you think might be related to coffee sells. There's a similar relationship between ANOVA and ANCOVA, or analysis of covariance. Analysis of covariance, or ANCOVA, is a statistical technique that tests the difference of means between three or more groups while controlling for the effects of covariates.
2:31:58
Covariates are the variables that are not of direct interest to the question we are trying to address. By taking the covariates into account, we can better isolate the relationship between the categorical variable we are interested in and the Y variable. This allows us to draw more accurate conclusions about the relationships among the variables. For example, if examining ice coffee sales, ANCOVA allows you to analyze how the sales are different on workdays versus the weekend,
2:32:26
while controlling for the temperature of the day. Let's say we have a drop in sales one weekend. We could assume that no one buys coffee on weekends because they're not going to work, but perhaps it was especially cold that weekend. ANCOVA allows us to double check whether temperature was a factor. You might wonder why a data analytics professional would use ANCOVA when we already have linear regression analysis we can use. There are many similarities between ANCOVA and linear regression.
2:32:56
For example, both allow for continuous and categorical independent variables. Both focus on a continuous Y variable, and both center on understanding relationships between variables, but the use-cases depend on which variable we're most interested in understanding. With ANCOVA, while we are not focused on the covariates, we are including covariates to gain a clear understanding of the categorical variable. With regression, we might be interested in all of the independent variables
2:33:28
or in predicting the Y variable for unseen data. That was a lot. We've gone through one-way ANOVA, two-way ANOVA and ANCOVA, after covering various linear regression models. All of these are important concepts to be able to talk about as a data analytics professional, but don't feel like you have to remember everything we're going through verbatim. That would be really hard. When you're working on a new project, you'll be able to refresh your memory by revisiting this course,
2:33:58
doing a quick Google search or asking other people. I literally do this every day. For instance, I don't remember every equation off the top of my head, but what's important is knowing that there are assumptions and equations, and when I need them, I can either look at my past code or search for them. If you're anything like me, you'll get really good at fine tuning your searches. Going back to ANCOVA, we're dealing with hypothesis tests. So it's important to state
2:34:26
our null and alternative hypothesis before running a test. We'll cover how to run the test in Python later in this course. In this video, we'll focus on understanding what kinds of questions ANCOVA can help us answer. Let's say that you are working at a bookstore, and you're interested in the relationship between book genres and sales. It seems new books tend to get more attention because authors are traveling to promote their recent work. The good news is that
2:34:54
ANCOVA will let us control for publication year. Controlling for other variables is important so we don't draw conclusions that are not accurate. In this example, the categorical independent variable is book genre. The covariates is the year since the book was published. The continuous dependent variable is the number of books sold in the last month. The null hypothesis, or H-naught, is that book sales are equal for all genres regardless of the number of years since publication.
2:35:26
The alternative hypothesis, or H-one, is that book sales are not equal for all genres regardless of number of years since publication. Just like with ANOVA, you have Python and our computers available to run the test and the math, but you need to interpret the results. Typically, if the test yields a P-value of less than 0.05, you can reject the null hypothesis that all of the means were the same, even when controlling for the covariates. Great work.
2:35:57
We've reviewed a lot about the importance of ANCOVA, which you can add to your data analysis toolkit now. As always, remember that there are additional resources for you to keep exploring ANOVA and its variance. Good luck and until next time, keep exploring interesting questions and telling compelling stories.
2:36:19
Remember how far you've come on your data journey so far. You've built and extended a number of models and tests from their most basic form to include more variables and different kinds of data. As you've explored various models, you've encountered different questions you can answer and different use-cases for each model. You've made the shift from ANOVA to ANCOVA, much like you extended simple linear regression to multiple regression. You added more independent variables
2:36:49
to help isolate the effect of each X variable on the Y variable in question. In this video, we'll add more dependent variables to allow us to perform new and different kinds of comparative analyses. The two tests we'll introduce our MANOVA and MANCOVA. Based on the names, you might be able to guess how these relate to ANOVA and ANCOVA. MANOVA, or multivariate analysis of variance, is an extension of ANOVA that compares how two or more continuous outcome variables
2:37:23
vary according to categorical independent variables. Like ANOVA, the two most common versions of MANOVA involve one or two categorical independent variables and are referred to as a one-way and two-way MANOVA. The independent variable must be categorical, and the outcome variables must be continuous. Since we're still dealing with hypothesis testing, we need some hypotheses to test. Let's return to the bookstore example to generate and test new hypotheses about factors relating to book sales.
2:37:58
The one categorical variable will be book genre. The two continuous dependent variables could be the number of books sold per month and profits from the book sales. Let's say you're working with a one-way MANOVA test. In this case, the null hypothesis would be that the means of both continuous variables are equal for every book genre. So the number of books sold per month is the same for each book genre, and the profit from selling books is the same for each book genre.
2:38:30
The alternative hypothesis would be that the means of both continuous variables are not equal for every book genre. This indicates that the only two genres of books must differ for just one of the outcome variables we're examining. For example, perhaps the profit made from self-help books differs from the profit made from science-fiction books, or maybe the number of graphic novels sold per month differs from the number of historical fiction books sold per month.
2:38:59
In either case, you could reject the null hypothesis. MANOVA allows us to think of each data point as having a number of characteristics, which are the continuous Y variables that we want to understand based on one or two sets of groups we care about, the one or two categorical X variables. If, however, we're only interested in one categorical variable, and we want to control for another variable, we can use MANCOVA. MANCOVA, or multivariate analysis of covariance,
2:39:35
is an extension of ANCOVA and MANOVA that compares how two or more continuous outcome variables vary according to categorical independent variables while controlling for covariates. Let's say you're still interested in whether book genre is related to the number of books sold and the amount of profit, but you want to control for the popularity of the author. Then you could use MANCOVA. The categorical X variable is still book genre, but now you add a covariate,
2:40:10
which is the follower count an author has across social media platforms which you are controlling for. Then the two Y outcome variables remain the same, the number of books sold per month and the monthly profit. The null hypothesis is that book sells and monthly profits are equal for all genres regardless of the author's social media following. The alternative hypothesis, or H-one, is that book sales and monthly profits are not equal for all genres
2:40:39
regardless of the author's social media following. As you continue expanding your data toolkit, you'll encounter more test, tools and models that build off each other. Identifying the connections and use-cases is important as you continue throughout your career as a data professional. I'm excited for you to continue testing your hypotheses.
2:41:02
Everything we've covered so far has allowed us to explore categorical variables in different ways. By fully utilizing the assortment of tools available with regards to categorical variables, you'll be able to tell a wider variety of stories in your data. After all, we are scientists, and sometimes we need to try out a few tests in order to figure out the story that the data holds. Starting with just understanding one categorical variable, we examined the chi-squared goodness-of-fit test,
2:41:32
which determines whether a variable matches a theoretical distribution. Then the test of independence helped us determine whether two categorical variables were independent of one another. Moving beyond the chi-squared distribution, we discovered ANOVA, or analysis of variance. ANOVA forms the basis of all the other tests we covered in this section. One-way ANOVA is a powerful way to determine if there are differences in a continuous outcome variable between groups you're interested in.
2:42:02
Two-way ANOVA allows you to gain similar insights while incorporating another set of groups as well. ANCOVA built upon the idea of ANOVA by allowing us to control for another variable to isolate the effect of the groups from covariates. MANCOVA built upon ANOVA and ANCOVA to allow testing of multiple outcome variables of interest. As with any hypothesis testing, all of these tests had null and alternative hypotheses. Stating these hypotheses helps you to articulate
2:42:37
what you're trying to learn from running these tests. I encourage you to continue working through scenarios with hypothesis tests. The more questions you ask, the more the data will reveal. Keep in mind the data requirements of each test and that there is always room for error. With practice and time, you'll be telling impressive stories soon. I'll join you again next time.
2:43:04
Hello, it's great to be with you again. When we learned about linear regression, we started thinking about different kinds of questions that regression analysis could help us answer. For example, we could consider the factors that contribute to penguins' body mass or factors that affect click-throughs on a website. We're going to expand the kinds of questions we can ask. This time, we'll focus on modeling the probability that an event will occur. There are lots of problems that we could explore.
2:43:33
For example, what factors influence the odds that a customer buys from a company again? What impacts the likelihood a worker receives high performance ratings? What contributes to a user commenting or not commenting on a video? Logistic regression can help us in these situations. Remember that logistic regression is a technique that models a categorical dependent variable Y based on one or more independent variables X. The dependent variable can have two or more possible discrete values.
2:44:05
We'll focus mostly on binomial logistic regression, which models the probability of an observation falling into one of two categories based on one or more independent variables. We use a binary variable Y to indicate the category. For example, let's say that you're a data professional working for a basketball team, and you want to understand the probability that any given player on your team will score more than 10 points in a game. There are many variables you might want to consider.
2:44:35
For example, how well did the player perform last season? What's their average playing time? How many points did they score this season? This might seem like a multiple linear regression problem, but consider the outcome variable, whether or not the player will score more than 10 points in a basketball game. This is a binary outcome variable. Since there are only two possible outcomes, we can't draw a best-fit line in the same way we did for linear regression.
2:45:06
For example, if you plot playing times against whether or not the player will score more than 10 points in a game, your data will look like two flat lines, one at Y equals zero and one at Y equals one. This is very different from the linear relationship you observed in prior scenarios. Later in the program, we'll learn more about binomial logistic regression and review the stages of regression in PACE. We'll analyze our data by understanding the model assumptions to the best of our ability
2:45:37
and ensure we have a binary outcome variable. We'll construct our model and evaluate it using several different metrics. Then we'll execute and share the results with stakeholders and other team members. Logistic regression is a versatile and powerful model, and I'm excited to review it with you. So let's get started.
2:45:59
So far, we've learned that binomial logistic regression is a method for modeling the probability of a binary outcome, such as how likely it is that a player player will score more than 10 points in a basketball game, will a user comment or not? But just like any other statistical method, we have to make assumptions about the data to have confidence in the results. Now we'll discuss the main assumptions of binomial logistic regression and consider how to find the best logistic regression model
2:46:29
given a set of data. Logistic regression is a bit more complex than linear regression. So our goal is to understand the basics of how it works, not to understand every detail of the model. Recall that as we navigate the first two stages of PACE, we can figure out if a logistic regression model is the best choice to address the question we're working on. We consider the assumptions of the model as we analyze our approach. Some assumptions are similar to those of linear regression,
2:46:58
and some are different. The first and most important assumption of binomial logistic regression is the linearity assumption, which is a bit different from the linearity assumption for linear regression. In binomial logistic regression, the linearity assumption states that there should be a linear relationship between each X variable and the logit of the probability that Y equals one. The linearity assumption is the key assumption that explains how we can estimate a logistic regression model
2:47:28
that fits the data best. To understand logit, we must first define the odds. The odds of a given probability P is equal to P divided by one minus P. We can think of the equation as the probability of P occurring divided by the probability of P not occurring. For example, let's imagine that in a package of cookies with different flavors, you know that about 60% are chocolate. You'll represent this as 0.6. Then the probability of a cookie not being chocolate is 0.4 because one minus 0.6 is 0.4.
2:48:05
The odds a given cookie is chocolate is 0.6 divided by 0.4, which is 1.5. The logit, or log-odds function, is the logarithm of the odds of a given probability. so the logit of probability P is equal to the logarithm of P divided by one minus P. Logit is the most common link function used to linearly relate the X variables to the probability of Y. To translate this into less technical language, let's explore the basketball example further. If you're working for a basketball team
2:48:41
as a data practitioner, you'll wanna know the likelihood of your players scoring many points in a game rather than the other outcome that they don't score many points. By assuming that there is a linear relationship between the X variables and the logit of the probability that Y equals our outcome of interest, or one, you can then find some beta co-efficient that explains the data you've observed. You can write the logit of P in terms of the X variables.
2:49:10
So logit of P equals beta-zero plus beta-one times X-one plus beta-two times X-two, all the way up to beta-N times X-N, where N is the number of independent variables you are considering in your model. And like linear regression, we don't want just any set of beta coefficients. We want the best set of beta coefficients to make sure our model fits the data. In linear regression, we minimize the sum of squared residuals, which is a measure of error to figure out the best model.
2:49:44
In logistic regression, we'll often use maximum likelihood estimation to find the best logistic regression model. Maximum likelihood estimation, or MLE, is a technique for estimating the beta parameters that maximize the likelihood of the model producing the observed data. We can think of likelihood as the probability of observing the actual data given some set of beta parameters. To understand these definitions, we need to revisit the assumptions of binomial logistic regression.
2:50:18
Aside from linearity between each X variable and the logit of Y variable, we assume that the observations are independent. This assumption relates to how the data was collected. Because the observations are assumed to be independent, we can say that the probability of observing data point A and observing data point B is equal to the probability of observing A times the probability of observing B. Therefore, if you have N basketball players on your team, you can calculate the likelihood
2:50:53
of observing the outcome for each player and then multiply all of the likelihoods together to determine the likelihood of observing all of the sample data. The best logistic regression model estimates the set of beta coefficients that maximizes the likelihood of observing all of the sample data. Now that we have estimated how maximum likelihood estimation works, we'll consider two other assumptions of binomial logistic regression. We assume that there is little to no multicollinearity
2:51:25
between the independent variables. If we include multiple X variables, they should not be highly correlated with one another, just like with linear regression. Lastly, we assume that there are no extreme outliers in the dataset. Outliers are a complex topic in regression modeling and can be detected after the model is fit. Sometimes, it is appropriate to transform or adjust variables to maintain model validity. Other times, it can be appropriate to remove outlier data.
2:51:57
To recap, we've defined the main assumptions of logistic regression and how to fit the best logistic regression model to the data using MLE. Coming up, we'll explore how to build and evaluate a logistic regression model in Python using real data. Meet you there.
2:52:17
We've explored the kinds of questions logistic regression can help answer, reviewed the theory supporting logistic regression, and why it's a reliable and foundational data analysis tool. In this video, we'll build a logistic regression model in Python, which puts us in the construct phase of PACE. To illustrate how to code a binomial logistic regression in Python, we'll go through an example with data together. The dataset we'll use comes from a study about motion detection in older adults.
2:52:46
We've selected a subset of the data to work with. We've already loaded our dataset in a data frame called activity. You can call the describe function to get more statistical information about the variables. There are 494 rows of data based on the count row. You can also observe the mean, standard deviation, minimum value, maximum value, and the value at each quartile for every variable. Now examine the first few rows using the head function. There are two variables in the dataset.
2:53:17
The first variable measures acceleration in the vertical direction. The second variable indicates whether a person is lying down. You want to use logistic regression to predict whether the person is lying down or not. Load in scikit-learn's train test split and logistic regression functions. Now prepare the data for the Python functions to split the data into training set and a holdout dataset. Divide the activity data frame into the X variables and Y variables as you've done for other models.
2:53:47
Here, you'll just focus on one X variable, vertical acceleration. To follow good data practices, split the data into a training and holdout dataset using scikit-learns train test split method you imported earlier. You'll use the holdout or testing data later when you evaluate the model you're building now. We set the random state so that the results are repeatable, but you don't always have to set the random state. If you use a different random state, your results will differ.
2:54:14
Next, build the logistic regression model and save it as a variable called CLF. This is a name you'll encounter frequently. CLF stands for the classifier. Call on the logistic regression function to build a classifier. Then use the fit method of the logistic regression classifier and input the training dataset. Then call the classifier's coefficient and intercept attributes to get the parameter estimates. The coefficient or parameter estimate for beta-one is negative 0.118
2:54:46
rounded to the nearest thousandth, and the estimate of the intercept or beta-zero is 6.102
2:54:53
rounded to the nearest thousandth. If you want to create a plot of your model to visualize results, you can use the Seaborn package, so import Seaborn using SNS alias if you haven't already. Then call on the reg plot function. You have to tell the function which column the X variable is in, which is labeled ACC parenthesis vertical close parenthesis. Then specify Y variable column. Next, tell the function where the data is stored. Specify that you're interested in logistic regression
2:55:26
by setting the logistic argument is true. The plot displays a sharp S-shaped curve. The shaded region around the line indicates the confidence ban. You can observe two horizontal lines of data points, one where the person is lying down when the variable equals one. The other one is when the person is not lying down when the variable equals zero. We'll discuss how to interpret logistic regression results next. In this video, you told the computer to find the best way
2:55:54
to determine the likelihood of someone lying down based on vertical acceleration. You also began the work of interpreting the results and were able to plot the model. Coming up, we'll consider a variety of ways to evaluate the quality of the model and how to tell a meaningful story about the data.
2:56:15
You have built your first binomial logistic regression. You were able to calculate the parameter estimates and graph the logistic regression. In this video, we'll use the metrics and graphs produce in Python to evaluate how good your model actually was. To recap, you're working with the activity data that measured acceleration and whether or not a person was lying down. You previously saved the data into training and holdout datasets and the logistic regression model as a variable CLF.
2:56:45
Now input the holdout dataset into the predict method to get the predicted labels from the model. Save these predictions as a variable called y_pred. Note that MLE predicts a probability that an observation is a zero or one. The predict function from scikit-learn actually labels each observation with a zero or one. The predict function works by assuming a threshold of 0.5. So if MLE predicts a value greater than or equal to 0.5, the predict function will label that observation a one.
2:57:20
If MLE predicts a value less than 0.5, the predict function will label that observation a zero. The predict_proba function, on the other hand, will allow you to check what probability was predicted for each data point. Now that you have predictions about whether each observation is a zero or one, you can create a confusion matrix that will give you a quick overview of how well your model categorize each data point in the holdout dataset. A confusion matrix is a graphical representation
2:57:51
of how accurate a classifier is at predicting the labels for a categorical variable. The confusion matrix displays how many data points were accurately categorized by a classifier for each category. The other squares in the grid convey how many data points were misclassified. Now build your own confusion matrix, and together, we'll review each part of the plot. Import scikit-learn's metrics module to create your own confusion matrix. To graph the confusion matrix,
2:58:20
you can use two methods from the metrics module, confusion matrix and confusion matrix display. Use the confusion matrix to generate values for the matrix and save the output as a variable called cm.
2:58:35
Then save the graph as a variable called disp using the confusion matrix display method. Now show the confusion matrix by plotting the confusion matrix display function's output. The diagonal from upper left to lower right displays how many data points were accurately categorized by the classifier for each category. The other squares in the grid show how many data points were misclassified. On the axis of the confusion matrix, there are labels indicating the category of data point
2:59:04
as zero or one. Zero means the person was labeled as not lying down for that observation. Data professionals call a data point that is labeled zero a negative data point. One means a person was labeled as lying down for that observation. A data point that is labeled one is a positive data point. In the upper left corner of the confusion matrix, we have the count of observations that our classifier predicted as not lying down, and they actually were not lying down.
2:59:31
These are called true negatives. Moving to the lower right, we have our true positives. This is our count of observations that our classifier predicted as lying down, and they actually were lying down. Now let's consider where our model's predictions were wrong. In the upper right is the count of observations where our classifier predicted the person was lying down, but they were not lying down. These were our false positives. In the lower left is the count of observations
2:59:59
where our classifier predicted the person was not lying down but they were lying down. These were our false negatives. In a great model, we should observe a high proportion of true positives and true negatives and a low proportion of false positives and false negatives. In this video, you learned about the confusion matrix. This versatile tool can help you add depth to stories you tell about logistic regression models. In the following videos, we'll discuss even more ways
3:00:27
to evaluate the quality of a logistic regression model. I'll meet you there.
3:00:34
Earlier we covered confusion matrices which help to visually represent and quantify how well a logistic regression performed. In this video, we'll continue to work through the analyze and construct phase of PACE and discuss evaluation metrics for logistic regression. These metrics summarize the true positives, true negatives, false positives and false negatives presented by the confusion matrix. The three metrics we'll discuss are precision, recall and accuracy.
3:01:05
You can access the functions used in this video through scikit-learn's metrics module. And we'll continue to consider our activity data example. Precision measures the proportion of positive predictions that were true positives. Precision is equal to the number of true positives divided by the sum of true positives and false positives. In this case, precision would tell us among the people we predicted to be lying down, how many of them are actually lying down.
3:01:36
Scikit-learn's metrics library has a convenient function that does the math for us. We input the Y values in our holdout dataset and the Y values that our model predicted from the X values in our holdout dataset. Since the range of precision is zero to one, a score of 0.97 is great. Recall measures the proportion of positives that the model was able to identify correctly. Recall is equal to the number of true positives divided by the sum of true positives and false negatives.
3:02:11
Remember that false negatives are those that were lying down, but the model did not detect as lying down. Recall measures the proportion of people the model correctly identified as laying down out of the people who were actually lying down. We input the same data that we input to get precision, but using the recall score function from scikit-learn. We get a recall of about 0.98, and since recall also ranges from zero to one, this model performs pretty well. Lastly, we have accuracy,
3:02:45
which is the proportion of data points that were correctly categorized. Accuracy is equal to the sum of true positives and true negatives divided by the total number of predictions. In our activity example, accuracy will measure the proportion of people our model correctly identified as laying down and the proportion of people our model correctly identified as not laying down. We can use the accuracy score function from scikit-learn. In this case, our model achieved in accuracy score of 0.97,
3:03:20
but note that most of the time, precision, recall and accuracy are not all this high. This is to be expected. Two other common evaluation techniques that may be helpful when working with classifiers are the ROC curve and AUC. These concepts are related to thresholds, true positives and false positives. Although we use a threshold of 0.5 to generate our predictions, sometimes the threshold is determined based on the scenario. Notice that when we decrease the threshold, the true positive increases
3:03:57
because we are predicting more observations to be positive, but the false positive rate also increases. The model's true positive rate and false positive rate changes at every threshold. With an ideal model, there would exist a threshold where the true positive rate is high and the false positive rate is low. We can use an ROC curve and AUC to examine how the true positive and false positive rate change together at every threshold. Data professionals may use ROC curves and AUC
3:04:31
when comparing classification models. So we'll explore them later in this program. In this video, we covered a number of important metrics. These concepts are similar and related. Don't expect to grasp all the nuances of the content on your first pass. Allow yourself some extra time and practice to understand how best to apply these measurement tools to support the stories you're telling with data. Now we'll start constructing the kinds of stories you can share with logistic regression.
3:05:05
So far, we've discussed how to apply PACE to logistic regression. We've analyzed our data by understanding the model assumptions to the best of our ability, ensuring we have a binary outcome variable. We've constructed our model and created several evaluation metrics. Now we can begin executing and sharing the results. For example, just like linear regression, we have coefficients that the maximum likelihood estimation technique found that would best model the data,
3:05:34
but the interpretation is slightly different. Let's revisit the role of logit in logistic regression using our example. To recap, we wanted to understand how changes in vertical acceleration are related to whether a person is lying down or not. Our focus now is on what the beta coefficient means in our equation. In this case, a one unit increase in vertical acceleration is associated with a beta-one unit increase in the log odds of P, but if we exponentiate the log odds,
3:06:07
then we can determine how much the odds change as a percentage based on changes in vertical acceleration. So E to the power of beta-one is how many times the odds of P will increase or decrease for every one unit increase in vertical acceleration. We know our coefficient is negative 0.118 from the output, and now we can exponentiate the coefficient. E raised to the power of negative 0.118 is 0.89.
3:06:42
We can say that for every one unit increase in vertical acceleration, we expect that the odds the person is lying down decreases by 11%. This makes sense because if someone is moving faster in a vertical direction, they're probably not lying down. Given that we have found a strong predictor of whether a person is lying down or not, we can consider the larger picture. For example, classifying motion can help detect falls or suspicions of injury.
3:07:12
Our strong predictor can be a piece of a larger story about providing care to older adults. Next, let's examine how the interpretation would change in different scenarios. Let's try an example where the coefficient is positive. Imagine the coefficient is 0.25. Then we exponential and get E raised to the power of 0.25, which is 1.28 rounded to the nearest hundredth. Then we could say that for every one unit increase in X, we expect the odds of Y being one to increase by 28%.
3:07:49
Now imagine there are other factors in the model, such as acceleration in other directions. When there are multiple independent variables in a logistic regression model, just like in a linear regression model, we have to report coefficients while holding other variables constant. So we could say, for every one unit increase in X, holding other variables constant, we expect the odds of Y being one to increase by 28%. Moving beyond the coefficient, it is always helpful to state the P-value
3:08:21
and confidence interval to give additional information about how likely the result is just by chance.
3:08:28
Scikit-learn does not have a built-in way of getting P-values or confidence intervals, but Statsmodel does. This is a great example of how different tools and packages can give you different kinds of information. As a data practitioner, you have to choose your tools depending on your priorities. When presenting the results, it can be helpful to include a confusion matrix and some statistics on precision, recall, accuracy and/or ROC, AUC. However, depending on the situation,
3:08:59
you might want to include all of them or focus on a subset of metrics. Consider that certain industries or organizations may have preferred metrics given how they manage their modeling process. It's important to check with your team about this as you get started as a data analytics professional. Take for example the case of detecting spam text messages. Spam texts are unsolicited messages sent to many recipients. Only a small fraction of text messages that you receive are probably spam,
3:09:29
but if your goal is to accurately classify spam text messages so you don't give away private information or click on a bad link, then you're really only focusing on how well you can detect spam messages. In fact, in this case, accuracy is not a good metric. Let's say only 3% of text messages are spam. That means 97% are text messages you want to receive from friends and family or automated messages from service providers. So if a model predicts that 100% of messages are not spam,
3:10:02
that model would have a 97% accuracy rate, which seems great, but it's not that good in this context because the model will not have detected any spam at all, even though 3% of messages are, in fact, spam. In the case of spam messages, recall is probably a more meaningful metric. Recall will tell us the proportion of spam messages that the model was actually able to detect. Precision on the other hand, would tell us the proportion of data points labeled spam that were actually spam.
3:10:37
Precision essentially measures the quality of our spam labeling. There are other metrics that you can explore as well, such as AIC and BIC, which can help determine how good a model is while factoring in how complex the model is. As with many of the topics covered in this course, we've reviewed many concepts and now understand the terminology associated with the material. Work in the data space is dynamic. There is no one solution that works every time. Data professionals are constantly learning
3:11:12
and thinking about better approaches. We always have to contextualize the data, the problem and the available solutions. Next up, we'll focus on comparing some of the models and tests you've learned so far. Meet you there.
3:11:28
You've learned a lot about binomial logistic regression and the kinds of problems that can help you solve. Now you'll review some of the other techniques that you've learned and focus on the kinds of questions you might encounter in your work as a data professional. So far, you've learned about linear regression models, hypothesis testing, specifically chi-squared test and ANOVA variance, and logistic regression. You've also used the PACE framework, plan, analyze, construct and execute,
3:11:58
to help guide your work. You started with a plan phase to understand which needs were being prioritized. Based on the questions being asked and the data you had access to, you determine which models or tests would be most appropriate. Let's consider an example. Imagine you're a data professional at a recording studio. You have a team meeting, and the conversation is about the number of times each song is streamed. Some questions that come up include,
3:12:24
what factors influence the number of music streams, and how much does each factor influence the number streams? Since the number of music streams is the outcome variable and the number of streams is a continuous variable, you could consider linear regression or hypothesis testing, but because the question asks about how much each factor influences music streams, linear regression is a better model to answer the question. Remember that linear regression allows
3:12:53
for accessible interpretation of the coefficients, and R-squared to help explain which factors impact the outcome variable and by how much, but you want to make sure that the model assumptions are met to add validity to your conclusions and insights. Let's consider another example. Imagine you are working for a cafe. They're sampling different coffee beans from different countries and want to figure out which kinds of coffee beans sell better. The team has put together some projections
3:13:22
about how they expect the beans to sell, but they're curious if these beans sells are independent of their pastry sells. The country of origin of the beans is a known differentiator. The pastry sales would be more of the covariate. In this case, although the outcome variable, sales, is still continuous, the focus of the question is about comparing different groups, such as different kinds of coffee beans and different countries of origin. Therefore, you should focus more on hypothesis testing,
3:13:54
which can be a great way to conduct AB tests. The null hypothesis would be that the cafe sells approximately the same number of bags of each coffee bean type. The alternative hypothesis would be that the cafe does not sell the same number of bags of each coffee bean type. By conducting a series of tests, you can either accept or reject the null hypothesis with a particular P-value to help understand how well the model explains the trends, and the team will be able to better understand
3:14:26
which beans to order to sell more coffee at the cafe. Let's consider one last example in which you're working at a social media company, and you're interested in exploring why some posts do or do not go viral. You decide on a question, how can I predict whether a post will go viral? Since the outcome variable is binary, either the post go viral or does not, binomial logistic regression might be the first model you consider. The best way to determine if a logistic regression is the right choice
3:15:00
is to build and evaluate the model. Luckily, there are many metrics you can use, including P-value, confusion matrices, precision recall accuracy ROC, AUC, AIC and BIC. Choosing the best metrics will depend on the situation. When interpreting the coefficients for logistic regression models, remember to exponential them. Recall that when sharing results, logistic regression coefficients report in percentages how much a factor increases or decreases the likelihood of an outcome.
3:15:37
As a data professional, you will encounter many different questions that require a variety of approaches to address. By focusing on the fundamentals of what each model does well, you'll be able to find ways to put these tools into practice.
3:15:53
We've arrived at the end of the course, congratulations. You should be proud of how much you've learned and can apply to your work as a future data analytics professional. Before we finish, let's review what you've learned. We've defined binomial logistic regression as a foundational data technique used to model the probability of one or two outcomes occurring. In learning about binomial logistic regression, we focused on the importance of the logit function,
3:16:20
both as an assumption of logistic regression and in terms of how to interpret the results. You learned about maximum likelihood estimation, a common technique for estimating the best parameter for maximizing the likelihood of observing the data used to build a logistic regression model. Then you learned how to construct a logistic regression model in Python using the scikit-learn library. Lastly, you considered some evaluation metrics, including confusion matrices,
3:16:50
precision, recall and accuracy. Scikit-learn has many convenient functions that can help you evaluate and interpret a logistic regression, but sometimes you'll need other Python libraries or packages. For example, it can be helpful to go to Statsmodel, which you used when learning about linear regression, chi-squared and ANOVA testing. Python packages, like regression models, have their own strengths and weaknesses. Your ongoing experience as a data professional will help guide you to the tools
3:17:22
that's most appropriate for each situation. You also learned how to interpret logistic regression coefficients and what considerations to make when selecting metrics and plots based on the questions you're trying to answer. Lastly, you reviewed example scenarios to figure out why certain models or tests might be most appropriate in each situation. You've worked very hard up to this point. Celebrate all that you've learned and consider the many questions you've asked and answered
3:17:51
throughout this learning journey. You are well on your way, so keep up the good work.
3:17:58
- Hi, I'm Tiffany. It's great to be with you again. I'm here to talk more with you about your portfolio ideas and how you can use it in your job search. Completing the project for this course is a great way to present your knowledge and experience about data center tasks to potential employers. This time your project will demonstrate what you've learned about regression analysis. Ready to get started? As you learned in earlier courses, each portfolio project provides an opportunity
3:18:25
to complete tasks that demonstrate what you've learned in a course and create artifacts that you can add to your portfolio. While completing this portfolio project, you will also develop your interview skills. When interviewing for a job, I recommend that you discuss specific experiences that apply to data professional work. You can also use this portfolio project in these instances to show and explain your transferrable skills. Additionally, some employers might ask you
3:18:53
to assess a business scenario that requires multiple linear regression model. This project was designed to help you practice these skills. In order to complete the project, you'll be presented with details about a business case. Use the instructions to complete a new entry in your PACE strategy document and create a multiple linear regression model with ANOVA testing. You'll also use your PACE strategy document to continue recording your methods and considerations for approaching data projects.
3:19:22
Remember, you can go back to other videos and resources in this course if you'd like a refresher on any of the material. Ready? Let's go.
3:19:34
In this course, you learned about regression models, how they work, how to build them in Python, and how to interpret them. In the previous course, you practiced statistical skills to analyze and interpret data. Everything you've been learning will help you complete this new portfolio project. To complete this project, you'll be presented with a business scenario and a dataset that requires a multiple linear regression with the goal of solving a unique challenge.
3:20:02
Your PACE strategy document will help you to complete the project and encourage you to think like a data professional. As you learned in this and other courses, data professionals solve problems by finding and explaining relationships between variables and data. They tell stories based on these relationships that inform stakeholders to adjust their business strategies. As a data professional, the work that you do has meaningful impact and can shape the way organizations make informed decisions.
3:20:33
This part of the portfolio project is a great opportunity to demonstrate your technical skills, your ability to address complex problems, and your ability to communicate effectively about solutions. And remember, these skills take time to develop. The more portfolio projects you complete, the better prepared you'll be to face difficult job interview questions and handle challenges in your future role as a data professional.
3:20:59
Hello again. I'm back to check in on how you're doing with your progress so far. You've already accomplished so much. In addition to the AB test you built and analyzed, the tidy dataset you organized, and the data visualizations you've composed, you've also built a multiple linear regression model. Taking a moment to reflect on each of our portfolio projects creates an opportunity to really acknowledge everything you've learned, practiced and accomplished up to this point in the program.
3:21:29
These projects are collectively preparing you for future interviews as you begin to navigate the data career space. And your PACE strategy document has notes for you to refer to about your process, considerations, and steps for completing these artifacts in your portfolio. This is all stuff you'll want to discuss with potential employers and hiring managers during interviews. So far, you identified the importance of understanding assumptions for linear regression.
3:21:57
You practiced evaluating models, and you have been taught to recognize the importance of explaining your process when working with regression models. As you begin preparing for interviews, you'll want to think about questions you may be asked like, what kind of assumptions do we have for linear regression? What should we do if there are outliers? How do we determine whether outliers are influential points? How do you check multicollinearity, and what should be done if there is multicollinearity?
3:22:28
Of course, there will be additional questions for you to answer in your interview. You might even find yourself in a situation where potential employers and hiring managers ask you to describe how you would approach a project that you have little to no domain knowledge about.
3:22:45
This portfolio project requires you to assess a unique business scenario. In response, you built a multiple linear regression model with ANOVA testing. In addition to showcasing your modeling skills, you also demonstrated your ability to communicate findings effectively in your professional writeup, which included data visualizations, evaluation and interpretation of the model, and key business insights. Coming up, you're going to learn all about machine learning models.
3:23:12
Then you'll have an opportunity to solve a problem by building your own. You're doing a great job putting together your portfolio.
3:23:22
- Congratulations on reaching the end of this course. Incredible work. I hope you take a moment to celebrate and reflect on everything you've learned and accomplished throughout this course and the program overall. You began by applying the PACE framework, plan, analyze, construct and execute, to modeling relationships between variables, and learning about two fundamental models, linear and logistic regression. Then you learned how to check assumptions and to construct, evaluate and interpret
3:23:51
a simple linear regression model using examples. You were able to practice your Python and modeling skills on real data too. You extended your understanding of simple linear regression concepts, which you applied to multiple linear regression. With more variables, the interpretation became more complex, and the number of considerations increased. However, the scope of questions you could ask and answer also expanded. Next, you turned your focus to categorical variables with hypothesis testing.
3:24:21
You learned about chi-squared and ANOVA testing. This allowed you to ask questions about how groups differed from one another. Finally, you learned about binomial logistic regression, which focuses on the probability of a particular outcome. I hope you're proud of every line you've coded and every question you've asked. Being able to talk about and apply concepts we covered will serve you in whichever industry you work and the roles you take on the rest of your data analytics career.
3:24:50
You now have a much faster skillset that includes regression models, evaluation metrics and hypothesis testing. As you move through the rest of the program, you'll be able to draw connections using statistics and regression modeling. In the next course, you'll learn from Sushila, a fellow Googler, about the machine learning landscape. You'll explore supervised and unsupervised machine learning. I can't wait for you to meet Sushila, and for all she has to teach you.
3:25:16
You'll learn a lot of new and interesting techniques and solve problems with big data. I'm so grateful I could be here with you on your regression journey. I hope you feel more confident about your data professional knowledge and are prepared to keep developing your skills. You're ready to continue to the next step of your career as a data analytics professional.
3:25:40
(upbeat music)
3:25:50
(upbeat music continues)
Here to help you learn
Sign in to keep a private conversation with your course tutor.
Ask me anything about the material…
Sign in to chat