Skip to main content
Loading Explore guides…
Differentiability · Acedra
Home
Explore
Mathematics
Differentiability
Sign in to save
Chat
If YouTube asks you to sign in, open the video on YouTube.
Watch on YouTube
Reload player
Transcript
73 segments
Copy
Hide video
0:00
The goal of this video is to introduce the definition of differentiability, which is going to be a huge topic in Real Analysis 2. So what I would like for us to do in this first look at it is talk through this kind of grand definition of differentiability. Then I will tie it to the more familiar notion of differentiability for a function from R to R, so a function of the form f(x). I will then give you a specific example of a function,
0:24
I believe from R^2 to R^3, if memory serves, and we will prove that it is differentiable at any point in its domain. Then I will wrap up this first look at differentiability by giving you two other ways to characterize what’s being expressed here. If you are following analysis in a different text, what you have may be phrased differently than what I've put here, but I would like for you to be able
0:45
to connect the two because they are expressing the same idea—it’s the same equivalent notion. So, what is this notion of differentiability? Well, imagine we have a function f which is a map from some domain A in R^n to the codomain R^m. I have a note here that A will be assumed to be open—that’s not strictly required, but it is a convenience. I’ll explain why in a bit. We say the function f is differentiable at a point p in its domain if there is a
1:17
linear transformation from R^n to R^m so that this quotient, when you set it up and compute the limit as x approaches p, exists and equals zero. Okay, I’ll talk through that a little more slowly. The name of the linear transformation is going to be Df(p) for us. This is fairly common notation. D might be replaced with J in a different text, and that’s totally fine—there’s a reason for it. But this whole expression here represents
1:45
the name of the associated linear transformation. It’s like calling a function g; for us, the name of the function is Df. This is an associated transformation that we associate to f at p. Since it’s a linear transformation, it will have an associated matrix, which will be m × n. In the numerator, you can kind of read this two ways. When we look at the linear transformation here, you could read this as the linear transformation acting on x - p, or you could read it as the
2:16
associated matrix times this vector. As we talked about in our review of linear transformations, every linear transformation from R^n to R^m has an associated m × n matrix so that the action of that linear transformation can be characterized as matrix multiplication on the left. So here, this expression is kind of being used in two different ways. One, it’s the name of the linear transformation, but we also think of it as the name of
2:44
the associated matrix, just to blur the line between the two. Okay, there’s a lot going on here, and what I would like to do is talk through this limit very thoroughly. I think it’s a good moment to pause and just kind of reread this, look at this limit, and analyze it a bit. A warm-up exercise, for example, would be identifying the dimensions of all of the pieces in this expression. For example, what type of object is f(x)? What type of object is f(p)?
3:16
What kind of norm is this? Maybe go through and label with n’s and m’s the dimensions of all of the characters in this expression. I will come through afterwards and say exactly what’s going on here algebraically, but then I want to talk about the meaning. So, what is this limit telling us about f? All right, let’s start all the way on the left. We have the limit as x approaches p. If A is open, so if we make that assumption, then hopefully you can see this marker. Typically,
3:50
I stay away from blue, but, uh, so let’s say this is the open domain A and imagine p is, say, here. All points in open sets are interior points. So making this assumption means that we always know we can approach p, and we can approach it from any side or any direction that we’d like to. So if p is kind of well inside the domain so that it’s an interior point—which it will be if the domain is open—then, you know, if I start out here, that might be a bad place
4:19
to start. But whenever you take a limit, you imagine you’re getting close. So it’s silly to imagine what’s going on over here. Instead, I want to imagine that x is approaching p, and it doesn’t have to approach in a straight-line manner. You know, we can imagine a sequence of points in A that are kind of spiraling around. But if A is open, then we can do this, so we can approach p kind of however we want to.
4:42
Okay, so this limit makes sense. I will usually avoid an extra subscript where x is in A, but we imagine that this is happening in the domain. Then, looking at the numerator, we have f(x) and f(p). These are both outputs, so they are in R^m. Looking at the wrong one—they’re in R^m. So both of these are m by 1 vectors. Remember, when we say vectors, we look at elements in R^n and R^m; we imagine these as columns. So this is m by 1, m by 1.
5:15
Then I’ve already said that Df(p)—here you can read this as the linear transformation acting on this, or you can read it as a matrix times this, because those two actions are associated with each other. The matrix that we associate to this linear transformation is m by n, so you can throw that dimension in there. x - p, again, this is back in the domain, so this is in R^n. So this m by n matrix acting on this n by 1 vector produces a vector in R^m.
5:48
So that overall product is an m by 1 vector, which is good because it means that what we’ve set up here is dimensionally consistent. It’s a combination of vectors in R^m. So the norm that we’re computing in the numerator will return us a scalar number that we get by doing a norm on a vector in R^m. So it’s like an m-norm. The denominator is much easier to look at and eyeball. So here, we have two vectors back in R^n, in A,
6:15
back in R^n. The denominator has nothing to do with f, so we just have x - p. That’s a vector in R^n which is shrinking to zero, and then we’re computing the norm. So as x approaches p, regardless of f, the denominator goes to zero. So it will always be the case when you set up this expression that the denominator goes to zero, because if x is going to p, this is going to the norm of the origin in R^n.
6:43
Which takes us to the meaning of this expression. So, we talked through the algebra, but let’s look at the meaning. The denominator is going to zero, and yet if my function is differentiable, I’m claiming that this quotient—so like the ratio of the numerator to the denominator—is zero. Which means that this whole numerator must also be going to zero. And more. But let me, let me pause there—it also has to be going to zero.
7:11
So if your function has a chance of being differentiable, this better be an indeterminate form. I’m mentioning that because sometimes students will write this down, and then they’ll kind of say, like, “Oh, the numerator is going to zero.” Don’t stop as soon as you realize you’ve got something like 0 over 0. That’s expected for a differentiable function. But more has to happen, because if this numerator were going to zero at a rate perhaps,
7:35
uh, equivalent to the denominator, this quotient could go to one, for example—not zero. So in order for this quotient to go to zero, you need the numerator to go to zero and to go to zero faster than the denominator. Speaking kind of casually there. What does it mean, faster? It means that this quotient goes to zero, right? So there’s, like, the kind of casual way of thinking about it, but then we can actually quantify it by saying the ratio goes to zero.
8:00
Okay, so the numerator better vanish, and it needs to go to zero so fast that it drives this whole expression to zero. What is that telling us? Well, I’ve tried to color it in green here, if you can see this. So from here to here, this whole expression is what I will call the first-order approximation for f centered at p. I said I will call it that because often people call it the linear approximation
8:27
to f at p. I don’t like that phrase because it’s not technically linear—it’s technically affine. But let me write it out, and then we’ll talk through it. So this whole expression here, f at p plus the linear transformation Df(p) acting on (x - p), is the first-order approximation to this function f centered at p. p is fixed, so this is a function of x. Any linear transformation from R^n to R^m will send this expression to f(p) when x equals p.
9:08
So if I plug in x = p, this becomes the origin in R^n, which means this is the origin in R^m. All linear transformations send the origin to the origin. So when x equals p, we exactly match f(p).
9:24
So if you look at this numerator expression—just kind of focus on this for a moment—and plug in x = p, it should always vanish at p. The question is, what happens when x is near p? When x is near p, this is close to zero. Linear transformations are continuous, so we expect the result of this linear transformation acting on this vector—which is close to zero—to send us close to the origin in R^m. So it’s like we’ll take f(p) and just change it a little bit. The question is,
9:57
can you find a linear transformation so that that change closely matches the real behavior of f near p? Because if you can, then when we compare f(p) to, like, this kind of adjusted approximation of f, it would be such a good approximation that this numerator would vanish—and vanish faster, you know, speaking casually, than the denominator. So differentiability is all about finding a good first-order approximation for the function centered at p.
10:32
Then let's look at the second quantity. I'm calling it R for remainder. So it looks like left and right, but really this is L, kind of sticking with the tradition that this is called a linear approximation even if it’s not technically linear, and R for remainder. That is just f(x) minus Lp(x). This is exactly what we’re computing the norm of in the numerator. This is like the remainder or the error measuring the gap
10:59
between the function’s real behavior near p and the first-order approximation’s behavior near p. So again, f is differentiable at p if we can find a linear transformation so that this error is very small—so small that again, this quotient gets driven to zero. This is just the start of differentiability, so we are going to keep working with these notions, like, for a long time. Um, I want you to memorize this—like, have this expression really ready to go. You should get
11:34
to the point where you just kind of write it down automatically, like you were trying to write something else and you found yourself just writing this. Because when we start talking about differentiability for, say, specific examples, or when we start building up the theory of differentiability and proving theorems, we will almost always need to go back to this expression. All right, let’s see how that quotient we were just looking at gives us the familiar
12:01
difference quotient for single-variable calculus. Suppose here that f is known to be differentiable at p. That means I don’t need to prove the limit from the definition goes to zero—I know it goes to zero. So what that means is, for this function, I’m going to set up, say, zero on the left, and then we’ll simplify the quotient on the right. I’m going to begin with the definition of differentiability that we just had,
12:26
but for this special case of looking at a function from R to R. So this will be the limit as x goes to p. I will drop vector notation because x and p here are scalars. Then we will have the R^1 norm, which is just absolute value. I recommend writing absolute value when you recognize that that is actually what’s happening, because it helps you recognize, like, the dimension of the space that you’re working in. So here we are just working in R, so what were vector norms in both the
12:55
numerator and denominator for the kind of generalized version of that quotient is just going to be absolute value here. Okay, then we have f(x) minus the first-order approximation—that’s f(p) plus Df(p) acting on (x - p). Okay, so that’s our numerator. Recall that linear transformations from R to R are really just scalars. So let me just write above it that we actually expect this linear transformation acting on this number to turn into scaling this
13:31
number by something called m, where that’s just like the slope of a line. Okay, all right, that’s the numerator. The denominator, again, is the norm in R^1 on the real number line, so that’s just absolute value of (x - p). What I’m going to do now is, like, take this quotient of absolute values and turn it into the absolute value of the quotient. So I can, because we’re working in a very special situation where both the numerator and denominator are just scalar
14:05
quantities, we can write this as the limit as x goes to p of the absolute value of the quotient. And then I’m going to distribute this negative, and I will make this replacement: that whatever we’re looking at here, its associated matrix is a 1-by-1 matrix, which is going to be this number m. Okay, so let’s see: f(x) minus f(p)
14:30
minus m(x - p) is my numerator, divided by the denominator (x - p), like so. Okay, let’s break this up into two expressions. I’m going to group these together: f(x) minus f(p)
14:53
all over (x - p) minus m. Okay, probably worth taking a moment to pause there and try to arrive at your interpretation for what this is telling us, and then I will come back and summarize. We know this limit exists and equals zero, so I don’t need to prove this exists—I know it exists. If you look at this comparison that we’re now making, we are saying that the difference between this quotient and the number m vanishes as x approaches p. Since
15:23
m here is a fixed 1-by-1 matrix, this has no x-dependence. So in order for this to vanish, it must be the case that m equals the limit of this quotient as x approaches p. Thus, m equals the limit as x approaches p of (f(x) - f(p)) / (x - p), which is your familiar difference quotient for computing a derivative in Calc 1. And it also means that, while m is not a bad choice for the name of this number that we associate to the derivative,
15:58
you can actually give it a familiar expression, and that would be f'(p). f'(p) at some point in the domain is a number. We think of that as telling us the slope of the tangent line. But let’s now write down the first-order approximation and connect this to an actual slope. So Lp(x), our first-order approximation, will now be this: that is going to be f(p) plus—I could call it m, but maybe I’ll call it f'(p).
16:34
This might not look familiar written in this general form, but I hope it does. This is the equation of the line tangent to the graph of f(x) centered at p, f(p). So this is the very famous tangent line, or rather, written as the approximation, this is the first-order approximation—it’s like having a tangent line approximation.
17:03
Let me sketch a picture of what’s going on. Say this is p, and here’s my function f. So here’s the point (p, f(p)), like that. The equation of the line through this point, which lies tangent to the graph of this function, would have the form f(p) plus f'(p)(x - p). So here the slope is f'(p)—that’s the number that we famously compute with this difference quotient. And we are saying that this function f is differentiable because—it’s not clear in my
18:01
picture, but this tangent line is so tangent to the curve, it’s, it’s tangent, it’s like laying against it right at this point. So that—again, my picture is a little bit flawed—imagine that this is a little bit more tangent, you know, right at p they agree. That’s why I said the numerator is zero if x equals p. Right at p, they’re passing through the same point. So the difference between f and its first-order approximation,
18:27
that remainder error quantity, is literally zero at p. The question with differentiability is what happens if you move a little bit away from p. If my function—that’s a terrible picture—but imagine my function had some kind of corner, and I tried to draw a line. As soon as I move away from this point—if you can see this tiny picture I’ve drawn—my function is not well-approximated by that line. However,
18:51
if you have a differentiable function, like this one, if I move a little bit away from p, the line is still a good approximation. So good that when we set up and evaluate this limit, it would go to zero, even though the denominator goes to zero. So the comparison between the function and the tangent line is very good, driving this expression to zero. So this is visualizing differentiability for the special case of looking at a
19:15
function from R to R. You can see how this kind of generalized quotient that we set up for multivariable functions, which are potentially vector-valued, actually reduces down to the familiar form from calculus if we’re looking at a function from R to R. You may be wondering, can we do something like this for other types of functions from, say, R^n to R^m? And the answer is yes. We will start with this expression and get out
19:42
of it what this associated matrix would have to be, but it’s a little bit more work than what we did here. So we’ll have a whole video on just that computation. Okay, so I hope that this illustration helps you connect this expression, which might not have looked immediately familiar, to the familiar form. And then again, our idea of differentiability is to say that there’s a really good first-order approximation. From calculus, you’ve probably
20:08
seen that many times—that is this idea of a tangent line approximation for the function. What I want to do now is look at an example of a function from R^2 to R^3, where I will just give you the associated matrix. So I won’t say where it came from—although you might be able to deduce that. I won’t talk about where it came from; I just will say, use this associated matrix and this function in this definition and prove that it goes to zero,
20:37
thereby confirming that the function is differentiable. Okay, let me step away if you’re still copying this down, and then we will look at that example. The goal of this exercise is just to show you, specifically with an example, how that numerator in the definition of differentiability looks. In fact, we will evaluate the whole limit, but the focus is really on the numerator. So here, consider a function f, which is a map
21:03
from R^2 to R^3, whose coordinates are very nice polynomial expressions. So we have (x, y^2, xy). We would like to prove that this function is differentiable at any point (a, b) in its domain. So here, the point (a, b) is like what I was calling p in the definition. Then I’m telling you the associated matrix to try. You might know how I got to this associated matrix, but if you don’t, I actually won’t tell you right now—we will learn that in a couple of lessons. So for now,
21:33
let me just tell you that this is the correct matrix to put into the definition of differentiability with f to establish differentiability for this function. So this matrix is a 3-by-2 map, because of course our function is from R^2 to R^3. So the matrix is 3 by 2, with entries 1, 0, 0, 2b, b, a. Okay, so again, what I really want to focus on here is what the objects look like, what the dimensions of each of the objects look
21:58
like, just so that you don’t look at that limit and think that the terms are mysterious. It’s a big computation if I try to set up the whole limit at once. So instead of writing the entire quotient, let me focus first on that kind of error piece. So I’ll call it R_ab(x, y). This is what is inside the norm in the numerator for our limit. So this is the difference between f and its first-order approximation.
22:29
Here, that’s going to look like f(x, y) minus f(a, b) minus the matrix I’m just telling you to use, times (x - a, y - b). Like that. Now let me take this expression and actually put in the specific detail for this problem. f(x, y) is (x, y^2, xy), minus f(a, b), which is the same thing but with a and b in place of x and y: (a, b^2, ab), minus the matrix 1, 0, 2b, b, a, times (x - a, y - b). So let me go ahead and put that in this form. Okay, so vector in R^3,
23:33
vector in R^3, 3-by-2 matrix acting on a vector in R^2. This is what we will take the norm of when we want to conclude differentiability at the end, but let’s simplify this as much as we can. Let me start with these two: x - a in the first coordinate, y^2 - b^2 in the second. I hope it’s not too much if I go ahead and factor that using difference of squares. So y^2 - b^2 I will write as (y - b)(y + b). And then, xy - ab.
24:06
The reason I want to do that is partly because I’ve already worked through this and partly instinctive. Whenever you are working with particularly polynomial expressions, if x is going to a and y is going to b, you want to see terms in your computation of the form (x - a) and (y - b), because those are the pieces that we have control over. So it’s like, I want to see (y - b), and I recognize that I could see that by pairing up (y^2 - b^2) and doing the factoring.
24:33
Moving right along, let's do this matrix product. So, 1 times (x - a) plus 0 times (y - b) is going to be x - a. Then, 0 times (x - a) plus 2b times (y - b) gives us 2b(y - b). And then here, I think I’m just going to distribute everything—we have more going on here—so b(x - a) plus a(y - b).
25:08
Okay, I think I did this right. Let’s see. Cancel, cancel. When I subtract terms, on that second row I will recognize, as I do so, that I can factor (y - b) in front of this expression. So we’ll have (y - b) times (y + b - 2b). And then here, again, I’m going to distribute and combine these two terms: b(x - a) + a(y - b). As I do that, I notice that we can simplify further. So let’s factor out (y - b) here. That leaves (y - b)(y - b) in the second coordinate.
26:45
And now, the last term simplifies as well: (x - a) plus something involving (y - b). I’ll leave a factor of (y - b) in front of the whole thing so it’s easier to manage.
26:59
Okay, so at the end of the day, that error expression that we compute the size of in our definition of differentiability looks like this:
27:09
(y - b) times some vector in R^3. That’s the dimension that we expected for the numerator. I’m going to step aside in case you’re writing any of this down, and I’m blocking any of the details. When I come back, we will set up the quotient—meaning we will take the norm of this and divide it by the norm of (x - a, y - b)—and compute the limit of that quotient as (x, y) approaches (a, b). I think I can fit the whole limit right here—it’s actually pretty nice. Of course,
27:48
my handwriting is kind of bad, but I don’t always write this limit in this way. Here, I really want to emphasize that when we look at these kinds of vectors, always think of vectors as columns. So I might, in the future, write (x, y), you know, in traditional coordinate form. But here, I just want to emphasize where these two objects live and what they look like. Okay, this limit—let me just write in general—we’re taking the size of this error,
28:14
the difference between our function and its first-order approximation, and dividing that by the size of (x - a, y - b). Like that.
28:29
So what is the norm of this? Because that’s our desired numerator. It’s going to be the size of the scalar. So I will write the absolute value of (y - b) times the norm of this vector, which is each coordinate squared, added together, and then take the square root. So that will be 0^2 + (y - b)^2 + (x - a)^2, and then, in a similar way, the denominator is going to be the square root of (x - a)^2 + (y - b)^2.
29:20
So that’s the distance from (x, y) to (a, b). But look what we have. Terms are orderly here, but it’s the same radical, so this will cancel with that, leaving us with the limit of the absolute value of (y - b) as (x, y) approaches (a, b). So as y goes to b, this is going to go to 0, as desired. So, since the limit of this quotient went to 0, we actually learned two things in this exercise. One is that f is differentiable at the point (a, b)—so at any point in R^2—and
29:58
two, you learned that I didn’t mislead you with this choice of 3-by-2 matrix. So it’s like, if you can prove the limit of the quotient goes to 0, you get differentiability and confirmation of the matrix at the same time. Um, so here, since I didn’t tell you where this came from, you’re kind of testing it as part of this exercise. A couple of things I want you to take out of this. One is what this looks like dimensionally. So,
30:23
it’s a vector minus a vector, times a product of a matrix and a vector. That’s the kind of expression that we see in the numerator when we evaluate this limit to test differentiability. The other thing I want you to notice is now what it looks like to actually have the numerator driving the quotient to 0. So as x approaches a and y approaches b—so as (x, y) goes to (a, b)—both numerator and denominator go to 0. This should be in an indeterminate
30:46
form—it’s not surprising that it is. However, there’s a whole extra factor here driving the numerator to 0. So we have the same rate, times something extra in the numerator. So here we see the numerator is going to 0 faster than the denominator, leaving us with this term here. Had that canceled out—like, had that not been there—we would have had a limit of 1, not differentiable. Sorry. All right, let me go drink some water. I’ll let you finish copying this down,
31:12
and then, when we come back, we will just characterize the definition of differentiability with two other equivalent ways, in case you encounter a slightly different presentation of that limit in another text.
31:30
So, our first alternative way to characterize this information—we're just going to translate this limit equals zero into neighborhood expressions. So, what does it mean to say this limit? Like, what is the definition of limit? Going back a few lectures, we can say that this limit equals zero if, you know, for any epsilon greater than zero, there exists a delta greater than zero such that
32:00
whenever the distance from x to p is less than delta and actually greater than zero, it follows that the distance from this expression—well, this is a real number—I will write numerator divided by denominator. So the distance from the quotient to 0 is less than epsilon. I'm not done. Then what we're going to do is take this and recognize that, um, the absolute value of the quotient minus 0 is really just the quotient,
32:44
because this is non-negative. And so, okay, this is like really just the numerator norm divided by the denominator norm, less than epsilon. And then bring that denominator over
33:00
to say that the numerator is less than epsilon times the denominator. So, let me rewrite the sentence. Now it follows that whenever the distance from x to p is less than delta, we can say that f(x) minus f(p) minus Df(p) acting on (x - p)—this norm—is less than epsilon times the distance from x to p. The reason why this characterization is sometimes useful is that, you know, when we're writing proofs with differentiability, we might see this emerge. So we might be working
33:43
with neighborhoods—epsilon-delta neighborhoods—and we might just land on this. So, it's a nice way to, um, you know, recognize differentiability when you see it. And all it is, is just taking this notation—so what does this limit notation mean—and translating it into epsilons and deltas. You might wonder if we should replace this with delta, say that this is less than epsilon times delta. And typically, no. Typically, we actually want to see this on the right-hand side.
34:07
All right, so I've worked out the details for this. I'm going to step aside and just put up the kind of formal presentation of this characterization, just for a few moments if you'd like to read out what I wrote for it. And then I will come back and talk you through the second presentation of differentiability, which is actually fairly common.
34:34
Here’s the other form I wanted to mention because this is often how differentiability is presented in real analysis textbooks. So, this is the exact same idea as the definition of differentiability that we opened with—it just looks a little bit different when you first read it. So, what this says is that f is differentiable at p if there exists a linear transformation, which again I’m going to call Df(p) here, so that you can write f in a special way.
34:58
It looks like it’s first-order approximation plus some kind of error term. This is the same error that we’ve seen. In fact, it’s what I’m calling the expression inside the numerator norm. If you put that expression here, you would just have f(x) = f(x), so it’s a true statement. But what’s key here—what's capturing differentiability—is that as you compute the norm of this, which again just actually returns you the original numerator that we had,
35:27
compute that norm and put it in ratio with the size of the difference (x - p), that ratio vanishes as x approaches p. So this is actually the same limit that we’ve been looking at. This is all the same, um, idea of differentiability that we’ve seen. What’s nice about this is it really emphasizes that f is being closely approximated by this first-order approximation here. So it’s like f is basically that near p, plus just a little error,
35:54
where the error is so small that it vanishes rapidly as x approaches p. Okay, so again, this is just a way to reorganize the ideas in our definition of differentiability. Typically, I will work with that opening definition, but I might call on this or the form that we just saw a moment ago as needed. You know, sometimes when you're working through a proof, one form emerges more naturally than one of the other forms. So just have all of them ready to go.
36:20
And I hope you enjoyed this first look at differentiability.
Here to help you learn
Sign in to keep a private conversation with your course tutor.
Ask me anything about the material…
Sign in to chat