Skip to main content
Loading Explore guides…
Eigenvalues and Eigenvectors · Acedra
Home
Explore
Mathematics
Eigenvalues and Eigenvectors
Sign in to save
Chat
If YouTube asks you to sign in, open the video on YouTube.
Watch on YouTube
Reload player
Transcript
88 segments
Copy
Hide video
0:01
Namaste and welcome back to yet another lecture on linear algebra through geometry.
0:08
In the last lecture, we were looking at this transformation, say 2 by 2.
0:17
Which was taking a 2-component vector, giving me another 2-component vector. Right, this is what we were looking at. And we said that, okay, suppose you have this transformation, which is like a11, a12, a21, a22, and x1, x2.
0:48
is giving me, let us say, b1, b2.
0:54
So we ended the last lecture with the question: is it possible to have a vector, say u, which is not equal to the 0 vector, such that Au is equal to λu? We asked this question and then we wound up the last class. In today's class, we will basically look at the answer to this question, and that is exactly where we are heading: towards the ideas of eigenvalues and eigenvectors.
1:49
So, this class is on eigenvalues and eigenvectors.
1:58
Now let us get into the details of what we are trying to do here. Suppose I consider my matrix A to be a system.
2:10
Let us take this example. Let us consider this matrix A21, 12.
2:16
Right, this is what we have: this is going to be the system that I am going to have. Assume that A is equal to 2, 1, 1, 2, right? Now this matrix A is going to act on a two-component vector x1, x2, or a two-component vector x whose components are x1 and x2 to give a vector B, which is again a two-component vector whose components are B1 and B2.
2:41
Now what I am going to do is take some examples of X vectors and see what the B vectors are that I am going to get upon action by this matrix A or the system A. So I am going to look at some examples.
2:57
The first example that I am going to do is look at the X vector to be equal to the 1 0 vector.
3:06
The 1 0 vector is more so because that is a vector that is along the x1 direction. So, any vector along the x1 direction can be written as x1 times the vector (1, 0), and any vector along the e2 axis can be written as x2 times (0, 1). So, I am going to look at x1 into 10; what does this transformation to 1 do to 10?
3:35
We see that the resultant of this is the vector b, which is given by (2, 1).
3:44
Now let me take this vector x, which is equal to (0, 1), and that is going to give me this vector b, which is nothing but (1, 2).
3:58
What am I looking at? I am just looking at Ax equal to b, so how do we do this? This is basically saying Ax is equal to b, where the vectors x are all on the left-hand side and the vectors b are all on the right-hand side. So that is what I am trying to do.
4:16
Now then, I have this third guy which lets me take the x vector to be equal to minus one, let's say, minus 1, 2. So what does this do to minus 1, 2? I am going to have 2, 1, 1, 2 into minus 1, 2. So that is going to take me to this: 2 into minus 1 is minus 2, plus 1 into 2 is 0; the 1 into minus 1 is minus 1, plus 2 into 2 is 4. So 4 minus 1 is 3, so I am going to go to 3, 0, 3. So this is going to give me b is equal to 0, 3.
5:03
The fourth vector that I am going to look at is basically this input where I am going to take the x vector to be equal to 11, and I see that the output vector is basically b, which is equal to 33, right? The fourth vector that I am going to do is basically this guy, which is.
5:27
The x vector is equal to 1-1, and that is going to give me another b vector, which is 1-1.
5:37
Now, what does this all mean? We will get to understand what this all means when we see this pictorially; we get the following things.
5:50
So let me draw the axes, say E1 and E2; let us mark all the points: 0, 1, 2, 3, 4, 5, 6, 1, 2, 3, 4, 5, -1, -2, -3.
6:26
Minus 5, minus 1, minus 2, minus 3, minus 4, and so on.
6:33
Now the first vector that we took in the previous example was the (1, 0) vector, and the (1, 0) vector was along this direction. Let me take a moment.
6:52
The vector 1 0 was here, and that 1 0 vector, upon the action of 2 1 1 2, went here; right? That means the matrix A acted upon this vector 1 0 to give me another vector, which is 2 1. Similarly, I had this vector 0 1, which went to 1 2, so I have the 0 1 vector. The 0 1 vector was along this direction; this is the 0 1 vector, and that got thrown to 1 2, right?
7:35
And the third vector that we looked at was –12, which went to 03. Let us look at –12 minus 12; it was basically this vector, and that went to 03. 03 is along this direction; this is 03. Then we had this vector, which was 11; the 11 vector was here, and it went to
8:16
3 3, which is here, and last we took this last example, which was 1-1. 1-1 was this vector, and 1-1 upon action by 2 1 1 2 gives me 1-1 itself.
8:37
Now what is interesting here is that if you look at all those vector pairs that I have drawn here, other than the green one and this guy.
8:47
You see that the input direction was one thing and the output direction was something else. In a sense, this is like, you know, a cricketer playing shots around the ground, but there are times when the cricketer is going to play a shot which is a straight drive or probably the merrier shot, where he places the ball in the same direction but behind the stumps, right?
9:16
So there are, if you look at this red color one or the purple color one, all the blue pad arrows; these are all vectors which have, these are all the input vectors, in some direction, and the resultant, the transformed vector, was in a different direction. However, if you look at the vector which was in the green direction, or let us say the vector (11, 11), it was.
9:42
Transformed to the vector 33, and the vector 1-1 got transformed to 1-1 itself, which means we have certain vectors.
9:53
For the transformation A, such that they do not undergo a direction change, instead they just go through scaling, right? In this case, what had happened is that if I take this matrix 2112 and this transformation, we have this vector 11, and it gives me the vector 33, where the vector 33 was nothing but the scalar multiple of 11 itself.
10:22
Similarly, if you take this vector 1-1, 2, 1, 1, 2 into 1-1, that gave me 1-1 itself, which was essentially 1 times 1-1.
10:42
Right, that means we see that in these two cases alone, the direction of the resulting vector, the transformed vector, and the direction of the input vectors are the same, except that the effect of the matrix A, or the transformation A, on the vector (1, 1) or (1, -1) was just to scale the vector and not to change its direction. So, we are going to call such vectors eigenvectors and the scaling factor that is associated with these vectors eigenvalues.
11:15
So, let us write the definition: a vector u is called an eigenvector of the matrix A if Au is equal to lambda u for some scalar lambda, and lambda is called the eigenvalue.
12:09
In fact, we say that U is the eigenvector associated with the eigenvalue lambda, an eigenvector associated with the value eigenvalue lambda.
12:39
Yeah, that is associated with the eigenvalue lambda. Now the question is, how do we find these eigenvectors? How do we find the eigenvalues? What is the use of these eigenvalues and eigenvectors?
12:53
Where do we use these applications? These are things which we will actually look at at a later point in time, but right now what we will do is try to find a way to get the values of the eigenvalues and eigenvectors, okay? So that is what we are going to do.
13:13
So what is it that is given to us? We say that AX, or let us say AU, is equal to lambda U, right? Now, lambda is a scalar, which means we have A into U1 U2 is equal to, say, some scalar lambda into u1, u2, right? Lambda is a scalar, right? Okay. Now, how do we find the values of lambda, and how do we find u? So the question is, how do we find lambda and how do we find u? That is the question. So how are we going to do it?
14:15
So it is very easy; now what we do is look at this: we have Au is equal to lambda u.
14:28
A is basically this matrix: A11, A12, A21, A22; this is the matrix that we have.
14:41
So what am I going to do? I am going to take this matrix A into U. I am going to say this is equal to lambda times I times U, lambda into I times U, where I is the 2 by 2 identity matrix: 1 0 0 1. That is the identity transformation or the identity matrix that we have. So, which implies I can bring this to lambda I U to the left-hand side, so I am going to get A – lambda I times U gives me the 0 vector; this is what we are going to have; it is equal to the 0 vector.
15:19
Now, if you look at this expression carefully, let me call this equation 1. If you look at equation 1 carefully, this is an a-lambda i, which is yet another 2 by 2 system, structured like this: a11-lambda, a12, a21, and a22-lambda. What we want is for this to equal u1, u2 equal to 00.
15:53
Right, this is what we want. So this is basically equal to saying that some other system is equal to 0, which means you are looking for the solution to the homogeneous system of equations, which is nothing but A - lambda I times U is equal to 0. So what we want is u, the vector that is basically the solution to the homogeneous system. Let me put it this way: u is equal to (u1, u2), which is the solution to the homogeneous system A - λI times u is equal to 0; the right-hand side is the 0 vector.
16:47
Now we want this U to be a non-zero vector. Remember when we defined the eigenvector, we said that here again. We said that is it possible to have a vector U not equal to 0? That means the vector that we are looking for, U, must be a non-zero vector. So we want a non-zero solution to A minus lambda I times U equals 0; that means you are looking for a non-trivial solution to A minus lambda I times U equals 0.
17:21
The solution u to a-lambda i u is equal to 0; this is what we want.
17:35
We have already seen that if you want a 2 by 2 system of homogeneous equations to have a non-trivial solution, it corresponds to the determinant of the matrix being 0.
17:52
If the determinant of the matrix is 0, we are going to have either a non-trivial solution to the homogeneous system of equations. So let us write that recall that if the determinant of A 2 by 2 is equal to 0, then the homogeneous system of equations HSE, let me write that as HSE Ax is equal to 0, has non-trivial solutions.
18:34
Now, what does that mean? That simply means, here again, if you look at this, we want A minus lambda I times U to be equal to zero; this is what we want. We want our U to not be equal to the zero vector, and therefore, if that is the case, if you look at this equivalently, we are going to say that the determinant of A minus lambda I times U is equal to zero.
19:05
That means the matrix A minus lambda I is basically a singular matrix; the determinant of this must be equal to 0, and this is exactly what we refer to as the characteristic equation of A. The determinant of A – lambda I is equal to 0, which is the characteristic equation of A. So, if you take our 2 by 2 case, we had A11 – lambda A12 A21 A22 – lambda. This is what we had; this is the matrix A – lambda I, and we want the determinant of this to be equal to 0.
20:02
We want the determinant of this to be equal to 0, which corresponds to A11 minus lambda times A22 minus lambda minus A12 A21 is equal to 0.
20:16
We see that it is a quadratic equation that we have: we have lambda squared minus lambda times A11 plus A22.
20:27
Plus a11 a22 minus a21 a12 is equal to 0; this is what we have. The roots of this equation, which is basically the characteristic equation, give the eigenvalues; these eigenvalues are the scalars.
20:56
It is interesting to see that if you have a quadratic equation, the coefficient of lambda is basically the sum of the roots, and the constant term that you are going to have is the product of the roots; in this case, the product of the roots is basically the constant term, which corresponds to the determinant of A, and this corresponds to the sum of the diagonal elements, which is basically called the trace. Trace.
21:24
So what we are going to say is that we are going to make a statement here saying that the product of the eigenvalues is equal to the determinant of A, and the sum of the eigenvalues is equal to the eigenvalues of A.
21:53
It is equal to the trace of A. So, what we have now is that we will have 2 roots here, namely lambda1 and lambda2, which are the 2 roots of the characteristic equation of A. So, lambda1 is one eigenvalue and lambda2 is the second eigenvalue.
22:34
Now what we need to do is substitute this value to find the eigenvectors; we get A – lambda 1 times I into, let us say, U is equal to the 0 vector. This is one equation; the second equation is A – lambda 2 I into, let us say, V is equal to the 0 vector. So, U corresponds to the eigenvector associated with lambda 1.
23:05
The eigenvector associated with lambda 1 and V is the eigenvector associated with lambda 2.
23:33
So we can solve the homogeneous system of equations; these two are homogeneous systems of equations that can be solved to find the values of u and v, right? So now let us take our simple example where we had 2, 1, 1, 2, and we wanted to look at, let us say, x1, let us say, okay. So we wanted to find out the eigenvalues of this matrix, which is equal to 2, 1, 1, 2. So, we can do the characteristic equation here; the characteristic equation is.
24:11
The determinant of 2 – lambda 1 1 2 – lambda is equal to 0. So I am going to have 2 – λ (the whole square) – 1 is equal to 0; we are going to get λ squared – 4λ plus 3 is equal to 0. We are going to have λ equal to 3 and λ equal to 1. λ₁ is equal to 3 and λ₂ is equal to 1. This is exactly what we had seen before. in the in this. So, if you look here, we have seen that this year we had 3 and 1, and that is exactly what we see here.
24:51
Lambda 1 is equal to 3, lambda 2 is equal to 1; you can easily verify that the product of lambda 1 and lambda 2 is equal to 3 times 1, which is 3, which is nothing but the determinant of the matrix 2 1 1 2, and lambda 1 plus lambda 2 is basically 3 plus 1, which is equal to 4, which is nothing but the sum of the diagonal elements of a sum of diagrams.
25:29
The sum of the diagonal elements of A. Now we can solve the system of equations to find out the eigenvectors we have for lambda 1, which is equal to 3: we have 2, minus 3, 1, 1, 2, minus 3. Let us say u1, u2 is equal to 0, 0.
25:52
We have minus u1 plus u2 is equal to 0; you have u1 minus u1 minus u2 is equal to 0, which means I am going to get u1 is equal to u2. So the vector of the eigenvector is basically u1, u2, which is the same as u1 since u1 is equal to u2. I am going to write it as u1, u1, which I will write as u1 times (1, 1). That means you can put any scalar here, which is a multiple scalar of (1, 1). That means (1, 1) is basically the eigenvector that we are going to have, right?
26:45
So, the eigenvector corresponding to lambda 1 is equal to 3 is (1, 1); similarly, we can find that for lambda 2, which is equal to 1, the eigenvector is (1, -1).
27:08
There are a few questions that we want to answer here. So we know how to find the eigenvalues and the eigenvectors; now the questions that we have are: number 1, is it always possible that for a 2 by 2 matrix A we get real eigenvalues? This is the first question that we want to answer.
27:55
It is a very important question because we see that the characteristic equation of a 2 by 2 matrix is basically, let us say, lambda squared minus lambda times some trace plus the determinant of A is equal to 0.
28:19
Now this is a quadratic equation where both roots can be complex, which means the roots will appear as complex conjugate pairs, in which case we will not have eigenvalues that are real at all. So, which means no real eigenvalues implies that the eigenvectors are also not real, so no real eigenvalues and therefore no real eigenvectors. Now, what does that mean?
29:17
That simply means there are no such transformations; there is no fixed direction, which means any vector you give this transformation is going to be thrown in a different direction. Where do we see an example? The best example that we want to look at is basically the matrix R theta, which rotates every vector by an angle theta; of course, theta can be 360, 0, or any other angle that would correspond to the same direction.
30:09
So, what we will do is take a simple example in this case where I am going to look at the rotation by 90 degrees, R 90 degrees.
30:22
So we remember that R theta was cos theta, sin theta, minus sin theta, and cos theta.
30:35
So if you look at R theta now, R theta is cos 90, theta is 90, so let me mark this as 90 degrees. Theta is 90 degrees, so I am going to have 0, 1, -1, 0.
30:51
Now, if you look at the characteristic equation, the determinant of the matrix 0 – lambda, 1, 1, 0 – lambda is equal to 0. We have lambda squared plus 1 is equal to 0, which implies lambda squared is equal to minus 1, and lambda is equal to plus or minus i; that means this does not have a fixed rotation, a fixed direction at all; that means this rotation matrix will not have.
31:26
Real eigenvalues and hence real eigenvectors. So the second question that we ask is: what happens if the roots of the characteristic equation of a 2 by 2 matrix are repeated? What does that mean? We have this characteristic equation, which is lambda squared – lambda times the sum of the roots, meaning the sum of the diagonal plus the determinant of A is equal to 0. In this case, we are going to have lambda 1 equal to lambda 2 equal to lambda itself.
32:26
Now, if this happens in 2D, just so, if this happens, if eigenvalues are repeated for a 2 by 2 matrix A, we have only one eigenvector; that means we have only one fixed direction.
33:23
Now, where do we see this? We look at this example where I have A to be equal to 1, 1, 0, 1.
33:31
Now the characteristic equation in this case is the determinant of 1 – lambda, 1, 0, 1 – lambda, which is equal to 0. That means I am going to have (1 – lambda) squared is equal to 0, which means lambda is equal to 1 twice.
33:56
In this case, we will just get one eigenvector, only one eigenvector. So, because the roots are repeated, which in this case are especially A2 by 2 in a 2 by 2 case, if the eigenvalues are repeated, we just get one eigenvector, which we can very clearly figure out. So, let us figure out what the eigenvalue and eigenvector corresponding to lambda equal to 1 are. So, we wanted A minus lambda I times U to be equal to 0.
34:25
So we have 1-1, 0, 1, 1-1, u1, u2 equal to 0, 0. Let me write this: so we get 1-1 was 0, 1-1 was 0, this one was there, and that was 0 into u1, u2 equal to 0, 0. Excuse me.
34:55
So here we see that the first equation tells me that 0u1 plus u2 is equal to 0, which implies u2 is definitely 0, and since it says 0u1 is equal to 0, u1 can take any value; u1 can take any value, right? Therefore, the eigenvector u1, u2 is equal to, let us say, some k, 0, which is k into 10 for k real; that is the only direction that we have. And what is this matrix that we looked at? This is basically the shear matrix we have.
35:51
This is the shear matrix that we had, which has only one direction, or that means only one eigen direction. In all other cases, when you have two distinct eigenvalues, you will have two distinct eigenvectors; you will have two different directions, basically two eigenvectors. What is interesting is that there are many applications of eigenvalues and eigenvectors; one is to compute the powers of.
36:21
The matrix A encompasses all these things, which we will actually look at when we go to the vector space and examine invariant vector spaces and related concepts. There we will look at this diagonalization of a matrix and other things, but the most important thing that we want to emphasize here is the fact that for a transformation, let us say a 2 by 2 transformation A.
36:49
We can have two fixed directions; you can have only one fixed direction, or you do not have any fixed direction at all. That means you may have two eigenvectors, you may have only one eigenvector, or you do not have a real eigenvector at all. So, that is the conclusion that we have, and it is, in fact, more interesting. We will just look at this particular matrix, which we started working with: 2, 1, 1, 2.
37:14
Where we figured out that the eigenvalues are λ₁ = 3 and the corresponding eigenvector was (1, 1) and λ₂ = 1 and the corresponding eigenvector was (1, -1). Now it is interesting to see that the dot product of 11 with 1-1 gives me 0, which means this implies there is something very important happening, a property which we will just mention here but prove at a later stage, where we say that if A is a real symmetric matrix.
38:11
Let me write this as A2 by 2, which is a real symmetric matrix with distinct eigenvalues; the eigenvectors corresponding to them are orthogonal to each other.
39:00
So, that is a very interesting property that we have: if you have a 2 by 2 real symmetric matrix, now later we can generalize it to n by n. I mean, you can look at an n by n matrix; if it is a real symmetric matrix with distinct eigenvalues, then the corresponding eigenvectors will be orthogonal to each other. That is a very interesting property of real symmetric matrices with distinct eigenvalues.
39:24
And we will also see that the eigenvectors would turn out to be a basis; we will see the idea of basis, then vector spaces, dimension, basis, and all those things may be a couple of modules later. As of now, what we have seen is that we have looked at a lot of things in two dimensions; we looked at the ideas of vectors and points, combining vectors, then we looked at transformations on vectors, and we looked at solving systems of equations.
39:53
We tried to look at the system of equations from the column perspective, then we said that we talked about unique solutions, no solution, and infinitely many solution cases, which we looked at again from the column picture, and then we looked at the idea of transformations that result in just scaling.
40:14
That means suppose you have a matrix A and you have vectors that just get scaled by it. The direction does not change; we call them eigenvectors, and we say that the scaling factor is called the eigenvalue. We looked at a 2 by 2 matrix; you may have 2 eigendirections, or you may have only 1 eigenvector, or you may not have a real eigenvector at all.
40:44
So, these are the things that we have seen. And at this point in time, we have almost covered everything that needs to be covered as far as 2 by 2 systems are concerned. From the next module onwards, we will get into the idea of three dimensions; we will look at vectors in 3D, we will interpret the ideas of the determinant of a 3 by 3 matrix, we will examine what it means to say there are just two linearly independent vectors in 3D, and we will explore the new idea of the cross product.
41:16
These are the topics that we will be looking at in the next module. Thank you.
Here to help you learn
Sign in to keep a private conversation with your course tutor.
Ask me anything about the material…
Sign in to chat