Skip to main content
Loading Explore guides…
Eigenvectors & Eigenvalues · Acedra
Home
Explore
Mathematics
Eigenvectors & Eigenvalues
Sign in to save
Chat
If YouTube asks you to sign in, open the video on YouTube.
Watch on YouTube
Reload player
Transcript
181 segments
Copy
Hide video
0:13
Hi everyone, and welcome to the next instalment of Linear Algebra 2.
0:26
So today, we want to take a slightly different perspective on things and look at eigenvectors and eigenvalues.
0:47
So one of the key aims in this course is to try and find efficient ways to study linear transformations. And we think of linear transformations as things that are
1:00
some skewing and stretching and rescaling of
1:07
standard vector spaces or Euclidean space. Let's try and make that a bit straighter,
1:15
so this gets mapped to here. And in general these are represented by some matrix, when we're working with respect to some basis. And there's certain very simple to understand transformations, are just things where we stretch each coordinate by a fixed number. So these are the diagonal matrices
1:52
or
1:55
stretching each coordinate
2:03
by a fixed amount. So somehow these are the simplest linear transformations to understand.
2:16
But whether it looks like stretching each coordinate by a fixed amount depends on the choice of basis. If you're stretching just the, say z direction in the x, y, z plane by a factor of two,
2:44
this is very easy to understand if you're working in x, y, z coordinates. But if you had some different basis that didn't have the z direction as a basis element, then this would have a rather more complicated matrix representation. And so there's an actual question,
3:07
which linear transformations
3:16
correspond to these simple transformations,
3:26
simple matrices with respect to some basis?
3:39
It might be that we have some linear transformation and we are given a matrix representation for it and it looks very complicated. But if we just change basis to a nice basis, then it would become a diagonal matrix, and actually would be therefore very simple to understand transformation, because it's just stretching different vectors by different fixed quantities.
4:03
And that's the whole point of looking at eigenvectors and eigenvalues. It's trying to choose a nice basis so we can understand much more simply certain transformations.
4:20
So, that's maybe a bit of, again, geometric style heuristic motivation. So, but now let's move on to some definitions. So let T be a linear transformation
4:46
of a vector space V, and I'm thinking of this as being over the reals.
4:58
Then we have the following definitions. A vector
5:04
v in V is called an eigenvector
5:13
of the linear transformation T
5:17
if T times v is just some simple scaled version of the vector v. So T v is lambda times v for some constant scalar lambda, which is just a real number.
5:35
And correspondingly, lambda, that is some real number, is an eigenvector
5:48
of T if T v equals lambda v for
5:59
some vector v in our vector space. And, ok, this would hold always when v is just the trivial vector zero, so it has to be some nonzero vector in the vector space. This is our definition of eigenvector and eigenvalue.
6:19
Sorry, lambda's called an eigenvalue of T if it's the constant corresponding to an eigenvector. And maybe our first proposition
6:45
is going to be starting to get some properties of
6:49
eigenvalues of T and how they correspond to properties of determinants. So
6:59
lambda is an eigenvalue of T
7:08
is equivalent to the kernel of the linear transformation T minus lambda times the identity not being equal to just the trivial set. So this linear transformation doesn't have trivial nullity. So here I recall i is defined to be the identity
7:38
transformation, which has just the identity matrix as its matrix representation for any basis.
7:48
So the proof for this is quite straightforward, but it's allowing us to start getting a feeling for why eigenvalues and eigenvectors crop up. So lambda is an eigenvalue of T,
8:07
well, we, just repeating the definition, this means that T times v is equal to lambda v for some v which is nonzero.
8:25
But lambda v is what v goes to under lambda times the identity. So that means that, taking it over to the other side, T minus lambda i, if I apply that transformation to v, I do get a zero vector
8:47
for some nonzero v.
8:50
But that means that this nonzero vector v is in the kernel of the linear transformation.
8:58
And so I have nontrivial nullity and the kernel of T minus lambda i
9:07
is not just
9:12
the zero vector – we have at least v in this kernel.
9:18
So that completes the proof of the proposition. So that's
9:27
maybe a first key property of eigenvalues, that it corresponds to the kernel of T minus lambda i being rather larger than we would typically expect.
9:42
And then as a corollary, we can list actually several different equivalent
9:53
definitions for lambda being an eigenvalue. So,
10:01
the following are equivalent.
10:07
So statement a is that lambda is an eigenvalue of the linear transformation T.
10:21
Statement b is what we just had above, the kernel of T minus lambda i is nontrivial.
10:36
Statement c is that the linear transformation T minus lambda i is not invertible. And statement d is that the determinant of T minus lambda i is equal to zero as a linear map.
11:02
And the proof of this is actually exceptionally straightforward, because we've basically done all the hard work already.
11:09
So a is equivalent to b is the proposition above.
11:25
That c is equivalent to d
11:31
is just saying that the determinant being 0 is equivalent to a linear transformation not being invertible. And that was what we established last lecture.
11:52
So we know that a is equivalent to b, we know that c is equivalent to d, we just need to link them up. And b is equivalent to c is the only thing where we have to do
12:04
any work. And here, we just use the Rank-Nullity Theorem.
12:08
So by the Rank-Nullity Theorem,
12:17
so this is one of the important results in Linear Algebra 1, saying that the
12:26
size of the image and— the dimension of the image plus the dimension of the kernel has to add up to the dimension of the ambient space.
12:37
That certainly means that if you're invertible, the dimension of the image has to be everything, so the dimension of the kernel has to be zero.
12:47
A is invertible if and only if the kernel of A is not just the zero vector. But that gives precisely what we wanted, the equivalence between b and c. So that completes the proof of the corollary. So we've already therefore got four different statements that are all equivalent to lambda being an eigenvector.
13:24
I'd like to just extend this notion of being an eigenvector a little bit further by introducing a definition,
13:36
which is that of the characteristic polynomial now.
13:42
So another definition. The characteristic
13:53
polynomial
13:59
of, well, let's start off with a matrix A,
14:08
is, so I'll write it as, so it's a function which I'm writing chi_A of a parameter t.
14:18
And it's just the determinant of A minus t times the identity matrix.
14:28
And similarly we define
14:34
the characteristic
14:40
polynomial of a linear transformation T
14:54
is chi_T of t which is
15:02
chi_A of t for
15:13
any choice of matrix A representing T with respect to some basis.
15:27
So whenever we have one of these definitions of defining something for a linear map, we normally define it in terms of what we do with the matrix, because that's how we actually calculate things. But then we need to check that this doesn't depend on the choice of basis, because the matrix representing the linear transformation
15:46
does depend on the choice of basis. So note, chi_T of t is well-defined:
16:04
if A and B represent T with respect to different bases,
16:21
then we know that B is P inverse A P for some change of basis matrix P.
16:31
So the determinant of B minus t times I_n
16:40
is equal to the determinant of P inverse A P minus t times I_n. But then the trick is to notice that this is equal to the determinant of P inverse A minus t times I_n times P because we can just multiply this out and we know that P inverse I_n times P is equal to I_n. And this is,
17:14
by multiplicativity of the determinant, equal to the determinant of A minus t times I_n. So the characteristic polynomial doesn't depend on
17:28
the choice of basis because, actually always the same.
17:40
OK, so we've now introduced this characteristic polynomial, and it shouldn't come as a surprise to you that the characteristic polynomial is very closely related to the concept of eigenvalues.
17:56
So maybe we'll start that off with a theorem. Let T be just some linear transformation. Then lambda is an eigenvalue
18:24
of T is the same thing as saying that lambda is a root of the characteristic polynomial
18:52
chi_T of t. OK.
19:00
So the proof of this is, if lambda is— maybe I'll say, let A
19:16
represent T with respect to some basis. So A is just going to be a matrix representation. If lambda is an eigenvalue
19:35
of T, then we know that zero is equal to the determinant to T minus lambda i, where i is the identity transformation.
19:52
But this is equal to just this with respect to any basis. So it's A minus, I_n.
20:04
But this is precisely equal
20:11
to the characteristic polynomial evaluated at lambda.
20:24
If instead,
20:37
then we know that the determinant— if instead the characteristic polynomial has a root at lambda, then we know that the determinant of A minus I_n is equal to zero. And by the corollary above,
20:59
this means that lambda is an eigenvalue.
21:09
So, again, we've basically done all the work already,
21:13
and we see that this characteristic polynomial picks out the different eigenvalues of T as the points when the characteristic polynomial vanishes.
21:26
So this has all been slightly algebraic and abstract so far.
21:31
But again, we should be trying to think of this maybe geometrically. So let's just have an example to work through and think and calculate some of these quantities. So let V be R cubed and T be the linear transformation which is just
21:56
rotating by an angle theta around the z axis.
22:11
So now we can try and think about what vectors v are scaled by T and what vectors v aren't scaled by T. So if v lies on the z axis,
22:33
then T doesn't change v at all because it's just a rotation about the z axis then, so, T v is equal to v, and therefore v is an eigenvector
22:56
with eigenvalue one.
23:02
But if we think about any vector that's not on the z axis, then rotating by an angle theta is not possibly going to let it point in the same direction, lie on the same line, unless, the one possibility is if we're rotating by 180 degrees, in which case it'll point in the completely opposite direction, if it lied on the x, y plane.
23:32
So no other vectors v have T v proportional
23:47
to v unless we are rotating by an angle of 180 degrees, so pi radians, in which case
24:02
every vector on the x, y plane,
24:09
so with z coordinate zero, has T v equals minus v,
24:19
so is an eigenvector
24:25
with eigenvalue
24:29
minus one. So that's maybe what we get geometrically thinking about it, but we can also do it algebraically and concretely calculate, say, the characteristic polynomial.
24:45
So the matrix of T with respect to the standard basis is given by cos theta,
24:59
minus sin theta, sin theta, cos theta, then zero zero one zero zero.
25:12
And so we can now calculate the characteristic polynomial. So chi_A of t will be the determinant of A minus t times the identity. So this would be cos theta minus t, minus sin theta, zero,
25:35
sin theta, cos theta minus t, zero, zero, zero and one minus t. So we've just taken t away from all the elements on the diagonal. And now
25:47
by the Laplace expansion, this is one minus t times the determinant of the upper left hand corner. And that's going to be
25:59
cos theta minus t all squared plus sin theta all squared,
26:10
because we have a minus sin theta in the top row. And we can expand this out, and using cos squared plus sin squared is equal to one, this is t squared minus two t cos theta plus one. So that's the characteristic polynomial for this rotation theta, rotation by angle theta around the z axis. And the roots of
26:43
the characteristic polynomial are,
26:50
well, there's a factor of one minus t, so we always have one, and then we have the other roots are cos theta plus or minus i times sin theta, where i is the square root of minus one.
27:10
Therefore, if sin theta does not equal zero, which corresponds to theta not being equal to pi, or I guess the totally trivial transformation where it's just the identity,
27:30
then one is the only eigenvalue over R.
27:45
If theta equals pi, then one and
27:56
cos theta, which is equal to minus one, are the eigenvalues.
28:07
So this fits with what we geometrically thought beforehand. But this way, we are able to calculate it algebraically very explicitly.
28:20
OK, good. So
28:27
maybe we'll move on to investigating this characteristic polynomial a little bit further.
28:37
But to do this, first of all, I'd like to recall the trace. So
28:47
the trace of a matrix,
28:56
which you should have seen in Linear Algebra 1,
29:01
is just the sum of all the diagonal elements.
29:19
And a quick lemma is that if we take
29:29
two n by n matrices A and B, we have that the trace of AB is equal to the trace of BA for all matrices A and B.
29:49
So the proof for this is simple. So the trace of AB is just the sum over i of
30:02
the i,i-th entry of the matrix AB. But we can write down that explicitly.
30:12
That's just the sum over j of a_ij b_ji.
30:20
But these are finite sums, so you can just swap the i sum and the j sum around. So this is the sum over j, sum over i of b_ji a_ij.
30:34
And we see that then this is precisely the same expansion of the trace with matrix BA.
30:42
And so you can even though AB is not necessarily equal to BA, it's the case that the trace of AB is equal to the trace of BA.
30:54
And this is important, because this means that we can define the trace of a linear transformation
31:02
as just the trace of any matrix and it won't depend on the choice of basis. So the trace of a matrix doesn't depend on any— on the basis chosen. So let
31:21
T send V to V be a linear transformation.
31:33
The trace of a linear transformation is the trace of A where A is the matrix
31:54
representing T with respect to some basis.
32:04
So, again, we have to check this is well-defined.
32:12
So if A and B both represent T
32:22
with respect to different bases, then B is equal to P inverse A P, some change of basis matrix.
32:32
So the trace of B is equal to the trace of P inverse A P. But we know that we can change the order of elements in the multiplication in the trace and it doesn't change the trace at all. So this is equal to the trace of A times P inverse P. But P inverse P is just the identity. So this is just the trace of A.
33:06
And so therefore, the trace of B is equal to the trace of A. And so the trace is the same for any matrix representing T with respect to an arbitrary basis. And therefore, this is a well-defined definition of the trace.
33:27
So now we've recapped a little bit about the trace,
33:32
we can investigate our characteristic polynomial a little bit more to establish that it is indeed a polynomial, and moreover, it encompasses these two key invariants we've seen, the trace and the determinant, as coefficients of this polynomial.
33:56
Proposition.
34:00
For A in M_n R,
34:10
chi_A of t is a polynomial
34:18
of degree n with lead coefficient minus one to the n, then next biggest coefficient, minus one to n minus one times the trace of A times t to the n minus one. And then there's some intermediate coefficients. And then the constant coefficient is det A.
34:53
That's the proposition verifying that the characteristic polynomial is a polynomial. So for n by n matrices its a polynomial degree n. And moreover, it's telling us the zeroth coefficient, the lead coefficient, and the next to lead coefficient.
35:17
So we can deal with the zero coefficient first, because that's fairly straightforward. chi_A of zero is just the determinant of A,
35:32
so det A is this constant term.
35:46
We see that chi_A of t is the determinant of matrix B, where, as before, B is given by a_1 minus t— sorry, a_11 minus t, a_22 minus t up to a_nn minus t on the diagonals, but then everywhere else it's just equal to a_ij. So we have a_21 up to
36:24
a_n1. And here we would have a n, n minus one.
36:37
And similarly here we would have a n minus 1, n up to a_12, and a_1n.
36:50
So it's just A, but all the elements on the diagonal we've subtracted t from. And
37:00
using our definitions of the determinant in terms of permutations, we know that this is the sum over all permutations sigma of the sign of sigma
37:17
times b 1, sigma 1 up to b n, sigma n.
37:26
Now b_ij is a...
37:33
b_ij doesn't depend on t
37:43
if i does not equal j, and b_ij is a polynomial of degree one in t if i does equal j.
38:02
Therefore, if I look at this product b 1, sigma 1 up to b n, sigma n,
38:13
this is a polynomial of
38:24
degree at most the number of choices of i such that i equals sigma i.
38:40
So the number of fixed points. OK. But the identity permutation has n fixed points,
39:06
and any permutation that's not the identity has at most n minus two fixed points.
39:21
So this means that in this big sum over all permutations, the identity permutation contributes a
39:29
polynomial term that's of degree n. And all the others are going to contribute polynomial terms of degree at most n minus two.
39:39
Therefore chi_A of t is equal to the product of b_ii from i equals one to n, so that's the contribution from the identity permutation, plus some polynomial of degree
40:05
at most n minus two. But b_ii is equal to just
40:17
a_ii minus t.
40:29
And it's easy to calculate the lead terms when we expand out this product. This is minus one to the n, t to the n plus minus one to the n minus one times the sum of a_ii times t to the n minus one.
40:53
And then we're left over with some other polynomial of
41:01
degree at most n minus two. Therefore
41:08
chi_A of t is a polynomial of degree
41:16
n with lead coefficient minus one to the n, next to lead,
41:29
minus one to the n minus one times the trace of A, because that's what the sum of a_ii is. And it then has constant coefficient what we saw before, is just the determinant of A.
41:51
And so this gives the result that we had. OK, great. I think that's covered everything that I'd like to say today. Thanks a lot for listening. I hope you enjoyed the lecture. Goodbye.
Here to help you learn
Sign in to keep a private conversation with your course tutor.
Ask me anything about the material…
Sign in to chat