Skip to main content
Loading Explore guides…
Matrix Trace · Acedra
Home
Explore
Mathematics
Matrix Trace
Sign in to save
Chat
If YouTube asks you to sign in, open the video on YouTube.
Watch on YouTube
Reload player
Transcript
35 segments
Copy
Hide video
0:00
Given a particular matrix, if you want to compute its trace, you will be basically adding up the diagonal entries. But this is quite an algebraic approach in thinking about trace. Can we visualise what’s really happening when we add these diagonal entries? The visualisation will also explain many properties of trace you might have seen. There is one particular property that I’m saving for the end of the video series, which is tr(AB) = tr(BA).
0:32
However, in the majority of cases where one of A or B is invertible, we can derive it using one of the other properties. But in any case, let’s start the visualisation. At the end of the previous video, we have described a matrix as a vector field. Let’s expand on that a little bit. If you are given a matrix A, then the vector field will be constructed by attaching every point with position vector x by the vector Ax. A little note on the illustration on screen here.
1:10
Technically if this illustration really describes Ax, then the vector field would look very messy because some of the vectors would be too long, so the usual thing to do is to standardise the lengths of the vectors, which will still give you the direction. If you really need the length, then add colours to indicate how long the vector should have been. In particular, these two vectors are the important ones, because they are the vectors attached to the points (1,0) and (0,1) respectively.
1:46
The vector attached to (1,0) would be the first column of A, while the vector attached to (0,1) would be the second column of A. With this visualisation of matrices, the intuition for trace is just 1 sentence. The trace of A is the divergence of this vector field created by the matrix A. We are going to see what divergence is. I am aware that this has been talked about on YouTube, but I want to be more quantitative, so bear with me if you already know the visual intuition of divergence.
2:23
Given a particular vector field, we can define divergence at a point, say this red dot. We now consider a very small region around this dot, and write down the area of this small region. The next thing is to let every point in this small region evolve together along the vector field. As you see, the area of this region changes. For divergence, we are interested in thinking about how quickly the area changes initially per area, as the area of the initial region shrinks to 0.
3:03
In this case, let’s look at how the area evolves again. We can see that it initially decreases, and in fact keeps decreasing along the way, so because the area change is negative, we have the divergence at this red dot to also be negative. However, one reminder is that this rate of area change is per area. In this case, if we compare the final area with the initial area, we can record a change of -0.148 during the evolution.
3:39
Now if we quadruple the initial area, and again play the same game and let this area evolve under the flow of the vector field, and this time we record a change of -0.595 in area. If we compare this change to the previous area change, this is exactly a factor of 4, accounting for rounding errors in my computer animation. So in our intuition of divergence, the per area part is very important. So that’s the divergence part, but why is this related to the trace of the matrix generating
4:21
the vector field? Actually the first question that comes up would be that the divergence is a local property. This means that if we consider another point on the vector field, the divergence could be different. So why does this statement seem like the divergence is the same everywhere? Well, it’s because it is! But just in this specific case where everything is generated by the matrix A. To see why the divergence is same everywhere, let’s first denote this point with its position vector
4:56
x, and any point within our small region around x to be x + d, for small displacements d from x. Now the vectors attached to these two points are Ax and A(x+d) respectively. This means that the initial displacement between these two points is d, and after a very short time epsilon, the displacement is increased by epsilon times A d. Here, A d is the difference in their velocities, or the rate where they drift apart. However, notice that there is absolutely no x here.
5:38
This result does not depend on x. So if we have this exact shape of the small region, let’s illustrate it with just three points, each with displacement d_i from the point we try to compute divergence of. After a short epsilon period of time, the displacements are changed to d_i + epsilon A d_i. But these are independent of our position! So even if we have this initial region elsewhere, we will still have the exact same shape after time epsilon. So the change in area is the same everywhere.
6:21
Because the divergence is the area change per area, divergence is the same everywhere in this case. This also means we can choose a convenient region to see how its area changes, and this region doesn’t have to be infinitesimal either. This is because if we chop any normal region into very small ones, then from our intuition of divergence, each small region U_i has its area changed from A_i to A_i (1+ epsilon times divergence) after time epsilon.
7:00
Most importantly, we know now the divergence is the same across all these regions. Then for the entire region U, which has its area being the sum of these small regions, would also scale by the same factor of 1 + epsilon times divergence. Since we can basically choose any region to see how the area evolves, a convenient choice is the unit square containing points (1,0) and (0,1), as the initial area would automatically be 1, so we focus on rate of area change.
7:41
If we suppose that A is the square matrix (a,b; c,d), then as said before, the vector attached to (1,0) would be the first column (a,c); and the vector attached to (0,1) would be the second column (b,d). Let’s focus on the point (1,0) and its attached vector (a,c). Here, its vertical component doesn’t really matter here, because it doesn’t change the area of the region. The base and height of the parallelogram have not changed.
8:16
So the vertical component doesn’t matter, but what about the horizontal component? This time it absolutely matters. The area has changed. The rate of change here is given by precisely the horizontal component of the attached vector, which is a. You can run a very similar argument on the other point (0,1) here. This time, the horizontal component does not matter because again the area wouldn’t change in that case, and the vertical component provides the change in area.
8:53
In this case, the vertical component is d. So altogether, the rate of change in area would be a + d, which is exactly the sum of the diagonal entries of the matrix A, so we have demonstrated that trace is really the divergence of the vector field. Now if you are not convinced by these horizontal component or vertical component argument, all we are doing is computing the determinant of this slightly different matrix.
9:27
The first column is where the point (1,0) moves to after time epsilon with this velocity (a,c). Similarly, the second column is where the point (0,1) moves to after time epsilon with the velocity (b,d). If you actually compute this determinant, then we see that this is really the rate of change of the area, that is also the trace. Anyway, the fact that trace is just the divergence of this vector field is very useful for understanding the many properties of trace.
10:05
And these properties, one by one, would be the focus of the video from this point onwards. Let’s first start with why trace is the sum of eigenvalues. When visualising matrices this way with vector fields, if the position vector x is an eigenvector of A, then the vector attached to it, which is Ax, would face the same or opposite direction as the position vector. In fact, once you are on this line, you will stay on this line because the position and velocity vectors are on this line.
10:44
In the case where there are two such lines where the position vector and the velocity vector attached are on the same line, then to investigate the divergence of this vector field and hence the trace, one convenient choice of region is a parallelogram aligned with these two lines. Let’s label the two lines by their eigenvalues lambda_1 and lambda_2 respectively, and let’s say the side lengths of the parallelogram are a and b. Focus on this corner of the parallelogram.
11:19
The length of the vector attached to it should be lambda_1 a. That’s because on this line, every position vector satisfies Ax = lambda_1 x. Since the position vector has length a in our notation, the vector attached to it should be with length lambda_1 a. So after a very short time epsilon, we will have an extra length of epsilon lambda_1 a along this line. Using the same argument on the other corner of the parallelogram, the extra length on
11:56
the other line would be epsilon lambda_2 b after time epsilon. In this illustration, lambda_2 is negative, and so the so-called extra length is actually the length lost. So after a short time epsilon, the length along this lambda_2 line would be b(1+epsilon lambda_2), and similarly, the length along the lambda_1 line would be a(1+epsilon lambda_1). So the new area is the product of the side lengths and sin theta, where theta is the angle between these two lines.
12:38
Compare this with the old area which is ab sin theta. If we now compute the divergence of this vector field, which is the rate of area change per area, then we would have this expression, which is new area minus the old area over the time period of epsilon, divided by the initial area which is ab sin theta. If you really compute this, we would have lambda_1 + lambda_2 + a small bit involving epsilon. However, as epsilon tends to 0, then the epsilon bit does not contribute and we are left with
13:19
the sum of eigenvalues, so the trace is the sum of the eigenvalues in this illustration. So this is the first property of trace. The next property has something to do with the exponential of matrices, which I have described in the previous video as well. Essentially, given a vector field created by the matrix A, the flow of this point with position vector x along the vector field is precisely given by e^(tA) applied to x, where t is the time elapsed when you flow along the field.
14:00
The determinant of a matrix is the scaling factor of this unit square under the matrix transformation. Applying the matrix e^(tA) to this square is equivalent to letting the region flow along the vector field, so the area of whatever we end up with is the determinant of the matrix exponential. And we actually know how the areas evolve along the vector flow. Let’s denote the area of this region as S(t), which depends on time elapsed along the vector field.
14:35
The trace of A is the rate of area change per area, with the rate of change in area being the derivative of S, and the area being S(t). By rearranging this formula, we obtain a differential equation in S. Note that trace of A doesn’t depend on t, it is a constant, so the determinant S(t) can be solved as e^(t trA). This neatly explains why the determinant of exponential matrix is the exponential of trace. Now let’s go to the property that the trace is independent of basis vectors.
15:21
Given a matrix A, as usual we consider the vector field generated. This time, we apply a linear transformation Q to the whole picture. In general, where the vectors emanate from would change, and the vectors attached would also change. More specifically, what had been x is now transformed to Qx, and what used to be Ax is now QA x. This one needs a bit of explanation. Initially, the attached vector to the point x is going from point x to x + Ax, so when
16:02
we apply the transformation Q on the whole picture, then the attached vector goes from Qx to Qx + QAx, so this attached vector is QA x. Now let’s call this new position vector y. Then in terms of y, the attached vector is QAQ^(-1) y, if Q is invertible. So for any position vector y, the attached vector is QAQ^(-1)y, which suggests that this is actually the vector field of another matrix QAQ^(-1). Now the trace is the rate of change in area per area, but in applying Q to the whole thing,
16:52
all areas scale by the determinant of Q, and this factor of determinant cancels when we consider the quotient. So in other words, the trace of A equals the trace of QAQ^(-1). Or if you want to go for the usual change of basis formula P^-1 AP, simply substitute the matrix Q as P^(-1). Using this relation, we are able to derive another famous property of trace, which starts from rewriting trace of AB. If B is invertible, then we can simply add in B^(-1) B, and the expression stays the same.
17:38
However, written like this, we can use the property we just derived, and then we can obtain the trace of BA. Now this property of trace is perhaps more well-known, but we have done a bit of algebra to get here. There is a better way, but we will introduce the concept of Lie bracket, a story for another time. So we can obtain this property of trace by transforming the whole plane by P^(-1), and the final property we will see is this perhaps less famous formula.
18:16
This formula is known as Jacobi’s formula for a matrix A that depends on time t. Well, technically, this is only true if A(t) in this formula is invertible, because there is A^(-1) in the formula. A more general formula involves the adjoint matrix, which is defined even if A is not invertible. But for the purpose of visualisation, we are going to assume A is invertible, and try to understand this formula first. First off, what does the left hand side really mean?
18:53
The determinant can refer to the area of a parallelogram described by A(t) acting on the two basis vectors. This area will change because A itself evolves with time. In fact, these points would also change as A changes. The rate at which these corners move is exactly A’(t) acting on the basis vectors. If we can somehow make this into a whole vector field, generated by the matrix M, then we know how the area evolves. Specifically, we know that the trace of M is the rate of area change per area.
19:38
In this case, the rate of area change is the derivative of det A. And the area is of course just the determinant of A. So if we rearrange this, then we have something that really looks like the Jacobi formula. The problem now is what is M? What is the matrix M that generates this vector field? Now particularly at the points A(t) acting on the basis vectors, we know what the vectors attached should be. To go from the position vector to the attached vector, we can simply apply A’ A^(-1) on
20:20
the position vector. So the matrix M we are looking for is A’ A^(-1). If we go back to what we have obtained previously about derivative of determinant, we simply substitute the matrix M to be A’ A^(-1). This is *almost* the same as Jacobi's formula. Just that the matrix in the trace has the other order in multiplication. A^(-1) before A’ here. Don’t worry, there are two approaches here: either use the property we discovered that
20:58
trace AB = trace BA for A or B being invertible; or similar to how we dealt with the previous property, we just apply A inverse to the whole picture. In that case, the corners of the parallelogram go back to their standard places (1,0) and (0,1), but the vectors attached to these places are simply A^(-1) A’ acting on the basis vectors. This is because previously they were just A’ acting on them, but we have applied A^(-1) to the whole picture.
21:37
This means that the matrix generating the vector field would be A^(-1) A’, and the matrix M to be put into the formula that we derived previously would be A^(-1) A’. And this is exactly the Jacobi’s formula where the matrices in the trace are in the right order, A^(-1) A’. So this is the Jacobi’s formula derived in the case when A is invertible. We are able to derive and understand many properties of trace visually by thinking of
22:14
trace as divergence, or the rate of area change per area when the area evolves along the vector field. As always, thanks for the patrons, and please like, subscribe, comment so that more people can watch it. See you next time!
Here to help you learn
Sign in to keep a private conversation with your course tutor.
Ask me anything about the material…
Sign in to chat