Skip to main content
Loading Explore guides…
Eigenvalues · Acedra
Home
Explore
Mathematics
Eigenvalues
Sign in to save
Chat
If YouTube asks you to sign in, open the video on YouTube.
Watch on YouTube
Reload player
Transcript
40 segments
Copy
Hide video
0:00
Give me 25 minutes and I will make IGEN values click forever. We're going to move beyond seeing a matrix as just a grid of numbers. We will uncover the deep predictive information it holds about the system it represents. This tutorial is direct, no fluff. By the end, you'll understand this. First, the core idea. You'll finally get what AV equals lambda V really means. We'll see why igen vectors are the hidden axes of a transformation. Then the algorithm. We'll walk through the step-by-step
0:33
recipe. You know the determinant of a minus lambda i equals z to find values and IGEN vectors. And finally the application. The cool part. How IGEN values predict the long-term behavior of a system. This is the machinery that powers everything from Google's page rank to facial recognition. Let's get started. Quick recap. A matrix is an action. It acts on an input vector, let's call it V, to produce an output vector V prime. As you see in the formula, V prime equals A * V. This
1:07
action can be a rotation, a scaling, a shear, or a mix of all three. Let's look at a concrete example, a shear transformation. We're going to see what this matrix A does to a vector V. Our matrix A is 1 1 0 1. And our vector V is 2 3 in code. This is super simple. We just import numpy, then set A equals numpy.array with values 1 1 0 1 and V as a numpyarray with 2 3. The transformation is just V prime equals A at V. So what's the calculation here? We take our matrix 1 1 and multiply it by
1:46
our vector 2 3. That's just 1 * 2 + 1 * 3 for the top row and 0 * 2 + 1 * 3 for the bottom, which gives us the new vector 53. And the output confirms it. The original vector V was 23. The transformed vector V prime is 53. Visually, the vector was bent to the right. The matrix A took our vector V and knocked it completely off its original line. The direction changed. This leads us to the central question of this entire video. Are there any special nonzero vectors that a matrix doesn't
2:24
knock off their line? Are there vectors whose direction remains perfectly unchanged? The answer is yes. And now we're on a hunt for them. The [clears throat] hunt begins with our shear matrix A. It's the matrix 1 1 0 1. We saw it knocked the vector 23 completely off its line. But what about other vectors? Let's see if we can find one that holds its ground. Let's try our first case. We'll call this one the failure. We'll test a vertical vector V up, which is the vector 01. What does our matrix A
3:00
do to it? Well, we multiply our matrix A 1 1 0 1 by our vector 01. So the top element is 1 * 0 + 1 * 1 and the bottom is 0 * 0 + 1 * 1 which gives us the vector 1 1. The vector 01 was transformed into 1 1. Its direction was completely changed. This vector failed our test. Okay, that didn't work. But what about case two? Let's try a horizontal vector. Let's call it vright. the vector 1 0. What happens now? We take our matrix A 1 1 0 1 and multiply it by VR 1 0. For the
3:44
top, we get 1 * 1 + 1 * 0. And for the bottom, 0 * 1 + 1 * 0. And the result is the vector 1 0. Look at that. This is completely different. The vector 1 0 was transformed into itself. its direction is perfectly unchanged. What we just discovered visually are the two most important concepts in this field. Intuition number one, Igen vectors are directions that don't turn. An igen vector of a matrix A is any nonzero vector V that when you apply the transformation A does not change its
4:24
direction. For our matrix A, the vector V= 1 0 is an igen vector. And what's really cool is that any other vector on that line like 30 0 or -50 is also an igen vector. This entire special line in this case the x-axis is called an igen space. Okay, that's the direction. Now for intuition number two values are the stretch factors. When we found our igen vector v= 1 0, the result of av was 1 0. How much was it stretched? Well, it was multiplied by a factor of 1. This
5:02
stretch factor is the igen value denoted by the Greek letter lambda. Every vector has a matching igen value that tells you how much it was scaled along its line. If the absolute value of lambda is greater than one, the vector gets stretched. If it's less than one, it gets squished. And if lambda is negative, the vector gets flipped 180°, but it still stays on the same line. Now we can put all of this together. It's time for the formalization the IGEN equation.
5:35
AV equals lambda V. Let's really break this down into plain English because this is the key. On the left AV we have the action of the matrix A on its special igen vector V. And on the right lambda V we have is identical to simply scaling that vector V by a number lambda. That's it. That's the whole idea right there in that beautiful little formula. So for our shear matrix A, the one we started with 11 1, solution to AV= lambda V is the igen value lambda equals 1. That's the stretch vector. And
6:13
the igen vector V is the vector 1 0. That's the direction that doesn't turn. Hunting for those special directions visually was interesting, but it's not a real strategy. What happens with a 3x3 matrix? or a 100x100 matrix. We can't just look for the answer. We need a system, a process. Our goal [cough and clears throat] is to find a recipe. We need to solve this equation for any matrix A. AV equals lambda V. But here is the challenge. We have two unknowns. The scalar value
6:46
lambda and the vector V. We need a clever way to untangle them and solve for one at a time. Let's start with a bit of algebra. The algebraic rearrangement. One, we start with the igen equation. AV equals lambda V. Two, subtract lambda V from both sides. That gives us AV minus lambda V equals 0. And here that zero represents the zero vector. Okay, step three. Let's factor out the vector V. That should give us A minus lambda * V equals 0. And stop right there. We've
7:20
hit a major mathematical problem. Do you see it? Look closely at that equation. A minus lambda in parentheses * V equals 0. Here's the issue. A is a matrix. You know, a grid of numbers. But lambda is a scalar, a single number. You cannot subtract a single number from a matrix. It's like trying to subtract the number five from a photograph. It just doesn't make any sense. Now, this is a sticking point for a lot of people, but the fix is incredibly clever. The fix is the
7:52
identity matrix capital I. The identity matrix I is the matrix equivalent of the number 1. Anything you multiply it by stays the same. So I * V is just V. Because of that, we can rewrite lambda V as lambda * I * V without changing a single thing. Lambda V equals lambda I * V. Okay, let's try that again but correctly this time. We'll start with our intermediate step. AV minus lambda V equals zero. Now we substitute lambda V with our new matrix compatible version
8:25
lambda IV. So we have AV minus lambda IV equals zero. Now we can legally factor out V. Both A and lambda I are matrices. So we get A minus lambda I all in parenthesis* V equals 0. This is the standard form of the IGEN equation and it's the version we are going to use to build our entire algorithm. Now for the aha moment, the determinant is the key. Look at this equation. A minus lambda I * V equals 0. Let's call the matrix part M. M= A minus lambda I. So our equation is now just
9:05
MV= 0. This equation asks a really simple question. What vector V does the matrix M transform into the zero vector? Well, there's always one boring trivial answer. V equals 0. Of course, M * the zero vector is the zero vector. That's not interesting. But remember our definition. Igen vectors must be non zero. So the real question is when does MV equals 0 have a solution where V is not the zero vector? A nonzero solution exists only if the matrix M is singular.
9:39
A singular matrix is one that collapses space into a lower dimension. Think of a 2D transformation that squishes the entire plane down onto a single line. When that happens, a whole line of vectors gets mapped to the origin, giving us the nonzero solutions we need. And what is the single definitive test for a singular matrix? Its determinant is zero. This is the master key. If we want a minus lambda i * v equals 0 to have a nonzero solution for v, then the matrix a minus lambda i must have a
10:13
determinant of zero. So we get our next big equation. The determinant of a minus lambda i equals 0. Look at what we just did. The determinant of a minus lambda i equals zero. The vector v has vanished for a moment. This equation called the characteristic equation lets us solve for the igen values lambda first. We've successfully untangled the two unknowns. We now have our reliable two-step recipe. Here it is. The recipe for finding IGEN properties. Step one, find
10:44
the igen values lambda. To do this, you set up and solve the characteristic equation. The determinant of a minus lambda I equals zero. This will always be a polomial equation and its roots are your igen values. Step two, find the igen vectors V. For each Igen value lambda you just found, you plug it back into the standard form equation. A minus lambda I * V equals 0. Then you solve that system of linear equations to find the corresponding igen vector V. This is
11:14
the algorithm. A guaranteed process that gives us every lambda v pair that satisfies AV equals lambda V. All right, now it's time to stop talking and start calculating. This is where the theory becomes a tool. Our mission, the example matrix. We are going to find the igen properties for this matrix A. It's a 2x2 matrix with three, one in the first row and one three in the second. Now look closely. Notice something special. This matrix is symmetric across its main diagonal. Keep
11:52
that in mind. It's not a coincidence. I promise. Step one, find the igen values lambda. Let's start with action A. Construct the matrix A minus lambda I. This is just our starting matrix A minus lambda* the identity matrix. So that's our matrix 3 1 1 3 minus lambda 0 0 0 lambda which gives us our new matrix 3 - lambda 1 and 1 3 - lambda. Simple enough. Next action B. Calculate the determinant. For any 2x2 matrix the determinant is just a d minus b c. So for our matrix we get the equation the
12:37
determinant of 3 - lambda 1 1 3 - lambda equals 0. That works out to and that brings us to action C. Solve the polomial. Cleaning that up gives us a nice simple quadratic equation. Lambda^ 2 - 6 lambda + 8= 0. Quick quiz. How does this factor? It factors beautifully. It becomes lambda minus 4 * lambda minus 2 = 0. And just like that, the solutions fall right out. These are our igen values. lambda 1 = 4 and lambda 2= 2. We've found them. The two special stretch factors for our
13:25
matrix A are 4 and 2. This tells us that the transformation A has two special axes and we just found out how much it stretches along them. Now for step two, find the igen vectors V. Our goal here is simple. For each Igen value we just found, we need to solve for the vector V in our other key equation. A minus lambda I all in parenthesis time V equals 0. Let's start with case 1 for our first igen value lambda 1 = 4. We plug lambda= 4 into that equation. So we have the
14:06
matrix 3 - 41 and 1 3 - 4 * our vector xy equ= the 0 vector. That simplifies to the matrix -1 1 and 1 -1 * xy = 0. Now for action B, solve the system. This matrix equation really just represents two lines. The first row gives us -x + y = 0, which just means y equ= x. The second row gives us x - y = 0, which also means y equals x. Wait, the equations are identical. Is that a mistake? No, this is a guarantee. it's supposed to happen. The system is designed to have infinite solutions
14:58
because we're looking for an entire line of vectors, the Igen space. So, we just need to find any nonzero vector that satisfies y= x. What's the simplest possible choice? Let's just say x= 1. Well, then y must also be 1. And there it is. Our first igen vector is v1. the vector 1 1. All right, let's do it again for case 2. For our second igen value, lambda 2 equals 2. We're going to move a little faster this time. You know the drill. We plug lambda= 2 into the equation a - lambda i * v = 0. That
15:40
gives us the matrix 3 - 2 1 and 1 3 - 2 * x y = 0 which simplifies to the matrix 1 1 1. This gives us the simple equation x + y = 0 or in other words y = -x. So what's the simplest vector we can choose? Let x equal 1. Then y equ= -1. Our second igen vector is v2. The vector 1 -1. And we're done. For the igen value lambda 1 = 4, we found the igen vector v1 = 1 1. Along the y= x line, vectors are stretched by a factor of 4. For the igen value lambda 2= 2, we found the igen vector v2 = 1
16:37
-1. Along the y =x line, vectors are stretched by a factor of 2. But how do we know we're right? Let's verify. Does a * v1 really equal lambda 1 * v1? Left side. Our matrix 3113 * the vector 1 1. The result is 3 * 1 + 1 * 1 and 1 * 1 + 3 * 1 which gives us the vector 44 right side lambda 1 * v1 that's just our igen value 4 * the vector 1 1 which gives us the vector 44 they match [snorts] perfectly the algorithm works remember I said our matrix A was symmetric. Here comes the
17:33
payoff. Look at the beautiful results it gave us. First, the igen values 4 and 2 are real numbers. There are no imaginary numbers. No complex weirdness. This is always true for symmetric matrices. Second, the igen vectors, the vector 1 and the vector 1 -1 are orthogonal. We can prove that with the dotproduct v_sub_1 dot v2 is zero. A zero dotproduct means a perfect 90° angle. This is also always true for symmetric matrices. You can see it right here. Orthogonal igen vectors at a perfect 90°
18:15
angle. This clean geometry makes these matrices the bedrock of data science. They represent stable, well-behaved systems whose fundamental axes don't shear or skew. We just went through a lot of algebra. So what? Why did we do all that work? Because I en values let us predict the long-term behavior of a system. Let's look at the motivating problem, a population model. Imagine a population is split between a city and its suburbs. The yearly change is described by a
18:46
transition matrix A and a state vector V. As you can see on the slide, A is the matrix with 0.95 and 0.10 in the first row and 0.05 and 0.90 in the second. And the vector V subK just holds the city and suburb populations for a given year K. So to find the population after K years, we use this formula you see here. V subK equals A to the power of K * our starting population V sub0. So here's our goal. Predict the state after 50 years. We need to find V sub50. Now,
19:21
calculating A to the 55th by hand is painful. And worse, it gives you zero insight into what's actually happening. There has to be a more elegant way. And there is. We know if our starting vector V 0 was an igen vector, the problem would be easy. A 50th * V 0 would just be lambda 50th * V 0. But what if it isn't? Here's the key idea. For most matrices, their igen vectors form a basis. You can think of them as a set of fundamental directions, a kind of natural coordinate system for the
19:53
transformation. So let's look at the igen vector basis solution for our population matrix A. We can find two igen pairs. First a stable state lambda 1= 1.0 with its igen vector v1 and second a decaying state lambda 2= 0.85 with its igen vector v2. Because they form a basis, we can write any starting vector v 0 as a mix of them. Just like in this formula, v 0= c1 v1 + c2 v2. Now watch what happens. 1, we start with our decomposed vector v sub50 = a 55th * our
20:31
whole expression c1 v1 + c2 v2. Two, we distribute the a to the 50th matrix thanks to linearity. This gives us C1 * A to the 55th V1 + C2 * A to the 55th V2 3. Now we use our igen value magic on each part and we get the final result. V sub50 = C1 * lambda 1 to 55th * V1 plus C2 * lambda 2 55th * V2. This is the general powerful formula for predicting the future state of any linear system. Now for the aha moment, the dominant igen value. Let's look at our formula
21:10
again. V sub50= C1 * lambda 1 55th V1 plus C2 * lambda 2 to the 55th V2. Now let's plug in our actual values from the problem. Lambda 1 is 1.0 and lambda 2 is 0.85. So the equation becomes v sub50 = c1 * 1.0 to the 55th power v1 plus c2 * 0.85 85 to the 50 V2. Let's look at what these numbers become. 1.0 to the 50th power is well, it's just 1. But what about 0.85 to the 50th power? It's approximately 0.0000296. It's practically zero. You can see this effect in the visualization here. The
21:56
one line stays flat while the other just drops off a cliff. So, let's rewrite our equation with these new values. V sub50 is approximately C1 * 1 * V1 + C2 * 0296 * V2. The entire second term of the equation, it just withers away. We're left with this beautifully simple result. V sub50 is approximately C1 * V1. After 50 years, the part of our starting vector corresponding to the smaller igen value lambda 2= 0.85 has decayed into almost nothing. The final state of the system becomes almost
22:34
perfectly aligned with the igen vector of the largest or dominant igen value. The dominant igen vector isn't just a special direction. It is the destiny of the system. No matter the initial population, if you wait long enough, the system will inevitably converge towards a state proportional to this dominant igen vector. We didn't just solve for V sub50. We uncovered the fundamental stable trend buried deep inside the matrix A. You've made it. You went from seeing a
23:02
matrix as just a grid of numbers to understanding the profound story it tells. You started with a simple question. Are there any directions that don't change? And that question led you to the engine of linear algebra. So let's just take a moment to realize what you now understand intuitively. Igen vectors are the hidden axes of a transformation. Those special directions that remain stable. Een valids are the stretch factors along these axes telling you exactly how much they scale. You
23:33
know that the equation determinant of a minus lambda i equals 0 is the master key to finding these values for any matrix. And most powerfully the dominant igen vector reveals the ultimate destiny of a system predicting its long-term behavior. So what is your next step? It's a technique called principal component analysis, PCA. Think about it like this. So far, we found the axes of a transformation. Now, I want you to ask a new, more powerful question. What are the axes of a data
24:08
set? If you look at this visualization, you can see a cloud of data points. PCA is the technique that finds the most important directions within that data. PC1 the first principal component shows you the direction of maximum variance. This corresponds to the largest igen value. Then PC2 is orthogonal to the first one and it's the direction with the second largest igen value. And here is the amazing part. The concepts you mastered in this course are the engine that drives this incredibly powerful
24:40
technique. You are ready for the next step.
Here to help you learn
Sign in to keep a private conversation with your course tutor.
Ask me anything about the material…
Sign in to chat