Skip to main content
Loading Explore guides…
History of Linear Algebra · Acedra
Home
Explore
Mathematics
History of Linear Algebra
Sign in to save
Chat
If YouTube asks you to sign in, open the video on YouTube.
Watch on YouTube
Reload player
Transcript
67 segments
Copy
Hide video
0:00
Hello, everyone. I'm Norman Wildberger, and we're here at the University of New South Wales. In today's lecture, we're going to talk about matrices, determinants, and the birth of linear algebra. Linear algebra is a very important topic for 20th-century mathematics, almost at a par with calculus as far as undergraduate education goes. But it was not always so, and in the 19th century, linear algebra was just getting going. We're going to talk a little bit about the development,
0:49
both in the 19th century and earlier as well. The origins of the subject really go back to antiquity. They go back to a Chinese work called Nine Chapters of the Mathematical Art,
1:20
also known as the "Chiu Chang Suan Shu". My pronunciation is about right, and this was written 300 BC to about 179 AD. It was actually an anonymous work, in a sense that we don't actually know who wrote it; there were various contributors. It was composed over several generations, and it basically told us how to solve systems of linear equations and introduced basic ideas of matrices and Gaussian elimination.
2:14
Okay, so this is a very ancient and important work that really was much earlier than the European work in this direction. In the modern time, there was a Japanese mathematician whose name was Siki Kowa who, around 1683, introduced a more modern approach to linear algebra and systems of equations. And simultaneously, more or less in Europe, Gottfried Leibniz of calculus fame around 1678 considered the general problem and there's a fundamental
2:54
problem in the subject which we are going to put here centerfolds. So it's, we have a system of equations of the form xi equals summation k equals 1 to n ai k yk where i goes from 1 to N, and we have therefore n equations in n unknowns. So, n equations in n unknowns y1 through yn and the idea is that we want to solve for the Y's given the excises the basic fundamental problem in linear algebra. So for example, in two dimensions, we have the following situation: for example,
4:01
a11y1 plus a12y2 equals x1 and a21y1 plus a22y2 equals x2, a system of two equations where we have unknowns y1 and y2 and we have known quantities x1 and x2. We want to understand what y1, y2 are in terms of x1 and x2 and also in terms of these coefficients. So these days, we like to parcel this system in matrix form, so we write things like this by introducing a matrix or an array a11 a12 a21 a22 and then the vector y1 y2 equals x1 x2. You're using matrix multiplication,
4:55
how we multiply 2 by 2 matrix by a vector like this. This is a short form for this system here. Now, the fundamental fact about this is that this has a unique solution. In other words, we can solve for the Y's precisely when a certain number, let's call it Delta, is nonzero, and that number is a11 times a22 minus a12a21. So that number is nonzero, then we can solve for the y1 and y2 uniquely, right? So this is a fundamental number and in modern notation we
5:47
would say this is precisely when the matrix, so let's call this A, the matrix A is invertible.
6:00
Now, this quantity here, the quantity Delta, is the central object in this topic of determinants. It is an example of a determinant, and it also has the form we write sometimes with the matrix like this, with square or square brackets. This is also sometimes written as the determinant of the matrix a11, a12, a21, a22. So, this is an example of a determinant.
6:45
All right, so this is a basic example, a basic problem in linear algebra, the first non-trivial example of the 2x2 situation. This is the crucial quantity that tells us when we actually have a solution. Suppose we do have a solution, then what can we say about the solution? That's the next step: to actually solve the system. This was all worked out by the ancient Chinese and also Kowa and Leibniz. All right, so let's solve this system. Here is the system again,
7:15
and the solution is that y1, y2 is one over this determinant quantity times the matrix a22, a11, minus a12, minus a21 times x1, x2. This is a very fundamental and beautiful thing: the linear relationship here can be inverted to get a similar kind of relationship between the y's and the x's, but with this matrix having to be modified. Crucially, this determinant is not equal to zero. Now, around 1750, Cramer discovered another way of
8:15
expressing this solution. He realized that we can write it as follows: y1 is x1, a12, x2, a22, so the 2x2 determinant formed by that little matrix divided by the Delta, and y2 is similarly the determinant a11, a21, x1, x2, again over the determinant. So, what Cramer did is he discovered that there was a sort of closed formula for the solutions individually by taking determinants, actually ratios of determinants, where we take the original coefficient matrix,
8:59
remove the first column and replace it with this right-hand column x's, in this case, and over here replace the second column with the x's and create determinants. This was called Cramer's Rule, and pleasantly, it ends up generalizing to the case of n equations in n unknowns. Now, Cramer was interested in this rule because he had a very specific problem that he wanted to solve, a problem which, in fact, is a very common problem for students of geometry. The problem is,
9:41
he was motivated by the problem of how to find a conic through five points.
9:57
That's a basic fact of geometry: if you have five points in the plane, then there is a unique conic that passes through them, maybe in this case something like an ellipse. So, the conic is in general of the equation ax squared plus, say, bxy plus cy squared plus dx plus ey plus f equals zero. There are six coefficients that we have to find, but in fact, there are really only five coefficients because, up to a scaling,
10:36
the conic doesn't change. If we multiply all the coefficients by a number, it doesn't change. So, there are only five degrees of freedom, and if we have five points, we can substitute each one of those five points in there, and we'll get five equations in essentially five unknowns. This is a very familiar problem that, in fact, has been solved over and over again by many generations of geometry students. But it's useful to point out that there is a
11:02
little bit of a more pleasant way of solving this particular problem, which is quite interesting. So, it turns out that if we look at four of the points, those four points determine a pencil of conics. Four of the five points determine a pencil of conics, so it's a one-parameter family of conics. Perhaps it's one that is more elongated, or maybe a hyperbola, and so on. So, this one-parameter family of conics includes two,
11:47
or actually three, special ones which include products of lines. So, for example, if we take this line together with this line, the product of the two equations of those two lines is a second-degree equation. So, it's actually also a conic. If we call that product of lines C1 and then choose another pair of lines, say this one and this one, and call this product C2, then any combination of these equations, C1 and C2, will be another
12:35
conic which also passes through the same four points. So, if we take lambda C1 plus C2, this is another conic in the pencil for any scalar lambda. We just write down the equation of the lines multiplied together and the equations of the other two lines multiplied together, then set that equal to zero. That's going to be an equation which is still satisfied by the four points because if you plug in here and here, they both give you zero. So, they'll give you zero when
13:14
you plug into this linear combination. This is a general conic in this pencil.
13:21
To find the conic that passes through the fifth point, all we have to do is solve one equation. We have to only find this lambda here. So, if we substitute the fifth point, we get a linear equation for lambda. In fact, this canonical geometry problem can be solved in a more efficient way rather than writing down five equations in five unknowns. You can certainly do it that way too, but this is a historical problem that motivated the important rule given by Cramer.
14:04
So, now a natural question is: how are we going to generalize this? This is all fine and good for a 2x2 case, but the 17th- and 18th-century mathematicians, of course, were interested in larger systems of equations. For that, we need to generalize this notion of the determinant. Here is the 3x3 determinant. We have a matrix A, B, C, D, E, F, G, H, I. The determinant is defined to be the sum of these six terms with plus and minus signs. This is a sum of products,
14:39
one from each row and column of the matrix. For example, the first product is A times E times I, taking those three entries one from each row and each column. The others are all the same: A times F times H, B times D times I, B times E times G, C times D times H, and C times F times G. These correspond to the six possible ways of placing three rooks on a 3x3 chessboard such that none of the rooks can capture each other. In general,
15:25
if there's an N by N chessboard, there are N factorial ways of arranging these N rooks, and each of those N factorial positions will correspond to an entry in this sum.
15:39
There's another important ingredient in this definition, and that's the plus and minus signs. This relies on an important combinatorial result which goes back to the great French mathematician Cauchy, one of the main architects of the theory of determinants. The result is called the parity theorem. The parity theorem states that the number of transpositions required to express a permutation is always odd or always even. Let me illustrate that with the 4x4 case. Suppose
16:52
we have a permutation of four things, obtained by swapping entries. Suppose we swap the 1 and the 3 in the first step, then maybe in the next step we'll swap the 3 and the 4, and in the next step, we'll swap the 1 and the 3 again. To go from the original to the final permutation, we made three swaps, an odd number of transpositions. The parity theorem states that if you want to go from one permutation to another, you can do it in many different ways using a lot
17:31
more transpositions if you want, but you will always require an odd number of transpositions, never an even number. So, if one way of doing it is even, all other ways are even; if one way of doing it is odd, all other ways are odd. This is a very important combinatorial result that often doesn't get the attention it deserves when the theory of determinants is established.
17:56
So, how does this figure in the definition? It turns out that the coefficient of a term is minus 1 to the power of the parity, the number of transpositions required to get the given permutation from the original permutation 1, 2, up to N. For example, if we start with A, B, C, D, E, F, G, H, I, and we want to get A, F, and H, we can think about getting that by swapping the two columns. So, we swap the two columns, then these positions go to those positions,
18:54
thinking about rooks on a chessboard. If we want to get B, F, G, then we have to do two swaps. If we swap these two columns, we would have rooks here and here, and if we swap these two columns, we would end up with rooks here and here. That would be two swaps of columns required to go from the original to this one, and that's why there's a plus sign there.
19:20
So, that's the general pattern. Once you've seen this formula and given this description, you can then generalize the story to the 4x4 and higher-dimensional cases. Now, there was an alternate way of thinking about things which was discovered by Vandermonde in the 1700s. He realized that the above determinant could be written as A times the 2x2 determinant formed by that little submatrix, and then minus B times the determinant you get by
20:09
eliminating the row and column through that B, so the determinant of D, F, G, H, which is D times H minus E times G, and then plus C times the 2x2 determinant there, plus C times D, H minus E, G. In this way, a 3x3 determinant could be reduced to calculating three 2x2 determinants and then taking some appropriate combination of them with weightings, which are also the other entries which haven't been used yet. This is an inductive way of
20:49
doing this. If you have a 4x4 determinant, Vandermonde realized you can write that as the sum of essentially four 3x3 determinants multiplied by the corresponding other entries.
21:04
This is a very important way of physically calculating determinants. The way computers will often do it is to reduce things in a recursive fashion by calculating a bigger one in terms of smaller ones. However, there's another kind of algorithm in this direction, which was introduced by another great mathematician, Laplace, which these days is not so widely known. I'd like to show you that as well.
21:37
This general strategy of taking a complicated problem and then reducing it to a bunch of similar but smaller problems is sometimes called dynamic programming. Here's another example of that kind of thing. This is due to Laplace, another very important 18th-century mathematician. He realized that you could expand a 4x4 determinant in terms of a bunch of products of 2x2 determinants. What Laplace does here is first choose, say, the top two and
22:15
then create all the possible 2x2 determinants that you can from those two rows. For example, we could create this 2x2 determinant, giving us a, b, e, f, and then the remaining rows and columns give us this sort of opposite 2x2 matrix, which also has a determinant. So, we can multiply this determinant by that one, and then we can look at other 2 by 2 determinants formed by the top two rows. For example, the AC and EG determinant, and then we would have to
22:53
multiply that by the corresponding block down here of the rows and columns that we haven't used. So, we would multiply this determinant by this determinant, but this time with a minus sign. Basically, every time we need to change two things, we should introduce a minus sign. So, there are six possible such products corresponding to the six ways of choosing a 2 by 2 block for those first two rows. This generalizes nicely to more general matrices. You can
23:33
decompose a 5 by 5 determinant into a lot of products of 2 by 2 and 3 by 3 determinants. This kind of manipulation was very popular in the 18th and especially the 19th centuries and culminated in the very important work by Thomas Muir called "A Treatise on the Theory of Determinants," written around the late 1800s and early 1900s. It has hundreds of pages of fabulous formulas involving determinants. 19th-century mathematicians developed a very deep theory about
24:13
determinants, which is a very interesting precursor to linear algebra. These days, we study linear algebra, and determinants are only a small chapter. In the 19th-century point of view, linear algebra was almost the theory of determinants; there wasn't much else that wasn't subsumed in one way or another by the theory of determinants. That's a good book to peruse if you're interested in this subject.
24:41
So, we come to the main architect of the modern theory of determinants, Augustin-Louis Cauchy, the great French mathematician who lived from 1789 to 1857. He is second only to Euler in terms of mathematical output and contributions to various subjects. In the theory of determinants, he was the key architect. He introduced systematic notation, for example, the double subscript notation like a_ij to stand for a matrix whose entries are a sub_i sub_j. He actually introduced
25:30
the term "determinant" and was responsible for discovering the fundamental fact about determinants. It's one fact that is more important than any other: the rule that the determinant of a product of two matrices is the determinant of the first matrix times the determinant of the second.
25:57
This involves matrix multiplication: two matrices, and the determinant of a product is the product of the determinants. Another important contributor was J.J. Sylvester, a British mathematician who lived from 1814 to 1897. He also made important contributions to American mathematics. He ended up teaching at Johns Hopkins University and founded the American Journal of Mathematics. He had an
26:34
earlier stint at the University of Virginia, which he left after clobbering a student who insulted him in class with a stick. The student was so shocked that he fainted, and Sylvester thought he had killed him, so he subsequently left Virginia not having been there very long. Sylvester made lots of contributions to the theory; he introduced the term "matrix," for example.
27:00
I'd like to talk a little bit about his work on resultants. He redefined the theory of resultants.
27:18
The theory of resultants is an interesting chapter in linear algebra, which unfortunately is not often taught to undergraduates these days. The basic idea is that we want to eliminate variables from a system of equations, but not necessarily a linear system; we're interested in polynomial systems. For example, here's the fundamental problem in the subject. We have two degree-n polynomials: f(x) and g(x), and we want to know if they have a common zero.
28:32
One way, of course, is to solve both of the equations and see if the zeros actually coincide. But for an nth-degree equation, that can be problematic, and people wanted to know if there was a faster way to figure out whether there's a common zero without actually solving these equations, which might not be possible. The question is how to see if there is a common zero or solution.
29:06
This kind of question had been examined by many people. Newton was interested in it, Euler discussed it, and there was a classical theory given by Bézout, who wrote around 1764. He wrote a book called "Cours de Mathématiques," where, among other things, he outlined a theory of how to figure these things out. The key concept is something called the resultant.
29:44
Let me illustrate how that works for this situation, and then we'll see how Sylvester came along and redefined this theory, introducing another aspect to it. Our problem is to try to eliminate variables between these two equations to see whether there is a common solution. Let's call the first equation "equation 1" and the second one "equation 2." Here's the strategy that was discovered: first, multiply equation 1 by the
30:24
coefficient of the highest degree term of equation 2, and equation 2 by the coefficient of the highest degree term of equation 1. Then, subtract the two resulting equations. Okay, clearly if we multiply this by B sub n and this one by A sub n and we subtract, the X to the n terms will disappear, and we'll have an equation of degree n minus 1. That's the crucial thing.
30:55
Then, the next step is to do something similar but a little bit more clever: multiply equation 1 by B sub n X plus B sub n minus 1 and equation 2 by A sub n X plus A sub n minus 1 and subtract.
31:27
Alright, I'll leave it to you to think about what happens there, but what happens is that again the higher degree terms cancel, and we end up with an equation of degree n minus 1. Then, the next step is to multiply one by B sub n X squared plus B sub n minus 1 X plus B sub n minus 2 and equation 2 by the corresponding A sub n X squared plus A sub n minus 1 X plus A sub n minus 2 and subtract. We get another equation of degree n minus 1. Ultimately,
32:06
we get n equations of degree n minus 1. Now, we view those equations as equations in the variables 1, X, up to X to the n minus 1. On the right-hand side of the equations, we always have zero because we are just taking linear combinations of these things. So, when we look at these n minus 1 equations in these n variables, there is a nonzero solution,
32:49
assuming that there's a common solution. If there's a common solution to this system, then the corresponding determinant of coefficients must be zero. Since we have a solution by assumption, the determinant of the coefficients must equal zero. This is then a polynomial relation. This determinant of coefficients that you get with these n minus 1 equations is a polynomial relation on the original coefficients a0 up
33:42
to a sub n and b0 up to b sub n, and that relation is called the resultant. So, let's have a look at a specific example. Suppose that our first equation is quadratic: a1 X squared plus b1 X plus c1 equals zero. The second equation is a2 X squared plus b2 X plus c2 equals zero. Alright, what are we supposed to do? We're supposed to take equation 1, multiply by a2, and subtract a1 times equation 2. That gives us: a2 b1 minus a1 b2 X plus a2 c1 minus a1 c2 equals zero.
34:32
Then, the next step: a2 X plus b2 times equation 1 minus a1 X plus b1 times equation 2 will give us: a2 c1 minus a1 c2 X plus b2 c1 minus b1 c2 equals zero. So, in this case, we get two equations in the variables 1 and X. In order for this to have a solution, which we're assuming, the determinant here must be zero. So, the condition for a common solution is that the determinant of: a1 b1 a2 b2 times b1 c1 b2 c2
35:57
equals the determinant of: a1 c1 a2 c2
36:03
This is a beautiful relation that is necessary for these two general quadratic equations to have a common solution. If this equation is satisfied, then there's a common solution.
36:26
Once you know this condition, you can actually solve for X. The X is then:
36:38
(the determinant of a1 c1 a2 c2) divided by (the determinant of a1 b1 a2 b2).
36:50
This polynomial relation was called a resultant by 18th-century mathematicians, and they generalized that to not just 2 by 2 situations but to higher degrees and even when the two polynomials are not of the same degree. Then, Sylvester came along and discovered a remarkable alternate way, perhaps a more efficient way, of writing down what this resultant is.
37:22
The formula that Sylvester discovered involved a big determinant. In general, if the equations are degrees m and n, then we need an (m plus n)-squared determinant.
37:50
Let me illustrate it in the case of our system that we had before. In the case of 2 plus 2, we need a 4 by 4 matrix. The first equation had coefficients a1, b1, and c1, so we write down those coefficients in a row. In the next row, we take those same coefficients and move them over by one. At that stage, we've bumped into the right-hand side,
38:25
so we stop with that first equation and go to the second equation, which was a2, b2, and c2, and do the same stepwise translation to the right by one each time. If necessary, we have to go quite a few steps, each time moving it down to the right by one until it bumps into the wall. We end up with this 4x4 matrix formed by the coefficients of the two equations, and the relation is that the determinant of this should be zero. That's a polynomial
39:02
relation satisfied by the coefficients of the two polynomials that ensures they have a common zero. Can we see our earlier formula from here? Well, we can if we look at it from the point of view of Laplace. If we think about expanding this thing using these two rows, we could use that 2 by 2 block and multiply it by the corresponding determinant of that 2 by 2 block. Then, we'd have to go to another 2 by 2 block contained in these top two rows, say a1 c1 a2 c2. We could do this
39:58
one, this one, this one, and this one, and then the corresponding opposite one would be this one, this one, this one, and this one with a minus sign. A little thought shows that there are no other 2 by 2 determinants formed by these two rows because of that column of zeros at the end. So, Laplace's equation would give us immediately the product of that first determinant times a1 b1 a2 b2 determinants minus this a1 c1 a2 c2 determinant squared equals zero.
40:42
It's a lovely and very useful formula of Sylvester because this problem actually comes up quite a lot in computational algebraic geometry and other applications. The story of determinants, of course, extends beyond that. There were a lot of other things involved. Determinants were being used as volumes. If you want to compute the volume of a parallelepiped in three-dimensional space,
41:20
then you take a determinant of a matrix formed by the three vectors. There was also the Cayley-Hamilton theorem, which gave an algebraic relation identified by a general matrix, leading to the theory of eigenvalues and eigenvectors.
41:56
Generally, what happened in the 20th century is there was a shift from the 19th-century orientation to quite a different orientation. In the late 19th century, vectors started emerging as an important way of thinking about engineering and physical problems, originally motivated by Hamilton's quaternion theory but then ended up having a life of its own.
Here to help you learn
Sign in to keep a private conversation with your course tutor.
Ask me anything about the material…
Sign in to chat