From 2001 until the time I retired I taught Linear Algebra every spring. In the 1970's Linear Algebra was a gateway to higher math for math majors and was taught in a very abstract fashion. Through the years Linear Algebra became more applied, catering more to engineers and computer scientists. A second course in Linear Algebra includes all the theory that was previously taught in the first course, and then some. My course changed with the times.
I was pleasantly surprised in my last few years to have a fine bunch of computer science students. Some of them came into the class with a list of topics they wanted to learn. These bright students were well read in their field. There was this barrier that they could not cross, and it was mostly Linear Algebra. Linear Algebra had emerged as a central course for AI in general, and specifically for machine learning. Linear Algebra is also central for computer vision, computer graphics, gaming and robotics. Suffice it to say, Linear Algebra is a very important course, and its importance is growing as we speak.
A simple problem in Linear Algebra is to take a bunch of points that represent a surface in 3D, multiply each point by an appropriate matrix, and surprise, the new set of points is the original surface rotated about a given axis. Encoded in the rotational matrix is the axis of rotation and the number of degrees of the rotation. Humans see by viewing two 2D images - one rotated a fixed number of degrees from the other - and constructing a 3D image in our brain. In Linear Algebra we can replicate this for a machine.
Linear Algebra is also useful in just about every engineering course. It is not required in engineering because there is already 150+ hours required by ABET. Engineering students are faced with taking 153+ hours or learning the subject on the fly while they are taking 4 or 5 engineering courses in their junior year. A handful of engineering students ended up in Linear Algebra, although they were not as curious about the subject matter, mostly because they were curious about how they would survive Dynamics, Mechanics of Materials, Circuits and Differential Equations, along with Linear Algebra.
In my next-to-last semester I had one particular student who, on the first day, asked me if we were going to learn how to factor matrices using QR Decomposition, and I think one other method. There are several different types of factoring, all with specific uses, and multiple algorithms for each type of factorization.
One use of factoring is to filter out noise. With a sound file we can literally filter out background noise using linear algebra and some other math, and a specific factoring technique. In machine learning, we sometimes want to ignore extraneous variables, and isolate the important ones. These extraneous variables are called noise. Factoring matrices allows us to do this. We can factor a matrix of data into a rule matrix, and a noise matrix.
As humans we make rules as we live in the world. We drive for a while, and decide that 15 mph is a good speed to make a right turn. We do not repeat an experiment - we take lots of little experiences and ignore what is not important, and make the rule, even if it is not a conscious rule. The human mind is good at making such rules. We go from facts to rules without even making a conscious effort. And so it is with matrix factorization: we can take a handful of driving experiences, and calculate a 15 mph rule for right turns. Human reasoning is replicated, although with different means. I seriously doubt children are born with matrices in their brain.
Scientists will isolate variables as much as they can, and do an experiment. Epidemiologists cannot always isolate variables because they do studies in hindsight. Thus, the epidemiologist has to do more sophisticated math to isolate the variables after the fact. And so it is with machine learning. We want the machine to be able to learn as it "experiences" its environment. Linear algebra helps us do that. We can factor a matrix of experiences into the product of two matrices, where one matrix gives us the rule, while the other has the noise we can ignore.
Anyway, a student asked me about a specific kind of matrix factoring, so I told him I would oblige. Weeks alter, he asked again, and I said not yet. I can't remember how many times it was brought up, but I stalled until the end of the semester.
When I did finally show the student how to do QR factorization, the algorithm I used was easily and fully understood by the whole class, because various elements were based on all they had learned throughout the whole semester. During Week 1 of the class, I would have lost them 5 minutes in. By week 15 The class not only understood how to do the procedure, but they fully understood why the procedure worked, because of all the theory that was covered in the class. A topic that was once nn impenetrable barrier was now a joy to learn!
I explained how pretty much anything else these students needed to know in Linear Algebra, at least applied to machine learning, etc., could likely be self taught and fully understood. The course I put together made it so. It was a great meta-lesson, brought about by the students' curiosity.
The above picture is from one of many short courses available for students entering various fields that require Linear Algebra. I used to pass one or two of these out, allowing students to follow along as we developed the course. My course always covered about 95% of each of these short courses. I told students they were welcome to ask any question about Linear Algebra that pertained to any other field, if they chose. Sometimes they would take me up on that offer, successfully. Through the years I got all sorts of crazy questions.
On a humorous note, I would always reserve the last day of class to "attack Canada." There was always a build up to this point, where I would promise to give them enough tools to launch an attack. The students, of course, knew I was not serious. It was very funny because it was such a preposterous idea.
It is easy to formulate such a problem where we are adding soldiers by breeding, and subtracting them due to combat loss and natural deaths. Since it is an easy problem to visualize, it is a good problem. The usual solution requires solving a linear, first order system of differential equations. Using eigenvectors (part of Linear Algebra,) the whole solution can be solved very simply.
Since our population is about 10 times that of Canada, we need to make Canadians great lovers and even better fighters to win a war. We had a lot of fun with that!
Eigenvectors are a central topic for all the various subjects listed above, and a subject that is often given little attention at universities. At the end of the semester we would bring to bear all the theory we learned, to give students a great understanding of such things.
No comments:
Post a Comment