| Projection and Least Squares |
This page is an over-simplified preview of projection and least squares (Ch 5.3 etc). It should also give you some practice with words from earlier chapters.
What
do you see? Well, my graphics
program makes it look 3D, but you can think of it as the usual
xy-plane (or x1,x2 plane, actually). It doesn't really matter
much. The red line is the subspace
"S", where x1 = x2 (which means adding 2 vectors in
this set gives a third vector that's still in the set; see Ch
3.2). The green vector was chosen randomly, and is [3,9]^T.
(Sorry! column vectors are hard to type). It's not in S.
The blue vector [6,6]^T is in S, and is "as close as possible" to the green one. Believe me? That is, the distance between the 2 blue balls is minimal. In fact, this distance is the square root of (6-3)^2 + (6-9)^2. We say that the blue vector solves a (very simple) least squares problem. We also call the blue vector the projection of the green vector onto S. Projection is another example of a linear transformation (a function that changes vectors into other vectors - see Ch 4.1). You might also notice that the vector from (6,6) to (3,9), namely [-3,3]^T, is orthogonal to S.
Why would we want to solve such a problem? You will see many examples in Ch 5, perhaps even involving signal processing (pg 282). Most involve approximation in some way. One possible story is that Jane estimates her father to be x1 =3 feet tall, and Jose estimates him to be x2 = 9 feet tall, which is the green vector. But these numbers are inconsistent, because they should be equal, x1 = x2. They should be in the red subspace S. So, we'll have to change one or both numbers. We assume they both have some accuracy, and want to change them as little as possible. The solution is to make x1 = 6 and x2 =6.
How to solve it? One general procedure is to set it up as an inconsistent system, Ax = b, [the "A" is related to the subspace "S", the "x" is the blue vector, and the "b" is the green one). Then use the formula of Thm 5.3.2, which we'll discuss later. For this particular example, the simplest solution is to just average the two coordinates of the green vector [ex: (3+9)/2 = 6]. There are other approaches.
Please read this over a few times, and try to retain some of the vocabulary. But do not assume that you know anything yet! I haven't scratched the surface. Learning the precise definitions and using them correctly will be quite important.