Orthogonal Sets
Summary
orthogonal set: A set of vectors in is said to be an orthogonal set if
In many cases we like our bases to be orthogonal (that is, the vectors to be mutually perpendicular). Even better are orthonormal bases, in which the orthogonal vectors are of unit length.
Theorem 4: If is an orthogonal set of nonzero vectors in , then S is linearly independent and hence is a basis for the subspace spanned by S.
orthogonal basis: an orthogonal basis for a subspace W of is a basis for W that is also an orthogonal set.
Theorem 5: Let be an orthogonal basis for a subspace W of . For each in W, the weights in the linear combination
are given by
Whoops! We have a notational collision: the author wants to use the ``hat'' symbol to indicate the orthogonal projection of y onto another vector. I don't like this, because the vector is different for different vectors. Mathworld (maintainers of Mathematica), many other mathematicians, and I like to reserve the ``hat'' for unit vectors
To make your lives easier, I'll give up my notation, albeit unhappily. I'll indicate unit vectors by the notation : hence,
So
I prefer this right-most form of the projection, as it makes clear what's going on: we form a unit vector in the direction of u, cast a shadow along this unit vector using the inner product, and then weight the normal vector by this coefficient. This corresponds to the ``shadow'' cast by the vector onto the of vector u. Then we can write
where z is orthogonal to u. We can rewrite equation 1 as
which just says that we break vector y into its components along the orthogonal direction to represent it. This is what we do with our ordinary basis of vectors , , and .
The fact of the matter is that orthonormal bases are used more often than orthogonal bases, so we generally are working with normalized vectors.
Theorem 6: An matrix U has orthonormal columns if and only if .
Theorem 7: Let U be an matrix with orthonormal columns, and let x and y be in . Then
Exercise #25, p. 393
Orthogonal matrix: a square matrix such that , having orthonormal columns. It's ironic that the name is ``orthogonal'', rather than ``orthonormal''. Feel free to call such a matrix an orthonormal matrix.
Curiously enough, orthonormal columns in an orthogonal matrix imply that the rows are also orthonormal:
Example: #28, p. 393