In this post, I want to talk about something called an orthogonal complement. The set is the orthogonal complement of the set if for all and for all , . We write this as since everything in is orthogonal to everything in .
Note that the definition of orthogonal complements is symmetric, meaning that if , then we also know that . This is because the dot product is commutative ().
Let’s look at an example of a set and its orthogonal complement. Consider the plane through the origin in This can be written as span, or any vector that has as its component. So, the orthogonal complement of the plane is any vector of the form , where .
Now let’s look at an example of orthogonal complements in the context of matrix multiplication. To do this, we’ll relate two features of a matrix, the null space and the row space.
Consider an matrix , which represents a transformation from to . The null space of , written is the set of all such that . The row space of is the set of all linear combinations of the rows of .
To understand why these two sets are orthogonal to each other, let’s write out matrix multiplication in a way that makes it easier to see. We’ll do this with an example of matrix, but the idea extends to a matrix with any dimensions.
We would normally understand this matrix vector multiplication as a weighted sum of the columns, or , but we are going to look at it a different way.
Then, we can rewrite as .
We can see that these two ways of multiplying matrices are the same because, for example, the first component of the final vector we just output is . And if you multiply the ‘s through in the first definition of matrix multiplication above, you’ll get the same thing for the first component of the vector.
Now, let’s consider the vector . Then, . Since is in the null space of , it must be orthogonal to each row that matrix.
Now, all we need to show is that if a vector is orthogonal to each row, then it is orthogonal to any linear combination of those rows.
Suppose is orthogonal to every vector in the set . Then, let’s compute (the right side of the dot is any vector in the span of the s). We can rewrite this as . We know that for , so this dot product is zero.
Thus, is orthogonal to any vector in the row space, and not just each vector that spans the row space.
Overall, we showed that if , then .
In the next post, we’ll connect this back to the Rank-Nullity theorem by talking about how the row space and column space of a matrix relate to each other.