Orthogonal Complements

In this post, I want to talk about something called an orthogonal complement. The set $B$ is the orthogonal complement of the set $A$ if for all $a\in A$ and for all $b\in B$ , $a\cdot b = 0$ . We write this as $A^{\perp} = B$ since everything in $A$ is orthogonal to everything in $B$ .

Note that the definition of orthogonal complements is symmetric, meaning that if $A^{\perp} = B$ , then we also know that $B^{\perp} = A$ . This is because the dot product is commutative ( $a\cdot b=b\cdot a$ ).

Let’s look at an example of a set and its orthogonal complement. Consider the $xy-$ plane through the origin in $\mathbb{R}^3$ This can be written as span $\left\{\begin{pmatrix}1\\0\\0\end{pmatrix},\begin{pmatrix}0\\1\\0\end{pmatrix}\right\}$ , or any vector that has $0$ as its $z$ component. So, the orthogonal complement of the $xy-$ plane is any vector of the form $\begin{pmatrix}0\\0\\k\end{pmatrix}$ , where $k\in\mathbb{R}$ .

Now let’s look at an example of orthogonal complements in the context of matrix multiplication. To do this, we’ll relate two features of a matrix, the null space and the row space.

Consider an $m\times n$ matrix $A$ , which represents a transformation from $\mathbb{R}^n$ to $\mathbb{R}^m$ . The null space of $A$ , written $Nul(A)$ is the set of all $x$ such that $Ax = \vec{0}$ . The row space of $A$ is the set of all linear combinations of the rows of $A$ .

To understand why these two sets are orthogonal to each other, let’s write out matrix multiplication in a way that makes it easier to see. We’ll do this with an example of $3\times 3$ matrix, but the idea extends to a matrix with any dimensions.

$\begin{pmatrix} 1&2&2\\3&-1&-2\\5&-4&-3\end{pmatrix}\begin{pmatrix}x_1\\x_2\\x_3\end{pmatrix}$ .

We would normally understand this matrix vector multiplication as a weighted sum of the columns, or $x_1\begin{pmatrix}1\\3\\5\end{pmatrix} + x_2\begin{pmatrix}2\\-1\\-4\end{pmatrix} +x_3\begin{pmatrix}2\\-2\\-3\end{pmatrix}$ , but we are going to look at it a different way.

Then, we can rewrite $\begin{pmatrix} 1&2&2\\3&-1&-2\\5&-4&-3\end{pmatrix}\begin{pmatrix}x_1\\x_2\\x_3\end{pmatrix}$ as $\begin{pmatrix}(1,2,2)\cdot (x_1,x_2,x_3)\\(3,-1,2)\cdot (x_1,x_2,x_3)\\(5,-4,-3)\cdot (x_1,x_2,x_3)\end{pmatrix}$ .

We can see that these two ways of multiplying matrices are the same because, for example, the first component of the final vector we just output is $x_1+2x_2+2x_3$ . And if you multiply the $x$ ‘s through in the first definition of matrix multiplication above, you’ll get the same thing for the first component of the vector.

Now, let’s consider the vector $\begin{pmatrix}1\\-2\\1\end{pmatrix}\in Nul\left\{\begin{pmatrix}1&2&3\\4&5&6\\7&8&9\end{pmatrix}\right\}$ . Then, $\begin{pmatrix}1&2&3\\4&5&6\\7&8&9\end{pmatrix} \begin{pmatrix}1\\-2\\1\end{pmatrix}=\begin{pmatrix}0\\0\\0\end{pmatrix}$ . Since $\begin{pmatrix}1\\-2\\1\end{pmatrix}$ is in the null space of $\begin{pmatrix}1&2&3\\4&5&6\\7&8&9\end{pmatrix}$ , it must be orthogonal to each row that matrix.

Now, all we need to show is that if a vector is orthogonal to each row, then it is orthogonal to any linear combination of those rows.

Suppose $\vec{x}$ is orthogonal to every vector in the set $\{\vec{a_1}, \cdots,\vec{a_m}\}$ . Then, let’s compute $\vec{x}\cdot (c_1\vec{a_1}+\ldots \cdot c_m\vec{a_m})$ (the right side of the dot is any vector in the span of the $a_i$ s). We can rewrite this as $c_1(\vec{x}\cdot\vec{a_1})+\ldots + c_m(\vec{x}\cdot\vec{a_m})$ . We know that $\vec{x}\cdot a_i=0$ for $1\leq i\leq m$ , so this dot product is zero.

Thus, $\vec{x}$ is orthogonal to any vector in the row space, and not just each vector that spans the row space.

Overall, we showed that if $\vec{x}\in Nul(A)$ , then $\vec{x}\perp Row(A)$ .

In the next post, we’ll connect this back to the Rank-Nullity theorem by talking about how the row space and column space of a matrix relate to each other.

Orthogonal Complements

Published by benjaminyellin6391

Leave a Reply Cancel reply

Share this:

Like this:

Related

Published by benjaminyellin6391

Leave a Reply Cancel reply