Quadratic Optimization and Matrices

In this post, I want to talk about a way to connect critical points of two variable polynomials of degree to the null space of a matrix. A critical point of a function is a place where all partial derivatives of that function are equal to zero. All polynomials like this are of the form , where . As a reminder, the null space of a matrix is the set .

Let’s look at a few examples.

Example 1: Let’s find the critical points of the polynomial . To do this, we will find all partial derivatives of . First, we see that and . So, to find the critical points of , we need to solve and .

This is a system of two equations with two unknowns! Let’s rewrite it using matrices. .

This matrix is invertible, since , so the Invertible Matrix Theorem tell us that the only vector that solves this equation is .

This means that the only critical point of occurs at the point .

Now let’s look at another example with a different polynomial.

Example 2: Let’s find the critical points of the polynomial . The partial derivatives of are and . This means we want to solve and .

That’s just the same equation twice! So, the corresponding matrix equation to examine is .

This matrix is not invertible since , which, by the Invertible Matrix Theorem, means that this matrix equation has infinitely many solutions.

That also tells us that the corresponding polynomial has infinitely many critical points! If you graph this polynomial it makes a lot of sense. There’s a flat line along , which all make attain the same value.

What’s cool about this is that there is a pretty simple way of finding critical points of a two variable quadratic function using linear algebra. And solutions to matrix equations like the ones we made will always either have no solution, one solution, or infinitely many solutions (that’s just a fact about systems of linear equations). Since we’re only every setting the left hand side of the matrix equation equal to a vector of zeros, we will never be in the “no solution” case (i.e. will be a valid solution).

This means that polynomials of this form will always either have a single critical point OR infinitely many critical points. (If you had a polynomial with critical points, for example, you would know that it is not a quadratic polynomial.) This is fun to think about because critical points are either local minima, maxima, or saddle points. By just finding the matrix associated with a polynomial like this, you can understand the shape of that polynomial without having to try and visualize it!