In this post, I want to talk about a way to connect critical points of two variable polynomials of degree to the null space of a matrix. A critical point of a function is a place where all partial derivatives of that function are equal to zero. All polynomials like this are of the form , where . As a reminder, the null space of a matrix is the set .

Let’s look at a few examples.

Example 1: Let’s find the critical points of the polynomial . To do this, we will find all partial derivatives of . First, we see that and . So, to find the critical points of , we need to solve and .

This is a system of two equations with two unknowns! Let’s rewrite it using matrices. .

This matrix is invertible, since , so the Invertible Matrix Theorem tell us that the only vector that solves this equation is .

This means that the only critical point of occurs at the point .

Now let’s look at another example with a different polynomial.

Example 2: Let’s find the critical points of the polynomial . The partial derivatives of are and . This means we want to solve and .

That’s just the same equation twice! So, the corresponding matrix equation to examine is .

This matrix is not invertible since , which, by the Invertible Matrix Theorem, means that this matrix equation has infinitely many solutions.

That also tells us that the corresponding polynomial has infinitely many critical points! If you graph this polynomial it makes a lot of sense. There’s a flat line along , which all make attain the same value.

What’s cool about this is that there is a pretty simple way of finding critical points of a two variable quadratic function using linear algebra. And solutions to matrix equations like the ones we made will always either have no solution, one solution, or infinitely many solutions (that’s just a fact about systems of linear equations). Since we’re only every setting the left hand side of the matrix equation equal to a vector of zeros, we will never be in the “no solution” case (i.e. will be a valid solution).

This means that polynomials of this form will always either have a single critical point OR infinitely many critical points. (If you had a polynomial with critical points, for example, you would know that it is not a quadratic polynomial.) This is fun to think about because critical points are either local minima, maxima, or saddle points. By just finding the matrix associated with a polynomial like this, you can understand the shape of that polynomial without having to try and visualize it!

In this post, I want to talk about the equation for the variance of a random variable use this to get a geometric intuition behind some properties of random variables. First, let’s define what a random variable is.

A random variable is a function that takes in a set of possible outcomes and outputs the probability of that particular outcome. For example, we could define a random variable that takes the value with probability and takes the value with probability .

Once you have a random variable, there are some things you might want to know about it. For example, you might be curious what its expected value is. This is like an average, but it extends to random variables that take on a continuous set of values as well. To find the expected value in this case, we calculate . For a discrete random variable, the expected value is really a weighted average of the possible values of the random variable (weighted by the probability of occurring).

The expected value of random variable is often written as . Once you know how to calculate this, you might ask what happens if you transform the values that can take and then recalculate expected values. In particular, you might ask what happens if you calculate , or in general, for some . For any value of , these are called “moments of a distribution,” and they tell you about properties of the distribution.

In our example above, to calculate , we just calculate . In this case, since the possible values of the random variable were and , we get the same answer if we square them and recompute the expression. So, in this case, but that won’t always be true.

The variance of a random variable, which tells you about how much the values of the random variable are spread out, is defined as , where is the standard deviation of .

Now let’s do something kind of fun and rearrange that equation a little bit. If we add to both sides, we get that .

Looking at it like this, we can imagine a right triangle with legs of length and and hypotenuse of length .

If we let be a moment generating function for , then we can replace with and with . That’s kind of fun!

I hope this helped you care more about triangles and probability!

In the last post, we talked about what the orthogonal complement of a set is and we showed why . This was kind of cool because for me, it’s hard to have a geometric understanding of the row space of matrix based on how matrix multiplication is usually defined ( is kind of hard to picture) but it’s not that hard for me to have a geometric understanding of the null space of a matrix. The fact that they are orthogonal means that if you can understand what it means to be in the null space, then to be in the row space just means that you are orthogonal to everything in the null space.

Now, let’s talk about how to connect the fact that the to the Rank-Nullity Theorem. As a reminder, the Rank-Nullity Theorem says that for an matrix , .

The first fact that we need to talk about before starting to understand why this is true is that for an matrix , . This should seem really surprising at first. The row space of lives in and the column space of lives in ! So, the dimension of the row and column spaces are the same, but they don’t even live in the same ambient dimensions! That’s pretty crazy, if you ask me.

Now, let’s look at an example to see why this makes sense. Consider the matrix . This represents a transformation from to , and every output is of the form , where represents any real number. In this case, the output lives in , but the output itself is only dimensional (since there’s only one vector that spans the output).

Now, let’s consider the matrix . This represents a transformation from to , and every output is of the form , where represents any real number. In this case, the output lives in , but the output itself is only dimensional (since there’s only one vector that spans the output).

So, even though the transformations given by and don’t map into the same dimension, the same number of vectors can still be used to represent their outputs! That’s so cool!

Now, let’s connect this to the Rank-Nullity Theorem. The Rank-Nullity Theorem gives a connection between the dimension of the column space of matrix and the dimension of the null space of a matrix. We just did an example that showed why dimension of the column space of a matrix is the same as the dimension of the row space of a matrix. So, if is an matrix, let’s rewrite the Rank-Nullity Theorem as . This has a nice geometric interpretation since any deficiency in the size of the row space is made up for by the size of the null space. It’s amazing how nicely they work together.

I hope this made you appreciate what it really means to be a matrix just a little bit more!

In this post, I want to talk about something called an orthogonal complement. The set is the orthogonal complement of the set if for all and for all , . We write this as since everything in is orthogonal to everything in .

Note that the definition of orthogonal complements is symmetric, meaning that if , then we also know that . This is because the dot product is commutative ().

Let’s look at an example of a set and its orthogonal complement. Consider the plane through the origin in This can be written as span, or any vector that has as its component. So, the orthogonal complement of the plane is any vector of the form , where .

Now let’s look at an example of orthogonal complements in the context of matrix multiplication. To do this, we’ll relate two features of a matrix, the null space and the row space.

Consider an matrix , which represents a transformation from to . The null space of , written is the set of all such that . The row space of is the set of all linear combinations of the rows of .

To understand why these two sets are orthogonal to each other, let’s write out matrix multiplication in a way that makes it easier to see. We’ll do this with an example of matrix, but the idea extends to a matrix with any dimensions.

.

We would normally understand this matrix vector multiplication as a weighted sum of the columns, or , but we are going to look at it a different way.

Then, we can rewrite as .

We can see that these two ways of multiplying matrices are the same because, for example, the first component of the final vector we just output is . And if you multiply the ‘s through in the first definition of matrix multiplication above, you’ll get the same thing for the first component of the vector.

Now, let’s consider the vector . Then, . Since is in the null space of , it must be orthogonal to each row that matrix.

Now, all we need to show is that if a vector is orthogonal to each row, then it is orthogonal to any linear combination of those rows.

Suppose is orthogonal to every vector in the set . Then, let’s compute (the right side of the dot is any vector in the span of the s). We can rewrite this as . We know that for , so this dot product is zero.

Thus, is orthogonal to any vector in the row space, and not just each vector that spans the row space.

Overall, we showed that if , then .

In the next post, we’ll connect this back to the Rank-Nullity theorem by talking about how the row space and column space of a matrix relate to each other.

The goal of linear regression is to find the equation of the polynomial that most closely approximates a set of data points. It turns out that you don’t actually have to restrict yourself to looking at equations where the highest degree on is 1, but we can talk about that some other time. Linear regression is really useful in settings where you’re trying to figure out how one (several) variable(s) impact(s) a single output variable.

To perform linear regression, though, we need to make some pretty strong assumptions about our data. One of these assumptions can be stated as follows:

In case you haven’t seen the notation before, means “the probability of Y given X.” This comes up a lot when you want to know how the probability of an event changes given that you know whether something already happened. An example of this would be “the probability that I get at A in the class given that I got an 87 on the midterm.”

At first, it seems like this would contradict the definition of conditional probability, which says that

,

but let’s talk about why it actually makes sense in this context.

When you perform a linear regression, there are kind of two ways to think about what’s going on: prediction and simultaneous measurements. Let’s think about a specific example to make this easier to talk about.

In this example, let’s collect people’s heights and weights and plot them on the and axes respectively. One way to think about this is that you are trying to find an equation that uses heights to predict weights. Said a different way, you are trying to maximize . We’ll come back to this idea in the next post.

Another way to think about this is that every time you observe a person, you collect two pieces of information about them: their height AND their weight. Here, INSTEAD of thinking about using height to PREDICT weight, we’re just simultaneously measuring two attributes of a person.

In this example, we see how we can think about either using people’s height to predict their weight, or just think about measuring people’s heights and weights simultaneously and understanding how those two values are related.

In the next post, we’ll say a little more about the assumptions that linear regression makes and why they make sense! Stay tuned!

I want to talk about some intuition behind the Rank-Nullity Theorem. In particular, let’s talk about why, even though the null space and column space of an matrix live in different ambient dimensions ( and respectively) it still makes sense why there is a connection between them. As a reminder, here’s the Rank-Nullity Theorem.

Theorem: If an matrix has a column space of dimension , then it has a null space of dimension .

The rank of a matrix tells you how many columns in the matrix are actually contributing to the dimension of the column space. For example, the matrix has a dimensional column space. Since has two rows, the column space could be at most . Since the second two columns are just multiples of the first, though, neither of them provide any new information. In other words, those columns are redundant.

Now let’s examine the redundancy of the columns of by looking at its null space. First, let’s note that we can write . This also means that we can write since multiplying a matrix by a vector is just taking a weighted sum of the columns of the matrix. Then you can just move one vector to the other side of the equals sign.

This means that finding a nonzero vector in the null space of a matrix is equivalent to writing some of the columns of the matrix as a linear combination of the other columns. In other words, not all of these columns are needed to tell you everything you wanted to know about the matrix.

I hope this helped you see how redundancy in a matrix can be seen through a not fully spanning column space AND a non-trivial null space!

In this post, I want to talk about a way to think about the Fundamental Theorem of Calculus that will hopefully give more intuition for it. As a reminder, the Fundamental Theorem of Calculus describes the connection between differentiation and integration, which are the two main operations in calculus. This theorem comes in two parts: one that says what happens when you differentiate an integral, and one that says what happens when you integrate a derivative. Let’s state the two parts now.

First Fundamental Theorem of Calculus:

Second Fundamental Theorem of Calculus: .

Let’s start by talking about the First Fundamental Theorem of Calculus. In this, we are differentiating an integral. To understand what this is saying, let’s break it down into parts. The integral is calculating how much of some quantity has accumulated by a certain point in time (this could be any quantity, like distance traveled, water in a container,…). Then, the derivative is asking about how much the total quantity is changing at a particular moment. For example, how much is the amount of water in the container increasing right this second?

The First Fundamental Theorem of Calculus says that the answer to that question is just equal the function evaluated at the upper bound of the integral, or .

Let’s think about why this makes sense. The integral is keeping track of how much total has accumulated until a certain point. Then, the derivative comes along and says “how much are you changing right this second?” The answer to that question should just be the current value of the function at that the time , or . You would at least expect the rate at which the area under a function increases to be proportional to the height of the function, and the First Fundamental Theorem of Calculus just says that the proportionality constant is equal to .

The Second Fundamental Theorem of Calculus says that if you want to find the total effect that a function’s derivative has over a region , you just have to compute the difference in the function’s values at those two points. This makes sense because if someone asks what the total effect of the slope of a mountain was in a region, you would just tell them how high you climbed in that region, or the difference between the height of where you ended and where you started.

Alright, now let’s talk about the fun stuff. Both parts of the Fundamental Theorem of Calculus give connections between differentiation and integration. Even though it’s not too hard to understand some intuition behind why each part of the theorem is true, there’s actually something pretty cool going on here! Differentiation is really asking a local question about a function. It’s asking, “How steep is the function at this particular point?” Whereas integration is really asking a global question about a function. It’s asking “In this region, how much have we accumulated by a certain point?”

It’s interesting that these two topics are related because it means that the Fundamental Theorem of Calculus acts as a way to connect the local behavior of a function to the global behavior of that function!

The concept of local/global statements about an object comes up a lot in Differential Geometry, which studies curves and surfaces. A lot of theorems in Differential Geometry talk about which properties of a surface you can measure from the curve/surface (local) (these are called intrinsic properties) and from observing the curve/surface from a distance (global) (these are called extrinsic properties).

So, the Fundamental Theorem of Calculus is kind of a little peek into how local and global properties of functions are related to each other!

Let’s talk about a way to visualize the orientations of the Rubik’s as a graph (the kind with vertices and edges, like the ones shown below). Then, we’ll use this representation to talk about what happens when you repeat a fixed set of moves over and over again on the Rubik’s cube.

The dots labeled with letters are called nodes and the lines connecting the nodes are called edges.

In the case of the Rubik’s cube, each node of the graph represents an orientation of the cube, and an edge connecting two cubes represents the move that takes one orientation to the other. It might be easier to imagine two directed edges (specify at which node it starts and ends) between each node, where the move on one of the edges undoes what the move on the edge in the opposite direction does.

Now, let’s just start picturing what this massive graph looks like. First of all, the graph has over 43 quintillion nodes, since there are roughly that many orientations of the Rubik’s cube. To simplify our graph a little, let’s not allow turns that directly affect any of the center pieces. Let’s also only consider edges that correspond to quarter turns.

At each orientation of the cube, you have 8 different options for how to move to another orientation (since there are four different sections you can rotate and each of them can be rotated two different directions.)

So, imagine building this graph. Out of each node, there are 8 edges. HANG ON… If we did that forever, this graph would have infinitely many nodes! And that would correspond to an infinite number of Rubik’s cube orientations. There ARE a lot of Rubik’s cube orientations, but there are definitely NOT infinitely many Rubik’s cube orientations. So, what does this tell us? It tells us that there must be closed loops within our graph! So, even if you think you’re getting farther and farther away from the correct solution, if you follow some path along the graph, you can only ever get so far away from the node corresponding to a solved cube. Another way of saying that is that given some starting orientation and some ending orientation (our target), there must more than one way of getting from the start to the end!

What else does the loopiness of this graph tell us? Let’s consider a finite set of moves that we repeat over and over again. For example, in Rubik’s cube notation, let’s consider the combination of moves (each letter corresponds to a different twist of the cube, and the prime symbol means turn it the way that’s not implied). Whereas before, each edge of the graph corresponded to a single twist of the cube, now each edge represents multiple twists. But, here’s the crazy thing…this graph will still have loops in it no matter what finite set of moves you repeat! It has to, since the graph has finitely many edges!

This is true because the only option is for it to either visit every node of the graph (orientation of the cube) or eventually come back to the starting orientation before hitting all of them. Since the set of moves you’re repeating is fixed, you’ll never jump around the graph and not get back to where you started.

This is so cool! Given any fixed set of twists of the cube, you know that you can repeat them and eventually get back to where you started!

In this post, I want to talk about synthetic polynomial division and why the final value you get is BOTH the value of the function at a particular point AND the remainder when you perform the division. We’ll use one running example with coefficients , and even though our example will be with a cubic polynomial, the same idea generalizes to polynomials of any degree.

Let’s let , and lets let . We want to compute . To do this, we’ll set up synthetic divison as follows.

Inside the box, we put the value that makes the denominator zero. One of the assumptions of synthetic division is that the highest degree on the term is . So, if is inside the box, we know that we are dividing by .

To the right of the box, we put the coefficients of the polynomial in question (in decreasing order of exponent). If there were a term missing as the powers decreased (like if the polynomial were (no term)), we would put a zero in that position.

Now, let’s do some synthetic division. This is a combination of two steps: add and multiply. In the first step, we’ll just be bringing down the first constant, but that’s the same as adding zero to the first constant. Here’s the result if you carry out the synthetic division in this example.

The last term in the last column above the horizontal line that’s hard to read is .

The expression in last column of synthetic division below the blue line always gives the remainder of the polynomial division. However, we can ALSO see that this expression is the same as , or the function evaluated at .

Hang on, that’s pretty cool. The remainder of the polynomial division by is the same as value of the function at .

Now let’s talk about why the final term being zero when plugging in corresponds to being a factor of . If , then we can write as a product of two polynomials, , where is some other polynomial (Note that if you evaluate , you’ll get ).

Even if the term you’re dividing by is not a factor of , we see that synthetic division still gives you the $y-$value at that point. Thinking in terms of remainders, this also makes a lot of sense. The remainder after division can be thought of as a measure of how far the denominator was from being a factor of the numerator. If, at , the value of the function was , then the numerator was more than a multiple of the denominator. In other words, if you subtract from the numerator, it will be divisible by the denominator with no remainder. And, this will have created a polynomial which IS a multiple of the denominator.

Let’s look at an example to make this a little more concrete. Let’s let , and let’s let . Then, the synthetic division will look like this.

The final value in the bottom right is $7$, so this should be the $y-$value of the function at . We can check that . Also, if the remainder is , then we should be able to subtract and obtain a multiple of . Let’s see if that works.

. Indeed, is a factor of that new polynomial!

I hope this helped you see a connection between zeros and factors of polynomials! And how the output of synthetic division relates to the numerator function’s value at a particular point!

Remember to post this on Facebook and tell your friends about it! Also, you can subscribe to posts so you can be the first to read the next one!

This post is a follow-up to the last post about the Fundamental Theorem of Algebra. As a recap, the Fundamental Theorem of Algebra says that if you have a polynomial of degree with coefficients in , then the polynomial will have roots in .

As we talked about last time, the Fundamental Theorem of Algebra is an existence proof because for a polynomial of degree whose coefficients are in , it says that you’ll be able to find values which, when plugged into the polynomial, make the polynomial equal zero. In other words, you’re guaranteeing the existence of the values that will make the polynomial equal zero.

Existence proofs are cool and all, but the Fundamental Theorem of Algebra doesn’t tell us anything about how to find the values that make the polynomial equal zero. Wouldn’t it be nice if there were a way to know where these zeros were?

It turns out that if we restrict ourselves to looking for rational zeros of a polynomial, then we can say exactly what those kinds of zeros would have to look like. Now, let’s state the Rational Root Theorem.

Rational Root Theorem: If a polynomial has any rational zeros, then they will have to occur at , where is some factor of and is some factor of .

To prove the Rational Root Theorem, we’re going to write a polynomial as a product of terms, which the Fundamental Theorem of Algebra guarantees we can do.

Proof: Let , where the “” includes the product of other linear terms. First, let’s find the zeros of this function. To do that, we want to solve . Note that we won’t always be able to get factors multiplied together if there are some factors that don’t yield rational roots. But even if you can’t, that gives you information, too! For example, there might be a polynomial with five roots, only three of which are rational.

Before we solve for the values of that make this function equal zero, let’s just think about what the highest degree term and lowest degree term will look like. We don’t need to think about the terms that have degree . The highest degree term comes from multiplying all of the terms that have an in them. And the lowest degree term comes from multiplying all of the terms that are just constants. This means that the highest degree term will be . And the lowest degree term will be .

Let’s remember that we’re trying to solve . Since the left-hand side is a product of terms that equals zero, the whole left-hand side will equal zero when any of these terms individually equals zero. So, this function has zeros when . 

Okay, we’re so close! We know that if our function has a rational zero, it will definitely be in the set , even though not all of these quotients have to be rational. And if we want to find these values, we just have to search the set of quotients , where is a factor of and is a factor of because that will check all of the values in , which is the set of all zeros.

This theorem says that IF a function has rational zeros, then here’s how to find them. But there’s no guarantee that a function will have rational zeros to begin with.

So, to recap: The Fundamental Theorem of Algebra states the existence of complex-valued zeros for a polynomial of degree , but is kind of like “good luck!” when it comes to actually finding these zeros. Then the Rational Root Theorem comes along and says, “hey, if you don’t mind only looking for rational zeros of this function, I can help you out!” It says that any rational root of your polynomial can be written as the quotient of two integers, where the numerator is a factor of the constant term and the denominator is a factor of the highest degree term.

I hope this helped you see a connection between the Fundamental Theorem of Algebra and the Rational Root Theorem!

Remember to pass this page along to your friends and ask them to share it!