The Shoelace Formula

The shoelace formula, sometimes referred to as Gauss's area formula, is an algorithm that can be used to calculate the area of a polygon with known Cartesian coordinates for it's vertices. It is called the shoelace formula because there is a way to visualize the pattern of multiplications that resembles a shoelace.

We will explain why the formula works later on this page.

Let's first consider the following right triangle:

We could calculate the area of this triangle using \(A=\dfrac{bh}{2}\). We can see from the figure that the height of the triangle is \(7\) and the base of the triangle is \(4\). Therefore, the area of the triangle is \(14\) square units.

Now let's use the shoelace formula to calculate the area.

Step 1: List the points vertically, starting and ending at point \(A\), moving in counterclockwise order.

Sum all of these values and subtract from the value found in Step 2. Half of this value is equal to the area of the triangle.

Notice that if we combine the images from Step 2 and Step 3, the operations resemble a tied shoelace.

If we substitute the values above into the shoelace formula we can combine Steps 1 through 3. 

\(\begin{align*} A&=\dfrac{1}{2} \left ( \class{hl1}{\Big((2)(-2)+(2)(-2)+(6)(5)\Big)} - \class{hl2}{ \Big((5)(2)+(-2)(6)+(-2)(2)\Big)} \right)\\[5px] A&=\dfrac{1}{2} \Big(22-(-6)\Big)\\[5px] A&=\dfrac{1}{2}\Big(28\Big)\\[5px] A&=14 \end{align*}\)

Notice that the shoelace formula has the same result as using \(A=\dfrac{bh}{2}\)!

Explaining the Shoelace Formula

Let's consider the following triangle which has been enclosed in a rectangle of smallest area with all sides parallel to the axes.

We will denote the area of a figure using vertical bars. For example, \(\mid \triangle ABC\mid\) is the area of \(\triangle ABC\).

To calculate \(\mid \triangle ABC\mid\), we can subtract \(\mid\triangle ADC\mid\), \(\mid\triangle AEB\mid\), and \(\mid\triangle CFB\mid\) from \(\mid DEBF\mid\).

The side lengths of \(DEBF\) are equal to \((x_2-x_1)\) and \((y_3-y_2)\). Therefore, 

\(\mid DEBF\mid =(x_2-x_1)(y_3-y_2)=x_2y_3-x_2y_2-x_1y_3+x_1y_2\)

Similarly, we can calculate \(\mid\triangle ADC\mid\), \(\mid\triangle AEB\mid\), and \(\mid\triangle CFB\mid\) as follows:

• \(\mid\triangle ADC\mid=\dfrac{1}{2}(x_3-x_1)(y_3-y_1)=\dfrac{1}{2}(x_3y_3-x_3y_1-x_1y_3+x_1y_1)\)
• \(\mid\triangle AEB\mid=\dfrac{1}{2}(x_2-x_1)(y_1-y_2)=\dfrac{1}{2}(x_2y_1-x_2y_2-x_1y_1+x_1y_2)\)
• \(\mid\triangle CFB\mid=\dfrac{1}{2}(x_2-x_3)(y_3-y_2)=\dfrac{1}{2}(x_2y_3-x_2y_2-x_3y_3+x_3y_2)\)

We know 

\(\mid\triangle ABC\mid=\mid DEBF \mid - \mid\triangle ADC\mid - \mid\triangle AEB\mid - \mid\triangle CFB\mid\)

We can now substitute as follows:

\(\begin{align*} \lvert\triangle ABC \rvert &= x_2y_3-x_2y_2-x_1y_3+x_1y_2 \\[2px] & \hspace{60px} -\dfrac{1}{2}(x_3y_3-x_3y_1-x_1y_3+x_1y_1)\\[2px] &\hspace{60px} - \dfrac{1}{2}(x_2y_1-x_2y_2-x_1y_1+x_1y_2)\\[2px] & \hspace{60px} - \dfrac{1}{2}(x_2y_3-x_2y_2-x_3y_3+x_3y_2) \end{align*}\)

Collecting like terms and simplifying, we get:

\(\begin{align*} \mid\triangle ABC \mid &= \dfrac{1}{2}x_2y_3-\dfrac{1}{2}x_1y_3+\dfrac{1}{2}x_1y_2+\dfrac{1}{2}x_3y_1-\dfrac{1}{2}x_2y_1-\dfrac{1}{2}x_3y_2\\[5px] &=\dfrac{1}{2}(x_2y_3+x_1y_2+x_3y_1)-\dfrac{1}{2}(x_1y_3+x_2y_1+x_3y_2)\\[5px] &=\dfrac{1}{2}\Big((x_1y_2+x_2y_3+x_3y_1) - (y_1x_2+y_2x_3+y_3x_1)\Big) \end{align*}\)

which is equivalent to the shoelace formula for triangles.

A complete proof of the shoelace formula would involve looking at all possible cases. For example, what happens if point \(C\) is to the right of point \(B\), meaning \(x_2\lt x_3\)?

Let's now look at a quadrilateral which can further be extended to other polygons by breaking them into more triangles.

Consider the following figure. It has the same coordinates from \(\triangle ABC\) above, but we have added a fourth point \(D(x_4,y_4)\) to create the quadrilateral \(ABDC\). Notice now that in counterclockwise order our points are not in numerical order (i.e., \((x_4,y_4)\) comes before \((x_3,y_3)\)). 

Now we will extend the result from finding our triangle areas above, to a quadrilateral by dividing the quadrilateral into triangles.

Using the triangles \(\triangle ABC\) and \(\triangle BDC\), working counterclockwise, and using the results from above we get:

• \(\begin{align*} \mid\triangle ABC \mid &=\dfrac{1}{2}\Big((x_1y_2+x_2y_3+x_3y_1) - (y_1x_2+y_2x_3+y_3x_1)\Big) \end{align*}\)
• \(\begin{align*} \mid\triangle BDC \mid  &=\dfrac{1}{2}\Big((x_2y_4+x_4y_3+x_3y_2)-(y_2x_4+y_4x_3+y_3x_2) \Big) \end{align*}\)

Adding the two areas gives us:

\(\begin{align*} \mid ABDC \mid &=\dfrac{1}{2}\Big((x_1y_2+x_2y_3+x_3y_1) - (y_1x_2+y_2x_3+y_3x_1)\Big) \\[2px] &\hspace{60px} + \dfrac{1}{2}\Big((x_2y_4+x_4y_3+x_3y_2)-(y_2x_4+y_4x_3+y_3x_2)\Big)\\[5px] &=\dfrac{1}{2}(x_1y_2+x_2y_3+x_3y_1-y_1x_2-y_2x_3-y_3x_1 \\[2px] & \hspace{60px}+x_2y_4+x_4y_3+x_3y_2-y_2x_4-y_4x_3-y_3x_2)\\[5px] &=\dfrac{1}{2}(x_1y_2+x_3y_1-y_1x_2-y_3x_1+x_2y_4+x_4y_3-y_2x_4-y_4x_3)\\[5px] &=\dfrac{1}{2}(x_1y_2+x_3y_1+x_2y_4+x_4y_3-y_1x_2-y_3x_1-y_2x_4-y_4x_3)\\[5px] &=\dfrac{1}{2}\Big((x_1y_2+x_2y_4+x_4y_3+x_3y_1)-(y_1x_2+y_2x_4+y_4x_3+y_3y_1)\Big) \end{align*}\)

which is again equivalent to the shoelace formula for a quadrilateral.