We’re all fatigued from talking about and making everyday concessions in our lives to COVID. However, given we are over 20 months into this pandemic and I have yet to hear of a proper evaluation metric for a region’s policies towards controlling the outbreak, I am compelled to put one forward.
This will NOT be an article criticizing or supporting any political stances towards COVID itself or the policies put in place. This is only critiquing and proposing how we should be looking at the data so we can make better decisions moving forward.
I will also not be going into hard math. In other words I won’t be actually grading different regions for their responses. Hopefully, the philosophy and approach advocated for in this piece finds its way to people who have access to the data and the technology to run the numbers.
Let’s look at a simple statement, which quite frankly was the inspiration for this article. “Florida has done better at protecting its population from COVID than New York.” The term “Florida” will be interpreted as being the government of Florida and the policies it put into place, since “protecting its population” conveys a direct relationship that the state has with its population.
If you were to look at COVID deaths per 100k in the population, New York has more than Florida: 322 vs 291.
Thus, using this data as supporting material, the statement above would be accurate. As we will see, this is FAR from the best way to measure COVID response.
When evaluating the impact of COVID on a region, there are external and internal factors. We will define an external factor as being a factor that is essentially outside of the control of the government, for better or for worse. An internal factor will be defined as a factor that a government can control, or levers they can pull to impact outcomes.
Let’s work with the basic assumption that (hopefully) everyone can agree on:
COVID spreads more when people are closer together than farther apart
Nothing too crazy.
Below is a graphic to represent two theoretical states:
These states are of identical land area, and each human figure can represent a particular number in the population. Which state do you expect to have more COVID deaths? A, of course.
But wait, A has more deaths because there are more people there. Let’s divide the number of deaths by 100k, and that should take care of it!
Baloney. As we have established, DENSITY impacts the spread of COVID. If one person from state A and one person from state B travel outside of the state, catch COVID, and then return, are they equally likely to spread it to others? Absolutely not. The citizen of state A is around more people, so they are more likely to spread it.
Fine, you might think, so let’s divide COVID deaths not only by 100k in the population, but also adjust for the density of the state’s population. That doesn’t work either. Here’s why:
States A and B not only have the identical land area, they also have identical populations. Thus, if you were to calculate the population density of each state, you would conclude (correctly) that they were identical. However, within the context of COVID, do both states have equal risk of COVID spreading? You see the pattern here.
That’s why the PROPER way to account for population distributions in states is to do something like this:
1. Take a smaller land area to work with, say 10 square miles
2. Take the population in each land area from step 1
3. Square it (or make whatever alerations to reflect the quantitative relationship between density and COVID spread)
4. Add all of these numbers together for a state. Now, we will create a number that’s easier to work with.
5. Divide the total land area for the state by the land area from step 1
6. The average population density of the US is roughly 90 people per square mile. Therefore, we will square it (8100), and multiply it by the number from step 5. This gives us an approximation of what a state’s step 4 might look like if its density was average.
7. Finally, divide the number from step 4 by step 6.
We’ll call this total metric “Population Risk Factor.” If the number is above 1, it means the state is at greater external risk to COVID spread due to population density, and a number below 1 means it is less so. This number takes into account BOTH total population AND population density.
Therefore, when we divide the total COVID deaths (or cases, whatever you want to measure) for the state by this Population Risk Factor, you will be accounting for the massively significant external factor of population density on the COVID death rate, and you will be MUCH closer to isolating and evaluating the performance the internal factors.
The process proposed above can be utilized by the right people TODAY, so we can have a much better idea as to whether Florida was actually better at protecting people than New York.