If you’re pursuing any goal, quantifying it and measuring regularly is a good way to get there in incremental steps, whether it’s paying off debt, getting in shape or mastering a subject.
Statistics have also helped public policy in innumerable ways. One of the earliest was tracking outbreaks of disease in Victorian London to identify the cause. Others include determining the efficacy of medical treatments, supplying armies and setting interest rates.
However, this focus on measurement can also lead us down the garden path.
What gets measured, gets overmanaged
If we have figures on any issue that people care about, that will naturally make us more inclined to care about that thing, especially if those figures are prominently published and widely discussed.
This can be a good thing. The nightly news death tallies of modern wars put a cap on developed countries’ participation in distant conflicts. The road toll reminds us not to drive drunk. High unemployment figures put pressure on governments to Do Something.
However, any such statistics need context. The daily Covid infection and death numbers caused a lot of panic that would have eased had people known that most of the infections don’t matter, and that most of the deaths were among those already unwell.
A number only means something once you compare it to other numbers. To hear that a man is six foot tall means nothing if you don’t know how tall any other man is.
Further, improved data collection has reduced many policy debates to ‘which approach would cause more deaths,’ as though the only purpose of public policy is to maximize years of life.
Another example of statistics leading us astray is standardized tests in schools. These are potentially useful numbers, but in many countries they are obsessed over at the expense of things like extracurricular activities, sport and even recess. ‘One number to rule them all.’
On the other hand, I reject the common assertion that GDP is overused as a measure of a country’s progress. It’s not perfect but it is pretty good, and I think overall GDP is used about the right amount, and is considered in light of other important measures.
What gets managed, gets mismeasured
Once any statistic becomes politically significant, there is an incentive for people to massage those numbers.
Read More