Numbers – it’s all about the context
Numbers are used a lot to make a point or serve as evidence. When you come across a number, in a report or on the news, always remember its meaning depends almost entirely on its context.
Certainly, numbers often seem inherently more reliable than descriptions. In the social sciences, there is a distinct preference for quantitative vs. qualitative findings. There is a bias, but perhaps it’s justified?
When you are presented with a number, there seems to be less intermediation. Numbers don’t lie, they say. But then the phrase “lies, damned lies and statistics,” often attributed to 19th century British Prime Minister Disraeli, is also pretty popular. It highlights inherent distrust toward statisticians and politicians. Because even if on the face of it numbers seem to be objective and verifiable, often there is quite a bit of intermediation behind their production.
While intermediation (statistical analysis) is acceptable, indeed necessary in order to collect and organize information and ensure that the results represent what they are supposed to, manipulation is not.
Forgive me for being blindingly obvious, but we need hard numbers. They are extremely useful. They allow us to have a common basis for discussion.
On a side note, numbers even serve different purposes. There are cardinal numbers (indicators of quantity, e.g. how many cups of flour in a bread recipe), there are ordinal numbers (indicating the order of things, who came in first place), and there are nominal numbers (for identifying things, like your seat number in the theatre). I’m going to focus on cardinal numbers for now.
Remember that a number is just a number. Its meaning (good or bad, important or trivial, expected or unexpected) is imposed externally. Its value is not inherent but rather depends on context. The context is what tells the story.
It is not enough to say that the unemployment rate was low. The description “low” can mean any number of things. Your first reaction should be “how low?” and “low compared to what?” The thing is, plugging in an actual number only gets you part of the way there. Let’s say unemployment was 7%. How does that compare to last year? It may be low if last year it was 9%, or it may be high if it was 5%. Or if in similar economies is above 10%, 7% looks pretty good, whereas if everywhere else it is below 6%, it looks less good.
Not only are comparisons across time and space important, but comparisons with purely fictitious numbers also matter. What I’m referring to by this is the number in your head, which doesn’t’ actually exist, right? This is the number you were expecting.
When unemployment numbers are predicted to fall, but they actually go up, the stock market may fall. When you expect to be offered a salary of $60,000 but then get an offer of $80,000, that will make you very happy. The age of newly elected French President Emanuel Macron’s wife, Brigitte (64) is not interesting – there are many women in their sixties in France – until you know that he is only 39. Their age difference is unexpected because of the gender markers. Its unexpectedness is in inverse relation to the age difference between Donald Trump and his wife, Melania. Who wouldn’t expect a billionaire property developer to have a wife a quarter century his junior?
Placing bets is an extreme example of what we can call the arbitrage between expected and actual numbers (or current and future numbers, if you will.) So a lot of money is often at stake.
You can apply this type of thinking to pretty much everything – from the price of a new refrigerator, to your annual salary to the number of people without access to drinking water. Context is everything. What that means is that reference points are critical. Numbers floating out there on their own don’t mean much.
Of course, you want to know how a number is derived. You should ask how they are collected, what they represent (often a matter of definition), and whether they truly represent what they are meant to? Obviously, in the case of the Brexit referendum and the last US election, the numbers which the vast majority of polls gave us did not accurately represent the people who actually voted.
Statistics can be presented misleadingly even when they are accurate. One of the most common issues perpetuated by the media and researchers themselves, involves comparing differences to a (seemingly arbitrary) base, or even not revealing the actual base. For example, findings reported in the American Journal of Epidemiology indicated that drinking more coffee reduced the chances of dying from oral/pharyngeal cancer by 49%. Sounds impressive, right? But you also need to know that your chances of dying from this type of cancer are about 0.09%, i.e. less than one in a 1,000. They weren’t high to begin with. And what about all the other cancers? Did they check those? And what about negative effects? Maybe those cancel out the positive ones.
Responsible reporting of statistics, whether in academic journals or the media, involves providing the reader or viewer with meaningful context. When you see a number, ask yourself the following questions:
- What does this actually represent? How is the number defined?
- How does it compare with alternatives, similar situations?
- How does it compare with the past?
- How does it compare with what was expected? Does the number fall in line with a trend or go against the trend?
- What are the possible interests of the person or organization in sharing this number? Journalists and evaluators have a code of ethics they are supposed to follow.