Down With Descriptive Statistics
.
Let’s say that we have some dataset in front of us, such as a list of patients who were diagnosed with a particular strain of cancer along with various longitudinal health outcomes, such as mortality. Traditionally, we segment the set of statistical methods usable to analyze this dataset as either descriptive or inferential. Descriptive statistics are beholden only to the dataset at hand. They seek to represent the dataset as accurately as possible, treating it as the source of truth. Descriptive statistics that you are likely familiar with are means (such as the average life expectancy of patients) or various graphs (such as a line graph showing percentage survival each year after initial diagnosis).
In contrast, inferential statistics view a dataset as a stepping stone to some greater truth, in one of two ways. One is learning about a bigger, unseen dataset that our seen dataset is a subset, or sample, of. For our cancer example, this bigger dataset might be all people who have ever contracted this strain of cancer. The other objective is to identify causality. In our example, we might be interested in how effective chemotherapy is in putting the cancer in remission.
I’ve come to believe that while descriptive statistical methods only make statements about the dataset at hand, our use of descriptive statistics is almost exclusively inferential. Consider sitting down with an oncologist and hearing the news about your positive diagnosis for the first time. Surely, some of your questions will be about what you should expect for your future - how likely are you to beat the tumor? What is your life expectancy? The oncologist is likely to refer to descriptive statistics such as average 5-year life expectancy to answer these fundamentally inferential questions.
I challenge you to think of a situation where you are honestly interested in descriptive statistics with no goal of inference, outside of simple academic interest. I have only come up with a single metric, which is barely even a “statistic” - the humble sum. For example, a military general would want to know how many soldiers they have in their army after a large battle with high casualties. Beyond that, I’ve got nothing.
Consider that all datasets are definitionally historical - anything in a dataset must definitionally have occurred in the past. In a meaningful way, statistics is the study of history, though the methods used are more quantitative than what is traditionally used in the social sciences. I’d contest that this same argument holds in the social sciences as well - that much of our interest in history is in trying to learn about what is currently happening in the world and what might come in the future. To illustrate this point, look no further than Winston Churchill’s famous quote that, “Those that fail to learn from history are doomed to repeat it”.
This is not merely a pedantic argument. I’ve observed this classification produce two opposing consequences, as a result of the lower rigor applied to descriptive statistics. The first is that we put too much trust in descriptive statistic when they are applied in inferential ways, because they never have to clear the hurdles present for inferential statistics, such as those required to reject a null hypothesis. A mean based on tiny small sample size may be worse than useless, given it may be trusted inappropriately. In contrast, we may overcorrect in the other direction, unwilling to trust any descriptive statistics because we “aren’t supposed to infer from them”, even though they can be genuinely useful - cancer patients can learn valuable information from historical life expectancies, even if it might be better to look at more sophisticated metrics in a perfect world.
At its core, this argument is a microcosm of a larger fight between Bayesians and Frequentists. I fundamentally reject the Frequentist hostility towards the application of human reasoning to statistics, and think that all statistics are imperfect but possibly useful ways to make sense of an uncertain world. Minimally, the applicability of statistics is a function of how much we expect the future to reflect the past, and I reject the implied suggestion that the best default heuristic is continuity at all times. Descriptive statistics may be “good enough” to not demand the effort required by inferential statistics, or they frankly might be the best available, if we cannot come to a coherent view on certain assumptions underlying more complex methods (such as the distribution of the underlying population).
In short - I view the division of descriptive and inferential statistics as denying the similarity in how these statistics are applied while obfuscating the applicability descriptive statistics. I believe we would be better served abolishing this delineation, and viewing all statistics on the same spectrum, all with the stated goal of helping us form views about the future.
Preview image from Isaac Smith on Unsplash.