As a scientist, you love models. You use models to make predictions. You use models to gain new knowledge. Who knows, you might even use models to brainstorm the next experiment! Models are THE ultimate tool for understanding the world around us.
Over the last decade, scientists have witnessed the rise of a new class of models — machine learning models. But they are strange. Indeed, just like traditional models they allow to make predictions. But unlike traditional models (and toys), you cannot open up a machine learning model, figure out what the pieces do, and then put them back together. A machine learning model is more like a juggernaut full of innumerable tiny pieces, like the weights in an artificial neural network, none of which bears individually an intuitive interpretation. This lack of interpretability of the individual elements makes it hard to use these otherwise powerfully predictive models to understand the world's phenomena.
But, you say, machine learning models must contain a lot of knowledge about our data and consequently about our phenomenon, otherwise how could they provide such accurate predictions? Can we somehow squeeze this knowledge out of the model and make it accessible to scientists?
In our recent publication in Minds and Machines, we show that we might, if we systematically analyze the behavior of the whole model. Such holistic model analysis can indeed reveal relevant properties of our data that have traditionally been studied via interpretable model parameters, like the masses and speeds of physical models, or the feature effects and importances of statistical models for the social sciences. We show how to construct methods — we call them property descriptors — to answer precise scientific questions by extracting such properties, and even estimate their uncertainties.
Luckily, you do not have to start from scratch. As it turns out, many of the existing machine learning interpretability techniques are already property descriptors. For a range of scientific questions, which we listed in the paper, you can use this arsenal to query expensively-trained machine learning models for insights about their training data, and, ultimately the phenomenon that produced it.
When coming up with this, we had a lot of fun and sometimes deep discussions about what scientific models really are, coming as we do from such different backgrounds as physics, data science, statistics and philosophy of science (with offshoots into computational neuroscience, causality, and pure math). And it only got better during the review process, thanks to our extraordinary reviewers at Minds and Machines — you know who you are.
Timo Freiesleben and Álvaro Tejero-Cantero.
Freiesleben, T., König, G., Molnar, C. et al. Scientific Inference with Interpretable Machine Learning: Analyzing Models to Learn About Real-World Phenomena. Minds & Machines 34, 32 (2024)