How is our misunderstanding of human decision-making muddying the waters of the AI debate

One of my early memories biking in the Italian countryside was sometimes hearing the distinctive roar of a Ferrari echoing through the roads, farms and trees. I instantly liked that sound. A few decades later, I recall the time when I had to take a complex decision on which PhD program to join… and the feeling I had first time I met some of my colleagues in my PhD program at Boston University. Just a few conversations and exchanges of idea, and I liked them instantly, a feeling that I carry over the decades with these long-time friends. Fast forward a few decades, when I take a break in the morning and go get a coffee, I know that I do not like the long American version and always reach for Espresso for its smooth, rich aroma and consistency… some things never change!

Taking decisions is very natural to us, whether these are simple “I like, I don’t like” ones, or complex and intricate ones, e.g. set interest rates for the whole US economy by sitting in a Federal Reserve meeting. Human decision-making is rarely simple or linear. One thing we perhaps do not appreciate is that human (or biological…) decisions are often less explainable than we like to admit.

This has been confirmed by decades of work in Behavioral and Cognitive Neuroscience. Decision making is fundamentally probabilistic, determined by the process used by our brains to integrate sensory and cognitive information. Nobel laureate Kahneman and his colleague Tversky have shown that humans rely on heuristics — mental shortcuts that simplify decision-making — demonstrating how people evaluate potential gains and losses in a non-linear, probability-weighted way.

Under the hood, in our brains, what happens has been studied for many decades: 25 years ago, Platt and Glimcher showed that the pattern of firing of neurons in the cerebral cortex reflect how the brain weighs rewards based on magnitude and uncertainty, and more than 30 years ago, Simonson, and Tversky demonstrated that choices often violate logical consistency, influenced heavily by context and framing.

The studies above are just examples of a many scientific articles and books — from Blink to Thinking, Fast and Slow — that have popularized the fallacy of human thinking. When you ask a human “why did you take such a decision”, you will get usually a fairly well-thought set of reasons. That’s great, and gives us the illusion of human really knowing what they are doing, and being in control of their decision in a rational, deterministic way. This holds true for everyday decisions, but most dramatically, also for more important and consequential ones, from choosing your candidate as US president, to setting the interest rate for a whole country.

We take decisions, but we live under the illusion we can explain them well.

The process of making up these explanation is known as confabulation: when one is asked to explain a decision, the process pf constructing a plausible-sounding story goes under this funny name. This happens constantly behind our awareness and in spite of the true reasons behind our choices.

This is not an intentional act of deception but rather a cognitive mechanism the brain uses to make sense of its own behavior. Why?

Because our choices are the result of a much complex, probabilistic, distributed, and convoluted process that is very much similar to the one under the hood in Artificial Intelligence models.

Our 90 billion neurons and 100 trillion synapses determine, with their firing and data transmission, what car sound, coffee type, and interest rate hike we are going to choose, in a way that is very similar to the one AI decides whether there is a cat in the picture or the paragraph you just wrote makes sense or needs edit.

AI nostalgia? Good-Old-Fashioned Artificial Intelligence vs Neural Networks

AI wasn’t always like this opaque. In the early days of artificial intelligence, systems were designed to be highly explainable. Traditional, Good-Old-Fashioned Artificial Intelligence (GOFAI) methods, such as decision trees and rule-based systems, had the virtue of being transparent in terms of the way they took their decisions. One could trace every step and understand exactly why a particular outcome was reached.

However, these systems had a fundamental problem: they didn’t work well in complex, real-world scenarios. Rule-based systems relied on the intelligence of the programmer to come up with their decisions, they did not learn those from the data. They were too rigid and failed to handle the variability and unpredictability of complex tasks like image recognition, language processing, or autonomous driving.

This very reason was the root of the success of Neural Networks, which powers today’s all-winning AI systems. From the early 90s, researchers (I was one of them) pioneered algorithms inspired by the human brain that were complex, but more powerful precisely because they use a fuzzy decision-making process. These systems process vast amounts of input data, processing it through layers of neurons and weights to derive conclusions. The tradeoff is obvious: unlike GOFAI, one sacrifices some interpretability for efficiency, e.g. surpassing humans in real-time object detection and human-like language understanding.

These systems are not unlike the brain in many ways: they use neurons and synapses in large quantities to learn tasks. Also, like brains, are probabilistic. As neuroscientist Karl Friston’s eloquently explains, the brain operates by constantly making predictions and updating billions of neurons/synapses based on the error between expectations and reality. Biological brains work much like deep neural networks: they rely on probabilistic processing to process their environments and, just as like a Neural Network, adjusts its weights to minimize error. This similarity is not coincidental, rather, it points us to the very reason why deep learning, inspired by biological neural networks, is so successful.

The myth of Explainability and “black boxes”

Therefore, when AI critics describe deep learning models as “black boxes,” they are both right and wrong. There is a real challenge behind understanding and making explicit the inner workings of these complex systems. However, they are missing the fact that human brains are at least as opaque as Neural Networks.

However, biological brains are opaquer!

AI can, in principle, offer a level of transparency that human brains do not. While researchers can examine every weight and activation pattern in an AI model, run experiments, and systematically analyze its behavior, today’s neuroimaging technologies has no chance to map out how our own neurons encode decisions.

Rather than complaining about the loss of GOFAI’s Explainability and arguing that “this new AI is not as explainable as humans”, we should take a hard look at the scientific evidence that points to the contrary.

Human decisions are NOT transparent, and AI has the chance to become more explainable than human brains by the simple fact that we can directly probe and access every element that an AI system has running on a computer, vs the impossibility to do the same for a human.

While today’s AI might still have some of a feel of a black box, by comparing AI with human brains, we gain a deeper appreciation of the complexities involved in both systems. Progress in AI and the need to explain their processing since they now work are all powerful forces that will make AI systems more transparent in the very near term.