It has probably happened to everyone at one time or another. You're driving to a restaurant for the very first time. At a crossroads, you make a turn. You drive for several minutes, and then several minutes more. Nothing in sight. The disturbing thought creeps into your mind: "I should be there by now. Did I make the wrong turn?"At what point will you make a u-turn and go back? It all depends on how confident you are of the decision you made at the crossroads. If you are following a mere hunch, you may decide to go back. If you are following printed directions issued by the restaurant, you may have much more confidence in your decision and continue along in the same direction.
A sense of what we know -- and don't know
Having a sense of what we know -- and don't know -- is a universal human experience, and has often been assumed to be the hallmark of self-consciousness. But new research by neuroscientists at Cold Spring Harbor Laboratory (CSHL) suggests that the estimation of confidence that underlies decisions may be the product of a very basic kind of information processing in the brain, shared widely across species and not strictly confined to those, like us, that are self-aware.
This remarkable prospect arises from experiments performed on laboratory rats and reported this week in Nature by CSHL Assistant Professor Adam Kepecs, Ph.D., in collaboration with Zachary F. Mainen, Ph.D., CSHL Associate Professor and Group Leader in the Champalimaud Neuroscience Programme at the Instituto Gulbenkian de Ciênçia near Lisbon, Portugal.
"We all possess some intuitive sense; we know our convictions from our mere hunches," said Kepecs, who heads a lab at CSHL devoted to uncovering neural circuitry underlying decision-making. "This sense of confidence, or lack thereof, is critical to our success, but how it arises in the brain has long been a mystery."
Rats take the smell test
To solve it, Kepecs, Mainen and colleagues trained rats to make decisions of different difficulty. Because rats excel at olfaction, this was achieved by repeatedly presenting them with odors composed of mixtures of two chemicals and asking them to determine which component was stronger in order to receive a small reward.
By precisely varying the exact mixture of components, it was possible to manipulate the difficulty of the decision and therefore the animals' predicted level of uncertainty. This task is akin to asking a human whether a particular blend of blue and green colors is more blue or more green. Confidence is typically highest when the blend is mostly green or mostly blue. Uncertainty is highest when the blend contains nearly equal amounts of each color.
The team recorded signals from individual neurons in the rodents' brains while they were put to the test of distinguishing smells. They found that neurons in a part of the brain known as the orbitofrontal cortex (an area of the brain found in both rats and humans) signal the uncertainty of the decisions, "firing" much more vigorously in difficult tests compared with easier tests.
"These neurons seem to have been registering, after the rat made its decision, how uncertain the animal was that it was about to receive its desired reward," Mainen explained. "We tested several alternative explanations but the best explanation for the neural activity we observed was that these neurons were signaling the confidence of the animal about its decisions."
The results were surprising. "Although previous work had suggested that confidence estimates might be a complex process restricted to humans and perhaps other primates," said Kepecs, "our results, to the contrary, show that for the brain to estimate confidence in a choice is no more complex than calculating the choice itself."
How confidence can guide behavior
Having demonstrated that rat brains make confidence calculations, the researchers sought a way to demonstrate whether such calculations informed the rats' behavior. They ran a series of olfaction trials that featured an important modification of the original task design. As in the first set of trials, rats made decisions involving the discrimination of two smells; they were rewarded, after a defined period of delay, if they decided correctly, and received nothing if their decision was incorrect.
In the modified task, the reward delay was increased substantially. However, while waiting for the reward, the rats were given the option to abort the trial – short of learning whether their decision was right or wrong – and return to the beginning to start a new trial. "This new option to abort and restart constitutes a decision that should be made based on the level of confidence about getting a reward," Kepecs said. This is similar to the decision to make a u-turn when not finding the restaurant in the example above: whether and when you will turn back depends on how confident you are about the decision made at the crossroads.
Kepecs and Mainen surmised that if the rats were not confident about their original decision about the smell, they would be more likely to abort the trial. "Ultimately, confidence about getting a reward is a direct function of the animals' confidence about the decision they have just made. In this way we sought to measure a variable internal to the animals – how confident they were about whether they made the right choice or not – by observing how it influenced their behavior."
The researchers did indeed find that rats preferentially aborted uncertain trials. This showed that they could not only calculate their level of confidence in a given decision, but also use that calculation in subsequent decisions to guide behavior.
Taken together, these experiments reveal "that confidence estimation is not a complex function specific to humans but a core component of the process of decision-making probably found throughout the animal kingdom," said Kepecs. According to Mainen, "future studies of this kind may illuminate the question of how we form an intuitive sense of the solidity of a belief, how we distinguish fact from fiction itself."
Skinner Rules, OK
Quoting the researchers:
> the estimation of confidence that underlies decisions may be the product of a very basic kind of information processing in the brain, shared widely across species and not strictly confined to those, like us, that are self-aware.
Who says rats are not self-aware?
> not a complex function specific to humans but a core component of the process of decision-making probably found throughout the animal kingdom
What?? Not only are rats being demoted from the ranks of the self-aware, they're considered far enough removed from us wonderful humans that the existence of a trait in both humans and rats implies it exists across the entire animal kingdom???
Rats are very closely related to us, which is one reason they're used in experiments. They're far, far more closely related to humans than most of the rest of the animal kingdom is: spiders, jellyfish, tuna, ostriches, sponges...
Come on! Rats are NOT simplistic little associative learning machines, they're complex, intelligent creatures with advanced brains. It's about time that Behaviorism lost its choking grip on the minds of psychologists, imho.
Having said that, this is a really interesting piece of research. In fact it demonstrates just how far from a simple associative system a rat's brain really is.
- Steve
self awareness
Couple thoughts... the terminology of consciousness (including self-awareness, attention, etc) is woefully lacking in consensus, so that it is certainly possible that the researchers meant "self-reflective" in describing the difference between humans and rats, as opposed to "self-aware", which sets a much lower bar in terms of cognitive sophistication. That kind of distinction is rarely spelled out however.
Also, my interpretation of the "across the animal kingdom" remark is that it says less about the cognitive sophistication of rats than it does about the degree to which cog. sci. has misunderstood how fundamental confidence is.
Finally, in support of what you're saying, if I worked as a lab researcher, I'd either hate my job, or have a strong bias towards seeing the "subjects" as clinically as possible. Actually, I could never work in that kind of lab. In that sense I'm lucky I'm good at programming. :-]
Rats
Hey Terren,
> if I worked as a lab researcher, I'd either hate my job, or have a strong bias towards seeing the "subjects" as clinically as possible.
Yeah, I can understand that! Although my son was a lab researcher and he really liked his rats. What pisses me (and him) off, though, is not this sanity-preserving tendency to be clinical about lopping the heads off living beings, but the overwhelming dogma of Behaviorism. You should see some of the neural network models that purport to "explain" what's going on in a rat's brain. There's more going on in the controller of the average microwave oven. Rats are mammals - they have complex brains capable of forming rules, among other things. Associationist networks can't explain the formation of rules beyond simple generalisation from categories. My son got his PhD by breaking an important class of connectionist model and showing that nothing that simple is adequate to explain a rat's behavior. So how did people respond to having their simple networks broken? Why they just added a few extra layers of kludge until it seemed to work around the specific problem. It just doesn't seem to occur to them that 16-node, feed-forward, sum-of-products networks are pathetically inadequate for explaining the behaviour of the mammalian brain and just plain wrong. Many of us in the AI world, from which such networks originally came, figured this out long ago. But the legacy of Pavlov and Skinner and Morgan got turned into dogma - the moderate (but not necessarily helpful) proposition that we can't directly see what's going on inside the black box of an animal's psyche so we should concern ourselves solely with the relationships between its inputs and outputs, got turned into the dogma that there isn't anything going on inside the black box at all; that animals (other than we fabulous humans, of course, and maybe the occasional chimp if we're feeling generous) don't have minds and are nothing more than simple conditioned reflex machines. That's what really lies behind the belief that rats aren't self aware (or even self reflexive - how would we know?), and that what is true for a rat is also likely to be true for a squid and a sponge. It's chauvinism. It's ideological reductionism.
My theory is that the actions of many behavioral psychologists can be described using a network of sixteen summation nodes ... in which fifteen of them don't work.
behaviorism
Hi Steve,
The ideological reductionism perfectly serves the ongoing practice of animal experimentation (and the grant money that goes with it). Note that I'm not taking a position on animal experimentation (my own view is ambivalent), just noting that the ethics would get far dicier if behaviorism was finally discarded as the 8-track-era idea it is. Humankind's usage of animals in so many different ways rewards seeing animals as inferior, so I don't see that happening anytime soon.
Unfortunately it's had a big negative impact on the field of AI, in my view. Instead of 50's and 60's researchers starting a new field of AI with a sophisticated view of animal intelligence, one that might have suggested a progression of intelligence tests, they leapt straight for the prize of human intelligence (with its Turing Test). That can be forgiven, because why wouldn't you go for the gusto when you're first starting, right? But each "AI winter" that followed presented a new opportunity to see that high-level human cognition was aiming a tad high. If the mood towards animal intelligence had changed at any of those points, it might have spurred on productive new directions in AI. Instead we have people jumping as high as they can to touch the moon.
Absolutely...
...couldn't agree more.
Not only wouldn't we have people aiming so high, but we wouldn't have some of them aiming so low, either. Modern "insect-like" AI is distinctly behaviourist. Again it's admirable stuff until it becomes dogma, which it all too often does: "Insects can do X; a robot with a small recurrent network can, on a good day, do something vaguely similar to X; therefore insects probably work in a similar way to the robot (they imply, and sometimes they're right). What's more, humans can do X and Y and Z and a billion other things; therefore (they also imply), their brains must contain one little recurrent network for each of these behaviours." Ah, the dangers of extrapolation...
recurrent networks
I'd actually consider that to be a step forward if most AI researchers started thinking in terms of recurrent networks, even if just in the patchwork sense you're describing. It boggles my mind that many researchers are still thinking in terms of feed-forward or linear approaches to intelligence.