Here were the simple instructions given by a Harvard University assistant professor to people participating in a recent cognitive science study:
“Imagine the following scene. Visualize it in your mind’s eye, as vividly as you can: a person walks into a room and knocks a ball off a table.”
The professor, Tomer Ullman, then asked those in the study about nine properties of their mental images, including the color and size of the ball, the shape and size of the table, and the person’s hair color and height. If you are anything like the people in the study, you only visualized a subset of all of these properties. Did you see how big the ball was? How about the person’s hair color? Most participants visualized the former but not the latter.
Ullman and his colleagues term this absence of details “noncommitment” to mental imagery. Psychologists and philosophers have noted the phenomenon before, but a new study published online May 18 in Cognition is the first attempt to gather data on it. The findings show noncommitment is the norm. It has nothing to do with a person forgetting the contents of a mental image, and it also is found in people with vivid imaginations.
The authors frame their findings with an eloquent description at the end of the study: “Mental images fill our daydreams, fuel our fancies, and color our memories. People often experience these images as richly detailed, making the imagination seem like a talented artist quickly painting a lifelike scene before our mind’s eye. Our results suggest that while the imagination may indeed be a good artist, it’s on a deadline, and stingy about paint.”
Beyond providing insight into the nature of the mind’s eye, the research raises a number of questions for further investigation and has implications for gathering eyewitness testimonies. Noncommitment, in fact, plays a role in a dispute known as the “imagery debate” that raged in philosophy and cognitive psychology for decades. One side argues that mental images are like actual pictures that our brain uses directly to reason about things. The opposing camp claims the images are more like sentences describing scenes, and the experience of seeing something in our mind’s eye is smoke and mirrors—it plays no role in cognition.
It’s true that some people lack mental imagery entirely, a phenomenon known as “aphantasia,” but most people say they see things vividly in their mind’s eye, and their bodies even respond as expected to what is imagined. When people visualize dark and light objects, their pupils dilate—unless they happen to have aphantasia. Brain imaging studies also show that mental imagery engages the same neurons in similar ways as perception. Visualizing things seems to have much in common with actually seeing them. But if mental images are indeed pictures, why do they lack such simple details?
Ullman and colleagues conducted a series of experiments in which participants visualized the ball and table scene and were then presented with the nine properties selected by the researchers. They had to respond either “Yes, it was part of my mental image,” or “No, it was not.” Seventy-eight percent of participants did not visualize at least two of the nine details. Ball size and table shape were usually included, whereas the person’s height and hair color often were not. The rest fell somewhere in between. “People are often unaware of how little detail their mental images contain until you ask them about it,” says cognitive scientist John McCoy of the University of Pennsylvania, co-senior author of the study. “You don’t notice how much you don’t notice.”
A second experiment required picturing four other scenes. In the third experiment, participants were given other options to describe each property in addition to yes or no: “don’t know,” “don’t remember” and “other.” Almost nobody chose these, implying that not visualizing a detail is not the same as forgetting or being uncertain about it.
The researchers next looked at the relationship between the vividness of people’s imaginations and noncommitment by using both standard questionnaire measures and participants’ own ratings of how vividly they pictured the scenes. There was an association between these measures of vividness and how many properties people visualized, but it was very weak, suggesting noncommitment has little to do with vividness. “You get people saying the image in their head is ‘super vivid, like real life.’ Then, when asked the color of the ball, they say, ‘Oh, I didn’t bother with that,’” Ullman says. “Conversely, people say they don’t see an image in their head but still say, ‘The ball? It was red, why?’”
In the final experiment, participants were instead asked to describe properties of their images, and they gave rich descriptions for everything. Being asked to describe an image likely prompts people to summon the relevant information, either by reimagining scenes and making sure to picture the requested detail or just making stuff up. “We know people confabulate details in many situations, but it was neat to see this play out in the context of imagination,” McCoy says.
The findings are actually not surprising to researchers who study mental imagery. “There’s a lot of behavioral data indicating images are constructed—not simply retrieved like a photograph but built up over time,” says neuroscientist Stephen Kosslyn of Harvard University, who was not involved in the work.
Perception itself is far from infallible. Research into “inattentional blindness” shows that even highly conspicuous things, such as a gorilla on a basketball court, can go completely unnoticed. These limits suggest a potential next step: “It would be interesting to briefly flash up a scene and have participants report what they saw,” Kosslyn says, “then compare that with the corresponding imagery study.” Researchers agree seeing and visualizing are not exactly the same, and this experiment could start to tease out when and how they differ.
The study may spur further research. “It opens a lot of questions for the field to follow up on,” says Jorge Morales, a psychologist and philosopher at Northeastern University who was not involved in the study. Most obviously, what kind of properties do people commit to and why? The findings of the Cognition study offer hints: “Nearly everyone can tell you sizes and shapes but not the person’s clothes,” Ullman says. “It’s like there’s a hierarchy when we construct images, and spatial properties are high up. Then things like colors are further down.”
This fits with Kosslyn’s “skeletal image” theory, in which overall shape is generated first, and other details are added as needed. Ullman likens it to a video game engine, which calculates spatial relationships and movement before rendering anything on-screen. “That kind of simulation can be used for answering a lot of questions that seem like they require an image but don’t,” Ullman says.
One limitation is that the study used participants’ subjective reports. “It’s a great study but still introspective. Can we get more objective evidence?” Morales asks. He suggests one possibility would be to ask participants to visualize a lamp and then ask whether it was on or off and measure pupil dilation. Another method that could be used is a technique called priming, where asking people to visualize objects influences subsequent judgements of something presented visually. Researchers could even use brain imaging to try to identify what is being imagined. “There are so many things to do now. I feel it’s going to reignite the question of the nature of mental images,” Morales says.
There is also an important practical application. “There are imagery-based protocols for interviewing people who witnessed a crime to guide them through trying to visualize it as accurately as possible,” Kosslyn says. Confabulation is an issue, but understanding noncommitment better could help develop ways of eliciting more accurate eyewitness testimony, he says. “That’s worth a lot.”
