When millions of people argued in 2015 over whether a photograph of a dress was blue and black or white and gold, they were looking at the same image while their brains made different guesses about the light.
A growing number of neuroscientists think that all perception works this way. Under a theory known as predictive processing, the brain does not simply record what arrives through the eyes and ears and assemble it into a picture. It continuously generates predictions about what is out there, based on memory and past experience, and uses the incoming signals mainly to check and correct those predictions. What a person experiences as the world is the brain’s current best guess.
The idea has a long history, going back to the 19th-century German physicist Hermann von Helmholtz, and it has been developed in computer models since the 1990s. Its supporters, including the British neuroscientist Karl Friston, argue that reducing the mismatch between prediction and reality may be the basic organizing principle of the brain. Much of the evidence is still indirect, and critics say the theory may explain too much to be easily tested.
A Flood of Data and a Trickle of Thought
The brain takes in far more information than it can use. In a study published in the journal Neuron in December 2024, Jieyu Zheng and Markus Meister of the California Institute of Technology estimated the information rate of human behavior. Drawing on tasks from typing and speaking to solving a Rubik’s Cube blindfolded, they put it at about 10 bits per second.
The senses deliver data many millions of times faster than that. Zheng and Meister calculated that a single cone cell in the human retina can transmit about 270 bits per second, so the 6 million cones in one eye together carry about 1.6 billion bits per second. Across the senses, the Caltech team estimated that the body gathers information about a billion bits a second, roughly 100 million times the rate at which people act on it.
“Every moment, we are extracting just 10 bits from the trillion that our senses are taking in and using those 10 to perceive the world around us and make decisions,” Meister said. “This raises a paradox: What is the brain doing to filter all of this information?” Predictive processing offers one answer to Meister’s question. If the brain already expects most of what it sees and hears, it needs to pay attention only to the part it did not expect. The puzzle is sharper because the brain has about 86 billion neurons, roughly 16 billion of them in the cerebral cortex, and individual neurons can easily transmit more than 10 bits a second on their own.
From Helmholtz to Prediction Errors
The Arab scholar Ibn al-Haytham described a version of the idea in his Book of Optics about a thousand years ago. In the 1860s Helmholtz argued that the brain infers the external causes of its sensations rather than building perceptions from the bottom up, a process he called unconscious inference. He used it to explain images such as the duck-rabbit drawing, which the mind flips between two readings even though the picture on the retina does not change.
In 1980 the British psychologist Richard Gregory published an influential paper, “Perceptions as Hypotheses,” arguing that visual illusions are the brain’s mistaken guesses about the causes of what it senses. In 1999 the computer scientists Rajesh Rao and Dana Ballard published a model of the visual cortex in the journal Nature Neuroscience that turned the idea into a working mechanism. In their model, feedback connections from higher brain areas carry predictions down to lower ones, and feedforward connections send back only the errors, the difference between what was predicted and what arrived.
This scheme, now called predictive coding, arranges the brain as a hierarchy. The highest levels hold the most abstract expectations, such as a snake lying in the shadows ahead, and each level predicts the activity of the one below it. Where the prediction fails, an error signal travels upward until the higher level updates its hypothesis, and the snake becomes a coiled rope. Floris de Lange of the Predictive Brain Lab at Radboud University in the Netherlands describes the brain as having two populations of neurons. One encodes the current best prediction, and the other signals errors in it.
Read next
Apollo 15’s Hammer and Feather Drop That Proved Galileo Right
Surprise and the Free Energy Principle
In the predictive view of the brain, surprise is the signal that matters most. Input that matches the prediction carries little new information, while an unexpected sound, an unfamiliar face or a sudden movement produces a large prediction error and forces the model to change. The updated model then generates the next round of predictions.
Karl Friston of University College London has taken the idea furthest. He set out his free energy principle in the journal Nature Reviews Neuroscience in 2010. In it, any living system that stays alive must keep its states within limited bounds and so must minimize surprise over time. Because surprise cannot be measured directly, the brain minimizes a quantity called free energy that sets an upper limit on it.
Friston’s framework gives the brain two ways to reduce prediction error. It can change its predictions, which is perception, or it can act to change the sensations themselves, which is movement. In his account, even motor control works by the brain predicting the feel of the movement it intends and then moving the body until the prediction comes true. “Trying to make sense of data without a generative model is doomed to failure,” Friston told Quanta Magazine. “All one can do is make statements about patterns in data.”
The Sussex neuroscientist Anil Seth has popularized a related idea in his 2021 book Being You, describing perception as a “controlled hallucination.” In his account, the brain’s predictions produce the experience, and the senses keep that experience tied to the real world.
The Dress That Divided the Internet
The dress photograph gave researchers an unusually public test of how expectations shape what people see. The dress, worn by the mother of a bride at a wedding in Scotland, was photographed in an overexposed picture that left the source of the light unclear. Viewers split sharply between blue and black and white and gold.
In 2017 the New York University neuroscientist Pascal Wallisch published the results of an online study of more than 13,000 people in the Journal of Vision. He found that the colors people saw depended on what they assumed about the light. Among participants who believed the dress was in shadow, four in five saw it as white and gold, compared with only about half of those who did not.
“Shadows are blue, so we mentally subtract the blue light in order to view the image, which then appears in bright colors, gold and white,” Wallisch explained. Artificial light tends to be yellowish, so viewers who assumed indoor lighting subtracted yellow and saw black and blue. People who go to bed and rise early spend more of their waking hours in daylight. They were significantly more likely to see white and gold than night owls, who live more under artificial light.
Evidence Still Being Gathered
Supporters of predictive processing point to the range of findings it can account for. “The beauty of the predictive processing framework [is] that it has a really large, sometimes critics might say too large, capacity to explain a lot of different phenomena in many different systems,” de Lange told Quanta Magazine. That breadth is also the main criticism, since a theory that can explain almost any result is hard to prove wrong.
Much of the support from studies of the human brain remains indirect. “If you look into cognitive neuroscience and neuro-imaging in humans, [there’s] a lot of evidence, but super-implicit, indirect evidence,” said Tim Kietzmann of Radboud University. Researchers have turned to computer models to test the idea. In 2016 William Lotter, David Cox and Gabriel Kreiman at Harvard built a four-layer network called PredNet on predictive coding principles. It learned to predict the next frame of video filmed from a car driving through city streets, and in 2020 Lotter’s team reported that it responded to unexpected images in ways seen in monkey brains.
At Radboud, Kietzmann and Marcel van Gerven asked whether the prediction machinery had to be built in at all. Because neural signaling is costly and the brain is the body’s most energy-intensive organ, they trained a network to predict sequences of digits while using as little energy as possible. The network developed its own “prediction units” and “error units,” and the error units fell quiet once the predictions were right. “Our system does it out of the box, as an emergent thing to do, to be energy-efficient,” Kietzmann said.
When Wallisch reported his dress results, he answered the old question of whether one person sees the same color as another. “The answer, based on this research, is ‘not necessarily’,” he said. “If illumination conditions are unclear, your assumptions about the illumination source will matter, and those might depend on lifestyle choices, such as when you go to sleep.”




