The Visual Cortex is not a Strictly Ascending Computational Pipeline; The Same Pipeline has Descending Computations Directed by Attention

Hermann von Helmholtz, pioneer of experimental psychology, is considered the first scientist to provide an experimental demonstration of covert attention (ca. 1860). Looking into a wooden box through two pinholes, Helmholtz would attend to a particular region of his visual field (without moving his eyes in that direction). When a spark was lit to briefly illuminate the box, he got an impression of only the objects in the region he had been attending to, thus showing that attention could be deployed independently of eye position and accommodation. Humans deploy covert attention routinely in many everyday situations, such as searching for objects, driving, crossing the street, playing sports and dancing. Covert attention allows us to monitor the environment and guides our eye movements, known as overt attention, to locations of the visual field that are salient and/or relevant.

The mechanism in the brain that enables this functionality has remained elusive over the many decades since Helmholtz. In a string of papers from 1987 to the present, Tsotsos and colleagues have described a theory of visual attention that proposed a solution. Known as the Selective Tuning model, this theory described the reasons for, the representation of, and the mathematical formalisms of how a top-down flow of computations within the visual cortex could enable the ability to covertly attend to a location of interest in a visual field. Although several other aspects of the theory have now received significant experimental support, the part dealing with this particular computation had not , until this paper :

Bartsch,M., Merkel, H., Strumpf, C., Schoenfeld, M. A.,Tsotsos, J.K., Hopf,J.-M., A cortical zoom-in operation underlies covert shifts of visual spatial attention, Science Advances 9(10), March 8, 2023, https://www.science.org/doi/10.1126/sciadv.ade7996

The lead author Mandy Bartsch is with the Leibniz-Institute for Neurobiology in Magdeburg Germany and the Donders Institute for Brain, Cognition and Behaviour, Radboud University Nijmegen, Netherlands. The research director Jens-Max Hopf is with the Leibniz-Institute for Neurobiology and the Otto-von-Guericke University, both in Magdeburg, Germany. The remaining study authors are from Germany and Canada. Experimental work was performed in Hopf’s lab examining human subjects with MEG imaging (magnetoencephalography).

Classically, visual processing has been considered as a separate module of the brain, as an input source, where processing is entirely accomplished in a feedforward manner, from input at the eyes towards moving higher levels such as prefrontal cortex. This is the view in the classic 1980 David Marr volume, has been a mainstay of most research whether biological or computational since then, and is incorporated usually tacitly within virtually all modern machine learning methods for visual processes. Any recurrence or feedback within this view has played a supporting role only by potentially modulating or biasing the feedforward computation.

Although there have been many proposals for how top-down processing might function in vision, none have been sufficiently detailed nor corroborated to enable predictions to be neurophysiologically tested. The Selective Tuning model, on the other hand, can be so tested. Now, the Bartsch et al. paper shows a major processing functionality as Selective Tuning prescribed, that challenges the feedforward view, namely, that visual processing can also occur in a different manner – from higher levels towards early ones, specifically for the computation of a critical element for all physical behaviour, stimulus location.

The ability to localize a visual element within a scene is critical not only for its specific analysis but also to direct any motor behaviour that might wish to affect that location (eye movement, grasping, locomotion). This top-down processing functionality is essentially as important as the feedforward function; the latter provides identity while the former provides location. In addition to solving a decades-old mystery, this work is the first to show that the visual cortex – and thus potentially any sensory brain area – plays a far more complex role in intelligent behaviour than has been previously acknowledged. The human visual cortex is not a strictly an ascending computational pipeline as has been the dominant belief for decades. The same pipeline also performs descending computations directed by attention.