[visionlist] What can retinotopy tell us about how the brain allows us to see?

Fri Sep 29 08:15:43 -04 2023

Dear colleagues,

Very interesting discussion and still an open debate... a shameless self plug, hoping it is a possible contribution to this discussion.
We have recently challenged the efficiency of retinotopy in our presentation this week at ICANN:

 Jean-Nicolas Jérémie<https://laurentperrinet.github.io/author/jean-nicolas-jeremie/>, Emmanuel Daucé<https://laurentperrinet.github.io/author/emmanuel-dauce/>, Laurent U Perrinet<https://laurentperrinet.github.io/author/laurent-u-perrinet/> (2023). Retinotopy improves the categorisation and localisation of visual objects in CNNs<https://laurentperrinet.github.io/publication/jeremie-23-icann/>. 32nd International Conference on Artificial Neural Networks (ICANN 2023).
https://laurentperrinet.github.io/publication/jeremie-23-icann/

Our modeling work suggests that standard visual object categorization models such as CNNs can easily handle retinotopic input (surprisingly !), and that this gives them some relative robustness to zoom and rotation:

[image]

Compared to traditional CNNs, the retinotopic mapping appears to have more density around the fixation point, so that "looking around" the scene shows better localization of objects:

[image]

The model is certainly wrong, but I hope it can be useful for predictions, maybe psychophysics?) :-)

Cheers,
Laurent & Jean-Nicolas

Ps: would be glad to share our paper / code - drop us a message!
--

Laurent Perrinet - INT (UMR 7289)/CNRS
https://laurentperrinet.github.io/

Le 28 sept. 2023 à 18:13, Peter Neri <neri.peter at googlemail.com> a écrit :

also relevant to this discussion, and to some of Andrew's comments:

https://pubmed.ncbi.nlm.nih.gov/29249282/
. 2018 Jan 3;97(1):164-180.e7.
doi: 10.1016/j.neuron.2017.11.017. Epub 2017 Dec 14.
Spatial Information in a Non-retinotopic Visual Cortex
Julien Fournier<https://pubmed.ncbi.nlm.nih.gov/?term=Fournier+J&cauthor_id=29249282>  1 <https://pubmed.ncbi.nlm.nih.gov/29249282/#full-view-affiliation-1> , Christian M Müller<https://pubmed.ncbi.nlm.nih.gov/?term=M%C3%BCller+CM&cauthor_id=29249282>  1 <https://pubmed.ncbi.nlm.nih.gov/29249282/#full-view-affiliation-1> , Ingmar Schneider<https://pubmed.ncbi.nlm.nih.gov/?term=Schneider+I&cauthor_id=29249282>  1 <https://pubmed.ncbi.nlm.nih.gov/29249282/#full-view-affiliation-1> , Gilles Laurent<https://pubmed.ncbi.nlm.nih.gov/?term=Laurent+G&cauthor_id=29249282>  2 <https://pubmed.ncbi.nlm.nih.gov/29249282/#full-view-affiliation-2>

On Wed, Sep 27, 2023 at 10:26 PM Andrew Parker <andrew.parker at dpag.ox.ac.uk<mailto:andrew.parker at dpag.ox.ac.uk>> wrote:
Nice to have the challenge: here are a few remarks.

1)    Retinotopy is probably efficient but not essential for visual processing. The only relationships that matter in the nervous system are the connections between nerve cells. So, the idea that retinotopy is important because the cortical representation is a distorted replica of the retinal image is surely for the birds. We might as well worry that the image of the world is “upside down” on the visual cortex, an issue that used to present problems but now poses us no problems at all nowadays.

2)    Retinotopy arises in part because that’s how the nervous system builds itself, as it does for other sensory systems. Several constraints arise from the principle of “fire together, wire together”. But there are other factors related to contact molecules (ephrins) in visual development. These matters bite before visual input to the cortex really gets going. So, by the time vision truly arrives, the system is already topographic, if not fully retinotopic.

3)    For humans, alignment of retinotopy for left and right eyes is critical. It really looks as if part of amblyopia is due to the “scrambling” of the retinotopy of one eye in its mapping onto the cortex. A powerful piece of evidence here are the reports of spatial distortions in geometric figures when viewed by the amblyopic eye, as compared with viewing by the fellow eye. Stereo-depth from binocularity will of course fail if the mapping is not correct. Contrariwise, in animals like mice, with limited binocular visual fields, retinotopy marches on across the cortex regardless of engagement from the other eye.

4)    There’s another point about binocular retinotopic mapping, which is that it is really hard to see how stereo could ever develop without going through the alignment of two monocular retinotopic maps. It’s not impossible, but binocular alignment of retinotopy provides a simple developmental mechanism that enables stereo through a controlled, local mismatching of inputs from left and right eyes. This is a sort of counter-argument to point 1 above, but the issue with stereo is that it depends on the relationship of left and right eye retinotopies, not just the existence of retinotopic maps for each eye taken alone.

5)    There are some properties of perception that are hard to explain with retinotopic maps. One example is perception of symmetry, which may involve matching of corresponding points that are separated by many millimeters of cortical distance, at least in V1.

6)    For many of the spatial relationships that we need to process computationally from the visual inputs, near-neighbour relationships are dominant. Thus, the nervous system can save on economy of wiring by setting things up on a retinotopic basis. The saving is more than just the cost of building connections with “long wires”, because, in long-term use, long wires also cost more metabolic energy, as there are more sodium pumps to be maintained along each mm of wire length.

7)    For some developmental cases, the topography becomes substantially rearranged without complete loss of spatial visual performance. A recent example is https://www.pnas.org/doi/full/10.1073/pnas.0809688106, which is interesting, particularly because it involves a very early deficit at the cortical level.  Another example to consider is the rearrangement of visual projections in albino individuals. In both these cases, the details of the topography are different from typical human patters, but nonetheless basic spatial function of visuo-motor responses are preserved and geometrically correct.

There’s a nice discussion of a number of these points in
Kremkow J, Alonso JM. Thalamocortical Circuits and Functional Architecture. Annu Rev Vis Sci. 2018 Sep 15;4:263-285. doi: 10.1146/annurev-vision-091517-034122. Epub 2018 Jun 1. PMID: 29856937; PMCID: PMC7525828.

Andrew

On 27. Sep 2023, at 17:34, Paul Linton <paul.linton at columbia.edu<mailto:paul.linton at columbia.edu>> wrote:

Hi Jorge,

Echoing Tomas Knapen's excellent discussion (and work!), you might also be interested in the discussion in:

Linton, P. (2021), ‘V1 as an Egocentric Cognitive Map’, Neuroscience of Consciousness, 2021(2), 17:
https://academic.oup.com/nc/article/2021/2/niab017/6369785

Thanks so much,

Paul

On 26 Sep 2023, at 16:15, Tomas Knapen <tknapen at gmail.com<mailto:tknapen at gmail.com>> wrote:

Hi Jorge,

We recently wrote a review about exactly this issue, prompted by a slew of recent studies showing coding of visual space throughout the brain (1). This builds on many ideas, also from the ones you’re citing. I personally love Koenderink earlier work on this subject too (2).

Interesting directions (again, personal opinion) are how different regions have specific biases in the representation of visual space, like in Uri Hasson’s earlier work (3). And, recent ideas by Mike Arcaro and Marge Livingstone about how different sensory topographies relate to one another (4).

Hope this helps, would love to discuss more.

Tomas

1.Groen, I. I. A., Dekker, T. M., Knapen, T. & Silson, E. H. Visuospatial coding as ubiquitous scaffolding for human cognition. Trends Cogn Sci (2021) doi:10.1016/j.tics.2021.10.011.
2.Koenderink, J. J. The brain a geometry engine. Psychological Res 52, 122–127 (1990).
3.Hasson, U., Levy, I., Behrmann, M., Hendler, T. & Malach, R. Eccentricity Bias as an Organizing Principle for Human High-Order Object Areas. Neuron 34, 479–490 (2002).
4.Arcaro, M. J. & Livingstone, M. S. On the relationship between maps and domains in inferotemporal cortex. Nat Rev Neurosci 22, 573–583 (2021).

On 26 Sep 2023, at 20:23, Jorge Almeida <jorgecbalmeida at gmail.com<mailto:jorgecbalmeida at gmail.com>> wrote:

Just some context that I should have given (sorry!!!). This comes from 1) the beautiful discussions on the functional role of retinotopy for instance between people like Jon Kaas and others (e.g., "Kaas, J (1997). Topographic maps are fundamental to sensory processing. Brain Research Bulletin, 44(2), 107-112. vs. Weinberg, R. (1997). Are topographic maps fundamental to sensory processing?  Brain Research Bulletin, 44(2), 113-116.); and 2) trying to understand how, in general, topographic maps such as retinotopy can be important functionally (and not just neurally) and can guide our understanding of how the mind works (at different levels of abstraction). Thank you all so much!
jorge almeida

On Mon, Sep 25, 2023 at 11:51 PM Jorge Almeida <jorgecbalmeida at gmail.com<mailto:jorgecbalmeida at gmail.com>> wrote:
Dear all,

I was wondering if some of you can point me to a set of papers (or just send out some ideas) on whether and how the fact that we show that visual cortex is organized in a retinotipic map (or tonotopy in auditory cortex) is important in understanding how vision works/the brain allows for visual processes.

That is, is there a function for retinotopy as it comes to vision? How does showing retinotopic maps tell us anything about how vision works (mostly we focus on things like reducing connections and thus saving energy)? How does it impact visual processing? What have we learnt about visual cognition from retinotopy? Perhaps even, are there visual effects that are a consequence of retinotopy?

Sorry if the formulation of the question is not super clear and thanks in advance!

Jorge Almeida
_______________________________________________
visionlist mailing list
visionlist at visionscience.com<mailto:visionlist at visionscience.com>
http://visionscience.com/mailman/listinfo/visionlist_visionscience.com
_______________________________________________
visionlist mailing list
visionlist at visionscience.com<mailto:visionlist at visionscience.com>
http://visionscience.com/mailman/listinfo/visionlist_visionscience.com

_______________________________________________
visionlist mailing list
visionlist at visionscience.com<mailto:visionlist at visionscience.com>
http://visionscience.com/mailman/listinfo/visionlist_visionscience.com

_______________________________________________
visionlist mailing list
visionlist at visionscience.com<mailto:visionlist at visionscience.com>
http://visionscience.com/mailman/listinfo/visionlist_visionscience.com

--
Peter Neri
Head of Vision Team
Laboratoire des Systèmes Perceptifs (UMR8248)
École Normale Supérieure
29 rue d'Ulm, 75005 Paris (France)

https://sites.google.com/site/neripeter/home
https://lsp.dec.ens.fr/en/member/652/peter-neri
_______________________________________________
visionlist mailing list
visionlist at visionscience.com
http://visionscience.com/mailman/listinfo/visionlist_visionscience.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://visionscience.com/pipermail/visionlist_visionscience.com/attachments/20230929/03f1ae4a/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image.png
Type: image/png
Size: 166202 bytes
Desc: image.png
URL: <http://visionscience.com/pipermail/visionlist_visionscience.com/attachments/20230929/03f1ae4a/attachment-0002.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image.png
Type: image/png
Size: 332489 bytes
Desc: image.png
URL: <http://visionscience.com/pipermail/visionlist_visionscience.com/attachments/20230929/03f1ae4a/attachment-0003.png>