-
Zero-shot counting with a dual-stream neural network model
Authors:
Jessica A. F. Thompson,
Hannah Sheahan,
Tsvetomira Dumbalska,
Julian Sandbrink,
Manuela Piazza,
Christopher Summerfield
Abstract:
Deep neural networks have provided a computational framework for understanding object recognition, grounded in the neurophysiology of the primate ventral stream, but fail to account for how we process relational aspects of a scene. For example, deep neural networks fail at problems that involve enumerating the number of elements in an array, a problem that in humans relies on parietal cortex. Here…
▽ More
Deep neural networks have provided a computational framework for understanding object recognition, grounded in the neurophysiology of the primate ventral stream, but fail to account for how we process relational aspects of a scene. For example, deep neural networks fail at problems that involve enumerating the number of elements in an array, a problem that in humans relies on parietal cortex. Here, we build a 'dual-stream' neural network model which, equipped with both dorsal and ventral streams, can generalise its counting ability to wholly novel items ('zero-shot' counting). In doing so, it forms spatial response fields and lognormal number codes that resemble those observed in macaque posterior parietal cortex. We use the dual-stream network to make successful predictions about behavioural studies of the human gaze during similar counting tasks.
△ Less
Submitted 16 May, 2024;
originally announced May 2024.
-
The effect of task and training on intermediate representations in convolutional neural networks revealed with modified RV similarity analysis
Authors:
Jessica A. F. Thompson,
Yoshua Bengio,
Marc Schoenwiesner
Abstract:
Centered Kernel Alignment (CKA) was recently proposed as a similarity metric for comparing activation patterns in deep networks. Here we experiment with the modified RV-coefficient (RV2), which has very similar properties as CKA while being less sensitive to dataset size. We compare the representations of networks that received varying amounts of training on different layers: a standard trained ne…
▽ More
Centered Kernel Alignment (CKA) was recently proposed as a similarity metric for comparing activation patterns in deep networks. Here we experiment with the modified RV-coefficient (RV2), which has very similar properties as CKA while being less sensitive to dataset size. We compare the representations of networks that received varying amounts of training on different layers: a standard trained network (all parameters updated at every step), a freeze trained network (layers gradually frozen during training), random networks (only some layers trained), and a completely untrained network. We found that RV2 was able to recover expected similarity patterns and provide interpretable similarity matrices that suggested hypotheses about how representations are affected by different training recipes. We propose that the superior performance achieved by freeze training can be attributed to representational differences in the penultimate layer. Our comparisons of random networks suggest that the inputs and targets serve as anchors on the representations in the lowest and highest layers.
△ Less
Submitted 4 December, 2019;
originally announced December 2019.
-
How can deep learning advance computational modeling of sensory information processing?
Authors:
Jessica A. F. Thompson,
Yoshua Bengio,
Elia Formisano,
Marc Schönwiesner
Abstract:
Deep learning, computational neuroscience, and cognitive science have overlap** goals related to understanding intelligence such that perception and behaviour can be simulated in computational systems. In neuroimaging, machine learning methods have been used to test computational models of sensory information processing. Recently, these model comparison techniques have been used to evaluate deep…
▽ More
Deep learning, computational neuroscience, and cognitive science have overlap** goals related to understanding intelligence such that perception and behaviour can be simulated in computational systems. In neuroimaging, machine learning methods have been used to test computational models of sensory information processing. Recently, these model comparison techniques have been used to evaluate deep neural networks (DNNs) as models of sensory information processing. However, the interpretation of such model evaluations is muddied by imprecise statistical conclusions. Here, we make explicit the types of conclusions that can be drawn from these existing model comparison techniques and how these conclusions change when the model in question is a DNN. We discuss how DNNs are amenable to new model comparison techniques that allow for stronger conclusions to be made about the computational mechanisms underlying sensory information processing.
△ Less
Submitted 25 September, 2018;
originally announced October 2018.
