-
Neural networks with motivation
Authors:
Sergey A. Shuvaev,
Ngoc B. Tran,
Marcus Stephenson-Jones,
Bo Li,
Alexei A. Koulakov
Abstract:
How can animals behave effectively in conditions involving different motivational contexts? Here, we propose how reinforcement learning neural networks can learn optimal behavior for dynamically changing motivational salience vectors. First, we show that Q-learning neural networks with motivation can navigate in environment with dynamic rewards. Second, we show that such networks can learn complex…
▽ More
How can animals behave effectively in conditions involving different motivational contexts? Here, we propose how reinforcement learning neural networks can learn optimal behavior for dynamically changing motivational salience vectors. First, we show that Q-learning neural networks with motivation can navigate in environment with dynamic rewards. Second, we show that such networks can learn complex behaviors simultaneously directed towards several goals distributed in an environment. Finally, we show that in Pavlovian conditioning task, the responses of the neurons in our model resemble the firing patterns of neurons in the ventral pallidum (VP), a basal ganglia structure involved in motivated behaviors. We show that, similarly to real neurons, recurrent networks with motivation are composed of two oppositely-tuned classes of neurons, responding to positive and negative rewards. Our model generates predictions for the VP connectivity. We conclude that networks with motivation can rapidly adapt their behavior to varying conditions without changes in synaptic strength when expected reward is modulated by motivation. Such networks may also provide a mechanism for how hierarchical reinforcement learning is implemented in the brain.
△ Less
Submitted 18 November, 2019; v1 submitted 22 June, 2019;
originally announced June 2019.
-
Representations of Sound in Deep Learning of Audio Features from Music
Authors:
Sergey Shuvaev,
Hamza Giaffar,
Alexei A. Koulakov
Abstract:
The work of a single musician, group or composer can vary widely in terms of musical style. Indeed, different stylistic elements, from performance medium and rhythm to harmony and texture, are typically exploited and developed across an artist's lifetime. Yet, there is often a discernable character to the work of, for instance, individual composers at the perceptual level - an experienced listener…
▽ More
The work of a single musician, group or composer can vary widely in terms of musical style. Indeed, different stylistic elements, from performance medium and rhythm to harmony and texture, are typically exploited and developed across an artist's lifetime. Yet, there is often a discernable character to the work of, for instance, individual composers at the perceptual level - an experienced listener can often pick up on subtle clues in the music to identify the composer or performer. Here we suggest that a convolutional network may learn these subtle clues or features given an appropriate representation of the music. In this paper, we apply a deep convolutional neural network to a large audio dataset and empirically evaluate its performance on audio classification tasks. Our trained network demonstrates accurate performance on such classification tasks when presented with 5 s examples of music obtained by simple transformations of the raw audio waveform. A particularly interesting example is the spectral representation of music obtained by application of a logarithmically spaced filter bank, mirroring the early stages of auditory signal transduction in mammals. The most successful representation of music to facilitate discrimination was obtained via a random matrix transform (RMT). Networks based on logarithmic filter banks and RMT were able to correctly guess the one composer out of 31 possibilities in 68 and 84 percent of cases respectively.
△ Less
Submitted 7 December, 2017;
originally announced December 2017.
-
Ocular dominance patterns in mammalian visual cortex: A wire length minimization approach
Authors:
Dmitri B. Chklovskii,
Alexei A. Koulakov
Abstract:
We propose a theory for ocular dominance (OD) patterns in mammalian primary visual cortex. This theory is based on the premise that OD pattern is an adaptation to minimize the length of intra-cortical wiring. Thus we can understand the existing OD patterns by solving a wire length minimization problem. We divide all the neurons into two classes: left-eye dominated and right-eye dominated. We fin…
▽ More
We propose a theory for ocular dominance (OD) patterns in mammalian primary visual cortex. This theory is based on the premise that OD pattern is an adaptation to minimize the length of intra-cortical wiring. Thus we can understand the existing OD patterns by solving a wire length minimization problem. We divide all the neurons into two classes: left-eye dominated and right-eye dominated. We find that segregation of neurons into monocular regions reduces wire length if the number of connections with the neurons of the same class differs from that with the other class. The shape of the regions depends on the relative fraction of neurons in the two classes. If the numbers are close we find that the optimal OD pattern consists of interdigitating stripes. If one class is less numerous than the other, the optimal OD pattern consists of patches of the first class neurons in the sea of the other class neurons. We predict the transition from stripes to patches when the fraction of neurons dominated by the ipsilateral eye is about 40%. This prediction agrees with the data in macaque and Cebus monkeys. This theory can be applied to other binary cortical systems.
△ Less
Submitted 14 June, 1999;
originally announced June 1999.
