-
Universal representation by Boltzmann machines with Regularised Axons
Authors:
Przemysław R. Grzybowski,
Antoni Jankiewicz,
Eloy Piñol,
David Cirauqui,
Dorota H. Grzybowska,
Paweł M. Petrykowski,
Miguel Ángel García-March,
Maciej Lewenstein,
Gorka Muñoz-Gil,
Alejandro Pozas-Kerstjens
Abstract:
It is widely known that Boltzmann machines are capable of representing arbitrary probability distributions over the values of their visible neurons, given enough hidden ones. However, sampling -- and thus training -- these models can be numerically hard. Recently we proposed a regularisation of the connections of Boltzmann machines, in order to control the energy landscape of the model, paving a w…
▽ More
It is widely known that Boltzmann machines are capable of representing arbitrary probability distributions over the values of their visible neurons, given enough hidden ones. However, sampling -- and thus training -- these models can be numerically hard. Recently we proposed a regularisation of the connections of Boltzmann machines, in order to control the energy landscape of the model, paving a way for efficient sampling and training. Here we formally prove that such regularised Boltzmann machines preserve the ability to represent arbitrary distributions. This is in conjunction with controlling the number of energy local minima, thus enabling easy \emph{guided} sampling and training. Furthermore, we explicitly show that regularised Boltzmann machines can store exponentially many arbitrarily correlated visible patterns with perfect retrieval, and we connect them to the Dense Associative Memory networks.
△ Less
Submitted 30 November, 2023; v1 submitted 22 October, 2023;
originally announced October 2023.
-
Characterization of anomalous diffusion through convolutional transformers
Authors:
Nicolás Firbas,
Òscar Garibo-i-Orts,
Miguel Ángel Garcia-March,
J. Alberto Conejero
Abstract:
The results of the Anomalous Diffusion Challenge (AnDi Challenge) have shown that machine learning methods can outperform classical statistical methodology at the characterization of anomalous diffusion in both the inference of the anomalous diffusion exponent alpha associated with each trajectory (Task 1), and the determination of the underlying diffusive regime which produced such trajectories (…
▽ More
The results of the Anomalous Diffusion Challenge (AnDi Challenge) have shown that machine learning methods can outperform classical statistical methodology at the characterization of anomalous diffusion in both the inference of the anomalous diffusion exponent alpha associated with each trajectory (Task 1), and the determination of the underlying diffusive regime which produced such trajectories (Task 2). Furthermore, of the five teams that finished in the top three across both tasks of the AnDi challenge, three of those teams used recurrent neural networks (RNNs). While RNNs, like the long short-term memory (LSTM) network, are effective at learning long-term dependencies in sequential data, their key disadvantage is that they must be trained sequentially. In order to facilitate training with larger data sets, by training in parallel, we propose a new transformer based neural network architecture for the characterization of anomalous diffusion. Our new architecture, the Convolutional Transformer (ConvTransformer) uses a bi-layered convolutional neural network to extract features from our diffusive trajectories that can be thought of as being words in a sentence. These features are then fed to two transformer encoding blocks that perform either regression or classification. To our knowledge, this is the first time transformers have been used for characterizing anomalous diffusion. Moreover, this may be the first time that a transformer encoding block has been used with a convolutional neural network and without the need for a transformer decoding block or positional encoding. Apart from being able to train in parallel, we show that the ConvTransformer is able to outperform the previous state of the art at determining the underlying diffusive regime in short trajectories (length 10-50 steps), which are the most important for experimental researchers.
△ Less
Submitted 10 October, 2022;
originally announced October 2022.
-
Efficient recurrent neural network methods for anomalously diffusing single particle short and noisy trajectories
Authors:
Òscar Garibo i Orts,
Miguel A. Garcia-March,
J. Alberto Conejero
Abstract:
Anomalous diffusion occurs at very different scales in nature, from atomic systems to motions in cell organelles, biological tissues or ecology, and also in artificial materials, such as cement. Being able to accurately measure the anomalous exponent associated with a given particle trajectory, thus determining whether the particle subdiffuses, superdiffuses or performs normal diffusion is of key…
▽ More
Anomalous diffusion occurs at very different scales in nature, from atomic systems to motions in cell organelles, biological tissues or ecology, and also in artificial materials, such as cement. Being able to accurately measure the anomalous exponent associated with a given particle trajectory, thus determining whether the particle subdiffuses, superdiffuses or performs normal diffusion is of key importance to understand the diffusion process. Also, it is often important to trustingly identify the model behind the trajectory, as this gives a large amount of information on the system dynamics. Both aspects are particularly difficult when the input data are short and noisy trajectories. It is even more difficult if one cannot guarantee that the trajectories output in experiments is homogeneous, hindering the statistical methods based on ensembles of trajectories. We present a data-driven method able to infer the anomalous exponent and to identify the type of anomalous diffusion process behind single, noisy and short trajectories, with good accuracy. This model was used in our participation in the Anomalous Diffusion (AnDi) Challenge. A combination of convolutional and recurrent neural networks were used to achieve state-of-the-art results when compared to methods participating in the AnDi Challenge, ranking top 4 in both classification and diffusion exponent regression.
△ Less
Submitted 5 August, 2021;
originally announced August 2021.
-
Efficient training of energy-based models via spin-glass control
Authors:
Alejandro Pozas-Kerstjens,
Gorka Muñoz-Gil,
Eloy Piñol,
Miguel Ángel García-March,
Antonio Acín,
Maciej Lewenstein,
Przemysław R. Grzybowski
Abstract:
We introduce a new family of energy-based probabilistic graphical models for efficient unsupervised learning. Its definition is motivated by the control of the spin-glass properties of the Ising model described by the weights of Boltzmann machines. We use it to learn the Bars and Stripes dataset of various sizes and the MNIST dataset, and show how they quickly achieve the performance offered by st…
▽ More
We introduce a new family of energy-based probabilistic graphical models for efficient unsupervised learning. Its definition is motivated by the control of the spin-glass properties of the Ising model described by the weights of Boltzmann machines. We use it to learn the Bars and Stripes dataset of various sizes and the MNIST dataset, and show how they quickly achieve the performance offered by standard methods for unsupervised learning. Our results indicate that the standard initialization of Boltzmann machines with random weights equivalent to spin-glass models is an unnecessary bottleneck in the process of training. Furthermore, this new family allows for very easy access to low-energy configurations, which points to new, efficient training algorithms. The simplest variant of such algorithms approximates the negative phase of the log-likelihood gradient with no Markov chain Monte Carlo sampling costs at all, and with an accuracy sufficient to achieve good learning and generalization.
△ Less
Submitted 15 April, 2021; v1 submitted 3 October, 2019;
originally announced October 2019.
-
Machine learning method for single trajectory characterization
Authors:
Gorka Muñoz-Gil,
Miguel Angel Garcia-March,
Carlo Manzo,
José D. Martín-Guerrero,
Maciej Lewenstein
Abstract:
In order to study transport in complex environments, it is extremely important to determine the physical mechanism underlying diffusion, and precisely characterize its nature and parameters. Often, this task is strongly impacted by data consisting of trajectories with short length and limited localization precision. In this paper, we propose a machine learning method based on a random forest archi…
▽ More
In order to study transport in complex environments, it is extremely important to determine the physical mechanism underlying diffusion, and precisely characterize its nature and parameters. Often, this task is strongly impacted by data consisting of trajectories with short length and limited localization precision. In this paper, we propose a machine learning method based on a random forest architecture, which is able to associate even very short trajectories to the underlying diffusion mechanism with a high accuracy. In addition, the method is able to classify the motion according to normal or anomalous diffusion, and determine its anomalous exponent with a small error. The method provides highly accurate outputs even when working with very short trajectories and in the presence of experimental noise. We further demonstrate the application of transfer learning to experimental and simulated data not included in the training/testing dataset. This allows for a full, high-accuracy characterization of experimental trajectories without the need of any prior information.
△ Less
Submitted 7 January, 2020; v1 submitted 7 March, 2019;
originally announced March 2019.
