-
Semantic Embeddings in Semilattices
Authors:
Fernando Martin-Maroto,
Gonzalo G. de Polavieja
Abstract:
To represent anything from mathematical concepts to real-world objects, we have to resort to an encoding. Encodings, such as written language, usually assume a decoder that understands a rich shared code. A semantic embedding is a form of encoding that assumes a decoder with no knowledge, or little knowledge, beyond the basic rules of a mathematical formalism such as an algebra. Here we give a for…
▽ More
To represent anything from mathematical concepts to real-world objects, we have to resort to an encoding. Encodings, such as written language, usually assume a decoder that understands a rich shared code. A semantic embedding is a form of encoding that assumes a decoder with no knowledge, or little knowledge, beyond the basic rules of a mathematical formalism such as an algebra. Here we give a formal definition of a semantic embedding in a semilattice which can be used to resolve machine learning and classic computer science problems. Specifically, a semantic embedding of a problem is here an encoding of the problem as sentences in an algebraic theory that extends the theory of semilattices. We use the recently introduced formalism of finite atomized semilattices to study the properties of the embeddings and their finite models. For a problem embedded in a semilattice, we show that every solution has a model atomized by an irreducible subset of the non-redundant atoms of the freest model of the embedding. We give examples of semantic embeddings that can be used to find solutions for the N-Queen's completion, the Sudoku, and the Hamiltonian Path problems.
△ Less
Submitted 25 May, 2022;
originally announced May 2022.
-
Finite Atomized Semilattices
Authors:
Fernando Martin-Maroto,
Gonzalo G. de Polavieja
Abstract:
We show that every finite semilattice can be represented as an atomized semilattice, an algebraic structure with additional elements (atoms) that extend the semilattice's partial order. Each atom maps to one subdirectly irreducible component, and the set of atoms forms a hypergraph that fully defines the semilattice. An atomization always exists and is unique up to "redundant atoms". Atomized semi…
▽ More
We show that every finite semilattice can be represented as an atomized semilattice, an algebraic structure with additional elements (atoms) that extend the semilattice's partial order. Each atom maps to one subdirectly irreducible component, and the set of atoms forms a hypergraph that fully defines the semilattice. An atomization always exists and is unique up to "redundant atoms". Atomized semilattices are representations that can be used as computational tools for building semilattice models from sentences, as well as building its subalgebras and products. Atomized semilattices can be applied to machine learning and to the study of semantic embeddings into algebras with idempotent operators.
△ Less
Submitted 16 February, 2021;
originally announced February 2021.
-
Supervised dimensionality reduction by a Linear Discriminant Analysis on pre-trained CNN features
Authors:
Francisco J. H. Heras,
Gonzalo G. de Polavieja
Abstract:
We explore the application of linear discriminant analysis (LDA) to the features obtained in different layers of pretrained deep convolutional neural networks (CNNs). The advantage of LDA compared to other techniques in dimensionality reduction is that it reduces dimensions while preserving the global structure of data, so distances in the low-dimensional structure found are meaningful. The LDA ap…
▽ More
We explore the application of linear discriminant analysis (LDA) to the features obtained in different layers of pretrained deep convolutional neural networks (CNNs). The advantage of LDA compared to other techniques in dimensionality reduction is that it reduces dimensions while preserving the global structure of data, so distances in the low-dimensional structure found are meaningful. The LDA applied to the CNN features finds that the centroids of classes corresponding to the similar data lay closer than classes corresponding to different data. We applied the method to a modification of the MNIST dataset with ten additional classes, each new class with half of the images from one of the standard ten classes. The method finds the new classes close to the corresponding standard classes we took the data form. We also applied the method to a dataset of images of butterflies to find that related subspecies are found to be close. For both datasets, we find a performance similar to state-of-the-art methods.
△ Less
Submitted 22 June, 2020;
originally announced June 2020.
-
Algebraic Machine Learning
Authors:
Fernando Martin-Maroto,
Gonzalo G. de Polavieja
Abstract:
Machine learning algorithms use error function minimization to fit a large set of parameters in a preexisting model. However, error minimization eventually leads to a memorization of the training dataset, losing the ability to generalize to other datasets. To achieve generalization something else is needed, for example a regularization method or stop** the training when error in a validation dat…
▽ More
Machine learning algorithms use error function minimization to fit a large set of parameters in a preexisting model. However, error minimization eventually leads to a memorization of the training dataset, losing the ability to generalize to other datasets. To achieve generalization something else is needed, for example a regularization method or stop** the training when error in a validation dataset is minimal. Here we propose a different approach to learning and generalization that is parameter-free, fully discrete and that does not use function minimization. We use the training data to find an algebraic representation with minimal size and maximal freedom, explicitly expressed as a product of irreducible components. This algebraic representation is shown to directly generalize, giving high accuracy in test data, more so the smaller the representation. We prove that the number of generalizing representations can be very large and the algebra only needs to find one. We also derive and test a relationship between compression and error rate. We give results for a simple problem solved step by step, hand-written character recognition, and the Queens Completion problem as an example of unsupervised learning. As an alternative to statistical learning, algebraic learning may offer advantages in combining bottom-up and top-down information, formal concept derivation from data and large-scale parallelization.
△ Less
Submitted 15 March, 2018; v1 submitted 14 March, 2018;
originally announced March 2018.
-
idtracker.ai: Tracking all individuals in large collectives of unmarked animals
Authors:
Francisco Romero-Ferrero,
Mattia G. Bergomi,
Robert Hinz,
Francisco J. H. Heras,
Gonzalo G. de Polavieja
Abstract:
Our understanding of collective animal behavior is limited by our ability to track each of the individuals. We describe an algorithm and software, idtracker.ai, that extracts from video all trajectories with correct identities at a high accuracy for collectives of up to 100 individuals. It uses two deep networks, one detecting when animals touch or cross and another one for animal identification,…
▽ More
Our understanding of collective animal behavior is limited by our ability to track each of the individuals. We describe an algorithm and software, idtracker.ai, that extracts from video all trajectories with correct identities at a high accuracy for collectives of up to 100 individuals. It uses two deep networks, one detecting when animals touch or cross and another one for animal identification, trained adaptively to conditions and difficulty of the video.
△ Less
Submitted 12 March, 2018;
originally announced March 2018.
-
Bayesian Social Influence in the Online Realm
Authors:
Przemyslaw A. Grabowicz,
Francisco Romero-Ferrero,
Theo Lins,
Fabrício Benevenuto,
Krishna P. Gummadi,
Gonzalo G. de Polavieja
Abstract:
Our opinions, which things we like or dislike, depend on the opinions of those around us. Nowadays, we are influenced by the opinions of online strangers, expressed in comments and ratings on online platforms. Here, we perform novel "academic A/B testing" experiments with over 2,500 participants to measure the extent of that influence. In our experiments, the participants watch and evaluate videos…
▽ More
Our opinions, which things we like or dislike, depend on the opinions of those around us. Nowadays, we are influenced by the opinions of online strangers, expressed in comments and ratings on online platforms. Here, we perform novel "academic A/B testing" experiments with over 2,500 participants to measure the extent of that influence. In our experiments, the participants watch and evaluate videos on mirror proxies of YouTube and Vimeo. We control the comments and ratings that are shown underneath each of these videos. Our study shows that from 5$\%$ up to 40$\%$ of subjects adopt the majority opinion of strangers expressed in the comments. Using Bayes' theorem, we derive a flexible and interpretable family of models of social influence, in which each individual forms posterior opinions stochastically following a logit model. The variants of our mixture model that maximize Akaike information criterion represent two sub-populations, i.e., non-influenceable and influenceable individuals. The prior opinions of the non-influenceable individuals are strongly correlated with the external opinions and have low standard error, whereas the prior opinions of influenceable individuals have high standard error and become correlated with the external opinions due to social influence. Our findings suggest that opinions are random variables updated via Bayes' rule whose standard deviation is correlated with opinion influenceability. Based on these findings, we discuss how to hinder opinion manipulation and misinformation diffusion in the online realm.
△ Less
Submitted 26 February, 2020; v1 submitted 2 December, 2015;
originally announced December 2015.
-
A Model of Decision-Making in Groups of Humans
Authors:
Gabriel Madirolas,
Alfonso Perez-Escudero,
Gonzalo G. de Polavieja
Abstract:
Decisions by humans depend on their estimations given some uncertain sensory data. These decisions can also be influenced by the behavior of others. Here we present a mathematical model to quantify this influence, inviting a further study on the cognitive consequences of social information. We also expect that the present model can be used for a better understanding of the neural circuits implicat…
▽ More
Decisions by humans depend on their estimations given some uncertain sensory data. These decisions can also be influenced by the behavior of others. Here we present a mathematical model to quantify this influence, inviting a further study on the cognitive consequences of social information. We also expect that the present model can be used for a better understanding of the neural circuits implicated in social processing.
△ Less
Submitted 24 September, 2012;
originally announced September 2012.
-
Collective Animal Behavior from Bayesian Estimation and Probability Matching
Authors:
Alfonso Pérez-Escudero,
Gonzalo G. de Polavieja
Abstract:
Animals living in groups make movement decisions that depend, among other factors, on social interactions with other group members. Our present understanding of social rules in animal collectives is mainly based on empirical fits to observations, with less emphasis in obtaining first-principles approaches that allow their derivation. Here we show that patterns of collective decisions can be derive…
▽ More
Animals living in groups make movement decisions that depend, among other factors, on social interactions with other group members. Our present understanding of social rules in animal collectives is mainly based on empirical fits to observations, with less emphasis in obtaining first-principles approaches that allow their derivation. Here we show that patterns of collective decisions can be derived from the basic ability of animals to make probabilistic estimations in the presence of uncertainty. We build a decision-making model with two stages: Bayesian estimation and probabilistic matching. In the first stage, each animal makes a Bayesian estimation of which behavior is best to perform taking into account personal information about the environment and social information collected by observing the behaviors of other animals. In the probability matching stage, each animal chooses a behavior with a probability equal to the Bayesian-estimated probability that this behavior is the most appropriate one. This model derives very simple rules of interaction in animal collectives that depend only on two types of reliability parameters, one that each animal assigns to the other animals and another given by the quality of the non-social information. We test our model by obtaining theoretically a rich set of observed collective patterns of decisions in three-spined sticklebacks, Gasterosteus aculeatus, a shoaling fish species. The quantitative link shown between probabilistic estimation and collective rules of behavior allows a better contact with other fields such as foraging, mate selection, neurobiology and psychology, and gives predictions for experiments directly testing the relationship between estimation and collective behavior.
△ Less
Submitted 7 November, 2011; v1 submitted 5 May, 2011;
originally announced May 2011.
