-
Evolving Self-Assembling Neural Networks: From Spontaneous Activity to Experience-Dependent Learning
Authors:
Erwan Plantec,
Joachin W. Pedersen,
Milton L. Montero,
Eleni Nisioti,
Sebastian Risi
Abstract:
Biological neural networks are characterized by their high degree of plasticity, a core property that enables the remarkable adaptability of natural organisms. Importantly, this ability affects both the synaptic strength and the topology of the nervous systems. Artificial neural networks, on the other hand, have been mainly designed as static, fully connected structures that can be notoriously bri…
▽ More
Biological neural networks are characterized by their high degree of plasticity, a core property that enables the remarkable adaptability of natural organisms. Importantly, this ability affects both the synaptic strength and the topology of the nervous systems. Artificial neural networks, on the other hand, have been mainly designed as static, fully connected structures that can be notoriously brittle in the face of changing environments and novel inputs. Building on previous works on Neural Developmental Programs (NDPs), we propose a class of self-organizing neural networks capable of synaptic and structural plasticity in an activity and reward-dependent manner which we call Lifelong Neural Developmental Program (LNDP). We present an instance of such a network built on the graph transformer architecture and propose a mechanism for pre-experience plasticity based on the spontaneous activity of sensory neurons. Our results demonstrate the ability of the model to learn from experiences in different control tasks starting from randomly connected or empty networks. We further show that structural plasticity is advantageous in environments necessitating fast adaptation or with non-stationary rewards.
△ Less
Submitted 14 June, 2024;
originally announced June 2024.
-
Meta-Learning an Evolvable Developmental Encoding
Authors:
Milton L. Montero,
Erwan Plantec,
Eleni Nisioti,
Joachim W. Pedersen,
Sebastian Risi
Abstract:
Representations for black-box optimisation methods (such as evolutionary algorithms) are traditionally constructed using a delicate manual process. This is in contrast to the representation that maps DNAs to phenotypes in biological organisms, which is at the hear of biological complexity and evolvability. Additionally, the core of this process is fundamentally the same across nearly all forms of…
▽ More
Representations for black-box optimisation methods (such as evolutionary algorithms) are traditionally constructed using a delicate manual process. This is in contrast to the representation that maps DNAs to phenotypes in biological organisms, which is at the hear of biological complexity and evolvability. Additionally, the core of this process is fundamentally the same across nearly all forms of life, reflecting their shared evolutionary origin. Generative models have shown promise in being learnable representations for black-box optimisation but they are not per se designed to be easily searchable. Here we present a system that can meta-learn such representation by directly optimising for a representation's ability to generate quality-diversity. In more detail, we show our meta-learning approach can find one Neural Cellular Automata, in which cells can attend to different parts of a "DNA" string genome during development, enabling it to grow different solvable 2D maze structures. We show that the evolved genotype-to-phenotype map**s become more and more evolvable, not only resulting in a faster search but also increasing the quality and diversity of grown artefacts.
△ Less
Submitted 5 July, 2024; v1 submitted 13 June, 2024;
originally announced June 2024.
-
MindSet: Vision. A toolbox for testing DNNs on key psychological experiments
Authors:
Valerio Biscione,
Dong Yin,
Gaurav Malhotra,
Marin Dujmovic,
Milton L. Montero,
Guillermo Puebla,
Federico Adolfi,
Rachel F. Heaton,
John E. Hummel,
Benjamin D. Evans,
Karim Habashy,
Jeffrey S. Bowers
Abstract:
Multiple benchmarks have been developed to assess the alignment between deep neural networks (DNNs) and human vision. In almost all cases these benchmarks are observational in the sense they are composed of behavioural and brain responses to naturalistic images that have not been manipulated to test hypotheses regarding how DNNs or humans perceive and identify objects. Here we introduce the toolbo…
▽ More
Multiple benchmarks have been developed to assess the alignment between deep neural networks (DNNs) and human vision. In almost all cases these benchmarks are observational in the sense they are composed of behavioural and brain responses to naturalistic images that have not been manipulated to test hypotheses regarding how DNNs or humans perceive and identify objects. Here we introduce the toolbox MindSet: Vision, consisting of a collection of image datasets and related scripts designed to test DNNs on 30 psychological findings. In all experimental conditions, the stimuli are systematically manipulated to test specific hypotheses regarding human visual perception and object recognition. In addition to providing pre-generated datasets of images, we provide code to regenerate these datasets, offering many configurable parameters which greatly extend the dataset versatility for different research contexts, and code to facilitate the testing of DNNs on these image datasets using three different methods (similarity judgments, out-of-distribution classification, and decoder method), accessible at https://github.com/MindSetVision/mindset-vision. We test ResNet-152 on each of these methods as an example of how the toolbox can be used.
△ Less
Submitted 8 April, 2024;
originally announced April 2024.
-
Neural Network Training on Encrypted Data with TFHE
Authors:
Luis Montero,
Jordan Frery,
Celia Kherfallah,
Roman Bredehoft,
Andrei Stoian
Abstract:
We present an approach to outsourcing of training neural networks while preserving data confidentiality from malicious parties. We use fully homomorphic encryption to build a unified training approach that works on encrypted data and learns quantized neural network models. The data can be horizontally or vertically split between multiple parties, enabling collaboration on confidential data. We tra…
▽ More
We present an approach to outsourcing of training neural networks while preserving data confidentiality from malicious parties. We use fully homomorphic encryption to build a unified training approach that works on encrypted data and learns quantized neural network models. The data can be horizontally or vertically split between multiple parties, enabling collaboration on confidential data. We train logistic regression and multi-layer perceptrons on several datasets.
△ Less
Submitted 29 January, 2024;
originally announced January 2024.
-
Privacy-Preserving Tree-Based Inference with TFHE
Authors:
Jordan Frery,
Andrei Stoian,
Roman Bredehoft,
Luis Montero,
Celia Kherfallah,
Benoit Chevallier-Mames,
Arthur Meyre
Abstract:
Privacy enhancing technologies (PETs) have been proposed as a way to protect the privacy of data while still allowing for data analysis. In this work, we focus on Fully Homomorphic Encryption (FHE), a powerful tool that allows for arbitrary computations to be performed on encrypted data. FHE has received lots of attention in the past few years and has reached realistic execution times and correctn…
▽ More
Privacy enhancing technologies (PETs) have been proposed as a way to protect the privacy of data while still allowing for data analysis. In this work, we focus on Fully Homomorphic Encryption (FHE), a powerful tool that allows for arbitrary computations to be performed on encrypted data. FHE has received lots of attention in the past few years and has reached realistic execution times and correctness.
More precisely, we explain in this paper how we apply FHE to tree-based models and get state-of-the-art solutions over encrypted tabular data. We show that our method is applicable to a wide range of tree-based models, including decision trees, random forests, and gradient boosted trees, and has been implemented within the Concrete-ML library, which is open-source at https://github.com/zama-ai/concrete-ml. With a selected set of use-cases, we demonstrate that our FHE version is very close to the unprotected version in terms of accuracy.
△ Less
Submitted 7 August, 2023; v1 submitted 13 February, 2023;
originally announced March 2023.
-
Deep Neural Networks for Encrypted Inference with TFHE
Authors:
Andrei Stoian,
Jordan Frery,
Roman Bredehoft,
Luis Montero,
Celia Kherfallah,
Benoit Chevallier-Mames
Abstract:
Fully homomorphic encryption (FHE) is an encryption method that allows to perform computation on encrypted data, without decryption. FHE preserves the privacy of the users of online services that handle sensitive data, such as health data, biometrics, credit scores and other personal information. A common way to provide a valuable service on such data is through machine learning and, at this time,…
▽ More
Fully homomorphic encryption (FHE) is an encryption method that allows to perform computation on encrypted data, without decryption. FHE preserves the privacy of the users of online services that handle sensitive data, such as health data, biometrics, credit scores and other personal information. A common way to provide a valuable service on such data is through machine learning and, at this time, Neural Networks are the dominant machine learning model for unstructured data. In this work we show how to construct Deep Neural Networks (DNN) that are compatible with the constraints of TFHE, an FHE scheme that allows arbitrary depth computation circuits. We discuss the constraints and show the architecture of DNNs for two computer vision tasks. We benchmark the architectures using the Concrete stack, an open-source implementation of TFHE.
△ Less
Submitted 13 February, 2023;
originally announced February 2023.
-
Lost in Latent Space: Disentangled Models and the Challenge of Combinatorial Generalisation
Authors:
Milton L. Montero,
Jeffrey S. Bowers,
Rui Ponte Costa,
Casimir J. H. Ludwig,
Gaurav Malhotra
Abstract:
Recent research has shown that generative models with highly disentangled representations fail to generalise to unseen combination of generative factor values. These findings contradict earlier research which showed improved performance in out-of-training distribution settings when compared to entangled representations. Additionally, it is not clear if the reported failures are due to (a) encoders…
▽ More
Recent research has shown that generative models with highly disentangled representations fail to generalise to unseen combination of generative factor values. These findings contradict earlier research which showed improved performance in out-of-training distribution settings when compared to entangled representations. Additionally, it is not clear if the reported failures are due to (a) encoders failing to map novel combinations to the proper regions of the latent space or (b) novel combinations being mapped correctly but the decoder/downstream process is unable to render the correct output for the unseen combinations. We investigate these alternatives by testing several models on a range of datasets and training settings. We find that (i) when models fail, their encoders also fail to map unseen combinations to correct regions of the latent space and (ii) when models succeed, it is either because the test conditions do not exclude enough examples, or because excluded generative factors determine independent parts of the output image. Based on these results, we argue that to generalise properly, models not only need to capture factors of variation, but also understand how to invert the generative process that was used to generate the data.
△ Less
Submitted 14 June, 2024; v1 submitted 5 April, 2022;
originally announced April 2022.
-
Vertex removal in biclique graphs
Authors:
Leandro Montero
Abstract:
A \textit{biclique} is a maximal induced complete bipartite subgraph. The \textit{biclique graph} of a graph $H$, denoted by $KB(H)$, is the intersection graph of the family of all bicliques of $H$. In this work we address the following question: Given a biclique graph $G=KB(H)$, is it possible to remove a vertex $q$ of $G$, such that $G - \{q\}$ is a biclique graph? And if possible, can we obtain…
▽ More
A \textit{biclique} is a maximal induced complete bipartite subgraph. The \textit{biclique graph} of a graph $H$, denoted by $KB(H)$, is the intersection graph of the family of all bicliques of $H$. In this work we address the following question: Given a biclique graph $G=KB(H)$, is it possible to remove a vertex $q$ of $G$, such that $G - \{q\}$ is a biclique graph? And if possible, can we obtain a graph $H'$ such that $G - \{q\} = KB(H')$? We show that the general question has a "no" for answer. However, we prove that if $G$ has a vertex $q$ such that $d(q) = 2$, then $G-\{q\}$ is a biclique graph and we show how to obtain $H'$.
△ Less
Submitted 30 May, 2022; v1 submitted 8 June, 2020;
originally announced June 2020.
-
On the edge-biclique graph and the iterated edge-biclique operator
Authors:
Leandro Montero,
Sylvain Legay
Abstract:
A biclique of a graph $G$ is a maximal induced complete bipartite subgraph of $G$. The edge-biclique graph of $G$, $KB_e(G)$, is the edge-intersection graph of the bicliques of $G$. A graph $G$ diverges (resp. converges or is periodic) under an operator $H$ whenever $\lim_{k \rightarrow \infty}|V(H^k(G))|=\infty$ (resp. $\lim_{k \rightarrow \infty}H^k(G)=H^m(G)$ for some $m$ or…
▽ More
A biclique of a graph $G$ is a maximal induced complete bipartite subgraph of $G$. The edge-biclique graph of $G$, $KB_e(G)$, is the edge-intersection graph of the bicliques of $G$. A graph $G$ diverges (resp. converges or is periodic) under an operator $H$ whenever $\lim_{k \rightarrow \infty}|V(H^k(G))|=\infty$ (resp. $\lim_{k \rightarrow \infty}H^k(G)=H^m(G)$ for some $m$ or $H^k(G)=H^{k+s}(G)$ for some $k$ and $s \geq 2$). The iterated edge-biclique graph of $G$, $KB_e^k(G)$, is the graph obtained by applying the edge-biclique operator $k$ successive times to $G$. In this paper, we first study the connectivity relation between $G$ and $KB_e(G)$. Next, we study the iterated edge-biclique operator $KB_e$. In particular, we give sufficient conditions for a graph to be convergent or divergent under the operator $KB_e$, we characterize the behavior of \textit{burgeon graphs} and we propose some general conjectures on the subject.
△ Less
Submitted 29 November, 2021; v1 submitted 19 August, 2019;
originally announced August 2019.
-
Structural properties of biclique graphs and the distance formula
Authors:
Marina Groshaus,
Leandro Montero
Abstract:
A \textit{biclique} is a maximal induced complete bipartite subgraph of $G$. The \textit{biclique graph} of a graph $G$, denoted by $KB(G)$, is the intersection graph of the family of all bicliques of $G$. In this work we study some structural properties of biclique graphs which are necessary conditions for a graph to be a biclique graph. In particular, we prove that for biclique graphs that are n…
▽ More
A \textit{biclique} is a maximal induced complete bipartite subgraph of $G$. The \textit{biclique graph} of a graph $G$, denoted by $KB(G)$, is the intersection graph of the family of all bicliques of $G$. In this work we study some structural properties of biclique graphs which are necessary conditions for a graph to be a biclique graph. In particular, we prove that for biclique graphs that are neither a $K_3$ nor a \textit{diamond}, the number of vertices of degree $2$ is less than half the number of vertices in the graph. Also, we present forbidden structures. For this, we introduce a natural definition of the distance between bicliques in a graph. We give a formula that relates the distance between bicliques in a graph $G$ and the distance between their respective vertices in $KB(G)$. Using these results, we can prove not only this new necessary condition involving the degree, but also that some graphs are not biclique graphs. For example, we show that the \textit{crown} is the smallest graph that is not a biclique graph although the known necessary condition for biclique graphs holds, answering an open problem about biclique graphs. Finally, we present some interesting related conjectures and open problems.
△ Less
Submitted 1 September, 2021; v1 submitted 31 August, 2017;
originally announced August 2017.
-
Tight lower bounds on the number of bicliques in false-twin-free graphs
Authors:
Marina Groshaus,
Leandro Montero
Abstract:
A \emph{biclique} is a maximal bipartite complete induced subgraph of $G$. Bicliques have been studied in the last years motivated by the large number of applications. In particular, enumeration of the maximal bicliques has been of interest in data analysis. Associated with this issue, bounds on the maximum number of bicliques were given. In this paper we study bounds on the minimun number of bicl…
▽ More
A \emph{biclique} is a maximal bipartite complete induced subgraph of $G$. Bicliques have been studied in the last years motivated by the large number of applications. In particular, enumeration of the maximal bicliques has been of interest in data analysis. Associated with this issue, bounds on the maximum number of bicliques were given. In this paper we study bounds on the minimun number of bicliques of a graph. Since adding false-twin vertices to $G$ does not change the number of bicliques, we restrict to false-twin-free graphs. We give a tight lower bound on the minimum number bicliques for a subclass of $\{C_4$,false-twin$\}$-free graphs and for the class of $\{K_3$,false-twin$\}$-free graphs. Finally we discuss the problem for general graphs.
△ Less
Submitted 11 May, 2016; v1 submitted 5 October, 2015;
originally announced October 2015.
-
Tropical Dominating Sets in Vertex-Coloured Graphs
Authors:
J. -A. Angles d'Auriac,
Cs. Bujtas,
A. El Maftouhi,
M. Karpinski,
Y. Manoussakis,
L. Montero,
N. Narayanan,
L. Rosaz,
J. Thapper,
Zs. Tuza
Abstract:
Given a vertex-coloured graph, a dominating set is said to be tropical if every colour of the graph appears at least once in the set. Here, we study minimum tropical dominating sets from structural and algorithmic points of view. First, we prove that the tropical dominating set problem is NP-complete even when restricted to a simple path. Then, we establish upper bounds related to various paramete…
▽ More
Given a vertex-coloured graph, a dominating set is said to be tropical if every colour of the graph appears at least once in the set. Here, we study minimum tropical dominating sets from structural and algorithmic points of view. First, we prove that the tropical dominating set problem is NP-complete even when restricted to a simple path. Then, we establish upper bounds related to various parameters of the graph such as minimum degree and number of edges. We also give upper bounds for random graphs. Last, we give approximability and inapproximability results for general and restricted classes of graphs, and establish a FPT algorithm for interval graphs.
△ Less
Submitted 18 January, 2016; v1 submitted 3 March, 2015;
originally announced March 2015.
-
Proper Hamiltonian Cycles in Edge-Colored Multigraphs
Authors:
Raquel Águeda,
Valentin Borozan,
Raquel Díaz,
Yannis Manoussakis,
Leandro Montero
Abstract:
A $c$-edge-colored multigraph has each edge colored with one of the $c$ available colors and no two parallel edges have the same color. A proper Hamiltonian cycle is a cycle containing all the vertices of the multigraph such that no two adjacent edges have the same color. In this work we establish sufficient conditions for a multigraph to have a proper Hamiltonian cycle, depending on several param…
▽ More
A $c$-edge-colored multigraph has each edge colored with one of the $c$ available colors and no two parallel edges have the same color. A proper Hamiltonian cycle is a cycle containing all the vertices of the multigraph such that no two adjacent edges have the same color. In this work we establish sufficient conditions for a multigraph to have a proper Hamiltonian cycle, depending on several parameters such as the number of edges and the rainbow degree.
△ Less
Submitted 13 February, 2017; v1 submitted 19 November, 2014;
originally announced November 2014.
-
Almost every graph is divergent under the biclique operator
Authors:
Marina Groshaus,
André Guedes,
Leandro Montero
Abstract:
A biclique of a graph $G$ is a maximal induced complete bipartite subgraph of $G$. The biclique graph of $G$ denoted by $KB(G)$, is the intersection graph of all the bicliques of $G$. The biclique graph can be thought as an operator between graphs. The iterated biclique graph of $G$ denoted by $KB^{k}(G)$, is the graph obtained by applying the biclique operator $k$ successive times to $G$. The ass…
▽ More
A biclique of a graph $G$ is a maximal induced complete bipartite subgraph of $G$. The biclique graph of $G$ denoted by $KB(G)$, is the intersection graph of all the bicliques of $G$. The biclique graph can be thought as an operator between graphs. The iterated biclique graph of $G$ denoted by $KB^{k}(G)$, is the graph obtained by applying the biclique operator $k$ successive times to $G$. The associated problem is deciding whether an input graph converges, diverges or is periodic under the biclique operator when $k$ grows to infinity. All possible behaviors were characterized recently and an $O(n^4)$ algorithm for deciding the behavior of any graph under the biclique operator was also given. In this work we prove new structural results of biclique graphs. In particular, we prove that every false-twin-free graph with at least $13$ vertices is divergent. These results lead to a linear time algorithm to solve the same problem.
△ Less
Submitted 31 August, 2015; v1 submitted 26 August, 2014;
originally announced August 2014.
-
Proper Hamiltonian Paths in Edge-Coloured Multigraphs
Authors:
Raquel Águeda,
Valentin Borozan,
Marina Groshaus,
Yannis Manoussakis,
Gervais Mendy,
Leandro Montero
Abstract:
Given a $c$-edge-coloured multigraph, a proper Hamiltonian path is a path that contains all the vertices of the multigraph such that no two adjacent edges have the same colour. In this work we establish sufficient conditions for an edge-coloured multigraph to guarantee the existence of a proper Hamiltonian path, involving various parameters as the number of edges, the number of colours, the rainbo…
▽ More
Given a $c$-edge-coloured multigraph, a proper Hamiltonian path is a path that contains all the vertices of the multigraph such that no two adjacent edges have the same colour. In this work we establish sufficient conditions for an edge-coloured multigraph to guarantee the existence of a proper Hamiltonian path, involving various parameters as the number of edges, the number of colours, the rainbow degree and the connectivity.
△ Less
Submitted 20 June, 2014;
originally announced June 2014.
-
Further results on strong edge-colourings in outerplanar graphs
Authors:
Valentin Borozan,
Leandro Montero,
Narayanan Narayanan
Abstract:
An edge-colouring is {\em strong} if every colour class is an induced matching. In this work we give a formulae that determines either the optimal or the optimal plus one strong chromatic index of bipartite outerplanar graphs. Further, we give an improved upper bound for any outerplanar graph which is close to optimal. All our proofs yield efficient algorithms to construct such colourings.
An edge-colouring is {\em strong} if every colour class is an induced matching. In this work we give a formulae that determines either the optimal or the optimal plus one strong chromatic index of bipartite outerplanar graphs. Further, we give an improved upper bound for any outerplanar graph which is close to optimal. All our proofs yield efficient algorithms to construct such colourings.
△ Less
Submitted 19 December, 2013;
originally announced December 2013.
-
A DDS-Based Scalable and Reconfigurable Framework for Cyber-Physical Systems
Authors:
Ismael Etxeberria-Agiriano,
Isidro Calvo,
Liliana Montero,
Ivan Alonso
Abstract:
Cyber-Physical Systems (CPSs) involve the interconnection of heterogeneous computing devices which are closely integrated with the physical processes under control. Often, these systems are resource-constrained and require specific features such as the ability to adapt in a timeliness and efficient fashion to dynamic environments. Also, they must support fault tolerance and avoid single points of…
▽ More
Cyber-Physical Systems (CPSs) involve the interconnection of heterogeneous computing devices which are closely integrated with the physical processes under control. Often, these systems are resource-constrained and require specific features such as the ability to adapt in a timeliness and efficient fashion to dynamic environments. Also, they must support fault tolerance and avoid single points of failure. This paper describes a scalable framework for CPSs based on the OMG DDS standard. The proposed solution allows reconfiguring this kind of systems at run-time and managing efficiently their resources.
△ Less
Submitted 10 August, 2012;
originally announced August 2012.
