-
Deep Learning as Ricci Flow
Authors:
Anthony Baptista,
Alessandro Barp,
Tapabrata Chakraborti,
Chris Harbron,
Ben D. MacArthur,
Christopher R. S. Banerji
Abstract:
Deep neural networks (DNNs) are powerful tools for approximating the distribution of complex data. It is known that data passing through a trained DNN classifier undergoes a series of geometric and topological simplifications. While some progress has been made toward understanding these transformations in neural networks with smooth activation functions, an understanding in the more general settin…
▽ More
Deep neural networks (DNNs) are powerful tools for approximating the distribution of complex data. It is known that data passing through a trained DNN classifier undergoes a series of geometric and topological simplifications. While some progress has been made toward understanding these transformations in neural networks with smooth activation functions, an understanding in the more general setting of non-smooth activation functions, such as the rectified linear unit (ReLU), which tend to perform better, is required. Here we propose that the geometric transformations performed by DNNs during classification tasks have parallels to those expected under Hamilton's Ricci flow - a tool from differential geometry that evolves a manifold by smoothing its curvature, in order to identify its topology. To illustrate this idea, we present a computational framework to quantify the geometric changes that occur as data passes through successive layers of a DNN, and use this framework to motivate a notion of `global Ricci network flow' that can be used to assess a DNN's ability to disentangle complex data geometries to solve classification problems. By training more than $1,500$ DNN classifiers of different widths and depths on synthetic and real-world data, we show that the strength of global Ricci network flow-like behaviour correlates with accuracy for well-trained DNNs, independently of depth, width and data set. Our findings motivate the use of tools from differential and discrete geometry to the problem of explainability in deep learning.
△ Less
Submitted 22 April, 2024;
originally announced April 2024.
-
Increased signaling entropy in cancer requires the scale-free property of protein interaction networks
Authors:
Andrew E. Teschendorff,
Christopher R. S. Banerji,
Simone Severini,
Reimer Kuehn,
Peter Sollich
Abstract:
One of the key characteristics of cancer cells is an increased phenotypic plasticity, driven by underlying genetic and epigenetic perturbations. However, at a systems-level it is unclear how these perturbations give rise to the observed increased plasticity. Elucidating such systems-level principles is key for an improved understanding of cancer. Recently, it has been shown that signaling entropy,…
▽ More
One of the key characteristics of cancer cells is an increased phenotypic plasticity, driven by underlying genetic and epigenetic perturbations. However, at a systems-level it is unclear how these perturbations give rise to the observed increased plasticity. Elucidating such systems-level principles is key for an improved understanding of cancer. Recently, it has been shown that signaling entropy, an overall measure of signaling pathway promiscuity, and computable from integrating a sample's gene expression profile with a protein interaction network, correlates with phenotypic plasticity and is increased in cancer compared to normal tissue. Here we develop a computational framework for studying the effects of network perturbations on signaling entropy. We demonstrate that the increased signaling entropy of cancer is driven by two factors: (i) the scale-free (or near scale-free) topology of the interaction network, and (ii) a subtle positive correlation between differential gene expression and node connectivity. Indeed, we show that if protein interaction networks were random graphs, described by Poisson degree distributions, that cancer would generally not exhibit an increased signaling entropy. In summary, this work exposes a deep connection between cancer, signaling entropy and interaction network topology.
△ Less
Submitted 1 April, 2015;
originally announced April 2015.
-
Cellular network entropy as the energy potential in Waddington's differentiation landscape
Authors:
Christopher R. S. Banerji,
Diego Miranda-Saavedra,
Simone Severini,
Martin Widschwendter,
Tariq Enver,
Joseph X. Zhou,
Andrew E. Teschendorff
Abstract:
Differentiation is a key cellular process in normal tissue development that is significantly altered in cancer. Although molecular signatures characterising pluripotency and multipotency exist, there is, as yet, no single quantitative mark of a cellular sample's position in the global differentiation hierarchy. Here we adopt a systems view and consider the sample's network entropy, a measure of si…
▽ More
Differentiation is a key cellular process in normal tissue development that is significantly altered in cancer. Although molecular signatures characterising pluripotency and multipotency exist, there is, as yet, no single quantitative mark of a cellular sample's position in the global differentiation hierarchy. Here we adopt a systems view and consider the sample's network entropy, a measure of signaling pathway promiscuity, computable from a sample's genome-wide expression profile. We demonstrate that network entropy provides a quantitative, in-silico, readout of the average undifferentiated state of the profiled cells, recapitulating the known hierarchy of pluripotent, multipotent and differentiated cell types. Network entropy further exhibits dynamic changes in time course differentiation data, and in line with a sample's differentiation stage. In disease, network entropy predicts a higher level of cellular plasticity in cancer stem cell populations compared to ordinary cancer cells. Importantly, network entropy also allows identification of key differentiation pathways. Our results are consistent with the view that pluripotency is a statistical property defined at the cellular population level, correlating with intra-sample heterogeneity, and driven by the degree of signaling promiscuity in cells. In summary, network entropy provides a quantitative measure of a cell's undifferentiated state, defining its elevation in Waddington's landscape.
△ Less
Submitted 26 October, 2013;
originally announced October 2013.
-
A notion of graph likelihood and an infinite monkey theorem
Authors:
Christopher R. S. Banerji,
Toufik Mansour,
Simone Severini
Abstract:
We play with a graph-theoretic analogue of the folklore infinite monkey theorem. We define a notion of graph likelihood as the probability that a given graph is constructed by a monkey in a number of time steps equal to the number of vertices. We present an algorithm to compute this graph invariant and closed formulas for some infinite classes. We have to leave the computational complexity of the…
▽ More
We play with a graph-theoretic analogue of the folklore infinite monkey theorem. We define a notion of graph likelihood as the probability that a given graph is constructed by a monkey in a number of time steps equal to the number of vertices. We present an algorithm to compute this graph invariant and closed formulas for some infinite classes. We have to leave the computational complexity of the likelihood as an open problem.
△ Less
Submitted 12 April, 2013;
originally announced April 2013.
-
Network Transfer Entropy and Metric Space for Causality Inference
Authors:
Christopher R. S. Banerji,
Simone Severini,
Andrew E. Teschendorff
Abstract:
A measure is derived to quantify directed information transfer between pairs of vertices in a weighted network, over paths of a specified maximal length. Our approach employs a general, probabilistic model of network traffic, from which the informational distance between dynamics on two weighted networks can be naturally expressed as a Jensen Shannon Divergence (JSD). Our network transfer entropy…
▽ More
A measure is derived to quantify directed information transfer between pairs of vertices in a weighted network, over paths of a specified maximal length. Our approach employs a general, probabilistic model of network traffic, from which the informational distance between dynamics on two weighted networks can be naturally expressed as a Jensen Shannon Divergence (JSD). Our network transfer entropy measure is shown to be able to distinguish and quantify causal relationships between network elements, in applications to simple synthetic networks and a biological signalling network. We conclude with a theoretical extension of our framework, in which the square root of the JSD induces a metric on the space of dynamics on weighted networks. We prove a convergence criterion, demonstrating that a form of convergence in the structure of weighted networks in a family of matrix metric spaces implies convergence of their dynamics with respect to the square root JSD metric.
△ Less
Submitted 2 November, 2013; v1 submitted 1 March, 2013;
originally announced March 2013.
