Showing 1–50 of 89 results for author: Zenil, H

Scientific Hypothesis Generation by a Large Language Model: Laboratory Validation in Breast Cancer Treatment

Authors: Abbi Abdel-Rehim, Hector Zenil, Oghenejokpeme Orhobor, Marie Fisher, Ross J. Collins, Elizabeth Bourne, Gareth W. Fearnley, Emma Tate, Holly X. Smith, Larisa N. Soldatova, Ross D. King

Abstract: Large language models (LLMs) have transformed AI and achieved breakthrough performance on a wide range of tasks that require human intelligence. In science, perhaps the most interesting application of LLMs is for hypothesis formation. A feature of LLMs, which results from their probabilistic structure, is that the output text is not necessarily a valid inference from the training text. These are '… ▽ More Large language models (LLMs) have transformed AI and achieved breakthrough performance on a wide range of tasks that require human intelligence. In science, perhaps the most interesting application of LLMs is for hypothesis formation. A feature of LLMs, which results from their probabilistic structure, is that the output text is not necessarily a valid inference from the training text. These are 'hallucinations', and are a serious problem in many applications. However, in science, hallucinations may be useful: they are novel hypotheses whose validity may be tested by laboratory experiments. Here we experimentally test the use of LLMs as a source of scientific hypotheses using the domain of breast cancer treatment. We applied the LLM GPT4 to hypothesize novel pairs of FDA-approved non-cancer drugs that target the MCF7 breast cancer cell line relative to the non-tumorigenic breast cell line MCF10A. In the first round of laboratory experiments GPT4 succeeded in discovering three drug combinations (out of 12 tested) with synergy scores above the positive controls. These combinations were itraconazole + atenolol, disulfiram + simvastatin and dipyridamole + mebendazole. GPT4 was then asked to generate new combinations after considering its initial results. It then discovered three more combinations with positive synergy scores (out of four tested), these were disulfiram + fulvestrant, mebendazole + quinacrine and disulfiram + quinacrine. A limitation of GPT4 as a generator of hypotheses was that its explanations for them were formulaic and unconvincing. We conclude that LLMs are an exciting novel source of scientific hypotheses. △ Less

Submitted 5 June, 2024; v1 submitted 20 May, 2024; originally announced May 2024.

Comments: 13 pages, 6 tables, 1 figure. Supplementary information available

Decoding Geometric Properties in Non-Random Data from First Information-Theoretic Principles

Authors: Hector Zenil, Felipe S. Abrahão

Abstract: Based on the principles of information theory, measure theory, and theoretical computer science, we introduce a univariate signal deconvolution method with a wide range of applications to coding theory, particularly in zero-knowledge one-way communication channels, such as in deciphering messages from unknown generating sources about which no prior knowledge is available and to which no return mes… ▽ More Based on the principles of information theory, measure theory, and theoretical computer science, we introduce a univariate signal deconvolution method with a wide range of applications to coding theory, particularly in zero-knowledge one-way communication channels, such as in deciphering messages from unknown generating sources about which no prior knowledge is available and to which no return message can be sent. Our multidimensional space reconstruction method from an arbitrary received signal is proven to be agnostic vis-a-vis the encoding-decoding scheme, computation model, programming language, formal theory, the computable (or semi-computable) method of approximation to algorithmic complexity, and any arbitrarily chosen (computable) probability measure of the events. The method derives from the principles of an approach to Artificial General Intelligence capable of building a general-purpose model of models independent of any arbitrarily assumed prior probability distribution. We argue that this optimal and universal method of decoding non-random data has applications to signal processing, causal deconvolution, topological and geometric properties encoding, cryptography, and bio- and technosignature detection. △ Less

Submitted 17 May, 2024; v1 submitted 13 May, 2024; originally announced May 2024.

Comments: arXiv:2303.16045 is based on this paper. arXiv admin note: substantial text overlap with arXiv:2303.16045

Fractal spatio-temporal scale-free messaging: amplitude modulation of self-executable carriers given by the Weierstrass function's components

Authors: Hector Zenil, Luan Carlos de Sena Monteiro

Abstract: In many communication contexts, the capabilities of the involved actors cannot be known beforehand, whether it is a cell, a plant, an insect, or even a life form unknown to Earth. Regardless of the recipient, the message space and time scale could be too fast, too slow, too large, or too small and may never be decoded. Therefore, it pays to devise a way to encode messages agnostic of space and tim… ▽ More In many communication contexts, the capabilities of the involved actors cannot be known beforehand, whether it is a cell, a plant, an insect, or even a life form unknown to Earth. Regardless of the recipient, the message space and time scale could be too fast, too slow, too large, or too small and may never be decoded. Therefore, it pays to devise a way to encode messages agnostic of space and time scales. We propose the use of fractal functions as self-executable infinite-frequency carriers for sending messages, given their properties of structural self-similarity and scale invariance. We call it `fractal messaging'. Starting from a spatial embedding, we introduce a framework for a space-time scale-free messaging approach to this challenge. When considering a space and time-agnostic framework for message transmission, it would be interesting to encode a message such that it could be decoded at several spatio-temporal scales. Hence, the core idea of the framework proposed herein is to encode a binary message as waves along infinitely many frequencies (in power-like distributions) and amplitudes, transmit such a message, and then decode and reproduce it. To do so, the components of the Weierstrass function, a known fractal, are used as carriers of the message. Each component will have its amplitude modulated to embed the binary stream, allowing for a space-time-agnostic approach to messaging. △ Less

Submitted 1 April, 2024; v1 submitted 11 March, 2024; originally announced March 2024.

Comments: 15 pages + appendix (21 pages total)

Assembly Theory is an approximation to algorithmic complexity based on LZ compression that does not explain selection or evolution

Authors: Felipe S. Abrahão, Santiago Hernández-Orozco, Narsis A. Kiani, Jesper Tegnér, Hector Zenil

Abstract: We prove the full equivalence between Assembly Theory (AT) and Shannon Entropy via a method based upon the principles of statistical compression renamed `assembly index' that belongs to the LZ family of popular compression algorithms (ZIP, GZIP, JPEG). Such popular algorithms have been shown to empirically reproduce the results of AT, results that have also been reported before in successful appli… ▽ More We prove the full equivalence between Assembly Theory (AT) and Shannon Entropy via a method based upon the principles of statistical compression renamed `assembly index' that belongs to the LZ family of popular compression algorithms (ZIP, GZIP, JPEG). Such popular algorithms have been shown to empirically reproduce the results of AT, results that have also been reported before in successful applications to separating organic from non-organic molecules and in the context of the study of selection and evolution. We show that the assembly index value is equivalent to the size of a minimal context-free grammar. The statistical compressibility of such a method is bounded by Shannon Entropy and other equivalent traditional LZ compression schemes, such as LZ77, LZ78, or LZW. In addition, we demonstrate that AT, and the algorithms supporting its pathway complexity, assembly index, and assembly number, define compression schemes and methods that are subsumed into the theory of algorithmic (Kolmogorov-Solomonoff-Chaitin) complexity. Due to AT's current lack of logical consistency in defining causality for non-stochastic processes and the lack of empirical evidence that it outperforms other complexity measures found in the literature capable of explaining the same phenomena, we conclude that the assembly index and the assembly number do not lead to an explanation or quantification of biases in generative (physical or biological) processes, including those brought about by (abiotic or Darwinian) selection and evolution, that could not have been arrived at using Shannon Entropy or that have not been reported before using classical information theory or algorithmic complexity. △ Less

Submitted 1 April, 2024; v1 submitted 11 March, 2024; originally announced March 2024.

Comments: 15 pages + appendix, 2 figures

The Future of Fundamental Science Led by Generative Closed-Loop Artificial Intelligence

Authors: Hector Zenil, Jesper Tegnér, Felipe S. Abrahão, Alexander Lavin, Vipin Kumar, Jeremy G. Frey, Adrian Weller, Larisa Soldatova, Alan R. Bundy, Nicholas R. Jennings, Koichi Takahashi, Lawrence Hunter, Saso Dzeroski, Andrew Briggs, Frederick D. Gregory, Carla P. Gomes, Jon Rowe, James Evans, Hiroaki Kitano, Ross King

Abstract: Recent advances in machine learning and AI, including Generative AI and LLMs, are disrupting technological innovation, product development, and society as a whole. AI's contribution to technology can come from multiple approaches that require access to large training data sets and clear performance evaluation criteria, ranging from pattern recognition and classification to generative models. Yet,… ▽ More Recent advances in machine learning and AI, including Generative AI and LLMs, are disrupting technological innovation, product development, and society as a whole. AI's contribution to technology can come from multiple approaches that require access to large training data sets and clear performance evaluation criteria, ranging from pattern recognition and classification to generative models. Yet, AI has contributed less to fundamental science in part because large data sets of high-quality data for scientific practice and model discovery are more difficult to access. Generative AI, in general, and Large Language Models in particular, may represent an opportunity to augment and accelerate the scientific discovery of fundamental deep science with quantitative models. Here we explore and investigate aspects of an AI-driven, automated, closed-loop approach to scientific discovery, including self-driven hypothesis generation and open-ended autonomous exploration of the hypothesis space. Integrating AI-driven automation into the practice of science would mitigate current problems, including the replication of findings, systematic production of data, and ultimately democratisation of the scientific process. Realising these possibilities requires a vision for augmented AI coupled with a diversity of AI approaches able to deal with fundamental aspects of causality analysis and model discovery while enabling unbiased search across the space of putative explanations. These advances hold the promise to unleash AI's potential for searching and discovering the fundamental structure of our world beyond what human scientists have been able to achieve. Such a vision would push the boundaries of new fundamental science rather than automatize current workflows and instead open doors for technological innovation to tackle some of the greatest challenges facing humanity today. △ Less

Submitted 29 August, 2023; v1 submitted 9 July, 2023; originally announced July 2023.

Comments: 35 pages, first draft of the final report from the Alan Turing Institute on AI for Scientific Discovery

Classical-to-Quantum Transfer Learning Facilitates Machine Learning with Variational Quantum Circuit

Authors: Jun Qi, Chao-Han Huck Yang, Pin-Yu Chen, Min-Hsiu Hsieh, Hector Zenil, Jesper Tegner

Abstract: While Quantum Machine Learning (QML) is an exciting emerging area, the accuracy of the loss function still needs to be improved by the number of available qubits. Here, we reformulate the QML problem such that the approximation error (representation power) does not depend on the number of qubits. We prove that a classical-to-quantum transfer learning architecture using a Variational Quantum Circui… ▽ More While Quantum Machine Learning (QML) is an exciting emerging area, the accuracy of the loss function still needs to be improved by the number of available qubits. Here, we reformulate the QML problem such that the approximation error (representation power) does not depend on the number of qubits. We prove that a classical-to-quantum transfer learning architecture using a Variational Quantum Circuit (VQC) improves the representation and generalization (estimation error) capabilities of the VQC model. We derive analytical bounds for the approximation and estimation error. We show that the architecture of classical-to-quantum transfer learning leverages pre-trained classical generative AI models, making it easier to find the optimal parameters for the VQC in the training stage. To validate our theoretical analysis, we perform experiments on single-dot and double-dot binary classification tasks for charge stability diagrams in semiconductor quantum dots, where the related empirical results support our theoretical findings. Our analytical and empirical results demonstrate the effectiveness of classical-to-quantum transfer learning architecture in realistic tasks. This sets the stage for accelerating QML applications beyond the current limits of available qubits. △ Less

Submitted 18 June, 2024; v1 submitted 17 May, 2023; originally announced June 2023.

Comments: The paper needs a major revision before it could be submitted to a new journal, and the authors agree that the latest version could not be open to public at the moment

An Optimal, Universal and Agnostic Decoding Method for Message Reconstruction, Bio and Technosignature Detection

Authors: Hector Zenil, Alyssa Adams, Felipe S. Abrahão

Abstract: We present a signal reconstruction method for zero-knowledge one-way communication channels in which a receiver aims to interpret a message sent by an unknown source about which no prior knowledge is available and to which no return message can be sent. Our reconstruction method is agnostic vis-à-vis the arbitrarily chosen encoding-decoding scheme and other observer-dependent characteristics, such… ▽ More We present a signal reconstruction method for zero-knowledge one-way communication channels in which a receiver aims to interpret a message sent by an unknown source about which no prior knowledge is available and to which no return message can be sent. Our reconstruction method is agnostic vis-à-vis the arbitrarily chosen encoding-decoding scheme and other observer-dependent characteristics, such as the arbitrarily chosen computation model or underlying mathematical theory. We investigate how non-random messages may encode information about the physical properties, such as dimension and length scales of the space in which a signal or message may have been originally encoded, embedded, or generated. We argue that our results have applications to life and technosignature detection and to coding theory in general. △ Less

Submitted 9 May, 2024; v1 submitted 28 March, 2023; originally announced March 2023.

On the Salient Limitations of the Methods of Assembly Theory and their Classification of Molecular Biosignatures

Authors: Abicumaran Uthamacumaran, Felipe S. Abrahão, Narsis A. Kiani, Hector Zenil

Abstract: We demonstrate that the assembly pathway method underlying Assembly Theory (AT) is a dictionary-based encoding scheme for `counting copies', widely used by popular statistical compression algorithms, some of which have been applied to many areas, including systems biology, chemistry, and biosignature classification. We show that AT performs similarly in all cases (synthetic or natural) to other si… ▽ More We demonstrate that the assembly pathway method underlying Assembly Theory (AT) is a dictionary-based encoding scheme for `counting copies', widely used by popular statistical compression algorithms, some of which have been applied to many areas, including systems biology, chemistry, and biosignature classification. We show that AT performs similarly in all cases (synthetic or natural) to other simple coding schemes and underperforms compared to system-related indexes based upon algorithmic probability that take into account the likelihood of related computable approximations of similar events. Our results also demonstrate that simple (and tractably computable) modular instructions can mislead AT, leading to failure in practice in capturing the properties of physical systems. These theoretical and empirical results imply that the assembly index, whose computable nature is not an advantage, does not offer substantial improvements over existing methods. In contrast, other resource-bounded (therefore also computable) indexes that approximate algorithmic (Kolmogorov) complexity show the ability to separate organic from inorganic molecules and even perform better on the mass spectral information used by the authors of AT. We show the predictive power of these other system-driven indexes based on their solid foundations and empirical results. △ Less

Submitted 16 April, 2024; v1 submitted 30 September, 2022; originally announced October 2022.

Comments: 25 pages + 13 from the appendix, 5 figures

Algorithmic Information Dynamics of Cellular Automata

Authors: Hector Zenil, Alyssa Adams

Abstract: We illustrate an application of Algorithmic Information Dynamics to Cellular Automata (CA) demonstrating how this digital calculus is able to quantify change in discrete dynamical systems. We demonstrate the sensitivity of the Block Decomposition Method on 1D and 2D CA, including Conway's Game of Life, against measures of statistical nature such as compression (LZW) and Shannon Entropy in two diff… ▽ More We illustrate an application of Algorithmic Information Dynamics to Cellular Automata (CA) demonstrating how this digital calculus is able to quantify change in discrete dynamical systems. We demonstrate the sensitivity of the Block Decomposition Method on 1D and 2D CA, including Conway's Game of Life, against measures of statistical nature such as compression (LZW) and Shannon Entropy in two different contexts (1) perturbation analysis and (2) dynamic-state colliding CA. The approach is interesting because it analyses a quintessential object native to software space (CA) in software space itself by using algorithmic information dynamics through a model-driven universal search instead of a traditional statistical approach e.g. LZW compression or Shannon entropy. The colliding example of two state-independent (if not three as one is regulating the collision itself) discrete dynamical systems offers a potential proof of concept for the development of a multivariate version of the AID calculus. △ Less

Submitted 12 January, 2022; v1 submitted 24 December, 2021; originally announced December 2021.

Comments: Invited contribution to The Mathematical Artist: A Tribute to John Horton Conway, by World Scientific Publishing Press

A Simplicity Bubble Problem in Formal-Theoretic Learning Systems

Authors: Felipe S. Abrahão, Hector Zenil, Fabio Porto, Michael Winter, Klaus Wehmuth, Itala M. L. D'Ottaviano

Abstract: When mining large datasets in order to predict new data, limitations of the principles behind statistical machine learning pose a serious challenge not only to the Big Data deluge, but also to the traditional assumptions that data generating processes are biased toward low algorithmic complexity. Even when one assumes an underlying algorithmic-informational bias toward simplicity in finite dataset… ▽ More When mining large datasets in order to predict new data, limitations of the principles behind statistical machine learning pose a serious challenge not only to the Big Data deluge, but also to the traditional assumptions that data generating processes are biased toward low algorithmic complexity. Even when one assumes an underlying algorithmic-informational bias toward simplicity in finite dataset generators, we show that current approaches to machine learning (including deep learning, or any formal-theoretic hybrid mix of top-down AI and statistical machine learning approaches), can always be deceived, naturally or artificially, by sufficiently large datasets. In particular, we demonstrate that, for every learning algorithm (with or without access to a formal theory), there is a sufficiently large dataset size above which the algorithmic probability of an unpredictable deceiver is an upper bound (up to a multiplicative constant that only depends on the learning algorithm) for the algorithmic probability of any other larger dataset. In other words, very large and complex datasets can deceive learning algorithms into a ``simplicity bubble'' as likely as any other particular non-deceiving dataset. These deceiving datasets guarantee that any prediction effected by the learning algorithm will unpredictably diverge from the high-algorithmic-complexity globally optimal solution while converging toward the low-algorithmic-complexity locally optimal solution, although the latter is deemed a global one by the learning algorithm. We discuss the framework and additional empirical conditions to be met in order to circumvent this deceptive phenomenon, moving away from statistical machine learning towards a stronger type of machine learning based on, and motivated by, the intrinsic power of algorithmic information theory and computability theory. △ Less

Submitted 25 April, 2023; v1 submitted 22 December, 2021; originally announced December 2021.

Computable Model Discovery and High-Level-Programming Approximations to Algorithmic Complexity

Authors: Vladimir Lemusa, Eduardo Acuña, Víctor Zamora, Francisco Hernandez-Quiroz, Hector Zenil

Abstract: Motivated by algorithmic information theory, the problem of program discovery can help find candidates of underlying generative mechanisms of natural and artificial phenomena. The uncomputability of such inverse problem, however, significantly restricts a wider application of exhaustive methods. Here we present a proof of concept of an approach based on IMP, a high-level imperative programming lan… ▽ More Motivated by algorithmic information theory, the problem of program discovery can help find candidates of underlying generative mechanisms of natural and artificial phenomena. The uncomputability of such inverse problem, however, significantly restricts a wider application of exhaustive methods. Here we present a proof of concept of an approach based on IMP, a high-level imperative programming language. Its main advantage is that conceptually complex computational routines are more succinctly expressed, unlike lower-level models such as Turing machines or cellular automata. We investigate if a more expressive higher-level programming language can be more efficient at generating approximations to algorithmic complexity of recursive functions, often of particular mathematical interest. △ Less

Submitted 22 December, 2021; originally announced December 2021.

Comments: 23 pages

Simulation Intelligence: Towards a New Generation of Scientific Methods

Authors: Alexander Lavin, David Krakauer, Hector Zenil, Justin Gottschlich, Tim Mattson, Johann Brehmer, Anima Anandkumar, Sanjay Choudry, Kamil Rocki, Atılım Güneş Baydin, Carina Prunkl, Brooks Paige, Olexandr Isayev, Erik Peterson, Peter L. McMahon, Jakob Macke, Kyle Cranmer, Jiaxin Zhang, Haruko Wainwright, Adi Hanuka, Manuela Veloso, Samuel Assefa, Stephan Zheng, Avi Pfeffer

Abstract: The original "Seven Motifs" set forth a roadmap of essential methods for the field of scientific computing, where a motif is an algorithmic method that captures a pattern of computation and data movement. We present the "Nine Motifs of Simulation Intelligence", a roadmap for the development and integration of the essential algorithms necessary for a merger of scientific computing, scientific simul… ▽ More The original "Seven Motifs" set forth a roadmap of essential methods for the field of scientific computing, where a motif is an algorithmic method that captures a pattern of computation and data movement. We present the "Nine Motifs of Simulation Intelligence", a roadmap for the development and integration of the essential algorithms necessary for a merger of scientific computing, scientific simulation, and artificial intelligence. We call this merger simulation intelligence (SI), for short. We argue the motifs of simulation intelligence are interconnected and interdependent, much like the components within the layers of an operating system. Using this metaphor, we explore the nature of each layer of the simulation intelligence operating system stack (SI-stack) and the motifs therein: (1) Multi-physics and multi-scale modeling; (2) Surrogate modeling and emulation; (3) Simulation-based inference; (4) Causal modeling and inference; (5) Agent-based modeling; (6) Probabilistic programming; (7) Differentiable programming; (8) Open-ended optimization; (9) Machine programming. We believe coordinated efforts between motifs offers immense opportunity to accelerate scientific discovery, from solving inverse problems in synthetic biology and climate science, to directing nuclear energy experiments and predicting emergent behavior in socioeconomic settings. We elaborate on each layer of the SI-stack, detailing the state-of-art methods, presenting examples to highlight challenges and opportunities, and advocating for specific ways to advance the motifs and the synergies from their combinations. Advancing and integrating these technologies can enable a robust and efficient hypothesis-simulation-analysis type of scientific method, which we introduce with several use-cases for human-machine teaming and automated science. △ Less

Submitted 27 November, 2022; v1 submitted 6 December, 2021; originally announced December 2021.

A Computable Piece of Uncomputable Art whose Expansion May Explain the Universe in Software Space

Authors: Hector Zenil

Abstract: At the intersection of what I call uncomputable art and computational epistemology, a form of experimental philosophy, we find an exciting and promising area of science related to causation with an alternative, possibly best possible, solution to the challenge of the inverse problem. That is the problem of finding the possible causes, mechanistic origins, first principles, and generative models of… ▽ More At the intersection of what I call uncomputable art and computational epistemology, a form of experimental philosophy, we find an exciting and promising area of science related to causation with an alternative, possibly best possible, solution to the challenge of the inverse problem. That is the problem of finding the possible causes, mechanistic origins, first principles, and generative models of a piece of data from a physical phenomenon. Here we explain how generating and exploring software space following the framework of Algorithmic Information Dynamics, it is possible to find small models and learn to navigate a sci-fi-looking space that can advance the field of scientific discovery with complementary tools to offer an opportunity to advance science itself. △ Less

Submitted 15 September, 2021; originally announced September 2021.

Comments: 20 pages. Invited contribution. arXiv admin note: substantial text overlap with arXiv:1904.10258, arXiv:1401.3613

Emergence and algorithmic information dynamics of systems and observers

Authors: Felipe S. Abrahão, Hector Zenil

Abstract: Previous work has shown that perturbation analysis in software space can produce candidate computable generative models and uncover possible causal properties from the finite description of an object or system quantifying the algorithmic contribution of each of its elements relative to the whole. One of the challenges for defining emergence is that one observer's prior knowledge may cause a phenom… ▽ More Previous work has shown that perturbation analysis in software space can produce candidate computable generative models and uncover possible causal properties from the finite description of an object or system quantifying the algorithmic contribution of each of its elements relative to the whole. One of the challenges for defining emergence is that one observer's prior knowledge may cause a phenomenon to present itself to such observer as emergent while for another as reducible. By formalising the act of observing as mutual perturbations between dynamical systems, we demonstrate that emergence of algorithmic information do depend on the observer's formal knowledge, while robust to other subjective factors, particularly: the choice of the programming language and the measurement method; errors or distortions during the information acquisition; and the informational cost of processing. This is called observer-dependent emergence (ODE). In addition, we demonstrate that the unbounded and fast increase of emergent algorithmic information implies asymptotically observer-independent emergence (AOIE). Unlike ODE, AOIE is a type of emergence for which emergent phenomena will remain considered to be emergent for every formal theory that any observer might devise. We demonstrate the existence of an evolutionary model that displays the diachronic variant of AOIE and a network model that displays the holistic variant of AOIE. Our results show that, restricted to the context of finite discrete deterministic dynamical systems, computable systems, and irreducible information content measures, AOIE is the strongest form of emergence that formal theories can attain. △ Less

Submitted 30 September, 2021; v1 submitted 31 May, 2021; originally announced May 2021.

An Algorithmic Information Distortion in Multidimensional Networks

Authors: Felipe S. Abrahão, Klaus Wehmuth, Hector Zenil, Artur Ziviani

Abstract: Network complexity, network information content analysis, and lossless compressibility of graph representations have been played an important role in network analysis and network modeling. As multidimensional networks, such as time-varying, multilayer, or dynamic multilayer networks, gain more relevancy in network science, it becomes crucial to investigate in which situations universal algorithmic… ▽ More Network complexity, network information content analysis, and lossless compressibility of graph representations have been played an important role in network analysis and network modeling. As multidimensional networks, such as time-varying, multilayer, or dynamic multilayer networks, gain more relevancy in network science, it becomes crucial to investigate in which situations universal algorithmic methods based on algorithmic information theory applied to graphs cannot be straightforwardly imported into the multidimensional case. In this direction, as a worst-case scenario of lossless compressibility distortion that increases linearly with the number of distinct dimensions, this article presents a counter-intuitive phenomenon that occurs when dealing with networks within non-uniform and sufficiently large multidimensional spaces. In particular, we demonstrate that the algorithmic information necessary to encode multidimensional networks that are isomorphic to logarithmically compressible monoplex networks may display exponentially larger distortions in the general case. △ Less

Submitted 5 October, 2020; v1 submitted 12 September, 2020; originally announced September 2020.

A Review of Methods for Estimating Algorithmic Complexity: Options, Challenges, and New Directions

Authors: Hector Zenil

Abstract: Some established and also novel techniques in the field of applications of algorithmic (Kolmogorov) complexity currently co-exist for the first time and are here reviewed, ranging from dominant ones such as statistical lossless compression to newer approaches that advance, complement and also pose new challenges and may exhibit their own limitations. Evidence suggesting that these different method… ▽ More Some established and also novel techniques in the field of applications of algorithmic (Kolmogorov) complexity currently co-exist for the first time and are here reviewed, ranging from dominant ones such as statistical lossless compression to newer approaches that advance, complement and also pose new challenges and may exhibit their own limitations. Evidence suggesting that these different methods complement each other for different regimes is presented and despite their many challenges, some of these methods can be better motivated by and better grounded in the principles of algorithmic information theory. It will be explained how different approaches to algorithmic complexity can explore the relaxation of different necessary and sufficient conditions in their pursuit of numerical applicability, with some of these approaches entailing greater risks than others in exchange for greater relevance. We conclude with a discussion of possible directions that may or should be taken into consideration to advance the field and encourage methodological innovation, but more importantly, to contribute to scientific discovery. This paper also serves as a rebuttal of claims made in a previously published minireview by another author, and offers an alternative account. △ Less

Submitted 27 May, 2020; v1 submitted 24 March, 2020; originally announced March 2020.

Comments: 39 pages; a rebuttal and answer to Paul Vitanyi's review. As accepted by the journal Entropy

Evolving Neural Networks through a Reverse Encoding Tree

Authors: Haoling Zhang, Chao-Han Huck Yang, Hector Zenil, Narsis A. Kiani, Yue Shen, Jesper N. Tegner

Abstract: NeuroEvolution is one of the most competitive evolutionary learning frameworks for designing novel neural networks for use in specific tasks, such as logic circuit design and digital gaming. However, the application of benchmark methods such as the NeuroEvolution of Augmenting Topologies (NEAT) remains a challenge, in terms of their computational cost and search time inefficiency. This paper advan… ▽ More NeuroEvolution is one of the most competitive evolutionary learning frameworks for designing novel neural networks for use in specific tasks, such as logic circuit design and digital gaming. However, the application of benchmark methods such as the NeuroEvolution of Augmenting Topologies (NEAT) remains a challenge, in terms of their computational cost and search time inefficiency. This paper advances a method which incorporates a type of topological edge coding, named Reverse Encoding Tree (RET), for evolving scalable neural networks efficiently. Using RET, two types of approaches -- NEAT with Binary search encoding (Bi-NEAT) and NEAT with Golden-Section search encoding (GS-NEAT) -- have been designed to solve problems in benchmark continuous learning environments such as logic gates, Cartpole, and Lunar Lander, and tested against classical NEAT and FS-NEAT as baselines. Additionally, we conduct a robustness test to evaluate the resilience of the proposed NEAT algorithms. The results show that the two proposed strategies deliver improved performance, characterized by (1) a higher accumulated reward within a finite number of time steps; (2) using fewer episodes to solve problems in targeted environments, and (3) maintaining adaptive robustness under noisy perturbations, which outperform the baselines in all tested cases. Our analysis also demonstrates that RET expends potential future research directions in dynamic environments. Code is available from https://github.com/HaolingZHANG/ReverseEncodingTree. △ Less

Submitted 31 March, 2020; v1 submitted 2 February, 2020; originally announced February 2020.

Comments: Accepted to IEEE Congress on Evolutionary Computation (IEEE CEC) 2020. Lecture Presentation

Journal ref: 2020 IEEE Congress on Evolutionary Computation (CEC)

Algorithmic Probability-guided Supervised Machine Learning on Non-differentiable Spaces

Authors: Santiago Hernández-Orozco, Hector Zenil, Jürgen Riedel, Adam Uccello, Narsis A. Kiani, Jesper Tegnér

Abstract: We show how complexity theory can be introduced in machine learning to help bring together apparently disparate areas of current research. We show that this new approach requires less training data and is more generalizable as it shows greater resilience to random attacks. We investigate the shape of the discrete algorithmic space when performing regression or classification using a loss function… ▽ More We show how complexity theory can be introduced in machine learning to help bring together apparently disparate areas of current research. We show that this new approach requires less training data and is more generalizable as it shows greater resilience to random attacks. We investigate the shape of the discrete algorithmic space when performing regression or classification using a loss function parametrized by algorithmic complexity, demonstrating that the property of differentiation is not necessary to achieve results similar to those obtained using differentiable programming approaches such as deep learning. In doing so we use examples which enable the two approaches to be compared (small, given the computational power required for estimations of algorithmic complexity). We find and report that (i) machine learning can successfully be performed on a non-smooth surface using algorithmic complexity; (ii) that parameter solutions can be found using an algorithmic-probability classifier, establishing a bridge between a fundamentally discrete theory of computability and a fundamentally continuous mathematical theory of optimization methods; (iii) a formulation of an algorithmically directed search technique in non-smooth manifolds can be defined and conducted; (iv) exploitation techniques and numerical methods for algorithmic search to navigate these discrete non-differentiable spaces can be performed; in application of the (a) identification of generative rules from data observations; (b) solutions to image classification problems more resilient against pixel attacks compared to neural networks; (c) identification of equation parameters from a small data-set in the presence of noise in continuous ODE system problem, (d) classification of Boolean NK networks by (1) network topology, (2) underlying Boolean function, and (3) number of incoming edges. △ Less

Submitted 8 October, 2019; v1 submitted 7 October, 2019; originally announced October 2019.

Comments: 33 pages including appendix

Compression is Comprehension, and the Unreasonable Effectiveness of Digital Computation in the Natural World

Authors: Hector Zenil

Abstract: Chaitin's work, in its depth and breadth, encompasses many areas of scientific and philosophical interest. It helped establish the accepted mathematical concept of randomness, which in turn is the basis of tools that I have developed to justify and quantify what I think is clear evidence of the algorithmic nature of the world. To illustrate the concept I will establish novel upper bounds of algori… ▽ More Chaitin's work, in its depth and breadth, encompasses many areas of scientific and philosophical interest. It helped establish the accepted mathematical concept of randomness, which in turn is the basis of tools that I have developed to justify and quantify what I think is clear evidence of the algorithmic nature of the world. To illustrate the concept I will establish novel upper bounds of algorithmic randomness for elementary cellular automata. I will discuss how the practice of science consists in conceiving a model that starts from certain initial values, running a computable instantiation, and awaiting a result in order to determine where the system may be in a future state--in a shorter time than the time taken by the actual unfolding of the phenomenon in question. If a model does not comply with all or some of these requirements it is traditionally considered useless or even unscientific, so the more precise and faster the better. A model is thus better if it can explain more with less, which is at the core of Chaitin's "compression is comprehension". I will pursue these questions related to the random versus possibly algorithmic nature of the world in two directions, drawing heavily on the work of Chaitin. I will also discuss how the algorithmic approach is related to the success of science at producing models of the world, allowing computer simulations to better understand it and make more accurate predictions and interventions. △ Less

Submitted 9 June, 2021; v1 submitted 23 April, 2019; originally announced April 2019.

Comments: 30 pages. Invited contribution to Chaitin's festschrift based on an invited talk delivered at the Workshop on 'Patterns in the World', Department of Philosophy, University of Barcelona on December 14, 2018

On incompressible multidimensional networks

Authors: Felipe S. Abrahão, Klaus Wehmuth, Hector Zenil, Artur Ziviani

Abstract: In order to deal with multidimensional structure representations of real-world networks, as well as with their worst-case irreducible information content analysis, the demand for new graph abstractions increases. This article presents an investigation of incompressible multidimensional networks defined by generalized graph representations. In particular, we mathematically study the lossless incomp… ▽ More In order to deal with multidimensional structure representations of real-world networks, as well as with their worst-case irreducible information content analysis, the demand for new graph abstractions increases. This article presents an investigation of incompressible multidimensional networks defined by generalized graph representations. In particular, we mathematically study the lossless incompressibility of snapshot-dynamic networks and multiplex networks in comparison to the lossless incompressibility of more general forms of dynamic networks and multilayer networks, from which snapshot-dynamic networks or multiplex networks are particular cases. We show that incompressible snapshot-dynamic (or multiplex) networks carry an amount of algorithmic information that is linearly dominated by the size of the set of time instants (or layers). This contrasts with the algorithmic information carried by incompressible general dynamic (or multilayer) networks that is of the quadratic order of the size of the set of time instants (or layers). Furthermore, we prove that incompressible general multidimensional networks have edges linking vertices at non-sequential time instants (layers or, in general, elements of a node dimension). Thus, representational incompressibility implies a necessary underlying constraint in the multidimensional network topology. △ Less

Submitted 29 April, 2020; v1 submitted 3 December, 2018; originally announced December 2018.

MSC Class: 05C82; 68Q30; 03D32; 05C80; 68P30; 94A29; 68R10; 05C60; 05C75; 94A16; 68T09;

Controllability, Multiplexing, and Transfer Learning in Networks using Evolutionary Learning

Authors: Rise Ooi, Chao-Han Huck Yang, Pin-Yu Chen, Vìctor Eguìluz, Narsis Kiani, Hector Zenil, David Gomez-Cabrero, Jesper Tegnèr

Abstract: Networks are fundamental building blocks for representing data, and computations. Remarkable progress in learning in structurally defined (shallow or deep) networks has recently been achieved. Here we introduce evolutionary exploratory search and learning method of topologically flexible networks under the constraint of producing elementary computational steady-state input-output operations. Our… ▽ More Networks are fundamental building blocks for representing data, and computations. Remarkable progress in learning in structurally defined (shallow or deep) networks has recently been achieved. Here we introduce evolutionary exploratory search and learning method of topologically flexible networks under the constraint of producing elementary computational steady-state input-output operations. Our results include; (1) the identification of networks, over four orders of magnitude, implementing computation of steady-state input-output functions, such as a band-pass filter, a threshold function, and an inverse band-pass function. Next, (2) the learned networks are technically controllable as only a small number of driver nodes are required to move the system to a new state. Furthermore, we find that the fraction of required driver nodes is constant during evolutionary learning, suggesting a stable system design. (3), our framework allows multiplexing of different computations using the same network. For example, using a binary representation of the inputs, the network can readily compute three different input-output functions. Finally, (4) the proposed evolutionary learning demonstrates transfer learning. If the system learns one function A, then learning B requires on average less number of steps as compared to learning B from tabula rasa. We conclude that the constrained evolutionary learning produces large robust controllable circuits, capable of multiplexing and transfer learning. Our study suggests that network-based computations of steady-state functions, representing either cellular modules of cell-to-cell communication networks or internal molecular circuits communicating within a cell, could be a powerful model for biologically inspired computing. This complements conceptualizations such as attractor based models, or reservoir computing. △ Less

Submitted 3 November, 2019; v1 submitted 13 November, 2018; originally announced November 2018.

Comments: A revised version. (word source code to pdf; owing to the algo package conflicts)

Algorithmic information distortions and incompressibility in uniform multidimensional networks

Authors: Felipe S. Abrahão, Klaus Wehmuth, Hector Zenil, Artur Ziviani

Abstract: This article presents a theoretical investigation of generalized encoded forms of networks in a uniform multidimensional space. First, we study encoded networks with (finite) arbitrary node dimensions (or aspects), such as time instants or layers. In particular, we study these networks that are formalized in the form of multiaspect graphs. In the context of node-aligned non-uniform (or node-unalig… ▽ More This article presents a theoretical investigation of generalized encoded forms of networks in a uniform multidimensional space. First, we study encoded networks with (finite) arbitrary node dimensions (or aspects), such as time instants or layers. In particular, we study these networks that are formalized in the form of multiaspect graphs. In the context of node-aligned non-uniform (or node-unaligned non-uniform and uniform) multidimensional spaces, previous results has shown that, unlike classical graphs, the algorithmic information of a multidimensional network is not in general dominated by the algorithmic information of the binary sequence that determines the presence or absence of edges. In the present work, first we demonstrate the existence of such algorithmic information distortions for node-aligned uniform multidimensional networks. Secondly, we show that there are particular cases of infinite nesting families of finite uniform multidimensional networks such that each member of these families is incompressible. From these results, we also recover the network topological properties and equivalences in irreducible information content of multidimensional networks in comparison to their isomorphic classical graph counterpart in the previous literature. These results together establish a universal algorithmic approach and set limitations and conditions for irreducible information content analysis in comparing arbitrary networks with a large number of dimensions, such as multilayer networks. △ Less

Submitted 21 April, 2023; v1 submitted 27 October, 2018; originally announced October 2018.

Report number: Article based on research report 08/2018 at the National Laboratory for Scientific Computing (LNCC), Brazil MSC Class: 05C82; 68Q30; 68P30; 94A29; 68R10; 05C75; 94A16; 03D32; 05C80; 05C60; 11U05; 68T09; 68Q01; 94A15; 05C30; 05C78; 62R07

On complexity of post-processing in analyzing GATE-driven X-ray spectrum

Authors: Neda Gholami, Mohammad Mahdi Dehshibi, Mahmood Fazlali, Antonio Rueda-Toicen, Hector Zenil, Andrew Adamatzky

Abstract: Computed Tomography (CT) imaging is one of the most influential diagnostic methods. In clinical reconstruction, an effective energy is used instead of total X-ray spectrum. This approximation causes an accuracy decline. To increase the contrast, single source or dual source dual energy CT can be used to reach optimal values of tissue differentiation. However, these infrastructures are still at the… ▽ More Computed Tomography (CT) imaging is one of the most influential diagnostic methods. In clinical reconstruction, an effective energy is used instead of total X-ray spectrum. This approximation causes an accuracy decline. To increase the contrast, single source or dual source dual energy CT can be used to reach optimal values of tissue differentiation. However, these infrastructures are still at the laboratory level, and their safeties for patients are still yet to mature. Therefore, computer modelling of DECT could be used. We propose a novel post-processing approach for converting a total X-ray spectrum into irregular intervals of quantized energy. We simulate a phantom in GATE/GEANT4 and irradiate it based on CT configuration. Inverse Radon transform is applied to the acquired sinogram to construct the Pixel-based Attenuation Matrix (PAM). To construct images represented by each interval, water attenuation coefficient of the interval is extracted from NIST and used in the Hounsfield unit (HU) scale in conjunction with PAM. The CT image is modified by using of an associated normalized photon flux and calculated HU corresponding to the interval. We demonstrate the proposed method efficiency via complexity analysis, using absolute and relative complexities, entropy measures, Kolmogorov complexity, morphological richness, and quantitative segmentation criteria associated with standard fuzzy C-means. The irregularity of the modified CT images decreases over the simulated ones. △ Less

Submitted 24 July, 2018; originally announced July 2018.

The Thermodynamics of Network Coding, and an Algorithmic Refinement of the Principle of Maximum Entropy

Authors: Hector Zenil, Narsis A. Kiani, Jesper Tegnér

Abstract: The principle of maximum entropy (Maxent) is often used to obtain prior probability distributions as a method to obtain a Gibbs measure under some restriction giving the probability that a system will be in a certain state compared to the rest of the elements in the distribution. Because classical entropy-based Maxent collapses cases confounding all distinct degrees of randomness and pseudo-random… ▽ More The principle of maximum entropy (Maxent) is often used to obtain prior probability distributions as a method to obtain a Gibbs measure under some restriction giving the probability that a system will be in a certain state compared to the rest of the elements in the distribution. Because classical entropy-based Maxent collapses cases confounding all distinct degrees of randomness and pseudo-randomness, here we take into consideration the generative mechanism of the systems considered in the ensemble to separate objects that may comply with the principle under some restriction and whose entropy is maximal but may be generated recursively from those that are actually algorithmically random offering a refinement to classical Maxent. We take advantage of a causal algorithmic calculus to derive a thermodynamic-like result based on how difficult it is to reprogram a computer code. Using the distinction between computable and algorithmic randomness we quantify the cost in information loss associated with reprogramming. To illustrate this we apply the algorithmic refinement to Maxent on graphs and introduce a Maximal Algorithmic Randomness Preferential Attachment (MARPA) Algorithm, a generalisation over previous approaches. We discuss practical implications of evaluation of network randomness. Our analysis provides insight in that the reprogrammability asymmetry appears to originate from a non-monotonic relationship to algorithmic probability. Our analysis motivates further analysis of the origin and consequences of the aforementioned asymmetries, reprogrammability, and computation. △ Less

Submitted 6 June, 2019; v1 submitted 18 May, 2018; originally announced May 2018.

Comments: 30 pages

Journal ref: Entropy, 21(6), 560, 2019

Symmetry and Algorithmic Complexity of Polyominoes and Polyhedral Graphs

Authors: Hector Zenil, Narsis A. Kiani, Jesper Tegnér

Abstract: We introduce a definition of algorithmic symmetry able to capture essential aspects of geometric symmetry. We review, study and apply a method for approximating the algorithmic complexity (also known as Kolmogorov-Chaitin complexity) of graphs and networks based on the concept of Algorithmic Probability (AP). AP is a concept (and method) capable of recursively enumeration all properties of computa… ▽ More We introduce a definition of algorithmic symmetry able to capture essential aspects of geometric symmetry. We review, study and apply a method for approximating the algorithmic complexity (also known as Kolmogorov-Chaitin complexity) of graphs and networks based on the concept of Algorithmic Probability (AP). AP is a concept (and method) capable of recursively enumeration all properties of computable (causal) nature beyond statistical regularities. We explore the connections of algorithmic complexity---both theoretical and numerical---with geometric properties mainly symmetry and topology from an (algorithmic) information-theoretic perspective. We show that approximations to algorithmic complexity by lossless compression and an Algorithmic Probability-based method can characterize properties of polyominoes, polytopes, regular and quasi-regular polyhedra as well as polyhedral networks, thereby demonstrating its profiling capabilities. △ Less

Submitted 24 February, 2018; originally announced March 2018.

Comments: 18 pages, 4 figures + Appendix (1 figure)

Algorithmic Causal Deconvolution of Intertwined Programs and Networks by Generative Mechanism

Authors: Hector Zenil, Narsis A. Kiani, Allan A. Zea, Jesper Tegnér

Abstract: Complex data usually results from the interaction of objects produced by different generating mechanisms. Here we introduce a universal, unsupervised and parameter-free model-oriented approach, based upon the seminal concept of algorithmic probability, that decomposes an observation into its most likely algorithmic generative sources. Our approach uses a causal calculus to infer model representati… ▽ More Complex data usually results from the interaction of objects produced by different generating mechanisms. Here we introduce a universal, unsupervised and parameter-free model-oriented approach, based upon the seminal concept of algorithmic probability, that decomposes an observation into its most likely algorithmic generative sources. Our approach uses a causal calculus to infer model representations. We demonstrate its ability to deconvolve interacting mechanisms regardless of whether the resultant objects are strings, space-time evolution diagrams, images or networks. While this is mostly a conceptual contribution and a novel framework, we provide numerical evidence evaluating the ability of our methods to separate data from observations produced by discrete dynamical systems such as cellular automata and complex networks. We think that these separating techniques can contribute to tackling the challenge of causation, thus complementing other statistically oriented approaches. △ Less

Submitted 12 September, 2018; v1 submitted 18 February, 2018; originally announced February 2018.

Comments: 29 pages + 7 Sup Inf. 9 figures in total

Algorithmic Information Dynamics of Persistent Patterns and Colliding Particles in the Game of Life

Authors: Hector Zenil, Narsis A. Kiani, Jesper Tegnér

Abstract: Without loss of generalisation to other systems, including possibly non-deterministic ones, we demonstrate the application of methods drawn from algorithmic information dynamics to the characterisation and classification of emergent and persistent patterns, motifs and colliding particles in Conway's Game of Life (GoL), a cellular automaton serving as a case study illustrating the way in which such… ▽ More Without loss of generalisation to other systems, including possibly non-deterministic ones, we demonstrate the application of methods drawn from algorithmic information dynamics to the characterisation and classification of emergent and persistent patterns, motifs and colliding particles in Conway's Game of Life (GoL), a cellular automaton serving as a case study illustrating the way in which such ideas can be applied to a typical discrete dynamical system. We explore the issue of local observations of closed systems whose orbits may appear open because of inaccessibility to the global rules governing the overall system. We also investigate aspects of symmetry related to complexity in the distribution of patterns that occur with high frequency in GoL (which we thus call motifs) and analyse the distribution of these motifs with a view to tracking the changes in their algorithmic probability over time. We demonstrate how the tools introduced are an alternative to other computable measures that are unable to capture changes in emergent structures in evolving complex systems that are often too small or too subtle to be properly characterised by methods such as lossless compression and Shannon entropy. △ Less

Submitted 5 April, 2018; v1 submitted 17 February, 2018; originally announced February 2018.

Comments: 18 pages + 1 sup page, 8 figures in total. Online complexity calculator: http://complexitycalculator.com/

Algorithmic Complexity and Reprogrammability of Chemical Structure Networks

Authors: Hector Zenil, Narsis A. Kiani, Ming-Mei Shang, Jesper Tegnér

Abstract: Here we address the challenge of profiling causal properties and tracking the transformation of chemical compounds from an algorithmic perspective. We explore the potential of applying a computational interventional calculus based on the principles of algorithmic probability to chemical structure networks. We profile the sensitivity of the elements and covalent bonds in a chemical structure networ… ▽ More Here we address the challenge of profiling causal properties and tracking the transformation of chemical compounds from an algorithmic perspective. We explore the potential of applying a computational interventional calculus based on the principles of algorithmic probability to chemical structure networks. We profile the sensitivity of the elements and covalent bonds in a chemical structure network algorithmically, asking whether reprogrammability affords information about thermodynamic and chemical processes involved in the transformation of different compound classes. We arrive at numerical results suggesting a correspondence between some physical, structural and functional properties. Our methods are capable of separating chemical classes that reflect functional and natural differences without considering any information about atomic and molecular properties. We conclude that these methods, with their links to chemoinformatics via algorithmic, probability hold promise for future research. △ Less

Submitted 18 March, 2018; v1 submitted 16 February, 2018; originally announced February 2018.

Comments: 19 pages + Appendix

Minimal Algorithmic Information Loss Methods for Dimension Reduction, Feature Selection and Network Sparsification

Authors: Hector Zenil, Narsis A. Kiani, Alyssa Adams, Felipe S. Abrahão, Antonio Rueda-Toicen, Allan A. Zea, Jesper Tegnér

Abstract: We introduce a family of unsupervised, domain-free, and asymptotically optimal model-independent algorithms based on the principles of algorithmic probability and information theory designed to minimize the loss of algorithmic information, and thereby avoiding certain deceiving phenomena and distortions known to occur in statistics and entropy-based approaches. Our methods include a lossless-compr… ▽ More We introduce a family of unsupervised, domain-free, and asymptotically optimal model-independent algorithms based on the principles of algorithmic probability and information theory designed to minimize the loss of algorithmic information, and thereby avoiding certain deceiving phenomena and distortions known to occur in statistics and entropy-based approaches. Our methods include a lossless-compression-based lossy compression algorithm that can select and coarse-grain data in an algorithmic-complexity fashion (without the use of popular compression algorithms) by collapsing regions that may procedurally be regenerated from a computable candidate model. We show that the method can perform dimension reduction, denoising, feature selection, and network sparsification, while preserving the properties of the objects. As validation case, we demonstrate the methods on image segmentation against popular methods like PCA and random selection, and also demonstrate that the method preserves the graph-theoretic indices measured on a well-known set of synthetic and real-world networks of very different nature, ranging from degree distribution and clustering coefficient to edge betweenness and degree and eigenvector centralities, achieving equal or significantly better results than other data reduction and the leading network sparsification methods (Spectral, Transitive). △ Less

Submitted 8 April, 2023; v1 submitted 16 February, 2018; originally announced February 2018.

Comments: Online implementation at http://complexitycalculator.com/MILS/

Slime mould: the fundamental mechanisms of cognition

Authors: Jordi Vallverdu, Oscar Castro, Richard Mayne, Max Talanov, Michael Levin, Frantisek Baluska, Yukio Gunji, Audrey Dussutour, Hector Zenil, Andrew Adamatzky

Abstract: The slime mould Physarum polycephalum has been used in develo** unconventional computing devices for in which the slime mould played a role of a sensing, actuating, and computing device. These devices treated the slime mould rather as an active living substrate yet the slime mould is a self-consistent living creature which evolved for millions of years and occupied most part of the world, but in… ▽ More The slime mould Physarum polycephalum has been used in develo** unconventional computing devices for in which the slime mould played a role of a sensing, actuating, and computing device. These devices treated the slime mould rather as an active living substrate yet the slime mould is a self-consistent living creature which evolved for millions of years and occupied most part of the world, but in any case, that living entity did not own true cognition, just automated biochemical mechanisms. To "rehabilitate" the slime mould from the rank of a purely living electronics element to a "creature of thoughts" we are analyzing the cognitive potential of P. polycephalum. We base our theory of minimal cognition of the slime mould on a bottom-up approach, from the biological and biophysical nature of the slime mould and its regulatory systems using frameworks suh as Lyon's biogenic cognition, Muller, di Primio-Lengelerś modifiable pathways, Bateson's "patterns that connect" framework, Maturana's autopoetic network, or proto-consciousness and Morgan's Canon. △ Less

Submitted 1 December, 2017; originally announced December 2017.

Coding-theorem Like Behaviour and Emergence of the Universal Distribution from Resource-bounded Algorithmic Probability

Authors: Hector Zenil, Liliana Badillo, Santiago Hernández-Orozco, Francisco Hernández-Quiroz

Abstract: Previously referred to as `miraculous' in the scientific literature because of its powerful properties and its wide application as optimal solution to the problem of induction/inference, (approximations to) Algorithmic Probability (AP) and the associated Universal Distribution are (or should be) of the greatest importance in science. Here we investigate the emergence, the rates of emergence and co… ▽ More Previously referred to as `miraculous' in the scientific literature because of its powerful properties and its wide application as optimal solution to the problem of induction/inference, (approximations to) Algorithmic Probability (AP) and the associated Universal Distribution are (or should be) of the greatest importance in science. Here we investigate the emergence, the rates of emergence and convergence, and the Coding-theorem like behaviour of AP in Turing-subuniversal models of computation. We investigate empirical distributions of computing models in the Chomsky hierarchy. We introduce measures of algorithmic probability and algorithmic complexity based upon resource-bounded computation, in contrast to previously thoroughly investigated distributions produced from the output distribution of Turing machines. This approach allows for numerical approximations to algorithmic (Kolmogorov-Chaitin) complexity-based estimations at each of the levels of a computational hierarchy. We demonstrate that all these estimations are correlated in rank and that they converge both in rank and values as a function of computational power, despite fundamental differences between computational models. In the context of natural processes that operate below the Turing universal level because of finite resources and physical degradation, the investigation of natural biases stemming from algorithmic rules may shed light on the distribution of outcomes. We show that up to 60\% of the simplicity/complexity bias in distributions produced even by the weakest of the computational models can be accounted for by Algorithmic Probability in its approximation to the Universal Distribution. △ Less

Submitted 13 April, 2018; v1 submitted 5 November, 2017; originally announced November 2017.

Comments: 27 pages main text, 39 pages including supplement. Online complexity calculator: http://complexitycalculator.com/

Journal ref: International Journal of Parallel, Emergent and Distributed Systems, DOI: 10.1080/17445760.2018.1448932

An Algorithmic Information Calculus for Causal Discovery and Reprogramming Systems

Authors: Hector Zenil, Narsis A. Kiani, Francesco Marabita, Yue Deng, Szabolcs Elias, Angelika Schmidt, Gordon Ball, Jesper Tegnér

Abstract: We demonstrate that the algorithmic information content of a system is deeply connected to its potential dynamics, thus affording an avenue for moving systems in the information-theoretic space and controlling them in the phase space. To this end we performed experiments and validated the results on (1) a very large set of small graphs, (2) a number of larger networks with different topologies, an… ▽ More We demonstrate that the algorithmic information content of a system is deeply connected to its potential dynamics, thus affording an avenue for moving systems in the information-theoretic space and controlling them in the phase space. To this end we performed experiments and validated the results on (1) a very large set of small graphs, (2) a number of larger networks with different topologies, and (3) biological networks from a widely studied and validated genetic network (e.coli) as well as on a significant number of differentiating (Th17) and differentiated human cells from high quality databases (Harvard's CellNet) with results conforming to experimentally validated biological data. Based on these results we introduce a conceptual framework, a model-based interventional calculus and a reprogrammability measure with which to steer, manipulate, and reconstruct the dynamics of non- linear dynamical systems from partial and disordered observations. The method consists in finding and applying a series of controlled interventions to a dynamical system to estimate how its algorithmic information content is affected when every one of its elements are perturbed. The approach represents an alternative to numerical simulation and statistical approaches for inferring causal mechanistic/generative models and finding first principles. We demonstrate the framework's capabilities by reconstructing the phase space of some discrete dynamical systems (cellular automata) as case study and reconstructing their generating rules. We thus advance tools for reprogramming artificial and living systems without full knowledge or access to the system's actual kinetic equations or probability distributions yielding a suite of universal and parameter-free algorithms of wide applicability ranging from causation, dimension reduction, feature selection and model generation. △ Less

Submitted 5 April, 2018; v1 submitted 15 September, 2017; originally announced September 2017.

Comments: 50 pages with Supplementary Information and Extended Figures. The Online Algorithmic Complexity Calculator implements the methods in this paper: http://complexitycalculator.com/ Animated video available at: https://youtu.be/ufzq2p5tVLI

Algorithmically probable mutations reproduce aspects of evolution such as convergence rate, genetic memory, and modularity

Authors: Santiago Hernández-Orozco, Narsis A. Kiani, Hector Zenil

Abstract: Natural selection explains how life has evolved over millions of years from more primitive forms. The speed at which this happens, however, has sometimes defied formal explanations when based on random (uniformly distributed) mutations. Here we investigate the application of a simplicity bias based on a natural but algorithmic distribution of mutations (no recombination) in various examples, parti… ▽ More Natural selection explains how life has evolved over millions of years from more primitive forms. The speed at which this happens, however, has sometimes defied formal explanations when based on random (uniformly distributed) mutations. Here we investigate the application of a simplicity bias based on a natural but algorithmic distribution of mutations (no recombination) in various examples, particularly binary matrices in order to compare evolutionary convergence rates. Results both on synthetic and on small biological examples indicate an accelerated rate when mutations are not statistical uniform but \textit{algorithmic uniform}. We show that algorithmic distributions can evolve modularity and genetic memory by preservation of structures when they first occur sometimes leading to an accelerated production of diversity but also population extinctions, possibly explaining naturally occurring phenomena such as diversity explosions (e.g. the Cambrian) and massive extinctions (e.g. the End Triassic) whose causes are currently a cause for debate. The natural approach introduced here appears to be a better approximation to biological evolution than models based exclusively upon random uniform mutations, and it also approaches a formal version of open-ended evolution based on previous formal results. These results validate some suggestions in the direction that computation may be an equally important driver of evolution. We also show that inducing the method on problems of optimization, such as genetic algorithms, has the potential to accelerate convergence of artificial evolutionary algorithms. △ Less

Submitted 20 June, 2018; v1 submitted 1 September, 2017; originally announced September 2017.

Comments: 13 pages, 10 figures

Training-free Measures Based on Algorithmic Probability Identify High Nucleosome Occupancy in DNA Sequences

Authors: Hector Zenil, Peter Minary

Abstract: We introduce and study a set of training-free methods of information-theoretic and algorithmic complexity nature applied to DNA sequences to identify their potential capabilities to determine nucleosomal binding sites. We test our measures on well-studied genomic sequences of different sizes drawn from different sources. The measures reveal the known in vivo versus in vitro predictive discrepancie… ▽ More We introduce and study a set of training-free methods of information-theoretic and algorithmic complexity nature applied to DNA sequences to identify their potential capabilities to determine nucleosomal binding sites. We test our measures on well-studied genomic sequences of different sizes drawn from different sources. The measures reveal the known in vivo versus in vitro predictive discrepancies and uncover their potential to pinpoint (high) nucleosome occupancy. We explore different possible signals within and beyond the nucleosome length and find that complexity indices are informative of nucleosome occupancy. We compare against the gold standard (Kaplan model) and find similar and complementary results with the main difference that our sequence complexity approach. For example, for high occupancy, complexity-based scores outperform the Kaplan model for predicting binding representing a significant advancement in predicting the highest nucleosome occupancy following a training-free approach. △ Less

Submitted 16 October, 2018; v1 submitted 5 August, 2017; originally announced August 2017.

Comments: 8 pages main text (4 figures), 12 total with Supplementary (1 figure)

Paths to Unconventional Computing: Causality in Complexity

Authors: Hector Zenil

Abstract: I describe my path to unconventionality in my exploration of theoretical and applied aspects of computation towards revealing the algorithmic and reprogrammable properties and capabilities of the world, in particular related to applications of algorithmic complexity in resha** molecular biology and tackling the challenges of causality in science. I describe my path to unconventionality in my exploration of theoretical and applied aspects of computation towards revealing the algorithmic and reprogrammable properties and capabilities of the world, in particular related to applications of algorithmic complexity in resha** molecular biology and tackling the challenges of causality in science. △ Less

Submitted 31 May, 2017; originally announced June 2017.

Comments: Extended version of an invited contribution to a special issue of the journal of Progress in Biophysics & Molecular Biology (Elsevier)

Reprogramming Matter, Life, and Purpose

Authors: Hector Zenil

Abstract: Reprogramming matter may sound far-fetched, but we have been doing it with increasing power and staggering efficiency for at least 60 years, and for centuries we have been paving the way toward the ultimate reprogrammed fate of the universe, the vessel of all programs. How will we be doing it in 60 years' time and how will it impact life and the purpose both of machines and of humans? Reprogramming matter may sound far-fetched, but we have been doing it with increasing power and staggering efficiency for at least 60 years, and for centuries we have been paving the way toward the ultimate reprogrammed fate of the universe, the vessel of all programs. How will we be doing it in 60 years' time and how will it impact life and the purpose both of machines and of humans? △ Less

Submitted 13 August, 2017; v1 submitted 2 April, 2017; originally announced April 2017.

Comments: Invited contribution to 'Computing in the year 2065', A. Adamatzky (Ed.), Springer Verlag and published in the International Journal of Unconventional Computing, 2017

A Decomposition Method for Global Evaluation of Shannon Entropy and Local Estimations of Algorithmic Complexity

Authors: Hector Zenil, Santiago Hernández-Orozco, Narsis A. Kiani, Fernando Soler-Toscano, Antonio Rueda-Toicen

Abstract: We investigate the properties of a Block Decomposition Method (BDM), which extends the power of a Coding Theorem Method (CTM) that approximates local estimations of algorithmic complexity based upon Solomonoff-Levin's theory of algorithmic probability providing a closer connection to algorithmic complexity than previous attempts based on statistical regularities e.g. as spotted by some popular los… ▽ More We investigate the properties of a Block Decomposition Method (BDM), which extends the power of a Coding Theorem Method (CTM) that approximates local estimations of algorithmic complexity based upon Solomonoff-Levin's theory of algorithmic probability providing a closer connection to algorithmic complexity than previous attempts based on statistical regularities e.g. as spotted by some popular lossless compression schemes. The strategy behind BDM is to find small computer programs that produce the components of a larger, decomposed object. The set of short computer programs can then be artfully arranged in sequence so as to produce the original object and to estimate an upper bound on the length of the shortest computer program that produces said original object. We show that the method provides efficient estimations of algorithmic complexity but that it performs like Shannon entropy when it loses accuracy. We estimate errors and study the behaviour of BDM for different boundary conditions, all of which are compared and assessed in detail. The measure may be adapted for use with more multi-dimensional objects than strings, objects such as arrays and tensors. To test the measure we demonstrate the power of CTM on low algorithmic-randomness objects that are assigned maximal entropy (e.g. $π$) but whose numerical approximations are closer to the theoretical low algorithmic-randomness expectation. We also test the measure on larger objects including dual, isomorphic and cospectral graphs for which we know that algorithmic randomness is low. We also release implementations of the methods in most major programming languages---Wolfram Language (Mathematica), Matlab, R, Perl, Python, Pascal, C++, and Haskell---and a free online algorithmic complexity calculator. △ Less

Submitted 18 June, 2018; v1 submitted 1 September, 2016; originally announced September 2016.

Comments: 39 pages, 46 with appendix. 15 figures total and 4 tables

ACM Class: H.1.1

Low Algorithmic Complexity Entropy-deceiving Graphs

Authors: Hector Zenil, Narsis Kiani, Jesper Tegnér

Abstract: In estimating the complexity of objects, in particular of graphs, it is common practice to rely on graph- and information-theoretic measures. Here, using integer sequences with properties such as Borel normality, we explain how these measures are not independent of the way in which an object, such as a graph, can be described or observed. From observations that can reconstruct the same graph and a… ▽ More In estimating the complexity of objects, in particular of graphs, it is common practice to rely on graph- and information-theoretic measures. Here, using integer sequences with properties such as Borel normality, we explain how these measures are not independent of the way in which an object, such as a graph, can be described or observed. From observations that can reconstruct the same graph and are therefore essentially translations of the same description, we will see that when applying a computable measure such as Shannon Entropy, not only is it necessary to pre-select a feature of interest where there is one, and to make an arbitrary selection where there is not, but also that more general properties, such as the causal likelihood of a graph as a measure (opposed to randomness), can be largely misrepresented by computable measures such as Entropy and Entropy rate. We introduce recursive and non-recursive (uncomputable) graphs and graph constructions based on these integer sequences, whose different lossless descriptions have disparate Entropy values, thereby enabling the study and exploration of a measure's range of applications and demonstrating the weaknesses of computable measures of complexity. △ Less

Submitted 10 May, 2017; v1 submitted 21 August, 2016; originally announced August 2016.

Comments: 28 pages

ACM Class: F.1.3

Journal ref: Phys. Rev. E 96, 012308 (2017)

Formal Definitions of Unbounded Evolution and Innovation Reveal Universal Mechanisms for Open-Ended Evolution in Dynamical Systems

Authors: Alyssa M Adams, Hector Zenil, Paul CW Davies, Sara I Walker

Abstract: Open-ended evolution (OEE) is relevant to a variety of biological, artificial and technological systems, but has been challenging to reproduce in silico. Most theoretical efforts focus on key aspects of open-ended evolution as it appears in biology. We recast the problem as a more general one in dynamical systems theory, providing simple criteria for open-ended evolution based on two hallmark feat… ▽ More Open-ended evolution (OEE) is relevant to a variety of biological, artificial and technological systems, but has been challenging to reproduce in silico. Most theoretical efforts focus on key aspects of open-ended evolution as it appears in biology. We recast the problem as a more general one in dynamical systems theory, providing simple criteria for open-ended evolution based on two hallmark features: unbounded evolution and innovation. We define unbounded evolution as patterns that are non-repeating within the expected Poincare recurrence time of an equivalent isolated system, and innovation as trajectories not observed in isolated systems. As a case study, we implement novel variants of cellular automata (CA) in which the update rules are allowed to vary with time in three alternative ways. Each is capable of generating conditions for open-ended evolution, but vary in their ability to do so. We find that state-dependent dynamics, widely regarded as a hallmark of life, statistically out-performs other candidate mechanisms, and is the only mechanism to produce open-ended evolution in a scalable manner, essential to the notion of ongoing evolution. This analysis suggests a new framework for unifying mechanisms for generating OEE with features distinctive to life and its artifacts, with broad applicability to biological and artificial systems. △ Less

Submitted 18 December, 2016; v1 submitted 6 July, 2016; originally announced July 2016.

Comments: Main document: 17 pages, Supplement: 21 pages Presented at OEE2: The Second Workshop on Open-Ended Evolution, 15th International Conference on the Synthesis and Simulation of Living Systems (ALIFE XV), Cancún, Mexico, 4-8 July 2016 (http://www.tim-taylor.com/oee2/)

Undecidability and Irreducibility Conditions for Open-Ended Evolution and Emergence

Authors: Santiago Hernández-Orozco, Francisco Hernández-Quiroz, Hector Zenil

Abstract: Is undecidability a requirement for open-ended evolution (OEE)? Using methods derived from algorithmic complexity theory, we propose robust computational definitions of open-ended evolution and the adaptability of computable dynamical systems. Within this framework, we show that decidability imposes absolute limits to the stable growth of complexity in computable dynamical systems. Conversely, sys… ▽ More Is undecidability a requirement for open-ended evolution (OEE)? Using methods derived from algorithmic complexity theory, we propose robust computational definitions of open-ended evolution and the adaptability of computable dynamical systems. Within this framework, we show that decidability imposes absolute limits to the stable growth of complexity in computable dynamical systems. Conversely, systems that exhibit (strong) open-ended evolution must be undecidable, establishing undecidability as a requirement for such systems. Complexity is assessed in terms of three measures: sophistication, coarse sophistication and busy beaver logical depth. These three complexity measures assign low complexity values to random (incompressible) objects. As time grows, the stated complexity measures allow for the existence of complex states during the evolution of a computable dynamical system. We show, however, that finding these states involves undecidable computations. We conjecture that for similar complexity measures that assign low complexity values, decidability imposes comparable limits to the stable growth of complexity, and that such behaviour is necessary for non-trivial evolutionary systems. We show that the undecidability of adapted states imposes novel and unpredictable behaviour on the individuals or populations being modelled. Such behaviour is irreducible. Finally, we offer an example of a system, first proposed by Chaitin, that exhibits strong OEE. △ Less

Submitted 27 December, 2016; v1 submitted 6 June, 2016; originally announced June 2016.

Comments: Reduced version of this article was submitted and accepted for oral presentation at ALife XV (July 4-8, 2016, Cancun, Mexico)

MSC Class: 92B20

Asymptotic Intrinsic Universality and Reprogrammability by Behavioural Emulation

Authors: Hector Zenil, Jürgen Riedel

Abstract: We advance a Bayesian concept of 'intrinsic asymptotic universality' taking to its final conclusions previous conceptual and numerical work based upon a concept of a reprogrammability test and an investigation of the complex qualitative behaviour of computer programs. Our method may quantify the trust and confidence of the computing capabilities of natural and classical systems, and quantify compu… ▽ More We advance a Bayesian concept of 'intrinsic asymptotic universality' taking to its final conclusions previous conceptual and numerical work based upon a concept of a reprogrammability test and an investigation of the complex qualitative behaviour of computer programs. Our method may quantify the trust and confidence of the computing capabilities of natural and classical systems, and quantify computers by their degree of reprogrammability. We test the method to provide evidence in favour of a conjecture concerning the computing capabilities of Busy Beaver Turing machines as candidates for Turing universality. The method has recently been used to quantify the number of 'intrinsically universal' cellular automata, with results that point towards the pervasiveness of universality due to a widespread capacity for emulation. Our method represents an unconventional approach to the classical and seminal concept of Turing universality, and it may be extended and applied in a broader context to natural computation, by (in something like the spirit of the Turing test) observing the behaviour of a system under circumstances where formal proofs of universality are difficult, if not impossible to come by. △ Less

Submitted 13 January, 2016; v1 submitted 3 January, 2016; originally announced January 2016.

Comments: 16 pages, 7 images. Invited contribution in Advances in Unconventional Computation. A. Adamatzky (ed), Springer Verlag

Interacting Behavior and Emerging Complexity

Authors: Alyssa Adams, Hector Zenil, Eduardo Hermo Reyes, Joost Joosten

Abstract: Can we quantify the change of complexity throughout evolutionary processes? We attempt to address this question through an empirical approach. In very general terms, we simulate two simple organisms on a computer that compete over limited available resources. We implement Global Rules that determine the interaction between two Elementary Cellular Automata on the same grid. Global Rules change the… ▽ More Can we quantify the change of complexity throughout evolutionary processes? We attempt to address this question through an empirical approach. In very general terms, we simulate two simple organisms on a computer that compete over limited available resources. We implement Global Rules that determine the interaction between two Elementary Cellular Automata on the same grid. Global Rules change the complexity of the state evolution output which suggests that some complexity is intrinsic to the interaction rules themselves. The largest increases in complexity occurred when the interacting elementary rules had very little complexity, suggesting that they are able to accept complexity through interaction only. We also found that some Class 3 or 4 CA rules are more fragile than others to Global Rules, while others are more robust, hence suggesting some intrinsic properties of the rules independent of the Global Rule choice. We provide statistical map**s of Elementary Cellular Automata exposed to Global Rules and different initial conditions onto different complexity classes. △ Less

Submitted 4 January, 2016; v1 submitted 23 December, 2015; originally announced December 2015.

Comments: 11 pages, 5 figures (in this version a minor typo corrected). Presented at AUTOMATA 2015 forthcoming in journal

Evaluating Network Inference Methods in Terms of Their Ability to Preserve the Topology and Complexity of Genetic Networks

Authors: Narsis A. Kiani, Hector Zenil, Jakub Olczak, Jesper Tegnér

Abstract: Network inference is a rapidly advancing field, with new methods being proposed on a regular basis. Understanding the advantages and limitations of different network inference methods is key to their effective application in different circumstances. The common structural properties shared by diverse networks naturally pose a challenge when it comes to devising accurate inference methods, but surpr… ▽ More Network inference is a rapidly advancing field, with new methods being proposed on a regular basis. Understanding the advantages and limitations of different network inference methods is key to their effective application in different circumstances. The common structural properties shared by diverse networks naturally pose a challenge when it comes to devising accurate inference methods, but surprisingly, there is a paucity of comparison and evaluation methods. Historically, every new methodology has only been tested against \textit{gold standard} (true values) purpose-designed synthetic and real-world (validated) biological networks. In this paper we aim to assess the impact of taking into consideration aspects of topological and information content in the evaluation of the final accuracy of an inference procedure. Specifically, we will compare the best inference methods, in both graph-theoretic and information-theoretic terms, for preserving topological properties and the original information content of synthetic and biological networks. New methods for performance comparison are introduced by borrowing ideas from gene set enrichment analysis and by applying concepts from algorithmic complexity. Experimental results show that no individual algorithm outperforms all others in all cases, and that the challenging and non-trivial nature of network inference is evident in the struggle of some of the algorithms to turn in a performance that is superior to random guesswork. Therefore special care should be taken to suit the method to the purpose at hand. Finally, we show that evaluations from data generated using different underlying topologies have different signatures that can be used to better choose a network reconstruction method. △ Less

Submitted 14 September, 2016; v1 submitted 3 December, 2015; originally announced December 2015.

Comments: main part: 18 pages. 21 pages with Sup Inf. Forthcoming in the journal of Seminars in Cell and Developmental Biology

Cross-boundary Behavioural Reprogrammability Reveals Evidence of Pervasive Universality

Authors: Jürgen Riedel, Hector Zenil

Abstract: We exhaustively explore the reprogrammability capabilities and the intrinsic universality of the Cartesian product $P \times C$ of the space $P$ of all possible computer programs of increasing size and the space $C$ of all possible compilers of increasing length such that $p \in P$ emulates $p^\prime \in P$ with $T|p^\prime|=|p|$ under a coarse-graining transformation $T$. Our approach yields a no… ▽ More We exhaustively explore the reprogrammability capabilities and the intrinsic universality of the Cartesian product $P \times C$ of the space $P$ of all possible computer programs of increasing size and the space $C$ of all possible compilers of increasing length such that $p \in P$ emulates $p^\prime \in P$ with $T|p^\prime|=|p|$ under a coarse-graining transformation $T$. Our approach yields a novel perspective on the complexity, controllability, causality and (re)programmability discrete dynamical systems. We find evidence that the density of (qualitatively different) computer programs that can be reprogrammed grows asymptotically as a function of program and compiler size. To illustrate these findings we show a series of behavioural boundary crossing results, including emulations (for all initial conditions) of Wolfram class 2 Elementary Cellular Automata (ECA) by Class 1 ECA, emulations of Classes 1, 2 and 3 ECA by Class 2 and 3 ECA, and of Classes 1, 2 and 3 by Class 3 ECA, along with results of even greater emulability for general CA (neighbourhood $r=3/2$), including Class 1 CA emulating Classes 2 and 3, and Classes 3 and 4 emulating all other classes (1, 2, 3 and 4). The emulations occur with only a linear overhead and can be considered computationally efficient. We also found that there is no hacking strategy to compress the search space based on compiler profiling in terms of e.g. similarity or complexity, suggesting that no strategy other than exhaustive search is viable. We also introduce emulation networks, derive a topologically-based measure of complexity based upon out- and in-degree connectivity, and establish bridges to fundamental ideas of complexity, universality, causality and dynamical systems. △ Less

Submitted 1 February, 2018; v1 submitted 6 October, 2015; originally announced October 2015.

Comments: 22 pages, 51 pages including Supplemental, 10 figures, code available online at URL in the paper

Approximations of Algorithmic and Structural Complexity Validate Cognitive-behavioural Experimental Results

Authors: Hector Zenil, James A. R. Marshall, Jesper Tegnér

Abstract: Being able to objectively characterise the intrinsic complexity of behavioural patterns resulting from human or animal decisions is fundamental for deconvolving cognition and designing autonomous artificial intelligence systems. Yet complexity is difficult in practice, particularly when strings are short. By numerically approximating algorithmic (Kolmogorov) complexity (K), we establish an objecti… ▽ More Being able to objectively characterise the intrinsic complexity of behavioural patterns resulting from human or animal decisions is fundamental for deconvolving cognition and designing autonomous artificial intelligence systems. Yet complexity is difficult in practice, particularly when strings are short. By numerically approximating algorithmic (Kolmogorov) complexity (K), we establish an objective tool to characterise behavioural complexity. Next, we approximate structural (Bennett's Logical Depth) complexity (LD) to assess the amount of computation required for generating a behavioural string. We apply our toolbox to three landmark studies of animal behaviour of increasing sophistication and degree of environmental influence, including studies of foraging communication by ants, flight patterns of fruit flies, and tactical deception and competition (e.g., predator-prey) strategies. We find that ants harness the environmental condition in their internal decision process, modulating their behavioural complexity accordingly. Our analysis of flight (fruit flies) invalidated the common hypothesis that animals navigating in an environment devoid of stimuli adopt a random strategy. Fruit flies exposed to a featureless environment deviated the most from Levy flight, suggesting an algorithmic bias in their attempt to devise a useful (navigation) strategy. Similarly, a logical depth analysis of rats revealed that the structural complexity of the rat always ends up matching the structural complexity of the competitor, with the rats' behaviour simulating algorithmic randomness. Finally, we discuss how experiments on how humans perceive randomness suggest the existence of an algorithmic bias in our reasoning and decision processes, in line with our analysis of the animal experiments. △ Less

Submitted 20 December, 2022; v1 submitted 21 September, 2015; originally announced September 2015.

Comments: 42 pages, 7 figures and 3 tables

Natural scene statistics mediate the perception of image complexity

Authors: Nicolas Gauvrit, Fernando Soler-Toscano, Hector Zenil

Abstract: Humans are sensitive to complexity and regularity in patterns. The subjective perception of pattern complexity is correlated to algorithmic (Kolmogorov-Chaitin) complexity as defined in computer science, but also to the frequency of naturally occurring patterns. However, the possible mediational role of natural frequencies in the perception of algorithmic complexity remains unclear. Here we reanal… ▽ More Humans are sensitive to complexity and regularity in patterns. The subjective perception of pattern complexity is correlated to algorithmic (Kolmogorov-Chaitin) complexity as defined in computer science, but also to the frequency of naturally occurring patterns. However, the possible mediational role of natural frequencies in the perception of algorithmic complexity remains unclear. Here we reanalyze Hsu et al. (2010) through a mediational analysis, and complement their results in a new experiment. We conclude that human perception of complexity seems partly shaped by natural scenes statistics, thereby establishing a link between the perception of complexity and the effect of natural scene statistics. △ Less

Submitted 14 September, 2015; originally announced September 2015.

Journal ref: Visual Cognition 22 (8), 2014, pages 1084-1091

Causality, Information and Biological Computation: An algorithmic software approach to life, disease and the immune system

Authors: Hector Zenil, Angelika Schmidt, Jesper Tegnér

Abstract: Biology has taken strong steps towards becoming a computer science aiming at reprogramming nature after the realisation that nature herself has reprogrammed organisms by harnessing the power of natural selection and the digital prescriptive nature of replicating DNA. Here we further unpack ideas related to computability, algorithmic information theory and software engineering, in the context of th… ▽ More Biology has taken strong steps towards becoming a computer science aiming at reprogramming nature after the realisation that nature herself has reprogrammed organisms by harnessing the power of natural selection and the digital prescriptive nature of replicating DNA. Here we further unpack ideas related to computability, algorithmic information theory and software engineering, in the context of the extent to which biology can be (re)programmed, and with how we may go about doing so in a more systematic way with all the tools and concepts offered by theoretical computer science in a translation exercise from computing to molecular biology and back. These concepts provide a means to a hierarchical organization thereby blurring previously clear-cut lines between concepts like matter and life, or between tumour types that are otherwise taken as different and may not have however a different cause. This does not diminish the properties of life or make its components and functions less interesting. On the contrary, this approach makes for a more encompassing and integrated view of nature, one that subsumes observer and observed within the same system, and can generate new perspectives and tools with which to view complex diseases like cancer, approaching them afresh from a software-engineering viewpoint that casts evolution in the role of programmer, cells as computing machines, DNA and genes as instructions and computer programs, viruses as hacking devices, the immune system as a software debugging tool, and diseases as an information-theoretic battlefield where all these forces deploy. We show how information theory and algorithmic programming may explain fundamental mechanisms of life and death. △ Less

Submitted 19 January, 2016; v1 submitted 24 August, 2015; originally announced August 2015.

Comments: 30 pages, 8 figures. Invited chapter contribution to Information and Causality: From Matter to Life. Sara I. Walker, Paul C.W. Davies and George Ellis (eds.), Cambridge University Press

Rare Speed-up in Automatic Theorem Proving Reveals Tradeoff Between Computational Time and Information Value

Authors: Santiago Hernández-Orozco, Francisco Hernández-Quiroz, Hector Zenil, Wilfried Sieg

Abstract: We show that strategies implemented in automatic theorem proving involve an interesting tradeoff between execution speed, proving speedup/computational time and usefulness of information. We advance formal definitions for these concepts by way of a notion of normality related to an expected (optimal) theoretical speedup when adding useful information (other theorems as axioms), as compared with ac… ▽ More We show that strategies implemented in automatic theorem proving involve an interesting tradeoff between execution speed, proving speedup/computational time and usefulness of information. We advance formal definitions for these concepts by way of a notion of normality related to an expected (optimal) theoretical speedup when adding useful information (other theorems as axioms), as compared with actual strategies that can be effectively and efficiently implemented. We propose the existence of an ineluctable tradeoff between this normality and computational time complexity. The argument quantifies the usefulness of information in terms of (positive) speed-up. The results disclose a kind of no-free-lunch scenario and a tradeoff of a fundamental nature. The main theorem in this paper together with the numerical experiment---undertaken using two different automatic theorem provers AProS and Prover9 on random theorems of propositional logic---provide strong theoretical and empirical arguments for the fact that finding new useful information for solving a specific problem (theorem) is, in general, as hard as the problem (theorem) itself. △ Less

Submitted 14 June, 2015; originally announced June 2015.

Comments: 14 pages, 7 figures

Quantifying Loss of Information in Network-based Dimensionality Reduction Techniques

Authors: Hector Zenil, Narsis A. Kiani, Jesper Tegnér

Abstract: To cope with the complexity of large networks, a number of dimensionality reduction techniques for graphs have been developed. However, the extent to which information is lost or preserved when these techniques are employed has not yet been clear. Here we develop a framework, based on algorithmic information theory, to quantify the extent to which information is preserved when network motif analys… ▽ More To cope with the complexity of large networks, a number of dimensionality reduction techniques for graphs have been developed. However, the extent to which information is lost or preserved when these techniques are employed has not yet been clear. Here we develop a framework, based on algorithmic information theory, to quantify the extent to which information is preserved when network motif analysis, graph spectra and spectral sparsification methods are applied to over twenty different biological and artificial networks. We find that the spectral sparsification is highly sensitive to high number of edge deletion, leading to significant inconsistencies, and that graph spectral methods are the most irregular, capturing algebraic information in a condensed fashion but largely losing most of the information content of the original networks. However, the approach shows that network motif analysis excels at preserving the relative algorithmic information content of a network, hence validating and generalizing the remarkable fact that despite their inherent combinatorial possibilities, local regularities preserve information to such an extent that essential properties are fully recoverable across different networks to determine their family group to which they belong to (eg genetic vs social network). Our algorithmic information methodology thus provides a rigorous framework enabling a fundamental assessment and comparison between different data dimensionality reduction methods thereby facilitating the identification and evaluation of the capabilities of old and new methods. △ Less

Submitted 27 August, 2015; v1 submitted 23 April, 2015; originally announced April 2015.

Comments: 29 pages, 6 figures

A Computable Measure of Algorithmic Probability by Finite Approximations with an Application to Integer Sequences

Authors: Fernando Soler-Toscano, Hector Zenil

Abstract: Given the widespread use of lossless compression algorithms to approximate algorithmic (Kolmogorov-Chaitin) complexity, and that lossless compression algorithms fall short at characterizing patterns other than statistical ones not different to entropy estimations, here we explore an alternative and complementary approach. We study formal properties of a Levin-inspired measure $m$ calculated from t… ▽ More Given the widespread use of lossless compression algorithms to approximate algorithmic (Kolmogorov-Chaitin) complexity, and that lossless compression algorithms fall short at characterizing patterns other than statistical ones not different to entropy estimations, here we explore an alternative and complementary approach. We study formal properties of a Levin-inspired measure $m$ calculated from the output distribution of small Turing machines. We introduce and justify finite approximations $m_k$ that have been used in some applications as an alternative to lossless compression algorithms for approximating algorithmic (Kolmogorov-Chaitin) complexity. We provide proofs of the relevant properties of both $m$ and $m_k$ and compare them to Levin's Universal Distribution. We provide error estimations of $m_k$ with respect to $m$. Finally, we present an application to integer sequences from the Online Encyclopedia of Integer Sequences which suggests that our AP-based measures may characterize non-statistical patterns, and we report interesting correlations with textual, function and program description lengths of the said sequences. △ Less

Submitted 14 August, 2017; v1 submitted 23 April, 2015; originally announced April 2015.

Comments: As accepted by the journal Complexity (Wiley/Hindawi)