-
A reliable Turing machine
Authors:
Ilir Çapuni,
Peter Gács
Abstract:
We consider computations of a Turing machine subjected to noise. In every step, the action (the new state and the new content of the observed cell, the direction of the head movement) can differ from that prescribed by the transition function with a small probability (independently of previous such events). We construct a universal 1-tape Turing machine that for a low enough (constant) noise proba…
▽ More
We consider computations of a Turing machine subjected to noise. In every step, the action (the new state and the new content of the observed cell, the direction of the head movement) can differ from that prescribed by the transition function with a small probability (independently of previous such events). We construct a universal 1-tape Turing machine that for a low enough (constant) noise probability performs arbitrarily large computations. For this unavoidably, the input needs to be encoded -- by a simple code depending on its size. The work uses a technique familiar from reliable cellular automata, complemented by some new ones.
△ Less
Submitted 3 December, 2021;
originally announced December 2021.
-
A new version of Toom's proof
Authors:
Peter Gacs
Abstract:
There are several proofs now for the stability of Toom's example of a two-dimensional stable cellular automaton and its application to fault-tolerant computation. Simon and Berman simplified and strengthened Toom's original proof: the present report is a simplified exposition of their proof.
There are several proofs now for the stability of Toom's example of a two-dimensional stable cellular automaton and its application to fault-tolerant computation. Simon and Berman simplified and strengthened Toom's original proof: the present report is a simplified exposition of their proof.
△ Less
Submitted 12 May, 2021;
originally announced May 2021.
-
Lecture notes on descriptional complexity and randomness
Authors:
Peter Gacs
Abstract:
A didactical survey of the foundations of Algorithmic Information Theory. These notes are short on motivation, history and background but introduce some of the main techniques and concepts of the field.
The "manuscript" has been evolving over the years. Please, look at "Version history" below to see what has changed when.
A didactical survey of the foundations of Algorithmic Information Theory. These notes are short on motivation, history and background but introduce some of the main techniques and concepts of the field.
The "manuscript" has been evolving over the years. Please, look at "Version history" below to see what has changed when.
△ Less
Submitted 10 May, 2021;
originally announced May 2021.
-
Inequalities for space-bounded Kolmogorov complexity
Authors:
Bruno Bauwens,
Peter Gács,
Andrei Romashchenko,
Alexander Shen
Abstract:
There is a parallelism between Shannon information theory and algorithmic information theory. In particular, the same linear inequalities are true for Shannon entropies of tuples of random variables and Kolmogorov complexities of tuples of strings (Hammer et al., 1997), as well as for sizes of subgroups and projections of sets (Chan, Yeung, Romashchenko, Shen, Vereshchagin, 1998--2002). This paral…
▽ More
There is a parallelism between Shannon information theory and algorithmic information theory. In particular, the same linear inequalities are true for Shannon entropies of tuples of random variables and Kolmogorov complexities of tuples of strings (Hammer et al., 1997), as well as for sizes of subgroups and projections of sets (Chan, Yeung, Romashchenko, Shen, Vereshchagin, 1998--2002). This parallelism started with the Kolmogorov-Levin formula (1968) for the complexity of pairs of strings with logarithmic precision. Longpré (1986) proved a version of this formula for space-bounded complexities.
In this paper we prove an improved version of Longpré's result with a tighter space bound, using Sipser's trick (1980). Then, using this space bound, we show that every linear inequality that is true for complexities or entropies, is also true for space-bounded Kolmogorov complexities with a polynomial space overhead.
△ Less
Submitted 9 September, 2022; v1 submitted 20 October, 2020;
originally announced October 2020.
-
Stable Multi-Level Monotonic Eroders
Authors:
Péter Gács,
Ilkka Törmä
Abstract:
Eroders are monotonic cellular automata with a linearly ordered state set that eventually wipe out any finite island of nonzero states. One-dimensional eroders were studied by Gal'perin in the 1970s, who presented a simple combinatorial characterization of the class. The multi-dimensional case has been studied by Toom and others, but no such characterization has been found. We prove a similar char…
▽ More
Eroders are monotonic cellular automata with a linearly ordered state set that eventually wipe out any finite island of nonzero states. One-dimensional eroders were studied by Gal'perin in the 1970s, who presented a simple combinatorial characterization of the class. The multi-dimensional case has been studied by Toom and others, but no such characterization has been found. We prove a similar characterization for those one-dimensional monotonic cellular automata that are eroders even in the presence of random noise.
△ Less
Submitted 25 September, 2018;
originally announced September 2018.
-
A Turing Machine Resisting Isolated Bursts Of Faults
Authors:
Ilir Capuni,
Peter Gacs
Abstract:
We consider computations of a Turing machine under noise that causes consecutive violations of the machine's transition function. Given a constant upper bound B on the size of bursts of faults, we construct a Turing machine M(B) subject to faults that can simulate any fault-free machine under the condition that bursts are not closer to each other than V for an appropriate V = O(B^2).
We consider computations of a Turing machine under noise that causes consecutive violations of the machine's transition function. Given a constant upper bound B on the size of bursts of faults, we construct a Turing machine M(B) subject to faults that can simulate any fault-free machine under the condition that bursts are not closer to each other than V for an appropriate V = O(B^2).
△ Less
Submitted 6 March, 2012;
originally announced March 2012.
-
Algorithmic tests and randomness with respect to a class of measures
Authors:
Laurent Bienvenu,
Peter Gacs,
Mathieu Hoyrup,
Cristobal Rojas,
Alexander Shen
Abstract:
The paper considers quantitative versions of different randomness notions: algorithmic test measures the amount of non-randomness (and is infinite for non-random sequences). We start with computable measures on Cantor space (and Martin-Lof randomness), then consider uniform randomness (test is a function of a sequence and a measure, not necessarily computable) and arbitrary constructive metric spa…
▽ More
The paper considers quantitative versions of different randomness notions: algorithmic test measures the amount of non-randomness (and is infinite for non-random sequences). We start with computable measures on Cantor space (and Martin-Lof randomness), then consider uniform randomness (test is a function of a sequence and a measure, not necessarily computable) and arbitrary constructive metric spaces. We also consider tests for classes of measures, in particular Bernoulli measures on Cantor space, and show how they are related to uniform tests and original Martin-Lof definition. We show that Hyppocratic (blind, oracle-free) randomness is equivalent to uniform randomness for measures in an effectively orthogonal effectively compact class. We also consider the notions of sparse set and on-line randomness and show how they can be expressed in our framework.
△ Less
Submitted 26 May, 2011; v1 submitted 8 March, 2011;
originally announced March 2011.
-
Information Distance
Authors:
Charles H. Bennett,
Peter Gacs,
Ming Li,
Paul M. B. Vitanyi,
Wojciech H. Zurek
Abstract:
While Kolmogorov complexity is the accepted absolute measure of information content in an individual finite object, a similarly absolute notion is needed for the information distance between two individual objects, for example, two pictures. We give several natural definitions of a universal information metric, based on length of shortest programs for either ordinary computations or reversible (di…
▽ More
While Kolmogorov complexity is the accepted absolute measure of information content in an individual finite object, a similarly absolute notion is needed for the information distance between two individual objects, for example, two pictures. We give several natural definitions of a universal information metric, based on length of shortest programs for either ordinary computations or reversible (dissipationless) computations. It turns out that these definitions are equivalent up to an additive logarithmic term. We show that the information distance is a universal cognitive similarity distance. We investigate the maximal correlation of the shortest programs involved, the maximal uncorrelation of programs (a generalization of the Slepian-Wolf theorem of classical information theory), and the density properties of the discrete metric spaces induced by the information distances. A related distance measures the amount of nonreversibility of a computation. Using the physical theory of reversible computation, we give an appropriate (universal, anti-symmetric, and transitive) measure of the thermodynamic work required to transform one object in another object by the most efficient process. Information distance between individual objects is needed in pattern recognition where one wants to express effective notions of "pattern similarity" or "cognitive similarity" between individual objects and in thermodynamics of computation where one wants to analyse the energy dissipation of a computation from a particular input to a particular output.
△ Less
Submitted 17 June, 2010;
originally announced June 2010.
-
Uniform test of algorithmic randomness over a general space
Authors:
Peter Gacs
Abstract:
The algorithmic theory of randomness is well developed when the underlying space is the set of finite or infinite sequences and the underlying probability distribution is the uniform distribution or a computable distribution. These restrictions seem artificial. Some progress has been made to extend the theory to arbitrary Bernoulli distributions (by Martin-Loef), and to arbitrary distributions (…
▽ More
The algorithmic theory of randomness is well developed when the underlying space is the set of finite or infinite sequences and the underlying probability distribution is the uniform distribution or a computable distribution. These restrictions seem artificial. Some progress has been made to extend the theory to arbitrary Bernoulli distributions (by Martin-Loef), and to arbitrary distributions (by Levin). We recall the main ideas and problems of Levin's theory, and report further progress in the same framework.
- We allow non-compact spaces (like the space of continuous functions, underlying the Brownian motion).
- The uniform test (deficiency of randomness) d_P(x) (depending both on the outcome x and the measure P should be defined in a general and natural way.
- We see which of the old results survive: existence of universal tests, conservation of randomness, expression of tests in terms of description complexity, existence of a universal measure, expression of mutual information as "deficiency of independence.
- The negative of the new randomness test is shown to be a generalization of complexity in continuous spaces; we show that the addition theorem survives.
The paper's main contribution is introducing an appropriate framework for studying these questions and related ones (like statistics for a general family of distributions).
△ Less
Submitted 20 September, 2006; v1 submitted 17 December, 2003;
originally announced December 2003.
-
Deterministic computations whose history is independent of the order of asynchronous updating
Authors:
Peter Gacs
Abstract:
Consider a network of processors (sites) in which each site x has a finite set N(x) of neighbors. There is a transition function f that for each site x computes the next state ξ(x) from the states in N(x). But these transitions (updates) are applied in arbitrary order, one or many at a time. If the state of site x at time t is η(x,t) then let us define the sequence ζ(x,0), ζ(x,1), ... by taking…
▽ More
Consider a network of processors (sites) in which each site x has a finite set N(x) of neighbors. There is a transition function f that for each site x computes the next state ξ(x) from the states in N(x). But these transitions (updates) are applied in arbitrary order, one or many at a time. If the state of site x at time t is η(x,t) then let us define the sequence ζ(x,0), ζ(x,1), ... by taking the sequence η(x,0), η(x,1), ..., and deleting repetitions. The function f is said to have invariant histories if the sequence ζ(x,i), (while it lasts, in case it is finite) depends only on the initial configuration, not on the order of updates.
This paper shows that though the invariant history property is typically undecidable, there is a useful simple sufficient condition, called commutativity: For any configuration, for any pair x,y of neighbors, if the updating would change both ξ(x) and ξ(y) then the result of updating first x and then y is the same as the result of doing this in the reverse order.
△ Less
Submitted 24 January, 2001;
originally announced January 2001.
-
Algorithmic Statistics
Authors:
Peter Gacs,
John Tromp,
Paul Vitanyi
Abstract:
While Kolmogorov complexity is the accepted absolute measure of information content of an individual finite object, a similarly absolute notion is needed for the relation between an individual data sample and an individual model summarizing the information in the data, for example, a finite set (or probability distribution) where the data sample typically came from. The statistical theory based…
▽ More
While Kolmogorov complexity is the accepted absolute measure of information content of an individual finite object, a similarly absolute notion is needed for the relation between an individual data sample and an individual model summarizing the information in the data, for example, a finite set (or probability distribution) where the data sample typically came from. The statistical theory based on such relations between individual objects can be called algorithmic statistics, in contrast to classical statistical theory that deals with relations between probabilistic ensembles. We develop the algorithmic theory of statistic, sufficient statistic, and minimal sufficient statistic. This theory is based on two-part codes consisting of the code for the statistic (the model summarizing the regularity, the meaningful information, in the data) and the model-to-data code. In contrast to the situation in probabilistic statistical theory, the algorithmic relation of (minimal) sufficiency is an absolute relation between the individual model and the individual data sample. We distinguish implicit and explicit descriptions of the models. We give characterizations of algorithmic (Kolmogorov) minimal sufficient statistic for all data samples for both description modes--in the explicit mode under some constraints. We also strengthen and elaborate earlier results on the ``Kolmogorov structure function'' and ``absolutely non-stochastic objects''--those rare objects for which the simplest models that summarize their relevant information (minimal sufficient statistics) are at least as complex as the objects themselves. We demonstrate a close relation between the probabilistic notions and the algorithmic ones.
△ Less
Submitted 9 October, 2001; v1 submitted 30 June, 2000;
originally announced June 2000.
-
Reliable Cellular Automata with Self-Organization
Authors:
Peter Gacs
Abstract:
In a probabilistic cellular automaton in which all local transitions have positive probability, the problem of kee** a bit of information indefinitely is nontrivial, even in an infinite automaton. Still, there is a solution in 2 dimensions, and this solution can be used to construct a simple 3-dimensional discrete-time universal fault-tolerant cellular automaton. This technique does not help muc…
▽ More
In a probabilistic cellular automaton in which all local transitions have positive probability, the problem of kee** a bit of information indefinitely is nontrivial, even in an infinite automaton. Still, there is a solution in 2 dimensions, and this solution can be used to construct a simple 3-dimensional discrete-time universal fault-tolerant cellular automaton. This technique does not help much to solve the following problems: remembering a bit of information in 1 dimension; computing in dimensions lower than 3; computing in any dimension with non-synchronized transitions.
Our more complex technique organizes the cells in blocks that perform a reliable simulation of a second (generalized) cellular automaton. The cells of the latter automaton are also organized in blocks, simulating even more reliably a third automaton, etc. Since all this (a possibly infinite hierarchy) is organized in ``software'', it must be under repair all the time from damage caused by errors. A large part of the problem is essentially self-stabilization recovering from a mess of arbitrary size and content. The present paper constructs an asynchronous one-dimensional fault-tolerant cellular automaton, with the further feature of ``self-organization''. The latter means that the initial configuration does not have to encode an infinite hierarchy -- this will be built up over time.
This is a corrected and strengthened version of the journal paper of 2001.
△ Less
Submitted 24 January, 2024; v1 submitted 20 March, 2000;
originally announced March 2000.
