Showing 301–318 of 318 results for author: Hutter, M

Distribution of Mutual Information

Authors: Marcus Hutter

Abstract: The mutual information of two random variables i and j with joint probabilities t_ij is commonly used in learning Bayesian nets as well as in many other fields. The chances t_ij are usually estimated by the empirical sampling frequency n_ij/n leading to a point estimate I(n_ij/n) for the mutual information. To answer questions like "is I(n_ij/n) consistent with zero?" or "what is the probability… ▽ More The mutual information of two random variables i and j with joint probabilities t_ij is commonly used in learning Bayesian nets as well as in many other fields. The chances t_ij are usually estimated by the empirical sampling frequency n_ij/n leading to a point estimate I(n_ij/n) for the mutual information. To answer questions like "is I(n_ij/n) consistent with zero?" or "what is the probability that the true mutual information is much larger than the point estimate?" one has to go beyond the point estimate. In the Bayesian framework one can answer these questions by utilizing a (second order) prior distribution p(t) comprising prior information about t. From the prior p(t) one can compute the posterior p(t|n), from which the distribution p(I|n) of the mutual information can be calculated. We derive reliable and quickly computable approximations for p(I|n). We concentrate on the mean, variance, skewness, and kurtosis, and non-informative priors. For the mean we also give an exact expression. Numerical issues and the range of validity are discussed. △ Less

Submitted 15 December, 2001; originally announced December 2001.

Comments: 8 pages

Report number: IDSIA-13-01 ACM Class: G.3; G.1.2

Journal ref: Advances in Neural Information Processing Systems 14 (NIPS-2001) pages 399-406

Gradient-based Reinforcement Planning in Policy-Search Methods

Authors: Ivo Kwee, Marcus Hutter, Juergen Schmidhuber

Abstract: We introduce a learning method called ``gradient-based reinforcement planning'' (GREP). Unlike traditional DP methods that improve their policy backwards in time, GREP is a gradient-based method that plans ahead and improves its policy before it actually acts in the environment. We derive formulas for the exact policy gradient that maximizes the expected future reward and confirm our ideas with… ▽ More We introduce a learning method called ``gradient-based reinforcement planning'' (GREP). Unlike traditional DP methods that improve their policy backwards in time, GREP is a gradient-based method that plans ahead and improves its policy before it actually acts in the environment. We derive formulas for the exact policy gradient that maximizes the expected future reward and confirm our ideas with numerical experiments. △ Less

Submitted 28 November, 2001; originally announced November 2001.

Comments: This is an extended version of the paper presented at the EWRL 2001 in Utrecht (The Netherlands)

Report number: 14-01 ACM Class: I.2; I.2.6; I.2.8

Instantons in QCD: Theory and Application of the Instanton Liquid Model

Authors: Marcus Hutter

Abstract: Numerical and anaytical studies of the instanton liquid model have allowed the determination of many hadronic parameters during the last 13 years. Most part of this thesis is devoted to the extension of the analytical methods. The meson correlation (polarization) functions are calculated in the instanton liquid model including dynamical quark loops. The correlators are plotted and masses and cou… ▽ More Numerical and anaytical studies of the instanton liquid model have allowed the determination of many hadronic parameters during the last 13 years. Most part of this thesis is devoted to the extension of the analytical methods. The meson correlation (polarization) functions are calculated in the instanton liquid model including dynamical quark loops. The correlators are plotted and masses and couplings of the $σ$, $ρ$, $ω$, $a1$ and $f1$ are obtained from a spectral fit. A separated analysis allows the determination of the $η'$ mass too. The results agree with the experimental values on a 10% level. Further I give some predictions for the proton form factors, which are related to the proton spin (problem). A gauge invariant gluon mass for small momenta is also calculated. At the end of the work some predictions are given, which do not rely on the instanton liquid model. A gauge invariant quark propagator is calculated in the one instanton background and is compared to the regular and singular propagator. An introduction to the skill of choosing a suitable gauge, especially a criterium for choosing regular or singular gauge, is given. An application is the derivation of a finite relation betweewn the quark condensate and the QCD scale $Λ$, where neither an infrared cutoff nor a specific instanton model has been used. In general the instanton liquid model exhibits an astonishing internal consistency and a good agreement with the experimental data. △ Less

Submitted 9 July, 2001; originally announced July 2001.

Comments: 100 pages, thesis translated into english

Convergence and Error Bounds for Universal Prediction of Nonbinary Sequences

Authors: Marcus Hutter

Abstract: Solomonoff's uncomputable universal prediction scheme $ξ$ allows to predict the next symbol $x_k$ of a sequence $x_1...x_{k-1}$ for any Turing computable, but otherwise unknown, probabilistic environment $μ$. This scheme will be generalized to arbitrary environmental classes, which, among others, allows the construction of computable universal prediction schemes $ξ$. Convergence of $ξ$ to $μ$ in… ▽ More Solomonoff's uncomputable universal prediction scheme $ξ$ allows to predict the next symbol $x_k$ of a sequence $x_1...x_{k-1}$ for any Turing computable, but otherwise unknown, probabilistic environment $μ$. This scheme will be generalized to arbitrary environmental classes, which, among others, allows the construction of computable universal prediction schemes $ξ$. Convergence of $ξ$ to $μ$ in a conditional mean squared sense and with $μ$ probability 1 is proven. It is shown that the average number of prediction errors made by the universal $ξ$ scheme rapidly converges to those made by the best possible informed $μ$ scheme. The schemes, theorems and proofs are given for general finite alphabet, which results in additional complications as compared to the binary case. Several extensions of the presented theory and results are outlined. They include general loss functions and bounds, games of chance, infinite alphabet, partial and delayed prediction, classification, and more active systems. △ Less

Submitted 15 June, 2001; originally announced June 2001.

Comments: 11 LaTeX pages

Report number: IDSIA-07-01 ACM Class: F.2.3

Journal ref: Lecture Notes in Artificial Intelligence (LNAI 2167), Proc. 12th Eurpean Conf. on Machine Learning (ECML) (2001) 239-250

Market-Based Reinforcement Learning in Partially Observable Worlds

Authors: Ivo Kwee, Marcus Hutter, Juergen Schmidhuber

Abstract: Unlike traditional reinforcement learning (RL), market-based RL is in principle applicable to worlds described by partially observable Markov Decision Processes (POMDPs), where an agent needs to learn short-term memories of relevant previous events in order to execute optimal actions. Most previous work, however, has focused on reactive settings (MDPs) instead of POMDPs. Here we reimplement a re… ▽ More Unlike traditional reinforcement learning (RL), market-based RL is in principle applicable to worlds described by partially observable Markov Decision Processes (POMDPs), where an agent needs to learn short-term memories of relevant previous events in order to execute optimal actions. Most previous work, however, has focused on reactive settings (MDPs) instead of POMDPs. Here we reimplement a recent approach to market-based RL and for the first time evaluate it in a toy POMDP setting. △ Less

Submitted 15 May, 2001; originally announced May 2001.

Comments: 8 LaTeX pages, 2 postscript figures

Report number: IDSIA-10-01 ACM Class: I.2

Journal ref: Lecture Notes in Computer Science (LNCS 2130), Proceeding of the International Conference on Artificial Neural Networks ICANN (2001) 865-873

Fitness Uniform Selection to Preserve Genetic Diversity

Authors: Marcus Hutter

Abstract: In evolutionary algorithms, the fitness of a population increases with time by mutating and recombining individuals and by a biased selection of more fit individuals. The right selection pressure is critical in ensuring sufficient optimization progress on the one hand and in preserving genetic diversity to be able to escape from local optima on the other. We propose a new selection scheme, which… ▽ More In evolutionary algorithms, the fitness of a population increases with time by mutating and recombining individuals and by a biased selection of more fit individuals. The right selection pressure is critical in ensuring sufficient optimization progress on the one hand and in preserving genetic diversity to be able to escape from local optima on the other. We propose a new selection scheme, which is uniform in the fitness values. It generates selection pressure towards sparsely populated fitness regions, not necessarily towards higher fitness, as is the case for all other selection schemes. We show that the new selection scheme can be much more effective than standard selection schemes. △ Less

Submitted 14 March, 2001; originally announced March 2001.

Comments: 13 LaTeX pages, 1 eps figure

Report number: IDSIA-01-01 ACM Class: I.2; I.2.6; I.2.8; F.2

Journal ref: Proceedings of the 2002 Congress on Evolutionary Computation (CEC-2002) 783-788

An effective Procedure for Speeding up Algorithms

Authors: Marcus Hutter

Abstract: The provably asymptotically fastest algorithm within a factor of 5 for formally described problems will be constructed. The main idea is to enumerate all programs provably equivalent to the original problem by enumerating all proofs. The algorithm could be interpreted as a generalization and improvement of Levin search, which is, within a multiplicative constant, the fastest algorithm for invert… ▽ More The provably asymptotically fastest algorithm within a factor of 5 for formally described problems will be constructed. The main idea is to enumerate all programs provably equivalent to the original problem by enumerating all proofs. The algorithm could be interpreted as a generalization and improvement of Levin search, which is, within a multiplicative constant, the fastest algorithm for inverting functions. Blum's speed-up theorem is avoided by taking into account only programs for which a correctness proof exists. Furthermore, it is shown that the fastest program that computes a certain function is also one of the shortest programs provably computing this function. To quantify this statement, the definition of Kolmogorov complexity is extended, and two new natural measures for the complexity of a function are defined. △ Less

Submitted 21 February, 2001; originally announced February 2001.

Comments: 10 LaTeX pages

Report number: IDSIA-16-00 ACM Class: F.2.3

Journal ref: Workshop on Mathematical approaches to Biological Computation (MaBiC 2001) and Workshop on Algorithmic Information Theory (TAI 2001)

General Loss Bounds for Universal Sequence Prediction

Authors: Marcus Hutter

Abstract: The Bayesian framework is ideally suited for induction problems. The probability of observing $x_t$ at time $t$, given past observations $x_1...x_{t-1}$ can be computed with Bayes' rule if the true distribution $μ$ of the sequences $x_1x_2x_3...$ is known. The problem, however, is that in many cases one does not even have a reasonable estimate of the true distribution. In order to overcome this… ▽ More The Bayesian framework is ideally suited for induction problems. The probability of observing $x_t$ at time $t$, given past observations $x_1...x_{t-1}$ can be computed with Bayes' rule if the true distribution $μ$ of the sequences $x_1x_2x_3...$ is known. The problem, however, is that in many cases one does not even have a reasonable estimate of the true distribution. In order to overcome this problem a universal distribution $ξ$ is defined as a weighted sum of distributions $μ_i\inM$, where $M$ is any countable set of distributions including $μ$. This is a generalization of Solomonoff induction, in which $M$ is the set of all enumerable semi-measures. Systems which predict $y_t$, given $x_1...x_{t-1}$ and which receive loss $l_{x_t y_t}$ if $x_t$ is the true next symbol of the sequence are considered. It is proven that using the universal $ξ$ as a prior is nearly as good as using the unknown true distribution $μ$. Furthermore, games of chance, defined as a sequence of bets, observations, and rewards are studied. The time needed to reach the winning zone is bounded in terms of the relative entropy of $μ$ and $ξ$. Extensions to arbitrary alphabets, partial and delayed prediction, and more active systems are discussed. △ Less

Submitted 19 September, 2001; v1 submitted 21 January, 2001; originally announced January 2001.

Comments: 8 two-column pages, LaTeX2e

Report number: IDSIA-03-01 ACM Class: I.2; I.2.6; I.2.8; F.1.3

Journal ref: Proc. 18th Int. Conf. on Machine Learning ICML (2001) 210-217

Towards a Universal Theory of Artificial Intelligence based on Algorithmic Probability and Sequential Decision Theory

Authors: Marcus Hutter

Abstract: Decision theory formally solves the problem of rational agents in uncertain worlds if the true environmental probability distribution is known. Solomonoff's theory of universal induction formally solves the problem of sequence prediction for unknown distribution. We unify both theories and give strong arguments that the resulting universal AIXI model behaves optimal in any computable environment… ▽ More Decision theory formally solves the problem of rational agents in uncertain worlds if the true environmental probability distribution is known. Solomonoff's theory of universal induction formally solves the problem of sequence prediction for unknown distribution. We unify both theories and give strong arguments that the resulting universal AIXI model behaves optimal in any computable environment. The major drawback of the AIXI model is that it is uncomputable. To overcome this problem, we construct a modified algorithm AIXI^tl, which is still superior to any other time t and space l bounded agent. The computation time of AIXI^tl is of the order t x 2^l. △ Less

Submitted 16 December, 2000; originally announced December 2000.

Comments: 8 two-column pages, latex2e, 1 figure, submitted to ijcai

ACM Class: I.2; I.2.3; I.2.6; I.2.8; F.1.3; F.2

Journal ref: Lecture Notes in Artificial Intelligence (LNAI 2167), Proc. 12th Eurpean Conf. on Machine Learning, ECML (2001) 226--238

A Theory of Universal Artificial Intelligence based on Algorithmic Complexity

Authors: Marcus Hutter

Abstract: Decision theory formally solves the problem of rational agents in uncertain worlds if the true environmental prior probability distribution is known. Solomonoff's theory of universal induction formally solves the problem of sequence prediction for unknown prior distribution. We combine both ideas and get a parameterless theory of universal Artificial Intelligence. We give strong arguments that t… ▽ More Decision theory formally solves the problem of rational agents in uncertain worlds if the true environmental prior probability distribution is known. Solomonoff's theory of universal induction formally solves the problem of sequence prediction for unknown prior distribution. We combine both ideas and get a parameterless theory of universal Artificial Intelligence. We give strong arguments that the resulting AIXI model is the most intelligent unbiased agent possible. We outline for a number of problem classes, including sequence prediction, strategic games, function minimization, reinforcement and supervised learning, how the AIXI model can formally solve them. The major drawback of the AIXI model is that it is uncomputable. To overcome this problem, we construct a modified algorithm AIXI-tl, which is still effectively more intelligent than any other time t and space l bounded agent. The computation time of AIXI-tl is of the order tx2^l. Other discussed topics are formal definitions of intelligence order relations, the horizon problem and relations of the AIXI theory to other AI approaches. △ Less

Submitted 3 April, 2000; originally announced April 2000.

Comments: 62 pages, LaTeX

ACM Class: I.2; F.1.3; E.4

New Error Bounds for Solomonoff Prediction

Authors: Marcus Hutter

Abstract: Solomonoff sequence prediction is a scheme to predict digits of binary strings without knowing the underlying probability distribution. We call a prediction scheme informed when it knows the true probability distribution of the sequence. Several new relations between universal Solomonoff sequence prediction and informed prediction and general probabilistic prediction schemes will be proved. Amon… ▽ More Solomonoff sequence prediction is a scheme to predict digits of binary strings without knowing the underlying probability distribution. We call a prediction scheme informed when it knows the true probability distribution of the sequence. Several new relations between universal Solomonoff sequence prediction and informed prediction and general probabilistic prediction schemes will be proved. Among others, they show that the number of errors in Solomonoff prediction is finite for computable distributions, if finite in the informed case. Deterministic variants will also be studied. The most interesting result is that the deterministic variant of Solomonoff prediction is optimal compared to any other probabilistic or deterministic prediction scheme apart from additive square root corrections only. This makes it well suited even for difficult prediction problems, where it does not suffice when the number of errors is minimal to within some factor greater than one. Solomonoff's original bound and the ones presented here complement each other in a useful way. △ Less

Submitted 26 January, 2001; v1 submitted 13 December, 1999; originally announced December 1999.

Comments: 13 pages, Journal of Computer and System Science, minor changes to 1st version

Report number: IDSIA-11-00 ACM Class: I.2.6; F.1.3; E.4; F.2

Journal ref: J. Computer and System Science 62:4 (2001) 653-667

Family Structure from Periodic Solutions of an Improved Gap Equation

Authors: Andreas Blumhofer, Marcus Hutter

Abstract: Fermion mass models usually contain a horizontal symmetry and therefore fail to predict the exponential mass spectrum of the Standard Model in a natural way. In dynamical symmetry breaking there are different concepts to introduce a fermion mass spectrum, which automatically has the desired hierarchy. In constructing a specific model we show that in some modified gap equations periodic solutions… ▽ More Fermion mass models usually contain a horizontal symmetry and therefore fail to predict the exponential mass spectrum of the Standard Model in a natural way. In dynamical symmetry breaking there are different concepts to introduce a fermion mass spectrum, which automatically has the desired hierarchy. In constructing a specific model we show that in some modified gap equations periodic solutions with several fermion poles appear. The stability of these excitations and the application of this toy model are discussed. The mass ratios turn out to be approximately $e^π$ and $e^{2π}$. Thus the model explains the large ratios of fermion masses between successive generations in the Standard Model without introducing large or small numbers by hand. △ Less

Submitted 24 May, 1996; originally announced May 1996.

Comments: 19 pages, 8 figures

Report number: LMU-05/96

Journal ref: Nucl.Phys. B484 (1997) 80-96; Erratum-ibid. B494 (1997) 485

Instantons in QCD: Theory and application of the instanton liquid model

Authors: Marcus Hutter

Abstract: Numerical and anaytical studies of the instanton liquid model have allowed the determination of many hadronic parameters during the last 13 years. Most part of this thesis is devoted to the extension of the analytical methods. The meson correlation (polarization) functions are calculated in the instanton liquid model including dynamical quark loops. The correlators are plotted and masses and cou… ▽ More Numerical and anaytical studies of the instanton liquid model have allowed the determination of many hadronic parameters during the last 13 years. Most part of this thesis is devoted to the extension of the analytical methods. The meson correlation (polarization) functions are calculated in the instanton liquid model including dynamical quark loops. The correlators are plotted and masses and couplings of the $σ$, $ρ$, $ω$, $a1$ and $f1$ are obtained from a spectral fit. A separated analysis allows the determination of the $η'$ mass too. The results agree with the experimental values on a 10\% level. Further I give some predictions for the proton form factors, which are related to the proton spin (problem). A gauge invariant gluon mass for small momenta is also calculated. At the end of the work some predictions are given, which do not rely on the instanton liquid model. A gauge invariant quark propagator is calculated in the one instanton background and is compared to the regular and singular propagator. An introduction to the skill of choosing a suitable gauge, especially a criterium for choosing regular or singular gauge, is given. An application is the derivation of a finite relation betweewn the quark condensate and the QCD scale $Λ$, where neither an infrared cutoff nor a specific instanton model has been used. In general the instanton liquid model exhibits an astonishing internal consistency and a good agreement with the experimental data. △ Less

Submitted 13 March, 1996; v1 submitted 12 March, 1996; originally announced March 1996.

Comments: 105 pages, thesis in german (dvi-file)

Proton Spin in the Instanton Background

Authors: Marcus Hutter

Abstract: The proton form factors are reduced to vacuum correlators of 4 quark fields by assuming independent constituent quarks. The axial singlet quark and gluonic form factors are calculated in the instanton liquid model. A discussion of gauge(in)dependence is given. The proton form factors are reduced to vacuum correlators of 4 quark fields by assuming independent constituent quarks. The axial singlet quark and gluonic form factors are calculated in the instanton liquid model. A discussion of gauge(in)dependence is given. △ Less

Submitted 27 September, 1995; originally announced September 1995.

Comments: 13 LaTeX pages

Report number: LMU-95-15

The mass of the eta' in self-dual QCD

Authors: Marcus Hutter

Abstract: The QCD gauge field is modeled as an ensemble of statistically independent selfdual and antiselfdual regions. This model is motivated from instanton physics. The scale anomaly then allows to relate the topological susceptibility to the gluon condensate. With the help of Wittens formula for m_eta' and an estimate of the suppression of the gluon condensate due to light quarks the mass of the eta'… ▽ More The QCD gauge field is modeled as an ensemble of statistically independent selfdual and antiselfdual regions. This model is motivated from instanton physics. The scale anomaly then allows to relate the topological susceptibility to the gluon condensate. With the help of Wittens formula for m_eta' and an estimate of the suppression of the gluon condensate due to light quarks the mass of the eta' can be related to f_pi and the physical gluon condensate. We get the quite satisfactory value m_eta'=884+-116 MeV. Using the physical eta' mass as an input it is in princple possible to get information about the interaction between instantons and anti-instantons. △ Less

Submitted 27 September, 1995; originally announced September 1995.

Comments: 6 LaTeX pages

Report number: LMU-95-14

Journal ref: Phys.Lett. B367 (1996) 275-278

Gauge Invariant Quark Propagator in the Instanton Background

Authors: Marcus Hutter

Abstract: After a general discussion on the choice of gauge, we compare the quark propagator in the background of one instanton in regular and singular gauge with a gauge invariant propagator obtained by inserting a path-ordered gluon exponential. Using a gauge motivated by this analysis, we were able to obtain a finite result for the quark condensate without introducing an infrared cutoff nor invoking some… ▽ More After a general discussion on the choice of gauge, we compare the quark propagator in the background of one instanton in regular and singular gauge with a gauge invariant propagator obtained by inserting a path-ordered gluon exponential. Using a gauge motivated by this analysis, we were able to obtain a finite result for the quark condensate without introducing an infrared cutoff nor invoking some instanton model. △ Less

Submitted 21 February, 1995; originally announced February 1995.

Comments: 15 latex pages, 1 figure, macros: epsf.sty

Report number: LMU-95-03

Gluon Mass from Instantons

Authors: Marcus Hutter

Abstract: The gluon propagator is calculated in the instanton background in a form appropriate for extracting the momentum dependent gluon mass. In background-$ξ$-gauge we get for the mass $400$MeV for small $p^2$ independent of the gauge parameter $ξ$. The gluon propagator is calculated in the instanton background in a form appropriate for extracting the momentum dependent gluon mass. In background-$ξ$-gauge we get for the mass $400$MeV for small $p^2$ independent of the gauge parameter $ξ$. △ Less

Submitted 19 January, 1995; v1 submitted 18 January, 1995; originally announced January 1995.

Comments: 13 pages, macros (axodraw.sty, epsf.sty) are included in part 2 (figures) Identical resubmission because of some transfere-Errors caused by invisible characters.

Report number: LMU-93-18

Instantons and Meson Correlators in QCD

Authors: Marcus Hutter

Abstract: Various QCD correlators are calculated in the instanton liquid model in zeromode approximation and $1/N_c$ expansion. Previous works are extended by including dynamical quark loops. In contrast to the original "perturbative" $1/N_c$ expansion not all quark loops are suppressed. In the flavor singlet meson correlators a chain of quark bubbles survives the $N_c\to\infty$ limit causing a massive… ▽ More Various QCD correlators are calculated in the instanton liquid model in zeromode approximation and $1/N_c$ expansion. Previous works are extended by including dynamical quark loops. In contrast to the original "perturbative" $1/N_c$ expansion not all quark loops are suppressed. In the flavor singlet meson correlators a chain of quark bubbles survives the $N_c\to\infty$ limit causing a massive $η^\prime$ in the pseudoscalar correlator while kee** massless pions in the triplet correlator. The correlators are plotted and meson masses and couplings are obtained from a spectral fit. They are compared to the values obtained from numerical studies of the instanton liquid and to experimental results. △ Less

Submitted 9 January, 1995; originally announced January 1995.

Comments: 43 latex pages, 7 1/2 figures included by epsf.tex-macro

Report number: LMU-Muenchen HEP-93-18

Journal ref: Z.Phys.C74:131-143,1997