Showing 1–17 of 17 results for author: Terilla, J

An enriched category theory of language: from syntax to semantics

Authors: Tai-Danae Bradley, John Terilla, Yiannis Vlassopoulos

Abstract: State of the art language models return a natural language text continuation from any piece of input text. This ability to generate coherent text extensions implies significant sophistication, including a knowledge of grammar and semantics. In this paper, we propose a mathematical framework for passing from probability distributions on extensions of given texts, such as the ones learned by today's… ▽ More State of the art language models return a natural language text continuation from any piece of input text. This ability to generate coherent text extensions implies significant sophistication, including a knowledge of grammar and semantics. In this paper, we propose a mathematical framework for passing from probability distributions on extensions of given texts, such as the ones learned by today's large language models, to an enriched category containing semantic information. Roughly speaking, we model probability distributions on texts as a category enriched over the unit interval. Objects of this category are expressions in language, and hom objects are conditional probabilities that one expression is an extension of another. This category is syntactical -- it describes what goes with what. Then, via the Yoneda embedding, we pass to the enriched category of unit interval-valued copresheaves on this syntactical category. This category of enriched copresheaves is semantic -- it is where we find meaning, logical operations such as entailment, and the building blocks for more elaborate semantic concepts. △ Less

Submitted 17 November, 2021; v1 submitted 15 June, 2021; originally announced June 2021.

Comments: 29 pages; v2 major revision with new proofs and computations

Tensor Networks for Probabilistic Sequence Modeling

Authors: Jacob Miller, Guillaume Rabusseau, John Terilla

Abstract: Tensor networks are a powerful modeling framework developed for computational many-body physics, which have only recently been applied within machine learning. In this work we utilize a uniform matrix product state (u-MPS) model for probabilistic modeling of sequence data. We first show that u-MPS enable sequence-level parallelism, with length-n sequences able to be evaluated in depth O(log n). We… ▽ More Tensor networks are a powerful modeling framework developed for computational many-body physics, which have only recently been applied within machine learning. In this work we utilize a uniform matrix product state (u-MPS) model for probabilistic modeling of sequence data. We first show that u-MPS enable sequence-level parallelism, with length-n sequences able to be evaluated in depth O(log n). We then introduce a novel generative algorithm giving trained u-MPS the ability to efficiently sample from a wide variety of conditional distributions, each one defined by a regular expression. Special cases of this algorithm correspond to autoregressive and fill-in-the-blank sampling, but more complex regular expressions permit the generation of richly structured data in a manner that has no direct analogue in neural generative models. Experiments on sequence modeling with synthetic and real text data show u-MPS outperforming a variety of baselines and effectively generalizing their predictions in the presence of limited data. △ Less

Submitted 23 April, 2021; v1 submitted 2 March, 2020; originally announced March 2020.

Comments: 18 pages, 2 figures; v4 conference version; v3 link to code for experiments; v2 major revision with new main result on regular expression sampling. International Conference on Artificial Intelligence and Statistics. PMLR, 2021

Modeling Sequences with Quantum States: A Look Under the Hood

Authors: Tai-Danae Bradley, E. Miles Stoudenmire, John Terilla

Abstract: Classical probability distributions on sets of sequences can be modeled using quantum states. Here, we do so with a quantum state that is pure and entangled. Because it is entangled, the reduced densities that describe subsystems also carry information about the complementary subsystem. This is in contrast to the classical marginal distributions on a subsystem in which information about the comple… ▽ More Classical probability distributions on sets of sequences can be modeled using quantum states. Here, we do so with a quantum state that is pure and entangled. Because it is entangled, the reduced densities that describe subsystems also carry information about the complementary subsystem. This is in contrast to the classical marginal distributions on a subsystem in which information about the complementary system has been integrated out and lost. A training algorithm based on the density matrix renormalization group (DMRG) procedure uses the extra information contained in the reduced densities and organizes it into a tensor network model. An understanding of the extra information contained in the reduced densities allow us to examine the mechanics of this DMRG algorithm and study the generalization error of the resulting model. As an illustration, we work with the even-parity dataset and produce an estimate for the generalization error as a function of the fraction of the dataset used in training. △ Less

Submitted 16 October, 2019; originally announced October 2019.

Comments: 27 pages

Journal ref: 2020 Mach. Learn.: Sci. Technol. 1 035008

Probabilistic Modeling with Matrix Product States

Authors: James Stokes, John Terilla

Abstract: Inspired by the possibility that generative models based on quantum circuits can provide a useful inductive bias for sequence modeling tasks, we propose an efficient training algorithm for a subset of classically simulable quantum circuit models. The gradient-free algorithm, presented as a sequence of exactly solvable effective models, is a modification of the density matrix renormalization group… ▽ More Inspired by the possibility that generative models based on quantum circuits can provide a useful inductive bias for sequence modeling tasks, we propose an efficient training algorithm for a subset of classically simulable quantum circuit models. The gradient-free algorithm, presented as a sequence of exactly solvable effective models, is a modification of the density matrix renormalization group procedure adapted for learning a probability distribution. The conclusion that circuit-based models offer a useful inductive bias for classical datasets is supported by experimental results on the parity learning problem. △ Less

Submitted 18 February, 2019; originally announced February 2019.

Language as a matrix product state

Authors: Vasily Pestun, John Terilla, Yiannis Vlassopoulos

Abstract: We propose a statistical model for natural language that begins by considering language as a monoid, then representing it in complex matrices with a compatible translation invariant probability measure. We interpret the probability measure as arising via the Born rule from a translation invariant matrix product state. We propose a statistical model for natural language that begins by considering language as a monoid, then representing it in complex matrices with a compatible translation invariant probability measure. We interpret the probability measure as arising via the Born rule from a translation invariant matrix product state. △ Less

Submitted 4 November, 2017; originally announced November 2017.

Comments: 10 pages

Homotopy probability theory on a Riemannian manifold and the Euler equation

Authors: Gabriel C. Drummond-Cole, John Terilla

Abstract: Homotopy probability theory is a version of probability theory in which the vector space of random variables is replaced with a chain complex. A natural example extends ordinary probability theory on a finite volume Riemannian manifold M. In this example, initial conditions for fluid flow on M are identified with collections of homotopy random variables and solutions to the Euler equation are iden… ▽ More Homotopy probability theory is a version of probability theory in which the vector space of random variables is replaced with a chain complex. A natural example extends ordinary probability theory on a finite volume Riemannian manifold M. In this example, initial conditions for fluid flow on M are identified with collections of homotopy random variables and solutions to the Euler equation are identified with homotopies between collections of homotopy random variables. Several ideas about using homotopy probability theory to study fluid flow are introduced. △ Less

Submitted 30 July, 2016; originally announced August 2016.

Comments: 17 pages

MSC Class: 55U35; 58Axx; 58Cxx; 60Axx; 76xx

Cones in homotopy probability theory

Authors: Gabriel C. Drummond-Cole, John Terilla

Abstract: This note defines cones in homotopy probability theory and demonstrates that a cone over a space is a reasonable replacement for the space. The homotopy Gaussian distribution in one variable is revisited as a cone on the ordinary Gaussian. This note defines cones in homotopy probability theory and demonstrates that a cone over a space is a reasonable replacement for the space. The homotopy Gaussian distribution in one variable is revisited as a cone on the ordinary Gaussian. △ Less

Submitted 16 November, 2014; v1 submitted 20 October, 2014; originally announced October 2014.

Comments: 8 pages. Missing reference added

MSC Class: 55U35; 46L53; 60Axx

Homotopy Probability Theory II

Authors: Gabriel C. Drummond-Cole, Jae-Suk Park, John Terilla

Abstract: This is the second of two papers that introduce a deformation theoretic framework to explain and broaden a link between homotopy algebra and probability theory. This paper outlines how the framework can assist in the development of homotopy probability theory, where a vector space of random variables is replaced by a chain complex of random variables. This allows the principles of derived mathemat… ▽ More This is the second of two papers that introduce a deformation theoretic framework to explain and broaden a link between homotopy algebra and probability theory. This paper outlines how the framework can assist in the development of homotopy probability theory, where a vector space of random variables is replaced by a chain complex of random variables. This allows the principles of derived mathematics to participate in classical and noncommutative probability theory. A simple example is presented. △ Less

Submitted 1 April, 2014; v1 submitted 21 February, 2013; originally announced February 2013.

Comments: 11 pages. Publication version

Homotopy Probability Theory I

Authors: Gabriel C. Drummond-Cole, Jae-Suk Park, John Terilla

Abstract: This is the first of two papers that introduce a deformation theoretic framework to explain and broaden a link between homotopy algebra and probability theory. In this paper, cumulants are proved to coincide with morphisms of homotopy algebras. The sequel paper outlines how the framework presented here can assist in the development of homotopy probability theory, allowing the principles of derived… ▽ More This is the first of two papers that introduce a deformation theoretic framework to explain and broaden a link between homotopy algebra and probability theory. In this paper, cumulants are proved to coincide with morphisms of homotopy algebras. The sequel paper outlines how the framework presented here can assist in the development of homotopy probability theory, allowing the principles of derived mathematics to participate in classical and noncommutative probability theory. △ Less

Submitted 14 October, 2013; v1 submitted 15 February, 2013; originally announced February 2013.

Comments: 9 pages, 2 figures. Revised

Homotopy DG algebras induce homotopy BV algebras

Authors: John Terilla, Thomas Tradler, Scott O. Wilson

Abstract: Let TA denote the space underlying the tensor algebra of a vector space A. In this short note, we show that if A is a differential graded algebra, then TA is a differential Batalin-Vilkovisky algebra. Moreover, if A is an A-infinity algebra, then TA is a commutative BV-infinity algebra. Let TA denote the space underlying the tensor algebra of a vector space A. In this short note, we show that if A is a differential graded algebra, then TA is a differential Batalin-Vilkovisky algebra. Moreover, if A is an A-infinity algebra, then TA is a commutative BV-infinity algebra. △ Less

Submitted 9 June, 2011; originally announced June 2011.

Comments: 6 pages, accepted by JHRS

Algebras over Cobar(coFrob)

Authors: Gabriel C. Drummond-Cole, John Terilla, Thomas Tradler

Abstract: We show that a square zero, degree one element in W(V), the Weyl algebra on a vector space V, is equivalent to providing V with the structure of an algebra over the properad Cobar(coFrob), the properad arising from the cobar construction applied to the cofrobenius coproperad. We show that a square zero, degree one element in W(V), the Weyl algebra on a vector space V, is equivalent to providing V with the structure of an algebra over the properad Cobar(coFrob), the properad arising from the cobar construction applied to the cofrobenius coproperad. △ Less

Submitted 25 September, 2009; v1 submitted 8 July, 2008; originally announced July 2008.

Comments: 24 pages, revised version

Smoothness Theorem for Differential BV Algebras

Authors: John Terilla

Abstract: Associated to a differential BV algebra are two differential graded Lie algebras: we call one classical and the other, which contains a formal h-bar parameter, quantum. The classical dgLa is always smooth formal. In this paper, we give necessary and sufficient conditions for the quantum dgLa to be smooth formal. These conditions are equivalent to the degeneration of a version of the noncommutati… ▽ More Associated to a differential BV algebra are two differential graded Lie algebras: we call one classical and the other, which contains a formal h-bar parameter, quantum. The classical dgLa is always smooth formal. In this paper, we give necessary and sufficient conditions for the quantum dgLa to be smooth formal. These conditions are equivalent to the degeneration of a version of the noncommutative Hodge to de Rham spectral sequence. References added. △ Less

Submitted 10 July, 2008; v1 submitted 9 July, 2007; originally announced July 2007.

Comments: 10 pages

MSC Class: 14B12; 55D99; 17B60

Journal ref: Journal of Topology 2008 1: 693-702

Quantum backgrounds and QFT

Authors: Jae-Suk Park, John Terilla, Thomas Tradler

Abstract: We introduce the concept of a quantum background and a functor QFT. In the case that the QFT moduli space is smooth formal, we construct a flat quantum superconnection on a bundle over QFT which defines algebraic structures relevant to correlation functions in quantum field theory. We go further and identify chain level generalizations of correlation functions which should be present in all quan… ▽ More We introduce the concept of a quantum background and a functor QFT. In the case that the QFT moduli space is smooth formal, we construct a flat quantum superconnection on a bundle over QFT which defines algebraic structures relevant to correlation functions in quantum field theory. We go further and identify chain level generalizations of correlation functions which should be present in all quantum field theories. △ Less

Submitted 21 September, 2009; v1 submitted 16 May, 2006; originally announced May 2006.

Comments: 28 pages, published version

MSC Class: 14D; 53D

Journal ref: Int.Math.Res.Not.5:765-801,2009

Deformations of associative algebras with inner products

Authors: John Terilla, Thomas Tradler

Abstract: We develop the deformation theory of A_\infty algebras together with \infty inner products and identify a differential graded Lie algebra that controls the theory. This generalizes the deformation theories of associative algebras, A_\infty algebras, associative algebras with inner products, and A_\infty algebras with inner products. We develop the deformation theory of A_\infty algebras together with \infty inner products and identify a differential graded Lie algebra that controls the theory. This generalizes the deformation theories of associative algebras, A_\infty algebras, associative algebras with inner products, and A_\infty algebras with inner products. △ Less

Submitted 2 May, 2003; originally announced May 2003.

Comments: 14 pages

MSC Class: 55P10;16S10;16S80

Probabilistic quantum error correction

Authors: Jesse Fern, John Terilla

Abstract: There are well known necessary and sufficient conditions for a quantum code to correct a set of errors. We study weaker conditions under which a quantum code may correct errors with probabilities that may be less than one. We work with stabilizer codes and as an application study how the nine qubit code, the seven qubit code, and the five qubit code perform when there are errors on more than one… ▽ More There are well known necessary and sufficient conditions for a quantum code to correct a set of errors. We study weaker conditions under which a quantum code may correct errors with probabilities that may be less than one. We work with stabilizer codes and as an application study how the nine qubit code, the seven qubit code, and the five qubit code perform when there are errors on more than one qubit. As a second application, we discuss the concept of syndrome quality and use it to suggest a way that quantum error correction can be practically improved. △ Less

Submitted 7 October, 2002; v1 submitted 6 September, 2002; originally announced September 2002.

Comments: RevTeX4, 9 pages. for v3, citations added and corrected

Thermodynamic interpretation of quantum error correcting criterion

Authors: Vladimir Korepin, John Terilla

Abstract: Shanon's fundamental coding theorems relate classical information theory to thermodynamics. More recent theoretical work has been successful in relating quantum information theory to thermodynamics. For example, Schumacher proved a quantum version of Shannon's 1948 classical noiseless coding theorem. In this note, we extend the connection between quantum information theory and thermodynamics to… ▽ More Shanon's fundamental coding theorems relate classical information theory to thermodynamics. More recent theoretical work has been successful in relating quantum information theory to thermodynamics. For example, Schumacher proved a quantum version of Shannon's 1948 classical noiseless coding theorem. In this note, we extend the connection between quantum information theory and thermodynamics to include quantum error correction. There is a standard mechanism for describing errors that may occur during the transmission, storage, and manipulation of quantum information. One can formulate a criterion of necessary and sufficient conditions for the errors to be detectable and correctable. We show that this criterion has a thermodynamical interpretation. △ Less

Submitted 1 March, 2004; v1 submitted 10 February, 2002; originally announced February 2002.

Comments: LATEX file, 18 pages, 1 figure

Report number: YITP-SB-02-07

Journal ref: Quantum Information Processing, vol 1, issue 4, pages 225-242, 2002

Hypersurfaces and generalized deformations

Authors: John Terilla

Abstract: The moduli space of generalized deformations of a Calabi-Yau hypersurface is computed in terms of the Jacobian ring of the defining polynomial. The fibers of the tangent bundle to this moduli space carry algebra structures, which are identified using subalgebras of a deformed Jacobian ring. The moduli space of generalized deformations of a Calabi-Yau hypersurface is computed in terms of the Jacobian ring of the defining polynomial. The fibers of the tangent bundle to this moduli space carry algebra structures, which are identified using subalgebras of a deformed Jacobian ring. △ Less

Submitted 24 December, 2003; v1 submitted 8 December, 2000; originally announced December 2000.

Comments: 25 pages

MSC Class: 14D15; 17B66