-
Position: Categorical Deep Learning is an Algebraic Theory of All Architectures
Authors:
Bruno Gavranović,
Paul Lessard,
Andrew Dudzik,
Tamara von Glehn,
João G. M. Araújo,
Petar Veličković
Abstract:
We present our position on the elusive quest for a general-purpose framework for specifying and studying deep learning architectures. Our opinion is that the key attempts made so far lack a coherent bridge between specifying constraints which models must satisfy and specifying their implementations. Focusing on building a such a bridge, we propose to apply category theory -- precisely, the univers…
▽ More
We present our position on the elusive quest for a general-purpose framework for specifying and studying deep learning architectures. Our opinion is that the key attempts made so far lack a coherent bridge between specifying constraints which models must satisfy and specifying their implementations. Focusing on building a such a bridge, we propose to apply category theory -- precisely, the universal algebra of monads valued in a 2-category of parametric maps -- as a single theory elegantly subsuming both of these flavours of neural network design. To defend our position, we show how this theory recovers constraints induced by geometric deep learning, as well as implementations of many architectures drawn from the diverse landscape of neural networks, such as RNNs. We also illustrate how the theory naturally encodes many standard constructs in computer science and automata theory.
△ Less
Submitted 5 June, 2024; v1 submitted 23 February, 2024;
originally announced February 2024.
-
On The Combinatorics of the Gray Cylinder
Authors:
Paul Lessard
Abstract:
This paper develops some combinatorics of the lax Gray cylinder on the cells of Θ understood as a full subcategory of the category of strict ω-categories. More, we construct a span relating the Cartesian cylinder, the Gray cylinder, and the shift functor.
This paper develops some combinatorics of the lax Gray cylinder on the cells of Θ understood as a full subcategory of the category of strict ω-categories. More, we construct a span relating the Cartesian cylinder, the Gray cylinder, and the shift functor.
△ Less
Submitted 27 June, 2022;
originally announced June 2022.
-
$\mathbf{Z}$-Categories I
Authors:
Paul Lessard
Abstract:
This paper is the first in a series of two papers, $\mathbf{Z}$-Categories I and $\mathbf{Z}$-Categories II, which develop the notion of $\mathbf{Z}$-category, the natural bi-infinite analog to strict $ω$-categories, and show that the $\left(\infty,1\right)$-category of spectra relates to the $\left(\infty,1\right)$-category of homotopy coherent $\mathbf{Z}$-categories as the pointed groupoids.…
▽ More
This paper is the first in a series of two papers, $\mathbf{Z}$-Categories I and $\mathbf{Z}$-Categories II, which develop the notion of $\mathbf{Z}$-category, the natural bi-infinite analog to strict $ω$-categories, and show that the $\left(\infty,1\right)$-category of spectra relates to the $\left(\infty,1\right)$-category of homotopy coherent $\mathbf{Z}$-categories as the pointed groupoids.
In this work we provide a $2$-categorical treatment of the combinatorial spectra of \cite{Kan} and argue that this description is a simplicial avatar of the abiding notion of homotopy coherent $\mathbf{Z}$-category. We then develop the theory of limits in the $2$-category of categories with arities of Berger, Mellies, and Weber to provide a cellular category which is to $\mathbf{Z}$-categories as $\triangle$ is to $1$-categories or $Θ_{n}$ is to $n$-categories. In an appendix we provide a generalization of the spectrification functors of 20$^{\mathrm{th}}$ century stable homotopy theory in the language of category-weighted limits.
△ Less
Submitted 1 June, 2022;
originally announced June 2022.
-
Joyal's Suspension Functor on $Θ$ and Kan's Combinatorial Spectra
Authors:
Paul Lessard
Abstract:
In [Joyal] where the category $Θ$ is first defined it is noted that the dimensional shift on $Θ$ suggests an elegant presentation of the unreduced suspension on cellular sets. In this note we prove that the reduced suspension associated to that presentation is left Quillen with respect to the Cisinski model category structure presenting the $\left(\infty,1\right)$-category of pointed spaces and en…
▽ More
In [Joyal] where the category $Θ$ is first defined it is noted that the dimensional shift on $Θ$ suggests an elegant presentation of the unreduced suspension on cellular sets. In this note we prove that the reduced suspension associated to that presentation is left Quillen with respect to the Cisinski model category structure presenting the $\left(\infty,1\right)$-category of pointed spaces and enjoys the correct universal property. More, we go on to describe how, in forthcoming work, inspired by the combinatorial spectra described in [Kan], this suspension functor entails a description of spectra which echoes the weaker form of the homotopy hypothesis, we describe the development of a presentation of spectra as locally finite weak $\mathbf{Z}$-groupoids.
△ Less
Submitted 27 June, 2022; v1 submitted 30 November, 2018;
originally announced December 2018.
-
La production de nitrites lors de la dénitrification des eaux usées par biofiltration - Stratégie de contrôle et de réduction des concentrations résiduelles
Authors:
Vincent Rocher,
Cédric Join,
Stéphane Mottelet,
Jean Bernier,
Sabrina Rechdaoui-Guérin,
Sam Azimi,
Paul Lessard,
André Pauss,
Michel Fliess
Abstract:
The recent popularity of post-denitrification processes in the greater Paris area wastewater treatment plants has caused a resurgence of the presence of nitrite in the Seine river. Controlling the production of nitrite during the post-denitrification has thus become a major technical issue. Research studies have been led in the MOCOPEE program (www.mocopee.com) to better understand the underlying…
▽ More
The recent popularity of post-denitrification processes in the greater Paris area wastewater treatment plants has caused a resurgence of the presence of nitrite in the Seine river. Controlling the production of nitrite during the post-denitrification has thus become a major technical issue. Research studies have been led in the MOCOPEE program (www.mocopee.com) to better understand the underlying mechanisms behind the production of nitrite during wastewater denitrification and to develop technical tools (measurement and control solutions) to assist on-site reductions of nitrite productions. Prior studies have shown that typical methanol dosage strategies produce a varying carbon-to-nitrogen ratio in the reactor, which in turn leads to unstable nitrite concentrations in the effluent. The possibility of adding a model-free control to the actual classical dosage strategy has thus been tested on the SimBio model, which simulates the behavior of wastewater biofilters. The corresponding "intelligent" feedback loop, which is using effluent nitrite concentrations, compensates the classical strategy only when needed. Simulation results show a clear improvement in average nitrite concentration level and level stability in the effluent, without a notable overcost in methanol.
△ Less
Submitted 28 November, 2017;
originally announced November 2017.
-
A Framework for the Numerical Computation and a Posteriori Verification of Invariant Objects of Evolution Equations
Authors:
Jordi-Lluís Figueras,
Marcio Gameiro,
Jean Philippe Lessard,
Rafael de la Llave
Abstract:
We develop a theoretical framework for computer-assisted proofs of the existence of invariant objects in semilinear PDEs. The invariant objects considered in this paper are equilibrium points, traveling waves, periodic orbits and invariant manifolds attached to fixed points or periodic orbits. The core of the study is writing down the invariance condition as a zero of an operator. These operators…
▽ More
We develop a theoretical framework for computer-assisted proofs of the existence of invariant objects in semilinear PDEs. The invariant objects considered in this paper are equilibrium points, traveling waves, periodic orbits and invariant manifolds attached to fixed points or periodic orbits. The core of the study is writing down the invariance condition as a zero of an operator. These operators are in general not continuous, so one needs to smooth them by means of preconditioners before classical fixed point theorems can be applied. We develop in detail all the aspects of how to work with these objects: how to precondition the equations, how to work with the nonlinear terms, which function spaces can be useful, and how to work with them in a computationally rigorous way. In two companion papers, we present two different implementations of the tools developed in this paper to study periodic orbits.
△ Less
Submitted 3 May, 2016;
originally announced May 2016.
-
Zipf's law holds for phrases, not words
Authors:
Jake Ryland Williams,
Paul R. Lessard,
Suma Desu,
Eric Clark,
James P. Bagrow,
Christopher M. Danforth,
Peter Sheridan Dodds
Abstract:
With Zipf's law being originally and most famously observed for word frequency, it is surprisingly limited in its applicability to human language, holding over no more than three to four orders of magnitude before hitting a clear break in scaling. Here, building on the simple observation that phrases of one or more words comprise the most coherent units of meaning in language, we show empirically…
▽ More
With Zipf's law being originally and most famously observed for word frequency, it is surprisingly limited in its applicability to human language, holding over no more than three to four orders of magnitude before hitting a clear break in scaling. Here, building on the simple observation that phrases of one or more words comprise the most coherent units of meaning in language, we show empirically that Zipf's law for phrases extends over as many as nine orders of rank magnitude. In doing so, we develop a principled and scalable statistical mechanical method of random text partitioning, which opens up a rich frontier of rigorous text analysis via a rank ordering of mixed length phrases.
△ Less
Submitted 4 March, 2015; v1 submitted 19 June, 2014;
originally announced June 2014.
