-
Subtractive Mixture Models via Squaring: Representation and Learning
Authors:
Lorenzo Loconte,
Aleksanteri M. Sladek,
Stefan Mengel,
Martin Trapp,
Arno Solin,
Nicolas Gillis,
Antonio Vergari
Abstract:
Mixture models are traditionally represented and learned by adding several distributions as components. Allowing mixtures to subtract probability mass or density can drastically reduce the number of components needed to model complex distributions. However, learning such subtractive mixtures while ensuring they still encode a non-negative function is challenging. We investigate how to learn and pe…
▽ More
Mixture models are traditionally represented and learned by adding several distributions as components. Allowing mixtures to subtract probability mass or density can drastically reduce the number of components needed to model complex distributions. However, learning such subtractive mixtures while ensuring they still encode a non-negative function is challenging. We investigate how to learn and perform inference on deep subtractive mixtures by squaring them. We do this in the framework of probabilistic circuits, which enable us to represent tensorized mixtures and generalize several other subtractive models. We theoretically prove that the class of squared circuits allowing subtractions can be exponentially more expressive than traditional additive mixtures; and, we empirically show this increased expressiveness on a series of real-world distribution estimation tasks.
△ Less
Submitted 26 April, 2024; v1 submitted 1 October, 2023;
originally announced October 2023.
-
How to Turn Your Knowledge Graph Embeddings into Generative Models
Authors:
Lorenzo Loconte,
Nicola Di Mauro,
Robert Peharz,
Antonio Vergari
Abstract:
Some of the most successful knowledge graph embedding (KGE) models for link prediction -- CP, RESCAL, TuckER, ComplEx -- can be interpreted as energy-based models. Under this perspective they are not amenable for exact maximum-likelihood estimation (MLE), sampling and struggle to integrate logical constraints. This work re-interprets the score functions of these KGEs as circuits -- constrained com…
▽ More
Some of the most successful knowledge graph embedding (KGE) models for link prediction -- CP, RESCAL, TuckER, ComplEx -- can be interpreted as energy-based models. Under this perspective they are not amenable for exact maximum-likelihood estimation (MLE), sampling and struggle to integrate logical constraints. This work re-interprets the score functions of these KGEs as circuits -- constrained computational graphs allowing efficient marginalisation. Then, we design two recipes to obtain efficient generative circuit models by either restricting their activations to be non-negative or squaring their outputs. Our interpretation comes with little or no loss of performance for link prediction, while the circuits framework unlocks exact learning by MLE, efficient sampling of new triples, and guarantee that logical constraints are satisfied by design. Furthermore, our models scale more gracefully than the original KGEs on graphs with millions of entities.
△ Less
Submitted 16 January, 2024; v1 submitted 25 May, 2023;
originally announced May 2023.
-
DeeProb-kit: a Python Library for Deep Probabilistic Modelling
Authors:
Lorenzo Loconte,
Gennaro Gala
Abstract:
DeeProb-kit is a unified library written in Python consisting of a collection of deep probabilistic models (DPMs) that are tractable and exact representations for the modelled probability distributions. The availability of a representative selection of DPMs in a single library makes it possible to combine them in a straightforward manner, a common practice in deep learning research nowadays. In ad…
▽ More
DeeProb-kit is a unified library written in Python consisting of a collection of deep probabilistic models (DPMs) that are tractable and exact representations for the modelled probability distributions. The availability of a representative selection of DPMs in a single library makes it possible to combine them in a straightforward manner, a common practice in deep learning research nowadays. In addition, it includes efficiently implemented learning techniques, inference routines, statistical algorithms, and provides high-quality fully-documented APIs. The development of DeeProb-kit will help the community to accelerate research on DPMs as well as to standardise their evaluation and better understand how they are related based on their expressivity.
△ Less
Submitted 8 December, 2022;
originally announced December 2022.
-
Charge ordering transition in GdBaCo2O5: evidence of reentrant behavior
Authors:
M. Allieta,
M. Scavini,
L. Lopresti,
M. Coduri,
L. Loconte,
S. Cappelli,
C. Oliva,
P. Ghigna,
P. Pattison,
V. Scagnoli
Abstract:
We present a detailed study on the charge ordering (CO) transition in GdBaCo2O5 system by combining high resolution synchrotron powder/single crystal diffraction with electron paramagnetic resonance (EPR) experiments as a function of temperature. We found a second order structural phase transition at TCO=247 K (Pmmm to Pmma) associated with the onset of long range CO. At Tmin = 1.2TCO, the EPR lin…
▽ More
We present a detailed study on the charge ordering (CO) transition in GdBaCo2O5 system by combining high resolution synchrotron powder/single crystal diffraction with electron paramagnetic resonance (EPR) experiments as a function of temperature. We found a second order structural phase transition at TCO=247 K (Pmmm to Pmma) associated with the onset of long range CO. At Tmin = 1.2TCO, the EPR linewidth rapidly broadens providing evidence of spin fluctuations due to magnetic interactions between Gd3+ ions and antiferromagnetic couplings of Co2+/Co3+ sublattices. This likely indicates that, analogously to manganites, the long-range antiferromagnetic order in GdBaCo2O5 sets in at TCO. Pair distribution function (PDF) analysis of diffraction data revealed signatures of structural inhomogeneities at low temperature. By comparing the average and local bond valences, we found that above TCO the local structure is consistent with a fully random occupation of Co2+ and Co3+ in a 1:1 ratio and with a complete charge ordering below TCO. Below T = 100 K the charge localization is partially melted at the local scale, suggesting a reentrant behavior of CO. This result is supported by the weakening of superstructure reflections and the temperature evolution of EPR linewidth that is consistent with paramagnetic (PM) reentrant behavior reported in the GdBaCo2O5.5 parent compound.
△ Less
Submitted 10 July, 2013;
originally announced July 2013.