Showing 1–18 of 18 results for author: Christiansen, H

Higher-Rank Irreducible Cartesian Tensors for Equivariant Message Passing

Authors: Viktor Zaverkin, Francesco Alesiani, Takashi Maruyama, Federico Errica, Henrik Christiansen, Makoto Takamoto, Nicolas Weber, Mathias Niepert

Abstract: The ability to perform fast and accurate atomistic simulations is crucial for advancing the chemical sciences. By learning from high-quality data, machine-learned interatomic potentials achieve accuracy on par with ab initio and first-principles methods at a fraction of their computational cost. The success of machine-learned interatomic potentials arises from integrating inductive biases such as… ▽ More The ability to perform fast and accurate atomistic simulations is crucial for advancing the chemical sciences. By learning from high-quality data, machine-learned interatomic potentials achieve accuracy on par with ab initio and first-principles methods at a fraction of their computational cost. The success of machine-learned interatomic potentials arises from integrating inductive biases such as equivariance to group actions on an atomic system, e.g., equivariance to rotations and reflections. In particular, the field has notably advanced with the emergence of equivariant message-passing architectures. Most of these models represent an atomic system using spherical tensors, tensor products of which require complicated numerical coefficients and can be computationally demanding. This work introduces higher-rank irreducible Cartesian tensors as an alternative to spherical tensors, addressing the above limitations. We integrate irreducible Cartesian tensor products into message-passing neural networks and prove the equivariance of the resulting layers. Through empirical evaluations on various benchmark data sets, we consistently observe on-par or better performance than that of state-of-the-art spherical models. △ Less

Submitted 23 May, 2024; originally announced May 2024.

Adaptive Message Passing: A General Framework to Mitigate Oversmoothing, Oversquashing, and Underreaching

Authors: Federico Errica, Henrik Christiansen, Viktor Zaverkin, Takashi Maruyama, Mathias Niepert, Francesco Alesiani

Abstract: Long-range interactions are essential for the correct description of complex systems in many scientific fields. The price to pay for including them in the calculations, however, is a dramatic increase in the overall computational costs. Recently, deep graph networks have been employed as efficient, data-driven surrogate models for predicting properties of complex systems represented as graphs. The… ▽ More Long-range interactions are essential for the correct description of complex systems in many scientific fields. The price to pay for including them in the calculations, however, is a dramatic increase in the overall computational costs. Recently, deep graph networks have been employed as efficient, data-driven surrogate models for predicting properties of complex systems represented as graphs. These models rely on a local and iterative message passing strategy that should, in principle, capture long-range information without explicitly modeling the corresponding interactions. In practice, most deep graph networks cannot really model long-range dependencies due to the intrinsic limitations of (synchronous) message passing, namely oversmoothing, oversquashing, and underreaching. This work proposes a general framework that learns to mitigate these limitations: within a variational inference framework, we endow message passing architectures with the ability to freely adapt their depth and filter messages along the way. With theoretical and empirical arguments, we show that this simple strategy better captures long-range interactions, by surpassing the state of the art on five node and graph prediction datasets suited for this problem. Our approach consistently improves the performances of the baselines tested on these tasks. We complement the exposition with qualitative analyses and ablations to get a deeper understanding of the framework's inner workings. △ Less

Submitted 20 March, 2024; v1 submitted 27 December, 2023; originally announced December 2023.

Self-Tuning Hamiltonian Monte Carlo for Accelerated Sampling

Authors: Henrik Christiansen, Federico Errica, Francesco Alesiani

Abstract: The performance of Hamiltonian Monte Carlo simulations crucially depends on both the integration timestep and the number of integration steps. We present an adaptive general-purpose framework to automatically tune such parameters, based on a local loss function which promotes the fast exploration of phase-space. We show that a good correspondence between loss and autocorrelation time can be establ… ▽ More The performance of Hamiltonian Monte Carlo simulations crucially depends on both the integration timestep and the number of integration steps. We present an adaptive general-purpose framework to automatically tune such parameters, based on a local loss function which promotes the fast exploration of phase-space. We show that a good correspondence between loss and autocorrelation time can be established, allowing for gradient-based optimization using a fully-differentiable set-up. The loss is constructed in such a way that it also allows for gradient-driven learning of a distribution over the number of integration steps. Our approach is demonstrated for the one-dimensional harmonic oscillator and alanine dipeptide, a small protein common as a test case for simulation methods. Through the application to the harmonic oscillator, we highlight the importance of not using a fixed timestep to avoid a rugged loss surface with many local minima, otherwise trap** the optimization. In the case of alanine dipeptide, by tuning the only free parameter of our loss definition, we find a good correspondence between it and the autocorrelation times, resulting in a $>100$ fold speed up in optimization of simulation parameters compared to a grid-search. For this system, we also extend the integrator to allow for atom-dependent timesteps, providing a further reduction of $25\%$ in autocorrelation times. △ Less

Submitted 26 November, 2023; v1 submitted 24 September, 2023; originally announced September 2023.

Deep Learning-based Signal Strength Prediction Using Geographical Images and Expert Knowledge

Authors: Jakob Thrane, Benjamin Sliwa, Christian Wietfeld, Henrik Christiansen

Abstract: Methods for accurate prediction of radio signal quality parameters are crucial for optimization of mobile networks, and a necessity for future autonomous driving solutions. The power-distance relation of current empirical models struggles with describing the specific local geo-statistics that influence signal quality parameters. The use of empirical models commonly results in an over- or under-est… ▽ More Methods for accurate prediction of radio signal quality parameters are crucial for optimization of mobile networks, and a necessity for future autonomous driving solutions. The power-distance relation of current empirical models struggles with describing the specific local geo-statistics that influence signal quality parameters. The use of empirical models commonly results in an over- or under-estimation of the signal quality parameters and require additional calibration studies. In this paper, we present a novel model-aided deep learning approach for path loss prediction, which implicitly extracts radio propagation characteristics from top-view geographical images of the receiver location. In a comprehensive evaluation campaign, we apply the proposed method on an extensive real-world data set consisting of five different scenarios and more than 125.000 individual measurements. It is found that 1) the novel approach reduces the average prediction error by up to 53% in comparison to ray-tracing techniques, 2) A distance of 250-300 meters spanned by the images offer the necessary level of detail, 3) Predictions with a root-mean-squared error of approximately 6 dB is achieved across inherently different data sources. △ Less

Submitted 18 August, 2020; originally announced August 2020.

Experimental Evaluation of Empirical NB-IoT Propagation Modelling in a Deep-Indoor Scenario

Authors: Jakob Thrane, Krzysztof Mateusz Malarski, Henrik Lehrmann Christiansen, Sarah Ruepp

Abstract: Path-loss modelling in deep-indoor scenarios is a difficult task. On one hand, the theoretical formulae solely dependent on transmitter-receiver distance are too simple; on the other hand, discovering all significant factors affecting the loss of signal power in a given situation may often be infeasible. In this paper, we experimentally investigate the influence of deep-indoor features such as ind… ▽ More Path-loss modelling in deep-indoor scenarios is a difficult task. On one hand, the theoretical formulae solely dependent on transmitter-receiver distance are too simple; on the other hand, discovering all significant factors affecting the loss of signal power in a given situation may often be infeasible. In this paper, we experimentally investigate the influence of deep-indoor features such as indoor depth, indoor distance and distance to the closest tunnel corridor and the effect on received power using NB-IoT. We describe a measurement campaign performed in a system of long underground tunnels, and we analyse linear regression models involving the engineered features. We show that the current empirical models for NB-IoT signal attenuation are inaccurate in a deep-indoor scenario. We observe that 1) indoor distance and penetration depth do not explain the signal attenuation well and increase the error of the prediction by 2-12 dB using existing models, and 2) a promising feature of average distance to the nearest corridor is identified. △ Less

Submitted 1 June, 2020; originally announced June 2020.

Comments: 6 pages, 6 figures, submitted to Globecom2020 conference, Selected Areas in Communications Symposium, Track on Internet of Things and Smart Connected Communities

The Danish Gigaword Project

Authors: Leon Strømberg-Derczynski, Manuel R. Ciosici, Rebekah Baglini, Morten H. Christiansen, Jacob Aarup Dalsgaard, Riccardo Fusaroli, Peter Juel Henrichsen, Rasmus Hvingelby, Andreas Kirkedal, Alex Speed Kjeldsen, Claus Ladefoged, Finn Årup Nielsen, Malte Lau Petersen, Jonathan Hvithamar Rystrøm, Daniel Varab

Abstract: Danish language technology has been hindered by a lack of broad-coverage corpora at the scale modern NLP prefers. This paper describes the Danish Gigaword Corpus, the result of a focused effort to provide a diverse and freely-available one billion word corpus of Danish text. The Danish Gigaword corpus covers a wide array of time periods, domains, speakers' socio-economic status, and Danish dialect… ▽ More Danish language technology has been hindered by a lack of broad-coverage corpora at the scale modern NLP prefers. This paper describes the Danish Gigaword Corpus, the result of a focused effort to provide a diverse and freely-available one billion word corpus of Danish text. The Danish Gigaword corpus covers a wide array of time periods, domains, speakers' socio-economic status, and Danish dialects. △ Less

Submitted 12 May, 2021; v1 submitted 7 May, 2020; originally announced May 2020.

Comments: Identical to the NoDaLiDa 2021 version

Memory limitations are hidden in grammar

Authors: Carlos Gómez-Rodríguez, Morten H. Christiansen, Ramon Ferrer-i-Cancho

Abstract: The ability to produce and understand an unlimited number of different sentences is a hallmark of human language. Linguists have sought to define the essence of this generative capacity using formal grammars that describe the syntactic dependencies between constituents, independent of the computational limitations of the human brain. Here, we evaluate this independence assumption by sampling sente… ▽ More The ability to produce and understand an unlimited number of different sentences is a hallmark of human language. Linguists have sought to define the essence of this generative capacity using formal grammars that describe the syntactic dependencies between constituents, independent of the computational limitations of the human brain. Here, we evaluate this independence assumption by sampling sentences uniformly from the space of possible syntactic structures. We find that the average dependency distance between syntactically related words, a proxy for memory limitations, is less than expected by chance in a collection of state-of-the-art classes of dependency grammars. Our findings indicate that memory limitations have permeated grammatical descriptions, suggesting that it may be impossible to build a parsimonious theory of human linguistic productivity independent of non-linguistic cognitive constraints. △ Less

Submitted 5 April, 2022; v1 submitted 19 August, 2019; originally announced August 2019.

Comments: Improved with reviewer feedback once again. In press in Glottometrics

Journal ref: Glottometrics (2022) 52, 39-64

UnsuperPoint: End-to-end Unsupervised Interest Point Detector and Descriptor

Authors: Peter Hviid Christiansen, Mikkel Fly Kragh, Yury Brodskiy, Henrik Karstoft

Abstract: It is hard to create consistent ground truth data for interest points in natural images, since interest points are hard to define clearly and consistently for a human annotator. This makes interest point detectors non-trivial to build. In this work, we introduce an unsupervised deep learning-based interest point detector and descriptor. Using a self-supervised approach, we utilize a siamese networ… ▽ More It is hard to create consistent ground truth data for interest points in natural images, since interest points are hard to define clearly and consistently for a human annotator. This makes interest point detectors non-trivial to build. In this work, we introduce an unsupervised deep learning-based interest point detector and descriptor. Using a self-supervised approach, we utilize a siamese network and a novel loss function that enables interest point scores and positions to be learned automatically. The resulting interest point detector and descriptor is UnsuperPoint. We use regression of point positions to 1) make UnsuperPoint end-to-end trainable and 2) to incorporate non-maximum suppression in the model. Unlike most trainable detectors, it requires no generation of pseudo ground truth points, no structure-from-motion-generated representations and the model is learned from only one round of training. Furthermore, we introduce a novel loss function to regularize network predictions to be uniformly distributed. UnsuperPoint runs in real-time with 323 frames per second (fps) at a resolution of $224\times320$ and 90 fps at $480\times640$. It is comparable or better than state-of-the-art performance when measured for speed, repeatability, localization, matching score and homography estimation on the HPatch dataset. △ Less

Submitted 9 July, 2019; originally announced July 2019.

Towards a constraint solver for proving confluence with invariant and equivalence of realistic CHR programs

Authors: Henning Christiansen, Maja Kirkeby

Abstract: Confluence of a nondeterministic program ensures a functional input-output relation, freeing the programmer from considering the actual scheduling strategy, and allowing optimized and perhaps parallel implementations. The more general property of confluence modulo equivalence ensures that equivalent inputs are related to equivalent outputs, that need not be identical. Confluence under invariants i… ▽ More Confluence of a nondeterministic program ensures a functional input-output relation, freeing the programmer from considering the actual scheduling strategy, and allowing optimized and perhaps parallel implementations. The more general property of confluence modulo equivalence ensures that equivalent inputs are related to equivalent outputs, that need not be identical. Confluence under invariants is also considered. Constraint Handling Rules (CHR) is an important example of a rewrite based logic programming language, and we aim at a mechanizable method for proving confluence modulo equivalence of terminating programs. While earlier approaches to confluence for CHR programs concern an idealized logic subset, we refer to a semantics compatible with standard Prolog-based implementations. We specify a meta-level constraint language in which invariants and equivalences can be expressed and manipulated, extending our previous theoretical results towards a practical implementation. △ Less

Submitted 13 September, 2018; v1 submitted 24 August, 2018; originally announced August 2018.

Comments: 17 pages, Accepted for presentation in WFLP 2018

Confluence of CHR revisited: invariants and modulo equivalence

Authors: Henning Christiansen, Maja H. Kirkeby

Abstract: Abstract simulation of one transition system by another is introduced as a means to simulate a potentially infinite class of similar transition sequences within a single transition sequence. This is useful for proving confluence under invariants of a given system, as it may reduce the number of proof cases to consider from infinity to a finite number. The classical confluence results for Constrain… ▽ More Abstract simulation of one transition system by another is introduced as a means to simulate a potentially infinite class of similar transition sequences within a single transition sequence. This is useful for proving confluence under invariants of a given system, as it may reduce the number of proof cases to consider from infinity to a finite number. The classical confluence results for Constraint Handling Rules (CHR) can be explained in this way, using CHR as a simulation of itself. Using an abstract simulation based on a ground representation, we extend these results to include confluence under invariant and modulo equivalence, which have not been done in a satisfactory way before. △ Less

Submitted 2 October, 2018; v1 submitted 26 May, 2018; originally announced May 2018.

Comments: Pre-proceedings paper presented at the 28th International Symposium on Logic-Based Program Synthesis and Transformation (LOPSTR 2018), Frankfurt am Main, Germany, 4-6 September 2018 (arXiv:1808.03326)

Report number: LOPSTR/2018/14

A simple script language for choreography of multiple, synchronizing non-anthropomorphic robots

Authors: Henning Christiansen

Abstract: The scripting language described in this document is (in the first place) intended to be used on robots developed by Anja Mølle Lindelof and Henning Christiansen as part of a research project about robots performing on stage. The target robots are expected to appear as familiar domestic objects that take their own life, so to speak, and perhaps perform together with human players, creating at il… ▽ More The scripting language described in this document is (in the first place) intended to be used on robots developed by Anja Mølle Lindelof and Henning Christiansen as part of a research project about robots performing on stage. The target robots are expected to appear as familiar domestic objects that take their own life, so to speak, and perhaps perform together with human players, creating at illusion of a communication between them. In the current version, these robots' common behaviour is determined uniquely by a script written in the language described here -- the only possible autonomy for the robots is action to correct dynamically for inaccuracies that arise during a performance. The present work is preliminary and has not been compared to properly to other research work in this area, and the testing is still limited. A first implementation on small Lego Mindstorms based robots is under development by Mads Saustrup Fox as part of his master thesis work. △ Less

Submitted 22 December, 2017; originally announced December 2017.

Comments: work in progress

Confluence and Convergence in Probabilistically Terminating Reduction Systems

Authors: Maja H. Kirkeby, Henning Christiansen

Abstract: Convergence of an abstract reduction system (ARS) is the property that any derivation from an initial state will end in the same final state, a.k.a. normal form. We generalize this for probabilistic ARS as almost-sure convergence, meaning that the normal form is reached with probability one, even if diverging derivations may exist. We show and exemplify properties that can be used for proving almo… ▽ More Convergence of an abstract reduction system (ARS) is the property that any derivation from an initial state will end in the same final state, a.k.a. normal form. We generalize this for probabilistic ARS as almost-sure convergence, meaning that the normal form is reached with probability one, even if diverging derivations may exist. We show and exemplify properties that can be used for proving almost-sure convergence of probabilistic ARS, generalizing known results from ARS. △ Less

Submitted 3 October, 2017; v1 submitted 15 September, 2017; originally announced September 2017.

Comments: Pre-proceedings paper presented at the 27th International Symposium on Logic-Based Program Synthesis and Transformation (LOPSTR 2017), Namur, Belgium, 10-12 October 2017 (arXiv:1708.07854)

Report number: LOPSTR/2017/13

On Proving Confluence Modulo Equivalence for Constraint Handling Rules

Authors: Henning Christiansen, Maja H. Kirkeby

Abstract: Previous results on proving confluence for Constraint Handling Rules are extended in two ways in order to allow a larger and more realistic class of CHR programs to be considered confluent. Firstly, we introduce the relaxed notion of confluence modulo equivalence into the context of CHR: while confluence for a terminating program means that all alternative derivations for a query lead to the exact… ▽ More Previous results on proving confluence for Constraint Handling Rules are extended in two ways in order to allow a larger and more realistic class of CHR programs to be considered confluent. Firstly, we introduce the relaxed notion of confluence modulo equivalence into the context of CHR: while confluence for a terminating program means that all alternative derivations for a query lead to the exact same final state, confluence modulo equivalence only requires the final states to be equivalent with respect to an equivalence relation tailored for the given program. Secondly, we allow non-logical built-in predicates such as var/1 and incomplete ones such as is/2, that are ignored in previous work on confluence. To this end, a new operational semantics for CHR is developed which includes such predicates. In addition, this semantics differs from earlier approaches by its simplicity without loss of generality, and it may also be recommended for future studies of CHR. For the purely logical subset of CHR, proofs can be expressed in first-order logic, that we show is not sufficient in the present case. We have introduced a formal meta-language that allows reasoning about abstract states and derivations with meta-level restrictions that reflect the non-logical and incomplete predicates. This language represents subproofs as diagrams, which facilitates a systematic enumeration of proof cases, pointing forward to a mechanical support for such proofs. △ Less

Submitted 11 November, 2016; originally announced November 2016.

Networks in Cognitive Science

Authors: Andrea Baronchelli, Ramon Ferrer-i-Cancho, Romualdo Pastor-Satorras, Nick Chater, Morten H. Christiansen

Abstract: Networks of interconnected nodes have long played a key role in Cognitive Science, from artificial neural net- works to spreading activation models of semantic mem- ory. Recently, however, a new Network Science has been developed, providing insights into the emergence of global, system-scale properties in contexts as diverse as the Internet, metabolic reactions, and collaborations among scientists… ▽ More Networks of interconnected nodes have long played a key role in Cognitive Science, from artificial neural net- works to spreading activation models of semantic mem- ory. Recently, however, a new Network Science has been developed, providing insights into the emergence of global, system-scale properties in contexts as diverse as the Internet, metabolic reactions, and collaborations among scientists. Today, the inclusion of network theory into Cognitive Sciences, and the expansion of complex- systems science, promises to significantly change the way in which the organization and dynamics of cognitive and behavioral processes are understood. In this paper, we review recent contributions of network theory at different levels and domains within the Cognitive Sciences. △ Less

Submitted 5 July, 2013; v1 submitted 24 April, 2013; originally announced April 2013.

Journal ref: Trends in Cognitive Sciences 17, 348-360 (2013)

The Biological Origin of Linguistic Diversity

Authors: Andrea Baronchelli, Nick Chater, Romualdo Pastor-Satorras, Morten H. Christiansen

Abstract: In contrast with animal communication systems, diversity is characteristic of almost every aspect of human language. Languages variously employ tones, clicks, or manual signs to signal differences in meaning; some languages lack the noun-verb distinction (e.g., Straits Salish), whereas others have a proliferation of fine-grained syntactic categories (e.g., Tzeltal); and some languages do without m… ▽ More In contrast with animal communication systems, diversity is characteristic of almost every aspect of human language. Languages variously employ tones, clicks, or manual signs to signal differences in meaning; some languages lack the noun-verb distinction (e.g., Straits Salish), whereas others have a proliferation of fine-grained syntactic categories (e.g., Tzeltal); and some languages do without morphology (e.g., Mandarin), while others pack a whole sentence into a single word (e.g., Cayuga). A challenge for evolutionary biology is to reconcile the diversity of languages with the high degree of biological uniformity of their speakers. Here, we model processes of language change and geographical dispersion and find a consistent pressure for flexible learning, irrespective of the language being spoken. This pressure arises because flexible learners can best cope with the observed high rates of linguistic change associated with divergent cultural evolution following human migration. Thus, rather than genetic adaptations for specific aspects of language, such as recursion, the coevolution of genes and fast-changing linguistic structure provides the biological basis for linguistic diversity. Only biological adaptations for flexible learning combined with cultural evolution can explain how each child has the potential to learn any human language. △ Less

Submitted 12 February, 2013; originally announced February 2013.

Journal ref: PLoS ONE 7(10): e48029 (2012)

Inference with Constrained Hidden Markov Models in PRISM

Authors: Henning Christiansen, Christian Theil Have, Ole Torp Lassen, Matthieu Petit

Abstract: A Hidden Markov Model (HMM) is a common statistical model which is widely used for analysis of biological sequence data and other sequential phenomena. In the present paper we show how HMMs can be extended with side-constraints and present constraint solving techniques for efficient inference. Defining HMMs with side-constraints in Constraint Logic Programming have advantages in terms of more comp… ▽ More A Hidden Markov Model (HMM) is a common statistical model which is widely used for analysis of biological sequence data and other sequential phenomena. In the present paper we show how HMMs can be extended with side-constraints and present constraint solving techniques for efficient inference. Defining HMMs with side-constraints in Constraint Logic Programming have advantages in terms of more compact expression and pruning opportunities during inference. We present a PRISM-based framework for extending HMMs with side-constraints and show how well-known constraints such as cardinality and all different are integrated. We experimentally validate our approach on the biologically motivated problem of global pairwise alignment. △ Less

Submitted 30 July, 2010; originally announced July 2010.

Journal ref: TPLP 2010, 10 (4-6) 449-464

CHR Grammars

Authors: Henning Christiansen

Abstract: A grammar formalism based upon CHR is proposed analogously to the way Definite Clause Grammars are defined and implemented on top of Prolog. These grammars execute as robust bottom-up parsers with an inherent treatment of ambiguity and a high flexibility to model various linguistic phenomena. The formalism extends previous logic programming based grammars with a form of context-sensitive rules a… ▽ More A grammar formalism based upon CHR is proposed analogously to the way Definite Clause Grammars are defined and implemented on top of Prolog. These grammars execute as robust bottom-up parsers with an inherent treatment of ambiguity and a high flexibility to model various linguistic phenomena. The formalism extends previous logic programming based grammars with a form of context-sensitive rules and the possibility to include extra-grammatical hypotheses in both head and body of grammar rules. Among the applications are straightforward implementations of Assumption Grammars and abduction under integrity constraints for language analysis. CHR grammars appear as a powerful tool for specification and implementation of language processors and may be proposed as a new standard for bottom-up grammars in logic programming. To appear in Theory and Practice of Logic Programming (TPLP), 2005 △ Less

Submitted 12 August, 2004; originally announced August 2004.

Comments: 36 pp. To appear in TPLP, 2005

CHR as grammar formalism. A first report

Authors: Henning Christiansen

Abstract: Grammars written as Constraint Handling Rules (CHR) can be executed as efficient and robust bottom-up parsers that provide a straightforward, non-backtracking treatment of ambiguity. Abduction with integrity constraints as well as other dynamic hypothesis generation techniques fit naturally into such grammars and are exemplified for anaphora resolution, coordination and text interpretation. Grammars written as Constraint Handling Rules (CHR) can be executed as efficient and robust bottom-up parsers that provide a straightforward, non-backtracking treatment of ambiguity. Abduction with integrity constraints as well as other dynamic hypothesis generation techniques fit naturally into such grammars and are exemplified for anaphora resolution, coordination and text interpretation. △ Less

Submitted 29 June, 2001; originally announced June 2001.

Comments: 12 pages. Presented at ERCIM Workshop on Constraints, Prague, Czech Republic, June 18-20, 2001

ACM Class: I.2.7; D.3.2; F.4.1; F.4.2

Journal ref: Proc. of ERCIM Workshop on Constraints, Prague, Czech Republic, June 18-20, 2001