Showing 1–5 of 5 results for author: Härkönen, M

EEND-M2F: Masked-attention mask transformers for speaker diarization

Authors: Marc Härkönen, Samuel J. Broughton, Lahiru Samarakoon

Abstract: In this paper, we make the explicit connection between image segmentation methods and end-to-end diarization methods. From these insights, we propose a novel, fully end-to-end diarization model, EEND-M2F, based on the Mask2Former architecture. Speaker representations are computed in parallel using a stack of transformer decoders, in which irrelevant frames are explicitly masked from the cross atte… ▽ More In this paper, we make the explicit connection between image segmentation methods and end-to-end diarization methods. From these insights, we propose a novel, fully end-to-end diarization model, EEND-M2F, based on the Mask2Former architecture. Speaker representations are computed in parallel using a stack of transformer decoders, in which irrelevant frames are explicitly masked from the cross attention using predictions from previous layers. EEND-M2F is lightweight, efficient, and truly end-to-end, as it does not require any additional diarization, speaker verification, or segmentation models to run, nor does it require running any clustering algorithms. Our model achieves state-of-the-art performance on several public datasets, such as AMI, AliMeeting and RAMC. Most notably our DER of 16.07% on DIHARD-III is the first major improvement upon the challenge winning system. △ Less

Submitted 23 January, 2024; originally announced January 2024.

Comments: 14 pages, 2 figures

Transformer Attractors for Robust and Efficient End-to-End Neural Diarization

Authors: Lahiru Samarakoon, Samuel J. Broughton, Marc Härkönen, Ivan Fung

Abstract: End-to-end neural diarization with encoder-decoder based attractors (EEND-EDA) is a method to perform diarization in a single neural network. EDA handles the diarization of a flexible number of speakers by using an LSTM-based encoder-decoder that generates a set of speaker-wise attractors in an autoregressive manner. In this paper, we propose to replace EDA with a transformer-based attractor calcu… ▽ More End-to-end neural diarization with encoder-decoder based attractors (EEND-EDA) is a method to perform diarization in a single neural network. EDA handles the diarization of a flexible number of speakers by using an LSTM-based encoder-decoder that generates a set of speaker-wise attractors in an autoregressive manner. In this paper, we propose to replace EDA with a transformer-based attractor calculation (TA) module. TA is composed of a Combiner block and a Transformer decoder. The main function of the combiner block is to generate conversational dependent (CD) embeddings by incorporating learned conversational information into a global set of embeddings. These CD embeddings will then serve as the input for the transformer decoder. Results on public datasets show that EEND-TA achieves 2.68% absolute DER improvement over EEND-EDA. EEND-TA inference is 1.28 times faster than that of EEND-EDA. △ Less

Submitted 11 December, 2023; originally announced December 2023.

Comments: 8 pages, 1 figure, ASRU2023

Gaussian Process Priors for Systems of Linear Partial Differential Equations with Constant Coefficients

Authors: Marc Härkönen, Markus Lange-Hegermann, Bogdan Raiţă

Abstract: Partial differential equations (PDEs) are important tools to model physical systems and including them into machine learning models is an important way of incorporating physical knowledge. Given any system of linear PDEs with constant coefficients, we propose a family of Gaussian process (GP) priors, which we call EPGP, such that all realizations are exact solutions of this system. We apply the Eh… ▽ More Partial differential equations (PDEs) are important tools to model physical systems and including them into machine learning models is an important way of incorporating physical knowledge. Given any system of linear PDEs with constant coefficients, we propose a family of Gaussian process (GP) priors, which we call EPGP, such that all realizations are exact solutions of this system. We apply the Ehrenpreis-Palamodov fundamental principle, which works as a non-linear Fourier transform, to construct GP kernels mirroring standard spectral methods for GPs. Our approach can infer probable solutions of linear PDE systems from any data such as noisy measurements, or pointwise defined initial and boundary conditions. Constructing EPGP-priors is algorithmic, generally applicable, and comes with a sparse version (S-EPGP) that learns the relevant spectral frequencies and works better for big data sets. We demonstrate our approach on three families of systems of PDEs, the heat equation, wave equation, and Maxwell's equations, where we improve upon the state of the art in computation time and precision, in some experiments by several orders of magnitude. △ Less

Submitted 2 November, 2023; v1 submitted 29 December, 2022; originally announced December 2022.

Comments: 26 pages, 8 figures; ICML 2023 (oral); updated with expanded appendices and ancillary files. Code available at https://github.com/haerski/EPGP. For animations, see https://mathrepo.mis.mpg.de/EPGP/index.html. For a presentation see https://icml.cc/virtual/2023/oral/25571. The paper and all ancillary files are released under CC-BY

MSC Class: 60G15; 13N10; 13P25; 60-08; 35G35

Journal ref: ICML 2023 (oral); PMLR 202:12587-12615, 2023

Linear PDE with Constant Coefficients

Authors: Rida Ait El Manssour, Marc Härkönen, Bernd Sturmfels

Abstract: We discuss practical methods for computing the space of solutions to an arbitrary homogeneous linear system of partial differential equations with constant coefficients. These rest on the Fundamental Principle of Ehrenpreis-Palamodov from the 1960s. We develop this further using recent advances in computational commutative algebra. We discuss practical methods for computing the space of solutions to an arbitrary homogeneous linear system of partial differential equations with constant coefficients. These rest on the Fundamental Principle of Ehrenpreis-Palamodov from the 1960s. We develop this further using recent advances in computational commutative algebra. △ Less

Submitted 12 October, 2021; v1 submitted 20 April, 2021; originally announced April 2021.

Comments: 31 pages, 1 figure

MSC Class: 13N10; 14-04; 14Q15; 35G35; 35C15

Noetherian operators and primary decomposition

Authors: Justin Chen, Marc Härkönen, Robert Krone, Anton Leykin

Abstract: Noetherian operators are differential operators that encode primary components of a polynomial ideal. We develop a framework, as well as algorithms, for computing Noetherian operators with local dual spaces, both symbolically and numerically. For a primary ideal, such operators provide an alternative representation to one given by a set of generators. This description fits well with numerical alge… ▽ More Noetherian operators are differential operators that encode primary components of a polynomial ideal. We develop a framework, as well as algorithms, for computing Noetherian operators with local dual spaces, both symbolically and numerically. For a primary ideal, such operators provide an alternative representation to one given by a set of generators. This description fits well with numerical algebraic geometry, taking a step toward the goal of numerical primary decomposition. △ Less

Submitted 24 June, 2020; originally announced June 2020.

Comments: 17 pages, codebase available at https://github.com/haerski/NoetherianOperators

MSC Class: 14Q15; 14-04; 13N05; 65L80; 65D05