-
PARMESAN: Parameter-Free Memory Search and Transduction for Dense Prediction Tasks
Authors:
Philip Matthias Winter,
Maria Wimmer,
David Major,
Dimitrios Lenis,
Astrid Berg,
Theresa Neubauer,
Gaia Romana De Paolis,
Johannes Novotny,
Sophia Ulonska,
Katja Bühler
Abstract:
In this work we address flexibility in deep learning by means of transductive reasoning. For adaptation to new tasks or new data, existing methods typically involve tuning of learnable parameters or even complete re-training from scratch, rendering such approaches unflexible in practice. We argue that the notion of separating computation from memory by the means of transduction can act as a steppi…
▽ More
In this work we address flexibility in deep learning by means of transductive reasoning. For adaptation to new tasks or new data, existing methods typically involve tuning of learnable parameters or even complete re-training from scratch, rendering such approaches unflexible in practice. We argue that the notion of separating computation from memory by the means of transduction can act as a step** stone for solving these issues. We therefore propose PARMESAN (parameter-free memory search and transduction), a scalable transduction method which leverages a memory module for solving dense prediction tasks. At inference, hidden representations in memory are being searched to find corresponding examples. In contrast to other methods, PARMESAN learns without the requirement for any continuous training or fine-tuning of learnable parameters simply by modifying the memory content. Our method is compatible with commonly used neural architectures and canonically transfers to 1D, 2D, and 3D grid-based data. We demonstrate the capabilities of our approach at complex tasks such as continual and few-shot learning. PARMESAN learns up to 370 times faster than common baselines while being on par in terms of predictive performance, knowledge retention, and data-efficiency.
△ Less
Submitted 18 March, 2024;
originally announced March 2024.
-
Chimera states in hierarchical networks of Van der Pol oscillators
Authors:
Stefan Ulonska,
Iryna Omelchenko,
Anna Zakharova,
Eckehard Schoell
Abstract:
Chimera states are complex spatio-temporal patterns that consist of coexisting domains of coherent and incoherent dynamics. We analyse chimera states in networks of Van der Pol oscillators with hierarchical coupling topology. We investigate the stepwise transition from a nonlocal to a hierarchical topology, and propose the network clustering coefficient as a measure to establish a link between the…
▽ More
Chimera states are complex spatio-temporal patterns that consist of coexisting domains of coherent and incoherent dynamics. We analyse chimera states in networks of Van der Pol oscillators with hierarchical coupling topology. We investigate the stepwise transition from a nonlocal to a hierarchical topology, and propose the network clustering coefficient as a measure to establish a link between the existence of chimera states and the compactness of the initial base pattern of a hierarchical topology; we show that a large clustering coefficient promotes the occurrence of chimeras. Depending on the level of hierarchy and base pattern, we obtain chimera states with different numbers of incoherent domains. We investigate the chimera regimes as a function of coupling strength and nonlinearity parameter of the individual oscillators. The analysis of a network with larger base pattern resulting in larger clustering coefficient reveals two different types of chimera states and highlights the increasing role of amplitude dynamics.
△ Less
Submitted 1 March, 2016;
originally announced March 2016.
-
Bright high-repetition-rate source of narrowband extreme-ultraviolet harmonics beyond 22 eV
Authors:
He Wang,
Yiming Xu,
Stefan Ulonska,
Joseph S. Robinson,
Predrag Ranitovic,
Robert A. Kaindl
Abstract:
Novel table-top sources of extreme-ultraviolet light based on high-harmonic generation yield unique insight into the fundamental properties of molecules, nanomaterials, or correlated solids, and enable advanced applications in imaging or metrology. Extending high-harmonic generation to high repetition rates portends great experimental benefits, yet efficient extreme-ultraviolet conversion of corre…
▽ More
Novel table-top sources of extreme-ultraviolet light based on high-harmonic generation yield unique insight into the fundamental properties of molecules, nanomaterials, or correlated solids, and enable advanced applications in imaging or metrology. Extending high-harmonic generation to high repetition rates portends great experimental benefits, yet efficient extreme-ultraviolet conversion of correspondingly weak driving pulses is challenging. Here, we demonstrate a highly efficient source of femtosecond extreme-ultraviolet pulses at 50-kHz repetition rate, utilizing the ultraviolet second-harmonic focused tightly into Kr gas. In this cascaded scheme, a photon flux beyond ~3e13 per second is generated at 22.3 eV, with 5e-5 conversion efficiency that surpasses similar harmonics directly driven by the fundamental by two orders of magnitude. The enhancement arises from both wavelength scaling of the atomic dipole and improved spatio-temporal phase-matching, confirmed by simulations. Spectral isolation of a single 72-meV wide harmonic renders this bright, 50-kHz extreme-ultraviolet source a powerful tool for ultrafast photoemission, nanoscale imaging and other applications.
△ Less
Submitted 28 December, 2017; v1 submitted 28 August, 2014;
originally announced August 2014.
