-
Generating Music with Structure Using Self-Similarity as Attention
Authors:
Sophia Hager,
Kathleen Hablutzel,
Katherine M. Kinnaird
Abstract:
Despite the innovations in deep learning and generative AI, creating long term structure as well as the layers of repeated structure common in musical works remains an open challenge in music generation. We propose an attention layer that uses a novel approach applying user-supplied self-similarity matrices to previous time steps, and demonstrate it in our Similarity Incentivized Neural Generator…
▽ More
Despite the innovations in deep learning and generative AI, creating long term structure as well as the layers of repeated structure common in musical works remains an open challenge in music generation. We propose an attention layer that uses a novel approach applying user-supplied self-similarity matrices to previous time steps, and demonstrate it in our Similarity Incentivized Neural Generator (SING) system, a deep learning autonomous music generation system with two layers. The first is a vanilla Long Short Term Memory layer, and the second is the proposed attention layer. During generation, this attention mechanism imposes a suggested structure from a template piece on the generated music. We train SING on the MAESTRO dataset using a novel variable batching method, and compare its performance to the same model without the attention mechanism. The addition of our proposed attention mechanism significantly improves the network's ability to replicate specific structures, and it performs better on an unseen test set than a model without the attention mechanism.
△ Less
Submitted 25 June, 2024; v1 submitted 21 June, 2024;
originally announced June 2024.
-
Learning to Generate Text in Arbitrary Writing Styles
Authors:
Aleem Khan,
Andrew Wang,
Sophia Hager,
Nicholas Andrews
Abstract:
Prior work in style-controlled text generation has focused on tasks such as emulating the style of prolific literary authors, producing formal or informal text, and mitigating toxicity of generated text. Plentiful demonstrations of these styles are available, and as a result modern language models are often able to emulate them, either via prompting or discriminative control. However, in applicati…
▽ More
Prior work in style-controlled text generation has focused on tasks such as emulating the style of prolific literary authors, producing formal or informal text, and mitigating toxicity of generated text. Plentiful demonstrations of these styles are available, and as a result modern language models are often able to emulate them, either via prompting or discriminative control. However, in applications such as writing assistants, it is desirable for language models to produce text in an author-specific style on the basis of a potentially small writing sample. For example, someone writing in a particular dialect may prefer writing suggestions that retain the same dialect. We find that instruction-tuned language models can struggle to reproduce author-specific style demonstrated in a prompt. Instead, we propose to guide a language model to generate text in a target style using contrastively-trained representations that capture stylometric features. Our approach (StyleMC) combines an author-adapted language model with sequence-level inference to improve stylistic consistency, and is found to be effective in a variety of conditions, including unconditional generation and style transfer. Additionally, we find that the proposed approach can serve as an effective anonymization method, by editing a document to mask authorship while preserving the original meaning
△ Less
Submitted 4 March, 2024; v1 submitted 28 December, 2023;
originally announced December 2023.
-
Phase-Based Breathing Rate Monitoring in Patient Rooms Using 6G Terahertz Technology
Authors:
Simon Haeger,
Akram Najjar,
Caner Bektas,
Dien Lessy,
Mohammed El-Absi,
Fawad Sheikh,
Stefan Boecker,
Thomas Kaiser,
Christian Wietfeld
Abstract:
The 6G standard aims to be an integral part of the future economy by providing high-performance communication and sensing services. At terahertz (THz) frequencies, indoor campus networks can offer the highest sensing quality. Health monitoring in hospitals is expected to be an application site for these. This work outlines a monostatic phase-based system for breathing rate monitoring. Our feasibil…
▽ More
The 6G standard aims to be an integral part of the future economy by providing high-performance communication and sensing services. At terahertz (THz) frequencies, indoor campus networks can offer the highest sensing quality. Health monitoring in hospitals is expected to be an application site for these. This work outlines a monostatic phase-based system for breathing rate monitoring. Our feasibility study observes motion measurement accuracy down to the micrometer level. However, we also find that the patient's pose needs to be considered for generalized applicability. Thus, a solution that leverages multiple propagation paths and beam orientations is proposed.
△ Less
Submitted 14 June, 2023; v1 submitted 25 May, 2023;
originally announced May 2023.
-
Learn2Reg: comprehensive multi-task medical image registration challenge, dataset and evaluation in the era of deep learning
Authors:
Alessa Hering,
Lasse Hansen,
Tony C. W. Mok,
Albert C. S. Chung,
Hanna Siebert,
Stephanie Häger,
Annkristin Lange,
Sven Kuckertz,
Stefan Heldmann,
Wei Shao,
Sulaiman Vesal,
Mirabela Rusu,
Geoffrey Sonn,
Théo Estienne,
Maria Vakalopoulou,
Luyi Han,
Yunzhi Huang,
Pew-Thian Yap,
Mikael Brudfors,
Yaël Balbastre,
Samuel Joutard,
Marc Modat,
Gal Lifshitz,
Dan Raviv,
**xin Lv
, et al. (28 additional authors not shown)
Abstract:
Image registration is a fundamental medical image analysis task, and a wide variety of approaches have been proposed. However, only a few studies have comprehensively compared medical image registration approaches on a wide range of clinically relevant tasks. This limits the development of registration methods, the adoption of research advances into practice, and a fair benchmark across competing…
▽ More
Image registration is a fundamental medical image analysis task, and a wide variety of approaches have been proposed. However, only a few studies have comprehensively compared medical image registration approaches on a wide range of clinically relevant tasks. This limits the development of registration methods, the adoption of research advances into practice, and a fair benchmark across competing approaches. The Learn2Reg challenge addresses these limitations by providing a multi-task medical image registration data set for comprehensive characterisation of deformable registration algorithms. A continuous evaluation will be possible at https://learn2reg.grand-challenge.org. Learn2Reg covers a wide range of anatomies (brain, abdomen, and thorax), modalities (ultrasound, CT, MR), availability of annotations, as well as intra- and inter-patient registration evaluation. We established an easily accessible framework for training and validation of 3D registration methods, which enabled the compilation of results of over 65 individual method submissions from more than 20 unique teams. We used a complementary set of metrics, including robustness, accuracy, plausibility, and runtime, enabling unique insight into the current state-of-the-art of medical image registration. This paper describes datasets, tasks, evaluation methods and results of the challenge, as well as results of further analysis of transferability to new datasets, the importance of label supervision, and resulting bias. While no single approach worked best across all tasks, many methodological aspects could be identified that push the performance of medical image registration to new state-of-the-art performance. Furthermore, we demystified the common belief that conventional registration methods have to be much slower than deep-learning-based methods.
△ Less
Submitted 7 October, 2022; v1 submitted 8 December, 2021;
originally announced December 2021.
-
CNN-based Lung CT Registration with Multiple Anatomical Constraints
Authors:
Alessa Hering,
Stephanie Häger,
Jan Moltz,
Nikolas Lessmann,
Stefan Heldmann,
Bram van Ginneken
Abstract:
Deep-learning-based registration methods emerged as a fast alternative to conventional registration methods. However, these methods often still cannot achieve the same performance as conventional registration methods because they are either limited to small deformation or they fail to handle a superposition of large and small deformations without producing implausible deformation fields with foldi…
▽ More
Deep-learning-based registration methods emerged as a fast alternative to conventional registration methods. However, these methods often still cannot achieve the same performance as conventional registration methods because they are either limited to small deformation or they fail to handle a superposition of large and small deformations without producing implausible deformation fields with foldings inside.
In this paper, we identify important strategies of conventional registration methods for lung registration and successfully developed the deep-learning counterpart. We employ a Gaussian-pyramid-based multilevel framework that can solve the image registration optimization in a coarse-to-fine fashion. Furthermore, we prevent foldings of the deformation field and restrict the determinant of the Jacobian to physiologically meaningful values by combining a volume change penalty with a curvature regularizer in the loss function. Keypoint correspondences are integrated to focus on the alignment of smaller structures.
We perform an extensive evaluation to assess the accuracy, the robustness, the plausibility of the estimated deformation fields, and the transferability of our registration approach. We show that it achieves state-of-the-art results on the COPDGene dataset compared to conventional registration method with much shorter execution time. In our experiments on the DIRLab exhale to inhale lung registration, we demonstrate substantial improvements (TRE below $1.2$ mm) over other deep learning methods. Our algorithm is publicly available at https://grand-challenge.org/algorithms/deep-learning-based-ct-lung-registration/.
△ Less
Submitted 14 June, 2021; v1 submitted 29 November, 2020;
originally announced November 2020.
