Showing 1–9 of 9 results for author: Stojanovski, D

Echo from noise: synthetic ultrasound image generation using diffusion models for real image segmentation

Authors: David Stojanovski, Uxio Hermida, Pablo Lamata, Arian Beqiri, Alberto Gomez

Abstract: We propose a novel pipeline for the generation of synthetic ultrasound images via Denoising Diffusion Probabilistic Models (DDPMs) guided by cardiac semantic label maps. We show that these synthetic images can serve as a viable substitute for real data in the training of deep-learning models for ultrasound image analysis tasks such as cardiac segmentation. To demonstrate the effectiveness of this… ▽ More We propose a novel pipeline for the generation of synthetic ultrasound images via Denoising Diffusion Probabilistic Models (DDPMs) guided by cardiac semantic label maps. We show that these synthetic images can serve as a viable substitute for real data in the training of deep-learning models for ultrasound image analysis tasks such as cardiac segmentation. To demonstrate the effectiveness of this approach, we generated synthetic 2D echocardiograms and trained a neural network for segmenting the left ventricle and left atrium. The performance of the network trained on exclusively synthetic images was evaluated on an unseen dataset of real images and yielded mean Dice scores of 88.6 $\pm 4.91$ , 91.9 $\pm 4.22$, 85.2 $\pm 4.83$ \% for left ventricular endocardium, epicardium and left atrial segmentation respectively. This represents a relative increase of $9.2$, $3.3$ and $13.9$ \% in Dice scores compared to the previous state-of-the-art. The proposed pipeline has potential for application to a wide range of other tasks across various medical imaging modalities. △ Less

Submitted 15 August, 2023; v1 submitted 9 May, 2023; originally announced May 2023.

Language-Family Adapters for Low-Resource Multilingual Neural Machine Translation

Authors: Alexandra Chronopoulou, Dario Stojanovski, Alexander Fraser

Abstract: Large multilingual models trained with self-supervision achieve state-of-the-art results in a wide range of natural language processing tasks. Self-supervised pretrained models are often fine-tuned on parallel data from one or multiple language pairs for machine translation. Multilingual fine-tuning improves performance on low-resource languages but requires modifying the entire model and can be p… ▽ More Large multilingual models trained with self-supervision achieve state-of-the-art results in a wide range of natural language processing tasks. Self-supervised pretrained models are often fine-tuned on parallel data from one or multiple language pairs for machine translation. Multilingual fine-tuning improves performance on low-resource languages but requires modifying the entire model and can be prohibitively expensive. Training a new adapter on each language pair or training a single adapter on all language pairs without updating the pretrained model has been proposed as a parameter-efficient alternative. However, the former does not permit any sharing between languages, while the latter shares parameters for all languages and is susceptible to negative interference. In this paper, we propose training language-family adapters on top of mBART-50 to facilitate cross-lingual transfer. Our approach outperforms related baselines, yielding higher translation scores on average when translating from English to 17 different low-resource languages. We also show that language-family adapters provide an effective method to translate to languages unseen during pretraining. △ Less

Submitted 29 March, 2023; v1 submitted 30 September, 2022; originally announced September 2022.

Comments: LoResMT (@EACL 2023) camera-ready version

Efficient Pix2Vox++ for 3D Cardiac Reconstruction from 2D echo views

Authors: David Stojanovski, Uxio Hermida, Marica Muffoletto, Pablo Lamata, Arian Beqiri, Alberto Gomez

Abstract: Accurate geometric quantification of the human heart is a key step in the diagnosis of numerous cardiac diseases, and in the management of cardiac patients. Ultrasound imaging is the primary modality for cardiac imaging, however acquisition requires high operator skill, and its interpretation and analysis is difficult due to artifacts. Reconstructing cardiac anatomy in 3D can enable discovery of n… ▽ More Accurate geometric quantification of the human heart is a key step in the diagnosis of numerous cardiac diseases, and in the management of cardiac patients. Ultrasound imaging is the primary modality for cardiac imaging, however acquisition requires high operator skill, and its interpretation and analysis is difficult due to artifacts. Reconstructing cardiac anatomy in 3D can enable discovery of new biomarkers and make imaging less dependent on operator expertise, however most ultrasound systems only have 2D imaging capabilities. We propose both a simple alteration to the Pix2Vox++ networks for a sizeable reduction in memory usage and computational complexity, and a pipeline to perform reconstruction of 3D anatomy from 2D standard cardiac views, effectively enabling 3D anatomical reconstruction from limited 2D data. We evaluate our pipeline using synthetically generated data achieving accurate 3D whole-heart reconstructions (peak intersection over union score > 0.88) from just two standard anatomical 2D views of the heart. We also show preliminary results using real echo images. △ Less

Submitted 27 July, 2022; originally announced July 2022.

Comments: 11 pages, 4 figures, July 27 2022 submitted to 3rd International Workshop, Advances in Simplifying Medical Ultrasound (ASMUS2022), https://miccai-ultrasound.github.io/#/asmus22

Improving the Lexical Ability of Pretrained Language Models for Unsupervised Neural Machine Translation

Authors: Alexandra Chronopoulou, Dario Stojanovski, Alexander Fraser

Abstract: Successful methods for unsupervised neural machine translation (UNMT) employ crosslingual pretraining via self-supervision, often in the form of a masked language modeling or a sequence generation task, which requires the model to align the lexical- and high-level representations of the two languages. While cross-lingual pretraining works for similar languages with abundant corpora, it performs po… ▽ More Successful methods for unsupervised neural machine translation (UNMT) employ crosslingual pretraining via self-supervision, often in the form of a masked language modeling or a sequence generation task, which requires the model to align the lexical- and high-level representations of the two languages. While cross-lingual pretraining works for similar languages with abundant corpora, it performs poorly in low-resource and distant languages. Previous research has shown that this is because the representations are not sufficiently aligned. In this paper, we enhance the bilingual masked language model pretraining with lexical-level information by using type-level cross-lingual subword embeddings. Empirical results demonstrate improved performance both on UNMT (up to 4.5 BLEU) and bilingual lexicon induction using our method compared to a UNMT baseline. △ Less

Submitted 14 April, 2021; v1 submitted 18 March, 2021; originally announced March 2021.

Comments: Accepted at NAACL 2021

The LMU Munich System for the WMT 2020 Unsupervised Machine Translation Shared Task

Authors: Alexandra Chronopoulou, Dario Stojanovski, Viktor Hangya, Alexander Fraser

Abstract: This paper describes the submission of LMU Munich to the WMT 2020 unsupervised shared task, in two language directions, German<->Upper Sorbian. Our core unsupervised neural machine translation (UNMT) system follows the strategy of Chronopoulou et al. (2020), using a monolingual pretrained language generation model (on German) and fine-tuning it on both German and Upper Sorbian, before initializing… ▽ More This paper describes the submission of LMU Munich to the WMT 2020 unsupervised shared task, in two language directions, German<->Upper Sorbian. Our core unsupervised neural machine translation (UNMT) system follows the strategy of Chronopoulou et al. (2020), using a monolingual pretrained language generation model (on German) and fine-tuning it on both German and Upper Sorbian, before initializing a UNMT model, which is trained with online backtranslation. Pseudo-parallel data obtained from an unsupervised statistical machine translation (USMT) system is used to fine-tune the UNMT model. We also apply BPE-Dropout to the low resource (Upper Sorbian) data to obtain a more robust system. We additionally experiment with residual adapters and find them useful in the Upper Sorbian->German direction. We explore sampling during backtranslation and curriculum learning to use SMT translations in a more principled way. Finally, we ensemble our best-performing systems and reach a BLEU score of 32.4 on German->Upper Sorbian and 35.2 on Upper Sorbian->German. △ Less

Submitted 25 October, 2020; originally announced October 2020.

Comments: WMT Unsupervised Shared Task 2020

Reusing a Pretrained Language Model on Languages with Limited Corpora for Unsupervised NMT

Authors: Alexandra Chronopoulou, Dario Stojanovski, Alexander Fraser

Abstract: Using a language model (LM) pretrained on two languages with large monolingual data in order to initialize an unsupervised neural machine translation (UNMT) system yields state-of-the-art results. When limited data is available for one language, however, this method leads to poor translations. We present an effective approach that reuses an LM that is pretrained only on the high-resource language.… ▽ More Using a language model (LM) pretrained on two languages with large monolingual data in order to initialize an unsupervised neural machine translation (UNMT) system yields state-of-the-art results. When limited data is available for one language, however, this method leads to poor translations. We present an effective approach that reuses an LM that is pretrained only on the high-resource language. The monolingual LM is fine-tuned on both languages and is then used to initialize a UNMT model. To reuse the pretrained LM, we have to modify its predefined vocabulary, to account for the new language. We therefore propose a novel vocabulary extension method. Our approach, RE-LM, outperforms a competitive cross-lingual pretraining model (XLM) in English-Macedonian (En-Mk) and English-Albanian (En-Sq), yielding more than +8.3 BLEU points for all four translation directions. △ Less

Submitted 6 October, 2020; v1 submitted 16 September, 2020; originally announced September 2020.

Comments: EMNLP 2020, main conference

Addressing Zero-Resource Domains Using Document-Level Context in Neural Machine Translation

Authors: Dario Stojanovski, Alexander Fraser

Abstract: Achieving satisfying performance in machine translation on domains for which there is no training data is challenging. Traditional supervised domain adaptation is not suitable for addressing such zero-resource domains because it relies on in-domain parallel data. We show that when in-domain parallel data is not available, access to document-level context enables better capturing of domain generali… ▽ More Achieving satisfying performance in machine translation on domains for which there is no training data is challenging. Traditional supervised domain adaptation is not suitable for addressing such zero-resource domains because it relies on in-domain parallel data. We show that when in-domain parallel data is not available, access to document-level context enables better capturing of domain generalities compared to only having access to a single sentence. Having access to more information provides a more reliable domain estimation. We present two document-level Transformer models which are capable of using large context sizes and we compare these models against strong Transformer baselines. We obtain improvements for the two zero resource domains we study. We additionally provide an analysis where we vary the amount of context and look at the case where in-domain data is available. △ Less

Submitted 19 April, 2021; v1 submitted 30 April, 2020; originally announced April 2020.

Secure Wireless Communications via Cooperative Transmitting

Authors: Toni Draganov Stojanovski, Ninoslav Marina

Abstract: Information theoretic secrecy is combined with cryptographic secrecy to create a secret-key exchange protocol for wireless networks. A network of transmitters, which already have cryptographically secured channels between them, cooperate to exchange a secret key with a new receiver at a random location, in the presence of passive eavesdroppers at unknown locations. Two spatial point processes: hom… ▽ More Information theoretic secrecy is combined with cryptographic secrecy to create a secret-key exchange protocol for wireless networks. A network of transmitters, which already have cryptographically secured channels between them, cooperate to exchange a secret key with a new receiver at a random location, in the presence of passive eavesdroppers at unknown locations. Two spatial point processes: homogeneous Poisson process and independent uniformly distributed points are used for the spatial distributions of transmitters and eavesdroppers. We analyse the impact of the number of cooperating transmitters and the number of eavesdroppers on the area fraction where secure communication is possible. Upper bounds on the probability of existence of positive secrecy between the cooperating transmitters and the receiver are derived. The closeness of the upper bounds to the real value is then estimated by means of numerical simulations. Simulations also indicate that a deterministic spatial distribution for the transmitters e.g. hexagonal and square lattices, increases the probability of existence of positive secrecy capacity compared to the random spatial distributions. For the same number of friendly nodes, cooperative transmitting provides a dramatically larger secrecy region than cooperative jamming and cooperative relaying. △ Less

Submitted 11 March, 2013; originally announced March 2013.

Comments: Submitted for presentation at the 2013 IEEE International Symposium on Information Theory, Istanbul, Turkey, July 2013

Interest Rate Risk of Bond Prices on Macedonian Stock Exchange - Empirical Test of the Duration, Modified Duration and Convexity and Bonds Valuation

Authors: Zoran Ivanovski, Toni Draganov Stojanovski, Nadica Ivanovska

Abstract: This article presents valuation of Treasury Bonds (T-Bonds) on Macedonian Stock Exchange (MSE) and empirical test of duration, modified duration and convexity of the T-bonds at MSE in order to determine sensitivity of bonds prices on interest rate changes. The main goal of this study is to determine how standard valuation models fit in case of T- Bonds that are traded on MSE and to verify whether… ▽ More This article presents valuation of Treasury Bonds (T-Bonds) on Macedonian Stock Exchange (MSE) and empirical test of duration, modified duration and convexity of the T-bonds at MSE in order to determine sensitivity of bonds prices on interest rate changes. The main goal of this study is to determine how standard valuation models fit in case of T- Bonds that are traded on MSE and to verify whether they offer reliable results compared with average bonds prices on MSE. We test the sensitivity of T- Bonds on MSE on interest rate changes and determine that convexity is more accurate measure as approximation of bond prices changes than duration. Final conclusion is that T-Bonds traded at MSE are not sensitive on interest rate changes due to institutional investors' permanent higher demand and at the same time market limited offer of risk-free instruments. △ Less

Submitted 29 June, 2012; originally announced June 2012.

Comments: Submitted to Economic Research, Juraj Dobrila University of Pula, Croatia