-
Exploring Optical Flow Inclusion into nnU-Net Framework for Surgical Instrument Segmentation
Authors:
Marcos Fernández-Rodríguez,
Bruno Silva,
Sandro Queirós,
Helena R. Torres,
Bruno Oliveira,
Pedro Morais,
Lukas R. Buschle,
Jorge Correia-Pinto,
Estevão Lima,
João L. Vilaça
Abstract:
Surgical instrument segmentation in laparoscopy is essential for computer-assisted surgical systems. Despite the Deep Learning progress in recent years, the dynamic setting of laparoscopic surgery still presents challenges for precise segmentation. The nnU-Net framework excelled in semantic segmentation analyzing single frames without temporal information. The framework's ease of use, including it…
▽ More
Surgical instrument segmentation in laparoscopy is essential for computer-assisted surgical systems. Despite the Deep Learning progress in recent years, the dynamic setting of laparoscopic surgery still presents challenges for precise segmentation. The nnU-Net framework excelled in semantic segmentation analyzing single frames without temporal information. The framework's ease of use, including its ability to be automatically configured, and its low expertise requirements, have made it a popular base framework for comparisons. Optical flow (OF) is a tool commonly used in video tasks to estimate motion and represent it in a single frame, containing temporal information. This work seeks to employ OF maps as an additional input to the nnU-Net architecture to improve its performance in the surgical instrument segmentation task, taking advantage of the fact that instruments are the main moving objects in the surgical field. With this new input, the temporal component would be indirectly added without modifying the architecture. Using CholecSeg8k dataset, three different representations of movement were estimated and used as new inputs, comparing them with a baseline model. Results showed that the use of OF maps improves the detection of classes with high movement, even when these are scarce in the dataset. To further improve performance, future work may focus on implementing other OF-preserving augmentations.
△ Less
Submitted 15 March, 2024;
originally announced March 2024.
-
Deep Spatiotemporal Clutter Filtering of Transthoracic Echocardiographic Images Using a 3D Convolutional Auto-Encoder
Authors:
Mahdi Tabassian,
Somayeh Akbari. S,
Sandro Queirós,
Jan D'hooge
Abstract:
This study presents a deep convolutional auto-encoder network for filtering reverberation artifacts, from transthoracic echocardiographic (TTE) image sequences. Given the spatiotemporal nature of these artifacts, the filtering network was built using 3D convolutional layers to suppress the clutter patterns throughout the cardiac cycle. The network was designed by taking advantage of: i) an attenti…
▽ More
This study presents a deep convolutional auto-encoder network for filtering reverberation artifacts, from transthoracic echocardiographic (TTE) image sequences. Given the spatiotemporal nature of these artifacts, the filtering network was built using 3D convolutional layers to suppress the clutter patterns throughout the cardiac cycle. The network was designed by taking advantage of: i) an attention mechanism to focus primarily on cluttered regions and ii) residual learning to preserve fine structures of the image frames. To train the deep network, a diverse set of artifact patterns was simulated and the simulated patterns were superimposed onto artifact-free ultra-realistic synthetic TTE sequences of six ultrasound vendors to generate input of the filtering network. The artifact-free sequences served as ground-truth. Performance of the filtering network was evaluated using unseen synthetic as well as in-vivo artifactual sequences. Satisfactory results obtained using the latter dataset confirmed the good generalization performance of the proposed network which was trained using the synthetic sequences and simulated artifact patterns. Suitability of the clutter-filtered sequences for further processing was assessed by computing segmental strain curves from them. The results showed that the large discrepancy between the strain profiles computed from the cluttered segments and their corresponding segments in the clutter-free images was significantly reduced after filtering the sequences using the proposed network. The trained deep network could process an artifactual TTE sequence in a fraction of a second and can be used for real-time clutter filtering. Moreover, it can improve the precision of the clinical indexes that are computed from the TTE sequences. The source code of the proposed method is available at: https://github.com/MahdiTabassian/Deep-Clutter-Filtering/tree/main.
△ Less
Submitted 23 January, 2024;
originally announced January 2024.
-
High-Resolution Maps of Left Atrial Displacements and Strains Estimated with 3D CINE MRI and Unsupervised Neural Networks
Authors:
Christoforos Galazis,
Samuel Shepperd,
Emma Brouwer,
Sandro Queirós,
Ebraham Alskaf,
Mustafa Anjari,
Amedeo Chiribiri,
Jack Lee,
Anil A. Bharath,
Marta Varela
Abstract:
The functional analysis of the left atrium (LA) is important for evaluating cardiac health and understanding diseases like atrial fibrillation. Cine MRI is ideally placed for the detailed 3D characterisation of LA motion and deformation, but it is lacking appropriate acquisition and analysis tools. In this paper, we present Analysis for Left Atrial Displacements and Deformations using unsupervIsed…
▽ More
The functional analysis of the left atrium (LA) is important for evaluating cardiac health and understanding diseases like atrial fibrillation. Cine MRI is ideally placed for the detailed 3D characterisation of LA motion and deformation, but it is lacking appropriate acquisition and analysis tools. In this paper, we present Analysis for Left Atrial Displacements and Deformations using unsupervIsed neural Networks, \textit{Aladdin}, to automatically and reliably characterise regional LA deformations from high-resolution 3D Cine MRI. The tool includes: an online few-shot segmentation network (Aladdin-S), an online unsupervised image registration network (Aladdin-R), and a strain calculations pipeline tailored to the LA. We create maps of LA Displacement Vector Field (DVF) magnitude and LA principal strain values from images of 10 healthy volunteers and 8 patients with cardiovascular disease (CVD). We additionally create an atlas of these biomarkers using the data from the healthy volunteers. Aladdin is able to accurately track the LA wall across the cardiac cycle and characterize its motion and deformation. The overall DVF magnitude and principal strain values are significantly higher in the healthy group vs CVD patients: $2.85 \pm 1.59~mm$ and $0.09 \pm 0.05$ vs $1.96 \pm 0.74~mm$ and $0.03 \pm 0.04$, respectively. The time course of these metrics is also different in the two groups, with a more marked active contraction phase observed in the healthy cohort. Finally, utilizing the LA atlas allows us to identify regional deviations from the population distribution that may indicate focal tissue abnormalities. The proposed tool for the quantification of novel regional LA deformation biomarkers should have important clinical applications. The source code, anonymized images, generated maps and atlas are publicly available: https://github.com/cgalaz01/aladdin_cmr_la.
△ Less
Submitted 14 December, 2023;
originally announced December 2023.
-
SurgT challenge: Benchmark of Soft-Tissue Trackers for Robotic Surgery
Authors:
Joao Cartucho,
Alistair Weld,
Samyakh Tukra,
Haozheng Xu,
Hiroki Matsuzaki,
Taiyo Ishikawa,
Minjun Kwon,
Yong Eun Jang,
Kwang-Ju Kim,
Gwang Lee,
Bizhe Bai,
Lueder Kahrs,
Lars Boecking,
Simeon Allmendinger,
Leopold Muller,
Yitong Zhang,
Yueming **,
Sophia Bano,
Francisco Vasconcelos,
Wolfgang Reiter,
Jonas Hajek,
Bruno Silva,
Estevao Lima,
Joao L. Vilaca,
Sandro Queiros
, et al. (1 additional authors not shown)
Abstract:
This paper introduces the ``SurgT: Surgical Tracking" challenge which was organised in conjunction with MICCAI 2022. There were two purposes for the creation of this challenge: (1) the establishment of the first standardised benchmark for the research community to assess soft-tissue trackers; and (2) to encourage the development of unsupervised deep learning methods, given the lack of annotated da…
▽ More
This paper introduces the ``SurgT: Surgical Tracking" challenge which was organised in conjunction with MICCAI 2022. There were two purposes for the creation of this challenge: (1) the establishment of the first standardised benchmark for the research community to assess soft-tissue trackers; and (2) to encourage the development of unsupervised deep learning methods, given the lack of annotated data in surgery. A dataset of 157 stereo endoscopic videos from 20 clinical cases, along with stereo camera calibration parameters, have been provided. Participants were assigned the task of develo** algorithms to track the movement of soft tissues, represented by bounding boxes, in stereo endoscopic videos. At the end of the challenge, the developed methods were assessed on a previously hidden test subset. This assessment uses benchmarking metrics that were purposely developed for this challenge, to verify the efficacy of unsupervised deep learning algorithms in tracking soft-tissue. The metric used for ranking the methods was the Expected Average Overlap (EAO) score, which measures the average overlap between a tracker's and the ground truth bounding boxes. Coming first in the challenge was the deep learning submission by ICVS-2Ai with a superior EAO score of 0.617. This method employs ARFlow to estimate unsupervised dense optical flow from cropped images, using photometric and regularization losses. Second, Jmees with an EAO of 0.583, uses deep learning for surgical tool segmentation on top of a non-deep learning baseline method: CSRT. CSRT by itself scores a similar EAO of 0.563. The results from this challenge show that currently, non-deep learning methods are still competitive. The dataset and benchmarking tool created for this challenge have been made publicly available at https://surgt.grand-challenge.org/.
△ Less
Submitted 30 August, 2023; v1 submitted 6 February, 2023;
originally announced February 2023.
-
Biomedical image analysis competitions: The state of current participation practice
Authors:
Matthias Eisenmann,
Annika Reinke,
Vivienn Weru,
Minu Dietlinde Tizabi,
Fabian Isensee,
Tim J. Adler,
Patrick Godau,
Veronika Cheplygina,
Michal Kozubek,
Sharib Ali,
Anubha Gupta,
Jan Kybic,
Alison Noble,
Carlos Ortiz de Solórzano,
Samiksha Pachade,
Caroline Petitjean,
Daniel Sage,
Donglai Wei,
Elizabeth Wilden,
Deepak Alapatt,
Vincent Andrearczyk,
Ujjwal Baid,
Spyridon Bakas,
Niranjan Balu,
Sophia Bano
, et al. (331 additional authors not shown)
Abstract:
The number of international benchmarking competitions is steadily increasing in various fields of machine learning (ML) research and practice. So far, however, little is known about the common practice as well as bottlenecks faced by the community in tackling the research questions posed. To shed light on the status quo of algorithm development in the specific field of biomedical imaging analysis,…
▽ More
The number of international benchmarking competitions is steadily increasing in various fields of machine learning (ML) research and practice. So far, however, little is known about the common practice as well as bottlenecks faced by the community in tackling the research questions posed. To shed light on the status quo of algorithm development in the specific field of biomedical imaging analysis, we designed an international survey that was issued to all participants of challenges conducted in conjunction with the IEEE ISBI 2021 and MICCAI 2021 conferences (80 competitions in total). The survey covered participants' expertise and working environments, their chosen strategies, as well as algorithm characteristics. A median of 72% challenge participants took part in the survey. According to our results, knowledge exchange was the primary incentive (70%) for participation, while the reception of prize money played only a minor role (16%). While a median of 80 working hours was spent on method development, a large portion of participants stated that they did not have enough time for method development (32%). 25% perceived the infrastructure to be a bottleneck. Overall, 94% of all solutions were deep learning-based. Of these, 84% were based on standard architectures. 43% of the respondents reported that the data samples (e.g., images) were too large to be processed at once. This was most commonly addressed by patch-based training (69%), downsampling (37%), and solving 3D analysis tasks as a series of 2D tasks. K-fold cross-validation on the training set was performed by only 37% of the participants and only 50% of the participants performed ensembling based on multiple identical models (61%) or heterogeneous models (39%). 48% of the respondents applied postprocessing steps.
△ Less
Submitted 12 September, 2023; v1 submitted 16 December, 2022;
originally announced December 2022.
-
Hysteresis and disorder-induced order in continuous kinetic-like opinion dynamics in complex networks
Authors:
A. L. Oestereich,
M. A. Pires,
S. M. Duarte Queirós,
N. Crokidakis
Abstract:
In this work we tackle a kinetic-like model of opinions dynamics in a networked population endued with a quenched plurality and polarization. Additionally, we consider pairwise interactions that are restrictive, which is modeled with a smooth bounded confidence. Our results show the interesting emergence of nonequilibrium hysteresis and heterogeneity-assisted ordering. Such counterintuitive phenom…
▽ More
In this work we tackle a kinetic-like model of opinions dynamics in a networked population endued with a quenched plurality and polarization. Additionally, we consider pairwise interactions that are restrictive, which is modeled with a smooth bounded confidence. Our results show the interesting emergence of nonequilibrium hysteresis and heterogeneity-assisted ordering. Such counterintuitive phenomena are robust to different types of network architectures such as random, small-world and scale-free.
△ Less
Submitted 21 February, 2020;
originally announced February 2020.
-
Optimal diffusion in ecological dynamics with Allee effect in a metapopulation
Authors:
Marcelo A. Pires,
Sílvio M. Duarte Queirós
Abstract:
How diffusion impacts on ecological dynamics under the Allee effect and spatial constraints? That is the question we address. Employing a microscopic minimal model in a metapopulation (without imposing nonlinear birth and death rates) we evince --- both numerically and analitically --- the emergence of an optimal diffusion that maximises the survival probability. Even though, at first such result…
▽ More
How diffusion impacts on ecological dynamics under the Allee effect and spatial constraints? That is the question we address. Employing a microscopic minimal model in a metapopulation (without imposing nonlinear birth and death rates) we evince --- both numerically and analitically --- the emergence of an optimal diffusion that maximises the survival probability. Even though, at first such result seems counter-intuitive, it has empirical support from recent experiments with engineered bacteria. Moreover, we show that this optimal diffusion disappears for loose spatial constraints.
△ Less
Submitted 27 September, 2018;
originally announced September 2018.
-
The Elephant Quantum Walk
Authors:
Giuseppe Di Molfetta,
Diogo O. Soares-Pinto,
Silvio M. Duarte Queiros
Abstract:
We explore the impact of long-range memory on the properties of a family of quantum walks in a one-dimensional lattice and discrete time, which can be understood as the quantum version of the classical "Elephant Random Walk" non-Markovian process. This Elephant Quantum Walk is robustly superballistic with the standard deviation showing a constant exponent, $σ\propto t^{3/ 2}$ , whatever the quantu…
▽ More
We explore the impact of long-range memory on the properties of a family of quantum walks in a one-dimensional lattice and discrete time, which can be understood as the quantum version of the classical "Elephant Random Walk" non-Markovian process. This Elephant Quantum Walk is robustly superballistic with the standard deviation showing a constant exponent, $σ\propto t^{3/ 2}$ , whatever the quantum coin operator, on which the diffusion coefficient is dependent. On the one hand, this result indicates that contrarily to the classical case, the degree of superdiffusivity in quantum non- Markovian processes of this kind is mainly ruled by the extension of memory rather than other microscopic parameters that explicitly define the process. On the other hand, these parameters reflect on the diffusion coefficient.
△ Less
Submitted 1 June, 2018; v1 submitted 27 September, 2017;
originally announced September 2017.
-
On the effectiveness of a binless entropy estimator for generalised entropic forms
Authors:
Silvio M. Duarte Queiros
Abstract:
In this manuscript we discuss the effectiveness of the Kozachenko-Leonenko entropy estimator when generalised to cope with entropic forms customarily applied to study systems evincing asymptotic scale invariance and dependence (either linear or non-linear type). We show that when the variables are independently and identically distributed the estimator is only valuable along the whole domain if…
▽ More
In this manuscript we discuss the effectiveness of the Kozachenko-Leonenko entropy estimator when generalised to cope with entropic forms customarily applied to study systems evincing asymptotic scale invariance and dependence (either linear or non-linear type). We show that when the variables are independently and identically distributed the estimator is only valuable along the whole domain if the data follow the uniform distribution, whereas for other distributions the estimator is only effectual in the limit of the Boltzmann-Gibbs-Shanon entropic form. We also analyse the influence of the dependence (linear and non-linear) between variables on the accuracy of the estimator between variables. As expected in the last case the estimator looses efficiency for the Boltzmann-Gibbs-Shanon entropic form as well.
△ Less
Submitted 3 January, 2010; v1 submitted 7 June, 2009;
originally announced June 2009.