-
Towards Safer Robot-Assisted Surgery: A Markerless Augmented Reality Framework
Authors:
Ziyang Chen,
Laura Cruciani,
Ke Fan,
Matteo Fontana,
Elena Lievore,
Ottavio De Cobelli,
Gennaro Musi,
Giancarlo Ferrigno,
Elena De Momi
Abstract:
Robot-assisted surgery is rapidly develo** in the medical field, and the integration of augmented reality shows the potential of improving the surgeons' operation performance by providing more visual information. In this paper, we proposed a markerless augmented reality framework to enhance safety by avoiding intra-operative bleeding which is a high risk caused by the collision between the surgi…
▽ More
Robot-assisted surgery is rapidly develo** in the medical field, and the integration of augmented reality shows the potential of improving the surgeons' operation performance by providing more visual information. In this paper, we proposed a markerless augmented reality framework to enhance safety by avoiding intra-operative bleeding which is a high risk caused by the collision between the surgical instruments and the blood vessel. Advanced stereo reconstruction and segmentation networks are compared to find out the best combination to reconstruct the intra-operative blood vessel in the 3D space for the registration of the pre-operative model, and the minimum distance detection between the instruments and the blood vessel is implemented. A robot-assisted lymphadenectomy is simulated on the da Vinci Research Kit in a dry lab, and ten human subjects performed this operation to explore the usability of the proposed framework. The result shows that the augmented reality framework can help the users to avoid the dangerous collision between the instruments and the blood vessel while not introducing an extra load. It provides a flexible framework that integrates augmented reality into the medical robot platform to enhance safety during the operation.
△ Less
Submitted 14 September, 2023;
originally announced September 2023.
-
When Multi-Task Learning Meets Partial Supervision: A Computer Vision Review
Authors:
Maxime Fontana,
Michael Spratling,
Miao**g Shi
Abstract:
Multi-Task Learning (MTL) aims to learn multiple tasks simultaneously while exploiting their mutual relationships. By using shared resources to simultaneously calculate multiple outputs, this learning paradigm has the potential to have lower memory requirements and inference times compared to the traditional approach of using separate methods for each task. Previous work in MTL has mainly focused…
▽ More
Multi-Task Learning (MTL) aims to learn multiple tasks simultaneously while exploiting their mutual relationships. By using shared resources to simultaneously calculate multiple outputs, this learning paradigm has the potential to have lower memory requirements and inference times compared to the traditional approach of using separate methods for each task. Previous work in MTL has mainly focused on fully-supervised methods, as task relationships can not only be leveraged to lower the level of data-dependency of those methods but they can also improve performance. However, MTL introduces a set of challenges due to a complex optimisation scheme and a higher labeling requirement. This review focuses on how MTL could be utilised under different partial supervision settings to address these challenges. First, this review analyses how MTL traditionally uses different parameter sharing techniques to transfer knowledge in between tasks. Second, it presents the different challenges arising from such a multi-objective optimisation scheme. Third, it introduces how task grou**s can be achieved by analysing task relationships. Fourth, it focuses on how partially supervised methods applied to MTL can tackle the aforementioned challenges. Lastly, this review presents the available datasets, tools and benchmarking results of such methods.
△ Less
Submitted 25 July, 2023;
originally announced July 2023.
-
Semi-supervised Bladder Tissue Classification in Multi-Domain Endoscopic Images
Authors:
Jorge F. Lazo,
Benoit Rosa,
Michele Catellani,
Matteo Fontana,
Francesco A. Mistretta,
Gennaro Musi,
Ottavio de Cobelli,
Michel de Mathelin,
Elena De Momi
Abstract:
Objective: Accurate visual classification of bladder tissue during Trans-Urethral Resection of Bladder Tumor (TURBT) procedures is essential to improve early cancer diagnosis and treatment. During TURBT interventions, White Light Imaging (WLI) and Narrow Band Imaging (NBI) techniques are used for lesion detection. Each imaging technique provides diverse visual information that allows clinicians to…
▽ More
Objective: Accurate visual classification of bladder tissue during Trans-Urethral Resection of Bladder Tumor (TURBT) procedures is essential to improve early cancer diagnosis and treatment. During TURBT interventions, White Light Imaging (WLI) and Narrow Band Imaging (NBI) techniques are used for lesion detection. Each imaging technique provides diverse visual information that allows clinicians to identify and classify cancerous lesions. Computer vision methods that use both imaging techniques could improve endoscopic diagnosis. We address the challenge of tissue classification when annotations are available only in one domain, in our case WLI, and the endoscopic images correspond to an unpaired dataset, i.e. there is no exact equivalent for every image in both NBI and WLI domains. Method: We propose a semi-surprised Generative Adversarial Network (GAN)-based method composed of three main components: a teacher network trained on the labeled WLI data; a cycle-consistency GAN to perform unpaired image-to-image translation, and a multi-input student network. To ensure the quality of the synthetic images generated by the proposed GAN we perform a detailed quantitative, and qualitative analysis with the help of specialists. Conclusion: The overall average classification accuracy, precision, and recall obtained with the proposed method for tissue classification are 0.90, 0.88, and 0.89 respectively, while the same metrics obtained in the unlabeled domain (NBI) are 0.92, 0.64, and 0.94 respectively. The quality of the generated images is reliable enough to deceive specialists. Significance: This study shows the potential of using semi-supervised GAN-based bladder tissue classification when annotations are limited in multi-domain data. The dataset is available at https://zenodo.org/record/7741476#.ZBQUK7TMJ6k
△ Less
Submitted 17 March, 2023; v1 submitted 21 December, 2022;
originally announced December 2022.
-
A Robust Learning Methodology for Uncertainty-aware Scientific Machine Learning models
Authors:
Erbet Costa Almeida,
Carine de Menezes Rebello,
Marcio Fontana,
Leizer Schnitman,
Idelfonso Bessa dos Reis Nogueira
Abstract:
Robust learning is an important issue in Scientific Machine Learning (SciML). There are several works in the literature addressing this topic. However, there is an increasing demand for methods that can simultaneously consider all the different uncertainty components involved in SciML model identification. Hence, this work proposes a comprehensive methodology for uncertainty evaluation of the SciM…
▽ More
Robust learning is an important issue in Scientific Machine Learning (SciML). There are several works in the literature addressing this topic. However, there is an increasing demand for methods that can simultaneously consider all the different uncertainty components involved in SciML model identification. Hence, this work proposes a comprehensive methodology for uncertainty evaluation of the SciML that also considers several possible sources of uncertainties involved in the identification process. The uncertainties considered in the proposed method are the absence of theory and causal models, the sensitiveness to data corruption or imperfection, and the computational effort. Therefore, it was possible to provide an overall strategy for the uncertainty-aware models in the SciML field. The methodology is validated through a case study, develo** a Soft Sensor for a polymerization reactor. The results demonstrated that the identified Soft Sensor are robust for uncertainties, corroborating with the consistency of the proposed approach.
△ Less
Submitted 5 September, 2022;
originally announced September 2022.
-
Diaphragm Ankle Actuation for Efficient Series Elastic Legged Robot Hop**
Authors:
Marco Bolignari,
An Mo,
Marco Fontana,
Alexander Badri-Spröwitz
Abstract:
Robots need lightweight legs for agile locomotion, and intrinsic series elastic compliance has proven to be a major ingredient for energy-efficient locomotion and robust locomotion control. Animals' anatomy and locomotion capabilities emphasize the importance of that lightweight legs and integrated, compact, series elastically actuated for distal leg joints. But unlike robots, animals achieve seri…
▽ More
Robots need lightweight legs for agile locomotion, and intrinsic series elastic compliance has proven to be a major ingredient for energy-efficient locomotion and robust locomotion control. Animals' anatomy and locomotion capabilities emphasize the importance of that lightweight legs and integrated, compact, series elastically actuated for distal leg joints. But unlike robots, animals achieve series elastic actuation by their muscle-tendon units. So far no designs are available that feature all characteristics of a perfect distal legged locomotion actuator; a low-weight and low-inertia design, with high mechanical efficiency, no stick and sliding friction, low mechanical complexity, high-power output while being easy to mount. Ideally, such an actuator can be controlled directly and without mechanical cross-coupling, for example remotely. With this goal in mind, we propose a low-friction, lightweight Series ELastic Diaphragm distal Actuator (SELDA) which meets many, although not all, of the above requirements. We develop, implement, and characterize a bioinspired robot leg that features a SELDA-actuated foot segment. We compare two leg configurations controlled by a central pattern generator that both feature agile forward hop**. By tuning SELDA's activation timing, we effectively adjust the robot's hop** height by 11% and its forward velocity by 14%, even with comparatively low power injection to the distal joint.
△ Less
Submitted 4 March, 2022; v1 submitted 3 March, 2022;
originally announced March 2022.
-
Landmark detection in Cardiac Magnetic Resonance Imaging Using A Convolutional Neural Network
Authors:
Hui Xue,
Jessica Artico,
Marianna Fontana,
James C Moon,
Rhodri H Davies,
Peter Kellman
Abstract:
Purpose: To develop a convolutional neural network (CNN) solution for robust landmark detection in cardiac MR images.
Methods: This retrospective study included cine, LGE and T1 map** scans from two hospitals. The training set included 2,329 patients and 34,019 images. A hold-out test set included 531 patients and 7,723 images. CNN models were developed to detect two mitral valve plane and api…
▽ More
Purpose: To develop a convolutional neural network (CNN) solution for robust landmark detection in cardiac MR images.
Methods: This retrospective study included cine, LGE and T1 map** scans from two hospitals. The training set included 2,329 patients and 34,019 images. A hold-out test set included 531 patients and 7,723 images. CNN models were developed to detect two mitral valve plane and apical points on long-axis (LAX) images. On short-axis (SAX) images, anterior and posterior RV insertion points and LV center were detected. Model outputs were compared to manual labels by two operators for accuracy with a t-test for statistical significance. The trained model was deployed to MR scanners.
Results: For the LAX images, success detection was 99.8% for cine, 99.4% for LGE. For the SAX, success rate was 96.6%, 97.6% and 98.9% for cine, LGE and T1-map**. The L2 distances between model and manual labels were 2 to 3.5 mm, indicating close agreement between model landmarks to manual labels. No significant differences were found for the anterior RV insertion angle and LV length by the models and operators for all views and imaging sequences. Model inference on MR scanner took 610ms/5.6s on GPU/CPU, respectively, for a typical cardiac cine series.
Conclusions: This study developed, validated and deployed a CNN solution for robust landmark detection in both long and short-axis CMR images for cine, LGE and T1 map** sequences, with the accuracy comparable to the inter-operator variation.
△ Less
Submitted 13 August, 2020;
originally announced August 2020.
-
Conformal Prediction: a Unified Review of Theory and New Challenges
Authors:
Matteo Fontana,
Gianluca Zeni,
Simone Vantini
Abstract:
In this work we provide a review of basic ideas and novel developments about Conformal Prediction -- an innovative distribution-free, non-parametric forecasting method, based on minimal assumptions -- that is able to yield in a very straightforward way predictions sets that are valid in a statistical sense also in in the finite sample case. The in-depth discussion provided in the paper covers the…
▽ More
In this work we provide a review of basic ideas and novel developments about Conformal Prediction -- an innovative distribution-free, non-parametric forecasting method, based on minimal assumptions -- that is able to yield in a very straightforward way predictions sets that are valid in a statistical sense also in in the finite sample case. The in-depth discussion provided in the paper covers the theoretical underpinnings of Conformal Prediction, and then proceeds to list the more advanced developments and adaptations of the original idea.
△ Less
Submitted 29 July, 2022; v1 submitted 16 May, 2020;
originally announced May 2020.
-
Automated Inline Analysis of Myocardial Perfusion MRI with Deep Learning
Authors:
Hui Xue,
Rhodri Davies,
Louis AE Brown,
Kristopher D Knott,
Tushar Kotecha,
Marianna Fontana,
Sven Plein,
James C Moon,
Peter Kellman
Abstract:
Recent development of quantitative myocardial blood flow (MBF) map** allows direct evaluation of absolute myocardial perfusion, by computing pixel-wise flow maps. Clinical studies suggest quantitative evaluation would be more desirable for objectivity and efficiency. Objective assessment can be further facilitated by segmenting the myocardium and automatically generating reports following the AH…
▽ More
Recent development of quantitative myocardial blood flow (MBF) map** allows direct evaluation of absolute myocardial perfusion, by computing pixel-wise flow maps. Clinical studies suggest quantitative evaluation would be more desirable for objectivity and efficiency. Objective assessment can be further facilitated by segmenting the myocardium and automatically generating reports following the AHA model. This will free user interaction for analysis and lead to a 'one-click' solution to improve workflow. This paper proposes a deep neural network based computational workflow for inline myocardial perfusion analysis. Adenosine stress and rest perfusion scans were acquired from three hospitals. Training set included N=1,825 perfusion series from 1,034 patients. Independent test set included 200 scans from 105 patients. Data were consecutively acquired at each site. A convolution neural net (CNN) model was trained to provide segmentation for LV cavity, myocardium and right ventricular by processing incoming 2D+T perfusion Gd series. Model outputs were compared to manual ground-truth for accuracy of segmentation and flow measures derived on global and per-sector basis. The trained models were integrated onto MR scanners for effective inference. Segmentation accuracy and myocardial flow measures were compared between CNN models and manual ground-truth. The mean Dice ratio of CNN derived myocardium was 0.93 +/- 0.04. Both global flow and per-sector values showed no significant difference, compared to manual results. The AHA 16 segment model was automatically generated and reported on the MR scanner. As a result, the fully automated analysis of perfusion flow map** was achieved. This solution was integrated on the MR scanner, enabling 'one-click' analysis and reporting of myocardial blood flow.
△ Less
Submitted 29 May, 2020; v1 submitted 1 November, 2019;
originally announced November 2019.
-
Multi-level conformal clustering: A distribution-free technique for clustering and anomaly detection
Authors:
Ilia Nouretdinov,
James Gammerman,
Matteo Fontana,
Daljit Rehal
Abstract:
In this work we present a clustering technique called \textit{multi-level conformal clustering (MLCC)}. The technique is hierarchical in nature because it can be performed at multiple significance levels which yields greater insight into the data than performing it at just one level. We describe the theoretical underpinnings of MLCC, compare and contrast it with the hierarchical clustering algorit…
▽ More
In this work we present a clustering technique called \textit{multi-level conformal clustering (MLCC)}. The technique is hierarchical in nature because it can be performed at multiple significance levels which yields greater insight into the data than performing it at just one level. We describe the theoretical underpinnings of MLCC, compare and contrast it with the hierarchical clustering algorithm, and then apply it to real world datasets to assess its performance. There are several advantages to using MLCC over more classical clustering techniques: Once a significance level has been set, MLCC is able to automatically select the number of clusters. Furthermore, thanks to the conformal prediction framework the resulting clustering model has a clear statistical meaning without any assumptions about the distribution of the data. This statistical robustness also allows us to perform clustering and anomaly detection simultaneously. Moreover, due to the flexibility of the conformal prediction framework, our algorithm can be used on top of many other machine learning algorithms.
△ Less
Submitted 21 October, 2019; v1 submitted 17 October, 2019;
originally announced October 2019.
-
Automated Detection of Left Ventricle in Arterial Input Function Images for Inline Perfusion Map** using Deep Learning: A study of 15,000 Patients
Authors:
Hui Xue,
Ethan Tseng,
Kristopher D Knott,
Tushar Kotecha,
Louise Brown,
Sven Plein,
Marianna Fontana,
James C Moon,
Peter Kellman
Abstract:
Quantification of myocardial perfusion has the potential to improve detection of regional and global flow reduction. Significant effort has been made to automate the workflow, where one essential step is the arterial input function (AIF) extraction. Since failure here invalidates quantification, high accuracy is required. For this purpose, this study presents a robust AIF detection method using th…
▽ More
Quantification of myocardial perfusion has the potential to improve detection of regional and global flow reduction. Significant effort has been made to automate the workflow, where one essential step is the arterial input function (AIF) extraction. Since failure here invalidates quantification, high accuracy is required. For this purpose, this study presents a robust AIF detection method using the convolutional neural net (CNN) model. CNN models were trained by assembling 25,027 scans (N=12,984 patients) from three hospitals, seven scanners. A test set of 5,721 scans (N=2,805 patients) evaluated model performance. The 2D+T AIF time series was inputted into CNN. Two variations were investigated: a) Two Classes (2CS) for background and foreground (LV mask); b) Three Classes (3CS) for background, foreground LV and RV. Final model was deployed on MR scanners via the Gadgetron InlineAI. Model loading time on MR scanner was ~340ms and applying it took ~180ms. The 3CS model successfully detect LV for 99.98% of all test cases (1 failed out of 5,721 cases). The mean Dice ratio for 3CS was 0.87+/-0.08 with 92.0% of all test cases having Dice ratio >0.75, while the 2CS model gave lower Dice of 0.82+/-0.22 (P<1e-5). Extracted AIF signals using CNN were further compared to manual ground-truth for foot-time, peak-time, first-pass duration, peak value and area-under-curve. No significant differences were found for all features (P>0.2). This study proposed, validated, and deployed a robust CNN solution to detect the LV for the extraction of the AIF signal used in fully automated perfusion flow map**. A very large data cohort was assembled and resulting models were deployed to MR scanners for fully inline AI in clinical hospitals.
△ Less
Submitted 6 April, 2020; v1 submitted 15 October, 2019;
originally announced October 2019.
