-
MEDPSeg: Hierarchical polymorphic multitask learning for the segmentation of ground-glass opacities, consolidation, and pulmonary structures on computed tomography
Authors:
Diedre S. Carmo,
Jean A. Ribeiro,
Alejandro P. Comellas,
Joseph M. Reinhardt,
Sarah E. Gerard,
Letícia Rittner,
Roberto A. Lotufo
Abstract:
The COVID-19 pandemic response highlighted the potential of deep learning methods in facilitating the diagnosis, prognosis and understanding of lung diseases through automated segmentation of pulmonary structures and lesions in chest computed tomography (CT). Automated separation of lung lesion into ground-glass opacity (GGO) and consolidation is hindered due to the labor-intensive and subjective…
▽ More
The COVID-19 pandemic response highlighted the potential of deep learning methods in facilitating the diagnosis, prognosis and understanding of lung diseases through automated segmentation of pulmonary structures and lesions in chest computed tomography (CT). Automated separation of lung lesion into ground-glass opacity (GGO) and consolidation is hindered due to the labor-intensive and subjective nature of this task, resulting in scarce availability of ground truth for supervised learning. To tackle this problem, we propose MEDPSeg. MEDPSeg learns from heterogeneous chest CT targets through hierarchical polymorphic multitask learning (HPML). HPML explores the hierarchical nature of GGO and consolidation, lung lesions, and the lungs, with further benefits achieved through multitasking airway and pulmonary artery segmentation. Over 6000 volumetric CT scans from different partially labeled sources were used for training and testing. Experiments show PML enabling new state-of-the-art performance for GGO and consolidation segmentation tasks. In addition, MEDPSeg simultaneously performs segmentation of the lung parenchyma, airways, pulmonary artery, and lung lesions, all in a single forward prediction, with performance comparable to state-of-the-art methods specialized in each of those targets. Finally, we provide an open-source implementation with a graphical user interface at https://github.com/MICLab-Unicamp/medpseg.
△ Less
Submitted 25 March, 2024; v1 submitted 4 December, 2023;
originally announced December 2023.
-
Automatic segmentation of lung findings in CT and application to Long COVID
Authors:
Diedre S. Carmo,
Rosarie A. Tudas,
Alejandro P. Comellas,
Leticia Rittner,
Roberto A. Lotufo,
Joseph M. Reinhardt,
Sarah E. Gerard
Abstract:
Automated segmentation of lung abnormalities in computed tomography is an important step for diagnosing and characterizing lung disease. In this work, we improve upon a previous method and propose S-MEDSeg, a deep learning based approach for accurate segmentation of lung lesions in chest CT images. S-MEDSeg combines a pre-trained EfficientNet backbone, bidirectional feature pyramid network, and mo…
▽ More
Automated segmentation of lung abnormalities in computed tomography is an important step for diagnosing and characterizing lung disease. In this work, we improve upon a previous method and propose S-MEDSeg, a deep learning based approach for accurate segmentation of lung lesions in chest CT images. S-MEDSeg combines a pre-trained EfficientNet backbone, bidirectional feature pyramid network, and modern network advancements to achieve improved segmentation performance. A comprehensive ablation study was performed to evaluate the contribution of the proposed network modifications. The results demonstrate modifications introduced in S-MEDSeg significantly improves segmentation performance compared to the baseline approach. The proposed method is applied to an independent dataset of long COVID inpatients to study the effect of post-acute infection vaccination on extent of lung findings. Open-source code, graphical user interface and pip package are available at https://github.com/MICLab-Unicamp/medseg.
△ Less
Submitted 13 October, 2023;
originally announced October 2023.
-
Challenges of the Creation of a Dataset for Vision Based Human Hand Action Recognition in Industrial Assembly
Authors:
Fabian Sturm,
Elke Hergenroether,
Julian Reinhardt,
Petar Smilevski Vojnovikj,
Melanie Siegel
Abstract:
This work presents the Industrial Hand Action Dataset V1, an industrial assembly dataset consisting of 12 classes with 459,180 images in the basic version and 2,295,900 images after spatial augmentation. Compared to other freely available datasets tested, it has an above-average duration and, in addition, meets the technical and legal requirements for industrial assembly lines. Furthermore, the da…
▽ More
This work presents the Industrial Hand Action Dataset V1, an industrial assembly dataset consisting of 12 classes with 459,180 images in the basic version and 2,295,900 images after spatial augmentation. Compared to other freely available datasets tested, it has an above-average duration and, in addition, meets the technical and legal requirements for industrial assembly lines. Furthermore, the dataset contains occlusions, hand-object interaction, and various fine-grained human hand actions for industrial assembly tasks that were not found in combination in examined datasets. The recorded ground truth assembly classes were selected after extensive observation of real-world use cases. A Gated Transformer Network, a state-of-the-art model from the transformer domain was adapted, and proved with a test accuracy of 86.25% before hyperparameter tuning by 18,269,959 trainable parameters, that it is possible to train sequential deep learning models with this dataset.
△ Less
Submitted 7 March, 2023;
originally announced March 2023.
-
Single volume lung biomechanics from chest computed tomography using a mode preserving generative adversarial network
Authors:
Muhammad F. A. Chaudhary,
Sarah E. Gerard,
Di Wang,
Gary E. Christensen,
Christopher B. Cooper,
Joyce D. Schroeder,
Eric A. Hoffman,
Joseph M. Reinhardt
Abstract:
Local tissue expansion of the lungs is typically derived by registering computed tomography (CT) scans acquired at multiple lung volumes. However, acquiring multiple scans incurs increased radiation dose, time, and cost, and may not be possible in many cases, thus restricting the applicability of registration-based biomechanics. We propose a generative adversarial learning approach for estimating…
▽ More
Local tissue expansion of the lungs is typically derived by registering computed tomography (CT) scans acquired at multiple lung volumes. However, acquiring multiple scans incurs increased radiation dose, time, and cost, and may not be possible in many cases, thus restricting the applicability of registration-based biomechanics. We propose a generative adversarial learning approach for estimating local tissue expansion directly from a single CT scan. The proposed framework was trained and evaluated on 2500 subjects from the SPIROMICS cohort. Once trained, the framework can be used as a registration-free method for predicting local tissue expansion. We evaluated model performance across varying degrees of disease severity and compared its performance with two image-to-image translation frameworks - UNet and Pix2Pix. Our model achieved an overall PSNR of 18.95 decibels, SSIM of 0.840, and Spearman's correlation of 0.61 at a high spatial resolution of 1 mm3.
△ Less
Submitted 15 October, 2021;
originally announced October 2021.
-
Recursive Refinement Network for Deformable Lung Registration between Exhale and Inhale CT Scans
Authors:
Xinzi He,
Jia Guo,
Xuzhe Zhang,
Hanwen Bi,
Sarah Gerard,
David Kaczka,
Amin Motahari,
Eric Hoffman,
Joseph Reinhardt,
R. Graham Barr,
Elsa Angelini,
Andrew Laine
Abstract:
Unsupervised learning-based medical image registration approaches have witnessed rapid development in recent years. We propose to revisit a commonly ignored while simple and well-established principle: recursive refinement of deformation vector fields across scales. We introduce a recursive refinement network (RRN) for unsupervised medical image registration, to extract multi-scale features, const…
▽ More
Unsupervised learning-based medical image registration approaches have witnessed rapid development in recent years. We propose to revisit a commonly ignored while simple and well-established principle: recursive refinement of deformation vector fields across scales. We introduce a recursive refinement network (RRN) for unsupervised medical image registration, to extract multi-scale features, construct normalized local cost correlation volume and recursively refine volumetric deformation vector fields. RRN achieves state of the art performance for 3D registration of expiratory-inspiratory pairs of CT lung scans. On DirLab COPDGene dataset, RRN returns an average Target Registration Error (TRE) of 0.83 mm, which corresponds to a 13% error reduction from the best result presented in the leaderboard. In addition to comparison with conventional methods, RRN leads to 89% error reduction compared to deep-learning-based peer approaches.
△ Less
Submitted 14 June, 2021;
originally announced June 2021.
-
CT Image Segmentation for Inflamed and Fibrotic Lungs Using a Multi-Resolution Convolutional Neural Network
Authors:
Sarah E. Gerard,
Jacob Herrmann,
Yi Xin,
Kevin T. Martin,
Emanuele Rezoagli,
Davide Ippolito,
Giacomo Bellani,
Maurizio Cereda,
Junfeng Guo,
Eric A. Hoffman,
David W. Kaczka,
Joseph M. Reinhardt
Abstract:
The purpose of this study was to develop a fully-automated segmentation algorithm, robust to various density enhancing lung abnormalities, to facilitate rapid quantitative analysis of computed tomography images. A polymorphic training approach is proposed, in which both specifically labeled left and right lungs of humans with COPD, and nonspecifically labeled lungs of animals with acute lung injur…
▽ More
The purpose of this study was to develop a fully-automated segmentation algorithm, robust to various density enhancing lung abnormalities, to facilitate rapid quantitative analysis of computed tomography images. A polymorphic training approach is proposed, in which both specifically labeled left and right lungs of humans with COPD, and nonspecifically labeled lungs of animals with acute lung injury, were incorporated into training a single neural network. The resulting network is intended for predicting left and right lung regions in humans with or without diffuse opacification and consolidation. Performance of the proposed lung segmentation algorithm was extensively evaluated on CT scans of subjects with COPD, confirmed COVID-19, lung cancer, and IPF, despite no labeled training data of the latter three diseases. Lobar segmentations were obtained using the left and right lung segmentation as input to the LobeNet algorithm. Regional lobar analysis was performed using hierarchical clustering to identify radiographic subtypes of COVID-19. The proposed lung segmentation algorithm was quantitatively evaluated using semi-automated and manually-corrected segmentations in 87 COVID-19 CT images, achieving an average symmetric surface distance of $0.495 \pm 0.309$ mm and Dice coefficient of $0.985 \pm 0.011$. Hierarchical clustering identified four radiographical phenotypes of COVID-19 based on lobar fractions of consolidated and poorly aerated tissue. Lower left and lower right lobes were consistently more afflicted with poor aeration and consolidation. However, the most severe cases demonstrated involvement of all lobes. The polymorphic training approach was able to accurately segment COVID-19 cases with diffuse consolidation without requiring COVID-19 cases for training.
△ Less
Submitted 14 January, 2021; v1 submitted 16 October, 2020;
originally announced October 2020.
-
Competing in a Complex Hidden Role Game with Information Set Monte Carlo Tree Search
Authors:
Jack Reinhardt
Abstract:
Advances in intelligent game playing agents have led to successes in perfect information games like Go and imperfect information games like Poker. The Information Set Monte Carlo Tree Search (ISMCTS) family of algorithms outperforms previous algorithms using Monte Carlo methods in imperfect information games. In this paper, Single Observer Information Set Monte Carlo Tree Search (SO-ISMCTS) is app…
▽ More
Advances in intelligent game playing agents have led to successes in perfect information games like Go and imperfect information games like Poker. The Information Set Monte Carlo Tree Search (ISMCTS) family of algorithms outperforms previous algorithms using Monte Carlo methods in imperfect information games. In this paper, Single Observer Information Set Monte Carlo Tree Search (SO-ISMCTS) is applied to Secret Hitler, a popular social deduction board game that combines traditional hidden role mechanics with the randomness of a card deck. This combination leads to a more complex information model than the hidden role and card deck mechanics alone. It is shown in 10108 simulated games that SO-ISMCTS plays as well as simpler rule based agents, and demonstrates the potential of ISMCTS algorithms in complicated information set domains.
△ Less
Submitted 14 May, 2020;
originally announced May 2020.
-
An Efficient Data Structure for Dynamic Two-Dimensional Reconfiguration
Authors:
Sándor P. Fekete,
Jan-Marc Reinhardt,
Christian Scheffer
Abstract:
In the presence of dynamic insertions and deletions into a partially reconfigurable FPGA, fragmentation is unavoidable. This poses the challenge of develo** efficient approaches to dynamic defragmentation and reallocation. One key aspect is to develop efficient algorithms and data structures that exploit the two-dimensional geometry of a chip, instead of just one. We propose a new method for thi…
▽ More
In the presence of dynamic insertions and deletions into a partially reconfigurable FPGA, fragmentation is unavoidable. This poses the challenge of develo** efficient approaches to dynamic defragmentation and reallocation. One key aspect is to develop efficient algorithms and data structures that exploit the two-dimensional geometry of a chip, instead of just one. We propose a new method for this task, based on the fractal structure of a quadtree, which allows dynamic segmentation of the chip area, along with dynamically adjusting the necessary communication infrastructure. We describe a number of algorithmic aspects, and present different solutions. We also provide a number of basic simulations that indicate that the theoretical worst-case bound may be pessimistic.
△ Less
Submitted 24 February, 2017;
originally announced February 2017.
-
Collecting a Swarm in a Grid Environment Using Shared, Global Inputs
Authors:
Arun V. Mahadev,
Dominik Krupke,
Jan-Marc Reinhardt,
Sándor P. Fekete,
Aaron T. Becker
Abstract:
This paper investigates efficient techniques to collect and concentrate an under-actuated particle swarm despite obstacles. Concentrating a swarm of particles is of critical importance in health-care for targeted drug delivery, where micro-scale particles must be steered to a goal location. Individual particles must be small in order to navigate through micro-vasculature, but decreasing size bring…
▽ More
This paper investigates efficient techniques to collect and concentrate an under-actuated particle swarm despite obstacles. Concentrating a swarm of particles is of critical importance in health-care for targeted drug delivery, where micro-scale particles must be steered to a goal location. Individual particles must be small in order to navigate through micro-vasculature, but decreasing size brings new challenges. Individual particles are too small to contain on-board power or computation and are instead controlled by a global input, such as an applied fluidic flow or electric field.
To make progress, this paper considers a swarm of robots initialized in a grid world in which each position is either free-space or obstacle. This paper provides algorithms that collect all the robots to one position and compares these algorithms on the basis of efficiency and implementation time.
△ Less
Submitted 2 January, 2017;
originally announced January 2017.
-
A Competitive Strategy for Distance-Aware Online Shape Allocation
Authors:
Sándor P. Fekete,
Nils Schweer,
Jan-Marc Reinhardt
Abstract:
We consider the following online allocation problem: Given a unit square S, and a sequence of numbers n_i between 0 and 1, with partial sum bounded by 1; at each step i, select a region C_i of previously unassigned area n_i in S. The objective is to make these regions compact in a distance-aware sense: minimize the maximum (normalized) average Manhattan distance between points from the same set C_…
▽ More
We consider the following online allocation problem: Given a unit square S, and a sequence of numbers n_i between 0 and 1, with partial sum bounded by 1; at each step i, select a region C_i of previously unassigned area n_i in S. The objective is to make these regions compact in a distance-aware sense: minimize the maximum (normalized) average Manhattan distance between points from the same set C_i. Related location problems have received a considerable amount of attention; in particular, the problem of determining the "optimal shape of a city", i.e., allocating a single n_i has been studied. We present an online strategy, based on an analysis of space-filling curves; for continuous shapes, we prove a factor of 1.8092, and 1.7848 for discrete point sets.
△ Less
Submitted 22 April, 2013;
originally announced April 2013.
