Showing 1–10 of 10 results for author: Marcinkiewicz, M

ScaleFold: Reducing AlphaFold Initial Training Time to 10 Hours

Authors: Feiwen Zhu, Arkadiusz Nowaczynski, Rundong Li, Jie Xin, Yifei Song, Michal Marcinkiewicz, Sukru Burc Eryilmaz, Jun Yang, Michael Andersch

Abstract: AlphaFold2 has been hailed as a breakthrough in protein folding. It can rapidly predict protein structures with lab-grade accuracy. However, its implementation does not include the necessary training code. OpenFold is the first trainable public reimplementation of AlphaFold. AlphaFold training procedure is prohibitively time-consuming, and gets diminishing benefits from scaling to more compute res… ▽ More AlphaFold2 has been hailed as a breakthrough in protein folding. It can rapidly predict protein structures with lab-grade accuracy. However, its implementation does not include the necessary training code. OpenFold is the first trainable public reimplementation of AlphaFold. AlphaFold training procedure is prohibitively time-consuming, and gets diminishing benefits from scaling to more compute resources. In this work, we conducted a comprehensive analysis on the AlphaFold training procedure based on Openfold, identified that inefficient communications and overhead-dominated computations were the key factors that prevented the AlphaFold training from effective scaling. We introduced ScaleFold, a systematic training method that incorporated optimizations specifically for these factors. ScaleFold successfully scaled the AlphaFold training to 2080 NVIDIA H100 GPUs with high resource utilization. In the MLPerf HPC v3.0 benchmark, ScaleFold finished the OpenFold benchmark in 7.51 minutes, shown over $6\times$ speedup than the baseline. For training the AlphaFold model from scratch, ScaleFold completed the pretraining in 10 hours, a significant improvement over the seven days required by the original AlphaFold pretraining baseline. △ Less

Submitted 17 April, 2024; originally announced April 2024.

Optimized U-Net for Brain Tumor Segmentation

Authors: Michał Futrega, Alexandre Milesi, Michal Marcinkiewicz, Pablo Ribalta

Abstract: We propose an optimized U-Net architecture for a brain tumor segmentation task in the BraTS21 challenge. To find the optimal model architecture and the learning schedule, we have run an extensive ablation study to test: deep supervision loss, Focal loss, decoder attention, drop block, and residual connections. Additionally, we have searched for the optimal depth of the U-Net encoder, number of con… ▽ More We propose an optimized U-Net architecture for a brain tumor segmentation task in the BraTS21 challenge. To find the optimal model architecture and the learning schedule, we have run an extensive ablation study to test: deep supervision loss, Focal loss, decoder attention, drop block, and residual connections. Additionally, we have searched for the optimal depth of the U-Net encoder, number of convolutional channels and post-processing strategy. Our method won the validation phase and took third place in the test phase. We have open-sourced the code to reproduce our BraTS21 submission at the NVIDIA Deep Learning Examples GitHub Repository. △ Less

Submitted 24 December, 2021; v1 submitted 7 October, 2021; originally announced October 2021.

Comments: 15 pages, 7 figures, MICCAI submission, BraTS21 submission

Quantitative Impact of Label Noise on the Quality of Segmentation of Brain Tumors on MRI scans

Authors: Michał Marcinkiewicz, Grzegorz Mrukwa

Abstract: Over the last few years, deep learning has proven to be a great solution to many problems, such as image or text classification. Recently, deep learning-based solutions have outperformed humans on selected benchmark datasets, yielding a promising future for scientific and real-world applications. Training of deep learning models requires vast amounts of high quality data to achieve such supreme pe… ▽ More Over the last few years, deep learning has proven to be a great solution to many problems, such as image or text classification. Recently, deep learning-based solutions have outperformed humans on selected benchmark datasets, yielding a promising future for scientific and real-world applications. Training of deep learning models requires vast amounts of high quality data to achieve such supreme performance. In real-world scenarios, obtaining a large, coherent, and properly labeled dataset is a challenging task. This is especially true in medical applications, where high-quality data and annotations are scarce and the number of expert annotators is limited. In this paper, we investigate the impact of corrupted ground-truth masks on the performance of a neural network for a brain tumor segmentation task. Our findings suggest that a) the performance degrades about 8% less than it could be expected from simulations, b) a neural network learns the simulated biases of annotators, c) biases can be partially mitigated by using an inversely-biased dice loss function. △ Less

Submitted 18 September, 2019; originally announced September 2019.

Fully-automated deep learning-powered system for DCE-MRI analysis of brain tumors

Authors: Jakub Nalepa, Pablo Ribalta Lorenzo, Michal Marcinkiewicz, Barbara Bobek-Billewicz, Pawel Wawrzyniak, Maksym Walczak, Michal Kawulok, Wojciech Dudzik, Grzegorz Mrukwa, Pawel Ulrych, Michael P. Hayball

Abstract: Dynamic contrast-enhanced magnetic resonance imaging (DCE-MRI) plays an important role in diagnosis and grading of brain tumor. Although manual DCE biomarker extraction algorithms boost the diagnostic yield of DCE-MRI by providing quantitative information on tumor prognosis and prediction, they are time-consuming and prone to human error. In this paper, we propose a fully-automated, end-to-end sys… ▽ More Dynamic contrast-enhanced magnetic resonance imaging (DCE-MRI) plays an important role in diagnosis and grading of brain tumor. Although manual DCE biomarker extraction algorithms boost the diagnostic yield of DCE-MRI by providing quantitative information on tumor prognosis and prediction, they are time-consuming and prone to human error. In this paper, we propose a fully-automated, end-to-end system for DCE-MRI analysis of brain tumors. Our deep learning-powered technique does not require any user interaction, it yields reproducible results, and it is rigorously validated against benchmark (BraTS'17 for tumor segmentation, and a test dataset released by the Quantitative Imaging Biomarkers Alliance for the contrast-concentration fitting) and clinical (44 low-grade glioma patients) data. Also, we introduce a cubic model of the vascular input function used for pharmacokinetic modeling which significantly decreases the fitting error when compared with the state of the art, alongside a real-time algorithm for determination of the vascular input region. An extensive experimental study, backed up with statistical tests, showed that our system delivers state-of-the-art results (in terms of segmentation accuracy and contrast-concentration fitting) while requiring less than 3 minutes to process an entire input DCE-MRI study using a single GPU. △ Less

Submitted 18 July, 2019; originally announced July 2019.

Comments: Submitted for publication in Artificial Intelligence in Medicine

Band Selection from Hyperspectral Images Using Attention-based Convolutional Neural Networks

Authors: Pablo Ribalta Lorenzo, Lukasz Tulczyjew, Michal Marcinkiewicz, Jakub Nalepa

Abstract: This paper introduces new attention-based convolutional neural networks for selecting bands from hyperspectral images. The proposed approach re-uses convolutional activations at different depths, identifying the most informative regions of the spectrum with the help of gating mechanisms. Our attention techniques are modular and easy to implement, and they can be seamlessly trained end-to-end using… ▽ More This paper introduces new attention-based convolutional neural networks for selecting bands from hyperspectral images. The proposed approach re-uses convolutional activations at different depths, identifying the most informative regions of the spectrum with the help of gating mechanisms. Our attention techniques are modular and easy to implement, and they can be seamlessly trained end-to-end using gradient descent. Our rigorous experiments showed that deep models equipped with the attention mechanism deliver high-quality classification, and repeatedly identify significant bands in the training data, permitting the creation of refined and extremely compact sets that retain the most meaningful features. △ Less

Submitted 9 January, 2020; v1 submitted 24 October, 2018; originally announced November 2018.

Comments: This is an initial draft of the paper submitted to IEEE ACCESS

Identifying the Best Machine Learning Algorithms for Brain Tumor Segmentation, Progression Assessment, and Overall Survival Prediction in the BRATS Challenge

Authors: Spyridon Bakas, Mauricio Reyes, Andras Jakab, Stefan Bauer, Markus Rempfler, Alessandro Crimi, Russell Takeshi Shinohara, Christoph Berger, Sung Min Ha, Martin Rozycki, Marcel Prastawa, Esther Alberts, Jana Lipkova, John Freymann, Justin Kirby, Michel Bilello, Hassan Fathallah-Shaykh, Roland Wiest, Jan Kirschke, Benedikt Wiestler, Rivka Colen, Aikaterini Kotrotsou, Pamela Lamontagne, Daniel Marcus, Mikhail Milchenko , et al. (402 additional authors not shown)

Abstract: Gliomas are the most common primary brain malignancies, with different degrees of aggressiveness, variable prognosis and various heterogeneous histologic sub-regions, i.e., peritumoral edematous/invaded tissue, necrotic core, active and non-enhancing core. This intrinsic heterogeneity is also portrayed in their radio-phenotype, as their sub-regions are depicted by varying intensity profiles dissem… ▽ More Gliomas are the most common primary brain malignancies, with different degrees of aggressiveness, variable prognosis and various heterogeneous histologic sub-regions, i.e., peritumoral edematous/invaded tissue, necrotic core, active and non-enhancing core. This intrinsic heterogeneity is also portrayed in their radio-phenotype, as their sub-regions are depicted by varying intensity profiles disseminated across multi-parametric magnetic resonance imaging (mpMRI) scans, reflecting varying biological properties. Their heterogeneous shape, extent, and location are some of the factors that make these tumors difficult to resect, and in some cases inoperable. The amount of resected tumor is a factor also considered in longitudinal scans, when evaluating the apparent tumor for potential diagnosis of progression. Furthermore, there is mounting evidence that accurate segmentation of the various tumor sub-regions can offer the basis for quantitative image analysis towards prediction of patient overall survival. This study assesses the state-of-the-art machine learning (ML) methods used for brain tumor image analysis in mpMRI scans, during the last seven instances of the International Brain Tumor Segmentation (BraTS) challenge, i.e., 2012-2018. Specifically, we focus on i) evaluating segmentations of the various glioma sub-regions in pre-operative mpMRI scans, ii) assessing potential tumor progression by virtue of longitudinal growth of tumor sub-regions, beyond use of the RECIST/RANO criteria, and iii) predicting the overall survival from pre-operative mpMRI scans of patients that underwent gross total resection. Finally, we investigate the challenge of identifying the best ML algorithms for each of these tasks, considering that apart from being diverse on each instance of the challenge, the multi-institutional mpMRI BraTS dataset has also been a continuously evolving/growing dataset. △ Less

Submitted 23 April, 2019; v1 submitted 5 November, 2018; originally announced November 2018.

Comments: The International Multimodal Brain Tumor Segmentation (BraTS) Challenge

Terahertz detection of magnetic field-driven topological phase transition in HgTe-based transistors

Authors: A. Kadykov, F. Teppe, C. Consejo, L. Viti, M. Vitiello, D. Coquillat, S. Ruffenach, S. Morozov, S. Kristopenko, M. Marcinkiewicz, N. Dyakonova, W. Knap, V. Gavrilenko, N. N. Michailov, S. A. Dvoretskii

Abstract: We report on Terahertz detection by inverted band structure HgTe-based Field Effect Transistor up to room temperature. At low temperature, we show that nonlinearities of the transistor channel allows for the observation of the quantum phase transition due to the avoided crossing of zero-mode Landau levels in HgTe 2D topological insulators. These results pave the way towards Terahertz topological F… ▽ More We report on Terahertz detection by inverted band structure HgTe-based Field Effect Transistor up to room temperature. At low temperature, we show that nonlinearities of the transistor channel allows for the observation of the quantum phase transition due to the avoided crossing of zero-mode Landau levels in HgTe 2D topological insulators. These results pave the way towards Terahertz topological Field Effect Transistors. △ Less

Submitted 27 April, 2018; originally announced April 2018.

Journal ref: APPLIED PHYSICS LETTERS 107, 152101 (2015)

Temperature-driven single-valley Dirac fermions in HgTe quantum wells

Authors: M. Marcinkiewicz, S. Ruffenach, S. S. Krishtopenko, A. M. Kadykov, C. Consejo, D. B. But, W. Desrat, W. Knap, J. Torres, A. V. Ikonnikov, K. E. Spirin, S. V. Morozov, V. I. Gavrilenko, N. N. Mikhailov, S. A. Dvoretskii, F. Teppe

Abstract: We report on temperature-dependent magnetospectroscopy of two HgTe/CdHgTe quantum wells below and above the critical well thickness $d_c$. Our results, obtained in magnetic fields up to 16 T and temperature range from 2 K to 150 K, clearly indicate a change of the band-gap energy with temperature. The quantum well wider than $d_c$ evidences a temperature-driven transition from topological insulato… ▽ More We report on temperature-dependent magnetospectroscopy of two HgTe/CdHgTe quantum wells below and above the critical well thickness $d_c$. Our results, obtained in magnetic fields up to 16 T and temperature range from 2 K to 150 K, clearly indicate a change of the band-gap energy with temperature. The quantum well wider than $d_c$ evidences a temperature-driven transition from topological insulator to semiconductor phases. At the critical temperature of 90 K, the merging of inter- and intra-band transitions in weak magnetic fields clearly specifies the formation of gapless state, revealing the appearance of single-valley massless Dirac fermions with velocity of $5.6\times10^5$ m$\times$s$^{-1}$. For both quantum wells, the energies extracted from experimental data are in good agreement with calculations on the basis of the 8-band Kane Hamiltonian with temperature-dependent parameters. △ Less

Submitted 12 July, 2017; v1 submitted 22 February, 2017; originally announced February 2017.

Comments: 5 pages, 3 figures and Supplemental Materials (4 pages)

Journal ref: Phys. Rev. B 96, 035405 (2017)

Temperature-dependent magnetospectroscopy of HgTe quantum wells

Authors: A. V. Ikonnikov, S. S. Krishtopenko, O. Drachenko, M. Goiran, M. S. Zholudev, V. V. Platonov, Yu. B. Kudasov, A. S. Korshunov, D. A. Maslov, I. V. Makarov, O. M. Surdin, A. V. Philippov, M. Marcinkiewicz, S. Ruffenach, F. Teppe, W. Knap, N. N. Mikhailov, S. A. Dvoretsky, V. I. Gavrilenko

Abstract: We report on magnetospectroscopy of HgTe quantum wells in magnetic fields up to 45 T in temperature range from 4.2 K up to 185 K. We observe intra- and inter-band transitions from zero-mode Landau levels, which split from the bottom conduction and upper valence subbands, and merge under the applied magnetic field. To describe experimental results, realistic temperature-dependent calculations of La… ▽ More We report on magnetospectroscopy of HgTe quantum wells in magnetic fields up to 45 T in temperature range from 4.2 K up to 185 K. We observe intra- and inter-band transitions from zero-mode Landau levels, which split from the bottom conduction and upper valence subbands, and merge under the applied magnetic field. To describe experimental results, realistic temperature-dependent calculations of Landau levels have been performed. We show that although our samples are topological insulators at low temperatures only, the signature of such phase persists in optical transitions at high temperatures and high magnetic fields. Our results demonstrate that temperature-dependent magnetospectroscopy is a powerful tool to discriminate trivial and topological insulator phases in HgTe quantum wells. △ Less

Submitted 30 August, 2016; v1 submitted 17 June, 2016; originally announced June 2016.

Journal ref: Phys. Rev. B 94, 155421 (2016)

Temperature-driven massless Kane fermions in HgCdTe crystals: verification of universal velocity and rest-mass description

Authors: F. Teppe, M. Marcinkiewicz, S. S. Krishtopenko, S. Ruffenach, C. Consejo, A. M. Kadykov, W. Desrat, D. But, W. Knap, J. Ludwig, S. Moon, D. Smirnov, M. Orlita, Z. Jiang, S. V. Morozov, V. I. Gavrilenko, N. N. Mikhailov, S. A. Dvoretskii

Abstract: It has recently been shown that the electronic states in bulk gapless HgCdTe offer another realization of pseudo-relativistic three-dimensional particles in a condensed matter system. These single valley relativistic states, referred to as massless Kane fermions, cannot be described by any other well-known relativistic massless particles. Furthermore, the HgCdTe band structure can be continuously… ▽ More It has recently been shown that the electronic states in bulk gapless HgCdTe offer another realization of pseudo-relativistic three-dimensional particles in a condensed matter system. These single valley relativistic states, referred to as massless Kane fermions, cannot be described by any other well-known relativistic massless particles. Furthermore, the HgCdTe band structure can be continuously tailored by modifying either the cadmium content or temperature. At the critical concentration or temperature, the bandgap, Eg, collapses as the system undergoes a semimetal-to-semiconductor topological phase transition between the inverted and normal alignments. Here, using far-infrared magneto-spectroscopy we explore the continuous evolution of band structure of bulk HgCdTe as temperature is tuned across the topological phase transition. We demonstrate that the rest-mass of the Dirac-like Kane fermions, m changes sign at the critical temperature, while their velocity, c remains constant. The relation Eg = 2mc2 with the universal value of c = (1.07 +- 0.05)10x6 m/s remains valid in a broad range of temperatures and Cd concentrations, indicating a striking universality of the pseudo-relativistic description of the Dirac-like Kane fermions in HgCdTe. △ Less

Submitted 18 February, 2016; originally announced February 2016.

Comments: 15 pages, 6 figures

Journal ref: Nature Communications 7, Article number: 12576 (2016)