Showing 1–38 of 38 results for author: Deng, R

HATs: Hierarchical Adaptive Taxonomy Segmentation for Panoramic Pathology Image Analysis

Authors: Ruining Deng, Quan Liu, Can Cui, Tianyuan Yao, Juming Xiong, Shunxing Bao, Hao Li, Mengmeng Yin, Yu Wang, Shilin Zhao, Yucheng Tang, Haichun Yang, Yuankai Huo

Abstract: Panoramic image segmentation in computational pathology presents a remarkable challenge due to the morphologically complex and variably scaled anatomy. For instance, the intricate organization in kidney pathology spans multiple layers, from regions like the cortex and medulla to functional units such as glomeruli, tubules, and vessels, down to various cell types. In this paper, we propose a novel… ▽ More Panoramic image segmentation in computational pathology presents a remarkable challenge due to the morphologically complex and variably scaled anatomy. For instance, the intricate organization in kidney pathology spans multiple layers, from regions like the cortex and medulla to functional units such as glomeruli, tubules, and vessels, down to various cell types. In this paper, we propose a novel Hierarchical Adaptive Taxonomy Segmentation (HATs) method, which is designed to thoroughly segment panoramic views of kidney structures by leveraging detailed anatomical insights. Our approach entails (1) the innovative HATs technique which translates spatial relationships among 15 distinct object classes into a versatile "plug-and-play" loss function that spans across regions, functional units, and cells, (2) the incorporation of anatomical hierarchies and scale considerations into a unified simple matrix representation for all panoramic entities, (3) the adoption of the latest AI foundation model (EfficientSAM) as a feature extraction tool to boost the model's adaptability, yet eliminating the need for manual prompt generation in conventional segment anything model (SAM). Experimental findings demonstrate that the HATs method offers an efficient and effective strategy for integrating clinical insights and imaging precedents into a unified segmentation model across more than 15 categories. The official implementation is publicly available at https://github.com/hrlblab/HATs. △ Less

Submitted 30 June, 2024; originally announced July 2024.

PrPSeg: Universal Proposition Learning for Panoramic Renal Pathology Segmentation

Authors: Ruining Deng, Quan Liu, Can Cui, Tianyuan Yao, Jialin Yue, Juming Xiong, Lining Yu, Yifei Wu, Mengmeng Yin, Yu Wang, Shilin Zhao, Yucheng Tang, Haichun Yang, Yuankai Huo

Abstract: Understanding the anatomy of renal pathology is crucial for advancing disease diagnostics, treatment evaluation, and clinical research. The complex kidney system comprises various components across multiple levels, including regions (cortex, medulla), functional units (glomeruli, tubules), and cells (podocytes, mesangial cells in glomerulus). Prior studies have predominantly overlooked the intrica… ▽ More Understanding the anatomy of renal pathology is crucial for advancing disease diagnostics, treatment evaluation, and clinical research. The complex kidney system comprises various components across multiple levels, including regions (cortex, medulla), functional units (glomeruli, tubules), and cells (podocytes, mesangial cells in glomerulus). Prior studies have predominantly overlooked the intricate spatial interrelations among objects from clinical knowledge. In this research, we introduce a novel universal proposition learning approach, called panoramic renal pathology segmentation (PrPSeg), designed to segment comprehensively panoramic structures within kidney by integrating extensive knowledge of kidney anatomy. In this paper, we propose (1) the design of a comprehensive universal proposition matrix for renal pathology, facilitating the incorporation of classification and spatial relationships into the segmentation process; (2) a token-based dynamic head single network architecture, with the improvement of the partial label image segmentation and capability for future data enlargement; and (3) an anatomy loss function, quantifying the inter-object relationships across the kidney. △ Less

Submitted 20 March, 2024; v1 submitted 29 February, 2024; originally announced February 2024.

Comments: IEEE / CVF Computer Vision and Pattern Recognition Conference 2024

Multi-target Detection for Reconfigurable Holographic Surfaces Enabled Radar

Authors: Xiaoyu Zhang, Haobo Zhang, Ruoqi Deng, Liang Liu, Boya Di

Abstract: Multi-target detection is one of the primary tasks in radar-based localization and sensing, typically built on phased array antennas. However, the bulky hardware in the phased array restricts its potential for enhancing detection accuracy, since the cost and power of the phased array can become unaffordable as its physical aperture scales up to pursue higher beam sha** capabilities. To resolve t… ▽ More Multi-target detection is one of the primary tasks in radar-based localization and sensing, typically built on phased array antennas. However, the bulky hardware in the phased array restricts its potential for enhancing detection accuracy, since the cost and power of the phased array can become unaffordable as its physical aperture scales up to pursue higher beam sha** capabilities. To resolve this issue, we propose a radar system enabled by reconfigurable holographic surfaces (RHSs), a novel meta-surface antenna composed of meta-material elements with cost-effective and power-efficient hardware, which performs multi-target detection in an adaptive manner. Different from the phase-control structure in the phased array, the RHS is able to apply beamforming by controlling the radiation amplitudes of its elements. Consequently, traditional beamforming schemes designed for phased arrays cannot be directly applied to RHSs due to this structural difference. To tackle this challenge, a waveform and amplitude optimization algorithm (WAOA) is designed to jointly optimize the radar waveform and RHS amplitudes in order to improve the detection accuracy. Simulation results reveal that the proposed RHS-enabled radar increases the probability of detection by 0.13 compared to phased array radars when six iterations of adaptive detection are performed given the same hardware cost. △ Less

Submitted 17 January, 2024; originally announced January 2024.

Evaluation Kidney Layer Segmentation on Whole Slide Imaging using Convolutional Neural Networks and Transformers

Authors: Muhao Liu, Chenyang Qi, Shunxing Bao, Quan Liu, Ruining Deng, Yu Wang, Shilin Zhao, Haichun Yang, Yuankai Huo

Abstract: The segmentation of kidney layer structures, including cortex, outer stripe, inner stripe, and inner medulla within human kidney whole slide images (WSI) plays an essential role in automated image analysis in renal pathology. However, the current manual segmentation process proves labor-intensive and infeasible for handling the extensive digital pathology images encountered at a large scale. In re… ▽ More The segmentation of kidney layer structures, including cortex, outer stripe, inner stripe, and inner medulla within human kidney whole slide images (WSI) plays an essential role in automated image analysis in renal pathology. However, the current manual segmentation process proves labor-intensive and infeasible for handling the extensive digital pathology images encountered at a large scale. In response, the realm of digital renal pathology has seen the emergence of deep learning-based methodologies. However, very few, if any, deep learning based approaches have been applied to kidney layer structure segmentation. Addressing this gap, this paper assesses the feasibility of performing deep learning based approaches on kidney layer structure segmetnation. This study employs the representative convolutional neural network (CNN) and Transformer segmentation approaches, including Swin-Unet, Medical-Transformer, TransUNet, U-Net, PSPNet, and DeepLabv3+. We quantitatively evaluated six prevalent deep learning models on renal cortex layer segmentation using mice kidney WSIs. The empirical results stemming from our approach exhibit compelling advancements, as evidenced by a decent Mean Intersection over Union (mIoU) index. The results demonstrate that Transformer models generally outperform CNN-based models. By enabling a quantitative evaluation of renal cortical structures, deep learning approaches are promising to empower these medical professionals to make more informed kidney layer segmentation. △ Less

Submitted 5 September, 2023; originally announced September 2023.

Vulnerability of Machine Learning Approaches Applied in IoT-based Smart Grid: A Review

Authors: Zhenyong Zhang, Mengxiang Liu, Mingyang Sun, Ruilong Deng, Peng Cheng, Dusit Niyato, Mo-Yuen Chow, Jiming Chen

Abstract: Machine learning (ML) sees an increasing prevalence of being used in the internet-of-things (IoT)-based smart grid. However, the trustworthiness of ML is a severe issue that must be addressed to accommodate the trend of ML-based smart grid applications (MLsgAPPs). The adversarial distortion injected into the power signal will greatly affect the system's normal control and operation. Therefore, it… ▽ More Machine learning (ML) sees an increasing prevalence of being used in the internet-of-things (IoT)-based smart grid. However, the trustworthiness of ML is a severe issue that must be addressed to accommodate the trend of ML-based smart grid applications (MLsgAPPs). The adversarial distortion injected into the power signal will greatly affect the system's normal control and operation. Therefore, it is imperative to conduct vulnerability assessment for MLsgAPPs applied in the context of safety-critical power systems. In this paper, we provide a comprehensive review of the recent progress in designing attack and defense methods for MLsgAPPs. Unlike the traditional survey about ML security, this is the first review work about the security of MLsgAPPs that focuses on the characteristics of power systems. We first highlight the specifics for constructing the adversarial attacks on MLsgAPPs. Then, the vulnerability of MLsgAPP is analyzed from both the aspects of the power system and ML model. Afterward, a comprehensive survey is conducted to review and compare existing studies about the adversarial attacks on MLsgAPPs in scenarios of generation, transmission, distribution, and consumption, and the countermeasures are reviewed according to the attacks that they defend against. Finally, the future research directions are discussed on the attacker's and defender's side, respectively. We also analyze the potential vulnerability of large language model-based (e.g., ChatGPT) power system applications. Overall, we encourage more researchers to contribute to investigating the adversarial issues of MLsgAPPs. △ Less

Submitted 24 December, 2023; v1 submitted 29 August, 2023; originally announced August 2023.

Cross-modality Attention-based Multimodal Fusion for Non-small Cell Lung Cancer (NSCLC) Patient Survival Prediction

Authors: Ruining Deng, Nazim Shaikh, Gareth Shannon, Yao Nie

Abstract: Cancer prognosis and survival outcome predictions are crucial for therapeutic response estimation and for stratifying patients into various treatment groups. Medical domains concerned with cancer prognosis are abundant with multiple modalities, including pathological image data and non-image data such as genomic information. To date, multimodal learning has shown potential to enhance clinical pred… ▽ More Cancer prognosis and survival outcome predictions are crucial for therapeutic response estimation and for stratifying patients into various treatment groups. Medical domains concerned with cancer prognosis are abundant with multiple modalities, including pathological image data and non-image data such as genomic information. To date, multimodal learning has shown potential to enhance clinical prediction model performance by extracting and aggregating information from different modalities of the same subject. This approach could outperform single modality learning, thus improving computer-aided diagnosis and prognosis in numerous medical applications. In this work, we propose a cross-modality attention-based multimodal fusion pipeline designed to integrate modality-specific knowledge for patient survival prediction in non-small cell lung cancer (NSCLC). Instead of merely concatenating or summing up the features from different modalities, our method gauges the importance of each modality for feature fusion with cross-modality relationship when infusing the multimodal features. Compared with single modality, which achieved c-index of 0.5772 and 0.5885 using solely tissue image data or RNA-seq data, respectively, the proposed fusion approach achieved c-index 0.6587 in our experiment, showcasing the capability of assimilating modality-specific knowledge from varied modalities. △ Less

Submitted 27 February, 2024; v1 submitted 18 August, 2023; originally announced August 2023.

Eosinophils Instance Object Segmentation on Whole Slide Imaging Using Multi-label Circle Representation

Authors: Yilin Liu, Ruining Deng, Juming Xiong, Regina N Tyree, Hernan Correa, Girish Hiremath, Yaohong Wang, Yuankai Huo

Abstract: Eosinophilic esophagitis (EoE) is a chronic and relapsing disease characterized by esophageal inflammation. Symptoms of EoE include difficulty swallowing, food impaction, and chest pain which significantly impact the quality of life, resulting in nutritional impairments, social limitations, and psychological distress. The diagnosis of EoE is typically performed with a threshold (15 to 20) of eosin… ▽ More Eosinophilic esophagitis (EoE) is a chronic and relapsing disease characterized by esophageal inflammation. Symptoms of EoE include difficulty swallowing, food impaction, and chest pain which significantly impact the quality of life, resulting in nutritional impairments, social limitations, and psychological distress. The diagnosis of EoE is typically performed with a threshold (15 to 20) of eosinophils (Eos) per high-power field (HPF). Since the current counting process of Eos is a resource-intensive process for human pathologists, automatic methods are desired. Circle representation has been shown as a more precise, yet less complicated, representation for automatic instance cell segmentation such as CircleSnake approach. However, the CircleSnake was designed as a single-label model, which is not able to deal with multi-label scenarios. In this paper, we propose the multi-label CircleSnake model for instance segmentation on Eos. It extends the original CircleSnake model from a single-label design to a multi-label model, allowing segmentation of multiple object types. Experimental results illustrate the CircleSnake model's superiority over the traditional Mask R-CNN model and DeepSnake model in terms of average precision (AP) in identifying and segmenting eosinophils, thereby enabling enhanced characterization of EoE. This automated approach holds promise for streamlining the assessment process and improving diagnostic accuracy in EoE analysis. The source code has been made publicly available at https://github.com/yilinliu610730/EoE. △ Less

Submitted 17 August, 2023; originally announced August 2023.

Deep Learning-Based Open Source Toolkit for Eosinophil Detection in Pediatric Eosinophilic Esophagitis

Authors: Juming Xiong, Yilin Liu, Ruining Deng, Regina N Tyree, Hernan Correa, Girish Hiremath, Yaohong Wang, Yuankai Huo

Abstract: Eosinophilic Esophagitis (EoE) is a chronic, immune/antigen-mediated esophageal disease, characterized by symptoms related to esophageal dysfunction and histological evidence of eosinophil-dominant inflammation. Owing to the intricate microscopic representation of EoE in imaging, current methodologies which depend on manual identification are not only labor-intensive but also prone to inaccuracies… ▽ More Eosinophilic Esophagitis (EoE) is a chronic, immune/antigen-mediated esophageal disease, characterized by symptoms related to esophageal dysfunction and histological evidence of eosinophil-dominant inflammation. Owing to the intricate microscopic representation of EoE in imaging, current methodologies which depend on manual identification are not only labor-intensive but also prone to inaccuracies. In this study, we develop an open-source toolkit, named Open-EoE, to perform end-to-end whole slide image (WSI) level eosinophil (Eos) detection using one line of command via Docker. Specifically, the toolkit supports three state-of-the-art deep learning-based object detection models. Furthermore, Open-EoE further optimizes the performance by implementing an ensemble learning strategy, and enhancing the precision and reliability of our results. The experimental results demonstrated that the Open-EoE toolkit can efficiently detect Eos on a testing set with 289 WSIs. At the widely accepted threshold of >= 15 Eos per high power field (HPF) for diagnosing EoE, the Open-EoE achieved an accuracy of 91%, showing decent consistency with pathologist evaluations. This suggests a promising avenue for integrating machine learning methodologies into the diagnostic process for EoE. The docker and source code has been made publicly available at https://github.com/hrlblab/Open-EoE. △ Less

Submitted 11 August, 2023; originally announced August 2023.

Spatial Pathomics Toolkit for Quantitative Analysis of Podocyte Nuclei with Histology and Spatial Transcriptomics Data in Renal Pathology

Authors: Jiayuan Chen, Yu Wang, Ruining Deng, Quan Liu, Can Cui, Tianyuan Yao, Yilin Liu, Jianyong Zhong, Agnes B. Fogo, Haichun Yang, Shilin Zhao, Yuankai Huo

Abstract: Podocytes, specialized epithelial cells that envelop the glomerular capillaries, play a pivotal role in maintaining renal health. The current description and quantification of features on pathology slides are limited, prompting the need for innovative solutions to comprehensively assess diverse phenotypic attributes within Whole Slide Images (WSIs). In particular, understanding the morphological c… ▽ More Podocytes, specialized epithelial cells that envelop the glomerular capillaries, play a pivotal role in maintaining renal health. The current description and quantification of features on pathology slides are limited, prompting the need for innovative solutions to comprehensively assess diverse phenotypic attributes within Whole Slide Images (WSIs). In particular, understanding the morphological characteristics of podocytes, terminally differentiated glomerular epithelial cells, is crucial for studying glomerular injury. This paper introduces the Spatial Pathomics Toolkit (SPT) and applies it to podocyte pathomics. The SPT consists of three main components: (1) instance object segmentation, enabling precise identification of podocyte nuclei; (2) pathomics feature generation, extracting a comprehensive array of quantitative features from the identified nuclei; and (3) robust statistical analyses, facilitating a comprehensive exploration of spatial relationships between morphological and spatial transcriptomics features.The SPT successfully extracted and analyzed morphological and textural features from podocyte nuclei, revealing a multitude of podocyte morphomic features through statistical analysis. Additionally, we demonstrated the SPT's ability to unravel spatial information inherent to podocyte distribution, shedding light on spatial patterns associated with glomerular injury. By disseminating the SPT, our goal is to provide the research community with a powerful and user-friendly resource that advances cellular spatial pathomics in renal pathology. The implementation and its complete source code of the toolkit are made openly accessible at https://github.com/hrlblab/spatial_pathomics. △ Less

Submitted 10 August, 2023; originally announced August 2023.

Leverage Weakly Annotation to Pixel-wise Annotation via Zero-shot Segment Anything Model for Molecular-empowered Learning

Authors: Xueyuan Li, Ruining Deng, Yucheng Tang, Shunxing Bao, Haichun Yang, Yuankai Huo

Abstract: Precise identification of multiple cell classes in high-resolution Giga-pixel whole slide imaging (WSI) is critical for various clinical scenarios. Building an AI model for this purpose typically requires pixel-level annotations, which are often unscalable and must be done by skilled domain experts (e.g., pathologists). However, these annotations can be prone to errors, especially when distinguish… ▽ More Precise identification of multiple cell classes in high-resolution Giga-pixel whole slide imaging (WSI) is critical for various clinical scenarios. Building an AI model for this purpose typically requires pixel-level annotations, which are often unscalable and must be done by skilled domain experts (e.g., pathologists). However, these annotations can be prone to errors, especially when distinguishing between intricate cell types (e.g., podocytes and mesangial cells) using only visual inspection. Interestingly, a recent study showed that lay annotators, when using extra immunofluorescence (IF) images for reference (referred to as molecular-empowered learning), can sometimes outperform domain experts in labeling. Despite this, the resource-intensive task of manual delineation remains a necessity during the annotation process. In this paper, we explore the potential of bypassing pixel-level delineation by employing the recent segment anything model (SAM) on weak box annotation in a zero-shot learning approach. Specifically, we harness SAM's ability to produce pixel-level annotations from box annotations and utilize these SAM-generated labels to train a segmentation model. Our findings show that the proposed SAM-assisted molecular-empowered learning (SAM-L) can diminish the labeling efforts for lay annotators by only requiring weak box annotations. This is achieved without compromising annotation accuracy or the performance of the deep learning-based segmentation. This research represents a significant advancement in democratizing the annotation process for training pathological image segmentation, relying solely on non-expert annotators. △ Less

Submitted 10 August, 2023; originally announced August 2023.

High-performance Data Management for Whole Slide Image Analysis in Digital Pathology

Authors: Haoju Leng, Ruining Deng, Shunxing Bao, Dazheng Fang, Bryan A. Millis, Yucheng Tang, Haichun Yang, Xiao Wang, Yifan Peng, Lipeng Wan, Yuankai Huo

Abstract: When dealing with giga-pixel digital pathology in whole-slide imaging, a notable proportion of data records holds relevance during each analysis operation. For instance, when deploying an image analysis algorithm on whole-slide images (WSI), the computational bottleneck often lies in the input-output (I/O) system. This is particularly notable as patch-level processing introduces a considerable I/O… ▽ More When dealing with giga-pixel digital pathology in whole-slide imaging, a notable proportion of data records holds relevance during each analysis operation. For instance, when deploying an image analysis algorithm on whole-slide images (WSI), the computational bottleneck often lies in the input-output (I/O) system. This is particularly notable as patch-level processing introduces a considerable I/O load onto the computer system. However, this data management process could be further paralleled, given the typical independence of patch-level image processes across different patches. This paper details our endeavors in tackling this data access challenge by implementing the Adaptable IO System version 2 (ADIOS2). Our focus has been constructing and releasing a digital pathology-centric pipeline using ADIOS2, which facilitates streamlined data management across WSIs. Additionally, we've developed strategies aimed at curtailing data retrieval times. The performance evaluation encompasses two key scenarios: (1) a pure CPU-based image analysis scenario ("CPU scenario"), and (2) a GPU-based deep learning framework scenario ("GPU scenario"). Our findings reveal noteworthy outcomes. Under the CPU scenario, ADIOS2 showcases an impressive two-fold speed-up compared to the brute-force approach. In the GPU scenario, its performance stands on par with the cutting-edge GPU I/O acceleration framework, NVIDIA Magnum IO GPU Direct Storage (GDS). From what we know, this appears to be among the initial instances, if any, of utilizing ADIOS2 within the field of digital pathology. The source code has been made publicly available at https://github.com/hrlblab/adios. △ Less

Submitted 20 August, 2023; v1 submitted 10 August, 2023; originally announced August 2023.

Multi-scale Multi-site Renal Microvascular Structures Segmentation for Whole Slide Imaging in Renal Pathology

Authors: Franklin Hu, Ruining Deng, Shunxing Bao, Haichun Yang, Yuankai Huo

Abstract: Segmentation of microvascular structures, such as arterioles, venules, and capillaries, from human kidney whole slide images (WSI) has become a focal point in renal pathology. Current manual segmentation techniques are time-consuming and not feasible for large-scale digital pathology images. While deep learning-based methods offer a solution for automatic segmentation, most suffer from a limitatio… ▽ More Segmentation of microvascular structures, such as arterioles, venules, and capillaries, from human kidney whole slide images (WSI) has become a focal point in renal pathology. Current manual segmentation techniques are time-consuming and not feasible for large-scale digital pathology images. While deep learning-based methods offer a solution for automatic segmentation, most suffer from a limitation: they are designed for and restricted to training on single-site, single-scale data. In this paper, we present Omni-Seg, a novel single dynamic network method that capitalizes on multi-site, multi-scale training data. Unique to our approach, we utilize partially labeled images, where only one tissue type is labeled per training image, to segment microvascular structures. We train a singular deep network using images from two datasets, HuBMAP and NEPTUNE, across different magnifications (40x, 20x, 10x, and 5x). Experimental results indicate that Omni-Seg outperforms in terms of both the Dice Similarity Coefficient (DSC) and Intersection over Union (IoU). Our proposed method provides renal pathologists with a powerful computational tool for the quantitative analysis of renal microvascular structures. △ Less

Submitted 10 August, 2023; originally announced August 2023.

Democratizing Pathological Image Segmentation with Lay Annotators via Molecular-empowered Learning

Authors: Ruining Deng, Yanwei Li, Peize Li, Jiacheng Wang, Lucas W. Remedios, Saydolimkhon Agzamkhodjaev, Zuhayr Asad, Quan Liu, Can Cui, Yaohong Wang, Yihan Wang, Yucheng Tang, Haichun Yang, Yuankai Huo

Abstract: Multi-class cell segmentation in high-resolution Giga-pixel whole slide images (WSI) is critical for various clinical applications. Training such an AI model typically requires labor-intensive pixel-wise manual annotation from experienced domain experts (e.g., pathologists). Moreover, such annotation is error-prone when differentiating fine-grained cell types (e.g., podocyte and mesangial cells) v… ▽ More Multi-class cell segmentation in high-resolution Giga-pixel whole slide images (WSI) is critical for various clinical applications. Training such an AI model typically requires labor-intensive pixel-wise manual annotation from experienced domain experts (e.g., pathologists). Moreover, such annotation is error-prone when differentiating fine-grained cell types (e.g., podocyte and mesangial cells) via the naked human eye. In this study, we assess the feasibility of democratizing pathological AI deployment by only using lay annotators (annotators without medical domain knowledge). The contribution of this paper is threefold: (1) We proposed a molecular-empowered learning scheme for multi-class cell segmentation using partial labels from lay annotators; (2) The proposed method integrated Giga-pixel level molecular-morphology cross-modality registration, molecular-informed annotation, and molecular-oriented segmentation model, so as to achieve significantly superior performance via 3 lay annotators as compared with 2 experienced pathologists; (3) A deep corrective learning (learning with imperfect label) method is proposed to further improve the segmentation performance using partially annotated noisy data. From the experimental results, our learning method achieved F1 = 0.8496 using molecular-informed annotations from lay annotators, which is better than conventional morphology-based annotations (F1 = 0.7015) from experienced pathologists. Our method democratizes the development of a pathological segmentation deep model to the lay annotator level, which consequently scales up the learning process similar to a non-medical computer vision task. The official implementation and cell annotations are publicly available at https://github.com/hrlblab/MolecularEL. △ Less

Submitted 21 July, 2023; v1 submitted 31 May, 2023; originally announced June 2023.

An Accelerated Pipeline for Multi-label Renal Pathology Image Segmentation at the Whole Slide Image Level

Authors: Haoju Leng, Ruining Deng, Zuhayr Asad, R. Michael Womick, Haichun Yang, Lipeng Wan, Yuankai Huo

Abstract: Deep-learning techniques have been used widely to alleviate the labour-intensive and time-consuming manual annotation required for pixel-level tissue characterization. Our previous study introduced an efficient single dynamic network - Omni-Seg - that achieved multi-class multi-scale pathological segmentation with less computational complexity. However, the patch-wise segmentation paradigm still a… ▽ More Deep-learning techniques have been used widely to alleviate the labour-intensive and time-consuming manual annotation required for pixel-level tissue characterization. Our previous study introduced an efficient single dynamic network - Omni-Seg - that achieved multi-class multi-scale pathological segmentation with less computational complexity. However, the patch-wise segmentation paradigm still applies to Omni-Seg, and the pipeline is time-consuming when providing segmentation for Whole Slide Images (WSIs). In this paper, we propose an enhanced version of the Omni-Seg pipeline in order to reduce the repetitive computing processes and utilize a GPU to accelerate the model's prediction for both better model performance and faster speed. Our proposed method's innovative contribution is two-fold: (1) a Docker is released for an end-to-end slide-wise multi-tissue segmentation for WSIs; and (2) the pipeline is deployed on a GPU to accelerate the prediction, achieving better segmentation quality in less time. The proposed accelerated implementation reduced the average processing time (at the testing stage) on a standard needle biopsy WSI from 2.3 hours to 22 minutes, using 35 WSIs from the Kidney Tissue Atlas (KPMP) Datasets. The source code and the Docker have been made publicly available at https://github.com/ddrrnn123/Omni-Seg. △ Less

Submitted 23 May, 2023; originally announced May 2023.

An End-to-end Pipeline for 3D Slide-wise Multi-stain Renal Pathology Registration

Authors: Peize Li, Ruining Deng, Yuankai Huo

Abstract: Tissue examination and quantification in a 3D context on serial section whole slide images (WSIs) were laborintensive and time-consuming tasks. Our previous study proposed a novel registration-based method (Map3D) to automatically align WSIs to the same physical space, reducing the human efforts of screening serial sections from WSIs. However, the registration performance of our Map3D method was o… ▽ More Tissue examination and quantification in a 3D context on serial section whole slide images (WSIs) were laborintensive and time-consuming tasks. Our previous study proposed a novel registration-based method (Map3D) to automatically align WSIs to the same physical space, reducing the human efforts of screening serial sections from WSIs. However, the registration performance of our Map3D method was only evaluated on single-stain WSIs with large-scale kidney tissue samples. In this paper, we provide a Docker for an end-to-end 3D slide-wise registration pipeline on needle biopsy serial sections in a multi-stain paradigm. The contribution of this study is three-fold: (1) We release a containerized Docker for an end-to-end multi-stain WSI registration. (2) We prove that the Map3D pipeline is capable of sectional registration from multi-stain WSI. (3) We verify that the Map3D pipeline can also be applied to needle biopsy tissue samples. The source code and the Docker have been made publicly available at https://github.com/hrlblab/Map3D. △ Less

Submitted 19 May, 2023; originally announced May 2023.

Comments: 6 pages, 4 figures

Journal ref: Proceedings Volume Medical Imaging 2023: Digital and Computational Pathology, 124710F (2023)

Enhancing Cyber-Resiliency of DER-based SmartGrid: A Survey

Authors: Mengxiang Liu, Fei Teng, Zhenyong Zhang, Pudong Ge, Ruilong Deng, Mingyang Sun, Peng Cheng, Jiming Chen

Abstract: The rapid development of information and communications technology has enabled the use of digital-controlled and software-driven distributed energy resources (DERs) to improve the flexibility and efficiency of power supply, and support grid operations. However, this evolution also exposes geographically-dispersed DERs to cyber threats, including hardware and software vulnerabilities, communication… ▽ More The rapid development of information and communications technology has enabled the use of digital-controlled and software-driven distributed energy resources (DERs) to improve the flexibility and efficiency of power supply, and support grid operations. However, this evolution also exposes geographically-dispersed DERs to cyber threats, including hardware and software vulnerabilities, communication issues, and personnel errors, etc. Therefore, enhancing the cyber-resiliency of DER-based smart grid - the ability to survive successful cyber intrusions - is becoming increasingly vital and has garnered significant attention from both industry and academia. In this survey, we aim to provide a systematical and comprehensive review regarding the cyber-resiliency enhancement (CRE) of DER-based smart grid. Firstly, an integrated threat modeling method is tailored for the hierarchical DER-based smart grid with special emphasis on vulnerability identification and impact analysis. Then, the defense-in-depth strategies encompassing prevention, detection, mitigation, and recovery are comprehensively surveyed, systematically classified, and rigorously compared. A CRE framework is subsequently proposed to incorporate the five key resiliency enablers. Finally, challenges and future directions are discussed in details. The overall aim of this survey is to demonstrate the development trend of CRE methods and motivate further efforts to improve the cyber-resiliency of DER-based smart grid. △ Less

Submitted 5 March, 2024; v1 submitted 9 May, 2023; originally announced May 2023.

Comments: Accepted by IEEE Transactions on Smart Grid

Exploring shared memory architectures for end-to-end gigapixel deep learning

Authors: Lucas W. Remedios, Leon Y. Cai, Samuel W. Remedios, Karthik Ramadass, Aravind Krishnan, Ruining Deng, Can Cui, Shunxing Bao, Lori A. Coburn, Yuankai Huo, Bennett A. Landman

Abstract: Deep learning has made great strides in medical imaging, enabled by hardware advances in GPUs. One major constraint for the development of new models has been the saturation of GPU memory resources during training. This is especially true in computational pathology, where images regularly contain more than 1 billion pixels. These pathological images are traditionally divided into small patches to… ▽ More Deep learning has made great strides in medical imaging, enabled by hardware advances in GPUs. One major constraint for the development of new models has been the saturation of GPU memory resources during training. This is especially true in computational pathology, where images regularly contain more than 1 billion pixels. These pathological images are traditionally divided into small patches to enable deep learning due to hardware limitations. In this work, we explore whether the shared GPU/CPU memory architecture on the M1 Ultra systems-on-a-chip (SoCs) recently released by Apple, Inc. may provide a solution. These affordable systems (less than \$5000) provide access to 128 GB of unified memory (Mac Studio with M1 Ultra SoC). As a proof of concept for gigapixel deep learning, we identified tissue from background on gigapixel areas from whole slide images (WSIs). The model was a modified U-Net (4492 parameters) leveraging large kernels and high stride. The M1 Ultra SoC was able to train the model directly on gigapixel images (16000$\times$64000 pixels, 1.024 billion pixels) with a batch size of 1 using over 100 GB of unified memory for the process at an average speed of 1 minute and 21 seconds per batch with Tensorflow 2/Keras. As expected, the model converged with a high Dice score of 0.989 $\pm$ 0.005. Training up until this point took 111 hours and 24 minutes over 4940 steps. Other high RAM GPUs like the NVIDIA A100 (largest commercially accessible at 80 GB, $\sim$\$15000) are not yet widely available (in preview for select regions on Amazon Web Services at \$40.96/hour as a group of 8). This study is a promising step towards WSI-wise end-to-end deep learning with prevalent network architectures. △ Less

Submitted 24 April, 2023; originally announced April 2023.

Segment Anything Model (SAM) for Digital Pathology: Assess Zero-shot Segmentation on Whole Slide Imaging

Authors: Ruining Deng, Can Cui, Quan Liu, Tianyuan Yao, Lucas W. Remedios, Shunxing Bao, Bennett A. Landman, Lee E. Wheless, Lori A. Coburn, Keith T. Wilson, Yaohong Wang, Shilin Zhao, Agnes B. Fogo, Haichun Yang, Yucheng Tang, Yuankai Huo

Abstract: The segment anything model (SAM) was released as a foundation model for image segmentation. The promptable segmentation model was trained by over 1 billion masks on 11M licensed and privacy-respecting images. The model supports zero-shot image segmentation with various segmentation prompts (e.g., points, boxes, masks). It makes the SAM attractive for medical image analysis, especially for digital… ▽ More The segment anything model (SAM) was released as a foundation model for image segmentation. The promptable segmentation model was trained by over 1 billion masks on 11M licensed and privacy-respecting images. The model supports zero-shot image segmentation with various segmentation prompts (e.g., points, boxes, masks). It makes the SAM attractive for medical image analysis, especially for digital pathology where the training data are rare. In this study, we evaluate the zero-shot segmentation performance of SAM model on representative segmentation tasks on whole slide imaging (WSI), including (1) tumor segmentation, (2) non-tumor tissue segmentation, (3) cell nuclei segmentation. Core Results: The results suggest that the zero-shot SAM model achieves remarkable segmentation performance for large connected objects. However, it does not consistently achieve satisfying performance for dense instance object segmentation, even with 20 prompts (clicks/boxes) on each image. We also summarized the identified limitations for digital pathology: (1) image resolution, (2) multiple scales, (3) prompt selection, and (4) model fine-tuning. In the future, the few-shot fine-tuning with images from downstream pathological segmentation tasks might help the model to achieve better performance in dense object segmentation. △ Less

Submitted 9 April, 2023; originally announced April 2023.

Cross-scale Multi-instance Learning for Pathological Image Diagnosis

Authors: Ruining Deng, Can Cui, Lucas W. Remedios, Shunxing Bao, R. Michael Womick, Sophie Chiron, Jia Li, Joseph T. Roland, Ken S. Lau, Qi Liu, Keith T. Wilson, Yaohong Wang, Lori A. Coburn, Bennett A. Landman, Yuankai Huo

Abstract: Analyzing high resolution whole slide images (WSIs) with regard to information across multiple scales poses a significant challenge in digital pathology. Multi-instance learning (MIL) is a common solution for working with high resolution images by classifying bags of objects (i.e. sets of smaller image patches). However, such processing is typically performed at a single scale (e.g., 20x magnifica… ▽ More Analyzing high resolution whole slide images (WSIs) with regard to information across multiple scales poses a significant challenge in digital pathology. Multi-instance learning (MIL) is a common solution for working with high resolution images by classifying bags of objects (i.e. sets of smaller image patches). However, such processing is typically performed at a single scale (e.g., 20x magnification) of WSIs, disregarding the vital inter-scale information that is key to diagnoses by human pathologists. In this study, we propose a novel cross-scale MIL algorithm to explicitly aggregate inter-scale relationships into a single MIL network for pathological image diagnosis. The contribution of this paper is three-fold: (1) A novel cross-scale MIL (CS-MIL) algorithm that integrates the multi-scale information and the inter-scale relationships is proposed; (2) A toy dataset with scale-specific morphological features is created and released to examine and visualize differential cross-scale attention; (3) Superior performance on both in-house and public datasets is demonstrated by our simple cross-scale MIL strategy. The official implementation is publicly available at https://github.com/hrlblab/CS-MIL. △ Less

Submitted 16 February, 2024; v1 submitted 31 March, 2023; originally announced April 2023.

Reconfigurable Holographic Surface: A New Paradigm to Implement Holographic Radio

Authors: Ruoqi Deng, Yutong Zhang, Haobo Zhang, Boya Di, Hongliang Zhang, Lingyang Song

Abstract: Ultra-massive multiple-input multiple-output (MIMO) is one of the key enablers in the forthcoming 6G networks to provide high-speed data services by exploiting spatial diversity. In this article, we consider a new paradigm termed holographic radio for ultra-massive MIMO, where numerous tiny and inexpensive antenna elements are integrated to realize high directive gain with low hardware cost. We pr… ▽ More Ultra-massive multiple-input multiple-output (MIMO) is one of the key enablers in the forthcoming 6G networks to provide high-speed data services by exploiting spatial diversity. In this article, we consider a new paradigm termed holographic radio for ultra-massive MIMO, where numerous tiny and inexpensive antenna elements are integrated to realize high directive gain with low hardware cost. We propose a practical way to enable holographic radio by a novel metasurface-based antenna, i.e., reconfigurable holographic surface (RHS). Specifically, RHSs incorporating densely packed tunable metamaterial elements are capable of holographic beamforming. Based on the working principle and hardware design of RHSs, we conduct full-wave analyses of RHSs and build an RHS-aided point-to-point communication platform supporting real-time data transmission. Both simulated and experimental results show that the RHS has great potential to achieve high directive gain with a limited size, thereby substantiating the feasibility of RHS-enabled holographic radio. Moreover, future research directions for RHS-enabled holographic radio are also discussed. △ Less

Submitted 27 December, 2022; originally announced December 2022.

Comments: 7 pages, 7 figures

Remote Photoplethysmography from Low Resolution videos: An end-to-end solution using Efficient ConvNets

Authors: Bharath Ramakrishnan, Ruijia Deng

Abstract: Measurement of the cardiac pulse from facial video has become an interesting pursuit of research over the last few years. This is mainly due to the increasing importance of obtaining the heart rate of an individual in a non-invasive manner, which can be highly useful for applications in gaming and the medical industry. Another instrumental area of research over the past few years has been the adve… ▽ More Measurement of the cardiac pulse from facial video has become an interesting pursuit of research over the last few years. This is mainly due to the increasing importance of obtaining the heart rate of an individual in a non-invasive manner, which can be highly useful for applications in gaming and the medical industry. Another instrumental area of research over the past few years has been the advent of Deep Learning and using Deep Neural networks to enhance task performance. In this work, we propose to use efficient convolutional networks to accurately measure the heart rate of user from low resolution facial videos. Furthermore, to ensure that we are able to obtain the heart rate in real time, we compress the deep learning model by pruning it, thereby reducing its memory footprint. We benchmark the performance of our approach on the MAHNOB dataset and compare its performance across multiple approaches. △ Less

Submitted 14 August, 2022; originally announced August 2022.

Comments: arXiv admin note: text overlap with arXiv:2208.04947

Visual Heart Rate Estimation from RGB Facial Video using Spectral Reflectance

Authors: Bharath Ramakrishnan, Ruijia Deng, Hassan Ali

Abstract: Estimation of the Heart rate from the facial video has a number of applications in the medical and fitness industries. Additionally, it has become useful in the field of gaming as well. Several approaches have been proposed to seamlessly obtain the Heart rate from the facial video, but these approaches have had issues in dealing with motion and illumination artifacts. In this work, we propose a re… ▽ More Estimation of the Heart rate from the facial video has a number of applications in the medical and fitness industries. Additionally, it has become useful in the field of gaming as well. Several approaches have been proposed to seamlessly obtain the Heart rate from the facial video, but these approaches have had issues in dealing with motion and illumination artifacts. In this work, we propose a reliable HR estimation framework using the spectral reflectance of the user, which makes it robust to motion and illumination disturbances. We employ deep learning-based frameworks such as Faster RCNNs to perform face detection as opposed to the Viola Jones algorithm employed by previous approaches. We evaluate our method on the MAHNOB HCI dataset and found that the proposed method is able to outperform previous approaches.Estimation of the Heart rate from facial video has a number of applications in the medical and the fitness industries. Additionally, it has become useful in the field of gaming as well. Several approaches have been proposed to seamlessly obtain the Heart rate from the facial video, but these approaches have had issues in dealing with motion and illumination artifacts. In this work, we propose a reliable HR estimation framework using the spectral reflectance of the user, which makes it robust to motion and illumination disturbances. We employ deep learning-based frameworks such as Faster RCNNs to perform face detection as opposed to the Viola-Jones algorithm employed by previous approaches. We evaluate our method on the MAHNOB HCI dataset and found that the proposed method is able to outperform previous approaches. △ Less

Submitted 9 August, 2022; originally announced August 2022.

Comments: Submitted as a student abstract to AAAI 2023

Omni-Seg: A Scale-aware Dynamic Network for Renal Pathological Image Segmentation

Authors: Ruining Deng, Quan Liu, Can Cui, Tianyuan Yao, Jun Long, Zuhayr Asad, R. Michael Womick, Zheyu Zhu, Agnes B. Fogo, Shilin Zhao, Haichun Yang, Yuankai Huo

Abstract: Comprehensive semantic segmentation on renal pathological images is challenging due to the heterogeneous scales of the objects. For example, on a whole slide image (WSI), the cross-sectional areas of glomeruli can be 64 times larger than that of the peritubular capillaries, making it impractical to segment both objects on the same patch, at the same scale. To handle this scaling issue, prior studi… ▽ More Comprehensive semantic segmentation on renal pathological images is challenging due to the heterogeneous scales of the objects. For example, on a whole slide image (WSI), the cross-sectional areas of glomeruli can be 64 times larger than that of the peritubular capillaries, making it impractical to segment both objects on the same patch, at the same scale. To handle this scaling issue, prior studies have typically trained multiple segmentation networks in order to match the optimal pixel resolution of heterogeneous tissue types. This multi-network solution is resource-intensive and fails to model the spatial relationship between tissue types. In this paper, we propose the Omni-Seg+ network, a scale-aware dynamic neural network that achieves multi-object (six tissue types) and multi-scale (5X to 40X scale) pathological image segmentation via a single neural network. The contribution of this paper is three-fold: (1) a novel scale-aware controller is proposed to generalize the dynamic neural network from single-scale to multi-scale; (2) semi-supervised consistency regularization of pseudo-labels is introduced to model the inter-scale correlation of unannotated tissue types into a single end-to-end learning paradigm; and (3) superior scale-aware generalization is evidenced by directly applying a model trained on human kidney images to mouse kidney images, without retraining. By learning from ~150,000 human pathological image patches from six tissue types at three different resolutions, our approach achieved superior segmentation performance according to human visual assessment and evaluation of image-omics (i.e., spatial transcriptomics). The official implementation is available at https://github.com/ddrrnn123/Omni-Seg. △ Less

Submitted 18 January, 2023; v1 submitted 27 June, 2022; originally announced June 2022.

Omni-Seg: A Single Dynamic Network for Multi-label Renal Pathology Image Segmentation using Partially Labeled Data

Authors: Ruining Deng, Quan Liu, Can Cui, Zuhayr Asad, Haichun Yang, Yuankai Huo

Abstract: Computer-assisted quantitative analysis on Giga-pixel pathology images has provided a new avenue in histology examination. The innovations have been largely focused on cancer pathology (i.e., tumor segmentation and characterization). In non-cancer pathology, the learning algorithms can be asked to examine more comprehensive tissue types simultaneously, as a multi-label setting. The prior arts typi… ▽ More Computer-assisted quantitative analysis on Giga-pixel pathology images has provided a new avenue in histology examination. The innovations have been largely focused on cancer pathology (i.e., tumor segmentation and characterization). In non-cancer pathology, the learning algorithms can be asked to examine more comprehensive tissue types simultaneously, as a multi-label setting. The prior arts typically needed to train multiple segmentation networks in order to match the domain-specific knowledge for heterogeneous tissue types (e.g., glomerular tuft, glomerular unit, proximal tubular, distal tubular, peritubular capillaries, and arteries). In this paper, we propose a dynamic single segmentation network (Omni-Seg) that learns to segment multiple tissue types using partially labeled images (i.e., only one tissue type is labeled for each training image) for renal pathology. By learning from ~150,000 patch-wise pathological images from six tissue types, the proposed Omni-Seg network achieved superior segmentation accuracy and less resource consumption when compared to the previous the multiple-network and multi-head design. In the testing stage, the proposed method obtains "completely labeled" tissue segmentation results using only "partially labeled" training images. The source code is available at https://github.com/ddrrnn123/Omni-Seg △ Less

Submitted 23 March, 2022; v1 submitted 23 December, 2021; originally announced December 2021.

Reconfigurable Holographic Surfaces for Future Wireless Communications

Authors: Ruoqi Deng, Boya Di, Hongliang Zhang, Dusit Niyato, Zhu Han, H. Vincent Poor, Lingyang Song

Abstract: Future wireless communications look forward to constructing a ubiquitous intelligent information network with high data rates through cost-efficient devices. Benefiting from the tunability and programmability of metamaterials, the reconfigurable holographic surface (RHS) composed of numerous metamaterial radiation elements is developed as a promising solution to fulfill such challenging visions. T… ▽ More Future wireless communications look forward to constructing a ubiquitous intelligent information network with high data rates through cost-efficient devices. Benefiting from the tunability and programmability of metamaterials, the reconfigurable holographic surface (RHS) composed of numerous metamaterial radiation elements is developed as a promising solution to fulfill such challenging visions. The RHS is more likely to serve as an ultra-thin and lightweight surface antenna integrated with the transceiver to generate beams with desirable directions by leveraging the holographic principle. This is different from reconfigurable intelligent surfaces (RISs) widely used as passive relays due to the reflection characteristic. In this article, we investigate RHS-aided wireless communications. Starting with a basic introduction of the RHS including its hardware structure, holographic principle, and fabrication methodologies, we propose a hybrid beamforming scheme for RHS-aided multi-user communication systems. A joint sum-rate maximization algorithm is then developed where the digital beamforming performed at the base station and the holographic beamforming performed at the RHS are optimized iteratively. Furthermore, key challenges in RHS-aided wireless communications are also discussed. △ Less

Submitted 12 December, 2021; originally announced December 2021.

Comments: 6 pages, 6 figures

Integrated Satellite-HAP-Terrestrial Networks for Dual-Band Connectivity

Authors: Wenwei Zhang, Ruoqi Deng, Boya Di, Lingyang Song

Abstract: The recent development of high-altitude platforms (HAPs) has attracted increasing attention since they can serve as a promising communication method to assist satellite-terrestrial networks. In this paper, we consider an integrated three-layer satellite-HAP-terrestrial network where the HAP support dual-band connectivity. Specifically, the HAP can not only communicate with terrestrial users over C… ▽ More The recent development of high-altitude platforms (HAPs) has attracted increasing attention since they can serve as a promising communication method to assist satellite-terrestrial networks. In this paper, we consider an integrated three-layer satellite-HAP-terrestrial network where the HAP support dual-band connectivity. Specifically, the HAP can not only communicate with terrestrial users over C-band directly, but also provide backhaul services to terrestrial user terminals over Ka-band. We formulate a sum-rate maximization problem and then propose a fractional programming based algorithm to solve the problem by optimizing the bandwidth and power allocation iteratively. The closed-form optimal solutions for bandwidth allocation and power allocation in each iteration are also derived. Simulation results show the capacity enhancement brought by the dual-band connectivity of the HAP. The influence of the power of the HAP and the power of the satellite is also discussed. △ Less

Submitted 5 July, 2021; originally announced July 2021.

VoxelEmbed: 3D Instance Segmentation and Tracking with Voxel Embedding based Deep Learning

Authors: Mengyang Zhao, Quan Liu, Aadarsh Jha, Ruining Deng, Tianyuan Yao, Anita Mahadevan-Jansen, Matthew J. Tyska, Bryan A. Millis, Yuankai Huo

Abstract: Recent advances in bioimaging have provided scientists a superior high spatial-temporal resolution to observe dynamics of living cells as 3D volumetric videos. Unfortunately, the 3D biomedical video analysis is lagging, impeded by resource insensitive human curation using off-the-shelf 3D analytic tools. Herein, biologists often need to discard a considerable amount of rich 3D spatial information… ▽ More Recent advances in bioimaging have provided scientists a superior high spatial-temporal resolution to observe dynamics of living cells as 3D volumetric videos. Unfortunately, the 3D biomedical video analysis is lagging, impeded by resource insensitive human curation using off-the-shelf 3D analytic tools. Herein, biologists often need to discard a considerable amount of rich 3D spatial information by compromising on 2D analysis via maximum intensity projection. Recently, pixel embedding-based cell instance segmentation and tracking provided a neat and generalizable computing paradigm for understanding cellular dynamics. In this work, we propose a novel spatial-temporal voxel-embedding (VoxelEmbed) based learning method to perform simultaneous cell instance segmenting and tracking on 3D volumetric video sequences. Our contribution is in four-fold: (1) The proposed voxel embedding generalizes the pixel embedding with 3D context information; (2) Present a simple multi-stream learning approach that allows effective spatial-temporal embedding; (3) Accomplished an end-to-end framework for one-stage 3D cell instance segmentation and tracking without heavy parameter tuning; (4) The proposed 3D quantification is memory efficient via a single GPU with 12 GB memory. We evaluate our VoxelEmbed method on four 3D datasets (with different cell types) from the ISBI Cell Tracking Challenge. The proposed VoxelEmbed method achieved consistent superior overall performance (OP) on two densely annotated datasets. The performance is also competitive on two sparsely annotated cohorts with 20.6% and 2% of data-set having segmentation annotations. The results demonstrate that the VoxelEmbed method is a generalizable and memory-efficient solution. △ Less

Submitted 21 June, 2021; originally announced June 2021.

Detecting False Data Injection Attacks in Smart Grids with Modeling Errors: A Deep Transfer Learning Based Approach

Authors: Bowen Xu, Fanghong Guo, Changyun Wen, Ruilong Deng, Wen-An Zhang

Abstract: Most traditional false data injection attack (FDIA) detection approaches rely on a key assumption, i.e., the power system can be accurately modeled. However, the transmission line parameters are dynamic and cannot be accurately known during operation and thus the involved modeling errors should not be neglected. In this paper, an illustrative case has revealed that modeling errors in transmission… ▽ More Most traditional false data injection attack (FDIA) detection approaches rely on a key assumption, i.e., the power system can be accurately modeled. However, the transmission line parameters are dynamic and cannot be accurately known during operation and thus the involved modeling errors should not be neglected. In this paper, an illustrative case has revealed that modeling errors in transmission lines significantly weaken the detection effectiveness of conventional FDIA approaches. To tackle this issue, we propose an FDIA detection mechanism from the perspective of transfer learning. Specifically, the simulated power system is treated as a source domain, which provides abundant simulated normal and attack data. The real world's running system whose transmission line parameters are unknown is taken as a target domain where sufficient real normal data are collected for tracking the latest system states online. The designed transfer strategy that aims at making full use of data in hand is divided into two optimization stages. In the first stage, a deep neural network (DNN) is built by simultaneously optimizing several well-designed objective terms with both simulated data and real data, and then it is fine-tuned via real data in the second stage. Several case studies on the IEEE 14-bus and 118-bus systems verify the effectiveness of the proposed mechanism. △ Less

Submitted 7 September, 2021; v1 submitted 9 April, 2021; originally announced April 2021.

Comments: 9 pages, 4 figures

Improve Global Glomerulosclerosis Classification with Imbalanced Data using CircleMix Augmentation

Authors: Yuzhe Lu, Haichun Yang, Zheyu Zhu, Ruining Deng, Agnes B. Fogo, Yuankai Huo

Abstract: The classification of glomerular lesions is a routine and essential task in renal pathology. Recently, machine learning approaches, especially deep learning algorithms, have been used to perform computer-aided lesion characterization of glomeruli. However, one major challenge of develo** such methods is the naturally imbalanced distribution of different lesions. In this paper, we propose CircleM… ▽ More The classification of glomerular lesions is a routine and essential task in renal pathology. Recently, machine learning approaches, especially deep learning algorithms, have been used to perform computer-aided lesion characterization of glomeruli. However, one major challenge of develo** such methods is the naturally imbalanced distribution of different lesions. In this paper, we propose CircleMix, a novel data augmentation technique, to improve the accuracy of classifying globally sclerotic glomeruli with a hierarchical learning strategy. Different from the recently proposed CutMix method, the CircleMix augmentation is optimized for the ball-shaped biomedical objects, such as glomeruli. 6,861 glomeruli with five classes (normal, periglomerular fibrosis, obsolescent glomerulosclerosis, solidified glomerulosclerosis, and disappearing glomerulosclerosis) were employed to develop and evaluate the proposed methods. From five-fold cross-validation, the proposed CircleMix augmentation achieved superior performance (Balanced Accuracy=73.0%) compared with the EfficientNet-B0 baseline (Balanced Accuracy=69.4%) △ Less

Submitted 16 January, 2021; originally announced January 2021.

ASIST: Annotation-free Synthetic Instance Segmentation and Tracking by Adversarial Simulations

Authors: Quan Liu, Isabella M. Gaeta, Mengyang Zhao, Ruining Deng, Aadarsh Jha, Bryan A. Millis, Anita Mahadevan-Jansen, Matthew J. Tyska, Yuankai Huo

Abstract: Background: The quantitative analysis of microscope videos often requires instance segmentation and tracking of cellular and subcellular objects. The traditional method consists of two stages: (1) performing instance object segmentation of each frame, and (2) associating objects frame-by-frame. Recently, pixel-embedding-based deep learning approaches these two steps simultaneously as a single stag… ▽ More Background: The quantitative analysis of microscope videos often requires instance segmentation and tracking of cellular and subcellular objects. The traditional method consists of two stages: (1) performing instance object segmentation of each frame, and (2) associating objects frame-by-frame. Recently, pixel-embedding-based deep learning approaches these two steps simultaneously as a single stage holistic solution. In computer vision, annotated training data with consistent segmentation and tracking is resource intensive, the severity of which is multiplied in microscopy imaging due to (1) dense objects (e.g., overlap** or touching), and (2) high dynamics (e.g., irregular motion and mitosis). Adversarial simulations have provided successful solutions to alleviate the lack of such annotations in dynamics scenes in computer vision, such as using simulated environments (e.g., computer games) to train real-world self-driving systems. Methods: In this paper, we propose an annotation-free synthetic instance segmentation and tracking (ASIST) method with adversarial simulation and single-stage pixel-embedding based learning. Contribution: The contribution of this paper is three-fold: (1) the proposed method aggregates adversarial simulations and single-stage pixel-embedding based deep learning; (2) the method is assessed with both the cellular (i.e., HeLa cells) and subcellular (i.e., microvilli) objects; and (3) to the best of our knowledge, this is the first study to explore annotation-free instance segmentation and tracking study for microscope videos. Results: The ASIST method achieved an important step forward, when compared with fully supervised approaches: ASIST shows 7% to 11% higher segmentation, detection and tracking performance on microvilli relative to fully supervised methods, and comparable performance on Hela cell videos. △ Less

Submitted 21 May, 2021; v1 submitted 3 January, 2021; originally announced January 2021.

ASIST: Annotation-free synthetic instance segmentation and tracking for microscope video analysis

Authors: Quan Liu, Isabella M. Gaeta, Mengyang Zhao, Ruining Deng, Aadarsh Jha, Bryan A. Millis, Anita Mahadevan-Jansen, Matthew J. Tyska, Yuankai Huo

Abstract: Instance object segmentation and tracking provide comprehensive quantification of objects across microscope videos. The recent single-stage pixel-embedding based deep learning approach has shown its superior performance compared with "segment-then-associate" two-stage solutions. However, one major limitation of applying a supervised pixel-embedding based method to microscope videos is the resource… ▽ More Instance object segmentation and tracking provide comprehensive quantification of objects across microscope videos. The recent single-stage pixel-embedding based deep learning approach has shown its superior performance compared with "segment-then-associate" two-stage solutions. However, one major limitation of applying a supervised pixel-embedding based method to microscope videos is the resource-intensive manual labeling, which involves tracing hundreds of overlapped objects with their temporal associations across video frames. Inspired by the recent generative adversarial network (GAN) based annotation-free image segmentation, we propose a novel annotation-free synthetic instance segmentation and tracking (ASIST) algorithm for analyzing microscope videos of sub-cellular microvilli. The contributions of this paper are three-fold: (1) proposing a new annotation-free video analysis paradigm is proposed. (2) aggregating the embedding based instance segmentation and tracking with annotation-free synthetic learning as a holistic framework; and (3) to the best of our knowledge, this is first study to investigate microvilli instance segmentation and tracking using embedding based deep learning. From the experimental results, the proposed annotation-free method achieved superior performance compared with supervised learning. △ Less

Submitted 2 November, 2020; originally announced November 2020.

EasierPath: An Open-source Tool for Human-in-the-loop Deep Learning of Renal Pathology

Authors: Zheyu Zhu, Yuzhe Lu, Ruining Deng, Haichun Yang, Agnes B. Fogo, Yuankai Huo

Abstract: Considerable morphological phenoty** studies in nephrology have emerged in the past few years, aiming to discover hidden regularities between clinical and imaging phenotypes. Such studies have been largely enabled by deep learning based image analysis to extract sparsely located targeting objects (e.g., glomeruli) on high-resolution whole slide images (WSI). However, such methods need to be trai… ▽ More Considerable morphological phenoty** studies in nephrology have emerged in the past few years, aiming to discover hidden regularities between clinical and imaging phenotypes. Such studies have been largely enabled by deep learning based image analysis to extract sparsely located targeting objects (e.g., glomeruli) on high-resolution whole slide images (WSI). However, such methods need to be trained using labor-intensive high-quality annotations, ideally labeled by pathologists. Inspired by the recent "human-in-the-loop" strategy, we developed EasierPath, an open-source tool to integrate human physicians and deep learning algorithms for efficient large-scale pathological image quantification as a loop. Using EasierPath, physicians are able to (1) optimize the recall and precision of deep learning object detection outcomes adaptively, (2) seamlessly support deep learning outcomes refining using either our EasierPath or prevalent ImageScope software without changing physician's user habit, and (3) manage and phenotype each object with user-defined classes. As a user case of EasierPath, we present the procedure of curating large-scale glomeruli in an efficient human-in-the-loop fashion (with two loops). From the experiments, the EasierPath saved 57 % of the annotation efforts to curate 8,833 glomeruli during the second loop. Meanwhile, the average precision of glomerular detection was leveraged from 0.504 to 0.620. The EasierPath software has been released as open-source to enable the large-scale glomerular prototy**. The code can be found in https://github.com/yuankaihuo/EasierPath △ Less

Submitted 27 July, 2020; originally announced July 2020.

Instance Segmentation for Whole Slide Imaging: End-to-End or Detect-Then-Segment

Authors: Aadarsh Jha, Haichun Yang, Ruining Deng, Meghan E. Kapp, Agnes B. Fogo, Yuankai Huo

Abstract: Automatic instance segmentation of glomeruli within kidney Whole Slide Imaging (WSI) is essential for clinical research in renal pathology. In computer vision, the end-to-end instance segmentation methods (e.g., Mask-RCNN) have shown their advantages relative to detect-then-segment approaches by performing complementary detection and segmentation tasks simultaneously. As a result, the end-to-end M… ▽ More Automatic instance segmentation of glomeruli within kidney Whole Slide Imaging (WSI) is essential for clinical research in renal pathology. In computer vision, the end-to-end instance segmentation methods (e.g., Mask-RCNN) have shown their advantages relative to detect-then-segment approaches by performing complementary detection and segmentation tasks simultaneously. As a result, the end-to-end Mask-RCNN approach has been the de facto standard method in recent glomerular segmentation studies, where downsampling and patch-based techniques are used to properly evaluate the high resolution images from WSI (e.g., >10,000x10,000 pixels on 40x). However, in high resolution WSI, a single glomerulus itself can be more than 1,000x1,000 pixels in original resolution which yields significant information loss when the corresponding features maps are downsampled via the Mask-RCNN pipeline. In this paper, we assess if the end-to-end instance segmentation framework is optimal for high-resolution WSI objects by comparing Mask-RCNN with our proposed detect-then-segment framework. Beyond such a comparison, we also comprehensively evaluate the performance of our detect-then-segment pipeline through: 1) two of the most prevalent segmentation backbones (U-Net and DeepLab_v3); 2) six different image resolutions (from 512x512 to 28x28); and 3) two different color spaces (RGB and LAB). Our detect-then-segment pipeline, with the DeepLab_v3 segmentation framework operating on previously detected glomeruli of 512x512 resolution, achieved a 0.953 dice similarity coefficient (DSC), compared with a 0.902 DSC from the end-to-end Mask-RCNN pipeline. Further, we found that neither RGB nor LAB color spaces yield better performance when compared against each other in the context of a detect-then-segment framework. Detect-then-segment pipeline achieved better segmentation performance compared with End-to-end method. △ Less

Submitted 7 July, 2020; originally announced July 2020.

Deep Reinforcement Learning Based Massive Access Management for Ultra-Reliable Low-Latency Communications

Authors: Helin Yang, Zehui Xiong, Jun Zhao, Dusit Niyato, Chau Yuen, Ruilong Deng

Abstract: With the rapid deployment of the Internet of Things (IoT), fifth-generation (5G) and beyond 5G networks are required to support massive access of a huge number of devices over limited radio spectrum radio. In wireless networks, different devices have various quality-of-service (QoS) requirements, ranging from ultra-reliable low latency communications (URLLC) to high transmission data rates. In thi… ▽ More With the rapid deployment of the Internet of Things (IoT), fifth-generation (5G) and beyond 5G networks are required to support massive access of a huge number of devices over limited radio spectrum radio. In wireless networks, different devices have various quality-of-service (QoS) requirements, ranging from ultra-reliable low latency communications (URLLC) to high transmission data rates. In this context, we present a joint energy-efficient subchannel assignment and power control approach to manage massive access requests while maximizing network energy efficiency (EE) and guaranteeing different QoS requirements. The latency constraint is transformed into a data rate constraint which makes the optimization problem tractable before modelling it as a multi-agent reinforcement learning problem. A distributed cooperative massive access approach based on deep reinforcement learning (DRL) is proposed to address the problem while meeting both reliability and latency constraints on URLLC services in massive access scenario. In addition, transfer learning and cooperative learning mechanisms are employed to enable communication links to work cooperatively in a distributed manner, which enhances the network performance and access success probability. Simulation results clearly show that the proposed distributed cooperative learning approach outperforms other existing approaches in terms of meeting EE and improving the transmission success probability in massive access scenario. △ Less

Submitted 17 December, 2020; v1 submitted 20 February, 2020; originally announced February 2020.

Comments: This paper appears in IEEE Transactions on Wireless Communications. Please feel free to contact us for questions or remarks. http://JunZhao.info/

False Data Injection Attacks and the Distributed Countermeasure in DC Microgrids

Authors: Mengxiang Liu, Peng Cheng, Chengcheng Zhao, Ruilong Deng, Wenhai Wang, Jiming Chen

Abstract: In this paper, we consider a hierarchical control based DC microgrid (DCmG) equipped with unknown input observer (UIO) based detectors, where the potential false data injection (FDI) attacks and the distributed countermeasure are investigated. First, we find that the vulnerability of the UIO-based detector originates from the lacked knowledge of true unknown inputs. Zero trace stealthy (ZTS) attac… ▽ More In this paper, we consider a hierarchical control based DC microgrid (DCmG) equipped with unknown input observer (UIO) based detectors, where the potential false data injection (FDI) attacks and the distributed countermeasure are investigated. First, we find that the vulnerability of the UIO-based detector originates from the lacked knowledge of true unknown inputs. Zero trace stealthy (ZTS) attacks can be launched by secretly faking the unknown inputs, under which the detection residual will not be altered, and the impact on the DCmG in terms of voltage balancing and current sharing is theoretically analyzed. Then, to mitigate the ZTS attack, we propose an automatic and timely countermeasure based on the average point of common coupling (PCC) voltage obtained from the dynamic average consensus (DAC) estimator. The integrity of the communicated data utilized in DAC estimators is guaranteed via UIO-based detectors, where the DAC parameters are perturbed in a fixed period to be concealed from attackers. Finally, the detection and mitigation performance of the proposed countermeasure is rigorously investigated, and extensive simulations are conducted in Simulink/PLECS to validate the theoretical results. △ Less

Submitted 28 July, 2021; v1 submitted 7 January, 2020; originally announced January 2020.

Channel Fingerprint Based Beam Tracking for Millimeter Wave Communications

Authors: Ruichen Deng, Sheng Chen, Sheng Zhou, Zhisheng Niu, Wei Zhang

Abstract: Beamforming structures with fixed beam codebooks provide economical solutions for millimeter wave (mmWave) communications due to the low hardware cost. However, the training overhead to search for the optimal beamforming configuration is proportional to the codebook size. To improve the efficiency of beam tracking, we propose a beam tracking scheme based on the channel fingerprint database, which… ▽ More Beamforming structures with fixed beam codebooks provide economical solutions for millimeter wave (mmWave) communications due to the low hardware cost. However, the training overhead to search for the optimal beamforming configuration is proportional to the codebook size. To improve the efficiency of beam tracking, we propose a beam tracking scheme based on the channel fingerprint database, which comprises map**s between statistical beamforming gains and user locations. The scheme tracks user movement by utilizing the trained beam configurations and estimating the gains of beam configurations that are not trained. Simulations show that the proposed scheme achieves significant beamforming performance gains over existing beam tracking schemes. △ Less

Submitted 24 December, 2019; originally announced December 2019.

Comments: 5 pages, 4 figures, accepted by IEEE Communications Letters

Analysis of Moving Target Defense Against False Data Injection Attacks on Power Grid

Authors: Zhenyong Zhang, Ruilong Deng, David K. Y. Yau, Peng Cheng, Jiming Chen

Abstract: Recent studies have considered thwarting false data injection (FDI) attacks against state estimation in power grids by proactively perturbing branch susceptances. This approach is known as moving target defense (MTD). However, despite of the deployment of MTD, it is still possible for the attacker to launch stealthy FDI attacks generated with former branch susceptances. In this paper, we prove tha… ▽ More Recent studies have considered thwarting false data injection (FDI) attacks against state estimation in power grids by proactively perturbing branch susceptances. This approach is known as moving target defense (MTD). However, despite of the deployment of MTD, it is still possible for the attacker to launch stealthy FDI attacks generated with former branch susceptances. In this paper, we prove that, an MTD has the capability to thwart all FDI attacks constructed with former branch susceptances only if (i) the number of branches $l$ in the power system is not less than twice that of the system states $n$ (i.e., $l \geq 2n$, where $n + 1$ is the number of buses); (ii) the susceptances of more than $n$ branches, which cover all buses, are perturbed. Moreover, we prove that the state variable of a bus that is only connected by a single branch (no matter it is perturbed or not) can always be modified by the attacker. Nevertheless, in order to reduce the attack opportunities of potential attackers, we first exploit the impact of the susceptance perturbation magnitude on the dimension of the \emph{stealthy attack space}, in which the attack vector is constructed with former branch susceptances. Then, we propose that, by perturbing an appropriate set of branches, we can minimize the dimension of the \emph{stealthy attack space} and maximize the number of covered buses. Besides, we consider the increasing operation cost caused by the activation of MTD. Finally, we conduct extensive simulations to illustrate our findings with IEEE standard test power systems. △ Less

Submitted 24 December, 2019; originally announced December 2019.

Robust UOWC systems against bubble-induced impairments via transmit/receive diversities

Authors: Lian-Kuan Chen, Yingjie Shao, Rui Deng

Abstract: We systematically investigate the bubble-induced performance degradation for underwater wireless optical communication (UOWC) with different bubble sizes and positions. By using different transmit and receive diversities, we investigate the effectiveness of transmit/receive diversity on the mitigation of the bubble-induced impairment to the UOWC link. With the help of a 2 by 2 MIMO using repetitio… ▽ More We systematically investigate the bubble-induced performance degradation for underwater wireless optical communication (UOWC) with different bubble sizes and positions. By using different transmit and receive diversities, we investigate the effectiveness of transmit/receive diversity on the mitigation of the bubble-induced impairment to the UOWC link. With the help of a 2 by 2 MIMO using repetition coding (RC) and maximum ratio combining (MRC), a robust 780-Mbit/s UOWC transmission is achieved. The corresponding outage probability can be significantly reduced from 34.6% for the system without diversity to less than 1%. △ Less

Submitted 5 August, 2019; originally announced August 2019.