-
Decentralized Personalized Federated Learning based on a Conditional Sparse-to-Sparser Scheme
Authors:
Qianyu Long,
Qiyuan Wang,
Christos Anagnostopoulos,
Daning Bi
Abstract:
Decentralized Federated Learning (DFL) has become popular due to its robustness and avoidance of centralized coordination. In this paradigm, clients actively engage in training by exchanging models with their networked neighbors. However, DFL introduces increased costs in terms of training and communication. Existing methods focus on minimizing communication often overlooking training efficiency a…
▽ More
Decentralized Federated Learning (DFL) has become popular due to its robustness and avoidance of centralized coordination. In this paradigm, clients actively engage in training by exchanging models with their networked neighbors. However, DFL introduces increased costs in terms of training and communication. Existing methods focus on minimizing communication often overlooking training efficiency and data heterogeneity. To address this gap, we propose a novel \textit{sparse-to-sparser} training scheme: DA-DPFL. DA-DPFL initializes with a subset of model parameters, which progressively reduces during training via \textit{dynamic aggregation} and leads to substantial energy savings while retaining adequate information during critical learning periods.
Our experiments showcase that DA-DPFL substantially outperforms DFL baselines in test accuracy, while achieving up to $5$ times reduction in energy costs. We provide a theoretical analysis of DA-DPFL's convergence by solidifying its applicability in decentralized and personalized learning. The code is available at:https://github.com/EricLoong/da-dpfl
△ Less
Submitted 25 April, 2024; v1 submitted 24 April, 2024;
originally announced April 2024.
-
FedDIP: Federated Learning with Extreme Dynamic Pruning and Incremental Regularization
Authors:
Qianyu Long,
Christos Anagnostopoulos,
Shameem Puthiya Parambath,
Daning Bi
Abstract:
Federated Learning (FL) has been successfully adopted for distributed training and inference of large-scale Deep Neural Networks (DNNs). However, DNNs are characterized by an extremely large number of parameters, thus, yielding significant challenges in exchanging these parameters among distributed nodes and managing the memory. Although recent DNN compression methods (e.g., sparsification, prunin…
▽ More
Federated Learning (FL) has been successfully adopted for distributed training and inference of large-scale Deep Neural Networks (DNNs). However, DNNs are characterized by an extremely large number of parameters, thus, yielding significant challenges in exchanging these parameters among distributed nodes and managing the memory. Although recent DNN compression methods (e.g., sparsification, pruning) tackle such challenges, they do not holistically consider an adaptively controlled reduction of parameter exchange while maintaining high accuracy levels. We, therefore, contribute with a novel FL framework (coined FedDIP), which combines (i) dynamic model pruning with error feedback to eliminate redundant information exchange, which contributes to significant performance improvement, with (ii) incremental regularization that can achieve \textit{extreme} sparsity of models. We provide convergence analysis of FedDIP and report on a comprehensive performance and comparative assessment against state-of-the-art methods using benchmark data sets and DNN models. Our results showcase that FedDIP not only controls the model sparsity but efficiently achieves similar or better performance compared to other model pruning methods adopting incremental regularization during distributed model training. The code is available at: https://github.com/EricLoong/feddip.
△ Less
Submitted 13 September, 2023;
originally announced September 2023.
-
CSK Realization for MC via Spatially Distributed Multicellular Consortia
Authors:
Rinrada Jadsadaphongphaibool,
Dadi Bi,
Yansha Deng
Abstract:
The design and engineering of molecular communication (MC) components capable of processing chemical concentration signals is the key to unleashing the potential of MC for interdisciplinary applications. By controlling the signaling pathway and molecule exchange between cell devices, synthetic biology provides the MC community with tools and techniques to achieve various signal processing function…
▽ More
The design and engineering of molecular communication (MC) components capable of processing chemical concentration signals is the key to unleashing the potential of MC for interdisciplinary applications. By controlling the signaling pathway and molecule exchange between cell devices, synthetic biology provides the MC community with tools and techniques to achieve various signal processing functions. In this paper, we propose a design framework to realize any order concentration shift keying (CSK) systems based on simple and reusable single-input single-output cells. The design framework also exploits the distributed multicellular consortia with spatial segregation, which has advantages in system scalability, low genetic manipulation, and signal orthogonality. We also create a small library of reusable engineered cells and apply them to implement binary CSK (BCSK) and quadruple CSK (QCSK) systems to demonstrate the feasibility of our proposed design framework. Importantly, we establish a mathematical framework to theoretically characterize our proposed distributed multicellular systems. Specially, we divide a system into fundamental building blocks, from which we derive the impulse response of each block and the cascade of the impulse responses leads to the end-to-end response of the system. Simulation results obtained from the agent-based simulator BSim not only validate our CSK design framework but also demonstrate the accuracy of the proposed mathematical analysis.
△ Less
Submitted 20 March, 2023;
originally announced March 2023.
-
Using word embedding for environmental violation analysis: Evidence from Pennsylvania unconventional oil and gas compliance reports
Authors:
Dan Bi,
Ju-e Guo,
Erlong Zhao,
Shaolong Sun,
Shouyang Wang
Abstract:
With the booming of the unconventional oil and gas industry, its inevitable damage to the environment and human health has attracted public attention. We applied text mining on a total 6057 the type of Environmental Health and Safety compliance reports from 2008 to 2018 lunched by the Department of Environmental Protection in Pennsylvania, USA, to discover the intern mechanism of environmental vio…
▽ More
With the booming of the unconventional oil and gas industry, its inevitable damage to the environment and human health has attracted public attention. We applied text mining on a total 6057 the type of Environmental Health and Safety compliance reports from 2008 to 2018 lunched by the Department of Environmental Protection in Pennsylvania, USA, to discover the intern mechanism of environmental violations.
△ Less
Submitted 9 November, 2021;
originally announced December 2021.
-
Digital Signal Processing for Molecular Communication via Chemical Reactions-based Microfluidic Circuits
Authors:
Dadi Bi,
Yansha Deng
Abstract:
Chemical reactions-based microfluidic circuits are expected to provide new opportunities to perform signal processing functions over molecular domain. To realize this vision, in this article, we exploit and present the digital signal processing capabilities of chemical reactions-based microfluidic circuits. Aiming to facilitate microfluidic circuit design, we describe a microfluidic circuit using…
▽ More
Chemical reactions-based microfluidic circuits are expected to provide new opportunities to perform signal processing functions over molecular domain. To realize this vision, in this article, we exploit and present the digital signal processing capabilities of chemical reactions-based microfluidic circuits. Aiming to facilitate microfluidic circuit design, we describe a microfluidic circuit using a five-level architecture: 1) Molecular Propagation; 2) Chemical Transformation; 3) Microfluidic Modules; 4) Microfluidic Logic Gates; and 5) Microfluidic Circuits. We first identify the components at Levels 1 and 2, and present how their combinations can build the basic modules for Level 3. We then assemble basic modules to construct five types of logic gate for Level 4, including AND, NAND, OR, NOR, and XOR gates, which show advantages of microfluidic circuits in reusability and modularity. Last but not least, we discuss challenges and potential solutions for designing, building, and testing microfluidic circuits with complex signal processing functions in Level 5 based on the digital logic gates at Level 4.
△ Less
Submitted 11 September, 2020; v1 submitted 10 September, 2020;
originally announced September 2020.
-
A Survey of Molecular Communication in Cell Biology: Establishing a New Hierarchy for Interdisciplinary Applications
Authors:
Dadi Bi,
Apostolos Almpanis,
Adam Noel,
Yansha Deng,
Robert Schober
Abstract:
Molecular communication (MC) engineering is inspired by the use of chemical signals as information carriers in cell biology. The biological nature of chemical signaling makes MC a promising methodology for interdisciplinary applications requiring communication between cells and other microscale devices. However, since the life sciences and communications engineering fields have distinct approaches…
▽ More
Molecular communication (MC) engineering is inspired by the use of chemical signals as information carriers in cell biology. The biological nature of chemical signaling makes MC a promising methodology for interdisciplinary applications requiring communication between cells and other microscale devices. However, since the life sciences and communications engineering fields have distinct approaches to formulating and solving research problems, the mismatch between them can hinder the translation of research results and impede the development and implementation of interdisciplinary solutions. To bridge this gap, this survey proposes a novel communication hierarchy for MC signaling in cell biology and maps phenomena, contributions, and problems to the hierarchy. The hierarchy includes: 1) the Physical Signal Propagation level; 2) the Physical and Chemical Signal Interaction level; 3) the Signal-Data Interface level; 4) the Local Data Abstraction level; and 5) the Application level. To further demonstrate the proposed hierarchy, it is applied to case studies on quorum sensing, neuronal signaling, and communication via DNA. Finally, several open problems are identified for each level and the integration of multiple levels. The proposed hierarchy provides language for communication engineers to study and interface with biological systems, and also helps biologists to understand how communications engineering concepts can be exploited to interpret, control, and manipulate signaling in cell biology.
△ Less
Submitted 31 August, 2020;
originally announced September 2020.
-
Microfluidic QCSK Transmitter and Receiver Design for Molecular Communication
Authors:
Dadi Bi,
Yansha Deng
Abstract:
The design of components with molecular communication (MC) functionalities can bring an opportunity to enable some emerging applications in fields from personal healthcare to modern industry. In this paper, we propose the designs of the microfluidic transmitter and receiver with quadruple concentration shift keying (QCSK) modulation and demodulation functionalities. To do so, we first present an A…
▽ More
The design of components with molecular communication (MC) functionalities can bring an opportunity to enable some emerging applications in fields from personal healthcare to modern industry. In this paper, we propose the designs of the microfluidic transmitter and receiver with quadruple concentration shift keying (QCSK) modulation and demodulation functionalities. To do so, we first present an AND gate design, and then apply it to the QCSK transmitter and receiver design. The QCSK transmitter is capable of modulating two input signals to four different concentration levels, and the QCSK receiver can demodulate a received signal to two outputs. More importantly, we also establish a mathematical framework to theoretically characterize our proposed microfluidic circuits. Based on this, we first derive the output concentration distribution of our proposed AND gate design, and provide the insight into the selection of design parameters to ensure an exhibition of desired behaviour. We further derive the output concentration distributions of the QCSK transmitter and receiver. Simulation results obtained in COMSOL Multiphysics not only show the desired behaviour of all the proposed microfluidic circuits, but also demonstrate the accuracy of the proposed mathematical framework.
△ Less
Submitted 11 May, 2020; v1 submitted 4 May, 2020;
originally announced May 2020.
-
New Research Trends in Unconventional Oil and Gas Environmental Issue: A Bibliometric Analysis
Authors:
Dan Bi,
Ju-e Guo,
Shouyang Wang,
Shaolong Sun
Abstract:
With the booming of unconventional gas production in the world, how to balance environment pollution risk and economy of unconventional gas have become a common dilemma around the world. The aim of this study is to elucidate the research about environmental issue brought with development of unconventional oil and gas industry. To achieve this goal, we present a bibliometrics overview of this field…
▽ More
With the booming of unconventional gas production in the world, how to balance environment pollution risk and economy of unconventional gas have become a common dilemma around the world. The aim of this study is to elucidate the research about environmental issue brought with development of unconventional oil and gas industry. To achieve this goal, we present a bibliometrics overview of this field from 1990 to 2018. Firstly, this study outlines a basic statistical analysis over journals, publications, authors, institutions and documents. Secondly, VOSviewer is employed to visualize the collaborative relationship to show the link between different author, institutions, regions and journals. Finally, document bibliographic coupling, cooccurrence and keyword burst detection are analyzed to reveal the emerging trend and hot topic. The results indicate that among all countries, America was the most productive country as well as cooperated the most with other countries, followed by China, while the China University of Petroleum is the most productive institution in the world, with 105 publications. Additionally, most articles were classified as energy fuels, environmental sciences and geosciences multidisciplinary. Furthermore, based on emerging trends analysis, it was concluded that hydraulic fracturing technology has become a hot topic, other popular research topics include: energy policy and regulation of unconventional gas development, greenhouse gas emissions, energy and water consumption of unconventional gas life cycle assessment.
△ Less
Submitted 13 March, 2020;
originally announced March 2020.
-
Seasonal and Trend Forecasting of Tourist Arrivals: An Adaptive Multiscale Ensemble Learning Approach
Authors:
Shaolong Suna,
Dan Bi,
Ju-e Guo,
Shouyang Wang
Abstract:
The accurate seasonal and trend forecasting of tourist arrivals is a very challenging task. In the view of the importance of seasonal and trend forecasting of tourist arrivals, and limited research work paid attention to these previously. In this study, a new adaptive multiscale ensemble (AME) learning approach incorporating variational mode decomposition (VMD) and least square support vector regr…
▽ More
The accurate seasonal and trend forecasting of tourist arrivals is a very challenging task. In the view of the importance of seasonal and trend forecasting of tourist arrivals, and limited research work paid attention to these previously. In this study, a new adaptive multiscale ensemble (AME) learning approach incorporating variational mode decomposition (VMD) and least square support vector regression (LSSVR) is developed for short-, medium-, and long-term seasonal and trend forecasting of tourist arrivals. In the formulation of our developed AME learning approach, the original tourist arrivals series are first decomposed into the trend, seasonal and remainders volatility components. Then, the ARIMA is used to forecast the trend component, the SARIMA is used to forecast seasonal component with a 12-month cycle, while the LSSVR is used to forecast remainder volatility components. Finally, the forecasting results of the three components are aggregated to generate an ensemble forecasting of tourist arrivals by the LSSVR based nonlinear ensemble approach. Furthermore, a direct strategy is used to implement multi-step-ahead forecasting. Taking two accuracy measures and the Diebold-Mariano test, the empirical results demonstrate that our proposed AME learning approach can achieve higher level and directional forecasting accuracy compared with other benchmarks used in this study, indicating that our proposed approach is a promising model for forecasting tourist arrivals with high seasonality and volatility.
△ Less
Submitted 10 March, 2020; v1 submitted 19 February, 2020;
originally announced February 2020.
-
Chemical Reactions-Based Microfluidic Transmitter and Receiver for Molecular Communication
Authors:
Dadi Bi,
Yansha Deng,
Massimiliano Pierobon,
Arumugam Nallanathan
Abstract:
The design of communication systems capable of processing and exchanging information through molecules and chemical processes is a rapidly growing interdisciplinary field, which holds the promise to revolutionize how we realize computing and communication devices. While molecular communication (MC) theory has had major developments in recent years, more practical aspects in designing components ca…
▽ More
The design of communication systems capable of processing and exchanging information through molecules and chemical processes is a rapidly growing interdisciplinary field, which holds the promise to revolutionize how we realize computing and communication devices. While molecular communication (MC) theory has had major developments in recent years, more practical aspects in designing components capable of MC functionalities remain less explored. Motivated by this, we design a microfluidic MC system with a microfluidic MC transmitter and a microfluidic MC receiver based on chemical reactions. Considering existing MC literature on information transmission via molecular pulse modulation, the proposed microfluidic MC transmitter is capable of generating continuously predefined pulse-shaped molecular concentrations upon rectangular triggering signals using chemical reactions inspired by how cells generate pulse-shaped molecular signals in biology. We further design a microfluidic MC receiver capable of demodulating a received signal to a rectangular output signal using a thresholding reaction and an amplifying reaction. Our chemical reactions-based microfluidic molecular communication system is reproducible and well-designed, and more importantly, it overcomes the slow-speed, unreliability, and non-scalability of biological processes in cells. To reveal design insights, we also derive the theoretical signal responses for our designed microfluidic transmitter and receiver, which further facilitate the transmitter design optimization. Our theoretical results are validated via simulations performed through the COMSOL Multiphysics finite element solver. We demonstrate the predefined nature of the generated pulse and the demodulated rectangular signal together with their dependence on design parameters.
△ Less
Submitted 8 August, 2019;
originally announced August 2019.
-
Handling boundary constraints for numerical optimization by particle swarm flying in periodic search space
Authors:
Wen-Jun Zhang,
Xiao-Feng Xie,
De-Chun Bi
Abstract:
The periodic mode is analyzed together with two conventional boundary handling modes for particle swarm. By providing an infinite space that comprises periodic copies of original search space, it avoids possible disorganizing of particle swarm that is induced by the undesired mutations at the boundary. The results on benchmark functions show that particle swarm with periodic mode is capable of i…
▽ More
The periodic mode is analyzed together with two conventional boundary handling modes for particle swarm. By providing an infinite space that comprises periodic copies of original search space, it avoids possible disorganizing of particle swarm that is induced by the undesired mutations at the boundary. The results on benchmark functions show that particle swarm with periodic mode is capable of improving the search performance significantly, by compared with that of conventional modes and other algorithms.
△ Less
Submitted 24 May, 2005;
originally announced May 2005.
-
Handling equality constraints by adaptive relaxing rule for swarm algorithms
Authors:
Xiao-Feng Xie,
Wen-Jun Zhang,
De-Chun Bi
Abstract:
The adaptive constraints relaxing rule for swarm algorithms to handle with the problems with equality constraints is presented. The feasible space of such problems may be similiar to ridge function class, which is hard for applying swarm algorithms. To enter the solution space more easily, the relaxed quasi feasible space is introduced and shrinked adaptively. The experimental results on benchma…
▽ More
The adaptive constraints relaxing rule for swarm algorithms to handle with the problems with equality constraints is presented. The feasible space of such problems may be similiar to ridge function class, which is hard for applying swarm algorithms. To enter the solution space more easily, the relaxed quasi feasible space is introduced and shrinked adaptively. The experimental results on benchmark functions are compared with the performance of other algorithms, which show its efficiency.
△ Less
Submitted 24 May, 2005;
originally announced May 2005.
-
Optimizing semiconductor devices by self-organizing particle swarm
Authors:
Xiao-Feng Xie,
Wen-Jun Zhang,
De-Chun Bi
Abstract:
A self-organizing particle swarm is presented. It works in dissipative state by employing the small inertia weight, according to experimental analysis on a simplified model, which with fast convergence. Then by recognizing and replacing inactive particles according to the process deviation information of device parameters, the fluctuation is introduced so as to driving the irreversible evolution…
▽ More
A self-organizing particle swarm is presented. It works in dissipative state by employing the small inertia weight, according to experimental analysis on a simplified model, which with fast convergence. Then by recognizing and replacing inactive particles according to the process deviation information of device parameters, the fluctuation is introduced so as to driving the irreversible evolution process with better fitness. The testing on benchmark functions and an application example for device optimization with designed fitness function indicates it improves the performance effectively.
△ Less
Submitted 24 May, 2005;
originally announced May 2005.
