Showing 1–50 of 103 results for author: Qin, K

An Alternative Channel to Black Hole Low-Mass X-ray Binaries: Dynamical Friction of Dark Matter?

Authors: Ke Qin, Wen-Cong Chen

Abstract: Both the anomalous magnetic braking of Ap/Bp stars and the surrounding circumbinary disk models can account for the formation of black hole (BH) low-mass X-ray binaries (LMXBs), while the simulated effective temperatures of the donor stars are significantly higher than the observed values. Therefore, the formation of BH LMXBs is not still completely understood. In this work, we diagnose whether th… ▽ More Both the anomalous magnetic braking of Ap/Bp stars and the surrounding circumbinary disk models can account for the formation of black hole (BH) low-mass X-ray binaries (LMXBs), while the simulated effective temperatures of the donor stars are significantly higher than the observed values. Therefore, the formation of BH LMXBs is not still completely understood. In this work, we diagnose whether the dynamical friction between dark matter and the companion stars can drive BH binaries to evolve toward the observed BH LMXBs and alleviate the effective temperature problem. Assuming that there exists a density spike of dark matter around BH, the dynamical friction can produce an efficient angular momentum loss, driving BH binaries with an intermediate-mass companion star to evolve into BH LMXBs for a spike index higher than $γ= 1.58$. Our detailed stellar evolution models show that the calculated effective temperatures can match the observed value of most BH LMXBs for a spike index range of $γ= 1.7-2.1$. However, the simulated mass-transfer rates when $γ= 2.0$ and $2.1$ are too high to be consistent with the observed properties that BH LMXBs appears as soft X-ray transients. Therefore, the dynamical friction of dark matter can only alleviate the effective temperature problem of those BH LMXBs with a relatively short orbital period. △ Less

Submitted 24 June, 2024; originally announced June 2024.

Comments: 12 pages, 6 figures, ApJ in press

Research on Foundation Model for Spatial Data Intelligence: China's 2024 White Paper on Strategic Development of Spatial Data Intelligence

Authors: Shaohua Wang, Xing Xie, Yong Li, Danhuai Guo, Zhi Cai, Yu Liu, Yang Yue, Xiao Pan, Feng Lu, Huayi Wu, Zhipeng Gui, Zhiming Ding, Bolong Zheng, Fuzheng Zhang, Tao Qin, **gyuan Wang, Chuang Tao, Zhengchao Chen, Hao Lu, Jiayi Li, Hongyang Chen, Peng Yue, Wenhao Yu, Yao Yao, Leilei Sun , et al. (9 additional authors not shown)

Abstract: This report focuses on spatial data intelligent large models, delving into the principles, methods, and cutting-edge applications of these models. It provides an in-depth discussion on the definition, development history, current status, and trends of spatial data intelligent large models, as well as the challenges they face. The report systematically elucidates the key technologies of spatial dat… ▽ More This report focuses on spatial data intelligent large models, delving into the principles, methods, and cutting-edge applications of these models. It provides an in-depth discussion on the definition, development history, current status, and trends of spatial data intelligent large models, as well as the challenges they face. The report systematically elucidates the key technologies of spatial data intelligent large models and their applications in urban environments, aerospace remote sensing, geography, transportation, and other scenarios. Additionally, it summarizes the latest application cases of spatial data intelligent large models in themes such as urban development, multimodal systems, remote sensing, smart transportation, and resource environments. Finally, the report concludes with an overview and outlook on the development prospects of spatial data intelligent large models. △ Less

Submitted 29 June, 2024; v1 submitted 30 May, 2024; originally announced May 2024.

Comments: in Chinese language

Armored Core of PKI: Removing Signing Keys for CA via Efficient and Trusted Physical Certification

Authors: Xiaolin Zhang, Chenghao Chen, Kailun Qin, Yuxuan Wang, Shipei Qu, Tengfei Wang, Chi Zhang, Dawu Gu

Abstract: The signing key protection for Certificate Authorities (CAs) remains a critical concern in PKI. These keys can be exposed by carefully designed attacks or operational errors even today. Traditional protections fail to eliminate such risk since attackers always manage to find an exploit path to capture the digital key leakage. Even a single successful attack can compromise the security. This everla… ▽ More The signing key protection for Certificate Authorities (CAs) remains a critical concern in PKI. These keys can be exposed by carefully designed attacks or operational errors even today. Traditional protections fail to eliminate such risk since attackers always manage to find an exploit path to capture the digital key leakage. Even a single successful attack can compromise the security. This everlasting dilemma motivates us to consider removing CA's signing keys and propose Armored Core, a PKI security extension using the trusted binding of Physically Unclonable Function (PUF) for certificate operations. By eliminating explicit signing keys, it makes key exposure attacks impossible. In Armored Core, we design a set of PUF-based X.509v3 TLS certificate functions for CAs, where they generate physically trusted "signatures" without using a fixed key. We formally prove the existential unforgeability of the certificates. We propose the first PUF transparency mechanism to effectively monitor the calling behaviors of PUF. We also provide an open-sourced implementation where Armored Core is integrated into real-world PKI systems like Let's Encrypt Pebble CA and Certbot. The results show that it achieves key removal without any additional performance overhead. It offers a more trusted basis for PKI security through efficient physical operations with compatible functions. △ Less

Submitted 13 June, 2024; v1 submitted 23 April, 2024; originally announced April 2024.

FedSPU: Personalized Federated Learning for Resource-constrained Devices with Stochastic Parameter Update

Authors: Ziru Niu, Hai Dong, A. K. Qin

Abstract: Personalized Federated Learning (PFL) is widely employed in IoT applications to handle high-volume, non-iid client data while ensuring data privacy. However, heterogeneous edge devices owned by clients may impose varying degrees of resource constraints, causing computation and communication bottlenecks for PFL. Federated Dropout has emerged as a popular strategy to address this challenge, wherein… ▽ More Personalized Federated Learning (PFL) is widely employed in IoT applications to handle high-volume, non-iid client data while ensuring data privacy. However, heterogeneous edge devices owned by clients may impose varying degrees of resource constraints, causing computation and communication bottlenecks for PFL. Federated Dropout has emerged as a popular strategy to address this challenge, wherein only a subset of the global model, i.e. a \textit{sub-model}, is trained on a client's device, thereby reducing computation and communication overheads. Nevertheless, the dropout-based model-pruning strategy may introduce bias, particularly towards non-iid local data. When biased sub-models absorb highly divergent parameters from other clients, performance degradation becomes inevitable. In response, we propose federated learning with stochastic parameter update (FedSPU). Unlike dropout that tailors the global model to small-size local sub-models, FedSPU maintains the full model architecture on each device but randomly freezes a certain percentage of neurons in the local model during training while updating the remaining neurons. This approach ensures that a portion of the local model remains personalized, thereby enhancing the model's robustness against biased parameters from other clients. Experimental results demonstrate that FedSPU outperforms federated dropout by 7.57\% on average in terms of accuracy. Furthermore, an introduced early stop** scheme leads to a significant reduction of the training time by \(24.8\%\sim70.4\%\) while maintaining high accuracy. △ Less

Submitted 18 March, 2024; originally announced March 2024.

Comments: 14 pages including ref

MSC Class: 68U35 ACM Class: C.2.4; I.2.11

FAGH: Accelerating Federated Learning with Approximated Global Hessian

Authors: Mrinmay Sen, A. K. Qin, Krishna Mohan C

Abstract: In federated learning (FL), the significant communication overhead due to the slow convergence speed of training the global model poses a great challenge. Specifically, a large number of communication rounds are required to achieve the convergence in FL. One potential solution is to employ the Newton-based optimization method for training, known for its quadratic convergence rate. However, the exi… ▽ More In federated learning (FL), the significant communication overhead due to the slow convergence speed of training the global model poses a great challenge. Specifically, a large number of communication rounds are required to achieve the convergence in FL. One potential solution is to employ the Newton-based optimization method for training, known for its quadratic convergence rate. However, the existing Newton-based FL training methods suffer from either memory inefficiency or high computational costs for local clients or the server. To address this issue, we propose an FL with approximated global Hessian (FAGH) method to accelerate FL training. FAGH leverages the first moment of the approximated global Hessian and the first moment of the global gradient to train the global model. By harnessing the approximated global Hessian curvature, FAGH accelerates the convergence of global model training, leading to the reduced number of communication rounds and thus the shortened training time. Experimental results verify FAGH's effectiveness in decreasing the number of communication rounds and the time required to achieve the pre-specified objectives of the global model performance in terms of training and test losses as well as test accuracy. Notably, FAGH outperforms several state-of-the-art FL training methods. △ Less

Submitted 16 March, 2024; originally announced March 2024.

Knowledge Transfer across Multiple Principal Component Analysis Studies

Authors: Zeyu Li, Kangxiang Qin, Yong He, Wang Zhou, Xinsheng Zhang

Abstract: Transfer learning has aroused great interest in the statistical community. In this article, we focus on knowledge transfer for unsupervised learning tasks in contrast to the supervised learning tasks in the literature. Given the transferable source populations, we propose a two-step transfer learning algorithm to extract useful information from multiple source principal component analysis (PCA) st… ▽ More Transfer learning has aroused great interest in the statistical community. In this article, we focus on knowledge transfer for unsupervised learning tasks in contrast to the supervised learning tasks in the literature. Given the transferable source populations, we propose a two-step transfer learning algorithm to extract useful information from multiple source principal component analysis (PCA) studies, thereby enhancing estimation accuracy for the target PCA task. In the first step, we integrate the shared subspace information across multiple studies by a proposed method named as Grassmannian barycenter, instead of directly performing PCA on the pooled dataset. The proposed Grassmannian barycenter method enjoys robustness and computational advantages in more general cases. Then the resulting estimator for the shared subspace from the first step is further utilized to estimate the target private subspace in the second step. Our theoretical analysis credits the gain of knowledge transfer between PCA studies to the enlarged eigenvalue gap, which is different from the existing supervised transfer learning tasks where sparsity plays the central role. In addition, we prove that the bilinear forms of the empirical spectral projectors have asymptotic normality under weaker eigenvalue gap conditions after knowledge transfer. When the set of informativesources is unknown, we endow our algorithm with the capability of useful dataset selection by solving a rectified optimization problem on the Grassmann manifold, which in turn leads to a computationally friendly rectified Grassmannian K-means procedure. In the end, extensive numerical simulation results and a real data case concerning activity recognition are reported to support our theoretical claims and to illustrate the empirical usefulness of the proposed transfer learning methods. △ Less

Submitted 12 March, 2024; originally announced March 2024.

No Language is an Island: Unifying Chinese and English in Financial Large Language Models, Instruction Data, and Benchmarks

Authors: Gang Hu, Ke Qin, Chenhan Yuan, Min Peng, Alejandro Lopez-Lira, Benyou Wang, Sophia Ananiadou, Wanlong Yu, Jimin Huang, Qianqian Xie

Abstract: While the progression of Large Language Models (LLMs) has notably propelled financial analysis, their application has largely been confined to singular language realms, leaving untapped the potential of bilingual Chinese-English capacity. To bridge this chasm, we introduce ICE-PIXIU, seamlessly amalgamating the ICE-INTENT model and ICE-FLARE benchmark for bilingual financial analysis. ICE-PIXIU un… ▽ More While the progression of Large Language Models (LLMs) has notably propelled financial analysis, their application has largely been confined to singular language realms, leaving untapped the potential of bilingual Chinese-English capacity. To bridge this chasm, we introduce ICE-PIXIU, seamlessly amalgamating the ICE-INTENT model and ICE-FLARE benchmark for bilingual financial analysis. ICE-PIXIU uniquely integrates a spectrum of Chinese tasks, alongside translated and original English datasets, enriching the breadth and depth of bilingual financial modeling. It provides unrestricted access to diverse model variants, a substantial compilation of diverse cross-lingual and multi-modal instruction data, and an evaluation benchmark with expert annotations, comprising 10 NLP tasks, 20 bilingual specific tasks, totaling 95k datasets. Our thorough evaluation emphasizes the advantages of incorporating these bilingual datasets, especially in translation tasks and utilizing original English data, enhancing both linguistic flexibility and analytical acuity in financial contexts. Notably, ICE-INTENT distinguishes itself by showcasing significant enhancements over conventional LLMs and existing financial LLMs in bilingual milieus, underscoring the profound impact of robust bilingual data on the accuracy and efficacy of financial NLP. △ Less

Submitted 16 April, 2024; v1 submitted 10 March, 2024; originally announced March 2024.

Comments: 24 pages, 5 figures, 12 tables, including Appendix

SOFIM: Stochastic Optimization Using Regularized Fisher Information Matrix

Authors: Mrinmay Sen, A. K. Qin, Gayathri C, Raghu Kishore N, Yen-Wei Chen, Balasubramanian Raman

Abstract: This paper introduces a new stochastic optimization method based on the regularized Fisher information matrix (FIM), named SOFIM, which can efficiently utilize the FIM to approximate the Hessian matrix for finding Newton's gradient update in large-scale stochastic optimization of machine learning models. It can be viewed as a variant of natural gradient descent, where the challenge of storing and… ▽ More This paper introduces a new stochastic optimization method based on the regularized Fisher information matrix (FIM), named SOFIM, which can efficiently utilize the FIM to approximate the Hessian matrix for finding Newton's gradient update in large-scale stochastic optimization of machine learning models. It can be viewed as a variant of natural gradient descent, where the challenge of storing and calculating the full FIM is addressed through making use of the regularized FIM and directly finding the gradient update direction via Sherman-Morrison matrix inversion. Additionally, like the popular Adam method, SOFIM uses the first moment of the gradient to address the issue of non-stationary objectives across mini-batches due to heterogeneous data. The utilization of the regularized FIM and Sherman-Morrison matrix inversion leads to the improved convergence rate with the same space and time complexities as stochastic gradient descent (SGD) with momentum. The extensive experiments on training deep learning models using several benchmark image classification datasets demonstrate that the proposed SOFIM outperforms SGD with momentum and several state-of-the-art Newton optimization methods in term of the convergence speed for achieving the pre-specified objectives of training and test losses as well as test accuracy. △ Less

Submitted 1 May, 2024; v1 submitted 5 March, 2024; originally announced March 2024.

Decentralised Traffic Incident Detection via Network Lasso

Authors: Qiyuan Zhu, A. K. Qin, Prabath Abeysekara, Hussein Dia, Hanna Grzybowska

Abstract: Traffic incident detection plays a key role in intelligent transportation systems, which has gained great attention in transport engineering. In the past, traditional machine learning (ML) based detection methods achieved good performance under a centralised computing paradigm, where all data are transmitted to a central server for building ML models therein. Nowadays, deep neural networks based f… ▽ More Traffic incident detection plays a key role in intelligent transportation systems, which has gained great attention in transport engineering. In the past, traditional machine learning (ML) based detection methods achieved good performance under a centralised computing paradigm, where all data are transmitted to a central server for building ML models therein. Nowadays, deep neural networks based federated learning (FL) has become a mainstream detection approach to enable the model training in a decentralised manner while warranting local data governance. Such neural networks-centred techniques, however, have overshadowed the utility of well-established ML-based detection methods. In this work, we aim to explore the potential of potent conventional ML-based detection models in modern traffic scenarios featured by distributed data. We leverage an elegant but less explored distributed optimisation framework named Network Lasso, with guaranteed global convergence for convex problem formulations, integrate the potent convex ML model with it, and compare it with centralised learning, local learning, and federated learning methods atop a well-known traffic incident detection dataset. Experimental results show that the proposed network lasso-based approach provides a promising alternative to the FL-based approach in data-decentralised traffic scenarios, with a strong convergence guarantee while rekindling the significance of conventional ML-based detection methods. △ Less

Submitted 28 February, 2024; originally announced February 2024.

Teamwork Makes TEE Work: Open and Resilient Remote Attestation on Decentralized Trust

Authors: Xiaolin Zhang, Kailun Qin, Shipei Qu, Tengfei Wang, Chi Zhang, Dawu Gu

Abstract: Remote Attestation (RA) enables the integrity and authenticity of applications in Trusted Execution Environment (TEE) to be verified. Existing TEE RA designs employ a centralized trust model where they rely on a single provisioned secret key and a centralized verifier to establish trust for remote parties. This model is however brittle and can be untrusted under advanced attacks nowadays. Besides,… ▽ More Remote Attestation (RA) enables the integrity and authenticity of applications in Trusted Execution Environment (TEE) to be verified. Existing TEE RA designs employ a centralized trust model where they rely on a single provisioned secret key and a centralized verifier to establish trust for remote parties. This model is however brittle and can be untrusted under advanced attacks nowadays. Besides, most designs only provide fixed functionalities once deployed, making them hard to adapt to different needs on availability, Quality of Service (QoS), etc. Therefore, we propose JANUS, an open and resilient TEE RA scheme. To decentralize trust, we, on one hand, introduce Physically Unclonable Function (PUF) as an intrinsic root of trust (RoT) in TEE to provide additional measurements and cryptographic enhancements. On the other hand, we use blockchain and smart contract to realize decentralized verification and result audit. Furthermore, we design an automated turnout mechanism that allows JANUS to remain resilient and offer flexible RA services under various situations. We provide a UC-based security proof and demonstrate the scalability and generality of JANUS by implementing an open-sourced prototype. △ Less

Submitted 13 February, 2024; originally announced February 2024.

Comments: 18 pages, 10 figures

Expert-Adaptive Medical Image Segmentation

Authors: Binyan Hu, A. K. Qin

Abstract: Medical image segmentation (MIS) plays an instrumental role in medical image analysis, where considerable effort has been devoted to automating the process. Currently, mainstream MIS approaches are based on deep neural networks (DNNs), which are typically trained on a dataset with annotations produced by certain medical experts. In the medical domain, the annotations generated by different experts… ▽ More Medical image segmentation (MIS) plays an instrumental role in medical image analysis, where considerable effort has been devoted to automating the process. Currently, mainstream MIS approaches are based on deep neural networks (DNNs), which are typically trained on a dataset with annotations produced by certain medical experts. In the medical domain, the annotations generated by different experts can be inherently distinct due to complexity of medical images and variations in expertise and post-segmentation missions. Consequently, the DNN model trained on the data annotated by some experts may hardly adapt to a new expert. In this work, we evaluate a customised expert-adaptive method, characterised by multi-expert annotation, multi-task DNN-based model training, and lightweight model fine-tuning, to investigate model's adaptivity to a new expert in the situation where the amount and mobility of training images are limited. Experiments conducted on brain MRI segmentation tasks with limited training data demonstrate its effectiveness and the impact of its key parameters. △ Less

Submitted 1 May, 2024; v1 submitted 11 February, 2024; originally announced February 2024.

Two-Stage Multi-task Self-Supervised Learning for Medical Image Segmentation

Authors: Binyan Hu, A. K. Qin

Abstract: Medical image segmentation has been significantly advanced by deep learning (DL) techniques, though the data scarcity inherent in medical applications poses a great challenge to DL-based segmentation methods. Self-supervised learning offers a solution by creating auxiliary learning tasks from the available dataset and then leveraging the knowledge acquired from solving auxiliary tasks to help bett… ▽ More Medical image segmentation has been significantly advanced by deep learning (DL) techniques, though the data scarcity inherent in medical applications poses a great challenge to DL-based segmentation methods. Self-supervised learning offers a solution by creating auxiliary learning tasks from the available dataset and then leveraging the knowledge acquired from solving auxiliary tasks to help better solve the target segmentation task. Different auxiliary tasks may have different properties and thus can help the target task to different extents. It is desired to leverage their complementary advantages to enhance the overall assistance to the target task. To achieve this, existing methods often adopt a joint training paradigm, which co-solves segmentation and auxiliary tasks by integrating their losses or intermediate gradients. However, direct coupling of losses or intermediate gradients risks undesirable interference because the knowledge acquired from solving each auxiliary task at every training step may not always benefit the target task. To address this issue, we propose a two-stage training approach. In the first stage, the target segmentation task will be independently co-solved with each auxiliary task in both joint training and pre-training modes, with the better model selected via validation performance. In the second stage, the models obtained with respect to each auxiliary task are converted into a single model using an ensemble knowledge distillation method. Our approach allows for making best use of each auxiliary task to create multiple elite segmentation models and then combine them into an even more powerful model. We employed five auxiliary tasks of different proprieties in our approach and applied it to train the U-Net model on an X-ray pneumothorax segmentation dataset. Experimental results demonstrate the superiority of our approach over several existing methods. △ Less

Submitted 11 February, 2024; originally announced February 2024.

HAYATE: Photometric redshift estimation by hybridising machine learning with template fitting

Authors: Shingo Tanigawa, Karl Glazebrook, Colin Jacobs, Ivo Labbe, Alex K. Qin

Abstract: Machine learning photo-z methods, trained directly on spectroscopic redshifts, provide a viable alternative to traditional template fitting methods but may not generalise well on new data that deviates from that in the training set. In this work, we present a Hybrid Algorithm for WI(Y)de-range photo-z estimation with Artificial neural networks and TEmplate fitting (HAYATE), a novel photo-z method… ▽ More Machine learning photo-z methods, trained directly on spectroscopic redshifts, provide a viable alternative to traditional template fitting methods but may not generalise well on new data that deviates from that in the training set. In this work, we present a Hybrid Algorithm for WI(Y)de-range photo-z estimation with Artificial neural networks and TEmplate fitting (HAYATE), a novel photo-z method that combines template fitting and data-driven approaches and whose training loss is optimised in terms of both redshift point estimates and probability distributions. We produce artificial training data from low-redshift galaxy SEDs at z<1.3, artificially redshifted up to z=5. We test the model on data from the ZFOURGE surveys, demonstrating that HAYATE can function as a reliable emulator of EAZY for the broad redshift range beyond the region of sufficient spectroscopic completeness. The network achieves precise photo-z estimations with smaller errors ($σ_{NMAD}$) than EAZY in the initial low-z region (z<1.3), while being comparable even in the high-z extrapolated regime (1.3<z<5). Meanwhile, it provides more robust photo-z estimations than EAZY with the lower outlier rate ($η_{0.2}\lesssim 1\%$) but runs $\sim100$ times faster than the original template fitting method. We also demonstrate HAYATE offers more reliable redshift PDFs, showing a flatter distribution of Probability Integral Transform scores than EAZY. The performance is further improved using transfer learning with spec-z samples. We expect that future large surveys will benefit from our novel methodology applicable to observations over a wide redshift range. △ Less

Submitted 31 January, 2024; originally announced February 2024.

Comments: 28 pages, 12 figures, accepted for publication in MNRAS

Towards Lifelong Scene Graph Generation with Knowledge-ware In-context Prompt Learning

Authors: Tao He, Tongtong Wu, Dongyang Zhang, Guiduo Duan, Ke Qin, Yuan-Fang Li

Abstract: Scene graph generation (SGG) endeavors to predict visual relationships between pairs of objects within an image. Prevailing SGG methods traditionally assume a one-off learning process for SGG. This conventional paradigm may necessitate repetitive training on all previously observed samples whenever new relationships emerge, mitigating the risk of forgetting previously acquired knowledge. This work… ▽ More Scene graph generation (SGG) endeavors to predict visual relationships between pairs of objects within an image. Prevailing SGG methods traditionally assume a one-off learning process for SGG. This conventional paradigm may necessitate repetitive training on all previously observed samples whenever new relationships emerge, mitigating the risk of forgetting previously acquired knowledge. This work seeks to address this pitfall inherent in a suite of prior relationship predictions. Motivated by the achievements of in-context learning in pretrained language models, our approach imbues the model with the capability to predict relationships and continuously acquire novel knowledge without succumbing to catastrophic forgetting. To achieve this goal, we introduce a novel and pragmatic framework for scene graph generation, namely Lifelong Scene Graph Generation (LSGG), where tasks, such as predicates, unfold in a streaming fashion. In this framework, the model is constrained to exclusive training on the present task, devoid of access to previously encountered training data, except for a limited number of exemplars, but the model is tasked with inferring all predicates it has encountered thus far. Rigorous experiments demonstrate the superiority of our proposed method over state-of-the-art SGG models in the context of LSGG across a diverse array of metrics. Besides, extensive experiments on the two mainstream benchmark datasets, VG and Open-Image(v6), show the superiority of our proposed model to a number of competitive SGG models in terms of continuous learning and conventional settings. Moreover, comprehensive ablation experiments demonstrate the effectiveness of each component in our model. △ Less

Submitted 25 January, 2024; originally announced January 2024.

Black Hole Ultracompact X-Ray Binaries as Galactic Low-frequency Gravitational Wave Sources: the He Star Channel

Authors: Ke Qin, Kun Xu, Dong-Dong Liu, Long Jiang, Bo Wang, Wen-Cong Chen

Abstract: Black hole (BH) ultracompact X-ray binaries (UCXBs) are potential Galactic low-frequency gravitational wave (GW) sources. As an alternative channel, BH UCXBs can evolve from BH+He star binaries. In this work, we perform a detailed stellar evolution model for the formation and evolution of BH UCXBs evolving from the He star channel to diagnose their detectability as low-frequency GW sources. Our ca… ▽ More Black hole (BH) ultracompact X-ray binaries (UCXBs) are potential Galactic low-frequency gravitational wave (GW) sources. As an alternative channel, BH UCXBs can evolve from BH+He star binaries. In this work, we perform a detailed stellar evolution model for the formation and evolution of BH UCXBs evolving from the He star channel to diagnose their detectability as low-frequency GW sources. Our calculations found that some nascent BH+He star binaries after the common-envelope (CE) phase could evolve into UCXB-LISA sources with a maximum GW frequency of $\sim5~\rm mHz$, which can be detected in a distance of 10 kpc (or 100 kpc). Once BH+He star systems become UCXBs through mass transfer, they would emit X-ray luminosities of $\sim10^{38}~\rm erg\, s^{-1}$, making them ideal multimessenger objects. If the initial He-star masses are $\geq 0.7 M_{\odot}$, those systems are likely to experience two Roche lobe overflows, and the X-ray luminosity can reach a maximum of $3.5\times 10^{39}~\rm erg\, s^{-1}$ in the second mass-transfer stage. The initial He-star masses and initial orbital periods of progenitors of Galactic BH UCXB-LISA sources are in the range of 0.32-2.9 $M_{\odot}$ and 0.02-0.19 days, respectively. Nearly all BH+He star binaries in the above parameter space can evolve into GW sources whose chirp masses can be accurately measured. Employing a population synthesis simulation, we predict the birthrate and detection number of Galactic BH UCXB-LISA source evolving from the He star channel are $R=2.2\times10^{-6}~\rm yr^{-1}$ and 33 for an optimistic CE parameter, respectively. △ Less

Submitted 27 November, 2023; originally announced November 2023.

Comments: 17 pages, 9 figures, ApJ in press

FLrce: Resource-Efficient Federated Learning with Early-Stop** Strategy

Authors: Ziru Niu, Hai Dong, A. Kai Qin, Tao Gu

Abstract: Federated learning (FL) achieves great popularity in the Internet of Things (IoT) as a powerful interface to offer intelligent services to customers while maintaining data privacy. Under the orchestration of a server, edge devices (also called clients in FL) collaboratively train a global deep-learning model without sharing any local data. Nevertheless, the unequal training contributions among cli… ▽ More Federated learning (FL) achieves great popularity in the Internet of Things (IoT) as a powerful interface to offer intelligent services to customers while maintaining data privacy. Under the orchestration of a server, edge devices (also called clients in FL) collaboratively train a global deep-learning model without sharing any local data. Nevertheless, the unequal training contributions among clients have made FL vulnerable, as clients with heavily biased datasets can easily compromise FL by sending malicious or heavily biased parameter updates. Furthermore, the resource shortage issue of edge devices also becomes a bottleneck. Due to overwhelming computation overheads generated by training deep-learning models on edge devices, and significant communication overheads for transmitting deep-learning models across the network, enormous amounts of resources are consumed in the FL process. This encompasses computation resources like energy and communication resources like bandwidth. To comprehensively address these challenges, in this paper, we present FLrce, an efficient FL framework with a relationship-based client selection and early-stop** strategy. FLrce accelerates the FL process by selecting clients with more significant effects, enabling the global model to converge to a high accuracy in fewer rounds. FLrce also leverages an early stop** mechanism that terminates FL in advance to save communication and computation resources. Experiment results show that, compared with existing efficient FL frameworks, FLrce improves the computation and communication efficiency by at least 47% and 43% respectively. △ Less

Submitted 15 February, 2024; v1 submitted 15 October, 2023; originally announced October 2023.

Comments: arxiv preprint

ACM Class: I.2.6

HiSEP-Q: A Highly Scalable and Efficient Quantum Control Processor for Superconducting Qubits

Authors: Xiaorang Guo, Kun Qin, Martin Schulz

Abstract: Quantum computing promises an effective way to solve targeted problems that are classically intractable. Among them, quantum computers built with superconducting qubits are considered one of the most advanced technologies, but they suffer from short coherence times. This can get exaggerated when they are controlled directly by general-purpose host machines, which leads to the loss of quantum infor… ▽ More Quantum computing promises an effective way to solve targeted problems that are classically intractable. Among them, quantum computers built with superconducting qubits are considered one of the most advanced technologies, but they suffer from short coherence times. This can get exaggerated when they are controlled directly by general-purpose host machines, which leads to the loss of quantum information. To mitigate this, we need quantum control processors (QCPs) positioned between quantum processing units and host machines to reduce latencies. However, existing QCPs are built on top of designs with no or inefficient scalability, requiring a large number of instructions when scaling to more qubits. In addition, interactions between current QCPs and host machines require frequent data transmissions and offline computations to obtain final results, which limits the performance of quantum computers. In this paper, we propose a QCP called HiSEP-Q featuring a novel quantum instruction set architecture (QISA) and its microarchitecture implementation. For efficient control, we utilize mixed-type addressing modes and mixed-length instructions in HiSEP-Q, which provides an efficient way to concurrently address more than 100 qubits. Further, for efficient read-out and analysis, we develop a novel onboard accumulation and sorting unit, which eliminates the data transmission of raw data between the QCPs and host machines and enables real-time result processing. Compared to the state-of-the-art, our proposed QISA achieves at least 62% and 28% improvements in encoding efficiency with real and synthetic quantum circuits, respectively. We also validate the microarchitecture on a field-programmable gate array, which exhibits low power and resource consumption. Both hardware and ISA evaluations demonstrate that HiSEP-Q features high scalability and efficiency toward the number of controlled qubits. △ Less

Submitted 31 August, 2023; originally announced August 2023.

Comments: The paper is accepted by the 41st IEEE International Conference on Computer Design (ICCD), 2023

One-Nearest Neighborhood Guides Inlier Estimation for Unsupervised Point Cloud Registration

Authors: Yongzhe Yuan, Yue Wu, Maoguo Gong, Qiguang Miao, A. K. Qin

Abstract: The precision of unsupervised point cloud registration methods is typically limited by the lack of reliable inlier estimation and self-supervised signal, especially in partially overlap** scenarios. In this paper, we propose an effective inlier estimation method for unsupervised point cloud registration by capturing geometric structure consistency between the source point cloud and its correspon… ▽ More The precision of unsupervised point cloud registration methods is typically limited by the lack of reliable inlier estimation and self-supervised signal, especially in partially overlap** scenarios. In this paper, we propose an effective inlier estimation method for unsupervised point cloud registration by capturing geometric structure consistency between the source point cloud and its corresponding reference point cloud copy. Specifically, to obtain a high quality reference point cloud copy, an One-Nearest Neighborhood (1-NN) point cloud is generated by input point cloud. This facilitates matching map construction and allows for integrating dual neighborhood matching scores of 1-NN point cloud and input point cloud to improve matching confidence. Benefiting from the high quality reference copy, we argue that the neighborhood graph formed by inlier and its neighborhood should have consistency between source point cloud and its corresponding reference copy. Based on this observation, we construct transformation-invariant geometric structure representations and capture geometric structure consistency to score the inlier confidence for estimated correspondences between source point cloud and its reference copy. This strategy can simultaneously provide the reliable self-supervised signal for model optimization. Finally, we further calculate transformation estimation by the weighted SVD algorithm with the estimated correspondences and corresponding inlier confidence. We train the proposed model in an unsupervised manner, and extensive experiments on synthetic and real-world datasets illustrate the effectiveness of the proposed method. △ Less

Submitted 26 July, 2023; originally announced July 2023.

What Drives the (In)stability of a Stablecoin?

Authors: Yu** Kwon, Kornrapat Pongmala, Kaihua Qin, Ariah Klages-Mundt, Philipp Jovanovic, Christine Parlour, Arthur Gervais, Dawn Song

Abstract: In May 2022, an apparent speculative attack, followed by market panic, led to the precipitous downfall of UST, one of the most popular stablecoins at that time. However, UST is not the only stablecoin to have been depegged in the past. Designing resilient and long-term stable coins, therefore, appears to present a hard challenge. To further scrutinize existing stablecoin designs and ultimately l… ▽ More In May 2022, an apparent speculative attack, followed by market panic, led to the precipitous downfall of UST, one of the most popular stablecoins at that time. However, UST is not the only stablecoin to have been depegged in the past. Designing resilient and long-term stable coins, therefore, appears to present a hard challenge. To further scrutinize existing stablecoin designs and ultimately lead to more robust systems, we need to understand where volatility emerges. Our work provides a game-theoretical model aiming to help identify why stablecoins suffer from a depeg. This game-theoretical model reveals that stablecoins have different price equilibria depending on the coin's architecture and mechanism to minimize volatility. Moreover, our theory is supported by extensive empirical data, spanning $1$ year. To that end, we collect daily prices for 22 stablecoins and on-chain data from five blockchains including the Ethereum and the Terra blockchain. △ Less

Submitted 25 July, 2023; v1 submitted 14 June, 2023; originally announced July 2023.

Training Physics-Informed Neural Networks via Multi-Task Optimization for Traffic Density Prediction

Authors: Bo Wang, A. K. Qin, Sajjad Shafiei, Hussein Dia, Adriana-Simona Mihaita, Hanna Grzybowska

Abstract: Physics-informed neural networks (PINNs) are a newly emerging research frontier in machine learning, which incorporate certain physical laws that govern a given data set, e.g., those described by partial differential equations (PDEs), into the training of the neural network (NN) based on such a data set. In PINNs, the NN acts as the solution approximator for the PDE while the PDE acts as the prior… ▽ More Physics-informed neural networks (PINNs) are a newly emerging research frontier in machine learning, which incorporate certain physical laws that govern a given data set, e.g., those described by partial differential equations (PDEs), into the training of the neural network (NN) based on such a data set. In PINNs, the NN acts as the solution approximator for the PDE while the PDE acts as the prior knowledge to guide the NN training, leading to the desired generalization performance of the NN when facing the limited availability of training data. However, training PINNs is a non-trivial task largely due to the complexity of the loss composed of both NN and physical law parts. In this work, we propose a new PINN training framework based on the multi-task optimization (MTO) paradigm. Under this framework, multiple auxiliary tasks are created and solved together with the given (main) task, where the useful knowledge from solving one task is transferred in an adaptive mode to assist in solving some other tasks, aiming to uplift the performance of solving the main task. We implement the proposed framework and apply it to train the PINN for addressing the traffic density prediction problem. Experimental results demonstrate that our proposed training framework leads to significant performance improvement in comparison to the traditional way of training the PINN. △ Less

Submitted 8 July, 2023; originally announced July 2023.

Comments: accepted by the 2023 IEEE International Joint Conference on Neural Networks (IJCNN 2023)

Do you still need a manual smart contract audit?

Authors: Isaac David, Liyi Zhou, Kaihua Qin, Dawn Song, Lorenzo Cavallaro, Arthur Gervais

Abstract: We investigate the feasibility of employing large language models (LLMs) for conducting the security audit of smart contracts, a traditionally time-consuming and costly process. Our research focuses on the optimization of prompt engineering for enhanced security analysis, and we evaluate the performance and accuracy of LLMs using a benchmark dataset comprising 52 Decentralized Finance (DeFi) smart… ▽ More We investigate the feasibility of employing large language models (LLMs) for conducting the security audit of smart contracts, a traditionally time-consuming and costly process. Our research focuses on the optimization of prompt engineering for enhanced security analysis, and we evaluate the performance and accuracy of LLMs using a benchmark dataset comprising 52 Decentralized Finance (DeFi) smart contracts that have previously been compromised. Our findings reveal that, when applied to vulnerable contracts, both GPT-4 and Claude models correctly identify the vulnerability type in 40% of the cases. However, these models also demonstrate a high false positive rate, necessitating continued involvement from manual auditors. The LLMs tested outperform a random model by 20% in terms of F1-score. To ensure the integrity of our study, we conduct mutation testing on five newly developed and ostensibly secure smart contracts, into which we manually insert two and 15 vulnerabilities each. This testing yielded a remarkable best-case 78.7% true positive rate for the GPT-4-32k model. We tested both, asking the models to perform a binary classification on whether a contract is vulnerable, and a non-binary prompt. We also examined the influence of model temperature variations and context length on the LLM's performance. Despite the potential for many further enhancements, this work lays the groundwork for a more efficient and economical approach to smart contract security audits. △ Less

Submitted 22 June, 2023; v1 submitted 21 June, 2023; originally announced June 2023.

A System of Monitoring and Analyzing Human Indoor Mobility and Air Quality

Authors: Kyle K. Qin, Mohammad S. Rahaman, Yongli Ren, Chi-Tsun Cheng, Ivan Cole, Flora D. Salim

Abstract: Human movements in the workspace usually have non-negligible relations with air quality parameters (e.g., CO$_2$, PM2.5, and PM10). We establish a system to monitor indoor human mobility with air quality and assess the interrelationship between these two types of time series data. More specifically, a sensor network was designed in indoor environments to observe air quality parameters continuously… ▽ More Human movements in the workspace usually have non-negligible relations with air quality parameters (e.g., CO$_2$, PM2.5, and PM10). We establish a system to monitor indoor human mobility with air quality and assess the interrelationship between these two types of time series data. More specifically, a sensor network was designed in indoor environments to observe air quality parameters continuously. Simultaneously, another sensing module detected participants' movements around the study areas. In this module, modern data analysis and machine learning techniques have been applied to reconstruct the trajectories of participants with relevant sensor information. Finally, a further study revealed the correlation between human indoor mobility patterns and indoor air quality parameters. Our experimental results demonstrate that human movements in different environments can significantly impact air quality during busy hours. With the results, we propose recommendations for future studies. △ Less

Submitted 20 June, 2023; originally announced June 2023.

Comments: 7 pages, accepted by the 24th IEEE International Conference on Mobile Data Management

MSC Class: 68T07 ACM Class: J.0

Graph Entropy Minimization for Semi-supervised Node Classification

Authors: Yi Luo, Guangchun Luo, Ke Qin, Aiguo Chen

Abstract: Node classifiers are required to comprehensively reduce prediction errors, training resources, and inference latency in the industry. However, most graph neural networks (GNN) concentrate only on one or two of them. The compromised aspects thus are the shortest boards on the bucket, hindering their practical deployments for industrial-level tasks. This work proposes a novel semi-supervised learnin… ▽ More Node classifiers are required to comprehensively reduce prediction errors, training resources, and inference latency in the industry. However, most graph neural networks (GNN) concentrate only on one or two of them. The compromised aspects thus are the shortest boards on the bucket, hindering their practical deployments for industrial-level tasks. This work proposes a novel semi-supervised learning method termed Graph Entropy Minimization (GEM) to resolve the three issues simultaneously. GEM benefits its one-hop aggregation from massive uncategorized nodes, making its prediction accuracy comparable to GNNs with two or more hops message passing. It can be decomposed to support stochastic training with mini-batches of independent edge samples, achieving extremely fast sampling and space-saving training. While its one-hop aggregation is faster in inference than deep GNNs, GEM can be further accelerated to an extreme by deriving a non-hop classifier via online knowledge distillation. Thus, GEM can be a handy choice for latency-restricted and error-sensitive services running on resource-constraint hardware. Code is available at https://github.com/cf020031308/GEM. △ Less

Submitted 30 May, 2023; originally announced May 2023.

Comments: 12 pages, 3 figures, 4 tables

Blockchain Censorship

Authors: Anton Wahrstätter, Jens Ernstberger, Aviv Yaish, Liyi Zhou, Kaihua Qin, Taro Tsuchiya, Sebastian Steinhorst, Davor Svetinovic, Nicolas Christin, Mikolaj Barczentewicz, Arthur Gervais

Abstract: Permissionless blockchains promise to be resilient against censorship by a single entity. This suggests that deterministic rules, and not third-party actors, are responsible for deciding if a transaction is appended to the blockchain or not. In 2022, the U.S. Office of Foreign Assets Control (OFAC) sanctioned a Bitcoin mixer and an Ethereum application, putting the neutrality of permissionless blo… ▽ More Permissionless blockchains promise to be resilient against censorship by a single entity. This suggests that deterministic rules, and not third-party actors, are responsible for deciding if a transaction is appended to the blockchain or not. In 2022, the U.S. Office of Foreign Assets Control (OFAC) sanctioned a Bitcoin mixer and an Ethereum application, putting the neutrality of permissionless blockchains to the test. In this paper, we formalize quantify and analyze the security impact of blockchain censorship. We start by defining censorship, followed by a quantitative assessment of current censorship practices. We find that 46% of Ethereum blocks were made by censoring actors that intend to comply with OFAC sanctions, indicating the significant impact of OFAC sanctions on the neutrality of public blockchains. We further uncover that censorship not only impacts neutrality, but also security. We show how after Ethereum's move to Proof-of-Stake (PoS) and adoption of Proposer-Builder Separation (PBS) the inclusion of censored transactions was delayed by an average of 85%. Inclusion delays compromise a transaction's security by, e.g., strengthening a sandwich adversary. Finally we prove a fundamental limitation of PoS and Proof-of-Work (PoW) protocols against censorship resilience. △ Less

Submitted 2 June, 2023; v1 submitted 29 May, 2023; originally announced May 2023.

Time to Bribe: Measuring Block Construction Market

Authors: Anton Wahrstätter, Liyi Zhou, Kaihua Qin, Davor Svetinovic, Arthur Gervais

Abstract: With the emergence of Miner Extractable Value (MEV), block construction markets on blockchains have evolved into a competitive arena. Following Ethereum's transition from Proof of Work (PoW) to Proof of Stake (PoS), the Proposer Builder Separation (PBS) mechanism has emerged as the dominant force in the Ethereum block construction market. This paper presents an in-depth longitudinal study of the… ▽ More With the emergence of Miner Extractable Value (MEV), block construction markets on blockchains have evolved into a competitive arena. Following Ethereum's transition from Proof of Work (PoW) to Proof of Stake (PoS), the Proposer Builder Separation (PBS) mechanism has emerged as the dominant force in the Ethereum block construction market. This paper presents an in-depth longitudinal study of the Ethereum block construction market, spanning from the introduction of PoS and PBS in September 2022 to May 2023. We analyze the market shares of builders and relays, their temporal changes, and the financial dynamics within the PBS system, including payments among builders and block proposers -- commonly referred to as bribes. We introduce an MEV-time law quantifying the expected MEV revenue wrt. the time elapsed since the last proposed block. We provide empirical evidence that moments of crisis (e.g. the FTX collapse, USDC stablecoin de-peg) coincide with significant spikes in MEV payments compared to the baseline. Despite the intention of the PBS architecture to enhance decentralization by separating actor roles, it remains unclear whether its design is optimal. Implicit trust assumptions and conflicts of interest may benefit particular parties and foster the need for vertical integration. MEV-Boost was explicitly designed to foster decentralization, causing the side effect of enabling risk-free sandwich extraction from unsuspecting users, potentially raising concerns for regulators. △ Less

Submitted 2 June, 2023; v1 submitted 25 May, 2023; originally announced May 2023.

Towards Automated Security Analysis of Smart Contracts based on Execution Property Graph

Authors: Kaihua Qin, Zhe Ye, Zhun Wang, Weilin Li, Liyi Zhou, Chao Zhang, Dawn Song, Arthur Gervais

Abstract: Identifying and mitigating vulnerabilities in smart contracts is crucial, especially considering the rapid growth and increasing complexity of Decentralized Finance (DeFi) platforms. To address the challenges associated with securing these contracts, we introduce a versatile dynamic analysis framework specifically designed for the Ethereum Virtual Machine (EVM). This comprehensive framework focuse… ▽ More Identifying and mitigating vulnerabilities in smart contracts is crucial, especially considering the rapid growth and increasing complexity of Decentralized Finance (DeFi) platforms. To address the challenges associated with securing these contracts, we introduce a versatile dynamic analysis framework specifically designed for the Ethereum Virtual Machine (EVM). This comprehensive framework focuses on tracking contract executions, capturing valuable runtime information, while introducing and employing the Execution Property Graph (EPG) to propose a unique graph traversal technique that swiftly detects potential smart contract attacks. Our approach showcases its efficacy with rapid average graph traversal time per transaction and high true positive rates. The successful identification of a zero-day vulnerability affecting Uniswap highlights the framework's potential to effectively uncover smart contract vulnerabilities in complex DeFi systems. △ Less

Submitted 24 May, 2023; v1 submitted 23 May, 2023; originally announced May 2023.

Blockchain Large Language Models

Authors: Yu Gai, Liyi Zhou, Kaihua Qin, Dawn Song, Arthur Gervais

Abstract: This paper presents a dynamic, real-time approach to detecting anomalous blockchain transactions. The proposed tool, BlockGPT, generates tracing representations of blockchain activity and trains from scratch a large language model to act as a real-time Intrusion Detection System. Unlike traditional methods, BlockGPT is designed to offer an unrestricted search space and does not rely on predefined… ▽ More This paper presents a dynamic, real-time approach to detecting anomalous blockchain transactions. The proposed tool, BlockGPT, generates tracing representations of blockchain activity and trains from scratch a large language model to act as a real-time Intrusion Detection System. Unlike traditional methods, BlockGPT is designed to offer an unrestricted search space and does not rely on predefined rules or patterns, enabling it to detect a broader range of anomalies. We demonstrate the effectiveness of BlockGPT through its use as an anomaly detection tool for Ethereum transactions. In our experiments, it effectively identifies abnormal transactions among a dataset of 68M transactions and has a batched throughput of 2284 transactions per second on average. Our results show that, BlockGPT identifies abnormal transactions by ranking 49 out of 124 attacks among the top-3 most abnormal transactions interacting with their victim contracts. This work makes contributions to the field of blockchain transaction analysis by introducing a custom data encoding compatible with the transformer architecture, a domain-specific tokenization technique, and a tree encoding method specifically crafted for the Ethereum Virtual Machine (EVM) trace representation. △ Less

Submitted 29 April, 2023; v1 submitted 25 April, 2023; originally announced April 2023.

The Blockchain Imitation Game

Authors: Kaihua Qin, Stefanos Chaliasos, Liyi Zhou, Benjamin Livshits, Dawn Song, Arthur Gervais

Abstract: The use of blockchains for automated and adversarial trading has become commonplace. However, due to the transparent nature of blockchains, an adversary is able to observe any pending, not-yet-mined transactions, along with their execution logic. This transparency further enables a new type of adversary, which copies and front-runs profitable pending transactions in real-time, yielding significant… ▽ More The use of blockchains for automated and adversarial trading has become commonplace. However, due to the transparent nature of blockchains, an adversary is able to observe any pending, not-yet-mined transactions, along with their execution logic. This transparency further enables a new type of adversary, which copies and front-runs profitable pending transactions in real-time, yielding significant financial gains. Shedding light on such "copy-paste" malpractice, this paper introduces the Blockchain Imitation Game and proposes a generalized imitation attack methodology called Ape. Leveraging dynamic program analysis techniques, Ape supports the automatic synthesis of adversarial smart contracts. Over a timeframe of one year (1st of August, 2021 to 31st of July, 2022), Ape could have yielded 148.96M USD in profit on Ethereum, and 42.70M USD on BNB Smart Chain (BSC). Not only as a malicious attack, we further show the potential of transaction and contract imitation as a defensive strategy. Within one year, we find that Ape could have successfully imitated 13 and 22 known Decentralized Finance (DeFi) attacks on Ethereum and BSC, respectively. Our findings suggest that blockchain validators can imitate attacks in real-time to prevent intrusions in DeFi. △ Less

Submitted 31 March, 2023; originally announced March 2023.

Mitigating Decentralized Finance Liquidations with Reversible Call Options

Authors: Kaihua Qin, Jens Ernstberger, Liyi Zhou, Philipp Jovanovic, Arthur Gervais

Abstract: Liquidations in Decentralized Finance (DeFi) are both a blessing and a curse -- whereas liquidations prevent lenders from capital loss, they simultaneously lead to liquidation spirals and system-wide failures. Since most lending and borrowing protocols assume liquidations are indispensable, there is an increased interest in alternative constructions that prevent immediate systemic-failure under un… ▽ More Liquidations in Decentralized Finance (DeFi) are both a blessing and a curse -- whereas liquidations prevent lenders from capital loss, they simultaneously lead to liquidation spirals and system-wide failures. Since most lending and borrowing protocols assume liquidations are indispensable, there is an increased interest in alternative constructions that prevent immediate systemic-failure under uncertain circumstances. In this work, we introduce reversible call options, a novel financial primitive that enables the seller of a call option to terminate it before maturity. We apply reversible call options to lending in DeFi and devise Miqado, a protocol for lending platforms to replace the liquidation mechanisms. To the best of our knowledge, Miqado is the first protocol that actively mitigates liquidations to reduce the risk of liquidation spirals. Instead of selling collateral, Miqado incentivizes external entities, so-called supporters, to top-up a borrowing position and grant the borrower additional time to rescue the debt. Our simulation shows that Miqado reduces the amount of liquidated collateral by 89.82% in a worst-case scenario. △ Less

Submitted 27 March, 2023; v1 submitted 10 February, 2023; originally announced March 2023.

TrafFormer: A Transformer Model for Predicting Long-term Traffic

Authors: David Alexander Tedjopurnomo, Farhana M. Choudhury, A. K. Qin

Abstract: Traffic prediction is a flourishing research field due to its importance in human mobility in the urban space. Despite this, existing studies only focus on short-term prediction of up to few hours in advance, with most being up to one hour only. Long-term traffic prediction can enable more comprehensive, informed, and proactive measures against traffic congestion and is therefore an important task… ▽ More Traffic prediction is a flourishing research field due to its importance in human mobility in the urban space. Despite this, existing studies only focus on short-term prediction of up to few hours in advance, with most being up to one hour only. Long-term traffic prediction can enable more comprehensive, informed, and proactive measures against traffic congestion and is therefore an important task to explore. In this paper, we explore the task of long-term traffic prediction; where we predict traffic up to 24 hours in advance. We note the weaknesses of existing models--which are based on recurrent structures--for long-term traffic prediction and propose a modified Transformer model "TrafFormer". Experiments comparing our model with existing hybrid neural network models show the superiority of our model. △ Less

Submitted 2 March, 2023; v1 submitted 23 February, 2023; originally announced February 2023.

Comments: 14 pages, 6 figures

MSC Class: ACM-class: I.2.1

Black Hole Ultra-compact X-ray Binaries: Galactic Low-Frequency Gravitational Wave Sources

Authors: Ke Qin, Long Jiang, Wen-Cong Chen

Abstract: In the Galaxy, close binaries with compact objects are important low-frequency gravitational wave (GW) sources. As potential low-frequency GW sources, neutron star/white dwarf (WD) ultra-compact X-ray binaries (UCXBs) have been investigated extensively. Using the MESA code, we systematically explored the evolution of black hole (BH)-main sequence star (MS) binaries to diagnose whether their descen… ▽ More In the Galaxy, close binaries with compact objects are important low-frequency gravitational wave (GW) sources. As potential low-frequency GW sources, neutron star/white dwarf (WD) ultra-compact X-ray binaries (UCXBs) have been investigated extensively. Using the MESA code, we systematically explored the evolution of black hole (BH)-main sequence star (MS) binaries to diagnose whether their descendants can be detected by space-borne GW detectors. Our simulations show that BH-MS binaries with an initial orbital period less than the bifurcation period can evolve into BH UCXBs that can be detected by LISA. Such an evolutionary channel would form compact mass-transferring BH-WD systems rather than detached BH-WD systems. The calculated X-ray luminosities of BH UCXBs detected by LISA at a distance $d=1$ kpc are $\sim10^{33}-10^{35}~\rm erg\,s^{-1}$ ($\sim10^{34}-10^{35}~\rm erg\,s^{-1}$ for $d=10$ kpc), hence it is possible to detect their electromagnetic counterparts. It is worth emphasizing only some BH-MS systems with an initial orbital period very close to the bifurcation period can evolve toward low-frequency GW sources whose chirp masses can be measured. The maximum GW frequency of BH UCXBs forming by BH-MS pathway is about 3 mHz, which is smaller than the minimum GW frequency (6.4 mHz) of mass-transferring BH-WD originating from a dynamic process. Furthermore, we obtain an initial parameter space (donor-star masses and orbital periods) of progenitors of BH UCXB-GW sources, which can be applied to future population synthesis simulations. By a rough estimation, we predict that LISA could detect only a few BH UCXB-GW sources forming by the BH-MS channel. △ Less

Submitted 15 January, 2023; originally announced January 2023.

Comments: 15 pages, 8 figures, 1 table. ApJ in press

Multiple-level Point Embedding for Solving Human Trajectory Imputation with Prediction

Authors: Kyle K. Qin, Yongli Ren, Wei Shao, Brennan Lake, Filippo Privitera, Flora D. Salim

Abstract: Sparsity is a common issue in many trajectory datasets, including human mobility data. This issue frequently brings more difficulty to relevant learning tasks, such as trajectory imputation and prediction. Nowadays, little existing work simultaneously deals with imputation and prediction on human trajectories. This work plans to explore whether the learning process of imputation and prediction cou… ▽ More Sparsity is a common issue in many trajectory datasets, including human mobility data. This issue frequently brings more difficulty to relevant learning tasks, such as trajectory imputation and prediction. Nowadays, little existing work simultaneously deals with imputation and prediction on human trajectories. This work plans to explore whether the learning process of imputation and prediction could benefit from each other to achieve better outcomes. And the question will be answered by studying the coexistence patterns between missing points and observed ones in incomplete trajectories. More specifically, the proposed model develops an imputation component based on the self-attention mechanism to capture the coexistence patterns between observations and missing points among encoder-decoder layers. Meanwhile, a recurrent unit is integrated to extract the sequential embeddings from newly imputed sequences for predicting the following location. Furthermore, a new implementation called Imputation Cycle is introduced to enable gradual imputation with prediction enhancement at multiple levels, which helps to accelerate the speed of convergence. The experimental results on three different real-world mobility datasets show that the proposed approach has significant advantages over the competitive baselines across both imputation and prediction tasks in terms of accuracy and stability. △ Less

Submitted 12 January, 2023; v1 submitted 11 January, 2023; originally announced January 2023.

Comments: 22 pages; accepted by ACM Transactions on Spatial Algorithms and Systems

MSC Class: 68T07 ACM Class: H.0

VQA and Visual Reasoning: An Overview of Recent Datasets, Methods and Challenges

Authors: Rufai Yusuf Zakari, Jim Wilson Owusu, Hailin Wang, Ke Qin, Zaharaddeen Karami Lawal, Yuezhou Dong

Abstract: Artificial Intelligence (AI) and its applications have sparked extraordinary interest in recent years. This achievement can be ascribed in part to advances in AI subfields including Machine Learning (ML), Computer Vision (CV), and Natural Language Processing (NLP). Deep learning, a sub-field of machine learning that employs artificial neural network concepts, has enabled the most rapid growth in t… ▽ More Artificial Intelligence (AI) and its applications have sparked extraordinary interest in recent years. This achievement can be ascribed in part to advances in AI subfields including Machine Learning (ML), Computer Vision (CV), and Natural Language Processing (NLP). Deep learning, a sub-field of machine learning that employs artificial neural network concepts, has enabled the most rapid growth in these domains. The integration of vision and language has sparked a lot of attention as a result of this. The tasks have been created in such a way that they properly exemplify the concepts of deep learning. In this review paper, we provide a thorough and an extensive review of the state of the arts approaches, key models design principles and discuss existing datasets, methods, their problem formulation and evaluation measures for VQA and Visual reasoning tasks to understand vision and language representation learning. We also present some potential future paths in this field of research, with the hope that our study may generate new ideas and novel approaches to handle existing difficulties and develop new applications. △ Less

Submitted 26 December, 2022; originally announced December 2022.

Deep Edge Intelligence: Architecture, Key Features, Enabling Technologies and Challenges

Authors: Prabath Abeysekara, Hai Dong, A. K. Qin

Abstract: With the breakthroughs in Deep Learning, recent years have witnessed a massive surge in Artificial Intelligence applications and services. Meanwhile, the rapid advances in Mobile Computing and Internet of Things has also given rise to billions of mobile and smart sensing devices connected to the Internet, generating zettabytes of data at the network edge. The opportunity to combine these two domai… ▽ More With the breakthroughs in Deep Learning, recent years have witnessed a massive surge in Artificial Intelligence applications and services. Meanwhile, the rapid advances in Mobile Computing and Internet of Things has also given rise to billions of mobile and smart sensing devices connected to the Internet, generating zettabytes of data at the network edge. The opportunity to combine these two domains of technologies to power interconnected devices with intelligence is likely to pave the way for a new wave of technology revolutions. Embracing this technology revolution, in this article, we present a novel computing vision named Deep Edge Intelligence (DEI). DEI employs Deep Learning, Artificial Intelligence, Cloud and Edge Computing, 5G/6G networks, Internet of Things, Microservices, etc. aiming to provision reliable and secure intelligence services to every person and organisation at any place with better user experience. The vision, system architecture, key layers and features of DEI are also detailed. Finally, we reveal the key enabling technologies and research challenges associated with it. △ Less

Submitted 24 October, 2022; originally announced October 2022.

Traffic disruption modelling with mode shift in multi-modal networks

Authors: Dong Zhao, Adriana-Simona Mihaita, Yuming Ou, Sajjad Shafiei, Hanna Grzybowska, A. K. Qin, Gary Tan, Mo Li, Hussein Dia

Abstract: A multi-modal transport system is acknowledged to have robust failure tolerance and can effectively relieve urban congestion issues. However, estimating the impact of disruptions across multi-transport modes is a challenging problem due to a dis-aggregated modelling approach applied to only individual modes at a time. To fill this gap, this paper proposes a new integrated modelling framework for a… ▽ More A multi-modal transport system is acknowledged to have robust failure tolerance and can effectively relieve urban congestion issues. However, estimating the impact of disruptions across multi-transport modes is a challenging problem due to a dis-aggregated modelling approach applied to only individual modes at a time. To fill this gap, this paper proposes a new integrated modelling framework for a multi-modal traffic state estimation and evaluation of the disruption impact across all modes under various traffic conditions. First, we propose an iterative trip assignment model to elucidate the association between travel demand and travel behaviour, including a multi-modal origin-to-destination estimation for private and public transport. Secondly, we provide a practical multi-modal travel demand re-adjustment that takes the mode shift of the affected travellers into consideration. The pros and cons of the mode shift strategy are showcased via several scenario-based transport simulating experiments. The results show that a well-balanced mode shift with flexible routing and early announcements of detours so that travellers can plan ahead can significantly benefit all travellers by a delay time reduction of 46%, while a stable route assignment maintains a higher average traffic flow and the inactive mode-route choice help relief density under the traffic disruptions. △ Less

Submitted 12 October, 2022; originally announced October 2022.

Comments: 8 pages, 7 figures, 1 table, The 25th IEEE International Conference on Intelligent Transportation Systems (IEEE ITSC 2022)

SoK: Decentralized Finance (DeFi) Attacks

Authors: Liyi Zhou, Xihan Xiong, Jens Ernstberger, Stefanos Chaliasos, Zhipeng Wang, Ye Wang, Kaihua Qin, Roger Wattenhofer, Dawn Song, Arthur Gervais

Abstract: Within just four years, the blockchain-based Decentralized Finance (DeFi) ecosystem has accumulated a peak total value locked (TVL) of more than 253 billion USD. This surge in DeFi's popularity has, unfortunately, been accompanied by many impactful incidents. According to our data, users, liquidity providers, speculators, and protocol operators suffered a total loss of at least 3.24 billion USD fr… ▽ More Within just four years, the blockchain-based Decentralized Finance (DeFi) ecosystem has accumulated a peak total value locked (TVL) of more than 253 billion USD. This surge in DeFi's popularity has, unfortunately, been accompanied by many impactful incidents. According to our data, users, liquidity providers, speculators, and protocol operators suffered a total loss of at least 3.24 billion USD from Apr 30, 2018 to Apr 30, 2022. Given the blockchain's transparency and increasing incident frequency, two questions arise: How can we systematically measure, evaluate, and compare DeFi incidents? How can we learn from past attacks to strengthen DeFi security? In this paper, we introduce a common reference frame to systematically evaluate and compare DeFi incidents, including both attacks and accidents. We investigate 77 academic papers, 30 audit reports, and 181 real-world incidents. Our data reveals several gaps between academia and the practitioners' community. For example, few academic papers address "price oracle attacks" and "permissonless interactions", while our data suggests that they are the two most frequent incident types (15% and 10.5% correspondingly). We also investigate potential defenses, and find that: (i) 103 (56%) of the attacks are not executed atomically, granting a rescue time frame for defenders; (ii) SoTA bytecode similarity analysis can at least detect 31 vulnerable/23 adversarial contracts; and (iii) 33 (15.3%) of the adversaries leak potentially identifiable information by interacting with centralized exchanges. △ Less

Submitted 7 April, 2023; v1 submitted 27 August, 2022; originally announced August 2022.

Multi-task Optimization Based Co-training for Electricity Consumption Prediction

Authors: Hui Song, A. K. Qin, Chenggang Yan

Abstract: Real-world electricity consumption prediction may involve different tasks, e.g., prediction for different time steps ahead or different geo-locations. These tasks are often solved independently without utilizing some common problem-solving knowledge that could be extracted and shared among these tasks to augment the performance of solving each task. In this work, we propose a multi-task optimizati… ▽ More Real-world electricity consumption prediction may involve different tasks, e.g., prediction for different time steps ahead or different geo-locations. These tasks are often solved independently without utilizing some common problem-solving knowledge that could be extracted and shared among these tasks to augment the performance of solving each task. In this work, we propose a multi-task optimization (MTO) based co-training (MTO-CT) framework, where the models for solving different tasks are co-trained via an MTO paradigm in which solving each task may benefit from the knowledge gained from when solving some other tasks to help its solving process. MTO-CT leverages long short-term memory (LSTM) based model as the predictor where the knowledge is represented via connection weights and biases. In MTO-CT, an inter-task knowledge transfer module is designed to transfer knowledge between different tasks, where the most helpful source tasks are selected by using the probability matching and stochastic universal selection, and evolutionary operations like mutation and crossover are performed for reusing the knowledge from selected source tasks in a target task. We use electricity consumption data from five states in Australia to design two sets of tasks at different scales: a) one-step ahead prediction for each state (five tasks) and b) 6-step, 12-step, 18-step, and 24-step ahead prediction for each state (20 tasks). The performance of MTO-CT is evaluated on solving each of these two sets of tasks in comparison to solving each task in the set independently without knowledge sharing under the same settings, which demonstrates the superiority of MTO-CT in terms of prediction accuracy. △ Less

Submitted 31 May, 2022; originally announced May 2022.

Comments: accepted by the 2022 IEEE International Joint Conference on Neural Networks (IJCNN 2022)

Sample-Efficient, Exploration-Based Policy Optimisation for Routing Problems

Authors: Nasrin Sultana, Jeffrey Chan, Tabinda Sarwar, A. K. Qin

Abstract: Model-free deep-reinforcement-based learning algorithms have been applied to a range of COPs~\cite{bello2016neural}~\cite{kool2018attention}~\cite{nazari2018reinforcement}. However, these approaches suffer from two key challenges when applied to combinatorial problems: insufficient exploration and the requirement of many training examples of the search space to achieve reasonable performance. Comb… ▽ More Model-free deep-reinforcement-based learning algorithms have been applied to a range of COPs~\cite{bello2016neural}~\cite{kool2018attention}~\cite{nazari2018reinforcement}. However, these approaches suffer from two key challenges when applied to combinatorial problems: insufficient exploration and the requirement of many training examples of the search space to achieve reasonable performance. Combinatorial optimisation can be complex, characterised by search spaces with many optimas and large spaces to search and learn. Therefore, a new method is needed to find good solutions that are more efficient by being more sample efficient. This paper presents a new reinforcement learning approach that is based on entropy. In addition, we design an off-policy-based reinforcement learning technique that maximises the expected return and improves the sample efficiency to achieve faster learning during training time. We systematically evaluate our approach on a range of route optimisation tasks typically used to evaluate learning-based optimisation, such as the such as the Travelling Salesman problems (TSP), Capacitated Vehicle Routing Problem (CVRP). In this paper, we show that our model can generalise to various route problems, such as the split-delivery VRP (SDVRP), and compare the performance of our method with that of current state-of-the-art approaches. The Empirical results show that the proposed method can improve on state-of-the-art methods in terms of solution quality and computation time and generalise to problems of different sizes. △ Less

Submitted 31 May, 2022; originally announced May 2022.

A General Multiple Data Augmentation Based Framework for Training Deep Neural Networks

Authors: Binyan Hu, Yu Sun, A. K. Qin

Abstract: Deep neural networks (DNNs) often rely on massive labelled data for training, which is inaccessible in many applications. Data augmentation (DA) tackles data scarcity by creating new labelled data from available ones. Different DA methods have different mechanisms and therefore using their generated labelled data for DNN training may help improving DNN's generalisation to different degrees. Combin… ▽ More Deep neural networks (DNNs) often rely on massive labelled data for training, which is inaccessible in many applications. Data augmentation (DA) tackles data scarcity by creating new labelled data from available ones. Different DA methods have different mechanisms and therefore using their generated labelled data for DNN training may help improving DNN's generalisation to different degrees. Combining multiple DA methods, namely multi-DA, for DNN training, provides a way to boost generalisation. Among existing multi-DA based DNN training methods, those relying on knowledge distillation (KD) have received great attention. They leverage knowledge transfer to utilise the labelled data sets created by multiple DA methods instead of directly combining them for training DNNs. However, existing KD-based methods can only utilise certain types of DA methods, incapable of utilising the advantages of arbitrary DA methods. We propose a general multi-DA based DNN training framework capable to use arbitrary DA methods. To train a DNN, our framework replicates a certain portion in the latter part of the DNN into multiple copies, leading to multiple DNNs with shared blocks in their former parts and independent blocks in their latter parts. Each of these DNNs is associated with a unique DA and a newly devised loss that allows comprehensively learning from the data generated by all DA methods and the outputs from all DNNs in an online and adaptive way. The overall loss, i.e., the sum of each DNN's loss, is used for training the DNN. Eventually, one of the DNNs with the best validation performance is chosen for inference. We implement the proposed framework by using three distinct DA methods and apply it for training representative DNNs. Experiments on the popular benchmarks of image classification demonstrate the superiority of our method to several existing single-DA and multi-DA based training methods. △ Less

Submitted 29 May, 2022; originally announced May 2022.

Comments: accepted by the 2022 IEEE International Joint Conference on Neural Networks (IJCNN 2022)

All Grains, One Scheme (AGOS): Learning Multi-grain Instance Representation for Aerial Scene Classification

Authors: Qi Bi, Beichen Zhou, Kun Qin, Qinghao Ye, Gui-Song Xia

Abstract: Aerial scene classification remains challenging as: 1) the size of key objects in determining the scene scheme varies greatly; 2) many objects irrelevant to the scene scheme are often flooded in the image. Hence, how to effectively perceive the region of interests (RoIs) from a variety of sizes and build more discriminative representation from such complicated object distribution is vital to under… ▽ More Aerial scene classification remains challenging as: 1) the size of key objects in determining the scene scheme varies greatly; 2) many objects irrelevant to the scene scheme are often flooded in the image. Hence, how to effectively perceive the region of interests (RoIs) from a variety of sizes and build more discriminative representation from such complicated object distribution is vital to understand an aerial scene. In this paper, we propose a novel all grains, one scheme (AGOS) framework to tackle these challenges. To the best of our knowledge, it is the first work to extend the classic multiple instance learning into multi-grain formulation. Specially, it consists of a multi-grain perception module (MGP), a multi-branch multi-instance representation module (MBMIR) and a self-aligned semantic fusion (SSF) module. Firstly, our MGP preserves the differential dilated convolutional features from the backbone, which magnifies the discriminative information from multi-grains. Then, our MBMIR highlights the key instances in the multi-grain representation under the MIL formulation. Finally, our SSF allows our framework to learn the same scene scheme from multi-grain instance representations and fuses them, so that the entire framework is optimized as a whole. Notably, our AGOS is flexible and can be easily adapted to existing CNNs in a plug-and-play manner. Extensive experiments on UCM, AID and NWPU benchmarks demonstrate that our AGOS achieves a comparable performance against the state-of-the-art methods. △ Less

Submitted 6 May, 2022; originally announced May 2022.

Comments: 16 pages, 9 figures

Huge out-of-plane piezoelectric response in ferromagnetic monolayer NiClI

Authors: San-Dong Guo, Yu-Tong Zhu, Ke Qin, Yee-Sin Ang

Abstract: The combination of piezoelectricity and ferromagnetic (FM) order in a two-dimensional (2D) material, namely 2D piezoelectric ferromagnetism (PFM), may open up unprecedented opportunities for novel device applications. Here, we predict an in-plane FM semiconductor Janus monolayer NiClI with considerably large magnetic anisotropy energy (MAE) of 1.439 meV, exhibiting dynamic, mechanical and thermal… ▽ More The combination of piezoelectricity and ferromagnetic (FM) order in a two-dimensional (2D) material, namely 2D piezoelectric ferromagnetism (PFM), may open up unprecedented opportunities for novel device applications. Here, we predict an in-plane FM semiconductor Janus monolayer NiClI with considerably large magnetic anisotropy energy (MAE) of 1.439 meV, exhibiting dynamic, mechanical and thermal stabilities. The NiClI monolayer possesses larger in-plane piezoelectricity ($d_{11}$$=$5.21 pm/V) comparable to that of $\mathrm{MoS_2}$. Furthermore, NiClI has huge out-of-plane piezoelectricity ($d_{31}$$=$1.89 pm/V), which is highly desirable for ultrathin piezoelectric device application. It is proved that huge out-of-plane piezoelectricity is robust against electronic correlation, which confirms reliability of huge $d_{31}$. Finally, being analogous to NiClI, PFM with large out-of-plane piezoelectricity can also be achieved in the Janus monolayers of NiClBr and NiBrI, with the predicted $d_{31}$ of 0.73 pm/V and 1.15 pm/V, respectively. The predicted huge out-of-plane piezoelectric response makes Janus monolayer NiClI a good platform for multifunctional semiconductor spintronic applications, which is also compatible with the bottom/top gate technologies of conventional semiconductor nanoelectronic devices. △ Less

Submitted 13 April, 2022; originally announced April 2022.

Comments: 6 pages, 4 figures

Development and Commissioning of a Compact Cosmic Ray Muon Imaging Prototype

Authors: Xujia Luo, Quanxiao Wang, Kemian Qin, Heng Tian, Zhiqiang Fu, Yanwei Zhao, Zhongtao Shen, Hao Liu, Yuanyong Fu, Guorui Liu, Kaiqiang Yao, ** Qian, Jian Rong, Weixiong Zhang, Xiaogang Luo, Chunxian Liu, Xiangsheng Tian, Minghai Yu, Feng Wu, **g**g Chen, Juntao Liu, Zhiyi Liu

Abstract: Due to the muon tomography's capability of imaging high Z materials, some potential applications have been reported on inspecting smuggled nuclear materials in customs. A compact Cosmic Ray Muons (CRM) imaging prototype, Lanzhou University Muon Imaging System (LUMIS), is comprehensively introduced in this paper including the structure design, assembly, data acquisition and analysis, detector perfo… ▽ More Due to the muon tomography's capability of imaging high Z materials, some potential applications have been reported on inspecting smuggled nuclear materials in customs. A compact Cosmic Ray Muons (CRM) imaging prototype, Lanzhou University Muon Imaging System (LUMIS), is comprehensively introduced in this paper including the structure design, assembly, data acquisition and analysis, detector performance test, and material imaging commissioning etc. Casted triangular prism plastic scintillators (PS) were coupled with Si-PMs for sensitive detector components in system. LUMIS's experimental results show that the detection efficiency of an individual detector layer is about 98%, the position resolution for vertical incident muons is 2.5 mm and the angle resolution is 8.73 mrad given a separation distance of 40.5 cm. Moreover, the image reconstruction software was developed based on the Point of Closest Approach (PoCA) to detect lead bricks as our target. The reconstructed images indicate that the profile of the lead bricks in the image is highly consistent with the target. Subsequently, the capability of LUMIS to distinguish different materials, such as Pb, Cu, Fe, and Al, was investigated as well. The lower limit of response time for rapidly alarming high-Z materials is also given and discussed. The successful development and commissioning of the LUMIS prototype have provided a new solution option in technology and craftsmanship for develo** compact CRM imaging systems that can be used in many applications. △ Less

Submitted 11 March, 2022; originally announced March 2022.

On How Zero-Knowledge Proof Blockchain Mixers Improve, and Worsen User Privacy

Authors: Zhipeng Wang, Stefanos Chaliasos, Kaihua Qin, Liyi Zhou, Lifeng Gao, Pascal Berrang, Ben Livshits, Arthur Gervais

Abstract: Zero-knowledge proof (ZKP) mixers are one of the most widely-used blockchain privacy solutions, operating on top of smart contract-enabled blockchains. We find that ZKP mixers are tightly intertwined with the growing number of Decentralized Finance (DeFi) attacks and Blockchain Extractable Value (BEV) extractions. Through coin flow tracing, we discover that 205 blockchain attackers and 2,595 BEV e… ▽ More Zero-knowledge proof (ZKP) mixers are one of the most widely-used blockchain privacy solutions, operating on top of smart contract-enabled blockchains. We find that ZKP mixers are tightly intertwined with the growing number of Decentralized Finance (DeFi) attacks and Blockchain Extractable Value (BEV) extractions. Through coin flow tracing, we discover that 205 blockchain attackers and 2,595 BEV extractors leverage mixers as their source of funds, while depositing a total attack revenue of 412.87M USD. Moreover, the US OFAC sanctions against the largest ZKP mixer, Tornado.Cash, have reduced the mixer's daily deposits by more than 80%. Further, ZKP mixers advertise their level of privacy through a so-called anonymity set size, which similarly to k-anonymity allows a user to hide among a set of k other users. Through empirical measurements, we, however, find that these anonymity set claims are mostly inaccurate. For the most popular mixers on Ethereum (ETH) and Binance Smart Chain (BSC), we show how to reduce the anonymity set size on average by 27.34% and 46.02% respectively. Our empirical evidence is also the first to suggest a differing privacy-predilection of users on ETH and BSC. State-of-the-art ZKP mixers are moreover interwoven with the DeFi ecosystem by offering anonymity mining (AM) incentives, i.e., users receive monetary rewards for mixing coins. However, contrary to the claims of related work, we find that AM does not necessarily improve the quality of a mixer's anonymity set. Our findings indicate that AM attracts privacy-ignorant users, who then do not contribute to improving the privacy of other mixer users. △ Less

Submitted 6 March, 2023; v1 submitted 22 January, 2022; originally announced January 2022.

Microbial transport and dispersion in heterogeneous flows created by pillar arrays

Authors: Kejie Chen, Kairong Qin

Abstract: Swimming microbes, such as bacteria and algae, live in diverse habitats including soil, ocean and human body which are characterized by structural boundaries and heterogeneous fluid flows. Although much progress has been made in understanding the Brownian ratchet motions of microbes and their hydrodynamic interactions with the wall over the last decades, the complex interplay between the structura… ▽ More Swimming microbes, such as bacteria and algae, live in diverse habitats including soil, ocean and human body which are characterized by structural boundaries and heterogeneous fluid flows. Although much progress has been made in understanding the Brownian ratchet motions of microbes and their hydrodynamic interactions with the wall over the last decades, the complex interplay between the structural and fluidic environment and the self-propelling microbial motions still remains elusive. Here, we developed a Langevin model to simulate and investigate the transport and dispersion of microbes in periodic pillar arrays. By tracing the spatial-temporal evolution of microbial trajectories, we show that the no-slip pillar surface induces local fluid shear which redirects microbial movements. In the vicinity of pillars, loo** trajectories and slowly moving speed lead to the transient accumulation and the sluggish transport of microbes. Comprehensive microscopic motions including the swinging, zigzag and adhesive motions are observed. In the pillar array of asymmetric pillar arrangements, the adjacent downstream pillars provide geometric guidance such that the microbial population has a deterministic shift perpendicular to the flow direction. Moreover, effects of the topology of the pillar array, fluid flowing properties and microbial properties on the microbial advection and dispersion in pillar arrays are quantitatively analyzed. These results highlight the importance of structures and flows on the microbial transport and distribution which should be carefully considered in the study of microbial processes. △ Less

Submitted 9 December, 2021; originally announced December 2021.

Exploring the interpretability of deep neural networks used for gravitational lens finding with a sensitivity probe

Authors: C. Jacobs, K. Glazebrook, A. K. Qin, T. Collett

Abstract: Artificial neural networks are finding increasing use in astronomy, but understanding the limitations of these models can be difficult. We utilize a statistical method, a sensitivity probe, designed to complement established methods for interpreting neural network behavior by quantifying the sensitivity of a model's performance to various properties of the inputs. We apply this method to neural ne… ▽ More Artificial neural networks are finding increasing use in astronomy, but understanding the limitations of these models can be difficult. We utilize a statistical method, a sensitivity probe, designed to complement established methods for interpreting neural network behavior by quantifying the sensitivity of a model's performance to various properties of the inputs. We apply this method to neural networks trained to classify images of galaxy-galaxy strong lenses in the Dark Energy Survey. We find that the networks are highly sensitive to color, the simulated PSF used in training, and occlusion of light from a lensed source, but are insensitive to Einstein radius, and performance degrades smoothly with source and lens magnitudes. From this we identify weaknesses in the training sets used to constrain the networks, particularly the over-sensitivity to PSF, and constrain the selection function of the lens-finder as a function of galaxy photometric magnitudes, with accuracy decreasing significantly where the g-band magnitude of the lens source is greater than 21.5 and the r-band magnitude of the lens is less than 19. △ Less

Submitted 5 December, 2021; originally announced December 2021.

Comments: 19 pages, 12 figures. Accepted for publication in Astronomy and Computing

Deep-learning Assisted Extraction of Fluid Velocity from Scalar Signal Transport in a Shallow Microfluidic Channel

Authors: Xiao Zeng, Chundong Xue, Kejie Chen, Yongjiang Li, Kai-Rong Qin

Abstract: Precise measurement of flow velocity in microfluidic channels is of importance in microfluidic applications, such as quantitative chemical analysis, sample preparation and drug synthesis. However, simple approaches for quickly and precisely measuring the flow velocity in microchannels are still lacking. Herein, we propose a deep neural networks assisted scalar image velocimetry (DNN-SIV) for quick… ▽ More Precise measurement of flow velocity in microfluidic channels is of importance in microfluidic applications, such as quantitative chemical analysis, sample preparation and drug synthesis. However, simple approaches for quickly and precisely measuring the flow velocity in microchannels are still lacking. Herein, we propose a deep neural networks assisted scalar image velocimetry (DNN-SIV) for quick and precise extraction of fluid velocity in a shallow microfluidic channel with a high aspect ratio, which is a basic geometry for cell culture, from a dye concentration field with spatiotemporal gradients. DNN-SIV is built on physics-informed neural networks and residual neural networks that integrate data of scalar field and physics laws to determine the velocity in the height direction. The underlying enforcing physics laws are derived from the Navier-Stokes equation and the scalar transport equation. Apart from this, dynamic concentration boundary condition is adopted to improve the velocity measurement of laminar flow with small Reynolds Number in microchannels. The proposed DNN-SIV is validated and analyzed by numerical simulations. Compared to integral minimization algorithm used in conventional SIV, DNN-SIV is robust to noise in the measured scalar field and more efficiently allowing real-time flow visualization. Furthermore, the fundamental significance of rational construction of concentration field in microchannels is also underscored. The proposed DNN-SIV in this paper is agnostic to initial and boundary conditions that can be a promising velocity measurement approach for many potential applications in microfluidic chips. △ Less

Submitted 1 December, 2021; originally announced December 2021.

ADDS: Adaptive Differentiable Sampling for Robust Multi-Party Learning

Authors: Maoguo Gong, Yuan Gao, Yue Wu, A. K. Qin

Abstract: Distributed multi-party learning provides an effective approach for training a joint model with scattered data under legal and practical constraints. However, due to the quagmire of a skewed distribution of data labels across participants and the computation bottleneck of local devices, how to build smaller customized models for clients in various scenarios while providing updates appliable to the… ▽ More Distributed multi-party learning provides an effective approach for training a joint model with scattered data under legal and practical constraints. However, due to the quagmire of a skewed distribution of data labels across participants and the computation bottleneck of local devices, how to build smaller customized models for clients in various scenarios while providing updates appliable to the central model remains a challenge. In this paper, we propose a novel adaptive differentiable sampling framework (ADDS) for robust and communication-efficient multi-party learning. Inspired by the idea of dropout in neural networks, we introduce a network sampling strategy in the multi-party setting, which distributes different subnets of the central model to clients for updating, and the differentiable sampling rates allow each client to extract optimal local architecture from the supernet according to its private data distribution. The approach requires minimal modifications to the existing multi-party learning structure, and it is capable of integrating local updates of all subnets into the supernet, improving the robustness of the central model. The proposed framework significantly reduces local computation and communication costs while speeding up the central model convergence, as we demonstrated through experiments on real-world datasets. △ Less

Submitted 28 October, 2021; originally announced October 2021.

Learning Enhanced Optimisation for Routing Problems

Authors: Nasrin Sultana, Jeffrey Chan, Tabinda Sarwar, Babak Abbasi, A. K. Qin

Abstract: Deep learning approaches have shown promising results in solving routing problems. However, there is still a substantial gap in solution quality between machine learning and operations research algorithms. Recently, another line of research has been introduced that fuses the strengths of machine learning and operational research algorithms. In particular, search perturbation operators have been us… ▽ More Deep learning approaches have shown promising results in solving routing problems. However, there is still a substantial gap in solution quality between machine learning and operations research algorithms. Recently, another line of research has been introduced that fuses the strengths of machine learning and operational research algorithms. In particular, search perturbation operators have been used to improve the solution. Nevertheless, using the perturbation may not guarantee a quality solution. This paper presents "Learning to Guide Local Search" (L2GLS), a learning-based approach for routing problems that uses a penalty term and reinforcement learning to adaptively adjust search efforts. L2GLS combines local search (LS) operators' strengths with penalty terms to escape local optimals. Routing problems have many practical applications, often presetting larger instances that are still challenging for many existing algorithms introduced in the learning to optimise field. We show that L2GLS achieves the new state-of-the-art results on larger TSP and CVRP over other machine learning methods. △ Less

Submitted 17 September, 2021; originally announced September 2021.

Evolutionary Ensemble Learning for Multivariate Time Series Prediction

Authors: Hui Song, A. K. Qin, Flora D. Salim

Abstract: Multivariate time series (MTS) prediction plays a key role in many fields such as finance, energy and transport, where each individual time series corresponds to the data collected from a certain data source, so-called channel. A typical pipeline of building an MTS prediction model (PM) consists of selecting a subset of channels among all available ones, extracting features from the selected chann… ▽ More Multivariate time series (MTS) prediction plays a key role in many fields such as finance, energy and transport, where each individual time series corresponds to the data collected from a certain data source, so-called channel. A typical pipeline of building an MTS prediction model (PM) consists of selecting a subset of channels among all available ones, extracting features from the selected channels, and building a PM based on the extracted features, where each component involves certain optimization tasks, i.e., selection of channels, feature extraction (FE) methods, and PMs as well as configuration of the selected FE method and PM. Accordingly, pursuing the best prediction performance corresponds to optimizing the pipeline by solving all of its involved optimization problems. This is a non-trivial task due to the vastness of the solution space. Different from most of the existing works which target at optimizing certain components of the pipeline, we propose a novel evolutionary ensemble learning framework to optimize the entire pipeline in a holistic manner. In this framework, a specific pipeline is encoded as a candidate solution and a multi-objective evolutionary algorithm is applied under different population sizes to produce multiple Pareto optimal sets (POSs). Finally, selective ensemble learning is designed to choose the optimal subset of solutions from the POSs and combine them to yield final prediction by using greedy sequential selection and least square methods. We implement the proposed framework and evaluate our implementation on two real-world applications, i.e., electricity consumption prediction and air quality prediction. The performance comparison with state-of-the-art techniques demonstrates the superiority of the proposed approach. △ Less

Submitted 22 August, 2021; originally announced August 2021.

AdvDrop: Adversarial Attack to DNNs by Drop** Information

Authors: Ranjie Duan, Yuefeng Chen, Dantong Niu, Yun Yang, A. K. Qin, Yuan He

Abstract: Human can easily recognize visual objects with lost information: even losing most details with only contour reserved, e.g. cartoon. However, in terms of visual perception of Deep Neural Networks (DNNs), the ability for recognizing abstract objects (visual objects with lost information) is still a challenge. In this work, we investigate this issue from an adversarial viewpoint: will the performance… ▽ More Human can easily recognize visual objects with lost information: even losing most details with only contour reserved, e.g. cartoon. However, in terms of visual perception of Deep Neural Networks (DNNs), the ability for recognizing abstract objects (visual objects with lost information) is still a challenge. In this work, we investigate this issue from an adversarial viewpoint: will the performance of DNNs decrease even for the images only losing a little information? Towards this end, we propose a novel adversarial attack, named \textit{AdvDrop}, which crafts adversarial examples by drop** existing information of images. Previously, most adversarial attacks add extra disturbing information on clean images explicitly. Opposite to previous works, our proposed work explores the adversarial robustness of DNN models in a novel perspective by drop** imperceptible details to craft adversarial examples. We demonstrate the effectiveness of \textit{AdvDrop} by extensive experiments, and show that this new type of adversarial examples is more difficult to be defended by current defense systems. △ Less

Submitted 20 August, 2021; originally announced August 2021.

Comments: Accepted to ICCV 2021