Showing 1–50 of 96 results for author: Chang, L

Human-like object concept representations emerge naturally in multimodal large language models

Authors: Changde Du, Kaicheng Fu, Bincheng Wen, Yi Sun, Jie Peng, Wei Wei, Ying Gao, Shengpei Wang, Chuncheng Zhang, **peng Li, Shuang Qiu, Le Chang, Huiguang He

Abstract: The conceptualization and categorization of natural objects in the human mind have long intrigued cognitive scientists and neuroscientists, offering crucial insights into human perception and cognition. Recently, the rapid development of Large Language Models (LLMs) has raised the attractive question of whether these models can also develop human-like object representations through exposure to vas… ▽ More The conceptualization and categorization of natural objects in the human mind have long intrigued cognitive scientists and neuroscientists, offering crucial insights into human perception and cognition. Recently, the rapid development of Large Language Models (LLMs) has raised the attractive question of whether these models can also develop human-like object representations through exposure to vast amounts of linguistic and multimodal data. In this study, we combined behavioral and neuroimaging analysis methods to uncover how the object concept representations in LLMs correlate with those of humans. By collecting large-scale datasets of 4.7 million triplet judgments from LLM and Multimodal LLM (MLLM), we were able to derive low-dimensional embeddings that capture the underlying similarity structure of 1,854 natural objects. The resulting 66-dimensional embeddings were found to be highly stable and predictive, and exhibited semantic clustering akin to human mental representations. Interestingly, the interpretability of the dimensions underlying these embeddings suggests that LLM and MLLM have developed human-like conceptual representations of natural objects. Further analysis demonstrated strong alignment between the identified model embeddings and neural activity patterns in many functionally defined brain ROIs (e.g., EBA, PPA, RSC and FFA). This provides compelling evidence that the object representations in LLMs, while not identical to those in the human, share fundamental commonalities that reflect key schemas of human conceptual knowledge. This study advances our understanding of machine intelligence and informs the development of more human-like artificial cognitive systems. △ Less

Submitted 1 July, 2024; originally announced July 2024.

ConStyle v2: A Strong Prompter for All-in-One Image Restoration

Authors: Dongqi Fan, Junhao Zhang, Liang Chang

Abstract: This paper introduces ConStyle v2, a strong plug-and-play prompter designed to output clean visual prompts and assist U-Net Image Restoration models in handling multiple degradations. The joint training process of IRConStyle, an Image Restoration framework consisting of ConStyle and a general restoration network, is divided into two stages: first, pre-training ConStyle alone, and then freezing its… ▽ More This paper introduces ConStyle v2, a strong plug-and-play prompter designed to output clean visual prompts and assist U-Net Image Restoration models in handling multiple degradations. The joint training process of IRConStyle, an Image Restoration framework consisting of ConStyle and a general restoration network, is divided into two stages: first, pre-training ConStyle alone, and then freezing its weights to guide the training of the general restoration network. Three improvements are proposed in the pre-training stage to train ConStyle: unsupervised pre-training, adding a pretext task (i.e. classification), and adopting knowledge distillation. Without bells and whistles, we can get ConStyle v2, a strong prompter for all-in-one Image Restoration, in less than two GPU days and doesn't require any fine-tuning. Extensive experiments on Restormer (transformer-based), NAFNet (CNN-based), MAXIM-1S (MLP-based), and a vanilla CNN network demonstrate that ConStyle v2 can enhance any U-Net style Image Restoration models to all-in-one Image Restoration models. Furthermore, models guided by the well-trained ConStyle v2 exhibit superior performance in some specific degradation compared to ConStyle. △ Less

Submitted 26 June, 2024; originally announced June 2024.

FLUX: Fast Software-based Communication Overlap On GPUs Through Kernel Fusion

Authors: Li-Wen Chang, Wenlei Bao, Qi Hou, Chengquan Jiang, Ningxin Zheng, Yinmin Zhong, Xuanrun Zhang, Zuquan Song, Ziheng Jiang, Haibin Lin, Xin **, Xin Liu

Abstract: Large deep learning models have demonstrated strong ability to solve many tasks across a wide range of applications. Those large models typically require training and inference to be distributed. Tensor parallelism is a common technique partitioning computation of an operation or layer across devices to overcome the memory capacity limitation of a single processor, and/or to accelerate computation… ▽ More Large deep learning models have demonstrated strong ability to solve many tasks across a wide range of applications. Those large models typically require training and inference to be distributed. Tensor parallelism is a common technique partitioning computation of an operation or layer across devices to overcome the memory capacity limitation of a single processor, and/or to accelerate computation to meet a certain latency requirement. However, this kind of parallelism introduces additional communication that might contribute a significant portion of overall runtime. Thus limits scalability of this technique within a group of devices with high speed interconnects, such as GPUs with NVLinks in a node. This paper proposes a novel method, Flux, to significantly hide communication latencies with dependent computations for GPUs. Flux over-decomposes communication and computation operations into much finer-grained operations and further fuses them into a larger kernel to effectively hide communication without compromising kernel efficiency. Flux can potentially overlap up to 96% of communication given a fused kernel. Overall, it can achieve up to 1.24x speedups for training over Megatron-LM on a cluster of 128 GPUs with various GPU generations and interconnects, and up to 1.66x and 1.30x speedups for prefill and decoding inference over vLLM on a cluster with 8 GPUs with various GPU generations and interconnects. △ Less

Submitted 18 June, 2024; v1 submitted 10 June, 2024; originally announced June 2024.

Privacy-Preserving Debiasing using Data Augmentation and Machine Unlearning

Authors: Zhixin Pan, Emma Andrews, Laura Chang, Prabhat Mishra

Abstract: Data augmentation is widely used to mitigate data bias in the training dataset. However, data augmentation exposes machine learning models to privacy attacks, such as membership inference attacks. In this paper, we propose an effective combination of data augmentation and machine unlearning, which can reduce data bias while providing a provable defense against known attacks. Specifically, we maint… ▽ More Data augmentation is widely used to mitigate data bias in the training dataset. However, data augmentation exposes machine learning models to privacy attacks, such as membership inference attacks. In this paper, we propose an effective combination of data augmentation and machine unlearning, which can reduce data bias while providing a provable defense against known attacks. Specifically, we maintain the fairness of the trained model with diffusion-based data augmentation, and then utilize multi-shard unlearning to remove identifying information of original data from the ML model for protection against privacy attacks. Experimental evaluation across diverse datasets demonstrates that our approach can achieve significant improvements in bias reduction as well as robustness against state-of-the-art privacy attacks. △ Less

Submitted 19 April, 2024; originally announced April 2024.

Taiyi: A high-performance CKKS accelerator for Practical Fully Homomorphic Encryption

Authors: Shengyu Fan, Xianglong Deng, Zhuoyu Tian, Zhicheng Hu, Liang Chang, Rui Hou, Dan Meng, Mingzhe Zhang

Abstract: Fully Homomorphic Encryption (FHE), a novel cryptographic theory enabling computation directly on ciphertext data, offers significant security benefits but is hampered by substantial performance overhead. In recent years, a series of accelerator designs have significantly enhanced the performance of FHE applications, bringing them closer to real-world applicability. However, these accelerators fac… ▽ More Fully Homomorphic Encryption (FHE), a novel cryptographic theory enabling computation directly on ciphertext data, offers significant security benefits but is hampered by substantial performance overhead. In recent years, a series of accelerator designs have significantly enhanced the performance of FHE applications, bringing them closer to real-world applicability. However, these accelerators face challenges related to large on-chip memory and area. Additionally, FHE algorithms undergo rapid development, rendering the previous accelerator designs less perfectly adapted to the evolving landscape of optimized FHE applications. In this paper, we conducted a detailed analysis of existing applications with the new FHE method, making two key observations: 1) the bottleneck of FHE applications shifts from NTT to the inner-product operation, and 2) the optimal α of KeySwitch changes with the decrease in multiplicative level. Based on these observations, we designed an accelerator named Taiyi, which includes specific hardware for the inner-product operation and optimizes the NTT and BConv operations through algorithmic derivation. A comparative evaluation of Taiyi against previous state-of-the-art designs reveals an average performance improvement of 1.5x and reduces the area overhead by 15.7%. △ Less

Submitted 15 March, 2024; originally announced March 2024.

Comments: 14 pages, 15 figures

Maximum Defective Clique Computation: Improved Time Complexities and Practical Performance

Authors: Lijun Chang

Abstract: The concept of $k$-defective clique, a relaxation of clique by allowing up-to $k$ missing edges, has been receiving increasing interests recently. Although the problem of finding the maximum $k$-defective clique is NP-hard, several practical algorithms have been recently proposed in the literature, with kDC being the state of the art. kDC not only runs the fastest in practice, but also achieves th… ▽ More The concept of $k$-defective clique, a relaxation of clique by allowing up-to $k$ missing edges, has been receiving increasing interests recently. Although the problem of finding the maximum $k$-defective clique is NP-hard, several practical algorithms have been recently proposed in the literature, with kDC being the state of the art. kDC not only runs the fastest in practice, but also achieves the best time complexity. Specifically, it runs in $O^*(γ_k^n)$ time when ignoring polynomial factors; here, $γ_k$ is a constant that is smaller than two and only depends on $k$, and $n$ is the number of vertices in the input graph $G$. In this paper, we propose the kDC-Two algorithm to improve the time complexity as well as practical performance. kDC-Two runs in $O^*( (αΔ)^{k+2} γ_{k-1}^α)$ time when the maximum $k$-defective clique size $ω_k(G)$ is at least $k+2$, and in $O^*(γ_{k-1}^n)$ time otherwise, where $α$ and $Δ$ are the degeneracy and maximum degree of $G$, respectively. In addition, with slight modification, kDC-Two also runs in $O^*( (αΔ)^{k+2} (k+1)^{α+k+1-ω_k(G)})$ time by using the degeneracy gap $α+k+1-ω_k(G)$ parameterization; this is better than $O^*( (αΔ)^{k+2}γ_{k-1}^α)$ when $ω_k(G)$ is close to the degeneracy-based upper bound $α+k+1$. Finally, to further improve the practical performance, we propose a new degree-sequence-based reduction rule that can be efficiently applied, and theoretically demonstrate its effectiveness compared with those proposed in the literature. Extensive empirical studies on three benchmark graph collections show that our algorithm outperforms the existing fastest algorithm by several orders of magnitude. △ Less

Submitted 12 March, 2024; originally announced March 2024.

IRConStyle: Image Restoration Framework Using Contrastive Learning and Style Transfer

Authors: Dongqi Fan, Xin Zhao, Liang Chang

Abstract: Recently, the contrastive learning paradigm has achieved remarkable success in high-level tasks such as classification, detection, and segmentation. However, contrastive learning applied in low-level tasks, like image restoration, is limited, and its effectiveness is uncertain. This raises a question: Why does the contrastive learning paradigm not yield satisfactory results in image restoration? I… ▽ More Recently, the contrastive learning paradigm has achieved remarkable success in high-level tasks such as classification, detection, and segmentation. However, contrastive learning applied in low-level tasks, like image restoration, is limited, and its effectiveness is uncertain. This raises a question: Why does the contrastive learning paradigm not yield satisfactory results in image restoration? In this paper, we conduct in-depth analyses and propose three guidelines to address the above question. In addition, inspired by style transfer and based on contrastive learning, we propose a novel module for image restoration called \textbf{ConStyle}, which can be efficiently integrated into any U-Net structure network. By leveraging the flexibility of ConStyle, we develop a \textbf{general restoration network} for image restoration. ConStyle and the general restoration network together form an image restoration framework, namely \textbf{IRConStyle}. To demonstrate the capability and compatibility of ConStyle, we replace the general restoration network with transformer-based, CNN-based, and MLP-based networks, respectively. We perform extensive experiments on various image restoration tasks, including denoising, deblurring, deraining, and dehazing. The results on 19 benchmarks demonstrate that ConStyle can be integrated with any U-Net-based network and significantly enhance performance. For instance, ConStyle NAFNet significantly outperforms the original NAFNet on SOTS outdoor (dehazing) and Rain100H (deraining) datasets, with PSNR improvements of 4.16 dB and 3.58 dB with 85% fewer parameters. △ Less

Submitted 7 March, 2024; v1 submitted 24 February, 2024; originally announced February 2024.

The Quantum Abstract Machine

Authors: Liyi Li, Le Chang, Rance Cleaveland, Mingwei Zhu, Xiaodi Wu

Abstract: This paper develops a model of quantum behavior that is intended to support the abstract yet accurate design and functional verification of quantum communication protocols. The work is motivated by the need for conceptual tools for the development of quantum-communication systems that are usable by non-specialists in quantum physics while also correctly capturing at a useful abstraction the underl… ▽ More This paper develops a model of quantum behavior that is intended to support the abstract yet accurate design and functional verification of quantum communication protocols. The work is motivated by the need for conceptual tools for the development of quantum-communication systems that are usable by non-specialists in quantum physics while also correctly capturing at a useful abstraction the underlying quantum phenomena. Our approach involves defining a quantum abstract machine (QAM) whose operations correspond to well-known quantum circuits; these operations, however, are given direct abstract semantics in a style similar to that of Berry's and Boudol's Chemical Abstract Machine. This paper defines the QAM's semantics and shows via examples how it may be used to model and reason about existing quantum communication protocols. △ Less

Submitted 20 February, 2024; originally announced February 2024.

Charting the COVID Long Haul Experience -- A Longitudinal Exploration of Symptoms, Activity, and Clinical Adherence

Authors: Jessica Pater, Shaan Chopra, Juliette Zaccour, Jeanne Carroll, Fayika Farhat Nova, Tammy Toscos, Shion Guha, Fen Lei Chang

Abstract: COVID Long Haul (CLH) is an emerging chronic illness with varied patient experiences. Our understanding of CLH is often limited to data from electronic health records (EHRs), such as diagnoses or problem lists, which do not capture the volatility and severity of symptoms or their impact. To better understand the unique presentation of CLH, we conducted a 3-month long cohort study with 14 CLH patie… ▽ More COVID Long Haul (CLH) is an emerging chronic illness with varied patient experiences. Our understanding of CLH is often limited to data from electronic health records (EHRs), such as diagnoses or problem lists, which do not capture the volatility and severity of symptoms or their impact. To better understand the unique presentation of CLH, we conducted a 3-month long cohort study with 14 CLH patients, collecting objective (EHR, daily Fitbit logs) and subjective (weekly surveys, interviews) data. Our findings reveal a complex presentation of symptoms, associated uncertainty, and the ensuing impact CLH has on patients' personal and professional lives. We identify patient needs, practices, and challenges around adhering to clinical recommendations, engaging with health data, and establishing "new normals" post COVID. We reflect on the potential found at the intersection of these various data streams and the persuasive heuristics possible when designing for this new population and their specific needs. △ Less

Submitted 7 February, 2024; originally announced February 2024.

Comments: 21 pages, 4 figures, 7 tables, ACM Conference CHI Conference on Human Factors in Computing Systems

ACM Class: K.4

LIR: A Lightweight Baseline for Image Restoration

Authors: Dongqi Fan, Ting Yue, Xin Zhao, Ren**g Xu, Liang Chang

Abstract: Recently, there have been significant advancements in Image Restoration based on CNN and transformer. However, the inherent characteristics of the Image Restoration task are often overlooked in many works. They, instead, tend to focus on the basic block design and stack numerous such blocks to the model, leading to parameters redundant and computations unnecessary. Thus, the efficiency of the imag… ▽ More Recently, there have been significant advancements in Image Restoration based on CNN and transformer. However, the inherent characteristics of the Image Restoration task are often overlooked in many works. They, instead, tend to focus on the basic block design and stack numerous such blocks to the model, leading to parameters redundant and computations unnecessary. Thus, the efficiency of the image restoration is hindered. In this paper, we propose a Lightweight Baseline network for Image Restoration called LIR to efficiently restore the image and remove degradations. First of all, through an ingenious structural design, LIR removes the degradations existing in the local and global residual connections that are ignored by modern networks. Then, a Lightweight Adaptive Attention (LAA) Block is introduced which is mainly composed of proposed Adaptive Filters and Attention Blocks. The proposed Adaptive Filter is used to adaptively extract high-frequency information and enhance object contours in various IR tasks, and Attention Block involves a novel Patch Attention module to approximate the self-attention part of the transformer. On the deraining task, our LIR achieves the state-of-the-art Structure Similarity Index Measure (SSIM) and comparable performance to state-of-the-art models on Peak Signal-to-Noise Ratio (PSNR). For denoising, dehazing, and deblurring tasks, LIR also achieves a comparable performance to state-of-the-art models with a parameter size of about 30\%. In addition, it is worth noting that our LIR produces better visual results that are more in line with the human aesthetic. △ Less

Submitted 24 June, 2024; v1 submitted 2 February, 2024; originally announced February 2024.

Distributed client selection with multi-objective in federated learning assisted Internet of Vehicles

Authors: Narisu Cha, Long Chang

Abstract: Federated learning is an emerging distributed machine learning framework in the Internet of Vehicles (IoV). In IoV, millions of vehicles are willing to train the model to share their knowledge. Maintaining an active state means the participants must update their state to the FL server in a fixed interval and participate to next round. However, the cost by maintaining an active state is very large… ▽ More Federated learning is an emerging distributed machine learning framework in the Internet of Vehicles (IoV). In IoV, millions of vehicles are willing to train the model to share their knowledge. Maintaining an active state means the participants must update their state to the FL server in a fixed interval and participate to next round. However, the cost by maintaining an active state is very large when there are a huge number of participating vehicles. In this paper, we proposed a distributed client selection scheme to reduce the cost of maintaining the active state for all participants. The clients with the highest evaluation are elected among the neighbours. In the evaluator, four variables are considered including sample quantity, throughput available, computational capability and the quality of the local dataset. We adopted fuzzy logic as the evaluator since the closed-form solution over four variables does not exist. Extensive simulation results show our proposal approximates the centralized client selection in terms of accuracy and can significantly reduce the communication overhead. △ Less

Submitted 6 January, 2024; originally announced January 2024.

Dual-Teacher De-biasing Distillation Framework for Multi-domain Fake News Detection

Authors: Jiayang Li, Xuan Feng, Tianlong Gu, Liang Chang

Abstract: Multi-domain fake news detection aims to identify whether various news from different domains is real or fake and has become urgent and important. However, existing methods are dedicated to improving the overall performance of fake news detection, ignoring the fact that unbalanced data leads to disparate treatment for different domains, i.e., the domain bias problem. To solve this problem, we prop… ▽ More Multi-domain fake news detection aims to identify whether various news from different domains is real or fake and has become urgent and important. However, existing methods are dedicated to improving the overall performance of fake news detection, ignoring the fact that unbalanced data leads to disparate treatment for different domains, i.e., the domain bias problem. To solve this problem, we propose the Dual-Teacher De-biasing Distillation framework (DTDBD) to mitigate bias across different domains. Following the knowledge distillation methods, DTDBD adopts a teacher-student structure, where pre-trained large teachers instruct a student model. In particular, the DTDBD consists of an unbiased teacher and a clean teacher that jointly guide the student model in mitigating domain bias and maintaining performance. For the unbiased teacher, we introduce an adversarial de-biasing distillation loss to instruct the student model in learning unbiased domain knowledge. For the clean teacher, we design domain knowledge distillation loss, which effectively incentivizes the student model to focus on representing domain features while maintaining performance. Moreover, we present a momentum-based dynamic adjustment algorithm to trade off the effects of two teachers. Extensive experiments on Chinese and English datasets show that the proposed method substantially outperforms the state-of-the-art baseline methods in terms of bias metrics while guaranteeing competitive performance. △ Less

Submitted 1 December, 2023; originally announced December 2023.

Comments: ICDE 2024

Challenges and Contributing Factors in the Utilization of Large Language Models (LLMs)

Authors: Xiaoliang Chen, Liangbin Li, Le Chang, Yunhe Huang, Yuxuan Zhao, Yuxiao Zhang, Dinuo Li

Abstract: With the development of large language models (LLMs) like the GPT series, their widespread use across various application scenarios presents a myriad of challenges. This review initially explores the issue of domain specificity, where LLMs may struggle to provide precise answers to specialized questions within niche fields. The problem of knowledge forgetting arises as these LLMs might find it har… ▽ More With the development of large language models (LLMs) like the GPT series, their widespread use across various application scenarios presents a myriad of challenges. This review initially explores the issue of domain specificity, where LLMs may struggle to provide precise answers to specialized questions within niche fields. The problem of knowledge forgetting arises as these LLMs might find it hard to balance old and new information. The knowledge repetition phenomenon reveals that sometimes LLMs might deliver overly mechanized responses, lacking depth and originality. Furthermore, knowledge illusion describes situations where LLMs might provide answers that seem insightful but are actually superficial, while knowledge toxicity focuses on harmful or biased information outputs. These challenges underscore problems in the training data and algorithmic design of LLMs. To address these issues, it's suggested to diversify training data, fine-tune models, enhance transparency and interpretability, and incorporate ethics and fairness training. Future technological trends might lean towards iterative methodologies, multimodal learning, model personalization and customization, and real-time learning and feedback mechanisms. In conclusion, future LLMs should prioritize fairness, transparency, and ethics, ensuring they uphold high moral and ethical standards when serving humanity. △ Less

Submitted 20 October, 2023; originally announced October 2023.

Efficient Maximum $k$-Defective Clique Computation with Improved Time Complexity

Authors: Lijun Chang

Abstract: $k$-defective cliques relax cliques by allowing up-to $k$ missing edges from being a complete graph. This relaxation enables us to find larger near-cliques and has applications in link prediction, cluster detection, social network analysis and transportation science. The problem of finding the largest $k… ▽ More $k$-defective cliques relax cliques by allowing up-to $k$ missing edges from being a complete graph. This relaxation enables us to find larger near-cliques and has applications in link prediction, cluster detection, social network analysis and transportation science. The problem of finding the largest $k$-defective clique has been recently studied with several algorithms being proposed in the literature. However, the currently fastest algorithm KDBB does not improve its time complexity from being the trivial $O(2^n)$, and also, KDBB's practical performance is still not satisfactory. In this paper, we advance the state of the art for exact maximum $k$-defective clique computation, in terms of both time complexity and practical performance. Moreover, we separate the techniques required for achieving the time complexity from others purely used for practical performance consideration; this design choice may facilitate the research community to further improve the practical efficiency while not sacrificing the worst case time complexity. In specific, we first develop a general framework kDC that beats the trivial time complexity of $O(2^n)$ and achieves a better time complexity than all existing algorithms. The time complexity of kDC is solely achieved by non-fully-adjacent-first branching rule, excess-removal reduction rule and high-degree reduction rule. Then, to make kDC practically efficient, we further propose a new upper bound, two reduction rules, and an algorithm for efficiently computing a large initial solution. Extensive empirical studies on three benchmark graph collections with $290$ graphs in total demonstrate that kDC outperforms the currently fastest algorithm KDBB by several orders of magnitude. △ Less

Submitted 5 September, 2023; originally announced September 2023.

Comments: Accepted by SIGMOD 2024 in May 2023

Improving Out-of-Distribution Detection in Echocardiographic View Classication through Enhancing Semantic Features

Authors: Jaeik Jeon, Seongmin Ha, Yeonggul Jang, Yeonyee E. Yoon, Jiyeon Kim, Hyunseok Jeong, Dawun Jeong, Youngtaek Hong, Seung-Ah Lee Hyuk-Jae Chang

Abstract: In echocardiographic view classification, accurately detecting out-of-distribution (OOD) data is essential but challenging, especially given the subtle differences between in-distribution and OOD data. While conventional OOD detection methods, such as Mahalanobis distance (MD) are effective in far-OOD scenarios with clear distinctions between distributions, they struggle to discern the less obviou… ▽ More In echocardiographic view classification, accurately detecting out-of-distribution (OOD) data is essential but challenging, especially given the subtle differences between in-distribution and OOD data. While conventional OOD detection methods, such as Mahalanobis distance (MD) are effective in far-OOD scenarios with clear distinctions between distributions, they struggle to discern the less obvious variations characteristic of echocardiographic data. In this study, we introduce a novel use of label smoothing to enhance semantic feature representation in echocardiographic images, demonstrating that these enriched semantic features are key for significantly improving near-OOD instance detection. By combining label smoothing with MD-based OOD detection, we establish a new benchmark for accuracy in echocardiographic OOD detection. △ Less

Submitted 23 November, 2023; v1 submitted 31 August, 2023; originally announced August 2023.

Interpretable (not just posthoc-explainable) heterogeneous survivor bias-corrected treatment effects for assignment of postdischarge interventions to prevent readmissions

Authors: Hong**g Xia, Joshua C. Chang, Sarah Nowak, Sonya Mahajan, Rohit Mahajan, Ted L. Chang, Carson C. Chow

Abstract: We used survival analysis to quantify the impact of postdischarge evaluation and management (E/M) services in preventing hospital readmission or death. Our approach avoids a specific pitfall of applying machine learning to this problem, which is an inflated estimate of the effect of interventions, due to survivors bias -- where the magnitude of inflation may be conditional on heterogeneous confoun… ▽ More We used survival analysis to quantify the impact of postdischarge evaluation and management (E/M) services in preventing hospital readmission or death. Our approach avoids a specific pitfall of applying machine learning to this problem, which is an inflated estimate of the effect of interventions, due to survivors bias -- where the magnitude of inflation may be conditional on heterogeneous confounders in the population. This bias arises simply because in order to receive an intervention after discharge, a person must not have been readmitted in the intervening period. After deriving an expression for this phantom effect, we controlled for this and other biases within an inherently interpretable Bayesian survival framework. We identified case management services as being the most impactful for reducing readmissions overall. △ Less

Submitted 3 August, 2023; v1 submitted 19 April, 2023; originally announced April 2023.

Comments: Submitted

Journal ref: PMLR 219:884-905, 2023

SikuGPT: A Generative Pre-trained Model for Intelligent Information Processing of Ancient Texts from the Perspective of Digital Humanities

Authors: Liu Chang, Wang Dongbo, Zhao Zhixiao, Hu Die, Wu Mengcheng, Lin Litao, Shen Si, Li Bin, Liu Jiangfeng, Zhang Hai, Zhao Lianzheng

Abstract: The rapid advance in artificial intelligence technology has facilitated the prosperity of digital humanities research. Against such backdrop, research methods need to be transformed in the intelligent processing of ancient texts, which is a crucial component of digital humanities research, so as to adapt to new development trends in the wave of AIGC. In this study, we propose a GPT model called Si… ▽ More The rapid advance in artificial intelligence technology has facilitated the prosperity of digital humanities research. Against such backdrop, research methods need to be transformed in the intelligent processing of ancient texts, which is a crucial component of digital humanities research, so as to adapt to new development trends in the wave of AIGC. In this study, we propose a GPT model called SikuGPT based on the corpus of Siku Quanshu. The model's performance in tasks such as intralingual translation and text classification exceeds that of other GPT-type models aimed at processing ancient texts. SikuGPT's ability to process traditional Chinese ancient texts can help promote the organization of ancient information and knowledge services, as well as the international dissemination of Chinese ancient culture. △ Less

Submitted 16 April, 2023; originally announced April 2023.

Comments: 20 pages,1 figure

On Decoder Ties for the Binary Symmetric Channel with Arbitrarily Distributed Input

Authors: Ling-Hua Chang, Po-Ning Chen, Fady Alajaji

Abstract: The error probability of block codes sent under a non-uniform input distribution over the memoryless binary symmetric channel (BSC) and decoded via the maximum a posteriori (MAP) decoding rule is investigated. It is proved that the ratio of the probability of MAP decoder ties to the probability of error when no MAP decoding ties occur grows at most linearly in blocklength, thus showing that decode… ▽ More The error probability of block codes sent under a non-uniform input distribution over the memoryless binary symmetric channel (BSC) and decoded via the maximum a posteriori (MAP) decoding rule is investigated. It is proved that the ratio of the probability of MAP decoder ties to the probability of error when no MAP decoding ties occur grows at most linearly in blocklength, thus showing that decoder ties do not affect the code's error exponent. This result generalizes a similar recent result shown for the case of block codes transmitted over the BSC under a uniform input distribution. △ Less

Submitted 13 April, 2023; v1 submitted 14 March, 2023; originally announced March 2023.

CheckedCBox: Type Directed Program Partitioning with Checked C for Incremental Spatial Memory Safety

Authors: Liyi Li, Arunkumar Bhattar, Le Chang, Mingwei Zhu, Aravind Machiry

Abstract: Spatial memory safety violation is still a major issue for C programs. Checked-C is a safe dialect of C and extends it with Checked pointer types and annotations that guarantee spatial memory safety in a backward-compatible manner, allowing the mix of checked pointers and regular (unchecked) pointer types. However, unchecked code vulnerabilities can violate the checked code's spatial safety guaran… ▽ More Spatial memory safety violation is still a major issue for C programs. Checked-C is a safe dialect of C and extends it with Checked pointer types and annotations that guarantee spatial memory safety in a backward-compatible manner, allowing the mix of checked pointers and regular (unchecked) pointer types. However, unchecked code vulnerabilities can violate the checked code's spatial safety guarantees. We present CheckedCBox, which adds a flexible, type-directed program partitioning mechanism to Checked-C, by enhancing the Checked-C type system with tainted types that enable flexible partitioning of the program into checked and unchecked regions, in a manner such that unchecked region code does not affect the spatial safety in the checked region. We formalize our type system and prove the non-crashing and non-exposure properties of a well-typed CheckedCBox program. We implemented CheckedCBox in a configurable manner, which enables us to use existing sandbox mechanisms (eg WebAssembly) to execute programs. Consequently, in doing so, CheckedCBox has prevented four known vulnerabilities by efficiently partitioning the program. △ Less

Submitted 3 February, 2023; originally announced February 2023.

Comments: Liyi Li and Arunkumar Bhattar contributed equally to this work

Profile-Guided Parallel Task Extraction and Execution for Domain Specific Heterogeneous SoC

Authors: Liangliang Chang, Joshua Mack, Benjamin Willis, Xing Chen, John Brunhaver, Ali Akoglu, Chaitali Chakrabarti

Abstract: In this study, we introduce a methodology for automatically transforming user applications in the radar and communication domain written in C/C++ based on dynamic profiling to a parallel representation targeted for a heterogeneous SoC. We present our approach for instrumenting the user application binary during the compilation process with barrier synchronization primitives that enable runtime sys… ▽ More In this study, we introduce a methodology for automatically transforming user applications in the radar and communication domain written in C/C++ based on dynamic profiling to a parallel representation targeted for a heterogeneous SoC. We present our approach for instrumenting the user application binary during the compilation process with barrier synchronization primitives that enable runtime system schedule and execute independent tasks concurrently over the available compute resources. We demonstrate the capabilities of our integrated compile time and runtime flow through task-level parallel and functionally correct execution of real-life applications. We perform validation of our integrated system by executing four distinct applications each carrying various degrees of task level parallelism over the Xeon-based multi-core homogeneous processor. We use the proposed compilation and code transformation methodology to re-target each application for execution on a heterogeneous SoC composed of three ARM cores and one FFT accelerator that is emulated on the Xilinx Zynq UltraScale+ platform. We demonstrate our runtime's ability to process application binary, dispatch independent tasks over the available compute resources of the emulated SoC on the Zynq FPGA based on three different scheduling heuristics. Finally we demonstrate execution of each application individually with task level parallelism on the Zynq FPGA and execution of workload scenarios composed of multiple instances of the same application as well as mixture of two distinct applications to demonstrate ability to realize both application and task level parallel execution. Our integrated approach offers a path forward for application developers to take full advantage of the target SoC without requiring users to become hardware and parallel programming experts. △ Less

Submitted 26 November, 2022; originally announced November 2022.

Comments: 8 pages, accepted by ISPA 2022

Persistence of the Omicron variant of SARS-CoV-2 in Australia: The impact of fluctuating social distancing

Authors: Sheryl L. Chang, Quang Dang Nguyen, Alexandra Martiniuk, Vitali Sintchenko, Tania C. Sorrell, Mikhail Prokopenko

Abstract: We modelled emergence and spread of the Omicron variant of SARS-CoV-2 in Australia between December 2021 and June 2022. This pandemic stage exhibited a diverse epidemiological profile with emergence of co-circulating sub-lineages of Omicron, further complicated by differences in social distancing behaviour which varied over time. Our study delineated distinct phases of the Omicron-associated pande… ▽ More We modelled emergence and spread of the Omicron variant of SARS-CoV-2 in Australia between December 2021 and June 2022. This pandemic stage exhibited a diverse epidemiological profile with emergence of co-circulating sub-lineages of Omicron, further complicated by differences in social distancing behaviour which varied over time. Our study delineated distinct phases of the Omicron-associated pandemic stage, and retrospectively quantified the adoption of social distancing measures, fluctuating over different time periods in response to the observable incidence dynamics. We also modelled the corresponding disease burden, in terms of hospitalisations, intensive care unit occupancy, and mortality. Supported by good agreement between simulated and actual health data, our study revealed that the nonlinear dynamics observed in the daily incidence and disease burden were determined not only by introduction of sub-lineages of Omicron, but also by the fluctuating adoption of social distancing measures. Our high-resolution model can be used in design and evaluation of public health interventions during future crises. △ Less

Submitted 3 April, 2023; v1 submitted 20 November, 2022; originally announced November 2022.

Comments: 30 pages, 12 figures, source code: https://doi.org/10.5281/zenodo.7325675

MSC Class: 92D30; 93A16 ACM Class: J.3; I.6

Qafny: A Quantum-Program Verifier

Authors: Liyi Li, Mingwei Zhu, Rance Cleaveland, Alexander Nicolellis, Yi Lee, Le Chang, Xiaodi Wu

Abstract: Because of the probabilistic/nondeterministic behavior of quantum programs, it is highly advisable to verify them formally to ensure that they correctly implement their specifications. Formal verification, however, also traditionally requires significant effort. To address this challenge, we present Qafny, an automated proof system based on the program verifier Dafny and designed for verifying qua… ▽ More Because of the probabilistic/nondeterministic behavior of quantum programs, it is highly advisable to verify them formally to ensure that they correctly implement their specifications. Formal verification, however, also traditionally requires significant effort. To address this challenge, we present Qafny, an automated proof system based on the program verifier Dafny and designed for verifying quantum programs. At its core, Qafny uses a type-guided quantum proof system that translates quantum operations to classical array operations modeled within a classical separation logic framework. We prove the soundness and completeness of our proof system and implement a prototype compiler that transforms Qafny programs and specifications into Dafny for automated verification purposes. We then illustrate the utility of Qafny's automated capabilities in efficiently verifying important quantum algorithms, including quantum-walk algorithms, Grover's algorithm, and Shor's algorithm. △ Less

Submitted 19 January, 2024; v1 submitted 11 November, 2022; originally announced November 2022.

Comments: Version 4

Understanding Acoustic Patterns of Human Teachers Demonstrating Manipulation Tasks to Robots

Authors: Akanksha Saran, Kush Desai, Mai Lee Chang, Rudolf Lioutikov, Andrea Thomaz, Scott Niekum

Abstract: Humans use audio signals in the form of spoken language or verbal reactions effectively when teaching new skills or tasks to other humans. While demonstrations allow humans to teach robots in a natural way, learning from trajectories alone does not leverage other available modalities including audio from human teachers. To effectively utilize audio cues accompanying human demonstrations, first it… ▽ More Humans use audio signals in the form of spoken language or verbal reactions effectively when teaching new skills or tasks to other humans. While demonstrations allow humans to teach robots in a natural way, learning from trajectories alone does not leverage other available modalities including audio from human teachers. To effectively utilize audio cues accompanying human demonstrations, first it is important to understand what kind of information is present and conveyed by such cues. This work characterizes audio from human teachers demonstrating multi-step manipulation tasks to a situated Sawyer robot using three feature types: (1) duration of speech used, (2) expressiveness in speech or prosody, and (3) semantic content of speech. We analyze these features along four dimensions and find that teachers convey similar semantic concepts via spoken words for different conditions of (1) demonstration types, (2) audio usage instructions, (3) subtasks, and (4) errors during demonstrations. However, differentiating properties of speech in terms of duration and expressiveness are present along the four dimensions, highlighting that human audio carries rich information, potentially beneficial for technological advancement of robot learning from demonstration methods. △ Less

Submitted 1 November, 2022; originally announced November 2022.

Comments: IROS 2022

Motion correction in MRI using deep learning and a novel hybrid loss function

Authors: Lei Zhang, Xiaoke Wang, Michael Rawson, Radu Balan, Edward H. Herskovits, Elias Melhem, Linda Chang, Ze Wang, Thomas Ernst

Abstract: Purpose To develop and evaluate a deep learning-based method (MC-Net) to suppress motion artifacts in brain magnetic resonance imaging (MRI). Methods MC-Net was derived from a UNet combined with a two-stage multi-loss function. T1-weighted axial brain images contaminated with synthetic motions were used to train the network. Evaluation used simulated T1 and T2-weighted axial, coronal, and sagittal… ▽ More Purpose To develop and evaluate a deep learning-based method (MC-Net) to suppress motion artifacts in brain magnetic resonance imaging (MRI). Methods MC-Net was derived from a UNet combined with a two-stage multi-loss function. T1-weighted axial brain images contaminated with synthetic motions were used to train the network. Evaluation used simulated T1 and T2-weighted axial, coronal, and sagittal images unseen during training, as well as T1-weighted images with motion artifacts from real scans. Performance indices included the peak signal to noise ratio (PSNR), structural similarity index measure (SSIM), and visual reading scores. Two clinical readers scored the images. Results The MC-Net outperformed other methods implemented in terms of PSNR and SSIM on the T1 axial test set. The MC-Net significantly improved the quality of all T1-weighted images (for all directions and for simulated as well as real motion artifacts), both on quantitative measures and visual scores. However, the MC-Net performed poorly on images of untrained contrast (T2-weighted). Conclusion The proposed two-stage multi-loss MC-Net can effectively suppress motion artifacts in brain MRI without compromising image context. Given the efficiency of the MC-Net (single image processing time ~40ms), it can potentially be used in real clinical settings. To facilitate further research, the code and trained model are available at https://github.com/MRIMoCo/DL_Motion_Correction. △ Less

Submitted 19 October, 2022; originally announced October 2022.

FedBA: Non-IID Federated Learning Framework in UAV Networks

Authors: Pei Li, Zhijun Liu, Luyi Chang, Jialiang Peng, Yi Wu

Abstract: With the development and progress of science and technology, the Internet of Things(IoT) has gradually entered people's lives, bringing great convenience to our lives and improving people's work efficiency. Specifically, the IoT can replace humans in jobs that they cannot perform. As a new type of IoT vehicle, the current status and trend of research on Unmanned Aerial Vehicle(UAV) is gratifying,… ▽ More With the development and progress of science and technology, the Internet of Things(IoT) has gradually entered people's lives, bringing great convenience to our lives and improving people's work efficiency. Specifically, the IoT can replace humans in jobs that they cannot perform. As a new type of IoT vehicle, the current status and trend of research on Unmanned Aerial Vehicle(UAV) is gratifying, and the development prospect is very promising. However, privacy and communication are still very serious issues in drone applications. This is because most drones still use centralized cloud-based data processing, which may lead to leakage of data collected by drones. At the same time, the large amount of data collected by drones may incur greater communication overhead when transferred to the cloud. Federated learning as a means of privacy protection can effectively solve the above two problems. However, federated learning when applied to UAV networks also needs to consider the heterogeneity of data, which is caused by regional differences in UAV regulation. In response, this paper proposes a new algorithm FedBA to optimize the global model and solves the data heterogeneity problem. In addition, we apply the algorithm to some real datasets, and the experimental results show that the algorithm outperforms other algorithms and improves the accuracy of the local model for UAVs. △ Less

Submitted 26 December, 2022; v1 submitted 10 October, 2022; originally announced October 2022.

Interpretable (not just posthoc-explainable) medical claims modeling for discharge placement to prevent avoidable all-cause readmissions or death

Authors: Joshua C. Chang, Ted L. Chang, Carson C. Chow, Rohit Mahajan, Sonya Mahajan, Joe Maisog, Shashaank Vattikuti, Hong**g Xia

Abstract: We developed an inherently interpretable multilevel Bayesian framework for representing variation in regression coefficients that mimics the piecewise linearity of ReLU-activated deep neural networks. We used the framework to formulate a survival model for using medical claims to predict hospital readmission and death that focuses on discharge placement, adjusting for confounding in estimating cau… ▽ More We developed an inherently interpretable multilevel Bayesian framework for representing variation in regression coefficients that mimics the piecewise linearity of ReLU-activated deep neural networks. We used the framework to formulate a survival model for using medical claims to predict hospital readmission and death that focuses on discharge placement, adjusting for confounding in estimating causal local average treatment effects. We trained the model on a 5% sample of Medicare beneficiaries from 2008 and 2011, based on their 2009--2011 inpatient episodes, and then tested the model on 2012 episodes. The model scored an AUROC of approximately 0.76 on predicting all-cause readmissions -- defined using official Centers for Medicare and Medicaid Services (CMS) methodology -- or death within 30-days of discharge, being competitive against XGBoost and a Bayesian deep neural network, demonstrating that one need-not sacrifice interpretability for accuracy. Crucially, as a regression model, we provide what blackboxes cannot -- the exact gold-standard global interpretation of the model, identifying relative risk factors and quantifying the effect of discharge placement. We also show that the posthoc explainer SHAP fails to provide accurate explanations. △ Less

Submitted 29 January, 2023; v1 submitted 28 August, 2022; originally announced August 2022.

Comments: In review

Log-linear Error State Model Derivation without Approximation for INS

Authors: Lubin Chang, Yarong Luo

Abstract: Through assembling the navigation parameters as matrix Lie group state, the corresponding inertial navigation system (INS) kinematic model possesses a group-affine property. The Lie logarithm of the navigation state estimation error satisfies a log-linear autonomous differential equation. These log-linear models are still applicable even with arbitrarily large initial errors, which is very attract… ▽ More Through assembling the navigation parameters as matrix Lie group state, the corresponding inertial navigation system (INS) kinematic model possesses a group-affine property. The Lie logarithm of the navigation state estimation error satisfies a log-linear autonomous differential equation. These log-linear models are still applicable even with arbitrarily large initial errors, which is very attractive for INS initial alignment. However, in existing works, the log-linear models are all derived based on first-order linearization approximation, which seemingly goes against their successful applications in INS initial alignment with large misalignments. In this work, it is shown that the log-linear models can also be derived without any approximation, the error dynamics for both left and right invariant error in continuous time are given in matrix Lie group SE_2 (3) for the first time. This work provides another evidence for the validity of the log-linear model in situations with arbitrarily large initial errors. △ Less

Submitted 5 August, 2022; originally announced August 2022.

ReLyMe: Improving Lyric-to-Melody Generation by Incorporating Lyric-Melody Relationships

Authors: Chen Zhang, Luchin Chang, Songruoyao Wu, Xu Tan, Tao Qin, Tie-Yan Liu, Kejun Zhang

Abstract: Lyric-to-melody generation, which generates melody according to given lyrics, is one of the most important automatic music composition tasks. With the rapid development of deep learning, previous works address this task with end-to-end neural network models. However, deep learning models cannot well capture the strict but subtle relationships between lyrics and melodies, which compromises the harm… ▽ More Lyric-to-melody generation, which generates melody according to given lyrics, is one of the most important automatic music composition tasks. With the rapid development of deep learning, previous works address this task with end-to-end neural network models. However, deep learning models cannot well capture the strict but subtle relationships between lyrics and melodies, which compromises the harmony between lyrics and generated melodies. In this paper, we propose ReLyMe, a method that incorporates Relationships between Lyrics and Melodies from music theory to ensure the harmony between lyrics and melodies. Specifically, we first introduce several principles that lyrics and melodies should follow in terms of tone, rhythm, and structure relationships. These principles are then integrated into neural network lyric-to-melody models by adding corresponding constraints during the decoding process to improve the harmony between lyrics and melodies. We use a series of objective and subjective metrics to evaluate the generated melodies. Experiments on both English and Chinese song datasets show the effectiveness of ReLyMe, demonstrating the superiority of incorporating lyric-melody relationships from the music domain into neural lyric-to-melody generation. △ Less

Submitted 12 July, 2022; originally announced July 2022.

Comments: Accepted by ACMMM 2022, oral

Mask2Hand: Learning to Predict the 3D Hand Pose and Shape from Shadow

Authors: Li-Jen Chang, Yu-Cheng Liao, Chia-Hui Lin, Hwann-Tzong Chen

Abstract: We present a self-trainable method, Mask2Hand, which learns to solve the challenging task of predicting 3D hand pose and shape from a 2D binary mask of hand silhouette/shadow without additional manually-annotated data. Given the intrinsic camera parameters and the parametric hand model in the camera space, we adopt the differentiable rendering technique to project 3D estimations onto the 2D binary… ▽ More We present a self-trainable method, Mask2Hand, which learns to solve the challenging task of predicting 3D hand pose and shape from a 2D binary mask of hand silhouette/shadow without additional manually-annotated data. Given the intrinsic camera parameters and the parametric hand model in the camera space, we adopt the differentiable rendering technique to project 3D estimations onto the 2D binary silhouette space. By applying a tailored combination of losses between the rendered silhouette and the input binary mask, we are able to integrate the self-guidance mechanism into our end-to-end optimization process for constraining global mesh registration and hand pose estimation. The experiments show that our method, which takes a single binary mask as the input, can achieve comparable prediction accuracy on both unaligned and aligned settings as state-of-the-art methods that require RGB or depth inputs. Our code is available at https://github.com/lijenchang/Mask2Hand. △ Less

Submitted 1 July, 2022; v1 submitted 31 May, 2022; originally announced May 2022.

6G-enabled Edge AI for Metaverse: Challenges, Methods, and Future Research Directions

Authors: Luyi Chang, Zhe Zhang, Pei Li, Shan Xi, Wei Guo, Yukang Shen, Zehui Xiong, Jiawen Kang, Dusit Niyato, Xiuquan Qiao, Yi Wu

Abstract: 6G-enabled edge intelligence opens up a new era of Internet of Everything and makes it possible to interconnect people-devices-cloud anytime, anywhere. More and more next-generation wireless network smart service applications are changing our way of life and improving our quality of life. As the hottest new form of next-generation Internet applications, Metaverse is striving to connect billions of… ▽ More 6G-enabled edge intelligence opens up a new era of Internet of Everything and makes it possible to interconnect people-devices-cloud anytime, anywhere. More and more next-generation wireless network smart service applications are changing our way of life and improving our quality of life. As the hottest new form of next-generation Internet applications, Metaverse is striving to connect billions of users and create a shared world where virtual and reality merge. However, limited by resources, computing power, and sensory devices, Metaverse is still far from realizing its full vision of immersion, materialization, and interoperability. To this end, this survey aims to realize this vision through the organic integration of 6G-enabled edge AI and Metaverse. Specifically, we first introduce three new types of edge-Metaverse architectures that use 6G-enabled edge AI to solve resource and computing constraints in Metaverse. Then we summarize technical challenges that these architectures face in Metaverse and the existing solutions. Furthermore, we explore how the edge-Metaverse architecture technology helps Metaverse to interact and share digital data. Finally, we discuss future research directions to realize the true vision of Metaverse with 6G-enabled edge AI. △ Less

Submitted 13 April, 2022; originally announced April 2022.

Comments: 16 pages

An End-to-End Cascaded Image Deraining and Object Detection Neural Network

Authors: Kaige Wang, Tianming Wang, Jianchuang Qu, Huatao Jiang, Qing Li, Lin Chang

Abstract: While the deep learning-based image deraining methods have made great progress in recent years, there are two major shortcomings in their application in real-world situations. Firstly, the gap between the low-level vision task represented by rain removal and the high-level vision task represented by object detection is significant, and the low-level vision task can hardly contribute to the high-le… ▽ More While the deep learning-based image deraining methods have made great progress in recent years, there are two major shortcomings in their application in real-world situations. Firstly, the gap between the low-level vision task represented by rain removal and the high-level vision task represented by object detection is significant, and the low-level vision task can hardly contribute to the high-level vision task. Secondly, the quality of the deraining dataset needs to be improved. In fact, the rain lines in many baselines have a large gap with the real rain lines, and the resolution of the deraining dataset images is generally not ideally. Meanwhile, there are few common datasets for both the low-level vision task and the high-level vision task. In this paper, we explore the combination of the low-level vision task with the high-level vision task. Specifically, we propose an end-to-end object detection network for reducing the impact of rainfall, which consists of two cascaded networks, an improved image deraining network and an object detection network, respectively. We also design the components of the loss function to accommodate the characteristics of the different sub-networks. We then propose a dataset based on the KITTI dataset for rainfall removal and object detection, on which our network surpasses the state-of-the-art with a significant improvement in metrics. Besides, our proposed network is measured on driving videos collected by self-driving vehicles and shows positive results for rain removal and object detection. △ Less

Submitted 22 February, 2022; originally announced February 2022.

Modeling unknown dynamical systems with hidden parameters

Authors: Xiaohan Fu, Weize Mao, Lo-Bin Chang, Dongbin Xiu

Abstract: We present a data-driven numerical approach for modeling unknown dynamical systems with missing/hidden parameters. The method is based on training a deep neural network (DNN) model for the unknown system using its trajectory data. A key feature is that the unknown dynamical system contains system parameters that are completely hidden, in the sense that no information about the parameters is availa… ▽ More We present a data-driven numerical approach for modeling unknown dynamical systems with missing/hidden parameters. The method is based on training a deep neural network (DNN) model for the unknown system using its trajectory data. A key feature is that the unknown dynamical system contains system parameters that are completely hidden, in the sense that no information about the parameters is available through either the measurement trajectory data or our prior knowledge of the system. We demonstrate that by training a DNN using the trajectory data with sufficient time history, the resulting DNN model can accurately model the unknown dynamical system. For new initial conditions associated with new, and unknown, system parameters, the DNN model can produce accurate system predictions over longer time. △ Less

Submitted 3 February, 2022; originally announced February 2022.

Semantic Search as Extractive Paraphrase Span Detection

Authors: Jenna Kanerva, Hanna Kitti, Li-Hsin Chang, Teemu Vahtola, Mathias Creutz, Filip Ginter

Abstract: In this paper, we approach the problem of semantic search by framing the search task as paraphrase span detection, i.e. given a segment of text as a query phrase, the task is to identify its paraphrase in a given document, the same modelling setup as typically used in extractive question answering. On the Turku Paraphrase Corpus of 100,000 manually extracted Finnish paraphrase pairs including thei… ▽ More In this paper, we approach the problem of semantic search by framing the search task as paraphrase span detection, i.e. given a segment of text as a query phrase, the task is to identify its paraphrase in a given document, the same modelling setup as typically used in extractive question answering. On the Turku Paraphrase Corpus of 100,000 manually extracted Finnish paraphrase pairs including their original document context, we find that our paraphrase span detection model outperforms two strong retrieval baselines (lexical similarity and BERT sentence embeddings) by 31.9pp and 22.4pp respectively in terms of exact match, and by 22.3pp and 12.9pp in terms of token-level F-score. This demonstrates a strong advantage of modelling the task in terms of span retrieval, rather than sentence similarity. Additionally, we introduce a method for creating artificial paraphrase data through back-translation, suitable for languages where manually annotated paraphrase resources for training the span detection model are not available. △ Less

Submitted 9 December, 2021; originally announced December 2021.

Graph-based Solutions with Residuals for Intrusion Detection: the Modified E-GraphSAGE and E-ResGAT Algorithms

Authors: Liyan Chang, Paula Branco

Abstract: The high volume of increasingly sophisticated cyber threats is drawing growing attention to cybersecurity, where many challenges remain unresolved. Namely, for intrusion detection, new algorithms that are more robust, effective, and able to use more information are needed. Moreover, the intrusion detection task faces a serious challenge associated with the extreme class imbalance between normal an… ▽ More The high volume of increasingly sophisticated cyber threats is drawing growing attention to cybersecurity, where many challenges remain unresolved. Namely, for intrusion detection, new algorithms that are more robust, effective, and able to use more information are needed. Moreover, the intrusion detection task faces a serious challenge associated with the extreme class imbalance between normal and malicious traffics. Recently, graph-neural network (GNN) achieved state-of-the-art performance to model the network topology in cybersecurity tasks. However, only a few works exist using GNNs to tackle the intrusion detection problem. Besides, other promising avenues such as applying the attention mechanism are still under-explored. This paper presents two novel graph-based solutions for intrusion detection, the modified E-GraphSAGE, and E-ResGATalgorithms, which rely on the established GraphSAGE and graph attention network (GAT), respectively. The key idea is to integrate residual learning into the GNN leveraging the available graph information. Residual connections are added as a strategy to deal with the high-class imbalance, aiming at retaining the original information and improving the minority classes' performance. An extensive experimental evaluation of four recent intrusion detection datasets shows the excellent performance of our approaches, especially when predicting minority classes. △ Less

Submitted 26 November, 2021; originally announced November 2021.

Comments: 11 pages, 4 figures

TYolov5: A Temporal Yolov5 Detector Based on Quasi-Recurrent Neural Networks for Real-Time Handgun Detection in Video

Authors: Mario Alberto Duran-Vega, Miguel Gonzalez-Mendoza, Leonardo Chang, Cuauhtemoc Daniel Suarez-Ramirez

Abstract: Timely handgun detection is a crucial problem to improve public safety; nevertheless, the effectiveness of many surveillance systems still depends of finite human attention. Much of the previous research on handgun detection is based on static image detectors, leaving aside valuable temporal information that could be used to improve object detection in videos. To improve the performance of surveil… ▽ More Timely handgun detection is a crucial problem to improve public safety; nevertheless, the effectiveness of many surveillance systems still depends of finite human attention. Much of the previous research on handgun detection is based on static image detectors, leaving aside valuable temporal information that could be used to improve object detection in videos. To improve the performance of surveillance systems, a real-time temporal handgun detection system should be built. Using Temporal Yolov5, an architecture based on Quasi-Recurrent Neural Networks, temporal information is extracted from video to improve the results of handgun detection. Moreover, two publicly available datasets are proposed, labeled with hands, guns, and phones. One containing 2199 static images to train static detectors, and another with 5960 frames of videos to train temporal modules. Additionally, we explore two temporal data augmentation techniques based on Mosaic and Mixup. The resulting systems are three temporal architectures: one focused in reducing inference with a mAP$_{50:95}$ of 55.9, another in having a good balance between inference and accuracy with a mAP$_{50:95}$ of 59, and a last one specialized in accuracy with a mAP$_{50:95}$ of 60.2. Temporal Yolov5 achieves real-time detection in the small and medium architectures. Moreover, it takes advantage of temporal features contained in videos to perform better than Yolov5 in our temporal dataset, making TYolov5 suitable for real-world applications. The source code is publicly available at https://github.com/MarioDuran/TYolov5. △ Less

Submitted 18 November, 2021; v1 submitted 16 November, 2021; originally announced November 2021.

Real-time Instance Segmentation of Surgical Instruments using Attention and Multi-scale Feature Fusion

Authors: Juan Carlos Angeles-Ceron, Gilberto Ochoa-Ruiz, Leonardo Chang, Sharib Ali

Abstract: Precise instrument segmentation aid surgeons to navigate the body more easily and increase patient safety. While accurate tracking of surgical instruments in real-time plays a crucial role in minimally invasive computer-assisted surgeries, it is a challenging task to achieve, mainly due to 1) complex surgical environment, and 2) model design with both optimal accuracy and speed. Deep learning give… ▽ More Precise instrument segmentation aid surgeons to navigate the body more easily and increase patient safety. While accurate tracking of surgical instruments in real-time plays a crucial role in minimally invasive computer-assisted surgeries, it is a challenging task to achieve, mainly due to 1) complex surgical environment, and 2) model design with both optimal accuracy and speed. Deep learning gives us the opportunity to learn complex environment from large surgery scene environments and placements of these instruments in real world scenarios. The Robust Medical Instrument Segmentation 2019 challenge (ROBUST-MIS) provides more than 10,000 frames with surgical tools in different clinical settings. In this paper, we use a light-weight single stage instance segmentation model complemented with a convolutional block attention module for achieving both faster and accurate inference. We further improve accuracy through data augmentation and optimal anchor localisation strategies. To our knowledge, this is the first work that explicitly focuses on both real-time performance and improved accuracy. Our approach out-performed top team performances in the ROBUST-MIS challenge with over 44% improvement on both area-based metric MI_DSC and distance-based metric MI_NSD. We also demonstrate real-time performance (> 60 frames-per-second) with different but competitive variants of our final approach. △ Less

Submitted 9 November, 2021; v1 submitted 8 November, 2021; originally announced November 2021.

Systematic definition and classification of data anomalies in DBMS (English Version)

Authors: Li Hai-Xiang, Li Xiao-Yan, Liu Chang, Du Xiao-Yong, Lu Wei, Pan An-Qun

Abstract: There is no unified definition of Data anomalies, which refers to the specific data operation mode that may violate the consistency of the database. Known data anomalies include Dirty Write, Dirty Read, Non-repeatable Read, Phantom, Read Skew and Write Skew, etc. In order to improve the efficiency of concurrency control algorithms, data anomalies are also used to define the isolation levels, becau… ▽ More There is no unified definition of Data anomalies, which refers to the specific data operation mode that may violate the consistency of the database. Known data anomalies include Dirty Write, Dirty Read, Non-repeatable Read, Phantom, Read Skew and Write Skew, etc. In order to improve the efficiency of concurrency control algorithms, data anomalies are also used to define the isolation levels, because the weak isolation level can improve the efficiency of transaction processing systems. This paper systematically studies the data anomalies and the corresponding isolation levels. We report twenty-two new data anomalies that other papers have not reported, and all data anomalies are classified miraculously. Based on the classification of data anomalies, two new isolation levels systems with different granularity are proposed, which reveals the rule of defining isolation levels based on data anomalies and makes the cognition of data anomalies and isolation levels more concise. △ Less

Submitted 27 October, 2021; originally announced October 2021.

MEKF Ignoring Initial Conditions for Attitude Estimation Using Vector Observations

Authors: Lubin Chang

Abstract: In this paper, the well-known multiplicative extended Kalman filter (MEKF) is re-investigated for attitude estimation using vector observations. From the Lie group theory, it is shown that the attitude estimation model is group affine and its error state model should be trajectory-independent. Moreover, with such trajectory-independent error state model, the linear Kalman filter is still effective… ▽ More In this paper, the well-known multiplicative extended Kalman filter (MEKF) is re-investigated for attitude estimation using vector observations. From the Lie group theory, it is shown that the attitude estimation model is group affine and its error state model should be trajectory-independent. Moreover, with such trajectory-independent error state model, the linear Kalman filter is still effective for large initialization errors. However, the measurement model of the traditional MEKF is dependent on the attitude prediction, which is therefore trajectory-dependent. This is also the main reason why the performance of traditional MEKF is degraded for large initialization errors. Through substitution of the attitude prediction related term with the vector observation in body frame, a trajectory-independent measurement model is derived for MEKF. Meanwhile, the MEKFs with reference attitude error definition and with global state formulating on special Euclidean group have also been studied, with main focus on derivation of the trajectory-independent measurement models. Extensive Monte Carlo simulations and field test of attitude estimation implementations demonstrate that the performance of MEKFs can be much improved with trajectory-independent measurement models. △ Less

Submitted 26 October, 2021; originally announced October 2021.

Performance Analysis for Covert Communications Under Faster-than-Nyquist Signaling

Authors: Yuan Li, Yuchen Zhang, Wanyu Xiang, Jianquan Wang, Sa Xiao, Liang Chang, Wanbin Tang

Abstract: In this letter, we analyze the performance of covert communications under faster-than-Nyquist (FTN) signaling in the Rayleigh block fading channel. Both Bayesian criterion- and Kullback-Leibler (KL) divergence-based covertness constraints are considered. Especially, for KL divergence-based one, we prove that both the maximum transmit power and covert rate under FTN signaling are higher than those… ▽ More In this letter, we analyze the performance of covert communications under faster-than-Nyquist (FTN) signaling in the Rayleigh block fading channel. Both Bayesian criterion- and Kullback-Leibler (KL) divergence-based covertness constraints are considered. Especially, for KL divergence-based one, we prove that both the maximum transmit power and covert rate under FTN signaling are higher than those under Nyquist signaling. Numerical results coincide with our analysis and validate the advantages of FTN signaling to realize covert data transmission. △ Less

Submitted 17 January, 2022; v1 submitted 20 October, 2021; originally announced October 2021.

Comments: We have corrected the typos and inappropriate description throughout the paper as reviewers suggested. We have proved the superiority of FTN signaling on covert communications with the same detection time duration at Willie in Theorem 3 and renewed the simulation results in Section V as Reviewer 2 suggested. This paper has been resubmitted to IEEE Communications Letters on 14-Jan-2022

PDAugment: Data Augmentation by Pitch and Duration Adjustments for Automatic Lyrics Transcription

Authors: Chen Zhang, Jiaxing Yu, LuChin Chang, Xu Tan, Jiawei Chen, Tao Qin, Kejun Zhang

Abstract: Automatic lyrics transcription (ALT), which can be regarded as automatic speech recognition (ASR) on singing voice, is an interesting and practical topic in academia and industry. ALT has not been well developed mainly due to the dearth of paired singing voice and lyrics datasets for model training. Considering that there is a large amount of ASR training data, a straightforward method is to lever… ▽ More Automatic lyrics transcription (ALT), which can be regarded as automatic speech recognition (ASR) on singing voice, is an interesting and practical topic in academia and industry. ALT has not been well developed mainly due to the dearth of paired singing voice and lyrics datasets for model training. Considering that there is a large amount of ASR training data, a straightforward method is to leverage ASR data to enhance ALT training. However, the improvement is marginal when training the ALT system directly with ASR data, because of the gap between the singing voice and standard speech data which is rooted in music-specific acoustic characteristics in singing voice. In this paper, we propose PDAugment, a data augmentation method that adjusts pitch and duration of speech at syllable level under the guidance of music scores to help ALT training. Specifically, we adjust the pitch and duration of each syllable in natural speech to those of the corresponding note extracted from music scores, so as to narrow the gap between natural speech and singing voice. Experiments on DSing30 and Dali corpus show that the ALT system equipped with our PDAugment outperforms previous state-of-the-art systems by 5.9% and 18.1% WERs respectively, demonstrating the effectiveness of PDAugment for ALT. △ Less

Submitted 17 September, 2021; v1 submitted 16 September, 2021; originally announced September 2021.

Comments: 7 pages

Recovering individual emotional states from sparse ratings using collaborative filtering

Authors: Eshin Jolly, Max Farrens, Nathan Greenstein, Hedwig Eisenbarth, Marianne Reddan, Eric Andrews, Tor D. Wager, Luke J. Chang

Abstract: A fundamental challenge in emotion research is measuring feeling states with high granularity and temporal precision without disrupting the emotion generation process. Here we introduce and validate a new approach in which responses are sparsely sampled and the missing data are recovered using a computational technique known as collaborative filtering (CF). This approach leverages structured covar… ▽ More A fundamental challenge in emotion research is measuring feeling states with high granularity and temporal precision without disrupting the emotion generation process. Here we introduce and validate a new approach in which responses are sparsely sampled and the missing data are recovered using a computational technique known as collaborative filtering (CF). This approach leverages structured covariation across individual experiences and is available in Neighbors, an open-source Python toolbox. We validate our approach across three different experimental contexts by recovering dense individual ratings using only a small subset of the original data. In dataset 1, participants (n=316) separately rated 112 emotional images on 6 different discrete emotions. In dataset 2, participants (n=203) watched 8 short emotionally engaging autobiographical stories while simultaneously providing moment-by-moment ratings of the intensity of their affective experience. In dataset 3, participants (n=60) with distinct social preferences made 76 decisions about how much money to return in a hidden multiplier trust game. Across all experimental contexts, CF was able to accurately recover missing data and importantly outperformed mean imputation, particularly in contexts with greater individual variability. This approach will enable new avenues for affective science research by allowing researchers to acquire high dimensional ratings from emotional experiences with minimal disruption to the emotion-generation process. △ Less

Submitted 4 October, 2022; v1 submitted 14 September, 2021; originally announced September 2021.

Comments: 21 pages, 8 figures

CE-Dedup: Cost-Effective Convolutional Neural Nets Training based on Image Deduplication

Authors: Xuan Li, Liqiong Chang, Xue Liu

Abstract: Attributed to the ever-increasing large image datasets, Convolutional Neural Networks (CNNs) have become popular for vision-based tasks. It is generally admirable to have larger-sized datasets for higher network training accuracies. However, the impact of dataset quality has not to be involved. It is reasonable to assume the near-duplicate images exist in the datasets. For instance, the Street Vie… ▽ More Attributed to the ever-increasing large image datasets, Convolutional Neural Networks (CNNs) have become popular for vision-based tasks. It is generally admirable to have larger-sized datasets for higher network training accuracies. However, the impact of dataset quality has not to be involved. It is reasonable to assume the near-duplicate images exist in the datasets. For instance, the Street View House Numbers (SVHN) dataset having cropped house plate digits from 0 to 9 are likely to have repetitive digits from the same/similar house plates. Redundant images may take up a certain portion of the dataset without consciousness. While contributing little to no accuracy improvement for the CNNs training, these duplicated images unnecessarily pose extra resource and computation consumption. To this end, this paper proposes a framework to assess the impact of the near-duplicate images on CNN training performance, called CE-Dedup. Specifically, CE-Dedup associates a hashing-based image deduplication approach with downstream CNNs-based image classification tasks. CE-Dedup balances the tradeoff between a large deduplication ratio and a stable accuracy by adjusting the deduplication threshold. The effectiveness of CE-Dedup is validated through extensive experiments on well-known CNN benchmarks. On one hand, while maintaining the same validation accuracy, CE-Dedup can reduce the dataset size by 23%. On the other hand, when allowing a small validation accuracy drop (by 5%), CE-Dedup can trim the dataset size by 75%. △ Less

Submitted 23 August, 2021; originally announced September 2021.

A Unifying Theory of Thompson Sampling for Continuous Risk-Averse Bandits

Authors: Joel Q. L. Chang, Vincent Y. F. Tan

Abstract: This paper unifies the design and the analysis of risk-averse Thompson sampling algorithms for the multi-armed bandit problem for a class of risk functionals $ρ$ that are continuous and dominant. We prove generalised concentration bounds for these continuous and dominant risk functionals and show that a wide class of popular risk functionals belong to this class. Using our newly developed analytic… ▽ More This paper unifies the design and the analysis of risk-averse Thompson sampling algorithms for the multi-armed bandit problem for a class of risk functionals $ρ$ that are continuous and dominant. We prove generalised concentration bounds for these continuous and dominant risk functionals and show that a wide class of popular risk functionals belong to this class. Using our newly developed analytical toolkits, we analyse the algorithm $ρ$-MTS (for multinomial distributions) and prove that they admit asymptotically optimal regret bounds of risk-averse algorithms under CVaR, proportional hazard, and other ubiquitous risk measures. More generally, we prove the asymptotic optimality of $ρ$-MTS for Bernoulli distributions for a class of risk measures known as empirical distribution performance measures (EDPMs); this includes the well-known mean-variance. Numerical simulations show that the regret bounds incurred by our algorithms are reasonably tight vis-à-vis algorithm-independent lower bounds. △ Less

Submitted 17 April, 2022; v1 submitted 25 August, 2021; originally announced August 2021.

Comments: Accepted to the Association for the Advancement of Artificial Intelligence (AAAI) 2022

Annotation Guidelines for the Turku Paraphrase Corpus

Authors: Jenna Kanerva, Filip Ginter, Li-Hsin Chang, Iiro Rastas, Valtteri Skantsi, Jemina Kilpeläinen, Hanna-Mari Kupari, Aurora Piirto, Jenna Saarni, Maija Sevón, Otto Tarkka

Abstract: This document describes the annotation guidelines used to construct the Turku Paraphrase Corpus. These guidelines were developed together with the corpus annotation, revising and extending the guidelines regularly during the annotation work. Our paraphrase annotation scheme uses the base scale 1-4, where labels 1 and 2 are used for negative candidates (not paraphrases), while labels 3 and 4 are pa… ▽ More This document describes the annotation guidelines used to construct the Turku Paraphrase Corpus. These guidelines were developed together with the corpus annotation, revising and extending the guidelines regularly during the annotation work. Our paraphrase annotation scheme uses the base scale 1-4, where labels 1 and 2 are used for negative candidates (not paraphrases), while labels 3 and 4 are paraphrases at least in the given context if not everywhere. In addition to base labeling, the scheme is enriched with additional subcategories (flags) for categorizing different types of paraphrases inside the two positive labels, making the annotation scheme suitable for more fine-grained paraphrase categorization. The annotation scheme is used to annotate over 100,000 Finnish paraphrase pairs. △ Less

Submitted 19 August, 2021; v1 submitted 17 August, 2021; originally announced August 2021.

Comments: The Turku Paraphrase Corpus is available at https://turkunlp.org/paraphrase.html

Simulating transmission scenarios of the Delta variant of SARS-CoV-2 in Australia

Authors: Sheryl L. Chang, Oliver M. Cliff, Cameron Zachreson, Mikhail Prokopenko

Abstract: An outbreak of the Delta (B.1.617.2) variant of SARS-CoV-2 that began around mid-June 2021 in Sydney, Australia, quickly developed into a nation-wide epidemic. The ongoing epidemic is of major concern as the Delta variant is more infectious than previous variants that circulated in Australia in 2020. Using a re-calibrated agent-based model, we explored a feasible range of non-pharmaceutical interv… ▽ More An outbreak of the Delta (B.1.617.2) variant of SARS-CoV-2 that began around mid-June 2021 in Sydney, Australia, quickly developed into a nation-wide epidemic. The ongoing epidemic is of major concern as the Delta variant is more infectious than previous variants that circulated in Australia in 2020. Using a re-calibrated agent-based model, we explored a feasible range of non-pharmaceutical interventions, including case isolation, home quarantine, school closures, and stay-at-home restrictions (i.e., "social distancing"). Our modelling indicated that the levels of reduced interactions in workplaces and across communities attained in Sydney and other parts of the nation were inadequate for controlling the outbreak. A counter-factual analysis suggested that if 70% of the population followed tight stay-at-home restrictions, then at least 45 days would have been needed for new daily cases to fall from their peak to below ten per day. Our model predicted that, under a progressive vaccination rollout, if 40-50% of the Australian population follow stay-at-home restrictions, the incidence will peak by mid-October 2021: the peak in incidence across the nation was indeed observed in mid-October. We also quantified an expected burden on the healthcare system and potential fatalities across Australia. △ Less

Submitted 10 March, 2022; v1 submitted 14 July, 2021; originally announced July 2021.

Comments: 36 pages, 15 figures, published in "Frontiers in Public Health", 24 February 2022

Journal ref: Frontiers in Public Health, 10 (2022)

Thompson Sampling for Gaussian Entropic Risk Bandits

Authors: Ming Liang Ang, Eloise Y. Y. Lim, Joel Q. L. Chang

Abstract: The multi-armed bandit (MAB) problem is a ubiquitous decision-making problem that exemplifies exploration-exploitation tradeoff. Standard formulations exclude risk in decision making. Risknotably complicates the basic reward-maximising objectives, in part because there is no universally agreed definition of it. In this paper, we consider an entropic risk (ER) measure and explore the performance of… ▽ More The multi-armed bandit (MAB) problem is a ubiquitous decision-making problem that exemplifies exploration-exploitation tradeoff. Standard formulations exclude risk in decision making. Risknotably complicates the basic reward-maximising objectives, in part because there is no universally agreed definition of it. In this paper, we consider an entropic risk (ER) measure and explore the performance of a Thompson sampling-based algorithm ERTS under this risk measure by providing regret bounds for ERTS and corresponding instance dependent lower bounds. △ Less

Submitted 14 May, 2021; originally announced May 2021.

Comments: arXiv admin note: text overlap with arXiv:2011.08046

Quantitative Evaluation of Alternative Translations in a Corpus of Highly Dissimilar Finnish Paraphrases

Authors: Li-Hsin Chang, Sampo Pyysalo, Jenna Kanerva, Filip Ginter

Abstract: In this paper, we present a quantitative evaluation of differences between alternative translations in a large recently released Finnish paraphrase corpus focusing in particular on non-trivial variation in translation. We combine a series of automatic steps detecting systematic variation with manual analysis to reveal regularities and identify categories of translation differences. We find the par… ▽ More In this paper, we present a quantitative evaluation of differences between alternative translations in a large recently released Finnish paraphrase corpus focusing in particular on non-trivial variation in translation. We combine a series of automatic steps detecting systematic variation with manual analysis to reveal regularities and identify categories of translation differences. We find the paraphrase corpus to contain highly non-trivial translation variants difficult to recognize through automatic approaches. △ Less

Submitted 6 May, 2021; originally announced May 2021.

Comments: Accepted to Workshop on MOdelling TRAnslation: Translatology in the Digital Age

Deep learning for sentence clustering in essay grading support

Authors: Li-Hsin Chang, Iiro Rastas, Sampo Pyysalo, Filip Ginter

Abstract: Essays as a form of assessment test student knowledge on a deeper level than short answer and multiple-choice questions. However, the manual evaluation of essays is time- and labor-consuming. Automatic clustering of essays, or their fragments, prior to manual evaluation presents a possible solution to reducing the effort required in the evaluation process. Such clustering presents numerous challen… ▽ More Essays as a form of assessment test student knowledge on a deeper level than short answer and multiple-choice questions. However, the manual evaluation of essays is time- and labor-consuming. Automatic clustering of essays, or their fragments, prior to manual evaluation presents a possible solution to reducing the effort required in the evaluation process. Such clustering presents numerous challenges due to the variability and ambiguity of natural language. In this paper, we introduce two datasets of undergraduate student essays in Finnish, manually annotated for salient arguments on the sentence level. Using these datasets, we evaluate several deep-learning embedding methods for their suitability to sentence clustering in support of essay grading. We find that the choice of the most suitable method depends on the nature of the exam question and the answers, with deep-learning methods being capable of, but not guaranteeing better performance over simpler methods based on lexical overlap. △ Less

Submitted 23 April, 2021; originally announced April 2021.

Comments: Accepted to EDM 2021

Py-Feat: Python Facial Expression Analysis Toolbox

Authors: ** Hyun Cheong, Eshin Jolly, Tiankang Xie, Sophie Byrne, Matthew Kenney, Luke J. Chang

Abstract: Studying facial expressions is a notoriously difficult endeavor. Recent advances in the field of affective computing have yielded impressive progress in automatically detecting facial expressions from pictures and videos. However, much of this work has yet to be widely disseminated in social science domains such as psychology. Current state of the art models require considerable domain expertise t… ▽ More Studying facial expressions is a notoriously difficult endeavor. Recent advances in the field of affective computing have yielded impressive progress in automatically detecting facial expressions from pictures and videos. However, much of this work has yet to be widely disseminated in social science domains such as psychology. Current state of the art models require considerable domain expertise that is not traditionally incorporated into social science training programs. Furthermore, there is a notable absence of user-friendly and open-source software that provides a comprehensive set of tools and functions that support facial expression research. In this paper, we introduce Py-Feat, an open-source Python toolbox that provides support for detecting, preprocessing, analyzing, and visualizing facial expression data. Py-Feat makes it easy for domain experts to disseminate and benchmark computer vision models and also for end users to quickly process, analyze, and visualize face expression data. We hope this platform will facilitate increased use of facial expression data in human behavior research. △ Less

Submitted 7 March, 2023; v1 submitted 8 April, 2021; originally announced April 2021.

Assessing YOLACT++ for real time and robust instance segmentation of medical instruments in endoscopic procedures

Authors: Juan Carlos Angeles Ceron, Leonardo Chang, Gilberto Ochoa-Ruiz, Sharib Ali

Abstract: Image-based tracking of laparoscopic instruments plays a fundamental role in computer and robotic-assisted surgeries by aiding surgeons and increasing patient safety. Computer vision contests, such as the Robust Medical Instrument Segmentation (ROBUST-MIS) Challenge, seek to encourage the development of robust models for such purposes, providing large, diverse, and annotated datasets. To date, mos… ▽ More Image-based tracking of laparoscopic instruments plays a fundamental role in computer and robotic-assisted surgeries by aiding surgeons and increasing patient safety. Computer vision contests, such as the Robust Medical Instrument Segmentation (ROBUST-MIS) Challenge, seek to encourage the development of robust models for such purposes, providing large, diverse, and annotated datasets. To date, most of the existing models for instance segmentation of medical instruments were based on two-stage detectors, which provide robust results but are nowhere near to the real-time (5 frames-per-second (fps)at most). However, in order for the method to be clinically applicable, real-time capability is utmost required along with high accuracy. In this paper, we propose the addition of attention mechanisms to the YOLACT architecture that allows real-time instance segmentation of instrument with improved accuracy on the ROBUST-MIS dataset. Our proposed approach achieves competitive performance compared to the winner ofthe 2019 ROBUST-MIS challenge in terms of robustness scores,obtaining 0.313 MI_DSC and 0.338 MI_NSD, while achieving real-time performance (37 fps) △ Less

Submitted 28 April, 2021; v1 submitted 29 March, 2021; originally announced March 2021.

Comments: Preprint under review for EMBC 2021 following IEEE guidelines