Showing 1–50 of 112 results for author: Cheung, W

Large Language Models as Partners in Student Essay Evaluation

Authors: Toru Ishida, Tongxi Liu, Hailong Wang, William K. Cheung

Abstract: As the importance of comprehensive evaluation in workshop courses increases, there is a growing demand for efficient and fair assessment methods that reduce the workload for faculty members. This paper presents an evaluation conducted with Large Language Models (LLMs) using actual student essays in three scenarios: 1) without providing guidance such as rubrics, 2) with pre-specified rubrics, and 3… ▽ More As the importance of comprehensive evaluation in workshop courses increases, there is a growing demand for efficient and fair assessment methods that reduce the workload for faculty members. This paper presents an evaluation conducted with Large Language Models (LLMs) using actual student essays in three scenarios: 1) without providing guidance such as rubrics, 2) with pre-specified rubrics, and 3) through pairwise comparison of essays. Quantitative analysis of the results revealed a strong correlation between LLM and faculty member assessments in the pairwise comparison scenario with pre-specified rubrics, although concerns about the quality and stability of evaluations remained. Therefore, we conducted a qualitative analysis of LLM assessment comments, showing that: 1) LLMs can match the assessment capabilities of faculty members, 2) variations in LLM assessments should be interpreted as diversity rather than confusion, and 3) assessments by humans and LLMs can differ and complement each other. In conclusion, this paper suggests that LLMs should not be seen merely as assistants to faculty members but as partners in evaluation committees and outlines directions for further research. △ Less

Submitted 28 May, 2024; originally announced May 2024.

Leveraging (Biased) Information: Multi-armed Bandits with Offline Data

Authors: Wang Chi Cheung, Lixing Lyu

Abstract: We leverage offline data to facilitate online learning in stochastic multi-armed bandits. The probability distributions that govern the offline data and the online rewards can be different. Without any non-trivial upper bound on their difference, we show that no non-anticipatory policy can outperform the UCB policy by (Auer et al. 2002), even in the presence of offline data. In complement, we prop… ▽ More We leverage offline data to facilitate online learning in stochastic multi-armed bandits. The probability distributions that govern the offline data and the online rewards can be different. Without any non-trivial upper bound on their difference, we show that no non-anticipatory policy can outperform the UCB policy by (Auer et al. 2002), even in the presence of offline data. In complement, we propose an online policy MIN-UCB, which outperforms UCB when a non-trivial upper bound is given. MIN-UCB adaptively chooses to utilize the offline data when they are deemed informative, and to ignore them otherwise. MIN-UCB is shown to be tight in terms of both instance independent and dependent regret bounds. Finally, we corroborate the theoretical results with numerical experiments. △ Less

Submitted 4 May, 2024; originally announced May 2024.

Comments: 24 pages, 5 figures. Accepted to ICML 2024

A Comprehensive Review of Latent Space Dynamics Identification Algorithms for Intrusive and Non-Intrusive Reduced-Order-Modeling

Authors: Christophe Bonneville, Xiaolong He, April Tran, Jun Sur Park, William Fries, Daniel A. Messenger, Siu Wun Cheung, Yeonjong Shin, David M. Bortz, Debojyoti Ghosh, Jiun-Shyan Chen, Jonathan Belof, Youngsoo Choi

Abstract: Numerical solvers of partial differential equations (PDEs) have been widely employed for simulating physical systems. However, the computational cost remains a major bottleneck in various scientific and engineering applications, which has motivated the development of reduced-order models (ROMs). Recently, machine-learning-based ROMs have gained significant popularity and are promising for addressi… ▽ More Numerical solvers of partial differential equations (PDEs) have been widely employed for simulating physical systems. However, the computational cost remains a major bottleneck in various scientific and engineering applications, which has motivated the development of reduced-order models (ROMs). Recently, machine-learning-based ROMs have gained significant popularity and are promising for addressing some limitations of traditional ROM methods, especially for advection dominated systems. In this chapter, we focus on a particular framework known as Latent Space Dynamics Identification (LaSDI), which transforms the high-fidelity data, governed by a PDE, to simpler and low-dimensional latent-space data, governed by ordinary differential equations (ODEs). These ODEs can be learned and subsequently interpolated to make ROM predictions. Each building block of LaSDI can be easily modulated depending on the application, which makes the LaSDI framework highly flexible. In particular, we present strategies to enforce the laws of thermodynamics into LaSDI models (tLaSDI), enhance robustness in the presence of noise through the weak form (WLaSDI), select high-fidelity training data efficiently through active learning (gLaSDI, GPLaSDI), and quantify the ROM prediction uncertainty through Gaussian processes (GPLaSDI). We demonstrate the performance of different LaSDI approaches on Burgers equation, a non-linear heat conduction problem, and a plasma physics problem, showing that LaSDI algorithms can achieve relative errors of less than a few percent and up to thousands of times speed-ups. △ Less

Submitted 15 March, 2024; originally announced March 2024.

DrFuse: Learning Disentangled Representation for Clinical Multi-Modal Fusion with Missing Modality and Modal Inconsistency

Authors: Wenfang Yao, Ke**g Yin, William K. Cheung, Jia Liu, **g Qin

Abstract: The combination of electronic health records (EHR) and medical images is crucial for clinicians in making diagnoses and forecasting prognosis. Strategically fusing these two data modalities has great potential to improve the accuracy of machine learning models in clinical prediction tasks. However, the asynchronous and complementary nature of EHR and medical images presents unique challenges. Miss… ▽ More The combination of electronic health records (EHR) and medical images is crucial for clinicians in making diagnoses and forecasting prognosis. Strategically fusing these two data modalities has great potential to improve the accuracy of machine learning models in clinical prediction tasks. However, the asynchronous and complementary nature of EHR and medical images presents unique challenges. Missing modalities due to clinical and administrative factors are inevitable in practice, and the significance of each data modality varies depending on the patient and the prediction target, resulting in inconsistent predictions and suboptimal model performance. To address these challenges, we propose DrFuse to achieve effective clinical multi-modal fusion. It tackles the missing modality issue by disentangling the features shared across modalities and those unique within each modality. Furthermore, we address the modal inconsistency issue via a disease-wise attention layer that produces the patient- and disease-wise weighting for each modality to make the final prediction. We validate the proposed method using real-world large-scale datasets, MIMIC-IV and MIMIC-CXR. Experimental results show that the proposed method significantly outperforms the state-of-the-art models. Our implementation is publicly available at https://github.com/dorothy-yao/drfuse. △ Less

Submitted 10 March, 2024; originally announced March 2024.

Comments: Accepted by AAAI-24

tLaSDI: Thermodynamics-informed latent space dynamics identification

Authors: Jun Sur Richard Park, Siu Wun Cheung, Youngsoo Choi, Yeonjong Shin

Abstract: We propose a latent space dynamics identification method, namely tLaSDI, that embeds the first and second principles of thermodynamics. The latent variables are learned through an autoencoder as a nonlinear dimension reduction model. The latent dynamics are constructed by a neural network-based model that precisely preserves certain structures for the thermodynamic laws through the GENERIC formali… ▽ More We propose a latent space dynamics identification method, namely tLaSDI, that embeds the first and second principles of thermodynamics. The latent variables are learned through an autoencoder as a nonlinear dimension reduction model. The latent dynamics are constructed by a neural network-based model that precisely preserves certain structures for the thermodynamic laws through the GENERIC formalism. An abstract error estimate is established, which provides a new loss formulation involving the Jacobian computation of autoencoder. The autoencoder and the latent dynamics are simultaneously trained to minimize the new loss. Computational examples demonstrate the effectiveness of tLaSDI, which exhibits robust generalization ability, even in extrapolation. In addition, an intriguing correlation is empirically observed between a quantity from tLaSDI in the latent space and the behaviors of the full-state solution. △ Less

Submitted 21 March, 2024; v1 submitted 9 March, 2024; originally announced March 2024.

Comments: 32 pages, 8 figures

Best Arm Identification with Resource Constraints

Authors: Zitian Li, Wang Chi Cheung

Abstract: Motivated by the cost heterogeneity in experimentation across different alternatives, we study the Best Arm Identification with Resource Constraints (BAIwRC) problem. The agent aims to identify the best arm under resource constraints, where resources are consumed for each arm pull. We make two novel contributions. We design and analyze the Successive Halving with Resource Rationing algorithm (SH-R… ▽ More Motivated by the cost heterogeneity in experimentation across different alternatives, we study the Best Arm Identification with Resource Constraints (BAIwRC) problem. The agent aims to identify the best arm under resource constraints, where resources are consumed for each arm pull. We make two novel contributions. We design and analyze the Successive Halving with Resource Rationing algorithm (SH-RR). The SH-RR achieves a near-optimal non-asymptotic rate of convergence in terms of the probability of successively identifying an optimal arm. Interestingly, we identify a difference in convergence rates between the cases of deterministic and stochastic resource consumption. △ Less

Submitted 29 February, 2024; originally announced February 2024.

Online Stochastic Allocation of Reusable Resources

Authors: Xilin Zhang, Wang Chi Cheung

Abstract: We study a multi-objective model on the allocation of reusable resources under model uncertainty. Heterogeneous customers arrive sequentially according to a latent stochastic process, request for certain amounts of resources, and occupy them for random durations of time. The decision maker's goal is to simultaneously maximize multiple types of rewards generated by the customers, while satisfying t… ▽ More We study a multi-objective model on the allocation of reusable resources under model uncertainty. Heterogeneous customers arrive sequentially according to a latent stochastic process, request for certain amounts of resources, and occupy them for random durations of time. The decision maker's goal is to simultaneously maximize multiple types of rewards generated by the customers, while satisfying the resource capacity constraints in each time step. We develop models and algorithms for deciding on the allocation actions. We show that when the usage duration is relatively small compared with the length of the planning horizon, our policy achieves $1-O(ε)$ fraction of the optimal expected rewards, where $ε$ decays to zero at a near optimal rate as the resource capacities grow. △ Less

Submitted 1 August, 2023; originally announced August 2023.

A 3D deep learning classifier and its explainability when assessing coronary artery disease

Authors: Wing Keung Cheung, Jeremy Kalindjian, Robert Bell, Arjun Nair, Leon J. Menezes, Riyaz Patel, Simon Wan, Kacy Chou, Jiahang Chen, Ryo Torii, Rhodri H. Davies, James C. Moon, Daniel C. Alexander, Joseph Jacob

Abstract: Early detection and diagnosis of coronary artery disease (CAD) could save lives and reduce healthcare costs. In this study, we propose a 3D Resnet-50 deep learning model to directly classify normal subjects and CAD patients on computed tomography coronary angiography images. Our proposed method outperforms a 2D Resnet-50 model by 23.65%. Explainability is also provided by using a Grad-GAM. Further… ▽ More Early detection and diagnosis of coronary artery disease (CAD) could save lives and reduce healthcare costs. In this study, we propose a 3D Resnet-50 deep learning model to directly classify normal subjects and CAD patients on computed tomography coronary angiography images. Our proposed method outperforms a 2D Resnet-50 model by 23.65%. Explainability is also provided by using a Grad-GAM. Furthermore, we link the 3D CAD classification to a 2D two-class semantic segmentation for improved explainability and accurate abnormality localisation. △ Less

Submitted 29 July, 2023; originally announced August 2023.

A data-centric deep learning approach to airway segmentation

Authors: Wing Keung Cheung, Ashkan Pakzad, Nesrin Mogulkoc, Sarah Needleman, Bojidar Rangelov, Eyjolfur Gudmundsson, An Zhao, Mariam Abbas, Davina McLaverty, Dimitrios Asimakopoulos, Robert Chapman, Recep Savas, Sam M Janes, Yipeng Hu, Daniel C. Alexander, John R Hurst, Joseph Jacob

Abstract: The morphology and distribution of airway tree abnormalities enables diagnosis and disease characterisation across a variety of chronic respiratory conditions. In this regard, airway segmentation plays a critical role in the production of the outline of the entire airway tree to enable estimation of disease extent and severity. In this study, we propose a data-centric deep learning technique to se… ▽ More The morphology and distribution of airway tree abnormalities enables diagnosis and disease characterisation across a variety of chronic respiratory conditions. In this regard, airway segmentation plays a critical role in the production of the outline of the entire airway tree to enable estimation of disease extent and severity. In this study, we propose a data-centric deep learning technique to segment the airway tree. The proposed technique utilises interpolation and image split to improve data usefulness and quality. Then, an ensemble learning strategy is implemented to aggregate the segmented airway trees at different scales. In terms of segmentation performance (dice similarity coefficient), our method outperforms the baseline model by 2.5% on average when a combined loss is used. Further, our proposed technique has a low GPU usage and high flexibility enabling it to be deployed on any 2D deep learning model. △ Less

Submitted 29 July, 2023; originally announced August 2023.

Towards Fairness in Personalized Ads Using Impression Variance Aware Reinforcement Learning

Authors: Aditya Srinivas Timmaraju, Mehdi Mashayekhi, Mingliang Chen, Qi Zeng, Quintin Fettes, Wesley Cheung, Yihan Xiao, Manojkumar Rangasamy Kannadasan, Pushkar Tripathi, Sean Gahagan, Miranda Bogen, Rob Roudani

Abstract: Variances in ad impression outcomes across demographic groups are increasingly considered to be potentially indicative of algorithmic bias in personalized ads systems. While there are many definitions of fairness that could be applicable in the context of personalized systems, we present a framework which we call the Variance Reduction System (VRS) for achieving more equitable outcomes in Meta's a… ▽ More Variances in ad impression outcomes across demographic groups are increasingly considered to be potentially indicative of algorithmic bias in personalized ads systems. While there are many definitions of fairness that could be applicable in the context of personalized systems, we present a framework which we call the Variance Reduction System (VRS) for achieving more equitable outcomes in Meta's ads systems. VRS seeks to achieve a distribution of impressions with respect to selected protected class (PC) attributes that more closely aligns the demographics of an ad's eligible audience (a function of advertiser targeting criteria) with the audience who sees that ad, in a privacy-preserving manner. We first define metrics to quantify fairness gaps in terms of ad impression variances with respect to PC attributes including gender and estimated race. We then present the VRS for re-ranking ads in an impression variance-aware manner. We evaluate VRS via extensive simulations over different parameter choices and study the effect of the VRS on the chosen fairness metric. We finally present online A/B testing results from applying VRS to Meta's ads systems, concluding with a discussion of future work. We have deployed the VRS to all users in the US for housing ads, resulting in significant improvement in our fairness metric. VRS is the first large-scale deployed framework for pursuing fairness for multiple PC attributes in online advertising. △ Less

Submitted 8 June, 2023; v1 submitted 5 June, 2023; originally announced June 2023.

Comments: 11 pages, 7 figure, KDD 2023

Data-scarce surrogate modeling of shock-induced pore collapse process

Authors: Siu Wun Cheung, Youngsoo Choi, H. Keo Springer, Teeratorn Kadeethum

Abstract: Understanding the mechanisms of shock-induced pore collapse is of great interest in various disciplines in sciences and engineering, including materials science, biological sciences, and geophysics. However, numerical modeling of the complex pore collapse processes can be costly. To this end, a strong need exists to develop surrogate models for generating economic predictions of pore collapse proc… ▽ More Understanding the mechanisms of shock-induced pore collapse is of great interest in various disciplines in sciences and engineering, including materials science, biological sciences, and geophysics. However, numerical modeling of the complex pore collapse processes can be costly. To this end, a strong need exists to develop surrogate models for generating economic predictions of pore collapse processes. In this work, we study the use of a data-driven reduced order model, namely dynamic mode decomposition, and a deep generative model, namely conditional generative adversarial networks, to resemble the numerical simulations of the pore collapse process at representative training shock pressures. Since the simulations are expensive, the training data are scarce, which makes training an accurate surrogate model challenging. To overcome the difficulties posed by the complex physics phenomena, we make several crucial treatments to the plain original form of the methods to increase the capability of approximating and predicting the dynamics. In particular, physics information is used as indicators or conditional inputs to guide the prediction. In realizing these methods, the training of each dynamic mode composition model takes only around 30 seconds on CPU. In contrast, training a generative adversarial network model takes 8 hours on GPU. Moreover, using dynamic mode decomposition, the final-time relative error is around 0.3% in the reproductive cases. We also demonstrate the predictive power of the methods at unseen testing shock pressures, where the error ranges from 1.3% to 5% in the interpolatory cases and 8% to 9% in extrapolatory cases. △ Less

Submitted 2 June, 2023; v1 submitted 31 May, 2023; originally announced June 2023.

Online Resource Allocation: Bandits feedback and Advice on Time-varying Demands

Authors: Lixing Lyu, Wang Chi Cheung

Abstract: We consider a general online resource allocation model with bandit feedback and time-varying demands. While online resource allocation has been well studied in the literature, most existing works make the strong assumption that the demand arrival process is stationary. In practical applications, such as online advertisement and revenue management, however, this process may be exogenous and non-sta… ▽ More We consider a general online resource allocation model with bandit feedback and time-varying demands. While online resource allocation has been well studied in the literature, most existing works make the strong assumption that the demand arrival process is stationary. In practical applications, such as online advertisement and revenue management, however, this process may be exogenous and non-stationary, like the constantly changing internet traffic. Motivated by the recent Online Algorithms with Advice framework [Mitazenmacher and Vassilvitskii, \emph{Commun. ACM} 2022], we explore how online advice can inform policy design. We establish an impossibility result that any algorithm perform poorly in terms of regret without any advice in our setting. In contrast, we design an robust online algorithm that leverages the online predictions on the total demand volumes. Empowered with online advice, our proposed algorithm is shown to have both theoretical performance and promising numerical results compared with other algorithms in literature. We also provide two explicit examples for the time-varying demand scenarios and derive corresponding theoretical performance guarantees. Finally, we adapt our model to a network revenue management problem, and numerically demonstrate that our algorithm can still performs competitively compared to existing baselines. △ Less

Submitted 12 June, 2023; v1 submitted 8 February, 2023; originally announced February 2023.

Comments: 74 pages. A preliminary short version entitled "Non-Stationary Bandits with Knapsack Problems with Advice" is accepted to ICML 2023

WL-Align: Weisfeiler-Lehman Relabeling for Aligning Users across Networks via Regularized Representation Learning

Authors: Li Liu, Penggang Chen, Xin Li, William K. Cheung, Youmin Zhang, Qun Liu, Guoyin Wang

Abstract: Aligning users across networks using graph representation learning has been found effective where the alignment is accomplished in a low-dimensional embedding space. Yet, achieving highly precise alignment is still challenging, especially when nodes with long-range connectivity to the labeled anchors are encountered. To alleviate this limitation, we purposefully designed WL-Align which adopts a re… ▽ More Aligning users across networks using graph representation learning has been found effective where the alignment is accomplished in a low-dimensional embedding space. Yet, achieving highly precise alignment is still challenging, especially when nodes with long-range connectivity to the labeled anchors are encountered. To alleviate this limitation, we purposefully designed WL-Align which adopts a regularized representation learning framework to learn distinctive node representations. It extends the Weisfeiler-Lehman Isormorphism Test and learns the alignment in alternating phases of "across-network Weisfeiler-Lehman relabeling" and "proximity-preserving representation learning". The across-network Weisfeiler-Lehman relabeling is achieved through iterating the anchor-based label propagation and a similarity-based hashing to exploit the known anchors' connectivity to different nodes in an efficient and robust manner. The representation learning module preserves the second-order proximity within individual networks and is regularized by the across-network Weisfeiler-Lehman hash labels. Extensive experiments on real-world and synthetic datasets have demonstrated that our proposed WL-Align outperforms the state-of-the-art methods, achieving significant performance improvements in the "exact matching" scenario. Data and code of WL-Align are available at https://github.com/ChenPengGang/WLAlignCode. △ Less

Submitted 29 December, 2022; originally announced December 2022.

The photometric observation of the quasi-simultaneous mutual eclipse and occultation between Europa and Ganymede on 22 August 2021

Authors: Chu Wing So, Godfrey Ho Ching Luk, Giann On Ching Chung, Po Kin Leung, Kenneith Ho Keung Hui, Jack Lap Chung Cheung, Ka Wo Chan, Edwin Lok Hei Yuen, Lawrence Wai Kwan Lee, Patrick Kai Ip Lau, Gloria Wing Shan Cheung, Prince Chun Lam Chan, Jason Chun Shing Pun

Abstract: Mutual events (MEs) are eclipses and occultations among planetary natural satellites. Most of the time, eclipses and occultations occur separately. However, the same satellite pair will exhibit an eclipse and an occultation quasi-simultaneously under particular orbital configurations. This kind of rare event is termed as a quasi-simultaneous mutual event (QSME). During the 2021 campaign of mutual… ▽ More Mutual events (MEs) are eclipses and occultations among planetary natural satellites. Most of the time, eclipses and occultations occur separately. However, the same satellite pair will exhibit an eclipse and an occultation quasi-simultaneously under particular orbital configurations. This kind of rare event is termed as a quasi-simultaneous mutual event (QSME). During the 2021 campaign of mutual events of jovian satellites, we observed a QSME between Europa and Ganymede. The present study aims to describe and study the event in detail. We observed the QSME with a CCD camera attached to a 300-mm telescope at the Hong Kong Space Museum Sai Kung iObservatory. We obtained the combined flux of Europa and Ganymede from aperture photometry. A geometric model was developed to explain the light curve observed. Our results are compared with theoretical predictions (O-C). We found that our simple geometric model can explain the QSME fairly accurately, and the QSME light curve is a superposition of the light curves of an eclipse and an occultation. Notably, the observed flux drops are within 2.6% of the theoretical predictions. The size of the event central time O-Cs ranges from -14.4 to 43.2 s. Both O-Cs of flux drop and timing are comparable to other studies adopting more complicated models. Given the event rarity, model simplicity and accuracy, we encourage more observations and analysis on QSMEs to improve Solar System ephemerides. △ Less

Submitted 10 December, 2022; originally announced December 2022.

Comments: 23 pages, 5 appendixes, 16 figures, 7 tables

Journal ref: Icarus, 392, 2023, 115348

Online Resource Allocation for Reusable Resources

Authors: Xilin Zhang, Wang Chi Cheung

Abstract: We study a general model on reusable resource allocation under model uncertainty. A heterogeneous population of customers arrive at the decision maker's (DM's) platform sequentially. Upon observing a customer's type, the DM selects an allocation decision, which leads to rewards earned and resources occupied. Each resource unit is occupied for a random duration, and the unit is available for anothe… ▽ More We study a general model on reusable resource allocation under model uncertainty. A heterogeneous population of customers arrive at the decision maker's (DM's) platform sequentially. Upon observing a customer's type, the DM selects an allocation decision, which leads to rewards earned and resources occupied. Each resource unit is occupied for a random duration, and the unit is available for another allocation after the usage duration. Our model captures numerous applications involving admission control and assortment planning. The DM aims to simultaneously maximize multiple types of rewards, while satisfying the resource constraints and being uncertain about the customers' arrival process. We develop a near-optimal algorithm that achieves $(1-ε)$ fraction of the optimal expected rewards, where the error parameter $ε$ decays to zero as the resource capacity units and the length of the horizon grow. The algorithm iteratively applies the Multiplicative Weight Update algorithm in a novel manner, which balances the trade-off among the amounts of rewards earned, resources occupied and usage durations. △ Less

Submitted 6 December, 2022; originally announced December 2022.

ILSGAN: Independent Layer Synthesis for Unsupervised Foreground-Background Segmentation

Authors: Qiran Zou, Yu Yang, Wing Yin Cheung, Chang Liu, Xiangyang Ji

Abstract: Unsupervised foreground-background segmentation aims at extracting salient objects from cluttered backgrounds, where Generative Adversarial Network (GAN) approaches, especially layered GANs, show great promise. However, without human annotations, they are typically prone to produce foreground and background layers with non-negligible semantic and visual confusion, dubbed "information leakage", res… ▽ More Unsupervised foreground-background segmentation aims at extracting salient objects from cluttered backgrounds, where Generative Adversarial Network (GAN) approaches, especially layered GANs, show great promise. However, without human annotations, they are typically prone to produce foreground and background layers with non-negligible semantic and visual confusion, dubbed "information leakage", resulting in notable degeneration of the generated segmentation mask. To alleviate this issue, we propose a simple-yet-effective explicit layer independence modeling approach, termed Independent Layer Synthesis GAN (ILSGAN), pursuing independent foreground-background layer generation by encouraging their discrepancy. Specifically, it targets minimizing the mutual information between visible and invisible regions of the foreground and background to spur interlayer independence. Through in-depth theoretical and experimental analyses, we justify that explicit layer independence modeling is critical to suppressing information leakage and contributes to impressive segmentation performance gains. Also, our ILSGAN achieves strong state-of-the-art generation quality and segmentation performance on complex real-world data. Code is available at: https://github.com/qrzou/ILSGAN △ Less

Submitted 7 October, 2023; v1 submitted 25 November, 2022; originally announced November 2022.

Comments: Accepted by AAAI 2023 (Oral). Code: https://github.com/qrzou/ILSGAN

Local Manifold Augmentation for Multiview Semantic Consistency

Authors: Yu Yang, Wing Yin Cheung, Chang Liu, Xiangyang Ji

Abstract: Multiview self-supervised representation learning roots in exploring semantic consistency across data of complex intra-class variation. Such variation is not directly accessible and therefore simulated by data augmentations. However, commonly adopted augmentations are handcrafted and limited to simple geometrical and color changes, which are unable to cover the abundant intra-class variation. In t… ▽ More Multiview self-supervised representation learning roots in exploring semantic consistency across data of complex intra-class variation. Such variation is not directly accessible and therefore simulated by data augmentations. However, commonly adopted augmentations are handcrafted and limited to simple geometrical and color changes, which are unable to cover the abundant intra-class variation. In this paper, we propose to extract the underlying data variation from datasets and construct a novel augmentation operator, named local manifold augmentation (LMA). LMA is achieved by training an instance-conditioned generator to fit the distribution on the local manifold of data and sampling multiview data using it. LMA shows the ability to create an infinite number of data views, preserve semantics, and simulate complicated variations in object pose, viewpoint, lighting condition, background etc. Experiments show that with LMA integrated, self-supervised learning methods such as MoCov2 and SimSiam gain consistent improvement on prevalent benchmarks including CIFAR10, CIFAR100, STL10, ImageNet100, and ImageNet. Furthermore, LMA leads to representations that obtain more significant invariance to the viewpoint, object pose, and illumination changes and stronger robustness to various real distribution shifts reflected by ImageNet-V2, ImageNet-R, ImageNet Sketch etc. △ Less

Submitted 4 November, 2022; originally announced November 2022.

Airway measurement by refinement of synthetic images improves mortality prediction in idiopathic pulmonary fibrosis

Authors: Ashkan Pakzad, Mou-Cheng Xu, Wing Keung Cheung, Marie Vermant, Tinne Goos, Laurens J De Sadeleer, Stijn E Verleden, Wim A Wuyts, John R Hurst, Joseph Jacob

Abstract: Several chronic lung diseases, like idiopathic pulmonary fibrosis (IPF) are characterised by abnormal dilatation of the airways. Quantification of airway features on computed tomography (CT) can help characterise disease progression. Physics based airway measurement algorithms have been developed, but have met with limited success in part due to the sheer diversity of airway morphology seen in cli… ▽ More Several chronic lung diseases, like idiopathic pulmonary fibrosis (IPF) are characterised by abnormal dilatation of the airways. Quantification of airway features on computed tomography (CT) can help characterise disease progression. Physics based airway measurement algorithms have been developed, but have met with limited success in part due to the sheer diversity of airway morphology seen in clinical practice. Supervised learning methods are also not feasible due to the high cost of obtaining precise airway annotations. We propose synthesising airways by style transfer using perceptual losses to train our model, Airway Transfer Network (ATN). We compare our ATN model with a state-of-the-art GAN-based network (simGAN) using a) qualitative assessment; b) assessment of the ability of ATN and simGAN based CT airway metrics to predict mortality in a population of 113 patients with IPF. ATN was shown to be quicker and easier to train than simGAN. ATN-based airway measurements were also found to be consistently stronger predictors of mortality than simGAN-derived airway metrics on IPF CTs. Airway synthesis by a transformation network that refines synthetic data using perceptual losses is a realistic alternative to GAN-based methods for clinical CT analyses of idiopathic pulmonary fibrosis. Our source code can be found at https://github.com/ashkanpakzad/ATN that is compatible with the existing open-source airway analysis framework, AirQuant. △ Less

Submitted 30 August, 2022; originally announced August 2022.

Comments: 11 Pages, 4 figures. Source code available: https://github.com/ashkanpakzad/ATN. Initial submission version, to be published in MICCAI Workshop on Deep Generative Models 2022

Attributed Abnormality Graph Embedding for Clinically Accurate X-Ray Report Generation

Authors: Sixing Yan, William K. Cheung, Keith Chiu, Terence M. Tong, Charles K. Cheung, Simon See

Abstract: Automatic generation of medical reports from X-ray images can assist radiologists to perform the time-consuming and yet important reporting task. Yet, achieving clinically accurate generated reports remains challenging. Modeling the underlying abnormalities using the knowledge graph approach has been found promising in enhancing the clinical accuracy. In this paper, we introduce a novel fined-grai… ▽ More Automatic generation of medical reports from X-ray images can assist radiologists to perform the time-consuming and yet important reporting task. Yet, achieving clinically accurate generated reports remains challenging. Modeling the underlying abnormalities using the knowledge graph approach has been found promising in enhancing the clinical accuracy. In this paper, we introduce a novel fined-grained knowledge graph structure called an attributed abnormality graph (ATAG). The ATAG consists of interconnected abnormality nodes and attribute nodes, allowing it to better capture the abnormality details. In contrast to the existing methods where the abnormality graph was constructed manually, we propose a methodology to automatically construct the fine-grained graph structure based on annotations, medical reports in X-ray datasets, and the RadLex radiology lexicon. We then learn the ATAG embedding using a deep model with an encoder-decoder architecture for the report generation. In particular, graph attention networks are explored to encode the relationships among the abnormalities and their attributes. A gating mechanism is adopted and integrated with various decoders for the generation. We carry out extensive experiments based on the benchmark datasets, and show that the proposed ATAG-based deep model outperforms the SOTA methods by a large margin and can improve the clinical accuracy of the generated reports. △ Less

Submitted 5 July, 2022; v1 submitted 4 July, 2022; originally announced July 2022.

Comments: 14 pages, 7 figures

ICME 2022 Few-shot LOGO detection top 9 solution

Authors: Ka Ho Tong, Ka Wai Cheung, Xiaochuan Yu

Abstract: ICME-2022 few-shot logo detection competition is held in May, 2022. Participants are required to develop a single model to detect logos by handling tiny logo instances, similar brands, and adversarial images at the same time, with limited annotations. Our team achieved rank 16 and 11 in the first and second round of the competition respectively, with a final rank of 9th. This technical report summ… ▽ More ICME-2022 few-shot logo detection competition is held in May, 2022. Participants are required to develop a single model to detect logos by handling tiny logo instances, similar brands, and adversarial images at the same time, with limited annotations. Our team achieved rank 16 and 11 in the first and second round of the competition respectively, with a final rank of 9th. This technical report summarized our major techniques used in this competitions, and potential improvement. △ Less

Submitted 22 June, 2022; originally announced June 2022.

Constraint Energy Minimizing Generalized Multiscale Finite Element Method for multi-continuum Richards equations

Authors: Tina Mai, Siu Wun Cheung, Jun Sur Richard Park

Abstract: In fluid flow simulation, the multi-continuum model is a useful strategy. When the heterogeneity and contrast of coefficients are high, the system becomes multiscale, and some kinds of reduced-order methods are demanded. Combining these techniques with nonlinearity, we will consider in this paper a dual-continuum model which is generalized as a multi-continuum model for a coupled system of nonline… ▽ More In fluid flow simulation, the multi-continuum model is a useful strategy. When the heterogeneity and contrast of coefficients are high, the system becomes multiscale, and some kinds of reduced-order methods are demanded. Combining these techniques with nonlinearity, we will consider in this paper a dual-continuum model which is generalized as a multi-continuum model for a coupled system of nonlinear Richards equations as unsaturated flows, in complex heterogeneous fractured porous media; and we will solve it by a novel multiscale approach utilizing the constraint energy minimizing generalized multiscale finite element method (CEM-GMsFEM). In particular, such a nonlinear system will be discretized in time and then linearized by Picard iteration (whose global convergence is proved theoretically). Subsequently, we tackle the resulting linearized equations by the CEM-GMsFEM and obtain proper offline multiscale basis functions to span the multiscale space (which contains the pressure solution). More specifically, we first introduce two new sources of samples, and the GMsFEM is used over each coarse block to build local auxiliary multiscale basis functions via solving local spectral problems, that are crucial for detecting high-contrast channels. Second, per oversampled coarse region, local multiscale basis functions are created through the CEM as constrainedly minimizing an energy functional. Various numerical tests for our approach reveal that the error converges with the coarse-grid size alone and that only a few oversampling layers, as well as basis functions, are needed. △ Less

Submitted 23 May, 2022; originally announced May 2022.

Comments: 22 pages, 7 figures, 4 tables, submitted to Journal of Computational Physics, fixed some typos and notation

MSC Class: 65M60; 65M12

S-OPT: A Points Selection Algorithm for Hyper-Reduction in Reduced Order Models

Authors: Jessica T. Lauzon, Siu Wun Cheung, Yeonjong Shin, Youngsoo Choi, Dylan Matthew Copeland, Kevin Huynh

Abstract: While projection-based reduced order models can reduce the dimension of full order solutions, the resulting reduced models may still contain terms that scale with the full order dimension. Hyper-reduction techniques are sampling-based methods that further reduce this computational complexity by approximating such terms with a much smaller dimension. The goal of this work is to introduce a points s… ▽ More While projection-based reduced order models can reduce the dimension of full order solutions, the resulting reduced models may still contain terms that scale with the full order dimension. Hyper-reduction techniques are sampling-based methods that further reduce this computational complexity by approximating such terms with a much smaller dimension. The goal of this work is to introduce a points selection algorithm developed by Shin and Xiu [SIAM J. Sci. Comput., 38 (2016), pp. A385--A411], as a hyper-reduction method. The selection algorithm is originally proposed as a stochastic collocation method for uncertainty quantification. Since the algorithm aims at maximizing a quantity S that measures both the column orthogonality and the determinant, we refer to the algorithm as S-OPT. Numerical examples are provided to demonstrate the performance of S-OPT and to compare its performance with an over-sampled Discrete Empirical Interpolation (DEIM) algorithm. We found that using the S-OPT algorithm is shown to predict the full order solutions with higher accuracy for a given number of indices. △ Less

Submitted 29 March, 2022; originally announced March 2022.

Comments: 26 pages, 15 figures, submitted to SIAM Journal of Scientific Computing

MSC Class: 37M99; 65M99; 76D05; 67Q05

Local Lagrangian reduced-order modeling for Rayleigh-Taylor instability by solution manifold decomposition

Authors: Siu Wun Cheung, Youngsoo Choi, Dylan Matthew Copeland, Kevin Huynh

Abstract: Rayleigh-Taylor instability is a classical hydrodynamic instability of great interest in various disciplines of science and engineering, including astrophyics, atmospheric sciences and climate, geophysics, and fusion energy. Analytical methods cannot be applied to explain the long-time behavior of Rayleigh-Taylor instability, and therefore numerical simulation of the full problem is required. Howe… ▽ More Rayleigh-Taylor instability is a classical hydrodynamic instability of great interest in various disciplines of science and engineering, including astrophyics, atmospheric sciences and climate, geophysics, and fusion energy. Analytical methods cannot be applied to explain the long-time behavior of Rayleigh-Taylor instability, and therefore numerical simulation of the full problem is required. However, in order to capture the growth of amplitude of perturbations accurately, both the spatial and temporal discretization need to be extremely fine for traditional numerical methods, and the long-time simulation may become prohibitively expensive. In this paper, we propose efficient reduced order model techniques to accelerate the simulation of Rayleigh-Taylor instability in compressible gas dynamics. We introduce a general framework for decomposing the solution manifold to construct the temporal domain partition and temporally-local reduced order model construction with varying Atwood number. We propose two practical approaches in this framework, namely decomposition by using physical time and penetration distance respectively. Numerical results are presented to examine the performance of the proposed approaches. △ Less

Submitted 18 January, 2022; originally announced January 2022.

Comments: arXiv admin note: substantial text overlap with arXiv:2104.11404

Towards Improving Embedding Based Models of Social Network Alignment via Pseudo Anchors

Authors: Zihan Yan, Li Liu, Xin Li, William K. Cheung, Youmin Zhang, Qun Liu, Guoyin Wang

Abstract: Social network alignment aims at aligning person identities across social networks. Embedding based models have been shown effective for the alignment where the structural proximity preserving objective is typically adopted for the model training. With the observation that ``overly-close'' user embeddings are unavoidable for such models causing alignment inaccuracy, we propose a novel learning fra… ▽ More Social network alignment aims at aligning person identities across social networks. Embedding based models have been shown effective for the alignment where the structural proximity preserving objective is typically adopted for the model training. With the observation that ``overly-close'' user embeddings are unavoidable for such models causing alignment inaccuracy, we propose a novel learning framework which tries to enforce the resulting embeddings to be more widely apart among the users via the introduction of carefully implanted pseudo anchors. We further proposed a meta-learning algorithm to guide the updating of the pseudo anchor embeddings during the learning process. The proposed intervention via the use of pseudo anchors and meta-learning allows the learning framework to be applicable to a wide spectrum of network alignment methods. We have incorporated the proposed learning framework into several state-of-the-art models. Our experimental results demonstrate its efficacy where the methods with the pseudo anchors implanted can outperform their counterparts without pseudo anchors by a fairly large margin, especially when there only exist very few labeled anchors. △ Less

Submitted 22 November, 2021; originally announced November 2021.

Journal ref: IEEE Transactions on Knowledge and Data Engineering 2021

Evaluation of automated airway morphological quantification for assessing fibrosing lung disease

Authors: Ashkan Pakzad, Wing Keung Cheung, Kin Quan, Nesrin Mogulkoc, Coline H. M. Van Moorsel, Brian J. Bartholmai, Hendrik W. Van Es, Alper Ezircan, Frouke Van Beek, Marcel Veltkamp, Ronald Karwoski, Tobias Peikert, Ryan D. Clay, Finbar Foley, Cassandra Braun, Recep Savas, Carole Sudre, Tom Doel, Daniel C. Alexander, Peter Wijeratne, David Hawkes, Yipeng Hu, John R Hurst, Joseph Jacob

Abstract: Abnormal airway dilatation, termed traction bronchiectasis, is a typical feature of idiopathic pulmonary fibrosis (IPF). Volumetric computed tomography (CT) imaging captures the loss of normal airway tapering in IPF. We postulated that automated quantification of airway abnormalities could provide estimates of IPF disease extent and severity. We propose AirQuant, an automated computational pipelin… ▽ More Abnormal airway dilatation, termed traction bronchiectasis, is a typical feature of idiopathic pulmonary fibrosis (IPF). Volumetric computed tomography (CT) imaging captures the loss of normal airway tapering in IPF. We postulated that automated quantification of airway abnormalities could provide estimates of IPF disease extent and severity. We propose AirQuant, an automated computational pipeline that systematically parcellates the airway tree into its lobes and generational branches from a deep learning based airway segmentation, deriving airway structural measures from chest CT. Importantly, AirQuant prevents the occurrence of spurious airway branches by thick wave propagation and removes loops in the airway-tree by graph search, overcoming limitations of existing airway skeletonisation algorithms. Tapering between airway segments (intertapering) and airway tortuosity computed by AirQuant were compared between 14 healthy participants and 14 IPF patients. Airway intertapering was significantly reduced in IPF patients, and airway tortuosity was significantly increased when compared to healthy controls. Differences were most marked in the lower lobes, conforming to the typical distribution of IPF-related damage. AirQuant is an open-source pipeline that avoids limitations of existing airway quantification algorithms and has clinical interpretability. Automated airway measurements may have potential as novel imaging biomarkers of IPF severity and disease extent. △ Less

Submitted 19 November, 2021; originally announced November 2021.

Comments: 14 pages, 8 Figures, for associated source code, see https://github.com/ashkanpakzad/AirQuant

Achieving the Pareto Frontier of Regret Minimization and Best Arm Identification in Multi-Armed Bandits

Authors: Zixin Zhong, Wang Chi Cheung, Vincent Y. F. Tan

Abstract: We study the Pareto frontier of two archetypal objectives in multi-armed bandits, namely, regret minimization (RM) and best arm identification (BAI) with a fixed horizon. It is folklore that the balance between exploitation and exploration is crucial for both RM and BAI, but exploration is more critical in achieving the optimal performance for the latter objective. To this end, we design and analy… ▽ More We study the Pareto frontier of two archetypal objectives in multi-armed bandits, namely, regret minimization (RM) and best arm identification (BAI) with a fixed horizon. It is folklore that the balance between exploitation and exploration is crucial for both RM and BAI, but exploration is more critical in achieving the optimal performance for the latter objective. To this end, we design and analyze the BoBW-lil'UCB$(γ)$ algorithm. Complementarily, by establishing lower bounds on the regret achievable by any algorithm with a given BAI failure probability, we show that (i) no algorithm can simultaneously perform optimally for both the RM and BAI objectives, and (ii) BoBW-lil'UCB$(γ)$ achieves order-wise optimal performance for RM or BAI under different values of $γ$. Our work elucidates the trade-off more precisely by showing how the constants in previous works depend on certain hardness parameters. Finally, we show that BoBW-lil'UCB outperforms a close competitor UCB$_α$ (Degenne et al., 2019) in terms of the time complexity and the regret on diverse datasets such as MovieLens and Published Kinase Inhibitor Set. △ Less

Submitted 9 June, 2023; v1 submitted 16 October, 2021; originally announced October 2021.

Comments: 43 pages, 10 figures

AdjointBackMapV2: Precise Reconstruction of Arbitrary CNN Unit's Activation via Adjoint Operators

Authors: Qing Wan, Siu Wun Cheung, Yoonsuck Choe

Abstract: Adjoint operators have been found to be effective in the exploration of CNN's inner workings [1]. However, the previous no-bias assumption restricted its generalization. We overcome the restriction via embedding input images into an extended normed space that includes bias in all CNN layers as part of the extended space and propose an adjoint-operator-based algorithm that maps high-level weights b… ▽ More Adjoint operators have been found to be effective in the exploration of CNN's inner workings [1]. However, the previous no-bias assumption restricted its generalization. We overcome the restriction via embedding input images into an extended normed space that includes bias in all CNN layers as part of the extended space and propose an adjoint-operator-based algorithm that maps high-level weights back to the extended input space for reconstructing an effective hypersurface. Such hypersurface can be computed for an arbitrary unit in the CNN, and we prove that this reconstructed hypersurface, when multiplied by the original input (through an inner product), will precisely replicate the output value of each unit. We show experimental results based on the CIFAR-10 and CIFAR-100 data sets where the proposed approach achieves near 0 activation value reconstruction error. △ Less

Submitted 9 November, 2023; v1 submitted 4 October, 2021; originally announced October 2021.

Comments: This is a preprint prior to peer-review. For the revised/finalized version, please see https://doi.org/10.1016/j.neunet.2023.11.009

Opportunistic Multi-Modal User Authentication for Health-Tracking IoT Wearables

Authors: Alexa Muratyan, William Cheung, Sayanton V. Dibbo, Sudip Vhaduri

Abstract: With the advancement of technologies, market wearables are becoming increasingly popular with a range of services, including providing access to bank accounts, accessing cars, monitoring patients remotely, among several others. However, often these wearables collect various sensitive personal information of a user with no to limited authentication, e.g., knowledge-based external authentication tec… ▽ More With the advancement of technologies, market wearables are becoming increasingly popular with a range of services, including providing access to bank accounts, accessing cars, monitoring patients remotely, among several others. However, often these wearables collect various sensitive personal information of a user with no to limited authentication, e.g., knowledge-based external authentication techniques, such as PINs. While most of these external authentication techniques suffer from multiple limitations, including recall burden, human errors, or biases, researchers have started using various physiological and behavioral data, such as gait and heart rate, collected by the wearables to authenticate a wearable user implicitly with a limited accuracy due to sensing and computing constraints of wearables. In this work, we explore the usefulness of blood oxygen saturation SpO2 values collected from the Oximeter device to distinguish a user from others. From a cohort of 25 subjects, we find that 92% of the cases SpO2 can distinguish pairs of users. From detailed modeling and performance analysis, we observe that while SpO2 alone can obtain an average accuracy of 0.69 and F1 score of 0.69, the addition of heart rate (HR) can improve the average identification accuracy by 15% and F1 score by 13%. These results show promise in using SpO2 along with other biometrics to develop implicit continuous authentications for wearables. △ Less

Submitted 9 October, 2021; v1 submitted 28 September, 2021; originally announced September 2021.

Comments: Accepted to publish in the 5th EAI International Conference on Safety and Security in Internet of Things (SaSeIoT 2021). (The previous version of this paper had an incorrect title due to a mistake. Aside from the title, all contents of this manuscript remain the same and match the version accepted at SaSeIoT 2021)

A Dynamic Programming Algorithm for Finding an Optimal Sequence of Informative Measurements

Authors: Peter N. Loxley, Ka-Wai Cheung

Abstract: An informative measurement is the most efficient way to gain information about an unknown state. We present a first-principles derivation of a general-purpose dynamic programming algorithm that returns an optimal sequence of informative measurements by sequentially maximizing the entropy of possible measurement outcomes. This algorithm can be used by an autonomous agent or robot to decide where be… ▽ More An informative measurement is the most efficient way to gain information about an unknown state. We present a first-principles derivation of a general-purpose dynamic programming algorithm that returns an optimal sequence of informative measurements by sequentially maximizing the entropy of possible measurement outcomes. This algorithm can be used by an autonomous agent or robot to decide where best to measure next, planning a path corresponding to an optimal sequence of informative measurements. The algorithm is applicable to states and controls that are either continuous or discrete, and agent dynamics that is either stochastic or deterministic; including Markov decision processes and Gaussian processes. Recent results from the fields of approximate dynamic programming and reinforcement learning, including on-line approximations such as rollout and Monte Carlo tree search, allow the measurement task to be solved in real time. The resulting solutions include non-myopic paths and measurement sequences that can generally outperform, sometimes substantially, commonly used greedy approaches. This is demonstrated for a global search task, where on-line planning for a sequence of local searches is found to reduce the number of measurements in the search by approximately half. A variant of the algorithm is derived for Gaussian processes for active sensing. △ Less

Submitted 30 January, 2023; v1 submitted 24 September, 2021; originally announced September 2021.

Journal ref: Entropy, 25, 251 (2023)

A review of computational tools for generating metagenome-assembled genomes from metagenomic sequencing data

Authors: Chao Yang, Debajyoti Chowdhury, Zhenmiao Zhang, William K. Cheung, Ai** Lu, Zhao Xiang Bian, Lu Zhang

Abstract: Microbes are essentially yet convolutedly linked with human lives on the earth. They critically interfere in different physiological processes and thus influence overall health status. Studying microbial species is used to be constrained to those that can be cultured in the lab. But it excluded a huge portion of the microbiome that could not survive on lab conditions. In the past few years, the cu… ▽ More Microbes are essentially yet convolutedly linked with human lives on the earth. They critically interfere in different physiological processes and thus influence overall health status. Studying microbial species is used to be constrained to those that can be cultured in the lab. But it excluded a huge portion of the microbiome that could not survive on lab conditions. In the past few years, the culture-independent metagenomic sequencing enabled us to explore the complex microbial community coexisting within and on us. Metagenomics has equipped us with new avenues of investigating the microbiome, from studying a single species to a complex community in a dynamic ecosystem. Thus, identifying the involved microbes and their genomes becomes one of the core tasks in metagenomic sequencing. Metagenome-assembled genomes are groups of contigs with similar sequence characteristics from de novo assembly and could represent the microbial genomes from metagenomic sequencing. In this paper, we reviewed a spectrum of tools for producing and annotating metagenome-assembled genomes from metagenomic sequencing data and discussed their technical and biological perspectives. △ Less

Submitted 2 September, 2021; originally announced September 2021.

TOHAN: A One-step Approach towards Few-shot Hypothesis Adaptation

Authors: Haoang Chi, Feng Liu, Wen**g Yang, Long Lan, Tongliang Liu, Bo Han, William K. Cheung, James T. Kwok

Abstract: In few-shot domain adaptation (FDA), classifiers for the target domain are trained with accessible labeled data in the source domain (SD) and few labeled data in the target domain (TD). However, data usually contain private information in the current era, e.g., data distributed on personal phones. Thus, the private information will be leaked if we directly access data in SD to train a target-domai… ▽ More In few-shot domain adaptation (FDA), classifiers for the target domain are trained with accessible labeled data in the source domain (SD) and few labeled data in the target domain (TD). However, data usually contain private information in the current era, e.g., data distributed on personal phones. Thus, the private information will be leaked if we directly access data in SD to train a target-domain classifier (required by FDA methods). In this paper, to thoroughly prevent the privacy leakage in SD, we consider a very challenging problem setting, where the classifier for the TD has to be trained using few labeled target data and a well-trained SD classifier, named few-shot hypothesis adaptation (FHA). In FHA, we cannot access data in SD, as a result, the private information in SD will be protected well. To this end, we propose a target orientated hypothesis adaptation network (TOHAN) to solve the FHA problem, where we generate highly-compatible unlabeled data (i.e., an intermediate domain) to help train a target-domain classifier. TOHAN maintains two deep networks simultaneously, where one focuses on learning an intermediate domain and the other takes care of the intermediate-to-target distributional adaptation and the target-risk minimization. Experimental results show that TOHAN outperforms competitive baselines significantly. △ Less

Submitted 7 September, 2022; v1 submitted 11 June, 2021; originally announced June 2021.

KRADA: Known-region-aware Domain Alignment for Open-set Domain Adaptation in Semantic Segmentation

Authors: Chenhong Zhou, Feng Liu, Chen Gong, Rongfei Zeng, Tongliang Liu, William K. Cheung, Bo Han

Abstract: In semantic segmentation, we aim to train a pixel-level classifier to assign category labels to all pixels in an image, where labeled training images and unlabeled test images are from the same distribution and share the same label set. However, in an open world, the unlabeled test images probably contain unknown categories and have different distributions from the labeled images. Hence, in this p… ▽ More In semantic segmentation, we aim to train a pixel-level classifier to assign category labels to all pixels in an image, where labeled training images and unlabeled test images are from the same distribution and share the same label set. However, in an open world, the unlabeled test images probably contain unknown categories and have different distributions from the labeled images. Hence, in this paper, we consider a new, more realistic, and more challenging problem setting where the pixel-level classifier has to be trained with labeled images and unlabeled open-world images -- we name it open-set domain adaptation segmentation (OSDAS). In OSDAS, the trained classifier is expected to identify unknown-class pixels and classify known-class pixels well. To solve OSDAS, we first investigate which distribution that unknown-class pixels obey. Then, motivated by the goodness-of-fit test, we use statistical measurements to show how a pixel fits the distribution of an unknown class and select highly-fitted pixels to form the unknown region in each test image. Eventually, we propose an end-to-end learning framework, known-region-aware domain alignment (KRADA), to distinguish unknown classes while aligning the distributions of known classes in labeled and unlabeled open-world images. The effectiveness of KRADA has been verified on two synthetic tasks and one COVID-19 segmentation task. △ Less

Submitted 19 February, 2023; v1 submitted 11 June, 2021; originally announced June 2021.

Comments: 18 pages

Journal ref: Transactions on Machine Learning Research, 2023

Reduced order models for Lagrangian hydrodynamics

Authors: Dylan Matthew Copeland, Siu Wun Cheung, Kevin Huynh, Youngsoo Choi

Abstract: As a mathematical model of high-speed flow and shock wave propagation in a complex multimaterial setting, Lagrangian hydrodynamics is characterized by moving meshes, advection-dominated solutions, and moving shock fronts with sharp gradients. These challenges hinder the existing projection-based model reduction schemes from being practical. We develop several variations of projection-based reduced… ▽ More As a mathematical model of high-speed flow and shock wave propagation in a complex multimaterial setting, Lagrangian hydrodynamics is characterized by moving meshes, advection-dominated solutions, and moving shock fronts with sharp gradients. These challenges hinder the existing projection-based model reduction schemes from being practical. We develop several variations of projection-based reduced order model techniques for Lagrangian hydrodynamics by introducing three different reduced bases for position, velocity, and energy fields. A time-windowing approach is also developed to address the challenge imposed by the advection-dominated solutions. Lagrangian hydrodynamics is formulated as a nonlinear problem, which requires a proper hyper-reduction technique. Therefore, we apply the over-sampling DEIM and SNS approaches to reduce the complexity due to the nonlinear terms. Finally, we also present both a posteriori and a priori error bounds associated with our reduced order model. We compare the performance of the spatial and time-windowing reduced order modeling approaches in terms of accuracy and speed-up with respect to the corresponding full order model for several numerical examples, namely Sedov blast, Gresho vortices, Taylor-Green vortices, and triple-point problems. △ Less

Submitted 25 October, 2021; v1 submitted 23 April, 2021; originally announced April 2021.

Comments: 42 pages, 7 figure, 8 tables

Learning Foreground-Background Segmentation from Improved Layered GANs

Authors: Yu Yang, Hakan Bilen, Qiran Zou, Wing Yin Cheung, Xiangyang Ji

Abstract: Deep learning approaches heavily rely on high-quality human supervision which is nonetheless expensive, time-consuming, and error-prone, especially for image segmentation task. In this paper, we propose a method to automatically synthesize paired photo-realistic images and segmentation masks for the use of training a foreground-background segmentation network. In particular, we learn a generative… ▽ More Deep learning approaches heavily rely on high-quality human supervision which is nonetheless expensive, time-consuming, and error-prone, especially for image segmentation task. In this paper, we propose a method to automatically synthesize paired photo-realistic images and segmentation masks for the use of training a foreground-background segmentation network. In particular, we learn a generative adversarial network that decomposes an image into foreground and background layers, and avoid trivial decompositions by maximizing mutual information between generated images and latent variables. The improved layered GANs can synthesize higher quality datasets from which segmentation networks of higher performance can be learned. Moreover, the segmentation networks are employed to stabilize the training of layered GANs in return, which are further alternately trained with Layered GANs. Experiments on a variety of single-object datasets show that our method achieves competitive generation quality and segmentation performance compared to related methods. △ Less

Submitted 3 December, 2021; v1 submitted 1 April, 2021; originally announced April 2021.

PriorityCut: Occlusion-guided Regularization for Warp-based Image Animation

Authors: Wai Ting Cheung, Gyeongsu Chae

Abstract: Image animation generates a video of a source image following the motion of a driving video. State-of-the-art self-supervised image animation approaches warp the source image according to the motion of the driving video and recover the war** artifacts by inpainting. These approaches mostly use vanilla convolution for inpainting, and vanilla convolution does not distinguish between valid and inva… ▽ More Image animation generates a video of a source image following the motion of a driving video. State-of-the-art self-supervised image animation approaches warp the source image according to the motion of the driving video and recover the war** artifacts by inpainting. These approaches mostly use vanilla convolution for inpainting, and vanilla convolution does not distinguish between valid and invalid pixels. As a result, visual artifacts are still noticeable after inpainting. CutMix is a state-of-the-art regularization strategy that cuts and mixes patches of images and is widely studied in different computer vision tasks. Among the remaining computer vision tasks, warp-based image animation is one of the fields that the effects of CutMix have yet to be studied. This paper first presents a preliminary study on the effects of CutMix on warp-based image animation. We observed in our study that CutMix helps improve only pixel values, but disturbs the spatial relationships between pixels. Based on such observation, we propose PriorityCut, a novel augmentation approach that uses the top-k percent occluded pixels of the foreground to regularize warp-based image animation. By leveraging the domain knowledge in warp-based image animation, PriorityCut significantly reduces the war** artifacts in state-of-the-art warp-based image animation models on diverse datasets. △ Less

Submitted 22 March, 2021; originally announced March 2021.

Online adaptive algorithm for Constraint Energy Minimizing Generalized Multiscale Discontinuous Galerkin Method

Authors: Sai-Mang Pun, Siu Wun Cheung

Abstract: In this research, we propose an online basis enrichment strategy within the framework of a recently developed constraint energy minimizing generalized multiscale discontinuous Galerkin method (CEM-GMsDGM). Combining the technique of oversampling, one makes use of the information of the current residuals to adaptively construct basis functions in the online stage to reduce the error of multiscale a… ▽ More In this research, we propose an online basis enrichment strategy within the framework of a recently developed constraint energy minimizing generalized multiscale discontinuous Galerkin method (CEM-GMsDGM). Combining the technique of oversampling, one makes use of the information of the current residuals to adaptively construct basis functions in the online stage to reduce the error of multiscale approximation. A complete analysis of the method is presented, which shows the proposed online enrichment leads to a fast convergence from multiscale approximation to the fine-scale solution. The error reduction can be made sufficiently large by suitably selecting oversampling regions and the number of oversampling layers. Further, the convergence rate of the enrichment algorithm depends on a factor of exponential decay regarding the number of oversampling layers and a user-defined parameter. Numerical results are provided to demonstrate the effectiveness and efficiency of the proposed online adaptive algorithm. △ Less

Submitted 3 March, 2021; originally announced March 2021.

Comments: 22 pages, 5 figures

MSC Class: 65N30

Role of Zealots on the Adaptive Voter Model

Authors: Ka Wai Cheung, Chung Him Liu, Kwok Yip Szeto

Abstract: The voter model has been extensive studied as an opinion dynamic model, and the role of the zealots has only been discussed recently. We introduce the adaptive voter model with zealots and show that the final distribution of the magnetism can be separated into two regions depending on the number of zealots as well as the probability of forming link. When the fraction of zealots is dominated in the… ▽ More The voter model has been extensive studied as an opinion dynamic model, and the role of the zealots has only been discussed recently. We introduce the adaptive voter model with zealots and show that the final distribution of the magnetism can be separated into two regions depending on the number of zealots as well as the probability of forming link. When the fraction of zealots is dominated in the population, the probability distribution of magnetism follows a Gaussian-like distribution and the relaxation time is population-size independent. When the population is dominated by the susceptible agents, the relaxation time is proportional to the exponential of the population size. We have found the analytical solution of the relaxation time in the limiting cases and explained the difference of the relaxation time in these two regions based on the approximation method. △ Less

Submitted 28 February, 2021; originally announced March 2021.

Iterative Oversampling Technique for Constraint Energy Minimizing Generalized Multiscale Finite Element Method in the Mixed Formulation

Authors: Siu Wun Cheung, Eric Chung, Yalchin Efendiev, Wing Tat Leung, Sai-Mang Pun

Abstract: In this paper, we develop an iterative scheme to construct multiscale basis functions within the framework of the Constraint Energy Minimizing Generalized Multiscale Finite Element Method (CEM-GMsFEM) for the mixed formulation. The iterative procedure starts with the construction of an energy minimizing snapshot space that can be used for approximating the solution of the model problem. A spectral… ▽ More In this paper, we develop an iterative scheme to construct multiscale basis functions within the framework of the Constraint Energy Minimizing Generalized Multiscale Finite Element Method (CEM-GMsFEM) for the mixed formulation. The iterative procedure starts with the construction of an energy minimizing snapshot space that can be used for approximating the solution of the model problem. A spectral decomposition is then performed on the snapshot space to form global multiscale space. Under this setting, each global multiscale basis function can be split into a non-decaying and a decaying parts. The non-decaying part of a global basis is localized and it is fixed during the iteration. Then, one can approximate the decaying part via a modified Richardson scheme with an appropriately defined preconditioner. Using this set of iterative-based multiscale basis functions, first-order convergence with respect to the coarse mesh size can be shown if sufficiently many times of iterations with regularization parameter being in an appropriate range are performed. Numerical results are presented to illustrate the effectiveness and efficiency of the proposed computational multiscale method. △ Less

Submitted 3 December, 2020; originally announced December 2020.

Learning Inter-Modal Correspondence and Phenotypes from Multi-Modal Electronic Health Records

Authors: Ke**g Yin, William K. Cheung, Benjamin C. M. Fung, Jonathan Poon

Abstract: Non-negative tensor factorization has been shown a practical solution to automatically discover phenotypes from the electronic health records (EHR) with minimal human supervision. Such methods generally require an input tensor describing the inter-modal interactions to be pre-established; however, the correspondence between different modalities (e.g., correspondence between medications and diagnos… ▽ More Non-negative tensor factorization has been shown a practical solution to automatically discover phenotypes from the electronic health records (EHR) with minimal human supervision. Such methods generally require an input tensor describing the inter-modal interactions to be pre-established; however, the correspondence between different modalities (e.g., correspondence between medications and diagnoses) can often be missing in practice. Although heuristic methods can be applied to estimate them, they inevitably introduce errors, and leads to sub-optimal phenotype quality. This is particularly important for patients with complex health conditions (e.g., in critical care) as multiple diagnoses and medications are simultaneously present in the records. To alleviate this problem and discover phenotypes from EHR with unobserved inter-modal correspondence, we propose the collective hidden interaction tensor factorization (cHITF) to infer the correspondence between multiple modalities jointly with the phenotype discovery. We assume that the observed matrix for each modality is marginalization of the unobserved inter-modal correspondence, which are reconstructed by maximizing the likelihood of the observed matrices. Extensive experiments conducted on the real-world MIMIC-III dataset demonstrate that cHITF effectively infers clinically meaningful inter-modal correspondence, discovers phenotypes that are more clinically relevant and diverse, and achieves better predictive performance compared with a number of state-of-the-art computational phenoty** models. △ Less

Submitted 12 November, 2020; originally announced November 2020.

Comments: Accepted by IEEE Transactions on Knowledge and Data Engineering (TKDE)

Analysis of Non-local Multicontinuum Upscaling for Dual Continuum Model

Authors: **gyan Zhang, Siu Wun Cheung

Abstract: In this paper, we develop and analyze a rigorous multiscale upscaling method for dual continuum model, which serves as a powerful tool in subsurface formation applications. Our proposed method is capable of identifying different continua and capturing non-local transfer and effective properties in the computational domain via constructing localized multiscale basis functions. The construction of t… ▽ More In this paper, we develop and analyze a rigorous multiscale upscaling method for dual continuum model, which serves as a powerful tool in subsurface formation applications. Our proposed method is capable of identifying different continua and capturing non-local transfer and effective properties in the computational domain via constructing localized multiscale basis functions. The construction of the basis functions consists of solving local problems defined on oversampling computational region, subject to the energy minimizing constraints that the mean values of the local solution are zero in all continua except for the one targeted. The basis functions constructed are shown to have good approximation properties. It is shown that the method has a coarse mesh dependent convergence. We present some numerical examples to illustrate the performance of the proposed method. △ Less

Submitted 2 November, 2020; originally announced November 2020.

Multiscale simulations for multi-continuum Richards equations

Authors: Jun Sur Richard Park, Siu Wun Cheung, Tina Mai

Abstract: In this paper, we study a multiscale method for simulating a dual-continuum unsaturated flow problem within complex heterogeneous fractured porous media. Mathematically, each of the dual continua is modeled by a multiscale Richards equation (for pressure head), and these equations are coupled to one another by transfer terms. On its own, Richards equation is already a nonlinear partial differentia… ▽ More In this paper, we study a multiscale method for simulating a dual-continuum unsaturated flow problem within complex heterogeneous fractured porous media. Mathematically, each of the dual continua is modeled by a multiscale Richards equation (for pressure head), and these equations are coupled to one another by transfer terms. On its own, Richards equation is already a nonlinear partial differential equation, and it is exceedingly difficult to solve numerically due to the extra nonlinear dependencies involving the soil water. To deal with multiple scales, our strategy is that starting from a microscopic scale, we upscale the coupled system of dual-continuum Richards equations via homogenization by the two-scale asymptotic expansion, to obtain a homogenized system, at an intermediate scale (level). Based on a hierarchical approach, the homogenization's effective coefficients are computed through solving the arising cell problems. To tackle the nonlinearity, after time discretization, we use Picard iteration procedure for linearization of the homogenized Richards equations. At each Picard iteration, some degree of multiscale still remains from the intermediate level, so we utilize the generalized multiscale finite element method (GMsFEM) combining with a multi-continuum approach, to upscale the homogenized system to a macroscopic (coarse-grid) level. This scheme involves building uncoupled and coupled multiscale basis functions, which are used not only to construct coarse-grid solution approximation with high accuracy but also (with the coupled multiscale basis) to capture the interactions among continua. These prospects and convergence are demonstrated by several numerical results for the proposed method. △ Less

Submitted 2 June, 2021; v1 submitted 18 October, 2020; originally announced October 2020.

Comments: 45 pages, 4 figures, 2 tables, major revision. This is the accepted manuscript by Journal of Computational and Applied Mathematics (2021). The published journal article is available at https://doi.org/10.1016/j.cam.2021.113648 (2021)

MSC Class: 65N30; 65N99

Journal ref: Journal of Computational and Applied Mathematics, 397:113648, 2021

Probabilistic Sequential Shrinking: A Best Arm Identification Algorithm for Stochastic Bandits with Corruptions

Authors: Zixin Zhong, Wang Chi Cheung, Vincent Y. F. Tan

Abstract: We consider a best arm identification (BAI) problem for stochastic bandits with adversarial corruptions in the fixed-budget setting of T steps. We design a novel randomized algorithm, Probabilistic Sequential Shrinking($u$) (PSS($u$)), which is agnostic to the amount of corruptions. When the amount of corruptions per step (CPS) is below a threshold, PSS($u$) identifies the best arm or item with pr… ▽ More We consider a best arm identification (BAI) problem for stochastic bandits with adversarial corruptions in the fixed-budget setting of T steps. We design a novel randomized algorithm, Probabilistic Sequential Shrinking($u$) (PSS($u$)), which is agnostic to the amount of corruptions. When the amount of corruptions per step (CPS) is below a threshold, PSS($u$) identifies the best arm or item with probability tending to $1$ as $T\rightarrow \infty$. Otherwise, the optimality gap of the identified item degrades gracefully with the CPS.We argue that such a bifurcation is necessary. In PSS($u$), the parameter $u$ serves to balance between the optimality gap and success probability. The injection of randomization is shown to be essential to mitigate the impact of corruptions. To demonstrate this, we design two attack strategies that are applicable to any algorithm. We apply one of them to a deterministic analogue of PSS($u$) known as Successive Halving (SH) by Karnin et al. (2013). The attack strategy results in a high failure probability for SH, but PSS($u$) remains robust. In the absence of corruptions, PSS($2$)'s performance guarantee matches SH's. We show that when the CPS is sufficiently large, no algorithm can achieve a BAI probability tending to $1$ as $T\rightarrow \infty$. Numerical experiments corroborate our theoretical findings. △ Less

Submitted 18 June, 2021; v1 submitted 15 October, 2020; originally announced October 2020.

Comments: 22 pages, 9 figures

Explicit and Energy-Conserving Constraint Energy Minimizing Generalized Multiscale Discontinuous Galerkin Method for Wave Propagation in Heterogeneous Media

Authors: Siu Wun Cheung, Eric T. Chung, Yalchin Efendiev, Wing Tat Leung

Abstract: In this work, we propose a local multiscale model reduction approach for the time-domain scalar wave equation in a heterogenous media. A fine mesh is used to capture the heterogeneities of the coefficient field, and the equation is solved globally on a coarse mesh in the discontinuous Galerkin discretization setting. The main idea of the model reduction approach is to extract dominant modes in loc… ▽ More In this work, we propose a local multiscale model reduction approach for the time-domain scalar wave equation in a heterogenous media. A fine mesh is used to capture the heterogeneities of the coefficient field, and the equation is solved globally on a coarse mesh in the discontinuous Galerkin discretization setting. The main idea of the model reduction approach is to extract dominant modes in local spectral problems for representation of important features, construct multiscale basis functions in coarse oversampled regions by constraint energy minimization problems, and perform a Petrov-Galerkin projection and a symmetrization onto the coarse grid. The method is expicit and energy conserving, and exhibits both coarse-mesh and spectral convergence, provided that the oversampling size is appropriately chosen. We study the stability and convergence of our method. We also present numerical results on the Marmousi model in order to test the performance of the method and verify the theoretical results. △ Less

Submitted 1 September, 2020; originally announced September 2020.

Comments: arXiv admin note: substantial text overlap with arXiv:1909.12461

Automated Storytelling via Causal, Commonsense Plot Ordering

Authors: Prithviraj Ammanabrolu, Wesley Cheung, William Broniec, Mark O. Riedl

Abstract: Automated story plot generation is the task of generating a coherent sequence of plot events. Causal relations between plot events are believed to increase the perception of story and plot coherence. In this work, we introduce the concept of soft causal relations as causal relations inferred from commonsense reasoning. We demonstrate C2PO, an approach to narrative generation that operationalizes t… ▽ More Automated story plot generation is the task of generating a coherent sequence of plot events. Causal relations between plot events are believed to increase the perception of story and plot coherence. In this work, we introduce the concept of soft causal relations as causal relations inferred from commonsense reasoning. We demonstrate C2PO, an approach to narrative generation that operationalizes this concept through Causal, Commonsense Plot Ordering. Using human-participant protocols, we evaluate our system against baseline systems with different commonsense reasoning reasoning and inductive biases to determine the role of soft causal relations in perceived story quality. Through these studies we also probe the interplay of how changes in commonsense norms across storytelling genres affect perceptions of story quality. △ Less

Submitted 30 December, 2020; v1 submitted 2 September, 2020; originally announced September 2020.

Comments: AAAI-21 Camera Ready Version

Context-Dependent Implicit Authentication for Wearable Device User

Authors: William Cheung, Sudip Vhaduri

Abstract: As market wearables are becoming popular with a range of services, including making financial transactions, accessing cars, etc. that they provide based on various private information of a user, security of this information is becoming very important. However, users are often flooded with PINs and passwords in this internet of things (IoT) world. Additionally, hard-biometric, such as facial or fin… ▽ More As market wearables are becoming popular with a range of services, including making financial transactions, accessing cars, etc. that they provide based on various private information of a user, security of this information is becoming very important. However, users are often flooded with PINs and passwords in this internet of things (IoT) world. Additionally, hard-biometric, such as facial or finger recognition, based authentications are not adaptable for market wearables due to their limited sensing and computation capabilities. Therefore, it is a time demand to develop a burden-free implicit authentication mechanism for wearables using the less-informative soft-biometric data that are easily obtainable from the market wearables. In this work, we present a context-dependent soft-biometric-based wearable authentication system utilizing the heart rate, gait, and breathing audio signals. From our detailed analysis, we find that a binary support vector machine (SVM) with radial basis function (RBF) kernel can achieve an average accuracy of $0.94 \pm 0.07$, $F_1$ score of $0.93 \pm 0.08$, an equal error rate (EER) of about $0.06$ at a lower confidence threshold of 0.52, which shows the promise of this work. △ Less

Submitted 25 August, 2020; originally announced August 2020.

Comments: 7 pages, 5 figures, accepted at IEEE International Symposium on Personal, Indoor and Mobile Radio Communications (PIMRC). arXiv admin note: substantial text overlap with arXiv:2008.10779

Continuous Authentication of Wearable Device Users from Heart Rate, Gait, and Breathing Data

Authors: William Cheung, Sudip Vhaduri

Abstract: The security of private information is becoming the bedrock of an increasingly digitized society. While the users are flooded with passwords and PINs, these gold-standard explicit authentications are becoming less popular and valuable. Recent biometric-based authentication methods, such as facial or finger recognition, are getting popular due to their higher accuracy. However, these hard-biometric… ▽ More The security of private information is becoming the bedrock of an increasingly digitized society. While the users are flooded with passwords and PINs, these gold-standard explicit authentications are becoming less popular and valuable. Recent biometric-based authentication methods, such as facial or finger recognition, are getting popular due to their higher accuracy. However, these hard-biometric-based systems require dedicated devices with powerful sensors and authentication models, which are often limited to most of the market wearables. Still, market wearables are collecting various private information of a user and are becoming an integral part of life: accessing cars, bank accounts, etc. Therefore, time demands a burden-free implicit authentication mechanism for wearables using the less-informative soft-biometric data that are easily obtainable from modern market wearables. In this work, we present a context-dependent soft-biometric-based authentication system for wearables devices using heart rate, gait, and breathing audio signals. From our detailed analysis using the "leave-one-out" validation, we find that a lighter $k$-Nearest Neighbor ($k$-NN) model with $k = 2$ can obtain an average accuracy of $0.93 \pm 0.06$, $F_1$ score $0.93 \pm 0.03$, and {\em false positive rate} (FPR) below $0.08$ at 50\% level of confidence, which shows the promise of this work. △ Less

Submitted 24 August, 2020; originally announced August 2020.

Comments: 6 pages, 3 figures, Accepted at IEEE Biomedical Robotics & Biomechatronics (BioRob)

Efficient and Optimal Algorithms for Tree Summarization with Weighted Terminologies

Authors: Xuliang Zhu, Xin Huang, Byron Choi, Jianliang Xu, William K. Cheung, Yanchun Zhang, Jiming Liu

Abstract: Data summarization that presents a small subset of a dataset to users has been widely applied in numerous applications and systems. Many datasets are coded with hierarchical terminologies, e.g., the international classification of Diseases-9, Medical Subject Heading, and Gene Ontology, to name a few. In this paper, we study the problem of selecting a diverse set of k elements to summarize an input… ▽ More Data summarization that presents a small subset of a dataset to users has been widely applied in numerous applications and systems. Many datasets are coded with hierarchical terminologies, e.g., the international classification of Diseases-9, Medical Subject Heading, and Gene Ontology, to name a few. In this paper, we study the problem of selecting a diverse set of k elements to summarize an input dataset with hierarchical terminologies, and visualize the summary in an ontology structure. We propose an efficient greedy algorithm to solve the problem with (1-1/e) = 62% -approximation guarantee. Although this greedy solution achieves quality-guaranteed answers approximately but it is still not optimal. To tackle the problem optimally, we further develop a dynamic programming algorithm to obtain optimal answers for graph visualization of log-data using ontology terminologies called OVDO . The complexity and correctness of OVDO are theoretically analyzed. In addition, we propose a useful optimization technique of tree reduction to remove useless nodes with zero weights and shrink the tree into a smaller one, which ensures the efficiency acceleration of OVDO in many real-world applications. Extensive experimental results on real-world datasets show the effectiveness and efficiency of our proposed approximate and exact algorithms for tree data summarization. △ Less

Submitted 14 October, 2021; v1 submitted 7 August, 2020; originally announced August 2020.

Reinforcement Learning for Non-Stationary Markov Decision Processes: The Blessing of (More) Optimism

Authors: Wang Chi Cheung, David Simchi-Levi, Ruihao Zhu

Abstract: We consider un-discounted reinforcement learning (RL) in Markov decision processes (MDPs) under drifting non-stationarity, i.e., both the reward and state transition distributions are allowed to evolve over time, as long as their respective total variations, quantified by suitable metrics, do not exceed certain variation budgets. We first develop the Sliding Window Upper-Confidence bound for Reinf… ▽ More We consider un-discounted reinforcement learning (RL) in Markov decision processes (MDPs) under drifting non-stationarity, i.e., both the reward and state transition distributions are allowed to evolve over time, as long as their respective total variations, quantified by suitable metrics, do not exceed certain variation budgets. We first develop the Sliding Window Upper-Confidence bound for Reinforcement Learning with Confidence Widening (SWUCRL2-CW) algorithm, and establish its dynamic regret bound when the variation budgets are known. In addition, we propose the Bandit-over-Reinforcement Learning (BORL) algorithm to adaptively tune the SWUCRL2-CW algorithm to achieve the same dynamic regret bound, but in a parameter-free manner, i.e., without knowing the variation budgets. Notably, learning non-stationary MDPs via the conventional optimistic exploration technique presents a unique challenge absent in existing (non-stationary) bandit learning settings. We overcome the challenge by a novel confidence widening technique that incorporates additional optimism. △ Less

Submitted 24 June, 2020; originally announced June 2020.

Comments: To appear in proceedings of the 37th International Conference on Machine Learning. Shortened conference version of its journal version (available at: arXiv:1906.02922)

On Hilbert's sum type inequalities

Authors: Chang-Jian Zhao, Wing Sum Cheung

Abstract: The main purpose of the present article is to give some new Hilbert's sum type inequalities, which in special cases yield the classical Hilbert's inequalities. Our results provide some new estimates to these types of inequalities. The main purpose of the present article is to give some new Hilbert's sum type inequalities, which in special cases yield the classical Hilbert's inequalities. Our results provide some new estimates to these types of inequalities. △ Less

Submitted 17 February, 2020; originally announced February 2020.

Bringing Stories Alive: Generating Interactive Fiction Worlds

Authors: Prithviraj Ammanabrolu, Wesley Cheung, Dan Tu, William Broniec, Mark O. Riedl

Abstract: World building forms the foundation of any task that requires narrative intelligence. In this work, we focus on procedurally generating interactive fiction worlds---text-based worlds that players "see" and "talk to" using natural language. Generating these worlds requires referencing everyday and thematic commonsense priors in addition to being semantically consistent, interesting, and coherent th… ▽ More World building forms the foundation of any task that requires narrative intelligence. In this work, we focus on procedurally generating interactive fiction worlds---text-based worlds that players "see" and "talk to" using natural language. Generating these worlds requires referencing everyday and thematic commonsense priors in addition to being semantically consistent, interesting, and coherent throughout. Using existing story plots as inspiration, we present a method that first extracts a partial knowledge graph encoding basic information regarding world structure such as locations and objects. This knowledge graph is then automatically completed utilizing thematic knowledge and used to guide a neural language generation model that fleshes out the rest of the world. We perform human participant-based evaluations, testing our neural model's ability to extract and fill-in a knowledge graph and to generate language conditioned on it against rule-based and human-made baselines. Our code is available at https://github.com/rajammanabrolu/WorldGeneration. △ Less

Submitted 27 January, 2020; originally announced January 2020.