Showing 1–50 of 221 results for author: Ong, Y

Generalized Entropy Implies Varying-G: Horizon Area Dependent Field Equations and Black Hole-Cosmology Coupling

Authors: Hengxin Lu, Sofia Di Gennaro, Yen Chin Ong

Abstract: When the Bekenstein-Hawking entropy is modified, ambiguity often arises concerning whether the Hawking temperature or the thermodynamic mass should be modified. The common practice, however, is to keep the black hole solution the same as that in general relativity. On the other hand, if Jacobson's method of deriving Einstein equations from thermodynamic is valid in the general settings, then given… ▽ More When the Bekenstein-Hawking entropy is modified, ambiguity often arises concerning whether the Hawking temperature or the thermodynamic mass should be modified. The common practice, however, is to keep the black hole solution the same as that in general relativity. On the other hand, if Jacobson's method of deriving Einstein equations from thermodynamic is valid in the general settings, then given a generalized entropy one should first derive the corresponding modified gravity, and then look for the compatible black hole solution before investigating its thermodynamics. We comment on some properties and subtleties in this approach. In particular, we point out that generically generalized entropy would lead to a varying effective gravitational "constant" theory, where $G_\text{eff}$ depends on the horizon area. We discuss in what ways such theories are discernible from general relativity despite its seemingly jarring differences, and how to make sense of area-dependent field equations. As a consequence we show that in the Jacobson's approach, the standard quantum gravitational logarithmic correction to Bekenstein-Hawking entropy is equivalent to a running gravitational "constant". A horizon area dependent $G_\text{eff}$ could also lead to a coupling between black hole masses and cosmological expansion, a scenario that has been studied recently in the literature, but so far lacks strong theoretical motivation. In the Tsallis case, we show that the thermodynamic mass for a Schwarzschild black hole is just a constant multiple of its ADM mass, which is considerably simpler than the approach not utilizing the Jacobson's method. △ Less

Submitted 29 June, 2024; originally announced July 2024.

LLM2FEA: Discover Novel Designs with Generative Evolutionary Multitasking

Authors: Melvin Wong, Jiao Liu, Thiago Rios, Stefan Menzel, Yew Soon Ong

Abstract: The rapid research and development of generative artificial intelligence has enabled the generation of high-quality images, text, and 3D models from text prompts. This advancement impels an inquiry into whether these models can be leveraged to create digital artifacts for both creative and engineering applications. Drawing on innovative designs from other domains may be one answer to this question… ▽ More The rapid research and development of generative artificial intelligence has enabled the generation of high-quality images, text, and 3D models from text prompts. This advancement impels an inquiry into whether these models can be leveraged to create digital artifacts for both creative and engineering applications. Drawing on innovative designs from other domains may be one answer to this question, much like the historical practice of ``bionics", where humans have sought inspiration from nature's exemplary designs. This raises the intriguing possibility of using generative models to simultaneously tackle design tasks across multiple domains, facilitating cross-domain learning and resulting in a series of innovative design solutions. In this paper, we propose LLM2FEA as the first attempt to discover novel designs in generative models by transferring knowledge across multiple domains. By utilizing a multi-factorial evolutionary algorithm (MFEA) to drive a large language model, LLM2FEA integrates knowledge from various fields to generate prompts that guide the generative model in discovering novel and practical objects. Experimental results in the context of 3D aerodynamic design verify the discovery capabilities of the proposed LLM2FEA. The designs generated by LLM2FEA not only satisfy practicality requirements to a certain degree but also feature novel and aesthetically pleasing shapes, demonstrating the potential applications of LLM2FEA in discovery tasks. △ Less

Submitted 21 June, 2024; originally announced June 2024.

Comments: This work has been submitted to the IEEE for possible publication. Copyright may be transferred without notice, after which this version may no longer be accessible

Generative AI-based Prompt Evolution Engineering Design Optimization With Vision-Language Model

Authors: Melvin Wong, Thiago Rios, Stefan Menzel, Yew Soon Ong

Abstract: Engineering design optimization requires an efficient combination of a 3D shape representation, an optimization algorithm, and a design performance evaluation method, which is often computationally expensive. We present a prompt evolution design optimization (PEDO) framework contextualized in a vehicle design scenario that leverages a vision-language model for penalizing impractical car designs sy… ▽ More Engineering design optimization requires an efficient combination of a 3D shape representation, an optimization algorithm, and a design performance evaluation method, which is often computationally expensive. We present a prompt evolution design optimization (PEDO) framework contextualized in a vehicle design scenario that leverages a vision-language model for penalizing impractical car designs synthesized by a generative model. The backbone of our framework is an evolutionary strategy coupled with an optimization objective function that comprises a physics-based solver and a vision-language model for practical or functional guidance in the generated car designs. In the prompt evolutionary search, the optimizer iteratively generates a population of text prompts, which embed user specifications on the aerodynamic performance and visual preferences of the 3D car designs. Then, in addition to the computational fluid dynamics simulations, the pre-trained vision-language model is used to penalize impractical designs and, thus, foster the evolutionary algorithm to seek more viable designs. Our investigations on a car design optimization problem show a wide spread of potential car designs generated at the early phase of the search, which indicates a good diversity of designs in the initial populations, and an increase of over 20\% in the probability of generating practical designs compared to a baseline framework without using a vision-language model. Visual inspection of the designs against the performance results demonstrates prompt evolution as a very promising paradigm for finding novel designs with good optimization performance while providing ease of use in specifying design specifications and preferences via a natural language interface. △ Less

Submitted 14 June, 2024; v1 submitted 13 June, 2024; originally announced June 2024.

Comments: Accepted and to be published in IEEE Congress on Evolutionary Computation (CEC) 2024. Copyright 2024 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses

Road Network Representation Learning with the Third Law of Geography

Authors: Haicang Zhou, Weiming Huang, Yile Chen, Tiantian He, Gao Cong, Yew-Soon Ong

Abstract: Road network representation learning aims to learn compressed and effective vectorized representations for road segments that are applicable to numerous tasks. In this paper, we identify the limitations of existing methods, particularly their overemphasis on the distance effect as outlined in the First Law of Geography. In response, we propose to endow road network representation with the principl… ▽ More Road network representation learning aims to learn compressed and effective vectorized representations for road segments that are applicable to numerous tasks. In this paper, we identify the limitations of existing methods, particularly their overemphasis on the distance effect as outlined in the First Law of Geography. In response, we propose to endow road network representation with the principles of the recent Third Law of Geography. To this end, we propose a novel graph contrastive learning framework that employs geographic configuration-aware graph augmentation and spectral negative sampling, ensuring that road segments with similar geographic configurations yield similar representations, and vice versa, aligning with the principles stated in the Third Law. The framework further fuses the Third Law with the First Law through a dual contrastive learning objective to effectively balance the implications of both laws. We evaluate our framework on two real-world datasets across three downstream tasks. The results show that the integration of the Third Law significantly improves the performance of road segment representations in downstream tasks. △ Less

Submitted 6 June, 2024; originally announced June 2024.

Covariance-Adaptive Sequential Black-box Optimization for Diffusion Targeted Generation

Authors: Yueming Lyu, Kim Yong Tan, Yew Soon Ong, Ivor W. Tsang

Abstract: Diffusion models have demonstrated great potential in generating high-quality content for images, natural language, protein domains, etc. However, how to perform user-preferred targeted generation via diffusion models with only black-box target scores of users remains challenging. To address this issue, we first formulate the fine-tuning of the targeted reserve-time stochastic differential equatio… ▽ More Diffusion models have demonstrated great potential in generating high-quality content for images, natural language, protein domains, etc. However, how to perform user-preferred targeted generation via diffusion models with only black-box target scores of users remains challenging. To address this issue, we first formulate the fine-tuning of the targeted reserve-time stochastic differential equation (SDE) associated with a pre-trained diffusion model as a sequential black-box optimization problem. Furthermore, we propose a novel covariance-adaptive sequential optimization algorithm to optimize cumulative black-box scores under unknown transition dynamics. Theoretically, we prove a $O(\frac{d^2}{\sqrt{T}})$ convergence rate for cumulative convex functions without smooth and strongly convex assumptions. Empirically, experiments on both numerical test problems and target-guided 3D-molecule generation tasks show the superior performance of our method in achieving better target scores. △ Less

Submitted 8 June, 2024; v1 submitted 2 June, 2024; originally announced June 2024.

SIG: Efficient Self-Interpretable Graph Neural Network for Continuous-time Dynamic Graphs

Authors: Lanting Fang, Yulian Yang, Kai Wang, Shanshan Feng, Kaiyu Feng, Jie Gui, Shuliang Wang, Yew-Soon Ong

Abstract: While dynamic graph neural networks have shown promise in various applications, explaining their predictions on continuous-time dynamic graphs (CTDGs) is difficult. This paper investigates a new research task: self-interpretable GNNs for CTDGs. We aim to predict future links within the dynamic graph while simultaneously providing causal explanations for these predictions. There are two key challen… ▽ More While dynamic graph neural networks have shown promise in various applications, explaining their predictions on continuous-time dynamic graphs (CTDGs) is difficult. This paper investigates a new research task: self-interpretable GNNs for CTDGs. We aim to predict future links within the dynamic graph while simultaneously providing causal explanations for these predictions. There are two key challenges: (1) capturing the underlying structural and temporal information that remains consistent across both independent and identically distributed (IID) and out-of-distribution (OOD) data, and (2) efficiently generating high-quality link prediction results and explanations. To tackle these challenges, we propose a novel causal inference model, namely the Independent and Confounded Causal Model (ICCM). ICCM is then integrated into a deep learning architecture that considers both effectiveness and efficiency. Extensive experiments demonstrate that our proposed model significantly outperforms existing methods across link prediction accuracy, explanation quality, and robustness to shortcut features. Our code and datasets are anonymously released at https://github.com/2024SIG/SIG. △ Less

Submitted 29 May, 2024; originally announced May 2024.

Comments: 19 pages

Learning Mixture-of-Experts for General-Purpose Black-Box Discrete Optimization

Authors: Shengcai Liu, Zhiyuan Wang, Yew-Soon Ong, Xin Yao, Ke Tang

Abstract: Real-world applications involve various discrete optimization problems. Designing a specialized optimizer for each of these problems is challenging, typically requiring significant domain knowledge and human efforts. Hence, develo** general-purpose optimizers as an off-the-shelf tool for a wide range of problems has been a long-standing research target. This article introduces MEGO, a novel gene… ▽ More Real-world applications involve various discrete optimization problems. Designing a specialized optimizer for each of these problems is challenging, typically requiring significant domain knowledge and human efforts. Hence, develo** general-purpose optimizers as an off-the-shelf tool for a wide range of problems has been a long-standing research target. This article introduces MEGO, a novel general-purpose neural optimizer trained through a fully data-driven learning-to-optimize (L2O) approach. MEGO consists of a mixture-of-experts trained on experiences from solving training problems and can be viewed as a foundation model for optimization problems with binary decision variables. When presented with a problem to solve, MEGO actively selects relevant expert models to generate high-quality solutions. MEGO can be used as a standalone sample-efficient optimizer or in conjunction with existing search methods as an initial solution generator. The generality of MEGO is validated across six problem classes, including three classic problem classes and three problem classes arising from real-world applications in compilers, network analysis, and 3D reconstruction. Trained solely on classic problem classes, MEGO performs very well on all six problem classes, significantly surpassing widely used general-purpose optimizers in both solution quality and efficiency. In some cases, MEGO even surpasses specialized state-of-the-art optimizers. Additionally, MEGO provides a similarity measure between problems, yielding a new perspective for problem classification. In the pursuit of general-purpose optimizers through L2O, MEGO represents an initial yet significant step forward. △ Less

Submitted 29 May, 2024; originally announced May 2024.

Comments: 34 pages, 6 figures

Human-Generative AI Collaborative Problem Solving Who Leads and How Students Perceive the Interactions

Authors: Gaoxia Zhu, Vidya Sudarshan, Jason Fok Kow, Yew Soon Ong

Abstract: This research investigates distinct human-generative AI collaboration types and students' interaction experiences when collaborating with generative AI (i.e., ChatGPT) for problem-solving tasks and how these factors relate to students' sense of agency and perceived collaborative problem solving. By analyzing the surveys and reflections of 79 undergraduate students, we identified three human-genera… ▽ More This research investigates distinct human-generative AI collaboration types and students' interaction experiences when collaborating with generative AI (i.e., ChatGPT) for problem-solving tasks and how these factors relate to students' sense of agency and perceived collaborative problem solving. By analyzing the surveys and reflections of 79 undergraduate students, we identified three human-generative AI collaboration types: even contribution, human leads, and AI leads. Notably, our study shows that 77.21% of students perceived they led or had even contributed to collaborative problem-solving when collaborating with ChatGPT. On the other hand, 15.19% of the human participants indicated that the collaborations were led by ChatGPT, indicating a potential tendency for students to rely on ChatGPT. Furthermore, 67.09% of students perceived their interaction experiences with ChatGPT to be positive or mixed. We also found a positive correlation between positive interaction experience and a sense of positive agency. The results of this study contribute to our understanding of the collaboration between students and generative AI and highlight the need to study further why some students let ChatGPT lead collaborative problem-solving and how to enhance their interaction experience through curriculum and technology design. △ Less

Submitted 18 May, 2024; originally announced May 2024.

Comments: This paper appears at the IEEE Conference on Artificial Intelligence (CAI) 2024

Testing Cotton gravity as dark matter substitute with weak lensing

Authors: Geyu Mo, Qingqing Wang, Xin Ren, Weitong Yan, Yen Chin Ong, Wentao Luo

Abstract: Harada proposed a modified theory of gravity called Cotton gravity, and argued that it successfully explains the rotation curves of $84$ galaxies without the need of dark matter. In this work we use galaxy-galaxy lensing technique to test whether the modification effect of Cotton gravity can indeed be a viable substitute for dark matter. Using the spherically symmetric solution of Cotton gravity,… ▽ More Harada proposed a modified theory of gravity called Cotton gravity, and argued that it successfully explains the rotation curves of $84$ galaxies without the need of dark matter. In this work we use galaxy-galaxy lensing technique to test whether the modification effect of Cotton gravity can indeed be a viable substitute for dark matter. Using the spherically symmetric solution of Cotton gravity, we obtain the deflection angle via Gauss-Bonnet theorem and the weak lensing shear. We use five galaxy catalogs divided in 5 stellar mass bins from the Sloan Digital Sky Survey Data Release 7 (SDSS DR7), each of which is further divided into blue star forming galaxy and red passive galaxy sub-catalogs. We find that Cotton gravity on its own has significant deviation from the measured galaxy-galaxy lensing signals, thus it cannot replace the role of dark matter. If we consider the combination of dark matter and Cotton gravity, the modification is tightly constrained. Our analysis also applies to other modified gravity theories whose an additional linear term appears in the Schwarzschild solution. △ Less

Submitted 12 May, 2024; originally announced May 2024.

Comments: 16 pages, 3 figures

Constrain the linear scalar perturbation theory of Cotton gravity

Authors: Pengbo Xia, Dongdong Zhang, Xin Ren, Bo Wang, Yen Chin Ong

Abstract: We perform a cosmological test of Cotton gravity, which describes gravity by cotton tensor. The model we consider allows for the same background evolution as the $Λ$CDM model. We derive the cosmological perturbation theory of the scalar mode at the linear level, where the difference from the $Λ$CDM model is characterized by the parameter $β$. We incorporate Cotton gravity with a neutrino model and… ▽ More We perform a cosmological test of Cotton gravity, which describes gravity by cotton tensor. The model we consider allows for the same background evolution as the $Λ$CDM model. We derive the cosmological perturbation theory of the scalar mode at the linear level, where the difference from the $Λ$CDM model is characterized by the parameter $β$. We incorporate Cotton gravity with a neutrino model and perform a Monte Carlo Markov Chain (MCMC) analysis using data from the Cosmic Microwave Background (CMB) and Sloan Digital Sky Survey (SDSS). The analysis constrains parameter $β=-0.00008^{+0.00080}_{-0.00104}$ at the 1-$σ$ confidence level. We conclude that currently, there is no obvious deviation between Cotton gravity and the $Λ$CDM model in the linear cosmological perturbation level for observations. △ Less

Submitted 12 May, 2024; originally announced May 2024.

Comments: 12 pages, 15 figures

Classical Acceleration Temperature (CAT) in a Box

Authors: Ahsan Mujtaba, Maksat Temirkhan, Yen Chin Ong, Michael R. R. Good

Abstract: A confined, slow-moving, accelerating electron is shown to emit thermal radiation. Since laboratories face spatial constraints when dealing with rectilinear motion, focusing on a finite total travel distance combines the benefits of simple theoretical analysis with prospects for table-top experimentation. We demonstrate an accelerated moving charge along an asymptotically static worldline with fix… ▽ More A confined, slow-moving, accelerating electron is shown to emit thermal radiation. Since laboratories face spatial constraints when dealing with rectilinear motion, focusing on a finite total travel distance combines the benefits of simple theoretical analysis with prospects for table-top experimentation. We demonstrate an accelerated moving charge along an asymptotically static worldline with fixed transit distance and slow maximum speed, emitting self-consistent analytic power, spectra, and energy. The classical radiation is Planck distributed with an associated acceleration temperature. This is the first fully parametrized, spectrum-solved, finite-distance worldline. △ Less

Submitted 6 May, 2024; originally announced May 2024.

Comments: 10 figures, 10 pages

A communication protocol based on NK boolean networks for coordinating collective action

Authors: Yori Ong

Abstract: In this paper, I describe a digital social communication protocol (Gridt) based on Kauffman's NK boolean networks. The main assertion is that a communication network with this topology supports infinitely scalable self-organization of collective action without requiring hierarchy or central control. The paper presents the functionality of this protocol and substantiates the following propositions… ▽ More In this paper, I describe a digital social communication protocol (Gridt) based on Kauffman's NK boolean networks. The main assertion is that a communication network with this topology supports infinitely scalable self-organization of collective action without requiring hierarchy or central control. The paper presents the functionality of this protocol and substantiates the following propositions about its function and implications: (1) Communication via NK boolean networks facilitates coordination on collective action games for any variable number of users, and justifies the assumption that the game's payoff structure is common knowledge; (2) Use of this protocol increases its users' transfer empowerment, a form of intrinsic motivation that motivates coordinated action independent of the task or outcome; (3) Communication via this network can be considered 'cheap talk' and benefits the strategy of players with aligned interests, but not of players with conflicting interests; (4) Absence of significant barriers for its realization warrants a timely and continuing discussion on the ethics and implications of this technology; (5) Full realization of the technology's potential calls for a free-to-use service with maximal transparency of design and associated economic incentives. △ Less

Submitted 24 April, 2024; originally announced April 2024.

Comments: 13 pages, 4 figures

Jamming memory into acoustically trained dense suspensions under shear

Authors: Edward Y. X. Ong, Anna R. Barth, Navneet Singh, Meera Ramaswamy, Abhishek Shetty, Bulbul Chakraborty, James P. Sethna, Itai Cohen

Abstract: Systems driven far from equilibrium often retain structural memories of their processing history. This memory has, in some cases, been shown to dramatically alter the material response. For example, work hardening in crystalline metals can alter the hardness, yield strength, and tensile strength to prevent catastrophic failure. Whether memory of processing history can be similarly exploited in flo… ▽ More Systems driven far from equilibrium often retain structural memories of their processing history. This memory has, in some cases, been shown to dramatically alter the material response. For example, work hardening in crystalline metals can alter the hardness, yield strength, and tensile strength to prevent catastrophic failure. Whether memory of processing history can be similarly exploited in flowing systems, where significantly larger changes in structure should be possible, remains poorly understood. Here, we demonstrate a promising route to embedding such useful memories. We build on work showing that exposing a sheared dense suspension to acoustic perturbations of different power allows for dramatically tuning the sheared suspension viscosity and underlying structure. We find that, for sufficiently dense suspensions, upon removing the acoustic perturbations, the suspension shear jams with shear stress contributions from the maximum compressive and maximum extensive axes that reflect the acoustic training. Because the contributions from these two orthogonal axes to the total shear stress are antagonistic, it is possible to tune the resulting suspension response in surprising ways. For example, we show that differently trained sheared suspensions exhibit: 1) different susceptibility to the same acoustic perturbation; 2) orders of magnitude changes in their instantaneous viscosities upon shear reversal; and 3) even a shear stress that increases in magnitude upon shear cessation. To further illustrate the power of this approach for controlling suspension properties, we demonstrate that flowing states well below the shear jamming threshold can be shear jammed via acoustic training. Collectively, our work paves the way for using acoustically induced memory in dense suspensions to generate rapidly and widely tunable materials. △ Less

Submitted 24 April, 2024; originally announced April 2024.

Comments: To be published in Physical Review X

Bridging the Gap Between Theory and Practice: Benchmarking Transfer Evolutionary Optimization

Authors: Yaqing Hou, Wenqiang Ma, Abhishek Gupta, Kavitesh Kumar Bali, Hongwei Ge, Qiang Zhang, Carlos A. Coello Coello, Yew-Soon Ong

Abstract: In recent years, the field of Transfer Evolutionary Optimization (TrEO) has witnessed substantial growth, fueled by the realization of its profound impact on solving complex problems. Numerous algorithms have emerged to address the challenges posed by transferring knowledge between tasks. However, the recently highlighted ``no free lunch theorem'' in transfer optimization clarifies that no single… ▽ More In recent years, the field of Transfer Evolutionary Optimization (TrEO) has witnessed substantial growth, fueled by the realization of its profound impact on solving complex problems. Numerous algorithms have emerged to address the challenges posed by transferring knowledge between tasks. However, the recently highlighted ``no free lunch theorem'' in transfer optimization clarifies that no single algorithm reigns supreme across diverse problem types. This paper addresses this conundrum by adopting a benchmarking approach to evaluate the performance of various TrEO algorithms in realistic scenarios. Despite the growing methodological focus on transfer optimization, existing benchmark problems often fall short due to inadequate design, predominantly featuring synthetic problems that lack real-world relevance. This paper pioneers a practical TrEO benchmark suite, integrating problems from the literature categorized based on the three essential aspects of Big Source Task-Instances: volume, variety, and velocity. Our primary objective is to provide a comprehensive analysis of existing TrEO algorithms and pave the way for the development of new approaches to tackle practical challenges. By introducing realistic benchmarks that embody the three dimensions of volume, variety, and velocity, we aim to foster a deeper understanding of algorithmic performance in the face of diverse and complex transfer scenarios. This benchmark suite is poised to serve as a valuable resource for researchers, facilitating the refinement and advancement of TrEO algorithms in the pursuit of solving real-world problems. △ Less

Submitted 20 April, 2024; originally announced April 2024.

Comments: 17 pages, 18 figures

Edge-wave phase-shifts versus normal-mode phase-tilts in an Eady problem with a slo** boundary

Authors: Julian Mak, Nili Harnik, Eyal Heifetz, Gautam Kumar, Ellie Q. Y. Ong

Abstract: One mechanistic interpretation of baroclinic instability is that of mutual constructive interference of Rossby edge-waves. While the two edge-waves and their relative phase-shifts are invoked as part of the mechanistic interpretation, the phase-tilts of the related normal modes are often presented instead. Here we highlight the differences between edge-wave phase-shifts and normal-mode phase-tilts… ▽ More One mechanistic interpretation of baroclinic instability is that of mutual constructive interference of Rossby edge-waves. While the two edge-waves and their relative phase-shifts are invoked as part of the mechanistic interpretation, the phase-tilts of the related normal modes are often presented instead. Here we highlight the differences between edge-wave phase-shifts and normal-mode phase-tilts, in the context of an Eady problem modified by the presence of a slo** boundary. We argue and present evidence that the normal-mode phase-tilt is potentially a misleading quantity to use, and edge-wave phase-shifts should be the ones that are mechanistically relevant. We also provide a clarification for the mechanistic rationalization for baroclinic instability in the presence of slopes (such as suppression of growth rates) that is valid over all parameter space, in contrast to previous attempts. We further present evidence that there is a strong correlation between quantities diagnosed from the GEOMETRIC framework with the edge-wave phase-shifts, but not the normal-mode phase-tilts. The result is noteworthy in that the geometric framework makes no explicit reference to the edge-wave structures in its construction, but the correlation suggests that in problems where edge-wave structures are not so well-defined or readily available, the GEOMETRIC framework should still capture mechanistic and dynamical information. Some implications for parameterization of baroclinic instability and relevant eddy-mean feedbacks are discussed. For completeness, we also provide an explicit demonstration that the linear instability problem of the present modified Eady problem is parity-time symmetric, and speculate on some suggestive links between parity-time symmetry, shear instability, and the edge-wave interaction mechanism. △ Less

Submitted 18 April, 2024; originally announced April 2024.

Comments: 9 figures; discrepancy of arXiv abstract and file abstract, because number of characters exceeded in arXiv version

Upper Bound of Barrow Entropy Index from Black Hole Fragmentation

Authors: Jiayi Xia, Yen Chin Ong

Abstract: Both classical and quantum arguments suggest that if Barrow entropy is correct, its index $δ$ must be energy dependent, which would affect the very early universe. Based on thermodynamic stability that sufficiently large black holes should not fragment, we argue that Barrow entropy correction must be small except possibly at the Planckian regime. Furthermore, the fact that a solar mass black hole… ▽ More Both classical and quantum arguments suggest that if Barrow entropy is correct, its index $δ$ must be energy dependent, which would affect the very early universe. Based on thermodynamic stability that sufficiently large black holes should not fragment, we argue that Barrow entropy correction must be small except possibly at the Planckian regime. Furthermore, the fact that a solar mass black hole does not fragment implies an upper bound $δ\lesssim O(10^{-3})$, which surprisingly lies in the same range as the bound obtained from some cosmological considerations assuming fixed $δ$. This indicates that allowing $δ$ to run does not raise its allowed value. We briefly comment on the case of Kaniadakis entropy. △ Less

Submitted 14 April, 2024; originally announced April 2024.

Comments: 3 pages; invited contribution

Journal ref: Universe 2024, 10(4), 177

Where to Move Next: Zero-shot Generalization of LLMs for Next POI Recommendation

Authors: Shanshan Feng, Haoming Lyu, Caishun Chen, Yew-Soon Ong

Abstract: Next Point-of-interest (POI) recommendation provides valuable suggestions for users to explore their surrounding environment. Existing studies rely on building recommendation models from large-scale users' check-in data, which is task-specific and needs extensive computational resources. Recently, the pretrained large language models (LLMs) have achieved significant advancements in various NLP tas… ▽ More Next Point-of-interest (POI) recommendation provides valuable suggestions for users to explore their surrounding environment. Existing studies rely on building recommendation models from large-scale users' check-in data, which is task-specific and needs extensive computational resources. Recently, the pretrained large language models (LLMs) have achieved significant advancements in various NLP tasks and have also been investigated for recommendation scenarios. However, the generalization abilities of LLMs still are unexplored to address the next POI recommendations, where users' geographical movement patterns should be extracted. Although there are studies that leverage LLMs for next-item recommendations, they fail to consider the geographical influence and sequential transitions. Hence, they cannot effectively solve the next POI recommendation task. To this end, we design novel prompting strategies and conduct empirical studies to assess the capability of LLMs, e.g., ChatGPT, for predicting a user's next check-in. Specifically, we consider several essential factors in human movement behaviors, including user geographical preference, spatial distance, and sequential transitions, and formulate the recommendation task as a ranking problem. Through extensive experiments on two widely used real-world datasets, we derive several key findings. Empirical evaluations demonstrate that LLMs have promising zero-shot recommendation abilities and can provide accurate and reasonable predictions. We also reveal that LLMs cannot accurately comprehend geographical context information and are sensitive to the order of presentation of candidate POIs, which shows the limitations of LLMs and necessitates further research on robust human mobility reasoning mechanisms. △ Less

Submitted 22 April, 2024; v1 submitted 2 April, 2024; originally announced April 2024.

A Simple Yet Effective Approach for Diversified Session-Based Recommendation

Authors: Qing Yin, Hui Fang, Zhu Sun, Yew-Soon Ong

Abstract: Session-based recommender systems (SBRSs) have become extremely popular in view of the core capability of capturing short-term and dynamic user preferences. However, most SBRSs primarily maximize recommendation accuracy but ignore user minor preferences, thus leading to filter bubbles in the long run. Only a handful of works, being devoted to improving diversity, depend on unique model designs and… ▽ More Session-based recommender systems (SBRSs) have become extremely popular in view of the core capability of capturing short-term and dynamic user preferences. However, most SBRSs primarily maximize recommendation accuracy but ignore user minor preferences, thus leading to filter bubbles in the long run. Only a handful of works, being devoted to improving diversity, depend on unique model designs and calibrated loss functions, which cannot be easily adapted to existing accuracy-oriented SBRSs. It is thus worthwhile to come up with a simple yet effective design that can be used as a plugin to facilitate existing SBRSs on generating a more diversified list in the meantime preserving the recommendation accuracy. In this case, we propose an end-to-end framework applied for every existing representative (accuracy-oriented) SBRS, called diversified category-aware attentive SBRS (DCA-SBRS), to boost the performance on recommendation diversity. It consists of two novel designs: a model-agnostic diversity-oriented loss function, and a non-invasive category-aware attention mechanism. Extensive experiments on three datasets showcase that our framework helps existing SBRSs achieve extraordinary performance in terms of recommendation diversity and comprehensive performance, without significantly deteriorating recommendation accuracy compared to state-of-the-art accuracy-oriented SBRSs. △ Less

Submitted 30 March, 2024; originally announced April 2024.

Virtual Co-Pilot: Multimodal Large Language Model-enabled Quick-access Procedures for Single Pilot Operations

Authors: Fan Li, Shanshan Feng, Yuqi Yan, Ching-Hung Lee, Yew Soon Ong

Abstract: Advancements in technology, pilot shortages, and cost pressures are driving a trend towards single-pilot and even remote operations in aviation. Considering the extensive workload and huge risks associated with single-pilot operations, the development of a Virtual Co-Pilot (V-CoP) is expected to be a potential way to ensure aviation safety. This study proposes a V-CoP concept and explores how huma… ▽ More Advancements in technology, pilot shortages, and cost pressures are driving a trend towards single-pilot and even remote operations in aviation. Considering the extensive workload and huge risks associated with single-pilot operations, the development of a Virtual Co-Pilot (V-CoP) is expected to be a potential way to ensure aviation safety. This study proposes a V-CoP concept and explores how humans and virtual assistants can effectively collaborate. A preliminary case study is conducted to explore a critical role of V-CoP, namely automated quick procedures searching, using the multimodal large language model (LLM). The LLM-enabled V-CoP integrates the pilot instruction and real-time cockpit instrumental data to prompt applicable aviation manuals and operation procedures. The results showed that the LLM-enabled V-CoP achieved high accuracy in situational analysis and effective retrieval of procedure information. The results showed that the LLM-enabled V-CoP achieved high accuracy in situational analysis (90.5%) and effective retrieval of procedure information (86.5%). The proposed V-CoP is expected to provide a foundation for future virtual intelligent assistant development, improve the performance of single pilots, and reduce the risk of human errors in aviation. △ Less

Submitted 25 March, 2024; originally announced March 2024.

Comments: 10 pages,7 figures

Multi-Task Learning with Multi-Task Optimization

Authors: Lu Bai, Abhishek Gupta, Yew-Soon Ong

Abstract: Multi-task learning solves multiple correlated tasks. However, conflicts may exist between them. In such circumstances, a single solution can rarely optimize all the tasks, leading to performance trade-offs. To arrive at a set of optimized yet well-distributed models that collectively embody different trade-offs in one algorithmic pass, this paper proposes to view Pareto multi-task learning throug… ▽ More Multi-task learning solves multiple correlated tasks. However, conflicts may exist between them. In such circumstances, a single solution can rarely optimize all the tasks, leading to performance trade-offs. To arrive at a set of optimized yet well-distributed models that collectively embody different trade-offs in one algorithmic pass, this paper proposes to view Pareto multi-task learning through the lens of multi-task optimization. Multi-task learning is first cast as a multi-objective optimization problem, which is then decomposed into a diverse set of unconstrained scalar-valued subproblems. These subproblems are solved jointly using a novel multi-task gradient descent method, whose uniqueness lies in the iterative transfer of model parameters among the subproblems during the course of optimization. A theorem proving faster convergence through the inclusion of such transfers is presented. We investigate the proposed multi-task learning with multi-task optimization for solving various problem settings including image classification, scene understanding, and multi-target regression. Comprehensive experiments confirm that the proposed method significantly advances the state-of-the-art in discovering sets of Pareto-optimized models. Notably, on the large image dataset we tested on, namely NYUv2, the hypervolume convergence achieved by our method was found to be nearly two times faster than the next-best among the state-of-the-art. △ Less

Submitted 24 March, 2024; originally announced March 2024.

Precise-Physics Driven Text-to-3D Generation

Authors: Qingshan Xu, Jiao Liu, Melvin Wong, Caishun Chen, Yew-Soon Ong

Abstract: Text-to-3D generation has shown great promise in generating novel 3D content based on given text prompts. However, existing generative methods mostly focus on geometric or visual plausibility while ignoring precise physics perception for the generated 3D shapes. This greatly hinders the practicality of generated 3D shapes in real-world applications. In this work, we propose Phy3DGen, a precise-phy… ▽ More Text-to-3D generation has shown great promise in generating novel 3D content based on given text prompts. However, existing generative methods mostly focus on geometric or visual plausibility while ignoring precise physics perception for the generated 3D shapes. This greatly hinders the practicality of generated 3D shapes in real-world applications. In this work, we propose Phy3DGen, a precise-physics-driven text-to-3D generation method. By analyzing the solid mechanics of generated 3D shapes, we reveal that the 3D shapes generated by existing text-to-3D generation methods are impractical for real-world applications as the generated 3D shapes do not conform to the laws of physics. To this end, we leverage 3D diffusion models to provide 3D shape priors and design a data-driven differentiable physics layer to optimize 3D shape priors with solid mechanics. This allows us to optimize geometry efficiently and learn precise physics information about 3D shapes at the same time. Experimental results demonstrate that our method can consider both geometric plausibility and precise physics perception, further bridging 3D virtual modeling and precise physical worlds. △ Less

Submitted 19 March, 2024; originally announced March 2024.

Make Me Happier: Evoking Emotions Through Image Diffusion Models

Authors: Qing Lin, **gfeng Zhang, Yew Soon Ong, Mengmi Zhang

Abstract: Despite the rapid progress in image generation, emotional image editing remains under-explored. The semantics, context, and structure of an image can evoke emotional responses, making emotional image editing techniques valuable for various real-world applications, including treatment of psychological disorders, commercialization of products, and artistic design. For the first time, we present a no… ▽ More Despite the rapid progress in image generation, emotional image editing remains under-explored. The semantics, context, and structure of an image can evoke emotional responses, making emotional image editing techniques valuable for various real-world applications, including treatment of psychological disorders, commercialization of products, and artistic design. For the first time, we present a novel challenge of emotion-evoked image generation, aiming to synthesize images that evoke target emotions while retaining the semantics and structures of the original scenes. To address this challenge, we propose a diffusion model capable of effectively understanding and editing source images to convey desired emotions and sentiments. Moreover, due to the lack of emotion editing datasets, we provide a unique dataset consisting of 340,000 pairs of images and their emotion annotations. Furthermore, we conduct human psychophysics experiments and introduce four new evaluation metrics to systematically benchmark all the methods. Experimental results demonstrate that our method surpasses all competitive baselines. Our diffusion model is capable of identifying emotional cues from original images, editing images that elicit desired emotions, and meanwhile, preserving the semantic structure of the original images. All code, model, and dataset will be made public. △ Less

Submitted 27 May, 2024; v1 submitted 13 March, 2024; originally announced March 2024.

LIST: Learning to Index Spatio-Textual Data for Embedding based Spatial Keyword Queries

Authors: Ziqi Yin, Shanshan Feng, Shang Liu, Gao Cong, Yew Soon Ong, Bin Cui

Abstract: With the proliferation of spatio-textual data, Top-k KNN spatial keyword queries (TkQs), which return a list of objects based on a ranking function that evaluates both spatial and textual relevance, have found many real-life applications. Existing geo-textual indexes for TkQs use traditional retrieval models like BM25 to compute text relevance and usually exploit a simple linear function to comput… ▽ More With the proliferation of spatio-textual data, Top-k KNN spatial keyword queries (TkQs), which return a list of objects based on a ranking function that evaluates both spatial and textual relevance, have found many real-life applications. Existing geo-textual indexes for TkQs use traditional retrieval models like BM25 to compute text relevance and usually exploit a simple linear function to compute spatial relevance, but its effectiveness is limited. To improve effectiveness, several deep learning models have recently been proposed, but they suffer severe efficiency issues. To the best of our knowledge, there are no efficient indexes specifically designed to accelerate the top-k search process for these deep learning models. To tackle these issues, we propose a novel technique, which Learns to Index the Spatio-Textual data for answering embedding based spatial keyword queries (called LIST). LIST is featured with two novel components. Firstly, we propose a lightweight and effective relevance model that is capable of learning both textual and spatial relevance. Secondly, we introduce a novel machine learning based Approximate Nearest Neighbor Search (ANNS) index, which utilizes a new learning-to-cluster technique to group relevant queries and objects together while separating irrelevant queries and objects. Two key challenges in building an effective and efficient index are the absence of high-quality labels and unbalanced clustering results. We develop a novel pseudo-label generation technique to address the two challenges. Experimental results show that LIST significantly outperforms state-of-the-art methods on effectiveness, with improvements up to 19.21% and 12.79% in terms of NDCG@1 and Recall@10, and is three orders of magnitude faster than the most effective baseline. △ Less

Submitted 18 March, 2024; v1 submitted 12 March, 2024; originally announced March 2024.

Automating the audit of electronic invoices with a soft robot

Authors: Tian Jun Cheng, Chia Jung Chen, Yao Lin Ong, Yi Fang Yang, Guang Yih Sheu

Abstract: Taiwan's Chi Mei Medical Center has completed four challenges mentioned in published robotic process automation (RPA) studies including automating a dynamic process, designing feasible human-robot collaboration, incorporating other emerging technologies, and bringing positive business impacts. Its executives called a committee to implement the electronic invoicing. This implementation includes the… ▽ More Taiwan's Chi Mei Medical Center has completed four challenges mentioned in published robotic process automation (RPA) studies including automating a dynamic process, designing feasible human-robot collaboration, incorporating other emerging technologies, and bringing positive business impacts. Its executives called a committee to implement the electronic invoicing. This implementation includes the creation of a software robot to download automatically cloud electronic invoice (E-invoice) data from Taiwan's E-invoice platform and detect the inconsistency between them and on-premise data. This bot operates when internal auditors are off their office. They satisfied this software robot since the remaining work is only verifying the resulting inconsistency. The Chi Mei Medical Center measured the time and costs before and after adopting software robots to audit E-invoice; consequently, it welcomed more bots automating other business processes. In conclusion, integrating a software robot with other emerging technologies mitigates the possible errors provided by this bot. A good human-robot collaboration relies on the consideration of human perspective in choosing RPA tasks. Free bot creators are sufficient to verify that automating a business process using a bot is a reasonable investment. △ Less

Submitted 6 February, 2024; originally announced February 2024.

Comments: 11 pages, 6 figures, 1 table

Dynamic In-Context Learning from Nearest Neighbors for Bundle Generation

Authors: Zhu Sun, Kaidong Feng, Jie Yang, Xinghua Qu, Hui Fang, Yew-Soon Ong, Wenyuan Liu

Abstract: Product bundling has evolved into a crucial marketing strategy in e-commerce. However, current studies are limited to generating (1) fixed-size or single bundles, and most importantly, (2) bundles that do not reflect consistent user intents, thus being less intelligible or useful to users. This paper explores two interrelated tasks, i.e., personalized bundle generation and the underlying intent in… ▽ More Product bundling has evolved into a crucial marketing strategy in e-commerce. However, current studies are limited to generating (1) fixed-size or single bundles, and most importantly, (2) bundles that do not reflect consistent user intents, thus being less intelligible or useful to users. This paper explores two interrelated tasks, i.e., personalized bundle generation and the underlying intent inference based on users' interactions in a session, leveraging the logical reasoning capability of large language models. We introduce a dynamic in-context learning paradigm, which enables ChatGPT to seek tailored and dynamic lessons from closely related sessions as demonstrations while performing tasks in the target session. Specifically, it first harnesses retrieval augmented generation to identify nearest neighbor sessions for each target session. Then, proper prompts are designed to guide ChatGPT to perform the two tasks on neighbor sessions. To enhance reliability and mitigate the hallucination issue, we develop (1) a self-correction strategy to foster mutual improvement in both tasks without supervision signals; and (2) an auto-feedback mechanism to recurrently offer dynamic supervision based on the distinct mistakes made by ChatGPT on various neighbor sessions. Thus, the target session can receive customized and dynamic lessons for improved performance by observing the demonstrations of its neighbor sessions. Finally, experimental results on three real-world datasets verify the effectiveness of our methods on both tasks. Additionally, the inferred intents can prove beneficial for other intriguing downstream tasks, such as crafting appealing bundle names. △ Less

Submitted 26 December, 2023; originally announced December 2023.

Inverse Transfer Multiobjective Optimization

Authors: Jiao Liu, Abhishek Gupta, Yew-Soon Ong

Abstract: Transfer optimization enables data-efficient optimization of a target task by leveraging experiential priors from related source tasks. This is especially useful in multiobjective optimization settings where a set of trade-off solutions is sought under tight evaluation budgets. In this paper, we introduce a novel concept of inverse transfer in multiobjective optimization. Inverse transfer stands o… ▽ More Transfer optimization enables data-efficient optimization of a target task by leveraging experiential priors from related source tasks. This is especially useful in multiobjective optimization settings where a set of trade-off solutions is sought under tight evaluation budgets. In this paper, we introduce a novel concept of inverse transfer in multiobjective optimization. Inverse transfer stands out by employing probabilistic inverse models to map performance vectors in the objective space to population search distributions in task-specific decision space, facilitating knowledge transfer through objective space unification. Building upon this idea, we introduce the first Inverse Transfer Multiobjective Evolutionary Optimizer (invTrEMO). A key highlight of invTrEMO is its ability to harness the common objective functions prevalent in many application areas, even when decision spaces do not precisely align between tasks. This allows invTrEMO to uniquely and effectively utilize information from heterogeneous source tasks as well. Furthermore, invTrEMO yields high-precision inverse models as a significant byproduct, enabling the generation of tailored solutions on-demand based on user preferences. Empirical studies on multi- and many-objective benchmark problems, as well as a practical case study, showcase the faster convergence rate and modelling accuracy of the invTrEMO relative to state-of-the-art evolutionary and Bayesian optimization algorithms. The source code of the invTrEMO is made available at https://github.com/LiuJ-2023/invTrEMO. △ Less

Submitted 27 February, 2024; v1 submitted 22 December, 2023; originally announced December 2023.

PR-NeuS: A Prior-based Residual Learning Paradigm for Fast Multi-view Neural Surface Reconstruction

Authors: Jianyao Xu, Qingshan Xu, Xinyao Liao, Wanjuan Su, Chen Zhang, Yew-Soon Ong, Wenbing Tao

Abstract: Neural surfaces learning has shown impressive performance in multi-view surface reconstruction. However, most existing methods use large multilayer perceptrons (MLPs) to train their models from scratch, resulting in hours of training for a single scene. Recently, how to accelerate the neural surfaces learning has received a lot of attention and remains an open problem. In this work, we propose a p… ▽ More Neural surfaces learning has shown impressive performance in multi-view surface reconstruction. However, most existing methods use large multilayer perceptrons (MLPs) to train their models from scratch, resulting in hours of training for a single scene. Recently, how to accelerate the neural surfaces learning has received a lot of attention and remains an open problem. In this work, we propose a prior-based residual learning paradigm for fast multi-view neural surface reconstruction. This paradigm consists of two optimization stages. In the first stage, we propose to leverage generalization models to generate a basis signed distance function (SDF) field. This initial field can be quickly obtained by fusing multiple local SDF fields produced by generalization models. This provides a coarse global geometry prior. Based on this prior, in the second stage, a fast residual learning strategy based on hash-encoding networks is proposed to encode an offset SDF field for the basis SDF field. Moreover, we introduce a prior-guided sampling scheme to help the residual learning stage converge better, and thus recover finer structures. With our designed paradigm, experimental results show that our method only takes about 3 minutes to reconstruct the surface of a single scene, while achieving competitive surface quality. Our code will be released upon publication. △ Less

Submitted 18 December, 2023; originally announced December 2023.

FedCompetitors: Harmonious Collaboration in Federated Learning with Competing Participants

Authors: Shanli Tan, Hao Cheng, Xiaohu Wu, Han Yu, Tiantian He, Yew-Soon Ong, Chongjun Wang, Xiaofeng Tao

Abstract: Federated learning (FL) provides a privacy-preserving approach for collaborative training of machine learning models. Given the potential data heterogeneity, it is crucial to select appropriate collaborators for each FL participant (FL-PT) based on data complementarity. Recent studies have addressed this challenge. Similarly, it is imperative to consider the inter-individual relationships among FL… ▽ More Federated learning (FL) provides a privacy-preserving approach for collaborative training of machine learning models. Given the potential data heterogeneity, it is crucial to select appropriate collaborators for each FL participant (FL-PT) based on data complementarity. Recent studies have addressed this challenge. Similarly, it is imperative to consider the inter-individual relationships among FL-PTs where some FL-PTs engage in competition. Although FL literature has acknowledged the significance of this scenario, practical methods for establishing FL ecosystems remain largely unexplored. In this paper, we extend a principle from the balance theory, namely ``the friend of my enemy is my enemy'', to ensure the absence of conflicting interests within an FL ecosystem. The extended principle and the resulting problem are formulated via graph theory and integer linear programming. A polynomial-time algorithm is proposed to determine the collaborators of each FL-PT. The solution guarantees high scalability, allowing even competing FL-PTs to smoothly join the ecosystem without conflict of interest. The proposed framework jointly considers competition and data heterogeneity. Extensive experiments on real-world and synthetic data demonstrate its efficacy compared to five alternative approaches, and its ability to establish efficient collaboration networks among FL-PTs. △ Less

Submitted 18 December, 2023; originally announced December 2023.

Comments: Accepted to AAAI-2024

Large Language Models for Intent-Driven Session Recommendations

Authors: Zhu Sun, Hongyang Liu, Xinghua Qu, Kaidong Feng, Yan Wang, Yew-Soon Ong

Abstract: Intent-aware session recommendation (ISR) is pivotal in discerning user intents within sessions for precise predictions. Traditional approaches, however, face limitations due to their presumption of a uniform number of intents across all sessions. This assumption overlooks the dynamic nature of user sessions, where the number and type of intentions can significantly vary. In addition, these method… ▽ More Intent-aware session recommendation (ISR) is pivotal in discerning user intents within sessions for precise predictions. Traditional approaches, however, face limitations due to their presumption of a uniform number of intents across all sessions. This assumption overlooks the dynamic nature of user sessions, where the number and type of intentions can significantly vary. In addition, these methods typically operate in latent spaces, thus hinder the model's transparency.Addressing these challenges, we introduce a novel ISR approach, utilizing the advanced reasoning capabilities of large language models (LLMs). First, this approach begins by generating an initial prompt that guides LLMs to predict the next item in a session, based on the varied intents manifested in user sessions. Then, to refine this process, we introduce an innovative prompt optimization mechanism that iteratively self-reflects and adjusts prompts. Furthermore, our prompt selection module, built upon the LLMs' broad adaptability, swiftly selects the most optimized prompts across diverse domains. This new paradigm empowers LLMs to discern diverse user intents at a semantic level, leading to more accurate and interpretable session recommendations. Our extensive experiments on three real-world datasets demonstrate the effectiveness of our method, marking a significant advancement in ISR systems. △ Less

Submitted 6 December, 2023; originally announced December 2023.

Generalizable Neural Physics Solvers by Baldwinian Evolution

Authors: Jian Cheng Wong, Chin Chun Ooi, Abhishek Gupta, Pao-Hsiung Chiu, Joshua Shao Zheng Low, My Ha Dao, Yew-Soon Ong

Abstract: Physics-informed neural networks (PINNs) are at the forefront of scientific machine learning, making possible the creation of machine intelligence that is cognizant of physical laws and able to accurately simulate them. In this paper, the potential of discovering PINNs that generalize over an entire family of physics tasks is studied, for the first time, through a biological lens of the Baldwin ef… ▽ More Physics-informed neural networks (PINNs) are at the forefront of scientific machine learning, making possible the creation of machine intelligence that is cognizant of physical laws and able to accurately simulate them. In this paper, the potential of discovering PINNs that generalize over an entire family of physics tasks is studied, for the first time, through a biological lens of the Baldwin effect. Drawing inspiration from the neurodevelopment of precocial species that have evolved to learn, predict and react quickly to their environment, we envision PINNs that are pre-wired with connection strengths inducing strong biases towards efficient learning of physics. To this end, evolutionary selection pressure (guided by proficiency over a family of tasks) is coupled with lifetime learning (to specialize on a smaller subset of those tasks) to produce PINNs that demonstrate fast and physics-compliant prediction capabilities across a range of empirically challenging problem instances. The Baldwinian approach achieves an order of magnitude improvement in prediction accuracy at a fraction of the computation cost compared to state-of-the-art results with PINNs meta-learned by gradient descent. This paper marks a leap forward in the meta-learning of PINNs as generalizable physics solvers. △ Less

Submitted 5 December, 2023; originally announced December 2023.

Large Language Models as Evolutionary Optimizers

Authors: Shengcai Liu, Caishun Chen, Xinghua Qu, Ke Tang, Yew-Soon Ong

Abstract: Evolutionary algorithms (EAs) have achieved remarkable success in tackling complex combinatorial optimization problems. However, EAs often demand carefully-designed operators with the aid of domain expertise to achieve satisfactory performance. In this work, we present the first study on large language models (LLMs) as evolutionary combinatorial optimizers. The main advantage is that it requires m… ▽ More Evolutionary algorithms (EAs) have achieved remarkable success in tackling complex combinatorial optimization problems. However, EAs often demand carefully-designed operators with the aid of domain expertise to achieve satisfactory performance. In this work, we present the first study on large language models (LLMs) as evolutionary combinatorial optimizers. The main advantage is that it requires minimal domain knowledge and human efforts, as well as no additional training of the model. This approach is referred to as LLM-driven EA (LMEA). Specifically, in each generation of the evolutionary search, LMEA instructs the LLM to select parent solutions from current population, and perform crossover and mutation to generate offspring solutions. Then, LMEA evaluates these new solutions and include them into the population for the next generation. LMEA is equipped with a self-adaptation mechanism that controls the temperature of the LLM. This enables it to balance between exploration and exploitation and prevents the search from getting stuck in local optima. We investigate the power of LMEA on the classical traveling salesman problems (TSPs) widely used in combinatorial optimization research. Notably, the results show that LMEA performs competitively to traditional heuristics in finding high-quality solutions on TSP instances with up to 20 nodes. Additionally, we also study the effectiveness of LLM-driven crossover/mutation and the self-adaptation mechanism in evolutionary search. In summary, our results reveal the great potentials of LLMs as evolutionary optimizers for solving combinatorial problems. We hope our research shall inspire future explorations on LLM-driven EAs for complex optimization challenges. △ Less

Submitted 26 April, 2024; v1 submitted 29 October, 2023; originally announced October 2023.

Comments: Accepted by CEC 2024

HPCR: Holistic Proxy-based Contrastive Replay for Online Continual Learning

Authors: Huiwei Lin, Shanshan Feng, Baoquan Zhang, Xutao Li, Yew-soon Ong, Yunming Ye

Abstract: Online continual learning (OCL) aims to continuously learn new data from a single pass over the online data stream. It generally suffers from the catastrophic forgetting issue. Existing replay-based methods effectively alleviate this issue by replaying part of old data in a proxy-based or contrastive-based replay manner. In this paper, we conduct a comprehensive analysis of these two replay manner… ▽ More Online continual learning (OCL) aims to continuously learn new data from a single pass over the online data stream. It generally suffers from the catastrophic forgetting issue. Existing replay-based methods effectively alleviate this issue by replaying part of old data in a proxy-based or contrastive-based replay manner. In this paper, we conduct a comprehensive analysis of these two replay manners and find they can be complementary. Inspired by this finding, we propose a novel replay-based method called proxy-based contrastive replay (PCR), which replaces anchor-to-sample pairs with anchor-to-proxy pairs in the contrastive-based loss to alleviate the phenomenon of forgetting. Based on PCR, we further develop a more advanced method named holistic proxy-based contrastive replay (HPCR), which consists of three components. The contrastive component conditionally incorporates anchor-to-sample pairs to PCR, learning more fine-grained semantic information with a large training batch. The second is a temperature component that decouples the temperature coefficient into two parts based on their impacts on the gradient and sets different values for them to learn more novel knowledge. The third is a distillation component that constrains the learning process to keep more historical knowledge. Experiments on four datasets consistently demonstrate the superiority of HPCR over various state-of-the-art methods. △ Less

Submitted 26 September, 2023; originally announced September 2023.

Comments: 18 pages, 11 figures

MosaicFusion: Diffusion Models as Data Augmenters for Large Vocabulary Instance Segmentation

Authors: Jiahao Xie, Wei Li, Xiangtai Li, Ziwei Liu, Yew Soon Ong, Chen Change Loy

Abstract: We present MosaicFusion, a simple yet effective diffusion-based data augmentation approach for large vocabulary instance segmentation. Our method is training-free and does not rely on any label supervision. Two key designs enable us to employ an off-the-shelf text-to-image diffusion model as a useful dataset generator for object instances and mask annotations. First, we divide an image canvas into… ▽ More We present MosaicFusion, a simple yet effective diffusion-based data augmentation approach for large vocabulary instance segmentation. Our method is training-free and does not rely on any label supervision. Two key designs enable us to employ an off-the-shelf text-to-image diffusion model as a useful dataset generator for object instances and mask annotations. First, we divide an image canvas into several regions and perform a single round of diffusion process to generate multiple instances simultaneously, conditioning on different text prompts. Second, we obtain corresponding instance masks by aggregating cross-attention maps associated with object prompts across layers and diffusion time steps, followed by simple thresholding and edge-aware refinement processing. Without bells and whistles, our MosaicFusion can produce a significant amount of synthetic labeled data for both rare and novel categories. Experimental results on the challenging LVIS long-tailed and open-vocabulary benchmarks demonstrate that MosaicFusion can significantly improve the performance of existing instance segmentation models, especially for rare and novel categories. Code will be released at https://github.com/Jiahao000/MosaicFusion. △ Less

Submitted 22 September, 2023; originally announced September 2023.

Comments: GitHub: https://github.com/Jiahao000/MosaicFusion

A Maximum Force Perspective on Black Hole Thermodynamics, Quantum Pressure, and Near-Extremality

Authors: Yen Chin Ong

Abstract: I re-examined the notion of the thermodynamic force constructed from the first law of black hole thermodynamics. In general relativity, the value of the charge (or angular momentum) at which the thermodynamic force equals the conjectured maximum force $F=1/4$ is found to correspond to $Q^2/M^2=8/9$ (respectively, $a^2/M^2=8/9$), which is known in the literature to exhibit some special properties.… ▽ More I re-examined the notion of the thermodynamic force constructed from the first law of black hole thermodynamics. In general relativity, the value of the charge (or angular momentum) at which the thermodynamic force equals the conjectured maximum force $F=1/4$ is found to correspond to $Q^2/M^2=8/9$ (respectively, $a^2/M^2=8/9$), which is known in the literature to exhibit some special properties. This provides a possible characterization of near-extremality. In addition, taking the maximum force conjecture seriously amounts to introducing a pressure term in the first law of black hole thermodynamics. This resolves the factor of two problem between the proposed maximum value $F=1/4$ and the thermodynamic force of Schwarzschild spacetime $F=1/2$. Surprisingly it also provides another indication for the instability of the inner horizon. For a Schwarzschild black hole, under some reasonable assumptions, this pressure can be interpreted as being induced by the quantum fluctuation of the horizon position, effectively giving rise to a diffused "shell" of characteristic width $\sqrt{M}$. The maximum force can therefore, in some contexts, be associated with inherently quantum phenomena. Some implications are discussed as more questions are raised. △ Less

Submitted 8 December, 2023; v1 submitted 8 September, 2023; originally announced September 2023.

Comments: Version to match the published article; expanded discussion

Journal ref: Eur.Phys.J.C 83 (2023) 11, 1068

GlassMessaging: Supporting Messaging Needs During Daily Activities Using OST-HMDs

Authors: Nuwan Janaka, Jie Gao, Lin Zhu, Shengdong Zhao, Lan Lyu, Peisen Xu, Maximilian Nabokow, Silang Wang, Yanch Ong

Abstract: The act of communicating with others during routine daily tasks is both common and intuitive for individuals. However, the hands- and eyes-engaged nature of present digital messaging applications makes it difficult to message someone amidst such activities. We introduce GlassMessaging, a messaging application designed for Optical See-Through Head-Mounted Displays (OST-HMDs). It facilitates messagi… ▽ More The act of communicating with others during routine daily tasks is both common and intuitive for individuals. However, the hands- and eyes-engaged nature of present digital messaging applications makes it difficult to message someone amidst such activities. We introduce GlassMessaging, a messaging application designed for Optical See-Through Head-Mounted Displays (OST-HMDs). It facilitates messaging through both voice and manual inputs, catering to situations where hands and eyes are preoccupied. GlassMessaging was iteratively developed through a formative study identifying current messaging behaviors and challenges in common multitasking with messaging scenarios △ Less

Submitted 30 August, 2023; originally announced August 2023.

Comments: 6 pages, 1 figure, ACM SUI 2023 (Demo)

Journal ref: SUI 2023

Neural Influence Estimator: Towards Real-time Solutions to Influence Blocking Maximization

Authors: Wenjie Chen, Shengcai Liu, Yew-Soon Ong, Ke Tang

Abstract: Real-time solutions to the influence blocking maximization (IBM) problems are crucial for promptly containing the spread of misinformation. However, achieving this goal is non-trivial, mainly because assessing the blocked influence of an IBM problem solution typically requires plenty of expensive Monte Carlo simulations (MCSs). Although several approaches have been proposed to enhance efficiency,… ▽ More Real-time solutions to the influence blocking maximization (IBM) problems are crucial for promptly containing the spread of misinformation. However, achieving this goal is non-trivial, mainly because assessing the blocked influence of an IBM problem solution typically requires plenty of expensive Monte Carlo simulations (MCSs). Although several approaches have been proposed to enhance efficiency, they still fail to achieve real-time solutions to IBM problems of practical scales. This work presents a novel approach that enables solving IBM problems with hundreds of thousands of nodes and edges in seconds. The key idea is to construct a fast-to-evaluate surrogate model, called neural influence estimator (NIE), as a substitute for the time-intensive MCSs. To this end, a learning problem is formulated to build the NIE that takes the false-and-true information instance as input, extracts features describing the topology and inter-relationship between two seed sets, and predicts the blocked influence. A well-trained NIE can generalize across different IBM problems defined on a social network, and can be readily combined with existing IBM optimization algorithms such as the greedy algorithm. The experiments on 25 IBM problems with up to millions of edges show that the NIE-based optimization method can be up to four orders of magnitude faster than MCSs-based optimization method to achieve the same solution quality. Moreover, given a real-time constraint of one minute, the NIE-based method can solve IBM problems with up to hundreds of thousands of nodes, which is at least one order of magnitude larger than what can be solved by existing methods. △ Less

Submitted 27 August, 2023; originally announced August 2023.

Meta-learning enhanced next POI recommendation by leveraging check-ins from auxiliary cities

Authors: **ze Wang, Lu Zhang, Zhu Sun, Yew-Soon Ong

Abstract: Most existing point-of-interest (POI) recommenders aim to capture user preference by employing city-level user historical check-ins, thus facilitating users' exploration of the city. However, the scarcity of city-level user check-ins brings a significant challenge to user preference learning. Although prior studies attempt to mitigate this challenge by exploiting various context information, e.g.,… ▽ More Most existing point-of-interest (POI) recommenders aim to capture user preference by employing city-level user historical check-ins, thus facilitating users' exploration of the city. However, the scarcity of city-level user check-ins brings a significant challenge to user preference learning. Although prior studies attempt to mitigate this challenge by exploiting various context information, e.g., spatio-temporal information, they ignore to transfer the knowledge (i.e., common behavioral pattern) from other relevant cities (i.e., auxiliary cities). In this paper, we investigate the effect of knowledge distilled from auxiliary cities and thus propose a novel Meta-learning Enhanced next POI Recommendation framework (MERec). The MERec leverages the correlation of check-in behaviors among various cities into the meta-learning paradigm to help infer user preference in the target city, by holding the principle of "paying more attention to more correlated knowledge". Particularly, a city-level correlation strategy is devised to attentively capture common patterns among cities, so as to transfer more relevant knowledge from more correlated cities. Extensive experiments verify the superiority of the proposed MERec against state-of-the-art algorithms. △ Less

Submitted 18 August, 2023; originally announced August 2023.

Chance-Constrained Multiple-Choice Knapsack Problem: Model, Algorithms, and Applications

Authors: Xuanfeng Li, Shengcai Liu, ** Wang, Xiao Chen, Yew-Soon Ong, Ke Tang

Abstract: The multiple-choice knapsack problem (MCKP) is a classic NP-hard combinatorial optimization problem. Motivated by several significant real-world applications, this work investigates a novel variant of MCKP called chance-constrained multiple-choice knapsack problem (CCMCKP), where the item weights are random variables. In particular, we focus on the practical scenario of CCMCKP, where the probabili… ▽ More The multiple-choice knapsack problem (MCKP) is a classic NP-hard combinatorial optimization problem. Motivated by several significant real-world applications, this work investigates a novel variant of MCKP called chance-constrained multiple-choice knapsack problem (CCMCKP), where the item weights are random variables. In particular, we focus on the practical scenario of CCMCKP, where the probability distributions of random weights are unknown but only sample data is available. We first present the problem formulation of CCMCKP and then establish two benchmark sets. The first set contains synthetic instances and the second set is devised to simulate a real-world application scenario of a certain telecommunication company. To solve CCMCKP, we propose a data-driven adaptive local search (DDALS) algorithm. The main novelty of DDALS lies in its data-driven solution evaluation approach that can effectively handle unknown probability distributions of item weights. Moreover, in cases with unknown distributions, high intensity of chance constraints, limited amount of sample data and large-scale problems, it still exhibits good performance. Experimental results demonstrate the superiority of DDALS over other baselines on the two benchmarks. Additionally, ablation studies confirm the effectiveness of each component of the algorithm. Finally, DDALS can serve as the baseline for future research, and the benchmark sets are open-sourced to further promote research on this challenging problem. △ Less

Submitted 14 December, 2023; v1 submitted 26 June, 2023; originally announced June 2023.

Towards Building Voice-based Conversational Recommender Systems: Datasets, Potential Solutions, and Prospects

Authors: Xinghua Qu, Hongyang Liu, Zhu Sun, Xiang Yin, Yew Soon Ong, Lu Lu, Zejun Ma

Abstract: Conversational recommender systems (CRSs) have become crucial emerging research topics in the field of RSs, thanks to their natural advantages of explicitly acquiring user preferences via interactive conversations and revealing the reasons behind recommendations. However, the majority of current CRSs are text-based, which is less user-friendly and may pose challenges for certain users, such as tho… ▽ More Conversational recommender systems (CRSs) have become crucial emerging research topics in the field of RSs, thanks to their natural advantages of explicitly acquiring user preferences via interactive conversations and revealing the reasons behind recommendations. However, the majority of current CRSs are text-based, which is less user-friendly and may pose challenges for certain users, such as those with visual impairments or limited writing and reading abilities. Therefore, for the first time, this paper investigates the potential of voice-based CRS (VCRSs) to revolutionize the way users interact with RSs in a natural, intuitive, convenient, and accessible fashion. To support such studies, we create two VCRSs benchmark datasets in the e-commerce and movie domains, after realizing the lack of such datasets through an exhaustive literature review. Specifically, we first empirically verify the benefits and necessity of creating such datasets. Thereafter, we convert the user-item interactions to text-based conversations through the ChatGPT-driven prompts for generating diverse and natural templates, and then synthesize the corresponding audios via the text-to-speech model. Meanwhile, a number of strategies are delicately designed to ensure the naturalness and high quality of voice conversations. On this basis, we further explore the potential solutions and point out possible directions to build end-to-end VCRSs by seamlessly extracting and integrating voice-based inputs, thus delivering performance-enhanced, self-explainable, and user-friendly VCRSs. Our study aims to establish the foundation and motivate further pioneering research in the emerging field of VCRSs. This aligns with the principles of explainable AI and AI for social good, viz., utilizing technology's potential to create a fair, sustainable, and just world. △ Less

Submitted 13 June, 2023; originally announced June 2023.

Comments: Accepted by SIGIR 2023 Resource Track

Prompt Evolution for Generative AI: A Classifier-Guided Approach

Authors: Melvin Wong, Yew-Soon Ong, Abhishek Gupta, Kavitesh K. Bali, Caishun Chen

Abstract: Synthesis of digital artifacts conditioned on user prompts has become an important paradigm facilitating an explosion of use cases with generative AI. However, such models often fail to connect the generated outputs and desired target concepts/preferences implied by the prompts. Current research addressing this limitation has largely focused on enhancing the prompts before output generation or imp… ▽ More Synthesis of digital artifacts conditioned on user prompts has become an important paradigm facilitating an explosion of use cases with generative AI. However, such models often fail to connect the generated outputs and desired target concepts/preferences implied by the prompts. Current research addressing this limitation has largely focused on enhancing the prompts before output generation or improving the model's performance up front. In contrast, this paper conceptualizes prompt evolution, imparting evolutionary selection pressure and variation during the generative process to produce multiple outputs that satisfy the target concepts/preferences better. We propose a multi-objective instantiation of this broader idea that uses a multi-label image classifier-guided approach. The predicted labels from the classifiers serve as multiple objectives to optimize, with the aim of producing diversified images that meet user preferences. A novelty of our evolutionary algorithm is that the pre-trained generative model gives us implicit mutation operations, leveraging the model's stochastic generative capability to automate the creation of Pareto-optimized images more faithful to user preferences. △ Less

Submitted 24 May, 2023; originally announced May 2023.

Comments: To appear in Proceedings of the 2023 IEEE Conference on Artificial Intelligence (CAI'23)

ACM Class: I.2

Large Language Models can be Guided to Evade AI-Generated Text Detection

Authors: Ning Lu, Shengcai Liu, Rui He, Qi Wang, Yew-Soon Ong, Ke Tang

Abstract: Large language models (LLMs) have shown remarkable performance in various tasks and have been extensively utilized by the public. However, the increasing concerns regarding the misuse of LLMs, such as plagiarism and spamming, have led to the development of multiple detectors, including fine-tuned classifiers and statistical methods. In this study, we equip LLMs with prompts, rather than relying on… ▽ More Large language models (LLMs) have shown remarkable performance in various tasks and have been extensively utilized by the public. However, the increasing concerns regarding the misuse of LLMs, such as plagiarism and spamming, have led to the development of multiple detectors, including fine-tuned classifiers and statistical methods. In this study, we equip LLMs with prompts, rather than relying on an external paraphraser, to evaluate the vulnerability of these detectors. We propose a novel Substitution-based In-Context example Optimization method (SICO) to automatically construct prompts for evading the detectors. SICO is cost-efficient as it requires only 40 human-written examples and a limited number of LLM inferences to generate a prompt. Moreover, once a task-specific prompt has been constructed, it can be universally used against a wide range of detectors. Extensive experiments across three real-world tasks demonstrate that SICO significantly outperforms the paraphraser baselines and enables GPT-3.5 to successfully evade six detectors, decreasing their AUC by 0.5 on average. Furthermore, a comprehensive human evaluation show that the SICO-generated text achieves human-level readability and task completion rates, while preserving high imperceptibility. Finally, we propose an ensemble approach to enhance the robustness of detectors against SICO attack. The code is publicly available at https://github.com/ColinLu50/Evade-GPT-Detector. △ Less

Submitted 15 May, 2024; v1 submitted 18 May, 2023; originally announced May 2023.

Comments: TMLR camera ready

Bayesian Federated Learning: A Survey

Authors: Longbing Cao, Hui Chen, Xuhui Fan, Joao Gama, Yew-Soon Ong, Vipin Kumar

Abstract: Federated learning (FL) demonstrates its advantages in integrating distributed infrastructure, communication, computing and learning in a privacy-preserving manner. However, the robustness and capabilities of existing FL methods are challenged by limited and dynamic data and conditions, complexities including heterogeneities and uncertainties, and analytical explainability. Bayesian federated lear… ▽ More Federated learning (FL) demonstrates its advantages in integrating distributed infrastructure, communication, computing and learning in a privacy-preserving manner. However, the robustness and capabilities of existing FL methods are challenged by limited and dynamic data and conditions, complexities including heterogeneities and uncertainties, and analytical explainability. Bayesian federated learning (BFL) has emerged as a promising approach to address these issues. This survey presents a critical overview of BFL, including its basic concepts, its relations to Bayesian learning in the context of FL, and a taxonomy of BFL from both Bayesian and federated perspectives. We categorize and discuss client- and server-side and FL-based BFL methods and their pros and cons. The limitations of the existing BFL methods and the future directions of BFL research further address the intricate requirements of real-life FL applications. △ Less

Submitted 25 April, 2023; originally announced April 2023.

Comments: Accepted by IJCAI 2023 Survey Track, copyright is owned to IJCAI

Intrinsically episodic Antarctic shelf intrusions of circumpolar deep water via canyons

Authors: Ellie Q. Y. Ong, Edward Doddridge, Navid C. Constantinou, Andrew McC. Hogg, Matthew H. England

Abstract: The structure of the Antarctic Slope Current at the continental shelf is crucial in governing the poleward transport of warm water. Canyons on the continental slope may provide a pathway for warm water to cross the slope current and intrude onto the continental shelf underneath ice shelves, which can increase rates of ice shelf melting, leading to reduced buttressing of ice shelves, accelerating g… ▽ More The structure of the Antarctic Slope Current at the continental shelf is crucial in governing the poleward transport of warm water. Canyons on the continental slope may provide a pathway for warm water to cross the slope current and intrude onto the continental shelf underneath ice shelves, which can increase rates of ice shelf melting, leading to reduced buttressing of ice shelves, accelerating glacial flow and hence increased sea level rise. Observations and modelling studies of the Antarctic Slope Current and cross-shelf warm water intrusions are limited, particularly in the East Antarctica region. To explore this topic, an idealised configuration of the Antarctic Slope Current is developed, using an eddy-resolving isopycnal model that emulates the dynamics and topography of the East Antarctic sector. Warm water intrusions via canyons are found to occur in discrete episodes of large onshore flow induced by eddies, even in the absence of any temporal variability in external forcings, demonstrating the intrinsic nature of these intrusions to the slope current system. Canyon width is found to play a key role in modulating cross-shelf exchanges; warm water transport through narrower canyons is more irregular than transport through wider canyons. The intrinsically episodic cross-shelf transport is found to be driven by feedbacks between wind energy input and eddy generation in the Antarctic Slope Current. Improved understanding of the intrinsic variability of warm water intrusions can help guide future observational and modelling studies in the analysis of eddy impacts on Antarctic shelf circulation. △ Less

Submitted 7 March, 2024; v1 submitted 25 April, 2023; originally announced April 2023.

Incorporating Experts' Judgment into Machine Learning Models

Authors: Hogun Park, Aly Megahed, Peifeng Yin, Yuya Ong, Pravar Mahajan, Pei Guo

Abstract: Machine learning (ML) models have been quite successful in predicting outcomes in many applications. However, in some cases, domain experts might have a judgment about the expected outcome that might conflict with the prediction of ML models. One main reason for this is that the training data might not be totally representative of the population. In this paper, we present a novel framework that ai… ▽ More Machine learning (ML) models have been quite successful in predicting outcomes in many applications. However, in some cases, domain experts might have a judgment about the expected outcome that might conflict with the prediction of ML models. One main reason for this is that the training data might not be totally representative of the population. In this paper, we present a novel framework that aims at leveraging experts' judgment to mitigate the conflict. The underlying idea behind our framework is that we first determine, using a generative adversarial network, the degree of representation of an unlabeled data point in the training data. Then, based on such degree, we correct the \textcolor{black}{machine learning} model's prediction by incorporating the experts' judgment into it, where the higher that aforementioned degree of representation, the less the weight we put on the expert intuition that we add to our corrected output, and vice-versa. We perform multiple numerical experiments on synthetic data as well as two real-world case studies (one from the IT services industry and the other from the financial industry). All results show the effectiveness of our framework; it yields much higher closeness to the experts' judgment with minimal sacrifice in the prediction accuracy, when compared to multiple baseline methods. We also develop a new evaluation metric that combines prediction accuracy with the closeness to experts' judgment. Our framework yields statistically significant results when evaluated on that metric. △ Less

Submitted 29 April, 2023; v1 submitted 24 April, 2023; originally announced April 2023.

Comments: Accepted to Expert Systems with Applications Journal, 2023

Some Symmetry and Duality Theorems on Multiple Zeta(-star) Values

Authors: Kwang-Wu Chen, Minking Eie, Yao Lin Ong

Abstract: In this paper, we provide a symmetric formula and a duality formula relating multiple zeta values and zeta-star values. Leveraging Zagier's formula for computing $ζ^\star(\{2\}^p,3,\{2\}^q)$, we employ our theorems to establish a formula for computing $ζ^\star(\{2\}^p,1,\{2\}^q)$ for any positive integers $p$ and $q$, along with other formulas of interest. In this paper, we provide a symmetric formula and a duality formula relating multiple zeta values and zeta-star values. Leveraging Zagier's formula for computing $ζ^\star(\{2\}^p,3,\{2\}^q)$, we employ our theorems to establish a formula for computing $ζ^\star(\{2\}^p,1,\{2\}^q)$ for any positive integers $p$ and $q$, along with other formulas of interest. △ Less

Submitted 18 April, 2023; originally announced April 2023.

Comments: 14 pages. I would appreciate any constructive feedback and alternative perspectives on my paper. Thank you!

MSC Class: 11M32; Secondary: 05A15; 33B15

Unfolded Self-Reconstruction LSH: Towards Machine Unlearning in Approximate Nearest Neighbour Search

Authors: Kim Yong Tan, Yueming Lyu, Yew Soon Ong, Ivor W. Tsang

Abstract: Approximate nearest neighbour (ANN) search is an essential component of search engines, recommendation systems, etc. Many recent works focus on learning-based data-distribution-dependent hashing and achieve good retrieval performance. However, due to increasing demand for users' privacy and security, we often need to remove users' data information from Machine Learning (ML) models to satisfy speci… ▽ More Approximate nearest neighbour (ANN) search is an essential component of search engines, recommendation systems, etc. Many recent works focus on learning-based data-distribution-dependent hashing and achieve good retrieval performance. However, due to increasing demand for users' privacy and security, we often need to remove users' data information from Machine Learning (ML) models to satisfy specific privacy and security requirements. This need requires the ANN search algorithm to support fast online data deletion and insertion. Current learning-based hashing methods need retraining the hash function, which is prohibitable due to the vast time-cost of large-scale data. To address this problem, we propose a novel data-dependent hashing method named unfolded self-reconstruction locality-sensitive hashing (USR-LSH). Our USR-LSH unfolded the optimization update for instance-wise data reconstruction, which is better for preserving data information than data-independent LSH. Moreover, our USR-LSH supports fast online data deletion and insertion without retraining. To the best of our knowledge, we are the first to address the machine unlearning of retrieval problems. Empirically, we demonstrate that USR-LSH outperforms the state-of-the-art data-distribution-independent LSH in ANN tasks in terms of precision and recall. We also show that USR-LSH has significantly faster data deletion and insertion time than learning-based data-dependent hashing. △ Less

Submitted 6 April, 2023; v1 submitted 5 April, 2023; originally announced April 2023.

Comments: correct author's name typo

A Critique on Some Aspects of GUP Effective Metric

Authors: Yen Chin Ong

Abstract: The generalized uncertainty principle (GUP) is a gravitational correction of Heisenberg's uncertainty principle, which allows us to probe some features of quantum gravity even without the full theory. We are used to working with metric tensors in general relativity; they are convenient to have available when we wish to calculate physical quantities like Hawking temperature and black hole shadow. V… ▽ More The generalized uncertainty principle (GUP) is a gravitational correction of Heisenberg's uncertainty principle, which allows us to probe some features of quantum gravity even without the full theory. We are used to working with metric tensors in general relativity; they are convenient to have available when we wish to calculate physical quantities like Hawking temperature and black hole shadow. Various authors have tried to incorporate GUP into an effective metric that allows such calculations. In this note, I point out that some of these results are not correct due to working with series truncation rather than the full GUP-corrected expressions. Perhaps more importantly, we rely too much on heuristic arguments and lack a guiding principle in constructing the correct effective metric. △ Less

Submitted 19 March, 2023; originally announced March 2023.

Journal ref: Eur. Phys. J. C 83 (2023) 209

Policy Dispersion in Non-Markovian Environment

Authors: Bohao Qu, Xiaofeng Cao, Jielong Yang, Hechang Chen, Chang Yi, Ivor W. Tsang, Yew-Soon Ong

Abstract: Markov Decision Process (MDP) presents a mathematical framework to formulate the learning processes of agents in reinforcement learning. MDP is limited by the Markovian assumption that a reward only depends on the immediate state and action. However, a reward sometimes depends on the history of states and actions, which may result in the decision process in a non-Markovian environment. In such env… ▽ More Markov Decision Process (MDP) presents a mathematical framework to formulate the learning processes of agents in reinforcement learning. MDP is limited by the Markovian assumption that a reward only depends on the immediate state and action. However, a reward sometimes depends on the history of states and actions, which may result in the decision process in a non-Markovian environment. In such environments, agents receive rewards via temporally-extended behaviors sparsely, and the learned policies may be similar. This leads the agents acquired with similar policies generally overfit to the given task and can not quickly adapt to perturbations of environments. To resolve this problem, this paper tries to learn the diverse policies from the history of state-action pairs under a non-Markovian environment, in which a policy dispersion scheme is designed for seeking diverse policy representation. Specifically, we first adopt a transformer-based method to learn policy embeddings. Then, we stack the policy embeddings to construct a dispersion matrix to induce a set of diverse policies. Finally, we prove that if the dispersion matrix is positive definite, the dispersed embeddings can effectively enlarge the disagreements across policies, yielding a diverse expression for the original policy embedding distribution. Experimental results show that this dispersion scheme can obtain more expressive diverse policies, which then derive more robust performance than recent learning baselines under various learning environments. △ Less

Submitted 2 June, 2024; v1 submitted 28 February, 2023; originally announced February 2023.

Comments: In further research, we found that the core content of the paper requires significant modification and that the entire paper needs to be restructured. To enhance the scientific quality and contributions of the paper, we have decided to resubmit it after completing the necessary revisions and improvements

LSA-PINN: Linear Boundary Connectivity Loss for Solving PDEs on Complex Geometry

Authors: Jian Cheng Wong, Pao-Hsiung Chiu, Chinchun Ooi, My Ha Dao, Yew-Soon Ong

Abstract: We present a novel loss formulation for efficient learning of complex dynamics from governing physics, typically described by partial differential equations (PDEs), using physics-informed neural networks (PINNs). In our experiments, existing versions of PINNs are seen to learn poorly in many problems, especially for complex geometries, as it becomes increasingly difficult to establish appropriate… ▽ More We present a novel loss formulation for efficient learning of complex dynamics from governing physics, typically described by partial differential equations (PDEs), using physics-informed neural networks (PINNs). In our experiments, existing versions of PINNs are seen to learn poorly in many problems, especially for complex geometries, as it becomes increasingly difficult to establish appropriate sampling strategy at the near boundary region. Overly dense sampling can adversely impede training convergence if the local gradient behaviors are too complex to be adequately modelled by PINNs. On the other hand, if the samples are too sparse, existing PINNs tend to overfit the near boundary region, leading to incorrect solution. To prevent such issues, we propose a new Boundary Connectivity (BCXN) loss function which provides linear local structure approximation (LSA) to the gradient behaviors at the boundary for PINN. Our BCXN-loss implicitly imposes local structure during training, thus facilitating fast physics-informed learning across entire problem domains with order of magnitude sparser training samples. This LSA-PINN method shows a few orders of magnitude smaller errors than existing methods in terms of the standard L2-norm metric, while using dramatically fewer training samples and iterations. Our proposed LSA-PINN does not pose any requirement on the differentiable property of the networks, and we demonstrate its benefits and ease of implementation on both multi-layer perceptron and convolutional neural network versions as commonly used in current PINN literature. △ Less

Submitted 2 March, 2023; v1 submitted 2 February, 2023; originally announced February 2023.

Comments: 11 pages, 7 figures

Journal ref: 2023 International Joint Conference on Neural Networks (IJCNN)

Electron as a Tiny Mirror: Radiation From a Worldline With Asymptotic Inertia

Authors: Michael R. R. Good, Yen Chin Ong

Abstract: We present a moving mirror analog of the electron, whose worldline possesses asymptotic constant velocity with corresponding beta Bogolubov coefficients that are consistent with finite total emitted energy. Furthermore, the quantum analog model is in agreement with the total energy obtained by integrating the classical Larmor power. We present a moving mirror analog of the electron, whose worldline possesses asymptotic constant velocity with corresponding beta Bogolubov coefficients that are consistent with finite total emitted energy. Furthermore, the quantum analog model is in agreement with the total energy obtained by integrating the classical Larmor power. △ Less

Submitted 1 February, 2023; originally announced February 2023.

Comments: 4 pages, 4 figures

Journal ref: Physics 2023, 5(1), 131-139