Showing 1–50 of 54 results for author: Doan, K

Non-Cooperative Backdoor Attacks in Federated Learning: A New Threat Landscape

Authors: Tuan Nguyen, Dung Thuy Nguyen, Khoa D Doan, Kok-Seng Wong

Abstract: Despite the promise of Federated Learning (FL) for privacy-preserving model training on distributed data, it remains susceptible to backdoor attacks. These attacks manipulate models by embedding triggers (specific input patterns) in the training data, forcing misclassification as predefined classes during deployment. Traditional single-trigger attacks and recent work on cooperative multiple-trigge… ▽ More Despite the promise of Federated Learning (FL) for privacy-preserving model training on distributed data, it remains susceptible to backdoor attacks. These attacks manipulate models by embedding triggers (specific input patterns) in the training data, forcing misclassification as predefined classes during deployment. Traditional single-trigger attacks and recent work on cooperative multiple-trigger attacks, where clients collaborate, highlight limitations in attack realism due to coordination requirements. We investigate a more alarming scenario: non-cooperative multiple-trigger attacks. Here, independent adversaries introduce distinct triggers targeting unique classes. These parallel attacks exploit FL's decentralized nature, making detection difficult. Our experiments demonstrate the alarming vulnerability of FL to such attacks, where individual backdoors can be successfully learned without impacting the main task. This research emphasizes the critical need for robust defenses against diverse backdoor attacks in the evolving FL landscape. While our focus is on empirical analysis, we believe it can guide backdoor research toward more realistic settings, highlighting the crucial role of FL in building robust defenses against diverse backdoor threats. The code is available at \url{https://anonymous.4open.science/r/nba-980F/}. △ Less

Submitted 5 July, 2024; originally announced July 2024.

Venomancer: Towards Imperceptible and Target-on-Demand Backdoor Attacks in Federated Learning

Authors: Son Nguyen, Thinh Nguyen, Khoa D Doan, Kok-Seng Wong

Abstract: Federated Learning (FL) is a distributed machine learning approach that maintains data privacy by training on decentralized data sources. Similar to centralized machine learning, FL is also susceptible to backdoor attacks, where an attacker can compromise some clients by injecting a backdoor trigger into local models of those clients, leading to the global model's behavior being manipulated as des… ▽ More Federated Learning (FL) is a distributed machine learning approach that maintains data privacy by training on decentralized data sources. Similar to centralized machine learning, FL is also susceptible to backdoor attacks, where an attacker can compromise some clients by injecting a backdoor trigger into local models of those clients, leading to the global model's behavior being manipulated as desired by the attacker. Most backdoor attacks in FL assume a predefined target class and require control over a large number of clients or knowledge of benign clients' information. Furthermore, they are not imperceptible and are easily detected by human inspection due to clear artifacts left on the poison data. To overcome these challenges, we propose Venomancer, an effective backdoor attack that is imperceptible and allows target-on-demand. Specifically, imperceptibility is achieved by using a visual loss function to make the poison data visually indistinguishable from the original data. Target-on-demand property allows the attacker to choose arbitrary target classes via conditional adversarial training. Additionally, experiments showed that the method is robust against state-of-the-art defenses such as Norm Clip**, Weak DP, Krum, Multi-Krum, RLR, FedRAD, Deepsight, and RFLBAT. The source code is available at https://github.com/nguyenhongson1902/Venomancer. △ Less

Submitted 11 July, 2024; v1 submitted 3 July, 2024; originally announced July 2024.

Forget but Recall: Incremental Latent Rectification in Continual Learning

Authors: Nghia D. Nguyen, Hieu Trung Nguyen, Ang Li, Hoang Pham, Viet Anh Nguyen, Khoa D. Doan

Abstract: Intrinsic capability to continuously learn a changing data stream is a desideratum of deep neural networks (DNNs). However, current DNNs suffer from catastrophic forgetting, which hinders remembering past knowledge. To mitigate this issue, existing Continual Learning (CL) approaches either retain exemplars for replay, regularize learning, or allocate dedicated capacity for new tasks. This paper in… ▽ More Intrinsic capability to continuously learn a changing data stream is a desideratum of deep neural networks (DNNs). However, current DNNs suffer from catastrophic forgetting, which hinders remembering past knowledge. To mitigate this issue, existing Continual Learning (CL) approaches either retain exemplars for replay, regularize learning, or allocate dedicated capacity for new tasks. This paper investigates an unexplored CL direction for incremental learning called Incremental Latent Rectification or ILR. In a nutshell, ILR learns to propagate with correction (or rectify) the representation from the current trained DNN backward to the representation space of the old task, where performing predictive decisions is easier. This rectification process only employs a chain of small representation map** networks, called rectifier units. Empirical experiments on several continual learning benchmarks, including CIFAR10, CIFAR100, and Tiny ImageNet, demonstrate the effectiveness and potential of this novel CL direction compared to existing representative CL methods. △ Less

Submitted 25 June, 2024; originally announced June 2024.

Towards Zero-Shot Text-To-Speech for Arabic Dialects

Authors: Khai Duy Doan, Abdul Waheed, Muhammad Abdul-Mageed

Abstract: Zero-shot multi-speaker text-to-speech (ZS-TTS) systems have advanced for English, however, it still lags behind due to insufficient resources. We address this gap for Arabic, a language of more than 450 million native speakers, by first adapting a sizeable existing dataset to suit the needs of speech synthesis. Additionally, we employ a set of Arabic dialect identification models to explore the i… ▽ More Zero-shot multi-speaker text-to-speech (ZS-TTS) systems have advanced for English, however, it still lags behind due to insufficient resources. We address this gap for Arabic, a language of more than 450 million native speakers, by first adapting a sizeable existing dataset to suit the needs of speech synthesis. Additionally, we employ a set of Arabic dialect identification models to explore the impact of pre-defined dialect labels on improving the ZS-TTS model in a multi-dialect setting. Subsequently, we fine-tune the XTTS\footnote{https://docs.coqui.ai/en/latest/models/xtts.html}\footnote{https://medium.com/machine-learns/xtts-v2-new-version-of-the-open-source-text-to-speech-model-af73914db81f}\footnote{https://medium.com/@erogol/xtts-v1-techincal-notes-eb83ff05bdc} model, an open-source architecture. We then evaluate our models on a dataset comprising 31 unseen speakers and an in-house dialectal dataset. Our automated and human evaluation results show convincing performance while capable of generating dialectal speech. Our study highlights significant potential for improvements in this emerging area of research in Arabic. △ Less

Submitted 7 July, 2024; v1 submitted 24 June, 2024; originally announced June 2024.

Composite Concept Extraction through Backdooring

Authors: Banibrata Ghosh, Haripriya Harikumar, Khoa D Doan, Svetha Venkatesh, Santu Rana

Abstract: Learning composite concepts, such as \textquotedbl red car\textquotedbl , from individual examples -- like a white car representing the concept of \textquotedbl car\textquotedbl{} and a red strawberry representing the concept of \textquotedbl red\textquotedbl -- is inherently challenging. This paper introduces a novel method called Composite Concept Extractor (CoCE), which leverages techniques fro… ▽ More Learning composite concepts, such as \textquotedbl red car\textquotedbl , from individual examples -- like a white car representing the concept of \textquotedbl car\textquotedbl{} and a red strawberry representing the concept of \textquotedbl red\textquotedbl -- is inherently challenging. This paper introduces a novel method called Composite Concept Extractor (CoCE), which leverages techniques from traditional backdoor attacks to learn these composite concepts in a zero-shot setting, requiring only examples of individual concepts. By repurposing the trigger-based model backdooring mechanism, we create a strategic distortion in the manifold of the target object (e.g., \textquotedbl car\textquotedbl ) induced by example objects with the target property (e.g., \textquotedbl red\textquotedbl ) from objects \textquotedbl red strawberry\textquotedbl , ensuring the distortion selectively affects the target objects with the target property. Contrastive learning is then employed to further refine this distortion, and a method is formulated for detecting objects that are influenced by the distortion. Extensive experiments with in-depth analysis across different datasets demonstrate the utility and applicability of our proposed approach. △ Less

Submitted 21 June, 2024; v1 submitted 19 June, 2024; originally announced June 2024.

Cold-start Recommendation by Personalized Embedding Region Elicitation

Authors: Hieu Trung Nguyen, Duy Nguyen, Khoa Doan, Viet Anh Nguyen

Abstract: Rating elicitation is a success element for recommender systems to perform well at cold-starting, in which the systems need to recommend items to a newly arrived user with no prior knowledge about the user's preference. Existing elicitation methods employ a fixed set of items to learn the user's preference and then infer the users' preferences on the remaining items. Using a fixed seed set can lim… ▽ More Rating elicitation is a success element for recommender systems to perform well at cold-starting, in which the systems need to recommend items to a newly arrived user with no prior knowledge about the user's preference. Existing elicitation methods employ a fixed set of items to learn the user's preference and then infer the users' preferences on the remaining items. Using a fixed seed set can limit the performance of the recommendation system since the seed set is unlikely optimal for all new users with potentially diverse preferences. This paper addresses this challenge using a 2-phase, personalized elicitation scheme. First, the elicitation scheme asks users to rate a small set of popular items in a ``burn-in'' phase. Second, it sequentially asks the user to rate adaptive items to refine the preference and the user's representation. Throughout the process, the system represents the user's embedding value not by a point estimate but by a region estimate. The value of information obtained by asking the user's rating on an item is quantified by the distance from the region center embedding space that contains with high confidence the true embedding value of the user. Finally, the recommendations are successively generated by considering the preference region of the user. We show that each subproblem in the elicitation scheme can be efficiently implemented. Further, we empirically demonstrate the effectiveness of the proposed method against existing rating-elicitation methods on several prominent datasets. △ Less

Submitted 3 June, 2024; originally announced June 2024.

Comments: Accepted at UAI 2024

Removing Undesirable Concepts in Text-to-Image Generative Models with Learnable Prompts

Authors: Anh Bui, Khanh Doan, Trung Le, Paul Montague, Tamas Abraham, Dinh Phung

Abstract: Generative models have demonstrated remarkable potential in generating visually impressive content from textual descriptions. However, training these models on unfiltered internet data poses the risk of learning and subsequently propagating undesirable concepts, such as copyrighted or unethical content. In this paper, we propose a novel method to remove undesirable concepts from text-to-image gene… ▽ More Generative models have demonstrated remarkable potential in generating visually impressive content from textual descriptions. However, training these models on unfiltered internet data poses the risk of learning and subsequently propagating undesirable concepts, such as copyrighted or unethical content. In this paper, we propose a novel method to remove undesirable concepts from text-to-image generative models by incorporating a learnable prompt into the cross-attention module. This learnable prompt acts as additional memory to transfer the knowledge of undesirable concepts into it and reduce the dependency of these concepts on the model parameters and corresponding textual inputs. Because of this knowledge transfer into the prompt, erasing these undesirable concepts is more stable and has minimal negative impact on other concepts. We demonstrate the effectiveness of our method on the Stable Diffusion model, showcasing its superiority over state-of-the-art erasure methods in terms of removing undesirable content while preserving other unrelated elements. △ Less

Submitted 18 March, 2024; originally announced March 2024.

Class-Prototype Conditional Diffusion Model with Gradient Projection for Continual Learning

Authors: Khanh Doan, Quyen Tran, Tung Lam Tran, Tuan Nguyen, Dinh Phung, Trung Le

Abstract: Mitigating catastrophic forgetting is a key hurdle in continual learning. Deep Generative Replay (GR) provides techniques focused on generating samples from prior tasks to enhance the model's memory capabilities using generative AI models ranging from Generative Adversarial Networks (GANs) to the more recent Diffusion Models (DMs). A major issue is the deterioration in the quality of generated dat… ▽ More Mitigating catastrophic forgetting is a key hurdle in continual learning. Deep Generative Replay (GR) provides techniques focused on generating samples from prior tasks to enhance the model's memory capabilities using generative AI models ranging from Generative Adversarial Networks (GANs) to the more recent Diffusion Models (DMs). A major issue is the deterioration in the quality of generated data compared to the original, as the generator continuously self-learns from its outputs. This degradation can lead to the potential risk of catastrophic forgetting (CF) occurring in the classifier. To address this, we propose the Gradient Projection Class-Prototype Conditional Diffusion Model (GPPDM), a GR-based approach for continual learning that enhances image quality in generators and thus reduces the CF in classifiers. The cornerstone of GPPDM is a learnable class prototype that captures the core characteristics of images in a given class. This prototype, integrated into the diffusion model's denoising process, ensures the generation of high-quality images of the old tasks, hence reducing the risk of CF in classifiers. Moreover, to further mitigate the CF of diffusion models, we propose a gradient projection technique tailored for the cross-attention layer of diffusion models to maximally maintain and preserve the representations of old task data in the current task as close as possible to their representations when they first arrived. Our empirical studies on diverse datasets demonstrate that our proposed method significantly outperforms existing state-of-the-art models, highlighting its satisfactory ability to preserve image quality and enhance the model's memory retention. △ Less

Submitted 21 March, 2024; v1 submitted 10 December, 2023; originally announced December 2023.

Synthesizing Physical Backdoor Datasets: An Automated Framework Leveraging Deep Generative Models

Authors: Sze Jue Yang, Chinh D. La, Quang H. Nguyen, Kok-Seng Wong, Anh Tuan Tran, Chee Seng Chan, Khoa D. Doan

Abstract: Backdoor attacks, representing an emerging threat to the integrity of deep neural networks, have garnered significant attention due to their ability to compromise deep learning systems clandestinely. While numerous backdoor attacks occur within the digital realm, their practical implementation in real-world prediction systems remains limited and vulnerable to disturbances in the physical world. Co… ▽ More Backdoor attacks, representing an emerging threat to the integrity of deep neural networks, have garnered significant attention due to their ability to compromise deep learning systems clandestinely. While numerous backdoor attacks occur within the digital realm, their practical implementation in real-world prediction systems remains limited and vulnerable to disturbances in the physical world. Consequently, this limitation has given rise to the development of physical backdoor attacks, where trigger objects manifest as physical entities within the real world. However, creating the requisite dataset to train or evaluate a physical backdoor model is a daunting task, limiting the backdoor researchers and practitioners from studying such physical attack scenarios. This paper unleashes a recipe that empowers backdoor researchers to effortlessly create a malicious, physical backdoor dataset based on advances in generative modeling. Particularly, this recipe involves 3 automatic modules: suggesting the suitable physical triggers, generating the poisoned candidate samples (either by synthesizing new samples or editing existing clean samples), and finally refining for the most plausible ones. As such, it effectively mitigates the perceived complexity associated with creating a physical backdoor dataset, transforming it from a daunting task into an attainable objective. Extensive experiment results show that datasets created by our "recipe" enable adversaries to achieve an impressive attack success rate on real physical world data and exhibit similar properties compared to previous physical backdoor attack studies. This paper offers researchers a valuable toolkit for studies of physical backdoors, all within the confines of their laboratories. △ Less

Submitted 15 March, 2024; v1 submitted 6 December, 2023; originally announced December 2023.

Local structure of the Teichmüller and the Riemann moduli stacks

Authors: An Khuong Doan

Abstract: The goal of this note is to introduce an interesting question proposed by D. Rydh on an analytic version of the local structure of Artin stacks saying that near points with linearly reductive stabilizers, Artin stacks are étale-locally quotient stacks. We give some supporting evidence by verifying it on two fundamental classes of classical analytic moduli spaces: the Teichmüller moduli space and t… ▽ More The goal of this note is to introduce an interesting question proposed by D. Rydh on an analytic version of the local structure of Artin stacks saying that near points with linearly reductive stabilizers, Artin stacks are étale-locally quotient stacks. We give some supporting evidence by verifying it on two fundamental classes of classical analytic moduli spaces: the Teichmüller moduli space and the Riemann moduli space of integrable complex structures whose analytic stack versions have been constructed by a recent work of L. Meersseman. △ Less

Submitted 26 November, 2023; originally announced November 2023.

Comments: 13 pages, comments are welcome

MSC Class: 14B99; 14D15; 32G13; 32G05

Fumbling in Babel: An Investigation into ChatGPT's Language Identification Ability

Authors: Wei-Rui Chen, Ife Adebara, Khai Duy Doan, Qisheng Liao, Muhammad Abdul-Mageed

Abstract: ChatGPT has recently emerged as a powerful NLP tool that can carry out a variety of tasks. However, the range of languages ChatGPT can handle remains largely a mystery. To uncover which languages ChatGPT `knows', we investigate its language identification (LID) abilities. For this purpose, we compile Babel-670, a benchmark comprising 670 languages representing 24 language families spoken in five c… ▽ More ChatGPT has recently emerged as a powerful NLP tool that can carry out a variety of tasks. However, the range of languages ChatGPT can handle remains largely a mystery. To uncover which languages ChatGPT `knows', we investigate its language identification (LID) abilities. For this purpose, we compile Babel-670, a benchmark comprising 670 languages representing 24 language families spoken in five continents. Languages in Babel-670 run the gamut from the very high-resource to the very low-resource. We then study ChatGPT's (both GPT-3.5 and GPT-4) ability to (i) identify language names and language codes (ii) under zero- and few-shot conditions (iii) with and without provision of a label set. When compared to smaller finetuned LID tools, we find that ChatGPT lags behind. For example, it has poor performance on African languages. We conclude that current large language models would benefit from further development before they can sufficiently serve diverse communities. △ Less

Submitted 8 April, 2024; v1 submitted 16 November, 2023; originally announced November 2023.

Comments: Accepted to NAACL 2024 Findings

The Skipped Beat: A Study of Sociopragmatic Understanding in LLMs for 64 Languages

Authors: Chiyu Zhang, Khai Duy Doan, Qisheng Liao, Muhammad Abdul-Mageed

Abstract: Instruction tuned large language models (LLMs), such as ChatGPT, demonstrate remarkable performance in a wide range of tasks. Despite numerous recent studies that examine the performance of instruction-tuned LLMs on various NLP benchmarks, there remains a lack of comprehensive investigation into their ability to understand cross-lingual sociopragmatic meaning (SM), i.e., meaning embedded within so… ▽ More Instruction tuned large language models (LLMs), such as ChatGPT, demonstrate remarkable performance in a wide range of tasks. Despite numerous recent studies that examine the performance of instruction-tuned LLMs on various NLP benchmarks, there remains a lack of comprehensive investigation into their ability to understand cross-lingual sociopragmatic meaning (SM), i.e., meaning embedded within social and interactive contexts. This deficiency arises partly from SM not being adequately represented in any of the existing benchmarks. To address this gap, we present SPARROW, an extensive multilingual benchmark specifically designed for SM understanding. SPARROW comprises 169 datasets covering 13 task types across six primary categories (e.g., anti-social language detection, emotion recognition). SPARROW datasets encompass 64 different languages originating from 12 language families representing 16 writing scripts. We evaluate the performance of various multilingual pretrained language models (e.g., mT5) and instruction-tuned LLMs (e.g., BLOOMZ, ChatGPT) on SPARROW through fine-tuning, zero-shot, and/or few-shot learning. Our comprehensive analysis reveals that existing open-source instruction tuned LLMs still struggle to understand SM across various languages, performing close to a random baseline in some cases. We also find that although ChatGPT outperforms many LLMs, it still falls behind task-specific finetuned models with a gap of 12.19 SPARROW score. Our benchmark is available at: https://github.com/UBC-NLP/SPARROW △ Less

Submitted 23 October, 2023; originally announced October 2023.

Comments: Accepted by EMNLP 2023 Main conference

Fooling the Textual Fooler via Randomizing Latent Representations

Authors: Duy C. Hoang, Quang H. Nguyen, Saurav Manchanda, MinLong Peng, Kok-Seng Wong, Khoa D. Doan

Abstract: Despite outstanding performance in a variety of NLP tasks, recent studies have revealed that NLP models are vulnerable to adversarial attacks that slightly perturb the input to cause the models to misbehave. Among these attacks, adversarial word-level perturbations are well-studied and effective attack strategies. Since these attacks work in black-box settings, they do not require access to the mo… ▽ More Despite outstanding performance in a variety of NLP tasks, recent studies have revealed that NLP models are vulnerable to adversarial attacks that slightly perturb the input to cause the models to misbehave. Among these attacks, adversarial word-level perturbations are well-studied and effective attack strategies. Since these attacks work in black-box settings, they do not require access to the model architecture or model parameters and thus can be detrimental to existing NLP applications. To perform an attack, the adversary queries the victim model many times to determine the most important words in an input text and to replace these words with their corresponding synonyms. In this work, we propose a lightweight and attack-agnostic defense whose main goal is to perplex the process of generating an adversarial example in these query-based black-box attacks; that is to fool the textual fooler. This defense, named AdvFooler, works by randomizing the latent representation of the input at inference time. Different from existing defenses, AdvFooler does not necessitate additional computational overhead during training nor relies on assumptions about the potential adversarial perturbation set while having a negligible impact on the model's accuracy. Our theoretical and empirical analyses highlight the significance of robustness resulting from confusing the adversary via randomizing the latent space, as well as the impact of randomization on clean accuracy. Finally, we empirically demonstrate near state-of-the-art robustness of AdvFooler against representative adversarial word-level attacks on two benchmark datasets. △ Less

Submitted 9 June, 2024; v1 submitted 2 October, 2023; originally announced October 2023.

Comments: Accepted to Findings of ACL 2024

Understanding the Robustness of Randomized Feature Defense Against Query-Based Adversarial Attacks

Authors: Quang H. Nguyen, Yingjie Lao, Tung Pham, Kok-Seng Wong, Khoa D. Doan

Abstract: Recent works have shown that deep neural networks are vulnerable to adversarial examples that find samples close to the original image but can make the model misclassify. Even with access only to the model's output, an attacker can employ black-box attacks to generate such adversarial examples. In this work, we propose a simple and lightweight defense against black-box attacks by adding random noi… ▽ More Recent works have shown that deep neural networks are vulnerable to adversarial examples that find samples close to the original image but can make the model misclassify. Even with access only to the model's output, an attacker can employ black-box attacks to generate such adversarial examples. In this work, we propose a simple and lightweight defense against black-box attacks by adding random noise to hidden features at intermediate layers of the model at inference time. Our theoretical analysis confirms that this method effectively enhances the model's resilience against both score-based and decision-based black-box attacks. Importantly, our defense does not necessitate adversarial training and has minimal impact on accuracy, rendering it applicable to any pre-trained model. Our analysis also reveals the significance of selectively adding noise to different parts of the model based on the gradient of the adversarial objective function, which can be varied during the attack. We demonstrate the robustness of our defense against multiple black-box attacks through extensive empirical experiments involving diverse models with various architectures. △ Less

Submitted 30 September, 2023; originally announced October 2023.

Everyone Can Attack: Repurpose Lossy Compression as a Natural Backdoor Attack

Authors: Sze Jue Yang, Quang Nguyen, Chee Seng Chan, Khoa D. Doan

Abstract: The vulnerabilities to backdoor attacks have recently threatened the trustworthiness of machine learning models in practical applications. Conventional wisdom suggests that not everyone can be an attacker since the process of designing the trigger generation algorithm often involves significant effort and extensive experimentation to ensure the attack's stealthiness and effectiveness. Alternativel… ▽ More The vulnerabilities to backdoor attacks have recently threatened the trustworthiness of machine learning models in practical applications. Conventional wisdom suggests that not everyone can be an attacker since the process of designing the trigger generation algorithm often involves significant effort and extensive experimentation to ensure the attack's stealthiness and effectiveness. Alternatively, this paper shows that there exists a more severe backdoor threat: anyone can exploit an easily-accessible algorithm for silent backdoor attacks. Specifically, this attacker can employ the widely-used lossy image compression from a plethora of compression tools to effortlessly inject a trigger pattern into an image without leaving any noticeable trace; i.e., the generated triggers are natural artifacts. One does not require extensive knowledge to click on the "convert" or "save as" button while using tools for lossy image compression. Via this attack, the adversary does not need to design a trigger generator as seen in prior works and only requires poisoning the data. Empirically, the proposed attack consistently achieves 100% attack success rate in several benchmark datasets such as MNIST, CIFAR-10, GTSRB and CelebA. More significantly, the proposed attack can still achieve almost 100% attack success rate with very small (approximately 10%) poisoning rates in the clean label setting. The generated trigger of the proposed attack using one lossy compression algorithm is also transferable across other related compression algorithms, exacerbating the severity of this backdoor threat. This work takes another crucial step toward understanding the extensive risks of backdoor attacks in practice, urging practitioners to investigate similar attacks and relevant backdoor mitigation methods. △ Less

Submitted 3 September, 2023; v1 submitted 31 August, 2023; originally announced August 2023.

Comments: 14 pages. This paper shows everyone can mount a powerful and stealthy backdoor attack with the widely-used lossy image compression

Clustering-based Identification of Precursors of Extreme Events in Chaotic Systems

Authors: Urszula Golyska, Nguyen Anh Khoa Doan

Abstract: Abrupt and rapid high-amplitude changes in a dynamical system's states known as extreme event appear in many processes occurring in nature, such as drastic climate patterns, rogue waves, or avalanches. These events often entail catastrophic effects, therefore their description and prediction is of great importance. However, because of their chaotic nature, their modelling represents a great challe… ▽ More Abrupt and rapid high-amplitude changes in a dynamical system's states known as extreme event appear in many processes occurring in nature, such as drastic climate patterns, rogue waves, or avalanches. These events often entail catastrophic effects, therefore their description and prediction is of great importance. However, because of their chaotic nature, their modelling represents a great challenge up to this day. The applicability of a data-driven modularity-based clustering technique to identify precursors of rare and extreme events in chaotic systems is here explored. The proposed identification framework based on clustering of system states, probability transition matrices and state space tessellation was developed and tested on two different chaotic systems that exhibit extreme events: the Moehliss-Faisst-Eckhardt model of self-sustained turbulence and the 2D Kolmogorov flow. Both exhibit extreme events in the form of bursts in kinetic energy and dissipation. It is shown that the proposed framework provides a way to identify pathways towards extreme events and predict their occurrence from a probabilistic standpoint. The clustering algorithm correctly identifies the precursor states leading to extreme events and allows for a statistical description of the system's states and its precursors to extreme events. △ Less

Submitted 20 June, 2023; originally announced June 2023.

A Cosine Similarity-based Method for Out-of-Distribution Detection

Authors: Nguyen Ngoc-Hieu, Nguyen Hung-Quang, The-Anh Ta, Thanh Nguyen-Tang, Khoa D Doan, Hoang Thanh-Tung

Abstract: The ability to detect OOD data is a crucial aspect of practical machine learning applications. In this work, we show that cosine similarity between the test feature and the typical ID feature is a good indicator of OOD data. We propose Class Typical Matching (CTM), a post hoc OOD detection algorithm that uses a cosine similarity scoring function. Extensive experiments on multiple benchmarks show t… ▽ More The ability to detect OOD data is a crucial aspect of practical machine learning applications. In this work, we show that cosine similarity between the test feature and the typical ID feature is a good indicator of OOD data. We propose Class Typical Matching (CTM), a post hoc OOD detection algorithm that uses a cosine similarity scoring function. Extensive experiments on multiple benchmarks show that CTM outperforms existing post hoc OOD detection methods. △ Less

Submitted 23 June, 2023; originally announced June 2023.

Comments: Accepted paper at ICML 2023 Workshop on Spurious Correlations, Invariance, and Stability. 10 pages (4 main + appendix)

Backdoor Attacks and Defenses in Federated Learning: Survey, Challenges and Future Research Directions

Authors: Thuy Dung Nguyen, Tuan Nguyen, Phi Le Nguyen, Hieu H. Pham, Khoa Doan, Kok-Seng Wong

Abstract: Federated learning (FL) is a machine learning (ML) approach that allows the use of distributed data without compromising personal privacy. However, the heterogeneous distribution of data among clients in FL can make it difficult for the orchestration server to validate the integrity of local model updates, making FL vulnerable to various threats, including backdoor attacks. Backdoor attacks involv… ▽ More Federated learning (FL) is a machine learning (ML) approach that allows the use of distributed data without compromising personal privacy. However, the heterogeneous distribution of data among clients in FL can make it difficult for the orchestration server to validate the integrity of local model updates, making FL vulnerable to various threats, including backdoor attacks. Backdoor attacks involve the insertion of malicious functionality into a targeted model through poisoned updates from malicious clients. These attacks can cause the global model to misbehave on specific inputs while appearing normal in other cases. Backdoor attacks have received significant attention in the literature due to their potential to impact real-world deep learning applications. However, they have not been thoroughly studied in the context of FL. In this survey, we provide a comprehensive survey of current backdoor attack strategies and defenses in FL, including a comprehensive analysis of different approaches. We also discuss the challenges and potential future directions for attacks and defenses in the context of FL. △ Less

Submitted 3 March, 2023; originally announced March 2023.

Convolutional autoencoder for the spatiotemporal latent representation of turbulence

Authors: Nguyen Anh Khoa Doan, Alberto Racca, Luca Magri

Abstract: Turbulence is characterised by chaotic dynamics and a high-dimensional state space, which make this phenomenon challenging to predict. However, turbulent flows are often characterised by coherent spatiotemporal structures, such as vortices or large-scale modes, which can help obtain a latent description of turbulent flows. However, current approaches are often limited by either the need to use som… ▽ More Turbulence is characterised by chaotic dynamics and a high-dimensional state space, which make this phenomenon challenging to predict. However, turbulent flows are often characterised by coherent spatiotemporal structures, such as vortices or large-scale modes, which can help obtain a latent description of turbulent flows. However, current approaches are often limited by either the need to use some form of thresholding on quantities defining the isosurfaces to which the flow structures are associated or the linearity of traditional modal flow decomposition approaches, such as those based on proper orthogonal decomposition. This problem is exacerbated in flows that exhibit extreme events, which are rare and sudden changes in a turbulent state. The goal of this paper is to obtain an efficient and accurate reduced-order latent representation of a turbulent flow that exhibits extreme events. Specifically, we employ a three-dimensional multiscale convolutional autoencoder (CAE) to obtain such latent representation. We apply it to a three-dimensional turbulent flow. We show that the Multiscale CAE is efficient, requiring less than 10% degrees of freedom than proper orthogonal decomposition for compressing the data and is able to accurately reconstruct flow states related to extreme events. The proposed deep learning architecture opens opportunities for nonlinear reduced-order modeling of turbulent flows from data. △ Less

Submitted 20 June, 2023; v1 submitted 31 January, 2023; originally announced January 2023.

Modelling spatiotemporal turbulent dynamics with the convolutional autoencoder echo state network

Authors: Alberto Racca, Nguyen Anh Khoa Doan, Luca Magri

Abstract: The spatiotemporal dynamics of turbulent flows is chaotic and difficult to predict. This makes the design of accurate and stable reduced-order models challenging. The overarching objective of this paper is to propose a nonlinear decomposition of the turbulent state for a reduced-order representation of the dynamics. We divide the turbulent flow into a spatial problem and a temporal problem. First,… ▽ More The spatiotemporal dynamics of turbulent flows is chaotic and difficult to predict. This makes the design of accurate and stable reduced-order models challenging. The overarching objective of this paper is to propose a nonlinear decomposition of the turbulent state for a reduced-order representation of the dynamics. We divide the turbulent flow into a spatial problem and a temporal problem. First, we compute the latent space, which is the manifold onto which the turbulent dynamics live (i.e., it is a numerical approximation of the turbulent attractor). The latent space is found by a series of nonlinear filtering operations, which are performed by a convolutional autoencoder (CAE). The CAE provides the decomposition in space. Second, we predict the time evolution of the turbulent state in the latent space, which is performed by an echo state network (ESN). The ESN provides the decomposition in time. Third, by assembling the CAE and the ESN, we obtain an autonomous dynamical system: the convolutional autoncoder echo state network (CAE-ESN). This is the reduced-order model of the turbulent flow. We test the CAE-ESN on a two-dimensional flow. We show that, after training, the CAE-ESN (i) finds a latent-space representation of the turbulent flow that has less than 1% of the degrees of freedom than the physical space; (ii) time-accurately and statistically predicts the flow in both quasiperiodic and turbulent regimes; (iii) is robust for different flow regimes (Reynolds numbers); and (iv) takes less than 1% of computational time to predict the turbulent flow than solving the governing equations. This work opens up new possibilities for nonlinear decompositions and reduced-order modelling of turbulent flows from data. △ Less

Submitted 22 November, 2022; v1 submitted 21 November, 2022; originally announced November 2022.

On interpretability and proper latent decomposition of autoencoders

Authors: Luca Magri, Anh Khoa Doan

Abstract: The dynamics of a turbulent flow tend to occupy only a portion of the phase space at a statistically stationary regime. From a dynamical systems point of view, this portion is the attractor. The knowledge of the turbulent attractor is useful for two purposes, at least: (i) We can gain physical insight into turbulence (what is the shape and geometry of the attractor?), and (ii) it provides the mini… ▽ More The dynamics of a turbulent flow tend to occupy only a portion of the phase space at a statistically stationary regime. From a dynamical systems point of view, this portion is the attractor. The knowledge of the turbulent attractor is useful for two purposes, at least: (i) We can gain physical insight into turbulence (what is the shape and geometry of the attractor?), and (ii) it provides the minimal number of degrees of freedom to accurately describe the turbulent dynamics. Autoencoders enable the computation of an optimal latent space, which is a low-order representation of the dynamics. If properly trained and correctly designed, autoencoders can learn an approximation of the turbulent attractor, as shown by Doan, Racca and Magri (2022). In this paper, we theoretically interpret the transformations of an autoencoder. First, we remark that the latent space is a curved manifold with curvilinear coordinates, which can be analyzed with simple tools from Riemann geometry. Second, we characterize the geometrical properties of the latent space. We mathematically derive the metric tensor, which provides a mathematical description of the manifold. Third, we propose a method -- proper latent decomposition (PLD) -- that generalizes proper orthogonal decomposition of turbulent flows on the autoencoder latent space. This decomposition finds the dominant directions in the curved latent space. This theoretical work opens up computational opportunities for interpreting autoencoders and creating reduced-order models of turbulent flows. △ Less

Submitted 2 December, 2022; v1 submitted 15 November, 2022; originally announced November 2022.

Comments: 9 pages, 2 figures, Proceedings of the Summer Program, Center for Turbulence Research, Stanford University

Asymmetric Hashing for Fast Ranking via Neural Network Measures

Authors: Khoa Doan, Shulong Tan, Weijie Zhao, ** Li

Abstract: Fast item ranking is an important task in recommender systems. In previous works, graph-based Approximate Nearest Neighbor (ANN) approaches have demonstrated good performance on item ranking tasks with generic searching/matching measures (including complex measures such as neural network measures). However, since these ANN approaches must go through the neural measures several times during ranking… ▽ More Fast item ranking is an important task in recommender systems. In previous works, graph-based Approximate Nearest Neighbor (ANN) approaches have demonstrated good performance on item ranking tasks with generic searching/matching measures (including complex measures such as neural network measures). However, since these ANN approaches must go through the neural measures several times during ranking, the computation is not practical if the neural measure is a large network. On the other hand, fast item ranking using existing hashing-based approaches, such as Locality Sensitive Hashing (LSH), only works with a limited set of measures. Previous learning-to-hash approaches are also not suitable to solve the fast item ranking problem since they can take a significant amount of time and computation to train the hash functions. Hashing approaches, however, are attractive because they provide a principle and efficient way to retrieve candidate items. In this paper, we propose a simple and effective learning-to-hash approach for the fast item ranking problem that can be used for any type of measure, including neural network measures. Specifically, we solve this problem with an asymmetric hashing framework based on discrete inner product fitting. We learn a pair of related hash functions that map heterogeneous objects (e.g., users and items) into a common discrete space where the inner product of their binary codes reveals their true similarity defined via the original searching measure. The fast ranking problem is reduced to an ANN search via this asymmetric hashing scheme. Then, we propose a sampling strategy to efficiently select relevant and contrastive samples to train the hashing model. We empirically validate the proposed method against the existing state-of-the-art fast item ranking methods in several combinations of non-linear searching functions and prominent datasets. △ Less

Submitted 1 November, 2022; originally announced November 2022.

Optimal Task Offloading Policy in Edge Computing Systems with Firm Deadlines

Authors: Khai Doan, Wesley Araujo, Evangelos Kranakis, Ioannis Lambadaris, Yannis Viniotis

Abstract: The recent drastic increase in mobile data traffic has pushed the mobile edge computing systems to the limit of their capacity. A promising solution to this problem is the task migration provided by unmanned aerial vehicles (UAV). Key factors to be taken into account in the design of UAV offloading schemes must include the number of tasks waiting in the system as well as their corresponding deadli… ▽ More The recent drastic increase in mobile data traffic has pushed the mobile edge computing systems to the limit of their capacity. A promising solution to this problem is the task migration provided by unmanned aerial vehicles (UAV). Key factors to be taken into account in the design of UAV offloading schemes must include the number of tasks waiting in the system as well as their corresponding deadlines. An appropriate system cost which is used as an objective function to be minimized comprises two parts. First, an offloading cost which can be interpreted as the cost of using computational resources at the UAV. Second, a penalty cost due to potential task expiration. In order to minimize the expected (time average) cost over a time horizon, we formulate a Dynamic Programming (DP) equation and analyze it to describe properties of a candidate optimal offloading policy. The DP equation suffers from the well-known "Curse of Dimensionality" that makes computations intractable, especially when the state space is infinite. In order to reduce the computational burden, we identify three important properties of the optimal policy. Based on these properties, we show that it suffices to evaluate the DP equation on a finite subset of the state space only. We then show that the optimal task offloading decision associated with a state can be inferred from the decision taken at its "adjacent" states, further reducing the computational load. Finally, we provide numerical results to evaluate the influence of different parameters on the system performance as well as verify the theoretical results. △ Less

Submitted 21 June, 2023; v1 submitted 27 October, 2022; originally announced October 2022.

Marksman Backdoor: Backdoor Attacks with Arbitrary Target Class

Authors: Khoa D. Doan, Yingjie Lao, ** Li

Abstract: In recent years, machine learning models have been shown to be vulnerable to backdoor attacks. Under such attacks, an adversary embeds a stealthy backdoor into the trained model such that the compromised models will behave normally on clean inputs but will misclassify according to the adversary's control on maliciously constructed input with a trigger. While these existing attacks are very effecti… ▽ More In recent years, machine learning models have been shown to be vulnerable to backdoor attacks. Under such attacks, an adversary embeds a stealthy backdoor into the trained model such that the compromised models will behave normally on clean inputs but will misclassify according to the adversary's control on maliciously constructed input with a trigger. While these existing attacks are very effective, the adversary's capability is limited: given an input, these attacks can only cause the model to misclassify toward a single pre-defined or target class. In contrast, this paper exploits a novel backdoor attack with a much more powerful payload, denoted as Marksman, where the adversary can arbitrarily choose which target class the model will misclassify given any input during inference. To achieve this goal, we propose to represent the trigger function as a class-conditional generative model and to inject the backdoor in a constrained optimization framework, where the trigger function learns to generate an optimal trigger pattern to attack any target class at will while simultaneously embedding this generative backdoor into the trained model. Given the learned trigger-generation function, during inference, the adversary can specify an arbitrary backdoor attack target class, and an appropriate trigger causing the model to classify toward this target class is created accordingly. We show empirically that the proposed framework achieves high attack performance while preserving the clean-data performance in several benchmark datasets, including MNIST, CIFAR10, GTSRB, and TinyImageNet. The proposed Marksman backdoor attack can also easily bypass existing backdoor defenses that were originally designed against backdoor attacks with a single target class. Our work takes another significant step toward understanding the extensive risks of backdoor attacks in practice. △ Less

Submitted 17 October, 2022; originally announced October 2022.

Comments: Accepted to NeurIPS 2022

CoopHash: Cooperative Learning of Multipurpose Descriptor and Contrastive Pair Generator via Variational MCMC Teaching for Supervised Image Hashing

Authors: Khoa D. Doan, Jianwen Xie, Yaxuan Zhu, Yang Zhao, ** Li

Abstract: Leveraging supervised information can lead to superior retrieval performance in the image hashing domain but the performance degrades significantly without enough labeled data. One effective solution to boost performance is to employ generative models, such as Generative Adversarial Networks (GANs), to generate synthetic data in an image hashing model. However, GAN-based methods are difficult to t… ▽ More Leveraging supervised information can lead to superior retrieval performance in the image hashing domain but the performance degrades significantly without enough labeled data. One effective solution to boost performance is to employ generative models, such as Generative Adversarial Networks (GANs), to generate synthetic data in an image hashing model. However, GAN-based methods are difficult to train, which prevents the hashing approaches from jointly training the generative models and the hash functions. This limitation results in sub-optimal retrieval performance. To overcome this limitation, we propose a novel framework, the generative cooperative hashing network, which is based on energy-based cooperative learning. This framework jointly learns a powerful generative representation of the data and a robust hash function via two components: a top-down contrastive pair generator that synthesizes contrastive images and a bottom-up multipurpose descriptor that simultaneously represents the images from multiple perspectives, including probability density, hash code, latent code, and category. The two components are jointly learned via a novel likelihood-based cooperative learning scheme. We conduct experiments on several real-world datasets and show that the proposed method outperforms the competing hashing supervised methods, achieving up to 10\% relative improvement over the current state-of-the-art supervised hashing methods, and exhibits a significantly better performance in out-of-distribution retrieval. △ Less

Submitted 12 June, 2024; v1 submitted 9 October, 2022; originally announced October 2022.

Formal moduli problems with cohomological constraints

Authors: An Khuong Doan

Abstract: We aim to generalize Lurie's famous $\infty$-correspondence between formal moduli problems and differential graded Lie algebras to the context where some cohomological conditions are imposed. Specifically, a natural equivalence between formal moduli problems with cohomological constraints and derived cohomology jump functors is provided, thereby answering affirmatively a question posed by N. Budur… ▽ More We aim to generalize Lurie's famous $\infty$-correspondence between formal moduli problems and differential graded Lie algebras to the context where some cohomological conditions are imposed. Specifically, a natural equivalence between formal moduli problems with cohomological constraints and derived cohomology jump functors is provided, thereby answering affirmatively a question posed by N. Budur and B. Wang in \cite{1}. △ Less

Submitted 26 September, 2022; originally announced September 2022.

Comments: 22 pages, comments are welcome!

MSC Class: 14D15; 14B10; 13D10

Velocity Reconstruction in Puffing Pool Fires with Physics-Informed Neural Networks

Authors: Michael Philip Sitte, Nguyen Anh Khoa Doan

Abstract: Pool fires are canonical representations of many accidental fires, which can exhibit an unstable unsteady behaviour, known as puffing, which involves a strong coupling between the temperature and velocity fields. Despite their practical relevance to fire research, their experimental study can be limited due to the complexity of measuring relevant quantities in parallel. In this work, we analyse th… ▽ More Pool fires are canonical representations of many accidental fires, which can exhibit an unstable unsteady behaviour, known as puffing, which involves a strong coupling between the temperature and velocity fields. Despite their practical relevance to fire research, their experimental study can be limited due to the complexity of measuring relevant quantities in parallel. In this work, we analyse the use of a recent physics-informed machine learning approach, called Hidden Fluid Mechanics (HFM), to reconstruct unmeasured quantities in a puffing pool fire from measured quantities. The HFM framework relies on a Physics-Informed Neural Network (PINN) for this task. A PINN is a neural network that uses both the available data, here the measured quantities, and the physical equations governing the system, here the reacting Navier-Stokes equations, to infer the full fluid dynamic state. This framework is used to infer the velocity field in a puffing pool fire from measurements of density, pressure and temperature. In this work, the dataset used for this test was generated from numerical simulations. It is shown that the PINN is able to reconstruct the velocity field accurately and to infer most features of the velocity field. In addition, it is shown that the reconstruction accuracy is robust with respect to noisy data, and a reduction in the number of measured quantities is explored and discussed. This study opens up the possibility of using PINNs for the reconstruction of unmeasured quantities from measured ones, providing the potential groundwork for their use in experiments for fire research. △ Less

Submitted 6 August, 2022; originally announced August 2022.

Defending Backdoor Attacks on Vision Transformer via Patch Processing

Authors: Khoa D. Doan, Yingjie Lao, Peng Yang, ** Li

Abstract: Vision Transformers (ViTs) have a radically different architecture with significantly less inductive bias than Convolutional Neural Networks. Along with the improvement in performance, security and robustness of ViTs are also of great importance to study. In contrast to many recent works that exploit the robustness of ViTs against adversarial examples, this paper investigates a representative caus… ▽ More Vision Transformers (ViTs) have a radically different architecture with significantly less inductive bias than Convolutional Neural Networks. Along with the improvement in performance, security and robustness of ViTs are also of great importance to study. In contrast to many recent works that exploit the robustness of ViTs against adversarial examples, this paper investigates a representative causative attack, i.e., backdoor. We first examine the vulnerability of ViTs against various backdoor attacks and find that ViTs are also quite vulnerable to existing attacks. However, we observe that the clean-data accuracy and backdoor attack success rate of ViTs respond distinctively to patch transformations before the positional encoding. Then, based on this finding, we propose an effective method for ViTs to defend both patch-based and blending-based trigger backdoor attacks via patch processing. The performances are evaluated on several benchmark datasets, including CIFAR10, GTSRB, and TinyImageNet, which show the proposed novel defense is very successful in mitigating backdoor attacks for ViTs. To the best of our knowledge, this paper presents the first defensive strategy that utilizes a unique characteristic of ViTs against backdoor attacks. The paper will appear in the Proceedings of the AAAI'23 Conference. This work was initially submitted in November 2021 to CVPR'22, then it was re-submitted to ECCV'22. The paper was made public in June 2022. The authors sincerely thank all the referees from the Program Committees of CVPR'22, ECCV'22, and AAAI'23. △ Less

Submitted 16 January, 2023; v1 submitted 24 June, 2022; originally announced June 2022.

One Loss for Quantization: Deep Hashing with Discrete Wasserstein Distributional Matching

Authors: Khoa D. Doan, Peng Yang, ** Li

Abstract: Image hashing is a principled approximate nearest neighbor approach to find similar items to a query in a large collection of images. Hashing aims to learn a binary-output function that maps an image to a binary vector. For optimal retrieval performance, producing balanced hash codes with low-quantization error to bridge the gap between the learning stage's continuous relaxation and the inference… ▽ More Image hashing is a principled approximate nearest neighbor approach to find similar items to a query in a large collection of images. Hashing aims to learn a binary-output function that maps an image to a binary vector. For optimal retrieval performance, producing balanced hash codes with low-quantization error to bridge the gap between the learning stage's continuous relaxation and the inference stage's discrete quantization is important. However, in the existing deep supervised hashing methods, coding balance and low-quantization error are difficult to achieve and involve several losses. We argue that this is because the existing quantization approaches in these methods are heuristically constructed and not effective to achieve these objectives. This paper considers an alternative approach to learning the quantization constraints. The task of learning balanced codes with low quantization error is re-formulated as matching the learned distribution of the continuous codes to a pre-defined discrete, uniform distribution. This is equivalent to minimizing the distance between two distributions. We then propose a computationally efficient distributional distance by leveraging the discrete property of the hash functions. This distributional distance is a valid distance and enjoys lower time and sample complexities. The proposed single-loss quantization objective can be integrated into any existing supervised hashing method to improve code balance and quantization error. Experiments confirm that the proposed approach substantially improves the performance of several representative hashing~methods. △ Less

Submitted 31 May, 2022; originally announced May 2022.

Comments: CVPR 2022

Modeling of the nonlinear flame response of a Bunsen-type flame via multi-layer perceptron

Authors: Nilam Tathawadekar, Nguyen Anh Khoa Doan, Camilo F. Silva, Nils Thuerey

Abstract: This paper demonstrates the ability of neural networks to reliably learn the nonlinear flame response of a laminar premixed flame, while carrying out only one unsteady CFD simulation. The system is excited with a broadband, low-pass filtered velocity signal that exhibits a uniform distribution of amplitudes within a predetermined range. The obtained time series of flow velocity upstream of the fla… ▽ More This paper demonstrates the ability of neural networks to reliably learn the nonlinear flame response of a laminar premixed flame, while carrying out only one unsteady CFD simulation. The system is excited with a broadband, low-pass filtered velocity signal that exhibits a uniform distribution of amplitudes within a predetermined range. The obtained time series of flow velocity upstream of the flame and heat release rate fluctuations are used to train the nonlinear model using a multi-layer perceptron. Several models with varying hyperparameters are trained and the dropout strategy is used as regularizer to avoid overfitting. The best performing model is subsequently used to compute the flame describing function (FDF) using mono-frequent excitations. In addition to accurately predicting the FDF, the trained neural network model also captures the presence of higher harmonics in the flame response. As a result, when coupled with an acoustic solver, the obtained neural network model is better suited than a classical FDF model to predict limit cycle oscillations characterized by more than one frequency. The latter is demonstrated in the final part of the present study. We show that the RMS value of the predicted acoustic oscillations together with the associated dominant frequencies are in excellent agreement with CFD reference data. △ Less

Submitted 11 April, 2022; originally announced April 2022.

Journal ref: Proceedings of the Combustion Institute, 38(4), 6261-6269 (2021)

Group actions on local moduli space of holomorphic vector bundles

Authors: An Khuong Doan

Abstract: We prove that actions of complex reductive Lie groups on a holomorphic vector bundle over a complex compact manifold are locally extendable to its local moduli space. We prove that actions of complex reductive Lie groups on a holomorphic vector bundle over a complex compact manifold are locally extendable to its local moduli space. △ Less

Submitted 26 September, 2022; v1 submitted 26 January, 2022; originally announced January 2022.

Comments: 16 pages, 1 figure, final version

MSC Class: 14D15; 14B10; 32G

Journal ref: Canadian Mathematical Bulletin (2022)

A posteriori assessment of consumption speed correction for LES with tabulated methods

Authors: Ivan Langella, Nguyen Anh Khoa Doan

Abstract: Large eddy simulation of propane/air jet flame in the wrinkled flamelets regime of the Borghi diagram is used to assess the performance of a recently developed consumption speed correction model in premixed combustion. The combustion is modelled using flamelet tabulation with a presumed probability density function. The investigated flame does not lie in the shear layer and is subjected to self-dr… ▽ More Large eddy simulation of propane/air jet flame in the wrinkled flamelets regime of the Borghi diagram is used to assess the performance of a recently developed consumption speed correction model in premixed combustion. The combustion is modelled using flamelet tabulation with a presumed probability density function. The investigated flame does not lie in the shear layer and is subjected to self-driven oscillations, which is ideal to test the model performance. The flame behaviour is first discussed using an accurate simulation performed on a refined mesh requiring no correction. The same modelling framework used on a coarser mesh is observed to overestimate the consumption speed, leading to a shorter flame. The application of the consumption speed correction leads to the recovery of the flame length and width observed for the refined mesh, demonstrating its effectiveness. An extended model for partially-premixed combustion is also proposed and preliminarily tested on a high-pressure combustor. △ Less

Submitted 2 December, 2021; originally announced December 2021.

Incomplete to complete multiphysics forecasting -- a hybrid approach for learning unknown phenomena

Authors: Nilam Tathawadekar, Nguyen Anh Khoa Doan, Camilo F. Silva, Nils Thuerey

Abstract: Modeling complex dynamical systems with only partial knowledge of their physical mechanisms is a crucial problem across all scientific and engineering disciplines. Purely data-driven approaches, which only make use of an artificial neural network and data, often fail to accurately simulate the evolution of the system dynamics over a sufficiently long time and in a physically consistent manner. The… ▽ More Modeling complex dynamical systems with only partial knowledge of their physical mechanisms is a crucial problem across all scientific and engineering disciplines. Purely data-driven approaches, which only make use of an artificial neural network and data, often fail to accurately simulate the evolution of the system dynamics over a sufficiently long time and in a physically consistent manner. Therefore, we propose a hybrid approach that uses a neural network model in combination with an incomplete partial differential equations (PDE) solver that provides known, but incomplete physical information. In this study, we demonstrate that the results obtained from the incomplete PDEs can be efficiently corrected at every time step by the proposed hybrid neural network - PDE solver model, so that the effect of the unknown physics present in the system is correctly accounted for. For validation purposes, the obtained simulations of the hybrid model are successfully compared against results coming from the complete set of PDEs describing the full physics of the considered system. We demonstrate the validity of the proposed approach on a reactive flow, an archetypal multi-physics system that combines fluid mechanics and chemistry, the latter being the physics considered unknown. Experiments are made on planar and Bunsen-type flames at various operating conditions. The hybrid neural network - PDE approach correctly models the flame evolution of the cases under study for significantly long time windows, yields improved generalization, and allows for larger simulation time steps. △ Less

Submitted 2 October, 2023; v1 submitted 22 November, 2021; originally announced November 2021.

A note on the group extension problem to semi-universal deformation

Authors: An Khuong Doan

Abstract: The aim of this note is twofold. Firstly, we explain in detail Remark 4.1 in \cite{doan-a} by showing that the action of the automorphism group of the second Hirzebruch surface $\mathbb{F}_2$ on itself extends to its formal semi-universal deformation only up to the first order. Secondly, we show that for reductive group actions, the locality of the extended actions on the Kuranishi space construct… ▽ More The aim of this note is twofold. Firstly, we explain in detail Remark 4.1 in \cite{doan-a} by showing that the action of the automorphism group of the second Hirzebruch surface $\mathbb{F}_2$ on itself extends to its formal semi-universal deformation only up to the first order. Secondly, we show that for reductive group actions, the locality of the extended actions on the Kuranishi space constructed in \cite{doan-equivariant} is the best one could expect in general. △ Less

Submitted 10 October, 2023; v1 submitted 9 August, 2021; originally announced August 2021.

Comments: 6 pages, a more revised version

MSC Class: 14D15; 14B10; 13D10

Journal ref: Indagationes Mathematicae 2023

Semi-prorepresentability of formal moduli problems and equivariant structures

Authors: An Khuong Doan

Abstract: We generalize the notion of semi-universality in the classical deformation problems to the context of derived deformation theories. A criterion for a formal moduli problem to be semi-prorepresentable is produced. This can be seen as an analogue of Schlessinger's conditions for a functor of Artinian rings to have a semi-universal element. We also give a sufficient condition for a semi-prorepresenta… ▽ More We generalize the notion of semi-universality in the classical deformation problems to the context of derived deformation theories. A criterion for a formal moduli problem to be semi-prorepresentable is produced. This can be seen as an analogue of Schlessinger's conditions for a functor of Artinian rings to have a semi-universal element. We also give a sufficient condition for a semi-prorepresentable formal moduli problem to admit a $G$-equivariant structure in a sense specified below, where $G$ is a linearly reductive group. Finally, by making use of these criteria, we derive many classical results including the existence of ($G$-equivariant) formal semi-universal deformations of algebraic schemes and that of complex compact manifolds. △ Less

Submitted 26 September, 2023; v1 submitted 20 July, 2021; originally announced July 2021.

Comments: 19 pages, final version, to appear in Homology, Homotopy and Applications. (all parts related to equivariant deformations of schemes will appear in another work, as suggested by the referee)

MSC Class: 14D15; 14B10; 13D10

Short- and long-term prediction of a chaotic flow: A physics-constrained reservoir computing approach

Authors: Nguyen Anh Khoa Doan, Wolfgang Polifke, Luca Magri

Abstract: We propose a physics-constrained machine learning method-based on reservoir computing- to time-accurately predict extreme events and long-term velocity statistics in a model of turbulent shear flow. The method leverages the strengths of two different approaches: empirical modelling based on reservoir computing, which it learns the chaotic dynamics from data only, and physical modelling based on co… ▽ More We propose a physics-constrained machine learning method-based on reservoir computing- to time-accurately predict extreme events and long-term velocity statistics in a model of turbulent shear flow. The method leverages the strengths of two different approaches: empirical modelling based on reservoir computing, which it learns the chaotic dynamics from data only, and physical modelling based on conservation laws, which extrapolates the dynamics when training data becomes unavailable. We show that the combination of the two approaches is able to accurately reproduce the velocity statistics and to predict the occurrence and amplitude of extreme events in a model of self-sustaining process in turbulence. In this flow, the extreme events are abrupt transitions from turbulent to quasi-laminar states, which are deterministic phenomena that cannot be traditionally predicted because of chaos. Furthermore, the physics-constrained machine learning method is shown to be robust with respect to noise. This work opens up new possibilities for synergistically enhancing data-driven methods with physical knowledge for the time-accurate prediction of chaotic flows. △ Less

Submitted 13 April, 2021; v1 submitted 15 February, 2021; originally announced February 2021.

Comments: 10 pages, 10 figures. arXiv admin note: substantial text overlap with arXiv:1912.10994

Auto-Encoded Reservoir Computing for Turbulence Learning

Authors: Nguyen Anh Khoa Doan, Wolfgang Polifke, Luca Magri

Abstract: We present an Auto-Encoded Reservoir-Computing (AE-RC) approach to learn the dynamics of a 2D turbulent flow. The AE-RC consists of an Autoencoder, which discovers an efficient manifold representation of the flow state, and an Echo State Network, which learns the time evolution of the flow in the manifold. The AE-RC is able to both learn the time-accurate dynamics of the flow and predict its first… ▽ More We present an Auto-Encoded Reservoir-Computing (AE-RC) approach to learn the dynamics of a 2D turbulent flow. The AE-RC consists of an Autoencoder, which discovers an efficient manifold representation of the flow state, and an Echo State Network, which learns the time evolution of the flow in the manifold. The AE-RC is able to both learn the time-accurate dynamics of the flow and predict its first-order statistical moments. The AE-RC approach opens up new possibilities for the spatio-temporal prediction of turbulence with machine learning. △ Less

Submitted 24 March, 2021; v1 submitted 20 December, 2020; originally announced December 2020.

Comments: 7 pages, 5 figures

Physics-Informed Echo State Networks

Authors: Nguyen Anh Khoa Doan, Wolfgang Polifke, Luca Magri

Abstract: We propose a physics-informed Echo State Network (ESN) to predict the evolution of chaotic systems. Compared to conventional ESNs, the physics-informed ESNs are trained to solve supervised learning tasks while ensuring that their predictions do not violate physical laws. This is achieved by introducing an additional loss function during the training, which is based on the system's governing equati… ▽ More We propose a physics-informed Echo State Network (ESN) to predict the evolution of chaotic systems. Compared to conventional ESNs, the physics-informed ESNs are trained to solve supervised learning tasks while ensuring that their predictions do not violate physical laws. This is achieved by introducing an additional loss function during the training, which is based on the system's governing equations. The additional loss function penalizes non-physical predictions without the need of any additional training data. This approach is demonstrated on a chaotic Lorenz system and a truncation of the Charney-DeVore system. Compared to the conventional ESNs, the physics-informed ESNs improve the predictability horizon by about two Lyapunov times. This approach is also shown to be robust with regard to noise. The proposed framework shows the potential of using machine learning combined with prior physical knowledge to improve the time-accurate prediction of chaotic dynamical systems. △ Less

Submitted 31 October, 2020; originally announced November 2020.

Comments: 10 pages, 11 figures. arXiv admin note: substantial text overlap with arXiv:1906.11122

Journal ref: Journal of computational science, 2020

First-principles machine learning modelling of COVID-19

Authors: Luca Magri, Nguyen Anh Khoa Doan

Abstract: The coronavirus disease 2019 (COVID-19) has changed the world since the World Health Organization declared its outbreak on 30th January 2020, recognizing the outbreak as a pandemic on 11th March 2020. As often said by politicians and scientific advisors, the objective is "to flatten the curve", or "push the peak down", or similar wording, of the virus spreading. Central to the official advice are… ▽ More The coronavirus disease 2019 (COVID-19) has changed the world since the World Health Organization declared its outbreak on 30th January 2020, recognizing the outbreak as a pandemic on 11th March 2020. As often said by politicians and scientific advisors, the objective is "to flatten the curve", or "push the peak down", or similar wording, of the virus spreading. Central to the official advice are mathematical models and data, which provide estimates on the evolution of the number of infected, recovered and deaths. The accuracy of the models is improved day by day by inferring the contact, recovery, and death rates from data (confirmed cases). A data-driven model trained with {\it both} data {\it and} first principles is proposed. The model can quickly be re-trained any time that new data becomes available. The method can be applied to more detailed epidemic models with virtually no conceptual modification. △ Less

Submitted 20 April, 2020; originally announced April 2020.

Comments: 39 pages, 53 figures

Equivariant Kuranishi family of complex compact manifolds

Authors: An Khuong Doan

Abstract: We prove that actions of complex reductive Lie groups on a complex compact manifold are locally extendable to its Kuranishi family. This can be seen as an analogue of Rim's result (see [11]) in the analytic setting. We prove that actions of complex reductive Lie groups on a complex compact manifold are locally extendable to its Kuranishi family. This can be seen as an analogue of Rim's result (see [11]) in the analytic setting. △ Less

Submitted 26 March, 2021; v1 submitted 15 April, 2020; originally announced April 2020.

Comments: 17 pages, Published version

MSC Class: 14D15; 14B10; 32G

Journal ref: Manuscripta Mathematica (2021)

Image Generation Via Minimizing Fréchet Distance in Discriminator Feature Space

Authors: Khoa D. Doan, Saurav Manchanda, Fengjiao Wang, Sathiya Keerthi, Avradeep Bhowmik, Chandan K. Reddy

Abstract: For a given image generation problem, the intrinsic image manifold is often low dimensional. We use the intuition that it is much better to train the GAN generator by minimizing the distributional distance between real and generated images in a small dimensional feature space representing such a manifold than on the original pixel-space. We use the feature space of the GAN discriminator for such a… ▽ More For a given image generation problem, the intrinsic image manifold is often low dimensional. We use the intuition that it is much better to train the GAN generator by minimizing the distributional distance between real and generated images in a small dimensional feature space representing such a manifold than on the original pixel-space. We use the feature space of the GAN discriminator for such a representation. For distributional distance, we employ one of two choices: the Fréchet distance or direct optimal transport (OT); these respectively lead us to two new GAN methods: Fréchet-GAN and OT-GAN. The idea of employing Fréchet distance comes from the success of Fréchet Inception Distance as a solid evaluation metric in image generation. Fréchet-GAN is attractive in several ways. We propose an efficient, numerically stable approach to calculate the Fréchet distance and its gradient. The Fréchet distance estimation requires a significantly less computation time than OT; this allows Fréchet-GAN to use much larger mini-batch size in training than OT. More importantly, we conduct experiments on a number of benchmark datasets and show that Fréchet-GAN (in particular) and OT-GAN have significantly better image generation capabilities than the existing representative primal and dual GAN approaches based on the Wasserstein distance. △ Less

Submitted 30 March, 2020; v1 submitted 26 March, 2020; originally announced March 2020.

Regression via Implicit Models and Optimal Transport Cost Minimization

Authors: Saurav Manchanda, Khoa Doan, Pranjul Yadav, S. Sathiya Keerthi

Abstract: This paper addresses the classic problem of regression, which involves the inductive learning of a map, $y=f(x,z)$, $z$ denoting noise, $f:\mathbb{R}^n\times \mathbb{R}^k \rightarrow \mathbb{R}^m$. Recently, Conditional GAN (CGAN) has been applied for regression and has shown to be advantageous over the other standard approaches like Gaussian Process Regression, given its ability to implicitly mod… ▽ More This paper addresses the classic problem of regression, which involves the inductive learning of a map, $y=f(x,z)$, $z$ denoting noise, $f:\mathbb{R}^n\times \mathbb{R}^k \rightarrow \mathbb{R}^m$. Recently, Conditional GAN (CGAN) has been applied for regression and has shown to be advantageous over the other standard approaches like Gaussian Process Regression, given its ability to implicitly model complex noise forms. However, the current CGAN implementation for regression uses the classical generator-discriminator architecture with the minimax optimization approach, which is notorious for being difficult to train due to issues like training instability or failure to converge. In this paper, we take another step towards regression models that implicitly model the noise, and propose a solution which directly optimizes the optimal transport cost between the true probability distribution $p(y|x)$ and the estimated distribution $\hat{p}(y|x)$ and does not suffer from the issues associated with the minimax approach. On a variety of synthetic and real-world datasets, our proposed solution achieves state-of-the-art results. The code accompanying this paper is available at "https://github.com/gurdaspuriya/ot_regression". △ Less

Submitted 2 March, 2020; originally announced March 2020.

Image Hashing by Minimizing Discrete Component-wise Wasserstein Distance

Authors: Khoa D. Doan, Saurav Manchanda, Sarkhan Badirli, Chandan K. Reddy

Abstract: Image hashing is one of the fundamental problems that demand both efficient and effective solutions for various practical scenarios. Adversarial autoencoders are shown to be able to implicitly learn a robust, locality-preserving hash function that generates balanced and high-quality hash codes. However, the existing adversarial hashing methods are inefficient to be employed for large-scale image r… ▽ More Image hashing is one of the fundamental problems that demand both efficient and effective solutions for various practical scenarios. Adversarial autoencoders are shown to be able to implicitly learn a robust, locality-preserving hash function that generates balanced and high-quality hash codes. However, the existing adversarial hashing methods are inefficient to be employed for large-scale image retrieval applications. Specifically, they require an exponential number of samples to be able to generate optimal hash codes and a significantly high computational cost to train. In this paper, we show that the high sample-complexity requirement often results in sub-optimal retrieval performance of the adversarial hashing methods. To address this challenge, we propose a new adversarial-autoencoder hashing approach that has a much lower sample requirement and computational cost. Specifically, by exploiting the desired properties of the hash function in the low-dimensional, discrete space, our method efficiently estimates a better variant of Wasserstein distance by averaging a set of easy-to-compute one-dimensional Wasserstein distances. The resulting hashing approach has an order-of-magnitude better sample complexity, thus better generalization property, compared to the other adversarial hashing methods. In addition, the computational cost is significantly reduced using our approach. We conduct experiments on several real-world datasets and show that the proposed method outperforms the competing hashing methods, achieving up to 10% improvement over the current state-of-the-art image hashing methods. The code accompanying this paper is available on Github (https://github.com/khoadoan/adversarial-hashing). △ Less

Submitted 25 May, 2020; v1 submitted 28 February, 2020; originally announced March 2020.

Gradient Boosting Neural Networks: GrowNet

Authors: Sarkhan Badirli, Xuanqing Liu, Zhengming Xing, Avradeep Bhowmik, Khoa Doan, Sathiya S. Keerthi

Abstract: A novel gradient boosting framework is proposed where shallow neural networks are employed as ``weak learners''. General loss functions are considered under this unified framework with specific examples presented for classification, regression, and learning to rank. A fully corrective step is incorporated to remedy the pitfall of greedy function approximation of classic gradient boosting decision… ▽ More A novel gradient boosting framework is proposed where shallow neural networks are employed as ``weak learners''. General loss functions are considered under this unified framework with specific examples presented for classification, regression, and learning to rank. A fully corrective step is incorporated to remedy the pitfall of greedy function approximation of classic gradient boosting decision tree. The proposed model rendered outperforming results against state-of-the-art boosting methods in all three tasks on multiple datasets. An ablation study is performed to shed light on the effect of each model components and model hyperparameters. △ Less

Submitted 14 June, 2020; v1 submitted 18 February, 2020; originally announced February 2020.

Comments: Supplementary material starts after references

Targeted display advertising: the case of preferential attachment

Authors: Saurav Manchanda, Pranjul Yadav, Khoa Doan, S. Sathiya Keerthi

Abstract: An average adult is exposed to hundreds of digital advertisements daily (https://www.mediadynamicsinc.com/uploads/files/PR092214-Note-only-150-Ads-2mk.pdf), making the digital advertisement industry a classic example of a big-data-driven platform. As such, the ad-tech industry relies on historical engagement logs (clicks or purchases) to identify potentially interested users for the advertisement… ▽ More An average adult is exposed to hundreds of digital advertisements daily (https://www.mediadynamicsinc.com/uploads/files/PR092214-Note-only-150-Ads-2mk.pdf), making the digital advertisement industry a classic example of a big-data-driven platform. As such, the ad-tech industry relies on historical engagement logs (clicks or purchases) to identify potentially interested users for the advertisement campaign of a partner (a seller who wants to target users for its products). The number of advertisements that are shown for a partner, and hence the historical campaign data available for a partner depends upon the budget constraints of the partner. Thus, enough data can be collected for the high-budget partners to make accurate predictions, while this is not the case with the low-budget partners. This skewed distribution of the data leads to "preferential attachment" of the targeted display advertising platforms towards the high-budget partners. In this paper, we develop "domain-adaptation" approaches to address the challenge of predicting interested users for the partners with insufficient data, i.e., the tail partners. Specifically, we develop simple yet effective approaches that leverage the similarity among the partners to transfer information from the partners with sufficient data to cold-start partners, i.e., partners without any campaign data. Our approaches readily adapt to the new campaign data by incremental fine-tuning, and hence work at varying points of a campaign, and not just the cold-start. We present an experimental analysis on the historical logs of a major display advertising platform (https://www.criteo.com/). Specifically, we evaluate our approaches across 149 partners, at varying points of their campaigns. Experimental results show that the proposed approaches outperform the other "domain-adaptation" approaches at different time points of the campaigns. △ Less

Submitted 7 February, 2020; originally announced February 2020.

Comments: IEEE BigData 2019 paper

Learning Hidden States in a Chaotic System: A Physics-Informed Echo State Network Approach

Authors: Nguyen Anh Khoa Doan, Wolfgang Polifke, Luca Magri

Abstract: We extend the Physics-Informed Echo State Network (PI-ESN) framework to reconstruct the evolution of an unmeasured state (hidden state) in a chaotic system. The PI-ESN is trained by using (i) data, which contains no information on the unmeasured state, and (ii) the physical equations of a prototypical chaotic dynamical system. Non-noisy and noisy datasets are considered. First, it is shown that th… ▽ More We extend the Physics-Informed Echo State Network (PI-ESN) framework to reconstruct the evolution of an unmeasured state (hidden state) in a chaotic system. The PI-ESN is trained by using (i) data, which contains no information on the unmeasured state, and (ii) the physical equations of a prototypical chaotic dynamical system. Non-noisy and noisy datasets are considered. First, it is shown that the PI-ESN can accurately reconstruct the unmeasured state. Second, the reconstruction is shown to be robust with respect to noisy data, which means that the PI-ESN acts as a denoiser. This paper opens up new possibilities for leveraging the synergy between physical knowledge and machine learning to enhance the reconstruction and prediction of unmeasured states in chaotic dynamical systems. △ Less

Submitted 7 April, 2020; v1 submitted 6 January, 2020; originally announced January 2020.

Comments: 7 pages, 4 figures

A physics-aware machine to predict extreme events in turbulence

Authors: Nguyen Anh Khoa Doan, Wolfgang Polifke, Luca Magri

Abstract: We propose a physics-aware machine learning method to time-accurately predict extreme events in a turbulent flow. The method combines two radically different approaches: empirical modelling based on reservoir computing, which learns the chaotic dynamics from data only, and physical modelling based on conservation laws. We show that the combination of the two approaches is able to predict the occur… ▽ More We propose a physics-aware machine learning method to time-accurately predict extreme events in a turbulent flow. The method combines two radically different approaches: empirical modelling based on reservoir computing, which learns the chaotic dynamics from data only, and physical modelling based on conservation laws. We show that the combination of the two approaches is able to predict the occurrence and amplitude of extreme events in the self-sustaining process in turbulence-the abrupt transitions from turbulent to quasi-laminar states-which cannot be achieved by using either approach separately. This opens up new possibilities for enhancing synergistically data-driven methods with physical knowledge for the accurate prediction of extreme events in chaotic dynamical systems. △ Less

Submitted 23 December, 2019; originally announced December 2019.

Comments: 5 pages, 5 figures

Power Allocation in Cache-Aided NOMA Systems: Optimization and Deep Reinforcement Learning Approaches

Authors: Khai Nguyen Doan, Mojtaba Vaezi, Wonjae Shin, H. Vincent Poor, Hyundong Shin, Tony Q. S. Quek

Abstract: This work exploits the advantages of two prominent techniques in future communication networks, namely caching and non-orthogonal multiple access (NOMA). Particularly, a system with Rayleigh fading channels and cache-enabled users is analyzed. It is shown that the caching-NOMA combination provides a new opportunity of cache hit which enhances the cache utility as well as the effectiveness of NOMA.… ▽ More This work exploits the advantages of two prominent techniques in future communication networks, namely caching and non-orthogonal multiple access (NOMA). Particularly, a system with Rayleigh fading channels and cache-enabled users is analyzed. It is shown that the caching-NOMA combination provides a new opportunity of cache hit which enhances the cache utility as well as the effectiveness of NOMA. Importantly, this comes without requiring users' collaboration, and thus, avoids many complicated issues such as users' privacy and security, selfishness, etc. In order to optimize users' quality of service and, concurrently, ensure the fairness among users, the probability that all users can decode the desired signals is maximized. In NOMA, a combination of multiple messages are sent to users, and the defined objective is approached by finding an appropriate power allocation for message signals. To address the power allocation problem, two novel methods are proposed. The first one is a divide-and-conquer-based method for which closed-form expressions for the optimal resource allocation policy are derived, making this method simple and flexible to the system context. The second one is based on the deep reinforcement learning method that allows all users to share the full bandwidth. Finally, simulation results are provided to demonstrate the effectiveness of the proposed methods and to compare their performance. △ Less

Submitted 24 September, 2019; originally announced September 2019.

Comments: To appear in the IEEE Transactions on Communications

Physics-Informed Echo State Networks for Chaotic Systems Forecasting

Authors: Nguyen Anh Khoa Doan, Wolfgang Polifke, Luca Magri

Abstract: We propose a physics-informed Echo State Network (ESN) to predict the evolution of chaotic systems. Compared to conventional ESNs, the physics-informed ESNs are trained to solve supervised learning tasks while ensuring that their predictions do not violate physical laws. This is achieved by introducing an additional loss function during the training of the ESNs, which penalizes non-physical predic… ▽ More We propose a physics-informed Echo State Network (ESN) to predict the evolution of chaotic systems. Compared to conventional ESNs, the physics-informed ESNs are trained to solve supervised learning tasks while ensuring that their predictions do not violate physical laws. This is achieved by introducing an additional loss function during the training of the ESNs, which penalizes non-physical predictions without the need of any additional training data. This approach is demonstrated on a chaotic Lorenz system, where the physics-informed ESNs improve the predictability horizon by about two Lyapunov times as compared to conventional ESNs. The proposed framework shows the potential of using machine learning combined with prior physical knowledge to improve the time-accurate prediction of chaotic dynamical systems. △ Less

Submitted 9 April, 2019; originally announced June 2019.

Comments: 7 pages, 3 figures

Journal ref: Computational Science - ICCS 2019. ICCS 2019. Lecture Notes in Computer Science, vol 11539. Springer, Cham

A counter-example to the equivariance structure on semi-universal deformation

Authors: An Khuong Doan

Abstract: If $X$ is a projective variety and $G$ is an algebraic group acting algebraically on $X$, we provide a counter-example to the existence of a $G$-equivariant extension on the formal semi-universal deformation of $X$. If $X$ is a projective variety and $G$ is an algebraic group acting algebraically on $X$, we provide a counter-example to the existence of a $G$-equivariant extension on the formal semi-universal deformation of $X$. △ Less

Submitted 6 April, 2020; v1 submitted 31 May, 2019; originally announced June 2019.

Comments: 13 pages, Remark 1.1, Definition 1.6, Definition 1.7, and Remark 4.1 have been added. A slight change in the proof of Theorem 4.1. To appear in The Journal of Geometric analysis

MSC Class: 14D15; 14B10; 13D10

Journal ref: The Journal of Geometric Analysis, Volume 31, Pages 3698-3712(2021)