Showing 1–39 of 39 results for author: Yeo, K

Neural Pose Representation Learning for Generating and Transferring Non-Rigid Object Poses

Authors: Seungwoo Yoo, Juil Koo, Kyeongmin Yeo, Minhyuk Sung

Abstract: We propose a novel method for learning representations of poses for 3D deformable objects, which specializes in 1) disentangling pose information from the object's identity, 2) facilitating the learning of pose variations, and 3) transferring pose information to other object identities. Based on these properties, our method enables the generation of 3D deformable objects with diversity in both ide… ▽ More We propose a novel method for learning representations of poses for 3D deformable objects, which specializes in 1) disentangling pose information from the object's identity, 2) facilitating the learning of pose variations, and 3) transferring pose information to other object identities. Based on these properties, our method enables the generation of 3D deformable objects with diversity in both identities and poses, using variations of a single object. It does not require explicit shape parameterization such as skeletons or joints, point-level or shape-level correspondence supervision, or variations of the target object for pose transfer. To achieve pose disentanglement, compactness for generative models, and transferability, we first design the pose extractor to represent the pose as a keypoint-based hybrid representation and the pose applier to learn an implicit deformation field. To better distill pose information from the object's geometry, we propose the implicit pose applier to output an intrinsic mesh property, the face Jacobian. Once the extracted pose information is transferred to the target object, the pose applier is fine-tuned in a self-supervised manner to better describe the target object's shapes with pose variations. The extracted poses are also used to train a cascaded diffusion model to enable the generation of novel poses. Our experiments with the DeformThings4D and Human datasets demonstrate state-of-the-art performance in pose transfer and the ability to generate diverse deformed shapes with various objects and poses. △ Less

Submitted 14 June, 2024; originally announced June 2024.

General Item Representation Learning for Cold-start Content Recommendations

Authors: Jooeun Kim, **ri Kim, Kwangeun Yeo, Eungi Kim, Kyoung-Woon On, Jonghwan Mun, Joonseok Lee

Abstract: Cold-start item recommendation is a long-standing challenge in recommendation systems. A common remedy is to use a content-based approach, but rich information from raw contents in various forms has not been fully utilized. In this paper, we propose a domain/data-agnostic item representation learning framework for cold-start recommendations, naturally equipped with multimodal alignment among vario… ▽ More Cold-start item recommendation is a long-standing challenge in recommendation systems. A common remedy is to use a content-based approach, but rich information from raw contents in various forms has not been fully utilized. In this paper, we propose a domain/data-agnostic item representation learning framework for cold-start recommendations, naturally equipped with multimodal alignment among various features by adopting a Transformer-based architecture. Our proposed model is end-to-end trainable completely free from classification labels, not just costly to collect but suboptimal for recommendation-purpose representation learning. From extensive experiments on real-world movie and news recommendation benchmarks, we verify that our approach better preserves fine-grained user taste than state-of-the-art baselines, universally applicable to multiple domains at large scale. △ Less

Submitted 21 April, 2024; originally announced April 2024.

Comments: 14 pages

SyncTweedies: A General Generative Framework Based on Synchronized Diffusions

Authors: Jaihoon Kim, Juil Koo, Kyeongmin Yeo, Minhyuk Sung

Abstract: We introduce a general framework for generating diverse visual content, including ambiguous images, panorama images, mesh textures, and Gaussian splat textures, by synchronizing multiple diffusion processes. We present exhaustive investigation into all possible scenarios for synchronizing multiple diffusion processes through a canonical space and analyze their characteristics across applications.… ▽ More We introduce a general framework for generating diverse visual content, including ambiguous images, panorama images, mesh textures, and Gaussian splat textures, by synchronizing multiple diffusion processes. We present exhaustive investigation into all possible scenarios for synchronizing multiple diffusion processes through a canonical space and analyze their characteristics across applications. In doing so, we reveal a previously unexplored case: averaging the outputs of Tweedie's formula while conducting denoising in multiple instance spaces. This case also provides the best quality with the widest applicability to downstream tasks. We name this case SyncTweedies. In our experiments generating visual content aforementioned, we demonstrate the superior quality of generation by SyncTweedies compared to other synchronization methods, optimization-based and iterative-update-based methods. △ Less

Submitted 20 June, 2024; v1 submitted 21 March, 2024; originally announced March 2024.

Comments: Project page: https://synctweedies.github.io/

A 3D super-resolution of wind fields via physics-informed pixel-wise self-attention generative adversarial network

Authors: Takuya Kurihana, Kyongmin Yeo, Daniela Szwarcman, Bruce Elmegreen, Karthik Mukkavilli, Johannes Schmude, Levente Klein

Abstract: To mitigate global warming, greenhouse gas sources need to be resolved at a high spatial resolution and monitored in time to ensure the reduction and ultimately elimination of the pollution source. However, the complexity of computation in resolving high-resolution wind fields left the simulations impractical to test different time lengths and model configurations. This study presents a preliminar… ▽ More To mitigate global warming, greenhouse gas sources need to be resolved at a high spatial resolution and monitored in time to ensure the reduction and ultimately elimination of the pollution source. However, the complexity of computation in resolving high-resolution wind fields left the simulations impractical to test different time lengths and model configurations. This study presents a preliminary development of a physics-informed super-resolution (SR) generative adversarial network (GAN) that super-resolves the three-dimensional (3D) low-resolution wind fields by upscaling x9 times. We develop a pixel-wise self-attention (PWA) module that learns 3D weather dynamics via a self-attention computation followed by a 2D convolution. We also employ a loss term that regularizes the self-attention map during pretraining, capturing the vertical convection process from input wind data. The new PWA SR-GAN shows the high-fidelity super-resolved 3D wind data, learns a wind structure at the high-frequency domain, and reduces the computational cost of a high-resolution wind simulation by x89.7 times. △ Less

Submitted 20 December, 2023; originally announced December 2023.

Comments: 7 pages, 4 figures, NeurIPS 2023 Workshop: Tackling Climate Change with Machine Learning

A Supervised Contrastive Learning Pretrain-Finetune Approach for Time Series

Authors: Trang H. Tran, Lam M. Nguyen, Kyongmin Yeo, Nam Nguyen, Roman Vaculin

Abstract: Foundation models have recently gained attention within the field of machine learning thanks to its efficiency in broad data processing. While researchers had attempted to extend this success to time series models, the main challenge is effectively extracting representations and transferring knowledge from pretraining datasets to the target finetuning dataset. To tackle this issue, we introduce a… ▽ More Foundation models have recently gained attention within the field of machine learning thanks to its efficiency in broad data processing. While researchers had attempted to extend this success to time series models, the main challenge is effectively extracting representations and transferring knowledge from pretraining datasets to the target finetuning dataset. To tackle this issue, we introduce a novel pretraining procedure that leverages supervised contrastive learning to distinguish features within each pretraining dataset. This pretraining phase enables a probabilistic similarity metric, which assesses the likelihood of a univariate sample being closely related to one of the pretraining datasets. Subsequently, using this similarity metric as a guide, we propose a fine-tuning procedure designed to enhance the accurate prediction of the target data by aligning it more closely with the learned dynamics of the pretraining datasets. Our experiments have shown promising results which demonstrate the efficacy of our approach. △ Less

Submitted 20 November, 2023; originally announced November 2023.

Transferability of Representations Learned using Supervised Contrastive Learning Trained on a Multi-Domain Dataset

Authors: Alvin De Jun Tan, Clement Tan, Chai Kiat Yeo

Abstract: Contrastive learning has shown to learn better quality representations than models trained using cross-entropy loss. They also transfer better to downstream datasets from different domains. However, little work has been done to explore the transferability of representations learned using contrastive learning when trained on a multi-domain dataset. In this paper, a study has been conducted using th… ▽ More Contrastive learning has shown to learn better quality representations than models trained using cross-entropy loss. They also transfer better to downstream datasets from different domains. However, little work has been done to explore the transferability of representations learned using contrastive learning when trained on a multi-domain dataset. In this paper, a study has been conducted using the Supervised Contrastive Learning framework to learn representations from the multi-domain DomainNet dataset and then evaluate the transferability of the representations learned on other downstream datasets. The fixed feature linear evaluation protocol will be used to evaluate the transferability on 7 downstream datasets that were chosen across different domains. The results obtained are compared to a baseline model that was trained using the widely used cross-entropy loss. Empirical results from the experiments showed that on average, the Supervised Contrastive Learning model performed 6.05% better than the baseline model on the 7 downstream datasets. The findings suggest that Supervised Contrastive Learning models can potentially learn more robust representations that transfer better across domains than cross-entropy models when trained on a multi-domain dataset. △ Less

Submitted 27 September, 2023; originally announced September 2023.

DeformToon3D: Deformable 3D Toonification from Neural Radiance Fields

Authors: Junzhe Zhang, Yushi Lan, Shuai Yang, Fangzhou Hong, Quan Wang, Chai Kiat Yeo, Ziwei Liu, Chen Change Loy

Abstract: In this paper, we address the challenging problem of 3D toonification, which involves transferring the style of an artistic domain onto a target 3D face with stylized geometry and texture. Although fine-tuning a pre-trained 3D GAN on the artistic domain can produce reasonable performance, this strategy has limitations in the 3D domain. In particular, fine-tuning can deteriorate the original GAN la… ▽ More In this paper, we address the challenging problem of 3D toonification, which involves transferring the style of an artistic domain onto a target 3D face with stylized geometry and texture. Although fine-tuning a pre-trained 3D GAN on the artistic domain can produce reasonable performance, this strategy has limitations in the 3D domain. In particular, fine-tuning can deteriorate the original GAN latent space, which affects subsequent semantic editing, and requires independent optimization and storage for each new style, limiting flexibility and efficient deployment. To overcome these challenges, we propose DeformToon3D, an effective toonification framework tailored for hierarchical 3D GAN. Our approach decomposes 3D toonification into subproblems of geometry and texture stylization to better preserve the original latent space. Specifically, we devise a novel StyleField that predicts conditional 3D deformation to align a real-space NeRF to the style space for geometry stylization. Thanks to the StyleField formulation, which already handles geometry stylization well, texture stylization can be achieved conveniently via adaptive style mixing that injects information of the artistic domain into the decoder of the pre-trained 3D GAN. Due to the unique design, our method enables flexible style degree control and shape-texture-specific style swap. Furthermore, we achieve efficient training without any real-world 2D-3D training pairs but proxy samples synthesized from off-the-shelf 2D toonification models. △ Less

Submitted 8 September, 2023; originally announced September 2023.

Comments: ICCV 2023. Code: https://github.com/junzhezhang/DeformToon3D Project page: https://www.mmlab-ntu.com/project/deformtoon3d/

Optimal Non-Adaptive Cell Probe Dictionaries and Hashing

Authors: Kasper Green Larsen, Rasmus Pagh, Giuseppe Persiano, Toniann Pitassi, Kevin Yeo, Or Zamir

Abstract: We present a simple and provably optimal non-adaptive cell probe data structure for the static dictionary problem. Our data structure supports storing a set of n key-value pairs from [u]x[u] using s words of space and answering key lookup queries in t = O(lg(u/n)/ lg(s/n)) nonadaptive probes. This generalizes a solution to the membership problem (i.e., where no values are associated with keys) due… ▽ More We present a simple and provably optimal non-adaptive cell probe data structure for the static dictionary problem. Our data structure supports storing a set of n key-value pairs from [u]x[u] using s words of space and answering key lookup queries in t = O(lg(u/n)/ lg(s/n)) nonadaptive probes. This generalizes a solution to the membership problem (i.e., where no values are associated with keys) due to Buhrman et al. We also present matching lower bounds for the non-adaptive static membership problem in the deterministic setting. Our lower bound implies that both our dictionary algorithm and the preceding membership algorithm are optimal, and in particular that there is an inherent complexity gap in these problems between no adaptivity and one round of adaptivity (with which hashing-based algorithms solve these problems in constant time). Using the ideas underlying our data structure, we also obtain the first implementation of a n-wise independent family of hash functions with optimal evaluation time in the cell probe model. △ Less

Submitted 19 April, 2024; v1 submitted 30 August, 2023; originally announced August 2023.

Comments: Appears at ICALP 2024. This paper is a merge and revision of two previous reports [PY20] and [LPPZ23]

Cuckoo Hashing in Cryptography: Optimal Parameters, Robustness and Applications

Authors: Kevin Yeo

Abstract: Cuckoo hashing is a powerful primitive that enables storing items using small space with efficient querying. At a high level, cuckoo hashing maps $n$ items into $b$ entries storing at most $\ell$ items such that each item is placed into one of $k$ randomly chosen entries. Additionally, there is an overflow stash that can store at most $s$ items. Many cryptographic primitives rely upon cuckoo hashi… ▽ More Cuckoo hashing is a powerful primitive that enables storing items using small space with efficient querying. At a high level, cuckoo hashing maps $n$ items into $b$ entries storing at most $\ell$ items such that each item is placed into one of $k$ randomly chosen entries. Additionally, there is an overflow stash that can store at most $s$ items. Many cryptographic primitives rely upon cuckoo hashing to privately embed and query data where it is integral to ensure small failure probability when constructing cuckoo hashing tables as it directly relates to the privacy guarantees. As our main result, we present a more query-efficient cuckoo hashing construction using more hash functions. For construction failure probability $ε$, the query overhead of our scheme is $O(1 + \sqrt{\log(1/ε)/\log n})$. Our scheme has quadratically smaller query overhead than prior works for any target failure probability $ε$. We also prove lower bounds matching our construction. Our improvements come from a new understanding of the locality of cuckoo hashing failures for small sets of items. We also initiate the study of robust cuckoo hashing where the input set may be chosen with knowledge of the hash functions. We present a cuckoo hashing scheme using more hash functions with query overhead $\tilde{O}(\log λ)$ that is robust against poly$(λ)$ adversaries. Furthermore, we present lower bounds showing that this construction is tight and that extending previous approaches of large stashes or entries cannot obtain robustness except with $Ω(n)$ query overhead. As applications of our results, we obtain improved constructions for batch codes and PIR. In particular, we present the most efficient explicit batch code and blackbox reduction from single-query PIR to batch PIR. △ Less

Submitted 19 June, 2023; originally announced June 2023.

Comments: Full version of CRYPTO 2023 paper, 45 pages

An End-to-End Time Series Model for Simultaneous Imputation and Forecast

Authors: Trang H. Tran, Lam M. Nguyen, Kyongmin Yeo, Nam Nguyen, Dzung Phan, Roman Vaculin, Jayant Kalagnanam

Abstract: Time series forecasting using historical data has been an interesting and challenging topic, especially when the data is corrupted by missing values. In many industrial problem, it is important to learn the inference function between the auxiliary observations and target variables as it provides additional knowledge when the data is not fully observed. We develop an end-to-end time series model th… ▽ More Time series forecasting using historical data has been an interesting and challenging topic, especially when the data is corrupted by missing values. In many industrial problem, it is important to learn the inference function between the auxiliary observations and target variables as it provides additional knowledge when the data is not fully observed. We develop an end-to-end time series model that aims to learn the such inference relation and make a multiple-step ahead forecast. Our framework trains jointly two neural networks, one to learn the feature-wise correlations and the other for the modeling of temporal behaviors. Our model is capable of simultaneously imputing the missing entries and making a multiple-step ahead prediction. The experiments show good overall performance of our framework over existing methods in both imputation and forecasting tasks. △ Less

Submitted 1 June, 2023; originally announced June 2023.

Code Will Tell: Visual Identification of Ponzi Schemes on Ethereum

Authors: Xiaolin Wen, Kim Siang Yeo, Yong Wang, Ling Cheng, Feida Zhu, Min Zhu

Abstract: Ethereum has become a popular blockchain with smart contracts for investors nowadays. Due to the decentralization and anonymity of Ethereum, Ponzi schemes have been easily deployed and caused significant losses to investors. However, there are still no explainable and effective methods to help investors easily identify Ponzi schemes and validate whether a smart contract is actually a Ponzi scheme.… ▽ More Ethereum has become a popular blockchain with smart contracts for investors nowadays. Due to the decentralization and anonymity of Ethereum, Ponzi schemes have been easily deployed and caused significant losses to investors. However, there are still no explainable and effective methods to help investors easily identify Ponzi schemes and validate whether a smart contract is actually a Ponzi scheme. To fill the research gap, we propose PonziLens, a novel visualization approach to help investors achieve early identification of Ponzi schemes by investigating the operation codes of smart contracts. Specifically, we conduct symbolic execution of opcode and extract the control flow for investing and rewarding with critical opcode instructions. Then, an intuitive directed-graph based visualization is proposed to display the investing and rewarding flows and the crucial execution paths, enabling easy identification of Ponzi schemes on Ethereum. Two usage scenarios involving both Ponzi and non-Ponzi schemes demonstrate the effectiveness of PonziLens. △ Less

Submitted 14 March, 2023; originally announced March 2023.

Multi-task Learning for Source Attribution and Field Reconstruction for Methane Monitoring

Authors: Arka Daw, Kyongmin Yeo, Anuj Karpatne, Levente Klein

Abstract: Inferring the source information of greenhouse gases, such as methane, from spatially sparse sensor observations is an essential element in mitigating climate change. While it is well understood that the complex behavior of the atmospheric dispersion of such pollutants is governed by the Advection-Diffusion equation, it is difficult to directly apply the governing equations to identify the source… ▽ More Inferring the source information of greenhouse gases, such as methane, from spatially sparse sensor observations is an essential element in mitigating climate change. While it is well understood that the complex behavior of the atmospheric dispersion of such pollutants is governed by the Advection-Diffusion equation, it is difficult to directly apply the governing equations to identify the source location and magnitude (inverse problem) because of the spatially sparse and noisy observations, i.e., the pollution concentration is known only at the sensor locations and sensors sensitivity is limited. Here, we develop a multi-task learning framework that can provide high-fidelity reconstruction of the concentration field and identify emission characteristics of the pollution sources such as their location, emission strength, etc. from sparse sensor observations. We demonstrate that our proposed framework is able to achieve accurate reconstruction of the methane concentrations from sparse sensor measurements as well as precisely pin-point the location and emission strength of these pollution sources. △ Less

Submitted 2 November, 2022; originally announced November 2022.

Comments: 7 pages, 8 figures, 1 table

Abductive Action Inference

Authors: Clement Tan, Chai Kiat Yeo, Cheston Tan, Basura Fernando

Abstract: Abductive reasoning aims to make the most likely inference for a given set of incomplete observations. In this paper, we introduce a novel research task known as "abductive action inference" which addresses the question of which actions were executed by a human to reach a specific state shown in a single snapshot. The research explores three key abductive inference problems: action set prediction,… ▽ More Abductive reasoning aims to make the most likely inference for a given set of incomplete observations. In this paper, we introduce a novel research task known as "abductive action inference" which addresses the question of which actions were executed by a human to reach a specific state shown in a single snapshot. The research explores three key abductive inference problems: action set prediction, action sequence prediction, and abductive action verification. To tackle these challenging tasks, we investigate various models, including established ones such as Transformers, Graph Neural Networks, CLIP, BLIP, GPT3, end-to-end trained Slow-Fast, Resnet50-3D, and ViT models. Furthermore, the paper introduces several innovative models tailored for abductive action inference, including a relational graph neural network, a relational bilinear pooling model, a relational rule-based inference model, a relational GPT-3 prompt method, and a relational Transformer model. Notably, the newly proposed object-relational bilinear graph encoder-decoder (BiGED) model emerges as the most effective among all methods evaluated, demonstrating good proficiency in handling the intricacies of the Action Genome dataset. The contributions of this research offer significant progress toward comprehending the implications of human actions and making highly plausible inferences concerning the outcomes of these actions. △ Less

Submitted 7 August, 2023; v1 submitted 24 October, 2022; originally announced October 2022.

Comments: 16 pages, 9 figures

Monocular 3D Object Reconstruction with GAN Inversion

Authors: Junzhe Zhang, Daxuan Ren, Zhongang Cai, Chai Kiat Yeo, Bo Dai, Chen Change Loy

Abstract: Recovering a textured 3D mesh from a monocular image is highly challenging, particularly for in-the-wild objects that lack 3D ground truths. In this work, we present MeshInversion, a novel framework to improve the reconstruction by exploiting the generative prior of a 3D GAN pre-trained for 3D textured mesh synthesis. Reconstruction is achieved by searching for a latent space in the 3D GAN that be… ▽ More Recovering a textured 3D mesh from a monocular image is highly challenging, particularly for in-the-wild objects that lack 3D ground truths. In this work, we present MeshInversion, a novel framework to improve the reconstruction by exploiting the generative prior of a 3D GAN pre-trained for 3D textured mesh synthesis. Reconstruction is achieved by searching for a latent space in the 3D GAN that best resembles the target mesh in accordance with the single view observation. Since the pre-trained GAN encapsulates rich 3D semantics in terms of mesh geometry and texture, searching within the GAN manifold thus naturally regularizes the realness and fidelity of the reconstruction. Importantly, such regularization is directly applied in the 3D space, providing crucial guidance of mesh parts that are unobserved in the 2D space. Experiments on standard benchmarks show that our framework obtains faithful 3D reconstructions with consistent geometry and texture across both observed and unobserved parts. Moreover, it generalizes well to meshes that are less commonly seen, such as the extended articulation of deformable objects. Code is released at https://github.com/junzhezhang/mesh-inversion △ Less

Submitted 20 July, 2022; originally announced July 2022.

Comments: ECCV 2022. Project page: https://www.mmlab-ntu.com/project/meshinversion/

Super Resolution for Turbulent Flows in 2D: Stabilized Physics Informed Neural Networks

Authors: Mykhaylo Zayats, Małgorzata J. Zimoń, Kyongmin Yeo, Sergiy Zhuk

Abstract: We propose a new design of a neural network for solving a zero shot super resolution problem for turbulent flows. We embed Luenberger-type observer into the network's architecture to inform the network of the physics of the process, and to provide error correction and stabilization mechanisms. In addition, to compensate for decrease of observer's performance due to the presence of unknown destabil… ▽ More We propose a new design of a neural network for solving a zero shot super resolution problem for turbulent flows. We embed Luenberger-type observer into the network's architecture to inform the network of the physics of the process, and to provide error correction and stabilization mechanisms. In addition, to compensate for decrease of observer's performance due to the presence of unknown destabilizing forcing, the network is designed to estimate the contribution of the unknown forcing implicitly from the data over the course of training. By running a set of numerical experiments, we demonstrate that the proposed network does recover unknown forcing from data and is capable of predicting turbulent flows in high resolution from low resolution noisy observations. △ Less

Submitted 15 April, 2022; originally announced April 2022.

MSC Class: 65P20 (Primary) 68T07; 37M05 (Secondary)

SoK: SCT Auditing in Certificate Transparency

Authors: Sarah Meiklejohn, Joe DeBlasio, Devon O'Brien, Chris Thompson, Kevin Yeo, Emily Stark

Abstract: The Web public key infrastructure is essential to providing secure communication on the Internet today, and certificate authorities play a crucial role in this ecosystem by issuing certificates. These authorities may misissue certificates or suffer misuse attacks, however, which has given rise to the Certificate Transparency (CT) project. The goal of CT is to store all issued certificates in publi… ▽ More The Web public key infrastructure is essential to providing secure communication on the Internet today, and certificate authorities play a crucial role in this ecosystem by issuing certificates. These authorities may misissue certificates or suffer misuse attacks, however, which has given rise to the Certificate Transparency (CT) project. The goal of CT is to store all issued certificates in public logs, which can then be checked for the presence of potentially misissued certificates. Thus, the requirement that a given certificate is indeed in one (or several) of these logs lies at the core of CT. In its current deployment, however, most individual clients do not check that the certificates they see are in logs, as requesting a proof of inclusion directly reveals the certificate and thus creates the clear potential for a violation of that client's privacy. In this paper, we explore the techniques that have been proposed for privacy-preserving auditing of certificate inclusion, focusing on their effectiveness, efficiency, and suitability in a near-term deployment. In doing so, we also explore the parallels with related problems involving browser clients. Guided by a set of constraints that we develop, we ultimately observe several key limitations in many proposals, ranging from their privacy provisions to the fact that they focus on the interaction between a client and a log but leave open the question of how a client could privately report any certificates that are missing. △ Less

Submitted 3 March, 2022; originally announced March 2022.

Comments: PETS 2022, issue 3

NSGZero: Efficiently Learning Non-Exploitable Policy in Large-Scale Network Security Games with Neural Monte Carlo Tree Search

Authors: Wanqi Xue, Bo An, Chai Kiat Yeo

Abstract: How resources are deployed to secure critical targets in networks can be modelled by Network Security Games (NSGs). While recent advances in deep learning (DL) provide a powerful approach to dealing with large-scale NSGs, DL methods such as NSG-NFSP suffer from the problem of data inefficiency. Furthermore, due to centralized control, they cannot scale to scenarios with a large number of resources… ▽ More How resources are deployed to secure critical targets in networks can be modelled by Network Security Games (NSGs). While recent advances in deep learning (DL) provide a powerful approach to dealing with large-scale NSGs, DL methods such as NSG-NFSP suffer from the problem of data inefficiency. Furthermore, due to centralized control, they cannot scale to scenarios with a large number of resources. In this paper, we propose a novel DL-based method, NSGZero, to learn a non-exploitable policy in NSGs. NSGZero improves data efficiency by performing planning with neural Monte Carlo Tree Search (MCTS). Our main contributions are threefold. First, we design deep neural networks (DNNs) to perform neural MCTS in NSGs. Second, we enable neural MCTS with decentralized control, making NSGZero applicable to NSGs with many resources. Third, we provide an efficient learning paradigm, to achieve joint training of the DNNs in NSGZero. Compared to state-of-the-art algorithms, our method achieves significantly better data efficiency and scalability. △ Less

Submitted 17 January, 2022; originally announced January 2022.

Comments: Published as a conference paper in AAAI 2022

S3RP: Self-Supervised Super-Resolution and Prediction for Advection-Diffusion Process

Authors: Chulin Wang, Kyongmin Yeo, Xiao **, Andres Codas, Levente J. Klein, Bruce Elmegreen

Abstract: We present a super-resolution model for an advection-diffusion process with limited information. While most of the super-resolution models assume high-resolution (HR) ground-truth data in the training, in many cases such HR dataset is not readily accessible. Here, we show that a Recurrent Convolutional Network trained with physics-based regularizations is able to reconstruct the HR information wit… ▽ More We present a super-resolution model for an advection-diffusion process with limited information. While most of the super-resolution models assume high-resolution (HR) ground-truth data in the training, in many cases such HR dataset is not readily accessible. Here, we show that a Recurrent Convolutional Network trained with physics-based regularizations is able to reconstruct the HR information without having the HR ground-truth data. Moreover, considering the ill-posed nature of a super-resolution problem, we employ the Recurrent Wasserstein Autoencoder to model the uncertainty. △ Less

Submitted 8 November, 2021; originally announced November 2021.

Comments: 9 pages, 8 figures

Journal ref: Neural Information Processing Systems (NeurIPS 2021) Workshop

Generative Adversarial Network for Probabilistic Forecast of Random Dynamical System

Authors: Kyongmin Yeo, Zan Li, Wesley M. Gifford

Abstract: We present a deep learning model for data-driven simulations of random dynamical systems without a distributional assumption. The deep learning model consists of a recurrent neural network, which aims to learn the time marching structure, and a generative adversarial network (GAN) to learn and sample from the probability distribution of the random dynamical system. Although GANs provide a powerful… ▽ More We present a deep learning model for data-driven simulations of random dynamical systems without a distributional assumption. The deep learning model consists of a recurrent neural network, which aims to learn the time marching structure, and a generative adversarial network (GAN) to learn and sample from the probability distribution of the random dynamical system. Although GANs provide a powerful tool to model a complex probability distribution, the training often fails without a proper regularization. Here, we propose a regularization strategy for a GAN based on consistency conditions for the sequential inference problems. First, the maximum mean discrepancy (MMD) is used to enforce the consistency between conditional and marginal distributions of a stochastic process. Then, the marginal distributions of the multiple-step predictions are regularized by using MMD or from multiple discriminators. The behavior of the proposed model is studied by using three stochastic processes with complex noise structures. △ Less

Submitted 11 April, 2022; v1 submitted 4 November, 2021; originally announced November 2021.

Mis-spoke or mis-lead: Achieving Robustness in Multi-Agent Communicative Reinforcement Learning

Authors: Wanqi Xue, Wei Qiu, Bo An, Zinovi Rabinovich, Svetlana Obraztsova, Chai Kiat Yeo

Abstract: Recent studies in multi-agent communicative reinforcement learning (MACRL) have demonstrated that multi-agent coordination can be greatly improved by allowing communication between agents. Meanwhile, adversarial machine learning (ML) has shown that ML models are vulnerable to attacks. Despite the increasing concern about the robustness of ML algorithms, how to achieve robust communication in multi… ▽ More Recent studies in multi-agent communicative reinforcement learning (MACRL) have demonstrated that multi-agent coordination can be greatly improved by allowing communication between agents. Meanwhile, adversarial machine learning (ML) has shown that ML models are vulnerable to attacks. Despite the increasing concern about the robustness of ML algorithms, how to achieve robust communication in multi-agent reinforcement learning has been largely neglected. In this paper, we systematically explore the problem of adversarial communication in MACRL. Our main contributions are threefold. First, we propose an effective method to perform attacks in MACRL, by learning a model to generate optimal malicious messages. Second, we develop a defence method based on message reconstruction, to maintain multi-agent coordination under message attacks. Third, we formulate the adversarial communication problem as a two-player zero-sum game and propose a game-theoretical method R-MACRL to improve the worst-case defending performance. Empirical results demonstrate that many state-of-the-art MACRL methods are vulnerable to message attacks, and our method can significantly improve their robustness. △ Less

Submitted 26 January, 2022; v1 submitted 9 August, 2021; originally announced August 2021.

Comments: Published as a conference paper in AAMAS 2022

Solving Large-Scale Extensive-Form Network Security Games via Neural Fictitious Self-Play

Authors: Wanqi Xue, Youzhi Zhang, Shuxin Li, Xinrun Wang, Bo An, Chai Kiat Yeo

Abstract: Securing networked infrastructures is important in the real world. The problem of deploying security resources to protect against an attacker in networked domains can be modeled as Network Security Games (NSGs). Unfortunately, existing approaches, including the deep learning-based approaches, are inefficient to solve large-scale extensive-form NSGs. In this paper, we propose a novel learning parad… ▽ More Securing networked infrastructures is important in the real world. The problem of deploying security resources to protect against an attacker in networked domains can be modeled as Network Security Games (NSGs). Unfortunately, existing approaches, including the deep learning-based approaches, are inefficient to solve large-scale extensive-form NSGs. In this paper, we propose a novel learning paradigm, NSG-NFSP, to solve large-scale extensive-form NSGs based on Neural Fictitious Self-Play (NFSP). Our main contributions include: i) reforming the best response (BR) policy network in NFSP to be a map** from action-state pair to action-value, to make the calculation of BR possible in NSGs; ii) converting the average policy network of an NFSP agent into a metric-based classifier, hel** the agent to assign distributions only on legal actions rather than all actions; iii) enabling NFSP with high-level actions, which can benefit training efficiency and stability in NSGs; and iv) leveraging information contained in graphs of NSGs by learning efficient graph node embeddings. Our algorithm significantly outperforms state-of-the-art algorithms in both scalability and solution quality. △ Less

Submitted 1 June, 2021; originally announced June 2021.

Comments: Published as a conference paper in IJCAI 2021

Unsupervised 3D Shape Completion through GAN Inversion

Authors: Junzhe Zhang, Xinyi Chen, Zhongang Cai, Liang Pan, Haiyu Zhao, Shuai Yi, Chai Kiat Yeo, Bo Dai, Chen Change Loy

Abstract: Most 3D shape completion approaches rely heavily on partial-complete shape pairs and learn in a fully supervised manner. Despite their impressive performances on in-domain data, when generalizing to partial shapes in other forms or real-world partial scans, they often obtain unsatisfactory results due to domain gaps. In contrast to previous fully supervised approaches, in this paper we present Sha… ▽ More Most 3D shape completion approaches rely heavily on partial-complete shape pairs and learn in a fully supervised manner. Despite their impressive performances on in-domain data, when generalizing to partial shapes in other forms or real-world partial scans, they often obtain unsatisfactory results due to domain gaps. In contrast to previous fully supervised approaches, in this paper we present ShapeInversion, which introduces Generative Adversarial Network (GAN) inversion to shape completion for the first time. ShapeInversion uses a GAN pre-trained on complete shapes by searching for a latent code that gives a complete shape that best reconstructs the given partial input. In this way, ShapeInversion no longer needs paired training data, and is capable of incorporating the rich prior captured in a well-trained generative model. On the ShapeNet benchmark, the proposed ShapeInversion outperforms the SOTA unsupervised method, and is comparable with supervised methods that are learned using paired data. It also demonstrates remarkable generalization ability, giving robust results for real-world scans and partial inputs of various forms and incompleteness levels. Importantly, ShapeInversion naturally enables a series of additional abilities thanks to the involvement of a pre-trained GAN, such as producing multiple valid complete shapes for an ambiguous partial input, as well as shape manipulation and interpolation. △ Less

Submitted 29 April, 2021; v1 submitted 27 April, 2021; originally announced April 2021.

Comments: Accepted in CVPR 2021, project webpage: https://junzhezhang.github.io/projects/ShapeInversion/

Automated Deep Learning Analysis of Angiography Video Sequences for Coronary Artery Disease

Authors: Chengyang Zhou, Thao Vy Dinh, Heyi Kong, Jonathan Yap, Khung Keong Yeo, Hwee Kuan Lee, Kaicheng Liang

Abstract: The evaluation of obstructions (stenosis) in coronary arteries is currently done by a physician's visual assessment of coronary angiography video sequences. It is laborious, and can be susceptible to interobserver variation. Prior studies have attempted to automate this process, but few have demonstrated an integrated suite of algorithms for the end-to-end analysis of angiograms. We report an auto… ▽ More The evaluation of obstructions (stenosis) in coronary arteries is currently done by a physician's visual assessment of coronary angiography video sequences. It is laborious, and can be susceptible to interobserver variation. Prior studies have attempted to automate this process, but few have demonstrated an integrated suite of algorithms for the end-to-end analysis of angiograms. We report an automated analysis pipeline based on deep learning to rapidly and objectively assess coronary angiograms, highlight coronary vessels of interest, and quantify potential stenosis. We propose a 3-stage automated analysis method consisting of key frame extraction, vessel segmentation, and stenosis measurement. We combined powerful deep learning approaches such as ResNet and U-Net with traditional image processing and geometrical analysis. We trained and tested our algorithms on the Left Anterior Oblique (LAO) view of the right coronary artery (RCA) using anonymized angiograms obtained from a tertiary cardiac institution, then tested the generalizability of our technique to the Right Anterior Oblique (RAO) view. We demonstrated an overall improvement on previous work, with key frame extraction top-5 precision of 98.4%, vessel segmentation F1-Score of 0.891 and stenosis measurement 20.7% Type I Error rate. △ Less

Submitted 29 January, 2021; originally announced January 2021.

Self-Organizing Map assisted Deep Autoencoding Gaussian Mixture Model for Intrusion Detection

Authors: Yang Chen, Nami Ashizawa, Seanglidet Yean, Chai Kiat Yeo, Naoto Yanai

Abstract: In the information age, a secure and stable network environment is essential and hence intrusion detection is critical for any networks. In this paper, we propose a self-organizing map assisted deep autoencoding Gaussian mixture model (SOMDAGMM) supplemented with well-preserved input space topology for more accurate network intrusion detection. The deep autoencoding Gaussian mixture model comprise… ▽ More In the information age, a secure and stable network environment is essential and hence intrusion detection is critical for any networks. In this paper, we propose a self-organizing map assisted deep autoencoding Gaussian mixture model (SOMDAGMM) supplemented with well-preserved input space topology for more accurate network intrusion detection. The deep autoencoding Gaussian mixture model comprises a compression network and an estimation network which is able to perform unsupervised joint training. However, the code generated by the autoencoder is inept at preserving the topology of the input space, which is rooted in the bottleneck of the adopted deep structure. A self-organizing map has been introduced to construct SOMDAGMM for addressing this issue. The superiority of the proposed SOM-DAGMM is empirically demonstrated with extensive experiments conducted upon two datasets. Experimental results show that SOM-DAGMM outperforms state-of-the-art DAGMM on all tests, and achieves up to 15.58% improvement in F1 score and with better stability. △ Less

Submitted 28 August, 2020; originally announced August 2020.

MessyTable: Instance Association in Multiple Camera Views

Authors: Zhongang Cai, Junzhe Zhang, Daxuan Ren, Cunjun Yu, Haiyu Zhao, Shuai Yi, Chai Kiat Yeo, Chen Change Loy

Abstract: We present an interesting and challenging dataset that features a large number of scenes with messy tables captured from multiple camera views. Each scene in this dataset is highly complex, containing multiple object instances that could be identical, stacked and occluded by other instances. The key challenge is to associate all instances given the RGB image of all views. The seemingly simple task… ▽ More We present an interesting and challenging dataset that features a large number of scenes with messy tables captured from multiple camera views. Each scene in this dataset is highly complex, containing multiple object instances that could be identical, stacked and occluded by other instances. The key challenge is to associate all instances given the RGB image of all views. The seemingly simple task surprisingly fails many popular methods or heuristics that we assume good performance in object association. The dataset challenges existing methods in mining subtle appearance differences, reasoning based on contexts, and fusing appearance with geometric cues for establishing an association. We report interesting findings with some popular baselines, and discuss how this dataset could help inspire new problems and catalyse more robust formulations to tackle real-world instance association problems. Project page: $\href{https://caizhongang.github.io/projects/MessyTable/}{\text{MessyTable}}$ △ Less

Submitted 29 July, 2020; originally announced July 2020.

Comments: Accepted in ECCV 2020

Variational inference formulation for a model-free simulation of a dynamical system with unknown parameters by a recurrent neural network

Authors: Kyongmin Yeo, Dylan E. C. Grullon, Fan-Keng Sun, Duane S. Boning, Jayant R. Kalagnanam

Abstract: We propose a recurrent neural network for a "model-free" simulation of a dynamical system with unknown parameters without prior knowledge. The deep learning model aims to jointly learn the nonlinear time marching operator and the effects of the unknown parameters from a time series dataset. We assume that the time series data set consists of an ensemble of trajectories for a range of the parameter… ▽ More We propose a recurrent neural network for a "model-free" simulation of a dynamical system with unknown parameters without prior knowledge. The deep learning model aims to jointly learn the nonlinear time marching operator and the effects of the unknown parameters from a time series dataset. We assume that the time series data set consists of an ensemble of trajectories for a range of the parameters. The learning task is formulated as a statistical inference problem by considering the unknown parameters as random variables. A latent variable is introduced to model the effects of the unknown parameters, and a variational inference method is employed to simultaneously train probabilistic models for the time marching operator and an approximate posterior distribution for the latent variable. Unlike the classical variational inference, where a factorized distribution is used to approximate the posterior, we employ a feedforward neural network supplemented by an encoder recurrent neural network to develop a more flexible probabilistic model. The approximate posterior distribution makes an inference on a trajectory to identify the effects of the unknown parameters. The time marching operator is approximated by a recurrent neural network, which takes a latent state sampled from the approximate posterior distribution as one of the input variables, to compute the time evolution of the probability distribution conditioned on the latent variable. In the numerical experiments, it is shown that the proposed variational inference model makes a more accurate simulation compared to the standard recurrent neural networks. It is found that the proposed deep learning model is capable of correctly identifying the dimensions of the random parameters and learning a representation of complex time series data. △ Less

Submitted 26 February, 2021; v1 submitted 2 March, 2020; originally announced March 2020.

Tight Static Lower Bounds for Non-Adaptive Data Structures

Authors: Giuseppe Persiano, Kevin Yeo

Abstract: In this paper, we study the static cell probe complexity of non-adaptive data structures that maintain a subset of $n$ points from a universe consisting of $m=n^{1+Ω(1)}$ points. A data structure is defined to be non-adaptive when the memory locations that are chosen to be accessed during a query depend only on the query inputs and not on the contents of memory. We prove an… ▽ More In this paper, we study the static cell probe complexity of non-adaptive data structures that maintain a subset of $n$ points from a universe consisting of $m=n^{1+Ω(1)}$ points. A data structure is defined to be non-adaptive when the memory locations that are chosen to be accessed during a query depend only on the query inputs and not on the contents of memory. We prove an $Ω(\log m / \log (sw/n\log m))$ static cell probe complexity lower bound for non-adaptive data structures that solve the fundamental dictionary problem where $s$ denotes the space of the data structure in the number of cells and $w$ is the cell size in bits. Our lower bounds hold for all word sizes including the bit probe model ($w = 1$) and are matched by the upper bounds of Boninger et al. [FSTTCS'17]. Our results imply a sharp dichotomy between dictionary data structures with one round of adaptive and at least two rounds of adaptivity. We show that $O(1)$, or $O(\log^{1-ε}(m))$, overhead dictionary constructions are only achievable with at least two rounds of adaptivity. In particular, we show that many $O(1)$ dictionary constructions with two rounds of adaptivity such as cuckoo hashing are optimal in terms of adaptivity. On the other hand, non-adaptive dictionaries must use significantly more overhead. Finally, our results also imply static lower bounds for the non-adaptive predecessor problem. Our static lower bounds peak higher than the previous, best known lower bounds of $Ω(\log m / \log w)$ for the dynamic predecessor problem by Boninger et al. [FSTTCS'17] and Ramamoorthy and Rao [CCC'18] in the natural setting of linear space $s = Θ(n)$ where each point can fit in a single cell $w = Θ(\log m)$. Furthermore, our results are stronger as they apply to the static setting unlike the previous lower bounds that only applied in the dynamic setting. △ Less

Submitted 17 April, 2024; v1 submitted 14 January, 2020; originally announced January 2020.

Comments: This paper has been superceded and merged with arXiv:2308.16042

Data-driven Reconstruction of Nonlinear Dynamics from Sparse Observation

Authors: Kyongmin Yeo

Abstract: We present a data-driven model to reconstruct nonlinear dynamics from a very sparse times series data, which relies on the strength of the echo state network (ESN) in learning nonlinear representation of data. With an assumption of the universal function approximation capability of ESN, it is shown that the reconstruction problem can be formulated as a fixed-point problem, in which the trajectory… ▽ More We present a data-driven model to reconstruct nonlinear dynamics from a very sparse times series data, which relies on the strength of the echo state network (ESN) in learning nonlinear representation of data. With an assumption of the universal function approximation capability of ESN, it is shown that the reconstruction problem can be formulated as a fixed-point problem, in which the trajectory of the dynamical system is a fixed point of the ESN. An under-relaxed fixed-point iteration is proposed to reconstruct the nonlinear dynamics from a sparse observation. The proposed fixed-point ESN is tested against both univariate and multivariate chaotic dynamical systems by randomly removing up to 95% of the data. It is shown that the fixed-point ESN is able to reconstruct the complex dynamics from only 5 ~ 10% of the data. For a relatively simple non-chaotic dynamical system, the numerical experiments on a forced van der Pol oscillator show that it is possible to reconstruct the nonlinear dynamics from only 1~2% of the data. △ Less

Submitted 10 June, 2019; originally announced June 2019.

What Storage Access Privacy is Achievable with Small Overhead?

Authors: Sarvar Patel, Giuseppe Persiano, Kevin Yeo

Abstract: Oblivious RAM (ORAM) and private information retrieval (PIR) are classic cryptographic primitives used to hide the access pattern to data whose storage has been outsourced to an untrusted server. Unfortunately, both primitives require considerable overhead compared to plaintext access. For large-scale storage infrastructure with highly frequent access requests, the degradation in response time and… ▽ More Oblivious RAM (ORAM) and private information retrieval (PIR) are classic cryptographic primitives used to hide the access pattern to data whose storage has been outsourced to an untrusted server. Unfortunately, both primitives require considerable overhead compared to plaintext access. For large-scale storage infrastructure with highly frequent access requests, the degradation in response time and the exorbitant increase in resource costs incurred by either ORAM or PIR prevent their usage. In an ideal scenario, a privacy-preserving storage protocols with small overhead would be implemented for these heavily trafficked storage systems to avoid negatively impacting either performance and/or costs. In this work, we study the problem of the best $\mathit{storage\ access\ privacy}$ that is achievable with only $\mathit{small\ overhead}$ over plaintext access. To answer this question, we consider $\mathit{differential\ privacy\ access}$ which is a generalization of the $\mathit{oblivious\ access}$ security notion that are considered by ORAM and PIR. Quite surprisingly, we present strong evidence that constant overhead storage schemes may only be achieved with privacy budgets of $ε= Ω(\log n)$. We present asymptotically optimal constructions for differentially private variants of both ORAM and PIR with privacy budgets $ε= Θ(\log n)$ with only $O(1)$ overhead. In addition, we consider a more complex storage primitive called key-value storage in which data is indexed by keys from a large universe (as opposed to consecutive integers in ORAM and PIR). We present a differentially private key-value storage scheme with $ε= Θ(\log n)$ and $O(\log\log n)$ overhead. This construction uses a new oblivious, two-choice hashing scheme that may be of independent interest. △ Less

Submitted 10 April, 2019; originally announced April 2019.

Comments: To appear at PODS'19

Short note on the behavior of recurrent neural network for noisy dynamical system

Authors: Kyongmin Yeo

Abstract: The behavior of recurrent neural network for the data-driven simulation of noisy dynamical systems is studied by training a set of Long Short-Term Memory Networks (LSTM) on the Mackey-Glass time series with a wide range of noise level. It is found that, as the training noise becomes larger, LSTM learns to depend more on its autonomous dynamics than the noisy input data. As a result, LSTM trained o… ▽ More The behavior of recurrent neural network for the data-driven simulation of noisy dynamical systems is studied by training a set of Long Short-Term Memory Networks (LSTM) on the Mackey-Glass time series with a wide range of noise level. It is found that, as the training noise becomes larger, LSTM learns to depend more on its autonomous dynamics than the noisy input data. As a result, LSTM trained on noisy data becomes less susceptible to the perturbation in the data, but has a longer relaxation timescale. On the other hand, when trained on noiseless data, LSTM becomes extremely sensitive to a small perturbation, but is able to adjusts to the changes in the input data. △ Less

Submitted 5 April, 2019; originally announced April 2019.

Lower Bounds for Oblivious Near-Neighbor Search

Authors: Kasper Green Larsen, Tal Malkin, Omri Weinstein, Kevin Yeo

Abstract: We prove an $Ω(d \lg n/ (\lg\lg n)^2)$ lower bound on the dynamic cell-probe complexity of statistically $\mathit{oblivious}$ approximate-near-neighbor search ($\mathsf{ANN}$) over the $d$-dimensional Hamming cube. For the natural setting of $d = Θ(\log n)$, our result implies an $\tildeΩ(\lg^2 n)$ lower bound, which is a quadratic improvement over the highest (non-oblivious) cell-probe lower boun… ▽ More We prove an $Ω(d \lg n/ (\lg\lg n)^2)$ lower bound on the dynamic cell-probe complexity of statistically $\mathit{oblivious}$ approximate-near-neighbor search ($\mathsf{ANN}$) over the $d$-dimensional Hamming cube. For the natural setting of $d = Θ(\log n)$, our result implies an $\tildeΩ(\lg^2 n)$ lower bound, which is a quadratic improvement over the highest (non-oblivious) cell-probe lower bound for $\mathsf{ANN}$. This is the first super-logarithmic $\mathit{unconditional}$ lower bound for $\mathsf{ANN}$ against general (non black-box) data structures. We also show that any oblivious $\mathit{static}$ data structure for decomposable search problems (like $\mathsf{ANN}$) can be obliviously dynamized with $O(\log n)$ overhead in update and query time, strengthening a classic result of Bentley and Saxe (Algorithmica, 1980). △ Less

Submitted 9 April, 2019; originally announced April 2019.

Comments: 28 pages

Deep learning algorithm for data-driven simulation of noisy dynamical system

Authors: Kyongmin Yeo, Igor Melnyk

Abstract: We present a deep learning model, DE-LSTM, for the simulation of a stochastic process with an underlying nonlinear dynamics. The deep learning model aims to approximate the probability density function of a stochastic process via numerical discretization and the underlying nonlinear dynamics is modeled by the Long Short-Term Memory (LSTM) network. It is shown that, when the numerical discretizatio… ▽ More We present a deep learning model, DE-LSTM, for the simulation of a stochastic process with an underlying nonlinear dynamics. The deep learning model aims to approximate the probability density function of a stochastic process via numerical discretization and the underlying nonlinear dynamics is modeled by the Long Short-Term Memory (LSTM) network. It is shown that, when the numerical discretization is used, the function estimation problem can be solved by a multi-label classification problem. A penalized maximum log likelihood method is proposed to impose a smoothness condition in the prediction of the probability distribution. We show that the time evolution of the probability distribution can be computed by a high-dimensional integration of the transition probability of the LSTM internal states. A Monte Carlo algorithm to approximate the high-dimensional integration is outlined. The behavior of DE-LSTM is thoroughly investigated by using the Ornstein-Uhlenbeck process and noisy observations of nonlinear dynamical systems; Mackey-Glass time series and forced Van der Pol oscillator. It is shown that DE-LSTM makes a good prediction of the probability distribution without assuming any distributional properties of the stochastic process. For a multiple-step forecast of the Mackey-Glass time series, the prediction uncertainty, denoted by the 95\% confidence interval, first grows, then dynamically adjusts following the evolution of the system, while in the simulation of the forced Van der Pol oscillator, the prediction uncertainty does not grow in time even for a 3,000-step forecast. △ Less

Submitted 5 September, 2018; v1 submitted 22 February, 2018; originally announced February 2018.

Development of hp-inverse model by using generalized polynomial chaos

Authors: Kyongmin Yeo, Youngdeok Hwang, Xiao Liu, Jayant Kalagnanam

Abstract: We present a hp-inverse model to estimate a smooth, non-negative source function from a limited number of observations for a two-dimensional linear source inversion problem. A standard least-square inverse model is formulated by using a set of Gaussian radial basis functions (GRBF) on a rectangular mesh system with a uniform grid space. Here, the choice of the mesh system is modeled as a random va… ▽ More We present a hp-inverse model to estimate a smooth, non-negative source function from a limited number of observations for a two-dimensional linear source inversion problem. A standard least-square inverse model is formulated by using a set of Gaussian radial basis functions (GRBF) on a rectangular mesh system with a uniform grid space. Here, the choice of the mesh system is modeled as a random variable and the generalized polynomial chaos (gPC) expansion is used to represent the random mesh system. It is shown that the convolution of gPC and GRBF provides hierarchical basis functions for the linear source inverse model with the $hp$-refinement capability. We propose a mixed l_1 and l_2 regularization to exploit the hierarchical nature of the basis functions to find a sparse solution. The $hp$-inverse model has an advantage over the standard least-square inverse model when the number of data is limited. It is shown that the hp-inverse model provides a good estimate of the source function even when the number of unknown parameters ($m$) is much larger the number of data ($n$), e.g., m/n > 40. △ Less

Submitted 14 December, 2018; v1 submitted 9 January, 2018; originally announced January 2018.

Model-free prediction of noisy chaotic time series by deep learning

Authors: Kyongmin Yeo

Abstract: We present a deep neural network for a model-free prediction of a chaotic dynamical system from noisy observations. The proposed deep learning model aims to predict the conditional probability distribution of a state variable. The Long Short-Term Memory network (LSTM) is employed to model the nonlinear dynamics and a softmax layer is used to approximate a probability distribution. The LSTM model i… ▽ More We present a deep neural network for a model-free prediction of a chaotic dynamical system from noisy observations. The proposed deep learning model aims to predict the conditional probability distribution of a state variable. The Long Short-Term Memory network (LSTM) is employed to model the nonlinear dynamics and a softmax layer is used to approximate a probability distribution. The LSTM model is trained by minimizing a regularized cross-entropy function. The LSTM model is validated against delay-time chaotic dynamical systems, Mackey-Glass and Ikeda equations. It is shown that the present LSTM makes a good prediction of the nonlinear dynamics by effectively filtering out the noise. It is found that the prediction uncertainty of a multiple-step forecast of the LSTM model is not a monotonic function of time; the predicted standard deviation may increase or decrease dynamically in time. △ Less

Submitted 29 September, 2017; originally announced October 2017.

CacheShuffle: An Oblivious Shuffle Algorithm Using Caches

Authors: Sarvar Patel, Giuseppe Persiano, Kevin Yeo

Abstract: We consider Oblivious Shuffling and K-Oblivious Shuffling, a refinement thereof. We provide efficient algorithms for both and discuss their application to the design of Oblivious RAM. The task of K-Oblivious Shuffling is to obliviously shuffle N encrypted blocks that have been randomly allocated on the server in such a way that an adversary learns nothing about the new allocation of blocks. The se… ▽ More We consider Oblivious Shuffling and K-Oblivious Shuffling, a refinement thereof. We provide efficient algorithms for both and discuss their application to the design of Oblivious RAM. The task of K-Oblivious Shuffling is to obliviously shuffle N encrypted blocks that have been randomly allocated on the server in such a way that an adversary learns nothing about the new allocation of blocks. The security guarantee should hold also with respect to an adversary that has learned the initial position of K touched blocks out of the N blocks. The classical notion of Oblivious Shuffling is obtained for K = N. We present a family of algorithms for Oblivious Shuffling. Our first construction, CacheShuffleRoot, is tailored for clients with $O(\sqrt{N})$ blocks of memory and uses $(4+ε)N$ blocks of bandwidth, for every $ε> 0$. CacheShuffleRoot is a 4.5x improvement over previous best known results on practical sizes of N. We also present CacheShuffle that obliviously shuffles using O(S) blocks of client memory with $O(N\log_S N)$ blocks of bandwidth. We then turn to K-Oblivious Shuffling and give algorithms that require 2N + f(K) blocks of bandwidth, for some function f. That is, any extra bandwidth above the 2N lower bound depends solely on K. We present KCacheShuffleBasic that uses O(K) client storage and exactly 2N blocks of bandwidth. For smaller client storage requirements, we show KCacheShuffle, which uses O(S) client storage and requires $2N+(1+ε)O(K\log_S K)$ blocks of bandwidth. Finally, we consider the case in which, in addition to the N blocks, the server stores D dummy blocks whose content is is irrelevant but still their positions must be hidden by the shuffling. For this case, we design algorithm KCacheShuffleDummy that, for N + D blocks and K touched blocks, uses O(K) client storage and $D+(2+ε)N$ blocks of bandwidth. △ Less

Submitted 17 October, 2017; v1 submitted 19 May, 2017; originally announced May 2017.

Comments: 29 pages, 4 figures

A Self-Organization Framework for Wireless Ad Hoc Networks as Small Worlds

Authors: Abhik Banerjee, Rachit Agarwal, Vincent Gauthier, Chai Kiat Yeo, Hossam Afifi, Bu Sung Lee

Abstract: Motivated by the benefits of small world networks, we propose a self-organization framework for wireless ad hoc networks. We investigate the use of directional beamforming for creating long-range short cuts between nodes. Using simulation results for randomized beamforming as a guideline, we identify crucial design issues for algorithm design. Our results show that, while significant path length r… ▽ More Motivated by the benefits of small world networks, we propose a self-organization framework for wireless ad hoc networks. We investigate the use of directional beamforming for creating long-range short cuts between nodes. Using simulation results for randomized beamforming as a guideline, we identify crucial design issues for algorithm design. Our results show that, while significant path length reduction is achievable, this is accompanied by the problem of asymmetric paths between nodes. Subsequently, we propose a distributed algorithm for small world creation that achieves path length reduction while maintaining connectivity. We define a new centrality measure that estimates the structural importance of nodes based on traffic flow in the network, which is used to identify the optimum nodes for beamforming. We show, using simulations, that this leads to significant reduction in path length while maintaining connectivity. △ Less

Submitted 6 March, 2012; originally announced March 2012.

Comments: Submitted to IEEE Transactions on Vehicular Technology

Achieving Small World Properties using Bio-Inspired Techniques in Wireless Networks

Authors: Rachit Agarwal, Abhik Banerjee, Vincent Gauthier, Monique Becker, Chai Kiat Yeo, Bu Sung Lee

Abstract: It is highly desirable and challenging for a wireless ad hoc network to have self-organization properties in order to achieve network wide characteristics. Studies have shown that Small World properties, primarily low average path length and high clustering coefficient, are desired properties for networks in general. However, due to the spatial nature of the wireless networks, achieving small worl… ▽ More It is highly desirable and challenging for a wireless ad hoc network to have self-organization properties in order to achieve network wide characteristics. Studies have shown that Small World properties, primarily low average path length and high clustering coefficient, are desired properties for networks in general. However, due to the spatial nature of the wireless networks, achieving small world properties remains highly challenging. Studies also show that, wireless ad hoc networks with small world properties show a degree distribution that lies between geometric and power law. In this paper, we show that in a wireless ad hoc network with non-uniform node density with only local information, we can significantly reduce the average path length and retain the clustering coefficient. To achieve our goal, our algorithm first identifies logical regions using Lateral Inhibition technique, then identifies the nodes that beamform and finally the beam properties using Flocking. We use Lateral Inhibition and Flocking because they enable us to use local state information as opposed to other techniques. We support our work with simulation results and analysis, which show that a reduction of up to 40% can be achieved for a high-density network. We also show the effect of hopcount used to create regions on average path length, clustering coefficient and connectivity. △ Less

Submitted 3 March, 2012; v1 submitted 21 November, 2011; originally announced November 2011.

Comments: Accepted for publication: Special Issue on Security and Performance of Networks and Clouds (The Computer Journal)

Self-organization of Nodes using Bio-Inspired Techniques for Achieving Small World Properties

Authors: Rachit Agarwal, Abhik Banerjee, Vincent Gauthier, Monique Becker, Chai Kiat Yeo, Bu Sung Lee

Abstract: In an autonomous wireless sensor network, self-organization of the nodes is essential to achieve network wide characteristics. We believe that connectivity in wireless autonomous networks can be increased and overall average path length can be reduced by using beamforming and bio-inspired algorithms. Recent works on the use of beamforming in wireless networks mostly assume the knowledge of the net… ▽ More In an autonomous wireless sensor network, self-organization of the nodes is essential to achieve network wide characteristics. We believe that connectivity in wireless autonomous networks can be increased and overall average path length can be reduced by using beamforming and bio-inspired algorithms. Recent works on the use of beamforming in wireless networks mostly assume the knowledge of the network in aggregation to either heterogeneous or hybrid deployment. We propose that without the global knowledge or the introduction of any special feature, the average path length can be reduced with the help of inspirations from the nature and simple interactions between neighboring nodes. Our algorithm also reduces the number of disconnected components within the network. Our results show that reduction in the average path length and the number of disconnected components can be achieved using very simple local rules and without the full network knowledge. △ Less

Submitted 27 September, 2011; originally announced September 2011.

Comments: Accepted to Joint workshop on complex networks and pervasive group communication (CCNet/PerGroup), in conjunction with IEEE Globecom 2011

Self-Organization of Wireless Ad Hoc Networks as Small Worlds Using Long Range Directional Beams

Authors: Abhik Banerjee, Rachit Agarwal, Vincent Gauthier, Chai Kiat Yeo, Hossam Afifi, Bu Sung Lee

Abstract: We study how long range directional beams can be used for self-organization of a wireless network to exhibit small world properties. Using simulation results for randomized beamforming as a guideline, we identify crucial design issues for algorithm design. Subsequently, we propose an algorithm for deterministic creation of small worlds. We define a new centrality measure that estimates the structu… ▽ More We study how long range directional beams can be used for self-organization of a wireless network to exhibit small world properties. Using simulation results for randomized beamforming as a guideline, we identify crucial design issues for algorithm design. Subsequently, we propose an algorithm for deterministic creation of small worlds. We define a new centrality measure that estimates the structural importance of nodes based on traffic flow in the network, which is used to identify the optimum nodes for beamforming. This results in significant reduction in path length while maintaining connectivity. △ Less

Submitted 25 September, 2011; originally announced September 2011.

Comments: Accepted to Joint workshop on complex networks and pervasive group communication (CCNet/PerGroup), in conjunction with IEEE Globecom 2011