-
Uplift Modeling based on Graph Neural Network Combined with Causal Knowledge
Authors:
Haowen Wang,
Xinyan Ye,
Yangze Zhou,
Zhiyi Zhang,
Longhan Zhang,
**g Jiang
Abstract:
Uplift modeling is a fundamental component of marketing effect modeling, which is commonly employed to evaluate the effects of treatments on outcomes. Through uplift modeling, we can identify the treatment with the greatest benefit. On the other side, we can identify clients who are likely to make favorable decisions in response to a certain treatment. In the past, uplift modeling approaches relie…
▽ More
Uplift modeling is a fundamental component of marketing effect modeling, which is commonly employed to evaluate the effects of treatments on outcomes. Through uplift modeling, we can identify the treatment with the greatest benefit. On the other side, we can identify clients who are likely to make favorable decisions in response to a certain treatment. In the past, uplift modeling approaches relied heavily on the difference-in-difference (DID) architecture, paired with a machine learning model as the estimation learner, while neglecting the link and confidential information between features. We proposed a framework based on graph neural networks that combine causal knowledge with an estimate of uplift value. Firstly, we presented a causal representation technique based on CATE (conditional average treatment effect) estimation and adjacency matrix structure learning. Secondly, we suggested a more scalable uplift modeling framework based on graph convolution networks for combining causal knowledge. Our findings demonstrate that this method works effectively for predicting uplift values, with small errors in typical simulated data, and its effectiveness has been verified in actual industry marketing data.
△ Less
Submitted 14 November, 2023;
originally announced November 2023.
-
Flexible and Hierarchical Prior for Bayesian Nonnegative Matrix Factorization
Authors:
Jun Lu,
Xuanyu Ye
Abstract:
In this paper, we introduce a probabilistic model for learning nonnegative matrix factorization (NMF) that is commonly used for predicting missing values and finding hidden patterns in the data, in which the matrix factors are latent variables associated with each data dimension. The nonnegativity constraint for the latent factors is handled by choosing priors with support on the nonnegative subsp…
▽ More
In this paper, we introduce a probabilistic model for learning nonnegative matrix factorization (NMF) that is commonly used for predicting missing values and finding hidden patterns in the data, in which the matrix factors are latent variables associated with each data dimension. The nonnegativity constraint for the latent factors is handled by choosing priors with support on the nonnegative subspace. Bayesian inference procedure based on Gibbs sampling is employed. We evaluate the model on several real-world datasets including MovieLens 100K and MovieLens 1M with different sizes and dimensions and show that the proposed Bayesian NMF GRRN model leads to better predictions and avoids overfitting compared to existing Bayesian NMF approaches.
△ Less
Submitted 19 June, 2022; v1 submitted 22 May, 2022;
originally announced May 2022.
-
Low-rank Matrix Recovery With Unknown Correspondence
Authors:
Zhiwei Tang,
Tsung-Hui Chang,
Xiao**g Ye,
Hongyuan Zha
Abstract:
We study a matrix recovery problem with unknown correspondence: given the observation matrix $M_o=[A,\tilde P B]$, where $\tilde P$ is an unknown permutation matrix, we aim to recover the underlying matrix $M=[A,B]$. Such problem commonly arises in many applications where heterogeneous data are utilized and the correspondence among them are unknown, e.g., due to privacy concerns. We show that it i…
▽ More
We study a matrix recovery problem with unknown correspondence: given the observation matrix $M_o=[A,\tilde P B]$, where $\tilde P$ is an unknown permutation matrix, we aim to recover the underlying matrix $M=[A,B]$. Such problem commonly arises in many applications where heterogeneous data are utilized and the correspondence among them are unknown, e.g., due to privacy concerns. We show that it is possible to recover $M$ via solving a nuclear norm minimization problem under a proper low-rank condition on $M$, with provable non-asymptotic error bound for the recovery of $M$. We propose an algorithm, $\text{M}^3\text{O}$ (Matrix recovery via Min-Max Optimization) which recasts this combinatorial problem as a continuous minimax optimization problem and solves it by proximal gradient with a Max-Oracle. $\text{M}^3\text{O}$ can also be applied to a more general scenario where we have missing entries in $M_o$ and multiple groups of data with distinct unknown correspondence. Experiments on simulated data, the MovieLens 100K dataset and Yale B database show that $\text{M}^3\text{O}$ achieves state-of-the-art performance over several baselines and can recover the ground-truth correspondence with high accuracy.
△ Less
Submitted 17 October, 2021; v1 submitted 15 October, 2021;
originally announced October 2021.
-
Nonlinear model reduction for slow-fast stochastic systems near unknown invariant manifolds
Authors:
Felix X. -F. Ye,
Sichen Yang,
Mauro Maggioni
Abstract:
We introduce a nonlinear stochastic model reduction technique for high-dimensional stochastic dynamical systems that have a low-dimensional invariant effective manifold with slow dynamics, and high-dimensional, large fast modes. Given only access to a black box simulator from which short bursts of simulation can be obtained, we design an algorithm that outputs an estimate of the invariant manifold…
▽ More
We introduce a nonlinear stochastic model reduction technique for high-dimensional stochastic dynamical systems that have a low-dimensional invariant effective manifold with slow dynamics, and high-dimensional, large fast modes. Given only access to a black box simulator from which short bursts of simulation can be obtained, we design an algorithm that outputs an estimate of the invariant manifold, a process of the effective stochastic dynamics on it, which has averaged out the fast modes, and a simulator thereof. This simulator is efficient in that it exploits of the low dimension of the invariant manifold, and takes time steps of size dependent on the regularity of the effective process, and therefore typically much larger than that of the original simulator, which had to resolve the fast modes. The algorithm and the estimation can be performed on-the-fly, leading to efficient exploration of the effective state space, without losing consistency with the underlying dynamics. This construction enables fast and efficient simulation of paths of the effective dynamics, together with estimation of crucial features and observables of such dynamics, including the stationary distribution, identification of metastable states, and residence times and transition rates between them.
△ Less
Submitted 24 October, 2023; v1 submitted 5 April, 2021;
originally announced April 2021.
-
ISALT: Inference-based schemes adaptive to large time-step** for locally Lipschitz ergodic systems
Authors:
Xingjie Li,
Fei Lu,
Felix X. -F. Ye
Abstract:
Efficient simulation of SDEs is essential in many applications, particularly for ergodic systems that demand efficient simulation of both short-time dynamics and large-time statistics. However, locally Lipschitz SDEs often require special treatments such as implicit schemes with small time-steps to accurately simulate the ergodic measure. We introduce a framework to construct inference-based schem…
▽ More
Efficient simulation of SDEs is essential in many applications, particularly for ergodic systems that demand efficient simulation of both short-time dynamics and large-time statistics. However, locally Lipschitz SDEs often require special treatments such as implicit schemes with small time-steps to accurately simulate the ergodic measure. We introduce a framework to construct inference-based schemes adaptive to large time-steps (ISALT) from data, achieving a reduction in time by several orders of magnitudes. The key is the statistical learning of an approximation to the infinite-dimensional discrete-time flow map. We explore the use of numerical schemes (such as the Euler-Maruyama, a hybrid RK4, and an implicit scheme) to derive informed basis functions, leading to a parameter inference problem. We introduce a scalable algorithm to estimate the parameters by least squares, and we prove the convergence of the estimators as data size increases.
We test the ISALT on three non-globally Lipschitz SDEs: the 1D double-well potential, a 2D multi-scale gradient system, and the 3D stochastic Lorenz equation with degenerate noise. Numerical results show that ISALT can tolerate time-step magnitudes larger than plain numerical schemes. It reaches optimal accuracy in reproducing the invariant measure when the time-step is medium-large.
△ Less
Submitted 24 February, 2021;
originally announced February 2021.
-
A Hypergradient Approach to Robust Regression without Correspondence
Authors:
Yujia Xie,
Yixiu Mao,
Simiao Zuo,
Hongteng Xu,
Xiao**g Ye,
Tuo Zhao,
Hongyuan Zha
Abstract:
We consider a variant of regression problem, where the correspondence between input and output data is not available. Such shuffled data is commonly observed in many real world problems. Taking flow cytometry as an example, the measuring instruments may not be able to maintain the correspondence between the samples and the measurements. Due to the combinatorial nature of the problem, most existing…
▽ More
We consider a variant of regression problem, where the correspondence between input and output data is not available. Such shuffled data is commonly observed in many real world problems. Taking flow cytometry as an example, the measuring instruments may not be able to maintain the correspondence between the samples and the measurements. Due to the combinatorial nature of the problem, most existing methods are only applicable when the sample size is small, and limited to linear regression models. To overcome such bottlenecks, we propose a new computational framework -- ROBOT -- for the shuffled regression problem, which is applicable to large data and complex nonlinear models. Specifically, we reformulate the regression without correspondence as a continuous optimization problem. Then by exploiting the interaction between the regression model and the data correspondence, we develop a hypergradient approach based on differentiable programming techniques. Such a hypergradient approach essentially views the data correspondence as an operator of the regression, and therefore allows us to find a better descent direction for the model parameter by differentiating through the data correspondence. ROBOT can be further extended to the inexact correspondence setting, where there may not be an exact alignment between the input and output data. Thorough numerical experiments show that ROBOT achieves better performance than existing methods in both linear and nonlinear regression tasks, including real-world applications such as flow cytometry and multi-object tracking.
△ Less
Submitted 11 February, 2021; v1 submitted 30 November, 2020;
originally announced December 2020.
-
Evaluating the effect of city lock-down on controlling COVID-19 propagation through deep learning and network science models
Authors:
Xiaoqi Zhang,
Zheng Ji,
Yanqiao Zheng,
Xinyue Ye,
Dong Li
Abstract:
The special epistemic characteristics of the COVID-19, such as the long incubation period and the infection through asymptomatic cases, put severe challenge to the containment of its outbreak. By the end of March 2020, China has successfully controlled the within-spreading of COVID-19 at a high cost of locking down most of its major cities, including the epicenter, Wuhan. Since the low accuracy of…
▽ More
The special epistemic characteristics of the COVID-19, such as the long incubation period and the infection through asymptomatic cases, put severe challenge to the containment of its outbreak. By the end of March 2020, China has successfully controlled the within-spreading of COVID-19 at a high cost of locking down most of its major cities, including the epicenter, Wuhan. Since the low accuracy of outbreak data before the mid of Feb. 2020 forms a major technical concern on those studies based on statistic inference from the early outbreak. We apply the supervised learning techniques to identify and train NP-Net-SIR model which turns out robust under poor data quality condition. By the trained model parameters, we analyze the connection between population flow and the cross-regional infection connection strength, based on which a set of counterfactual analysis is carried out to study the necessity of lock-down and substitutability between lock-down and the other containment measures. Our findings support the existence of non-lock-down-typed measures that can reach the same containment consequence as the lock-down, and provide useful guideline for the design of a more flexible containment strategy.
△ Less
Submitted 4 September, 2020;
originally announced September 2020.
-
Network Embedding with Completely-imbalanced Labels
Authors:
Zheng Wang,
Xiaojun Ye,
Chaokun Wang,
Jian Cui,
Philip S. Yu
Abstract:
Network embedding, aiming to project a network into a low-dimensional space, is increasingly becoming a focus of network research. Semi-supervised network embedding takes advantage of labeled data, and has shown promising performance. However, existing semi-supervised methods would get unappealing results in the completely-imbalanced label setting where some classes have no labeled nodes at all. T…
▽ More
Network embedding, aiming to project a network into a low-dimensional space, is increasingly becoming a focus of network research. Semi-supervised network embedding takes advantage of labeled data, and has shown promising performance. However, existing semi-supervised methods would get unappealing results in the completely-imbalanced label setting where some classes have no labeled nodes at all. To alleviate this, we propose two novel semi-supervised network embedding methods. The first one is a shallow method named RSDNE. Specifically, to benefit from the completely-imbalanced labels, RSDNE guarantees both intra-class similarity and inter-class dissimilarity in an approximate way. The other method is RECT which is a new class of graph neural networks. Different from RSDNE, to benefit from the completely-imbalanced labels, RECT explores the class-semantic knowledge. This enables RECT to handle networks with node features and multi-label setting. Experimental results on several real-world datasets demonstrate the superiority of the proposed methods.
△ Less
Submitted 7 July, 2020;
originally announced July 2020.
-
Network Diffusions via Neural Mean-Field Dynamics
Authors:
Shushan He,
Hongyuan Zha,
Xiao**g Ye
Abstract:
We propose a novel learning framework based on neural mean-field dynamics for inference and estimation problems of diffusion on networks. Our new framework is derived from the Mori-Zwanzig formalism to obtain an exact evolution of the node infection probabilities, which renders a delay differential equation with memory integral approximated by learnable time convolution operators, resulting in a h…
▽ More
We propose a novel learning framework based on neural mean-field dynamics for inference and estimation problems of diffusion on networks. Our new framework is derived from the Mori-Zwanzig formalism to obtain an exact evolution of the node infection probabilities, which renders a delay differential equation with memory integral approximated by learnable time convolution operators, resulting in a highly structured and interpretable RNN. Directly using cascade data, our framework can jointly learn the structure of the diffusion network and the evolution of infection probabilities, which are cornerstone to important downstream applications such as influence maximization. Connections between parameter learning and optimal control are also established. Empirical study shows that our approach is versatile and robust to variations of the underlying diffusion network models, and significantly outperform existing approaches in accuracy and efficiency on both synthetic and real-world data.
△ Less
Submitted 19 January, 2021; v1 submitted 16 June, 2020;
originally announced June 2020.
-
Learning Cost Functions for Optimal Transport
Authors:
Shaojun Ma,
Haodong Sun,
Xiao**g Ye,
Hongyuan Zha,
Haomin Zhou
Abstract:
Inverse optimal transport (OT) refers to the problem of learning the cost function for OT from observed transport plan or its samples. In this paper, we derive an unconstrained convex optimization formulation of the inverse OT problem, which can be further augmented by any customizable regularization. We provide a comprehensive characterization of the properties of inverse OT, including uniqueness…
▽ More
Inverse optimal transport (OT) refers to the problem of learning the cost function for OT from observed transport plan or its samples. In this paper, we derive an unconstrained convex optimization formulation of the inverse OT problem, which can be further augmented by any customizable regularization. We provide a comprehensive characterization of the properties of inverse OT, including uniqueness of solutions. We also develop two numerical algorithms, one is a fast matrix scaling method based on the Sinkhorn-Knopp algorithm for discrete OT, and the other one is a learning based algorithm that parameterizes the cost function as a deep neural network for continuous OT. The novel framework proposed in the work avoids repeatedly solving a forward OT in each iteration which has been a thorny computational bottleneck for the bi-level optimization in existing inverse OT approaches. Numerical results demonstrate promising efficiency and accuracy advantages of the proposed algorithms over existing state-of-the-art methods.
△ Less
Submitted 5 July, 2021; v1 submitted 22 February, 2020;
originally announced February 2020.
-
Multiclass spectral feature scaling method for dimensionality reduction
Authors:
Momo Matsuda,
Keiichi Morikuni,
Akira Imakura,
Xiucai Ye,
Tetsuya Sakurai
Abstract:
Irregular features disrupt the desired classification. In this paper, we consider aggressively modifying scales of features in the original space according to the label information to form well-separated clusters in low-dimensional space. The proposed method exploits spectral clustering to derive scaling factors that are used to modify the features. Specifically, we reformulate the Laplacian eigen…
▽ More
Irregular features disrupt the desired classification. In this paper, we consider aggressively modifying scales of features in the original space according to the label information to form well-separated clusters in low-dimensional space. The proposed method exploits spectral clustering to derive scaling factors that are used to modify the features. Specifically, we reformulate the Laplacian eigenproblem of the spectral clustering as an eigenproblem of a linear matrix pencil whose eigenvector has the scaling factors. Numerical experiments show that the proposed method outperforms well-established supervised dimensionality reduction methods for toy problems with more samples than features and real-world problems with more features than samples.
△ Less
Submitted 16 October, 2019;
originally announced October 2019.
-
Accelerating CNN Training by Pruning Activation Gradients
Authors:
Xucheng Ye,
Pengcheng Dai,
Junyu Luo,
Xin Guo,
Yingjie Qi,
Jianlei Yang,
Yiran Chen
Abstract:
Sparsification is an efficient approach to accelerate CNN inference, but it is challenging to take advantage of sparsity in training procedure because the involved gradients are dynamically changed. Actually, an important observation shows that most of the activation gradients in back-propagation are very close to zero and only have a tiny impact on weight-updating. Hence, we consider pruning thes…
▽ More
Sparsification is an efficient approach to accelerate CNN inference, but it is challenging to take advantage of sparsity in training procedure because the involved gradients are dynamically changed. Actually, an important observation shows that most of the activation gradients in back-propagation are very close to zero and only have a tiny impact on weight-updating. Hence, we consider pruning these very small gradients randomly to accelerate CNN training according to the statistical distribution of activation gradients. Meanwhile, we theoretically analyze the impact of pruning algorithm on the convergence. The proposed approach is evaluated on AlexNet and ResNet-\{18, 34, 50, 101, 152\} with CIFAR-\{10, 100\} and ImageNet datasets. Experimental results show that our training approach could substantially achieve up to $5.92 \times$ speedups at back-propagation stage with negligible accuracy loss.
△ Less
Submitted 20 July, 2020; v1 submitted 31 July, 2019;
originally announced August 2019.
-
Incremental Classifier Learning Based on PEDCC-Loss and Cosine Distance
Authors:
Qiuyu Zhu,
Zikuang He,
Xin Ye
Abstract:
The main purpose of incremental learning is to learn new knowledge while not forgetting the knowledge which have been learned before. At present, the main challenge in this area is the catastrophe forgetting, namely the network will lose their performance in the old tasks after training for new tasks. In this paper, we introduce an ensemble method of incremental classifier to alleviate this proble…
▽ More
The main purpose of incremental learning is to learn new knowledge while not forgetting the knowledge which have been learned before. At present, the main challenge in this area is the catastrophe forgetting, namely the network will lose their performance in the old tasks after training for new tasks. In this paper, we introduce an ensemble method of incremental classifier to alleviate this problem, which is based on the cosine distance between the output feature and the pre-defined center, and can let each task to be preserved in different networks. During training, we make use of PEDCC-Loss to train the CNN network. In the stage of testing, the prediction is determined by the cosine distance between the network latent features and pre-defined center. The experimental results on EMINST and CIFAR100 show that our method outperforms the recent LwF method, which use the knowledge distillation, and iCaRL method, which keep some old samples while training for new task. The method can achieve the goal of not forgetting old knowledge while training new classes, and solve the problem of catastrophic forgetting better.
△ Less
Submitted 11 June, 2019;
originally announced June 2019.
-
Identifying the Best Machine Learning Algorithms for Brain Tumor Segmentation, Progression Assessment, and Overall Survival Prediction in the BRATS Challenge
Authors:
Spyridon Bakas,
Mauricio Reyes,
Andras Jakab,
Stefan Bauer,
Markus Rempfler,
Alessandro Crimi,
Russell Takeshi Shinohara,
Christoph Berger,
Sung Min Ha,
Martin Rozycki,
Marcel Prastawa,
Esther Alberts,
Jana Lipkova,
John Freymann,
Justin Kirby,
Michel Bilello,
Hassan Fathallah-Shaykh,
Roland Wiest,
Jan Kirschke,
Benedikt Wiestler,
Rivka Colen,
Aikaterini Kotrotsou,
Pamela Lamontagne,
Daniel Marcus,
Mikhail Milchenko
, et al. (402 additional authors not shown)
Abstract:
Gliomas are the most common primary brain malignancies, with different degrees of aggressiveness, variable prognosis and various heterogeneous histologic sub-regions, i.e., peritumoral edematous/invaded tissue, necrotic core, active and non-enhancing core. This intrinsic heterogeneity is also portrayed in their radio-phenotype, as their sub-regions are depicted by varying intensity profiles dissem…
▽ More
Gliomas are the most common primary brain malignancies, with different degrees of aggressiveness, variable prognosis and various heterogeneous histologic sub-regions, i.e., peritumoral edematous/invaded tissue, necrotic core, active and non-enhancing core. This intrinsic heterogeneity is also portrayed in their radio-phenotype, as their sub-regions are depicted by varying intensity profiles disseminated across multi-parametric magnetic resonance imaging (mpMRI) scans, reflecting varying biological properties. Their heterogeneous shape, extent, and location are some of the factors that make these tumors difficult to resect, and in some cases inoperable. The amount of resected tumor is a factor also considered in longitudinal scans, when evaluating the apparent tumor for potential diagnosis of progression. Furthermore, there is mounting evidence that accurate segmentation of the various tumor sub-regions can offer the basis for quantitative image analysis towards prediction of patient overall survival. This study assesses the state-of-the-art machine learning (ML) methods used for brain tumor image analysis in mpMRI scans, during the last seven instances of the International Brain Tumor Segmentation (BraTS) challenge, i.e., 2012-2018. Specifically, we focus on i) evaluating segmentations of the various glioma sub-regions in pre-operative mpMRI scans, ii) assessing potential tumor progression by virtue of longitudinal growth of tumor sub-regions, beyond use of the RECIST/RANO criteria, and iii) predicting the overall survival from pre-operative mpMRI scans of patients that underwent gross total resection. Finally, we investigate the challenge of identifying the best ML algorithms for each of these tasks, considering that apart from being diverse on each instance of the challenge, the multi-institutional mpMRI BraTS dataset has also been a continuously evolving/growing dataset.
△ Less
Submitted 23 April, 2019; v1 submitted 5 November, 2018;
originally announced November 2018.
-
Horizon: Facebook's Open Source Applied Reinforcement Learning Platform
Authors:
Jason Gauci,
Edoardo Conti,
Yitao Liang,
Kittipat Virochsiri,
Yuchen He,
Zachary Kaden,
Vivek Narayanan,
Xiaohui Ye,
Zhengxing Chen,
Scott Fujimoto
Abstract:
In this paper we present Horizon, Facebook's open source applied reinforcement learning (RL) platform. Horizon is an end-to-end platform designed to solve industry applied RL problems where datasets are large (millions to billions of observations), the feedback loop is slow (vs. a simulator), and experiments must be done with care because they don't run in a simulator. Unlike other RL platforms, w…
▽ More
In this paper we present Horizon, Facebook's open source applied reinforcement learning (RL) platform. Horizon is an end-to-end platform designed to solve industry applied RL problems where datasets are large (millions to billions of observations), the feedback loop is slow (vs. a simulator), and experiments must be done with care because they don't run in a simulator. Unlike other RL platforms, which are often designed for fast prototy** and experimentation, Horizon is designed with production use cases as top of mind. The platform contains workflows to train popular deep RL algorithms and includes data preprocessing, feature transformation, distributed training, counterfactual policy evaluation, optimized serving, and a model-based data understanding tool. We also showcase and describe real examples where reinforcement learning models trained with Horizon significantly outperformed and replaced supervised learning systems at Facebook.
△ Less
Submitted 4 September, 2019; v1 submitted 1 November, 2018;
originally announced November 2018.
-
Learning to Match via Inverse Optimal Transport
Authors:
Ruilin Li,
Xiao**g Ye,
Haomin Zhou,
Hongyuan Zha
Abstract:
We propose a unified data-driven framework based on inverse optimal transport that can learn adaptive, nonlinear interaction cost function from noisy and incomplete empirical matching matrix and predict new matching in various matching contexts. We emphasize that the discrete optimal transport plays the role of a variational principle which gives rise to an optimization-based framework for modelin…
▽ More
We propose a unified data-driven framework based on inverse optimal transport that can learn adaptive, nonlinear interaction cost function from noisy and incomplete empirical matching matrix and predict new matching in various matching contexts. We emphasize that the discrete optimal transport plays the role of a variational principle which gives rise to an optimization-based framework for modeling the observed empirical matching data. Our formulation leads to a non-convex optimization problem which can be solved efficiently by an alternating optimization method. A key novel aspect of our formulation is the incorporation of marginal relaxation via regularized Wasserstein distance, significantly improving the robustness of the method in the face of noisy or missing empirical matching data. Our model falls into the category of prescriptive models, which not only predict potential future matching, but is also able to explain what leads to empirical matching and quantifies the impact of changes in matching factors. The proposed approach has wide applicability including predicting matching in online dating, labor market, college application and crowdsourcing. We back up our claims with numerical experiments on both synthetic data and real world data sets.
△ Less
Submitted 30 October, 2018; v1 submitted 10 February, 2018;
originally announced February 2018.
-
Estimate exponential memory decay in Hidden Markov Model and its applications
Authors:
Felix X. -F. Ye,
Yi-an Ma,
Hong Qian
Abstract:
Inference in hidden Markov model has been challenging in terms of scalability due to dependencies in the observation data. In this paper, we utilize the inherent memory decay in hidden Markov models, such that the forward and backward probabilities can be carried out with subsequences, enabling efficient inference over long sequences of observations. We formulate this forward filtering process in…
▽ More
Inference in hidden Markov model has been challenging in terms of scalability due to dependencies in the observation data. In this paper, we utilize the inherent memory decay in hidden Markov models, such that the forward and backward probabilities can be carried out with subsequences, enabling efficient inference over long sequences of observations. We formulate this forward filtering process in the setting of the random dynamical system and there exist Lyapunov exponents in the i.i.d random matrices production. And the rate of the memory decay is known as $λ_2-λ_1$, the gap of the top two Lyapunov exponents almost surely. An efficient and accurate algorithm is proposed to numerically estimate the gap after the soft-max parametrization. The length of subsequences $B$ given the controlled error $ε$ is $B=\log(ε)/(λ_2-λ_1)$. We theoretically prove the validity of the algorithm and demonstrate the effectiveness with numerical examples. The method developed here can be applied to widely used algorithms, such as mini-batch stochastic gradient method. Moreover, the continuity of Lyapunov spectrum ensures the estimated $B$ could be reused for the nearby parameter during the inference.
△ Less
Submitted 16 October, 2017;
originally announced October 2017.
-
Wasserstein Learning of Deep Generative Point Process Models
Authors:
Shuai Xiao,
Mehrdad Farajtabar,
Xiao**g Ye,
Junchi Yan,
Le Song,
Hongyuan Zha
Abstract:
Point processes are becoming very popular in modeling asynchronous sequential data due to their sound mathematical foundation and strength in modeling a variety of real-world phenomena. Currently, they are often characterized via intensity function which limits model's expressiveness due to unrealistic assumptions on its parametric form used in practice. Furthermore, they are learned via maximum l…
▽ More
Point processes are becoming very popular in modeling asynchronous sequential data due to their sound mathematical foundation and strength in modeling a variety of real-world phenomena. Currently, they are often characterized via intensity function which limits model's expressiveness due to unrealistic assumptions on its parametric form used in practice. Furthermore, they are learned via maximum likelihood approach which is prone to failure in multi-modal distributions of sequences. In this paper, we propose an intensity-free approach for point processes modeling that transforms nuisance processes to a target one. Furthermore, we train the model using a likelihood-free leveraging Wasserstein distance between point processes. Experiments on various synthetic and real-world data substantiate the superiority of the proposed point process model over conventional ones.
△ Less
Submitted 22 May, 2017;
originally announced May 2017.
-
A New Theoretical Interpretation of Measurement Error and Its Uncertainty
Authors:
Huisheng Shi,
Xiaoming Ye,
Cheng Xing,
Shijun Ding
Abstract:
The traditional measurement theory interprets the variance as the dispersion of a measured value, which is actually contrary to a general mathematical concept that the variance of a constant is 0. This paper will fully demonstrate that the variance in measurement theory is actually the evaluation of probability interval of an error instead of the dispersion of a measured value, point out the key p…
▽ More
The traditional measurement theory interprets the variance as the dispersion of a measured value, which is actually contrary to a general mathematical concept that the variance of a constant is 0. This paper will fully demonstrate that the variance in measurement theory is actually the evaluation of probability interval of an error instead of the dispersion of a measured value, point out the key point of mistake in the traditional interpretation, and fully interpret a series of changes in conceptual logic and processing method brought about by this new concept.
△ Less
Submitted 18 September, 2020; v1 submitted 12 April, 2017;
originally announced April 2017.
-
The new concepts of measurement error's regularities and effect characteristics
Authors:
Xiaoming Ye,
Haibo Liu,
Xuebin Xiao,
Mo Ling
Abstract:
In several literatures, the authors give a new thinking of measurement theory system based on error non-classification philosophy, which completely overthrows the existing measurement concept system of precision, trueness and accuracy. In this paper, by focusing on the issues of error's regularities and effect characteristics, the authors will do a thematic interpretation, and prove that the error…
▽ More
In several literatures, the authors give a new thinking of measurement theory system based on error non-classification philosophy, which completely overthrows the existing measurement concept system of precision, trueness and accuracy. In this paper, by focusing on the issues of error's regularities and effect characteristics, the authors will do a thematic interpretation, and prove that the error's regularities actually come from different cognitive perspectives, are also unable to be used for classifying errors, and that the error's effect characteristics actually depend on artificial condition rules of repeated measurement, and are still unable to be used for classifying errors. Thus, from the perspectives of error's regularities and effect characteristics, the existing error classification philosophy is still incorrect; and an uncertainty concept system, which must be interpreted by the error non-classification philosophy, naturally becomes the only way out of measurement theory.
△ Less
Submitted 18 May, 2018; v1 submitted 25 March, 2017;
originally announced March 2017.
