-
COMET Flows: Towards Generative Modeling of Multivariate Extremes and Tail Dependence
Authors:
Andrew McDonald,
Pang-Ning Tan,
Lifeng Luo
Abstract:
Normalizing flows, a popular class of deep generative models, often fail to represent extreme phenomena observed in real-world processes. In particular, existing normalizing flow architectures struggle to model multivariate extremes, characterized by heavy-tailed marginal distributions and asymmetric tail dependence among variables. In light of this shortcoming, we propose COMET (COpula Multivaria…
▽ More
Normalizing flows, a popular class of deep generative models, often fail to represent extreme phenomena observed in real-world processes. In particular, existing normalizing flow architectures struggle to model multivariate extremes, characterized by heavy-tailed marginal distributions and asymmetric tail dependence among variables. In light of this shortcoming, we propose COMET (COpula Multivariate ExTreme) Flows, which decompose the process of modeling a joint distribution into two parts: (i) modeling its marginal distributions, and (ii) modeling its copula distribution. COMET Flows capture heavy-tailed marginal distributions by combining a parametric tail belief at extreme quantiles of the marginals with an empirical kernel density function at mid-quantiles. In addition, COMET Flows capture asymmetric tail dependence among multivariate extremes by viewing such dependence as inducing a low-dimensional manifold structure in feature space. Experimental results on both synthetic and real-world datasets demonstrate the effectiveness of COMET Flows in capturing both heavy-tailed marginals and asymmetric tail dependence compared to other state-of-the-art baseline architectures. All code is available on GitHub at https://github.com/andrewmcdonald27/COMETFlows.
△ Less
Submitted 2 May, 2022;
originally announced May 2022.
-
Multi-Robot Gaussian Process Estimation and Coverage: A Deterministic Sequencing Algorithm and Regret Analysis
Authors:
Lai Wei,
Andrew McDonald,
Vaibhav Srivastava
Abstract:
We study the problem of distributed multi-robot coverage over an unknown, nonuniform sensory field. Modeling the sensory field as a realization of a Gaussian Process and using Bayesian techniques, we devise a policy which aims to balance the tradeoff between learning the sensory function and covering the environment. We propose an adaptive coverage algorithm called Deterministic Sequencing of Lear…
▽ More
We study the problem of distributed multi-robot coverage over an unknown, nonuniform sensory field. Modeling the sensory field as a realization of a Gaussian Process and using Bayesian techniques, we devise a policy which aims to balance the tradeoff between learning the sensory function and covering the environment. We propose an adaptive coverage algorithm called Deterministic Sequencing of Learning and Coverage (DSLC) that schedules learning and coverage epochs such that its emphasis gradually shifts from exploration to exploitation while never fully ceasing to learn. Using a novel definition of coverage regret which characterizes overall coverage performance of a multi-robot team over a time horizon $T$, we analyze DSLC to provide an upper bound on expected cumulative coverage regret. Finally, we illustrate the empirical performance of the algorithm through simulations of the coverage task over an unknown distribution of wildfires.
△ Less
Submitted 31 May, 2021; v1 submitted 12 January, 2021;
originally announced January 2021.
-
Crash Themes in Automated Vehicles: A Topic Modeling Analysis of the California Department of Motor Vehicles Automated Vehicle Crash Database
Authors:
Hananeh Alambeigi,
Anthony D. McDonald,
Srinivas R. Tankasala
Abstract:
Automated vehicle technology promises to reduce the societal impact of traffic crashes. Early investigations of this technology suggest that significant safety issues remain during control transfers between the automation and human drivers and automation interactions with the transportation system. In order to address these issues, it is critical to understand both the behavior of human drivers du…
▽ More
Automated vehicle technology promises to reduce the societal impact of traffic crashes. Early investigations of this technology suggest that significant safety issues remain during control transfers between the automation and human drivers and automation interactions with the transportation system. In order to address these issues, it is critical to understand both the behavior of human drivers during these events and the environments where they occur. This article analyzes automated vehicle crash narratives from the California Department of Motor Vehicles automated vehicle crash database to identify safety concerns and gaps between crash types and current areas of focus in the current research. The database was analyzed using probabilistic topic modeling of open-ended crash narratives. Topic modeling analysis identified five themes in the database: driver-initiated transition crashes, sideswipe crashes during left-side overtakes, and rear-end collisions while the vehicle was stopped at an intersection, in a turn lane, and when the crash involved oncoming traffic. Many crashes represented by the driver-initiated transitions topic were also associated with the side-swipe collisions. A substantial portion of the side-swipe collisions also involved motorcycles. These findings highlight previously raised safety concerns with transitions of control and interactions between vehicles in automated mode and the transportation social network. In response to these findings, future empirical work should focus on driver-initiated transitions, overtakes, silent failures, complex traffic situations, and adverse driving environments. Beyond this future work, the topic modeling analysis method may be used as a tool to monitor emergent safety issues.
△ Less
Submitted 29 January, 2020;
originally announced January 2020.
-
Fitting Spectral Decay with the $k$-Support Norm
Authors:
Andrew M. McDonald,
Massimiliano Pontil,
Dimitris Stamos
Abstract:
The spectral $k$-support norm enjoys good estimation properties in low rank matrix learning problems, empirically outperforming the trace norm. Its unit ball is the convex hull of rank $k$ matrices with unit Frobenius norm. In this paper we generalize the norm to the spectral $(k,p)$-support norm, whose additional parameter $p$ can be used to tailor the norm to the decay of the spectrum of the und…
▽ More
The spectral $k$-support norm enjoys good estimation properties in low rank matrix learning problems, empirically outperforming the trace norm. Its unit ball is the convex hull of rank $k$ matrices with unit Frobenius norm. In this paper we generalize the norm to the spectral $(k,p)$-support norm, whose additional parameter $p$ can be used to tailor the norm to the decay of the spectrum of the underlying model. We characterize the unit ball and we explicitly compute the norm. We further provide a conditional gradient method to solve regularization problems with the norm, and we derive an efficient algorithm to compute the Euclidean projection on the unit ball in the case $p=\infty$. In numerical experiments, we show that allowing $p$ to vary significantly improves performance over the spectral $k$-support norm on various matrix completion benchmarks, and better captures the spectral decay of the underlying model.
△ Less
Submitted 4 January, 2016;
originally announced January 2016.
-
New Perspectives on $k$-Support and Cluster Norms
Authors:
Andrew M. McDonald,
Massimiliano Pontil,
Dimitris Stamos
Abstract:
We study a regularizer which is defined as a parameterized infimum of quadratics, and which we call the box-norm. We show that the k-support norm, a regularizer proposed by [Argyriou et al, 2012] for sparse vector prediction problems, belongs to this family, and the box-norm can be generated as a perturbation of the former. We derive an improved algorithm to compute the proximity operator of the s…
▽ More
We study a regularizer which is defined as a parameterized infimum of quadratics, and which we call the box-norm. We show that the k-support norm, a regularizer proposed by [Argyriou et al, 2012] for sparse vector prediction problems, belongs to this family, and the box-norm can be generated as a perturbation of the former. We derive an improved algorithm to compute the proximity operator of the squared box-norm, and we provide a method to compute the norm. We extend the norms to matrices, introducing the spectral k-support norm and spectral box-norm. We note that the spectral box-norm is essentially equivalent to the cluster norm, a multitask learning regularizer introduced by [Jacob et al. 2009a], and which in turn can be interpreted as a perturbation of the spectral k-support norm. Centering the norm is important for multitask learning and we also provide a method to use centered versions of the norms as regularizers. Numerical experiments indicate that the spectral k-support and box-norms and their centered variants provide state of the art performance in matrix completion and multitask learning problems respectively.
△ Less
Submitted 27 December, 2015;
originally announced December 2015.
-
New Perspectives on k-Support and Cluster Norms
Authors:
Andrew M. McDonald,
Massimiliano Pontil,
Dimitris Stamos
Abstract:
The $k$-support norm is a regularizer which has been successfully applied to sparse vector prediction problems. We show that it belongs to a general class of norms which can be formulated as a parameterized infimum over quadratics. We further extend the $k$-support norm to matrices, and we observe that it is a special case of the matrix cluster norm. Using this formulation we derive an efficient a…
▽ More
The $k$-support norm is a regularizer which has been successfully applied to sparse vector prediction problems. We show that it belongs to a general class of norms which can be formulated as a parameterized infimum over quadratics. We further extend the $k$-support norm to matrices, and we observe that it is a special case of the matrix cluster norm. Using this formulation we derive an efficient algorithm to compute the proximity operator of both norms. This improves upon the standard algorithm for the $k$-support norm and allows us to apply proximal gradient methods to the cluster norm. We also describe how to solve regularization problems which employ centered versions of these norms. Finally, we apply the matrix regularizers to different matrix completion and multitask learning datasets. Our results indicate that the spectral $k$-support norm and the cluster norm give state of the art performance on these problems, significantly outperforming trace norm and elastic net penalties.
△ Less
Submitted 6 March, 2014;
originally announced March 2014.
