-
The Artificial Regression Market
Authors:
Nathan Lay,
Adrian Barbu
Abstract:
The Artificial Prediction Market is a recent machine learning technique for multi-class classification, inspired from the financial markets. It involves a number of trained market participants that bet on the possible outcomes and are rewarded if they predict correctly. This paper generalizes the scope of the Artificial Prediction Markets to regression, where there are uncountably many possible ou…
▽ More
The Artificial Prediction Market is a recent machine learning technique for multi-class classification, inspired from the financial markets. It involves a number of trained market participants that bet on the possible outcomes and are rewarded if they predict correctly. This paper generalizes the scope of the Artificial Prediction Markets to regression, where there are uncountably many possible outcomes and the error is usually the MSE. For that, we introduce the reward kernel that rewards each participant based on its prediction error and we derive the price equations. Using two reward kernels we obtain two different learning rules, one of which is approximated using Hermite-Gauss quadrature. The market setting makes it easy to aggregate specialized regressors that only predict when an observation falls into their specialization domain. Experiments show that regression markets based on the two learning rules outperform Random Forest Regression on many UCI datasets and are rarely outperformed.
△ Less
Submitted 18 April, 2012;
originally announced April 2012.
-
Large-Scale Automatic Labeling of Video Events with Verbs Based on Event-Participant Interaction
Authors:
Andrei Barbu,
Alexander Bridge,
Dan Coroian,
Sven Dickinson,
Sam Mussman,
Siddharth Narayanaswamy,
Dhaval Salvi,
Lara Schmidt,
Jiangnan Shangguan,
Jeffrey Mark Siskind,
Jarrell Waggoner,
Song Wang,
**lian Wei,
Yifan Yin,
Zhiqi Zhang
Abstract:
We present an approach to labeling short video clips with English verbs as event descriptions. A key distinguishing aspect of this work is that it labels videos with verbs that describe the spatiotemporal interaction between event participants, humans and objects interacting with each other, abstracting away all object-class information and fine-grained image characteristics, and relying solely on…
▽ More
We present an approach to labeling short video clips with English verbs as event descriptions. A key distinguishing aspect of this work is that it labels videos with verbs that describe the spatiotemporal interaction between event participants, humans and objects interacting with each other, abstracting away all object-class information and fine-grained image characteristics, and relying solely on the coarse-grained motion of the event participants. We apply our approach to a large set of 22 distinct verb classes and a corpus of 2,584 videos, yielding two surprising outcomes. First, a classification accuracy of greater than 70% on a 1-out-of-22 labeling task and greater than 85% on a variety of 1-out-of-10 subsets of this labeling task is independent of the choice of which of two different time-series classifiers we employ. Second, we achieve this level of accuracy using a highly impoverished intermediate representation consisting solely of the bounding boxes of one or two event participants as a function of time. This indicates that successful event recognition depends more on the choice of appropriate features that characterize the linguistic invariants of the event classes than on the particular classifier algorithms.
△ Less
Submitted 16 April, 2012;
originally announced April 2012.
-
Seeing Unseeability to See the Unseeable
Authors:
Siddharth Narayanaswamy,
Andrei Barbu,
Jeffrey Mark Siskind
Abstract:
We present a framework that allows an observer to determine occluded portions of a structure by finding the maximum-likelihood estimate of those occluded portions consistent with visible image evidence and a consistency model. Doing this requires determining which portions of the structure are occluded in the first place. Since each process relies on the other, we determine a solution to both prob…
▽ More
We present a framework that allows an observer to determine occluded portions of a structure by finding the maximum-likelihood estimate of those occluded portions consistent with visible image evidence and a consistency model. Doing this requires determining which portions of the structure are occluded in the first place. Since each process relies on the other, we determine a solution to both problems in tandem. We extend our framework to determine confidence of one's assessment of which portions of an observed structure are occluded, and the estimate of that occluded structure, by determining the sensitivity of one's assessment to potential new observations. We further extend our framework to determine a robotic action whose execution would allow a new observation that would maximally increase one's confidence.
△ Less
Submitted 12 April, 2012;
originally announced April 2012.
-
Video In Sentences Out
Authors:
Andrei Barbu,
Alexander Bridge,
Zachary Burchill,
Dan Coroian,
Sven Dickinson,
Sanja Fidler,
Aaron Michaux,
Sam Mussman,
Siddharth Narayanaswamy,
Dhaval Salvi,
Lara Schmidt,
Jiangnan Shangguan,
Jeffrey Mark Siskind,
Jarrell Waggoner,
Song Wang,
**lian Wei,
Yifan Yin,
Zhiqi Zhang
Abstract:
We present a system that produces sentential descriptions of video: who did what to whom, and where and how they did it. Action class is rendered as a verb, participant objects as noun phrases, properties of those objects as adjectival modifiers in those noun phrases,spatial relations between those participants as prepositional phrases, and characteristics of the event as prepositional-phrase adju…
▽ More
We present a system that produces sentential descriptions of video: who did what to whom, and where and how they did it. Action class is rendered as a verb, participant objects as noun phrases, properties of those objects as adjectival modifiers in those noun phrases,spatial relations between those participants as prepositional phrases, and characteristics of the event as prepositional-phrase adjuncts and adverbial modifiers. Extracting the information needed to render these linguistic entities requires an approach to event recognition that recovers object tracks, the track-to-role assignments, and changing body posture.
△ Less
Submitted 12 April, 2012;
originally announced April 2012.
-
Simultaneous Object Detection, Tracking, and Event Recognition
Authors:
Andrei Barbu,
Aaron Michaux,
Siddharth Narayanaswamy,
Jeffrey Mark Siskind
Abstract:
The common internal structure and algorithmic organization of object detection, detection-based tracking, and event recognition facilitates a general approach to integrating these three components. This supports multidirectional information flow between these components allowing object detection to influence tracking and event recognition and event recognition to influence tracking and object dete…
▽ More
The common internal structure and algorithmic organization of object detection, detection-based tracking, and event recognition facilitates a general approach to integrating these three components. This supports multidirectional information flow between these components allowing object detection to influence tracking and event recognition and event recognition to influence tracking and object detection. The performance of the combination can exceed the performance of the components in isolation. This can be done with linear asymptotic complexity.
△ Less
Submitted 12 April, 2012;
originally announced April 2012.
-
Hierarchical Object Parsing from Structured Noisy Point Clouds
Authors:
Adrian Barbu
Abstract:
Object parsing and segmentation from point clouds are challenging tasks because the relevant data is available only as thin structures along object boundaries or other features, and is corrupted by large amounts of noise. To handle this kind of data, flexible shape models are desired that can accurately follow the object boundaries. Popular models such as Active Shape and Active Appearance models…
▽ More
Object parsing and segmentation from point clouds are challenging tasks because the relevant data is available only as thin structures along object boundaries or other features, and is corrupted by large amounts of noise. To handle this kind of data, flexible shape models are desired that can accurately follow the object boundaries. Popular models such as Active Shape and Active Appearance models lack the necessary flexibility for this task, while recent approaches such as the Recursive Compositional Models make model simplifications in order to obtain computational guarantees. This paper investigates a hierarchical Bayesian model of shape and appearance in a generative setting. The input data is explained by an object parsing layer, which is a deformation of a hidden PCA shape model with Gaussian prior. The paper also introduces a novel efficient inference algorithm that uses informed data-driven proposals to initialize local searches for the hidden variables. Applied to the problem of object parsing from structured point clouds such as edge detection images, the proposed approach obtains state of the art parsing errors on two standard datasets without using any intensity information.
△ Less
Submitted 15 September, 2012; v1 submitted 17 August, 2011;
originally announced August 2011.
-
An Introduction to Artificial Prediction Markets for Classification
Authors:
Adrian Barbu,
Nathan Lay
Abstract:
Prediction markets are used in real life to predict outcomes of interest such as presidential elections. This paper presents a mathematical theory of artificial prediction markets for supervised learning of conditional probability estimators. The artificial prediction market is a novel method for fusing the prediction information of features or trained classifiers, where the fusion result is the c…
▽ More
Prediction markets are used in real life to predict outcomes of interest such as presidential elections. This paper presents a mathematical theory of artificial prediction markets for supervised learning of conditional probability estimators. The artificial prediction market is a novel method for fusing the prediction information of features or trained classifiers, where the fusion result is the contract price on the possible outcomes. The market can be trained online by updating the participants' budgets using training examples. Inspired by the real prediction markets, the equations that govern the market are derived from simple and reasonable assumptions. Efficient numerical algorithms are presented for solving these equations. The obtained artificial prediction market is shown to be a maximum likelihood estimator. It generalizes linear aggregation, existent in boosting and random forest, as well as logistic regression and some kernel methods. Furthermore, the market mechanism allows the aggregation of specialized classifiers that participate only on specific instances. Experimental comparisons show that the artificial prediction markets often outperform random forest and implicit online learning on synthetic data and real UCI datasets. Moreover, an extensive evaluation for pelvic and abdominal lymph node detection in CT data shows that the prediction market improves adaboost's detection rate from 79.6% to 81.2% at 3 false positives/volume.
△ Less
Submitted 9 July, 2012; v1 submitted 7 February, 2011;
originally announced February 2011.
-
Dimension reduction and variable selection in case control studies via regularized likelihood optimization
Authors:
Florentina Bunea,
Adrian Barbu
Abstract:
Dimension reduction and variable selection are performed routinely in case-control studies, but the literature on the theoretical aspects of the resulting estimates is scarce. We bring our contribution to this literature by studying estimators obtained via L1 penalized likelihood optimization. We show that the optimizers of the L1 penalized retrospective likelihood coincide with the optimizers o…
▽ More
Dimension reduction and variable selection are performed routinely in case-control studies, but the literature on the theoretical aspects of the resulting estimates is scarce. We bring our contribution to this literature by studying estimators obtained via L1 penalized likelihood optimization. We show that the optimizers of the L1 penalized retrospective likelihood coincide with the optimizers of the L1 penalized prospective likelihood. This extends the results of Prentice and Pyke (1979), obtained for non-regularized likelihoods. We establish both the sup-norm consistency of the odds ratio, after model selection, and the consistency of subset selection of our estimators. The novelty of our theoretical results consists in the study of these properties under the case-control sampling scheme. Our results hold for selection performed over a large collection of candidate variables, with cardinality allowed to depend and be greater than the sample size. We complement our theoretical results with a novel approach of determining data driven tuning parameters, based on the bisection method. The resulting procedure offers significant computational savings when compared with grid search based methods. All our numerical experiments support strongly our theoretical findings.
△ Less
Submitted 20 November, 2009; v1 submitted 13 May, 2009;
originally announced May 2009.
-
SPADES and mixture models
Authors:
Florentina Bunea,
Alexandre B. Tsybakov,
Marten H. Wegkamp,
Adrian Barbu
Abstract:
This paper studies sparse density estimation via $\ell_1$ penalization (SPADES). We focus on estimation in high-dimensional mixture models and nonparametric adaptive density estimation. We show, respectively, that SPADES can recover, with high probability, the unknown components of a mixture of probability densities and that it yields minimax adaptive density estimates. These results are based on…
▽ More
This paper studies sparse density estimation via $\ell_1$ penalization (SPADES). We focus on estimation in high-dimensional mixture models and nonparametric adaptive density estimation. We show, respectively, that SPADES can recover, with high probability, the unknown components of a mixture of probability densities and that it yields minimax adaptive density estimates. These results are based on a general sparsity oracle inequality that the SPADES estimates satisfy. We offer a data driven method for the choice of the tuning parameter used in the construction of SPADES. The method uses the generalized bisection method first introduced in \citebb09. The suggested procedure bypasses the need for a grid search and offers substantial computational savings. We complement our theoretical results with a simulation study that employs this method for approximations of one and two-dimensional densities with mixtures. The numerical results strongly support our theoretical findings.
△ Less
Submitted 21 October, 2010; v1 submitted 14 January, 2009;
originally announced January 2009.
-
Cluster Dynamics Modeling of Materials: Advantages and Limitations
Authors:
Alain Barbu,
Emmanuel Clouet
Abstract:
The aim of this paper is to give a short review on cluster dynamics modeling in the field of atoms and point defects clustering in materials. It is shown that this method, due to its low computer cost, can handle long term evolution that cannot, in many cases, be obtained by Lattice Kinetic Monte Carlo methods. Indeed, such a possibility is achieved thanks to an important drawback that is the lo…
▽ More
The aim of this paper is to give a short review on cluster dynamics modeling in the field of atoms and point defects clustering in materials. It is shown that this method, due to its low computer cost, can handle long term evolution that cannot, in many cases, be obtained by Lattice Kinetic Monte Carlo methods. Indeed, such a possibility is achieved thanks to an important drawback that is the loss of space correlations of the elements of the microstructures. Some examples, in the field of precipitation and irradiation of metallic materials are given. The limitations and difficulties of this method are also discussed. Unsurprisingly, it is shown that it goes in a very satisfactory way when the objects are distributed homogeneously. Conversely, the source term describing the primary damage under irradiation, by nature heterogeneous in space and time, is tricky to introduce especially when displacement cascades are produced.
△ Less
Submitted 12 September, 2007;
originally announced September 2007.
-
Using Cluster Dynamics to Model Electrical Resistivity Measurements in Precipitating Al-Sc Alloys
Authors:
Emmanuel Clouet,
Alain Barbu
Abstract:
Electrical resistivity evolution during precipitation in Al-Sc alloys is modeled using cluster dynamics. This mesoscopic modeling has already been shown to correctly predict the time evolution of the precipitate size distribution. In this work, we show that it leads too to resistivity predictions in quantitative agreement with experimental data. We only assume that all clusters contribute to the…
▽ More
Electrical resistivity evolution during precipitation in Al-Sc alloys is modeled using cluster dynamics. This mesoscopic modeling has already been shown to correctly predict the time evolution of the precipitate size distribution. In this work, we show that it leads too to resistivity predictions in quantitative agreement with experimental data. We only assume that all clusters contribute to the resistivity and that each cluster contribution is proportional to its area. One interesting result is that the resistivity excess observed during coarsening mainly arises from large clusters and not really from the solid solution. As a consequence, one cannot assume that resistivity asymptotic behavior obeys a simple power law as predicted by LSW theory for the solid solution supersaturation. This forbids any derivation of the precipitate interface free energy or of the solute diffusion coefficient from resistivity experimental data in a phase-separating system like Al-Sc supersaturated alloys.
△ Less
Submitted 20 November, 2006;
originally announced November 2006.
-
Precipitation in Al-Zr-Sc alloys: a comparison between kinetic Monte Carlo, cluster dynamics and classical nucleation theory
Authors:
Emmanuel Clouet,
Maylise Nastar,
Alain Barbu,
Christophe Sigli,
Georges Martin
Abstract:
Zr and Sc precipitate in aluminum alloys to form the Al\_3Zr\_xSc\_{1-x} compound which, for low supersaturations of the solid solution, exhibits the L1\_2 structure. The aim of the present study is to model at an atomic scale the kinetics of precipitation and to build mesoscopic models so as to extend the range of supersaturations and annealing times that can be simulated up to values of practi…
▽ More
Zr and Sc precipitate in aluminum alloys to form the Al\_3Zr\_xSc\_{1-x} compound which, for low supersaturations of the solid solution, exhibits the L1\_2 structure. The aim of the present study is to model at an atomic scale the kinetics of precipitation and to build mesoscopic models so as to extend the range of supersaturations and annealing times that can be simulated up to values of practical interest. In this purpose, we use some ab initio calculations and experimental data to fit an Ising type model describing thermodynamics of the Al-Zr-Sc system. Kinetics of precipitation are studied with a kinetic Monte Carlo algorithm based on an atom-vacancy exchange mechanism. Cluster dynamics is then used to model at a mesoscopic scale all the different stages of homogeneous precipitation in the two binary Al-Zr and Al-Sc alloys. This technique correctly manages to reproduce both the kinetics of precipitation simulated with kinetic Monte Carlo as well as experimental observations. Focusing on the nucleation stage, it is shown that classical theory well applies as long as the short range order tendency of the system is considered. This allows us to propose an extension of classical nucleation theory for the ternary Al-Zr-Sc alloy.
△ Less
Submitted 18 July, 2005; v1 submitted 12 July, 2005;
originally announced July 2005.
-
Precipitation kinetics of Al3Zr and Al3Sc in aluminum alloys modeled with cluster dynamics
Authors:
Emmanuel Clouet,
Alain Barbu,
Ludovic Laé,
Georges Martin
Abstract:
Precipitation kinetics of Al3Zr and Al3Sc in aluminum supersaturated solid solutions is studied using cluster dynamics, a mesoscopic modeling technique which describes the various stages of homogeneous precipitation by a single set of rate equations. The only parameters needed are the interface free energy and the diffusion coefficient which are deduced from an atomic model previously developed…
▽ More
Precipitation kinetics of Al3Zr and Al3Sc in aluminum supersaturated solid solutions is studied using cluster dynamics, a mesoscopic modeling technique which describes the various stages of homogeneous precipitation by a single set of rate equations. The only parameters needed are the interface free energy and the diffusion coefficient which are deduced from an atomic model previously developed to study the same alloys. A comparison with kinetic Monte Carlo simulations based on the vacancy diffusion mechanism shows that cluster dynamics correctly predicts the precipitation kinetics provided a size dependent interface free energy is used. It also manages to reproduce reasonably well existing experimental data.
△ Less
Submitted 21 March, 2005; v1 submitted 18 March, 2005;
originally announced March 2005.
