-
Memory Bounded Deep Convolutional Networks
Authors:
Maxwell D. Collins,
Pushmeet Kohli
Abstract:
In this work, we investigate the use of sparsity-inducing regularizers during training of Convolution Neural Networks (CNNs). These regularizers encourage that fewer connections in the convolution and fully connected layers take non-zero values and in effect result in sparse connectivity between hidden units in the deep network. This in turn reduces the memory and runtime cost involved in deployin…
▽ More
In this work, we investigate the use of sparsity-inducing regularizers during training of Convolution Neural Networks (CNNs). These regularizers encourage that fewer connections in the convolution and fully connected layers take non-zero values and in effect result in sparse connectivity between hidden units in the deep network. This in turn reduces the memory and runtime cost involved in deploying the learned CNNs. We show that training with such regularization can still be performed using stochastic gradient descent implying that it can be used easily in existing codebases. Experimental evaluation of our approach on MNIST, CIFAR, and ImageNet datasets shows that our regularizers can result in dramatic reductions in memory requirements. For instance, when applied on AlexNet, our method can reduce the memory consumption by a factor of four with minimal loss in accuracy.
△ Less
Submitted 3 December, 2014;
originally announced December 2014.
-
Consensus Message Passing for Layered Graphical Models
Authors:
Varun Jampani,
S. M. Ali Eslami,
Daniel Tarlow,
Pushmeet Kohli,
John Winn
Abstract:
Generative models provide a powerful framework for probabilistic reasoning. However, in many domains their use has been hampered by the practical difficulties of inference. This is particularly the case in computer vision, where models of the imaging process tend to be large, loopy and layered. For this reason bottom-up conditional models have traditionally dominated in such domains. We find that…
▽ More
Generative models provide a powerful framework for probabilistic reasoning. However, in many domains their use has been hampered by the practical difficulties of inference. This is particularly the case in computer vision, where models of the imaging process tend to be large, loopy and layered. For this reason bottom-up conditional models have traditionally dominated in such domains. We find that widely-used, general-purpose message passing inference algorithms such as Expectation Propagation (EP) and Variational Message Passing (VMP) fail on the simplest of vision models. With these models in mind, we introduce a modification to message passing that learns to exploit their layered structure by passing 'consensus' messages that guide inference towards good solutions. Experiments on a variety of problems show that the proposed technique leads to significantly more accurate inference results, not only when compared to standard EP and VMP, but also when compared to competitive bottom-up conditional models.
△ Less
Submitted 26 January, 2015; v1 submitted 27 October, 2014;
originally announced October 2014.
-
Inverse Graphics with Probabilistic CAD Models
Authors:
Tejas D. Kulkarni,
Vikash K. Mansinghka,
Pushmeet Kohli,
Joshua B. Tenenbaum
Abstract:
Recently, multiple formulations of vision problems as probabilistic inversions of generative models based on computer graphics have been proposed. However, applications to 3D perception from natural images have focused on low-dimensional latent scenes, due to challenges in both modeling and inference. Accounting for the enormous variability in 3D object shape and 2D appearance via realistic genera…
▽ More
Recently, multiple formulations of vision problems as probabilistic inversions of generative models based on computer graphics have been proposed. However, applications to 3D perception from natural images have focused on low-dimensional latent scenes, due to challenges in both modeling and inference. Accounting for the enormous variability in 3D object shape and 2D appearance via realistic generative models seems intractable, as does inverting even simple versions of the many-to-many computations that link 3D scenes to 2D images. This paper proposes and evaluates an approach that addresses key aspects of both these challenges. We show that it is possible to solve challenging, real-world 3D vision problems by approximate inference in generative models for images based on rendering the outputs of probabilistic CAD (PCAD) programs. Our PCAD object geometry priors generate deformable 3D meshes corresponding to plausible objects and apply affine transformations to place them in a scene. Image likelihoods are based on similarity in a feature space based on standard mid-level image representations from the vision literature. Our inference algorithm integrates single-site and locally blocked Metropolis-Hastings proposals, Hamiltonian Monte Carlo and discriminative data-driven proposals learned from training data generated from our models. We apply this approach to 3D human pose estimation and object shape reconstruction from single images, achieving quantitative and qualitative performance improvements over state-of-the-art baselines.
△ Less
Submitted 4 July, 2014;
originally announced July 2014.
-
Multi-utility Learning: Structured-output Learning with Multiple Annotation-specific Loss Functions
Authors:
Roman Shapovalov,
Dmitry Vetrov,
Anton Osokin,
Pushmeet Kohli
Abstract:
Structured-output learning is a challenging problem; particularly so because of the difficulty in obtaining large datasets of fully labelled instances for training. In this paper we try to overcome this difficulty by presenting a multi-utility learning framework for structured prediction that can learn from training instances with different forms of supervision. We propose a unified technique for…
▽ More
Structured-output learning is a challenging problem; particularly so because of the difficulty in obtaining large datasets of fully labelled instances for training. In this paper we try to overcome this difficulty by presenting a multi-utility learning framework for structured prediction that can learn from training instances with different forms of supervision. We propose a unified technique for inferring the loss functions most suitable for quantifying the consistency of solutions with the given weak annotation. We demonstrate the effectiveness of our framework on the challenging semantic image segmentation problem for which a wide variety of annotations can be used. For instance, the popular training datasets for semantic segmentation are composed of images with hard-to-generate full pixel labellings, as well as images with easy-to-obtain weak annotations, such as bounding boxes around objects, or image-level labels that specify which object categories are present in an image. Experimental evaluation shows that the use of annotation-specific loss functions dramatically improves segmentation accuracy compared to the baseline system where only one type of weak annotation is used.
△ Less
Submitted 23 June, 2014;
originally announced June 2014.
-
Partition-Merge: Distributed Inference and Modularity Optimization
Authors:
Vincent Blondel,
Kyomin Jung,
Pushmeet Kohli,
Devavrat Shah
Abstract:
This paper presents a novel meta algorithm, Partition-Merge (PM), which takes existing centralized algorithms for graph computation and makes them distributed and faster. In a nutshell, PM divides the graph into small subgraphs using our novel randomized partitioning scheme, runs the centralized algorithm on each partition separately, and then stitches the resulting solutions to produce a global s…
▽ More
This paper presents a novel meta algorithm, Partition-Merge (PM), which takes existing centralized algorithms for graph computation and makes them distributed and faster. In a nutshell, PM divides the graph into small subgraphs using our novel randomized partitioning scheme, runs the centralized algorithm on each partition separately, and then stitches the resulting solutions to produce a global solution. We demonstrate the efficiency of the PM algorithm on two popular problems: computation of Maximum A Posteriori (MAP) assignment in an arbitrary pairwise Markov Random Field (MRF), and modularity optimization for community detection. We show that the resulting distributed algorithms for these problems essentially run in time linear in the number of nodes in the graph, and perform as well -- or even better -- than the original centralized algorithm as long as the graph has geometric structures. Here we say a graph has geometric structures, or polynomial growth property, when the number of nodes within distance r of any given node grows no faster than a polynomial function of r. More precisely, if the centralized algorithm is a C-factor approximation with constant C \ge 1, the resulting distributed algorithm is a (C+δ)-factor approximation for any small δ>0; but if the centralized algorithm is a non-constant (e.g. logarithmic) factor approximation, then the resulting distributed algorithm becomes a constant factor approximation. For general graphs, we compute explicit bounds on the loss of performance of the resulting distributed algorithm with respect to the centralized algorithm.
△ Less
Submitted 24 September, 2013;
originally announced September 2013.
-
Efficient Energy Minimization for Enforcing Statistics
Authors:
Yongsub Lim,
Kyomin Jung,
Pushmeet Kohli
Abstract:
Energy minimization algorithms, such as graph cuts, enable the computation of the MAP solution under certain probabilistic models such as Markov random fields. However, for many computer vision problems, the MAP solution under the model is not the ground truth solution. In many problem scenarios, the system has access to certain statistics of the ground truth. For instance, in image segmentation,…
▽ More
Energy minimization algorithms, such as graph cuts, enable the computation of the MAP solution under certain probabilistic models such as Markov random fields. However, for many computer vision problems, the MAP solution under the model is not the ground truth solution. In many problem scenarios, the system has access to certain statistics of the ground truth. For instance, in image segmentation, the area and boundary length of the object may be known. In these cases, we want to estimate the most probable solution that is consistent with such statistics, i.e., satisfies certain equality or inequality constraints.
The above constrained energy minimization problem is NP-hard in general, and is usually solved using Linear Programming formulations, which relax the integrality constraints. This paper proposes a novel method that finds the discrete optimal solution of such problems by maximizing the corresponding Lagrangian dual. This method can be applied to any constrained energy minimization problem whose unconstrained version is polynomial time solvable, and can handle multiple, equality or inequality, and linear or non-linear constraints. We demonstrate the efficacy of our method on the foreground/background image segmentation problem, and show that it produces impressive segmentation results with less error, and runs more than 20 times faster than the state-of-the-art LP relaxation based approaches.
△ Less
Submitted 29 July, 2013;
originally announced July 2013.
-
Multi-dimensional Parametric Mincuts for Constrained MAP Inference
Authors:
Yongsub Lim,
Kyomin Jung,
Pushmeet Kohli
Abstract:
In this paper, we propose novel algorithms for inferring the Maximum a Posteriori (MAP) solution of discrete pairwise random field models under multiple constraints. We show how this constrained discrete optimization problem can be formulated as a multi-dimensional parametric mincut problem via its Lagrangian dual, and prove that our algorithm isolates all constraint instances for which the proble…
▽ More
In this paper, we propose novel algorithms for inferring the Maximum a Posteriori (MAP) solution of discrete pairwise random field models under multiple constraints. We show how this constrained discrete optimization problem can be formulated as a multi-dimensional parametric mincut problem via its Lagrangian dual, and prove that our algorithm isolates all constraint instances for which the problem can be solved exactly. These multiple solutions enable us to even deal with `soft constraints' (higher order penalty functions). Moreover, we propose two practical variants of our algorithm to solve problems with hard constraints. We also show how our method can be applied to solve various constrained discrete optimization problems such as submodular minimization and shortest path computation. Experimental evaluation using the foreground-background image segmentation problem with statistic constraints reveals that our method is faster and its results are closer to the ground truth labellings compared with the popular continuous relaxation based methods.
△ Less
Submitted 29 July, 2013;
originally announced July 2013.
-
A two-layer Conditional Random Field for the classification of partially occluded objects
Authors:
Sergey Kosov,
Pushmeet Kohli,
Franz Rottensteiner,
Christian Heipke
Abstract:
Conditional Random Fields (CRF) are among the most popular techniques for image labelling because of their flexibility in modelling dependencies between the labels and the image features. This paper proposes a novel CRF-framework for image labeling problems which is capable to classify partially occluded objects. Our approach is evaluated on aerial near-vertical images as well as on urban street-v…
▽ More
Conditional Random Fields (CRF) are among the most popular techniques for image labelling because of their flexibility in modelling dependencies between the labels and the image features. This paper proposes a novel CRF-framework for image labeling problems which is capable to classify partially occluded objects. Our approach is evaluated on aerial near-vertical images as well as on urban street-view images and compared with another methods.
△ Less
Submitted 13 September, 2013; v1 submitted 11 July, 2013;
originally announced July 2013.
-
Exact and Approximate Inference in Associative Hierarchical Networks using Graph Cuts
Authors:
Chris Russell,
L'ubor Ladicky,
Pushmeet Kohli,
Philip H. S. Torr
Abstract:
Markov Networks are widely used through out computer vision and machine learning. An important subclass are the Associative Markov Networks which are used in a wide variety of applications. For these networks a good approximate minimum cost solution can be found efficiently using graph cut based move making algorithms such as alpha-expansion. Recently a related model has been proposed, the associa…
▽ More
Markov Networks are widely used through out computer vision and machine learning. An important subclass are the Associative Markov Networks which are used in a wide variety of applications. For these networks a good approximate minimum cost solution can be found efficiently using graph cut based move making algorithms such as alpha-expansion. Recently a related model has been proposed, the associative hierarchical network, which provides a natural generalisation of the Associative Markov Network for higher order cliques (i.e. clique size greater than two). This method provides a good model for object class segmentation problem in computer vision. Within this paper we briefly describe the associative hierarchical network and provide a computationally efficient method for approximate inference based on graph cuts. Our method performs well for networks containing hundreds of thousand of variables, and higher order potentials are defined over cliques containing tens of thousands of variables. Due to the size of these problems standard linear programming techniques are inapplicable. We show that our method has a bound of 4 for the solution of general associative hierarchical network with arbitrary clique size noting that few results on bounds exist for the solution of labelling of Markov Networks with higher order cliques.
△ Less
Submitted 15 March, 2012;
originally announced March 2012.
-
Curvature Prior for MRF-based Segmentation and Shape Inpainting
Authors:
Alexander Shekhovtsov,
Pushmeet Kohli,
Carsten Rother
Abstract:
Most image labeling problems such as segmentation and image reconstruction are fundamentally ill-posed and suffer from ambiguities and noise. Higher order image priors encode high level structural dependencies between pixels and are key to overcoming these problems. However, these priors in general lead to computationally intractable models. This paper addresses the problem of discovering compact…
▽ More
Most image labeling problems such as segmentation and image reconstruction are fundamentally ill-posed and suffer from ambiguities and noise. Higher order image priors encode high level structural dependencies between pixels and are key to overcoming these problems. However, these priors in general lead to computationally intractable models. This paper addresses the problem of discovering compact representations of higher order priors which allow efficient inference. We propose a framework for solving this problem which uses a recently proposed representation of higher order functions where they are encoded as lower envelopes of linear functions. Maximum a Posterior inference on our learned models reduces to minimizing a pairwise function of discrete variables, which can be done approximately using standard methods. Although this is a primarily theoretical paper, we also demonstrate the practical effectiveness of our framework on the problem of learning a shape prior for image segmentation and reconstruction. We show that our framework can learn a compact representation that approximates a prior that encourages low curvature shapes. We evaluate the approximation accuracy, discuss properties of the trained model, and show various results for shape inpainting and image segmentation.
△ Less
Submitted 7 September, 2011;
originally announced September 2011.
-
Optimal Coalition Structures in Cooperative Graph Games
Authors:
Yoram Bachrach,
Pushmeet Kohli,
Vladimir Kolmogorov,
Morteza Zadimoghaddam
Abstract:
Representation languages for coalitional games are a key research area in algorithmic game theory. There is an inherent tradeoff between how general a language is, allowing it to capture more elaborate games, and how hard it is computationally to optimize and solve such games. One prominent such language is the simple yet expressive Weighted Graph Games (WGGs) representation [14], which maintains…
▽ More
Representation languages for coalitional games are a key research area in algorithmic game theory. There is an inherent tradeoff between how general a language is, allowing it to capture more elaborate games, and how hard it is computationally to optimize and solve such games. One prominent such language is the simple yet expressive Weighted Graph Games (WGGs) representation [14], which maintains knowledge about synergies between agents in the form of an edge weighted graph.
We consider the problem of finding the optimal coalition structure in WGGs. The agents in such games are vertices in a graph, and the value of a coalition is the sum of the weights of the edges present between coalition members. The optimal coalition structure is a partition of the agents to coalitions, that maximizes the sum of utilities obtained by the coalitions. We show that finding the optimal coalition structure is not only hard for general graphs, but is also intractable for restricted families such as planar graphs which are amenable for many other combinatorial problems. We then provide algorithms with constant factor approximations for planar, minor-free and bounded degree graphs.
△ Less
Submitted 14 April, 2013; v1 submitted 26 August, 2011;
originally announced August 2011.
-
Learning an Interactive Segmentation System
Authors:
Hannes Nickisch,
Pushmeet Kohli,
Carsten Rother
Abstract:
Many successful applications of computer vision to image or video manipulation are interactive by nature. However, parameters of such systems are often trained neglecting the user. Traditionally, interactive systems have been treated in the same manner as their fully automatic counterparts. Their performance is evaluated by computing the accuracy of their solutions under some fixed set of user i…
▽ More
Many successful applications of computer vision to image or video manipulation are interactive by nature. However, parameters of such systems are often trained neglecting the user. Traditionally, interactive systems have been treated in the same manner as their fully automatic counterparts. Their performance is evaluated by computing the accuracy of their solutions under some fixed set of user interactions. This paper proposes a new evaluation and learning method which brings the user in the loop. It is based on the use of an active robot user - a simulated model of a human user. We show how this approach can be used to evaluate and learn parameters of state-of-the-art interactive segmentation systems. We also show how simulated user models can be integrated into the popular max-margin method for parameter learning and propose an algorithm to solve the resulting optimisation problem.
△ Less
Submitted 13 December, 2009;
originally announced December 2009.
-
Coarse-grained Dynamic Taint Analysis for Defeating Control and Non-control Data Attacks
Authors:
Pankaj Kohli
Abstract:
Memory corruption attacks remain the primary threat for computer security. Information flow tracking or taint analysis has been proven to be effective against most memory corruption attacks. However, there are two shortcomings with current taint analysis based techniques. First, these techniques cause application slowdown by about 76% thereby limiting their practicality. Second, these techniques…
▽ More
Memory corruption attacks remain the primary threat for computer security. Information flow tracking or taint analysis has been proven to be effective against most memory corruption attacks. However, there are two shortcomings with current taint analysis based techniques. First, these techniques cause application slowdown by about 76% thereby limiting their practicality. Second, these techniques cannot handle non-control data attacks i.e., attacks that do not overwrite control data such as return address, but instead overwrite critical application configuration data or user identity data. In this work, to address these problems, we describe a coarse-grained taint analysis technique that uses information flow tracking at the level of application data objects. We propagate a one-bit taint over each application object that is modified by untrusted data thereby reducing the taint management overhead considerably. We performed extensive experimental evaluation of our approach and show that it can detect all critical attacks such as buffer overflows, and format string attacks, including non-control data attacks. Unlike the currently known approaches that can detect such a wide range of attacks, our approach does not require the source code or any hardware extensions. Run-time performance overhead evaluation shows that, on an average, our approach causes application slowdown by only 37% which is an order of magnitude improvement over existing approaches. Finally, since our approach performs run-time binary instrumentation, it is easier to integrate it with existing applications and systems.
△ Less
Submitted 1 July, 2009; v1 submitted 24 June, 2009;
originally announced June 2009.
