-
Task-Driven Dictionary Learning for Hyperspectral Image Classification with Structured Sparsity Constraints
Authors:
Xiaoxia Sun,
Nasser M. Nasrabadi,
Trac D. Tran
Abstract:
Sparse representation models a signal as a linear combination of a small number of dictionary atoms. As a generative model, it requires the dictionary to be highly redundant in order to ensure both a stable high sparsity level and a low reconstruction error for the signal. However, in practice, this requirement is usually impaired by the lack of labelled training samples. Fortunately, previous res…
▽ More
Sparse representation models a signal as a linear combination of a small number of dictionary atoms. As a generative model, it requires the dictionary to be highly redundant in order to ensure both a stable high sparsity level and a low reconstruction error for the signal. However, in practice, this requirement is usually impaired by the lack of labelled training samples. Fortunately, previous research has shown that the requirement for a redundant dictionary can be less rigorous if simultaneous sparse approximation is employed, which can be carried out by enforcing various structured sparsity constraints on the sparse codes of the neighboring pixels. In addition, numerous works have shown that applying a variety of dictionary learning methods for the sparse representation model can also improve the classification performance. In this paper, we highlight the task-driven dictionary learning algorithm, which is a general framework for the supervised dictionary learning method. We propose to enforce structured sparsity priors on the task-driven dictionary learning method in order to improve the performance of the hyperspectral classification. Our approach is able to benefit from both the advantages of the simultaneous sparse representation and those of the supervised dictionary learning. We enforce two different structured sparsity priors, the joint and Laplacian sparsity, on the task-driven dictionary learning method and provide the details of the corresponding optimization algorithms. Experiments on numerous popular hyperspectral images demonstrate that the classification performance of our approach is superior to sparse representation classifier with structured priors or the task-driven dictionary learning method.
△ Less
Submitted 3 February, 2015;
originally announced February 2015.
-
Multi-task Image Classification via Collaborative, Hierarchical Spike-and-Slab Priors
Authors:
Hojjat Seyed Mousavi,
Umamahesh Srinivas,
Vishal Monga,
Yuanming Suo,
Minh Dao,
Trac. D. Tran
Abstract:
Promising results have been achieved in image classification problems by exploiting the discriminative power of sparse representations for classification (SRC). Recently, it has been shown that the use of \emph{class-specific} spike-and-slab priors in conjunction with the class-specific dictionaries from SRC is particularly effective in low training scenarios. As a logical extension, we build on t…
▽ More
Promising results have been achieved in image classification problems by exploiting the discriminative power of sparse representations for classification (SRC). Recently, it has been shown that the use of \emph{class-specific} spike-and-slab priors in conjunction with the class-specific dictionaries from SRC is particularly effective in low training scenarios. As a logical extension, we build on this framework for multitask scenarios, wherein multiple representations of the same physical phenomena are available. We experimentally demonstrate the benefits of mining joint information from different camera views for multi-view face recognition.
△ Less
Submitted 30 January, 2015;
originally announced January 2015.
-
Collaborative Multi-sensor Classification via Sparsity-based Representation
Authors:
Minh Dao,
Nam H. Nguyen,
Nasser M. Nasrabadi,
Trac D. Tran
Abstract:
In this paper, we propose a general collaborative sparse representation framework for multi-sensor classification, which takes into account the correlations as well as complementary information between heterogeneous sensors simultaneously while considering joint sparsity within each sensor's observations. We also robustify our models to deal with the presence of sparse noise and low-rank interfere…
▽ More
In this paper, we propose a general collaborative sparse representation framework for multi-sensor classification, which takes into account the correlations as well as complementary information between heterogeneous sensors simultaneously while considering joint sparsity within each sensor's observations. We also robustify our models to deal with the presence of sparse noise and low-rank interference signals. Specifically, we demonstrate that incorporating the noise or interference signal as a low-rank component in our models is essential in a multi-sensor classification problem when multiple co-located sources/sensors simultaneously record the same physical event. We further extend our frameworks to kernelized models which rely on sparsely representing a test sample in terms of all the training samples in a feature space induced by a kernel function. A fast and efficient algorithm based on alternative direction method is proposed where its convergence to an optimal solution is guaranteed. Extensive experiments are conducted on several real multi-sensor data sets and results are compared with the conventional classifiers to verify the effectiveness of the proposed methods.
△ Less
Submitted 16 June, 2016; v1 submitted 29 October, 2014;
originally announced October 2014.
-
Hierarchical Sparse and Collaborative Low-Rank Representation for Emotion Recognition
Authors:
Xiang Xiang,
Minh Dao,
Gregory D. Hager,
Trac D. Tran
Abstract:
In this paper, we design a Collaborative-Hierarchical Sparse and Low-Rank (C-HiSLR) model that is natural for recognizing human emotion in visual data. Previous attempts require explicit expression components, which are often unavailable and difficult to recover. Instead, our model exploits the lowrank property over expressive facial frames and rescue inexact sparse representations by incorporatin…
▽ More
In this paper, we design a Collaborative-Hierarchical Sparse and Low-Rank (C-HiSLR) model that is natural for recognizing human emotion in visual data. Previous attempts require explicit expression components, which are often unavailable and difficult to recover. Instead, our model exploits the lowrank property over expressive facial frames and rescue inexact sparse representations by incorporating group sparsity. For the CK+ dataset, C-HiSLR on raw expressive faces performs as competitive as the Sparse Representation based Classification (SRC) applied on manually prepared emotions. C-HiSLR performs even better than SRC in terms of true positive rate.
△ Less
Submitted 1 April, 2015; v1 submitted 6 October, 2014;
originally announced October 2014.
-
The uniqueness of hierarchically extended backward solutions of the Wright-Fisher model
Authors:
Julian Hofrichter,
Tat Dat Tran,
Jürgen Jost
Abstract:
The diffusion approximation of the Wright-Fisher model of population genetics leads to partial differentiable equations, the so-called Kolmogorov equations, with an operator that degenerates at the boundary. Standard tools do not apply, and in fact, solutions lack regularity properties. In this paper, we develop a regularising blow-up scheme for a certain class of solutions of the backward Kolmogo…
▽ More
The diffusion approximation of the Wright-Fisher model of population genetics leads to partial differentiable equations, the so-called Kolmogorov equations, with an operator that degenerates at the boundary. Standard tools do not apply, and in fact, solutions lack regularity properties. In this paper, we develop a regularising blow-up scheme for a certain class of solutions of the backward Kolmogorov equation, the iteratively extended global solutions presented in \cite{THJ5}, and establish their uniqueness. As the model describes the random genetic drift of several alleles at the same locus from a backward perspective, the singularities result from the loss of an allele. While in an analytical approach, this causes substantial difficulties, from a biological or geometric perspective, this is a natural process that can be analyzed in detail. The presented scheme regularises the solution via a tailored successive transformation of the domain.
△ Less
Submitted 19 July, 2014; v1 submitted 11 July, 2014;
originally announced July 2014.
-
A hierarchical extension scheme for solutions of the Wright-Fisher model
Authors:
Julian Hofrichter,
Tat Dat Tran,
Jürgen Jost
Abstract:
We develop a global and hierarchical scheme for the forward Kolmogorov (Fokker-Planck) equation of the diffusion approximation of the Wright-Fisher model of population genetics. That model describes the random genetic drift of several alleles at the same locus in a population. The key of our scheme is to connect the solutions before and after the loss of an allele. Whereas in an approach via stoch…
▽ More
We develop a global and hierarchical scheme for the forward Kolmogorov (Fokker-Planck) equation of the diffusion approximation of the Wright-Fisher model of population genetics. That model describes the random genetic drift of several alleles at the same locus in a population. The key of our scheme is to connect the solutions before and after the loss of an allele. Whereas in an approach via stochastic processes or partial differential equations, such a loss of an allele leads to a boundary singularity, from a biological or geometric perspective, this is a natural process that can be analyzed in detail. Our method depends on evolution equations for the moments of the process and a careful analysis of the boundary flux.
△ Less
Submitted 18 September, 2015; v1 submitted 19 June, 2014;
originally announced June 2014.
-
A hierarchical extension scheme for backward solutions of the Wright-Fisher model
Authors:
Julian Hofrichter,
Tat Dat Tran,
Jürgen Jost
Abstract:
We develop an iterative global solution scheme for the backward Kolmogorov equation of the diffusion approximation of the Wright-Fisher model of population genetics. That model describes the random genetic drift of several alleles at the same locus in a population from a backward perspective. The key of our scheme is to connect the solutions before and after the loss of an allele. Whereas in an ap…
▽ More
We develop an iterative global solution scheme for the backward Kolmogorov equation of the diffusion approximation of the Wright-Fisher model of population genetics. That model describes the random genetic drift of several alleles at the same locus in a population from a backward perspective. The key of our scheme is to connect the solutions before and after the loss of an allele. Whereas in an approach via stochastic processes or partial differential equations, such a loss of an allele leads to a boundary singularity, from a biological or geometric perspective, this is a natural process that can be analyzed in detail. A clarification of the role of the boundary resolves certain uniqueness issues and enlucidates the construction of hierarchical solutions.
△ Less
Submitted 19 June, 2014;
originally announced June 2014.
-
Structured Dictionary Learning for Classification
Authors:
Yuanming Suo,
Minh Dao,
Umamahesh Srinivas,
Vishal Monga,
Trac D. Tran
Abstract:
Sparsity driven signal processing has gained tremendous popularity in the last decade. At its core, the assumption is that the signal of interest is sparse with respect to either a fixed transformation or a signal dependent dictionary. To better capture the data characteristics, various dictionary learning methods have been proposed for both reconstruction and classification tasks. For classificat…
▽ More
Sparsity driven signal processing has gained tremendous popularity in the last decade. At its core, the assumption is that the signal of interest is sparse with respect to either a fixed transformation or a signal dependent dictionary. To better capture the data characteristics, various dictionary learning methods have been proposed for both reconstruction and classification tasks. For classification particularly, most approaches proposed so far have focused on designing explicit constraints on the sparse code to improve classification accuracy while simply adopting $l_0$-norm or $l_1$-norm for sparsity regularization. Motivated by the success of structured sparsity in the area of Compressed Sensing, we propose a structured dictionary learning framework (StructDL) that incorporates the structure information on both group and task levels in the learning process. Its benefits are two-fold: (i) the label consistency between dictionary atoms and training data are implicitly enforced; and (ii) the classification performance is more robust in the cases of a small dictionary size or limited training data than other techniques. Using the subspace model, we derive the conditions for StructDL to guarantee the performance and show theoretically that StructDL is superior to $l_0$-norm or $l_1$-norm regularized dictionary learning for classification. Extensive experiments have been performed on both synthetic simulations and real world applications, such as face recognition and object classification, to demonstrate the validity of the proposed DL framework.
△ Less
Submitted 7 June, 2014;
originally announced June 2014.
-
The evolution of moment generating functions for the Wright Fisher model of population genetics
Authors:
Tat Dat Tran,
Julian Hofrichter,
Juergen Jost
Abstract:
We derive and apply a partial differential equation for the moment generating function of the Wright-Fisher model of population genetics.
We derive and apply a partial differential equation for the moment generating function of the Wright-Fisher model of population genetics.
△ Less
Submitted 21 January, 2014;
originally announced January 2014.
-
Structured Priors for Sparse-Representation-Based Hyperspectral Image Classification
Authors:
Xiaoxia Sun,
Qing Qu,
Nasser M. Nasrabadi,
Trac D. Tran
Abstract:
Pixel-wise classification, where each pixel is assigned to a predefined class, is one of the most important procedures in hyperspectral image (HSI) analysis. By representing a test pixel as a linear combination of a small subset of labeled pixels, a sparse representation classifier (SRC) gives rather plausible results compared with that of traditional classifiers such as the support vector machine…
▽ More
Pixel-wise classification, where each pixel is assigned to a predefined class, is one of the most important procedures in hyperspectral image (HSI) analysis. By representing a test pixel as a linear combination of a small subset of labeled pixels, a sparse representation classifier (SRC) gives rather plausible results compared with that of traditional classifiers such as the support vector machine (SVM). Recently, by incorporating additional structured sparsity priors, the second generation SRCs have appeared in the literature and are reported to further improve the performance of HSI. These priors are based on exploiting the spatial dependencies between the neighboring pixels, the inherent structure of the dictionary, or both. In this paper, we review and compare several structured priors for sparse-representation-based HSI classification. We also propose a new structured prior called the low rank group prior, which can be considered as a modification of the low rank prior. Furthermore, we will investigate how different structured priors improve the result for the HSI classification.
△ Less
Submitted 15 January, 2014;
originally announced January 2014.
-
Single Crystalline InGaAs Nanopillar Grown on Polysilicon with Dimensions beyond Substrate Grain Size Limit
Authors:
Kar Wei Ng,
Thai-Truong D. Tran,
Wai Son Ko,
Roger Chen,
Fanglu Lu,
Connie J. Chang-Hasnain
Abstract:
Monolithic integration of III-V optoelectronic devices with materials for various functionalities inexpensively is always desirable. Polysilicon (poly-Si) is an ideal platform because it is dopable and semi-conducting and can be deposited and patterned easily on a wide range of low cost substrates. However, the lack of crystalline coherency in poly-Si poses an immense challenge for high-quality ep…
▽ More
Monolithic integration of III-V optoelectronic devices with materials for various functionalities inexpensively is always desirable. Polysilicon (poly-Si) is an ideal platform because it is dopable and semi-conducting and can be deposited and patterned easily on a wide range of low cost substrates. However, the lack of crystalline coherency in poly-Si poses an immense challenge for high-quality epitaxial growth. In this work, we demonstrate, for the first time, direct growth of micron-sized InGaAs/GaAs nanopillars on polysilicon. Transmission electron microscopy shows that the micron-sized pillars are single-crystalline and single Wurzite-phase, far exceeding the substrate crystal grain size ~100nm. The high quality growth is enabled by the unique tapering geometry at the base of the nanostructure, which reduces the effective InGaAs/Si contact area to < 40 nm in diameter. The small footprint not only reduces stress due to lattice mismatch but also prevents the nanopillar from nucleating on multiple Si crystal grains. This relaxes the grain size requirement for poly-Si, potentially reducing the cost for poly-Si deposition. Lasing is achieved in the as-grown pillars under optical pum**, attesting their excellent crystalline and optical quality. These promising results open up a pathway for low-cost synergy of optoelectronics with other technologies such as CMOS integrated circuits, sensing, nanofluidics, thin film transistor display, photovoltaics, etc.
△ Less
Submitted 28 October, 2013;
originally announced October 2013.
-
Robust Lasso with missing and grossly corrupted observations
Authors:
Nam H. Nguyen,
Trac D. Tran
Abstract:
This paper studies the problem of accurately recovering a sparse vector $β^{\star}$ from highly corrupted linear measurements $y = X β^{\star} + e^{\star} + w$ where $e^{\star}$ is a sparse error vector whose nonzero entries may be unbounded and $w$ is a bounded noise. We propose a so-called extended Lasso optimization which takes into consideration sparse prior information of both $β^{\star}$ and…
▽ More
This paper studies the problem of accurately recovering a sparse vector $β^{\star}$ from highly corrupted linear measurements $y = X β^{\star} + e^{\star} + w$ where $e^{\star}$ is a sparse error vector whose nonzero entries may be unbounded and $w$ is a bounded noise. We propose a so-called extended Lasso optimization which takes into consideration sparse prior information of both $β^{\star}$ and $e^{\star}$. Our first result shows that the extended Lasso can faithfully recover both the regression as well as the corruption vector. Our analysis relies on the notion of extended restricted eigenvalue for the design matrix $X$. Our second set of results applies to a general class of Gaussian design matrix $X$ with i.i.d rows $\oper N(0, Σ)$, for which we can establish a surprising result: the extended Lasso can recover exact signed supports of both $β^{\star}$ and $e^{\star}$ from only $Ω(k \log p \log n)$ observations, even when the fraction of corruption is arbitrarily close to one. Our analysis also shows that this amount of observations required to achieve exact signed support is indeed optimal.
△ Less
Submitted 6 December, 2011; v1 submitted 2 December, 2011;
originally announced December 2011.
-
Discriminative Local Sparse Representations for Robust Face Recognition
Authors:
Yi Chen,
Umamahesh Srinivas,
Thong T. Do,
Vishal Monga,
Trac D. Tran
Abstract:
A key recent advance in face recognition models a test face image as a sparse linear combination of a set of training face images. The resulting sparse representations have been shown to possess robustness against a variety of distortions like random pixel corruption, occlusion and disguise. This approach however makes the restrictive (in many scenarios) assumption that test faces must be perfectl…
▽ More
A key recent advance in face recognition models a test face image as a sparse linear combination of a set of training face images. The resulting sparse representations have been shown to possess robustness against a variety of distortions like random pixel corruption, occlusion and disguise. This approach however makes the restrictive (in many scenarios) assumption that test faces must be perfectly aligned (or registered) to the training data prior to classification. In this paper, we propose a simple yet robust local block-based sparsity model, using adaptively-constructed dictionaries from local features in the training data, to overcome this misalignment problem. Our approach is inspired by human perception: we analyze a series of local discriminative features and combine them to arrive at the final classification decision. We propose a probabilistic graphical model framework to explicitly mine the conditional dependencies between these distinct sparse local features. In particular, we learn discriminative graphs on sparse representations obtained from distinct local slices of a face. Conditional correlations between these sparse features are first discovered (in the training phase), and subsequently exploited to bring about significant improvements in recognition rates. Experimental results obtained on benchmark face databases demonstrate the effectiveness of the proposed algorithms in the presence of multiple registration errors (such as translation, rotation, and scaling) as well as under variations of pose and illumination.
△ Less
Submitted 8 November, 2011;
originally announced November 2011.
-
Fast and Efficient Compressive Sensing using Structurally Random Matrices
Authors:
Thong T. Do,
Lu Gan,
Nam H. Nguyen,
Trac D. Tran
Abstract:
This paper introduces a new framework of fast and efficient sensing matrices for practical compressive sensing, called Structurally Random Matrix (SRM). In the proposed framework, we pre-randomize a sensing signal by scrambling its samples or flip** its sample signs and then fast-transform the randomized samples and finally, subsample the transform coefficients as the final sensing measurements.…
▽ More
This paper introduces a new framework of fast and efficient sensing matrices for practical compressive sensing, called Structurally Random Matrix (SRM). In the proposed framework, we pre-randomize a sensing signal by scrambling its samples or flip** its sample signs and then fast-transform the randomized samples and finally, subsample the transform coefficients as the final sensing measurements. SRM is highly relevant for large-scale, real-time compressive sensing applications as it has fast computation and supports block-based processing. In addition, we can show that SRM has theoretical sensing performance comparable with that of completely random sensing matrices. Numerical simulation results verify the validity of the theory as well as illustrate the promising potentials of the proposed sensing framework.
△ Less
Submitted 24 June, 2011;
originally announced June 2011.
-
Exact recoverability from dense corrupted observations via $L_1$ minimization
Authors:
Nam H. Nguyen,
Trac. D. Tran
Abstract:
This paper confirms a surprising phenomenon first observed by Wright \textit{et al.} \cite{WYGSM_Face_2009_J} \cite{WM_denseError_2010_J} under different setting: given $m$ highly corrupted measurements $y = A_{Ω\bullet} x^{\star} + e^{\star}$, where $A_{Ω\bullet}$ is a submatrix whose rows are selected uniformly at random from rows of an orthogonal matrix $A$ and $e^{\star}$ is an unknown sparse…
▽ More
This paper confirms a surprising phenomenon first observed by Wright \textit{et al.} \cite{WYGSM_Face_2009_J} \cite{WM_denseError_2010_J} under different setting: given $m$ highly corrupted measurements $y = A_{Ω\bullet} x^{\star} + e^{\star}$, where $A_{Ω\bullet}$ is a submatrix whose rows are selected uniformly at random from rows of an orthogonal matrix $A$ and $e^{\star}$ is an unknown sparse error vector whose nonzero entries may be unbounded, we show that with high probability $\ell_1$-minimization can recover the sparse signal of interest $x^{\star}$ exactly from only $m = C μ^2 k (\log n)^2$ where $k$ is the number of nonzero components of $x^{\star}$ and $μ= n \max_{ij} A_{ij}^2$, even if nearly 100% of the measurements are corrupted. We further guarantee that stable recovery is possible when measurements are polluted by both gross sparse and small dense errors: $y = A_{Ω\bullet} x^{\star} + e^{\star}+ ν$ where $ν$ is the small dense noise with bounded energy. Numerous simulation results under various settings are also presented to verify the validity of the theory as well as to illustrate the promising potential of the proposed framework.
△ Less
Submitted 23 November, 2011; v1 submitted 6 February, 2011;
originally announced February 2011.
-
Nanolasers grown on silicon
Authors:
Roger Chen,
Thai-Truong D. Tran,
Kar Wei Ng,
Wai Son Ko,
Linus C. Chuang,
Forrest G. Sedgwick,
Connie Chang-Hasnain
Abstract:
Integration of optical interconnects with silicon-based electronics can address the growing limitations facing chip-scale data transport as microprocessors become progressively faster. However, material lattice mismatch and incompatible growth temperatures have fundamentally limited monolithic integration of lasers onto silicon substrates until now. Here, we use a novel growth scheme to overcome t…
▽ More
Integration of optical interconnects with silicon-based electronics can address the growing limitations facing chip-scale data transport as microprocessors become progressively faster. However, material lattice mismatch and incompatible growth temperatures have fundamentally limited monolithic integration of lasers onto silicon substrates until now. Here, we use a novel growth scheme to overcome this roadblock and directly grow on-chip InGaAs nanopillar lasers, demonstrating the potency of bottom-up nano-optoelectronic integration. Unique helically-propagating cavity modes are employed to strongly confine light within subwavelength nanopillars despite low refractive index contrast between InGaAs and silicon. These modes thereby provide an avenue for engineering on-chip nanophotonic devices such as lasers. Nanopillar lasers are as-grown on silicon, offer tiny footprints and scalability, and are thereby particularly suited to high-density optoelectronics. They may ultimately form the basis of the missing monolithic light sources needed to bridge the existing gap between photonic and electronic circuits.
△ Less
Submitted 17 January, 2011;
originally announced January 2011.
-
Tensor sparsification via a bound on the spectral norm of random tensors
Authors:
Nam H. Nguyen,
Petros Drineas,
Trac D. Tran
Abstract:
Given an order-$d$ tensor $\tensor A \in \R^{n \times n \times...\times n}$, we present a simple, element-wise sparsification algorithm that zeroes out all sufficiently small elements of $\tensor A$, keeps all sufficiently large elements of $\tensor A$, and retains some of the remaining elements with probabilities proportional to the square of their magnitudes. We analyze the approximation accurac…
▽ More
Given an order-$d$ tensor $\tensor A \in \R^{n \times n \times...\times n}$, we present a simple, element-wise sparsification algorithm that zeroes out all sufficiently small elements of $\tensor A$, keeps all sufficiently large elements of $\tensor A$, and retains some of the remaining elements with probabilities proportional to the square of their magnitudes. We analyze the approximation accuracy of the proposed algorithm using a powerful inequality that we derive. This inequality bounds the spectral norm of a random tensor and is of independent interest. As a result, we obtain novel bounds for the tensor sparsification problem.
△ Less
Submitted 3 February, 2015; v1 submitted 26 May, 2010;
originally announced May 2010.
