-
Understanding the Effects of Projectors in Knowledge Distillation
Authors:
Yudong Chen,
Sen Wang,
Jiajun Liu,
Xuwei Xu,
Frank de Hoog,
Brano Kusy,
Zi Huang
Abstract:
Conventionally, during the knowledge distillation process (e.g. feature distillation), an additional projector is often required to perform feature transformation due to the dimension mismatch between the teacher and the student networks. Interestingly, we discovered that even if the student and the teacher have the same feature dimensions, adding a projector still helps to improve the distillatio…
▽ More
Conventionally, during the knowledge distillation process (e.g. feature distillation), an additional projector is often required to perform feature transformation due to the dimension mismatch between the teacher and the student networks. Interestingly, we discovered that even if the student and the teacher have the same feature dimensions, adding a projector still helps to improve the distillation performance. In addition, projectors even improve logit distillation if we add them to the architecture too. Inspired by these surprising findings and the general lack of understanding of the projectors in the knowledge distillation process from existing literature, this paper investigates the implicit role that projectors play but so far have been overlooked. Our empirical study shows that the student with a projector (1) obtains a better trade-off between the training accuracy and the testing accuracy compared to the student without a projector when it has the same feature dimensions as the teacher, (2) better preserves its similarity to the teacher beyond shallow and numeric resemblance, from the view of Centered Kernel Alignment (CKA), and (3) avoids being over-confident as the teacher does at the testing phase. Motivated by the positive effects of projectors, we propose a projector ensemble-based feature distillation method to further improve distillation performance. Despite the simplicity of the proposed strategy, empirical results from the evaluation of classification tasks on benchmark datasets demonstrate the superior classification performance of our method on a broad range of teacher-student pairs and verify from the aspects of CKA and model calibration that the student's features are of improved quality with the projector ensemble design.
△ Less
Submitted 26 October, 2023;
originally announced October 2023.
-
Improved Feature Distillation via Projector Ensemble
Authors:
Yudong Chen,
Sen Wang,
Jiajun Liu,
Xuwei Xu,
Frank de Hoog,
Zi Huang
Abstract:
In knowledge distillation, previous feature distillation methods mainly focus on the design of loss functions and the selection of the distilled layers, while the effect of the feature projector between the student and the teacher remains under-explored. In this paper, we first discuss a plausible mechanism of the projector with empirical evidence and then propose a new feature distillation method…
▽ More
In knowledge distillation, previous feature distillation methods mainly focus on the design of loss functions and the selection of the distilled layers, while the effect of the feature projector between the student and the teacher remains under-explored. In this paper, we first discuss a plausible mechanism of the projector with empirical evidence and then propose a new feature distillation method based on a projector ensemble for further performance improvement. We observe that the student network benefits from a projector even if the feature dimensions of the student and the teacher are the same. Training a student backbone without a projector can be considered as a multi-task learning process, namely achieving discriminative feature extraction for classification and feature matching between the student and the teacher for distillation at the same time. We hypothesize and empirically verify that without a projector, the student network tends to overfit the teacher's feature distributions despite having different architecture and weights initialization. This leads to degradation on the quality of the student's deep features that are eventually used in classification. Adding a projector, on the other hand, disentangles the two learning tasks and helps the student network to focus better on the main feature extraction task while still being able to utilize teacher features as a guidance through the projector. Motivated by the positive effect of the projector in feature distillation, we propose an ensemble of projectors to further improve the quality of student features. Experimental results on different datasets with a series of teacher-student pairs illustrate the effectiveness of the proposed method.
△ Less
Submitted 28 February, 2023; v1 submitted 27 October, 2022;
originally announced October 2022.
-
A Note on Error Bounds for Pseudo Skeleton Approximations of Matrices
Authors:
Frank de Hoog,
Markus Hegland
Abstract:
Due to their importance in both data analysis and numerical algorithms, low rank approximations have recently been widely studied. They enable the handling of very large matrices. Tight error bounds for the computationally efficient Gaussian elimination based methods (skeleton approximations) are available. In practice, these bounds are useful for matrices with singular values which decrease quick…
▽ More
Due to their importance in both data analysis and numerical algorithms, low rank approximations have recently been widely studied. They enable the handling of very large matrices. Tight error bounds for the computationally efficient Gaussian elimination based methods (skeleton approximations) are available. In practice, these bounds are useful for matrices with singular values which decrease quickly. Using the Chebyshev norm, this paper provides improved bounds for the errors of the matrix elements. These bounds are substantially better in the practically relevant cases where the eigenvalues decrease polynomially. Results are proven for general real rectangular matrices. Even stronger bounds are obtained for symmetric positive definite matrices. A simple example is given, comparing these new bounds to earlier ones.
△ Less
Submitted 14 August, 2022; v1 submitted 5 July, 2021;
originally announced July 2021.
-
Vaccination strategies on dynamic networks with indirect transmission links and limited contact information
Authors:
Md Shahzamal,
Raja Jurdak,
Bernard Mans,
Frank de Hoog,
Dean Paini
Abstract:
Infectious diseases are still a major global burden for modern society causing 13 million deaths annually. One way to reduce the morbidity and mortality rates from infectious diseases is through preventative or targeted vaccinations. Current vaccination strategies, however, rely on the highly specific individual contact information that is difficult and costly to obtain, in order to identify influ…
▽ More
Infectious diseases are still a major global burden for modern society causing 13 million deaths annually. One way to reduce the morbidity and mortality rates from infectious diseases is through preventative or targeted vaccinations. Current vaccination strategies, however, rely on the highly specific individual contact information that is difficult and costly to obtain, in order to identify influential spreading individuals. Current approaches also focus only on direct contacts between individuals for spreading, and disregard indirect transmission where a pathogen can spread between one infected individual and one susceptible individual that visit the same location within a short time-frame without meeting. This paper presents a novel vaccination strategy that relies on coarse-grained contact information, both direct and indirect, that can be easily and efficiently collected. Rather than tracking exact contact degrees of individuals, our strategy uses the types of places people visit to estimate a range of contact degrees for individuals, considering both direct and indirect contacts. We conduct extensive simulations to evaluate the performance of our strategy in comparison to the state of the art's vaccination strategies. Results show that our strategy achieves comparable performance to the oracle approach and outperforms all existing strategies when considering indirect links.
△ Less
Submitted 12 April, 2020;
originally announced April 2020.
-
Generating dynamic contact graphs with indirect links
Authors:
Md Shahzamal,
Raja Jurdak,
Bernard Mans,
Frank De Hoog,
Dean Paini
Abstract:
Graph models are widely used to study diffusion processes in contact networks. Recent data-driven research has highlighted the significance of indirect links, where interactions are possible when two nodes visit the same place at different times (SPDT), in determining network structure and diffusion dynamics. However, how to generate dynamic graphs with indirect links for modeling diffusion remain…
▽ More
Graph models are widely used to study diffusion processes in contact networks. Recent data-driven research has highlighted the significance of indirect links, where interactions are possible when two nodes visit the same place at different times (SPDT), in determining network structure and diffusion dynamics. However, how to generate dynamic graphs with indirect links for modeling diffusion remains an unsolved challenge. Here, we present a dynamic contact graph model for generating contact networks with direct and indirect links. Our model introduces the concept of multiple concurrently active copies of a node for capturing indirect transmission links. The SPDT graph model builds on activity driven time-varying network modelling for generating dynamic contact networks using simple statistical distributions. This model is fitted with a large city-scale empirical dataset using maximum likelihood estimation methods. Finally, the performance of the model is evaluated by analysing the capability of capturing the network properties observed in empirical graphs constructed using the location updates of a social networking app and simulating SPDT diffusion processes. Our results show that, in comparison to current graph models that only include direct links, our graph model with indirect links match empirical network properties and diffusion dynamics much more closely.
△ Less
Submitted 9 November, 2019;
originally announced November 2019.
-
Indirect interactions influence contact network structure and diffusion dynamics
Authors:
Md Shahzamal,
Raja Jurdak,
Bernard Mans,
Frank de Hoog
Abstract:
Interaction patterns at the individual level influence the behaviour of diffusion over contact networks. Most of the current diffusion models only consider direct interactions among individuals to build underlying infectious items transmission networks. However, delayed indirect interactions, where a susceptible individual interacts with infectious items after the infected individual has left the…
▽ More
Interaction patterns at the individual level influence the behaviour of diffusion over contact networks. Most of the current diffusion models only consider direct interactions among individuals to build underlying infectious items transmission networks. However, delayed indirect interactions, where a susceptible individual interacts with infectious items after the infected individual has left the interaction space, can also cause transmission events. We define a diffusion model called the same place different time transmission (SPDT) based diffusion that considers transmission links for these indirect interactions. Our SPDT model changes the network dynamics where the connectivity among individuals varies with the decay rates of link infectivity. We investigate SPDT diffusion behaviours by simulating airborne disease spreading on data-driven contact networks. The SPDT model significantly increases diffusion dynamics (particularly for networks with low link densities where indirect interactions create new infection pathways) and is capable of producing realistic disease reproduction number. Our results show that the SPDT model is significantly more likely to lead to outbreaks compared to current diffusion models with direct interactions. We find that the diffusion dynamics with including indirect links are not reproducible by the current models, highlighting the importance of the indirect links for predicting outbreaks.
△ Less
Submitted 6 June, 2019;
originally announced June 2019.
-
A Graph Model with Indirect Co-location Links
Authors:
Md Shahzamal,
Raja Jurdak,
Bernard Mans,
Frank de Hoog
Abstract:
Graph models are widely used to analyse diffusion processes embedded in social contacts and to develop applications. A range of graph models are available to replicate the underlying social structures and dynamics realistically. However, most of the current graph models can only consider concurrent interactions among individuals in the co-located interaction networks. However, they do not account…
▽ More
Graph models are widely used to analyse diffusion processes embedded in social contacts and to develop applications. A range of graph models are available to replicate the underlying social structures and dynamics realistically. However, most of the current graph models can only consider concurrent interactions among individuals in the co-located interaction networks. However, they do not account for indirect interactions that can transmit spreading items to individuals who visit the same locations at different times but within a certain time limit. The diffusion phenomena occurring through direct and indirect interactions is called same place different time (SPDT) diffusion. This paper introduces a model to synthesize co-located interaction graphs capturing both direct interactions, where individuals meet at a location, and indirect interactions, where individuals visit the same location at different times within a set timeframe. We analyze 60 million location updates made by 2 million users from a social networking application to characterize the graph properties, including the space-time correlations and its time evolving characteristics, such as bursty or ongoing behaviors. The generated synthetic graph reproduces diffusion dynamics of a realistic contact graph, and reduces the prediction error by up to 82% when compare to other contact graph models demonstrating its potential for forecasting epidemic spread.
△ Less
Submitted 26 July, 2018; v1 submitted 8 June, 2018;
originally announced June 2018.
-
Impact of Indirect Contacts in Emerging Infectious Disease on Social Networks
Authors:
Md Shahzamal,
Raja Jurdak,
Bernard Mans,
Ahmad El Shoghri,
Frank De Hoog
Abstract:
Interaction patterns among individuals play vital roles in spreading infectious diseases. Understanding these patterns and integrating their impact in modeling diffusion dynamics of infectious diseases are important for epidemiological studies. Current network-based diffusion models assume that diseases transmit through interactions where both infected and susceptible individuals are co-located at…
▽ More
Interaction patterns among individuals play vital roles in spreading infectious diseases. Understanding these patterns and integrating their impact in modeling diffusion dynamics of infectious diseases are important for epidemiological studies. Current network-based diffusion models assume that diseases transmit through interactions where both infected and susceptible individuals are co-located at the same time. However, there are several infectious diseases that can transmit when a susceptible individual visits a location after an infected individual has left. Recently, we introduced a diffusion model called same place different time (SPDT) transmission to capture the indirect transmissions that happen when an infected individual leaves before a susceptible individual's arrival along with direct transmissions. In this paper, we demonstrate how these indirect transmission links significantly enhance the emergence of infectious diseases simulating airborne disease spreading on a synthetic social contact network. We denote individuals having indirect links but no direct links during their infectious periods as hidden spreaders. Our simulation shows that indirect links play similar roles of direct links and a single hidden spreader can cause large outbreak in the SPDT model which causes no infection in the current model based on direct link. Our work opens new direction in modeling infectious diseases.
△ Less
Submitted 30 March, 2018; v1 submitted 21 March, 2018;
originally announced March 2018.
-
On the van Cittert - Zernike theorem for intensity correlations and its applications
Authors:
Timur E. Gureyev,
Alexander Kozlov,
David M. Paganin,
Yakov I. Nesterets,
Frank De Hoog,
Harry M. Quiney
Abstract:
A reciprocal relationship between the autocovariance of the light intensity in the source plane and in the far-field detector plane is presented in a form analogous to the classical van Cittert - Zernike theorem, but involving intensity correlation functions. A "classical" version of the reciprocity relationship is considered first, based on the assumption of circular Gaussian statistics of the co…
▽ More
A reciprocal relationship between the autocovariance of the light intensity in the source plane and in the far-field detector plane is presented in a form analogous to the classical van Cittert - Zernike theorem, but involving intensity correlation functions. A "classical" version of the reciprocity relationship is considered first, based on the assumption of circular Gaussian statistics of the complex amplitudes in the source plane. The result is consistent with the theory of Hanbury Brown - Twiss interferometry, but it is shown to be also applicable to estimation of the source size or the spatial resolution of the detector from the noise power spectrum of flat-field images. An alternative version of the van Cittert - Zernike theorem for intensity correlations is then derived for a quantized electromagnetic beam in a coherent state, which leads to Poisson statistics for the intrinsic intensity of the beam.
△ Less
Submitted 8 June, 2017;
originally announced June 2017.
-
Compressive hyperspectral imaging via adaptive sampling and dictionary learning
Authors:
Mingrui Yang,
Frank de Hoog,
Yuqi Fan,
Wen Hu
Abstract:
In this paper, we propose a new sampling strategy for hyperspectral signals that is based on dictionary learning and singular value decomposition (SVD). Specifically, we first learn a sparsifying dictionary from training spectral data using dictionary learning. We then perform an SVD on the dictionary and use the first few left singular vectors as the rows of the measurement matrix to obtain the c…
▽ More
In this paper, we propose a new sampling strategy for hyperspectral signals that is based on dictionary learning and singular value decomposition (SVD). Specifically, we first learn a sparsifying dictionary from training spectral data using dictionary learning. We then perform an SVD on the dictionary and use the first few left singular vectors as the rows of the measurement matrix to obtain the compressive measurements for reconstruction. The proposed method provides significant improvement over the conventional compressive sensing approaches. The reconstruction performance is further improved by reconditioning the sensing matrix using matrix balancing. We also demonstrate that the combination of dictionary learning and SVD is robust by applying them to different datasets.
△ Less
Submitted 2 December, 2015;
originally announced December 2015.
-
Hyperspectral Image Recovery via Hybrid Regularization
Authors:
Reza Arablouei,
Frank de Hoog
Abstract:
Natural images tend to mostly consist of smooth regions with individual pixels having highly correlated spectra. This information can be exploited to recover hyperspectral images of natural scenes from their incomplete and noisy measurements. To perform the recovery while taking full advantage of the prior knowledge, we formulate a composite cost function containing a square-error data-fitting ter…
▽ More
Natural images tend to mostly consist of smooth regions with individual pixels having highly correlated spectra. This information can be exploited to recover hyperspectral images of natural scenes from their incomplete and noisy measurements. To perform the recovery while taking full advantage of the prior knowledge, we formulate a composite cost function containing a square-error data-fitting term and two distinct regularization terms pertaining to spatial and spectral domains. The regularization for the spatial domain is the sum of total-variation of the image frames corresponding to all spectral bands. The regularization for the spectral domain is the l1-norm of the coefficient matrix obtained by applying a suitable sparsifying transform to the spectra of the pixels. We use an accelerated proximal-subgradient method to minimize the formulated cost function. We analyze the performance of the proposed algorithm and prove its convergence. Numerical simulations using real hyperspectral images exhibit that the proposed algorithm offers an excellent recovery performance with a number of measurements that is only a small fraction of the hyperspectral image data size. Simulation results also show that the proposed algorithm significantly outperforms an accelerated proximal-gradient algorithm that solves the classical basis-pursuit denoising problem to recover the hyperspectral image.
△ Less
Submitted 25 August, 2016; v1 submitted 9 November, 2015;
originally announced November 2015.
-
On spatial resolution, signal-to-noise and information capacity of linear imaging systems
Authors:
Timur Gureyev,
Yakov Nesterets,
Frank de Hoog
Abstract:
A simple model for image formation in linear shift-invariant systems is considered, in which both the detected signal and the noise variance are varying slowly compared to the point-spread function of the system. It is shown that within the constraints of this model, the square of the signal-to-noise ratio is always proportional to the "volume" of the spatial resolution unit. In the case of Poisso…
▽ More
A simple model for image formation in linear shift-invariant systems is considered, in which both the detected signal and the noise variance are varying slowly compared to the point-spread function of the system. It is shown that within the constraints of this model, the square of the signal-to-noise ratio is always proportional to the "volume" of the spatial resolution unit. In the case of Poisson statistics, the ratio of these two quantities divided by the incident density of the imaging particles (e.g. photons) represents a dimensionless invariant of the imaging system, which was previously termed the intrinsic imaging quality. The relationship of this invariant to the notion of information capacity of communication and imaging systems, which was previously considered by Shannon, Gabor and others, is investigated. The results are then applied to a simple generic model of quantitative imaging of weakly scattering objects, leading to an estimate of the upper limit for the amount of information about the sample that can be obtained in such experiments. It is shown that this limit depends only on the total number of imaging particles incident on the sample, the average scattering coefficient, the size of the sample and the number of spatial resolution units.
△ Less
Submitted 8 February, 2016; v1 submitted 1 November, 2015;
originally announced November 2015.
-
Evaluating the Performance of BSBL Methodology for EEG Source Localization On a Realistic Head Model
Authors:
Sajib Saha,
Rajib Rana,
Ya. I. Nesterets,
M. Tahtali,
Frank de Hoog,
T. E. Gureyev
Abstract:
Source localization in EEG represents a high dimensional inverse problem, which is severely ill-posed by nature. Fortunately, sparsity constraints have come into rescue as it helps solving the ill-posed problems when the signal is sparse. When the signal has a structure such as block structure, consideration of block sparsity produces better results. Knowing sparse Bayesian learning is an importan…
▽ More
Source localization in EEG represents a high dimensional inverse problem, which is severely ill-posed by nature. Fortunately, sparsity constraints have come into rescue as it helps solving the ill-posed problems when the signal is sparse. When the signal has a structure such as block structure, consideration of block sparsity produces better results. Knowing sparse Bayesian learning is an important member in the family of sparse recovery, and a superior choice when the projection matrix is highly coherent (which is typical the case for EEG), in this work we evaluate the performance of block sparse Bayesian learning (BSBL) method for EEG source localization. It is already accepted by the EEG community that a group of dipoles rather than a single dipole are activated during brain activities; thus, block structure is a reasonable choice for EEG. In this work we use two definitions of blocks: Brodmann areas and automated anatomical labelling (AAL), and analyze the reconstruction performance of BSBL methodology for them. A realistic head model is used for the experiment, which was obtained from segmentation of MRI images. When the number of simultaneously active blocks is 2, the BSBL produces overall localization accuracy of less than 5 mm without the presence of noise. The presence of more than 3 simultaneously active blocks and noise significantly affect the localization performance. Consideration of AAL based blocks results more accurate source localization in comparison to Brodmann area based blocks.
△ Less
Submitted 27 April, 2015;
originally announced April 2015.
-
On the noise-resolution duality, Heisenberg uncertainty and Shannon's information
Authors:
T. E. Gureyev,
F. de Hoog,
Ya. I. Nesterets,
D. M. Paganin
Abstract:
Several variations of the Heisenberg uncertainty inequality are derived on the basis of "noise-resolution duality" recently proposed by the authors. The same approach leads to a related inequality that provides an upper limit for the information capacity of imaging systems in terms of the number of imaging quanta (particles) used in the experiment. These results can be useful in the context of bio…
▽ More
Several variations of the Heisenberg uncertainty inequality are derived on the basis of "noise-resolution duality" recently proposed by the authors. The same approach leads to a related inequality that provides an upper limit for the information capacity of imaging systems in terms of the number of imaging quanta (particles) used in the experiment. These results can be useful in the context of biomedical imaging constrained by the radiation dose delivered to the sample, or in imaging (e.g. astronomical) problems under "low light" conditions.
△ Less
Submitted 14 March, 2015;
originally announced March 2015.
-
Sparse Bayesian Learning for EEG Source Localization
Authors:
Sajib Saha,
Frank de Hoog,
Ya. I. Nesterets,
Rajib Rana,
M. Tahtali,
T. E. Gureyev
Abstract:
Purpose: Localizing the sources of electrical activity from electroencephalographic (EEG) data has gained considerable attention over the last few years. In this paper, we propose an innovative source localization method for EEG, based on Sparse Bayesian Learning (SBL). Methods: To better specify the sparsity profile and to ensure efficient source localization, the proposed approach considers grou…
▽ More
Purpose: Localizing the sources of electrical activity from electroencephalographic (EEG) data has gained considerable attention over the last few years. In this paper, we propose an innovative source localization method for EEG, based on Sparse Bayesian Learning (SBL). Methods: To better specify the sparsity profile and to ensure efficient source localization, the proposed approach considers grou** of the electrical current dipoles inside human brain. SBL is used to solve the localization problem in addition with imposed constraint that the electric current dipoles associated with the brain activity are isotropic. Results: Numerical experiments are conducted on a realistic head model that is obtained by segmentation of MRI images of the head and includes four major components, namely the scalp, the skull, the cerebrospinal fluid (CSF) and the brain, with appropriate relative conductivity values. The results demonstrate that the isotropy constraint significantly improves the performance of SBL. In a noiseless environment, the proposed method was 1 found to accurately (with accuracy of >75%) locate up to 6 simultaneously active sources, whereas for SBL without the isotropy constraint, the accuracy of finding just 3 simultaneously active sources was <75%. Conclusions: Compared to the state-of-the-art algorithms, the proposed method is potentially more consistent in specifying the sparsity profile of human brain activity and is able to produce better source localization for EEG.
△ Less
Submitted 19 January, 2015;
originally announced January 2015.
-
EEG source localization using a sparsity prior based on Brodmann areas
Authors:
S. Saha,
Ya. I. Nesterets,
Rajib Rana,
M. Tahtali,
Frank de Hoog,
T. E. Gureyev
Abstract:
Localizing the sources of electrical activity in the brain from Electroencephalographic (EEG) data is an important tool for non-invasive study of brain dynamics. Generally, the source localization process involves a high-dimensional inverse problem that has an infinite number of solutions and thus requires additional constraints to be considered to have a unique solution. In the context of EEG sou…
▽ More
Localizing the sources of electrical activity in the brain from Electroencephalographic (EEG) data is an important tool for non-invasive study of brain dynamics. Generally, the source localization process involves a high-dimensional inverse problem that has an infinite number of solutions and thus requires additional constraints to be considered to have a unique solution. In the context of EEG source localization, we propose a novel approach that is based on dividing the cerebral cortex of the brain into a finite number of Functional Zones which correspond to unitary functional areas in the brain. In this paper we investigate the use of Brodmanns areas as the Functional Zones. This approach allows us to apply a sparsity constraint to find a unique solution for the inverse EEG problem. Compared to previously published algorithms which use different sparsity constraints to solve this problem, the proposed method is potentially more consistent with the known sparsity profile of the human brain activity and thus may be able to ensure better localization. Numerical experiments are conducted on a realistic head model obtained from segmentation of MRI images of the head and includes four major compartments namely scalp, skull, cerebrospinal fluid (CSF) and brain with relative conductivity values. Three different electrode setups are tested in the numerical experiments.
△ Less
Submitted 10 June, 2014;
originally announced June 2014.
-
New Coherence and RIP Analysis for Weak Orthogonal Matching Pursuit
Authors:
Mingrui Yang,
Frank de Hoog
Abstract:
In this paper we define a new coherence index, named the global 2-coherence, of a given dictionary and study its relationship with the traditional mutual coherence and the restricted isometry constant. By exploring this relationship, we obtain more general results on sparse signal reconstruction using greedy algorithms in the compressive sensing (CS) framework. In particular, we obtain an improved…
▽ More
In this paper we define a new coherence index, named the global 2-coherence, of a given dictionary and study its relationship with the traditional mutual coherence and the restricted isometry constant. By exploring this relationship, we obtain more general results on sparse signal reconstruction using greedy algorithms in the compressive sensing (CS) framework. In particular, we obtain an improved bound over the best known results on the restricted isometry constant for successful recovery of sparse signals using orthogonal matching pursuit (OMP).
△ Less
Submitted 13 May, 2014;
originally announced May 2014.
-
Orthogonal Matching Pursuit with Thresholding and its Application in Compressive Sensing
Authors:
Mingrui Yang,
Frank de Hoog
Abstract:
Greed is good. However, the tighter you squeeze, the less you have. In this paper, a less greedy algorithm for sparse signal reconstruction in compressive sensing, named orthogonal matching pursuit with thresholding is studied. Using the global 2-coherence , which provides a "bridge" between the well known mutual coherence and the restricted isometry constant, the performance of orthogonal matchin…
▽ More
Greed is good. However, the tighter you squeeze, the less you have. In this paper, a less greedy algorithm for sparse signal reconstruction in compressive sensing, named orthogonal matching pursuit with thresholding is studied. Using the global 2-coherence , which provides a "bridge" between the well known mutual coherence and the restricted isometry constant, the performance of orthogonal matching pursuit with thresholding is analyzed and more general results for sparse signal reconstruction are obtained. It is also shown that given the same assumption on the coherence index and the restricted isometry constant as required for orthogonal matching pursuit, the thresholding variation gives exactly the same reconstruction performance with significantly less complexity.
△ Less
Submitted 1 July, 2015; v1 submitted 8 July, 2013;
originally announced July 2013.
-
The application of compressive sampling to radio astronomy I: Deconvolution
Authors:
Feng Li,
Tim J. Cornwell,
Frank de Hoog
Abstract:
Compressive sampling is a new paradigm for sampling, based on sparseness of signals or signal representations. It is much less restrictive than Nyquist-Shannon sampling theory and thus explains and systematises the widespread experience that methods such as the Högbom CLEAN can violate the Nyquist-Shannon sampling requirements. In this paper, a CS-based deconvolution method for extended sources is…
▽ More
Compressive sampling is a new paradigm for sampling, based on sparseness of signals or signal representations. It is much less restrictive than Nyquist-Shannon sampling theory and thus explains and systematises the widespread experience that methods such as the Högbom CLEAN can violate the Nyquist-Shannon sampling requirements. In this paper, a CS-based deconvolution method for extended sources is introduced. This method can reconstruct both point sources and extended sources (using the isotropic undecimated wavelet transform as a basis function for the reconstruction step). We compare this CS-based deconvolution method with two CLEAN-based deconvolution methods: the Högbom CLEAN and the multiscale CLEAN. This new method shows the best performance in deconvolving extended sources for both uniform and natural weighting of the sampled visibilities. Both visual and numerical results of the comparison are provided.
△ Less
Submitted 9 June, 2011;
originally announced June 2011.
-
The application of compressive sampling to radio astronomy II: Faraday rotation measure synthesis
Authors:
Feng Li,
Shea Brown,
Tim J. Cornwell,
Frank de Hoog
Abstract:
Faraday rotation measure (RM) synthesis is an important tool to study and analyze galactic and extra-galactic magnetic fields. Since there is a Fourier relation between the Faraday dispersion function and the polarized radio emission, full reconstruction of the dispersion function requires knowledge of the polarized radio emission at both positive and negative square wavelengths $λ^2$. However, on…
▽ More
Faraday rotation measure (RM) synthesis is an important tool to study and analyze galactic and extra-galactic magnetic fields. Since there is a Fourier relation between the Faraday dispersion function and the polarized radio emission, full reconstruction of the dispersion function requires knowledge of the polarized radio emission at both positive and negative square wavelengths $λ^2$. However, one can only make observations for $λ^2 > 0$. Furthermore observations are possible only for a limited range of wavelengths. Thus reconstructing the Faraday dispersion function from these limited measurements is ill-conditioned. In this paper, we propose three new reconstruction algorithms for RM synthesis based upon compressive sensing/sampling (CS). These algorithms are designed to be appropriate for Faraday thin sources only, thick sources only, and mixed sources respectively. Both visual and numerical results show that the new RM synthesis methods provide superior reconstructions of both magnitude and phase information than RM-CLEAN
△ Less
Submitted 9 June, 2011;
originally announced June 2011.
-
A weakly stable algorithm for general Toeplitz systems
Authors:
Adam W. Bojanczyk,
Richard P. Brent,
Frank R. de Hoog
Abstract:
We show that a fast algorithm for the QR factorization of a Toeplitz or Hankel matrix A is weakly stable in the sense that R^T.R is close to A^T.A. Thus, when the algorithm is used to solve the semi-normal equations R^T.Rx = A^Tb, we obtain a weakly stable method for the solution of a nonsingular Toeplitz or Hankel linear system Ax = b. The algorithm also applies to the solution of the full-rank T…
▽ More
We show that a fast algorithm for the QR factorization of a Toeplitz or Hankel matrix A is weakly stable in the sense that R^T.R is close to A^T.A. Thus, when the algorithm is used to solve the semi-normal equations R^T.Rx = A^Tb, we obtain a weakly stable method for the solution of a nonsingular Toeplitz or Hankel linear system Ax = b. The algorithm also applies to the solution of the full-rank Toeplitz or Hankel least squares problem.
△ Less
Submitted 4 May, 2010;
originally announced May 2010.
-
On the stability of the Bareiss and related Toeplitz factorization algorithms
Authors:
Adam W. Bojanczyk,
Richard P. Brent,
Frank R. de Hoog,
Douglas R. Sweet
Abstract:
This report contains a numerical stability analysis of factorization algorithms for computing the Cholesky decomposition of symmetric positive definite matrices of displacement rank 2. The algorithms in the class can be expressed as sequences of elementary downdating steps. The stability of the factorization algorithms follows directly from the numerical properties of algorithms for realizing elem…
▽ More
This report contains a numerical stability analysis of factorization algorithms for computing the Cholesky decomposition of symmetric positive definite matrices of displacement rank 2. The algorithms in the class can be expressed as sequences of elementary downdating steps. The stability of the factorization algorithms follows directly from the numerical properties of algorithms for realizing elementary downdating operations. It is shown that the Bareiss algorithm for factorizing a symmetric positive definite Toeplitz matrix is in the class and hence the Bareiss algorithm is stable. Some numerical experiments that compare behavior of the Bareiss algorithm and the Levinson algorithm are presented. These experiments indicate that in general (when the reflection coefficients are not all positive) the Levinson algorithm is not stable; certainly it can give much larger residuals than the Bareiss algorithm.
△ Less
Submitted 30 April, 2010;
originally announced April 2010.
