-
Utilisation de la notion de copule en tomographie
Authors:
Doriano-Boris Pougaza,
Ali Mohammad-Djafari,
Jean-Francois Bercher
Abstract:
Un problème important en statistique est la détermination d'une loi de probabilité jointe à partir de ses lois marginales. Dans le cas bidimensionnel, les lois de probabilité marginales f1 (x) et f2(y) sont reliées à la loi jointe f(x,y) par les intégrales suivant les lignes horizontale et verticale (les deux axes x et y). Ainsi, le problème de la détermination de f(x,y) connaissant f1 (x) et f2(y…
▽ More
Un problème important en statistique est la détermination d'une loi de probabilité jointe à partir de ses lois marginales. Dans le cas bidimensionnel, les lois de probabilité marginales f1 (x) et f2(y) sont reliées à la loi jointe f(x,y) par les intégrales suivant les lignes horizontale et verticale (les deux axes x et y). Ainsi, le problème de la détermination de f(x,y) connaissant f1 (x) et f2(y) est un problème inverse mal posé. En statistique la notion de copule est introduite pour obtenir une solution à ce problème. Un problème similaire en tomographie à rayon X est la reconstruction d'une image f(x,y) représentant la répartition de la densité d'une quantité à l'intérieur de l'objet à partir de ses deux projections horizontale et verticale, f1 (x) et f2(y). Il existe aussi un grand nombre de méthodes pour de tels problèmes fondées sur la transformée de Radon. Dans cet article, nous montrons les liens entre la notion de copule et celle de la tomographie à rayon X et voyons si on peut utiliser les méthodes d'un domaine à l'autre.
△ Less
Submitted 14 August, 2010; v1 submitted 18 August, 2009;
originally announced August 2009.
-
Joint Image Restoration and Segmentation using Gauss-Markov-Potts Prior Models and Variational Bayesian Computation: Technical Details
Authors:
Hacheme Ayasso,
Ali Mohammad-Djafari
Abstract:
We propose a method to restore and to segment simultaneously images degraded by a known point spread function (PSF) and additive white noise. For this purpose, we propose a joint Bayesian estimation framework, where a family of non-homogeneous Gauss-Markov fields with Potts region labels models are chosen to serve as priors for images. Since neither the joint maximum a posteriori estimator nor p…
▽ More
We propose a method to restore and to segment simultaneously images degraded by a known point spread function (PSF) and additive white noise. For this purpose, we propose a joint Bayesian estimation framework, where a family of non-homogeneous Gauss-Markov fields with Potts region labels models are chosen to serve as priors for images. Since neither the joint maximum a posteriori estimator nor posterior mean one are tractable, the joint posterior law of the image, its segmentation and all the hyper-parameters, is approximated by a separable probability laws using the Variational Bayes technique. This yields a known probability laws of the posterior with mutually dependent sha** parameter, which aims to enhance the convergence speed of the estimator compared to stochastic sampling based estimator. The main work is description is given in [1], while technical details of the variational calculations are presented in the current paper.
△ Less
Submitted 13 August, 2009;
originally announced August 2009.
-
Using the Notion of Copula in Tomography
Authors:
Doriano-Boris Pougaza,
A. Mohammad-Djafari,
Jean-François Bercher
Abstract:
In 1917 Johann Radon introduced the Radon transform which is used in 1963 by A. M. Cormack for application in the context of tomographic image reconstruction. He proposed to reconstruct the spatial variation of the material density of the body from X-Ray images (radiographies) for different directions. Independently G. N. Hounsfield derived an algorithm and built the first medical CT scanner. Ba…
▽ More
In 1917 Johann Radon introduced the Radon transform which is used in 1963 by A. M. Cormack for application in the context of tomographic image reconstruction. He proposed to reconstruct the spatial variation of the material density of the body from X-Ray images (radiographies) for different directions. Independently G. N. Hounsfield derived an algorithm and built the first medical CT scanner. Basically the idea of the X-ray CT is to get an image of the interior structure of an object by X-raying the object from many different directions. The mathematical problem is then estimating a multivariate function from its line integrals.
Four year before Cormack's idea, Abe Sklar introduced a theory in the context of Statistics called copula. Shortly copulas are functions that link multivariate distributions to theirs univariate marginal functions. It appeared that copulas captivated all dependence structure concerning the marginal functions and offer a wide range of parametric family model which could be used as a model for the joint distribution function. This statistical problem is the same as in Tomography, because a marginal density is obtained from a line integral of its joint distribution. In the particular case of only given horizontal and vertical projections corresponding to a given two marginal functions, we link the theory of copula to tomography via the Radon transform and Sklar's theorem. The result we propose seems to be new as mathematical approach to solve this tomographic inverse problem.
△ Less
Submitted 6 December, 2008;
originally announced December 2008.
-
Bayesian segmentation of hyperspectral images
Authors:
Adel Mohammadpour,
Olivier Féron,
Ali Mohammad-Djafari
Abstract:
In this paper we consider the problem of joint segmentation of hyperspectral images in the Bayesian framework. The proposed approach is based on a Hidden Markov Modeling (HMM) of the images with common segmentation, or equivalently with common hidden classification label variables which is modeled by a Potts Markov Random Field. We introduce an appropriate Markov Chain Monte Carlo (MCMC) algorit…
▽ More
In this paper we consider the problem of joint segmentation of hyperspectral images in the Bayesian framework. The proposed approach is based on a Hidden Markov Modeling (HMM) of the images with common segmentation, or equivalently with common hidden classification label variables which is modeled by a Potts Markov Random Field. We introduce an appropriate Markov Chain Monte Carlo (MCMC) algorithm to implement the method and show some simulation results.
△ Less
Submitted 22 August, 2007;
originally announced August 2007.
-
On the estimation of a parameter with incomplete knowledge on a nuisance parameter
Authors:
Ali Mohammad-Djafari,
Adel Mohammadpour
Abstract:
In this paper we consider the problem of estimating a parameter of a probability distribution when we have some prior information on a nuisance parameter. We start by the very simple case where we know perfectly the value of the nuisance parameter. The complete likelihood is the classical tool in this case. Then, progressively, we consider the case where we are given a prior probability distribu…
▽ More
In this paper we consider the problem of estimating a parameter of a probability distribution when we have some prior information on a nuisance parameter. We start by the very simple case where we know perfectly the value of the nuisance parameter. The complete likelihood is the classical tool in this case. Then, progressively, we consider the case where we are given a prior probability distribution on this nuisance parameter. The marginal likelihood is then the classical tool in this case. Then, we consider the case where we only have a fixed number of its moments. Here, we may use the maximum entropy (ME) principle to assign a prior law and thus go back to the previous case. Finally, we consider the case where we know only its median. In our knowledge, there is not any classical tool for this case. We propose then a new tool for this case based on a recently proposed alternative distribution to the marginal probability distribution. This new criterion is obtained by first remarking that the marginal distribution can be considered as the mean value of the original distribution over the prior probability law of the nuisance parameter, and then, by using the median in place of the mean. In this paper, we first summarize the classical tools used for the three first cases, then we give the precise definition of this new criterion and its properties and, finally, present a few examples to show the differences of these cases.
Key Words: Nuisance parameter, Bayesian inference, Maximum Entropy, Marginalization, Incomplete knowledge, Mean and Median of the Likelihood over the prior distribution
△ Less
Submitted 22 August, 2007;
originally announced August 2007.
-
Approche variationnelle pour le calcul bayésien dans les problèmes inverses en imagerie
Authors:
Ali Mohammad-Djafari
Abstract:
In a non supervised Bayesian estimation approach for inverse problems in imaging systems, one tries to estimate jointly the unknown image pixels $\fb$ and the hyperparameters $\thetab$. This is, in general, done through the joint posterior law $p(\fb,\thetab|\gb)$. The expression of this joint law is often very complex and its exploration through sampling and computation of the point estimators…
▽ More
In a non supervised Bayesian estimation approach for inverse problems in imaging systems, one tries to estimate jointly the unknown image pixels $\fb$ and the hyperparameters $\thetab$. This is, in general, done through the joint posterior law $p(\fb,\thetab|\gb)$. The expression of this joint law is often very complex and its exploration through sampling and computation of the point estimators such as MAP and posterior means need either optimization of non convex criteria or intégration of non Gaussian and multi variate probability laws. In any of these cases, we need to do approximations. We had explored before the possibilities of Laplace approximation and sampling by MCMC. In this paper, we explore the possibility of approximating this joint law by a separable one in $\fb$ and in $\thetab$. This gives the possibility of develo** iterative algorithms with more reasonable computational cost, in particular, if the approximating laws are choosed in the exponential conjugate families. The main objective of this paper is to give details of different algorithms we obtain with different choices of these families.
△ Less
Submitted 13 June, 2007;
originally announced June 2007.
-
Inverse problems in imaging systems and the general Bayesian inversion frawework
Authors:
Ali Mohammad-Djafari
Abstract:
In this paper, first a great number of inverse problems which arise in instrumentation, in computer imaging systems and in computer vision are presented. Then a common general forward modeling for them is given and the corresponding inversion problem is presented. Then, after showing the inadequacy of the classical analytical and least square methods for these ill posed inverse problems, a Bayes…
▽ More
In this paper, first a great number of inverse problems which arise in instrumentation, in computer imaging systems and in computer vision are presented. Then a common general forward modeling for them is given and the corresponding inversion problem is presented. Then, after showing the inadequacy of the classical analytical and least square methods for these ill posed inverse problems, a Bayesian estimation framework is presented which can handle, in a coherent way, all these problems. One of the main steps, in Bayesian inversion framework is the prior modeling of the unknowns. For this reason, a great number of such models and in particular the compound hidden Markov models are presented. Then, the main computational tools of the Bayesian estimation are briefly presented. Finally, some particular cases are studied in detail and new results are presented.
△ Less
Submitted 18 May, 2007;
originally announced May 2007.
-
Computed tomography image reconstruction from only two projections
Authors:
Ali Mohammad-Djafari
Abstract:
English: This paper concerns the image reconstruction from a few projections in Computed Tomography (CT). The main objective of this paper is to show that the problem is so ill posed that no classical method, such as analytical methods based on inverse Radon transform, nor the algebraic methods such as Least squares (LS) or regularization theory can give satisfactory result. As an example, we co…
▽ More
English: This paper concerns the image reconstruction from a few projections in Computed Tomography (CT). The main objective of this paper is to show that the problem is so ill posed that no classical method, such as analytical methods based on inverse Radon transform, nor the algebraic methods such as Least squares (LS) or regularization theory can give satisfactory result. As an example, we consider in detail the case of image reconstruction from two horizontal and vertical projections. We then show how a particular composite Markov modeling and the Bayesian estimation framework can possibly propose satisfactory solutions to the problem. For demonstration and educational purpose a set of Matlab programs are given for a live presentation of the results.
-----
French: Ce travail, à but pédagogique, présente le problème inverse de la reconstruction d'image en tomographie X lorsque le nombre des projections est très limité. voir le texte en Anglais et en Français.
△ Less
Submitted 18 May, 2007;
originally announced May 2007.
-
Bayesian Separation of Document Images with Hidden Markov Model
Authors:
Feng Su,
Ali Mohammad-Djafari
Abstract:
this paper we consider the problem of separating noisy instantaneous linear mixtures of document images in the Bayesian framework. The source image is modeled hierarchically by a latent labeling process representing the common classifications of document objects among different color channels and the intensity process of pixels given the class labels. A Potts Markov random field is used to model…
▽ More
this paper we consider the problem of separating noisy instantaneous linear mixtures of document images in the Bayesian framework. The source image is modeled hierarchically by a latent labeling process representing the common classifications of document objects among different color channels and the intensity process of pixels given the class labels. A Potts Markov random field is used to model regional regularity of the classification labels inside object regions. Local dependency between neighboring pixels can also be accounted by smoothness constraint on their intensities. Within the Bayesian approach, all unknowns including the source, the classification, the mixing coefficients and the distribution parameters of these variables are estimated from their posterior laws. The corresponding Bayesian computations are done by MCMC sampling algorithm. Results from experiments on synthetic and real image mixtures are presented to illustrate the performance of the proposed method.
△ Less
Submitted 16 May, 2007;
originally announced May 2007.
-
Hierarchical Markovian models for hyperspectral image segmentation
Authors:
Ali Mohammad-Djafari,
Adel Mohammadpoor,
Nadia Bali
Abstract:
Hyperspectral images can be represented either as a set of images or as a set of spectra. Spectral classification and segmentation and data reduction are the main problems in hyperspectral image analysis. In this paper we propose a Bayesian estimation approach with an appropriate hiearchical model with hidden markovian variables which gives the possibility to jointly do data reduction, spectral…
▽ More
Hyperspectral images can be represented either as a set of images or as a set of spectra. Spectral classification and segmentation and data reduction are the main problems in hyperspectral image analysis. In this paper we propose a Bayesian estimation approach with an appropriate hiearchical model with hidden markovian variables which gives the possibility to jointly do data reduction, spectral classification and image segmentation. In the proposed model, the desired independent components are piecewise homogeneous images which share the same common hidden segmentation variable. Thus, the joint Bayesian estimation of this hidden variable as well as the sources and the mixing matrix of the source separation problem gives a solution for all the three problems of dimensionality reduction, spectra classification and segmentation of hyperspectral images. A few simulation results illustrate the performances of the proposed method compared to other classical methods usually used in hyperspectral image processing.
△ Less
Submitted 16 May, 2007;
originally announced May 2007.
-
Non Gaussianity and Non Stationarity modeled through Hidden Variables and their use in ICA and Blind Source Separation
Authors:
Ali Mohammad-Djafari
Abstract:
Modeling non Gaussian and non stationary signals and images has always been one of the most important part of signal and image processing methods. In this paper, first we propose a few new models, all based on using hidden variables for modeling either stationary but non Gaussian or Gaussian but non stationary or non Gaussian and non stationary signals and images. Then, we will see how to use th…
▽ More
Modeling non Gaussian and non stationary signals and images has always been one of the most important part of signal and image processing methods. In this paper, first we propose a few new models, all based on using hidden variables for modeling either stationary but non Gaussian or Gaussian but non stationary or non Gaussian and non stationary signals and images. Then, we will see how to use these models in independent component analysis (ICA) or blind source separation (BSS). The computational aspects of the Bayesian estimation framework associated with these prior models are also discussed.
△ Less
Submitted 16 May, 2007;
originally announced May 2007.
-
Dirichlet or Potts ?
Authors:
Ali Mohammad-Djafari
Abstract:
When modeling the distribution of a set of data by a mixture of Gaussians, there are two possibilities: i) the classical one is using a set of parameters which are the proportions, the means and the variances; ii) the second is to consider the proportions as the probabilities of a discrete valued hidden variable. In the first case a usual prior distribution for the proportions is the Dirichlet w…
▽ More
When modeling the distribution of a set of data by a mixture of Gaussians, there are two possibilities: i) the classical one is using a set of parameters which are the proportions, the means and the variances; ii) the second is to consider the proportions as the probabilities of a discrete valued hidden variable. In the first case a usual prior distribution for the proportions is the Dirichlet which accounts for the fact that they have to sum up to one. In the second case, to each data is associated a hidden variable for which we consider two possibilities: a) assuming those variables to be i.i.d. We show then that this scheme is equivalent to the classical mixture model with Dirichlet prior; b) assuming a Markovian structure. Then we choose the simplest markovian model which is the Potts distribution. As we will see this model is more appropriate for the case where the data represents the pixels of an image for which the hidden variables represent a segmentation of that image. The main object of this paper is to give some details on these models and different algorithms used for their simulation and the estimation of their parameters.
Key Words: Mixture of Gaussians, Dirichlet, Potts, Classification, Segmentation.
△ Less
Submitted 16 May, 2007;
originally announced May 2007.
-
Information and Covariance Matrices for Multivariate Burr III and Logistic distributions
Authors:
Gholamhossein Yari,
Ali Mohammad-Djafari
Abstract:
Main result of this paper is to derive the exact analytical expressions of information and covariance matrices for multivariate Burr III and logistic distributions. These distributions arise as tractable parametric models in price and income distributions, reliability, economics, populations growth and survival data. We showed that all the calculations can be obtained from one main moment multi…
▽ More
Main result of this paper is to derive the exact analytical expressions of information and covariance matrices for multivariate Burr III and logistic distributions. These distributions arise as tractable parametric models in price and income distributions, reliability, economics, populations growth and survival data. We showed that all the calculations can be obtained from one main moment multi dimensional integral whose expression is obtained through some particular change of variables. Indeed, we consider that this calculus technique for improper integral has its own importance in applied probability calculus.
△ Less
Submitted 13 April, 2004;
originally announced April 2004.
-
Entropy, Information Matrix and order statistics of Multivariate Pareto, Burr and related distributions
Authors:
Gholamhossein Yari,
Ali Mohammad-Djafari
Abstract:
In this paper we derive the exact analytical expressions for the information and covariance matrices of the multivariate Burr and related distributions. These distributions arise as tractable parametric models in reliability, actuarial science, economics, finance and telecommunications. We show that all the calculations can be obtained from one main moment multi dimensional integral whose expres…
▽ More
In this paper we derive the exact analytical expressions for the information and covariance matrices of the multivariate Burr and related distributions. These distributions arise as tractable parametric models in reliability, actuarial science, economics, finance and telecommunications. We show that all the calculations can be obtained from one main moment multi dimensional integral whose expression is obtained through some particular change of variables.
△ Less
Submitted 13 April, 2004;
originally announced April 2004.
-
A hidden Markov Model for image fusion and their joint segmentation in medical image computing
Authors:
Olivier Feron,
Ali Mohammad-Djafari
Abstract:
In this work we propose a Bayesian framework for fully automated image fusion and their joint segmentation. More specifically, we consider the case where we have observed images of the same object through different image processes or through different spectral bands. The objective of this work is then to propose a coherent approach to combine these data sets and obtain a segmented image which ca…
▽ More
In this work we propose a Bayesian framework for fully automated image fusion and their joint segmentation. More specifically, we consider the case where we have observed images of the same object through different image processes or through different spectral bands. The objective of this work is then to propose a coherent approach to combine these data sets and obtain a segmented image which can be considered as the fusion result of these observations. The proposed approach is based on a Hidden Markov Modeling (HMM) of the images with common segmentation, or equivalently, with common hidden classification label variables which are modeled by the Potts Markov Random Field. We propose an appropriate Markov Chain Monte Carlo (MCMC) algorithm to implement the method and show some simulation results and applications.
△ Less
Submitted 31 March, 2004;
originally announced March 2004.
-
A Hidden Markov model for Bayesian data fusion of multivariate signals
Authors:
Olivier Feron,
Ali Mohammad-Djafari
Abstract:
In this work we propose a Bayesian framework for data fusion of multivariate signals which arises in imaging systems. More specifically, we consider the case where we have observed two images of the same object through two different imaging processes. The objective of this work is then to propose a coherent approach to combine these data sets to obtain a segmented image which can be considered a…
▽ More
In this work we propose a Bayesian framework for data fusion of multivariate signals which arises in imaging systems. More specifically, we consider the case where we have observed two images of the same object through two different imaging processes. The objective of this work is then to propose a coherent approach to combine these data sets to obtain a segmented image which can be considered as the fusion result of these two images. The proposed approach is based on a Hidden Markov Modeling (HMM) of the images with common segmentation, or equivalently, with common hidden classification label variables which is modeled by the Potts Markov Random Field. We propose then an appropriate Markov Chain Monte Carlo (MCMC) algorithm to implement the method and show some simulation results and applications.
△ Less
Submitted 31 March, 2004;
originally announced March 2004.
-
A Bayesian approach to change point analysis of discrete time series
Authors:
Ali Mohammad-Djafari,
Olivier Feron
Abstract:
In this work we consider time series with a finite number of discrete point changes. We assume that the data in each segment follows a different probability density functions (pdf). We focus on the case where the data in all segments are modeled by Gaussian probability density functions with different means, variances and correlation lengths. We put a prior law on the change point instances (Poi…
▽ More
In this work we consider time series with a finite number of discrete point changes. We assume that the data in each segment follows a different probability density functions (pdf). We focus on the case where the data in all segments are modeled by Gaussian probability density functions with different means, variances and correlation lengths. We put a prior law on the change point instances (Poisson process) as well as on these different parameters(conjugate priors) and give the expression of the posterior probality distributions of these change points. The computations are done by using an appropriate Markov Chain Monte Carlo (MCMC) technique.
The problem as we stated can also be considered as an unsupervised classification and/or segmentation of the time serie. This analogy gives us the possibility to propose alternative modeling and computation of change points, which are more appropriate for multivariate signals, for example in image processing.
△ Less
Submitted 31 March, 2004;
originally announced March 2004.
-
Bayesian Wavelet Based Signal and Image Separation
Authors:
Mahieddine M. Ichir,
Ali Mohammad-Djafari
Abstract:
In this contribution, we consider the problem of blind source separation in a Bayesian estimation framework. The wavelet representation allows us to assign an adequate prior distribution to the wavelet coefficients of the sources. MCMC algorithms are implemented to test the validity of the proposed approach, and the non linear approximation of the wavelet transform is exploited to aleviate the a…
▽ More
In this contribution, we consider the problem of blind source separation in a Bayesian estimation framework. The wavelet representation allows us to assign an adequate prior distribution to the wavelet coefficients of the sources. MCMC algorithms are implemented to test the validity of the proposed approach, and the non linear approximation of the wavelet transform is exploited to aleviate the algorithm.
△ Less
Submitted 3 December, 2003; v1 submitted 7 November, 2003;
originally announced November 2003.
-
Wavelet Domain Image Separation
Authors:
Ali Mohammad-Djafari,
Mahieddine Ichir
Abstract:
In this paper, we consider the problem of blind signal and image separation using a sparse representation of the images in the wavelet domain. We consider the problem in a Bayesian estimation framework using the fact that the distribution of the wavelet coefficients of real world images can naturally be modeled by an exponential power probability density function. The Bayesian approach which has…
▽ More
In this paper, we consider the problem of blind signal and image separation using a sparse representation of the images in the wavelet domain. We consider the problem in a Bayesian estimation framework using the fact that the distribution of the wavelet coefficients of real world images can naturally be modeled by an exponential power probability density function. The Bayesian approach which has been used with success in blind source separation gives also the possibility of including any prior information we may have on the mixing matrix elements as well as on the hyperparameters (parameters of the prior laws of the noise and the sources). We consider two cases: first the case where the wavelet coefficients are assumed to be i.i.d. and second the case where we model the correlation between the coefficients of two adjacent scales by a first order Markov chain. This paper only reports on the first case, the second case results will be reported in a near future. The estimation computations are done via a Monte Carlo Markov Chain (MCMC) procedure. Some simulations show the performances of the proposed method. Keywords: Blind source separation, wavelets, Bayesian estimation, MCMC Hasting-Metropolis algorithm.
△ Less
Submitted 14 November, 2002; v1 submitted 12 November, 2002;
originally announced November 2002.
-
MCMC joint separation and segmentation of hidden Markov fields
Authors:
Hichem Snoussi,
Ali Mohammad-Djafari
Abstract:
In this contribution, we consider the problem of the blind separation of noisy instantaneously mixed images. The images are modelized by hidden Markov fields with unknown parameters. Given the observed images, we give a Bayesian formulation and we propose to solve the resulting data augmentation problem by implementing a Monte Carlo Markov Chain (MCMC) procedure. We separate the unknown variable…
▽ More
In this contribution, we consider the problem of the blind separation of noisy instantaneously mixed images. The images are modelized by hidden Markov fields with unknown parameters. Given the observed images, we give a Bayesian formulation and we propose to solve the resulting data augmentation problem by implementing a Monte Carlo Markov Chain (MCMC) procedure. We separate the unknown variables into two categories:
1. The parameters of interest which are the mixing matrix, the noise covariance and the parameters of the sources distributions. 2. The hidden variables which are the unobserved sources and the unobserved pixels classification labels.
The proposed algorithm provides in the stationary regime samples drawn from the posterior distributions of all the variables involved in the problem leading to a flexibility in the cost function choice.
We discuss and characterize some problems of non identifiability and degeneracies of the parameters likelihood and the behavior of the MCMC algorithm in this case.
Finally, we show the results for both synthetic and real data to illustrate the feasibility of the proposed solution. keywords: MCMC, blind source separation, hidden Markov fields, segmentation, Bayesian approach
△ Less
Submitted 12 November, 2002;
originally announced November 2002.
-
Yet Another Analysis of Dice Problems
Authors:
Ali Mohammad-Djafari
Abstract:
During the MaxEnt 2002 workshop in Moscow, Idaho, Tony Vignaux asked again a few simple questions about using Maximum Entropy or Bayesian approaches for the famous Dice problems which have been analyzed many times through this workshop and also in other places. Here, there is another analysis of these problems. I hope that, this paper will answer a few questions of Tony and other participants of…
▽ More
During the MaxEnt 2002 workshop in Moscow, Idaho, Tony Vignaux asked again a few simple questions about using Maximum Entropy or Bayesian approaches for the famous Dice problems which have been analyzed many times through this workshop and also in other places. Here, there is another analysis of these problems. I hope that, this paper will answer a few questions of Tony and other participants of the workshop on the situations where we can use Maximum Entropy or Bayesian approaches or even the cases where we can actually use both of them.
Keywords: Dice problems and probability theory, Maximum Likelihood, Bayesian inference, Maximum A Posteriori, Entropy, Maximum entropy, Maximum entropy in the mean.
△ Less
Submitted 12 November, 2002;
originally announced November 2002.
-
A Matlab Program to Calculate the Maximum Entropy Distributions
Authors:
A. Mohammad-Djafari
Abstract:
The classical Maximum Entropy (ME) problem consists of determining a probability distribution function (pdf) from a finite set of expectations of known functions. The solution depends on $N+1$ Lagrange multipliers which are determined by solving the set of nonlinear equations formed by the $N$ data constraints and the normalization constraint. In this short communication we give three Matlab pro…
▽ More
The classical Maximum Entropy (ME) problem consists of determining a probability distribution function (pdf) from a finite set of expectations of known functions. The solution depends on $N+1$ Lagrange multipliers which are determined by solving the set of nonlinear equations formed by the $N$ data constraints and the normalization constraint. In this short communication we give three Matlab programs to calculate these Lagrange multipliers. The first considers the general case where the functions can be any functions. The second considers the special case of power functions $x^n$. In this case the data are the geometrical moments of $p(x)$. The third considers the special case of Fourier series functions $\exp(-j n ωx)$. In this case the data are the trigonometrical moments of $p(x)$. Some examples are also given to illustrate the usefullness of these programs.
△ Less
Submitted 14 November, 2001;
originally announced November 2001.
-
A scale invariant Bayesian method to solve linear inverse problems
Authors:
A. Mohammad-Djafari,
Jérôme Idier
Abstract:
In this paper we propose a new Bayesian estimation method to solve linear inverse problems in signal and image restoration and reconstruction problems which has the property to be scale invariant. In general, Bayesian estimators are {\em nonlinear} functions of the observed data. The only exception is the Gaussian case. When dealing with linear inverse problems the linearity is sometimes a too s…
▽ More
In this paper we propose a new Bayesian estimation method to solve linear inverse problems in signal and image restoration and reconstruction problems which has the property to be scale invariant. In general, Bayesian estimators are {\em nonlinear} functions of the observed data. The only exception is the Gaussian case. When dealing with linear inverse problems the linearity is sometimes a too strong property, while {\em scale invariance} often remains a desirable property. As everybody knows one of the main difficulties with using the Bayesian approach in real applications is the assignment of the direct (prior) probability laws before applying the Bayes' rule. We discuss here how to choose prior laws to obtain scale invariant Bayesian estimators. In this paper we discuss and propose a familly of generalized exponential probability distributions functions for the direct probabilities (the prior $p(\xb)$ and the likelihood $p(\yb|\xb)$), for which the posterior $p(\xb|\yb)$, and, consequently, the main posterior estimators are scale invariant. Among many properties, generalized exponential can be considered as the maximum entropy probability distributions subject to the knowledge of a finite set of expectation values of some knwon functions.
△ Less
Submitted 14 November, 2001;
originally announced November 2001.
-
Scale Invariant Markov Models for Bayesian Inversion of Linear Inverse Problems
Authors:
Stéphane Brette,
Ali Mohammad-Djafari,
Jérôme Idier
Abstract:
In a Bayesian approach for solving linear inverse problems one needs to specify the prior laws for calculation of the posterior law. A cost function can also be defined in order to have a common tool for various Bayesian estimators which depend on the data and the hyperparameters. The Gaussian case excepted, these estimators are not linear and so depend on the scale of the measurements. In this…
▽ More
In a Bayesian approach for solving linear inverse problems one needs to specify the prior laws for calculation of the posterior law. A cost function can also be defined in order to have a common tool for various Bayesian estimators which depend on the data and the hyperparameters. The Gaussian case excepted, these estimators are not linear and so depend on the scale of the measurements. In this paper a weaker property than linearity is imposed on the Bayesian estimator, namely the scale invariance property (SIP).First, we state some results on linear estimation and then we introduce and justify a scale invariance axiom. We show that arbitrary choice of scale measurement can be avoided if the estimator has this SIP. Some examples of classical regularization procedures are shown to be scale invariant. Then we investigate general conditions on classes of Bayesian estimators which satisfy this SIP, as well as their consequences on the cost function and prior laws. We also show that classical methods for hyperparameters estimation (i.e., Maximum Likelihood and Generalized Maximum Likelihood) can be introduced for hyperparameters estimation, and we verify the SIP property for them. Finally we discuss how to choose the prior laws to obtain scale invariant Bayesian estimators. For this, we consider two cases of prior laws: {\em entropic prior laws} and {\em first-order Markov models}. In related preceding works [Mohammad-Djafari90,Mohammad-Djafari93], the SIP constraints have been studied for the case of entropic prior laws. In this paper extension to the case of first-order Markov models is provided. KEYWORDS: Bayesian estimation, Scale invariance, Markov modelling, Inverse Problems, Image reconstruction, Prior model selection
△ Less
Submitted 14 November, 2001;
originally announced November 2001.
-
A full Bayesian approach for inverse problems
Authors:
A. Mohammad-Djafari
Abstract:
The main object of this paper is to present some general concepts of Bayesian inference and more specifically the estimation of the hyperparameters in inverse problems. We consider a general linear situation where we are given some data $\yb$ related to the unknown parameters $\xb$ by $\yb=\Ab \xb+\nb$ and where we can assign the probability laws $p(\xb|\thetab)$, $p(\yb|\xb,\betab)$,…
▽ More
The main object of this paper is to present some general concepts of Bayesian inference and more specifically the estimation of the hyperparameters in inverse problems. We consider a general linear situation where we are given some data $\yb$ related to the unknown parameters $\xb$ by $\yb=\Ab \xb+\nb$ and where we can assign the probability laws $p(\xb|\thetab)$, $p(\yb|\xb,\betab)$, $p(\betab)$ and $p(\thetab)$. The main discussion is then how to infer $\xb$, $\thetab$ and $\betab$ either individually or any combinations of them. Different situations are considered and discussed. As an important example, we consider the case where $θ$ and $β$ are the precision parameters of the Gaussian laws to whom we assign Gamma priors and we propose some new and practical algorithms to estimate them simultaneously. Comparisons and links with other classical methods such as maximum likelihood are presented. Keywords: Bayesian inference, Hyperparameter estimation, Inverse problems, Maximum likelihood.
△ Less
Submitted 14 November, 2001;
originally announced November 2001.
-
A Comparison of Two Approaches: Maximum Entropy on the Mean (MEM) and Bayesian Estimation (BAYES) for Inverse Problems
Authors:
A. Mohammad-Djafari
Abstract:
To handle with inverse problems, two probabilistic approaches have been proposed: the maximum entropy on the mean (MEM) and the Bayesian estimation (BAYES). The main object of this presentation is to compare these two approaches which are in fact two different inference procedures to define the solution of an inverse problem as the optimizer of a compound criterion. Keywords: Inverse problems, M…
▽ More
To handle with inverse problems, two probabilistic approaches have been proposed: the maximum entropy on the mean (MEM) and the Bayesian estimation (BAYES). The main object of this presentation is to compare these two approaches which are in fact two different inference procedures to define the solution of an inverse problem as the optimizer of a compound criterion. Keywords: Inverse problems, Maximum Entropy on the Mean, Bayesian inference, Convex analysis.
△ Less
Submitted 14 November, 2001;
originally announced November 2001.
-
New Advances in Bayesian Calculation for Linear and Nonlinear Inverse Problems
Authors:
A. Mohammad-Djafari
Abstract:
The Bayesian approach has proved to be a coherent approach to handle ill posed Inverse problems. However, the Bayesian calculations need either an optimization or an integral calculation. The maximum a posteriori (MAP) estimation requires the minimization of a compound criterion which, in general, has two parts: a data fitting part and a prior part. In many situations the criterion to be minimiz…
▽ More
The Bayesian approach has proved to be a coherent approach to handle ill posed Inverse problems. However, the Bayesian calculations need either an optimization or an integral calculation. The maximum a posteriori (MAP) estimation requires the minimization of a compound criterion which, in general, has two parts: a data fitting part and a prior part. In many situations the criterion to be minimized becomes multimodal. The cost of the Simulated Annealing (SA) based techniques is in general huge for inverse problems. Recently a deterministic optimization technique, based on Graduated Non Convexity (GNC), have been proposed to overcome this difficulty. The objective of this paper is to show two specific implementations of this technique for the following situations: -- Linear inverse problems where the solution is modeled as a piecewise continuous function. The non convexity of the criterion is then due to the special choice of the prior; -- A nonlinear inverse problem which arises in inverse scattering where the non convexity of the criterion is due to the likelihood part. Keywords: Inverse problems, Regularization, Bayesian calculation, Global optimization, Graduated Non Convexity.
△ Less
Submitted 14 November, 2001;
originally announced November 2001.
-
A Bayesian Approach to Shape Reconstruction of a Compact Object from a Few Number of Projections
Authors:
A. Mohammad-Djafari
Abstract:
Image reconstruction in X ray tomography consists in determining an object from its projections. In many applications such as non destructive testing, we look for an image who has a constant value inside a region (default) and another constant value outside that region (homogeneous region surrounding the default). The image reconstruction problem becomes then the determination of the shape of th…
▽ More
Image reconstruction in X ray tomography consists in determining an object from its projections. In many applications such as non destructive testing, we look for an image who has a constant value inside a region (default) and another constant value outside that region (homogeneous region surrounding the default). The image reconstruction problem becomes then the determination of the shape of that region. In this work we model the object (the default region) as a polygonal disc and propose a new method for the estimation of the coordinates of its vertices directly from a very limited number of its projections. Keywords: Computed Imaging, Tomography, Shape reconstruction, Non destructive testing, Regularization, Bayesian estimation, Deformable contours.
△ Less
Submitted 14 November, 2001;
originally announced November 2001.
-
A Bayesian Approach for the Determination of the Charge Density from Elastic Electron Scattering Data
Authors:
A. Mohammad-Djafari,
H. G. Miller
Abstract:
The problem of the determination of the charge density from limited information about the charge form factor is an ill-posed inverse problem. A Bayesian probabilistic approach to this problem which permits to take into account both errors and prior information about the solution is presented. We will show that many classical methods can be considered as special cases of the proposed approach. We…
▽ More
The problem of the determination of the charge density from limited information about the charge form factor is an ill-posed inverse problem. A Bayesian probabilistic approach to this problem which permits to take into account both errors and prior information about the solution is presented. We will show that many classical methods can be considered as special cases of the proposed approach. We address also the problem of the basis function choice for the discretization and the uncertainty of the solution. Some numerical results for an analytical model are presented to show the performance of the proposed method.
△ Less
Submitted 14 November, 2001;
originally announced November 2001.
-
Probabilistic methods for data fusion
Authors:
A. Mohammad-Djafari
Abstract:
The main object of this paper is to show how we can use classical probabilistic methods such as Maximum Entropy (ME), maximum likelihood (ML) and/or Bayesian (BAYES) approaches to do microscopic and macroscopic data fusion. Actually ME can be used to assign a probability law to an unknown quantity when we have macroscopic data (expectations) on it. ML can be used to estimate the parameters of a…
▽ More
The main object of this paper is to show how we can use classical probabilistic methods such as Maximum Entropy (ME), maximum likelihood (ML) and/or Bayesian (BAYES) approaches to do microscopic and macroscopic data fusion. Actually ME can be used to assign a probability law to an unknown quantity when we have macroscopic data (expectations) on it. ML can be used to estimate the parameters of a probability law when we have microscopic data (direct observation). BAYES can be used to update a prior probability law when we have microscopic data through the likelihood. When we have both microscopic and macroscopic data we can use first ME to assign a prior and then use BAYES to update it to the posterior law thus doing the desired data fusion. However, in practical data fusion applications, we may still need some engineering feeling to propose realistic data fusion solutions. Some simple examples in sensor data fusion and image reconstruction using different kind of data are presented to illustrate these ideas. Keywords: Data fusion, Maximum entropy, Maximum likelihood, Bayesian data fusion, EM algorithm.
△ Less
Submitted 14 November, 2001;
originally announced November 2001.
-
Shape reconstruction in X-ray tomography from a small number of projections using deformable models
Authors:
A. Mohammad-Djafari,
Ken Sauer
Abstract:
X-ray tomographic image reconstruction consists of determining an object function from its projections. In many applications such as non-destructive testing, we look for a fault region (air) in a homogeneous, known background (metal). The image reconstruction problem then becomes the determination of the shape of the default region. Two approaches can be used: modeling the image as a binary Mark…
▽ More
X-ray tomographic image reconstruction consists of determining an object function from its projections. In many applications such as non-destructive testing, we look for a fault region (air) in a homogeneous, known background (metal). The image reconstruction problem then becomes the determination of the shape of the default region. Two approaches can be used: modeling the image as a binary Markov random field and estimating the pixels of the image, or modeling the shape of the fault and estimating it directly from the projections. In this work we model the fault shape by a deformable polygonal disc or a deformable polyhedral volume and propose a new method for directly estimating the coordinates of its vertices from a very limited number of its projections. The basic idea is not new, but in other competing methods, in general, the fault shape is modeled by a small number of parameters (polygonal shapes with very small number of vertices, snakes and deformable templates) and these parameters are estimated either by least squares or by maximum likelihood methods. We propose modeling the shape of the fault region by a polygon with a large number of vertices, allowing modeling of nearly any shape and estimation of its vertices' coordinates directly from the projections by defining the solution as the minimizer of an appropriate regularized criterion. This formulation can also be interpreted as a maximum a posteriori (MAP) estimate in a Bayesian estimation framework. To optimize this criterion we use either a simulated annealing or a special purpose deterministic algorithm based on iterated conditional modes (ICM). The simulated results are very encouraging, especially when the number and the angles of projections are very limited.
△ Less
Submitted 14 November, 2001;
originally announced November 2001.
-
Entropy in Signal Processing (Entropie en Traitement du Signal)
Authors:
Ali Mohammad-Djafari
Abstract:
Résumé: Le principal objet de cette communication est de faire une rétro perspective succincte de l'utilisation de l'entropie et du principe du maximum d'entropie dans le domaine du traitement du signal. Après un bref rappel de quelques définitions et du principe du maximum d'entropie, nous verrons successivement comment l'entropie est utilisée en séparation de sources, en modélisation de signau…
▽ More
Résumé: Le principal objet de cette communication est de faire une rétro perspective succincte de l'utilisation de l'entropie et du principe du maximum d'entropie dans le domaine du traitement du signal. Après un bref rappel de quelques définitions et du principe du maximum d'entropie, nous verrons successivement comment l'entropie est utilisée en séparation de sources, en modélisation de signaux, en analyse spectrale et pour la résolution des problèmes inverses linéaires. Mots clés : Entropie, Entropie croisée, Distance de Kullback, Information mutuelle, Estimation spectrale, Problèmes inverses
Abstract: The main object of this work is to give a brief overview of the different ways the entropy has been used in signal and image processing. After a short introduction of different quantities related to the entropy and the maximum entropy principle, we will study their use in different fields of signal processing such as: source separation, model order selection, spectral estimation and, finally, general linear inverse problems. Keywords : Entropy, Relative entropy, Kullback distance, Mutual information, Spectral estimation, Inverse problems.
△ Less
Submitted 6 November, 2001;
originally announced November 2001.
-
Model selection for inverse problems: Best choice of basis functions and model order selection
Authors:
A. Mohammad-Djafari
Abstract:
A complete solution for an inverse problem needs five main steps: choice of basis functions for discretization, determination of the order of the model, estimation of the hyperparameters, estimation of the solution, and finally, characterization of the proposed solution. Many works have been done for the three last steps. The first two have been neglected for a while, in part due to the complexi…
▽ More
A complete solution for an inverse problem needs five main steps: choice of basis functions for discretization, determination of the order of the model, estimation of the hyperparameters, estimation of the solution, and finally, characterization of the proposed solution. Many works have been done for the three last steps. The first two have been neglected for a while, in part due to the complexity of the problem. However, in many inverse problems, particularly when the number of data is very low, a good choice of the basis functions and a good selection of the order become primary. In this paper, we first propose a complete solution within a Bayesian framework. Then, we apply the proposed method to an inverse elastic electron scattering problem.
△ Less
Submitted 6 November, 2001;
originally announced November 2001.
-
Bayesian source separation with mixture of Gaussians prior for sources and Gaussian prior for mixture coefficients
Authors:
Hichem Snoussi,
Ali Mohammad-Djafari
Abstract:
In this contribution, we present new algorithms to source separation for the case of noisy instantaneous linear mixture, within the Bayesian statistical framework. The source distribution prior is modeled by a mixture of Gaussians [Moulines97] and the mixing matrix elements distributions by a Gaussian [Djafari99a]. We model the mixture of Gaussians hierarchically by mean of hidden variables repr…
▽ More
In this contribution, we present new algorithms to source separation for the case of noisy instantaneous linear mixture, within the Bayesian statistical framework. The source distribution prior is modeled by a mixture of Gaussians [Moulines97] and the mixing matrix elements distributions by a Gaussian [Djafari99a]. We model the mixture of Gaussians hierarchically by mean of hidden variables representing the labels of the mixture. Then, we consider the joint a posteriori distribution of sources, mixing matrix elements, labels of the mixture and other parameters of the mixture with appropriate prior probability laws to eliminate degeneracy of the likelihood function of variance parameters and we propose two iterative algorithms to estimate jointly sources, mixing matrix and hyperparameters: Joint MAP (Maximum a posteriori) algorithm and penalized EM algorithm. The illustrative example is taken in [Macchi99] to compare with other algorithms proposed in literature. Keywords: Source separation, Gaussian mixture, classification, JMAP algorithm, Penalized EM algorithm.
△ Less
Submitted 6 November, 2001;
originally announced November 2001.
-
Penalized maximum likelihood for multivariate Gaussian mixture
Authors:
Hichem Snoussi,
Ali Mohammad-Djafari
Abstract:
In this paper, we first consider the parameter estimation of a multivariate random process distribution using multivariate Gaussian mixture law. The labels of the mixture are allowed to have a general probability law which gives the possibility to modelize a temporal structure of the process under study. We generalize the case of univariate Gaussian mixture in [Ridolfi99] to show that the likeli…
▽ More
In this paper, we first consider the parameter estimation of a multivariate random process distribution using multivariate Gaussian mixture law. The labels of the mixture are allowed to have a general probability law which gives the possibility to modelize a temporal structure of the process under study. We generalize the case of univariate Gaussian mixture in [Ridolfi99] to show that the likelihood is unbounded and goes to infinity when one of the covariance matrices approaches the boundary of singularity of the non negative definite matrices set. We characterize the parameter set of these singularities. As a solution to this degeneracy problem, we show that the penalization of the likelihood by an Inverse Wishart prior on covariance matrices results to a penalized or maximum a posteriori criterion which is bounded. Then, the existence of positive definite matrices optimizing this criterion can be guaranteed. We also show that with a modified EM procedure or with a Bayesian sampling scheme, we can constrain covariance matrices to belong to a particular subclass of covariance matrices. Finally, we study degeneracies in the source separation problem where the characterization of parameter singularity set is more complex. We show, however, that Inverse Wishart prior on covariance matrices eliminates the degeneracies in this case too.
△ Less
Submitted 2 November, 2001;
originally announced November 2001.
-
Bayesian inference for inverse problems
Authors:
Ali Mohammad-Djafari
Abstract:
Traditionally, the MaxEnt workshops start by a tutorial day. This paper summarizes my talk during 2001'th workshop at John Hopkins University. The main idea in this talk is to show how the Bayesian inference can naturally give us all the necessary tools we need to solve real inverse problems: starting by simple inversion where we assume to know exactly the forward model and all the input model p…
▽ More
Traditionally, the MaxEnt workshops start by a tutorial day. This paper summarizes my talk during 2001'th workshop at John Hopkins University. The main idea in this talk is to show how the Bayesian inference can naturally give us all the necessary tools we need to solve real inverse problems: starting by simple inversion where we assume to know exactly the forward model and all the input model parameters up to more realistic advanced problems of myopic or blind inversion where we may be uncertain about the forward model and we may have noisy data. Starting by an introduction to inverse problems through a few examples and explaining their ill posedness nature, I briefly presented the main classical deterministic methods such as data matching and classical regularization methods to show their limitations. I then presented the main classical probabilistic methods based on likelihood, information theory and maximum entropy and the Bayesian inference framework for such problems. I show that the Bayesian framework, not only generalizes all these methods, but also gives us natural tools, for example, for inferring the uncertainty of the computed solutions, for the estimation of the hyperparameters or for handling myopic or blind inversion problems. Finally, through a deconvolution problem example, I presented a few state of the art methods based on Bayesian inference particularly designed for some of the mass spectrometry data processing problems.
△ Less
Submitted 31 October, 2001;
originally announced October 2001.
