-
Fisher's Legacy of Directional Statistics, and Beyond to Statistics on Manifolds
Authors:
Kanti V. Mardia
Abstract:
It will not be an exaggeration to say that R A Fisher is the Albert Einstein of Statistics. He pioneered almost all the main branches of statistics, but it is not as well known that he opened the area of Directional Statistics with his 1953 paper introducing a distribution on the sphere which is now known as the Fisher distribution. He stressed that for spherical data one should take into account…
▽ More
It will not be an exaggeration to say that R A Fisher is the Albert Einstein of Statistics. He pioneered almost all the main branches of statistics, but it is not as well known that he opened the area of Directional Statistics with his 1953 paper introducing a distribution on the sphere which is now known as the Fisher distribution. He stressed that for spherical data one should take into account that the data is on a manifold. We will describe this Fisher distribution and reanalyse his geological data. We also comment on the two goals he set himself in that paper, and how he reinvented the von Mises distribution on the circle. Since then, many extensions of this distribution have appeared bearing Fisher's name such as the von Mises Fisher distribution and the matrix Fisher distribution. In fact, the subject of Directional Statistics has grown tremendously in the last two decades with new applications emerging in Life Sciences, Image Analysis, Machine Learning and so on. We give a recent new method of constructing the Fisher type distribution which has been motivated by some problems in Machine Learning. The subject related to his distribution has evolved since then more broadly as Statistics on Manifolds which also includes the new field of Shape Analysis. We end with a historical note pointing out some correspondence between D'Arcy Thompson and R A Fisher related to Shape Analysis.
△ Less
Submitted 28 May, 2024;
originally announced May 2024.
-
Fisher's Pioneering work on Discriminant Analysis and its Impact on AI
Authors:
Kanti V. Mardia
Abstract:
Fisher opened many new areas in Multivariate Analysis, and the one which we will consider is discriminant analysis. Several papers by Fisher and others followed from his seminal paper in 1936 where he coined the name discrimination function. Historically, his four papers on discriminant analysis during 1936-1940 connect to the contemporaneous pioneering work of Hotelling and Mahalanobis. We revisi…
▽ More
Fisher opened many new areas in Multivariate Analysis, and the one which we will consider is discriminant analysis. Several papers by Fisher and others followed from his seminal paper in 1936 where he coined the name discrimination function. Historically, his four papers on discriminant analysis during 1936-1940 connect to the contemporaneous pioneering work of Hotelling and Mahalanobis. We revisit the famous iris data which Fisher used in his 1936 paper and in particular, test the hypothesis of multivariate normality for the data which he assumed. Fisher constructed his genetic discriminant motivated by this application and we provide a deeper insight into this construction; however, this construction has not been well understood as far as we know. We also indicate how the subject has developed along with the computer revolution, noting newer methods to carry out discriminant analysis, such as kernel classifiers, classification trees, support vector machines, neural networks, and deep learning. Overall, with computational power, the whole subject of Multivariate Analysis has changed its emphasis but the impact of this Fisher's pioneering work continues as an integral part of supervised learning in Artificial Intelligence.
△ Less
Submitted 9 September, 2023;
originally announced September 2023.
-
The Classical Multidimensional Scaling Revisited
Authors:
Kanti V. Mardia,
Anthony D. Riley
Abstract:
We reexamine the the classical multidimensional scaling (MDS). We study some special cases, in particular, the exact solution for the sub-space formed by the 3 dimensional principal coordinates is derived. Also we give the extreme case when the points are collinear. Some insight into the effect on the MDS solution of the excluded eigenvalues (could be both positive as well as negative) of the doub…
▽ More
We reexamine the the classical multidimensional scaling (MDS). We study some special cases, in particular, the exact solution for the sub-space formed by the 3 dimensional principal coordinates is derived. Also we give the extreme case when the points are collinear. Some insight into the effect on the MDS solution of the excluded eigenvalues (could be both positive as well as negative) of the doubly centered matrix is provided. As an illustration, we work through an example to understand the distortion in the MDS construction with positive and negative eigenvalues.
△ Less
Submitted 29 December, 2021;
originally announced December 2021.
-
Mixture models for spherical data with applications to protein bioinformatics
Authors:
Kanti V. Mardia,
Stuart Barber,
Philippa M. Burdett,
John T. Kent,
Thomas Hamelryck
Abstract:
Finite mixture models are fitted to spherical data. Kent distributions are used for the components of the mixture because they allow considerable flexibility. Previous work on such mixtures has used an approximate maximum likelihood estimator for the parameters of a single component. However, the approximation causes problems when using the EM algorithm to estimate the parameters in a mixture mode…
▽ More
Finite mixture models are fitted to spherical data. Kent distributions are used for the components of the mixture because they allow considerable flexibility. Previous work on such mixtures has used an approximate maximum likelihood estimator for the parameters of a single component. However, the approximation causes problems when using the EM algorithm to estimate the parameters in a mixture model. Hence the exact maximum likelihood estimator is used here for the individual components. This paper is motivated by a challenging prize problem in structural bioinformatics of how proteins fold. It is known that hydrogen bonds play a key role in the folding of a protein. We explore this hydrogen bond geometry using a data set describing bonds between two amino acids in proteins. An appropriate coordinate system to represent the hydrogen bond geometry is proposed, with each bond represented as a point on a sphere. We fit mixtures of Kent distributions to different subsets of the hydrogen bond data to gain insight into how the secondary structure elements bond together, since the distribution of hydrogen bonds depends on which secondary structure elements are involved.
△ Less
Submitted 27 April, 2021;
originally announced April 2021.
-
Clustering Schemes on the Torus with Application to RNA Clashes
Authors:
Henrik Wiechers,
Benjamin Eltzner,
Stephan F. Huckemann,
Kanti V. Mardia
Abstract:
Molecular structures of RNA molecules reconstructed from X-ray crystallography frequently contain errors. Motivated by this problem we examine clustering on a torus since RNA shapes can be described by dihedral angles. A previously developed clustering method for torus data involves two tuning parameters and we assess clustering results for different parameter values in relation to the problem of…
▽ More
Molecular structures of RNA molecules reconstructed from X-ray crystallography frequently contain errors. Motivated by this problem we examine clustering on a torus since RNA shapes can be described by dihedral angles. A previously developed clustering method for torus data involves two tuning parameters and we assess clustering results for different parameter values in relation to the problem of so-called RNA clashes. This clustering problem is part of the dynamically evolving field of statistics on manifolds. Statistical problems on the torus highlight general challenges for statistics on manifolds. Therefore, the torus PCA and clustering methods we propose make an important contribution to directional statistics and statistics on manifolds in general.
△ Less
Submitted 28 February, 2021;
originally announced April 2021.
-
Families of discrete circular distributions with some novel applications
Authors:
Kanti V. Mardia,
Karthik Sriram
Abstract:
Motivated by some cutting edge circular data such as from Smart Home technologies and roulette spins from online and casino, we construct some new rich classes of discrete distributions on the circle. We give four new general methods of construction, namely (i) maximum entropy, (ii) centered wrap**, (iii) marginalized and (iv) conditionalized methods. We motivate these methods on the line and th…
▽ More
Motivated by some cutting edge circular data such as from Smart Home technologies and roulette spins from online and casino, we construct some new rich classes of discrete distributions on the circle. We give four new general methods of construction, namely (i) maximum entropy, (ii) centered wrap**, (iii) marginalized and (iv) conditionalized methods. We motivate these methods on the line and then work on the circular case and provide some properties to gain insight into these constructions. We mainly focus on the last two methods (iii) and (iv) in the context of circular location families, as they are amenable to general methodology. We show that the marginalized and conditionalized discrete circular location families inherit important properties from their parent continuous families. In particular, for the von Mises and wrapped Cauchy as the parent distribution, we examine their properties including the maximum likelihood estimators, the hypothesis test for uniformity and give a test of serial independence. Using our discrete circular distributions, we demonstrate how to determine changepoint when the data arise in a sequence and how to fit mixtures of this distribution. Illustrative examples are given which triggered the work. For example, for roulette data, we test for uniformity (unbiasedness) , test for serial correlation, detect changepoint in streaming roulette-spins data, and fit mixtures. We analyse a smart home data using our mixtures. We examine the effect of ignoring discreteness of the underlying population, and discuss marginalized versus conditionalized approaches. We give various extensions of the families with skewness and kurtosis, to those supported on an irregular lattice, and discuss potential extension to general manifolds by showing a construction on the torus
△ Less
Submitted 29 April, 2022; v1 submitted 11 September, 2020;
originally announced September 2020.
-
Helix modelling through the Mardia-Holmes model framework and an extension of the Mardia-Holmes model
Authors:
Mai F Alfahad,
John T Kent,
Kanti V Mardia
Abstract:
For noisy two-dimensional data, which are approximately uniformly distributed near the circumference of an ellipse, Mardia and Holmes (1980) developed a model to fit the ellipse. In this paper we adapt their methodology to the analysis of helix data in three dimensions. If the helix axis is known, then the Mardia-Holmes model for the circular case can be fitted after projecting the helix data onto…
▽ More
For noisy two-dimensional data, which are approximately uniformly distributed near the circumference of an ellipse, Mardia and Holmes (1980) developed a model to fit the ellipse. In this paper we adapt their methodology to the analysis of helix data in three dimensions. If the helix axis is known, then the Mardia-Holmes model for the circular case can be fitted after projecting the helix data onto the plane normal to the helix axis. If the axis is unknown, an iterative algorithm has been developed to estimate the axis. The methodology is illustrated using simulated protein alpha-helices. We also give a multivariate version of the Mardia-Holmes model which will be applicable for fitting an ellipsoid and in particular a cylinder.
△ Less
Submitted 21 October, 2018;
originally announced October 2018.
-
Toroidal diffusions and protein structure evolution
Authors:
Eduardo García-Portugués,
Michael Golden,
Michael Sørensen,
Kanti V. Mardia,
Thomas Hamelryck,
Jotun Hein
Abstract:
This chapter shows how toroidal diffusions are convenient methodological tools for modelling protein evolution in a probabilistic framework. The chapter addresses the construction of ergodic diffusions with stationary distributions equal to well-known directional distributions, which can be regarded as toroidal analogues of the Ornstein-Uhlenbeck process. The important challenges that arise in the…
▽ More
This chapter shows how toroidal diffusions are convenient methodological tools for modelling protein evolution in a probabilistic framework. The chapter addresses the construction of ergodic diffusions with stationary distributions equal to well-known directional distributions, which can be regarded as toroidal analogues of the Ornstein-Uhlenbeck process. The important challenges that arise in the estimation of the diffusion parameters require the consideration of tractable approximate likelihoods and, among the several approaches introduced, the one yielding a specific approximation to the transition density of the wrapped normal process is shown to give the best empirical performance on average. This provides the methodological building block for Evolutionary Torus Dynamic Bayesian Network (ETDBN), a hidden Markov model for protein evolution that emits a wrapped normal process and two continuous-time Markov chains per hidden state. The chapter describes the main features of ETDBN, which allows for both "smooth" conformational changes and "catastrophic" conformational jumps, and several empirical benchmarks. The insights into the relationship between sequence and structure evolution that ETDBN provides are illustrated in a case study.
△ Less
Submitted 21 September, 2020; v1 submitted 1 April, 2018;
originally announced April 2018.
-
Langevin diffusions on the torus: estimation and applications
Authors:
Eduardo García-Portugués,
Michael Sørensen,
Kanti V. Mardia,
Thomas Hamelryck
Abstract:
We introduce stochastic models for continuous-time evolution of angles and develop their estimation. We focus on studying Langevin diffusions with stationary distributions equal to well-known distributions from directional statistics, since such diffusions can be regarded as toroidal analogues of the Ornstein-Uhlenbeck process. Their likelihood function is a product of transition densities with no…
▽ More
We introduce stochastic models for continuous-time evolution of angles and develop their estimation. We focus on studying Langevin diffusions with stationary distributions equal to well-known distributions from directional statistics, since such diffusions can be regarded as toroidal analogues of the Ornstein-Uhlenbeck process. Their likelihood function is a product of transition densities with no analytical expression, but that can be calculated by solving the Fokker-Planck equation numerically through adequate schemes. We propose three approximate likelihoods that are computationally tractable: (i) a likelihood based on the stationary distribution; (ii) toroidal adaptations of the Euler and Shoji-Ozaki pseudo-likelihoods; (iii) a likelihood based on a specific approximation to the transition density of the wrapped normal process. A simulation study compares, in dimensions one and two, the approximate transition densities to the exact ones, and investigates the empirical performance of the approximate likelihoods. Finally, two diffusions are used to model the evolution of the backbone angles of the protein G (PDB identifier 1GB1) during a molecular dynamics simulation. The software package sdetorus implements the estimation methods and applications presented in the paper.
△ Less
Submitted 21 September, 2020; v1 submitted 30 April, 2017;
originally announced May 2017.
-
A generative angular model of protein structure evolution
Authors:
Michael Golden,
Eduardo García-Portugués,
Michael Sørensen,
Kanti V. Mardia,
Thomas Hamelryck,
Jotun Hein
Abstract:
Recently described stochastic models of protein evolution have demonstrated that the inclusion of structural information in addition to amino acid sequences leads to a more reliable estimation of evolutionary parameters. We present a generative, evolutionary model of protein structure and sequence that is valid on a local length scale. The model concerns the local dependencies between sequence and…
▽ More
Recently described stochastic models of protein evolution have demonstrated that the inclusion of structural information in addition to amino acid sequences leads to a more reliable estimation of evolutionary parameters. We present a generative, evolutionary model of protein structure and sequence that is valid on a local length scale. The model concerns the local dependencies between sequence and structure evolution in a pair of homologous proteins. The evolutionary trajectory between the two structures in the protein pair is treated as a random walk in dihedral angle space, which is modelled using a novel angular diffusion process on the two-dimensional torus. Coupling sequence and structure evolution in our model allows for modelling both "smooth" conformational changes and "catastrophic" conformational jumps, conditioned on the amino acid changes. The model has interpretable parameters and is comparatively more realistic than previous stochastic models, providing new insights into the relationship between sequence and structure evolution. For example, using the trained model we were able to identify an apparent sequence-structure evolutionary motif present in a large number of homologous protein pairs. The generative nature of our model enables us to evaluate its validity and its ability to simulate aspects of protein evolution conditioned on an amino acid sequence, a related amino acid sequence, a related structure or any combination thereof.
△ Less
Submitted 21 September, 2020; v1 submitted 30 December, 2016;
originally announced December 2016.
-
Score matching estimators for directional distributions
Authors:
Kanti V Mardia,
John T Kent,
Arnab K Laha
Abstract:
One of the major problems for maximum likelihood estimation in the well-established directional models is that the normalising constants can be difficult to evaluate. A new general method of "score matching estimation" is presented here on a compact oriented Riemannian manifold. Important applications include von Mises-Fisher, Bingham and joint models on the sphere and related spaces. The estimato…
▽ More
One of the major problems for maximum likelihood estimation in the well-established directional models is that the normalising constants can be difficult to evaluate. A new general method of "score matching estimation" is presented here on a compact oriented Riemannian manifold. Important applications include von Mises-Fisher, Bingham and joint models on the sphere and related spaces. The estimator is consistent and asymptotically normally distributed under mild regularity conditions. Further, it is easy to compute as a solution of a linear set of equations and requires no knowledge of the normalizing constant. Several examples are given, both analytic and numerical, to demonstrate its good performance.
△ Less
Submitted 28 April, 2016;
originally announced April 2016.
-
Torus Principal Component Analysis with an Application to RNA Structures
Authors:
Benjamin Eltzner,
Stephan Huckemann,
Kanti V. Mardia
Abstract:
There are several cutting edge applications needing PCA methods for data on tori and we propose a novel torus-PCA method with important properties that can be generally applied. There are two existing general methods: tangent space PCA and geodesic PCA. However, unlike tangent space PCA, our torus-PCA honors the cyclic topology of the data space whereas, unlike geodesic PCA, our torus-PCA produces…
▽ More
There are several cutting edge applications needing PCA methods for data on tori and we propose a novel torus-PCA method with important properties that can be generally applied. There are two existing general methods: tangent space PCA and geodesic PCA. However, unlike tangent space PCA, our torus-PCA honors the cyclic topology of the data space whereas, unlike geodesic PCA, our torus-PCA produces a variety of non-winding, non-dense descriptors. This is achieved by deforming tori into spheres and then using a variant of the recently developed principle nested spheres analysis. This PCA analysis involves a step of small sphere fitting and we provide an improved test to avoid overfitting. However, deforming tori into spheres creates singularities. We introduce a data-adaptive pre-clustering technique to keep the singularities away from the data. For the frequently encountered case that the residual variance around the PCA main component is small, we use a post-mode hunting technique for more fine-grained clustering. Thus in general, there are three successive interrelated key steps of torus-PCA in practice: pre-clustering, deformation, and post-mode hunting. We illustrate our method with two recently studied RNA structure (tori) data sets: one is a small RNA data set which is established as the benchmark for PCA and we validate our method through this data. Another is a large RNA data set (containing the small RNA data set) for which we show that our method provides interpretable principal components as well as giving further insight into its structure.
△ Less
Submitted 16 November, 2015;
originally announced November 2015.
-
A Fast Algorithm for Sampling from the Posterior of a von Mises distribution
Authors:
Peter G. M. Forbes,
Kanti V. Mardia
Abstract:
Motivated by molecular biology, there has been an upsurge of research activities in directional statistics in general and its Bayesian aspect in particular. The central distribution for the circular case is von Mises distribution which has two parameters (mean and concentration) akin to the univariate normal distribution. However, there has been a challenge to sample efficiently from the posterior…
▽ More
Motivated by molecular biology, there has been an upsurge of research activities in directional statistics in general and its Bayesian aspect in particular. The central distribution for the circular case is von Mises distribution which has two parameters (mean and concentration) akin to the univariate normal distribution. However, there has been a challenge to sample efficiently from the posterior distribution of the concentration parameter. We describe a novel, highly efficient algorithm to sample from the posterior distribution and fill this long-standing gap.
△ Less
Submitted 23 June, 2014; v1 submitted 14 February, 2014;
originally announced February 2014.
-
Bayesian alignment of similarity shapes
Authors:
Kanti V. Mardia,
Christopher J. Fallaize,
Stuart Barber,
Richard M. Jackson,
Douglas L. Theobald
Abstract:
We develop a Bayesian model for the alignment of two point configurations under the full similarity transformations of rotation, translation and scaling. Other work in this area has concentrated on rigid body transformations, where scale information is preserved, motivated by problems involving molecular data; this is known as form analysis. We concentrate on a Bayesian formulation for statistical…
▽ More
We develop a Bayesian model for the alignment of two point configurations under the full similarity transformations of rotation, translation and scaling. Other work in this area has concentrated on rigid body transformations, where scale information is preserved, motivated by problems involving molecular data; this is known as form analysis. We concentrate on a Bayesian formulation for statistical shape analysis. We generalize the model introduced by Green and Mardia [Biometrika 93 (2006) 235-254] for the pairwise alignment of two unlabeled configurations to full similarity transformations by introducing a scaling factor to the model. The generalization is not straightforward, since the model needs to be reformulated to give good performance when scaling is included. We illustrate our method on the alignment of rat growth profiles and a novel application to the alignment of protein domains. Here, scaling is applied to secondary structure elements when comparing protein folds; additionally, we find that one global scaling factor is not in general sufficient to model these data and, hence, we develop a model in which multiple scale factors can be included to handle different scalings of shape components.
△ Less
Submitted 6 December, 2013;
originally announced December 2013.
-
A new method to simulate the Bingham and related distributions in directional data analysis with applications
Authors:
John T. Kent,
Asaad M. Ganeiber,
Kanti V. Mardia
Abstract:
A new acceptance-rejection method is proposed and investigated for the Bingham distribution on the sphere using the angular central Gaussian distribution as an envelope. It is shown to have high efficiency and to be straightfoward to use. The method can also be extended to Fisher and Fisher-Bingham distributions on spheres and related manifolds.
A new acceptance-rejection method is proposed and investigated for the Bingham distribution on the sphere using the angular central Gaussian distribution as an envelope. It is shown to have high efficiency and to be straightfoward to use. The method can also be extended to Fisher and Fisher-Bingham distributions on spheres and related manifolds.
△ Less
Submitted 30 October, 2013;
originally announced October 2013.
-
Matching markers and unlabeled configurations in protein gels
Authors:
Kanti V. Mardia,
Emma M. Petty,
Charles C. Taylor
Abstract:
Unlabeled shape analysis is a rapidly emerging and challenging area of statistics. This has been driven by various novel applications in bioinformatics. We consider here the situation where two configurations are matched under various constraints, namely, the configurations have a subset of manually located "markers" with high probability of matching each other while a larger subset consists of un…
▽ More
Unlabeled shape analysis is a rapidly emerging and challenging area of statistics. This has been driven by various novel applications in bioinformatics. We consider here the situation where two configurations are matched under various constraints, namely, the configurations have a subset of manually located "markers" with high probability of matching each other while a larger subset consists of unlabeled points. We consider a plausible model and give an implementation using the EM algorithm. The work is motivated by a real experiment of gels for renal cancer and our approach allows for the possibility of missing and misallocated markers. The methodology is successfully used to automatically locate and remove a grossly misallocated marker within the given data set.
△ Less
Submitted 27 September, 2012;
originally announced September 2012.
-
Some Fundamental Properties of a Multivariate von Mises Distribution
Authors:
Kanti V. Mardia,
Jochen Voss
Abstract:
In application areas like bioinformatics multivariate distributions on angles are encountered which show significant clustering. One approach to statistical modelling of such situations is to use mixtures of unimodal distributions. In the literature (Mardia et al., 2011), the multivariate von Mises distribution, also known as the multivariate sine distribution, has been suggested for components of…
▽ More
In application areas like bioinformatics multivariate distributions on angles are encountered which show significant clustering. One approach to statistical modelling of such situations is to use mixtures of unimodal distributions. In the literature (Mardia et al., 2011), the multivariate von Mises distribution, also known as the multivariate sine distribution, has been suggested for components of such models, but work in the area has been hampered by the fact that no good criteria for the von Mises distribution to be unimodal were available. In this article we study the question about when a multivariate von Mises distribution is unimodal. We give sufficient criteria for this to be the case and show examples of distributions with multiple modes when these criteria are violated. In addition, we propose a method to generate samples from the von Mises distribution in the case of high concentration.
△ Less
Submitted 5 July, 2013; v1 submitted 27 September, 2011;
originally announced September 2011.
-
Directions and projective shapes
Authors:
Kanti V. Mardia,
Vic Patrangenaru
Abstract:
This paper deals with projective shape analysis, which is a study of finite configurations of points modulo projective transformations. The topic has various applications in machine vision. We introduce a convenient projective shape space, as well as an appropriate coordinate system for this shape space. For generic configurations of k points in m dimensions, the resulting projective shape space…
▽ More
This paper deals with projective shape analysis, which is a study of finite configurations of points modulo projective transformations. The topic has various applications in machine vision. We introduce a convenient projective shape space, as well as an appropriate coordinate system for this shape space. For generic configurations of k points in m dimensions, the resulting projective shape space is identified as a product of k-m-2 copies of axial spaces RP^m. This identification leads to the need for develo** multivariate directional and multivariate axial analysis and we propose parametric models, as well as nonparametric methods, for these areas. In particular, we investigate the Frechet extrinsic mean for the multivariate axial case. Asymptotic distributions of the appropriate parametric and nonparametric tests are derived. We illustrate our methodology with examples from machine vision.
△ Less
Submitted 16 August, 2005;
originally announced August 2005.
