-
Removing Bias from Maximum Likelihood Estimation with Model Autophagy
Authors:
Paul Mayer,
Lorenzo Luzi,
Ali Siahkoohi,
Don H. Johnson,
Richard G. Baraniuk
Abstract:
We propose autophagy penalized likelihood estimation (PLE), an unbiased alternative to maximum likelihood estimation (MLE) which is more fair and less susceptible to model autophagy disorder (madness). Model autophagy refers to models trained on their own output; PLE ensures the statistics of these outputs coincide with the data statistics. This enables PLE to be statistically unbiased in certain…
▽ More
We propose autophagy penalized likelihood estimation (PLE), an unbiased alternative to maximum likelihood estimation (MLE) which is more fair and less susceptible to model autophagy disorder (madness). Model autophagy refers to models trained on their own output; PLE ensures the statistics of these outputs coincide with the data statistics. This enables PLE to be statistically unbiased in certain scenarios where MLE is biased. When biased, MLE unfairly penalizes minority classes in unbalanced datasets and exacerbates the recently discovered issue of self-consuming generative modeling. Theoretical and empirical results show that 1) PLE is more fair to minority classes and 2) PLE is more stable in a self-consumed setting. Furthermore, we provide a scalable and portable implementation of PLE with a hypernetwork framework, allowing existing deep learning architectures to be easily trained with PLE. Finally, we show PLE can bridge the gap between Bayesian and frequentist paradigms in statistics.
△ Less
Submitted 22 May, 2024;
originally announced May 2024.
-
Mutual Information in Frequency and its Application to Measure Cross-Frequency Coupling in Epilepsy
Authors:
Rakesh Malladi,
Don H Johnson,
Giridhar P Kalamangalam,
Nitin Tandon,
Behnaam Aazhang
Abstract:
We define a metric, mutual information in frequency (MI-in-frequency), to detect and quantify the statistical dependence between different frequency components in the data, referred to as cross-frequency coupling and apply it to electrophysiological recordings from the brain to infer cross-frequency coupling. The current metrics used to quantify the cross-frequency coupling in neuroscience cannot…
▽ More
We define a metric, mutual information in frequency (MI-in-frequency), to detect and quantify the statistical dependence between different frequency components in the data, referred to as cross-frequency coupling and apply it to electrophysiological recordings from the brain to infer cross-frequency coupling. The current metrics used to quantify the cross-frequency coupling in neuroscience cannot detect if two frequency components in non-Gaussian brain recordings are statistically independent or not. Our MI-in-frequency metric, based on Shannon's mutual information between the Cramer's representation of stochastic processes, overcomes this shortcoming and can detect statistical dependence in frequency between non-Gaussian signals. We then describe two data-driven estimators of MI-in-frequency: one based on kernel density estimation and the other based on the nearest neighbor algorithm and validate their performance on simulated data. We then use MI-in-frequency to estimate mutual information between two data streams that are dependent across time, without making any parametric model assumptions. Finally, we use the MI-in- frequency metric to investigate the cross-frequency coupling in seizure onset zone from electrocorticographic recordings during seizures. The inferred cross-frequency coupling characteristics are essential to optimize the spatial and spectral parameters of electrical stimulation based treatments of epilepsy.
△ Less
Submitted 15 March, 2018; v1 submitted 5 November, 2017;
originally announced November 2017.
-
Data-Driven Estimation Of Mutual Information Between Dependent Data
Authors:
Rakesh Malladi,
Don H Johnson,
Behnaam Aazhang
Abstract:
We consider the problem of estimating mutual information between dependent data, an important problem in many science and engineering applications. We propose a data-driven, non-parametric estimator of mutual information in this paper. The main novelty of our solution lies in transforming the data to frequency domain to make the problem tractable. We define a novel metric--mutual information in fr…
▽ More
We consider the problem of estimating mutual information between dependent data, an important problem in many science and engineering applications. We propose a data-driven, non-parametric estimator of mutual information in this paper. The main novelty of our solution lies in transforming the data to frequency domain to make the problem tractable. We define a novel metric--mutual information in frequency--to detect and quantify the dependence between two random processes across frequency using Cramér's spectral representation. Our solution calculates mutual information as a function of frequency to estimate the mutual information between the dependent data over time. We validate its performance on linear and nonlinear models. In addition, mutual information in frequency estimated as a part of our solution can also be used to infer cross-frequency coupling in the data.
△ Less
Submitted 7 March, 2017;
originally announced March 2017.
-
Mechanism of preferential adsorption of SO$_2$ into two microporous paddle wheel frameworks M(bdc)(ted)0.5
Authors:
Kui Tan,
Pieremanuele Canepa,
Qihan Gong,
Jian Liu,
Daniel H. Johnson,
Allison Dyevoich,
Praveen K. Thallapally,
Timo Thonhauser,
**g Li,
Yves J. Chabal
Abstract:
The selective adsorption of a corrosive gas, SO$_2$, into two microporous pillared paddle-wheel frameworks M(bdc)(ted)0.5 [M = Ni, Zn; bdc = 1,4-benzenedicarboxylate; ted = triethylenediamine] is studied by volumetric adsorption measurements and a combination of in-situ infrared spectroscopy and ab initio density functional theory (DFT) calculations. The uptake of SO$_2$ in M(bdc)(ted)0.5 at room…
▽ More
The selective adsorption of a corrosive gas, SO$_2$, into two microporous pillared paddle-wheel frameworks M(bdc)(ted)0.5 [M = Ni, Zn; bdc = 1,4-benzenedicarboxylate; ted = triethylenediamine] is studied by volumetric adsorption measurements and a combination of in-situ infrared spectroscopy and ab initio density functional theory (DFT) calculations. The uptake of SO$_2$ in M(bdc)(ted)0.5 at room temperature is quite significant, 9.97 mol/kg at 1.13 bar. The major adsorbed SO$_2$ molecules contributing to the isotherm measurements are characterized by stretching bands at 1326 and 1144 cm$^{-1}$. Theoretical calculations including van der Waals interactions (based on vdW-DF) suggest that two adsorption configurations are possible for these SO$_2$ molecules. One geometry involves an SO$_2$ molecule bonded through its sulfur atom to the oxygen atom of the paddle-wheel building unit and its two oxygen atoms to the C-H groups of the organic linkers by formation of hydrogen bonds. Such a configuration results in a distortion of the benzene rings, which is consistent with the experimentally observed shift of the ring deformation mode. In the other geometry, SO$_2$ establishes hydrogen bonding with -CH$_2$ group of the ted linker through its two oxygen atoms simultaneously. The vdW-DF-simulated frequency shifts of the SO$_2$ stretching bands in these two configurations are similar and in good agreement with spectroscopically measured values of physisorbed SO$_2$.In addition, the IR spectra reveal the presence of another minor species, characterized by stretching modes at 1242 and 1105 cm$^{-1}$ and causing significant perturbations of MOFs vibrational modes (CH$_x$ and carboxylate groups). This species is more strongly bound, requiring a higher temperature ($\sim$150 $^\circ$C) to remove it than for the main physisorbed species.
△ Less
Submitted 26 October, 2013;
originally announced October 2013.
-
High-throughput screening of small-molecule adsorption in MOF
Authors:
Pieremanuele Canepa,
Calvin A. Arter,
Eliot M. Conwill,
Daniel H. Johnson,
Brian A. Shoemaker,
Karim Z. Soliman,
T. Thonhauser
Abstract:
Using high-throughput screening coupled with state-of-the-art van der Waals density functional theory, we investigate the adsorption properties of four important molecules, H_2, CO_2, CH_4, and H_2O in MOF-74-M with M = Be, Mg, Al, Ca, Sc, Ti, V, Cr, Mn, Fe, Co, Ni, Cu, Zn, Sr, Zr, Nb, Ru, Rh, Pd, La, W, Os, Ir, and Pt. We show that high-throughput techniques can aid in speeding up the development…
▽ More
Using high-throughput screening coupled with state-of-the-art van der Waals density functional theory, we investigate the adsorption properties of four important molecules, H_2, CO_2, CH_4, and H_2O in MOF-74-M with M = Be, Mg, Al, Ca, Sc, Ti, V, Cr, Mn, Fe, Co, Ni, Cu, Zn, Sr, Zr, Nb, Ru, Rh, Pd, La, W, Os, Ir, and Pt. We show that high-throughput techniques can aid in speeding up the development and refinement of effective materials for hydrogen storage, carbon capture, and gas separation. The exploration of the configurational adsorption space allows us to extract crucial information concerning, for example, the competition of water with CO_2 for the adsorption "pockets." We find that only a few noble metals---Rh, Pd, Os, Ir, and Pt---favor the adsorption of CO_2 and hence are potential candidates for effective carbon-capture materials. Our findings further reveal significant differences in the binding characteristics of H_2, CO_2, CH_4, and H_2O within the MOF structure, indicating that molecular blends can be successfully separated by these nano-porous materials.
△ Less
Submitted 7 June, 2013;
originally announced June 2013.
-
Jointly Poisson processes
Authors:
D. H. Johnson,
I. N. Goodman
Abstract:
What constitutes jointly Poisson processes remains an unresolved issue. This report reviews the current state of the theory and indicates how the accepted but unproven model equals that resulting from the small time-interval limit of jointly Bernoulli processes. One intriguing consequence of these models is that jointly Poisson processes can only be positively correlated as measured by the corre…
▽ More
What constitutes jointly Poisson processes remains an unresolved issue. This report reviews the current state of the theory and indicates how the accepted but unproven model equals that resulting from the small time-interval limit of jointly Bernoulli processes. One intriguing consequence of these models is that jointly Poisson processes can only be positively correlated as measured by the correlation coefficient defined by cumulants of the probability generating functional.
△ Less
Submitted 12 November, 2009;
originally announced November 2009.
-
The Correlation Function of Multiple Dependent Poisson Processes Generated by the Alternating Renewal Process Method
Authors:
Don H. Johnson
Abstract:
We derive conditions under which alternating renewal processes can be used to construct correlated Poisson processes. The pairwise correlation function is also derived, showing that the resulting correlations can be negative. The technique and the analysis can be extended to the generation of two or more dependent renewal processes.
We derive conditions under which alternating renewal processes can be used to construct correlated Poisson processes. The pairwise correlation function is also derived, showing that the resulting correlations can be negative. The technique and the analysis can be extended to the generation of two or more dependent renewal processes.
△ Less
Submitted 22 November, 2008;
originally announced November 2008.
