Showing 1–8 of 8 results for author: Vitek, O

Experimental design for causal query estimation in partially observed biomolecular networks

Authors: Sara Mohammad-Taheri, Vartika Tewari, Rohan Kapre, Ehsan Rahiminasab, Karen Sachs, Charles Tapley Hoyt, Jeremy Zucker, Olga Vitek

Abstract: Estimating a causal query from observational data is an essential task in the analysis of biomolecular networks. Estimation takes as input a network topology, a query estimation method, and observational measurements on the network variables. However, estimations involving many variables can be experimentally expensive, and computationally intractable. Moreover, using the full set of variables can… ▽ More Estimating a causal query from observational data is an essential task in the analysis of biomolecular networks. Estimation takes as input a network topology, a query estimation method, and observational measurements on the network variables. However, estimations involving many variables can be experimentally expensive, and computationally intractable. Moreover, using the full set of variables can be detrimental, leading to bias, or increasing the variance in the estimation. Therefore, designing an experiment based on a well-chosen subset of network components can increase estimation accuracy, and reduce experimental and computational costs. We propose a simulation-based algorithm for selecting sub-networks that support unbiased estimators of the causal query under a constraint of cost, ranked with respect to the variance of the estimators. The simulations are constructed based on historical experimental data, or based on known properties of the biological system. Three case studies demonstrated the effectiveness of well-chosen network subsets for estimating causal queries from observational data. All the case studies are reproducible and available at https://github.com/srtaheri/Simplified_LVM. △ Less

Submitted 28 November, 2022; v1 submitted 24 October, 2022; originally announced October 2022.

Do-calculus enables estimation of causal effects in partially observed biomolecular pathways

Authors: Sara Mohammad-Taheri, Jeremy Zucker, Charles Tapley Hoyt, Karen Sachs, Vartika Tewari, Robert Ness, and Olga Vitek

Abstract: Estimating causal queries, such as changes in protein abundance in response to a perturbation, is a fundamental task in the analysis of biomolecular pathways. The estimation requires experimental measurements on the pathway components. However, in practice many pathway components are left unobserved (latent) because they are either unknown, or difficult to measure. Latent variable models (LVMs) ar… ▽ More Estimating causal queries, such as changes in protein abundance in response to a perturbation, is a fundamental task in the analysis of biomolecular pathways. The estimation requires experimental measurements on the pathway components. However, in practice many pathway components are left unobserved (latent) because they are either unknown, or difficult to measure. Latent variable models (LVMs) are well-suited for such estimation. Unfortunately, LVM-based estimation of causal queries can be inaccurate when parameters of the latent variables are not uniquely identified, or when the number of latent variables is misspecified. This has limited the use of LVMs for causal inference in biomolecular pathways. In this manuscript, we propose a general and practical approach for LVM-based estimation of causal queries. We prove that, despite the challenges above, LVM-based estimators of causal queries are accurate if the queries are identifiable according to Pearl's do-calculus, and describe an algorithm for its estimation. We illustrate the breadth and the practical utility of this approach for estimating causal queries in four synthetic and two experimental case studies, where structures of biomolecular pathways challenge the existing methods for causal query estimation. The code and the data documenting all the case studies are available at \url{https://github.com/srtaheri/LVMwithDoCalculus} △ Less

Submitted 24 October, 2022; v1 submitted 12 February, 2021; originally announced February 2021.

Comments: https://academic.oup.com/bioinformatics/article/38/Supplement_1/i350/6617530

Journal ref: Bioinformatics 38(2022): i350-i358

Leveraging Structured Biological Knowledge for Counterfactual Inference: a Case Study of Viral Pathogenesis

Authors: Jeremy Zucker, Kaushal Paneri, Sara Mohammad-Taheri, Somya Bhargava, Pallavi Kolambkar, Craig Bakker, Jeremy Teuton, Charles Tapley Hoyt, Kristie Oxford, Robert Ness, Olga Vitek

Abstract: Counterfactual inference is a useful tool for comparing outcomes of interventions on complex systems. It requires us to represent the system in form of a structural causal model, complete with a causal diagram, probabilistic assumptions on exogenous variables, and functional assignments. Specifying such models can be extremely difficult in practice. The process requires substantial domain expertis… ▽ More Counterfactual inference is a useful tool for comparing outcomes of interventions on complex systems. It requires us to represent the system in form of a structural causal model, complete with a causal diagram, probabilistic assumptions on exogenous variables, and functional assignments. Specifying such models can be extremely difficult in practice. The process requires substantial domain expertise, and does not scale easily to large systems, multiple systems, or novel system modifications. At the same time, many application domains, such as molecular biology, are rich in structured causal knowledge that is qualitative in nature. This manuscript proposes a general approach for querying a causal biological knowledge graph, and converting the qualitative result into a quantitative structural causal model that can learn from data to answer the question. We demonstrate the feasibility, accuracy and versatility of this approach using two case studies in systems biology. The first demonstrates the appropriateness of the underlying assumptions and the accuracy of the results. The second demonstrates the versatility of the approach by querying a knowledge base for the molecular determinants of a severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2)-induced cytokine storm, and performing counterfactual inference to estimate the causal effect of medical countermeasures for severely ill patients. △ Less

Submitted 13 January, 2021; originally announced January 2021.

Comments: In proceeding of IEEE, Transactions on Big Data

New mixture models for decoy-free false discovery rate estimation in mass-spectrometry proteomics

Authors: Yisu Peng, Shantanu Jain, Yong Fuga Li, Michal Gregus, Alexander R. Ivanov, Olga Vitek, Predrag Radivojac

Abstract: Motivation: Accurate estimation of false discovery rate (FDR) of spectral identification is a central problem in mass spectrometry-based proteomics. Over the past two decades, target decoy approaches (TDAs) and decoy-free approaches (DFAs), have been widely used to estimate FDR. TDAs use a database of decoy species to faithfully model score distributions of incorrect peptide-spectrum matches (PSMs… ▽ More Motivation: Accurate estimation of false discovery rate (FDR) of spectral identification is a central problem in mass spectrometry-based proteomics. Over the past two decades, target decoy approaches (TDAs) and decoy-free approaches (DFAs), have been widely used to estimate FDR. TDAs use a database of decoy species to faithfully model score distributions of incorrect peptide-spectrum matches (PSMs). DFAs, on the other hand, fit two-component mixture models to learn the parameters of correct and incorrect PSM score distributions. While conceptually straightforward, both approaches lead to problems in practice, particularly in experiments that push instrumentation to the limit and generate low fragmentation-efficiency and low signal-to-noise-ratio spectra. Results: We introduce a new decoy-free framework for FDR estimation that generalizes present DFAs while exploiting more search data in a manner similar to TDAs. Our approach relies on multi-component mixtures, in which score distributions corresponding to the correct PSMs, best incorrect PSMs, and second-best incorrect PSMs are modeled by the skew normal family. We derive EM algorithms to estimate parameters of these distributions from the scores of best and second-best PSMs associated with each experimental spectrum. We evaluate our models on multiple proteomics datasets and a HeLa cell digest case study consisting of more than a million spectra in total. We provide evidence of improved performance over existing DFAs and improved stability and speed over TDAs without any performance degradation. We propose that the new strategy has the potential to extend beyond peptide identification and reduce the need for TDA on all analytical platforms. △ Less

Submitted 16 September, 2020; originally announced September 2020.

Investigating usability of MSstatsQC software

Authors: Sara Mohammad Taheri, Omkar Terse, Eralp Dogu, Magy Seif El-Nasr, Olga Vitek

Abstract: MSstatsQC [3] is an open-source software that provides longitudinal system suitability monitoring tools in the form of control charts for proteomic experiments. It includes simultaneous tools for the mean and dispersion of suitability metrics and presents alternative methods of monitoring through different tabs that are designed in the interface. This research focuses on investigating the usabilit… ▽ More MSstatsQC [3] is an open-source software that provides longitudinal system suitability monitoring tools in the form of control charts for proteomic experiments. It includes simultaneous tools for the mean and dispersion of suitability metrics and presents alternative methods of monitoring through different tabs that are designed in the interface. This research focuses on investigating the usability of MSstatsQC software and the interpretability of the designed plots. In this study, we ask 4 test users, from the proteomics field, to complete a series of tasks and questionnaires. The tasks are designed to test the usability of the software in terms of importing data files, selecting appropriate metrics, guide set, and peptides, and finally creating decision rules (tasks 1 and 3 in appendix). The questionnaires ask about interpretability of the plots including control charts, box plots, heat maps, river plots, and radar plots (tasks 1 and 4 in appendix). The goal of the questions is to determine if the test users understand the plots and can interpret them. Results show limitations in usability and plot interpretability, especially in the data import section. We suggest the following modifications. I) providing conspicuous guides close to the window related to up-loading a datafile as well as providing error messages that pop-up when the data set has a wrong format II) providing plot descriptions, hints to interpret plots, plot titles and appropriate axis labels, and, III) Numbering tabs to show the flow of procedures in the software. △ Less

Submitted 2 February, 2020; originally announced February 2020.

Comments: 19 pages

FSE/CACM Rebuttal$^2$: Correcting A Large-Scale Study of Programming Languages and Code Quality in GitHub

Authors: Emery D. Berger, Petr Maj, Olga Vitek, Jan Vitek

Abstract: Ray, Devanbu and Filkov issued a rebuttal of our TOPLAS paper "On the Impact of Programming Languages on Code Quality: A Reproduction Study". Our paper reproduced "A Large-Scale Study of Programming Languages and Code Quality in GitHub", which appeared at FSE 2014 and was subsequently republished as a CACM research highlight in 2017. This article is a rebuttal to that rebuttal. Ray, Devanbu and Filkov issued a rebuttal of our TOPLAS paper "On the Impact of Programming Languages on Code Quality: A Reproduction Study". Our paper reproduced "A Large-Scale Study of Programming Languages and Code Quality in GitHub", which appeared at FSE 2014 and was subsequently republished as a CACM research highlight in 2017. This article is a rebuttal to that rebuttal. △ Less

Submitted 26 November, 2019; originally announced November 2019.

Integrating Markov processes with structural causal modeling enables counterfactual inference in complex systems

Authors: Robert Osazuwa Ness, Kaushal Paneri, Olga Vitek

Abstract: This manuscript contributes a general and practical framework for casting a Markov process model of a system at equilibrium as a structural causal model, and carrying out counterfactual inference. Markov processes mathematically describe the mechanisms in the system, and predict the system's equilibrium behavior upon intervention, but do not support counterfactual inference. In contrast, structura… ▽ More This manuscript contributes a general and practical framework for casting a Markov process model of a system at equilibrium as a structural causal model, and carrying out counterfactual inference. Markov processes mathematically describe the mechanisms in the system, and predict the system's equilibrium behavior upon intervention, but do not support counterfactual inference. In contrast, structural causal models support counterfactual inference, but do not identify the mechanisms. This manuscript leverages the benefits of both approaches. We define the structural causal models in terms of the parameters and the equilibrium dynamics of the Markov process models, and counterfactual inference flows from these settings. The proposed approach alleviates the identifiability drawback of the structural causal models, in that the counterfactual inference is consistent with the counterfactual trajectories simulated from the Markov process model. We showcase the benefits of this framework in case studies of complex biomolecular systems with nonlinear dynamics. We illustrate that, in presence of Markov process model misspecification, counterfactual inference leverages prior data, and therefore estimates the outcome of an intervention more accurately than a direct simulation. △ Less

Submitted 5 November, 2019; originally announced November 2019.

Comments: Accepted to Thirty-third Conference on Neural Information Processing Systems (2019)

On the Impact of Programming Languages on Code Quality

Authors: Emery D. Berger, Celeste Hollenbeck, Petr Maj, Olga Vitek, Jan Vitek

Abstract: This paper is a reproduction of work by Ray et al. which claimed to have uncovered a statistically significant association between eleven programming languages and software defects in projects hosted on GitHub. First we conduct an experimental repetition, repetition is only partially successful, but it does validate one of the key claims of the original work about the association of ten programmin… ▽ More This paper is a reproduction of work by Ray et al. which claimed to have uncovered a statistically significant association between eleven programming languages and software defects in projects hosted on GitHub. First we conduct an experimental repetition, repetition is only partially successful, but it does validate one of the key claims of the original work about the association of ten programming languages with defects. Next, we conduct a complete, independent reanalysis of the data and statistical modeling steps of the original study. We uncover a number of flaws that undermine the conclusions of the original study as only four languages are found to have a statistically significant association with defects, and even for those the effect size is exceedingly small. We conclude with some additional sources of bias that should be investigated in follow up work and a few best practice recommendations for similar efforts. △ Less

Submitted 24 April, 2019; v1 submitted 29 January, 2019; originally announced January 2019.

Comments: Accepted, to appear in TOPLAS (ACM Transactions of Programming Languages and Systems)