-
Krylov fractality and complexity in generic random matrix ensembles
Authors:
Budhaditya Bhattacharjee,
Pratik Nandy
Abstract:
Krylov space methods provide an efficient framework for analyzing the dynamical aspects of quantum systems, with tridiagonal matrices playing a key role. Despite their importance, the behavior of such matrices from chaotic to integrable states, transitioning through an intermediate phase, remains unexplored. We aim to fill this gap by considering the properties of the tridiagonal matrix elements a…
▽ More
Krylov space methods provide an efficient framework for analyzing the dynamical aspects of quantum systems, with tridiagonal matrices playing a key role. Despite their importance, the behavior of such matrices from chaotic to integrable states, transitioning through an intermediate phase, remains unexplored. We aim to fill this gap by considering the properties of the tridiagonal matrix elements and the associated basis vectors for appropriate random matrix ensembles. We utilize the Rosenzweig-Porter model as our primary example, which hosts a fractal regime in addition to the ergodic and localized phases. We discuss the characteristics of the matrix elements and basis vectors across the three (ergodic, fractal, and localized) regimes and introduce tools to identify the transition points. The exact expressions of the Lanczos coefficients are provided in terms of $q$-logarithmic function across the full parameter regime. The numerical results are corroborated with analytical reasoning for certain features of the Krylov spectra. Additionally, we investigate the Krylov state complexity within these regimes, showcasing the efficacy of our methods in pinpointing these transitions.
△ Less
Submitted 10 July, 2024;
originally announced July 2024.
-
Probing quantum chaos through singular-value correlations in sparse non-Hermitian SYK model
Authors:
Pratik Nandy,
Tanay Pathak,
Masaki Tezuka
Abstract:
Utilizing singular value decomposition, our investigation focuses on the spectrum of the singular values within a sparse non-Hermitian Sachdev-Ye-Kitaev (SYK) model. Unlike the complex eigenvalues typical of non-Hermitian systems, singular values are inherently real and positive. Our findings reveal a congruence between the statistics of singular values and those of the analogous Hermitian Gaussia…
▽ More
Utilizing singular value decomposition, our investigation focuses on the spectrum of the singular values within a sparse non-Hermitian Sachdev-Ye-Kitaev (SYK) model. Unlike the complex eigenvalues typical of non-Hermitian systems, singular values are inherently real and positive. Our findings reveal a congruence between the statistics of singular values and those of the analogous Hermitian Gaussian ensembles. An increase in sparsity results in the non-Hermitian SYK model deviating from its chaotic behavior, a phenomenon precisely captured by the singular value ratios. Our analysis of the singular form factor ({\upsigma}FF), analogous to the spectral form factor (SFF) indicates the disappearance of the linear ramp with increased sparsity. Additionally, we define singular complexity, inspired by the spectral complexity in Hermitian systems, whose saturation provides a critical threshold of sparseness. Such disintegration is likely associated with the breakdown of the existing holographic dual for non-Hermitian systems.
△ Less
Submitted 17 June, 2024;
originally announced June 2024.
-
Quantum Dynamics in Krylov Space: Methods and Applications
Authors:
Pratik Nandy,
Apollonas S. Matsoukas-Roubeas,
Pablo Martínez-Azcona,
Anatoly Dymarsky,
Adolfo del Campo
Abstract:
The dynamics of quantum systems unfolds within a subspace of the state space or operator space, known as the Krylov space. This review presents the use of Krylov subspace methods to provide a compact and computationally efficient description of quantum evolution, with emphasis on nonequilibrium phenomena of many-body systems with a large Hilbert space. It provides a comprehensive update of recent…
▽ More
The dynamics of quantum systems unfolds within a subspace of the state space or operator space, known as the Krylov space. This review presents the use of Krylov subspace methods to provide a compact and computationally efficient description of quantum evolution, with emphasis on nonequilibrium phenomena of many-body systems with a large Hilbert space. It provides a comprehensive update of recent developments, focused on the quantum evolution of operators in the Heisenberg picture as well as pure and mixed states. It further explores the notion of Krylov complexity and associated metrics as tools for quantifying operator growth, their bounds by generalized quantum speed limits, the universal operator growth hypothesis, and its relation to quantum chaos, scrambling, and generalized coherent states. A comparison of several generalizations of the Krylov construction for open quantum systems is presented. A closing discussion addresses the application of Krylov subspace methods in quantum field theory, holography, integrability, quantum control, and quantum computing, as well as current open problems.
△ Less
Submitted 5 June, 2024; v1 submitted 15 May, 2024;
originally announced May 2024.
-
Operator dynamics in Lindbladian SYK: a Krylov complexity perspective
Authors:
Budhaditya Bhattacharjee,
Pratik Nandy,
Tanay Pathak
Abstract:
We use Krylov complexity to study operator growth in the $q$-body dissipative SYK model, where the dissipation is modeled by linear and random $p$-body Lindblad operators. In the large $q$ limit, we analytically establish the linear growth of two sets of coefficients for any generic jump operators. We numerically verify this by implementing the bi-Lanczos algorithm, which transforms the Lindbladia…
▽ More
We use Krylov complexity to study operator growth in the $q$-body dissipative SYK model, where the dissipation is modeled by linear and random $p$-body Lindblad operators. In the large $q$ limit, we analytically establish the linear growth of two sets of coefficients for any generic jump operators. We numerically verify this by implementing the bi-Lanczos algorithm, which transforms the Lindbladian into a pure tridiagonal form. We find that the Krylov complexity saturates inversely with the dissipation strength, while the dissipative timescale grows logarithmically. This is akin to the behavior of other $\mathfrak{q}$-complexity measures, namely out-of-time-order correlator (OTOC) and operator size, which we also demonstrate. We connect these observations to continuous quantum measurement processes. We further investigate the pole structure of a generic auto-correlation and the high-frequency behavior of the spectral function in the presence of dissipation, thereby revealing a general principle for operator growth in dissipative quantum chaotic systems.
△ Less
Submitted 17 January, 2024; v1 submitted 1 November, 2023;
originally announced November 2023.
-
Learning to Detect Touches on Cluttered Tables
Authors:
Norberto Adrian Goussies,
Kenji Hata,
Shruthi Prabhakara,
Abhishek Amit,
Tony Aube,
Carl Cepress,
Diana Chang,
Li-Te Cheng,
Horia Stefan Ciurdar,
Mike Cleron,
Chelsey Fleming,
Ashwin Ganti,
Divyansh Garg,
Niloofar Gheissari,
Petra Luna Grutzik,
David Hendon,
Daniel Iglesia,
** Kim,
Stuart Kyle,
Chris LaRosa,
Roman Lewkow,
Peter F McDermott,
Chris Melancon,
Paru Nackeeran,
Neal Norwitz
, et al. (6 additional authors not shown)
Abstract:
We present a novel self-contained camera-projector tabletop system with a lamp form-factor that brings digital intelligence to our tables. We propose a real-time, on-device, learning-based touch detection algorithm that makes any tabletop interactive. The top-down configuration and learning-based algorithm makes our method robust to the presence of clutter, a main limitation of existing camera-pro…
▽ More
We present a novel self-contained camera-projector tabletop system with a lamp form-factor that brings digital intelligence to our tables. We propose a real-time, on-device, learning-based touch detection algorithm that makes any tabletop interactive. The top-down configuration and learning-based algorithm makes our method robust to the presence of clutter, a main limitation of existing camera-projector tabletop systems. Our research prototype enables a set of experiences that combine hand interactions and objects present on the table. A video can be found at https://youtu.be/hElC_c25Fg8.
△ Less
Submitted 10 April, 2023;
originally announced April 2023.
-
On Krylov complexity in open systems: an approach via bi-Lanczos algorithm
Authors:
Aranya Bhattacharya,
Pratik Nandy,
**al Pratyush Nath,
Himanshu Sahu
Abstract:
Continuing the previous initiatives arXiv: 2207.05347 and arXiv: 2212.06180, we pursue the exploration of operator growth and Krylov complexity in dissipative open quantum systems. In this paper, we resort to the bi-Lanczos algorithm generating two bi-orthogonal Krylov spaces, which individually generate non-orthogonal subspaces. Unlike the previously studied Arnoldi iteration, this algorithm rend…
▽ More
Continuing the previous initiatives arXiv: 2207.05347 and arXiv: 2212.06180, we pursue the exploration of operator growth and Krylov complexity in dissipative open quantum systems. In this paper, we resort to the bi-Lanczos algorithm generating two bi-orthogonal Krylov spaces, which individually generate non-orthogonal subspaces. Unlike the previously studied Arnoldi iteration, this algorithm renders the Lindbladian into a purely tridiagonal form, thus opening up a possibility to study a wide class of dissipative integrable and chaotic systems by computing Krylov complexity at late times. Our study relies on two specific systems, the dissipative transverse-field Ising model (TFIM) and the dissipative interacting XXZ chain. We find that, for the weak coupling, initial Lanczos coefficients can efficiently distinguish integrable and chaotic evolution before the dissipative effect sets in, which results in more fluctuations in higher Lanczos coefficients. This results in the equal saturation of late-time complexity for both integrable and chaotic cases, making the notion of late-time chaos dubious.
△ Less
Submitted 14 December, 2023; v1 submitted 7 March, 2023;
originally announced March 2023.
-
Operator growth in open quantum systems: lessons from the dissipative SYK
Authors:
Budhaditya Bhattacharjee,
Xiangyu Cao,
Pratik Nandy,
Tanay Pathak
Abstract:
We study the operator growth in open quantum systems with dephasing dissipation terms, extending the Krylov complexity formalism of Phys. Rev. X 9, 041017. Our results are based on the study of the dissipative $q$-body Sachdev-Ye-Kitaev (SYK$_q$) model, governed by the Markovian dynamics. We introduce a notion of ''operator size concentration'' which allows a diagrammatic and combinatorial proof o…
▽ More
We study the operator growth in open quantum systems with dephasing dissipation terms, extending the Krylov complexity formalism of Phys. Rev. X 9, 041017. Our results are based on the study of the dissipative $q$-body Sachdev-Ye-Kitaev (SYK$_q$) model, governed by the Markovian dynamics. We introduce a notion of ''operator size concentration'' which allows a diagrammatic and combinatorial proof of the asymptotic linear behavior of the two sets of Lanczos coefficients ($a_n$ and $b_n$) in the large $q$ limit. Our results corroborate with the semi-analytics in finite $q$ in the large $N$ limit, and the numerical Arnoldi iteration in finite $q$ and finite $N$ limit. As a result, Krylov complexity exhibits exponential growth following a saturation at a time that grows logarithmically with the inverse dissipation strength. The growth of complexity is suppressed compared to the closed system results, yet it upper bounds the growth of the normalized out-of-time-ordered correlator (OTOC). We provide a plausible explanation of the results from the dual gravitational side.
△ Less
Submitted 8 March, 2023; v1 submitted 12 December, 2022;
originally announced December 2022.
-
Krylov complexity in large-$q$ and double-scaled SYK model
Authors:
Budhaditya Bhattacharjee,
Pratik Nandy,
Tanay Pathak
Abstract:
Considering the large-$q$ expansion of the Sachdev-Ye-Kitaev (SYK) model in the two-stage limit, we compute the Lanczos coefficients, Krylov complexity, and the higher Krylov cumulants in subleading order, along with the $t/q$ effects. The Krylov complexity naturally describes the "size" of the distribution, while the higher cumulants encode richer information. We further consider the double-scale…
▽ More
Considering the large-$q$ expansion of the Sachdev-Ye-Kitaev (SYK) model in the two-stage limit, we compute the Lanczos coefficients, Krylov complexity, and the higher Krylov cumulants in subleading order, along with the $t/q$ effects. The Krylov complexity naturally describes the "size" of the distribution, while the higher cumulants encode richer information. We further consider the double-scaled limit of SYK$_q$ at infinite temperature, where $q \sim \sqrt{N}$. In such a limit, we find that the scrambling time shrinks to zero, and the Lanczos coefficients diverge. The growth of Krylov complexity appears to be "hyperfast", which is previously conjectured to be associated with scrambling in de Sitter space.
△ Less
Submitted 17 August, 2023; v1 submitted 5 October, 2022;
originally announced October 2022.
-
Pushing the limits of fairness impossibility: Who's the fairest of them all?
Authors:
Brian Hsu,
Rahul Mazumder,
Preetam Nandy,
Kinjal Basu
Abstract:
The impossibility theorem of fairness is a foundational result in the algorithmic fairness literature. It states that outside of special cases, one cannot exactly and simultaneously satisfy all three common and intuitive definitions of fairness - demographic parity, equalized odds, and predictive rate parity. This result has driven most works to focus on solutions for one or two of the metrics. Ra…
▽ More
The impossibility theorem of fairness is a foundational result in the algorithmic fairness literature. It states that outside of special cases, one cannot exactly and simultaneously satisfy all three common and intuitive definitions of fairness - demographic parity, equalized odds, and predictive rate parity. This result has driven most works to focus on solutions for one or two of the metrics. Rather than follow suit, in this paper we present a framework that pushes the limits of the impossibility theorem in order to satisfy all three metrics to the best extent possible. We develop an integer-programming based approach that can yield a certifiably optimal post-processing method for simultaneously satisfying multiple fairness criteria under small violations. We show experiments demonstrating that our post-processor can improve fairness across the different definitions simultaneously with minimal model performance reduction. We also discuss applications of our framework for model selection and fairness explainability, thereby attempting to answer the question: who's the fairest of them all?
△ Less
Submitted 24 August, 2022;
originally announced August 2022.
-
Probing quantum scars and weak ergodicity-breaking through quantum complexity
Authors:
Budhaditya Bhattacharjee,
Samudra Sur,
Pratik Nandy
Abstract:
Scar states are special many-body eigenstates that weakly violate the eigenstate thermalization hypothesis (ETH). Using the explicit formalism of the Lanczos algorithm, usually known as the forward scattering approximation in this context, we compute the Krylov state (spread) complexity of typical states generated by the time evolution of the PXP Hamiltonian, hosting such states. We show that the…
▽ More
Scar states are special many-body eigenstates that weakly violate the eigenstate thermalization hypothesis (ETH). Using the explicit formalism of the Lanczos algorithm, usually known as the forward scattering approximation in this context, we compute the Krylov state (spread) complexity of typical states generated by the time evolution of the PXP Hamiltonian, hosting such states. We show that the complexity for the Neel state revives in an approximate sense, while complexity for the generic ETH-obeying state always increases. This can be attributed to the approximate SU(2) structure of the corresponding generators of the Hamiltonian. We quantify such ''closeness'' by the q-deformed SU(2) algebra and provide an analytic expression of Lanczos coefficients for the Neel state within the approximate Krylov subspace. We intuitively explain the results in terms of a tight-binding model. We further consider a deformation of the PXP Hamiltonian and compute the corresponding Lanczos coefficients and the complexity. We find that complexity for the Neel state shows nearly perfect revival while the same does not hold for a generic ETH-obeying state.
△ Less
Submitted 29 November, 2022; v1 submitted 10 August, 2022;
originally announced August 2022.
-
Operator growth and Krylov construction in dissipative open quantum systems
Authors:
Aranya Bhattacharya,
Pratik Nandy,
**al Pratyush Nath,
Himanshu Sahu
Abstract:
Inspired by the universal operator growth hypothesis, we extend the formalism of Krylov construction in dissipative open quantum systems connected to a Markovian bath. Our construction is based upon the modification of the Liouvillian superoperator by the appropriate Lindbladian, thereby following the vectorized Lanczos algorithm and the Arnoldi iteration. This is well justified due to the incorpo…
▽ More
Inspired by the universal operator growth hypothesis, we extend the formalism of Krylov construction in dissipative open quantum systems connected to a Markovian bath. Our construction is based upon the modification of the Liouvillian superoperator by the appropriate Lindbladian, thereby following the vectorized Lanczos algorithm and the Arnoldi iteration. This is well justified due to the incorporation of non-Hermitian effects due to the environment. We study the growth of Lanczos coefficients in the transverse field Ising model (integrable and chaotic limits) for boundary amplitude dam** and bulk dephasing. Although the direct implementation of the Lanczos algorithm fails to give physically meaningful results, the Arnoldi iteration retains the generic nature of the integrability and chaos as well as the signature of non-Hermiticity through separate sets of coefficients (Arnoldi coefficients) even after including the dissipative environment. Our results suggest that the Arnoldi iteration is meaningful and more appropriate in dealing with open systems.
△ Less
Submitted 3 December, 2022; v1 submitted 12 July, 2022;
originally announced July 2022.
-
Detection and Mitigation of Algorithmic Bias via Predictive Rate Parity
Authors:
Cyrus DiCiccio,
Brian Hsu,
YinYin Yu,
Preetam Nandy,
Kinjal Basu
Abstract:
Predictive parity (PP), also known as sufficiency, is a core definition of algorithmic fairness essentially stating that model outputs must have the same interpretation of expected outcomes regardless of group. Testing and satisfying PP is especially important in many settings where model scores are interpreted by humans or directly provide access to opportunity, such as healthcare or banking. Sol…
▽ More
Predictive parity (PP), also known as sufficiency, is a core definition of algorithmic fairness essentially stating that model outputs must have the same interpretation of expected outcomes regardless of group. Testing and satisfying PP is especially important in many settings where model scores are interpreted by humans or directly provide access to opportunity, such as healthcare or banking. Solutions for PP violations have primarily been studied through the lens of model calibration. However, we find that existing calibration-based tests and mitigation methods are designed for independent data, which is often not assumable in large-scale applications such as social media or medical testing. In this work, we address this issue by develo** a statistically rigorous non-parametric regression based test for PP with dependent observations. We then apply our test to illustrate that PP testing can significantly vary under the two assumptions. Lastly, we provide a mitigation solution to provide a minimally-biased post-processing transformation function to achieve PP.
△ Less
Submitted 30 May, 2023; v1 submitted 12 April, 2022;
originally announced April 2022.
-
Long-term Dynamics of Fairness Intervention in Connection Recommender Systems
Authors:
Nil-Jana Akpinar,
Cyrus DiCiccio,
Preetam Nandy,
Kinjal Basu
Abstract:
Recommender system fairness has been studied from the perspectives of a variety of stakeholders including content producers, the content itself and recipients of recommendations. Regardless of which type of stakeholders are considered, most works in this area assess the efficacy of fairness intervention by evaluating a single fixed fairness criterion through the lens of a one-shot, static setting.…
▽ More
Recommender system fairness has been studied from the perspectives of a variety of stakeholders including content producers, the content itself and recipients of recommendations. Regardless of which type of stakeholders are considered, most works in this area assess the efficacy of fairness intervention by evaluating a single fixed fairness criterion through the lens of a one-shot, static setting. Yet recommender systems constitute dynamical systems with feedback loops from the recommendations to the underlying population distributions which could lead to unforeseen and adverse consequences if not taken into account. In this paper, we study a connection recommender system patterned after the systems employed by web-scale social networks and analyze the long-term effects of intervening on fairness in the recommendations. We find that, although seemingly fair in aggregate, common exposure and utility parity interventions fail to mitigate amplification of biases in the long term. We theoretically characterize how certain fairness interventions impact the bias amplification dynamics in a stylized Pólya urn model.
△ Less
Submitted 20 September, 2022; v1 submitted 30 March, 2022;
originally announced March 2022.
-
Krylov complexity in saddle-dominated scrambling
Authors:
Budhaditya Bhattacharjee,
Xiangyu Cao,
Pratik Nandy,
Tanay Pathak
Abstract:
In semi-classical systems, the exponential growth of the out-of-timeorder correlator (OTOC) is believed to be the hallmark of quantum chaos. However,on several occasions, it has been argued that, even in integrable systems, OTOC can grow exponentially due to the presence of unstable saddle points in the phase space. In this work, we probe such an integrable system exhibiting saddle dominated scram…
▽ More
In semi-classical systems, the exponential growth of the out-of-timeorder correlator (OTOC) is believed to be the hallmark of quantum chaos. However,on several occasions, it has been argued that, even in integrable systems, OTOC can grow exponentially due to the presence of unstable saddle points in the phase space. In this work, we probe such an integrable system exhibiting saddle dominated scrambling through Krylov complexity and the associated Lanczos coefficients. In the realm of the universal operator growth hypothesis, we demonstrate that the Lanczos coefficients follow the linear growth, which ensures the exponential behavior of Krylov complexity at early times. The linear growth arises entirely due to the saddle, which dominates other phase-space points even away from itself. Our results reveal that the exponential growth of Krylov complexity can be observed in integrable systems with saddle-dominated scrambling and thus need not be associated with the presence of chaos.
△ Less
Submitted 5 June, 2022; v1 submitted 7 March, 2022;
originally announced March 2022.
-
Offline Reinforcement Learning for Mobile Notifications
Authors:
Yi** Yuan,
Ajith Muralidharan,
Preetam Nandy,
Miao Cheng,
Prakruthi Prabhakar
Abstract:
Mobile notification systems have taken a major role in driving and maintaining user engagement for online platforms. They are interesting recommender systems to machine learning practitioners with more sequential and long-term feedback considerations. Most machine learning applications in notification systems are built around response-prediction models, trying to attribute both short-term impact a…
▽ More
Mobile notification systems have taken a major role in driving and maintaining user engagement for online platforms. They are interesting recommender systems to machine learning practitioners with more sequential and long-term feedback considerations. Most machine learning applications in notification systems are built around response-prediction models, trying to attribute both short-term impact and long-term impact to a notification decision. However, a user's experience depends on a sequence of notifications and attributing impact to a single notification is not always accurate, if not impossible. In this paper, we argue that reinforcement learning is a better framework for notification systems in terms of performance and iteration speed. We propose an offline reinforcement learning framework to optimize sequential notification decisions for driving user engagement. We describe a state-marginalized importance sampling policy evaluation approach, which can be used to evaluate the policy offline and tune learning hyperparameters. Through simulations that approximate the notifications ecosystem, we demonstrate the performance and benefits of the offline evaluation approach as a part of the reinforcement learning modeling approach. Finally, we collect data through online exploration in the production system, train an offline Double Deep Q-Network and launch a successful policy online. We also discuss the practical considerations and results obtained by deploying these policies for a large-scale recommendation system use-case.
△ Less
Submitted 4 February, 2022;
originally announced February 2022.
-
Generalized Causal Tree for Uplift Modeling
Authors:
Preetam Nandy,
Xiufan Yu,
Wanjun Liu,
Ye Tu,
Kinjal Basu,
Shaunak Chatterjee
Abstract:
Uplift modeling is crucial in various applications ranging from marketing and policy-making to personalized recommendations. The main objective is to learn optimal treatment allocations for a heterogeneous population. A primary line of existing work modifies the loss function of the decision tree algorithm to identify cohorts with heterogeneous treatment effects. Another line of work estimates the…
▽ More
Uplift modeling is crucial in various applications ranging from marketing and policy-making to personalized recommendations. The main objective is to learn optimal treatment allocations for a heterogeneous population. A primary line of existing work modifies the loss function of the decision tree algorithm to identify cohorts with heterogeneous treatment effects. Another line of work estimates the individual treatment effects separately for the treatment group and the control group using off-the-shelf supervised learning algorithms. The former approach that directly models the heterogeneous treatment effect is known to outperform the latter in practice. However, the existing tree-based methods are mostly limited to a single treatment and a single control use case, except for a handful of extensions to multiple discrete treatments. In this paper, we propose a generalization of tree-based approaches to tackle multiple discrete and continuous-valued treatments. We focus on a generalization of the well-known causal tree algorithm due to its desirable statistical properties, but our generalization technique can be applied to other tree-based approaches as well. The efficacy of our proposed method is demonstrated using experiments and real data examples.
△ Less
Submitted 19 December, 2023; v1 submitted 4 February, 2022;
originally announced February 2022.
-
Balanced Partial Entanglement and Mixed State Correlations
Authors:
Hugo A. Camargo,
Pratik Nandy,
Qiang Wen,
Haocheng Zhong
Abstract:
Recently in Ref.\cite{Wen:2021qgx}, one of the authors introduced the balanced partial entanglement (BPE), which has been proposed to be dual to the entanglement wedge cross-section (EWCS). In this paper, we explicitly demonstrate that the BPE could be considered as a proper measure of the total intrinsic correlation between two subsystems in a mixed state. The total correlation includes certain c…
▽ More
Recently in Ref.\cite{Wen:2021qgx}, one of the authors introduced the balanced partial entanglement (BPE), which has been proposed to be dual to the entanglement wedge cross-section (EWCS). In this paper, we explicitly demonstrate that the BPE could be considered as a proper measure of the total intrinsic correlation between two subsystems in a mixed state. The total correlation includes certain crossing correlations which are minimized on some balance conditions. By constructing a class of purifications from Euclidean path-integrals, we find that the balanced crossing correlations show universality and can be considered as the generalization of the Markov gap for canonical purification. We also test the relation between the BPE and the EWCS in three-dimensional asymptotically flat holography. We find that the balanced crossing correlation vanishes for the field theory invariant under BMS$_3$ symmetry (BMSFT) and dual to the Einstein gravity, indicating the possibility of a perfect Markov recovery. We further elucidate these crossing correlations as a signature of tripartite entanglement and explain their interpretation in both AdS and non-AdS holography.
△ Less
Submitted 15 April, 2022; v1 submitted 31 January, 2022;
originally announced January 2022.
-
Q-curvature and Path Integral Complexity
Authors:
Hugo A. Camargo,
Pawel Caputa,
Pratik Nandy
Abstract:
We discuss the interpretation of path integral optimization as a uniformization problem in even dimensions. This perspective allows for a systematical construction of the higher-dimensional path integral complexity in holographic conformal field theories in terms of Q-curvature actions. We explore the properties and consequences of these actions from the perspective of the optimization programme,…
▽ More
We discuss the interpretation of path integral optimization as a uniformization problem in even dimensions. This perspective allows for a systematical construction of the higher-dimensional path integral complexity in holographic conformal field theories in terms of Q-curvature actions. We explore the properties and consequences of these actions from the perspective of the optimization programme, tensor networks and penalty factors. Moreover, in the context of recently proposed holographic path integral optimization, we consider higher curvature contributions on the Hartle-Hawking bulk slice and study their impact on the optimization as well as their relation to Q-curvature actions and finite cut-off holography.
△ Less
Submitted 15 April, 2022; v1 submitted 3 January, 2022;
originally announced January 2022.
-
Bath deformations, islands and holographic complexity
Authors:
Aranya Bhattacharya,
Arpan Bhattacharyya,
Pratik Nandy,
Ayan K. Patra
Abstract:
Considering a doubly holographic model, we study the evolution of holographic subregion complexity corresponding to deformations of bath state by a relevant scalar operator, which corresponds to a renormalization group flow from the AdS-Schwarzschild to the Kasner universe in the bulk. The subregion complexity shows a discontinuous jump at Page time at a fixed perturbation, where the discontinuity…
▽ More
Considering a doubly holographic model, we study the evolution of holographic subregion complexity corresponding to deformations of bath state by a relevant scalar operator, which corresponds to a renormalization group flow from the AdS-Schwarzschild to the Kasner universe in the bulk. The subregion complexity shows a discontinuous jump at Page time at a fixed perturbation, where the discontinuity depends solely on the system's parameters. We show that the amount of discontinuity decreases with the perturbation as well as with the scaling dimension of the relevant scalar operator.
△ Less
Submitted 15 December, 2021; v1 submitted 13 December, 2021;
originally announced December 2021.
-
Partial islands and subregion complexity in geometric secret-sharing model
Authors:
Aranya Bhattacharya,
Arpan Bhattacharyya,
Pratik Nandy,
Ayan K. Patra
Abstract:
We compute the holographic subregion complexity of a radiation subsystem in a geometric secret-sharing model of Hawking radiation in the "complexity = volume" proposal. The model is constructed using multiboundary wormhole geometries in AdS$_{3}$. The entanglement curve for secret-sharing captures a crossover between two minimal curves in the geometry apart from the usual eternal Page curve presen…
▽ More
We compute the holographic subregion complexity of a radiation subsystem in a geometric secret-sharing model of Hawking radiation in the "complexity = volume" proposal. The model is constructed using multiboundary wormhole geometries in AdS$_{3}$. The entanglement curve for secret-sharing captures a crossover between two minimal curves in the geometry apart from the usual eternal Page curve present for the complete radiation entanglement. We compute the complexity dual to the secret-sharing minimal surfaces and study their "time" evolution. When we have access to a small part of the radiation, the complexity shows a jump at the secret-sharing time larger than the Page time. Moreover, the minimal surfaces do not have access to the entire island region for this particular case. They can only access it partially. We describe this inaccessibility in the context of "classical" Markov recovery.
△ Less
Submitted 5 December, 2021; v1 submitted 16 September, 2021;
originally announced September 2021.
-
Eigenstate capacity and Page curve in fermionic Gaussian states
Authors:
Budhaditya Bhattacharjee,
Pratik Nandy,
Tanay Pathak
Abstract:
Capacity of entanglement (CoE), an information-theoretic measure of entanglement, defined as the variance of modular Hamiltonian, is known to capture the deviation from the maximal entanglement. We derive an exact expression for the average eigenstate CoE in fermionic Gaussian states as a finite series, valid for arbitrary bi-partition of the total system. Further, we consider the complex SYK$_2$…
▽ More
Capacity of entanglement (CoE), an information-theoretic measure of entanglement, defined as the variance of modular Hamiltonian, is known to capture the deviation from the maximal entanglement. We derive an exact expression for the average eigenstate CoE in fermionic Gaussian states as a finite series, valid for arbitrary bi-partition of the total system. Further, we consider the complex SYK$_2$ model in the thermodynamic limit and we obtain a closed-form expression of average CoE. In this limit, the variance of the average CoE becomes independent of the system size. Moreover, when the subsystem size is half of the total system, the leading volume-law coefficient approaches a value of $π^{2}/8 - 1$. We identify this as a distinguishing feature between integrable and quantum-chaotic systems. We confirm our analytical results by numerical computations.
△ Less
Submitted 8 December, 2021; v1 submitted 1 September, 2021;
originally announced September 2021.
-
A/B Testing for Recommender Systems in a Two-sided Marketplace
Authors:
Preetam Nandy,
Divya Venugopalan,
Chun Lo,
Shaunak Chatterjee
Abstract:
Two-sided marketplaces are standard business models of many online platforms (e.g., Amazon, Facebook, LinkedIn), wherein the platforms have consumers, buyers or content viewers on one side and producers, sellers or content-creators on the other. Consumer side measurement of the impact of a treatment variant can be done via simple online A/B testing. Producer side measurement is more challenging be…
▽ More
Two-sided marketplaces are standard business models of many online platforms (e.g., Amazon, Facebook, LinkedIn), wherein the platforms have consumers, buyers or content viewers on one side and producers, sellers or content-creators on the other. Consumer side measurement of the impact of a treatment variant can be done via simple online A/B testing. Producer side measurement is more challenging because the producer experience depends on the treatment assignment of the consumers. Existing approaches for producer side measurement are either based on graph cluster-based randomization or on certain treatment propagation assumptions. The former approach results in low-powered experiments as the producer-consumer network density increases and the latter approach lacks a strict notion of error control. In this paper, we propose (i) a quantification of the quality of a producer side experiment design, and (ii) a new experiment design mechanism that generates high-quality experiments based on this quantification. Our approach, called UniCoRn (Unifying Counterfactual Rankings), provides explicit control over the quality of the experiment and its computation cost. Further, we prove that our experiment design is optimal to the proposed design quality measure. Our approach is agnostic to the density of the producer-consumer network and does not rely on any treatment propagation assumption. Moreover, unlike the existing approaches, we do not need to know the underlying network in advance, making this widely applicable to the industrial setting where the underlying network is unknown and challenging to predict a priori due to its dynamic nature. We use simulations to validate our approach and compare it against existing methods. We also deployed UniCoRn in an edge recommendation application that serves tens of millions of members and billions of edge recommendations daily.
△ Less
Submitted 26 October, 2021; v1 submitted 28 May, 2021;
originally announced June 2021.
-
Capacity of Entanglement in Local Operators
Authors:
Pratik Nandy
Abstract:
We study the time evolution of the excess value of capacity of entanglement between a locally excited state and ground state in free, massless fermionic theory and free Yang-Mills theory in four spacetime dimensions. Capacity has non-trivial time evolution and is sensitive to the partial entanglement structure, and shows a universal peak at early times. We define a quantity, the normalized "Page t…
▽ More
We study the time evolution of the excess value of capacity of entanglement between a locally excited state and ground state in free, massless fermionic theory and free Yang-Mills theory in four spacetime dimensions. Capacity has non-trivial time evolution and is sensitive to the partial entanglement structure, and shows a universal peak at early times. We define a quantity, the normalized "Page time", which measures the timescale when capacity reaches its peak. This quantity turns out to be a characteristic property of the inserted operator. This firmly establishes capacity as a valuable measure of entanglement structure of an operator, especially at early times similar in spirit to the Renyi entropies at late times. Interestingly, the time evolution of capacity closely resembles its evolution in microcanonical and canonical ensemble of the replica wormhole model in the context of the black hole information paradox.
△ Less
Submitted 17 June, 2021; v1 submitted 1 June, 2021;
originally announced June 2021.
-
Islands and complexity of eternal black hole and radiation subsystems for a doubly holographic model
Authors:
Aranya Bhattacharya,
Arpan Bhattacharyya,
Pratik Nandy,
Ayan K. Patra
Abstract:
We study the entanglement islands and subsystem volume complexity corresponding to the left/ right entanglement of a conformal defect in $d$-dimensions in Randall-Sundrum (RS) braneworld model with subcritical tension brane. The left and right modes of the defect mimic the eternal black hole and radiation system respectively. Hence the entanglement entropy between the two follows an eternal black…
▽ More
We study the entanglement islands and subsystem volume complexity corresponding to the left/ right entanglement of a conformal defect in $d$-dimensions in Randall-Sundrum (RS) braneworld model with subcritical tension brane. The left and right modes of the defect mimic the eternal black hole and radiation system respectively. Hence the entanglement entropy between the two follows an eternal black hole Page curve which is unitarity compatible. We compute the volumes corresponding to the left and right branes with preferred Ryu-Takanayagi (RT) surfaces at different times, which provide a probe of the subregion complexity of the black hole and the radiation states respectively. An interesting jump in volume is found at Page time, where the entanglement curve is saturated due to the inclusion of the island surfaces. We explain various possibilities of this phase transition in complexity at Page time and argue how these results match with a covariant proposal qualitatively.
△ Less
Submitted 3 May, 2021; v1 submitted 29 March, 2021;
originally announced March 2021.
-
Achieving Fairness via Post-Processing in Web-Scale Recommender Systems
Authors:
Preetam Nandy,
Cyrus Diciccio,
Divya Venugopalan,
Heloise Logan,
Kinjal Basu,
Noureddine El Karoui
Abstract:
Building fair recommender systems is a challenging and crucial area of study due to its immense impact on society. We extended the definitions of two commonly accepted notions of fairness to recommender systems, namely equality of opportunity and equalized odds. These fairness measures ensure that equally "qualified" (or "unqualified") candidates are treated equally regardless of their protected a…
▽ More
Building fair recommender systems is a challenging and crucial area of study due to its immense impact on society. We extended the definitions of two commonly accepted notions of fairness to recommender systems, namely equality of opportunity and equalized odds. These fairness measures ensure that equally "qualified" (or "unqualified") candidates are treated equally regardless of their protected attribute status (such as gender or race). We propose scalable methods for achieving equality of opportunity and equalized odds in rankings in the presence of position bias, which commonly plagues data generated from recommender systems. Our algorithms are model agnostic in the sense that they depend only on the final scores provided by a model, making them easily applicable to virtually all web-scale recommender systems. We conduct extensive simulations as well as real-world experiments to show the efficacy of our approach.
△ Less
Submitted 11 August, 2022; v1 submitted 19 June, 2020;
originally announced June 2020.
-
Use of Artificial Intelligence to Analyse Risk in Legal Documents for a Better Decision Support
Authors:
Dipankar Chakrabarti,
Neelam Patodia,
Udayan Bhattacharya,
Indranil Mitra,
Satyaki Roy,
Jayanta Mandi,
Nandini Roy,
Prasun Nandy
Abstract:
Assessing risk for voluminous legal documents such as request for proposal; contracts is tedious and error prone. We have developed "risk-o-meter", a framework, based on machine learning and natural language processing to review and assess risks of any legal document. Our framework uses Paragraph Vector, an unsupervised model to generate vector representation of text. This enables the framework to…
▽ More
Assessing risk for voluminous legal documents such as request for proposal; contracts is tedious and error prone. We have developed "risk-o-meter", a framework, based on machine learning and natural language processing to review and assess risks of any legal document. Our framework uses Paragraph Vector, an unsupervised model to generate vector representation of text. This enables the framework to learn contextual relations of legal terms and generate sensible context aware embedding. The framework then feeds the vector space into a supervised classification algorithm to predict whether a paragraph belongs to a per-defined risk category or not. The framework thus extracts risk prone paragraphs. This technique efficiently overcomes the limitations of keyword-based search. We have achieved an accuracy of 91% for the risk category having the largest training dataset. This framework will help organizations optimize effort to identify risk from large document base with minimal human intervention and thus will help to have risk mitigated sustainable growth. Its machine learning capability makes it scalable to uncover relevant information from any type of document apart from legal documents, provided the library is per-populated and rich.
△ Less
Submitted 22 November, 2019;
originally announced December 2019.
-
Renormalized Circuit Complexity
Authors:
Arpan Bhattacharyya,
Pratik Nandy,
Aninda Sinha
Abstract:
We propose a modification to Nielsen's circuit complexity for Hamiltonian simulation using the Suzuki-Trotter (ST) method, which provides a network like structure for the quantum circuit. This leads to an optimized gate counting linear in the geodesic distance and spatial volume, unlike in the original proposal. The optimized ST iteration order is correlated with the error tolerance and plays the…
▽ More
We propose a modification to Nielsen's circuit complexity for Hamiltonian simulation using the Suzuki-Trotter (ST) method, which provides a network like structure for the quantum circuit. This leads to an optimized gate counting linear in the geodesic distance and spatial volume, unlike in the original proposal. The optimized ST iteration order is correlated with the error tolerance and plays the role of an anti-de Sitter (AdS) radial coordinate. The density of gates is shown to be monotonic with the tolerance and a holographic interpretation using path-integral optimization is given.
△ Less
Submitted 20 February, 2020; v1 submitted 18 July, 2019;
originally announced July 2019.
-
Optimal Convergence for Stochastic Optimization with Multiple Expectation Constraints
Authors:
Kinjal Basu,
Preetam Nandy
Abstract:
In this paper, we focus on the problem of stochastic optimization where the objective function can be written as an expectation function over a closed convex set. We also consider multiple expectation constraints which restrict the domain of the problem. We extend the cooperative stochastic approximation algorithm from Lan and Zhou [2016] to solve the particular problem. We close the gaps in the p…
▽ More
In this paper, we focus on the problem of stochastic optimization where the objective function can be written as an expectation function over a closed convex set. We also consider multiple expectation constraints which restrict the domain of the problem. We extend the cooperative stochastic approximation algorithm from Lan and Zhou [2016] to solve the particular problem. We close the gaps in the previous analysis and provide a novel proof technique to show that our algorithm attains the optimal rate of convergence for both optimality gap and constraint violation when the functions are generally convex. We also compare our algorithm empirically to the state-of-the-art and show improved convergence in many situations.
△ Less
Submitted 15 June, 2019; v1 submitted 8 June, 2019;
originally announced June 2019.
-
Personalized Treatment Selection using Causal Heterogeneity
Authors:
Ye Tu,
Kinjal Basu,
Cyrus DiCiccio,
Romil Bansal,
Preetam Nandy,
Padmini Jaikumar,
Shaunak Chatterjee
Abstract:
Randomized experimentation (also known as A/B testing or bucket testing) is widely used in the internet industry to measure the metric impact obtained by different treatment variants. A/B tests identify the treatment variant showing the best performance, which then becomes the chosen or selected treatment for the entire population. However, the effect of a given treatment can differ across experim…
▽ More
Randomized experimentation (also known as A/B testing or bucket testing) is widely used in the internet industry to measure the metric impact obtained by different treatment variants. A/B tests identify the treatment variant showing the best performance, which then becomes the chosen or selected treatment for the entire population. However, the effect of a given treatment can differ across experimental units and a personalized approach for treatment selection can greatly improve upon the usual global selection strategy. In this work, we develop a framework for personalization through (i) estimation of heterogeneous treatment effect at either a cohort or member-level, followed by (ii) selection of optimal treatment variants for cohorts (or members) obtained through (deterministic or stochastic) constrained optimization.
We perform a two-fold evaluation of our proposed methods. First, a simulation analysis is conducted to study the effect of personalized treatment selection under carefully controlled settings. This simulation illustrates the differences between the proposed methods and the suitability of each with increasing uncertainty. We also demonstrate the effectiveness of the method through a real-life example related to serving notifications at Linkedin. The solution significantly outperformed both heuristic solutions and the global treatment selection baseline leading to a sizable win on top-line metrics like member visits.
△ Less
Submitted 21 December, 2020; v1 submitted 29 January, 2019;
originally announced January 2019.
-
A/B Testing in Dense Large-Scale Networks: Design and Inference
Authors:
Preetam Nandy,
Kinjal Basu,
Shaunak Chatterjee,
Ye Tu
Abstract:
Design of experiments and estimation of treatment effects in large-scale networks, in the presence of strong interference, is a challenging and important problem. Most existing methods' performance deteriorates as the density of the network increases. In this paper, we present a novel strategy for accurately estimating the causal effects of a class of treatments in a dense large-scale network. Fir…
▽ More
Design of experiments and estimation of treatment effects in large-scale networks, in the presence of strong interference, is a challenging and important problem. Most existing methods' performance deteriorates as the density of the network increases. In this paper, we present a novel strategy for accurately estimating the causal effects of a class of treatments in a dense large-scale network. First, we design an approximate randomized controlled experiment by solving an optimization problem to allocate treatments in the presence of competition among neighboring nodes. Then we apply an importance sampling adjustment to correct for any leftover bias (from the approximation) in estimating average treatment effects. We provide theoretical guarantees, verify robustness in a simulation study, and validate the scalability and usefulness of our procedure in a real-world experiment on a large social network.
△ Less
Submitted 13 December, 2020; v1 submitted 29 January, 2019;
originally announced January 2019.
-
Inference for Individual Mediation Effects and Interventional Effects in Sparse High-Dimensional Causal Graphical Models
Authors:
Abhishek Chakrabortty,
Preetam Nandy,
Hongzhe Li
Abstract:
We consider the problem of identifying intermediate variables (or mediators) that regulate the effect of a treatment on a response variable. While there has been significant research on this classical topic, little work has been done when the set of potential mediators is high-dimensional (HD). A further complication arises when these mediators are interrelated (with unknown dependencies). In part…
▽ More
We consider the problem of identifying intermediate variables (or mediators) that regulate the effect of a treatment on a response variable. While there has been significant research on this classical topic, little work has been done when the set of potential mediators is high-dimensional (HD). A further complication arises when these mediators are interrelated (with unknown dependencies). In particular, we assume that the causal structure of the treatment, the confounders, the potential mediators and the response is a (possibly unknown) directed acyclic graph (DAG). HD DAG models have previously been used for the estimation of causal effects from observational data. In particular, methods called IDA and joint-IDA have been developed for estimating the effects of single and multiple simultaneous interventions, respectively. In this paper, we propose an IDA-type method called MIDA for estimating so-called individual mediation effects from HD observational data. Although IDA and joint-IDA estimators have been shown to be consistent in certain sparse HD settings, their asymptotic properties such as convergence in distribution and inferential tools in such settings have remained unknown. In this paper, we prove HD consistency of MIDA for linear structural equation models with sub-Gaussian errors. More importantly, we derive distributional convergence results for MIDA in similar HD settings, which are applicable to IDA and joint-IDA estimators as well. To our knowledge, these are the first such distributional convergence results facilitating inference for IDA-type estimators. These are built on our novel theoretical results regarding uniform bounds for linear regression estimators over varying subsets of HD covariates which may be of independent interest. Finally, we empirically validate our asymptotic theory for MIDA and demonstrate its usefulness via simulations and a real data application.
△ Less
Submitted 28 July, 2021; v1 submitted 27 September, 2018;
originally announced September 2018.
-
Robust causal structure learning with some hidden variables
Authors:
Benjamin Frot,
Preetam Nandy,
Marloes H. Maathuis
Abstract:
We introduce a new method to estimate the Markov equivalence class of a directed acyclic graph (DAG) in the presence of hidden variables, in settings where the underlying DAG among the observed variables is sparse, and there are a few hidden variables that have a direct effect on many of the observed ones. Building on the so-called low rank plus sparse framework, we suggest a two-stage approach wh…
▽ More
We introduce a new method to estimate the Markov equivalence class of a directed acyclic graph (DAG) in the presence of hidden variables, in settings where the underlying DAG among the observed variables is sparse, and there are a few hidden variables that have a direct effect on many of the observed ones. Building on the so-called low rank plus sparse framework, we suggest a two-stage approach which first removes the effect of the hidden variables, and then estimates the Markov equivalence class of the underlying DAG under the assumption that there are no remaining hidden variables. This approach is consistent in certain high-dimensional regimes and performs favourably when compared to the state of the art, both in terms of graphical structure recovery and total causal effect estimation.
△ Less
Submitted 4 August, 2018; v1 submitted 3 August, 2017;
originally announced August 2017.
-
Structure Learning of Linear Gaussian Structural Equation Models with Weak Edges
Authors:
Marco F. Eigenmann,
Preetam Nandy,
Marloes H. Maathuis
Abstract:
We consider structure learning of linear Gaussian structural equation models with weak edges. Since the presence of weak edges can lead to a loss of edge orientations in the true underlying CPDAG, we define a new graphical object that can contain more edge orientations. We show that this object can be recovered from observational data under a type of strong faithfulness assumption. We present a ne…
▽ More
We consider structure learning of linear Gaussian structural equation models with weak edges. Since the presence of weak edges can lead to a loss of edge orientations in the true underlying CPDAG, we define a new graphical object that can contain more edge orientations. We show that this object can be recovered from observational data under a type of strong faithfulness assumption. We present a new algorithm for this purpose, called aggregated greedy equivalence search (AGES), that aggregates the solution path of the greedy equivalence search (GES) algorithm for varying values of the penalty parameter. We prove consistency of AGES and demonstrate its performance in a simulation study and on single cell data from Sachs et al. (2005). The algorithm will be made available in the R-package pcalg.
△ Less
Submitted 24 July, 2017;
originally announced July 2017.
-
Large-Sample Theory for the Bergsma-Dassios Sign Covariance
Authors:
Preetam Nandy,
Luca Weihs,
Mathias Drton
Abstract:
The Bergsma-Dassios sign covariance is a recently proposed extension of Kendall's tau. In contrast to tau or also Spearman's rho, the new sign covariance $τ^*$ vanishes if and only if the two considered random variables are independent. Specifically, this result has been shown for continuous as well as discrete variables. We develop large-sample distribution theory for the empirical version of…
▽ More
The Bergsma-Dassios sign covariance is a recently proposed extension of Kendall's tau. In contrast to tau or also Spearman's rho, the new sign covariance $τ^*$ vanishes if and only if the two considered random variables are independent. Specifically, this result has been shown for continuous as well as discrete variables. We develop large-sample distribution theory for the empirical version of $τ^*$. In particular, we use theory for degenerate U-statistics to derive asymptotic null distributions under independence and demonstrate in simulations that the limiting distributions give useful approximations.
△ Less
Submitted 13 February, 2016;
originally announced February 2016.
-
High-dimensional consistency in score-based and hybrid structure learning
Authors:
Preetam Nandy,
Alain Hauser,
Marloes H. Maathuis
Abstract:
Main approaches for learning Bayesian networks can be classified as constraint-based, score-based or hybrid methods. Although high-dimensional consistency results are available for constraint-based methods like the PC algorithm, such results have not been proved for score-based or hybrid methods, and most of the hybrid methods have not even shown to be consistent in the classical setting where the…
▽ More
Main approaches for learning Bayesian networks can be classified as constraint-based, score-based or hybrid methods. Although high-dimensional consistency results are available for constraint-based methods like the PC algorithm, such results have not been proved for score-based or hybrid methods, and most of the hybrid methods have not even shown to be consistent in the classical setting where the number of variables remains fixed and the sample size tends to infinity. In this paper, we show that consistency of hybrid methods based on greedy equivalence search (GES) can be achieved in the classical setting with adaptive restrictions on the search space that depend on the current state of the algorithm. Moreover, we prove consistency of GES and adaptively restricted GES (ARGES) in several sparse high-dimensional settings. ARGES scales well to sparse graphs with thousands of variables and our simulation study indicates that both GES and ARGES generally outperform the PC algorithm.
△ Less
Submitted 3 February, 2018; v1 submitted 9 July, 2015;
originally announced July 2015.
-
A review of some recent advances in causal inference
Authors:
Marloes H. Maathuis,
Preetam Nandy
Abstract:
We give a selective review of some recent developments in causal inference, intended for researchers who are not familiar with graphical models and causality, and with a focus on methods that are applicable to large data sets. We mainly address the problem of estimating causal effects from observational data. For example, one can think of estimating the effect of single or multiple gene knockouts…
▽ More
We give a selective review of some recent developments in causal inference, intended for researchers who are not familiar with graphical models and causality, and with a focus on methods that are applicable to large data sets. We mainly address the problem of estimating causal effects from observational data. For example, one can think of estimating the effect of single or multiple gene knockouts from wild-type gene expression data, that is, from gene expression measurements that were obtained without doing any gene knockout experiments.
We assume that the observational data are generated from a causal structure that can be represented by a directed acyclic graph (DAG). First, we discuss estimation of causal effects when the underlying causal DAG is known. In large-scale networks, however, the causal DAG is often unknown. Next, we therefore discuss causal structure learning, that is, learning information about the causal structure from observational data. We then combine these two parts and discuss methods to estimate (bounds on) causal effects from observational data when the causal structure is unknown. We also illustrate this method on a yeast gene expression data set. We close by mentioning several extensions of the discussed work.
△ Less
Submitted 25 June, 2015;
originally announced June 2015.
-
Estimating the effect of joint interventions from observational data in sparse high-dimensional settings
Authors:
Preetam Nandy,
Marloes H. Maathuis,
Thomas S. Richardson
Abstract:
We consider the estimation of joint causal effects from observational data. In particular, we propose new methods to estimate the effect of multiple simultaneous interventions (e.g., multiple gene knockouts), under the assumption that the observational data come from an unknown linear structural equation model with independent errors. We derive asymptotic variances of our estimators when the under…
▽ More
We consider the estimation of joint causal effects from observational data. In particular, we propose new methods to estimate the effect of multiple simultaneous interventions (e.g., multiple gene knockouts), under the assumption that the observational data come from an unknown linear structural equation model with independent errors. We derive asymptotic variances of our estimators when the underlying causal structure is partly known, as well as high-dimensional consistency when the causal structure is fully unknown and the joint distribution is multivariate Gaussian. We also propose a generalization of our methodology to the class of nonparanormal distributions. We evaluate the estimators in simulation studies and also illustrate them on data from the DREAM4 challenge.
△ Less
Submitted 9 March, 2016; v1 submitted 9 July, 2014;
originally announced July 2014.
