Showing 1–46 of 46 results for author: Štefankovič, D

Lecture Notes on Spectral Independence and Bases of a Matroid: Local-to-Global and Trickle-Down from a Markov Chain Perspective

Authors: Daniel Stefankovic, Eric Vigoda

Abstract: These are self-contained lecture notes for spectral independence. For an $n$-vertex graph, the spectral independence condition is a bound on the maximum eigenvalue of the $n\times n$ influence matrix whose entries capture the influence between pairs of vertices, it is closely related to the covariance matrix. We will present recent results showing that spectral independence implies the mixing time… ▽ More These are self-contained lecture notes for spectral independence. For an $n$-vertex graph, the spectral independence condition is a bound on the maximum eigenvalue of the $n\times n$ influence matrix whose entries capture the influence between pairs of vertices, it is closely related to the covariance matrix. We will present recent results showing that spectral independence implies the mixing time of the Glauber dynamics is polynomial (where the degree of the polynomial depends on certain parameters). The proof utilizes local-to-global theorems which we will detail in these notes. Finally, we will present more recent results showing that spectral independence implies an optimal bound on the relaxation time (inverse spectral gap) and with some additional conditions implies an optimal mixing time bound of $O(n\log{n})$ for the Glauber dynamics. We also present the results of Anari, Liu, Oveis Gharan, and Vinzant (2019) for generating a random basis of a matroid. The analysis of the associated bases-exchange walk utilizes the local-to-global theorems used for spectral independence with the Trickle-Down Theorem of Oppenheim (2018) to analyze the local walks. Our focus in these notes is on the analysis of the spectral gap of the associated Markov chains from a functional analysis perspective, and we present proofs of the associated local-to-global theorems from this same Markov chain perspective. △ Less

Submitted 14 December, 2023; v1 submitted 25 July, 2023; originally announced July 2023.

Comments: Small corrections

Optimal Mixing via Tensorization for Random Independent Sets on Arbitrary Trees

Authors: Charilaos Efthymiou, Thomas P. Hayes, Daniel Stefankovic, Eric Vigoda

Abstract: We study the mixing time of the single-site update Markov chain, known as the Glauber dynamics, for generating a random independent set of a tree. Our focus is obtaining optimal convergence results for arbitrary trees. We consider the more general problem of sampling from the Gibbs distribution in the hard-core model where independent sets are weighted by a parameter $λ>0$; the special case $λ=1$… ▽ More We study the mixing time of the single-site update Markov chain, known as the Glauber dynamics, for generating a random independent set of a tree. Our focus is obtaining optimal convergence results for arbitrary trees. We consider the more general problem of sampling from the Gibbs distribution in the hard-core model where independent sets are weighted by a parameter $λ>0$; the special case $λ=1$ corresponds to the uniform distribution over all independent sets. Previous work of Martinelli, Sinclair and Weitz (2004) obtained optimal mixing time bounds for the complete $Δ$-regular tree for all $λ$. However, Restrepo et al. (2014) showed that for sufficiently large $λ$ there are bounded-degree trees where optimal mixing does not hold. Recent work of Eppstein and Frishberg (2022) proved a polynomial mixing time bound for the Glauber dynamics for arbitrary trees, and more generally for graphs of bounded tree-width. We establish an optimal bound on the relaxation time (i.e., inverse spectral gap) of $O(n)$ for the Glauber dynamics for unweighted independent sets on arbitrary trees. We stress that our results hold for arbitrary trees and there is no dependence on the maximum degree $Δ$. Interestingly, our results extend (far) beyond the uniqueness threshold which is on the order $λ=O(1/Δ)$. Our proof approach is inspired by recent work on spectral independence. In fact, we prove that spectral independence holds with a constant independent of the maximum degree for any tree, but this does not imply mixing for general trees as the optimal mixing results of Chen, Liu, and Vigoda (2021) only apply for bounded degree graphs. We instead utilize the combinatorial nature of independent sets to directly prove approximate tensorization of variance via a non-trivial inductive proof. △ Less

Submitted 18 February, 2024; v1 submitted 15 July, 2023; originally announced July 2023.

Comments: The optimum mixing result (Theorem 1.2) of version 1 of the manuscript has been removed due to an error

Beyond the Existential Theory of the Reals

Authors: Marcus Schaefer, Daniel Stefankovic

Abstract: We show that completeness at higher levels of the theory of the reals is a robust notion (under changing the signature and bounding the domain of the quantifiers). This mends recognized gaps in the hierarchy, and leads to stronger completeness results for various computational problems. We exhibit several families of complete problems which can be used for future completeness results in the real h… ▽ More We show that completeness at higher levels of the theory of the reals is a robust notion (under changing the signature and bounding the domain of the quantifiers). This mends recognized gaps in the hierarchy, and leads to stronger completeness results for various computational problems. We exhibit several families of complete problems which can be used for future completeness results in the real hierarchy. As an application we sharpen some results by Bürgisser and Cucker on the complexity of properties of semialgebraic sets, including the Hausdorff distance problem also studied by Jungeblut, Kleist, and Miltzow. △ Less

Submitted 17 February, 2024; v1 submitted 2 October, 2022; originally announced October 2022.

MSC Class: 68Q15; 68Q17; 14P10

Complexity of High-Dimensional Identity Testing with Coordinate Conditional Sampling

Authors: Antonio Blanca, Zongchen Chen, Daniel Štefankovič, Eric Vigoda

Abstract: We study the identity testing problem for high-dimensional distributions. Given as input an explicit distribution $μ$, an $\varepsilon>0$, and access to sampling oracle(s) for a hidden distribution $π$, the goal in identity testing is to distinguish whether the two distributions $μ$ and $π$ are identical or are at least $\varepsilon$-far apart. When there is only access to full samples from the hi… ▽ More We study the identity testing problem for high-dimensional distributions. Given as input an explicit distribution $μ$, an $\varepsilon>0$, and access to sampling oracle(s) for a hidden distribution $π$, the goal in identity testing is to distinguish whether the two distributions $μ$ and $π$ are identical or are at least $\varepsilon$-far apart. When there is only access to full samples from the hidden distribution $π$, it is known that exponentially many samples (in the dimension) may be needed for identity testing, and hence previous works have studied identity testing with additional access to various "conditional" sampling oracles. We consider a significantly weaker conditional sampling oracle, which we call the $\mathsf{Coordinate\ Oracle}$, and provide a computational and statistical characterization of the identity testing problem in this new model. We prove that if an analytic property known as approximate tensorization of entropy holds for an $n$-dimensional visible distribution $μ$, then there is an efficient identity testing algorithm for any hidden distribution $π$ using $\tilde{O}(n/\varepsilon)$ queries to the $\mathsf{Coordinate\ Oracle}$. Approximate tensorization of entropy is a pertinent condition as recent works have established it for a large class of high-dimensional distributions. We also prove a computational phase transition: for a well-studied class of $n$-dimensional distributions, specifically sparse antiferromagnetic Ising models over $\{+1,-1\}^n$, we show that in the regime where approximate tensorization of entropy fails, there is no efficient identity testing algorithm unless $\mathsf{RP}=\mathsf{NP}$. We complement our results with a matching $Ω(n/\varepsilon)$ statistical lower bound for the sample complexity of identity testing in the $\mathsf{Coordinate\ Oracle}$ model. △ Less

Submitted 7 November, 2022; v1 submitted 19 July, 2022; originally announced July 2022.

Approximating observables is as hard as counting

Authors: Andreas Galanis, Daniel Stefankovic, Eric Vigoda

Abstract: We study the computational complexity of estimating local observables for Gibbs distributions. A simple combinatorial example is the average size of an independent set in a graph. In a recent work, we established NP-hardness of approximating the average size of an independent set utilizing hardness of the corresponding optimization problem and the related phase transition behavior. Here, we instea… ▽ More We study the computational complexity of estimating local observables for Gibbs distributions. A simple combinatorial example is the average size of an independent set in a graph. In a recent work, we established NP-hardness of approximating the average size of an independent set utilizing hardness of the corresponding optimization problem and the related phase transition behavior. Here, we instead consider settings where the underlying optimization problem is easily solvable. Our main contribution is to classify the complexity of approximating a wide class of observables via a generic reduction from approximate counting to the problem of estimating local observables. The key idea is to use the observables to interpolate the counting problem. Using this new approach, we are able to study observables on bipartite graphs where the underlying optimization problem is easy but the counting problem is believed to be hard. The most-well studied class of graphs that was excluded from previous hardness results were bipartite graphs. We establish hardness for estimating the average size of the independent set in bipartite graphs of maximum degree 6; more generally, we show tight hardness results for general vertex-edge observables for antiferromagnetic 2-spin systems on bipartite graphs. Our techniques go beyond 2-spin systems, and for the ferromagnetic Potts model we establish hardness of approximating the number of monochromatic edges in the same region as known hardness of approximate counting results. △ Less

Submitted 23 June, 2022; originally announced June 2022.

Spiraling and Folding: The Topological View

Authors: Jan Kynčl, Marcus Schaefer, Eric Sedgwick, Daniel Štefankovič

Abstract: For every $n$, we construct two curves in the plane that intersect at least $n$ times and do not form spirals. The construction is in three stages: we first exhibit closed curves on the torus that do not form double spirals, then arcs on the torus that do not form spirals, and finally pairs of planar arcs that do not form spirals. These curves provide a counterexample to a proof of Pach and Tóth c… ▽ More For every $n$, we construct two curves in the plane that intersect at least $n$ times and do not form spirals. The construction is in three stages: we first exhibit closed curves on the torus that do not form double spirals, then arcs on the torus that do not form spirals, and finally pairs of planar arcs that do not form spirals. These curves provide a counterexample to a proof of Pach and Tóth concerning string graphs. △ Less

Submitted 15 June, 2022; originally announced June 2022.

Comments: 23 pages, 18 figures

MSC Class: 57M15; 57M50; 57N35; 68R10; 68U05

Journal ref: Discrete and Computational Geometry 72 (2024), no. 1, 246-268

Metastability of the Potts ferromagnet on random regular graphs

Authors: Amin Coja-Oghlan, Andreas Galanis, Leslie Ann Goldberg, Jean Bernoulli Ravelomanana, Daniel Stefankovic, Eric Vigoda

Abstract: We study the performance of Markov chains for the $q$-state ferromagnetic Potts model on random regular graphs. It is conjectured that their performance is dictated by metastability phenomena, i.e., the presence of "phases" (clusters) in the sample space where Markov chains with local update rules, such as the Glauber dynamics, are bound to take exponential time to escape. The phases that are beli… ▽ More We study the performance of Markov chains for the $q$-state ferromagnetic Potts model on random regular graphs. It is conjectured that their performance is dictated by metastability phenomena, i.e., the presence of "phases" (clusters) in the sample space where Markov chains with local update rules, such as the Glauber dynamics, are bound to take exponential time to escape. The phases that are believed to drive these metastability phenomena in the case of the Potts model emerge as local, rather than global, maxima of the so-called Bethe functional, and previous approaches of analysing these phases based on optimisation arguments fall short of the task. Our first contribution is to detail the emergence of the metastable phases for the $q$-state Potts model on the $d$-regular random graph for all integers $q,d\geq 3$, and establish that for an interval of temperatures, which is delineated by the uniqueness and a broadcasting threshold on the $d$-regular tree, the two phases coexist. The proofs are based on a conceptual connection between spatial properties and the structure of the Potts distribution on the random regular graph, rather than complicated moment calculations. Based on this new structural understanding of the model, we obtain various algorithmic consequences. We first complement recent fast mixing results for Glauber dynamics by Blanca and Gheissari below the uniqueness threshold, showing an exponential lower bound on the mixing time above the uniqueness threshold. Then, we obtain tight results even for the non-local Swendsen-Wang chain, where we establish slow mixing/metastability for the whole interval of temperatures where the chain is conjectured to mix slowly on the random regular graph. The key is to bound the conductance of the chains using a random graph "planting" argument combined with delicate bounds on random-graph percolation. △ Less

Submitted 10 January, 2023; v1 submitted 11 February, 2022; originally announced February 2022.

Comments: Abstract shortened for arXiv. To appear in Communications in Mathematical Physics (CIMP)

Fast sampling via spectral independence beyond bounded-degree graphs

Authors: Ivona Bezáková, Andreas Galanis, Leslie Ann Goldberg, Daniel Štefankovič

Abstract: Spectral independence is a recently-developed framework for obtaining sharp bounds on the convergence time of the classical Glauber dynamics. This new framework has yielded optimal $O(n \log n)$ sampling algorithms on bounded-degree graphs for a large class of problems throughout the so-called uniqueness regime, including, for example, the problems of sampling independent sets, matchings, and Isin… ▽ More Spectral independence is a recently-developed framework for obtaining sharp bounds on the convergence time of the classical Glauber dynamics. This new framework has yielded optimal $O(n \log n)$ sampling algorithms on bounded-degree graphs for a large class of problems throughout the so-called uniqueness regime, including, for example, the problems of sampling independent sets, matchings, and Ising-model configurations. Our main contribution is to relax the bounded-degree assumption that has so far been important in establishing and applying spectral independence. Previous methods for avoiding degree bounds rely on using $L^p$-norms to analyse contraction on graphs with bounded connective constant (Sinclair, Srivastava, Yin; FOCS'13). The non-linearity of $L^p$-norms is an obstacle to applying these results to bound spectral independence. Our solution is to capture the $L^p$-analysis recursively by amortising over the subtrees of the recurrence used to analyse contraction. Our method generalises previous analyses that applied only to bounded-degree graphs. As a main application of our techniques, we consider the random graph $G(n,d/n)$, where the previously known algorithms run in time $n^{O(\log d)}$ or applied only to large $d$. We refine these algorithmic bounds significantly, and develop fast $n^{1+o(1)}$ algorithms based on Glauber dynamics that apply to all $d$, throughout the uniqueness regime. △ Less

Submitted 13 October, 2023; v1 submitted 7 November, 2021; originally announced November 2021.

Comments: TALG, To Appear

Sampling Colorings and Independent Sets of Random Regular Bipartite Graphs in the Non-Uniqueness Region

Authors: Zongchen Chen, Andreas Galanis, Daniel Štefankovič, Eric Vigoda

Abstract: For spin systems, such as the $q$-colorings and independent-set models, approximating the partition function in the so-called non-uniqueness region, where the model exhibits long-range correlations, is typically computationally hard for bounded-degree graphs. We present new algorithmic results for approximating the partition function and sampling from the Gibbs distribution for spin systems in the… ▽ More For spin systems, such as the $q$-colorings and independent-set models, approximating the partition function in the so-called non-uniqueness region, where the model exhibits long-range correlations, is typically computationally hard for bounded-degree graphs. We present new algorithmic results for approximating the partition function and sampling from the Gibbs distribution for spin systems in the non-uniqueness region on random regular bipartite graphs. We give an $\mathsf{FPRAS}$ for counting $q$-colorings for even $q=O\big(\tfracΔ{\logΔ}\big)$ on almost every $Δ$-regular bipartite graph. This is within a factor $O(\logΔ)$ of the sampling algorithm for general graphs in the uniqueness region and improves significantly upon the previous best bound of $q=O\big(\tfrac{\sqrtΔ}{(\logΔ)^2}\big)$ by Jenssen, Keevash, and Perkins (SODA'19). Analogously, for the hard-core model on independent sets weighted by $λ>0$, we present an $\mathsf{FPRAS}$ for estimating the partition function when $λ=Ω\big(\tfrac{\logΔ}Δ\big)$, which improves upon previous results by an $Ω(\log Δ)$ factor. Our results for the colorings and hard-core models follow from a general result that applies to arbitrary spin systems. Our main contribution is to show how to elevate probabilistic/analytic bounds on the marginal probabilities for the typical structure of phases on random bipartite regular graphs into efficient algorithms, using the polymer method. We further show evidence that our result for colorings is within a constant factor of best possible using current polymer-method approaches. △ Less

Submitted 4 May, 2021; originally announced May 2021.

On Mixing of Markov Chains: Coupling, Spectral Independence, and Entropy Factorization

Authors: Antonio Blanca, Pietro Caputo, Zongchen Chen, Daniel Parisi, Daniel Štefankovič, Eric Vigoda

Abstract: For general spin systems, we prove that a contractive coupling for any local Markov chain implies optimal bounds on the mixing time and the modified log-Sobolev constant for a large class of Markov chains including the Glauber dynamics, arbitrary heat-bath block dynamics, and the Swendsen-Wang dynamics. This reveals a novel connection between probabilistic techniques for bounding the convergence t… ▽ More For general spin systems, we prove that a contractive coupling for any local Markov chain implies optimal bounds on the mixing time and the modified log-Sobolev constant for a large class of Markov chains including the Glauber dynamics, arbitrary heat-bath block dynamics, and the Swendsen-Wang dynamics. This reveals a novel connection between probabilistic techniques for bounding the convergence to stationarity and analytic tools for analyzing the decay of relative entropy. As a corollary of our general results, we obtain $O(n\log{n})$ mixing time and $Ω(1/n)$ modified log-Sobolev constant of the Glauber dynamics for sampling random $q$-colorings of an $n$-vertex graph with constant maximum degree $Δ$ when $q > (11/6 - ε_0)Δ$ for some fixed $ε_0>0$. We also obtain $O(\log{n})$ mixing time and $Ω(1)$ modified log-Sobolev constant of the Swendsen-Wang dynamics for the ferromagnetic Ising model on an $n$-vertex graph of constant maximum degree when the parameters of the system lie in the tree uniqueness region. At the heart of our results are new techniques for establishing spectral independence of the spin system and block factorization of the relative entropy. On one hand we prove that a contractive coupling of a local Markov chain implies spectral independence of the Gibbs distribution. On the other hand we show that spectral independence implies factorization of entropy for arbitrary blocks, establishing optimal bounds on the modified log-Sobolev constant of the corresponding block dynamics. △ Less

Submitted 12 March, 2021; originally announced March 2021.

The Swendsen-Wang Dynamics on Trees

Authors: Antonio Blanca, Zongchen Chen, Daniel Štefankovič, Eric Vigoda

Abstract: The Swendsen-Wang algorithm is a sophisticated, widely-used Markov chain for sampling from the Gibbs distribution for the ferromagnetic Ising and Potts models. This chain has proved difficult to analyze, due in part to the global nature of its updates. We present optimal bounds on the convergence rate of the Swendsen-Wang algorithm for the complete $d$-ary tree. Our bounds extend to the non-unique… ▽ More The Swendsen-Wang algorithm is a sophisticated, widely-used Markov chain for sampling from the Gibbs distribution for the ferromagnetic Ising and Potts models. This chain has proved difficult to analyze, due in part to the global nature of its updates. We present optimal bounds on the convergence rate of the Swendsen-Wang algorithm for the complete $d$-ary tree. Our bounds extend to the non-uniqueness region and apply to all boundary conditions. We show that the spatial mixing conditions known as Variance Mixing and Entropy Mixing, introduced in the study of local Markov chains by Martinelli et al. (2003), imply $Ω(1)$ spectral gap and $O(\log{n})$ mixing time, respectively, for the Swendsen-Wang dynamics on the $d$-ary tree. We also show that these bounds are asymptotically optimal. As a consequence, we establish $Θ(\log{n})$ mixing for the Swendsen-Wang dynamics for all boundary conditions throughout the tree uniqueness region; in fact, our bounds hold beyond the uniqueness threshold for the Ising model, and for the $q$-state Potts model when $q$ is small with respect to $d$. Our proofs feature a novel spectral view of the Variance Mixing condition inspired by several recent rapid mixing results on high-dimensional expanders and utilize recent work on block factorization of entropy under spatial mixing conditions. △ Less

Submitted 10 May, 2021; v1 submitted 15 July, 2020; originally announced July 2020.

Rapid Mixing for Colorings via Spectral Independence

Authors: Zongchen Chen, Andreas Galanis, Daniel Štefankovič, Eric Vigoda

Abstract: The spectral independence approach of Anari et al. (2020) utilized recent results on high-dimensional expanders of Alev and Lau (2020) and established rapid mixing of the Glauber dynamics for the hard-core model defined on weighted independent sets. We develop the spectral independence approach for colorings, and obtain new algorithmic results for the corresponding counting/sampling problems. Le… ▽ More The spectral independence approach of Anari et al. (2020) utilized recent results on high-dimensional expanders of Alev and Lau (2020) and established rapid mixing of the Glauber dynamics for the hard-core model defined on weighted independent sets. We develop the spectral independence approach for colorings, and obtain new algorithmic results for the corresponding counting/sampling problems. Let $α^*\approx 1.763$ denote the solution to $\exp(1/x)=x$ and let $α>α^*$. We prove that, for any triangle-free graph $G=(V,E)$ with maximum degree $Δ$, for all $q\geqαΔ+1$, the mixing time of the Glauber dynamics for $q$-colorings is polynomial in $n=|V|$, with the exponent of the polynomial independent of $Δ$ and $q$. In comparison, previous approximate counting results for colorings held for a similar range of $q$ (asymptotically in $Δ$) but with larger girth requirement or with a running time where the polynomial exponent depended on $Δ$ and $q$ (exponentially). One further feature of using the spectral independence approach to study colorings is that it avoids many of the technical complications in previous approaches caused by coupling arguments or by passing to the complex plane; the key improvement on the running time is based on relatively simple combinatorial arguments which are then translated into spectral bounds. △ Less

Submitted 15 July, 2020; originally announced July 2020.

Statistical Mechanical Analysis of Neural Network Pruning

Authors: Rupam Acharyya, Ankani Chattoraj, Boyu Zhang, Shouman Das, Daniel Stefankovic

Abstract: Deep learning architectures with a huge number of parameters are often compressed using pruning techniques to ensure computational efficiency of inference during deployment. Despite multitude of empirical advances, there is a lack of theoretical understanding of the effectiveness of different pruning methods. We inspect different pruning techniques under the statistical mechanics formulation of a… ▽ More Deep learning architectures with a huge number of parameters are often compressed using pruning techniques to ensure computational efficiency of inference during deployment. Despite multitude of empirical advances, there is a lack of theoretical understanding of the effectiveness of different pruning methods. We inspect different pruning techniques under the statistical mechanics formulation of a teacher-student framework and derive their generalization error (GE) bounds. It has been shown that Determinantal Point Process (DPP) based node pruning method is notably superior to competing approaches when tested on real datasets. Using GE bounds in the aforementioned setup we provide theoretical guarantees for their empirical observations. Another consistent finding in literature is that sparse neural networks (edge pruned) generalize better than dense neural networks (node pruned) for a fixed number of parameters. We use our theoretical setup to prove this finding and show that even the baseline random edge pruning method performs better than the DPP node pruning method. We also validate this empirically on real datasets. △ Less

Submitted 11 June, 2021; v1 submitted 30 June, 2020; originally announced June 2020.

Comments: Authors Ankani Chattoraj and Boyu Zhang made an equal contribution

Hardness of Identity Testing for Restricted Boltzmann Machines and Potts models

Authors: Antonio Blanca, Zongchen Chen, Daniel Štefankovič, Eric Vigoda

Abstract: We study identity testing for restricted Boltzmann machines (RBMs), and more generally for undirected graphical models. Given sample access to the Gibbs distribution corresponding to an unknown or hidden model $M^*$ and given an explicit model $M$, can we distinguish if either $M = M^*$ or if they are (statistically) far apart? Daskalakis et al. (2018) presented a polynomial-time algorithm for ide… ▽ More We study identity testing for restricted Boltzmann machines (RBMs), and more generally for undirected graphical models. Given sample access to the Gibbs distribution corresponding to an unknown or hidden model $M^*$ and given an explicit model $M$, can we distinguish if either $M = M^*$ or if they are (statistically) far apart? Daskalakis et al. (2018) presented a polynomial-time algorithm for identity testing for the ferromagnetic (attractive) Ising model. In contrast, for the antiferromagnetic (repulsive) Ising model, Bezáková et al. (2019) proved that unless $RP=NP$ there is no identity testing algorithm when $βd=ω(\log{n})$, where $d$ is the maximum degree of the visible graph and $β$ is the largest edge weight in absolute value. We prove analogous hardness results for RBMs (i.e., mixed Ising models on bipartite graphs), even when there are no latent variables or an external field. Specifically, we show that if $RP \neq NP$, then when $βd=ω(\log{n})$ there is no polynomial-time algorithm for identity testing for RBMs; when $βd =O(\log{n})$ there is an efficient identity testing algorithm that utilizes the structure learning algorithm of Klivans and Meka (2017). In addition, we prove similar lower bounds for purely ferromagnetic RBMs with inconsistent external fields, and for the ferromagnetic Potts model. Previous hardness results for identity testing of Bezáková et al. (2019) utilized the hardness of finding the maximum cuts, which corresponds to the ground states of the antiferromagnetic Ising model. Since RBMs are on bipartite graphs such an approach is not feasible. We instead introduce a general methodology to reduce from the corresponding approximate counting problem and utilize the phase transition that is exhibited by RBMs and the mean-field Potts model. △ Less

Submitted 22 April, 2020; originally announced April 2020.

The complexity of approximating averages on bounded-degree graphs

Authors: Andreas Galanis, Daniel Stefankovic, Eric Vigoda

Abstract: We prove that, unless P=NP, there is no polynomial-time algorithm to approximate within some multiplicative constant the average size of an independent set in graphs of maximum degree 6. This is a special case of a more general result for the hard-core model defined on independent sets weighted by a parameter $λ>0$. In the general setting, we prove that, unless P=NP, for all $Δ\geq 3$, all… ▽ More We prove that, unless P=NP, there is no polynomial-time algorithm to approximate within some multiplicative constant the average size of an independent set in graphs of maximum degree 6. This is a special case of a more general result for the hard-core model defined on independent sets weighted by a parameter $λ>0$. In the general setting, we prove that, unless P=NP, for all $Δ\geq 3$, all $λ>λ_c(Δ)$, there is no FPTAS which applies to all graphs of maximum degree $Δ$ for computing the average size of the independent set in the Gibbs distribution, where $λ_c(Δ)$ is the critical point for the uniqueness/non-uniqueness phase transition on the $Δ$-regular tree. Moreover, we prove that for $λ$ in a dense set of this non-uniqueness region the problem is NP-hard to approximate within some constant factor. Our work extends to the antiferromagnetic Ising model and generalizes to all 2-spin antiferromagnetic models, establishing hardness of computing the average magnetization in the tree non-uniqueness region. Previously, Schulman, Sinclair and Srivastava (2015) showed that it is #P-hard to compute the average magnetization exactly, but no hardness of approximation results were known. Hardness results of Sly (2010) and Sly and Sun (2014) for approximating the partition function do not imply hardness of computing averages. The new ingredient in our reduction is an intricate construction of pairs of rooted trees whose marginal distributions at the root agree but their derivatives disagree. The main technical contribution is controlling what marginal distributions and derivatives are achievable and using Cauchy's functional equation to argue existence of the gadgets. △ Less

Submitted 19 July, 2021; v1 submitted 20 April, 2020; originally announced April 2020.

Comments: Minor update to make "field gadgets" bipartite (Lemma 30)

Improved Strong Spatial Mixing for Colorings on Trees

Authors: Charilaos Efthymiou, Andreas Galanis, Thomas P. Hayes, Daniel Stefankovic, Eric Vigoda

Abstract: Strong spatial mixing (SSM) is a form of correlation decay that has played an essential role in the design of approximate counting algorithms for spin systems. A notable example is the algorithm of Weitz (2006) for the hard-core model on weighted independent sets. We study SSM for the $q$-colorings problem on the infinite $(d+1)$-regular tree. Weak spatial mixing (WSM) captures whether the influen… ▽ More Strong spatial mixing (SSM) is a form of correlation decay that has played an essential role in the design of approximate counting algorithms for spin systems. A notable example is the algorithm of Weitz (2006) for the hard-core model on weighted independent sets. We study SSM for the $q$-colorings problem on the infinite $(d+1)$-regular tree. Weak spatial mixing (WSM) captures whether the influence of the leaves on the root vanishes as the height of the tree grows. Jonasson (2002) established WSM when $q>d+1$. In contrast, in SSM, we first fix a coloring on a subset of internal vertices, and we again ask if the influence of the leaves on the root is vanishing. It was known that SSM holds on the $(d+1)$-regular tree when $q>αd$ where $α\approx 1.763...$ is a constant that has arisen in a variety of results concerning random colorings. Here we improve on this bound by showing SSM for $q>1.59d$. Our proof establishes an $L^2$ contraction for the BP operator. For the contraction we bound the norm of the BP Jacobian by exploiting combinatorial properties of the coloring of the tree. △ Less

Submitted 16 September, 2019; originally announced September 2019.

Lower bounds for testing graphical models: colorings and antiferromagnetic Ising models

Authors: Ivona Bezakova, Antonio Blanca, Zongchen Chen, Daniel Štefankovič, Eric Vigoda

Abstract: We study the identity testing problem in the context of spin systems or undirected graphical models, where it takes the following form: given the parameter specification of the model $M$ and a sampling oracle for the distribution $μ_{\hat{M}}$ of an unknown model $\hat{M}$, can we efficiently determine if the two models $M$ and $\hat{M}$ are the same? We consider identity testing for both soft-con… ▽ More We study the identity testing problem in the context of spin systems or undirected graphical models, where it takes the following form: given the parameter specification of the model $M$ and a sampling oracle for the distribution $μ_{\hat{M}}$ of an unknown model $\hat{M}$, can we efficiently determine if the two models $M$ and $\hat{M}$ are the same? We consider identity testing for both soft-constraint and hard-constraint systems. In particular, we prove hardness results in two prototypical cases, the Ising model and proper colorings, and explore whether identity testing is any easier than structure learning. For the ferromagnetic (attractive) Ising model, Daskalakis et al. (2018) presented a polynomial time algorithm for identity testing. We prove hardness results in the antiferromagnetic (repulsive) setting in the same regime of parameters where structure learning is known to require a super-polynomial number of samples. In particular, for $n$-vertex graphs of maximum degree $d$, we prove that if $|β| d = ω(\log{n})$ (where $β$ is the inverse temperature parameter), then there is no polynomial running time identity testing algorithm unless $RP=NP$. We also establish computational lower bounds for a broader set of parameters under the (randomized) exponential time hypothesis. Our proofs utilize insights into the design of gadgets using random graphs in recent works concerning the hardness of approximate counting by Sly (2010). In the hard-constraint setting, we present hardness results for identity testing for proper colorings. Our results are based on the presumed hardness of #BIS, the problem of (approximately) counting independent sets in bipartite graphs. In particular, we prove that identity testing is hard in the same range of parameters where structure learning is known to be hard. △ Less

Submitted 20 June, 2019; v1 submitted 22 January, 2019; originally announced January 2019.

The complexity of approximating the matching polynomial in the complex plane

Authors: Ivona Bezakova, Andreas Galanis, Leslie Ann Goldberg, Daniel Stefankovic

Abstract: We study the problem of approximating the value of the matching polynomial on graphs with edge parameter $γ$, where $γ$ takes arbitrary values in the complex plane. When $γ$ is a positive real, Jerrum and Sinclair showed that the problem admits an FPRAS on general graphs. For general complex values of $γ$, Patel and Regts, building on methods developed by Barvinok, showed that the problem admits… ▽ More We study the problem of approximating the value of the matching polynomial on graphs with edge parameter $γ$, where $γ$ takes arbitrary values in the complex plane. When $γ$ is a positive real, Jerrum and Sinclair showed that the problem admits an FPRAS on general graphs. For general complex values of $γ$, Patel and Regts, building on methods developed by Barvinok, showed that the problem admits an FPTAS on graphs of maximum degree $Δ$ as long as $γ$ is not a negative real number less than or equal to $-1/(4(Δ-1))$. Our first main result completes the picture for the approximability of the matching polynomial on bounded degree graphs. We show that for all $Δ\geq 3$ and all real $γ$ less than $-1/(4(Δ-1))$, the problem of approximating the value of the matching polynomial on graphs of maximum degree $Δ$ with edge parameter $γ$ is #P-hard. We then explore whether the maximum degree parameter can be replaced by the connective constant. Sinclair et al. showed that for positive real $γ$ it is possible to approximate the value of the matching polynomial using a correlation decay algorithm on graphs with bounded connective constant (and potentially unbounded maximum degree). We first show that this result does not extend in general in the complex plane; in particular, the problem is #P-hard on graphs with bounded connective constant for a dense set of $γ$ values on the negative real axis. Nevertheless, we show that the result does extend for any complex value $γ$ that does not lie on the negative real axis. Our analysis accounts for complex values of $γ$ using geodesic distances in the complex plane in the metric defined by an appropriate density function. △ Less

Submitted 11 January, 2021; v1 submitted 13 July, 2018; originally announced July 2018.

Sampling in Uniqueness from the Potts and Random-Cluster Models on Random Regular Graphs

Authors: Antonio Blanca, Andreas Galanis, Leslie Ann Goldberg, Daniel Stefankovic, Eric Vigoda, Kuan Yang

Abstract: We consider the problem of sampling from the Potts model on random regular graphs. It is conjectured that sampling is possible when the temperature of the model is in the uniqueness regime of the regular tree, but positive algorithmic results have been for the most part elusive. In this paper, for all integers $q\geq 3$ and $Δ\geq 3$, we develop algorithms that produce samples within error $o(1)$… ▽ More We consider the problem of sampling from the Potts model on random regular graphs. It is conjectured that sampling is possible when the temperature of the model is in the uniqueness regime of the regular tree, but positive algorithmic results have been for the most part elusive. In this paper, for all integers $q\geq 3$ and $Δ\geq 3$, we develop algorithms that produce samples within error $o(1)$ from the $q$-state Potts model on random $Δ$-regular graphs, whenever the temperature is in uniqueness, for both the ferromagnetic and antiferromagnetic cases. The algorithm for the antiferromagnetic Potts model is based on iteratively adding the edges of the graph and resampling a bichromatic class that contains the endpoints of the newly added edge. Key to the algorithm is how to perform the resampling step efficiently since bichromatic classes may induce linear-sized components. To this end, we exploit the tree uniqueness to show that the average growth of bichromatic components is typically small, which allows us to use correlation decay algorithms for the resampling step. While the precise uniqueness threshold on the tree is not known for general values of $q$ and $Δ$ in the antiferromagnetic case, our algorithm works throughout uniqueness regardless of its value. In the case of the ferromagnetic Potts model, we simplify the algorithm significantly by utilising the random-cluster representation of the model. In particular, we show that a percolation-type algorithm succeeds in sampling from the random-cluster model with parameters $p,q$ on random $Δ$-regular graphs for all values of $q\geq 1$ and $p<p_c(q,Δ)$, where $p_c(q,Δ)$ corresponds to a uniqueness threshold for the model on the $Δ$-regular tree. When restricted to integer values of $q$, this yields a simplified algorithm for the ferromagnetic Potts model on random $Δ$-regular graphs. △ Less

Submitted 1 December, 2019; v1 submitted 22 April, 2018; originally announced April 2018.

On Counting Perfect Matchings in General Graphs

Authors: Daniel Štefankovič, Eric Vigoda, John Wilmes

Abstract: Counting perfect matchings has played a central role in the theory of counting problems. The permanent, corresponding to bipartite graphs, was shown to be #P-complete to compute exactly by Valiant (1979), and a fully polynomial randomized approximation scheme (FPRAS) was presented by Jerrum, Sinclair, and Vigoda (2004) using a Markov chain Monte Carlo (MCMC) approach. However, it has remained an o… ▽ More Counting perfect matchings has played a central role in the theory of counting problems. The permanent, corresponding to bipartite graphs, was shown to be #P-complete to compute exactly by Valiant (1979), and a fully polynomial randomized approximation scheme (FPRAS) was presented by Jerrum, Sinclair, and Vigoda (2004) using a Markov chain Monte Carlo (MCMC) approach. However, it has remained an open question whether there exists an FPRAS for counting perfect matchings in general graphs. In fact, it was unresolved whether the same Markov chain defined by JSV is rapidly mixing in general. In this paper, we show that it is not. We prove torpid mixing for any weighting scheme on hole patterns in the JSV chain. As a first step toward overcoming this obstacle, we introduce a new algorithm for counting matchings based on the Gallai-Edmonds decomposition of a graph, and give an FPRAS for counting matchings in graphs that are sufficiently close to bipartite. In particular, we obtain a fixed-parameter tractable algorithm for counting matchings in general graphs, parameterized by the greatest "order" of a factor-critical subgraph. △ Less

Submitted 20 December, 2017; originally announced December 2017.

Comments: To appear in LATIN 2018

MSC Class: 68Q25; 60J10

Inapproximability of the independent set polynomial in the complex plane

Authors: Ivona Bezakova, Andreas Galanis, Leslie Ann Goldberg, Daniel Stefankovic

Abstract: We study the complexity of approximating the independent set polynomial $Z_G(λ)$ of a graph $G$ with maximum degree $Δ$ when the activity $λ$ is a complex number. This problem is already well understood when $λ$ is real using connections to the $Δ$-regular tree $T$. The key concept in that case is the "occupation ratio" of the tree $T$. This ratio is the contribution to $Z_T(λ)$ from independent… ▽ More We study the complexity of approximating the independent set polynomial $Z_G(λ)$ of a graph $G$ with maximum degree $Δ$ when the activity $λ$ is a complex number. This problem is already well understood when $λ$ is real using connections to the $Δ$-regular tree $T$. The key concept in that case is the "occupation ratio" of the tree $T$. This ratio is the contribution to $Z_T(λ)$ from independent sets containing the root of the tree, divided by $Z_T(λ)$ itself. If $λ$ is such that the occupation ratio converges to a limit, as the height of $T$ grows, then there is an FPTAS for approximating $Z_G(λ)$ on a graph $G$ with maximum degree $Δ$. Otherwise, the approximation problem is NP-hard. Unsurprisingly, the case where $λ$ is complex is more challenging. Peters and Regts identified the complex values of $λ$ for which the occupation ratio of the $Δ$-regular tree converges. These values carve a cardioid-shaped region $Λ_Δ$ in the complex plane. Motivated by the picture in the real case, they asked whether $Λ_Δ$ marks the true approximability threshold for general complex values $λ$. Our main result shows that for every $λ$ outside of $Λ_Δ$, the problem of approximating $Z_G(λ)$ on graphs $G$ with maximum degree at most $Δ$ is indeed NP-hard. In fact, when $λ$ is outside of $Λ_Δ$ and is not a positive real number, we give the stronger result that approximating $Z_G(λ)$ is actually #P-hard. If $λ$ is a negative real number outside of $Λ_Δ$, we show that it is #P-hard to even decide whether $Z_G(λ)>0$, resolving in the affirmative a conjecture of Harvey, Srivastava and Vondrak. Our proof techniques are based around tools from complex analysis - specifically the study of iterative multivariate rational maps. △ Less

Submitted 5 July, 2020; v1 submitted 1 November, 2017; originally announced November 2017.

Structure Learning of $H$-colorings

Authors: Antonio Blanca, Zongchen Chen, Daniel Štefankovič, Eric Vigoda

Abstract: We study the structure learning problem for $H$-colorings, an important class of Markov random fields that capture key combinatorial structures on graphs, including proper colorings and independent sets, as well as spin systems from statistical physics. The learning problem is as follows: for a fixed (and known) constraint graph $H$ with $q$ colors and an unknown graph $G=(V,E)$ with $n$ vertices,… ▽ More We study the structure learning problem for $H$-colorings, an important class of Markov random fields that capture key combinatorial structures on graphs, including proper colorings and independent sets, as well as spin systems from statistical physics. The learning problem is as follows: for a fixed (and known) constraint graph $H$ with $q$ colors and an unknown graph $G=(V,E)$ with $n$ vertices, given uniformly random $H$-colorings of $G$, how many samples are required to learn the edges of the unknown graph $G$? We give a characterization of $H$ for which the problem is identifiable for every $G$, i.e., we can learn $G$ with an infinite number of samples. We also show that there are identifiable constraint graphs for which one cannot hope to learn every graph $G$ efficiently. We focus particular attention on the case of proper vertex $q$-colorings of graphs of maximum degree $d$ where intriguing connections to statistical physics phase transitions appear. We prove that in the tree uniqueness region (when $q>d$) the problem is identifiable and we can learn $G$ in ${\rm poly}(d,q) \times O(n^2\log{n})$ time. In contrast for soft-constraint systems, such as the Ising model, the best possible running time is exponential in $d$. In the tree non-uniqueness region (when $q\leq d$) we prove that the problem is not identifiable and thus $G$ cannot be learned. Moreover, when $q<d-\sqrt{d} + Θ(1)$ we prove that even learning an equivalent graph (any graph with the same set of $H$-colorings) is computationally hard---sample complexity is exponential in $n$ in the worst case. We further explore the connection between the efficiency/hardness of the structure learning problem and the uniqueness/non-uniqueness phase transition for general $H$-colorings and prove that under the well-known Dobrushin uniqueness condition, we can learn $G$ in ${\rm poly}(d,q)\times O(n^2\log{n})$ time. △ Less

Submitted 24 April, 2018; v1 submitted 16 August, 2017; originally announced August 2017.

Sampling Random Colorings of Sparse Random Graphs

Authors: Charilaos Efthymiou, Thomas P. Hayes, Daniel Stefankovic, Eric Vigoda

Abstract: We study the mixing properties of the single-site Markov chain known as the Glauber dynamics for sampling $k$-colorings of a sparse random graph $G(n,d/n)$ for constant $d$. The best known rapid mixing results for general graphs are in terms of the maximum degree $Δ$ of the input graph $G$ and hold when $k>11Δ/6$ for all $G$. Improved results hold when $k>αΔ$ for graphs with girth $\geq 5$ and… ▽ More We study the mixing properties of the single-site Markov chain known as the Glauber dynamics for sampling $k$-colorings of a sparse random graph $G(n,d/n)$ for constant $d$. The best known rapid mixing results for general graphs are in terms of the maximum degree $Δ$ of the input graph $G$ and hold when $k>11Δ/6$ for all $G$. Improved results hold when $k>αΔ$ for graphs with girth $\geq 5$ and $Δ$ sufficiently large where $α\approx 1.7632\ldots$ is the root of $α=\exp(1/α)$; further improvements on the constant $α$ hold with stronger girth and maximum degree assumptions. For sparse random graphs the maximum degree is a function of $n$ and the goal is to obtain results in terms of the expected degree $d$. The following rapid mixing results for $G(n,d/n)$ hold with high probability over the choice of the random graph for sufficiently large constant~$d$. Mossel and Sly (2009) proved rapid mixing for constant $k$, and Efthymiou (2014) improved this to $k$ linear in~$d$. The condition was improved to $k>3d$ by Yin and Zhang (2016) using non-MCMC methods. Here we prove rapid mixing when $k>αd$ where $α\approx 1.7632\ldots$ is the same constant as above. Moreover we obtain $O(n^{3})$ mixing time of the Glauber dynamics, while in previous rapid mixing results the exponent was an increasing function in $d$. As in previous results for random graphs our proof analyzes an appropriately defined block dynamics to "hide" high-degree vertices. One new aspect in our improved approach is utilizing so-called local uniformity properties for the analysis of block dynamics. To analyze the "burn-in" phase we prove a concentration inequality for the number of disagreements propagating in large blocks. △ Less

Submitted 12 July, 2017; originally announced July 2017.

Rapid Mixing Swendsen-Wang Sampler for Stochastic Partitioned Attractive Models

Authors: Sejun Park, Yunhun Jang, Andreas Galanis, **woo Shin, Daniel Stefankovic, Eric Vigoda

Abstract: The Gibbs sampler is a particularly popular Markov chain used for learning and inference problems in Graphical Models (GMs). These tasks are computationally intractable in general, and the Gibbs sampler often suffers from slow mixing. In this paper, we study the Swendsen-Wang dynamics which is a more sophisticated Markov chain designed to overcome bottlenecks that impede the Gibbs sampler. We prov… ▽ More The Gibbs sampler is a particularly popular Markov chain used for learning and inference problems in Graphical Models (GMs). These tasks are computationally intractable in general, and the Gibbs sampler often suffers from slow mixing. In this paper, we study the Swendsen-Wang dynamics which is a more sophisticated Markov chain designed to overcome bottlenecks that impede the Gibbs sampler. We prove O(\log n) mixing time for attractive binary pairwise GMs (i.e., ferromagnetic Ising models) on stochastic partitioned graphs having n vertices, under some mild conditions, including low temperature regions where the Gibbs sampler provably mixes exponentially slow. Our experiments also confirm that the Swendsen-Wang sampler significantly outperforms the Gibbs sampler when they are used for learning parameters of attractive GMs. △ Less

Submitted 6 April, 2017; originally announced April 2017.

On The Projection Operator to A Three-view Cardinality Constrained Set

Authors: Haichuan Yang, Shupeng Gui, Chuyang Ke, Daniel Stefankovic, Ryohei Fujimaki, Ji Liu

Abstract: The cardinality constraint is an intrinsic way to restrict the solution structure in many domains, for example, sparse learning, feature selection, and compressed sensing. To solve a cardinality constrained problem, the key challenge is to solve the projection onto the cardinality constraint set, which is NP-hard in general when there exist multiple overlapped cardinality constraints. In this pape… ▽ More The cardinality constraint is an intrinsic way to restrict the solution structure in many domains, for example, sparse learning, feature selection, and compressed sensing. To solve a cardinality constrained problem, the key challenge is to solve the projection onto the cardinality constraint set, which is NP-hard in general when there exist multiple overlapped cardinality constraints. In this paper, we consider the scenario where the overlapped cardinality constraints satisfy a Three-view Cardinality Structure (TVCS), which reflects the natural restriction in many applications, such as identification of gene regulatory networks and task-worker assignment problem. We cast the projection into a linear programming, and show that for TVCS, the vertex solution of this linear programming is the solution for the original projection problem. We further prove that such solution can be found with the complexity proportional to the number of variables and constraints. We finally use synthetic experiments and two interesting applications in bioinformatics and crowdsourcing to validate the proposed TVCS model and method. △ Less

Submitted 14 June, 2017; v1 submitted 21 March, 2017; originally announced March 2017.

Implementations and the independent set polynomial below the Shearer threshold

Authors: Andreas Galanis, Leslie Ann Goldberg, Daniel Stefankovic

Abstract: The independent set polynomial is important in many areas. For every integer $Δ\geq 2$, the Shearer threshold is the value $λ^*(Δ)=(Δ-1)^{Δ-1}/Δ^Δ$ . It is known that for $λ< - λ^*(Δ)$, there are graphs~$G$ with maximum degree~$Δ$ whose independent set polynomial, evaluated at~$λ$, is at most~$0$. Also, there are no such graphs for any $λ> -λ^*(Δ)$. This paper is motivated by the computational pro… ▽ More The independent set polynomial is important in many areas. For every integer $Δ\geq 2$, the Shearer threshold is the value $λ^*(Δ)=(Δ-1)^{Δ-1}/Δ^Δ$ . It is known that for $λ< - λ^*(Δ)$, there are graphs~$G$ with maximum degree~$Δ$ whose independent set polynomial, evaluated at~$λ$, is at most~$0$. Also, there are no such graphs for any $λ> -λ^*(Δ)$. This paper is motivated by the computational problem of approximating the independent set polynomial when $λ< - λ^*(Δ)$. The key issue in complexity bounds for this problem is "implementation". Informally, an implementation of a real number $λ'$ is a graph whose hard-core partition function, evaluated at~$λ$, simulates a vertex-weight of~$λ'$ in the sense that $λ'$ is the ratio between the contribution to the partition function from independent sets containing a certain vertex and the contribution from independent sets that do not contain that vertex. Implementations are the cornerstone of intractability results for the problem of approximately evaluating the independent set polynomial. Our main result is that, for any $λ< - λ^*(Δ)$, it is possible to implement a set of values that is dense over the reals. The result is tight in the sense that it is not possible to implement a set of values that is dense over the reals for any $λ> λ^*(Δ)$. Our result has already been used in a paper with \bezakova{} (STOC 2018) to show that it is \#P-hard to approximate the evaluation of the independent set polynomial on graphs of degree at most~$Δ$ at any value $λ<-λ^*(Δ)$. In the appendix, we give an additional incomparable inapproximability result (strengthening the inapproximability bound to an exponential factor, but weakening the hardness to NP-hardness). △ Less

Submitted 22 October, 2022; v1 submitted 17 December, 2016; originally announced December 2016.

Comments: To appear in TCS

The Complexity of Tensor Rank

Authors: Marcus Schaefer, Daniel Stefankovic

Abstract: We show that determining the rank of a tensor over a field has the same complexity as deciding the existential theory of that field. This implies earlier NP-hardness results by Håstad~\cite{H90}. The hardness proof also implies an algebraic universality result. We show that determining the rank of a tensor over a field has the same complexity as deciding the existential theory of that field. This implies earlier NP-hardness results by Håstad~\cite{H90}. The hardness proof also implies an algebraic universality result. △ Less

Submitted 9 January, 2024; v1 submitted 13 December, 2016; originally announced December 2016.

Comments: fixed error in Section 3.3

Convergence of MCMC and Loopy BP in the Tree Uniqueness Region for the Hard-Core Model

Authors: Charilaos Efthymiou, Thomas P. Hayes, Daniel Stefankovic, Eric Vigoda, Yitong Yin

Abstract: We study the hard-core model defined on independent sets of an input graph where the independent sets are weighted by a parameter $λ>0$. For constant $Δ$, previous work of Weitz (2006) established an FPTAS for the partition function for graphs of maximum degree $Δ$ when $λ< λ_c(Δ)$. The threshold $λ_c(Δ)$ is the critical point for the phase transition for uniqueness/non-uniqueness on the infinite… ▽ More We study the hard-core model defined on independent sets of an input graph where the independent sets are weighted by a parameter $λ>0$. For constant $Δ$, previous work of Weitz (2006) established an FPTAS for the partition function for graphs of maximum degree $Δ$ when $λ< λ_c(Δ)$. The threshold $λ_c(Δ)$ is the critical point for the phase transition for uniqueness/non-uniqueness on the infinite $Δ$-regular trees. Sly (2010) showed that there is no FPRAS, unless NP=RP, when $λ>λ_c(Δ)$. The running time of Weitz's algorithm is exponential in $\log(Δ)$. Here we present an FPRAS for the partition function whose running time is $O^*(n^2)$. We analyze the simple single-site Glauber dynamics for sampling from the associated Gibbs distribution. We prove there exists a constant $Δ_0$ such that for all graphs with maximum degree $Δ\geqΔ_0$ and girth $\geq 7$, the mixing time of the Glauber dynamics is $O(n\log(n))$ when $λ<λ_c(Δ)$. Our work complements that of Weitz which applies for constant $Δ$ whereas our work applies for all $Δ\geq Δ_0$. We utilize loopy BP (belief propagation), a widely-used inference algorithm. A novel aspect of our work is using the principal eigenvector for the BP operator to design a distance function which contracts in expectation for pairs of states that behave like the BP fixed point. We also prove that the Glauber dynamics behaves locally like loopy BP. As a byproduct we obtain that the Glauber dynamics converges, after a short burn-in period, close to the BP fixed point, and this implies that the fixed point of loopy BP is a close approximation to the Gibbs distribution. Using these connections we establish that loopy BP quickly converges to the Gibbs distribution when the girth $\geq 6$ and $λ<λ_c(Δ)$. △ Less

Submitted 29 August, 2016; v1 submitted 5 April, 2016; originally announced April 2016.

ACM Class: G.2.1; F.2.2

Approximation via Correlation Decay when Strong Spatial Mixing Fails

Authors: Ivona Bezakova, Andreas Galanis, Leslie Ann Goldberg, Heng Guo, Daniel Stefankovic

Abstract: Approximate counting via correlation decay is the core algorithmic technique used in the sharp delineation of the computational phase transition that arises in the approximation of the partition function of anti-ferromagnetic two-spin models. Previous analyses of correlation-decay algorithms implicitly depended on the occurrence of strong spatial mixing (SSM). This means that one uses worst-case… ▽ More Approximate counting via correlation decay is the core algorithmic technique used in the sharp delineation of the computational phase transition that arises in the approximation of the partition function of anti-ferromagnetic two-spin models. Previous analyses of correlation-decay algorithms implicitly depended on the occurrence of strong spatial mixing (SSM). This means that one uses worst-case analysis of the recursive procedure that creates the sub-instances. We develop a new analysis method that is more refined than the worst-case analysis. We take the shape of instances in the computation tree into consideration and amortise against certain "bad" instances that are created as the recursion proceeds. This enables us to show correlation decay and to obtain an FPTAS even when SSM fails. We apply our technique to the problem of approximately counting independent sets in hypergraphs with degree upper-bound Delta and with a lower bound k on the arity of hyperedges. Liu and Lin gave an FPTAS for k>=2 and Delta<=5 (lack of SSM was the obstacle preventing this algorithm from being generalised to Delta=6). Our technique gives a tight result for Delta=6, showing that there is an FPTAS for k>=3 and Delta<=6. The best previously-known approximation scheme for Delta=6 is the Markov-chain simulation based FPRAS of Bordewich, Dyer and Karpinski, which only works for k>=8. Our technique also applies for larger values of k, giving an FPTAS for k>=Delta. This bound is not substantially stronger than existing randomised results in the literature. Nevertheless, it gives the first deterministic approximation scheme in this regime. Moreover, unlike existing results, it leads to an FPTAS for counting dominating sets in regular graphs with sufficiently large degree. We further demonstrate that approximately counting independent sets in hypergraphs is NP-hard even within the uniqueness regime. △ Less

Submitted 1 February, 2019; v1 submitted 30 October, 2015; originally announced October 2015.

Comments: To appear in SICOMP

Swendsen-Wang Algorithm on the Mean-Field Potts Model

Authors: Andreas Galanis, Daniel Stefankovic, Eric Vigoda

Abstract: We study the $q$-state ferromagnetic Potts model on the $n$-vertex complete graph known as the mean-field (Curie-Weiss) model. We analyze the Swendsen-Wang algorithm which is a Markov chain that utilizes the random cluster representation for the ferromagnetic Potts model to recolor large sets of vertices in one step and potentially overcomes obstacles that inhibit single-site Glauber dynamics. Lon… ▽ More We study the $q$-state ferromagnetic Potts model on the $n$-vertex complete graph known as the mean-field (Curie-Weiss) model. We analyze the Swendsen-Wang algorithm which is a Markov chain that utilizes the random cluster representation for the ferromagnetic Potts model to recolor large sets of vertices in one step and potentially overcomes obstacles that inhibit single-site Glauber dynamics. Long et al. studied the case $q=2$, the Swendsen-Wang algorithm for the mean-field ferromagnetic Ising model, and showed that the mixing time satisfies: (i) $Θ(1)$ for $β<β_c$, (ii) $Θ(n^{1/4})$ for $β=β_c$, (iii) $Θ(\log n)$ for $β>β_c$, where $β_c$ is the critical temperature for the ordered/disordered phase transition. In contrast, for $q\geq 3$ there are two critical temperatures $0<β_u<β_{rc}$ that are relevant. We prove that the mixing time of the Swendsen-Wang algorithm for the ferromagnetic Potts model on the $n$-vertex complete graph satisfies: (i) $Θ(1)$ for $β<β_u$, (ii) $Θ(n^{1/3})$ for $β=β_u$, (iii) $\exp(n^{Ω(1)})$ for $β_u<β<β_{rc}$, and (iv) $Θ(\log{n})$ for $β\geqβ_{rc}$. These results complement refined results of Cuff et al. on the mixing time of the Glauber dynamics for the ferromagnetic Potts model. △ Less

Submitted 23 November, 2017; v1 submitted 23 February, 2015; originally announced February 2015.

Comments: To appear in Random Structures & Algorithms

Spatial mixing and the connective constant: Optimal bounds

Authors: Alistair Sinclair, Piyush Srivastava, Daniel Štefankovič, Yitong Yin

Abstract: We study the problem of deterministic approximate counting of matchings and independent sets in graphs of bounded connective constant. More generally, we consider the problem of evaluating the partition functions of the monomer-dimer model (which is defined as a weighted sum over all matchings where each matching is given a weight $γ^{|V| - 2 |M|}$ in terms of a fixed parameter gamma called the mo… ▽ More We study the problem of deterministic approximate counting of matchings and independent sets in graphs of bounded connective constant. More generally, we consider the problem of evaluating the partition functions of the monomer-dimer model (which is defined as a weighted sum over all matchings where each matching is given a weight $γ^{|V| - 2 |M|}$ in terms of a fixed parameter gamma called the monomer activity) and the hard core model (which is defined as a weighted sum over all independent sets where an independent set I is given a weight $λ^{|I|}$ in terms of a fixed parameter lambda called the vertex activity). The connective constant is a natural measure of the average degree of a graph which has been studied extensively in combinatorics and mathematical physics, and can be bounded by a constant even for certain unbounded degree graphs such as those sampled from the sparse Erdős-Rényi model $G(n, d/n)$. Our main technical contribution is to prove the best possible rates of decay of correlations in the natural probability distributions induced by both the hard core model and the monomer-dimer model in graphs with a given bound on the connective constant. These results on decay of correlations are obtained using a new framework based on the so-called message approach that has been extensively used recently to prove such results for bounded degree graphs. We then use these optimal decay of correlations results to obtain FPTASs for the two problems on graphs of bounded connective constant. Our techniques also allow us to improve upon known bounds for decay of correlations for the hard core model on various regular lattices, including those obtained by Restrepo, Shin, Vigoda and Tetali (2011) for the special case of Z^2 using sophisticated numerically intensive methods tailored to that special case. △ Less

Submitted 8 October, 2014; originally announced October 2014.

Comments: This paper supersedes arxiv:1308.1762, in which weaker versions of some of the results in this paper appeared. The current paper strengthens the main result of 1308.1762 (Theorem 1.3) to obtain an optimal setting of the parameters, and also adds new results for the monomer-dimer model

Ferromagnetic Potts Model: Refined #BIS-hardness and Related Results

Authors: Andreas Galanis, Daniel Stefankovic, Eric Vigoda, Linji Yang

Abstract: Recent results establish for 2-spin antiferromagnetic systems that the computational complexity of approximating the partition function on graphs of maximum degree D undergoes a phase transition that coincides with the uniqueness phase transition on the infinite D-regular tree. For the ferromagnetic Potts model we investigate whether analogous hardness results hold. Goldberg and Jerrum showed that… ▽ More Recent results establish for 2-spin antiferromagnetic systems that the computational complexity of approximating the partition function on graphs of maximum degree D undergoes a phase transition that coincides with the uniqueness phase transition on the infinite D-regular tree. For the ferromagnetic Potts model we investigate whether analogous hardness results hold. Goldberg and Jerrum showed that approximating the partition function of the ferromagnetic Potts model is at least as hard as approximating the number of independent sets in bipartite graphs (#BIS-hardness). We improve this hardness result by establishing it for bipartite graphs of maximum degree D. We first present a detailed picture for the phase diagram for the infinite D-regular tree, giving a refined picture of its first-order phase transition and establishing the critical temperature for the coexistence of the disordered and ordered phases. We then prove for all temperatures below this critical temperature that it is #BIS-hard to approximate the partition function on bipartite graphs of maximum degree D. As a corollary, it is #BIS-hard to approximate the number of k-colorings on bipartite graphs of maximum degree D when k <= D/(2 ln D). The #BIS-hardness result for the ferromagnetic Potts model uses random bipartite regular graphs as a gadget in the reduction. The analysis of these random graphs relies on recent connections between the maxima of the expectation of their partition function, attractive fixpoints of the associated tree recursions, and induced matrix norms. We extend these connections to random regular graphs for all ferromagnetic models and establish the Bethe prediction for every ferromagnetic spin system on random regular graphs. We also prove for the ferromagnetic Potts model that the Swendsen-Wang algorithm is torpidly mixing on random D-regular graphs at the critical temperature for large q. △ Less

Submitted 13 September, 2016; v1 submitted 19 November, 2013; originally announced November 2013.

Comments: To appear in SIAM J. Computing

#BIS-Hardness for 2-Spin Systems on Bipartite Bounded Degree Graphs in the Tree Nonuniqueness Region

Authors: **-Yi Cai, Andreas Galanis, Leslie Ann Goldberg, Heng Guo, Mark Jerrum, Daniel Stefankovic, Eric Vigoda

Abstract: Counting independent sets on bipartite graphs (#BIS) is considered a canonical counting problem of intermediate approximation complexity. It is conjectured that #BIS neither has an FPRAS nor is as hard as #SAT to approximate. We study #BIS in the general framework of two-state spin systems on bipartite graphs. We define two notions, nearly-independent phase-correlated spins and unary symmetry brea… ▽ More Counting independent sets on bipartite graphs (#BIS) is considered a canonical counting problem of intermediate approximation complexity. It is conjectured that #BIS neither has an FPRAS nor is as hard as #SAT to approximate. We study #BIS in the general framework of two-state spin systems on bipartite graphs. We define two notions, nearly-independent phase-correlated spins and unary symmetry breaking. We prove that it is #BIS-hard to approximate the partition function of any 2-spin system on bipartite graphs supporting these two notions. As a consequence, we classify the complexity of approximating the partition function of antiferromagnetic 2-spin systems on bounded-degree bipartite graphs. △ Less

Submitted 21 September, 2015; v1 submitted 18 November, 2013; originally announced November 2013.

Reasoning Under the Principle of Maximum Entropy for Modal Logics K45, KD45, and S5

Authors: Tivadar Papai, Henry Kautz, Daniel Stefankovic

Abstract: We propose modal Markov logic as an extension of propositional Markov logic to reason under the principle of maximum entropy for modal logics K45, KD45, and S5. Analogous to propositional Markov logic, the knowledge base consists of weighted formulas, whose weights are learned from data. However, in contrast to Markov logic, in our framework we use the knowledge base to define a probability distri… ▽ More We propose modal Markov logic as an extension of propositional Markov logic to reason under the principle of maximum entropy for modal logics K45, KD45, and S5. Analogous to propositional Markov logic, the knowledge base consists of weighted formulas, whose weights are learned from data. However, in contrast to Markov logic, in our framework we use the knowledge base to define a probability distribution over non-equivalent epistemic situations (pointed Kripke structures) rather than over atoms, and use this distribution to assign probabilities to modal formulas. As in all probabilistic representations, the central task in our framework is inference. Although the size of the state space grows doubly exponentially in the number of propositions in the domain, we provide an algorithm that scales only exponentially in the size of the knowledge base. Finally, we briefly discuss the case of languages with an infinite number of propositions. △ Less

Submitted 23 October, 2013; originally announced October 2013.

Comments: 7 pages, Poster presentation at TARK 2013 (arXiv:1310.6382) http://www.tark.org

Report number: TARK/2013/p222

Inapproximability for Antiferromagnetic Spin Systems in the Tree Non-Uniqueness Region

Authors: Andreas Galanis, Daniel Stefankovic, Eric Vigoda

Abstract: A remarkable connection has been established for antiferromagnetic 2-spin systems, including the Ising and hard-core models, showing that the computational complexity of approximating the partition function for graphs with maximum degree D undergoes a phase transition that coincides with the statistical physics uniqueness/non-uniqueness phase transition on the infinite D-regular tree. Despite this… ▽ More A remarkable connection has been established for antiferromagnetic 2-spin systems, including the Ising and hard-core models, showing that the computational complexity of approximating the partition function for graphs with maximum degree D undergoes a phase transition that coincides with the statistical physics uniqueness/non-uniqueness phase transition on the infinite D-regular tree. Despite this clear picture for 2-spin systems, there is little known for multi-spin systems. We present the first analog of the above inapproximability results for multi-spin systems. The main difficulty in previous inapproximability results was analyzing the behavior of the model on random D-regular bipartite graphs, which served as the gadget in the reduction. To this end one needs to understand the moments of the partition function. Our key contribution is connecting: (i) induced matrix norms, (ii) maxima of the expectation of the partition function, and (iii) attractive fixed points of the associated tree recursions (belief propagation). The view through matrix norms allows a simple and generic analysis of the second moment for any spin system on random D-regular bipartite graphs. This yields concentration results for any spin system in which one can analyze the maxima of the first moment. The connection to fixed points of the tree recursions enables an analysis of the maxima of the first moment for specific models of interest. For k-colorings we prove that for even k, in the tree non-uniqueness region (which corresponds to k<D) it is NP-hard, unless NP=RP, to approximate the number of colorings for triangle-free D-regular graphs. Our proof extends to the antiferromagnetic Potts model, and, in fact, to every antiferromagnetic model under a mild condition. △ Less

Submitted 4 November, 2014; v1 submitted 13 May, 2013; originally announced May 2013.

Subset Selection for Gaussian Markov Random Fields

Authors: Satyaki Mahalanabis, Daniel Stefankovic

Abstract: Given a Gaussian Markov random field, we consider the problem of selecting a subset of variables to observe which minimizes the total expected squared prediction error of the unobserved variables. We first show that finding an exact solution is NP-hard even for a restricted class of Gaussian Markov random fields, called Gaussian free fields, which arise in semi-supervised learning and computer vis… ▽ More Given a Gaussian Markov random field, we consider the problem of selecting a subset of variables to observe which minimizes the total expected squared prediction error of the unobserved variables. We first show that finding an exact solution is NP-hard even for a restricted class of Gaussian Markov random fields, called Gaussian free fields, which arise in semi-supervised learning and computer vision. We then give a simple greedy approximation algorithm for Gaussian free fields on arbitrary graphs. Finally, we give a message passing algorithm for general Gaussian Markov random fields on bounded tree-width graphs. △ Less

Submitted 26 September, 2012; originally announced September 2012.

Comments: 40 pages

MSC Class: 68Q32

Inapproximability of the Partition Function for the Antiferromagnetic Ising and Hard-Core Models

Authors: Andreas Galanis, Daniel Stefankovic, Eric Vigoda

Abstract: Recent inapproximability results of Sly (2010), together with an approximation algorithm presented by Weitz (2006) establish a beautiful picture for the computational complexity of approximating the partition function of the hard-core model. Let $λ_c(T_Δ)$ denote the critical activity for the hard-model on the infinite $Δ$-regular tree. Weitz presented an FPTAS for the partition function when… ▽ More Recent inapproximability results of Sly (2010), together with an approximation algorithm presented by Weitz (2006) establish a beautiful picture for the computational complexity of approximating the partition function of the hard-core model. Let $λ_c(T_Δ)$ denote the critical activity for the hard-model on the infinite $Δ$-regular tree. Weitz presented an FPTAS for the partition function when $λ<λ_c(T_Δ)$ for graphs with constant maximum degree $Δ$. In contrast, Sly showed that for all $Δ\geq 3$, there exists $ε_Δ>0$ such that (unless RP=NP) there is no FPRAS for approximating the partition function on graphs of maximum degree $Δ$ for activities $λ$ satisfying $λ_c(T_Δ)<λ<λ_c(T_Δ)+ε_Δ$. We prove that a similar phenomenon holds for the antiferromagnetic Ising model. Recent results of Li et al. and Sinclair et al. extend Weitz's approach to any 2-spin model, which includes the antiferromagnetic Ising model, to yield an FPTAS for the partition function for all graphs of constant maximum degree $Δ$ when the parameters of the model lie in the uniqueness regime of the infinite tree $T_Δ$. We prove the complementary result that for the antiferrogmanetic Ising model without external field that, unless RP=NP, for all $Δ\geq 3$, there is no FPRAS for approximating the partition function on graphs of maximum degree $Δ$ when the inverse temperature lies in the non-uniqueness regime of the infinite tree $T_Δ$. Our results extend to a region of the parameter space for general 2-spin models. Our proof works by relating certain second moment calculations for random $Δ$-regular bipartite graphs to the tree recursions used to establish the critical points on the infinite tree. △ Less

Submitted 13 September, 2016; v1 submitted 9 March, 2012; originally announced March 2012.

Comments: Journal version (no changes)

Journal ref: Combinator. Probab. Comp. 25 (2016) 500-559

Improved Inapproximability Results for Counting Independent Sets in the Hard-Core Model

Authors: Andreas Galanis, Qi Ge, Daniel Stefankovic, Eric Vigoda, Linji Yang

Abstract: We study the computational complexity of approximately counting the number of independent sets of a graph with maximum degree Delta. More generally, for an input graph G=(V,E) and an activity lambda>0, we are interested in the quantity Z_G(lambda) defined as the sum over independent sets I weighted as w(I) = lambda^|I|. In statistical physics, Z_G(lambda) is the partition function for the hard-c… ▽ More We study the computational complexity of approximately counting the number of independent sets of a graph with maximum degree Delta. More generally, for an input graph G=(V,E) and an activity lambda>0, we are interested in the quantity Z_G(lambda) defined as the sum over independent sets I weighted as w(I) = lambda^|I|. In statistical physics, Z_G(lambda) is the partition function for the hard-core model, which is an idealized model of a gas where the particles have non-negibile size. Recently, an interesting phase transition was shown to occur for the complexity of approximating the partition function. Weitz showed an FPAS for the partition function for any graph of maximum degree Delta when Delta is constant and lambda< lambda_c(Tree_Delta):=(Delta-1)^(Delta-1)/(Delta-2)^Delta. The quantity lambda_c(Tree_Delta) is the critical point for the so-called uniqueness threshold on the infinite, regular tree of degree Delta. On the other side, Sly proved that there does not exist efficient (randomized) approximation algorithms for lambda_c(Tree_Delta) < lambda < lambda_c(Tree_Delta)+epsilon(Delta), unless NP=RP, for some function epsilon(Delta)>0. We remove the upper bound in the assumptions of Sly's result for Delta not equal to 4 and 5, that is, we show that there does not exist efficient randomized approximation algorithms for all lambda>lambda_c(Tree_Delta) for Delta=3 and Delta>= 6. Sly's inapproximability result uses a clever reduction, combined with a second-moment analysis of Mossel, Weitz and Wormald which prove torpid mixing of the Glauber dynamics for sampling from the associated Gibbs distribution on almost every regular graph of degree Delta for the same range of lambda as in Sly's result. We extend Sly's result by improving upon the technical work of Mossel et al., via a more detailed analysis of independent sets in random regular graphs. △ Less

Submitted 11 December, 2012; v1 submitted 25 May, 2011; originally announced May 2011.

Comments: to appear in Random Structures and Algorithms

ACM Class: F.2.2; G.3

The Complexity of Counting Eulerian Tours in 4-Regular Graphs

Authors: Qi Ge, Daniel Stefankovic

Abstract: We investigate the complexity of counting Eulerian tours ({\sc #ET}) and its variations from two perspectives---the complexity of exact counting and the complexity w.r.t. approximation-preserving reductions (AP-reductions \cite{MR2044886}). We prove that {\sc #ET} is #P-complete even for planar 4-regular graphs. A closely related problem is that of counting A-trails ({\sc #A-trails}) in graphs w… ▽ More We investigate the complexity of counting Eulerian tours ({\sc #ET}) and its variations from two perspectives---the complexity of exact counting and the complexity w.r.t. approximation-preserving reductions (AP-reductions \cite{MR2044886}). We prove that {\sc #ET} is #P-complete even for planar 4-regular graphs. A closely related problem is that of counting A-trails ({\sc #A-trails}) in graphs with rotational embedding schemes (so called maps). Kotzig \cite{MR0248043} showed that {\sc #A-trails} can be computed in polynomial time for 4-regular plane graphs (embedding in the plane is equivalent to giving a rotational embedding scheme). We show that for 4-regular maps the problem is #P-hard. Moreover, we show that from the approximation viewpoint {\sc #A-trails} in 4-regular maps captures the essence of {\sc #ET}, that is, we give an AP-reduction from {\sc #ET} in general graphs to {\sc #A-trails} in 4-regular maps. The reduction uses a fast mixing result for a card shuffling problem \cite{MR2023023}. In order to understand whether #{\sc A-trails} in 4-regular maps can AP-reduce to #{\sc ET} in 4-regular graphs, we investigate a problem in which transitions in vertices are weighted (this generalizes both #{\sc A-trails} and #{\sc ET}). In the 4-regular case we show that {\sc A-trails} can be used to simulate any vertex weights and provide evidence that {\sc ET} can simulate only a limited set of vertex weights. △ Less

Submitted 25 September, 2010; originally announced September 2010.

A Deterministic Polynomial-time Approximation Scheme for Counting Knapsack Solutions

Authors: Daniel Stefankovic, Santosh Vempala, Eric Vigoda

Abstract: Given n elements with nonnegative integer weights w1,..., wn and an integer capacity C, we consider the counting version of the classic knapsack problem: find the number of distinct subsets whose weights add up to at most the given capacity. We give a deterministic algorithm that estimates the number of solutions to within relative error 1+-eps in time polynomial in n and 1/eps (fully polynomial a… ▽ More Given n elements with nonnegative integer weights w1,..., wn and an integer capacity C, we consider the counting version of the classic knapsack problem: find the number of distinct subsets whose weights add up to at most the given capacity. We give a deterministic algorithm that estimates the number of solutions to within relative error 1+-eps in time polynomial in n and 1/eps (fully polynomial approximation scheme). More precisely, our algorithm takes time O(n^3 (1/eps) log (n/eps)). Our algorithm is based on dynamic programming. Previously, randomized polynomial time approximation schemes were known first by Morris and Sinclair via Markov chain Monte Carlo techniques, and subsequently by Dyer via dynamic programming and rejection sampling. △ Less

Submitted 10 August, 2010; originally announced August 2010.

Comments: 11 pages

ACM Class: F.2.2; G.2.1

Phase Transition for Glauber Dynamics for Independent Sets on Regular Trees

Authors: Ricardo Restrepo, Daniel Stefankovic, Juan C. Vera, Eric Vigoda, Linji Yang

Abstract: We study the effect of boundary conditions on the relaxation time of the Glauber dynamics for the hard-core model on the tree. The hard-core model is defined on the set of independent sets weighted by a parameter $λ$, called the activity. The Glauber dynamics is the Markov chain that updates a randomly chosen vertex in each step. On the infinite tree with branching factor $b$, the hard-core model… ▽ More We study the effect of boundary conditions on the relaxation time of the Glauber dynamics for the hard-core model on the tree. The hard-core model is defined on the set of independent sets weighted by a parameter $λ$, called the activity. The Glauber dynamics is the Markov chain that updates a randomly chosen vertex in each step. On the infinite tree with branching factor $b$, the hard-core model can be equivalently defined as a broadcasting process with a parameter $ω$ which is the positive solution to $λ=ω(1+ω)^b$, and vertices are occupied with probability $ω/(1+ω)$ when their parent is unoccupied. This broadcasting process undergoes a phase transition between the so-called reconstruction and non-reconstruction regions at $ω_r\approx \ln{b}/b$. Reconstruction has been of considerable interest recently since it appears to be intimately connected to the efficiency of local algorithms on locally tree-like graphs, such as sparse random graphs. In this paper we show that the relaxation time of the Glauber dynamics on regular $b$-ary trees $T_h$ of height $h$ and $n$ vertices, undergoes a phase transition around the reconstruction threshold. In particular, we construct a boundary condition for which the relaxation time slows down at the reconstruction threshold. More precisely, for any $ω\le \ln{b}/b$, for $T_h$ with any boundary condition, the relaxation time is $Ω(n)$ and $O(n^{1+o_b(1)})$. In contrast, above the reconstruction threshold we show that for every $δ>0$, for $ω=(1+δ)\ln{b}/b$, the relaxation time on $T_h$ with any boundary condition is $O(n^{1+δ+ o_b(1)})$, and we construct a boundary condition where the relaxation time is $Ω(n^{1+δ/2 - o_b(1)})$. △ Less

Submitted 14 July, 2010; originally announced July 2010.

MSC Class: 60J10

Fast Convergence of MCMC Algorithms for Phylogenetic Reconstruction with Homogeneous Data on Closely Related Species

Authors: Daniel Stefankovic, Eric Vigoda

Abstract: This paper studies a Markov chain for phylogenetic reconstruction which uses a popular transition between tree topologies known as subtree pruning-and-regrafting (SPR). We analyze the Markov chain in the simpler setting that the generating tree consists of very short edge lengths, short enough so that each sample from the generating tree (or character in phylogenetic terminology) is likely to have… ▽ More This paper studies a Markov chain for phylogenetic reconstruction which uses a popular transition between tree topologies known as subtree pruning-and-regrafting (SPR). We analyze the Markov chain in the simpler setting that the generating tree consists of very short edge lengths, short enough so that each sample from the generating tree (or character in phylogenetic terminology) is likely to have only one mutation, and that there enough samples so that the data looks like the generating distribution. We prove in this setting that the Markov chain is rapidly mixing, i.e., it quickly converges to its stationary distribution, which is the posterior distribution over tree topologies. Our proofs use that the leading term of the maximum likelihood function of a tree T is the maximum parsimony score, which is the size of the minimum cut in T needed to realize single edge cuts of the generating tree. Our main contribution is a combinatorial proof that in our simplified setting, SPR moves are guaranteed to converge quickly to the maximum parsimony tree. Our results are in contrast to recent works showing examples with heterogeneous data (namely, the data is generated from a mixture distribution) where many natural Markov chains are exponentially slow to converge to the stationary distribution. △ Less

Submitted 5 May, 2011; v1 submitted 30 March, 2010; originally announced March 2010.

Comments: To appear in SIAM Journal of Discrete Mathematics (SIDMA)

A graph polynomial for independent sets of bipartite graphs

Authors: Qi Ge, Daniel Stefankovic

Abstract: We introduce a new graph polynomial that encodes interesting properties of graphs, for example, the number of matchings and the number of perfect matchings. Most importantly, for bipartite graphs the polynomial encodes the number of independent sets (#BIS). We analyze the complexity of exact evaluation of the polynomial at rational points and show that for most points exact evaluation is #P-ha… ▽ More We introduce a new graph polynomial that encodes interesting properties of graphs, for example, the number of matchings and the number of perfect matchings. Most importantly, for bipartite graphs the polynomial encodes the number of independent sets (#BIS). We analyze the complexity of exact evaluation of the polynomial at rational points and show that for most points exact evaluation is #P-hard (assuming the generalized Riemann hypothesis) and for the rest of the points exact evaluation is trivial. We conjecture that a natural Markov chain can be used to approximately evaluate the polynomial for a range of parameters. The conjecture, if true, would imply an approximate counting algorithm for #BIS, a problem shown, by [Dyer et al. 2004], to be complete (with respect to, so called, AP-reductions) for a rich logically defined sub-class of #P. We give a mild support for our conjecture by proving that the Markov chain is rapidly mixing on trees. As a by-product we show that the "single bond flip" Markov chain for the random cluster model is rapidly mixing on constant tree-width graphs. △ Less

Submitted 10 February, 2010; v1 submitted 24 November, 2009; originally announced November 2009.

Approximating L1-distances between mixture distributions using random projections

Authors: Satyaki Mahalanabis, Daniel Stefankovic

Abstract: We consider the problem of computing L1-distances between every pair ofcprobability densities from a given family. We point out that the technique of Cauchy random projections (Indyk'06) in this context turns into stochastic integrals with respect to Cauchy motion. For piecewise-linear densities these integrals can be sampled from if one can sample from the stochastic integral of the function… ▽ More We consider the problem of computing L1-distances between every pair ofcprobability densities from a given family. We point out that the technique of Cauchy random projections (Indyk'06) in this context turns into stochastic integrals with respect to Cauchy motion. For piecewise-linear densities these integrals can be sampled from if one can sample from the stochastic integral of the function x->(1,x). We give an explicit density function for this stochastic integral and present an efficient sampling algorithm. As a consequence we obtain an efficient algorithm to approximate the L1-distances with a small relative error. For piecewise-polynomial densities we show how to approximately sample from the distributions resulting from the stochastic integrals. This also results in an efficient algorithm to approximate the L1-distances, although our inability to get exact samples worsens the dependence on the parameters. △ Less

Submitted 7 April, 2008; originally announced April 2008.

Density estimation in linear time

Authors: Satyaki Mahalanabis, Daniel Stefankovic

Abstract: We consider the problem of choosing a density estimate from a set of distributions F, minimizing the L1-distance to an unknown distribution (Devroye, Lugosi 2001). Devroye and Lugosi analyze two algorithms for the problem: Scheffe tournament winner and minimum distance estimate. The Scheffe tournament estimate requires fewer computations than the minimum distance estimate, but has strictly weake… ▽ More We consider the problem of choosing a density estimate from a set of distributions F, minimizing the L1-distance to an unknown distribution (Devroye, Lugosi 2001). Devroye and Lugosi analyze two algorithms for the problem: Scheffe tournament winner and minimum distance estimate. The Scheffe tournament estimate requires fewer computations than the minimum distance estimate, but has strictly weaker guarantees than the latter. We focus on the computational aspect of density estimation. We present two algorithms, both with the same guarantee as the minimum distance estimate. The first one, a modification of the minimum distance estimate, uses the same number (quadratic in |F|) of computations as the Scheffe tournament. The second one, called ``efficient minimum loss-weight estimate,'' uses only a linear number of computations, assuming that F is preprocessed. We also give examples showing that the guarantees of the algorithms cannot be improved and explore randomized algorithms for density estimation. △ Less

Submitted 17 December, 2007; originally announced December 2007.

Comments: 11 pages

Adaptive Simulated Annealing: A Near-optimal Connection between Sampling and Counting

Authors: Daniel Stefankovic, Santosh Vempala, Eric Vigoda

Abstract: We present a near-optimal reduction from approximately counting the cardinality of a discrete set to approximately sampling elements of the set. An important application of our work is to approximating the partition function $Z$ of a discrete system, such as the Ising model, matchings or colorings of a graph. The typical approach to estimating the partition function $Z(β^*)$ at some desired inve… ▽ More We present a near-optimal reduction from approximately counting the cardinality of a discrete set to approximately sampling elements of the set. An important application of our work is to approximating the partition function $Z$ of a discrete system, such as the Ising model, matchings or colorings of a graph. The typical approach to estimating the partition function $Z(β^*)$ at some desired inverse temperature $β^*$ is to define a sequence, which we call a {\em cooling schedule}, $β_0=0<β_1<...<β_\ell=β^*$ where Z(0) is trivial to compute and the ratios $Z(β_{i+1})/Z(β_i)$ are easy to estimate by sampling from the distribution corresponding to $Z(β_i)$. Previous approaches required a cooling schedule of length $O^*(\ln{A})$ where $A=Z(0)$, thereby ensuring that each ratio $Z(β_{i+1})/Z(β_i)$ is bounded. We present a cooling schedule of length $\ell=O^*(\sqrt{\ln{A}})$. For well-studied problems such as estimating the partition function of the Ising model, or approximating the number of colorings or matchings of a graph, our cooling schedule is of length $O^*(\sqrt{n})$, which implies an overall savings of $O^*(n)$ in the running time of the approximate counting algorithm (since roughly $\ell$ samples are needed to estimate each ratio). △ Less

Submitted 10 December, 2006; originally announced December 2006.

ACM Class: G.3