-
Single and Multi-Hop Question-Answering Datasets for Reticular Chemistry with GPT-4-Turbo
Authors:
Nakul Rampal,
Kaiyu Wang,
Matthew Burigana,
Lingxiang Hou,
Juri Al-Johani,
Anna Sackmann,
Hanan S. Murayshid,
Walaa Abdullah Al-Sumari,
Arwa M. Al-Abdulkarim,
Nahla Eid Al-Hazmi,
Majed O. Al-Awad,
Christian Borgs,
Jennifer T. Chayes,
Omar M. Yaghi
Abstract:
The rapid advancement in artificial intelligence and natural language processing has led to the development of large-scale datasets aimed at benchmarking the performance of machine learning models. Herein, we introduce 'RetChemQA,' a comprehensive benchmark dataset designed to evaluate the capabilities of such models in the domain of reticular chemistry. This dataset includes both single-hop and m…
▽ More
The rapid advancement in artificial intelligence and natural language processing has led to the development of large-scale datasets aimed at benchmarking the performance of machine learning models. Herein, we introduce 'RetChemQA,' a comprehensive benchmark dataset designed to evaluate the capabilities of such models in the domain of reticular chemistry. This dataset includes both single-hop and multi-hop question-answer pairs, encompassing approximately 45,000 Q&As for each type. The questions have been extracted from an extensive corpus of literature containing about 2,530 research papers from publishers including NAS, ACS, RSC, Elsevier, and Nature Publishing Group, among others. The dataset has been generated using OpenAI's GPT-4 Turbo, a cutting-edge model known for its exceptional language understanding and generation capabilities. In addition to the Q&A dataset, we also release a dataset of synthesis conditions extracted from the corpus of literature used in this study. The aim of RetChemQA is to provide a robust platform for the development and evaluation of advanced machine learning algorithms, particularly for the reticular chemistry community. The dataset is structured to reflect the complexities and nuances of real-world scientific discourse, thereby enabling nuanced performance assessments across a variety of tasks. The dataset is available at the following link: https://github.com/nakulrampal/RetChemQA
△ Less
Submitted 3 May, 2024;
originally announced May 2024.
-
Image and Data Mining in Reticular Chemistry Using GPT-4V
Authors:
Zhiling Zheng,
Zhiguo He,
Omar Khattab,
Nakul Rampal,
Matei A. Zaharia,
Christian Borgs,
Jennifer T. Chayes,
Omar M. Yaghi
Abstract:
The integration of artificial intelligence into scientific research has reached a new pinnacle with GPT-4V, a large language model featuring enhanced vision capabilities, accessible through ChatGPT or an API. This study demonstrates the remarkable ability of GPT-4V to navigate and obtain complex data for metal-organic frameworks, especially from graphical sources. Our approach involved an automate…
▽ More
The integration of artificial intelligence into scientific research has reached a new pinnacle with GPT-4V, a large language model featuring enhanced vision capabilities, accessible through ChatGPT or an API. This study demonstrates the remarkable ability of GPT-4V to navigate and obtain complex data for metal-organic frameworks, especially from graphical sources. Our approach involved an automated process of converting 346 scholarly articles into 6240 images, which represents a benchmark dataset in this task, followed by deploying GPT-4V to categorize and analyze these images using natural language prompts. This methodology enabled GPT-4V to accurately identify and interpret key plots integral to MOF characterization, such as nitrogen isotherms, PXRD patterns, and TGA curves, among others, with accuracy and recall above 93%. The model's proficiency in extracting critical information from these plots not only underscores its capability in data mining but also highlights its potential in aiding the creation of comprehensive digital databases for reticular chemistry. In addition, the extracted nitrogen isotherm data from the selected literature allowed for a comparison between theoretical and experimental porosity values for over 200 compounds, highlighting certain discrepancies and underscoring the importance of integrating computational and experimental data. This work highlights the potential of AI in accelerating scientific discovery and innovation, bridging the gap between computational tools and experimental research, and paving the way for more efficient, inclusive, and comprehensive scientific inquiry.
△ Less
Submitted 9 December, 2023;
originally announced December 2023.
-
A GPT-4 Reticular Chemist for Guiding MOF Discovery
Authors:
Zhiling Zheng,
Zichao Rong,
Nakul Rampal,
Christian Borgs,
Jennifer T. Chayes,
Omar M. Yaghi
Abstract:
We present a new framework integrating the AI model GPT-4 into the iterative process of reticular chemistry experimentation, leveraging a cooperative workflow of interaction between AI and a human researcher. This GPT-4 Reticular Chemist is an integrated system composed of three phases. Each of these utilizes GPT-4 in various capacities, wherein GPT-4 provides detailed instructions for chemical ex…
▽ More
We present a new framework integrating the AI model GPT-4 into the iterative process of reticular chemistry experimentation, leveraging a cooperative workflow of interaction between AI and a human researcher. This GPT-4 Reticular Chemist is an integrated system composed of three phases. Each of these utilizes GPT-4 in various capacities, wherein GPT-4 provides detailed instructions for chemical experimentation and the human provides feedback on the experimental outcomes, including both success and failures, for the in-context learning of AI in the next iteration. This iterative human-AI interaction enabled GPT-4 to learn from the outcomes, much like an experienced chemist, by a prompt-learning strategy. Importantly, the system is based on natural language for both development and operation, eliminating the need for coding skills, and thus, make it accessible to all chemists. Our collaboration with GPT-4 Reticular Chemist guided the discovery of an isoreticular series of MOFs, with each synthesis fine-tuned through iterative feedback and expert suggestions. This workflow presents a potential for broader applications in scientific research by harnessing the capability of large language models like GPT-4 to enhance the feasibility and efficiency of research activities.
△ Less
Submitted 3 October, 2023; v1 submitted 20 June, 2023;
originally announced June 2023.
-
ChatGPT Chemistry Assistant for Text Mining and Prediction of MOF Synthesis
Authors:
Zhiling Zheng,
Oufan Zhang,
Christian Borgs,
Jennifer T. Chayes,
Omar M. Yaghi
Abstract:
We use prompt engineering to guide ChatGPT in the automation of text mining of metal-organic frameworks (MOFs) synthesis conditions from diverse formats and styles of the scientific literature. This effectively mitigates ChatGPT's tendency to hallucinate information -- an issue that previously made the use of Large Language Models (LLMs) in scientific fields challenging. Our approach involves the…
▽ More
We use prompt engineering to guide ChatGPT in the automation of text mining of metal-organic frameworks (MOFs) synthesis conditions from diverse formats and styles of the scientific literature. This effectively mitigates ChatGPT's tendency to hallucinate information -- an issue that previously made the use of Large Language Models (LLMs) in scientific fields challenging. Our approach involves the development of a workflow implementing three different processes for text mining, programmed by ChatGPT itself. All of them enable parsing, searching, filtering, classification, summarization, and data unification with different tradeoffs between labor, speed, and accuracy. We deploy this system to extract 26,257 distinct synthesis parameters pertaining to approximately 800 MOFs sourced from peer-reviewed research articles. This process incorporates our ChemPrompt Engineering strategy to instruct ChatGPT in text mining, resulting in impressive precision, recall, and F1 scores of 90-99%. Furthermore, with the dataset built by text mining, we constructed a machine-learning model with over 86% accuracy in predicting MOF experimental crystallization outcomes and preliminarily identifying important factors in MOF crystallization. We also developed a reliable data-grounded MOF chatbot to answer questions on chemical reactions and synthesis procedures. Given that the process of using ChatGPT reliably mines and tabulates diverse MOF synthesis information in a unified format, while using only narrative language requiring no coding expertise, we anticipate that our ChatGPT Chemistry Assistant will be very useful across various other chemistry sub-disciplines.
△ Less
Submitted 19 July, 2023; v1 submitted 20 June, 2023;
originally announced June 2023.
-
Estimating Total Treatment Effect in Randomized Experiments with Unknown Network Structure
Authors:
Christina Lee Yu,
Edoardo M Airoldi,
Christian Borgs,
Jennifer T Chayes
Abstract:
Randomized experiments are widely used to estimate the causal effects of a proposed treatment in many areas of science, from medicine and healthcare to the physical and biological sciences, from the social sciences to engineering, to public policy and to the technology industry at large. Here, we consider situations where classical methods for estimating the total treatment effect on a target popu…
▽ More
Randomized experiments are widely used to estimate the causal effects of a proposed treatment in many areas of science, from medicine and healthcare to the physical and biological sciences, from the social sciences to engineering, to public policy and to the technology industry at large. Here, we consider situations where classical methods for estimating the total treatment effect on a target population are considerably biased due to confounding network effects, i.e., the fact that the treatment of an individual may impact their neighbors' outcomes, an issue referred to as network interference or as non-individualized treatment response. A key challenge in these situations, is that the network is often unknown, and difficult, or costly, to measure. In this paper, we characterize the limitations in estimating the total treatment effect without knowledge of the network that drives interference, assuming a potential outcomes model with heterogeneous additive network effects. This model encompasses a broad class of network interference sources, including spillover, peer effects, and contagion. Within this framework, we show that, surprisingly, given access to average historical baseline measurements prior to the experiment, we can develop a simple estimator and efficient randomized design that outputs an unbiased estimate with low variance. Our solution does not require knowledge of the underlying network structure, and it comes with statistical guarantees for a broad class of models. We believe our results are poised to impact current randomized experimentation strategies due to its ease of interpretation and implementation, alongside its provable theoretical insights under heterogeneous network effects.
△ Less
Submitted 24 September, 2022; v1 submitted 25 May, 2022;
originally announced May 2022.
-
A correction to Kallenberg's theorem for jointly exchangeable random measures
Authors:
Christian Borgs,
Jennifer T. Chayes,
Souvik Dhara,
Subhabrata Sen
Abstract:
Kallenberg (2005) provided a necessary and sufficient condition for the local finiteness of a jointly exchangeable random measure on $\R_+^2$. Here we note an additional condition that was missing in Kallenberg's theorem, but was implicitly used in the proof. We also provide a counter-example when the additional condition does not hold.
Kallenberg (2005) provided a necessary and sufficient condition for the local finiteness of a jointly exchangeable random measure on $\R_+^2$. Here we note an additional condition that was missing in Kallenberg's theorem, but was implicitly used in the proof. We also provide a counter-example when the additional condition does not hold.
△ Less
Submitted 2 July, 2019;
originally announced July 2019.
-
Limits of Sparse Configuration Models and Beyond: Graphexes and Multi-Graphexes
Authors:
Christian Borgs,
Jennifer T. Chayes,
Souvik Dhara,
Subhabrata Sen
Abstract:
We investigate structural properties of large, sparse random graphs through the lens of "sampling convergence" (Borgs et. al. (2017)). Sampling convergence generalizes left convergence to sparse graphs, and describes the limit in terms of a "graphex". We introduce a notion of sampling convergence for sequences of multigraphs, and establish the graphex limit for the configuration model, a preferent…
▽ More
We investigate structural properties of large, sparse random graphs through the lens of "sampling convergence" (Borgs et. al. (2017)). Sampling convergence generalizes left convergence to sparse graphs, and describes the limit in terms of a "graphex". We introduce a notion of sampling convergence for sequences of multigraphs, and establish the graphex limit for the configuration model, a preferential attachment model, the generalized random graph, and a bipartite variant of the configuration model. The results for the configuration model, preferential attachment model and bipartite configuration model provide necessary and sufficient conditions for these random graph models to converge. The limit for the configuration model and the preferential attachment model is an augmented version of an exchangeable random graph model introduced by Caron and Fox (2017).
△ Less
Submitted 2 July, 2019;
originally announced July 2019.
-
Identifiability for graphexes and the weak kernel metric
Authors:
Christian Borgs,
Jennifer T. Chayes,
Henry Cohn,
László Miklós Lovász
Abstract:
In two recent papers by Veitch and Roy and by Borgs, Chayes, Cohn, and Holden, a new class of sparse random graph processes based on the concept of graphexes over $σ$-finite measure spaces has been introduced. In this paper, we introduce a metric for graphexes that generalizes the cut metric for the graphons of the dense theory of graph convergence. We show that a sequence of graphexes converges i…
▽ More
In two recent papers by Veitch and Roy and by Borgs, Chayes, Cohn, and Holden, a new class of sparse random graph processes based on the concept of graphexes over $σ$-finite measure spaces has been introduced. In this paper, we introduce a metric for graphexes that generalizes the cut metric for the graphons of the dense theory of graph convergence. We show that a sequence of graphexes converges in this metric if and only if the sequence of graph processes generated by the graphexes converges in distribution. In the course of the proof, we establish a regularity lemma and determine which sets of graphexes are precompact under our metric. Finally, we establish an identifiability theorem, characterizing when two graphexes are equivalent in the sense that they lead to the same process of random graphs.
△ Less
Submitted 9 April, 2018;
originally announced April 2018.
-
Sampling perspectives on sparse exchangeable graphs
Authors:
Christian Borgs,
Jennifer T. Chayes,
Henry Cohn,
Victor Veitch
Abstract:
Recent work has introduced sparse exchangeable graphs and the associated graphex framework, as a generalization of dense exchangeable graphs and the associated graphon framework. The development of this subject involves the interplay between the statistical modeling of network data, the theory of large graph limits, exchangeability, and network sampling. The purpose of the present paper is to clar…
▽ More
Recent work has introduced sparse exchangeable graphs and the associated graphex framework, as a generalization of dense exchangeable graphs and the associated graphon framework. The development of this subject involves the interplay between the statistical modeling of network data, the theory of large graph limits, exchangeability, and network sampling. The purpose of the present paper is to clarify the relationships between these subjects by explaining each in terms of a certain natural sampling scheme associated with the graphex model. The first main technical contribution is the introduction of sampling convergence, a new notion of graph limit that generalizes left convergence so that it becomes meaningful for the sparse graph regime. The second main technical contribution is the demonstration that the (somewhat cryptic) notion of exchangeability underpinning the graphex framework is equivalent to a more natural probabilistic invariance expressed in terms of the sampling scheme.
△ Less
Submitted 10 February, 2020; v1 submitted 10 August, 2017;
originally announced August 2017.
-
Graphons: A Nonparametric Method to Model, Estimate, and Design Algorithms for Massive Networks
Authors:
Christian Borgs,
Jennifer T. Chayes
Abstract:
Many social and economic systems are naturally represented as networks, from off-line and on-line social networks, to bipartite networks, like Netflix and Amazon, between consumers and products. Graphons, developed as limits of graphs, form a natural, nonparametric method to describe and estimate large networks like Facebook and LinkedIn. Here we describe the development of the theory of graphons,…
▽ More
Many social and economic systems are naturally represented as networks, from off-line and on-line social networks, to bipartite networks, like Netflix and Amazon, between consumers and products. Graphons, developed as limits of graphs, form a natural, nonparametric method to describe and estimate large networks like Facebook and LinkedIn. Here we describe the development of the theory of graphons, for both dense and sparse networks, over the last decade. We also review theorems showing that we can consistently estimate graphons from massive networks in a wide variety of models. Finally, we show how to use graphons to estimate missing links in a sparse network, which has applications from estimating social and information networks in development economics, to rigorously and efficiently doing collaborative filtering with applications to movie recommendations in Netflix and product suggestions in Amazon.
△ Less
Submitted 4 June, 2017;
originally announced June 2017.
-
Sparse exchangeable graphs and their limits via graphon processes
Authors:
Christian Borgs,
Jennifer T. Chayes,
Henry Cohn,
Nina Holden
Abstract:
In a recent paper, Caron and Fox suggest a probabilistic model for sparse graphs which are exchangeable when associating each vertex with a time parameter in $\mathbb{R}_+$. Here we show that by generalizing the classical definition of graphons as functions over probability spaces to functions over $σ$-finite measure spaces, we can model a large family of exchangeable graphs, including the Caron-F…
▽ More
In a recent paper, Caron and Fox suggest a probabilistic model for sparse graphs which are exchangeable when associating each vertex with a time parameter in $\mathbb{R}_+$. Here we show that by generalizing the classical definition of graphons as functions over probability spaces to functions over $σ$-finite measure spaces, we can model a large family of exchangeable graphs, including the Caron-Fox graphs and the traditional exchangeable dense graphs as special cases. Explicitly, modelling the underlying space of features by a $σ$-finite measure space $(S,\mathcal{S},μ)$ and the connection probabilities by an integrable function $W\colon S\times S\to [0,1]$, we construct a random family $(G_t)_{t\geq 0}$ of growing graphs such that the vertices of $G_t$ are given by a Poisson point process on $S$ with intensity $tμ$, with two points $x,y$ of the point process connected with probability $W(x,y)$. We call such a random family a graphon process. We prove that a graphon process has convergent subgraph frequencies (with possibly infinite limits) and that, in the natural extension of the cut metric to our setting, the sequence converges to the generating graphon. We also show that the underlying graphon is identifiable only as an equivalence class over graphons with cut distance zero. More generally, we study metric convergence for arbitrary (not necessarily random) sequences of graphs, and show that a sequence of graphs has a convergent subsequence if and only if it has a subsequence satisfying a property we call uniform regularity of tails. Finally, we prove that every graphon is equivalent to a graphon on $\mathbb{R}_+$ equipped with Lebesgue measure.
△ Less
Submitted 20 June, 2018; v1 submitted 26 January, 2016;
originally announced January 2016.
-
Consistent nonparametric estimation for heavy-tailed sparse graphs
Authors:
Christian Borgs,
Jennifer T. Chayes,
Henry Cohn,
Shirshendu Ganguly
Abstract:
We study graphons as a non-parametric generalization of stochastic block models, and show how to obtain compactly represented estimators for sparse networks in this framework. Our algorithms and analysis go beyond previous work in several ways. First, we relax the usual boundedness assumption for the generating graphon and instead treat arbitrary integrable graphons, so that we can handle networks…
▽ More
We study graphons as a non-parametric generalization of stochastic block models, and show how to obtain compactly represented estimators for sparse networks in this framework. Our algorithms and analysis go beyond previous work in several ways. First, we relax the usual boundedness assumption for the generating graphon and instead treat arbitrary integrable graphons, so that we can handle networks with long tails in their degree distributions. Second, again motivated by real-world applications, we relax the usual assumption that the graphon is defined on the unit interval, to allow latent position graphs where the latent positions live in a more general space, and we characterize identifiability for these graphons and their underlying position spaces.
We analyze three algorithms. The first is a least squares algorithm, which gives an approximation we prove to be consistent for all square-integrable graphons, with errors expressed in terms of the best possible stochastic block model approximation to the generating graphon. Next, we analyze a generalization based on the cut norm, which works for any integrable graphon (not necessarily square-integrable). Finally, we show that clustering based on degrees works whenever the underlying degree distribution is atomless. Unlike the previous two algorithms, this third one runs in polynomial time.
△ Less
Submitted 24 February, 2016; v1 submitted 26 August, 2015;
originally announced August 2015.
-
Private Graphon Estimation for Sparse Graphs
Authors:
Christian Borgs,
Jennifer T. Chayes,
Adam Smith
Abstract:
We design algorithms for fitting a high-dimensional statistical model to a large, sparse network without revealing sensitive information of individual members. Given a sparse input graph $G$, our algorithms output a node-differentially-private nonparametric block model approximation. By node-differentially-private, we mean that our output hides the insertion or removal of a vertex and all its adja…
▽ More
We design algorithms for fitting a high-dimensional statistical model to a large, sparse network without revealing sensitive information of individual members. Given a sparse input graph $G$, our algorithms output a node-differentially-private nonparametric block model approximation. By node-differentially-private, we mean that our output hides the insertion or removal of a vertex and all its adjacent edges. If $G$ is an instance of the network obtained from a generative nonparametric model defined in terms of a graphon $W$, our model guarantees consistency, in the sense that as the number of vertices tends to infinity, the output of our algorithm converges to $W$ in an appropriate version of the $L_2$ norm. In particular, this means we can estimate the sizes of all multi-way cuts in $G$.
Our results hold as long as $W$ is bounded, the average degree of $G$ grows at least like the log of the number of vertices, and the number of blocks goes to infinity at an appropriate rate. We give explicit error bounds in terms of the parameters of the model; in several settings, our bounds improve on or match known nonprivate results.
△ Less
Submitted 19 June, 2015;
originally announced June 2015.
-
An $L^p$ theory of sparse graph convergence II: LD convergence, quotients, and right convergence
Authors:
Christian Borgs,
Jennifer T. Chayes,
Henry Cohn,
Yufei Zhao
Abstract:
We extend the $L^p$ theory of sparse graph limits, which was introduced in a companion paper, by analyzing different notions of convergence. Under suitable restrictions on node weights, we prove the equivalence of metric convergence, quotient convergence, microcanonical ground state energy convergence, microcanonical free energy convergence, and large deviation convergence. Our theorems extend the…
▽ More
We extend the $L^p$ theory of sparse graph limits, which was introduced in a companion paper, by analyzing different notions of convergence. Under suitable restrictions on node weights, we prove the equivalence of metric convergence, quotient convergence, microcanonical ground state energy convergence, microcanonical free energy convergence, and large deviation convergence. Our theorems extend the broad applicability of dense graph convergence to all sparse graphs with unbounded average degree, while the proofs require new techniques based on uniform upper regularity. Examples to which our theory applies include stochastic block models, power law graphs, and sparse versions of $W$-random graphs.
△ Less
Submitted 4 August, 2014;
originally announced August 2014.
-
An $L^p$ theory of sparse graph convergence I: limits, sparse random graph models, and power law distributions
Authors:
Christian Borgs,
Jennifer T. Chayes,
Henry Cohn,
Yufei Zhao
Abstract:
We introduce and develop a theory of limits for sequences of sparse graphs based on $L^p$ graphons, which generalizes both the existing $L^\infty$ theory of dense graph limits and its extension by Bollobás and Riordan to sparse graphs without dense spots. In doing so, we replace the no dense spots hypothesis with weaker assumptions, which allow us to analyze graphs with power law degree distributi…
▽ More
We introduce and develop a theory of limits for sequences of sparse graphs based on $L^p$ graphons, which generalizes both the existing $L^\infty$ theory of dense graph limits and its extension by Bollobás and Riordan to sparse graphs without dense spots. In doing so, we replace the no dense spots hypothesis with weaker assumptions, which allow us to analyze graphs with power law degree distributions. This gives the first broadly applicable limit theory for sparse graphs with unbounded average degrees. In this paper, we lay the foundations of the $L^p$ theory of graphons, characterize convergence, and develop corresponding random graph models, while we prove the equivalence of several alternative metrics in a companion paper.
△ Less
Submitted 29 December, 2014; v1 submitted 13 January, 2014;
originally announced January 2014.
-
Asymptotic behavior and distributional limits of preferential attachment graphs
Authors:
Noam Berger,
Christian Borgs,
Jennifer T. Chayes,
Amin Saberi
Abstract:
We give an explicit construction of the weak local limit of a class of preferential attachment graphs. This limit contains all local information and allows several computations that are otherwise hard, for example, joint degree distributions and, more generally, the limiting distribution of subgraphs in balls of any given radius $k$ around a random vertex in the preferential attachment graph. We a…
▽ More
We give an explicit construction of the weak local limit of a class of preferential attachment graphs. This limit contains all local information and allows several computations that are otherwise hard, for example, joint degree distributions and, more generally, the limiting distribution of subgraphs in balls of any given radius $k$ around a random vertex in the preferential attachment graph. We also establish the finite-volume corrections which give the approach to the limit.
△ Less
Submitted 13 January, 2014;
originally announced January 2014.
-
Tight Bounds for Mixing of the Swendsen-Wang Algorithm at the Potts Transition Point
Authors:
Christian Borgs,
Jennifer T. Chayes,
Prasad Tetali
Abstract:
We study two widely used algorithms for the Potts model on rectangular subsets of the hypercubic lattice Z^d - heat bath dynamics and the Swendsen-Wang algorithm - and prove that, under certain circumstances, the mixing in these algorithms is torpid or slow. In particular, we show that for heat bath dynamics throughout the region of phase coexistence, and for the Swendsen-Wang algorithm at the tra…
▽ More
We study two widely used algorithms for the Potts model on rectangular subsets of the hypercubic lattice Z^d - heat bath dynamics and the Swendsen-Wang algorithm - and prove that, under certain circumstances, the mixing in these algorithms is torpid or slow. In particular, we show that for heat bath dynamics throughout the region of phase coexistence, and for the Swendsen-Wang algorithm at the transition point, the mixing time in a box of side length L with periodic boundary conditions has upper and lower bounds which are exponential in L^{d-1}. This work provides the first upper bound of this form for the Swendsen-Wang algorithm, and gives lower bounds for both algorithms which significantly improve the previous lower bounds that were exponential in L/(log L)^2.
△ Less
Submitted 12 November, 2010;
originally announced November 2010.
-
Convergent Sequences of Dense Graphs I: Subgraph Frequencies, Metric Properties and Testing
Authors:
C. Borgs,
J. T. Chayes,
L. Lovasz,
V. T. Sos,
K. Vesztergombi
Abstract:
We consider sequences of graphs and define various notions of convergence related to these sequences: ``left convergence'' defined in terms of the densities of homomorphisms from small graphs into the graphs of the sequence, and ``right convergence'' defined in terms of the densities of homomorphisms from the graphs of the sequence into small graphs; and convergence in a suitably defined metric.…
▽ More
We consider sequences of graphs and define various notions of convergence related to these sequences: ``left convergence'' defined in terms of the densities of homomorphisms from small graphs into the graphs of the sequence, and ``right convergence'' defined in terms of the densities of homomorphisms from the graphs of the sequence into small graphs; and convergence in a suitably defined metric.
In Part I of this series, we show that left convergence is equivalent to convergence in metric, both for simple graphs, and for graphs with nodeweights and edgeweights. One of the main steps here is the introduction of a cut-distance comparing graphs, not necessarily of the same size. We also show how these notions of convergence provide natural formulations of Szemeredi partitions, sampling and testing of large graphs.
△ Less
Submitted 31 January, 2007;
originally announced February 2007.
-
Fitting the WHOIS Internet data
Authors:
R. M. D'Souza,
C. Borgs,
J. T. Chayes,
N. Berger,
R. D. Kleinberg
Abstract:
We consider the RIPE WHOIS Internet data as characterized by the Cooperative Association for Internet Data Analysis (CAIDA), and show that the Tempered Preferential Attachment model [1] provides an excellent fit to this data.
[1] D'Souza, Borgs, Chayes, Berger and Kleinberg, to appear PNAS USA, 2007.
We consider the RIPE WHOIS Internet data as characterized by the Cooperative Association for Internet Data Analysis (CAIDA), and show that the Tempered Preferential Attachment model [1] provides an excellent fit to this data.
[1] D'Souza, Borgs, Chayes, Berger and Kleinberg, to appear PNAS USA, 2007.
△ Less
Submitted 30 January, 2007;
originally announced January 2007.
-
Degree Distribution of Competition-Induced Preferential Attachment Graphs
Authors:
N. Berger,
C. Borgs,
J. T. Chayes,
R. M. D'Souza,
R. D. Kleinberg
Abstract:
We introduce a family of one-dimensional geometric growth models, constructed iteratively by locally optimizing the tradeoffs between two competing metrics, and show that this family is equivalent to a family of preferential attachment random graph models with upper cutoffs. This is the first explanation of how preferential attachment can arise from a more basic underlying mechanism of local com…
▽ More
We introduce a family of one-dimensional geometric growth models, constructed iteratively by locally optimizing the tradeoffs between two competing metrics, and show that this family is equivalent to a family of preferential attachment random graph models with upper cutoffs. This is the first explanation of how preferential attachment can arise from a more basic underlying mechanism of local competition. We rigorously determine the degree distribution for the family of random graph models, showing that it obeys a power law up to a finite threshold and decays exponentially above this threshold.
We also rigorously analyze a generalized version of our graph process, with two natural parameters, one corresponding to the cutoff and the other a ``fertility'' parameter. We prove that the general model has a power-law degree distribution up to a cutoff, and establish monotonicity of the power as a function of the two parameters. Limiting cases of the general model include the standard preferential attachment model without cutoff and the uniform attachment model.
△ Less
Submitted 8 February, 2005; v1 submitted 8 February, 2005;
originally announced February 2005.
-
Competition-Induced Preferential Attachment
Authors:
N. Berger,
C. Borgs,
J. T. Chayes,
R. M. D'Souza,
R. D. Kleinberg
Abstract:
Models based on preferential attachment have had much success in reproducing the power law degree distributions which seem ubiquitous in both natural and engineered systems. Here, rather than assuming preferential attachment, we give an explanation of how it can arise from a more basic underlying mechanism of competition between opposing forces.
We introduce a family of one-dimensional geometr…
▽ More
Models based on preferential attachment have had much success in reproducing the power law degree distributions which seem ubiquitous in both natural and engineered systems. Here, rather than assuming preferential attachment, we give an explanation of how it can arise from a more basic underlying mechanism of competition between opposing forces.
We introduce a family of one-dimensional geometric growth models, constructed iteratively by locally optimizing the tradeoffs between two competing metrics. This family admits an equivalent description as a graph process with no reference to the underlying geometry. Moreover, the resulting graph process is shown to be preferential attachment with an upper cutoff. We rigorously determine the degree distribution for the family of random graph models, showing that it obeys a power law up to a finite threshold and decays exponentially above this threshold.
We also introduce and rigorously analyze a generalized version of our graph process, with two natural parameters, one corresponding to the cutoff and the other a ``fertility'' parameter. Limiting cases of this process include the standard Barabasi-Albert preferential attachment model and the uniform attachment model. In the general case, we prove that the process has a power law degree distribution up to a cutoff, and establish monotonicity of the power as a function of the two parameters.
△ Less
Submitted 10 February, 2004;
originally announced February 2004.
-
Random subgraphs of finite graphs: III. The phase transition for the $n$-cube
Authors:
Christian Borgs,
Jennifer T. Chayes,
Remco van der Hofstad,
Gordon Slade,
Joel Spencer
Abstract:
We study random subgraphs of the $n$-cube $\{0,1\}^n$, where nearest-neighbor edges are occupied with probability $p$. Let $p_c(n)$ be the value of $p$ for which the expected cluster size of a fixed vertex attains the value $λ2^{n/3}$, where $λ$ is a small positive constant. Let $ε=n(p-p_c(n))$. In two previous papers, we showed that the largest cluster inside a scaling window given by…
▽ More
We study random subgraphs of the $n$-cube $\{0,1\}^n$, where nearest-neighbor edges are occupied with probability $p$. Let $p_c(n)$ be the value of $p$ for which the expected cluster size of a fixed vertex attains the value $λ2^{n/3}$, where $λ$ is a small positive constant. Let $ε=n(p-p_c(n))$. In two previous papers, we showed that the largest cluster inside a scaling window given by $|ε|=Θ(2^{-n/3})$ is of size $Θ(2^{2n/3})$, below this scaling window it is at most $2(\log2) nε^{-2}$, and above this scaling window it is at most $O(ε2^n)$. In this paper, we prove that for $p - p_c(n) \geq e^{-cn^{1/3}}$ the size of the largest cluster is at least $Θ(ε2^n)$, which is of the same order as the upper bound. This provides an understanding of the phase transition that goes far beyond that obtained by previous authors. The proof is based on a method that has come to be known as ``sprinkling,'' and relies heavily on the specific geometry of the $n$-cube.
△ Less
Submitted 8 January, 2004;
originally announced January 2004.
-
Random subgraphs of finite graphs: II. The lace expansion and the triangle condition
Authors:
Christian Borgs,
Jennifer T. Chayes,
Remco van der Hofstad,
Gordon Slade,
Joel Spencer
Abstract:
In a previous paper, we defined a version of the percolation triangle condition that is suitable for the analysis of bond percolation on a finite connected transitive graph, and showed that this triangle condition implies that the percolation phase transition has many features in common with the phase transition on the complete graph. In this paper, we use a new and simplified approach to the la…
▽ More
In a previous paper, we defined a version of the percolation triangle condition that is suitable for the analysis of bond percolation on a finite connected transitive graph, and showed that this triangle condition implies that the percolation phase transition has many features in common with the phase transition on the complete graph. In this paper, we use a new and simplified approach to the lace expansion to prove quite generally that for finite graphs that are tori the triangle condition for percolation is implied by a certain triangle condition for simple random walks on the graph.
The latter is readily verified for several graphs with vertex set $\{0,1,..., r-1\}^n$, including the Hamming cube on an alphabet of $r$ letters (the $n$-cube, for $r=2$), the $n$-dimensional torus with nearest-neighbor bonds and $n$ sufficiently large, and the $n$-dimensional torus with $n>6$ and sufficiently spread-out (long range) bonds. The conclusions of our previous paper thus apply to the percolation phase transition for each of the above examples.
△ Less
Submitted 8 January, 2004;
originally announced January 2004.
-
Random subgraphs of finite graphs: I. The scaling window under the triangle condition
Authors:
Christian Borgs,
Jennifer T. Chayes,
Remco van der Hofstad,
Gordon Slade,
Joel Spencer
Abstract:
We study random subgraphs of an arbitrary finite connected transitive graph $\mathbb G$ obtained by independently deleting edges with probability $1-p$. Let $V$ be the number of vertices in $\mathbb G$, and let $Ω$ be their degree. We define the critical threshold $p_c=p_c(\mathbb G,λ)$ to be the value of $p$ for which the expected cluster size of a fixed vertex attains the value $λV^{1/3}$, whe…
▽ More
We study random subgraphs of an arbitrary finite connected transitive graph $\mathbb G$ obtained by independently deleting edges with probability $1-p$. Let $V$ be the number of vertices in $\mathbb G$, and let $Ω$ be their degree. We define the critical threshold $p_c=p_c(\mathbb G,λ)$ to be the value of $p$ for which the expected cluster size of a fixed vertex attains the value $λV^{1/3}$, where $λ$ is fixed and positive. We show that for any such model, there is a phase transition at $p_c$ analogous to the phase transition for the random graph, provided that a quantity called the triangle diagram is sufficiently small at the threshold $p_c$. In particular, we show that the largest cluster inside a scaling window of size $|p-p_c|=Θ(\cn^{-1}V^{-1/3})$ is of size $Θ(V^{2/3})$, while below this scaling window, it is much smaller, of order $O(ε^{-2}\log(Vε^3))$, with $ε=\cn(p_c-p)$. We also obtain an upper bound $O(\cn(p-p_c)V)$ for the expected size of the largest cluster above the window. In addition, we define and analyze the percolation probability above the window and show that it is of order $Θ(\cn(p-p_c))$. Among the models for which the triangle diagram is small enough to allow us to draw these conclusions are the random graph, the $n$-cube and certain Hamming cubes, as well as the spread-out $n$-dimensional torus for $n>6$.
△ Less
Submitted 8 January, 2004;
originally announced January 2004.
-
Partition function zeros at first-order phase transitions: Pirogov-Sinai theory
Authors:
Marek Biskup,
Christian Borgs,
Jennifer T. Chayes,
Roman Kotecky
Abstract:
This paper is a continuation of our previous analysis [BBCKK] of partition functions zeros in models with first-order phase transitions and periodic boundary conditions. Here it is shown that the assumptions under which the results of [BBCKK] were established are satisfied by a large class of lattice models. These models are characterized by two basic properties: The existence of only a finite n…
▽ More
This paper is a continuation of our previous analysis [BBCKK] of partition functions zeros in models with first-order phase transitions and periodic boundary conditions. Here it is shown that the assumptions under which the results of [BBCKK] were established are satisfied by a large class of lattice models. These models are characterized by two basic properties: The existence of only a finite number of ground states and the availability of an appropriate contour representation. This setting includes, for instance, the Ising, Potts and Blume-Capel models at low temperatures. The combined results of [BBCKK] and the present paper provide complete control of the zeros of the partition function with periodic boundary conditions for all models in the above class.
△ Less
Submitted 6 May, 2004; v1 submitted 14 December, 2003;
originally announced December 2003.
-
Partition function zeros at first-order phase transitions: A general analysis
Authors:
Marek Biskup,
Christian Borgs,
Jennifer T. Chayes,
Logan J. Kleinwaks,
Roman Kotecky
Abstract:
We present a general, rigorous theory of partition function zeros for lattice spin models depending on one complex parameter. First, we formulate a set of natural assumptions which are verified for a large class of spin models in a companion paper [BBCKK2, math-ph/0304007]. Under these assumptions, we derive equations whose solutions give the location of the zeros of the partition function with…
▽ More
We present a general, rigorous theory of partition function zeros for lattice spin models depending on one complex parameter. First, we formulate a set of natural assumptions which are verified for a large class of spin models in a companion paper [BBCKK2, math-ph/0304007]. Under these assumptions, we derive equations whose solutions give the location of the zeros of the partition function with periodic boundary conditions, up to an error which we prove is (generically) exponentially small in the linear size of the system. For asymptotically large systems, the zeros concentrate on phase boundaries which are simple curves ending in multiple points. For models with an Ising-like plus-minus symmetry, we also establish a local version of the Lee-Yang Circle Theorem. This result allows us to control situations when in one region of the complex plane the zeros lie precisely on the unit circle, while in the complement of this region the zeros concentrate on less symmetric curves.
△ Less
Submitted 6 May, 2004; v1 submitted 3 April, 2003;
originally announced April 2003.
-
Phase Diagram for the Constrained Integer Partitioning Problem
Authors:
C. Borgs,
J. T. Chayes,
S. Mertens,
B. Pittel
Abstract:
We consider the problem of partitioning $n$ integers into two subsets of given cardinalities such that the discrepancy, the absolute value of the difference of their sums, is minimized. The integers are i.i.d. random variables chosen uniformly from the set $\{1,...,M\}$. We study how the typical behavior of the optimal partition depends on $n,M$ and the bias $s$, the difference between the cardi…
▽ More
We consider the problem of partitioning $n$ integers into two subsets of given cardinalities such that the discrepancy, the absolute value of the difference of their sums, is minimized. The integers are i.i.d. random variables chosen uniformly from the set $\{1,...,M\}$. We study how the typical behavior of the optimal partition depends on $n,M$ and the bias $s$, the difference between the cardinalities of the two subsets in the partition. In particular, we rigorously establish this typical behavior as a function of the two parameters $κ:=n^{-1}\log_2M$ and $b:=|s|/n$ by proving the existence of three distinct ``phases'' in the $κb$-plane, characterized by the value of the discrepancy and the number of optimal solutions: a ``perfect phase'' with exponentially many optimal solutions with discrepancy 0 or 1; a ``hard phase'' with minimal discrepancy of order $Me^{-Θ(n)}$; and a ``sorted phase'' with an unique optimal partition of order $Mn$, obtained by putting the $(s+n)/2$ smallest integers in one subset. Our phase diagram covers all but a relatively small region in the $κb$-plane. We also show that the three phases can be alternatively characterized by the number of basis solutions of the associated linear programming problem, and by the fraction of these basis solutions whose $\pm 1$-valued components form optimal integer partitions of the subproblem with the corresponding weights. We show in particular that this fraction is one in the sorted phase, and exponentially small in both the perfect and hard phases, and strictly exponentially smaller in the hard phase than in the perfect phase. Open problems are discussed, and numerical experiments are presented.
△ Less
Submitted 26 February, 2003;
originally announced February 2003.
-
General Theory of Lee-Yang Zeros in Models with First-Order Phase Transitions
Authors:
Marek Biskup,
Christian Borgs,
Jennifer T. Chayes,
Logan J. Kleinwaks,
Roman Kotecky
Abstract:
We present a general, rigorous theory of Lee-Yang zeros for models with first-order phase transitions that admit convergent contour expansions. We derive formulas for the positions and the density of the zeros. In particular, we show that for models without symmetry, the curves on which the zeros lie are generically not circles, and can have topologically nontrivial features, such as bifurcation…
▽ More
We present a general, rigorous theory of Lee-Yang zeros for models with first-order phase transitions that admit convergent contour expansions. We derive formulas for the positions and the density of the zeros. In particular, we show that for models without symmetry, the curves on which the zeros lie are generically not circles, and can have topologically nontrivial features, such as bifurcation. Our results are illustrated in three models in a complex field: the low-temperature Ising and Blume-Capel models, and the $q$-state Potts model for $q$ large enough.
△ Less
Submitted 30 September, 2003; v1 submitted 4 April, 2000;
originally announced April 2000.
-
The Scaling Window of the 2-SAT Transition
Authors:
Béla Bollobás,
Christian Borgs,
Jennifer T. Chayes,
Jeong Han Kim,
David B. Wilson
Abstract:
We consider the random 2-satisfiability problem, in which each instance is a formula that is the conjunction of m clauses of the form (x or y), chosen uniformly at random from among all 2-clauses on n Boolean variables and their negations. As m and n tend to infinity in the ratio m/n --> alpha, the problem is known to have a phase transition at alpha_c = 1, below which the probability that the f…
▽ More
We consider the random 2-satisfiability problem, in which each instance is a formula that is the conjunction of m clauses of the form (x or y), chosen uniformly at random from among all 2-clauses on n Boolean variables and their negations. As m and n tend to infinity in the ratio m/n --> alpha, the problem is known to have a phase transition at alpha_c = 1, below which the probability that the formula is satisfiable tends to one and above which it tends to zero. We determine the finite-size scaling about this transition, namely the scaling of the maximal window W(n,delta) = (alpha_-(n,delta),alpha_+(n,delta)) such that the probability of satisfiability is greater than 1-delta for alpha < alpha_- and is less than delta for alpha > alpha_+. We show that W(n,delta)=(1-Theta(n^{-1/3}),1+Theta(n^{-1/3})), where the constants implicit in Theta depend on delta. We also determine the rates at which the probability of satisfiability approaches one and zero at the boundaries of the window. Namely, for m=(1+epsilon)n, where epsilon may depend on n as long as |epsilon| is sufficiently small and |epsilon|*n^(1/3) is sufficiently large, we show that the probability of satisfiability decays like exp(-Theta(n*epsilon^3)) above the window, and goes to one like 1-Theta(1/(n*|epsilon|^3)) below the window. We prove these results by defining an order parameter for the transition and establishing its scaling behavior in n both inside and outside the window. Using this order parameter, we prove that the 2-SAT phase transition is continuous with an order parameter critical exponent of 1. We also determine the values of two other critical exponents, showing that the exponents of 2-SAT are identical to those of the random graph.
△ Less
Submitted 26 February, 2001; v1 submitted 5 September, 1999;
originally announced September 1999.
-
The covariance matrix of the potts model: A random cluster analysis
Authors:
C. Borgs,
J. T. Chayes
Abstract:
We consider the covariance matrix $G^{mn}(x-y)$ of the d-dimensional q-states Potts model, rewriting it in terms of the connectivity, the finite-cluster connectivity and the infinite-cluster covariance in the random cluster repre- sentation of Fortuin and Kasteleyn. In any of the $q$ ordered phases, we show that the matrix $G^{mn}(x-y)$ has one tivial eigenvalue 0, one simple eigen- value…
▽ More
We consider the covariance matrix $G^{mn}(x-y)$ of the d-dimensional q-states Potts model, rewriting it in terms of the connectivity, the finite-cluster connectivity and the infinite-cluster covariance in the random cluster repre- sentation of Fortuin and Kasteleyn. In any of the $q$ ordered phases, we show that the matrix $G^{mn}(x-y)$ has one tivial eigenvalue 0, one simple eigen- value $G_{\wir}^{(1)}(x-y)$ and one ($q-2$)-fold degenerate eigenvalue $G_{\wir}^{(2)}(x-y)$. Furthermore, we identify the eigenvalues both in terms of representations of the unbroken symmetry group of the model, and in terms of connectivities and cluster covariances, thereby attributing algebraic signifi- cance to these stochastic geometric quantities. In addition to establishing the existence of the correlation lengths $ξ_{\wir}^{(1)}$ and $ξ_{\wir}^{(2)}$ corresponding to $G_{\wir}^{(1)}(x-y)$ and $G_{\wir}^{(2)}(x-y)$, we show that $ξ_{\wir}^{(1)}(β)\geq ξ_{\wir}^{(2)}(β)$ for all inverse tempera- tures $β$. For dimension $d=2$ and $q \geq 1$, we establish a duality relation between $ξ_{\wir}^{(2)}$ and $ξ_{\free}$, the correlation length of the two-point function with free boundary conditions: We show $ξ_{\wir}^{(2)}(β) = \frac{1}{2} ξ_{\free}(β^\ast)$ for all $β\geq β_o$, where $β^\ast$ is the dual inverse temperature and $β_o$ is the self-dual point. In order to prove the above results, we introduce two new inequalities. The first is similar to the FKG inequality, but holds for events which are neither increasing nor decreasing, and replaces independence in the standard percolation model; the second replaces the van den Berg - Kesten inequality.
△ Less
Submitted 14 November, 1994;
originally announced November 1994.
