-
Hydra: Computer Vision for Data Quality Monitoring
Authors:
Thomas Britton,
Torri Jeske,
David Lawrence,
Kishansingh Rajput
Abstract:
Hydra is a system which utilizes computer vision to perform near real time data quality management, initially developed for Hall-D in 2019. Since then, it has been deployed across all experimental halls at Jefferson Lab, with the CLAS12 collaboration in Hall-B being the first outside of GlueX to fully utilize Hydra. The system comprises back end processes that manage the models, their inferences,…
▽ More
Hydra is a system which utilizes computer vision to perform near real time data quality management, initially developed for Hall-D in 2019. Since then, it has been deployed across all experimental halls at Jefferson Lab, with the CLAS12 collaboration in Hall-B being the first outside of GlueX to fully utilize Hydra. The system comprises back end processes that manage the models, their inferences, and the data flow. The front-end components, accessible via web pages, allow detector experts and shift crews to view and interact with the system. This talk will give an overview of the Hydra system as well as highlight significant developments in Hydra's feature set, acute challenges with operating Hydra in all halls, and lessons learned along the way.
△ Less
Submitted 1 March, 2024;
originally announced March 2024.
-
Artificial Intelligence for the Electron Ion Collider (AI4EIC)
Authors:
C. Allaire,
R. Ammendola,
E. -C. Aschenauer,
M. Balandat,
M. Battaglieri,
J. Bernauer,
M. Bondì,
N. Branson,
T. Britton,
A. Butter,
I. Chahrour,
P. Chatagnon,
E. Cisbani,
E. W. Cline,
S. Dash,
C. Dean,
W. Deconinck,
A. Deshpande,
M. Diefenthaler,
R. Ent,
C. Fanelli,
M. Finger,
M. Finger, Jr.,
E. Fol,
S. Furletov
, et al. (70 additional authors not shown)
Abstract:
The Electron-Ion Collider (EIC), a state-of-the-art facility for studying the strong force, is expected to begin commissioning its first experiments in 2028. This is an opportune time for artificial intelligence (AI) to be included from the start at this facility and in all phases that lead up to the experiments. The second annual workshop organized by the AI4EIC working group, which recently took…
▽ More
The Electron-Ion Collider (EIC), a state-of-the-art facility for studying the strong force, is expected to begin commissioning its first experiments in 2028. This is an opportune time for artificial intelligence (AI) to be included from the start at this facility and in all phases that lead up to the experiments. The second annual workshop organized by the AI4EIC working group, which recently took place, centered on exploring all current and prospective application areas of AI for the EIC. This workshop is not only beneficial for the EIC, but also provides valuable insights for the newly established ePIC collaboration at EIC. This paper summarizes the different activities and R&D projects covered across the sessions of the workshop and provides an overview of the goals, approaches and strategies regarding AI/ML in the EIC community, as well as cutting-edge techniques currently studied in other experiments.
△ Less
Submitted 17 July, 2023;
originally announced July 2023.
-
Distance Preserving Machine Learning for Uncertainty Aware Accelerator Capacitance Predictions
Authors:
Steven Goldenberg,
Malachi Schram,
Kishansingh Rajput,
Thomas Britton,
Chris Pappas,
Dan Lu,
Jared Walden,
Majdi I. Radaideh,
Sarah Cousineau,
Sudarshan Harave
Abstract:
Providing accurate uncertainty estimations is essential for producing reliable machine learning models, especially in safety-critical applications such as accelerator systems. Gaussian process models are generally regarded as the gold standard method for this task, but they can struggle with large, high-dimensional datasets. Combining deep neural networks with Gaussian process approximation techni…
▽ More
Providing accurate uncertainty estimations is essential for producing reliable machine learning models, especially in safety-critical applications such as accelerator systems. Gaussian process models are generally regarded as the gold standard method for this task, but they can struggle with large, high-dimensional datasets. Combining deep neural networks with Gaussian process approximation techniques have shown promising results, but dimensionality reduction through standard deep neural network layers is not guaranteed to maintain the distance information necessary for Gaussian process models. We build on previous work by comparing the use of the singular value decomposition against a spectral-normalized dense layer as a feature extractor for a deep neural Gaussian process approximation model and apply it to a capacitance prediction problem for the High Voltage Converter Modulators in the Oak Ridge Spallation Neutron Source. Our model shows improved distance preservation and predicts in-distribution capacitance values with less than 1% error.
△ Less
Submitted 5 July, 2023;
originally announced July 2023.
-
Difficult Lessons on Social Prediction from Wisconsin Public Schools
Authors:
Juan C. Perdomo,
Tolani Britton,
Moritz Hardt,
Rediet Abebe
Abstract:
Early warning systems (EWS) are predictive tools at the center of recent efforts to improve graduation rates in public schools across the United States. These systems assist in targeting interventions to individual students by predicting which students are at risk of drop** out. Despite significant investments in their widespread adoption, there remain large gaps in our understanding of the effi…
▽ More
Early warning systems (EWS) are predictive tools at the center of recent efforts to improve graduation rates in public schools across the United States. These systems assist in targeting interventions to individual students by predicting which students are at risk of drop** out. Despite significant investments in their widespread adoption, there remain large gaps in our understanding of the efficacy of EWS, and the role of statistical risk scores in education.
In this work, we draw on nearly a decade's worth of data from a system used throughout Wisconsin to provide the first large-scale evaluation of the long-term impact of EWS on graduation outcomes. We present empirical evidence that the prediction system accurately sorts students by their dropout risk. We also find that it may have caused a single-digit percentage increase in graduation rates, though our empirical analyses cannot reliably rule out that there has been no positive treatment effect.
Going beyond a retrospective evaluation of DEWS, we draw attention to a central question at the heart of the use of EWS: Are individual risk scores necessary for effectively targeting interventions? We propose a simple mechanism that only uses information about students' environments -- such as their schools, and districts -- and argue that this mechanism can target interventions just as efficiently as the individual risk score-based mechanism. Our argument holds even if individual predictions are highly accurate and effective interventions exist. In addition to motivating this simple targeting mechanism, our work provides a novel empirical backbone for the robust qualitative understanding among education researchers that dropout is structurally determined. Combined, our insights call into question the marginal value of individual predictions in settings where outcomes are driven by high levels of inequality.
△ Less
Submitted 18 September, 2023; v1 submitted 12 April, 2023;
originally announced April 2023.
-
Lost in Translation: Reimagining the Machine Learning Life Cycle in Education
Authors:
Lydia T. Liu,
Serena Wang,
Tolani Britton,
Rediet Abebe
Abstract:
Machine learning (ML) techniques are increasingly prevalent in education, from their use in predicting student dropout, to assisting in university admissions, and facilitating the rise of MOOCs. Given the rapid growth of these novel uses, there is a pressing need to investigate how ML techniques support long-standing education principles and goals. In this work, we shed light on this complex lands…
▽ More
Machine learning (ML) techniques are increasingly prevalent in education, from their use in predicting student dropout, to assisting in university admissions, and facilitating the rise of MOOCs. Given the rapid growth of these novel uses, there is a pressing need to investigate how ML techniques support long-standing education principles and goals. In this work, we shed light on this complex landscape drawing on qualitative insights from interviews with education experts. These interviews comprise in-depth evaluations of ML for education (ML4Ed) papers published in preeminent applied ML conferences over the past decade. Our central research goal is to critically examine how the stated or implied education and societal objectives of these papers are aligned with the ML problems they tackle. That is, to what extent does the technical problem formulation, objectives, approach, and interpretation of results align with the education problem at hand. We find that a cross-disciplinary gap exists and is particularly salient in two parts of the ML life cycle: the formulation of an ML problem from education goals and the translation of predictions to interventions. We use these insights to propose an extended ML life cycle, which may also apply to the use of ML in other domains. Our work joins a growing number of meta-analytical studies across education and ML research, as well as critical analyses of the societal impact of ML. Specifically, it fills a gap between the prevailing technical understanding of machine learning and the perspective of education researchers working with students and in policy.
△ Less
Submitted 8 September, 2022;
originally announced September 2022.
-
Gender issues in fundamental physics: Strumia's bibliometric analysis fails to account for key confounders and confuses correlation with causation
Authors:
Philip Ball,
T. Benjamin Britton,
Erin Hengel,
Philip Moriarty,
Rachel A. Oliver,
Gina Rippon,
Angela Saini,
Jessica Wade
Abstract:
Alessandro Strumia recently published a survey of gender differences in publications and citations in high-energy physics (HEP). In addition to providing full access to the data, code, and methodology, Strumia (2020) systematically describes and accounts for gender differences in HEP citation networks. His analysis points both to ongoing difficulties in attracting women to high-energy physics and…
▽ More
Alessandro Strumia recently published a survey of gender differences in publications and citations in high-energy physics (HEP). In addition to providing full access to the data, code, and methodology, Strumia (2020) systematically describes and accounts for gender differences in HEP citation networks. His analysis points both to ongoing difficulties in attracting women to high-energy physics and an encouraging-though slow-trend in improvement. Unfortunately, however, the time and effort Strumia (2020) devoted to collating and quantifying the data are not matched by a similar rigour in interpreting the results. To support his conclusions, he selectively cites available literature and fails to adequately adjust for a range of confounding factors. For example, his analyses do not consider how unobserved factors -- e.g., a tendency to overcite well-known authors -- drive a wedge between quality and citations and correlate with author gender. He also fails to take into account many structural and non-structural factors -- including, but not limited to, direct discrimination and the expectations women form (and actions they take) in response to it -- that undoubtedly lead to gender differences in productivity. We therefore believe that a number of Strumia's conclusions are not supported by his analysis. Indeed, we re-analyse a subsample of solo-authored papers from his data, adjusting for year and journal of publication, authors' research age and their lifetime "fame". Our re-analysis suggests that female-authored papers are actually cited more than male-authored papers. This finding is inconsistent with the "greater male variability" hypothesis Strumia (2020) proposes to explain many of his results.
△ Less
Submitted 3 December, 2020;
originally announced June 2021.
-
AI Enabled Data Quality Monitoring with Hydra
Authors:
Thomas Britton,
David Lawrence,
Kishansingh Rajput
Abstract:
Data quality monitoring is critical to all experiments impacting the quality of any physics results. Traditionally, this is done through an alarm system, which detects low level faults, leaving higher level monitoring to human crews. Artificial Intelligence is beginning to find its way into scientific applications, but comes with difficulties, relying on the acquisition of new skill sets, either t…
▽ More
Data quality monitoring is critical to all experiments impacting the quality of any physics results. Traditionally, this is done through an alarm system, which detects low level faults, leaving higher level monitoring to human crews. Artificial Intelligence is beginning to find its way into scientific applications, but comes with difficulties, relying on the acquisition of new skill sets, either through education or acquisition, in data science. This paper will discuss the development and deployment of the Hydra monitoring system in production at Gluex. It will show how "off-the-shelf" technologies can be rapidly developed, as well as discuss what sociological hurdles must be overcome to successfully deploy such a system. Early results from production running of Hydra will also be shared as well as a future outlook for development of Hydra.
△ Less
Submitted 28 April, 2021;
originally announced May 2021.
-
A network epidemic model with preventive rewiring: comparative analysis of the initial phase
Authors:
Tom Britton,
David Juher,
Joan Saldana
Abstract:
This paper is concerned with stochastic SIR and SEIR epidemic models on random networks in which individuals may rewire away from infected neighbors at some rate $ω$ (and reconnect to non-infectious individuals with probability $α$ or else simply drop the edge if $α=0$), so-called preventive rewiring. The models are denoted SIR-$ω$ and SEIR-$ω$, and we focus attention on the early stages of an out…
▽ More
This paper is concerned with stochastic SIR and SEIR epidemic models on random networks in which individuals may rewire away from infected neighbors at some rate $ω$ (and reconnect to non-infectious individuals with probability $α$ or else simply drop the edge if $α=0$), so-called preventive rewiring. The models are denoted SIR-$ω$ and SEIR-$ω$, and we focus attention on the early stages of an outbreak, where we derive expression for the basic reproduction number $R_0$ and the expected degree of the infectious nodes $E(D_I)$ using two different approximation approaches. The first approach approximates the early spread of an epidemic by a branching process, whereas the second one uses pair approximation. The expressions are compared with the corresponding empirical means obtained from stochastic simulations of SIR-$ω$ and SEIR-$ω$ epidemics on Poisson and scale-free networks. Without rewiring of exposed nodes, the two approaches predict the same epidemic threshold and the same $E(D_I)$ for both types of epidemics, the latter being very close to the mean degree obtained from simulated epidemics over Poisson networks. Above the epidemic threshold, pairwise models overestimate the value of $R_0$ computed from simulations, which turns out to be very close to the one predicted by the branching process approximation. When exposed individuals also rewire with $α> 0$ (perhaps unaware of being infected), the two approaches give different epidemic thresholds, with the branching process approximation being more in agreement with simulations.
△ Less
Submitted 18 October, 2016; v1 submitted 1 December, 2015;
originally announced December 2015.
-
The Configuration Model for Partially Directed Graphs
Authors:
Kristoffer Spricer,
Tom Britton
Abstract:
The configuration model was originally defined for undirected networks and has recently been extended to directed networks. Many empirical networks are however neither undirected nor completely directed, but instead usually partially directed meaning that certain edges are directed and others are undirected. In the paper we define a configuration model for such networks where nodes have in-, out-,…
▽ More
The configuration model was originally defined for undirected networks and has recently been extended to directed networks. Many empirical networks are however neither undirected nor completely directed, but instead usually partially directed meaning that certain edges are directed and others are undirected. In the paper we define a configuration model for such networks where nodes have in-, out-, and undirected degrees that may be dependent. We prove conditions under which the resulting degree distributions converge to the intended degree distributions. The new model is shown to better approximate several empirical networks compared to undirected and completely directed networks.
△ Less
Submitted 17 March, 2015;
originally announced March 2015.
-
Respondent-driven sampling and an unusual epidemic
Authors:
Jens Malmros,
Fredrik Liljeros,
Tom Britton
Abstract:
Respondent-driven sampling (RDS) is frequently used when sampling hard-to-reach and/or stigmatized communities. RDS utilizes a peer-driven recruitment mechanism where sampled individuals pass on participation coupons to at most $c$ of their acquaintances in the community ($c=3$ being a common choice), who then in turn pass on to their acquaintances if they choose to participate, and so on. This pr…
▽ More
Respondent-driven sampling (RDS) is frequently used when sampling hard-to-reach and/or stigmatized communities. RDS utilizes a peer-driven recruitment mechanism where sampled individuals pass on participation coupons to at most $c$ of their acquaintances in the community ($c=3$ being a common choice), who then in turn pass on to their acquaintances if they choose to participate, and so on. This process of distributing coupons is shown to behave like a new Reed-Frost type network epidemic model, in which becoming infected corresponds to receiving a coupon. The difference from existing network epidemic models is that an infected individual can not infect (i.e.\ sample) all of its contacts, but only at most $c$ of them. We calculate $R_0$, the probability of a major "outbreak", and the relative size of a major outbreak in the limit of infinite population size and evaluate their adequacy in finite populations. We study the effect of varying $c$ and compare RDS to the corresponding usual epidemic models, i.e.\ the case of $c=\infty$. Our results suggest that the number of coupons has a large effect on RDS recruitment. Additionally, we use our findings to explain previous empirical observations.
△ Less
Submitted 11 November, 2014;
originally announced November 2014.
-
Random Walks on Directed Networks: Inference and Respondent-driven Sampling
Authors:
Jens Malmros,
Naoki Masuda,
Tom Britton
Abstract:
Respondent driven sampling (RDS) is a method often used to estimate population properties (e.g. sexual risk behavior) in hard-to-reach populations. It combines an effective modified snowball sampling methodology with an estimation procedure that yields unbiased population estimates under the assumption that the sampling process behaves like a random walk on the social network of the population. Cu…
▽ More
Respondent driven sampling (RDS) is a method often used to estimate population properties (e.g. sexual risk behavior) in hard-to-reach populations. It combines an effective modified snowball sampling methodology with an estimation procedure that yields unbiased population estimates under the assumption that the sampling process behaves like a random walk on the social network of the population. Current RDS estimation methodology assumes that the social network is undirected, i.e. that all edges are reciprocal. However, empirical social networks in general also have non-reciprocated edges. To account for this fact, we develop a new estimation method for RDS in the presence of directed edges on the basis of random walks on directed networks. We distinguish directed and undirected edges and consider the possibility that the random walk returns to its current position in two steps through an undirected edge. We derive estimators of the selection probabilities of individuals as a function of the number of outgoing edges of sampled individuals. We evaluate the performance of the proposed estimators on artificial and empirical networks to show that they generally perform better than existing methods. This is in particular the case when the fraction of directed edges in the network is large.
△ Less
Submitted 16 August, 2013;
originally announced August 2013.
-
A network with tunable clustering, degree correlation and degree distribution, and an epidemic thereon
Authors:
Frank Ball,
Tom Britton,
David Sirl
Abstract:
A random network model which allows for tunable, quite general forms of clustering, degree correlation and degree distribution is defined. The model is an extension of the configuration model, in which stubs (half-edges) are paired to form a network. Clustering is obtained by forming small completely connected subgroups, and positive (negative) degree correlation is obtained by connecting a fracti…
▽ More
A random network model which allows for tunable, quite general forms of clustering, degree correlation and degree distribution is defined. The model is an extension of the configuration model, in which stubs (half-edges) are paired to form a network. Clustering is obtained by forming small completely connected subgroups, and positive (negative) degree correlation is obtained by connecting a fraction of the stubs with stubs of similar (dissimilar) degree. An SIR (Susceptible -> Infective -> Recovered) epidemic model is defined on this network. Asymptotic properties of both the network and the epidemic, as the population size tends to infinity, are derived: the degree distribution, degree correlation and clustering coefficient, as well as a reproduction number $R_*$, the probability of a major outbreak and the relative size of such an outbreak. The theory is illustrated by Monte Carlo simulations and numerical examples. The main findings are that clustering tends to decrease the spread of disease, the effect of degree correlation is appreciably greater when the disease is close to threshold than when it is well above threshold and disease spread broadly increases with degree correlation $ρ$ when $R_*$ is just above its threshold value of one and decreases with $ρ$ when $R_*$ is well above one.
△ Less
Submitted 30 July, 2012; v1 submitted 13 July, 2012;
originally announced July 2012.
-
Inferring global network properties from egocentric data with applications to epidemics
Authors:
Tom Britton,
Pieter Trapman
Abstract:
Social networks are rarely observed in full detail. In many situations properties are known for only a sample of the individuals in the network and it is desirable to induce global properties of the full social network from this "egocentric" network data. In the current paper we study a few different types of egocentric data, and show what global network properties are consistent with those egocen…
▽ More
Social networks are rarely observed in full detail. In many situations properties are known for only a sample of the individuals in the network and it is desirable to induce global properties of the full social network from this "egocentric" network data. In the current paper we study a few different types of egocentric data, and show what global network properties are consistent with those egocentric data. Two global network properties are considered: the size of the largest connected component in the network (the giant), and secondly, the possible size of an epidemic outbreak taking place on the network, in which transmission occurs only between network neighbours, and with probability $p$. The main conclusion is that in most cases, egocentric data allow for a large range of possible sizes of the giant and the outbreak. However, there is an upper bound for the latter. For the case that the network is selected uniformly among networks with prescribed egocentric data (satisfying some conditions), the asymptotic size of the giant and the outbreak is characterised.
△ Less
Submitted 13 January, 2012;
originally announced January 2012.
-
Inhomogeneous epidemics on weighted networks
Authors:
Tom Britton,
David Lindenstrand
Abstract:
A social (sexual) network is modeled by an extension of the configuration model to the situation where edges have weights, e.g. reflecting the number of sex-contacts between the individuals. An epidemic model is defined on the network such that individuals are heterogeneous in terms of how susceptible and infectious they are. The basic reproduction number R_0 is derived and studied for various exa…
▽ More
A social (sexual) network is modeled by an extension of the configuration model to the situation where edges have weights, e.g. reflecting the number of sex-contacts between the individuals. An epidemic model is defined on the network such that individuals are heterogeneous in terms of how susceptible and infectious they are. The basic reproduction number R_0 is derived and studied for various examples, but also the size and probability of a major outbreak. The qualitative conclusion is that R_0 gets larger as the community becomes more heterogeneous but that different heterogeneities (degree distribution, weight, susceptibility and infectivity) can sometimes have the cumulative effect of homogenizing the community, thus making $R_0$ smaller. The effect on the probability and final size of an outbreak is more complicated.
△ Less
Submitted 20 December, 2011;
originally announced December 2011.
-
A dynamic network in a dynamic population: asymptotic properties
Authors:
Tom Britton,
Mathias Lindholm,
Tatyana Turova
Abstract:
We derive asymptotic properties for a stochastic dynamic network model in a stochastic dynamic population. In the model, nodes give birth to new nodes until they die, each node being equipped with a social index given at birth. During the life of a node it creates edges to other nodes, nodes with high social index at higher rate, and edges disappear randomly in time. For this model we derive crite…
▽ More
We derive asymptotic properties for a stochastic dynamic network model in a stochastic dynamic population. In the model, nodes give birth to new nodes until they die, each node being equipped with a social index given at birth. During the life of a node it creates edges to other nodes, nodes with high social index at higher rate, and edges disappear randomly in time. For this model we derive criterion for when a giant connected component exists after the process has evolved for a long period of time, assuming the node population grows to infinity. We also obtain an explicit expression for the degree correlation $ρ$ (of neighbouring nodes) which shows that $ρ$ is always positive irrespective of parameter values in one of the two treated submodels, and may be either positive or negative in the other model, depending on the parameters.
△ Less
Submitted 1 April, 2011;
originally announced April 2011.
