Showing 1–17 of 17 results for author: Foley, J

Predicting the Age of Astronomical Transients from Real-Time Multivariate Time Series

Authors: Hali Huang, Daniel Muthukrishna, Prajna Nair, Zimi Zhang, Michael Fausnaugh, Torsha Majumder, Ryan J. Foley, George R. Ricker

Abstract: Astronomical transients, such as supernovae and other rare stellar explosions, have been instrumental in some of the most significant discoveries in astronomy. New astronomical sky surveys will soon record unprecedented numbers of transients as sparsely and irregularly sampled multivariate time series. To improve our understanding of the physical mechanisms of transients and their progenitor syste… ▽ More Astronomical transients, such as supernovae and other rare stellar explosions, have been instrumental in some of the most significant discoveries in astronomy. New astronomical sky surveys will soon record unprecedented numbers of transients as sparsely and irregularly sampled multivariate time series. To improve our understanding of the physical mechanisms of transients and their progenitor systems, early-time measurements are necessary. Prioritizing the follow-up of transients based on their age along with their class is crucial for new surveys. To meet this demand, we present the first method of predicting the age of transients in real-time from multi-wavelength time-series observations. We build a Bayesian probabilistic recurrent neural network. Our method can accurately predict the age of a transient with robust uncertainties as soon as it is initially triggered by a survey telescope. This work will be essential for the advancement of our understanding of the numerous young transients being detected by ongoing and upcoming astronomical surveys. △ Less

Submitted 28 November, 2023; originally announced November 2023.

Comments: 6 pages, 4 figures. Accepted at the NeurIPS 2023 Machine Learning and the Physical Sciences workshop

Generalized partitioned local depth

Authors: Kenneth S. Berenhaut, John D. Foley, Liangdongsheng Lyu

Abstract: In this paper we provide a generalization of the concept of cohesion as introduced recently by Berenhaut, Moore and Melvin [Proceedings of the National Academy of Sciences, 119 (4) (2022)]. The formulation presented builds on the technique of partitioned local depth by distilling two key probabilistic concepts: local relevance and support division. Earlier results are extended within the new conte… ▽ More In this paper we provide a generalization of the concept of cohesion as introduced recently by Berenhaut, Moore and Melvin [Proceedings of the National Academy of Sciences, 119 (4) (2022)]. The formulation presented builds on the technique of partitioned local depth by distilling two key probabilistic concepts: local relevance and support division. Earlier results are extended within the new context, and examples of applications to revealing communities in data with uncertainty are included. The work sheds light on the foundations of partitioned local depth, and extends the original ideas to enable probabilistic consideration of uncertain, variable and potentially conflicting information. △ Less

Submitted 14 November, 2023; v1 submitted 17 March, 2023; originally announced March 2023.

Comments: Typos correct & clarifying comments, 19 pages, 6 figures

Journal ref: Journal of Statistical Theory and Practice, Vol. 18, No. 10, 2024

Operads for complex system design specification, analysis and synthesis

Authors: John D. Foley, Spencer Breiner, Eswaran Subrahmanian, John M. Dusel

Abstract: As the complexity and heterogeneity of a system grows, the challenge of specifying, documenting and synthesizing correct, machine-readable designs increases dramatically. Separation of the system into manageable parts is needed to support analysis at various levels of granularity so that the system is maintainable and adaptable over its life cycle. In this paper, we argue that operads provide an e… ▽ More As the complexity and heterogeneity of a system grows, the challenge of specifying, documenting and synthesizing correct, machine-readable designs increases dramatically. Separation of the system into manageable parts is needed to support analysis at various levels of granularity so that the system is maintainable and adaptable over its life cycle. In this paper, we argue that operads provide an effective knowledge representation to address these challenges. Formal documentation of a syntactically correct design is built up during design synthesis, guided by semantic reasoning about design effectiveness. Throughout, the ability to decompose the system into parts and reconstitute the whole is maintained. We describe recent progress in effective modeling under this paradigm and directions for future work to systematically address scalability challenges for complex system design. △ Less

Submitted 25 May, 2021; v1 submitted 26 January, 2021; originally announced January 2021.

Comments: 33 pages; 14 figures; revised presentation, esp. first three sections and removal of appendix; new style file

Report number: G2021-0140

Journal ref: Proc. Royal Soc. A , Vol. 477, No. 2250, 2021, 20210099

Operads for Designing Systems of Systems

Authors: John C. Baez, John Foley

Abstract: System of systems engineering seeks to analyze, design and deploy collections of systems that together can flexibly address an array of complex tasks. In the Complex Adaptive System Composition and Design Environment program, we developed "network operads" as a tool for designing and tasking systems of systems, and applied them to domains including maritime search and rescue. The network operad fo… ▽ More System of systems engineering seeks to analyze, design and deploy collections of systems that together can flexibly address an array of complex tasks. In the Complex Adaptive System Composition and Design Environment program, we developed "network operads" as a tool for designing and tasking systems of systems, and applied them to domains including maritime search and rescue. The network operad formalism offers new ways to handle changing levels of abstraction in system-of-system design and tasking. △ Less

Submitted 26 September, 2020; originally announced September 2020.

Comments: 2 pages, TikZ figure

Journal ref: Notices of the American Mathematical Society, Vol. 68, No. 6, 2021, pp 1005-1007

Optimal Use of Multi-spectral Satellite Data with Convolutional Neural Networks

Authors: Sagar Vaze, James Foley, Mohamed Seddiq, Alexey Unagaev, Natalia Efremova

Abstract: The analysis of satellite imagery will prove a crucial tool in the pursuit of sustainable development. While Convolutional Neural Networks (CNNs) have made large gains in natural image analysis, their application to multi-spectral satellite images (wherein input images have a large number of channels) remains relatively unexplored. In this paper, we compare different methods of leveraging multi-ba… ▽ More The analysis of satellite imagery will prove a crucial tool in the pursuit of sustainable development. While Convolutional Neural Networks (CNNs) have made large gains in natural image analysis, their application to multi-spectral satellite images (wherein input images have a large number of channels) remains relatively unexplored. In this paper, we compare different methods of leveraging multi-band information with CNNs, demonstrating the performance of all compared methods on the task of semantic segmentation of agricultural vegetation (vineyards). We show that standard industry practice of using bands selected by a domain expert leads to a significantly worse test accuracy than the other methods compared. Specifically, we compare: using bands specified by an expert; using all available bands; learning attention maps over the input bands; and leveraging Bayesian optimisation to dictate band choice. We show that simply using all available band information already increases test time performance, and show that the Bayesian optimisation, first applied to band selection in this work, can be used to further boost accuracy. △ Less

Submitted 15 September, 2020; originally announced September 2020.

Comments: AI for Social Good workshop - Harvard CRCS

A summary of the prevalence of Genetic Algorithms in Bioinformatics from 2015 onwards

Authors: Mekaal Swerhun, Jasmine Foley, Brandon Massop, Vijay Mago

Abstract: In recent years, machine learning has seen an increasing presencein a large variety of fields, especially in health care and bioinformatics.More specifically, the field where machine learning algorithms have found most applications is Genetic Algorithms.The objective of this paper is to conduct a survey of articles published from 2015 onwards that deal with Genetic Algorithms(GA) and how they are… ▽ More In recent years, machine learning has seen an increasing presencein a large variety of fields, especially in health care and bioinformatics.More specifically, the field where machine learning algorithms have found most applications is Genetic Algorithms.The objective of this paper is to conduct a survey of articles published from 2015 onwards that deal with Genetic Algorithms(GA) and how they are used in bioinformatics.To achieve the objective, a sco** review was conducted that utilized Google Scholar alongside Publish or Perish and the Scimago Journal & CountryRank to search for respectable sources. Upon analyzing 31 articles from the field of bioinformatics, it became apparent that genetic algorithms rarely form a full application, instead they rely on other vital algorithms such as support vector machines.Indeed, support vector machines were the most prevalent algorithms used alongside genetic algorithms; however, while the usage of such algorithms contributes to the heavy focus on accuracy by GA programs, it often sidelines computation times in the process. In fact, most applications employing GAs for classification and feature selectionare nearing or at 100% success rate, and the focus of future GA development should be directed elsewhere. Population-based searches, like GA, are often combined with other machine learning algorithms. In this sco** review, genetic algorithms combined with Support Vector Machines were found to perform best. The performance metric that was evaluated most often was accuracy. Measuring the accuracy avoids measuring the main weakness of GAs, which is computational time. The future of genetic algorithms could be open-ended evolutionary algorithms, which attempt to increase complexity and find diverse solutions, rather than optimize a fitness function and converge to a single best solution from the initial population of solutions. △ Less

Submitted 20 August, 2020; originally announced August 2020.

Comments: 20 pages and 2 figures

SMArtCast: Predicting soil moisture interpolations into the future using Earth observation data in a deep learning framework

Authors: Conrad James Foley, Sagar Vaze, Mohamed El Amine Seddiq, Alexey Unagaev, Natalia Efremova

Abstract: Soil moisture is critical component of crop health and monitoring it can enable further actions for increasing yield or preventing catastrophic die off. As climate change increases the likelihood of extreme weather events and reduces the predictability of weather, and non-optimal soil moistures for crops may become more likely. In this work, we a series of LSTM architectures to analyze measurement… ▽ More Soil moisture is critical component of crop health and monitoring it can enable further actions for increasing yield or preventing catastrophic die off. As climate change increases the likelihood of extreme weather events and reduces the predictability of weather, and non-optimal soil moistures for crops may become more likely. In this work, we a series of LSTM architectures to analyze measurements of soil moisture and vegetation indiced derived from satellite imagery. The system learns to predict the future values of these measurements. These spatially sparse values and indices are used as input features to an interpolation method that infer spatially dense moisture map for a future time point. This has the potential to provide advance warning for soil moistures that may be inhospitable to crops across an area with limited monitoring capacity. △ Less

Submitted 24 April, 2020; v1 submitted 16 March, 2020; originally announced March 2020.

Comments: Climate change AI workshop

Journal ref: ICLR 2020

Report on the First HIPstIR Workshop on the Future of Information Retrieval

Authors: Laura Dietz, Bhaskar Mitra, Jeremy Pickens, Hana Anber, Sandeep Avula, Asia Biega, Adrian Boteanu, Shubham Chatterjee, Jeff Dalton, Shiri Dori-Hacohen, John Foley, Henry Feild, Ben Gamari, Rosie Jones, Pallika Kanani, Sumanta Kashyapi, Widad Machmouchi, Matthew Mitsui, Steve Nole, Alexandre Tachard Passos, Jordan Ramsdell, Adam Roegiest, David Smith, Alessandro Sordoni

Abstract: The vision of HIPstIR is that early stage information retrieval (IR) researchers get together to develop a future for non-mainstream ideas and research agendas in IR. The first iteration of this vision materialized in the form of a three day workshop in Portsmouth, New Hampshire attended by 24 researchers across academia and industry. Attendees pre-submitted one or more topics that they want to pi… ▽ More The vision of HIPstIR is that early stage information retrieval (IR) researchers get together to develop a future for non-mainstream ideas and research agendas in IR. The first iteration of this vision materialized in the form of a three day workshop in Portsmouth, New Hampshire attended by 24 researchers across academia and industry. Attendees pre-submitted one or more topics that they want to pitch at the meeting. Then over the three days during the workshop, we self-organized into groups and worked on six specific proposals of common interest. In this report, we present an overview of the workshop and brief summaries of the six proposals that resulted from the workshop. △ Less

Submitted 20 December, 2019; originally announced December 2019.

Toybox: A Suite of Environments for Experimental Evaluation of Deep Reinforcement Learning

Authors: Emma Tosch, Kaleigh Clary, John Foley, David Jensen

Abstract: Evaluation of deep reinforcement learning (RL) is inherently challenging. In particular, learned policies are largely opaque, and hypotheses about the behavior of deep RL agents are difficult to test in black-box environments. Considerable effort has gone into addressing opacity, but almost no effort has been devoted to producing high quality environments for experimental evaluation of agent behav… ▽ More Evaluation of deep reinforcement learning (RL) is inherently challenging. In particular, learned policies are largely opaque, and hypotheses about the behavior of deep RL agents are difficult to test in black-box environments. Considerable effort has gone into addressing opacity, but almost no effort has been devoted to producing high quality environments for experimental evaluation of agent behavior. We present TOYBOX, a new high-performance, open-source* subset of Atari environments re-designed for the experimental evaluation of deep RL. We show that TOYBOX enables a wide range of experiments and analyses that are impossible in other environments. *https://kdl-umass.github.io/Toybox/ △ Less

Submitted 7 May, 2019; originally announced May 2019.

Let's Play Again: Variability of Deep Reinforcement Learning Agents in Atari Environments

Authors: Kaleigh Clary, Emma Tosch, John Foley, David Jensen

Abstract: Reproducibility in reinforcement learning is challenging: uncontrolled stochasticity from many sources, such as the learning algorithm, the learned policy, and the environment itself have led researchers to report the performance of learned agents using aggregate metrics of performance over multiple random seeds for a single environment. Unfortunately, there are still pernicious sources of variabi… ▽ More Reproducibility in reinforcement learning is challenging: uncontrolled stochasticity from many sources, such as the learning algorithm, the learned policy, and the environment itself have led researchers to report the performance of learned agents using aggregate metrics of performance over multiple random seeds for a single environment. Unfortunately, there are still pernicious sources of variability in reinforcement learning agents that make reporting common summary statistics an unsound metric for performance. Our experiments demonstrate the variability of common agents used in the popular OpenAI Baselines repository. We make the case for reporting post-training agent performance as a distribution, rather than a point estimate. △ Less

Submitted 12 April, 2019; originally announced April 2019.

Comments: NeurIPS 2018 Critiquing and Correcting Trends Workshop

Network Models from Petri Nets with Catalysts

Authors: John C. Baez, John Foley, Joe Moeller

Abstract: Petri networks and network models are two frameworks for the compositional design of systems of interacting entities. Here we show how to combine them using the concept of a "catalyst": an entity that is neither destroyed nor created by any process it engages in. In a Petri net, a place is a catalyst if its in-degree equals its out-degree for every transition. We show how a Petri net with a chosen… ▽ More Petri networks and network models are two frameworks for the compositional design of systems of interacting entities. Here we show how to combine them using the concept of a "catalyst": an entity that is neither destroyed nor created by any process it engages in. In a Petri net, a place is a catalyst if its in-degree equals its out-degree for every transition. We show how a Petri net with a chosen set of catalysts gives a network model. This network model maps any list of catalysts from the chosen set to the category whose morphisms are all the processes enabled by this list of catalysts. Applying the Grothendieck construction, we obtain a category fibered over the category whose objects are lists of catalysts. This category has as morphisms all processes enabled by some list of catalysts. While this category has a symmetric monoidal structure that describes doing processes in parallel, its fibers also have premonoidal structures that describe doing one process and then another while reusing the catalysts. △ Less

Submitted 23 December, 2019; v1 submitted 6 April, 2019; originally announced April 2019.

Comments: 15 pages, TikZ figures

Journal ref: Compositionality 1, 4 (2019)

ToyBox: Better Atari Environments for Testing Reinforcement Learning Agents

Authors: John Foley, Emma Tosch, Kaleigh Clary, David Jensen

Abstract: It is a widely accepted principle that software without tests has bugs. Testing reinforcement learning agents is especially difficult because of the stochastic nature of both agents and environments, the complexity of state-of-the-art models, and the sequential nature of their predictions. Recently, the Arcade Learning Environment (ALE) has become one of the most widely used benchmark suites for d… ▽ More It is a widely accepted principle that software without tests has bugs. Testing reinforcement learning agents is especially difficult because of the stochastic nature of both agents and environments, the complexity of state-of-the-art models, and the sequential nature of their predictions. Recently, the Arcade Learning Environment (ALE) has become one of the most widely used benchmark suites for deep learning research, and state-of-the-art Reinforcement Learning (RL) agents have been shown to routinely equal or exceed human performance on many ALE tasks. Since ALE is based on emulation of original Atari games, the environment does not provide semantically meaningful representations of internal game state. This means that ALE has limited utility as an environment for supporting testing or model introspection. We propose ToyBox, a collection of reimplementations of these games that solves this critical problem and enables robust testing of RL agents. △ Less

Submitted 25 January, 2019; v1 submitted 6 December, 2018; originally announced December 2018.

Comments: NeurIPS Systems for ML Workshop

Explainable Agreement through Simulation for Tasks with Subjective Labels

Authors: John Foley

Abstract: The field of information retrieval often works with limited and noisy data in an attempt to classify documents into subjective categories, e.g., relevance, sentiment and controversy. We typically quantify a notion of agreement to understand the difficulty of the labeling task, but when we present final results, we do so using measures that are unaware of agreement or the inherent subjectivity of t… ▽ More The field of information retrieval often works with limited and noisy data in an attempt to classify documents into subjective categories, e.g., relevance, sentiment and controversy. We typically quantify a notion of agreement to understand the difficulty of the labeling task, but when we present final results, we do so using measures that are unaware of agreement or the inherent subjectivity of the task. We propose using user simulation to understand the effect size of this noisy agreement data. By simulating truth and predictions, we can understand the maximum scores a dataset can support: for if a classifier is doing better than a reasonable model of a human, we cannot conclude that it is actually better, but that it may be learning noise present in the dataset. We present a brief case study on controversy detection that concludes that a commonly-used dataset has been exhausted: in order to advance the state-of-the-art, more data must be gathered at the current level of label agreement in order to distinguish between techniques with confidence. △ Less

Submitted 13 June, 2018; originally announced June 2018.

Comments: 2-page position paper at LND4IR

Named Entity Recognition with Extremely Limited Data

Authors: John Foley, Sheikh Muhammad Sarwar, James Allan

Abstract: Traditional information retrieval treats named entity recognition as a pre-indexing corpus annotation task, allowing entity tags to be indexed and used during search. Named entity taggers themselves are typically trained on thousands or tens of thousands of examples labeled by humans. However, there is a long tail of named entities classes, and for these cases, labeled data may be impossible to… ▽ More Traditional information retrieval treats named entity recognition as a pre-indexing corpus annotation task, allowing entity tags to be indexed and used during search. Named entity taggers themselves are typically trained on thousands or tens of thousands of examples labeled by humans. However, there is a long tail of named entities classes, and for these cases, labeled data may be impossible to find or justify financially. We propose exploring named entity recognition as a search task, where the named entity class of interest is a query, and entities of that class are the relevant "documents". What should that query look like? Can we even perform NER-style labeling with tens of labels? This study presents an exploration of CRF-based NER models with handcrafted features and of how we might transform them into search queries. △ Less

Submitted 13 June, 2018; v1 submitted 12 June, 2018; originally announced June 2018.

Comments: 6 pages

ACM Class: H.3.2; H.3.3

On the Equivalence of Generative and Discriminative Formulations of the Sequential Dependence Model

Authors: Laura Dietz, John Foley

Abstract: The sequential dependence model (SDM) is a popular retrieval model which is based on the theory of probabilistic graphical models. While it was originally introduced by Metzler and Croft as a Markov Random Field (aka discriminative probabilistic model), in this paper we demonstrate that it is equivalent to a generative probabilistic model. To build an foundation for future retrieval models, this… ▽ More The sequential dependence model (SDM) is a popular retrieval model which is based on the theory of probabilistic graphical models. While it was originally introduced by Metzler and Croft as a Markov Random Field (aka discriminative probabilistic model), in this paper we demonstrate that it is equivalent to a generative probabilistic model. To build an foundation for future retrieval models, this paper details the axiomatic underpinning of the SDM model as discriminative and generative probabilistic model. The only difference arises whether model parameters are estimated in log-space or Multinomial-space. We demonstrate that parameter-estimation with grid-tuning is negatively impacting the generative formulation, an effect that vanishes when parameters are estimated with coordinate-gradient descent. This is concerning, since empirical differences may be falsely attributed to improved models. △ Less

Submitted 30 April, 2018; originally announced May 2018.

Comments: SIGIR'17 Workshop on Axiomatic Thinking for Information Retrieval and Related Tasks (ATIR)

Term Relevance Feedback for Contextual Named Entity Retrieval

Authors: Sheikh Muhammad Sarwar, John Foley, James Allan

Abstract: We address the role of a user in Contextual Named Entity Retrieval (CNER), showing (1) that user identification of important context-bearing terms is superior to automated approaches, and (2) that further gains are possible if the user indicates the relative importance of those terms. CNER is similar in spirit to List Question answering and Entity disambiguation. However, the main focus of CNER is… ▽ More We address the role of a user in Contextual Named Entity Retrieval (CNER), showing (1) that user identification of important context-bearing terms is superior to automated approaches, and (2) that further gains are possible if the user indicates the relative importance of those terms. CNER is similar in spirit to List Question answering and Entity disambiguation. However, the main focus of CNER is to obtain user feedback for constructing a profile for a class of entities on the fly and use that to retrieve entities from free text. Given a sentence, and an entity selected from that sentence, CNER aims to retrieve sentences that have entities similar to query entity. This paper explores obtaining term relevance feedback and importance weighting from humans in order to improve a CNER system. We report our findings based on the efforts of IR researchers as well as crowdsourced workers. △ Less

Submitted 8 January, 2018; originally announced January 2018.

Wireless Scheduling Algorithms in Complex Environments

Authors: Helga Gudmundsdottir, Eyjólfur I Ásgeirsson, Marijke H. L. Bodlaender, Joseph T. Foley, Magnús M. Halldórsson, Ymir Vigfusson

Abstract: Efficient spectrum use in wireless sensor networks through spatial reuse requires effective models of packet reception at the physical layer in the presence of interference. Despite recent progress in analytic and simulations research into worst-case behavior from interference effects, these efforts generally assume geometric path loss and isotropic transmission, assumptions which have not been bo… ▽ More Efficient spectrum use in wireless sensor networks through spatial reuse requires effective models of packet reception at the physical layer in the presence of interference. Despite recent progress in analytic and simulations research into worst-case behavior from interference effects, these efforts generally assume geometric path loss and isotropic transmission, assumptions which have not been borne out in experiments. Our paper aims to provide a methodology for grounding theoretical results into wireless interference in experimental reality. We develop a new framework for wireless algorithms in which distance-based path loss is replaced by an arbitrary gain matrix, typically obtained by measurements of received signal strength (RSS). Gain matrices allow for the modeling of complex environments, e.g., with obstacles and walls. We experimentally evaluate the framework in two indoors testbeds with 20 and 60 motes, and confirm superior predictive performance in packet reception rate for a gain matrix model over a geometric distance-based model. At the heart of our approach is a new parameter $ζ$ called metricity which indicates how close the gain matrix is to a distance metric, effectively measuring the complexity of the environment. A powerful theoretical feature of this parameter is that all known SINR scheduling algorithms that work in general metric spaces carry over to arbitrary gain matrices and achieve equivalent performance guarantees in terms of $ζ$ as previously obtained in terms of the path loss constant. Our experiments confirm the sensitivity of $ζ$ to the nature of the environment. Finally, we show analytically and empirically how multiple channels can be leveraged to improve metricity and thereby performance. We believe our contributions will facilitate experimental validation for recent advances in algorithms for physical wireless interference models. △ Less

Submitted 16 May, 2014; v1 submitted 8 January, 2014; originally announced January 2014.