Showing 1–18 of 18 results for author: Jones, W

BrainSLAM: SLAM on Neural Population Activity Data

Authors: Kipp Freud, Nathan Lepora, Matt W. Jones, Cian O'Donnell

Abstract: Simultaneous localisation and map** (SLAM) algorithms are commonly used in robotic systems for learning maps of novel environments. Brains also appear to learn maps, but the mechanisms are not known and it is unclear how to infer these maps from neural activity data. We present BrainSLAM; a method for performing SLAM using only population activity (local field potential, LFP) data simultaneously… ▽ More Simultaneous localisation and map** (SLAM) algorithms are commonly used in robotic systems for learning maps of novel environments. Brains also appear to learn maps, but the mechanisms are not known and it is unclear how to infer these maps from neural activity data. We present BrainSLAM; a method for performing SLAM using only population activity (local field potential, LFP) data simultaneously recorded from three brain regions in rats: hippocampus, prefrontal cortex, and parietal cortex. This system uses a convolutional neural network (CNN) to decode velocity and familiarity information from wavelet scalograms of neural local field potential data recorded from rats as they navigate a 2D maze. The CNN's output drives a RatSLAM-inspired architecture, powering an attractor network which performs path integration plus a separate system which performs `loop closure' (detecting previously visited locations and correcting map aliasing errors). Together, these three components can construct faithful representations of the environment while simultaneously tracking the animal's location. This is the first demonstration of inference of a spatial map from brain recordings. Our findings expand SLAM to a new modality, enabling a new method of map** environments and facilitating a better understanding of the role of cognitive maps in navigation and decision making. △ Less

Submitted 1 February, 2024; originally announced February 2024.

Comments: Accepted to the 23rd International Conference on Autonomous Agents and Multiagent Systems. 2024

Perspectives on AI Architectures and Co-design for Earth System Predictability

Authors: Maruti K. Mudunuru, James A. Ang, Mahantesh Halappanavar, Simon D. Hammond, Maya B. Gokhale, James C. Hoe, Tushar Krishna, Sarat S. Sreepathi, Matthew R. Norman, Ivy B. Peng, Philip W. Jones

Abstract: Recently, the U.S. Department of Energy (DOE), Office of Science, Biological and Environmental Research (BER), and Advanced Scientific Computing Research (ASCR) programs organized and held the Artificial Intelligence for Earth System Predictability (AI4ESP) workshop series. From this workshop, a critical conclusion that the DOE BER and ASCR community came to is the requirement to develop a new par… ▽ More Recently, the U.S. Department of Energy (DOE), Office of Science, Biological and Environmental Research (BER), and Advanced Scientific Computing Research (ASCR) programs organized and held the Artificial Intelligence for Earth System Predictability (AI4ESP) workshop series. From this workshop, a critical conclusion that the DOE BER and ASCR community came to is the requirement to develop a new paradigm for Earth system predictability focused on enabling artificial intelligence (AI) across the field, lab, modeling, and analysis activities, called ModEx. The BER's `Model-Experimentation', ModEx, is an iterative approach that enables process models to generate hypotheses. The developed hypotheses inform field and laboratory efforts to collect measurement and observation data, which are subsequently used to parameterize, drive, and test model (e.g., process-based) predictions. A total of 17 technical sessions were held in this AI4ESP workshop series. This paper discusses the topic of the `AI Architectures and Co-design' session and associated outcomes. The AI Architectures and Co-design session included two invited talks, two plenary discussion panels, and three breakout rooms that covered specific topics, including: (1) DOE HPC Systems, (2) Cloud HPC Systems, and (3) Edge computing and Internet of Things (IoT). We also provide forward-looking ideas and perspectives on potential research in this co-design area that can be achieved by synergies with the other 16 session topics. These ideas include topics such as: (1) reimagining co-design, (2) data acquisition to distribution, (3) heterogeneous HPC solutions for integration of AI/ML and other data analytics like uncertainty quantification with earth system modeling and simulation, and (4) AI-enabled sensor integration into earth system measurements and observations. Such perspectives are a distinguishing aspect of this paper. △ Less

Submitted 7 April, 2023; originally announced April 2023.

Comments: 23 pages, 1 figure

Interpretable Classification of Early Stage Parkinson's Disease from EEG

Authors: Amarpal Sahota, Amber Roguski, Matthew W. Jones, Michal Rolinski, Alan Whone, Raul Santos-Rodriguez, Zahraa S. Abdallah

Abstract: Detecting Parkinson's Disease in its early stages using EEG data presents a significant challenge. This paper introduces a novel approach, representing EEG data as a 15-variate series of bandpower and peak frequency values/coefficients. The hypothesis is that this representation captures essential information from the noisy EEG signal, improving disease detection. Statistical features extracted fr… ▽ More Detecting Parkinson's Disease in its early stages using EEG data presents a significant challenge. This paper introduces a novel approach, representing EEG data as a 15-variate series of bandpower and peak frequency values/coefficients. The hypothesis is that this representation captures essential information from the noisy EEG signal, improving disease detection. Statistical features extracted from this representation are utilised as input for interpretable machine learning models, specifically Decision Tree and AdaBoost classifiers. Our classification pipeline is deployed within our proposed framework which enables high-importance data types and brain regions for classification to be identified. Interestingly, our analysis reveals that while there is no significant regional importance, the N1 sleep data type exhibits statistically significant predictive power (p < 0.01) for early-stage Parkinson's Disease classification. AdaBoost classifiers trained on the N1 data type consistently outperform baseline models, achieving over 80% accuracy and recall. Our classification pipeline statistically significantly outperforms baseline models indicating that the model has acquired useful information. Paired with the interpretability (ability to view feature importance's) of our pipeline this enables us to generate meaningful insights into the classification of early stage Parkinson's with our N1 models. In Future, these models could be deployed in the real world - the results presented in this paper indicate that more than 3 in 4 early-stage Parkinson's cases would be captured with our pipeline. △ Less

Submitted 8 December, 2023; v1 submitted 20 January, 2023; originally announced January 2023.

A Comparison of Model-Free and Model Predictive Control for Price Responsive Water Heaters

Authors: David J. Biagioni, Xiangyu Zhang, Peter Graf, Devon Sigler, Wesley Jones

Abstract: We present a careful comparison of two model-free control algorithms, Evolution Strategies (ES) and Proximal Policy Optimization (PPO), with receding horizon model predictive control (MPC) for operating simulated, price responsive water heaters. Four MPC variants are considered: a one-shot controller with perfect forecasting yielding optimal control; a limited-horizon controller with perfect forec… ▽ More We present a careful comparison of two model-free control algorithms, Evolution Strategies (ES) and Proximal Policy Optimization (PPO), with receding horizon model predictive control (MPC) for operating simulated, price responsive water heaters. Four MPC variants are considered: a one-shot controller with perfect forecasting yielding optimal control; a limited-horizon controller with perfect forecasting; a mean forecasting-based controller; and a two-stage stochastic programming controller using historical scenarios. In all cases, the MPC model for water temperature and electricity price are exact; only water demand is uncertain. For comparison, both ES and PPO learn neural network-based policies by directly interacting with the simulated environment under the same scenarios used by MPC. All methods are then evaluated on a separate one-week continuation of the demand time series. We demonstrate that optimal control for this problem is challenging, requiring more than 8-hour lookahead for MPC with perfect forecasting to attain the minimum cost. Despite this challenge, both ES and PPO learn good general purpose policies that outperform mean forecast and two-stage stochastic MPC controllers in terms of average cost and are more than two orders of magnitude faster at computing actions. We show that ES in particular can leverage parallelism to learn a policy in under 90 seconds using 1150 CPU cores. △ Less

Submitted 8 November, 2021; originally announced November 2021.

Comments: All authors are with the Computational Science Center at the National Renewable Energy Laboratory. Corresponding author: David Biagioni ([email protected])

Journal ref: In Proceedings of the 1st International Workshop on Reinforcement Learning for Energy Management in Buildings & Cities, pp. 29-33. 2020

Learning offline: memory replay in biological and artificial reinforcement learning

Authors: Emma L. Roscow, Raymond Chua, Rui Ponte Costa, Matt W. Jones, Nathan Lepora

Abstract: Learning to act in an environment to maximise rewards is among the brain's key functions. This process has often been conceptualised within the framework of reinforcement learning, which has also gained prominence in machine learning and artificial intelligence (AI) as a way to optimise decision-making. A common aspect of both biological and machine reinforcement learning is the reactivation of pr… ▽ More Learning to act in an environment to maximise rewards is among the brain's key functions. This process has often been conceptualised within the framework of reinforcement learning, which has also gained prominence in machine learning and artificial intelligence (AI) as a way to optimise decision-making. A common aspect of both biological and machine reinforcement learning is the reactivation of previously experienced episodes, referred to as replay. Replay is important for memory consolidation in biological neural networks, and is key to stabilising learning in deep neural networks. Here, we review recent developments concerning the functional roles of replay in the fields of neuroscience and AI. Complementary progress suggests how replay might support learning processes, including generalisation and continual learning, affording opportunities to transfer knowledge across the two fields to advance the understanding of biological and artificial learning and memory. △ Less

Submitted 21 September, 2021; originally announced September 2021.

Comments: In press at Trends in Neurosciences

Mobilkit: A Python Toolkit for Urban Resilience and Disaster Risk Management Analytics using High Frequency Human Mobility Data

Authors: Enrico Ubaldi, Takahiro Yabe, Nicholas K. W. Jones, Maham Faisal Khan, Satish V. Ukkusuri, Riccardo Di Clemente, Emanuele Strano

Abstract: Increasingly available high-frequency location datasets derived from smartphones provide unprecedented insight into trajectories of human mobility. These datasets can play a significant and growing role in informing preparedness and response to natural disasters. However, limited tools exist to enable rapid analytics using mobility data, and tend not to be tailored specifically for disaster risk m… ▽ More Increasingly available high-frequency location datasets derived from smartphones provide unprecedented insight into trajectories of human mobility. These datasets can play a significant and growing role in informing preparedness and response to natural disasters. However, limited tools exist to enable rapid analytics using mobility data, and tend not to be tailored specifically for disaster risk management. We present an open-source, Python-based toolkit designed to conduct replicable and scalable post-disaster analytics using GPS location data. Privacy, system capabilities, and potential expansions of \textit{Mobilkit} are discussed. △ Less

Submitted 16 September, 2021; v1 submitted 29 July, 2021; originally announced July 2021.

Comments: 3 pages, 1 figure, KDD KDD Workshop on Data-driven Humanitarian Map**, 27th ACM SIGKDD Conference

ACM Class: J.2

Journal ref: Journal of Open Source Software, 9(95), 5201, 2024

Personal Information Management

Authors: William Jones, Jesse David Dinneen, Robert Capra, Anne R. Diekema, Manuel A. Pérez-Quiñones

Abstract: Personal Information Management (PIM) refers to the practice and the study of the activities a person performs in order to acquire or create, store, organize, maintain, retrieve, use, and distribute information in each of its many forms (paper and digital, in e-mails, files, Web pages, text messages, tweets, posts, etc.) as needed to meet life's many goals (everyday and long-term, work-related and… ▽ More Personal Information Management (PIM) refers to the practice and the study of the activities a person performs in order to acquire or create, store, organize, maintain, retrieve, use, and distribute information in each of its many forms (paper and digital, in e-mails, files, Web pages, text messages, tweets, posts, etc.) as needed to meet life's many goals (everyday and long-term, work-related and not) and to fulfill life's many roles and responsibilities (as parent, spouse, friend, employee, member of community, etc.). PIM activities are an effort to establish, use, and maintain a map** between information and need. Activities of finding (and re-finding) move from a current need toward information while activities of kee** move from encountered information toward anticipated need. Meta-level activities such as maintaining, organizing, and managing the flow of information focus on the map** itself. Tools and techniques of PIM can promote information integration with benefits for each kind of PIM activity and across the life cycle of personal information. Understanding how best to accomplish this integration without inadvertently creating problems along the way is a key challenge of PIM. △ Less

Submitted 7 July, 2021; originally announced July 2021.

Comments: Final version available at https://www.routledgehandbooks.com/doi/10.1081/E-ELIS4-120053695

Journal ref: Chapter in Levine-Clark, M., & McDonald, J. (Eds.), Encyclopedia of Library and Information Science, Fourth Edition, 2017, pp. 3584-3605. New York, NY: Taylor & Francis

Linear solvers for power grid optimization problems: a review of GPU-accelerated linear solvers

Authors: Kasia Swirydowicz, Eric Darve, Wesley Jones, Jonathan Maack, Shaked Regev, Michael A. Saunders, Stephen J. Thomas, Slaven Peles

Abstract: The linear equations that arise in interior methods for constrained optimization are sparse symmetric indefinite and become extremely ill-conditioned as the interior method converges. These linear systems present a challenge for existing solver frameworks based on sparse LU or LDL^T decompositions. We benchmark five well known direct linear solver packages using matrices extracted from power grid… ▽ More The linear equations that arise in interior methods for constrained optimization are sparse symmetric indefinite and become extremely ill-conditioned as the interior method converges. These linear systems present a challenge for existing solver frameworks based on sparse LU or LDL^T decompositions. We benchmark five well known direct linear solver packages using matrices extracted from power grid optimization problems. The achieved solution accuracy varies greatly among the packages. None of the tested packages delivers significant GPU acceleration for our test cases. △ Less

Submitted 13 August, 2021; v1 submitted 25 June, 2021; originally announced June 2021.

NightVision: Generating Nighttime Satellite Imagery from Infra-Red Observations

Authors: Paula Harder, William Jones, Redouane Lguensat, Shahine Bouabid, James Fulton, Dánell Quesada-Chacón, Aris Marcolongo, Sofija Stefanović, Yuhan Rao, Peter Manshausen, Duncan Watson-Parris

Abstract: The recent explosion in applications of machine learning to satellite imagery often rely on visible images and therefore suffer from a lack of data during the night. The gap can be filled by employing available infra-red observations to generate visible images. This work presents how deep learning can be applied successfully to create those images by using U-Net based architectures. The proposed m… ▽ More The recent explosion in applications of machine learning to satellite imagery often rely on visible images and therefore suffer from a lack of data during the night. The gap can be filled by employing available infra-red observations to generate visible images. This work presents how deep learning can be applied successfully to create those images by using U-Net based architectures. The proposed methods show promising results, achieving a structural similarity index (SSIM) up to 86\% on an independent test set and providing visually convincing output images, generated from infra-red observations. △ Less

Submitted 8 December, 2020; v1 submitted 13 November, 2020; originally announced November 2020.

Uncovering socioeconomic gaps in mobility reduction during the COVID-19 pandemic using location data

Authors: Samuel P. Fraiberger, Pablo Astudillo, Lorenzo Candeago, Alex Chunet, Nicholas K. W. Jones, Maham Faisal Khan, Bruno Lepri, Nancy Lozano Gracia, Lorenzo Lucchini, Emanuele Massaro, Aleister Montfort

Abstract: Using smartphone location data from Colombia, Mexico, and Indonesia, we investigate how non-pharmaceutical policy interventions intended to mitigate the spread of the COVID-19 pandemic impact human mobility. In all three countries, we find that following the implementation of mobility restriction measures, human movement decreased substantially. Importantly, we also uncover large and persistent di… ▽ More Using smartphone location data from Colombia, Mexico, and Indonesia, we investigate how non-pharmaceutical policy interventions intended to mitigate the spread of the COVID-19 pandemic impact human mobility. In all three countries, we find that following the implementation of mobility restriction measures, human movement decreased substantially. Importantly, we also uncover large and persistent differences in mobility reduction between wealth groups: on average, users in the top decile of wealth reduced their mobility up to twice as much as users in the bottom decile. For decision-makers seeking to efficiently allocate resources to response efforts, these findings highlight that smartphone location data can be leveraged to tailor policies to the needs of specific socioeconomic groups, especially the most vulnerable. △ Less

Submitted 27 July, 2020; v1 submitted 26 June, 2020; originally announced June 2020.

Comments: to appear in KDD 2020, Workshop on Humanitarian Map**

K-spin Hamiltonian for quantum-resolvable Markov decision processes

Authors: Eric B. Jones, Peter Graf, Eliot Kapit, Wesley Jones

Abstract: The Markov decision process is the mathematical formalization underlying the modern field of reinforcement learning when transition and reward functions are unknown. We derive a pseudo-Boolean cost function that is equivalent to a K-spin Hamiltonian representation of the discrete, finite, discounted Markov decision process with infinite horizon. This K-spin Hamiltonian furnishes a starting point f… ▽ More The Markov decision process is the mathematical formalization underlying the modern field of reinforcement learning when transition and reward functions are unknown. We derive a pseudo-Boolean cost function that is equivalent to a K-spin Hamiltonian representation of the discrete, finite, discounted Markov decision process with infinite horizon. This K-spin Hamiltonian furnishes a starting point from which to solve for an optimal policy using heuristic quantum algorithms such as adiabatic quantum annealing and the quantum approximate optimization algorithm on near-term quantum hardware. In proving that the variational minimization of our Hamiltonian is equivalent to the Bellman optimality condition we establish an interesting analogy with classical field theory. Along with proof-of-concept calculations to corroborate our formulation by simulated and quantum annealing against classical Q-Learning, we analyze the scaling of physical resources required to solve our Hamiltonian on quantum hardware. △ Less

Submitted 13 April, 2020; originally announced April 2020.

On the Computational Viability of Quantum Optimization for PMU Placement

Authors: Eric B. Jones, Eliot Kapit, Chin-Yao Chang, David Biagioni, Deepthi Vaidhynathan, Peter Graf, Wesley Jones

Abstract: Using optimal phasor measurement unit placement as a prototypical problem, we assess the computational viability of the current generation D-Wave Systems 2000Q quantum annealer for power systems design problems. We reformulate minimum dominating set for the annealer hardware, solve the reformulation for a standard set of IEEE test systems, and benchmark solution quality and time to solution agains… ▽ More Using optimal phasor measurement unit placement as a prototypical problem, we assess the computational viability of the current generation D-Wave Systems 2000Q quantum annealer for power systems design problems. We reformulate minimum dominating set for the annealer hardware, solve the reformulation for a standard set of IEEE test systems, and benchmark solution quality and time to solution against the CPLEX Optimizer and simulated annealing. For some problem instances the 2000Q outpaces CPLEX. For instances where the 2000Q underperforms with respect to CPLEX and simulated annealing, we suggest hardware improvements for the next generation of quantum annealers. △ Less

Submitted 13 January, 2020; originally announced January 2020.

Cops that surround a robber

Authors: Andrea C. Burgess, Rosalind A. Cameron, Nancy E. Clarke, Peter Danziger, Stephen Finbow, Caleb W. Jones, David A. Pike

Abstract: We introduce the game of Surrounding Cops and Robbers on a graph, as a variant of the original game of Cops and Robbers. In contrast to the original game in which the cops win by occupying the same vertex as the robber, they now win by occupying each of the robber's neighbouring vertices. We denote by $σ(G)$ the {\em surrounding cop number} of $G$, namely the least number of cops required to surro… ▽ More We introduce the game of Surrounding Cops and Robbers on a graph, as a variant of the original game of Cops and Robbers. In contrast to the original game in which the cops win by occupying the same vertex as the robber, they now win by occupying each of the robber's neighbouring vertices. We denote by $σ(G)$ the {\em surrounding cop number} of $G$, namely the least number of cops required to surround a robber in the graph $G$. We present a number of results regarding this parameter, including general bounds as well as exact values for several classes of graphs. Particular classes of interest include product graphs, graphs arising from combinatorial designs, and generalised Petersen graphs. △ Less

Submitted 5 August, 2021; v1 submitted 30 October, 2019; originally announced October 2019.

Deep Learned Path Planning via Randomized Reward-Linked-Goals and Potential Space Applications

Authors: Tamir Blum, William Jones, Kazuya Yoshida

Abstract: Space exploration missions have seen use of increasingly sophisticated robotic systems with ever more autonomy. Deep learning promises to take this even a step further, and has applications for high-level tasks, like path planning, as well as low-level tasks, like motion control, which are critical components for mission efficiency and success. Using deep reinforcement end-to-end learning with ran… ▽ More Space exploration missions have seen use of increasingly sophisticated robotic systems with ever more autonomy. Deep learning promises to take this even a step further, and has applications for high-level tasks, like path planning, as well as low-level tasks, like motion control, which are critical components for mission efficiency and success. Using deep reinforcement end-to-end learning with randomized reward function parameters during training, we teach a simulated 8 degree-of-freedom quadruped ant-like robot to travel anywhere within a perimeter, conducting path plan and motion control on a single neural network, without any system model or prior knowledge of the terrain or environment. Our approach also allows for user specified waypoints, which could translate well to either fully autonomous or semi-autonomous/teleoperated space applications that encounter delay times. We trained the agent using randomly generated waypoints linked to the reward function and passed waypoint coordinates as inputs to the neural network. Such applications show promise on a variety of space exploration robots, including high speed rovers for fast locomotion and legged cave robots for rough terrain. △ Less

Submitted 13 September, 2019; originally announced September 2019.

Comments: 8 pages, 3 tables, 5 figures

The infinite Viterbi alignment and decay-convexity

Authors: Nick Whiteley, Matt W. Jones, Aleks P. F. Domanski

Abstract: The infinite Viterbi alignment is the limiting maximum a-posteriori estimate of the unobserved path in a hidden Markov model as the length of the time horizon grows. For models on state-space $\mathbb{R}^{d}$ satisfying a new ``decay-convexity'' condition, we develop an approach to existence of the infinite Viterbi alignment in an infinite dimensional Hilbert space. Quantitative bounds on the dist… ▽ More The infinite Viterbi alignment is the limiting maximum a-posteriori estimate of the unobserved path in a hidden Markov model as the length of the time horizon grows. For models on state-space $\mathbb{R}^{d}$ satisfying a new ``decay-convexity'' condition, we develop an approach to existence of the infinite Viterbi alignment in an infinite dimensional Hilbert space. Quantitative bounds on the distance to the infinite Viterbi alignment, which are the first of their kind, are derived and used to illustrate how approximate estimation via parallelization can be accurate and scaleable to high-dimensional problems because the rate of convergence to the infinite Viterbi alignment does not necessarily depend on $d$. The results are applied to approximate estimation via parallelization and a model of neural population activity. △ Less

Submitted 13 February, 2023; v1 submitted 8 October, 2018; originally announced October 2018.

Graffinity: Visualizing Connectivity In Large Graphs

Authors: Ethan Kerzner, Alexander Lex, Crystal Lynn Sigulinsky, Timothy Urness, Bryan William Jones, Robert E. Marc, Miriah Meyer

Abstract: Multivariate graphs are prolific across many fields, including transportation and neuroscience. A key task in graph analysis is the exploration of connectivity, to, for example, analyze how signals flow through neurons, or to explore how well different cities are connected by flights. While standard node-link diagrams are helpful in judging connectivity, they do not scale to large networks. Adjace… ▽ More Multivariate graphs are prolific across many fields, including transportation and neuroscience. A key task in graph analysis is the exploration of connectivity, to, for example, analyze how signals flow through neurons, or to explore how well different cities are connected by flights. While standard node-link diagrams are helpful in judging connectivity, they do not scale to large networks. Adjacency matrices also do not scale to large networks and are only suitable to judge connectivity of adjacent nodes. A key approach to realize scalable graph visualization are queries: instead of displaying the whole network, only a relevant subset is shown. Query-based techniques for analyzing connectivity in graphs, however, can also easily suffer from cluttering if the query result is big enough. To remedy this, we introduce techniques that provide an overview of the connectivity and reveal details on demand. We have two main contributions: (1) two novel visualization techniques that work in concert for summarizing graph connectivity; and (2) Graffinity, an open-source implementation of these visualizations supplemented by detail views to enable a complete analysis workflow. Graffinity was designed in a close collaboration with neuroscientists and is optimized for connectomics data analysis, yet the technique is applicable across domains. We validate the connectivity overview and our open-source tool with illustrative examples using flight and connectomics data. △ Less

Submitted 22 March, 2017; originally announced March 2017.

Comments: The definitive version is available at http://diglib.eg.org/ and http://onlinelibrary.wiley.com

Ganga: a tool for computational-task management and easy access to Grid resources

Authors: J. T. Mościcki, F. Brochu, J. Ebke, U. Egede, J. Elmsheuser, K. Harrison, R. W. L. Jones, H. C. Lee, D. Liko, A. Maier, A. Muraru, G. N. Patrick, K. Pajchel, W. Reece, B. H. Samset, M. W. Slater, A. Soroko, C. L. Tan, D. C. Vanderster, M. Williams

Abstract: In this paper, we present the computational task-management tool Ganga, which allows for the specification, submission, bookkee** and post-processing of computational tasks on a wide set of distributed resources. Ganga has been developed to solve a problem increasingly common in scientific projects, which is that researchers must regularly switch between different processing systems, each with… ▽ More In this paper, we present the computational task-management tool Ganga, which allows for the specification, submission, bookkee** and post-processing of computational tasks on a wide set of distributed resources. Ganga has been developed to solve a problem increasingly common in scientific projects, which is that researchers must regularly switch between different processing systems, each with its own command set, to complete their computational tasks. Ganga provides a homogeneous environment for processing data on heterogeneous resources. We give examples from High Energy Physics, demonstrating how an analysis can be developed on a local system and then transparently moved to a Grid system for processing of all available data. Ganga has an API that can be used via an interactive interface, in scripts, or through a GUI. Specific knowledge about types of tasks or computational resources is provided at run-time through a plugin system, making new developments easy to integrate. We give an overview of the Ganga architecture, give examples of current use, and demonstrate how Ganga can be used in many different areas of science. △ Less

Submitted 9 June, 2009; v1 submitted 16 February, 2009; originally announced February 2009.

Comments: Extended and clarified information on the Grid computing context for Ganga, supported job model etc. Additional minor corrections and clarifications. Updated the author list as agreed with the Ganga team

ACM Class: C.2.4; H.3.4; J.2; J.3

GANGA: a user-Grid interface for Atlas and LHCb

Authors: K. Harrison, W. T. L. P. Lavrijsen, P. Mato, A. Soroko, C. L. Tan, C. E. Tull, N. Brook, R. W. L. Jones

Abstract: The Gaudi/Athena and Grid Alliance (GANGA) is a front-end for the configuration, submission, monitoring, bookkee**, output collection, and reporting of computing jobs run on a local batch system or on the grid. In particular, GANGA handles jobs that use applications written for the Gaudi software framework shared by the Atlas and LHCb experiments. GANGA exploits the commonality of Gaudi-based… ▽ More The Gaudi/Athena and Grid Alliance (GANGA) is a front-end for the configuration, submission, monitoring, bookkee**, output collection, and reporting of computing jobs run on a local batch system or on the grid. In particular, GANGA handles jobs that use applications written for the Gaudi software framework shared by the Atlas and LHCb experiments. GANGA exploits the commonality of Gaudi-based computing jobs, while insulating against grid-, batch- and framework-specific technicalities, to maximize end-user productivity in defining, configuring, and executing jobs. Designed for a python-based component architecture, GANGA has a modular underpinning and is therefore well placed for contributing to, and benefiting from, work in related projects. Its functionality is accessible both from a scriptable command-line interface, for expert users and automated tasks, and through a graphical interface, which simplifies the interaction with GANGA for beginning and c1asual users. This paper presents the GANGA design and implementation, the development of the underlying software bus architecture, and the functionality of the first public GANGA release. △ Less

Submitted 13 June, 2003; originally announced June 2003.

Comments: 9 pages, 3 figures, CHEP 2003, March 2003, La Jolla, California, PSN TUCT002

ACM Class: D.2.6