Showing 1–14 of 14 results for author: Hale, J

FEniCSx Preconditioning Tools (FEniCSx-pctools)

Authors: Martin Řehoř, Jack S. Hale

Abstract: FEniCSx Preconditioning Tools (FEniCSx-pctools) is a software package for easing the specification of PETSc-based block preconditioning strategies in the DOLFINx finite element solver of the FEniCS Project. It attaches all of the necessary metadata to the block-structured linear systems in order that block-structured preconditioners can be applied straightforwardly via PETSc's options-based config… ▽ More FEniCSx Preconditioning Tools (FEniCSx-pctools) is a software package for easing the specification of PETSc-based block preconditioning strategies in the DOLFINx finite element solver of the FEniCS Project. It attaches all of the necessary metadata to the block-structured linear systems in order that block-structured preconditioners can be applied straightforwardly via PETSc's options-based configuration system. Fast prototy** is facilitated thanks to the implementation in Python, and all intensive operations are executed in C/C++. FEniCSx-pctools is available under the LGPLv3 or later license. △ Less

Submitted 4 February, 2024; originally announced February 2024.

Comments: 7 pages, 2 figures, 1 table

MSC Class: 65N22; 65F08; 65F10

Multipath parsing in the brain

Authors: Berta Franzluebbers, Donald Dunagan, Miloš Stanojević, Jan Buys, John T. Hale

Abstract: Humans understand sentences word-by-word, in the order that they hear them. This incrementality entails resolving temporary ambiguities about syntactic relationships. We investigate how humans process these syntactic ambiguities by correlating predictions from incremental generative dependency parsers with timecourse data from people undergoing functional neuroimaging while listening to an audiobo… ▽ More Humans understand sentences word-by-word, in the order that they hear them. This incrementality entails resolving temporary ambiguities about syntactic relationships. We investigate how humans process these syntactic ambiguities by correlating predictions from incremental generative dependency parsers with timecourse data from people undergoing functional neuroimaging while listening to an audiobook. In particular, we compare competing hypotheses regarding the number of develo** syntactic analyses in play during word-by-word comprehension: one vs more than one. This comparison involves evaluating syntactic surprisal from a state-of-the-art dependency parser with LLM-adapted encodings against an existing fMRI dataset. In both English and Chinese data, we find evidence for multipath parsing. Brain regions associated with this multipath effect include bilateral superior temporal gyrus. △ Less

Submitted 6 June, 2024; v1 submitted 31 January, 2024; originally announced January 2024.

Comments: Accepted at ACL2024, main conference. 15 pages

Transformation vs Tradition: Artificial General Intelligence (AGI) for Arts and Humanities

Authors: Zhengliang Liu, Yiwei Li, Qian Cao, Junwen Chen, Tianze Yang, Zihao Wu, John Hale, John Gibbs, Khaled Rasheed, Ninghao Liu, Gengchen Mai, Tianming Liu

Abstract: Recent advances in artificial general intelligence (AGI), particularly large language models and creative image generation systems have demonstrated impressive capabilities on diverse tasks spanning the arts and humanities. However, the swift evolution of AGI has also raised critical questions about its responsible deployment in these culturally significant domains traditionally seen as profoundly… ▽ More Recent advances in artificial general intelligence (AGI), particularly large language models and creative image generation systems have demonstrated impressive capabilities on diverse tasks spanning the arts and humanities. However, the swift evolution of AGI has also raised critical questions about its responsible deployment in these culturally significant domains traditionally seen as profoundly human. This paper provides a comprehensive analysis of the applications and implications of AGI for text, graphics, audio, and video pertaining to arts and the humanities. We survey cutting-edge systems and their usage in areas ranging from poetry to history, marketing to film, and communication to classical art. We outline substantial concerns pertaining to factuality, toxicity, biases, and public safety in AGI systems, and propose mitigation strategies. The paper argues for multi-stakeholder collaboration to ensure AGI promotes creativity, knowledge, and cultural values without undermining truth or human dignity. Our timely contribution summarizes a rapidly develo** field, highlighting promising directions while advocating for responsible progress centering on human flourishing. The analysis lays the groundwork for further research on aligning AGI's technological capacities with enduring social goods. △ Less

Submitted 30 October, 2023; originally announced October 2023.

ACM Class: J.5; I.2.7; I.2.10

Modeling structure-building in the brain with CCG parsing and large language models

Authors: Miloš Stanojević, Jonathan R. Brennan, Donald Dunagan, Mark Steedman, John T. Hale

Abstract: To model behavioral and neural correlates of language comprehension in naturalistic environments researchers have turned to broad-coverage tools from natural-language processing and machine learning. Where syntactic structure is explicitly modeled, prior work has relied predominantly on context-free grammars (CFG), yet such formalisms are not sufficiently expressive for human languages. Combinator… ▽ More To model behavioral and neural correlates of language comprehension in naturalistic environments researchers have turned to broad-coverage tools from natural-language processing and machine learning. Where syntactic structure is explicitly modeled, prior work has relied predominantly on context-free grammars (CFG), yet such formalisms are not sufficiently expressive for human languages. Combinatory Categorial Grammars (CCGs) are sufficiently expressive directly compositional models of grammar with flexible constituency that affords incremental interpretation. In this work we evaluate whether a more expressive CCG provides a better model than a CFG for human neural signals collected with fMRI while participants listen to an audiobook story. We further test between variants of CCG that differ in how they handle optional adjuncts. These evaluations are carried out against a baseline that includes estimates of next-word predictability from a Transformer neural network language model. Such a comparison reveals unique contributions of CCG structure-building predominantly in the left posterior temporal lobe: CCG-derived measures offer a superior fit to neural signals compared to those derived from a CFG. These effects are spatially distinct from bilateral superior temporal effects that are unique to predictability. Neural effects for structure-building are thus separable from predictability during naturalistic listening, and those effects are best characterized by a grammar whose expressive power is motivated on independent linguistic grounds. △ Less

Submitted 16 April, 2023; v1 submitted 28 October, 2022; originally announced October 2022.

Quantifying Discourse Support for Omitted Pronouns

Authors: Shulin Zhang, Jixing Li, John Hale

Abstract: Pro-drop is commonly seen in many languages, but its discourse motivations have not been well characterized. Inspired by the topic chain theory in Chinese, this study shows how character-verb usage continuity distinguishes dropped pronouns from overt references to story characters. We model the choice to drop vs. not drop as a function of character-verb continuity. The results show that omitted su… ▽ More Pro-drop is commonly seen in many languages, but its discourse motivations have not been well characterized. Inspired by the topic chain theory in Chinese, this study shows how character-verb usage continuity distinguishes dropped pronouns from overt references to story characters. We model the choice to drop vs. not drop as a function of character-verb continuity. The results show that omitted subjects have higher character history-current verb continuity salience than non-omitted subjects. This is consistent with the idea that discourse coherence with a particular topic, such as a story character, indeed facilitates the omission of pronouns in languages and contexts where they are optional. △ Less

Submitted 16 September, 2022; originally announced September 2022.

Comments: to be published in Proceedings of the Fifth Workshop on Computational Models of Reference, Anaphora and Coreference, 2022

SOniCS: Develop intuition on biomechanical systems through interactive error controlled simulations

Authors: Arnaud Mazier, Sidaty El Hadramy, Jean-Nicolas Brunet, Jack S. Hale, Stéphane Cotin, Stéphane P. A. Bordas

Abstract: This new approach allows the user to experiment with model choices easily and quickly without requiring in-depth expertise, as constitutive models can be modified by one line of code only. This ease in building new models makes SOniCS ideal to develop surrogate, reduced order models and to train machine learning algorithms for uncertainty quantification or to enable patient-specific simulations. S… ▽ More This new approach allows the user to experiment with model choices easily and quickly without requiring in-depth expertise, as constitutive models can be modified by one line of code only. This ease in building new models makes SOniCS ideal to develop surrogate, reduced order models and to train machine learning algorithms for uncertainty quantification or to enable patient-specific simulations. SOniCS is thus not only a tool that facilitates the development of surgical training simulations but also, and perhaps more importantly, paves the way to increase the intuition of users or otherwise non-intuitive behaviors of (bio)mechanical systems. The plugin uses new developments of the FEniCSx project enabling automatic generation with FFCx of finite element tensors such as the local residual vector and Jacobian matrix. We validate our approach with numerical simulations such as manufactured solutions, cantilever beams, and benchmarks provided by FEBio. We reach machine precision accuracy and demonstrate the use of the plugin for a real-time haptic simulation involving a surgical tool controlled by the user in contact with a hyperelastic liver. We include complete examples showing the use of our plugin for simulations involving Saint Venant-Kirchhoff, Neo-Hookean, Mooney-Rivlin, and Holzapfel Ogden anisotropic models as supplementary material. △ Less

Submitted 24 August, 2022; originally announced August 2022.

Neural Language Models are not Born Equal to Fit Brain Data, but Training Helps

Authors: Alexandre Pasquiou, Yair Lakretz, John Hale, Bertrand Thirion, Christophe Pallier

Abstract: Neural Language Models (NLMs) have made tremendous advances during the last years, achieving impressive performance on various linguistic tasks. Capitalizing on this, studies in neuroscience have started to use NLMs to study neural activity in the human brain during language processing. However, many questions remain unanswered regarding which factors determine the ability of a neural language mod… ▽ More Neural Language Models (NLMs) have made tremendous advances during the last years, achieving impressive performance on various linguistic tasks. Capitalizing on this, studies in neuroscience have started to use NLMs to study neural activity in the human brain during language processing. However, many questions remain unanswered regarding which factors determine the ability of a neural language model to capture brain activity (aka its 'brain score'). Here, we make first steps in this direction and examine the impact of test loss, training corpus and model architecture (comparing GloVe, LSTM, GPT-2 and BERT), on the prediction of functional Magnetic Resonance Imaging timecourses of participants listening to an audiobook. We find that (1) untrained versions of each model already explain significant amount of signal in the brain by capturing similarity in brain responses across identical words, with the untrained LSTM outperforming the transformerbased models, being less impacted by the effect of context; (2) that training NLP models improves brain scores in the same brain regions irrespective of the model's architecture; (3) that Perplexity (test loss) is not a good predictor of brain score; (4) that training data have a strong influence on the outcome and, notably, that off-the-shelf models may lack statistical power to detect brain activations. Overall, we outline the impact of modeltraining choices, and suggest good practices for future studies aiming at explaining the human language system using neural language models. △ Less

Submitted 7 July, 2022; originally announced July 2022.

Journal ref: ICML 2022 - 39th International Conference on Machine Learning, Jul 2022, Baltimore, United States. pp.18

Building on Huang et al. GlossBERT for Word Sense Disambiguation

Authors: Nikhil Patel, James Hale, Kanika **dal, Apoorva Sharma, Yichun Yu

Abstract: We propose to take on the problem ofWord Sense Disambiguation (WSD). In language, words of the same form can take different meanings depending on context. While humans easily infer the meaning or gloss of such words by their context, machines stumble on this task.As such, we intend to replicated and expand upon the results of Huang et al.GlossBERT, a model which they design to disambiguate these w… ▽ More We propose to take on the problem ofWord Sense Disambiguation (WSD). In language, words of the same form can take different meanings depending on context. While humans easily infer the meaning or gloss of such words by their context, machines stumble on this task.As such, we intend to replicated and expand upon the results of Huang et al.GlossBERT, a model which they design to disambiguate these words (Huang et al.,2019). Specifically, we propose the following augmentations: data-set tweaking(alpha hyper-parameter), ensemble methods, and replacement of BERT with BART andALBERT. The following GitHub repository contains all code used in this report, which extends on the code made available by Huang et al. △ Less

Submitted 13 December, 2021; originally announced December 2021.

Inverse deformation analysis: an experimental and numerical assessment using the FEniCS Project

Authors: Arnaud Mazier, Alexandre Bilger, Antonio E. Forte, Igor Peterlik, Jack S. Hale, Stéphane P. A. Bordas, ., Institute of Computational Engineering, Department of Engineering, University of Luxembourg, Esch-sur-Alzette, Luxembourg., Harvard University, Cambridge, USA., Department of Electronics, Information, Bioengineering, Politecnico di Milano, Milan, Italy., Institute of Computer Science, Masaryk University, Czech Republic., Institute of Research , et al. (3 additional authors not shown)

Abstract: In this paper, we develop a framework for solving inverse deformation problems using the FEniCS Project finite element software. We validate our approach with experimental imaging data acquired from a soft silicone beam under gravity. In contrast with inverse iterative algorithms that require multiple solutions of a standard elasticity problem, the proposed method can compute the undeformed config… ▽ More In this paper, we develop a framework for solving inverse deformation problems using the FEniCS Project finite element software. We validate our approach with experimental imaging data acquired from a soft silicone beam under gravity. In contrast with inverse iterative algorithms that require multiple solutions of a standard elasticity problem, the proposed method can compute the undeformed configuration by solving only one modified elasticity problem. This modified problem has a complexity comparable to the standard one. The framework is implemented within an open-source pipeline enabling the direct and inverse deformation simulation directly from imaging data. We use the high-level Unified Form Language (UFL) of the FEniCS Project to express the finite element model in variational form and to automatically derive the consistent Jacobian. Consequently, the design of the pipeline is flexible: for example, it allows the modification of the constitutive models by changing a single line of code. We include a complete working example showing the inverse deformation of a beam deformed by gravity as supplementary material. △ Less

Submitted 26 February, 2021; originally announced February 2021.

Comments: 33 pages, 12 figures, submitted to Elsevier

Hierarchical a posteriori error estimation of Bank-Weiser type in the FEniCS Project

Authors: Raphaël Bulle, Jack S. Hale, Alexei Lozinski, Stéphane P. A. Bordas, Franz Chouly

Abstract: In the seminal paper of Bank and Weiser [Math. Comp., 44 (1985), pp.283-301] a new a posteriori estimator was introduced. This estimator requires the solution of a local Neumann problem on every cell of the finite element mesh. Despite the promise of Bank-Weiser type estimators, namely locality, computational efficiency, and asymptotic sharpness, they have seen little use in practical computationa… ▽ More In the seminal paper of Bank and Weiser [Math. Comp., 44 (1985), pp.283-301] a new a posteriori estimator was introduced. This estimator requires the solution of a local Neumann problem on every cell of the finite element mesh. Despite the promise of Bank-Weiser type estimators, namely locality, computational efficiency, and asymptotic sharpness, they have seen little use in practical computational problems. The focus of this contribution is to describe a novel implementation of hierarchical estimators of the Bank-Weiser type in a modern high-level finite element software with automatic code generation capabilities. We show how to use the estimator to drive (goal-oriented) adaptive mesh refinement and to mixed approximations of the nearly-incompressible elasticity problems. We provide comparisons with various other used estimators. An open-source implementation based on the FEniCS Project finite element software is provided as supplementary material. △ Less

Submitted 7 February, 2022; v1 submitted 8 February, 2021; originally announced February 2021.

Towards Automated Sexual Violence Report Tracking

Authors: Naeemul Hassan, Amrit Poudel, Jason Hale, Claire Hubacek, Khandakar Tasnim Huq, Shubhra Kanti Karmaker Santu, Syed Ishtiaque Ahmed

Abstract: Tracking sexual violence is a challenging task. In this paper, we present a supervised learning-based automated sexual violence report tracking model that is more scalable, and reliable than its crowdsource based counterparts. We define the sexual violence report tracking problem by considering victim, perpetrator contexts and the nature of the violence. We find that our model could identify sexua… ▽ More Tracking sexual violence is a challenging task. In this paper, we present a supervised learning-based automated sexual violence report tracking model that is more scalable, and reliable than its crowdsource based counterparts. We define the sexual violence report tracking problem by considering victim, perpetrator contexts and the nature of the violence. We find that our model could identify sexual violence reports with a precision and recall of 80.4% and 83.4%, respectively. Moreover, we also applied the model during and after the \#MeToo movement. Several interesting findings are discovered which are not easily identifiable from a shallow analysis. △ Less

Submitted 16 November, 2019; originally announced November 2019.

Finding Syntax in Human Encephalography with Beam Search

Authors: John Hale, Chris Dyer, Adhiguna Kuncoro, Jonathan R. Brennan

Abstract: Recurrent neural network grammars (RNNGs) are generative models of (tree,string) pairs that rely on neural networks to evaluate derivational choices. Parsing with them using beam search yields a variety of incremental complexity metrics such as word surprisal and parser action count. When used as regressors against human electrophysiological responses to naturalistic text, they derive two amplitud… ▽ More Recurrent neural network grammars (RNNGs) are generative models of (tree,string) pairs that rely on neural networks to evaluate derivational choices. Parsing with them using beam search yields a variety of incremental complexity metrics such as word surprisal and parser action count. When used as regressors against human electrophysiological responses to naturalistic text, they derive two amplitude effects: an early peak and a P600-like later peak. By contrast, a non-syntactic neural language model yields no reliable effects. Model comparisons attribute the early peak to syntactic composition within the RNNG. This pattern of results recommends the RNNG+beam search combination as a mechanistic model of the syntactic processing that occurs during normal human language comprehension. △ Less

Submitted 11 June, 2018; originally announced June 2018.

Comments: ACL2018

Containers for portable, productive and performant scientific computing

Authors: Jack S. Hale, Lizao Li, Chris N. Richardson, Garth N. Wells

Abstract: Containers are an emerging technology that hold promise for improving productivity and code portability in scientific computing. We examine Linux container technology for the distribution of a non-trivial scientific computing software stack and its execution on a spectrum of platforms from laptop computers through to high performance computing (HPC) systems. We show on a workstation and a leadersh… ▽ More Containers are an emerging technology that hold promise for improving productivity and code portability in scientific computing. We examine Linux container technology for the distribution of a non-trivial scientific computing software stack and its execution on a spectrum of platforms from laptop computers through to high performance computing (HPC) systems. We show on a workstation and a leadership-class HPC system that when deployed appropriately there are no performance penalties running scientific programs inside containers. For Python code run on large parallel computers, the run time is reduced inside a container due to faster library imports. The software distribution approach and data that we present will help developers and users decide on whether container technology is appropriate for them. We also provide guidance for the vendors of HPC systems that rely on proprietary libraries for performance on what they can do to make containers work seamlessly and without performance penalty. △ Less

Submitted 3 November, 2016; v1 submitted 26 August, 2016; originally announced August 2016.

MSC Class: 68N99; 68U0 ACM Class: D.2.7; D.2.8; D.2.9

Bayesian inference for the stochastic identification of elastoplastic material parameters: Introduction, misconceptions and insights

Authors: Hussein Rappel, Lars A. A. Beex, Jack S. Hale, Stephane P. A. Bordas

Abstract: We discuss Bayesian inference (BI) for the probabilistic identification of material parameters. This contribution aims to shed light on the use of BI for the identification of elastoplastic material parameters. For this purpose a single spring is considered, for which the stress-strain curves are artificially created. Besides offering a didactic introduction to BI, this paper proposes an approach… ▽ More We discuss Bayesian inference (BI) for the probabilistic identification of material parameters. This contribution aims to shed light on the use of BI for the identification of elastoplastic material parameters. For this purpose a single spring is considered, for which the stress-strain curves are artificially created. Besides offering a didactic introduction to BI, this paper proposes an approach to incorporate statistical errors both in the measured stresses, and in the measured strains. It is assumed that the uncertainty is only due to measurement errors and the material is homogeneous. Furthermore, a number of possible misconceptions on BI are highlighted based on the purely elastic case. △ Less

Submitted 15 January, 2017; v1 submitted 8 June, 2016; originally announced June 2016.