Showing 1–10 of 10 results for author: Moreno, J G

Probing Pretrained Language Models with Hierarchy Properties

Authors: Jesús Lovón-Melgarejo, Jose G. Moreno, Romaric Besançon, Olivier Ferret, Lynda Tamine

Abstract: Since Pretrained Language Models (PLMs) are the cornerstone of the most recent Information Retrieval (IR) models, the way they encode semantic knowledge is particularly important. However, little attention has been given to studying the PLMs' capability to capture hierarchical semantic knowledge. Traditionally, evaluating such knowledge encoded in PLMs relies on their performance on a task-depende… ▽ More Since Pretrained Language Models (PLMs) are the cornerstone of the most recent Information Retrieval (IR) models, the way they encode semantic knowledge is particularly important. However, little attention has been given to studying the PLMs' capability to capture hierarchical semantic knowledge. Traditionally, evaluating such knowledge encoded in PLMs relies on their performance on a task-dependent evaluation approach based on proxy tasks, such as hypernymy detection. Unfortunately, this approach potentially ignores other implicit and complex taxonomic relations. In this work, we propose a task-agnostic evaluation method able to evaluate to what extent PLMs can capture complex taxonomy relations, such as ancestors and siblings. The evaluation is based on intrinsic properties that capture the hierarchical nature of taxonomies. Our experimental evaluation shows that the lexico-semantic knowledge implicitly encoded in PLMs does not always capture hierarchical relations. We further demonstrate that the proposed properties can be injected into PLMs to improve their understanding of hierarchy. Through evaluations on taxonomy reconstruction, hypernym discovery and reading comprehension tasks, we show that the knowledge about hierarchy is moderately but not systematically transferable across tasks. △ Less

Submitted 15 December, 2023; originally announced December 2023.

Comments: Accepted at ECIR 2024

Magnetospheric Venus Space Explorers (MVSE) Mission: A Proposal for Understanding the Dynamics of Induced Magnetospheres

Authors: Roland Albers, Henrik Andrews, Gabriele Boccacci, Vasco D. C Pires, Sunny Laddha, Ville Lundén, Nadim Maraqten, João Matias, Eva Krämer, Leonard Schulz, Ines Terraza Palanca, Daniel Teubenbacher, Claire Baskevitch, Francesca Covella, Luca Cressa, Juan Garrido Moreno, Jana Gillmayr, Joshua Hollowood, Kilian Huber, Viktoria Kutnohorsky, Sofia Lennerstrand, Adel Malatinszky, Davide Manzini, Manuel Maurer, Daiana Maria Alessandra Nidele , et al. (5 additional authors not shown)

Abstract: Induced magnetospheres form around planetary bodies with atmospheres through the interaction of the solar wind with their ionosphere. Induced magnetospheres are highly dependent on the solar wind conditions and have only been studied with single spacecraft missions in the past. This gap in knowledge could be addressed by a multi-spacecraft plasma mission, optimized for studying global spatial and… ▽ More Induced magnetospheres form around planetary bodies with atmospheres through the interaction of the solar wind with their ionosphere. Induced magnetospheres are highly dependent on the solar wind conditions and have only been studied with single spacecraft missions in the past. This gap in knowledge could be addressed by a multi-spacecraft plasma mission, optimized for studying global spatial and temporal variations in the magnetospheric system around Venus, which hosts the most prominent example of an induced magnetosphere in our solar system. The MVSE mission comprises four satellites, of which three are identical scientific spacecraft, carrying the same suite of instruments probing different regions of the induced magnetosphere and the solar wind simultaneously. The fourth spacecraft is the transfer vehicle which acts as a relay satellite for communications at Venus. In this way, changes in the solar wind conditions and extreme solar events can be observed, and their effects can be quantified as they propagate through the Venusian induced magnetosphere. Additionally, energy transfer in the Venusian induced magnetosphere can be investigated. The scientific payload includes instrumentation to measure the magnetic field, electric field, and ion-electron velocity distributions. This study presents the scientific motivation for the mission as well as requirements and the resulting mission design. Concretely, a mission timeline along with a complete spacecraft design, including mass, power, communication, propulsion and thermal budgets are given. This mission was initially conceived at the Alpbach Summer School 2022 and refined during a week-long study at ESAs Concurrent Design Facility in Redu, Belgium △ Less

Submitted 12 December, 2023; originally announced December 2023.

Comments: 23 pages, 5 figures, Submitted to Acta Astronautica

Yes but.. Can ChatGPT Identify Entities in Historical Documents?

Authors: Carlos-Emiliano González-Gallardo, Emanuela Boros, Nancy Girdhar, Ahmed Hamdi, Jose G. Moreno, Antoine Doucet

Abstract: Large language models (LLMs) have been leveraged for several years now, obtaining state-of-the-art performance in recognizing entities from modern documents. For the last few months, the conversational agent ChatGPT has "prompted" a lot of interest in the scientific community and public due to its capacity of generating plausible-sounding answers. In this paper, we explore this ability by probing… ▽ More Large language models (LLMs) have been leveraged for several years now, obtaining state-of-the-art performance in recognizing entities from modern documents. For the last few months, the conversational agent ChatGPT has "prompted" a lot of interest in the scientific community and public due to its capacity of generating plausible-sounding answers. In this paper, we explore this ability by probing it in the named entity recognition and classification (NERC) task in primary sources (e.g., historical newspapers and classical commentaries) in a zero-shot manner and by comparing it with state-of-the-art LM-based systems. Our findings indicate several shortcomings in identifying entities in historical text that range from the consistency of entity annotation guidelines, entity complexity, and code-switching, to the specificity of prompting. Moreover, as expected, the inaccessibility of historical archives to the public (and thus on the Internet) also impacts its performance. △ Less

Submitted 30 March, 2023; originally announced March 2023.

Comments: 5 pages, accepted to JCDL2023

Using contextual sentence analysis models to recognize ESG concepts

Authors: Elvys Linhares Pontes, Mohamed Benjannet, Jose G. Moreno, Antoine Doucet

Abstract: This paper summarizes the joint participation of the Trading Central Labs and the L3i laboratory of the University of La Rochelle on both sub-tasks of the Shared Task FinSim-4 evaluation campaign. The first sub-task aims to enrich the 'Fortia ESG taxonomy' with new lexicon entries while the second one aims to classify sentences to either 'sustainable' or 'unsustainable' with respect to ESG (Enviro… ▽ More This paper summarizes the joint participation of the Trading Central Labs and the L3i laboratory of the University of La Rochelle on both sub-tasks of the Shared Task FinSim-4 evaluation campaign. The first sub-task aims to enrich the 'Fortia ESG taxonomy' with new lexicon entries while the second one aims to classify sentences to either 'sustainable' or 'unsustainable' with respect to ESG (Environment, Social and Governance) related factors. For the first sub-task, we proposed a model based on pre-trained Sentence-BERT models to project sentences and concepts in a common space in order to better represent ESG concepts. The official task results show that our system yields a significant performance improvement compared to the baseline and outperforms all other submissions on the first sub-task. For the second sub-task, we combine the RoBERTa model with a feed-forward multi-layer perceptron in order to extract the context of sentences and classify them. Our model achieved high accuracy scores (over 92%) and was ranked among the top 5 systems. △ Less

Submitted 4 July, 2022; originally announced July 2022.

Exceptionally high saturation magnetisation in Eu-doped magnetite stabilised by spin-orbit interaction

Authors: M. Hussein N. Assadi, José Julio Gutiérrez Moreno, Dorian A. H. Hanaor, Hiroshi Katayama-Yoshida

Abstract: The significance of the spin-orbit interaction is very well known in compounds containing heavier elements such as the rare-earth Eu ion. Here, through density functional calculations, we investigated the effect of the spin-orbit interaction on the magnetic ground state of Eu doped magnetite ($\mathrm{Fe_3O_4:Eu_{Fe}}$). By examining all possible spin alignments between Eu and magnetite's Fe, we d… ▽ More The significance of the spin-orbit interaction is very well known in compounds containing heavier elements such as the rare-earth Eu ion. Here, through density functional calculations, we investigated the effect of the spin-orbit interaction on the magnetic ground state of Eu doped magnetite ($\mathrm{Fe_3O_4:Eu_{Fe}}$). By examining all possible spin alignments between Eu and magnetite's Fe, we demonstrate that Eu, which is most stable when doped at the tetrahedral site, adapts a spin almost opposite the substituted Fe. Consequently, because of smaller spin cancellation between the cations on the tetrahedral site ($\mathrm{Fe_{Tet}}$ and $\mathrm{Eu_{Tet}}$) and the cations on the octahedral sites ($\mathrm{Fe_{Oct}}$), $\mathrm{Fe_3O_4:Eu_{Fe}}$ exhibits a maximum saturation magnetisation of 9.451 $μ_B/$f.u. which is significantly larger than that of undoped magnetite (calculated to be 3.929 $μ_B/$f.u.). We further show that this large magnetisation persists through additional electron do**. However, additional hole do**, which may unintentionally occur in Fe deficient magnetite, can reduce the magnetisation to values smaller than that of the undoped magnetite. The results presented here can aid in designing highly efficient magnetically recoverable catalysts for which both magnetite and rare earth dopants are common materials. △ Less

Submitted 3 February, 2022; originally announced February 2022.

Comments: 12 pages, 9 figures, 3 tables

Journal ref: Phys. Chem. Chem. Phys., 2021, 23, 20129-20137

Named entity recognition architecture combining contextual and global features

Authors: Tran Thi Hong Hanh, Antoine Doucet, Nicolas Sidere, Jose G. Moreno, Senja Pollak

Abstract: Named entity recognition (NER) is an information extraction technique that aims to locate and classify named entities (e.g., organizations, locations,...) within a document into predefined categories. Correctly identifying these phrases plays a significant role in simplifying information access. However, it remains a difficult task because named entities (NEs) have multiple forms and they are cont… ▽ More Named entity recognition (NER) is an information extraction technique that aims to locate and classify named entities (e.g., organizations, locations,...) within a document into predefined categories. Correctly identifying these phrases plays a significant role in simplifying information access. However, it remains a difficult task because named entities (NEs) have multiple forms and they are context-dependent. While the context can be represented by contextual features, global relations are often misrepresented by those models. In this paper, we propose the combination of contextual features from XLNet and global features from Graph Convolution Network (GCN) to enhance NER performance. Experiments over a widely-used dataset, CoNLL 2003, show the benefits of our strategy, with results competitive with the state of the art (SOTA). △ Less

Submitted 15 December, 2021; originally announced December 2021.

Lifetime effects and satellites in the photoelectron spectrum of tungsten metal

Authors: Curran Kalha, Laura E. Ratcliff, Julio J. Gutiérrez Moreno, Stephan Mohr, Mervi Mantsinen, Nathalie K. Fernando, Pardeep K. Thakur, Tien-Lin Lee, Hsiang-Han Tseng, Tim S. Nunney, Juhan M. Kahk, Johannes Lischner, Anna Regoutz

Abstract: Tungsten is an important and versatile transition metal and has a firm place at the heart of many technologies. A popular experimental technique for the characterisation of tungsten and tungsten-based compounds is X-ray photoelectron spectroscopy (XPS), which enables the assessment of chemical states and electronic structure through the collection of core level and valence band spectra. However, i… ▽ More Tungsten is an important and versatile transition metal and has a firm place at the heart of many technologies. A popular experimental technique for the characterisation of tungsten and tungsten-based compounds is X-ray photoelectron spectroscopy (XPS), which enables the assessment of chemical states and electronic structure through the collection of core level and valence band spectra. However, in the case of metallic tungsten, open questions remain regarding the origin, nature, and position of satellite features that are prominent in the photoelectron spectrum. These satellites are a fingerprint of the electronic structure of the material and have not been thoroughly investigated, at times leading to their misinterpretation. The present work combines high-resolution soft and hard X-ray photoelectron spectroscopy (SXPS and HAXPES) with reflection electron energy loss spectroscopy (REELS) and a multi-tiered ab-initio theoretical approach, including density functional theory (DFT) and many-body perturbation theory (G0W0 and GW+C), to disentangle the complex set of experimentally observed satellite features attributed to the generation of plasmons and interband transitions. This combined experiment-theory strategy is able to uncover previously undocumented satellite features, improving our understanding of their direct relationship to tungsten's electronic structure. Furthermore, it lays the groundwork for future studies into tungsten based mixed-metal systems and holds promise for the re-assessment of the photoelectron spectra of other transition and post-transition metals, where similar questions regarding satellite features remain. △ Less

Submitted 10 September, 2021; originally announced September 2021.

Event Detection as Question Answering with Entity Information

Authors: Emanuela Boros, Jose G. Moreno, Antoine Doucet

Abstract: In this paper, we propose a recent and under-researched paradigm for the task of event detection (ED) by casting it as a question-answering (QA) problem with the possibility of multiple answers and the support of entities. The extraction of event triggers is, thus, transformed into the task of identifying answer spans from a context, while also focusing on the surrounding entities. The architectur… ▽ More In this paper, we propose a recent and under-researched paradigm for the task of event detection (ED) by casting it as a question-answering (QA) problem with the possibility of multiple answers and the support of entities. The extraction of event triggers is, thus, transformed into the task of identifying answer spans from a context, while also focusing on the surrounding entities. The architecture is based on a pre-trained and fine-tuned language model, where the input context is augmented with entities marked at different levels, their positions, their types, and, finally, the argument roles. Experiments on the ACE~2005 corpus demonstrate that the proposed paradigm is a viable solution for the ED task and it significantly outperforms the state-of-the-art models. Moreover, we prove that our methods are also able to extract unseen event types. △ Less

Submitted 14 April, 2021; originally announced April 2021.

High-Performance Thermoelectric Oxides Based on Spinel Structure

Authors: M. Hussein N. Assadi, J. Julio Gutiérrez Moreno, Marco Fronzi

Abstract: High-performance thermoelectric oxides could offer a great energy solution for integrated and embedded applications in sensing and electronics industries. Oxides, however, often suffer from low Seebeck coefficient when compared with other classes of thermoelectric materials. In search of high-performance thermoelectric oxides, we present a comprehensive density functional investigation, based on G… ▽ More High-performance thermoelectric oxides could offer a great energy solution for integrated and embedded applications in sensing and electronics industries. Oxides, however, often suffer from low Seebeck coefficient when compared with other classes of thermoelectric materials. In search of high-performance thermoelectric oxides, we present a comprehensive density functional investigation, based on GGA$+U$ formalism, surveying the 3d and 4d transition-metal-containing ferrites of the spinel structure. Consequently, we predict MnFe$_2$O$_4$ and RhFe$_2$O$_4$ have Seebeck coefficients of $\sim \pm 600$ $μ$V K$^{-1}$ at near room temperature, achieved by light hole and electron do**. Furthermore, CrFe$_2$O$_4$ and MoFe$_2$O$_4$ have even higher ambient Seebeck coefficients at $\sim \pm 700$ $μ$V K$^{-1}$. In the latter compounds, the Seebeck coefficient is approximately a flat function of temperature up to $\sim 700$ K, offering a tremendous operational convenience. Additionally, MoFe$_2$O$_4$ doped with $10^{19}$ holes/cm$^3$ has a calculated thermoelectric power factor of $689.81$ $μ$W K$^{-2}$ m$^{-1}$ at $300$ K, and $455.67$ $μ$W K$^{-2}$ m$^{-1}$ at $600$ K. The thermoelectric properties predicted here can bring these thermoelectric oxides to applications at lower temperatures traditionally fulfilled by more toxic and otherwise burdensome materials. △ Less

Submitted 22 August, 2020; originally announced August 2020.

Comments: 11 pages, 6 figures, 1 table, 2 supplementary files

Journal ref: ACS Appl. Energy Mater. 2020, 3(6) 5666

Everything You Always Wanted to Know About TREC RTS* (*But Were Afraid to Ask)

Authors: Gilles Hubert, Jose G. Moreno, Karen Pinel-Sauvagnat, Yoann Pitarch

Abstract: The TREC Real-Time Summarization (RTS) track provides a framework for evaluating systems monitoring the Twitter stream and pushing tweets to users according to given profiles. It includes metrics, files, settings and hypothesis provided by the organizers. In this work, we perform a thorough analysis of each component of the framework used in 2016 and 2017 and found some limitations for the Scenari… ▽ More The TREC Real-Time Summarization (RTS) track provides a framework for evaluating systems monitoring the Twitter stream and pushing tweets to users according to given profiles. It includes metrics, files, settings and hypothesis provided by the organizers. In this work, we perform a thorough analysis of each component of the framework used in 2016 and 2017 and found some limitations for the Scenario A of this track. Our main findings point out the weakness of the metrics and give clear recommendations to fairly reuse the collection. △ Less

Submitted 13 December, 2017; originally announced December 2017.