-
Using Synchronic Definitions and Semantic Relations to Classify Semantic Change Types
Authors:
Pierluigi Cassotti,
Stefano De Pascale,
Nina Tahmasebi
Abstract:
There is abundant evidence of the fact that the way words change their meaning can be classified in different types of change, highlighting the relationship between the old and new meanings (among which generalization, specialization and co-hyponymy transfer). In this paper, we present a way of detecting these types of change by constructing a model that leverages information both from synchronic…
▽ More
There is abundant evidence of the fact that the way words change their meaning can be classified in different types of change, highlighting the relationship between the old and new meanings (among which generalization, specialization and co-hyponymy transfer). In this paper, we present a way of detecting these types of change by constructing a model that leverages information both from synchronic lexical relations and definitions of word meanings. Specifically, we use synset definitions and hierarchy information from WordNet and test it on a digitized version of Blank's (1997) dataset of semantic change types. Finally, we show how the sense relationships can improve models for both approximation of human judgments of semantic relatedness as well as binary Lexical Semantic Change Detection.
△ Less
Submitted 11 June, 2024; v1 submitted 5 June, 2024;
originally announced June 2024.
-
Analyzing Semantic Change through Lexical Replacements
Authors:
Francesco Periti,
Pierluigi Cassotti,
Haim Dubossarsky,
Nina Tahmasebi
Abstract:
Modern language models are capable of contextualizing words based on their surrounding context. However, this capability is often compromised due to semantic change that leads to words being used in new, unexpected contexts not encountered during pre-training. In this paper, we model \textit{semantic change} by studying the effect of unexpected contexts introduced by \textit{lexical replacements}.…
▽ More
Modern language models are capable of contextualizing words based on their surrounding context. However, this capability is often compromised due to semantic change that leads to words being used in new, unexpected contexts not encountered during pre-training. In this paper, we model \textit{semantic change} by studying the effect of unexpected contexts introduced by \textit{lexical replacements}. We propose a \textit{replacement schema} where a target word is substituted with lexical replacements of varying relatedness, thus simulating different kinds of semantic change. Furthermore, we leverage the replacement schema as a basis for a novel \textit{interpretable} model for semantic change. We are also the first to evaluate the use of LLaMa for semantic change detection.
△ Less
Submitted 29 April, 2024;
originally announced April 2024.
-
A Systematic Comparison of Contextualized Word Embeddings for Lexical Semantic Change
Authors:
Francesco Periti,
Nina Tahmasebi
Abstract:
Contextualized embeddings are the preferred tool for modeling Lexical Semantic Change (LSC). Current evaluations typically focus on a specific task known as Graded Change Detection (GCD). However, performance comparison across work are often misleading due to their reliance on diverse settings. In this paper, we evaluate state-of-the-art models and approaches for GCD under equal conditions. We fur…
▽ More
Contextualized embeddings are the preferred tool for modeling Lexical Semantic Change (LSC). Current evaluations typically focus on a specific task known as Graded Change Detection (GCD). However, performance comparison across work are often misleading due to their reliance on diverse settings. In this paper, we evaluate state-of-the-art models and approaches for GCD under equal conditions. We further break the LSC problem into Word-in-Context (WiC) and Word Sense Induction (WSI) tasks, and compare models across these different levels. Our evaluation is performed across different languages on eight available benchmarks for LSC, and shows that (i) APD outperforms other approaches for GCD; (ii) XL-LEXEME outperforms other contextualized models for WiC, WSI, and GCD, while being comparable to GPT-4; (iii) there is a clear need for improving the modeling of word meanings, as well as focus on how, when, and why these meanings change, rather than solely focusing on the extent of semantic change.
△ Less
Submitted 8 March, 2024; v1 submitted 19 February, 2024;
originally announced February 2024.
-
(Chat)GPT v BERT: Dawn of Justice for Semantic Change Detection
Authors:
Francesco Periti,
Haim Dubossarsky,
Nina Tahmasebi
Abstract:
In the universe of Natural Language Processing, Transformer-based language models like BERT and (Chat)GPT have emerged as lexical superheroes with great power to solve open research problems. In this paper, we specifically focus on the temporal problem of semantic change, and evaluate their ability to solve two diachronic extensions of the Word-in-Context (WiC) task: TempoWiC and HistoWiC. In part…
▽ More
In the universe of Natural Language Processing, Transformer-based language models like BERT and (Chat)GPT have emerged as lexical superheroes with great power to solve open research problems. In this paper, we specifically focus on the temporal problem of semantic change, and evaluate their ability to solve two diachronic extensions of the Word-in-Context (WiC) task: TempoWiC and HistoWiC. In particular, we investigate the potential of a novel, off-the-shelf technology like ChatGPT (and GPT) 3.5 compared to BERT, which represents a family of models that currently stand as the state-of-the-art for modeling semantic change. Our experiments represent the first attempt to assess the use of (Chat)GPT for studying semantic change. Our results indicate that ChatGPT performs significantly worse than the foundational GPT version. Furthermore, our results demonstrate that (Chat)GPT achieves slightly lower performance than BERT in detecting long-term changes but performs significantly worse in detecting short-term changes.
△ Less
Submitted 29 April, 2024; v1 submitted 25 January, 2024;
originally announced January 2024.
-
The DURel Annotation Tool: Human and Computational Measurement of Semantic Proximity, Sense Clusters and Semantic Change
Authors:
Dominik Schlechtweg,
Shafqat Mumtaz Virk,
Pauline Sander,
Emma Sköldberg,
Lukas Theuer Linke,
Tuo Zhang,
Nina Tahmasebi,
Jonas Kuhn,
Sabine Schulte im Walde
Abstract:
We present the DURel tool that implements the annotation of semantic proximity between uses of words into an online, open source interface. The tool supports standardized human annotation as well as computational annotation, building on recent advances with Word-in-Context models. Annotator judgments are clustered with automatic graph clustering techniques and visualized for analysis. This allows…
▽ More
We present the DURel tool that implements the annotation of semantic proximity between uses of words into an online, open source interface. The tool supports standardized human annotation as well as computational annotation, building on recent advances with Word-in-Context models. Annotator judgments are clustered with automatic graph clustering techniques and visualized for analysis. This allows to measure word senses with simple and intuitive micro-task judgments between use pairs, requiring minimal preparation efforts. The tool offers additional functionalities to compare the agreement between annotators to guarantee the inter-subjectivity of the obtained judgments and to calculate summary statistics giving insights into sense frequency distributions, semantic variation or changes of senses over time.
△ Less
Submitted 5 February, 2024; v1 submitted 21 November, 2023;
originally announced November 2023.
-
Computational modeling of semantic change
Authors:
Nina Tahmasebi,
Haim Dubossarsky
Abstract:
In this chapter we provide an overview of computational modeling for semantic change using large and semi-large textual corpora. We aim to provide a key for the interpretation of relevant methods and evaluation techniques, and also provide insights into important aspects of the computational study of semantic change. We discuss the pros and cons of different classes of models with respect to the p…
▽ More
In this chapter we provide an overview of computational modeling for semantic change using large and semi-large textual corpora. We aim to provide a key for the interpretation of relevant methods and evaluation techniques, and also provide insights into important aspects of the computational study of semantic change. We discuss the pros and cons of different classes of models with respect to the properties of the data from which one wishes to model semantic change, and which avenues are available to evaluate the results.
△ Less
Submitted 13 April, 2023;
originally announced April 2023.
-
DWUG: A large Resource of Diachronic Word Usage Graphs in Four Languages
Authors:
Dominik Schlechtweg,
Nina Tahmasebi,
Simon Hengchen,
Haim Dubossarsky,
Barbara McGillivray
Abstract:
Word meaning is notoriously difficult to capture, both synchronically and diachronically. In this paper, we describe the creation of the largest resource of graded contextualized, diachronic word meaning annotation in four different languages, based on 100,000 human semantic proximity judgments. We thoroughly describe the multi-round incremental annotation process, the choice for a clustering algo…
▽ More
Word meaning is notoriously difficult to capture, both synchronically and diachronically. In this paper, we describe the creation of the largest resource of graded contextualized, diachronic word meaning annotation in four different languages, based on 100,000 human semantic proximity judgments. We thoroughly describe the multi-round incremental annotation process, the choice for a clustering algorithm to group usages into senses, and possible - diachronic and synchronic - uses for this dataset.
△ Less
Submitted 8 July, 2024; v1 submitted 17 April, 2021;
originally announced April 2021.
-
SuperSim: a test set for word similarity and relatedness in Swedish
Authors:
Simon Hengchen,
Nina Tahmasebi
Abstract:
Language models are notoriously difficult to evaluate. We release SuperSim, a large-scale similarity and relatedness test set for Swedish built with expert human judgments. The test set is composed of 1,360 word-pairs independently judged for both relatedness and similarity by five annotators. We evaluate three different models (Word2Vec, fastText, and GloVe) trained on two separate Swedish datase…
▽ More
Language models are notoriously difficult to evaluate. We release SuperSim, a large-scale similarity and relatedness test set for Swedish built with expert human judgments. The test set is composed of 1,360 word-pairs independently judged for both relatedness and similarity by five annotators. We evaluate three different models (Word2Vec, fastText, and GloVe) trained on two separate Swedish datasets, namely the Swedish Gigaword corpus and a Swedish Wikipedia dump, to provide a baseline for future comparison. We release the fully annotated test set, code, baseline models, and data.
△ Less
Submitted 12 April, 2021;
originally announced April 2021.
-
What Is the Generalized Representation of Dirac Equation in Two Dimensions?
Authors:
H. Moaiery,
A. Chenani,
A. Hakimifard,
N. Tahmasebi
Abstract:
In this work, the general form of $2\times2$ Dirac matrices for 2+1 dimension is found. In order to find this general representation, all relations among the elements of the matrices and matrices themselves are found,and the generalized Lorentz transform matrix is also found under the effect of the general representation of Dirac matrices. As we know, the well known equation of Dirac,…
▽ More
In this work, the general form of $2\times2$ Dirac matrices for 2+1 dimension is found. In order to find this general representation, all relations among the elements of the matrices and matrices themselves are found,and the generalized Lorentz transform matrix is also found under the effect of the general representation of Dirac matrices. As we know, the well known equation of Dirac, $ \left( iγ^μ\partial_μ-m\right) Ψ=0 $, is consist of matrices of even dimension known as the general representation of Dirac matrices or Dirac matrices. Our motivation for this study was lack of the general representation of these matrices despite the fact that more than nine decades have been passed since the discovery of this well known equation. Everyone has used a specific representation of this equation according to their need; such as the standard representation known as Dirac-Pauli Representation, Weyl Representation or Majorana representation. In this work, the general form which these matrices can have is found once for all.
△ Less
Submitted 1 April, 2021;
originally announced April 2021.
-
Challenges for Computational Lexical Semantic Change
Authors:
Simon Hengchen,
Nina Tahmasebi,
Dominik Schlechtweg,
Haim Dubossarsky
Abstract:
The computational study of lexical semantic change (LSC) has taken off in the past few years and we are seeing increasing interest in the field, from both computational sciences and linguistics. Most of the research so far has focused on methods for modelling and detecting semantic change using large diachronic textual data, with the majority of the approaches employing neural embeddings. While me…
▽ More
The computational study of lexical semantic change (LSC) has taken off in the past few years and we are seeing increasing interest in the field, from both computational sciences and linguistics. Most of the research so far has focused on methods for modelling and detecting semantic change using large diachronic textual data, with the majority of the approaches employing neural embeddings. While methods that offer easy modelling of diachronic text are one of the main reasons for the spiking interest in LSC, neural models leave many aspects of the problem unsolved. The field has several open and complex challenges. In this chapter, we aim to describe the most important of these challenges and outline future directions.
△ Less
Submitted 19 January, 2021;
originally announced January 2021.
-
SemEval-2020 Task 1: Unsupervised Lexical Semantic Change Detection
Authors:
Dominik Schlechtweg,
Barbara McGillivray,
Simon Hengchen,
Haim Dubossarsky,
Nina Tahmasebi
Abstract:
Lexical Semantic Change detection, i.e., the task of identifying words that change meaning over time, is a very active research area, with applications in NLP, lexicography, and linguistics. Evaluation is currently the most pressing problem in Lexical Semantic Change detection, as no gold standards are available to the community, which hinders progress. We present the results of the first shared t…
▽ More
Lexical Semantic Change detection, i.e., the task of identifying words that change meaning over time, is a very active research area, with applications in NLP, lexicography, and linguistics. Evaluation is currently the most pressing problem in Lexical Semantic Change detection, as no gold standards are available to the community, which hinders progress. We present the results of the first shared task that addresses this gap by providing researchers with an evaluation framework and manually annotated, high-quality datasets for English, German, Latin, and Swedish. 33 teams submitted 186 systems, which were evaluated on two subtasks.
△ Less
Submitted 28 August, 2020; v1 submitted 22 July, 2020;
originally announced July 2020.
-
Time-Out: Temporal Referencing for Robust Modeling of Lexical Semantic Change
Authors:
Haim Dubossarsky,
Simon Hengchen,
Nina Tahmasebi,
Dominik Schlechtweg
Abstract:
State-of-the-art models of lexical semantic change detection suffer from noise stemming from vector space alignment. We have empirically tested the Temporal Referencing method for lexical semantic change and show that, by avoiding alignment, it is less affected by this noise. We show that, trained on a diachronic corpus, the skip-gram with negative sampling architecture with temporal referencing o…
▽ More
State-of-the-art models of lexical semantic change detection suffer from noise stemming from vector space alignment. We have empirically tested the Temporal Referencing method for lexical semantic change and show that, by avoiding alignment, it is less affected by this noise. We show that, trained on a diachronic corpus, the skip-gram with negative sampling architecture with temporal referencing outperforms alignment models on a synthetic task as well as a manual testset. We introduce a principled way to simulate lexical semantic change and systematically control for possible biases.
△ Less
Submitted 4 June, 2019;
originally announced June 2019.
-
Survey of Computational Approaches to Lexical Semantic Change
Authors:
Nina Tahmasebi,
Lars Borin,
Adam Jatowt
Abstract:
Our languages are in constant flux driven by external factors such as cultural, societal and technological changes, as well as by only partially understood internal motivations. Words acquire new meanings and lose old senses, new words are coined or borrowed from other languages and obsolete words slide into obscurity. Understanding the characteristics of shifts in the meaning and in the use of wo…
▽ More
Our languages are in constant flux driven by external factors such as cultural, societal and technological changes, as well as by only partially understood internal motivations. Words acquire new meanings and lose old senses, new words are coined or borrowed from other languages and obsolete words slide into obscurity. Understanding the characteristics of shifts in the meaning and in the use of words is useful for those who work with the content of historical texts, the interested general public, but also in and of itself. The findings from automatic lexical semantic change detection, and the models of diachronic conceptual change are currently being incorporated in approaches for measuring document across-time similarity, information retrieval from long-term document archives, the design of OCR algorithms, and so on. In recent years we have seen a surge in interest in the academic community in computational methods and tools supporting inquiry into diachronic conceptual change and lexical replacement. This article is an extract of a survey of recent computational techniques to tackle lexical semantic change currently under review. In this article we focus on diachronic conceptual change as an extension of semantic change.
△ Less
Submitted 13 March, 2019; v1 submitted 15 November, 2018;
originally announced November 2018.
-
Named Entity Evolution Recognition on the Blogosphere
Authors:
Helge Holzmann,
Nina Tahmasebi,
Thomas Risse
Abstract:
Advancements in technology and culture lead to changes in our language. These changes create a gap between the language known by users and the language stored in digital archives. It affects user's possibility to firstly find content and secondly interpret that content. In previous work we introduced our approach for Named Entity Evolution Recognition~(NEER) in newspaper collections. Lately, incre…
▽ More
Advancements in technology and culture lead to changes in our language. These changes create a gap between the language known by users and the language stored in digital archives. It affects user's possibility to firstly find content and secondly interpret that content. In previous work we introduced our approach for Named Entity Evolution Recognition~(NEER) in newspaper collections. Lately, increasing efforts in Web preservation lead to increased availability of Web archives covering longer time spans. However, language on the Web is more dynamic than in traditional media and many of the basic assumptions from the newspaper domain do not hold for Web data. In this paper we discuss the limitations of existing methodology for NEER. We approach these by adapting an existing NEER method to work on noisy data like the Web and the Blogosphere in particular. We develop novel filters that reduce the noise and make use of Semantic Web resources to obtain more information about terms. Our evaluation shows the potentials of the proposed approach.
△ Less
Submitted 3 February, 2017;
originally announced February 2017.
