Showing 1–6 of 6 results for author: Dastgheib, D

Khayyam Challenge (PersianMMLU): Is Your LLM Truly Wise to The Persian Language?

Authors: Omid Ghahroodi, Marzia Nouri, Mohammad Vali Sanian, Alireza Sahebi, Doratossadat Dastgheib, Ehsaneddin Asgari, Mahdieh Soleymani Baghshah, Mohammad Hossein Rohban

Abstract: Evaluating Large Language Models (LLMs) is challenging due to their generative nature, necessitating precise evaluation methodologies. Additionally, non-English LLM evaluation lags behind English, resulting in the absence or weakness of LLMs for many languages. In response to this necessity, we introduce Khayyam Challenge (also known as PersianMMLU), a meticulously curated collection comprising 20… ▽ More Evaluating Large Language Models (LLMs) is challenging due to their generative nature, necessitating precise evaluation methodologies. Additionally, non-English LLM evaluation lags behind English, resulting in the absence or weakness of LLMs for many languages. In response to this necessity, we introduce Khayyam Challenge (also known as PersianMMLU), a meticulously curated collection comprising 20,192 four-choice questions sourced from 38 diverse tasks extracted from Persian examinations, spanning a wide spectrum of subjects, complexities, and ages. The primary objective of the Khayyam Challenge is to facilitate the rigorous evaluation of LLMs that support the Persian language. Distinctive features of the Khayyam Challenge are (i) its comprehensive coverage of various topics, including literary comprehension, mathematics, sciences, logic, intelligence testing, etc., aimed at assessing different facets of LLMs such as language comprehension, reasoning, and information retrieval across various educational stages, from lower primary school to upper secondary school (ii) its inclusion of rich metadata such as human response rates, difficulty levels, and descriptive answers (iii) its utilization of new data to avoid data contamination issues prevalent in existing frameworks (iv) its use of original, non-translated data tailored for Persian speakers, ensuring the framework is free from translation challenges and errors while encompassing cultural nuances (v) its inherent scalability for future data updates and evaluations without requiring special human effort. Previous works lacked an evaluation framework that combined all of these features into a single comprehensive benchmark. Furthermore, we evaluate a wide range of existing LLMs that support the Persian language, with statistical analyses and interpretations of their outputs. △ Less

Submitted 9 April, 2024; originally announced April 2024.

A proof for completeness of Łukasiewicz logic

Authors: Doratossadat Dastgheib, Hadi Farahani

Abstract: In this paper we give a new proof for the completeness of infinite valued propositional Łukasiewicz logic introduced by Łukasiewicz and Tarski in 1930. Our approach employs a Hilbert-style proof that relies on the concept of maximal consistent extensions, and unlike classical logic, in this context, the maximal extensions are not required to include all formulas or their negations. To illustrate t… ▽ More In this paper we give a new proof for the completeness of infinite valued propositional Łukasiewicz logic introduced by Łukasiewicz and Tarski in 1930. Our approach employs a Hilbert-style proof that relies on the concept of maximal consistent extensions, and unlike classical logic, in this context, the maximal extensions are not required to include all formulas or their negations. To illustrate this point, we provide examples of such formulas. △ Less

Submitted 28 August, 2023; v1 submitted 3 August, 2023; originally announced August 2023.

Doxastic Lukasiewicz Logic with Public Announcement

Authors: Doratossadat Dastgheib, Hadi Farahani

Abstract: In this paper, we propose a doxastic extension $BL^+$ of Lukasiewicz logic which is sound and complete relative to the introduced corresponding semantics. Also, we equip our doxastic Lukasiewicz logic $BL^+$ with public announcement and propose the logic $DL$. As an application, we model a fuzzy version of muddy children puzzle with public announcement using $DL$. Finally, we define a translation… ▽ More In this paper, we propose a doxastic extension $BL^+$ of Lukasiewicz logic which is sound and complete relative to the introduced corresponding semantics. Also, we equip our doxastic Lukasiewicz logic $BL^+$ with public announcement and propose the logic $DL$. As an application, we model a fuzzy version of muddy children puzzle with public announcement using $DL$. Finally, we define a translation between $DL$ and $BL^+$, and prove the soundness and completeness theorems for D L △ Less

Submitted 17 April, 2023; originally announced April 2023.

The Touché23-ValueEval Dataset for Identifying Human Values behind Arguments

Authors: Nailia Mirzakhmedova, Johannes Kiesel, Milad Alshomary, Maximilian Heinrich, Nicolas Handke, Xiaoni Cai, Barriere Valentin, Doratossadat Dastgheib, Omid Ghahroodi, Mohammad Ali Sadraei, Ehsaneddin Asgari, Lea Kawaletz, Henning Wachsmuth, Benno Stein

Abstract: We present the Touché23-ValueEval Dataset for Identifying Human Values behind Arguments. To investigate approaches for the automated detection of human values behind arguments, we collected 9324 arguments from 6 diverse sources, covering religious texts, political discussions, free-text arguments, newspaper editorials, and online democracy platforms. Each argument was annotated by 3 crowdworkers f… ▽ More We present the Touché23-ValueEval Dataset for Identifying Human Values behind Arguments. To investigate approaches for the automated detection of human values behind arguments, we collected 9324 arguments from 6 diverse sources, covering religious texts, political discussions, free-text arguments, newspaper editorials, and online democracy platforms. Each argument was annotated by 3 crowdworkers for 54 values. The Touché23-ValueEval dataset extends the Webis-ArgValues-22. In comparison to the previous dataset, the effectiveness of a 1-Baseline decreases, but that of an out-of-the-box BERT model increases. Therefore, though the classification difficulty increased as per the label distribution, the larger dataset allows for training better models. △ Less

Submitted 31 January, 2023; originally announced January 2023.

Some Doxastic Łukasiewicz Logic

Authors: Doratossadat Dastgheib, Hadi Farahani

Abstract: We propose a doxastic Łukasiewicz logic \textbf{BŁ} that is sound and complete with respect to the class of Kripke-based models in which atomic propositions and accessibility relations are both infinitely valued in the standard MV-algebra [0,1]. We also introduce some extensions of \textbf{BŁ} corresponding to axioms \textbf{D}, \textbf{4}, and \textbf{T} of classical epistemic logic. Furthermore,… ▽ More We propose a doxastic Łukasiewicz logic \textbf{BŁ} that is sound and complete with respect to the class of Kripke-based models in which atomic propositions and accessibility relations are both infinitely valued in the standard MV-algebra [0,1]. We also introduce some extensions of \textbf{BŁ} corresponding to axioms \textbf{D}, \textbf{4}, and \textbf{T} of classical epistemic logic. Furthermore, completeness of these extensions are established corresponding to the appropriate classes of models. △ Less

Submitted 11 December, 2023; v1 submitted 4 November, 2021; originally announced November 2021.

Some Epistemic Extensions of Gödel Fuzzy Logic

Authors: D. Dastgheib, H. Farahani, A. H. Sharafi

Abstract: In this paper we prove soundness and completeness of some epistemic extensions of Gödel fuzzy logic, based on Kripke models in which both propositions at each state and accessibility relations take values in [0,1]. We adopt belief as our epistemic operator, acknowledging that the axiom of Truth may not always hold. We propose the axiomatic system $\textbf{K}_\textbf{F}$ serves as a fuzzy variant… ▽ More In this paper we prove soundness and completeness of some epistemic extensions of Gödel fuzzy logic, based on Kripke models in which both propositions at each state and accessibility relations take values in [0,1]. We adopt belief as our epistemic operator, acknowledging that the axiom of Truth may not always hold. We propose the axiomatic system $\textbf{K}_\textbf{F}$ serves as a fuzzy variant of classical epistemic logic $\textbf{K}$, then by considering consistent belief and adding positive introspection and Truth axioms to the axioms of $\textbf{K}_\textbf{F}$, the axiomatic extensions $\textbf{B}_\textbf{F}$ and $\textbf{T}_\textbf{F}$ are established. To demonstrate the completeness of $\textbf{K}_\textbf{F}$, we present a novel approach that characterizes formulas semantically equivalent to $\perp$ and we introduce a grammar describing formulas with this property. Furthermore, it is revealed that validity in $\textbf{K}_\textbf{F}$ cannot be reduced to the class of all models having crisp accessibility relations, and also $\textbf{K}_\textbf{F}$ does not enjoy the finite model property. These properties distinguish $\textbf{K}_\textbf{F}$ as a new modal extension of Gödel fuzzy logic which differs from the standard Gödel Modal Logics $\mathcal{G}_\Box$ and $\mathcal{G}_\Diamond$ proposed by Caicedo and O. Rodriguez. △ Less

Submitted 2 March, 2024; v1 submitted 12 May, 2016; originally announced May 2016.