-
InCA: Rethinking In-Car Conversational System Assessment Leveraging Large Language Models
Authors:
Ken E. Friedl,
Abbas Goher Khan,
Soumya Ranjan Sahoo,
Md Rashad Al Hasan Rony,
Jana Germies,
Christian Süß
Abstract:
The assessment of advanced generative large language models (LLMs) poses a significant challenge, given their heightened complexity in recent developments. Furthermore, evaluating the performance of LLM-based applications in various industries, as indicated by Key Performance Indicators (KPIs), is a complex undertaking. This task necessitates a profound understanding of industry use cases and the…
▽ More
The assessment of advanced generative large language models (LLMs) poses a significant challenge, given their heightened complexity in recent developments. Furthermore, evaluating the performance of LLM-based applications in various industries, as indicated by Key Performance Indicators (KPIs), is a complex undertaking. This task necessitates a profound understanding of industry use cases and the anticipated system behavior. Within the context of the automotive industry, existing evaluation metrics prove inadequate for assessing in-car conversational question answering (ConvQA) systems. The unique demands of these systems, where answers may relate to driver or car safety and are confined within the car domain, highlight the limitations of current metrics. To address these challenges, this paper introduces a set of KPIs tailored for evaluating the performance of in-car ConvQA systems, along with datasets specifically designed for these KPIs. A preliminary and comprehensive empirical evaluation substantiates the efficacy of our proposed approach. Furthermore, we investigate the impact of employing varied personas in prompts and found that it enhances the model's capacity to simulate diverse viewpoints in assessments, mirroring how individuals with different backgrounds perceive a topic.
△ Less
Submitted 15 November, 2023; v1 submitted 13 November, 2023;
originally announced November 2023.
-
Bi-Asymptotic $c$-Expansivity
Authors:
Rohit Nageshwar,
Abdul Gaffar Khan,
Tarun Das
Abstract:
In this paper, we define bi-asymptotically $c$-expansive maps on metric spaces and study its relationship with other variants of expansivity such as bi-asymptotically expansive maps and $N$-expansive maps. We also provide an example to establish that expansive homeomorphisms need not be bi-asymptotically expansive. Finally we prove a spectral decomposition theorem for bi-asymptotically $c$-expansi…
▽ More
In this paper, we define bi-asymptotically $c$-expansive maps on metric spaces and study its relationship with other variants of expansivity such as bi-asymptotically expansive maps and $N$-expansive maps. We also provide an example to establish that expansive homeomorphisms need not be bi-asymptotically expansive. Finally we prove a spectral decomposition theorem for bi-asymptotically $c$-expansive continuous surjective maps with the shadowing property on compact metric spaces.
△ Less
Submitted 14 November, 2022;
originally announced November 2022.
-
Topologically stable and $β$-persistent points of group actions
Authors:
Abdul Gaffar Khan,
Tarun Das
Abstract:
In this paper, we introduce topologically stable points, $β$-persistent points, $β$-persistent property, $β$-persistent measures and almost $β$-persistent measures for first countable Hausdorff group actions of compact metric spaces. We prove that the set of all $β$-persistent points is measurable and it is closed if the action is equicontinuous. We also prove that the set of all $β$-persistent me…
▽ More
In this paper, we introduce topologically stable points, $β$-persistent points, $β$-persistent property, $β$-persistent measures and almost $β$-persistent measures for first countable Hausdorff group actions of compact metric spaces. We prove that the set of all $β$-persistent points is measurable and it is closed if the action is equicontinuous. We also prove that the set of all $β$-persistent measures is a convex set and every almost $β$-persistent measure is a $β$-persistent measure. Finally, we prove that every equicontinuous pointwise topologically stable first countable Hausdorff group action of a compact metric space is $β$-persistent. In particular, every equicontinuous pointwise topologically stable flow is $β$-persistent.
△ Less
Submitted 22 November, 2021; v1 submitted 13 August, 2020;
originally announced August 2020.
-
Stability Theorems in Pointwise Dynamics
Authors:
Abdul Gaffar Khan,
Tarun Das
Abstract:
We introduce minimally expansive and GH-stable points for homeomorphisms on metric spaces and $μ$-uniformly expansive, $μ$-shadowable and strong $μ$-topologically stable points for Borel measures (with respect to a homeomorphism on a metric space). We prove that: (i) minimally expansive shadowable point of a homeomorphism on a compact metric space is topologically stable and GH-stable. (ii) $μ$-un…
▽ More
We introduce minimally expansive and GH-stable points for homeomorphisms on metric spaces and $μ$-uniformly expansive, $μ$-shadowable and strong $μ$-topologically stable points for Borel measures (with respect to a homeomorphism on a metric space). We prove that: (i) minimally expansive shadowable point of a homeomorphism on a compact metric space is topologically stable and GH-stable. (ii) $μ$-uniformly expansive $μ$-shadowable point for a Borel measure $μ$ (with respect to a homeomorphism on a compact metric space) is strong $μ$-topologically stable.
△ Less
Submitted 26 August, 2019;
originally announced August 2019.
-
Pointwise dynamics under Orbital Convergence
Authors:
Abdul Gaffar Khan,
Pramod Kumar Das,
Tarun Das
Abstract:
We obtain sufficient conditions under which the limit of a sequence of functions exhibits a particular dynamical behaviour at a point like expansivity, shadowing, mixing, sensitivity and transitivity. We provide examples to show that the set of all expansive, positively expansive and sensitive points are neither open nor closed in general. We also observe that the set of all transitive and mixing…
▽ More
We obtain sufficient conditions under which the limit of a sequence of functions exhibits a particular dynamical behaviour at a point like expansivity, shadowing, mixing, sensitivity and transitivity. We provide examples to show that the set of all expansive, positively expansive and sensitive points are neither open nor closed in general. We also observe that the set of all transitive and mixing points are closed but not open in general. We give examples to show that properties like expansivity, sensitivity, shadowing, transitivity and mixing at a point need not be preserved under uniform convergence and properties like topological stability and $α$-persistence at a point need not be preserved under pointwise convergence.
△ Less
Submitted 12 July, 2019;
originally announced July 2019.
-
Topologically Stable Equicontinuous Non-Autonomous Systems
Authors:
Abdul Gaffar Khan,
Pramod Kumar Das,
Tarun Das
Abstract:
We find sufficient conditions for commutative non-autonomous systems on certain metric spaces to be topologically stable. In particular, we prove that (i) Every mean equicontinuous, mean expansive system with strong average shadowing property is topologically stable. (ii) Every equicontinuous, recurrently expansive system with almost shadowing property is topologically stable. (iii) Every equicont…
▽ More
We find sufficient conditions for commutative non-autonomous systems on certain metric spaces to be topologically stable. In particular, we prove that (i) Every mean equicontinuous, mean expansive system with strong average shadowing property is topologically stable. (ii) Every equicontinuous, recurrently expansive system with almost shadowing property is topologically stable. (iii) Every equicontinuous, expansive system with shadowing property is topologically stable.
△ Less
Submitted 24 June, 2019;
originally announced June 2019.
-
Specification for Group Actions on Uniform Spaces
Authors:
Abdul Gaffar Khan,
Pramod Kumar Das,
Tarun Das
Abstract:
We extend specification and periodic specification to finitely generated group actions on uniform spaces using a concept of specification point. We prove that certain group actions having two distinct specification points have positive entropy. We further prove that if a group containing an infinite order element acts on an infinite Hausdorff uniform space and the action possesses periodic specifi…
▽ More
We extend specification and periodic specification to finitely generated group actions on uniform spaces using a concept of specification point. We prove that certain group actions having two distinct specification points have positive entropy. We further prove that if a group containing an infinite order element acts on an infinite Hausdorff uniform space and the action possesses periodic specification, then it is Devaney chaotic.
△ Less
Submitted 24 June, 2019;
originally announced June 2019.
-
Measure Expansivity and Specification for Pointwise Dynamics
Authors:
Pramod Das,
Abdul Gaffar Khan,
Tarun Das
Abstract:
We introduce pointwise measure expansivity for bi-measurable maps. We show through examples that this notion is weaker than measure expansivity. In spite of this fact, we show that many results for measure expansive systems hold true for pointwise systems as well. Then, we study the concept of mixing, specification and chaos at a point in the phase space of a continuous map. We show that mixing at…
▽ More
We introduce pointwise measure expansivity for bi-measurable maps. We show through examples that this notion is weaker than measure expansivity. In spite of this fact, we show that many results for measure expansive systems hold true for pointwise systems as well. Then, we study the concept of mixing, specification and chaos at a point in the phase space of a continuous map. We show that mixing at a shadowable point is not sufficient for it to be a specification point, but mixing of the map force a shadowable point to be a specification point. We prove that periodic specification points are Devaney chaotic point. Finally, we show that existence of two distinct specification points is sufficient for a map to have positive Bowen entropy.
△ Less
Submitted 29 September, 2018; v1 submitted 7 June, 2018;
originally announced June 2018.
-
GH-stability and Spectral Decomposition for Group Actions
Authors:
Abdul Gaffar Khan,
Pramod Das,
Tarun Das
Abstract:
We study expansivity and the shadowing property for finitely generated group actions on metric spaces. We consider the projecting and lifting problems for actions having these properties. We prove that every expansive action with the shadowing property is strongly $GH$-stable (Gromov-Hausdorff stable). Finally, we introduce sequential shadowing property for finitely generated group actions on metr…
▽ More
We study expansivity and the shadowing property for finitely generated group actions on metric spaces. We consider the projecting and lifting problems for actions having these properties. We prove that every expansive action with the shadowing property is strongly $GH$-stable (Gromov-Hausdorff stable). Finally, we introduce sequential shadowing property for finitely generated group actions on metric spaces and show that such shadowing is strong enough to imply the spectral decomposition property.
△ Less
Submitted 4 December, 2018; v1 submitted 16 April, 2018;
originally announced April 2018.
