-
Exploring Ordinality in Text Classification: A Comparative Study of Explicit and Implicit Techniques
Authors:
Siva Rajesh Kasa,
Aniket Goel,
Karan Gupta,
Sumegh Roychowdhury,
Anish Bhanushali,
Nikhil Pattisapu,
Prasanna Srinivasa Murthy
Abstract:
Ordinal Classification (OC) is a widely encountered challenge in Natural Language Processing (NLP), with applications in various domains such as sentiment analysis, rating prediction, and more. Previous approaches to tackle OC have primarily focused on modifying existing or creating novel loss functions that \textbf{explicitly} account for the ordinal nature of labels. However, with the advent of…
▽ More
Ordinal Classification (OC) is a widely encountered challenge in Natural Language Processing (NLP), with applications in various domains such as sentiment analysis, rating prediction, and more. Previous approaches to tackle OC have primarily focused on modifying existing or creating novel loss functions that \textbf{explicitly} account for the ordinal nature of labels. However, with the advent of Pretrained Language Models (PLMs), it became possible to tackle ordinality through the \textbf{implicit} semantics of the labels as well. This paper provides a comprehensive theoretical and empirical examination of both these approaches. Furthermore, we also offer strategic recommendations regarding the most effective approach to adopt based on specific settings.
△ Less
Submitted 20 May, 2024;
originally announced May 2024.
-
How Robust are LLMs to In-Context Majority Label Bias?
Authors:
Karan Gupta,
Sumegh Roychowdhury,
Siva Rajesh Kasa,
Santhosh Kumar Kasa,
Anish Bhanushali,
Nikhil Pattisapu,
Prasanna Srinivasa Murthy
Abstract:
In the In-Context Learning (ICL) setup, various forms of label biases can manifest. One such manifestation is majority label bias, which arises when the distribution of labeled examples in the in-context samples is skewed towards one or more specific classes making Large Language Models (LLMs) more prone to predict those labels. Such discrepancies can arise from various factors, including logistic…
▽ More
In the In-Context Learning (ICL) setup, various forms of label biases can manifest. One such manifestation is majority label bias, which arises when the distribution of labeled examples in the in-context samples is skewed towards one or more specific classes making Large Language Models (LLMs) more prone to predict those labels. Such discrepancies can arise from various factors, including logistical constraints, inherent biases in data collection methods, limited access to diverse data sources, etc. which are unavoidable in a real-world industry setup. In this work, we study the robustness of in-context learning in LLMs to shifts that occur due to majority label bias within the purview of text classification tasks. Prior works have shown that in-context learning with LLMs is susceptible to such biases. In our study, we go one level deeper and show that the robustness boundary varies widely for different models and tasks, with certain LLMs being highly robust (~90%) to majority label bias. Additionally, our findings also highlight the impact of model size and the richness of instructional prompts contributing towards model robustness. We restrict our study to only publicly available open-source models to ensure transparency and reproducibility.
△ Less
Submitted 27 December, 2023;
originally announced December 2023.
-
Tackling Concept Shift in Text Classification using Entailment-style Modeling
Authors:
Sumegh Roychowdhury,
Karan Gupta,
Siva Rajesh Kasa,
Prasanna Srinivasa Murthy,
Alok Chandra
Abstract:
Pre-trained language models (PLMs) have seen tremendous success in text classification (TC) problems in the context of Natural Language Processing (NLP). In many real-world text classification tasks, the class definitions being learned do not remain constant but rather change with time - this is known as Concept Shift. Most techniques for handling concept shift rely on retraining the old classifie…
▽ More
Pre-trained language models (PLMs) have seen tremendous success in text classification (TC) problems in the context of Natural Language Processing (NLP). In many real-world text classification tasks, the class definitions being learned do not remain constant but rather change with time - this is known as Concept Shift. Most techniques for handling concept shift rely on retraining the old classifiers with the newly labelled data. However, given the amount of training data required to fine-tune large DL models for the new concepts, the associated labelling costs can be prohibitively expensive and time consuming. In this work, we propose a reformulation, converting vanilla classification into an entailment-style problem that requires significantly less data to re-train the text classifier to adapt to new concepts. We demonstrate the effectiveness of our proposed method on both real world & synthetic datasets achieving absolute F1 gains upto 7% and 40% respectively in few-shot settings. Further, upon deployment, our solution also helped save 75% of labeling costs overall.
△ Less
Submitted 6 November, 2023;
originally announced November 2023.
-
Infrared absorption study of charge ordered $La{}_{0.5}Ca{}_{0.5-x}Sr_{x}MnO_{3}$ $(0.1\leq x\leq0.5$
Authors:
Indu Dhiman,
A. Das,
K. R. Priolkar,
P. S. R. Murthy
Abstract:
Infrared absorption study has been carried out on a series of half doped manganites, $La_{0.5}Ca_{0.5-x}Sr_{x}MnO_{3}$ $(0.1\leq x\leq0.5)$, with varying magnetic ground state. The charge ordering transition observed in samples with {\normalsize $x\leq0.3$ is accompanied by a mode at $\sim525cm^{-1}$ in addition to the stretching mode at $615cm^{-1}$ and bending mode at $400cm^{-1}$. Phonon harden…
▽ More
Infrared absorption study has been carried out on a series of half doped manganites, $La_{0.5}Ca_{0.5-x}Sr_{x}MnO_{3}$ $(0.1\leq x\leq0.5)$, with varying magnetic ground state. The charge ordering transition observed in samples with {\normalsize $x\leq0.3$ is accompanied by a mode at $\sim525cm^{-1}$ in addition to the stretching mode at $615cm^{-1}$ and bending mode at $400cm^{-1}$. Phonon hardening is found to occur below the CE - type antiferromagnetic ordering temperature. The value of the insulating gap decreases on do** with Sr from $727cm^{-1}$ to $615cm^{-1}.$}
△ Less
Submitted 7 January, 2011;
originally announced January 2011.
-
Structure, Transport and Magnetic properties in La$_{2x}$Sr$_{2-2x}$Co$_{2x}$Ru$_{2-2x}$O$_{6}$
Authors:
P. S. R. Murthy,
K. R. Priolkar,
P. A. Bhobe,
A. Das,
P. R. Sarode,
A. K. Nigam
Abstract:
The perovskite solid solutions of the type La$_{2x}$Sr$_{2-2x}$Co$_{2x}$Ru$_{2-2x}$O$_{6}$ with 0.25 $\leq$ x $ \leq $ 0.75 have been investigated for their structural, magnetic and transport properties. All the compounds crystallize in double perovskite structure. The magnetization measurements indicate a complex magnetic ground state with strong competition between ferromagnetic and antiferromag…
▽ More
The perovskite solid solutions of the type La$_{2x}$Sr$_{2-2x}$Co$_{2x}$Ru$_{2-2x}$O$_{6}$ with 0.25 $\leq$ x $ \leq $ 0.75 have been investigated for their structural, magnetic and transport properties. All the compounds crystallize in double perovskite structure. The magnetization measurements indicate a complex magnetic ground state with strong competition between ferromagnetic and antiferromagnetic interactions. Resistivity of the compounds is in confirmation with hop** conduction behaviour though differences are noted especially for $x$ = 0.4 and 0.6. Most importantly, low field (50Oe) magnetization measurements display negative magnetization during the zero field cooled cycle. X-ray photoelectron spectroscopy measurements indicate presence of Co$^{2+}$/Co$^{3+}$ and Ru$^{4+}$/Ru$^{5+}$ redox couples in all compositions except $x$ = 0.5. Presence of magnetic ions like Ru$^{4+}$ and Co$^{3+}$ gives rise to additional ferromagnetic (Ru-rich) and antiferromagnetic sublattices and also explains the observed negative magnetization.
△ Less
Submitted 11 November, 2010;
originally announced November 2010.
-
Effect of B-site Dopants on Magnetic and Transport Properties of LaSrCoRuO$_6$
Authors:
P. S. Ramu Murthy,
K. R. Priolkar,
P. A. Bhobe,
A. Das,
P. R. Sarode,
A. K. Nigam
Abstract:
Effect of Co, Ru and Cu substitution at B and B' sites on the magnetic and transport properties of LaSrCoRuO$_6$ have been investigated. All the doped compositions crystallize in the monoclinic structure in the space group $P2_1/n$ indicating a double perovskite structure. While the magnetization and conductivity increase in Co and Ru doped compounds, antiferromagnetism is seen to strengthen in th…
▽ More
Effect of Co, Ru and Cu substitution at B and B' sites on the magnetic and transport properties of LaSrCoRuO$_6$ have been investigated. All the doped compositions crystallize in the monoclinic structure in the space group $P2_1/n$ indicating a double perovskite structure. While the magnetization and conductivity increase in Co and Ru doped compounds, antiferromagnetism is seen to strengthen in the Cu doped samples. These results are explained on the basis of a competition between linear Co-O-Ru-O-Co and perpendicular Co-O-O-Co antiferromagnetic interactions and due to formation of Ru-O-Ru ferromagnetic networks.
△ Less
Submitted 11 November, 2010;
originally announced November 2010.
-
Disorder Induced Negative Magnetization in LaSrCoRuO6
Authors:
P. S. R. Murthy,
K. R. Priolkar,
P. A. Bhobe,
A. Das,
P. R. Sarode,
A. K. Nigam
Abstract:
This paper reports effect of thermally induced disorder on the magnetic properties of LaSrCoRuO6 double perovskite. While the ordered sample is antiferromagnetic, the disordered sample exhibits negative values of magnetization measured in low applied fields. Isothermal magnetization on this sample shows hysteresis due to presence of ferromagnetic interactions. Based on neutron diffraction and X-ra…
▽ More
This paper reports effect of thermally induced disorder on the magnetic properties of LaSrCoRuO6 double perovskite. While the ordered sample is antiferromagnetic, the disordered sample exhibits negative values of magnetization measured in low applied fields. Isothermal magnetization on this sample shows hysteresis due to presence of ferromagnetic interactions. Based on neutron diffraction and X-ray Absorption Fine Structure (XAFS) studies, these results have been interpreted to be due disorder in site occupancy of Co and Ru leading to octahedral distortions and formation of Ru-O-Ru ferromagnetic linkages. Below 150K these ferromagnetic Ru spins polarize the Co spins in a direction opposite to that of the applied field resulting in observed negative magnetization.
△ Less
Submitted 7 August, 2010;
originally announced August 2010.
