-
ILLUMINER: Instruction-tuned Large Language Models as Few-shot Intent Classifier and Slot Filler
Authors:
Paramita Mirza,
Viju Sudhi,
Soumya Ranjan Sahoo,
Sinchana Ramakanth Bhat
Abstract:
State-of-the-art intent classification (IC) and slot filling (SF) methods often rely on data-intensive deep learning models, limiting their practicality for industry applications. Large language models on the other hand, particularly instruction-tuned models (Instruct-LLMs), exhibit remarkable zero-shot performance across various natural language tasks. This study evaluates Instruct-LLMs on popula…
▽ More
State-of-the-art intent classification (IC) and slot filling (SF) methods often rely on data-intensive deep learning models, limiting their practicality for industry applications. Large language models on the other hand, particularly instruction-tuned models (Instruct-LLMs), exhibit remarkable zero-shot performance across various natural language tasks. This study evaluates Instruct-LLMs on popular benchmark datasets for IC and SF, emphasizing their capacity to learn from fewer examples. We introduce ILLUMINER, an approach framing IC and SF as language generation tasks for Instruct-LLMs, with a more efficient SF-prompting method compared to prior work. A comprehensive comparison with multiple baselines shows that our approach, using the FLAN-T5 11B model, outperforms the state-of-the-art joint IC+SF method and in-context learning with GPT3.5 (175B), particularly in slot filling by 11.1--32.2 percentage points. Additionally, our in-depth ablation study demonstrates that parameter-efficient fine-tuning requires less than 6% of training data to yield comparable performance with traditional full-weight fine-tuning.
△ Less
Submitted 26 March, 2024;
originally announced March 2024.
-
InCA: Rethinking In-Car Conversational System Assessment Leveraging Large Language Models
Authors:
Ken E. Friedl,
Abbas Goher Khan,
Soumya Ranjan Sahoo,
Md Rashad Al Hasan Rony,
Jana Germies,
Christian Süß
Abstract:
The assessment of advanced generative large language models (LLMs) poses a significant challenge, given their heightened complexity in recent developments. Furthermore, evaluating the performance of LLM-based applications in various industries, as indicated by Key Performance Indicators (KPIs), is a complex undertaking. This task necessitates a profound understanding of industry use cases and the…
▽ More
The assessment of advanced generative large language models (LLMs) poses a significant challenge, given their heightened complexity in recent developments. Furthermore, evaluating the performance of LLM-based applications in various industries, as indicated by Key Performance Indicators (KPIs), is a complex undertaking. This task necessitates a profound understanding of industry use cases and the anticipated system behavior. Within the context of the automotive industry, existing evaluation metrics prove inadequate for assessing in-car conversational question answering (ConvQA) systems. The unique demands of these systems, where answers may relate to driver or car safety and are confined within the car domain, highlight the limitations of current metrics. To address these challenges, this paper introduces a set of KPIs tailored for evaluating the performance of in-car ConvQA systems, along with datasets specifically designed for these KPIs. A preliminary and comprehensive empirical evaluation substantiates the efficacy of our proposed approach. Furthermore, we investigate the impact of employing varied personas in prompts and found that it enhances the model's capacity to simulate diverse viewpoints in assessments, mirroring how individuals with different backgrounds perceive a topic.
△ Less
Submitted 15 November, 2023; v1 submitted 13 November, 2023;
originally announced November 2023.
-
Data Augmentation using Feature Generation for Volumetric Medical Images
Authors:
Khushboo Mehra,
Hassan Soliman,
Soumya Ranjan Sahoo
Abstract:
Medical image classification is one of the most critical problems in the image recognition area. One of the major challenges in this field is the scarcity of labelled training data. Additionally, there is often class imbalance in datasets as some cases are very rare to happen. As a result, accuracy in classification task is normally low. Deep Learning models, in particular, show promising results…
▽ More
Medical image classification is one of the most critical problems in the image recognition area. One of the major challenges in this field is the scarcity of labelled training data. Additionally, there is often class imbalance in datasets as some cases are very rare to happen. As a result, accuracy in classification task is normally low. Deep Learning models, in particular, show promising results on image segmentation and classification problems, but they require very large datasets for training. Therefore, there is a need to generate more of synthetic samples from the same distribution. Previous work has shown that feature generation is more efficient and leads to better performance than corresponding image generation. We apply this idea in the Medical Imaging domain. We use transfer learning to train a segmentation model for the small dataset for which gold-standard class annotations are available. We extracted the learnt features and use them to generate synthetic features conditioned on class labels, using Auxiliary Classifier GAN (ACGAN). We test the quality of the generated features in a downstream classification task for brain tumors according to their severity level. Experimental results show a promising result regarding the validity of these generated features and their overall contribution to balancing the data and improving the classification class-wise accuracy.
△ Less
Submitted 28 September, 2022;
originally announced September 2022.
-
Impact of sensor placement in soil water estimation: A real-case study
Authors:
Erfan Orouskhani,
Soumya R. Sahoo,
Bernard T. Agyeman,
Song Bo,
**feng Liu
Abstract:
One of the essential elements in implementing a closed-loop irrigation system is soil moisture estimation based on a limited number of available sensors. One associated problem is the determination of the optimal locations to install the sensors such that good soil moisture estimation can be obtained. In our previous work, the modal degree of observability was employed to address the problem of op…
▽ More
One of the essential elements in implementing a closed-loop irrigation system is soil moisture estimation based on a limited number of available sensors. One associated problem is the determination of the optimal locations to install the sensors such that good soil moisture estimation can be obtained. In our previous work, the modal degree of observability was employed to address the problem of optimal sensor placement for soil moisture estimation of agro-hydrological systems. It was demonstrated that the optimally placed sensors can improve the soil moisture estimation performance. However, it is unclear whether the optimal sensor placement can significantly improve the soil moisture estimation performance in actual applications. In this work, we investigate the impact of sensor placement in soil moisture estimation for an actual agricultural field in Lethbridge, Alberta, Canada. In an experiment on the studied field, 42 soil moisture sensors were installed at different depths to collect the soil moisture measurements for one growing season. A three-dimensional agro-hydrological model with heterogeneous soil parameters of the studied field is developed. The modal degree of observability is applied to the three-dimensional system to determine the optimal sensor locations. The extended Kalman filter (EKF) is chosen as the data assimilation tool to estimate the soil moisture content of the studied field. Soil moisture estimation results for different scenarios are obtained and analyzed to investigate the effects of sensor placement on the performance of soil moisture estimation in the actual applications.
△ Less
Submitted 12 March, 2022;
originally announced March 2022.
-
Eternal-Thing 2.0: Analog-Trojan Resilient Ripple-Less Solar Energy Harvesting System for Sustainable IoT in Smart Cities and Smart Villages
Authors:
Saswat K. Ram,
Sauvagya R. Sahoo,
Banee B. Das,
Kamalakanta Mahapatra,
Saraju P. Mohanty
Abstract:
Recently, harvesting natural energy is gaining more attention than other conventional approaches for sustainable Internet-of-Things (IoT). System on chip (SoC) power requirement for the IoT and generating higher voltages on-chip is a massive challenge for on-chip peripherals and systems. Many sensors are employed in smart cities and smart villages in decision-making, whose power requirement is an…
▽ More
Recently, harvesting natural energy is gaining more attention than other conventional approaches for sustainable Internet-of-Things (IoT). System on chip (SoC) power requirement for the IoT and generating higher voltages on-chip is a massive challenge for on-chip peripherals and systems. Many sensors are employed in smart cities and smart villages in decision-making, whose power requirement is an issue, and it must be uninterrupted. Previously, we presented Security-by-Design (SbD) principle to bring energy dissipation and cybersecurity together through our "Eternal-Thing". In this paper, an on-chip reliable energy harvesting system (EHS) is designed for IoT end node devices which is called "Eternal-Thing 2.0". The management section monitors the process load and also the recharging of the battery or super-capacitor. An efficient maximum power point tracking (MPPT) algorithm is used to avoid quiescent power consumption. The reliability of the proposed EHS is improved by using an aging tolerant ring oscillator. The proposed EHS is intended and simulated in CMOS 90nm technology. The output voltage is within the vary of 3-3.55V with an input of 1-1.5V. The EHS consumes 22 micro Watt of power, that satisfies the ultra-low-power necessities of IoT sensible nodes.
△ Less
Submitted 9 March, 2021;
originally announced March 2021.
