Showing 1–50 of 81 results for author: Sahay, S

Introducing v0.5 of the AI Safety Benchmark from MLCommons

Authors: Bertie Vidgen, Adarsh Agrawal, Ahmed M. Ahmed, Victor Akinwande, Namir Al-Nuaimi, Najla Alfaraj, Elie Alhajjar, Lora Aroyo, Trupti Bavalatti, Max Bartolo, Borhane Blili-Hamelin, Kurt Bollacker, Rishi Bomassani, Marisa Ferrara Boston, Siméon Campos, Kal Chakra, Canyu Chen, Cody Coleman, Zacharie Delpierre Coudert, Leon Derczynski, Debojyoti Dutta, Ian Eisenberg, James Ezick, Heather Frase, Brian Fuller , et al. (75 additional authors not shown)

Abstract: This paper introduces v0.5 of the AI Safety Benchmark, which has been created by the MLCommons AI Safety Working Group. The AI Safety Benchmark has been designed to assess the safety risks of AI systems that use chat-tuned language models. We introduce a principled approach to specifying and constructing the benchmark, which for v0.5 covers only a single use case (an adult chatting to a general-pu… ▽ More This paper introduces v0.5 of the AI Safety Benchmark, which has been created by the MLCommons AI Safety Working Group. The AI Safety Benchmark has been designed to assess the safety risks of AI systems that use chat-tuned language models. We introduce a principled approach to specifying and constructing the benchmark, which for v0.5 covers only a single use case (an adult chatting to a general-purpose assistant in English), and a limited set of personas (i.e., typical users, malicious users, and vulnerable users). We created a new taxonomy of 13 hazard categories, of which 7 have tests in the v0.5 benchmark. We plan to release version 1.0 of the AI Safety Benchmark by the end of 2024. The v1.0 benchmark will provide meaningful insights into the safety of AI systems. However, the v0.5 benchmark should not be used to assess the safety of AI systems. We have sought to fully document the limitations, flaws, and challenges of v0.5. This release of v0.5 of the AI Safety Benchmark includes (1) a principled approach to specifying and constructing the benchmark, which comprises use cases, types of systems under test (SUTs), language and context, personas, tests, and test items; (2) a taxonomy of 13 hazard categories with definitions and subcategories; (3) tests for seven of the hazard categories, each comprising a unique set of test items, i.e., prompts. There are 43,090 test items in total, which we created with templates; (4) a grading system for AI systems against the benchmark; (5) an openly available platform, and downloadable tool, called ModelBench that can be used to evaluate the safety of AI systems on the benchmark; (6) an example evaluation report which benchmarks the performance of over a dozen openly available chat-tuned language models; (7) a test specification for the benchmark. △ Less

Submitted 13 May, 2024; v1 submitted 18 April, 2024; originally announced April 2024.

Zero-shot Conversational Summarization Evaluations with small Large Language Models

Authors: Ramesh Manuvinakurike, Saurav Sahay, Sangeeta Manepalli, Lama Nachman

Abstract: Large Language Models (LLMs) exhibit powerful summarization abilities. However, their capabilities on conversational summarization remains under explored. In this work we evaluate LLMs (approx. 10 billion parameters) on conversational summarization and showcase their performance on various prompts. We show that the summaries generated by models depend on the instructions and the performance of LLM… ▽ More Large Language Models (LLMs) exhibit powerful summarization abilities. However, their capabilities on conversational summarization remains under explored. In this work we evaluate LLMs (approx. 10 billion parameters) on conversational summarization and showcase their performance on various prompts. We show that the summaries generated by models depend on the instructions and the performance of LLMs vary with different instructions sometimes resulting steep drop in ROUGE scores if prompts are not selected carefully. We also evaluate the models with human evaluations and discuss the limitations of the models on conversational summarization △ Less

Submitted 29 November, 2023; originally announced November 2023.

Comments: Accepted at RoF0Mo workshop at Neurips 2023

Design Theory for Societal Digital Transformation: The Case of Digital Global Health

Authors: Jorn Braa, Sundeep Sahay, Eric Monteiro

Abstract: With societal challenges, including but not limited to human development, equity, social justice, and climate change, societal-level digital transformation (SDT) is of imminent relevance and theoretical interest. While building on local-level efforts, societal-level transformation is a nonlinear extension of the local level. Unfortunately, academic discourse on digital transformation has largely l… ▽ More With societal challenges, including but not limited to human development, equity, social justice, and climate change, societal-level digital transformation (SDT) is of imminent relevance and theoretical interest. While building on local-level efforts, societal-level transformation is a nonlinear extension of the local level. Unfortunately, academic discourse on digital transformation has largely left SDT unaccounted for. Drawing on more than 25 years of intensive, interventionist research engagement with the digital transformation of public healthcare information management and delivery in more than 80 countries in the Global South, we contribute to theorizing SDT in the form of a design theory consisting of six interconnected design principles. These design principles articulate the interplay and tensions of accommodating over time increased diversity and flexibility in digital solutions, while simultaneously connecting local, national, and regional/ global efforts. △ Less

Submitted 15 November, 2023; originally announced November 2023.

Journal ref: Journal of the AIS, 24(6), 2023

Learning from Red Teaming: Gender Bias Provocation and Mitigation in Large Language Models

Authors: Hsuan Su, Cheng-Chu Cheng, Hua Farn, Shachi H Kumar, Saurav Sahay, Shang-Tse Chen, Hung-yi Lee

Abstract: Recently, researchers have made considerable improvements in dialogue systems with the progress of large language models (LLMs) such as ChatGPT and GPT-4. These LLM-based chatbots encode the potential biases while retaining disparities that can harm humans during interactions. The traditional biases investigation methods often rely on human-written test cases. However, these test cases are usually… ▽ More Recently, researchers have made considerable improvements in dialogue systems with the progress of large language models (LLMs) such as ChatGPT and GPT-4. These LLM-based chatbots encode the potential biases while retaining disparities that can harm humans during interactions. The traditional biases investigation methods often rely on human-written test cases. However, these test cases are usually expensive and limited. In this work, we propose a first-of-its-kind method that automatically generates test cases to detect LLMs' potential gender bias. We apply our method to three well-known LLMs and find that the generated test cases effectively identify the presence of biases. To address the biases identified, we propose a mitigation strategy that uses the generated test cases as demonstrations for in-context learning to circumvent the need for parameter fine-tuning. The experimental results show that LLMs generate fairer responses with the proposed approach. △ Less

Submitted 17 October, 2023; originally announced October 2023.

Inspecting Spoken Language Understanding from Kids for Basic Math Learning at Home

Authors: Eda Okur, Roddy Fuentes Alba, Saurav Sahay, Lama Nachman

Abstract: Enriching the quality of early childhood education with interactive math learning at home systems, empowered by recent advances in conversational AI technologies, is slowly becoming a reality. With this motivation, we implement a multimodal dialogue system to support play-based learning experiences at home, guiding kids to master basic math concepts. This work explores Spoken Language Understandin… ▽ More Enriching the quality of early childhood education with interactive math learning at home systems, empowered by recent advances in conversational AI technologies, is slowly becoming a reality. With this motivation, we implement a multimodal dialogue system to support play-based learning experiences at home, guiding kids to master basic math concepts. This work explores Spoken Language Understanding (SLU) pipeline within a task-oriented dialogue system developed for Kid Space, with cascading Automatic Speech Recognition (ASR) and Natural Language Understanding (NLU) components evaluated on our home deployment data with kids going through gamified math learning activities. We validate the advantages of a multi-task architecture for NLU and experiment with a diverse set of pretrained language representations for Intent Recognition and Entity Extraction tasks in the math learning domain. To recognize kids' speech in realistic home environments, we investigate several ASR systems, including the commercial Google Cloud and the latest open-source Whisper solutions with varying model sizes. We evaluate the SLU pipeline by testing our best-performing NLU models on noisy ASR output to inspect the challenges of understanding children for math learning in authentic homes. △ Less

Submitted 1 June, 2023; originally announced June 2023.

Comments: Proceedings of the 18th Workshop on Innovative Use of NLP for Building Educational Applications (BEA) at ACL 2023

Sample Efficient Multimodal Semantic Augmentation for Incremental Summarization

Authors: Sumanta Bhattacharyya, Ramesh Manuvinakurike, Sahisnu Mazumder, Saurav Sahay

Abstract: In this work, we develop a prompting approach for incremental summarization of task videos. We develop a sample-efficient few-shot approach for extracting semantic concepts as an intermediate step. We leverage an existing model for extracting the concepts from the images and extend it to videos and introduce a clustering and querying approach for sample efficiency, motivated by the recent advances… ▽ More In this work, we develop a prompting approach for incremental summarization of task videos. We develop a sample-efficient few-shot approach for extracting semantic concepts as an intermediate step. We leverage an existing model for extracting the concepts from the images and extend it to videos and introduce a clustering and querying approach for sample efficiency, motivated by the recent advances in perceiver-based architectures. Our work provides further evidence that an approach with richer input context with relevant entities and actions from the videos and using these as prompts could enhance the summaries generated by the model. We show the results on a relevant dataset and discuss possible directions for the work. △ Less

Submitted 7 March, 2023; originally announced March 2023.

Position Matters! Empirical Study of Order Effect in Knowledge-grounded Dialogue

Authors: Hsuan Su, Shachi H Kumar, Sahisnu Mazumder, Wenda Chen, Ramesh Manuvinakurike, Eda Okur, Saurav Sahay, Lama Nachman, Shang-Tse Chen, Hung-yi Lee

Abstract: With the power of large pretrained language models, various research works have integrated knowledge into dialogue systems. The traditional techniques treat knowledge as part of the input sequence for the dialogue system, prepending a set of knowledge statements in front of dialogue history. However, such a mechanism forces knowledge sets to be concatenated in an ordered manner, making models impl… ▽ More With the power of large pretrained language models, various research works have integrated knowledge into dialogue systems. The traditional techniques treat knowledge as part of the input sequence for the dialogue system, prepending a set of knowledge statements in front of dialogue history. However, such a mechanism forces knowledge sets to be concatenated in an ordered manner, making models implicitly pay imbalanced attention to the sets during training. In this paper, we first investigate how the order of the knowledge set can influence autoregressive dialogue systems' responses. We conduct experiments on two commonly used dialogue datasets with two types of transformer-based models and find that models view the input knowledge unequally. To this end, we propose a simple and novel technique to alleviate the order effect by modifying the position embeddings of knowledge input in these models. With the proposed position embedding method, the experimental results show that each knowledge statement is uniformly considered to generate responses. △ Less

Submitted 12 February, 2023; originally announced February 2023.

Systematic Analysis for Pretrained Language Model Priming for Parameter-Efficient Fine-tuning

Authors: Shih-Cheng Huang, Shih-Heng Wang, Min-Han Shih, Saurav Sahay, Hung-yi Lee

Abstract: Parameter-efficient (PE) methods (like Prompts or Adapters) for adapting pre-trained language models (PLM) to downstream tasks have been popular recently. However, hindrances still prevent these methods from reaching their full potential. For example, two significant challenges are few-shot adaptation and cross-task generalization. To tackle these issues, we propose a general PE priming framework… ▽ More Parameter-efficient (PE) methods (like Prompts or Adapters) for adapting pre-trained language models (PLM) to downstream tasks have been popular recently. However, hindrances still prevent these methods from reaching their full potential. For example, two significant challenges are few-shot adaptation and cross-task generalization. To tackle these issues, we propose a general PE priming framework to enhance and explore the few-shot adaptation and generalization ability of PE methods. In this framework, PLMs are primed with PE methods for rapidly adapting to various target tasks. To evaluate the generalization ability of these PE methods, we conduct experiments on a few-shot cross-domain benchmark containing 160 diverse NLP tasks. Our experiment not only reveals the best priming strategy but also verifies that priming facilitates the adaptation to target tasks. △ Less

Submitted 30 May, 2024; v1 submitted 2 December, 2022; originally announced December 2022.

End-to-End Evaluation of a Spoken Dialogue System for Learning Basic Mathematics

Authors: Eda Okur, Saurav Sahay, Roddy Fuentes Alba, Lama Nachman

Abstract: The advances in language-based Artificial Intelligence (AI) technologies applied to build educational applications can present AI for social-good opportunities with a broader positive impact. Across many disciplines, enhancing the quality of mathematics education is crucial in building critical thinking and problem-solving skills at younger ages. Conversational AI systems have started maturing to… ▽ More The advances in language-based Artificial Intelligence (AI) technologies applied to build educational applications can present AI for social-good opportunities with a broader positive impact. Across many disciplines, enhancing the quality of mathematics education is crucial in building critical thinking and problem-solving skills at younger ages. Conversational AI systems have started maturing to a point where they could play a significant role in hel** students learn fundamental math concepts. This work presents a task-oriented Spoken Dialogue System (SDS) built to support play-based learning of basic math concepts for early childhood education. The system has been evaluated via real-world deployments at school while the students are practicing early math concepts with multimodal interactions. We discuss our efforts to improve the SDS pipeline built for math learning, for which we explore utilizing MathBERT representations for potential enhancement to the Natural Language Understanding (NLU) module. We perform an end-to-end evaluation using real-world deployment outputs from the Automatic Speech Recognition (ASR), Intent Recognition, and Dialogue Manager (DM) components to understand how error propagation affects the overall performance in real-world scenarios. △ Less

Submitted 7 November, 2022; originally announced November 2022.

Comments: Proceedings of the 1st Workshop on Mathematical Natural Language Processing (MathNLP) at EMNLP 2022

Human in the loop approaches in multi-modal conversational task guidance system development

Authors: Ramesh Manuvinakurike, Sovan Biswas, Giuseppe Raffa, Richard Beckwith, Anthony Rhodes, Meng Shi, Gesem Gudino Mejia, Saurav Sahay, Lama Nachman

Abstract: Development of task guidance systems for aiding humans in a situated task remains a challenging problem. The role of search (information retrieval) and conversational systems for task guidance has immense potential to help the task performers achieve various goals. However, there are several technical challenges that need to be addressed to deliver such conversational systems, where common supervi… ▽ More Development of task guidance systems for aiding humans in a situated task remains a challenging problem. The role of search (information retrieval) and conversational systems for task guidance has immense potential to help the task performers achieve various goals. However, there are several technical challenges that need to be addressed to deliver such conversational systems, where common supervised approaches fail to deliver the expected results in terms of overall performance, user experience and adaptation to realistic conditions. In this preliminary work we first highlight some of the challenges involved during the development of such systems. We then provide an overview of existing datasets available and highlight their limitations. We finally develop a model-in-the-loop wizard-of-oz based data collection tool and perform a pilot experiment. △ Less

Submitted 3 November, 2022; originally announced November 2022.

Comments: SCAI @ SIGIR

Learning to Generate Prompts for Dialogue Generation through Reinforcement Learning

Authors: Hsuan Su, Pohan Chi, Shih-Cheng Huang, Chung Ho Lam, Saurav Sahay, Shang-Tse Chen, Hung-yi Lee

Abstract: Much literature has shown that prompt-based learning is an efficient method to make use of the large pre-trained language model. Recent works also exhibit the possibility of steering a chatbot's output by plugging in an appropriate prompt. Gradient-based methods are often used to perturb the prompts. However, some language models are not even available to the public. In this work, we first explore… ▽ More Much literature has shown that prompt-based learning is an efficient method to make use of the large pre-trained language model. Recent works also exhibit the possibility of steering a chatbot's output by plugging in an appropriate prompt. Gradient-based methods are often used to perturb the prompts. However, some language models are not even available to the public. In this work, we first explored the combination of prompting and reinforcement learning (RL) to steer models' generation without accessing any of the models' parameters. Second, to reduce the training effort and enhance the generalizability to the unseen task, we apply multi-task learning to make the model learn to generalize to new tasks better. The experiment results show that our proposed method can successfully control several state-of-the-art (SOTA) dialogue models without accessing their parameters. Furthermore, the model demonstrates the strong ability to quickly adapt to an unseen task in fewer steps than the baseline model. △ Less

Submitted 13 October, 2022; v1 submitted 8 June, 2022; originally announced June 2022.

Deep Reinforcement Learning for Cybersecurity Threat Detection and Protection: A Review

Authors: Mohit Sewak, Sanjay K. Sahay, Hemant Rathore

Abstract: The cybersecurity threat landscape has lately become overly complex. Threat actors leverage weaknesses in the network and endpoint security in a very coordinated manner to perpetuate sophisticated attacks that could bring down the entire network and many critical hosts in the network. Increasingly advanced deep and machine learning-based solutions have been used in threat detection and protection.… ▽ More The cybersecurity threat landscape has lately become overly complex. Threat actors leverage weaknesses in the network and endpoint security in a very coordinated manner to perpetuate sophisticated attacks that could bring down the entire network and many critical hosts in the network. Increasingly advanced deep and machine learning-based solutions have been used in threat detection and protection. The application of these techniques has been reviewed well in the scientific literature. Deep Reinforcement Learning has shown great promise in develo** AI-based solutions for areas that had earlier required advanced human cognizance. Different techniques and algorithms under deep reinforcement learning have shown great promise in applications ranging from games to industrial processes, where it is claimed to augment systems with general AI capabilities. These algorithms have recently also been used in cybersecurity, especially in threat detection and endpoint protection, where these are showing state-of-the-art results. Unlike supervised machines and deep learning, deep reinforcement learning is used in more diverse ways and is empowering many innovative applications in the threat defense landscape. However, there does not exist any comprehensive review of these unique applications and accomplishments. Therefore, in this paper, we intend to fill this gap and provide a comprehensive review of the different applications of deep reinforcement learning in cybersecurity threat detection and protection. △ Less

Submitted 6 June, 2022; originally announced June 2022.

Journal ref: International Conference On Secure Knowledge Management In Artificial Intelligence Era. Springer, Cham, 2021

NLU for Game-based Learning in Real: Initial Evaluations

Authors: Eda Okur, Saurav Sahay, Lama Nachman

Abstract: Intelligent systems designed for play-based interactions should be contextually aware of the users and their surroundings. Spoken Dialogue Systems (SDS) are critical for these interactive agents to carry out effective goal-oriented communication with users in real-time. For the real-world (i.e., in-the-wild) deployment of such conversational agents, improving the Natural Language Understanding (NL… ▽ More Intelligent systems designed for play-based interactions should be contextually aware of the users and their surroundings. Spoken Dialogue Systems (SDS) are critical for these interactive agents to carry out effective goal-oriented communication with users in real-time. For the real-world (i.e., in-the-wild) deployment of such conversational agents, improving the Natural Language Understanding (NLU) module of the goal-oriented SDS pipeline is crucial, especially with limited task-specific datasets. This study explores the potential benefits of a recently proposed transformer-based multi-task NLU architecture, mainly to perform Intent Recognition on small-size domain-specific educational game datasets. The evaluation datasets were collected from children practicing basic math concepts via play-based interactions in game-based learning settings. We investigate the NLU performances on the initial proof-of-concept game datasets versus the real-world deployment datasets and observe anticipated performance drops in-the-wild. We have shown that compared to the more straightforward baseline approaches, Dual Intent and Entity Transformer (DIET) architecture is robust enough to handle real-world data to a large extent for the Intent Recognition task on these domain-specific in-the-wild game datasets. △ Less

Submitted 26 May, 2022; originally announced May 2022.

Comments: Proceedings of the Games and Natural Language Processing Workshop at LREC 2022

Data Augmentation with Paraphrase Generation and Entity Extraction for Multimodal Dialogue System

Authors: Eda Okur, Saurav Sahay, Lama Nachman

Abstract: Contextually aware intelligent agents are often required to understand the users and their surroundings in real-time. Our goal is to build Artificial Intelligence (AI) systems that can assist children in their learning process. Within such complex frameworks, Spoken Dialogue Systems (SDS) are crucial building blocks to handle efficient task-oriented communication with children in game-based learni… ▽ More Contextually aware intelligent agents are often required to understand the users and their surroundings in real-time. Our goal is to build Artificial Intelligence (AI) systems that can assist children in their learning process. Within such complex frameworks, Spoken Dialogue Systems (SDS) are crucial building blocks to handle efficient task-oriented communication with children in game-based learning settings. We are working towards a multimodal dialogue system for younger kids learning basic math concepts. Our focus is on improving the Natural Language Understanding (NLU) module of the task-oriented SDS pipeline with limited datasets. This work explores the potential benefits of data augmentation with paraphrase generation for the NLU models trained on small task-specific datasets. We also investigate the effects of extracting entities for conceivably further data expansion. We have shown that paraphrasing with model-in-the-loop (MITL) strategies using small seed data is a promising approach yielding improved performance results for the Intent Recognition task. △ Less

Submitted 8 May, 2022; originally announced May 2022.

Comments: Proceedings of the 13th International Conference on Language Resources and Evaluation (LREC 2022)

Seamlessly Integrating Factual Information and Social Content with Persuasive Dialogue

Authors: Maximillian Chen, Weiyan Shi, Feifan Yan, Ryan Hou, **gwen Zhang, Saurav Sahay, Zhou Yu

Abstract: Complex conversation settings such as persuasion involve communicating changes in attitude or behavior, so users' perspectives need to be addressed, even when not directly related to the topic. In this work, we contribute a novel modular dialogue system framework that seamlessly integrates factual information and social content into persuasive dialogue. Our framework is generalizable to any dialog… ▽ More Complex conversation settings such as persuasion involve communicating changes in attitude or behavior, so users' perspectives need to be addressed, even when not directly related to the topic. In this work, we contribute a novel modular dialogue system framework that seamlessly integrates factual information and social content into persuasive dialogue. Our framework is generalizable to any dialogue tasks that have mixed social and task contents. We conducted a study that compared user evaluations of our framework versus a baseline end-to-end generation model. We found our framework was evaluated more favorably in all dimensions including competence and friendliness, compared to the end-to-end model which does not explicitly handle social content or factual questions. △ Less

Submitted 23 September, 2022; v1 submitted 15 March, 2022; originally announced March 2022.

Comments: To appear in Proceedings of AACL-IJCNLP 2022; 16 pages, 4 figures, 7 tables

Controllable Response Generation for Assistive Use-cases

Authors: Shachi H Kumar, Hsuan Su, Ramesh Manuvinakurike, Saurav Sahay, Lama Nachman

Abstract: Conversational agents have become an integral part of the general population for simple task enabling situations. However, these systems are yet to have any social impact on the diverse and minority population, for example, hel** people with neurological disorders, for example ALS, and people with speech, language and social communication disorders. Language model technology can play a huge role… ▽ More Conversational agents have become an integral part of the general population for simple task enabling situations. However, these systems are yet to have any social impact on the diverse and minority population, for example, hel** people with neurological disorders, for example ALS, and people with speech, language and social communication disorders. Language model technology can play a huge role to help these users carry out daily communication and social interactions. To enable this population, we build a dialog system that can be controlled by users using cues or keywords. We build models that can suggest relevant cues in the dialog response context which is used to control response generation and can speed up communication. We also introduce a keyword loss to lexically constrain the model output. We show both qualitatively and quantitatively that our models can effectively induce the keyword into the model response without degrading the quality of response. In the context of usage of such systems for people with degenerative disorders, we present human evaluation of our cue or keyword predictor and the controllable dialog system and show that our models perform significantly better than models without control. Our study shows that keyword control on end to end response generation models is powerful and can enable and empower users with degenerative disorders to carry out their day to day communication. △ Less

Submitted 4 December, 2021; originally announced December 2021.

Energy-Efficient Implementation of Generative Adversarial Networks on Passive RRAM Crossbar Arrays

Authors: Siddharth Satyam, Honey Nikam, Shubham Sahay

Abstract: Generative algorithms such as GANs are at the cusp of next revolution in the field of unsupervised learning and large-scale artificial data generation. However, the adversarial (competitive) co-training of the discriminative and generative networks in GAN makes them computationally intensive and hinders their deployment on the resource-constrained IoT edge devices. Moreover, the frequent data tran… ▽ More Generative algorithms such as GANs are at the cusp of next revolution in the field of unsupervised learning and large-scale artificial data generation. However, the adversarial (competitive) co-training of the discriminative and generative networks in GAN makes them computationally intensive and hinders their deployment on the resource-constrained IoT edge devices. Moreover, the frequent data transfer between the discriminative and generative networks during training significantly degrades the efficacy of the von-Neumann GAN accelerators such as those based on GPU and FPGA. Therefore, there is an urgent need for development of ultra-compact and energy-efficient hardware accelerators for GANs. To this end, in this work, we propose to exploit the passive RRAM crossbar arrays for performing key operations of a fully-connected GAN: (a) true random noise generation for the generator network, (b) vector-by-matrix-multiplication with unprecedented energy-efficiency during the forward pass and backward propagation and (C) in-situ adversarial training using a hardware friendly Manhattan's rule. Our extensive analysis utilizing an experimentally calibrated phenomological model for passive RRAM crossbar array reveals an unforeseen trade-off between the accuracy and the energy dissipated while training the GAN network with different noise inputs to the generator. Furthermore, our results indicate that the spatial and temporal variations and true random noise, which are otherwise undesirable for memory application, boost the energy-efficiency of the GAN implementation on passive RRAM crossbar arrays without degrading its accuracy. △ Less

Submitted 19 April, 2022; v1 submitted 29 November, 2021; originally announced November 2021.

Long Short-Term Memory Implementation Exploiting Passive RRAM Crossbar Array

Authors: Honey Nikam, Siddharth Satyam, Shubham Sahay

Abstract: The ever-increasing demand to extract temporal correlations across sequential data and perform context-based learning in this era of big data has led to the development of long short-term memory (LSTM) networks. Furthermore, there is an urgent need to perform these time-series data-dependent applications including speech/video processing and recognition, language modelling and translation, etc. on… ▽ More The ever-increasing demand to extract temporal correlations across sequential data and perform context-based learning in this era of big data has led to the development of long short-term memory (LSTM) networks. Furthermore, there is an urgent need to perform these time-series data-dependent applications including speech/video processing and recognition, language modelling and translation, etc. on compact internet-of-things (IoT) edge devices with limited energy. To this end, in this work, for the first time, we propose an extremely area- and energy-efficient LSTM network implementation exploiting the passive resistive random access memory (RRAM) crossbar array. We developed a hardware-aware LSTM network simulation framework and performed an extensive analysis of the proposed LSTM implementation considering the non-ideal hardware artifacts such as spatial (device-to-device) and temporal variations, non-linearity, noise, etc. utilizing an experimentally calibrated comprehensive phenomenological model for passive RRAM crossbar array. Our results indicate that the proposed passive RRAM crossbar-based LSTM network implementation not only outperforms the prior digital and active 1T-1R crossbar-based LSTM implementations by more than three orders of magnitude in terms of area and two orders of magnitude in terms of training energy for identical network accuracy, but also exhibits robustness against spatial and temporal variations and noise, and a faster convergence rate. Our work may provide the incentive for experimental realization of LSTM networks on passive RRAM crossbar arrays. △ Less

Submitted 8 November, 2021; originally announced November 2021.

Privacy-Preserving Mutual Authentication and Key Agreement Scheme for Multi-Server Healthcare System

Authors: Trupil Limbasiya, Sanjay K. Sahay, Bharath Sridharan

Abstract: The usage of different technologies and smart devices helps people to get medical services remotely for multiple benefits. Thus, critical and sensitive data is exchanged between a user and a doctor. When health data is transmitted over a common channel, it becomes essential to preserve various privacy and security properties in the system. Further, the number of users for remote services is increa… ▽ More The usage of different technologies and smart devices helps people to get medical services remotely for multiple benefits. Thus, critical and sensitive data is exchanged between a user and a doctor. When health data is transmitted over a common channel, it becomes essential to preserve various privacy and security properties in the system. Further, the number of users for remote services is increasing day-by-day exponentially, and thus, it is not adequate to deal with all users using the one server due to the verification overhead, server failure, and scalability issues. Thus, researchers proposed various authentication protocols for multi-server architecture, but most of them are vulnerable to different security attacks and require high computational resources during the implementation. To Tackle privacy and security issues using less computational resources, we propose a privacy-preserving mutual authentication and key agreement protocol for a multi-server healthcare system. We discuss the proposed scheme's security analysis and performance results to understand its security strengths and the computational resource requirement, respectively. Further, we do the comparison of security and performance results with recent relevant authentication protocols. △ Less

Submitted 13 October, 2021; originally announced October 2021.

Comments: 22 Pages

Journal ref: Information Systems Frontiers, Vol. 23, No. 4, p. 835, 2021

ADVERSARIALuscator: An Adversarial-DRL Based Obfuscator and Metamorphic Malware SwarmGenerator

Authors: Mohit Sewak, Sanjay K. Sahay, Hemant Rathore

Abstract: Advanced metamorphic malware and ransomware, by using obfuscation, could alter their internal structure with every attack. If such malware could intrude even into any of the IoT networks, then even if the original malware instance gets detected, by that time it can still infect the entire network. It is challenging to obtain training data for such evasive malware. Therefore, in this paper, we pres… ▽ More Advanced metamorphic malware and ransomware, by using obfuscation, could alter their internal structure with every attack. If such malware could intrude even into any of the IoT networks, then even if the original malware instance gets detected, by that time it can still infect the entire network. It is challenging to obtain training data for such evasive malware. Therefore, in this paper, we present ADVERSARIALuscator, a novel system that uses specialized Adversarial-DRL to obfuscate malware at the opcode level and create multiple metamorphic instances of the same. To the best of our knowledge, ADVERSARIALuscator is the first-ever system that adopts the Markov Decision Process-based approach to convert and find a solution to the problem of creating individual obfuscations at the opcode level. This is important as the machine language level is the least at which functionality could be preserved so as to mimic an actual attack effectively. ADVERSARIALuscator is also the first-ever system to use efficient continuous action control capable of deep reinforcement learning agents like the Proximal Policy Optimization in the area of cyber security. Experimental results indicate that ADVERSARIALuscator could raise the metamorphic probability of a corpus of malware by >0.45. Additionally, more than 33% of metamorphic instances generated by ADVERSARIALuscator were able to evade the most potent IDS. If such malware could intrude even into any of the IoT networks, then even if the original malware instance gets detected, by that time it can still infect the entire network. Hence ADVERSARIALuscator could be used to generate data representative of a swarm of very potent and coordinated AI-based metamorphic malware attacks. The so generated data and simulations could be used to bolster the defenses of an IDS against an actual AI-based metamorphic attack from advanced malware and ransomware. △ Less

Submitted 23 September, 2021; originally announced September 2021.

Journal ref: 2021 International Joint Conference on Neural Networks (IJCNN), 2021, pp. 1-9

LSTM Hyper-Parameter Selection for Malware Detection: Interaction Effects and Hierarchical Selection Approach

Authors: Mohit Sewak, Sanjay K. Sahay, Hemant Rathore

Abstract: Long-Short-Term-Memory (LSTM) networks have shown great promise in artificial intelligence (AI) based language modeling. Recently, LSTM networks have also become popular for designing AI-based Intrusion Detection Systems (IDS). However, its applicability in IDS is studied largely in the default settings as used in language models. Whereas security applications offer distinct conditions and hence w… ▽ More Long-Short-Term-Memory (LSTM) networks have shown great promise in artificial intelligence (AI) based language modeling. Recently, LSTM networks have also become popular for designing AI-based Intrusion Detection Systems (IDS). However, its applicability in IDS is studied largely in the default settings as used in language models. Whereas security applications offer distinct conditions and hence warrant careful consideration while applying such recurrent networks. Therefore, we conducted one of the most exhaustive works on LSTM hyper-parameters for IDS and experimented with approx. 150 LSTM configurations to determine its hyper-parameters relative importance, interaction effects, and optimal selection approach for designing an IDS. We conducted multiple analyses of the results of these experiments and empirically controlled for the interaction effects of different hyper-parameters covariate levels. We found that for security applications, especially for designing an IDS, neither similar relative importance as applicable to language models is valid, nor is the standard linear method for hyper-parameter selection ideal. We ascertained that the interaction effect plays a crucial role in determining the relative importance of hyper-parameters. We also discovered that after controlling for the interaction effect, the correct relative importance for LSTMs for an IDS is batch-size, followed by dropout ratio and padding. The findings are significant because when LSTM was first used for language models, the focus had mostly been on increasing the number of layers to enhance performance. △ Less

Submitted 23 September, 2021; originally announced September 2021.

Journal ref: 2021 International Joint Conference on Neural Networks (IJCNN), 2021, pp. 1-9

DRo: A data-scarce mechanism to revolutionize the performance of Deep Learning based Security Systems

Authors: Mohit Sewak, Sanjay K. Sahay, Hemant Rathore

Abstract: Supervised Deep Learning requires plenty of labeled data to converge, and hence perform optimally for task-specific learning. Therefore, we propose a novel mechanism named DRo (for Deep Routing) for data-scarce domains like security. The DRo approach builds upon some of the recent developments in Deep-Clustering. In particular, it exploits the self-augmented training mechanism using synthetically… ▽ More Supervised Deep Learning requires plenty of labeled data to converge, and hence perform optimally for task-specific learning. Therefore, we propose a novel mechanism named DRo (for Deep Routing) for data-scarce domains like security. The DRo approach builds upon some of the recent developments in Deep-Clustering. In particular, it exploits the self-augmented training mechanism using synthetically generated local perturbations. DRo not only allays the challenges with sparse-labeled data but also offers many unique advantages. We also developed a system named DRoID that uses the DRo mechanism for enhancing the performance of an existing Malware Detection System that uses (low information features like the) Android implicit Intent(s) as the only features. We conduct experiments on DRoID using a popular and standardized Android malware dataset and found that the DRo mechanism could successfully reduce the false-alarms generated by the downstream classifier by 67.9%, and also simultaneously boosts its accuracy by 11.3%. This is significant not only because the gains achieved are unparalleled but also because the features used were never considered rich enough to train a classifier on; and hence no decent performance could ever be reported by any malware classification system till-date using these features in isolation. Owing to the results achieved, the DRo mechanism claims a dominant position amongst all known systems that aims to enhance the classification performance of deep learning models with sparse-labeled data. △ Less

Submitted 12 September, 2021; originally announced September 2021.

Journal ref: 2021 IEEE 46th Conference on Local Computer Networks (LCN), 2021, pp. 581-588

Digital Resilience for What? Case Study of South Korea

Authors: Kyung Ryul Park, Sundeep Sahay, Jørn Braa, Pamod Amarakoon

Abstract: Resilience has become an emerging topic in various fields of academic research. In spite of its widespread use, there remains conceptual confusion over what resilience means particularly in multi-disciplinary studies including the field of ICT and Development. With the potential of digital technology, research is needed to critically question what key socio-institutional values related to resilien… ▽ More Resilience has become an emerging topic in various fields of academic research. In spite of its widespread use, there remains conceptual confusion over what resilience means particularly in multi-disciplinary studies including the field of ICT and Development. With the potential of digital technology, research is needed to critically question what key socio-institutional values related to resilience are being strengthened, for what and for whom through the different conceptualizations of resilience. In this study, we conduct an interpretive case study on South Korea's response to the pandemic and construct a chronological narrative to identify key aspects of digital resilience. We identify agility, diversity, and plurality - enabled by active roles of various stakeholders, including citizens, research communities, and private sector - as keys to digital resilience to the pandemic. Findings from the case of South Korea provide implications to ICT4D research while discussing how develo** countries, where a national single window platform is typically implemented with greater level of homogeneity, achieve digital resilience with inclusive innovation with plurality of diverse platforms. △ Less

Submitted 23 August, 2021; originally announced August 2021.

Comments: In proceedings of the 1st Virtual Conference on Implications of Information and Digital Technologies for Development, 2021

Reflections, Learnings and Proposed Interventions on Data Validation and Data Use for Action in Health: A Case of Mozambique

Authors: Nilza Collinson, Zeferino Saugene, Jørn Braa, Sundeep Sahay, Emilio Mosse

Abstract: The ideal of a country's health information system (HIS) is to develop processes that ensure easy collection of relevant data and enable their conversion to useful health indicators, which guide decision making and support health interventions. In many Low- and Middle-Income Countries (LMICs), actively engaged in health reform efforts, the role of HIS is crucial, particularly in terms of quality o… ▽ More The ideal of a country's health information system (HIS) is to develop processes that ensure easy collection of relevant data and enable their conversion to useful health indicators, which guide decision making and support health interventions. In many Low- and Middle-Income Countries (LMICs), actively engaged in health reform efforts, the role of HIS is crucial, particularly in terms of quality of data and its ability to inspire trust in decision makers to actively use routine HIS data. Recognizing digital platforms potential to support those efforts, several interventions have been implemented in many LMICs. In turn, while the transition from paper registers to digital platforms carries the promise of improving data quality processes, this promise has been notoriously complex to materialize in practice. The authors draw upon more than 15 years of experience implementing HIS in Mozambique to understand how the potential of digital platforms have been realized with respect to data quality, what are the gaps and required remedial steps. △ Less

Submitted 22 August, 2021; originally announced August 2021.

Comments: In proceedings of the 1st Virtual Conference on Implications of Information and Digital Technologies for Development, 2021

Building Agility in COVID-19 Information Systems Response in Sri Lanka: Recommendations for Practice

Authors: Pamod Amarakoon, Jorn Braa, Sundeep Sahay

Abstract: COVID-19 pandemic tested the capacity of information systems in countries on the ability to rapidly respond to requirements which were not anticipated. This article analyzes the socio-technical determinants of agility in building the IS response to the COVID-19 pandemic in Sri Lanka. We deploy qualitative research methods to explore the case study of implementation of COVID-19 surveillance system… ▽ More COVID-19 pandemic tested the capacity of information systems in countries on the ability to rapidly respond to requirements which were not anticipated. This article analyzes the socio-technical determinants of agility in building the IS response to the COVID-19 pandemic in Sri Lanka. We deploy qualitative research methods to explore the case study of implementation of COVID-19 surveillance system in Sri Lanka. Three key recommendations are developed for practice relating to high-level multisectoral governance, use of lightweight digital platforms and leveraging on existing capacities and infrastructure. △ Less

Submitted 22 August, 2021; originally announced August 2021.

Comments: In proceedings of the 1st Virtual Conference on Implications of Information and Digital Technologies for Development, 2021

Building Resilient Information Systems for Child Nutrition in Post-conflict Sri Lanka during COVID-19 Pandemic

Authors: Pamod Amarakoon, Jørn Braa, Sundeep Sahay, Lakmini Magodarathna, Rajeev Moorthy

Abstract: Post-conflict, low-resource settings are menaced with challenges related to low-resources, economic and social instability. The objective of the study is to understand the socio-technical determinants of resilience of resilience of routine information systems a backdrop of an implementation of a mobile-based nutrition information system in a post-conflict district in Sri Lanka. The longitudinal ev… ▽ More Post-conflict, low-resource settings are menaced with challenges related to low-resources, economic and social instability. The objective of the study is to understand the socio-technical determinants of resilience of resilience of routine information systems a backdrop of an implementation of a mobile-based nutrition information system in a post-conflict district in Sri Lanka. The longitudinal events in the study spans across several years into the period of COVID-19 pandemic and tries to understand the process of develo** resilience of in a vulnerable district. The qualitative study deploys interviews, observations and document analysis for collection of empirical data. The case study reveals the long-standing capacity building, leadership and local governance, multisector collaboration, platform resilience and empowering of field health staff contribute in building resilience in everyday context. The empirical insights include the mechanisms in building resilience in routine system in low resource settings while promoting data quality and data use at field level. △ Less

Submitted 22 August, 2021; originally announced August 2021.

Comments: In proceedings of the 1st Virtual Conference on Implications of Information and Digital Technologies for Development, 2021

Digital Global Public Goods

Authors: Johan Ivar Sæbø, Brian Nicholson, Petter Nielsen, Sundeep Sahay

Abstract: The purpose of this paper is to define and conceptualize digital global public goods (DGPGs) and illustrate the importance of contextual relevance in ICT4D projects. Recent studies have examined the importance of digital artefacts with public goods traits, emphasizing the significant potential for socio-economic development. However, we know little about the theoretical and practical dimensions of… ▽ More The purpose of this paper is to define and conceptualize digital global public goods (DGPGs) and illustrate the importance of contextual relevance in ICT4D projects. Recent studies have examined the importance of digital artefacts with public goods traits, emphasizing the significant potential for socio-economic development. However, we know little about the theoretical and practical dimensions of how we can align the public goods traits of such artefacts to create relevance in the context they are implemented. To address this gap we review the literature firstly to develop a definition and conceptual basis of DGPGs and then to illustrate the importance of relevance: how to align DGPGs with context to meet local needs. The illustration draws from a case study of the District Health Information systems (DHIS2). The paper advances both the theoretical and practical understanding of DPGs in development processes. △ Less

Submitted 22 August, 2021; originally announced August 2021.

Comments: In proceedings of the 1st Virtual Conference on Implications of Information and Digital Technologies for Development, 2021

Semi-supervised Interactive Intent Labeling

Authors: Saurav Sahay, Eda Okur, Nagib Hakim, Lama Nachman

Abstract: Building the Natural Language Understanding (NLU) modules of task-oriented Spoken Dialogue Systems (SDS) involves a definition of intents and entities, collection of task-relevant data, annotating the data with intents and entities, and then repeating the same process over and over again for adding any functionality/enhancement to the SDS. In this work, we showcase an Intent Bulk Labeling system w… ▽ More Building the Natural Language Understanding (NLU) modules of task-oriented Spoken Dialogue Systems (SDS) involves a definition of intents and entities, collection of task-relevant data, annotating the data with intents and entities, and then repeating the same process over and over again for adding any functionality/enhancement to the SDS. In this work, we showcase an Intent Bulk Labeling system where SDS developers can interactively label and augment training data from unlabeled utterance corpora using advanced clustering and visual labeling methods. We extend the Deep Aligned Clustering work with a better backbone BERT model, explore techniques to select the seed data for labeling, and develop a data balancing method using an oversampling technique that utilizes paraphrasing models. We also look at the effect of data augmentation on the clustering process. Our results show that we can achieve over 10% gain in clustering accuracy on some datasets using the combination of the above techniques. Finally, we extract utterance embeddings from the clustering model and plot the data to interactively bulk label the samples, reducing the time and effort for data labeling of the whole dataset significantly. △ Less

Submitted 11 May, 2021; v1 submitted 27 April, 2021; originally announced April 2021.

Comments: NAACL 2021 - Workshop on Data Science with Human-in-the-loop: Language Advances (DaSH-LA)

Put Chatbot into Its Interlocutor's Shoes: New Framework to Learn Chatbot Responding with Intention

Authors: Hsuan Su, Jiun-Hao Jhan, Fan-yun Sun, Saurav Sahay, Hung-yi Lee

Abstract: Most chatbot literature that focuses on improving the fluency and coherence of a chatbot, is dedicated to making chatbots more human-like. However, very little work delves into what really separates humans from chatbots -- humans intrinsically understand the effect their responses have on the interlocutor and often respond with an intention such as proposing an optimistic view to make the interloc… ▽ More Most chatbot literature that focuses on improving the fluency and coherence of a chatbot, is dedicated to making chatbots more human-like. However, very little work delves into what really separates humans from chatbots -- humans intrinsically understand the effect their responses have on the interlocutor and often respond with an intention such as proposing an optimistic view to make the interlocutor feel better. This paper proposes an innovative framework to train chatbots to possess human-like intentions. Our framework includes a guiding chatbot and an interlocutor model that plays the role of humans. The guiding chatbot is assigned an intention and learns to induce the interlocutor to reply with responses matching the intention, for example, long responses, joyful responses, responses with specific words, etc. We examined our framework using three experimental setups and evaluated the guiding chatbot with four different metrics to demonstrate flexibility and performance advantages. Additionally, we performed trials with human interlocutors to substantiate the guiding chatbot's effectiveness in influencing the responses of humans to a certain extent. Code will be made available to the public. △ Less

Submitted 23 April, 2021; v1 submitted 30 March, 2021; originally announced March 2021.

Comments: Accepted at NAACL-HLT 2021

Identification of Significant Permissions for Efficient Android Malware Detection

Authors: Hemant Rathore, Sanjay K. Sahay, Ritvik Rajvanshi, Mohit Sewak

Abstract: Since Google unveiled Android OS for smartphones, malware are thriving with 3Vs, i.e. volume, velocity, and variety. A recent report indicates that one out of every five business/industry mobile application leaks sensitive personal data. Traditional signature/heuristic-based malware detection systems are unable to cope up with current malware challenges and thus threaten the Android ecosystem. The… ▽ More Since Google unveiled Android OS for smartphones, malware are thriving with 3Vs, i.e. volume, velocity, and variety. A recent report indicates that one out of every five business/industry mobile application leaks sensitive personal data. Traditional signature/heuristic-based malware detection systems are unable to cope up with current malware challenges and thus threaten the Android ecosystem. Therefore recently researchers have started exploring machine learning and deep learning based malware detection systems. In this paper, we performed a comprehensive feature analysis to identify the significant Android permissions and propose an efficient Android malware detection system using machine learning and deep neural network. We constructed a set of $16$ permissions ($8\%$ of the total set) derived from variance threshold, auto-encoders, and principal component analysis to build a malware detection engine that consumes less train and test time without significant compromise on the model accuracy. Our experimental results show that the Android malware detection model based on the random forest classifier is most balanced and achieves the highest area under curve score of $97.7\%$, which is better than the current state-of-art systems. We also observed that deep neural networks attain comparable accuracy to the baseline results but with a massive computational penalty. △ Less

Submitted 28 February, 2021; originally announced March 2021.

Comments: BROADNETS, 2020

Detection of Malicious Android Applications: Classical Machine Learning vs. Deep Neural Network Integrated with Clustering

Authors: Hemant Rathore, Sanjay K. Sahay, Shivin Thukral, Mohit Sewak

Abstract: Today anti-malware community is facing challenges due to the ever-increasing sophistication and volume of malware attacks developed by adversaries. Traditional malware detection mechanisms are not able to cope-up with next-generation malware attacks. Therefore in this paper, we propose effective and efficient Android malware detection models based on machine learning and deep learning integrated w… ▽ More Today anti-malware community is facing challenges due to the ever-increasing sophistication and volume of malware attacks developed by adversaries. Traditional malware detection mechanisms are not able to cope-up with next-generation malware attacks. Therefore in this paper, we propose effective and efficient Android malware detection models based on machine learning and deep learning integrated with clustering. We performed a comprehensive study of different feature reduction, classification and clustering algorithms over various performance metrics to construct the Android malware detection models. Our experimental results show that malware detection models developed using Random Forest eclipsed deep neural network and other classifiers on the majority of performance metrics. The baseline Random Forest model without any feature reduction achieved the highest AUC of 99.4%. Also, the segregating of vector space using clustering integrated with Random Forest further boosted the AUC to 99.6% in one cluster and direct detection of Android malware in another cluster, thus reducing the curse of dimensionality. Additionally, we found that feature reduction in detection models does improve the model efficiency (training and testing time) many folds without much penalty on the effectiveness of the detection model. △ Less

Submitted 28 February, 2021; originally announced March 2021.

Comments: BROADNETS, 2020

DRLDO: A novel DRL based De-ObfuscationSystem for Defense against Metamorphic Malware

Authors: Mohit Sewak, Sanjay K. Sahay, Hemant Rathore

Abstract: In this paper, we propose a novel mechanism to normalize metamorphic and obfuscated malware down at the opcode level and hence create an advanced metamorphic malware de-obfuscation and defense system. We name this system DRLDO, for Deep Reinforcement Learning based De-Obfuscator. With the inclusion of the DRLDO as a sub-component, an existing Intrusion Detection System could be augmented with defe… ▽ More In this paper, we propose a novel mechanism to normalize metamorphic and obfuscated malware down at the opcode level and hence create an advanced metamorphic malware de-obfuscation and defense system. We name this system DRLDO, for Deep Reinforcement Learning based De-Obfuscator. With the inclusion of the DRLDO as a sub-component, an existing Intrusion Detection System could be augmented with defensive capabilities against 'zero-day' attacks from obfuscated and metamorphic variants of existing malware. This gains importance, not only because there exists no system to date that uses advanced DRL to intelligently and automatically normalize obfuscation down even to the opcode level, but also because the DRLDO system does not mandate any changes to the existing IDS. The DRLDO system does not even mandate the IDS' classifier to be retrained with any new dataset containing obfuscated samples. Hence DRLDO could be easily retrofitted into any existing IDS deployment. We designed, developed, and conducted experiments on the system to evaluate the same against multiple-simultaneous attacks from obfuscations generated from malware samples from a standardized dataset that contains multiple generations of malware. Experimental results prove that DRLDO was able to successfully make the otherwise un-detectable obfuscated variants of the malware detectable by an existing pre-trained malware classifier. The detection probability was raised well above the cut-off mark to 0.6 for the classifier to detect the obfuscated malware unambiguously. Further, the de-obfuscated variants generated by DRLDO achieved a very high correlation (of 0.99) with the base malware. This observation validates that the DRLDO system is actually learning to de-obfuscate and not exploiting a trivial trick. △ Less

Submitted 1 February, 2021; originally announced February 2021.

Journal ref: Defence Science Journal, 71(1), 55-65

Robust Android Malware Detection System against Adversarial Attacks using Q-Learning

Authors: Hemant Rathore, Sanjay K. Sahay, Piyush Nikam, Mohit Sewak

Abstract: The current state-of-the-art Android malware detection systems are based on machine learning and deep learning models. Despite having superior performance, these models are susceptible to adversarial attacks. Therefore in this paper, we developed eight Android malware detection models based on machine learning and deep neural network and investigated their robustness against adversarial attacks. F… ▽ More The current state-of-the-art Android malware detection systems are based on machine learning and deep learning models. Despite having superior performance, these models are susceptible to adversarial attacks. Therefore in this paper, we developed eight Android malware detection models based on machine learning and deep neural network and investigated their robustness against adversarial attacks. For this purpose, we created new variants of malware using Reinforcement Learning, which will be misclassified as benign by the existing Android malware detection models. We propose two novel attack strategies, namely single policy attack and multiple policy attack using reinforcement learning for white-box and grey-box scenario respectively. Putting ourselves in the adversary's shoes, we designed adversarial attacks on the detection models with the goal of maximizing fooling rate, while making minimum modifications to the Android application and ensuring that the app's functionality and behavior do not change. We achieved an average fooling rate of 44.21% and 53.20% across all the eight detection models with a maximum of five modifications using a single policy attack and multiple policy attack, respectively. The highest fooling rate of 86.09% with five changes was attained against the decision tree-based model using the multiple policy approach. Finally, we propose an adversarial defense strategy that reduces the average fooling rate by threefold to 15.22% against a single policy attack, thereby increasing the robustness of the detection models i.e. the proposed model can effectively detect variants (metamorphic) of malware. The experimental analysis shows that our proposed Android malware detection system using reinforcement learning is more robust against adversarial attacks. △ Less

Submitted 27 January, 2021; originally announced January 2021.

Comments: Inf Syst Front (2020)

Refine and Imitate: Reducing Repetition and Inconsistency in Persuasion Dialogues via Reinforcement Learning and Human Demonstration

Authors: Weiyan Shi, Yu Li, Saurav Sahay, Zhou Yu

Abstract: Persuasion dialogue systems reflect the machine's ability to make strategic moves beyond verbal communication, and therefore differentiate themselves from task-oriented or open-domain dialogue systems and have their own unique values. However, the repetition and inconsistency problems still persist in dialogue response generation and could substantially impact user experience and impede the persua… ▽ More Persuasion dialogue systems reflect the machine's ability to make strategic moves beyond verbal communication, and therefore differentiate themselves from task-oriented or open-domain dialogue systems and have their own unique values. However, the repetition and inconsistency problems still persist in dialogue response generation and could substantially impact user experience and impede the persuasion outcome. Besides, although reinforcement learning (RL) approaches have achieved big success in strategic tasks such as games, they require a sophisticated user simulator to provide real-time feedback to the dialogue system, which limits the application of RL on persuasion dialogues. To address these issues towards a better persuasion dialogue system, we apply RL to refine a language model baseline without user simulators, and distill sentence-level information about repetition, inconsistency, and task relevance through rewards. Moreover, to better accomplish the persuasion task, the model learns from human demonstration to imitate human persuasion behavior and selects the most persuasive responses. Experiments show that our model outperforms previous state-of-the-art dialogue models on both automatic metrics and human evaluation results on a donation persuasion task, and generates more diverse, consistent and persuasive conversations according to the user feedback. △ Less

Submitted 22 October, 2022; v1 submitted 30 December, 2020; originally announced December 2020.

Comments: EMNLP 2021 Findings

Assessment of the Relative Importance of different hyper-parameters of LSTM for an IDS

Authors: Mohit Sewak, Sanjay K. Sahay, Hemant Rathore

Abstract: Recurrent deep learning language models like the LSTM are often used to provide advanced cyber-defense for high-value assets. The underlying assumption for using LSTM networks for malware-detection is that the op-code sequence of malware could be treated as a (spoken) language representation. There are differences between any spoken-language (sequence of words/sentences) and the machine-language (… ▽ More Recurrent deep learning language models like the LSTM are often used to provide advanced cyber-defense for high-value assets. The underlying assumption for using LSTM networks for malware-detection is that the op-code sequence of malware could be treated as a (spoken) language representation. There are differences between any spoken-language (sequence of words/sentences) and the machine-language (sequence of op-codes). In this paper, we demonstrate that due to these inherent differences, an LSTM model with its default configuration as tuned for a spoken-language, may not work well to detect malware (using its op-code sequence) unless the network's essential hyper-parameters are tuned appropriately. In the process, we also determine the relative importance of all the different hyper-parameters of an LSTM network as applied to malware detection using their op-code sequence representations. We experimented with different configurations of LSTM networks, and altered hyper-parameters like the embedding-size, number of hidden layers, number of LSTM-units in a hidden layer, pruning/padding-length of the input-vector, activation-function, and batch-size. We discovered that owing to the enhanced complexity of the malware/machine-language, the performance of an LSTM network configured for an Intrusion Detection System, is very sensitive towards the number-of-hidden-layers, input sequence-length, and the choice of the activation-function. Also, for (spoken) language-modeling, the recurrent architectures by-far outperform their non-recurrent counterparts. Therefore, we also assess how sequential DL architectures like the LSTM compare against their non-sequential counterparts like the MLP-DNN for the purpose of malware-detection. △ Less

Submitted 26 December, 2020; originally announced December 2020.

Journal ref: 2020 IEEE REGION 10 CONFERENCE (TENCON), Osaka, Japan, 2020, pp. 414-419

DOOM: A Novel Adversarial-DRL-Based Op-Code Level Metamorphic Malware Obfuscator for the Enhancement of IDS

Authors: Mohit Sewak, Sanjay K. Sahay, Hemant Rathore

Abstract: We designed and developed DOOM (Adversarial-DRL based Opcode level Obfuscator to generate Metamorphic malware), a novel system that uses adversarial deep reinforcement learning to obfuscate malware at the op-code level for the enhancement of IDS. The ultimate goal of DOOM is not to give a potent weapon in the hands of cyber-attackers, but to create defensive-mechanisms against advanced zero-day at… ▽ More We designed and developed DOOM (Adversarial-DRL based Opcode level Obfuscator to generate Metamorphic malware), a novel system that uses adversarial deep reinforcement learning to obfuscate malware at the op-code level for the enhancement of IDS. The ultimate goal of DOOM is not to give a potent weapon in the hands of cyber-attackers, but to create defensive-mechanisms against advanced zero-day attacks. Experimental results indicate that the obfuscated malware created by DOOM could effectively mimic multiple-simultaneous zero-day attacks. To the best of our knowledge, DOOM is the first system that could generate obfuscated malware detailed to individual op-code level. DOOM is also the first-ever system to use efficient continuous action control based deep reinforcement learning in the area of malware generation and defense. Experimental results indicate that over 67% of the metamorphic malware generated by DOOM could easily evade detection from even the most potent IDS. This achievement gains significance, as with this, even IDS augment with advanced routing sub-system can be easily evaded by the malware generated by DOOM. △ Less

Submitted 16 October, 2020; originally announced October 2020.

DeepIntent: ImplicitIntent based Android IDS with E2E Deep Learning architecture

Authors: Mohit Sewak, Sanjay K. Sahay, Hemant Rathore

Abstract: The Intent in Android plays an important role in inter-process and intra-process communications. The implicit Intent that an application could accept are declared in its manifest and are amongst the easiest feature to extract from an apk. Implicit Intents could even be extracted online and in real-time. So far neither the feasibility of develo** an Intrusion Detection System solely on implicit I… ▽ More The Intent in Android plays an important role in inter-process and intra-process communications. The implicit Intent that an application could accept are declared in its manifest and are amongst the easiest feature to extract from an apk. Implicit Intents could even be extracted online and in real-time. So far neither the feasibility of develo** an Intrusion Detection System solely on implicit Intent has been explored, nor are any benchmarks available of a malware classifier that is based on implicit Intent alone. We demonstrate that despite Intent is implicit and well declared, it can provide very intuitive insights to distinguish malicious from non-malicious applications. We conducted exhaustive experiments with over 40 different end-to-end Deep Learning configurations of Auto-Encoders and Multi-Layer-Perceptron to create a benchmark for a malware classifier that works exclusively on implicit Intent. Using the results from the experiments we create an intrusion detection system using only the implicit Intents and end-to-end Deep Learning architecture. We obtained an area-under-curve statistic of 0.81, and accuracy of 77.2% along with false-positive-rate of 0.11 on Drebin dataset. △ Less

Submitted 16 October, 2020; originally announced October 2020.

A Novel Spatial-Spectral Framework for the Classification of Hyperspectral Satellite Imagery

Authors: Shriya TP Gupta, Sanjay K Sahay

Abstract: Hyper-spectral satellite imagery is now widely being used for accurate disaster prediction and terrain feature classification. However, in such classification tasks, most of the present approaches use only the spectral information contained in the images. Therefore, in this paper, we present a novel framework that takes into account both the spectral and spatial information contained in the data f… ▽ More Hyper-spectral satellite imagery is now widely being used for accurate disaster prediction and terrain feature classification. However, in such classification tasks, most of the present approaches use only the spectral information contained in the images. Therefore, in this paper, we present a novel framework that takes into account both the spectral and spatial information contained in the data for land cover classification. For this purpose, we use the Gaussian Maximum Likelihood (GML) and Convolutional Neural Network methods for the pixel-wise spectral classification and then, using segmentation maps generated by the Watershed algorithm, we incorporate the spatial contextual information into our model with a modified majority vote technique. The experimental analyses on two benchmark datasets demonstrate that our proposed methodology performs better than the earlier approaches by achieving an accuracy of 99.52% and 98.31% on the Pavia University and the Indian Pines datasets respectively. Additionally, our GML based approach, a non-deep learning algorithm, shows comparable performance to the state-of-the-art deep learning techniques, which indicates the importance of the proposed approach for performing a computationally efficient classification of hyper-spectral imagery. △ Less

Submitted 22 July, 2020; originally announced August 2020.

Comments: 13 Pages, 15 Figures, EANN-2020

Journal ref: Springer, INNS, Vol. 2, pp 227-239, 2020

Audio-Visual Understanding of Passenger Intents for In-Cabin Conversational Agents

Authors: Eda Okur, Shachi H Kumar, Saurav Sahay, Lama Nachman

Abstract: Building multimodal dialogue understanding capabilities situated in the in-cabin context is crucial to enhance passenger comfort in autonomous vehicle (AV) interaction systems. To this end, understanding passenger intents from spoken interactions and vehicle vision systems is a crucial component for develo** contextual and visually grounded conversational agents for AV. Towards this goal, we exp… ▽ More Building multimodal dialogue understanding capabilities situated in the in-cabin context is crucial to enhance passenger comfort in autonomous vehicle (AV) interaction systems. To this end, understanding passenger intents from spoken interactions and vehicle vision systems is a crucial component for develo** contextual and visually grounded conversational agents for AV. Towards this goal, we explore AMIE (Automated-vehicle Multimodal In-cabin Experience), the in-cabin agent responsible for handling multimodal passenger-vehicle interactions. In this work, we discuss the benefits of a multimodal understanding of in-cabin utterances by incorporating verbal/language input together with the non-verbal/acoustic and visual clues from inside and outside the vehicle. Our experimental results outperformed text-only baselines as we achieved improved performances for intent detection with a multimodal approach. △ Less

Submitted 7 July, 2020; originally announced July 2020.

Comments: ACL 2020 - Second Grand-Challenge and Workshop on Multimodal Language (Challenge-HML)

Low Rank Fusion based Transformers for Multimodal Sequences

Authors: Saurav Sahay, Eda Okur, Shachi H Kumar, Lama Nachman

Abstract: Our senses individually work in a coordinated fashion to express our emotional intentions. In this work, we experiment with modeling modality-specific sensory signals to attend to our latent multimodal emotional intentions and vice versa expressed via low-rank multimodal fusion and multimodal transformers. The low-rank factorization of multimodal fusion amongst the modalities helps represent appro… ▽ More Our senses individually work in a coordinated fashion to express our emotional intentions. In this work, we experiment with modeling modality-specific sensory signals to attend to our latent multimodal emotional intentions and vice versa expressed via low-rank multimodal fusion and multimodal transformers. The low-rank factorization of multimodal fusion amongst the modalities helps represent approximate multiplicative latent signal interactions. Motivated by the work of~\cite{tsai2019MULT} and~\cite{Liu_2018}, we present our transformer-based cross-fusion architecture without any over-parameterization of the model. The low-rank fusion helps represent the latent signal interactions while the modality-specific attention helps focus on relevant parts of the signal. We present two methods for the Multimodal Sentiment and Emotion Recognition results on CMU-MOSEI, CMU-MOSI, and IEMOCAP datasets and show that our models have lesser parameters, train faster and perform comparably to many larger fusion-based architectures. △ Less

Submitted 4 July, 2020; originally announced July 2020.

Comments: ACL 2020 workshop on Second Grand Challenge and Workshop on Multimodal Language

Secure and Energy-Efficient Key-Agreement Protocol for Multi-Server Architecture

Authors: Trupil Limbasiya, Sanjay K. Sahay

Abstract: Authentication schemes are practised globally to verify the legitimacy of users and servers for the exchange of data in different facilities. Generally, the server verifies a user to provide resources for different purposes. But due to the large network system, the authentication process has become complex and therefore, time-to-time different authentication protocols have been proposed for the mu… ▽ More Authentication schemes are practised globally to verify the legitimacy of users and servers for the exchange of data in different facilities. Generally, the server verifies a user to provide resources for different purposes. But due to the large network system, the authentication process has become complex and therefore, time-to-time different authentication protocols have been proposed for the multi-server architecture. However, most of the protocols are vulnerable to various security attacks and their performance is not efficient. In this paper, we propose a secure and energy-efficient remote user authentication protocol for multi-server systems. The results show that the proposed protocol is comparatively ~44% more efficient and needs ~38% less communication cost. We also demonstrate that with only two-factor authentication, the proposed protocol is more secure from the earlier related authentication schemes. △ Less

Submitted 19 April, 2020; originally announced April 2020.

Comments: 17 Pages, SKM-2019

Journal ref: Springer, CCIS, Vol. 1186, pp. 82-97, 2020

LinkedIn's Audience Engagements API: A Privacy Preserving Data Analytics System at Scale

Authors: Ryan Rogers, Subbu Subramaniam, Sean Peng, David Durfee, Seunghyun Lee, Santosh Kumar Kancha, Shraddha Sahay, Parvez Ahammad

Abstract: We present a privacy system that leverages differential privacy to protect LinkedIn members' data while also providing audience engagement insights to enable marketing analytics related applications. We detail the differentially private algorithms and other privacy safeguards used to provide results that can be used with existing real-time data analytics platforms, specifically with the open sourc… ▽ More We present a privacy system that leverages differential privacy to protect LinkedIn members' data while also providing audience engagement insights to enable marketing analytics related applications. We detail the differentially private algorithms and other privacy safeguards used to provide results that can be used with existing real-time data analytics platforms, specifically with the open sourced Pinot system. Our privacy system provides user-level privacy guarantees. As part of our privacy system, we include a budget management service that enforces a strict differential privacy budget on the returned results to the analyst. This budget management service brings together the latest research in differential privacy into a product to maintain utility given a fixed differential privacy budget. △ Less

Submitted 16 November, 2020; v1 submitted 13 February, 2020; originally announced February 2020.

Effects of Persuasive Dialogues: Testing Bot Identities and Inquiry Strategies

Authors: Weiyan Shi, Xuewei Wang, Yoo Jung Oh, **gwen Zhang, Saurav Sahay, Zhou Yu

Abstract: Intelligent conversational agents, or chatbots, can take on various identities and are increasingly engaging in more human-centered conversations with persuasive goals. However, little is known about how identities and inquiry strategies influence the conversation's effectiveness. We conducted an online study involving 790 participants to be persuaded by a chatbot for charity donation. We designed… ▽ More Intelligent conversational agents, or chatbots, can take on various identities and are increasingly engaging in more human-centered conversations with persuasive goals. However, little is known about how identities and inquiry strategies influence the conversation's effectiveness. We conducted an online study involving 790 participants to be persuaded by a chatbot for charity donation. We designed a two by four factorial experiment (two chatbot identities and four inquiry strategies) where participants were randomly assigned to different conditions. Findings showed that the perceived identity of the chatbot had significant effects on the persuasion outcome (i.e., donation) and interpersonal perceptions (i.e., competence, confidence, warmth, and sincerity). Further, we identified interaction effects among perceived identities and inquiry strategies. We discuss the findings for theoretical and practical implications for develo** ethical and effective persuasive chatbots. Our published data, codes, and analyses serve as the first step towards building competent ethical persuasive chatbots. △ Less

Submitted 18 January, 2020; v1 submitted 13 January, 2020; originally announced January 2020.

Comments: 15 pages, 10 figures. Full paper to appear at ACM CHI 2020

Secure Communication Protocol for Smart Transportation Based on Vehicular Cloud

Authors: Trupil Limbasiya, Debasis Das, Sanjay K. Sahay

Abstract: The pioneering concept of connected vehicles has transformed the way of thinking for researchers and entrepreneurs by collecting relevant data from nearby objects. However, this data is useful for a specific vehicle only. Moreover, vehicles get a high amount of data (e.g., traffic, safety, and multimedia infotainment) on the road. Thus, vehicles expect adequate storage devices for this data, but i… ▽ More The pioneering concept of connected vehicles has transformed the way of thinking for researchers and entrepreneurs by collecting relevant data from nearby objects. However, this data is useful for a specific vehicle only. Moreover, vehicles get a high amount of data (e.g., traffic, safety, and multimedia infotainment) on the road. Thus, vehicles expect adequate storage devices for this data, but it is infeasible to have a large memory in each vehicle. Hence, the vehicular cloud computing (VCC) framework came into the picture to provide a storage facility by connecting a road-side-unit (RSU) with the vehicular cloud (VC). In this, data should be saved in an encrypted form to preserve security, but there is a challenge to search for information over encrypted data. Next, we understand that many of vehicular communication schemes are inefficient for data transmissions due to its poor performance results and vulnerable to different fundamental security attacks. Accordingly, on-device performance is critical, but data damages and secure on-time connectivity are also significant challenges in a public environment. Therefore, we propose reliable data transmission protocols for cutting-edge architecture to search data from the storage, to resist against various security attacks, and provide better performance results. Thus, the proposed data transmission protocol is useful in diverse smart city applications (business, safety, and entertainment) for the benefits of society. △ Less

Submitted 4 January, 2020; v1 submitted 30 December, 2019; originally announced December 2019.

Comments: 10 Pages, 1 figure, Conference

Journal ref: ACM Proceedings of the 2019 ACM International Joint Conference on Pervasive and Ubiquitous Computing and Proceedings of the 2019 ACM International Symposium on Wearable Computers, pp. 372-376

Exploring Context, Attention and Audio Features for Audio Visual Scene-Aware Dialog

Authors: Shachi H Kumar, Eda Okur, Saurav Sahay, Jonathan Huang, Lama Nachman

Abstract: We are witnessing a confluence of vision, speech and dialog system technologies that are enabling the IVAs to learn audio-visual groundings of utterances and have conversations with users about the objects, activities and events surrounding them. Recent progress in visual grounding techniques and Audio Understanding are enabling machines to understand shared semantic concepts and listen to the var… ▽ More We are witnessing a confluence of vision, speech and dialog system technologies that are enabling the IVAs to learn audio-visual groundings of utterances and have conversations with users about the objects, activities and events surrounding them. Recent progress in visual grounding techniques and Audio Understanding are enabling machines to understand shared semantic concepts and listen to the various sensory events in the environment. With audio and visual grounding methods, end-to-end multimodal SDS are trained to meaningfully communicate with us in natural language about the real dynamic audio-visual sensory world around us. In this work, we explore the role of `topics' as the context of the conversation along with multimodal attention into such an end-to-end audio-visual scene-aware dialog system architecture. We also incorporate an end-to-end audio classification ConvNet, AclNet, into our models. We develop and test our approaches on the Audio Visual Scene-Aware Dialog (AVSD) dataset released as a part of the DSTC7. We present the analysis of our experiments and show that some of our model variations outperform the baseline system released for AVSD. △ Less

Submitted 20 December, 2019; originally announced December 2019.

Comments: Presented at the Visual Question Answering and Dialog Workshop, CVPR 2019, Long Beach, USA. arXiv admin note: substantial text overlap with arXiv:1912.10131

Leveraging Topics and Audio Features with Multimodal Attention for Audio Visual Scene-Aware Dialog

Authors: Shachi H Kumar, Eda Okur, Saurav Sahay, Jonathan Huang, Lama Nachman

Abstract: With the recent advancements in Artificial Intelligence (AI), Intelligent Virtual Assistants (IVA) such as Alexa, Google Home, etc., have become a ubiquitous part of many homes. Currently, such IVAs are mostly audio-based, but going forward, we are witnessing a confluence of vision, speech and dialog system technologies that are enabling the IVAs to learn audio-visual groundings of utterances. Thi… ▽ More With the recent advancements in Artificial Intelligence (AI), Intelligent Virtual Assistants (IVA) such as Alexa, Google Home, etc., have become a ubiquitous part of many homes. Currently, such IVAs are mostly audio-based, but going forward, we are witnessing a confluence of vision, speech and dialog system technologies that are enabling the IVAs to learn audio-visual groundings of utterances. This will enable agents to have conversations with users about the objects, activities and events surrounding them. In this work, we present three main architectural explorations for the Audio Visual Scene-Aware Dialog (AVSD): 1) investigating `topics' of the dialog as an important contextual feature for the conversation, 2) exploring several multimodal attention mechanisms during response generation, 3) incorporating an end-to-end audio classification ConvNet, AclNet, into our architecture. We discuss detailed analysis of the experimental results and show that our model variations outperform the baseline system presented for the AVSD task. △ Less

Submitted 20 December, 2019; originally announced December 2019.

Comments: Presented at the 3rd Visually Grounded Interaction and Language (ViGIL) Workshop, NeurIPS 2019, Vancouver, Canada. arXiv admin note: substantial text overlap with arXiv:1812.08407, arXiv:1912.10132

Modeling Intent, Dialog Policies and Response Adaptation for Goal-Oriented Interactions

Authors: Saurav Sahay, Shachi H Kumar, Eda Okur, Haroon Syed, Lama Nachman

Abstract: Building a machine learning driven spoken dialog system for goal-oriented interactions involves careful design of intents and data collection along with development of intent recognition models and dialog policy learning algorithms. The models should be robust enough to handle various user distractions during the interaction flow and should steer the user back into an engaging interaction for succ… ▽ More Building a machine learning driven spoken dialog system for goal-oriented interactions involves careful design of intents and data collection along with development of intent recognition models and dialog policy learning algorithms. The models should be robust enough to handle various user distractions during the interaction flow and should steer the user back into an engaging interaction for successful completion of the interaction. In this work, we have designed a goal-oriented interaction system where children can engage with agents for a series of interactions involving `Meet \& Greet' and `Simon Says' game play. We have explored various feature extractors and models for improved intent recognition and looked at leveraging previous user and system interactions in novel ways with attention models. We have also looked at dialog adaptation methods for entrained response selection. Our bootstrapped models from limited training data perform better than many baseline approaches we have looked at for intent recognition and dialog action prediction. △ Less

Submitted 20 December, 2019; originally announced December 2019.

Comments: Presented as a full-paper at the 23rd Workshop on the Semantics and Pragmatics of Dialogue (SemDial 2019 - LondonLogue), Sep 4-6, 2019, London, UK

Journal ref: Proceedings of the 23rd Workshop on the Semantics and Pragmatics of Dialogue (SEMDIAL), pp. 146-155, London, United Kingdom, September 2019

Towards Multimodal Understanding of Passenger-Vehicle Interactions in Autonomous Vehicles: Intent/Slot Recognition Utilizing Audio-Visual Data

Authors: Eda Okur, Shachi H Kumar, Saurav Sahay, Lama Nachman

Abstract: Understanding passenger intents from spoken interactions and car's vision (both inside and outside the vehicle) are important building blocks towards develo** contextual dialog systems for natural interactions in autonomous vehicles (AV). In this study, we continued exploring AMIE (Automated-vehicle Multimodal In-cabin Experience), the in-cabin agent responsible for handling certain multimodal p… ▽ More Understanding passenger intents from spoken interactions and car's vision (both inside and outside the vehicle) are important building blocks towards develo** contextual dialog systems for natural interactions in autonomous vehicles (AV). In this study, we continued exploring AMIE (Automated-vehicle Multimodal In-cabin Experience), the in-cabin agent responsible for handling certain multimodal passenger-vehicle interactions. When the passengers give instructions to AMIE, the agent should parse such commands properly considering available three modalities (language/text, audio, video) and trigger the appropriate functionality of the AV system. We had collected a multimodal in-cabin dataset with multi-turn dialogues between the passengers and AMIE using a Wizard-of-Oz scheme via realistic scavenger hunt game. In our previous explorations, we experimented with various RNN-based models to detect utterance-level intents (set destination, change route, go faster, go slower, stop, park, pull over, drop off, open door, and others) along with intent keywords and relevant slots (location, position/direction, object, gesture/gaze, time-guidance, person) associated with the action to be performed in our AV scenarios. In this recent work, we propose to discuss the benefits of multimodal understanding of in-cabin utterances by incorporating verbal/language input (text and speech embeddings) together with the non-verbal/acoustic and visual input from inside and outside the vehicle (i.e., passenger gestures and gaze from in-cabin video stream, referred objects outside of the vehicle from the road view camera stream). Our experimental results outperformed text-only baselines and with multimodality, we achieved improved performances for utterance-level intent detection and slot filling. △ Less

Submitted 19 September, 2019; originally announced September 2019.

Comments: Presented as a short-paper at the 23rd Workshop on the Semantics and Pragmatics of Dialogue (SemDial 2019 - LondonLogue), Sep 4-6, 2019, London, UK

Journal ref: Proceedings of the 23rd Workshop on the Semantics and Pragmatics of Dialogue (SEMDIAL), pp. 213-215, London, United Kingdom, September 2019

3D-aCortex: An Ultra-Compact Energy-Efficient Neurocomputing Platform Based on Commercial 3D-NAND Flash Memories

Authors: Mohammad Bavandpour, Shubham Sahay, Mohammad Reza Mahmoodi, Dmitri B. Strukov

Abstract: The first contribution of this paper is the development of extremely dense, energy-efficient mixed-signal vector-by-matrix-multiplication (VMM) circuits based on the existing 3D-NAND flash memory blocks, without any need for their modification. Such compatibility is achieved using time-domain-encoded VMM design. Our detailed simulations have shown that, for example, the 5-bit VMM of 200-element ve… ▽ More The first contribution of this paper is the development of extremely dense, energy-efficient mixed-signal vector-by-matrix-multiplication (VMM) circuits based on the existing 3D-NAND flash memory blocks, without any need for their modification. Such compatibility is achieved using time-domain-encoded VMM design. Our detailed simulations have shown that, for example, the 5-bit VMM of 200-element vectors, using the commercially available 64-layer gate-all-around macaroni-type 3D-NAND memory blocks designed in the 55-nm technology node, may provide an unprecedented area efficiency of 0.14 um2/byte and energy efficiency of ~10 fJ/Op, including the input/output and other peripheral circuitry overheads. Our second major contribution is the development of 3D-aCortex, a multi-purpose neuromorphic inference processor that utilizes the proposed 3D-VMM blocks as its core processing units. We have performed rigorous performance simulations of such a processor on both circuit and system levels, taking into account non-idealities such as drain-induced barrier lowering, capacitive coupling, charge injection, parasitics, process variations, and noise. Our modeling of the 3D-aCortex performing several state-of-the-art neuromorphic-network benchmarks has shown that it may provide the record-breaking storage efficiency of 4.34 MB/mm2, the peak energy efficiency of 70.43 TOps/J, and the computational throughput up to 10.66 TOps/s. The storage efficiency can be further improved seven-fold by aggressively sharing VMM peripheral circuits at the cost of slight decrease in energy efficiency and throughput. △ Less

Submitted 7 August, 2019; originally announced August 2019.

Comments: 14 pages, 9 figures, 2 tables

A Survey on the Detection of Android Malicious Apps

Authors: Sanjay K. Sahay, Ashu Sharma

Abstract: Android-based smart devices are exponentially growing, and due to the ubiquity of the Internet, these devices are globally connected to the different devices/networks. Its popularity, attractive features, and mobility make malware creator to put a number of malicious apps in the market to disrupt and annoy the victims. Although to identify the malicious apps, time-to-time various techniques are pr… ▽ More Android-based smart devices are exponentially growing, and due to the ubiquity of the Internet, these devices are globally connected to the different devices/networks. Its popularity, attractive features, and mobility make malware creator to put a number of malicious apps in the market to disrupt and annoy the victims. Although to identify the malicious apps, time-to-time various techniques are proposed. However, it appears that malware developers are always ahead of the anti-malware group, and the proposed techniques by the anti-malware groups are not sufficient to counter the advanced malicious apps. Therefore, to understand the various techniques proposed/used for the identification of Android malicious apps, in this paper, we present a survey conducted by us on the work done by the researchers in this field. △ Less

Submitted 29 May, 2019; originally announced May 2019.

Comments: Conference paper, 11 pages

Journal ref: Springer, Advances in Computer Communication and Computational Sciences, pp 437-446, 2019