Showing 151–200 of 675 results for author: Agarwal, S

Simple generic picture of toughness in solid polymer blends

Authors: Debashish Mukherji, Shubham Agarwal, Tiago Espinosa de Oliveira, Céline Ruscher, Jörg Rottler

Abstract: Toughness $\mathcal{T}$ of a brittle polymeric solid can be enhanced by blending another compatible and ductile polymer. While this common wisdom is generally valid, a generic picture is lacking that connects the atomistic details to the macroscopic non-linear mechanics. Using all-atom and complementary generic simulations we show how a delicate balance between the side group contact density of th… ▽ More Toughness $\mathcal{T}$ of a brittle polymeric solid can be enhanced by blending another compatible and ductile polymer. While this common wisdom is generally valid, a generic picture is lacking that connects the atomistic details to the macroscopic non-linear mechanics. Using all-atom and complementary generic simulations we show how a delicate balance between the side group contact density of the brittle polymers $ρ_{\rm c}$ and its dilution upon adding a second component controls $\mathcal{T}$. A broad range of systems follows a universal trend in $\mathcal{T}$ with ${\rm d}ρ_{\rm c}/{\rm d}\varepsilon$, where $\varepsilon$ is the tensile strain. The simulation data is consistent with a simple model based on the parallel spring analogy. △ Less

Submitted 26 July, 2022; originally announced July 2022.

Journal ref: Physical Review Materials 7, 115601 (2023)

Probing Semantic Grounding in Language Models of Code with Representational Similarity Analysis

Authors: Shounak Naik, Rajaswa Patil, Swati Agarwal, Veeky Baths

Abstract: Representational Similarity Analysis is a method from cognitive neuroscience, which helps in comparing representations from two different sources of data. In this paper, we propose using Representational Similarity Analysis to probe the semantic grounding in language models of code. We probe representations from the CodeBERT model for semantic grounding by using the data from the IBM CodeNet datas… ▽ More Representational Similarity Analysis is a method from cognitive neuroscience, which helps in comparing representations from two different sources of data. In this paper, we propose using Representational Similarity Analysis to probe the semantic grounding in language models of code. We probe representations from the CodeBERT model for semantic grounding by using the data from the IBM CodeNet dataset. Through our experiments, we show that current pre-training methods do not induce semantic grounding in language models of code, and instead focus on optimizing form-based patterns. We also show that even a little amount of fine-tuning on semantically relevant tasks increases the semantic grounding in CodeBERT significantly. Our ablations with the input modality to the CodeBERT model show that using bimodal inputs (code and natural language) over unimodal inputs (only code) gives better semantic grounding and sample efficiency during semantic fine-tuning. Finally, our experiments with semantic perturbations in code reveal that CodeBERT is able to robustly distinguish between semantically correct and incorrect code. △ Less

Submitted 15 July, 2022; originally announced July 2022.

Comments: Under review at ADMA 2022

NGAME: Negative Mining-aware Mini-batching for Extreme Classification

Authors: Kunal Dahiya, Nilesh Gupta, Deepak Saini, Akshay Soni, Yajun Wang, Kushal Dave, Jian Jiao, Gururaj K, Prasenjit Dey, Amit Singh, Deepesh Hada, Vidit Jain, Bhawna Paliwal, Anshul Mittal, Sonu Mehta, Ramachandran Ramjee, Sumeet Agarwal, Purushottam Kar, Manik Varma

Abstract: Extreme Classification (XC) seeks to tag data points with the most relevant subset of labels from an extremely large label set. Performing deep XC with dense, learnt representations for data points and labels has attracted much attention due to its superiority over earlier XC methods that used sparse, hand-crafted features. Negative mining techniques have emerged as a critical component of all dee… ▽ More Extreme Classification (XC) seeks to tag data points with the most relevant subset of labels from an extremely large label set. Performing deep XC with dense, learnt representations for data points and labels has attracted much attention due to its superiority over earlier XC methods that used sparse, hand-crafted features. Negative mining techniques have emerged as a critical component of all deep XC methods that allow them to scale to millions of labels. However, despite recent advances, training deep XC models with large encoder architectures such as transformers remains challenging. This paper identifies that memory overheads of popular negative mining techniques often force mini-batch sizes to remain small and slow training down. In response, this paper introduces NGAME, a light-weight mini-batch creation technique that offers provably accurate in-batch negative samples. This allows training with larger mini-batches offering significantly faster convergence and higher accuracies than existing negative sampling techniques. NGAME was found to be up to 16% more accurate than state-of-the-art methods on a wide array of benchmark datasets for extreme classification, as well as 3% more accurate at retrieving search engine queries in response to a user webpage visit to show personalized ads. In live A/B tests on a popular search engine, NGAME yielded up to 23% gains in click-through-rates. △ Less

Submitted 10 July, 2022; originally announced July 2022.

An Atlas for the Pinhole Camera

Authors: Sameer Agarwal, Timothy Duff, Max Lieblich, Rekha Thomas

Abstract: We introduce an atlas of algebro-geometric objects associated with image formation in pinhole cameras. The nodes of the atlas are algebraic varieties or their vanishing ideals related to each other by projection or elimination and restriction or specialization respectively. This atlas offers a unifying framework for the study of problems in 3D computer vision. We initiate the study of the atlas by… ▽ More We introduce an atlas of algebro-geometric objects associated with image formation in pinhole cameras. The nodes of the atlas are algebraic varieties or their vanishing ideals related to each other by projection or elimination and restriction or specialization respectively. This atlas offers a unifying framework for the study of problems in 3D computer vision. We initiate the study of the atlas by completely characterizing a part of the atlas stemming from the triangulation problem. We conclude with several open problems and generalizations of the atlas. △ Less

Submitted 3 October, 2022; v1 submitted 27 June, 2022; originally announced June 2022.

Comments: 47 pages with references and appendices, final version

MSC Class: 14Q25; 94A08

Journal ref: JFoCM, 2022

Cryptocurrency Bubble Detection: A New Stock Market Dataset, Financial Task & Hyperbolic Models

Authors: Ramit Sawhney, Shivam Agarwal, Vivek Mittal, Paolo Rosso, Vikram Nanda, Sudheer Chava

Abstract: The rapid spread of information over social media influences quantitative trading and investments. The growing popularity of speculative trading of highly volatile assets such as cryptocurrencies and meme stocks presents a fresh challenge in the financial realm. Investigating such "bubbles" - periods of sudden anomalous behavior of markets are critical in better understanding investor behavior and… ▽ More The rapid spread of information over social media influences quantitative trading and investments. The growing popularity of speculative trading of highly volatile assets such as cryptocurrencies and meme stocks presents a fresh challenge in the financial realm. Investigating such "bubbles" - periods of sudden anomalous behavior of markets are critical in better understanding investor behavior and market dynamics. However, high volatility coupled with massive volumes of chaotic social media texts, especially for underexplored assets like cryptocoins pose a challenge to existing methods. Taking the first step towards NLP for cryptocoins, we present and publicly release CryptoBubbles, a novel multi-span identification task for bubble detection, and a dataset of more than 400 cryptocoins from 9 exchanges over five years spanning over two million tweets. Further, we develop a set of sequence-to-sequence hyperbolic models suited to this multi-span identification task based on the power-law dynamics of cryptocurrencies and user behavior on social media. We further test the effectiveness of our models under zero-shot settings on a test set of Reddit posts pertaining to 29 "meme stocks", which see an increase in trade volume due to social media hype. Through quantitative, qualitative, and zero-shot analyses on Reddit and Twitter spanning cryptocoins and meme-stocks, we show the practical applicability of CryptoBubbles and hyperbolic models. △ Less

Submitted 11 May, 2022; originally announced June 2022.

Comments: Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies

Forecasting COVID- 19 cases using Statistical Models and Ontology-based Semantic Modelling: A real time data analytics approach

Authors: Sadhana Tiwari, Ritesh Chandra, Sonali Agarwal

Abstract: SARS-COV-19 is the most prominent issue which many countries face today. The frequent changes in infections, recovered and deaths represents the dynamic nature of this pandemic. It is very crucial to predict the spreading rate of this virus for accurate decision making against fighting with the situation of getting infected through the virus, tracking and controlling the virus transmission in the… ▽ More SARS-COV-19 is the most prominent issue which many countries face today. The frequent changes in infections, recovered and deaths represents the dynamic nature of this pandemic. It is very crucial to predict the spreading rate of this virus for accurate decision making against fighting with the situation of getting infected through the virus, tracking and controlling the virus transmission in the community. We develop a prediction model using statistical time series models such as SARIMA and FBProphet to monitor the daily active, recovered and death cases of COVID-19 accurately. Then with the help of various details across each individual patient (like height, weight, gender etc.), we designed a set of rules using Semantic Web Rule Language and some mathematical models for dealing with COVID19 infected cases on an individual basis. After combining all the models, a COVID-19 Ontology is developed and performs various queries using SPARQL query on designed Ontology which accumulate the risk factors, provide appropriate diagnosis, precautions and preventive suggestions for COVID Patients. After comparing the performance of SARIMA and FBProphet, it is observed that the SARIMA model performs better in forecasting of COVID cases. On individual basis COVID case prediction, approx. 497 individual samples have been tested and classified into five different levels of COVID classes such as Having COVID, No COVID, High Risk COVID case, Medium to High Risk case, and Control needed case. △ Less

Submitted 31 January, 2023; v1 submitted 6 June, 2022; originally announced June 2022.

Anomaly detection in surveillance videos using transformer based attention model

Authors: Kapil Deshpande, Narinder Singh Punn, Sanjay Kumar Sonbhadra, Sonali Agarwal

Abstract: Surveillance footage can catch a wide range of realistic anomalies. This research suggests using a weakly supervised strategy to avoid annotating anomalous segments in training videos, which is time consuming. In this approach only video level labels are used to obtain frame level anomaly scores. Weakly supervised video anomaly detection (WSVAD) suffers from the wrong identification of abnormal an… ▽ More Surveillance footage can catch a wide range of realistic anomalies. This research suggests using a weakly supervised strategy to avoid annotating anomalous segments in training videos, which is time consuming. In this approach only video level labels are used to obtain frame level anomaly scores. Weakly supervised video anomaly detection (WSVAD) suffers from the wrong identification of abnormal and normal instances during the training process. Therefore it is important to extract better quality features from the available videos. WIth this motivation, the present paper uses better quality transformer-based features named Videoswin Features followed by the attention layer based on dilated convolution and self attention to capture long and short range dependencies in temporal domain. This gives us a better understanding of available videos. The proposed framework is validated on real-world dataset i.e. ShanghaiTech Campus dataset which results in competitive performance than current state-of-the-art methods. The model and the code are available at https://github.com/kapildeshpande/Anomaly-Detection-in-Surveillance-Videos △ Less

Submitted 6 June, 2022; v1 submitted 3 June, 2022; originally announced June 2022.

Impact of the composition of feature extraction and class sampling in medicare fraud detection

Authors: Akrity Kumari, Narinder Singh Punn, Sanjay Kumar Sonbhadra, Sonali Agarwal

Abstract: With healthcare being critical aspect, health insurance has become an important scheme in minimizing medical expenses. Following this, the healthcare industry has seen a significant increase in fraudulent activities owing to increased insurance, and fraud has become a significant contributor to rising medical care expenses, although its impact can be mitigated using fraud detection techniques. To… ▽ More With healthcare being critical aspect, health insurance has become an important scheme in minimizing medical expenses. Following this, the healthcare industry has seen a significant increase in fraudulent activities owing to increased insurance, and fraud has become a significant contributor to rising medical care expenses, although its impact can be mitigated using fraud detection techniques. To detect fraud, machine learning techniques are used. The Centers for Medicaid and Medicare Services (CMS) of the United States federal government released "Medicare Part D" insurance claims is utilized in this study to develop fraud detection system. Employing machine learning algorithms on a class-imbalanced and high dimensional medicare dataset is a challenging task. To compact such challenges, the present work aims to perform feature extraction following data sampling, afterward applying various classification algorithms, to get better performance. Feature extraction is a dimensionality reduction approach that converts attributes into linear or non-linear combinations of the actual attributes, generating a smaller and more diversified set of attributes and thus reducing the dimensions. Data sampling is commonlya used to address the class imbalance either by expanding the frequency of minority class or reducing the frequency of majority class to obtain approximately equal numbers of occurrences for both classes. The proposed approach is evaluated through standard performance metrics. Thus, to detect fraud efficiently, this study applies autoencoder as a feature extraction technique, synthetic minority oversampling technique (SMOTE) as a data sampling technique, and various gradient boosted decision tree-based classifiers as a classification algorithm. The experimental results show the combination of autoencoders followed by SMOTE on the LightGBM classifier achieved best results. △ Less

Submitted 28 June, 2022; v1 submitted 3 June, 2022; originally announced June 2022.

Mechanistic framework for reduced-order models in soft materials: Application to three-dimensional granular intrusion

Authors: Shashank Agarwal, Daniel I Goldman, Ken Kamrin

Abstract: Soft materials often display complex behaviors that transition through apparent solid- and fluid-like regimes. While a growing number of microscale simulation methods exist for these materials, reduced-order models that encapsulate the global-scale physics are often desired to predict how external bodies interact with soft media, as occurs in diverse situations from impact and penetration problems… ▽ More Soft materials often display complex behaviors that transition through apparent solid- and fluid-like regimes. While a growing number of microscale simulation methods exist for these materials, reduced-order models that encapsulate the global-scale physics are often desired to predict how external bodies interact with soft media, as occurs in diverse situations from impact and penetration problems to locomotion over natural terrains. This work proposes a systematic program to develop three-dimensional reduced-order models for soft materials from a fundamental basis using continuum symmetries and rheological principles. In particular, we derive a reduced-order technique for modeling intrusion in granular media which we term three-dimensional Resistive Force Theory (3D-RFT), which is capable of accurately and quickly predicting the resistive stress distribution on arbitrary-shaped intruding bodies. Aided by a continuum description of the granular medium, a comprehensive set of spatial symmetry constraints, and a limited amount of reference data, we develop a self-consistent and accurate 3D-RFT. We verify the model capabilities in a wide range of cases and show it can be quickly recalibrated to different media and intruder surface types. The premises leading to 3D-RFT anticipate application to other soft materials with strongly hyperlocalized intrusion behavior. △ Less

Submitted 10 December, 2022; v1 submitted 30 May, 2022; originally announced May 2022.

Comments: 12 pages, 7 figures, 1 SI document (12 pages, 8 figures, and 4 tables)

AdaMix: Mixture-of-Adaptations for Parameter-efficient Model Tuning

Authors: Yaqing Wang, Sahaj Agarwal, Subhabrata Mukherjee, Xiaodong Liu, **g Gao, Ahmed Hassan Awadallah, Jianfeng Gao

Abstract: Standard fine-tuning of large pre-trained language models (PLMs) for downstream tasks requires updating hundreds of millions to billions of parameters, and storing a large copy of the PLM weights for every task resulting in increased cost for storing, sharing and serving the models. To address this, parameter-efficient fine-tuning (PEFT) techniques were introduced where small trainable components… ▽ More Standard fine-tuning of large pre-trained language models (PLMs) for downstream tasks requires updating hundreds of millions to billions of parameters, and storing a large copy of the PLM weights for every task resulting in increased cost for storing, sharing and serving the models. To address this, parameter-efficient fine-tuning (PEFT) techniques were introduced where small trainable components are injected in the PLM and updated during fine-tuning. We propose AdaMix as a general PEFT method that tunes a mixture of adaptation modules -- given the underlying PEFT method of choice -- introduced in each Transformer layer while kee** most of the PLM weights frozen. For instance, AdaMix can leverage a mixture of adapters like Houlsby or a mixture of low rank decomposition matrices like LoRA to improve downstream task performance over the corresponding PEFT methods for fully supervised and few-shot NLU and NLG tasks. Further, we design AdaMix such that it matches the same computational cost and the number of tunable parameters as the underlying PEFT method. By only tuning 0.1-0.2% of PLM parameters, we show that AdaMix outperforms SOTA parameter-efficient fine-tuning and full model fine-tuning for both NLU and NLG tasks. △ Less

Submitted 1 November, 2022; v1 submitted 24 May, 2022; originally announced May 2022.

Comments: Accepted by EMNLP 2022

Investigating the concentration of High Yield Investment Programs in the United Kingdom

Authors: Sharad Agarwal, Marie Vasek

Abstract: Ponzi schemes that offer absurdly high rates of return by relying on more and more people paying into the scheme have been documented since at least the mid-1800s. Ponzi schemes have shifted online in the Internet age, and some are re-branded as HYIPs or High Yield Investment Programs. This paper focuses on understanding HYIPs' continuous presence and presents various possible reasons behind their… ▽ More Ponzi schemes that offer absurdly high rates of return by relying on more and more people paying into the scheme have been documented since at least the mid-1800s. Ponzi schemes have shifted online in the Internet age, and some are re-branded as HYIPs or High Yield Investment Programs. This paper focuses on understanding HYIPs' continuous presence and presents various possible reasons behind their existence in today's world. A look into the countries where these schemes purport to exist, we find that 62.89% of all collected HYIPs claim to be in the United Kingdom (UK), and a further 55.56% are officially registered in the UK as a 'limited company' with a registration number provided by the UK Companies House, a UK agency that registers companies. We investigate other factors influencing these schemes, including the HYIPs' social media platforms and payment processors. The lifetime of the HYIPs helps to understand the success/failure of the investment schemes and helps indicate the schemes that could attract more investors. Using Cox proportional regression analysis, we find that having a valid UK address significantly affects the lifetime of an HYIP. △ Less

Submitted 21 April, 2022; originally announced May 2022.

Horizons: Nuclear Astrophysics in the 2020s and Beyond

Authors: H. Schatz, A. D. Becerril Reyes, A. Best, E. F. Brown, K. Chatziioannou, K. A. Chipps, C. M. Deibel, R. Ezzeddine, D. K. Galloway, C. J. Hansen, F. Herwig, A. P. Ji, M. Lugaro, Z. Meisel, D. Norman, J. S. Read, L. F. Roberts, A. Spyrou, I. Tews, F. X. Timmes, C. Travaglio, N. Vassh, C. Abia, P. Adsley, S. Agarwal , et al. (140 additional authors not shown)

Abstract: Nuclear Astrophysics is a field at the intersection of nuclear physics and astrophysics, which seeks to understand the nuclear engines of astronomical objects and the origin of the chemical elements. This white paper summarizes progress and status of the field, the new open questions that have emerged, and the tremendous scientific opportunities that have opened up with major advances in capabilit… ▽ More Nuclear Astrophysics is a field at the intersection of nuclear physics and astrophysics, which seeks to understand the nuclear engines of astronomical objects and the origin of the chemical elements. This white paper summarizes progress and status of the field, the new open questions that have emerged, and the tremendous scientific opportunities that have opened up with major advances in capabilities across an ever growing number of disciplines and subfields that need to be integrated. We take a holistic view of the field discussing the unique challenges and opportunities in nuclear astrophysics in regards to science, diversity, education, and the interdisciplinarity and breadth of the field. Clearly nuclear astrophysics is a dynamic field with a bright future that is entering a new era of discovery opportunities. △ Less

Submitted 16 May, 2022; originally announced May 2022.

Comments: 96 pages. Submitted to Journal of Physics G

Report number: LA-UR-22-23997

First Order Topological Phase Transitions and Disorder Induced Majorana Modes in Interacting Fermion Chains

Authors: Shruti Agarwal, Shreekant Gawande, Satoshi Nishimoto, Jeroen van den Brink, Sanjeev Kumar

Abstract: Using a combination of the mean-field Bogoliubov deGennes (BdG) approach and the Density Matrix Renormalization Group (DMRG) method, we discover first order topological transitions between topological superconducting and trivial insulating phases in a sawtooth lattice of inter-site attractive fermions. Topological characterization of different phases is achieved in terms of winding numbers, Majora… ▽ More Using a combination of the mean-field Bogoliubov deGennes (BdG) approach and the Density Matrix Renormalization Group (DMRG) method, we discover first order topological transitions between topological superconducting and trivial insulating phases in a sawtooth lattice of inter-site attractive fermions. Topological characterization of different phases is achieved in terms of winding numbers, Majorana edge modes and entanglement spectra. By studying the effect of disorder on the first order topological phase transitions, we establish the disorder-induced topological phase coexistence as a mechanism for generating a finite density of Majorana particles. △ Less

Submitted 13 April, 2022; originally announced April 2022.

Comments: Supplemental material included

Synthesizing Adversarial Visual Scenarios for Model-Based Robotic Control

Authors: Shubhankar Agarwal, Sandeep P. Chinchali

Abstract: Today's robots often interface with data-driven perception and planning models with classical model-predictive controllers (MPC). Often, such learned perception/planning models produce erroneous waypoint predictions on out-of-distribution (OoD) or even adversarial visual inputs, which increase control costs. However, today's methods to train robust perception models are largely task-agnostic - the… ▽ More Today's robots often interface with data-driven perception and planning models with classical model-predictive controllers (MPC). Often, such learned perception/planning models produce erroneous waypoint predictions on out-of-distribution (OoD) or even adversarial visual inputs, which increase control costs. However, today's methods to train robust perception models are largely task-agnostic - they augment a dataset using random image transformations or adversarial examples targeted at the vision model in isolation. As such, they often introduce pixel perturbations that are ultimately benign for control. In contrast to prior work that synthesizes adversarial examples for single-step vision tasks, our key contribution is to synthesize adversarial scenarios tailored to multi-step, model-based control. To do so, we use differentiable MPC methods to calculate the sensitivity of a model-based controller to errors in state estimation. We show that re-training vision models on these adversarial datasets improves control performance on OoD test scenarios by up to 36.2% compared to standard task-agnostic data augmentation. We demonstrate our method on examples of robotic navigation, manipulation in RoboSuite, and control of an autonomous air vehicle. △ Less

Submitted 2 December, 2022; v1 submitted 13 April, 2022; originally announced April 2022.

Comments: Conference on Robot Learning, 2022

"That Is a Suspicious Reaction!": Interpreting Logits Variation to Detect NLP Adversarial Attacks

Authors: Edoardo Mosca, Shreyash Agarwal, Javier Rando, Georg Groh

Abstract: Adversarial attacks are a major challenge faced by current machine learning research. These purposely crafted inputs fool even the most advanced models, precluding their deployment in safety-critical applications. Extensive research in computer vision has been carried to develop reliable defense strategies. However, the same issue remains less explored in natural language processing. Our work pres… ▽ More Adversarial attacks are a major challenge faced by current machine learning research. These purposely crafted inputs fool even the most advanced models, precluding their deployment in safety-critical applications. Extensive research in computer vision has been carried to develop reliable defense strategies. However, the same issue remains less explored in natural language processing. Our work presents a model-agnostic detector of adversarial text examples. The approach identifies patterns in the logits of the target classifier when perturbing the input text. The proposed detector improves the current state-of-the-art performance in recognizing adversarial inputs and exhibits strong generalization capabilities across different NLP models, datasets, and word-level attacks. △ Less

Submitted 29 June, 2023; v1 submitted 10 April, 2022; originally announced April 2022.

Comments: ACL 2022

An optimized hybrid solution for IoT based lifestyle disease classification using stress data

Authors: Sadhana Tiwari, Sonali Agarwal

Abstract: Stress, anxiety, and nervousness are all high-risk health states in everyday life. Previously, stress levels were determined by speaking with people and gaining insight into what they had experienced recently or in the past. Typically, stress is caused by an incidence that occurred a long time ago, but sometimes it is triggered by unknown factors. This is a challenging and complex task, but recent… ▽ More Stress, anxiety, and nervousness are all high-risk health states in everyday life. Previously, stress levels were determined by speaking with people and gaining insight into what they had experienced recently or in the past. Typically, stress is caused by an incidence that occurred a long time ago, but sometimes it is triggered by unknown factors. This is a challenging and complex task, but recent research advances have provided numerous opportunities to automate it. The fundamental features of most of these techniques are electro dermal activity (EDA) and heart rate values (HRV). We utilized an accelerometer to measure body motions to solve this challenge. The proposed novel method employs a test that measures a subject's electrocardiogram (ECG), galvanic skin values (GSV), HRV values, and body movements in order to provide a low-cost and time-saving solution for detecting stress lifestyle disease in modern times using cyber physical systems. This study provides a new hybrid model for lifestyle disease classification that decreases execution time while picking the best collection of characteristics and increases classification accuracy. The developed approach is capable of dealing with the class imbalance problem by using WESAD (wearable stress and affect dataset) dataset. The new model uses the Grid search (GS) method to select an optimized set of hyper parameters, and it uses a combination of the Correlation coefficient based Recursive feature elimination (CoC-RFE) method for optimal feature selection and gradient boosting as an estimator to classify the dataset, which achieves high accuracy and helps to provide smart, accurate, and high-quality healthcare systems. To demonstrate the validity and utility of the proposed methodology, its performance is compared to those of other well-established machine learning models. △ Less

Submitted 4 April, 2022; originally announced April 2022.

Comments: Data mining and Data analytics used for healthcare data

Semantic Sensor Network Ontology based Decision Support System for Forest Fire Management

Authors: Ritesh Chandra, Kumar Abhishek, Sonali Agarwal, Navjot Singh

Abstract: The forests are significant assets for every country. When it gets destroyed, it may negatively impact the environment, and forest fire is one of the primary causes. Fire weather indices are widely used to measure fire danger and are used to issue bushfire warnings. It can also be used to predict the demand for emergency management resources. Sensor networks have grown in popularity in data collec… ▽ More The forests are significant assets for every country. When it gets destroyed, it may negatively impact the environment, and forest fire is one of the primary causes. Fire weather indices are widely used to measure fire danger and are used to issue bushfire warnings. It can also be used to predict the demand for emergency management resources. Sensor networks have grown in popularity in data collection and processing capabilities for a variety of applications in industries such as medical, environmental monitoring, home automation etc. Semantic sensor networks can collect various climatic circumstances like wind speed, temperature, and relative humidity. However, estimating fire weather indices is challenging due to the various issues involved in processing the data streams generated by the sensors. Hence, the importance of forest fire detection has increased day by day. The underlying Semantic Sensor Network (SSN) ontologies are built to allow developers to create rules for calculating fire weather indices and also the convert dataset into Resource Description Framework (RDF). This research describes the various steps involved in develo** rules for calculating fire weather indices. Besides, this work presents a Web-based map** interface to help users visualize the changes in fire weather indices over time. With the help of the inference rule, it designed a decision support system using the SSN ontology and query on it through SPARQL. The proposed fire management system acts according to the situation, supports reasoning and the general semantics of the open-world followed by all the ontologies △ Less

Submitted 13 July, 2022; v1 submitted 3 April, 2022; originally announced April 2022.

Comments: Ontology and Semantic Modeling

Empirical Analysis of Lifelog Data using Optimal Feature Selection based Unsupervised Logistic Regression (OFS-ULR) Model with Spark Streaming

Authors: Sadhana Tiwari, Sonali Agarwal

Abstract: Recent advancement in the field of pervasive healthcare monitoring systems causes the generation of a huge amount of lifelog data in real-time. Chronic diseases are one of the most serious health challenges in develo** and developed countries. According to WHO, this accounts for 73% of all deaths and 60% of the global burden of diseases. Chronic disease classification models are now harnessing t… ▽ More Recent advancement in the field of pervasive healthcare monitoring systems causes the generation of a huge amount of lifelog data in real-time. Chronic diseases are one of the most serious health challenges in develo** and developed countries. According to WHO, this accounts for 73% of all deaths and 60% of the global burden of diseases. Chronic disease classification models are now harnessing the potential of lifelog data to explore better healthcare practices. This paper is to construct an optimal feature selection-based unsupervised logistic regression model (OFS-ULR) to classify chronic diseases. Since lifelog data analysis is crucial due to its sensitive nature; thus the conventional classification models show limited performance. Therefore, designing new classifiers for the classification of chronic diseases using lifelog data is the need of the age. The vital part of building a good model depends on pre-processing of the dataset, identifying important features, and then training a learning algorithm with suitable hyper parameters for better performance. The proposed approach improves the performance of existing methods using a series of steps such as (i) removing redundant or invalid instances, (ii) making the data labelled using clustering and partitioning the data into classes, (iii) identifying the suitable subset of features by applying either some domain knowledge or selection algorithm, (iv) hyper parameter tuning for models to get best results, and (v) performance evaluation using Spark streaming environment. For this purpose, two-time series datasets are used in the experiment to compute the accuracy, recall, precision, and f1-score. The experimental analysis proves the suitability of the proposed approach as compared to the conventional classifiers and our newly constructed model achieved highest accuracy and reduced training complexity among all among all. △ Less

Submitted 12 April, 2022; v1 submitted 4 April, 2022; originally announced April 2022.

Comments: Data analytics and Machine learning in healthcare

PublicCheck: Public Integrity Verification for Services of Run-time Deep Models

Authors: Shuo Wang, Sharif Abuadbba, Sidharth Agarwal, Kristen Moore, Ruoxi Sun, Minhui Xue, Surya Nepal, Seyit Camtepe, Salil Kanhere

Abstract: Existing integrity verification approaches for deep models are designed for private verification (i.e., assuming the service provider is honest, with white-box access to model parameters). However, private verification approaches do not allow model users to verify the model at run-time. Instead, they must trust the service provider, who may tamper with the verification results. In contrast, a publ… ▽ More Existing integrity verification approaches for deep models are designed for private verification (i.e., assuming the service provider is honest, with white-box access to model parameters). However, private verification approaches do not allow model users to verify the model at run-time. Instead, they must trust the service provider, who may tamper with the verification results. In contrast, a public verification approach that considers the possibility of dishonest service providers can benefit a wider range of users. In this paper, we propose PublicCheck, a practical public integrity verification solution for services of run-time deep models. PublicCheck considers dishonest service providers, and overcomes public verification challenges of being lightweight, providing anti-counterfeiting protection, and having fingerprinting samples that appear smooth. To capture and fingerprint the inherent prediction behaviors of a run-time model, PublicCheck generates smoothly transformed and augmented encysted samples that are enclosed around the model's decision boundary while ensuring that the verification queries are indistinguishable from normal queries. PublicCheck is also applicable when knowledge of the target model is limited (e.g., with no knowledge of gradients or model parameters). A thorough evaluation of PublicCheck demonstrates the strong capability for model integrity breach detection (100% detection accuracy with less than 10 black-box API queries) against various model integrity attacks and model compression attacks. PublicCheck also demonstrates the smooth appearance, feasibility, and efficiency of generating a plethora of encysted samples for fingerprinting. △ Less

Submitted 19 December, 2022; v1 submitted 21 March, 2022; originally announced March 2022.

Comments: 18 pages, 9 figures. Accepted to IEEE S&P 2023

Parametric Scaling of Preprocessing assisted U-net Architecture for Improvised Retinal Vessel Segmentation

Authors: Kundan Kumar, Sumanshu Agarwal

Abstract: Extracting blood vessels from retinal fundus images plays a decisive role in diagnosing the progression in pertinent diseases. In medical image analysis, vessel extraction is a semantic binary segmentation problem, where blood vasculature needs to be extracted from the background. Here, we present an image enhancement technique based on the morphological preprocessing coupled with a scaled U-net a… ▽ More Extracting blood vessels from retinal fundus images plays a decisive role in diagnosing the progression in pertinent diseases. In medical image analysis, vessel extraction is a semantic binary segmentation problem, where blood vasculature needs to be extracted from the background. Here, we present an image enhancement technique based on the morphological preprocessing coupled with a scaled U-net architecture. Despite a relatively less number of trainable network parameters, the scaled version of U-net architecture provides better performance compare to other methods in the domain. We validated the proposed method on retinal fundus images from the DRIVE database. A significant improvement as compared to the other algorithms in the domain, in terms of the area under ROC curve (>0.9762) and classification accuracy (>95.47%) are evident from the results. Furthermore, the proposed method is resistant to the central vessel reflex while sensitive to detect blood vessels in the presence of background items viz. exudates, optic disc, and fovea. △ Less

Submitted 18 March, 2022; originally announced March 2022.

Comments: 10 pages, 5 figures, ICAIHC-2022

Application of Top-hat Transformation for Enhanced Blood Vessel Extraction

Authors: Tithi Parna Das, Sheetal Praharaj, Sarita Swain, Sumanshu Agarwal, Kundan Kumar

Abstract: In the medical domain, different computer-aided diagnosis systems have been proposed to extract blood vessels from retinal fundus images for the clinical treatment of vascular diseases. Accurate extraction of blood vessels from the fundus images using a computer-generated method can help the clinician to produce timely and accurate reports for the patient suffering from these diseases. In this art… ▽ More In the medical domain, different computer-aided diagnosis systems have been proposed to extract blood vessels from retinal fundus images for the clinical treatment of vascular diseases. Accurate extraction of blood vessels from the fundus images using a computer-generated method can help the clinician to produce timely and accurate reports for the patient suffering from these diseases. In this article, we integrate top-hat based preprocessing approach with fine-tuned B-COSFIRE filter to achieve more accurate segregation of blood vessel pixels from the background. The use of top-hat transformation in the preprocessing stage enhances the efficacy of the algorithm to extract blood vessels in presence of structures like fovea, exudates, haemorrhages, etc. Furthermore, to reduce the false positives, small clusters of blood vessel pixels are removed in the postprocessing stage. Further, we find that the proposed algorithm is more efficient as compared to various modern algorithms reported in the literature. △ Less

Submitted 18 March, 2022; originally announced March 2022.

Comments: 9 pages, 3 figures, ICAIHC-2022

Training language models to follow instructions with human feedback

Authors: Long Ouyang, Jeff Wu, Xu Jiang, Diogo Almeida, Carroll L. Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, John Schulman, Jacob Hilton, Fraser Kelton, Luke Miller, Maddie Simens, Amanda Askell, Peter Welinder, Paul Christiano, Jan Leike, Ryan Lowe

Abstract: Making language models bigger does not inherently make them better at following a user's intent. For example, large language models can generate outputs that are untruthful, toxic, or simply not helpful to the user. In other words, these models are not aligned with their users. In this paper, we show an avenue for aligning language models with user intent on a wide range of tasks by fine-tuning wi… ▽ More Making language models bigger does not inherently make them better at following a user's intent. For example, large language models can generate outputs that are untruthful, toxic, or simply not helpful to the user. In other words, these models are not aligned with their users. In this paper, we show an avenue for aligning language models with user intent on a wide range of tasks by fine-tuning with human feedback. Starting with a set of labeler-written prompts and prompts submitted through the OpenAI API, we collect a dataset of labeler demonstrations of the desired model behavior, which we use to fine-tune GPT-3 using supervised learning. We then collect a dataset of rankings of model outputs, which we use to further fine-tune this supervised model using reinforcement learning from human feedback. We call the resulting models InstructGPT. In human evaluations on our prompt distribution, outputs from the 1.3B parameter InstructGPT model are preferred to outputs from the 175B GPT-3, despite having 100x fewer parameters. Moreover, InstructGPT models show improvements in truthfulness and reductions in toxic output generation while having minimal performance regressions on public NLP datasets. Even though InstructGPT still makes simple mistakes, our results show that fine-tuning with human feedback is a promising direction for aligning language models with human intent. △ Less

Submitted 4 March, 2022; originally announced March 2022.

BagPipe: Accelerating Deep Recommendation Model Training

Authors: Saurabh Agarwal, Chengpo Yan, Ziyi Zhang, Shivaram Venkataraman

Abstract: Deep learning based recommendation models (DLRM) are widely used in several business critical applications. Training such recommendation models efficiently is challenging because they contain billions of embedding-based parameters, leading to significant overheads from embedding access. By profiling existing systems for DLRM training, we observe that around 75\% of the iteration time is spent on e… ▽ More Deep learning based recommendation models (DLRM) are widely used in several business critical applications. Training such recommendation models efficiently is challenging because they contain billions of embedding-based parameters, leading to significant overheads from embedding access. By profiling existing systems for DLRM training, we observe that around 75\% of the iteration time is spent on embedding access and model synchronization. Our key insight in this paper is that embedding access has a specific structure which can be used to accelerate training. We observe that embedding accesses are heavily skewed, with around 1\% of embeddings representing more than 92\% of total accesses. Further, we observe that during offline training we can lookahead at future batches to determine exactly which embeddings will be needed at what iteration in the future. Based on these insights, we develop Bagpipe, a system for training deep recommendation models that uses caching and prefetching to overlap remote embedding accesses with the computation. We design an Oracle Cacher, a new component that uses a lookahead algorithm to generate optimal cache update decisions while providing strong consistency guarantees against staleness. We also design a logically replicated, physically partitioned cache and show that our design can reduce synchronization overheads in a distributed setting. Finally, we propose a disaggregated system architecture and show that our design can enable low-overhead fault tolerance. Our experiments using three datasets and four models show that Bagpipe provides a speed up of up to 5.6x compared to state of the art baselines, while providing the same convergence and reproducibility guarantees as synchronous training. △ Less

Submitted 1 November, 2023; v1 submitted 24 February, 2022; originally announced February 2022.

Can Open Domain Question Answering Systems Answer Visual Knowledge Questions?

Authors: Jiawen Zhang, Abhijit Mishra, Avinesh P. V. S, Siddharth Patwardhan, Sachin Agarwal

Abstract: The task of Outside Knowledge Visual Question Answering (OKVQA) requires an automatic system to answer natural language questions about pictures and images using external knowledge. We observe that many visual questions, which contain deictic referential phrases referring to entities in the image, can be rewritten as "non-grounded" questions and can be answered by existing text-based question answ… ▽ More The task of Outside Knowledge Visual Question Answering (OKVQA) requires an automatic system to answer natural language questions about pictures and images using external knowledge. We observe that many visual questions, which contain deictic referential phrases referring to entities in the image, can be rewritten as "non-grounded" questions and can be answered by existing text-based question answering systems. This allows for the reuse of existing text-based Open Domain Question Answering (QA) Systems for visual question answering. In this work, we propose a potentially data-efficient approach that reuses existing systems for (a) image analysis, (b) question rewriting, and (c) text-based question answering to answer such visual questions. Given an image and a question pertaining to that image (a visual question), we first extract the entities present in the image using pre-trained object and scene classifiers. Using these detected entities, the visual questions can be rewritten so as to be answerable by open domain QA systems. We explore two rewriting strategies: (1) an unsupervised method using BERT for masking and rewriting, and (2) a weakly supervised approach that combines adaptive rewriting and reinforcement learning techniques to use the implicit feedback from the QA system. We test our strategies on the publicly available OKVQA dataset and obtain a competitive performance with state-of-the-art models while using only 10% of the training data. △ Less

Submitted 9 February, 2022; originally announced February 2022.

Comments: 9 pages (including references), 5 figures

An isolated mass gap black hole or neutron star detected with astrometric microlensing

Authors: Casey Y. Lam, Jessica R. Lu, Andrzej Udalski, Ian Bond, David P. Bennett, Jan Skowron, Przemek Mroz, Radek Poleski, Takahiro Sumi, Michal K. Szymanski, Szymon Kozlowski, Pawel Pietrukowicz, Igor Soszynski, Krzysztof Ulaczyk, Lukasz Wyrzykowski, Shota Miyazaki, Daisuke Suzuki, Naoki Koshimoto, Nicholas J. Rattenbury, Matthew W. Hosek Jr., Fumio Abe, Richard Barry, Aparna Bhattacharya, Akihiko Fukui, Hirosane Fujii , et al. (20 additional authors not shown)

Abstract: We present the analysis of five black hole candidates identified from gravitational microlensing surveys. Hubble Space Telescope astrometric data and densely sampled lightcurves from ground-based microlensing surveys are fit with a single-source, single-lens microlensing model in order to measure the mass and luminosity of each lens and determine if it is a black hole. One of the five targets (OGL… ▽ More We present the analysis of five black hole candidates identified from gravitational microlensing surveys. Hubble Space Telescope astrometric data and densely sampled lightcurves from ground-based microlensing surveys are fit with a single-source, single-lens microlensing model in order to measure the mass and luminosity of each lens and determine if it is a black hole. One of the five targets (OGLE-2011-BLG-0462/MOA-2011-BLG-191 or OB110462 for short) shows a significant $>1$ mas coherent astrometric shift, little to no lens flux, and has an inferred lens mass of 1.6 - 4.4 $M_\odot$. This makes OB110462 the first definitive discovery of a compact object through astrometric microlensing and it is most likely either a neutron star or a low-mass black hole. This compact object lens is relatively nearby (0.70-1.92 kpc) and has a slow transverse motion of $<$30 km/s. OB110462 shows significant tension between models well-fit to photometry vs. astrometry, making it currently difficult to distinguish between a neutron star and a black hole. Additional observations and modeling with more complex system geometries, such as binary sources are needed to resolve the puzzling nature of this object. For the remaining four candidates, the lens masses are $<2 M_\odot$ and they are unlikely to be black holes; two of the four are likely white dwarfs or neutron stars. We compare the full sample of five candidates to theoretical expectations on the number of black holes in the Milky Way ($\sim 10^8$) and find reasonable agreement given the small sample size. △ Less

Submitted 31 May, 2022; v1 submitted 3 February, 2022; originally announced February 2022.

Comments: Accepted to ApJ Letters, with corresponding ApJ Supplement. 10 page Letter (6 figures, 2 tables) + 51 page Supplement (27 figures, 20 tables, 9 appendices). Some minor updates from the refereeing process, but no change to main conclusions

A Deep Learning Approach To Estimation Using Measurements Received Over a Network

Authors: Shivangi Agarwal, Sanjit K. Kaul, Saket Anand, P. B. Sujit

Abstract: We propose a novel deep neural network (DNN) based approximation architecture to learn estimates of measurements. We detail an algorithm that enables training of the DNN. The DNN estimator only uses measurements, if and when they are received over a communication network. The measurements are communicated over a network as packets, at a rate unknown to the estimator. Packets may suffer drops and n… ▽ More We propose a novel deep neural network (DNN) based approximation architecture to learn estimates of measurements. We detail an algorithm that enables training of the DNN. The DNN estimator only uses measurements, if and when they are received over a communication network. The measurements are communicated over a network as packets, at a rate unknown to the estimator. Packets may suffer drops and need retransmission. They may suffer waiting delays as they traverse a network path. Works on estimation often assume knowledge of the dynamic model of the measured system, which may not be available in practice. The DNN estimator doesn't assume knowledge of the dynamic system model or the communication network. It doesn't require a history of measurements, often used by other works. The DNN estimator results in significantly smaller average estimation error than the commonly used Time-varying Kalman Filter and the Unscented Kalman Filter, in simulations of linear and nonlinear dynamic systems. The DNN need not be trained separately for different communications network settings. It is robust to errors in estimation of network delays that occur due to imperfect time synchronization between the measurement source and the estimator. Last but not the least, our simulations shed light on the rate of updates that result in low estimation error. △ Less

Submitted 12 September, 2022; v1 submitted 20 January, 2022; originally announced January 2022.

Model Stability with Continuous Data Updates

Authors: Huiting Liu, Avinesh P. V. S., Siddharth Patwardhan, Peter Grasch, Sachin Agarwal

Abstract: In this paper, we study the "stability" of machine learning (ML) models within the context of larger, complex NLP systems with continuous training data updates. For this study, we propose a methodology for the assessment of model stability (which we refer to as jitter under various experimental conditions. We find that model design choices, including network architecture and input representation,… ▽ More In this paper, we study the "stability" of machine learning (ML) models within the context of larger, complex NLP systems with continuous training data updates. For this study, we propose a methodology for the assessment of model stability (which we refer to as jitter under various experimental conditions. We find that model design choices, including network architecture and input representation, have a critical impact on stability through experiments on four text classification tasks and two sequence labeling tasks. In classification tasks, non-RNN-based models are observed to be more stable than RNN-based ones, while the encoder-decoder model is less stable in sequence labeling tasks. Moreover, input representations based on pre-trained fastText embeddings contribute to more stability than other choices. We also show that two learning strategies -- ensemble models and incremental training -- have a significant influence on stability. We recommend ML model designers account for trade-offs in accuracy and jitter when making modeling choices. △ Less

Submitted 14 January, 2022; originally announced January 2022.

Cavity mediated level attraction and repulsion between magnons

Authors: Jayakrishnan M. P. Nair, Debsuvra Mukhopadhyay, Girish S. Agarwal

Abstract: We characterize some of the distinctive hallmarks of magnon-magnon interaction mediated by the intracavity field of a microwave cavity, along with their testable ramifications. In general, we foreground two widely dissimilar parameter domains that bring forth the contrasting possibilities of level splitting and level crossing. The former is observed in the regime of strong magnon-photon couplings,… ▽ More We characterize some of the distinctive hallmarks of magnon-magnon interaction mediated by the intracavity field of a microwave cavity, along with their testable ramifications. In general, we foreground two widely dissimilar parameter domains that bring forth the contrasting possibilities of level splitting and level crossing. The former is observed in the regime of strong magnon-photon couplings, particularly when the three modes bear comparable relaxation rates. This character is marked by the appearance of three distinguishable and non-converging polariton branches in the spectral response to a cavity drive. However, when the bare modes are resonant and the couplings perfectly symmetrical, one of the spectral peaks gets wiped out. This anomalous extinction of polaritonic response can be traced down to the existence of a conspicuous dark mode alongside two frequency-shifted bright modes. In an alternate parameter regime, where the magnon modes are weakly coupled to the cavity, features of level attraction unfold, subject to a large relaxation rate for the cavity mode. Concurrently, for antisymmetric detunings to the magnon modes, a transmission window springs into existence, exhibiting transparency in the limit of negligible dissipation from the magnons. The emergence of level attraction can be reconciled with a theoretical model that embodies the dynamics of the magnon-magnon subsystem when the cavity field decays rapidly into its steady state. In this limit, we identify a purely dissipative coupling between the magnon modes. △ Less

Submitted 14 January, 2022; originally announced January 2022.

Journal ref: Phys. Rev. B 105, 214418 (2022)

Conserved quantities in non-Hermitian systems via vectorization method

Authors: Kaustubh S. Agarwal, Jacob Muldoon, Yogesh N. Joglekar

Abstract: Open classical and quantum systems have attracted great interest in the past two decades. These include systems described by non-Hermitian Hamiltonians with parity-time $(\mathcal{PT})$ symmetry that are best understood as systems with balanced, separated gain and loss. Here, we present an alternative way to characterize and derive conserved quantities, or intertwining operators, in such open syst… ▽ More Open classical and quantum systems have attracted great interest in the past two decades. These include systems described by non-Hermitian Hamiltonians with parity-time $(\mathcal{PT})$ symmetry that are best understood as systems with balanced, separated gain and loss. Here, we present an alternative way to characterize and derive conserved quantities, or intertwining operators, in such open systems. As a consequence, we also obtain non-Hermitian or Hermitian operators whose expectations values show single exponential time dependence. By using a simple example of a $\mathcal{PT}$-symmetric dimer that arises in two distinct physical realizations, we demonstrate our procedure for static Hamiltonians and generalize it to time-periodic (Floquet) cases where intertwining operators are stroboscopically conserved. Inspired by the Lindblad density matrix equation, our approach provides a useful addition to the well-established methods for characterizing time-invariants in non-Hermitian systems. △ Less

Submitted 13 January, 2022; originally announced January 2022.

Comments: 7 pages, 2 figure: Proceedings of AAMP XVIII (Prague 2021)

Journal ref: Acta Polytechnica 62, 1 (2022)

Watch Those Words: Video Falsification Detection Using Word-Conditioned Facial Motion

Authors: Shruti Agarwal, Liwen Hu, Evonne Ng, Trevor Darrell, Hao Li, Anna Rohrbach

Abstract: In today's era of digital misinformation, we are increasingly faced with new threats posed by video falsification techniques. Such falsifications range from cheapfakes (e.g., lookalikes or audio dubbing) to deepfakes (e.g., sophisticated AI media synthesis methods), which are becoming perceptually indistinguishable from real videos. To tackle this challenge, we propose a multi-modal semantic foren… ▽ More In today's era of digital misinformation, we are increasingly faced with new threats posed by video falsification techniques. Such falsifications range from cheapfakes (e.g., lookalikes or audio dubbing) to deepfakes (e.g., sophisticated AI media synthesis methods), which are becoming perceptually indistinguishable from real videos. To tackle this challenge, we propose a multi-modal semantic forensic approach to discover clues that go beyond detecting discrepancies in visual quality, thereby handling both simpler cheapfakes and visually persuasive deepfakes. In this work, our goal is to verify that the purported person seen in the video is indeed themselves by detecting anomalous facial movements corresponding to the spoken words. We leverage the idea of attribution to learn person-specific biometric patterns that distinguish a given speaker from others. We use interpretable Action Units (AUs) to capture a person's face and head movement as opposed to deep CNN features, and we are the first to use word-conditioned facial motion analysis. We further demonstrate our method's effectiveness on a range of fakes not seen in training including those without video manipulation, that were not addressed in prior work. △ Less

Submitted 1 December, 2022; v1 submitted 20 December, 2021; originally announced December 2021.

Comments: Accepted in WACV 2023

BT-Unet: A self-supervised learning framework for biomedical image segmentation using Barlow Twins with U-Net models

Authors: Narinder Singh Punn, Sonali Agarwal

Abstract: Deep learning has brought the most profound contribution towards biomedical image segmentation to automate the process of delineation in medical imaging. To accomplish such task, the models are required to be trained using huge amount of annotated or labelled data that highlights the region of interest with a binary mask. However, efficient generation of the annotations for such huge data requires… ▽ More Deep learning has brought the most profound contribution towards biomedical image segmentation to automate the process of delineation in medical imaging. To accomplish such task, the models are required to be trained using huge amount of annotated or labelled data that highlights the region of interest with a binary mask. However, efficient generation of the annotations for such huge data requires expert biomedical analysts and extensive manual effort. It is a tedious and expensive task, while also being vulnerable to human error. To address this problem, a self-supervised learning framework, BT-Unet is proposed that uses the Barlow Twins approach to pre-train the encoder of a U-Net model via redundancy reduction in an unsupervised manner to learn data representation. Later, complete network is fine-tuned to perform actual segmentation. The BT-Unet framework can be trained with a limited number of annotated samples while having high number of unannotated samples, which is mostly the case in real-world problems. This framework is validated over multiple U-Net models over diverse datasets by generating scenarios of a limited number of labelled samples using standard evaluation metrics. With exhaustive experiment trials, it is observed that the BT-Unet framework enhances the performance of the U-Net models with significant margin under such circumstances. △ Less

Submitted 23 March, 2022; v1 submitted 7 December, 2021; originally announced December 2021.

Modular Pipe Climber III with Three-Output Open Differential

Authors: Rama Vadapalli, Saharsh Agarwal, Vishnu Kumar, Kartik Suryavanshi, Nagamanikandan, K Madhava Krishna

Abstract: The paper introduces the novel Modular Pipe Climber III with a Three-Output Open Differential (3-OOD) mechanism to eliminate slip** of the tracks due to the changing cross-sections of the pipe. This will be achieved in any orientation of the robot. Previous pipe climbers use three-wheel/track modules, each with an individual driving mechanism to achieve stable traversing. Slip** of tracks is p… ▽ More The paper introduces the novel Modular Pipe Climber III with a Three-Output Open Differential (3-OOD) mechanism to eliminate slip** of the tracks due to the changing cross-sections of the pipe. This will be achieved in any orientation of the robot. Previous pipe climbers use three-wheel/track modules, each with an individual driving mechanism to achieve stable traversing. Slip** of tracks is prevalent in such robots when it encounters the pipe turns. Thus, active control of each module's speed is employed to mitigate the slip, thereby requiring substantial control effort. The proposed pipe climber implements the 3-OOD to address this issue by allowing the robot to mechanically modulate the track speeds as it encounters a turn. The proposed 3-OOD is the first three-output differential to realize the functional abilities of a traditional two-output differential. △ Less

Submitted 8 January, 2022; v1 submitted 1 November, 2021; originally announced December 2021.

Quantum-Enhanced Stimulated Brillouin Scattering Spectroscopy and Imaging

Authors: Tian Li, Fu Li, Xinghua Liu, Vladislav V. Yakovlev, Girish S. Agarwal

Abstract: Brillouin microscopy is an emerging label-free imaging technique to assess local viscoelastic properties. Quantum-enhanced stimulated Brillouin scattering is demonstrated for the first time using low power continuous-wave lasers at 795~nm. A signal to noise ratio enhancement of 3.4~dB is reported by using two-mode intensity-difference squeezed light generated with the four-wave mixing process in a… ▽ More Brillouin microscopy is an emerging label-free imaging technique to assess local viscoelastic properties. Quantum-enhanced stimulated Brillouin scattering is demonstrated for the first time using low power continuous-wave lasers at 795~nm. A signal to noise ratio enhancement of 3.4~dB is reported by using two-mode intensity-difference squeezed light generated with the four-wave mixing process in atomic rubidium vapor. The low optical power and the excitation wavelengths in the water transparency window has the potential to provide a powerful bio-imaging technique for probing mechanical properties of biological samples prone to phototoxicity and thermal effects. The performance enhancement affordable through the use of quantum light may pave the way for significantly improved sensitivity that cannot be achieved classically. The proposed new way of utilizing squeezed light for enhanced stimulated Brillouin scattering can be easily adapted for both spectroscopic and imaging applications in materials science and biology. △ Less

Submitted 14 July, 2022; v1 submitted 5 December, 2021; originally announced December 2021.

Comments: 12 pages, 8 figures

Journal ref: Optica 9, 959-964 (2022)

Reinforcement Explanation Learning

Authors: Siddhant Agarwal, Owais Iqbal, Sree Aditya Buridi, Madda Manjusha, Abir Das

Abstract: Deep Learning has become overly complicated and has enjoyed stellar success in solving several classical problems like image classification, object detection, etc. Several methods for explaining these decisions have been proposed. Black-box methods to generate saliency maps are particularly interesting due to the fact that they do not utilize the internals of the model to explain the decision. Mos… ▽ More Deep Learning has become overly complicated and has enjoyed stellar success in solving several classical problems like image classification, object detection, etc. Several methods for explaining these decisions have been proposed. Black-box methods to generate saliency maps are particularly interesting due to the fact that they do not utilize the internals of the model to explain the decision. Most black-box methods perturb the input and observe the changes in the output. We formulate saliency map generation as a sequential search problem and leverage upon Reinforcement Learning (RL) to accumulate evidence from input images that most strongly support decisions made by a classifier. Such a strategy encourages to search intelligently for the perturbations that will lead to high-quality explanations. While successful black box explanation approaches need to rely on heavy computations and suffer from small sample approximation, the deterministic policy learned by our method makes it a lot more efficient during the inference. Experiments on three benchmark datasets demonstrate the superiority of the proposed approach in inference time over state-of-the-arts without hurting the performance. Project Page: https://cvir.github.io/projects/rexl.html △ Less

Submitted 26 November, 2021; originally announced November 2021.

Comments: Accepted in NeurIPS 2021 workshop on eXplainable AI approaches for debugging and diagnosis. Project Page: https://cvir.github.io/projects/rexl.html

Building Goal-Oriented Dialogue Systems with Situated Visual Context

Authors: Sanchit Agarwal, Jan Jezabek, Arijit Biswas, Emre Barut, Shuyang Gao, Tagyoung Chung

Abstract: Most popular goal-oriented dialogue agents are capable of understanding the conversational context. However, with the surge of virtual assistants with screen, the next generation of agents are required to also understand screen context in order to provide a proper interactive experience, and better understand users' goals. In this paper, we propose a novel multimodal conversational framework, wher… ▽ More Most popular goal-oriented dialogue agents are capable of understanding the conversational context. However, with the surge of virtual assistants with screen, the next generation of agents are required to also understand screen context in order to provide a proper interactive experience, and better understand users' goals. In this paper, we propose a novel multimodal conversational framework, where the dialogue agent's next action and their arguments are derived jointly conditioned both on the conversational and the visual context. Specifically, we propose a new model, that can reason over the visual context within a conversation and populate API arguments with visual entities given the user query. Our model can recognize visual features such as color and shape as well as the metadata based features such as price or star rating associated with a visual entity. In order to train our model, due to a lack of suitable multimodal conversational datasets, we also propose a novel multimodal dialog simulator to generate synthetic data and also collect realistic user data from MTurk to improve model robustness. The proposed model achieves a reasonable 85% model accuracy, without high inference latency. We also demonstrate the proposed approach in a prototypical furniture shop** experience for a multimodal virtual assistant. △ Less

Submitted 22 November, 2021; originally announced November 2021.

The RayGalGroupSims cosmological simulation suite for the study of relativistic effects: an application to lensing-matter clustering statistics

Authors: Y. Rasera, M-A. Breton, P-S. Corasaniti, J. Allingham, F. Roy, V. Reverdy, T. Pellegrin, S. Saga, A. Taruya, S. Agarwal, S. Anselmi

Abstract: General Relativistic effects on the clustering of matter in the universe provide a sensitive probe of cosmology and gravity theories that can be tested with the upcoming generation of galaxy surveys. Here, we present a suite of large volume high-resolution N-body simulations specifically designed to generate light-cone data for the study of relativistic effects on lensing-matter observables. RayGa… ▽ More General Relativistic effects on the clustering of matter in the universe provide a sensitive probe of cosmology and gravity theories that can be tested with the upcoming generation of galaxy surveys. Here, we present a suite of large volume high-resolution N-body simulations specifically designed to generate light-cone data for the study of relativistic effects on lensing-matter observables. RayGalGroupSims (or in short RayGal) consists of two N-body simulations of $(2625\,h^{-1}\,{\rm Mpc})^3$ volume with $4096^3$ particles of a standard flat $Λ$CDM model and a non-standard $w$CDM phantom dark energy model. Light-cone data from the simulations have been generated using a parallel ray-tracing algorithm that has accurately solved billion geodesic equations. Catalogues and maps with relativistic weak-lensing which include post-Born effects, magnification bias (MB) and redshift space distortions (RSD) due to gravitational redshift, Doppler, transverse Doppler, Integrated Sachs-Wolfe/Rees-Sciama effects, are publicly released. Using this dataset, we are able to reproduce the linear and quasi-linear predictions from the Class relativistic code for the 10 (cross-)power spectra (3$\times$2 points) of the matter density fluctuation field and the gravitational convergence at $z=0.7$ and $z=1.8$. We find $1-30\%$ level contribution from both MB and RSD to the matter power spectrum, while the Fingers-of-God effect is visible at lower redshift in the non-linear regime. MB contributes at the $10-30\%$ level to the convergence power spectrum leading to a deviation between the shear power-spectrum and the convergence power-spectrum. MB also plays a significant role in the galaxy-galaxy lensing by decreasing the density-convergence spectra by $20\%$, while coupling non-trivial configurations (such as the one with the convergence at the same or even lower redshift than the density field). △ Less

Submitted 28 July, 2023; v1 submitted 16 November, 2021; originally announced November 2021.

Comments: 25 pages, 19 figures, minor modifications to match A&A version, RayGal data available at https://cosmo.obspm.fr/public-datasets

Journal ref: A&A 661, A90 (2022)

DeepXML: A Deep Extreme Multi-Label Learning Framework Applied to Short Text Documents

Authors: Kunal Dahiya, Deepak Saini, Anshul Mittal, Ankush Shaw, Kushal Dave, Akshay Soni, Himanshu Jain, Sumeet Agarwal, Manik Varma

Abstract: Scalability and accuracy are well recognized challenges in deep extreme multi-label learning where the objective is to train architectures for automatically annotating a data point with the most relevant subset of labels from an extremely large label set. This paper develops the DeepXML framework that addresses these challenges by decomposing the deep extreme multi-label task into four simpler sub… ▽ More Scalability and accuracy are well recognized challenges in deep extreme multi-label learning where the objective is to train architectures for automatically annotating a data point with the most relevant subset of labels from an extremely large label set. This paper develops the DeepXML framework that addresses these challenges by decomposing the deep extreme multi-label task into four simpler sub-tasks each of which can be trained accurately and efficiently. Choosing different components for the four sub-tasks allows DeepXML to generate a family of algorithms with varying trade-offs between accuracy and scalability. In particular, DeepXML yields the Astec algorithm that could be 2-12% more accurate and 5-30x faster to train than leading deep extreme classifiers on publically available short text datasets. Astec could also efficiently train on Bing short text datasets containing up to 62 million labels while making predictions for billions of users and data points per day on commodity hardware. This allowed Astec to be deployed on the Bing search engine for a number of short text applications ranging from matching user queries to advertiser bid phrases to showing personalized ads where it yielded significant gains in click-through-rates, coverage, revenue and other online metrics over state-of-the-art techniques currently in production. DeepXML's code is available at https://github.com/Extreme-classification/deepxml △ Less

Submitted 12 November, 2021; originally announced November 2021.

ACM Class: F.2.2; I.2.7

Journal ref: Web Search and Data Mining 2021

Coupling Quantum Antennas to Fibers and Waveguides

Authors: Girish S. Agarwal, Debsuvra Mukhopadhyay

Abstract: We present a brief overview of the transport of quantum light across a one-dimensional waveguide which is integrated with a periodic string of quantum-scale dipoles. We demonstrate a scheme to implement transparency by suitably tuning the atomic frequencies without applying a coupling field and bring out the pronounced non-reciprocity of this optical device. The fiber-mediated interaction between… ▽ More We present a brief overview of the transport of quantum light across a one-dimensional waveguide which is integrated with a periodic string of quantum-scale dipoles. We demonstrate a scheme to implement transparency by suitably tuning the atomic frequencies without applying a coupling field and bring out the pronounced non-reciprocity of this optical device. The fiber-mediated interaction between integrated dipoles allows one to achieve both dispersive and dissipative couplings, level repulsion and attraction, and enhanced sensing capabilities. All these ideas can be translated to a wide variety of experimental setups of topical interest such as resonators on a transmission line, cold atoms near a fiber and quantum dots coupled to plasmonic excitations in a nanowire or photonic crystal waveguides. △ Less

Submitted 4 November, 2021; originally announced November 2021.

Comments: Manuscript based on an invited contribution to the special session on "Quantum Antennas and Photonic Quantum Sensing" at IEEE COMCAS 2021, Tel Aviv, Israel

Long-Time Memory and Ternary Logic Gate Using a Multistable Cavity Magnonic System

Authors: Rui-Chang Shen, Yi-Pu Wang, Jie Li, Shi-Yao Zhu, G. S. Agarwal, J. Q. You

Abstract: Multistability is an extraordinary nonlinear property of dynamical systems and can be explored to implement memory and switches. Here we experimentally realize the tristability in a three-mode cavity magnonic system with Kerr nonlinearity. The three stable states in the tristable region correspond to the stable solutions of the frequency shift of the cavity magnon polariton under specific driving… ▽ More Multistability is an extraordinary nonlinear property of dynamical systems and can be explored to implement memory and switches. Here we experimentally realize the tristability in a three-mode cavity magnonic system with Kerr nonlinearity. The three stable states in the tristable region correspond to the stable solutions of the frequency shift of the cavity magnon polariton under specific driving conditions. We find that the system staying in which stable state depends on the history experienced by the system, and this state can be harnessed to store the history information. In our experiment, the memory time can reach as long as 5.11 s. Moreover, we demonstrate the ternary logic gate with good on-off characteristics using this multistable hybrid system. Our new findings pave a way towards cavity magnonics-based information storage and processing. △ Less

Submitted 1 November, 2021; originally announced November 2021.

Journal ref: Physical Review Letters 127, 183202 (2021)

Anti-PT-symmetry-enhanced interconversion between microwave and optical fields

Authors: Debsuvra Mukhopadhyay, Jayakrishnan M. P. Nair, Girish S. Agarwal

Abstract: The intrinsic dissipation of systems into a shared reservoir introduces coherence between two systems, enabling anti-Parity-Time (anti-PT) symmetry. In this paper, we propose an anti-PT symmetric converter, consisting of a microwave cavity coupled dissipatively to a ferromagnetic sphere, which supports significant improvements in the conversion efficiency when compared to coherently coupled setups… ▽ More The intrinsic dissipation of systems into a shared reservoir introduces coherence between two systems, enabling anti-Parity-Time (anti-PT) symmetry. In this paper, we propose an anti-PT symmetric converter, consisting of a microwave cavity coupled dissipatively to a ferromagnetic sphere, which supports significant improvements in the conversion efficiency when compared to coherently coupled setups. In particular, when only the ferrite sample is driven, the strong coherence induced by the vacuum of the mediating channel leads to much stronger enhancements in the intended conversion. The enhancement is an inalienable artifact of the emergence of a long-lived, dark mode associated with a quasi-real singularity of the hybrid system. In addition, we observe considerable asymmetry in the efficiencies of microwave-to-optical and optical-to-microwave conversions, in spite of the symmetrical structure of the trilinear optomagnonic coupling stimulating both the transduction phenomena. The nonreciprocity stems from the intrinsic asymmetry in the couplings of the microwave and optical fields to the cavity-magnon network as well as the phase coupling entailed by the spatial separation. △ Less

Submitted 1 November, 2021; originally announced November 2021.

SIM-ECG: A Signal Importance Mask-driven ECGClassification System

Authors: Dharma KC, Chicheng Zhang, Chris Gniady, Parth Sandeep Agarwal, Sushil Sharma

Abstract: Heart disease is the number one killer, and ECGs can assist in the early diagnosis and prevention of deadly outcomes. Accurate ECG interpretation is critical in detecting heart diseases; however, they are often misinterpreted due to a lack of training or insufficient time spent to detect minute anomalies. Subsequently, researchers turned to machine learning to assist in the analysis. However, exis… ▽ More Heart disease is the number one killer, and ECGs can assist in the early diagnosis and prevention of deadly outcomes. Accurate ECG interpretation is critical in detecting heart diseases; however, they are often misinterpreted due to a lack of training or insufficient time spent to detect minute anomalies. Subsequently, researchers turned to machine learning to assist in the analysis. However, existing systems are not as accurate as skilled ECG readers, and black-box approaches to providing diagnosis result in a lack of trust by medical personnel in a given diagnosis. To address these issues, we propose a signal importance mask feedback-based machine learning system that continuously accepts feedback, improves accuracy, and ex-plains the resulting diagnosis. This allows medical personnel to quickly glance at the output and either accept the results, validate the explanation and diagnosis, or quickly correct areas of misinterpretation, giving feedback to the system for improvement. We have tested our system on a publicly available dataset consisting of healthy and disease-indicating samples. We empirically show that our algorithm is better in terms of standard performance measures such as F-score and MacroAUC compared to normal training baseline (without feedback); we also show that our model generates better interpretability maps. △ Less

Submitted 27 October, 2021; originally announced October 2021.

Comments: 9 pages

Parametric interaction induced avoided dressed state crossings in cavity QED:generation of quantum coherence and equally weighted superposition of Fock states

Authors: L. L. **, W. Li, C. J. Zhu, Y. P. Yang, G. S. Agarwal

Abstract: We present a new paradigm in the field of cavity QED by bringing out remarkable features associated with the avoided crossing of the dressed state levels of the Jaynes Cummings model. We demonstrate how the parametric couplings, realized by a second order nonlinearity in the cavity, can turn the crossing of dressed states into avoided crossings. We show how one can generate coherence between the a… ▽ More We present a new paradigm in the field of cavity QED by bringing out remarkable features associated with the avoided crossing of the dressed state levels of the Jaynes Cummings model. We demonstrate how the parametric couplings, realized by a second order nonlinearity in the cavity, can turn the crossing of dressed states into avoided crossings. We show how one can generate coherence between the avoided crossing of dressed states. Such coherences result, for example, in quantum beats in the excitation probability of the qubit. The quality of quantum beats can be considerably improved by adiabatically turning on the parametric interaction. We show how these avoided crossings can be used to generate superpositions of even or odd Fock states with the remarkable property of equal weights for the states in superposition. The fidelity of generation is more than 95\%. In addition, we show strong entanglement between the cavity field and the qubit with the concurrence parameter exceeding 90\%. △ Less

Submitted 24 October, 2021; originally announced October 2021.

Does Data Repair Lead to Fair Models? Curating Contextually Fair Data To Reduce Model Bias

Authors: Sharat Agarwal, Sumanyu Muku, Saket Anand, Chetan Arora

Abstract: Contextual information is a valuable cue for Deep Neural Networks (DNNs) to learn better representations and improve accuracy. However, co-occurrence bias in the training dataset may hamper a DNN model's generalizability to unseen scenarios in the real world. For example, in COCO, many object categories have a much higher co-occurrence with men compared to women, which can bias a DNN's prediction… ▽ More Contextual information is a valuable cue for Deep Neural Networks (DNNs) to learn better representations and improve accuracy. However, co-occurrence bias in the training dataset may hamper a DNN model's generalizability to unseen scenarios in the real world. For example, in COCO, many object categories have a much higher co-occurrence with men compared to women, which can bias a DNN's prediction in favor of men. Recent works have focused on task-specific training strategies to handle bias in such scenarios, but fixing the available data is often ignored. In this paper, we propose a novel and more generic solution to address the contextual bias in the datasets by selecting a subset of the samples, which is fair in terms of the co-occurrence with various classes for a protected attribute. We introduce a data repair algorithm using the coefficient of variation, which can curate fair and contextually balanced data for a protected class(es). This helps in training a fair model irrespective of the task, architecture or training methodology. Our proposed solution is simple, effective, and can even be used in an active learning setting where the data labels are not present or being generated incrementally. We demonstrate the effectiveness of our algorithm for the task of object detection and multi-label image classification across different datasets. Through a series of experiments, we validate that curating contextually fair data helps make model predictions fair by balancing the true positive rate for the protected class across groups without compromising on the model's overall performance. △ Less

Submitted 20 October, 2021; originally announced October 2021.

Comments: A variant of this report is accepted in WACV 2022

Quantum Fisher Information Perspective on Sensing in Anti-PT Symmetric Systems

Authors: J. Wang, D. Mukhopadhyay, G. S. Agarwal

Abstract: The efficient sensing of weak environmental perturbations via special degeneracies called exceptional points in non-Hermitian systems has gained enormous traction in the last few decades. However, in contrast to the extensive literature on parity-time (PT) symmetric systems, the exotic hallmarks of anti-PT symmetric systems are only beginning to be realized now. Very recently, a characteristic res… ▽ More The efficient sensing of weak environmental perturbations via special degeneracies called exceptional points in non-Hermitian systems has gained enormous traction in the last few decades. However, in contrast to the extensive literature on parity-time (PT) symmetric systems, the exotic hallmarks of anti-PT symmetric systems are only beginning to be realized now. Very recently, a characteristic resonance of vanishing linewidth in anti-PT symmetric systems was shown to exhibit tremendous sensitivity to intrinsic nonlinearities. Given the primacy of sensing in non-Hermitian systems, in general, and the immense topicality of anti-PT symmetry, we investigate the statistical bound to the measurement sensitivity for any arbitrary perturbation in a dissipatively coupled, anti-PT symmetric system. Using the framework of quantum Fisher information and the long-time solution to the full master equation, we analytically compute the Cramer-Rao bound for the system properties like the detunings and the couplings. As an illustrative example of this formulation, we inspect and reaffirm the role of a long-lived resonance in dissipatively interacting systems for sensing applications. \end{abstract} △ Less

Submitted 14 October, 2021; originally announced October 2021.

Comments: 6 pages, 1 figure

Report number: Phys. Rev. Research 4, 013131

Gap Statistics for Confined Particles with Power-Law Interactions

Authors: Saikat Santra, Jitendra Kethepalli, Sanaa Agarwal, Abhishek Dhar, Manas Kulkarni, Anupam Kundu

Abstract: We consider the $N$ particle classical Riesz gas confined in a one-dimensional external harmonic potential with power law interaction of the form $1/r^k$ where $r$ is the separation between particles. As special limits it contains several systems such as Dyson's log-gas ($k\to 0^+$), Calogero-Moser model ($k=2$), 1d one component plasma ($k=-1$) and the hard-rod gas ($k\to \infty$). Despite its gr… ▽ More We consider the $N$ particle classical Riesz gas confined in a one-dimensional external harmonic potential with power law interaction of the form $1/r^k$ where $r$ is the separation between particles. As special limits it contains several systems such as Dyson's log-gas ($k\to 0^+$), Calogero-Moser model ($k=2$), 1d one component plasma ($k=-1$) and the hard-rod gas ($k\to \infty$). Despite its growing importance, only large-$N$ field theory and average density profile are known for general $k$. In this Letter, we study the fluctuations in the system by looking at the statistics of the gap between successive particles. This quantity is analogous to the well-known level spacing statistics which is ubiquitous in several branches of physics. We show that the variance goes as $N^{-b_k}$ and we find the $k$ dependence of $b_k$ via direct Monte Carlo simulations. We provide supporting arguments based on microscopic Hessian calculation and a quadratic field theory approach. We compute the gap distribution and study its system size scaling. Except in the range $-1<k<0$, we find scaling for all $k>-2$ with both Gaussian and non-Gaussian scaling forms. △ Less

Submitted 16 May, 2022; v1 submitted 30 September, 2021; originally announced September 2021.

Comments: 13 pages, 11 figures

Journal ref: @article{year = 2022, month = {apr}, journal = {Physical Review Letters},Phys. Rev. Lett. 128, 170603}

Vronicle: A System for Producing Videos with Verifiable Provenance

Authors: Yuxin, Liu, Yoshimichi Nakatsuka, Ardalan Amiri Sani, Sharad Agarwal, Gene Tsudik

Abstract: Demonstrating the veracity of videos is a longstanding problem that has recently become more urgent and acute. It is extremely hard to accurately detect manipulated videos using content analysis, especially in the face of subtle, yet effective, manipulations, such as frame rate changes or skin tone adjustments. One prominent alternative to content analysis is to securely embed provenance informati… ▽ More Demonstrating the veracity of videos is a longstanding problem that has recently become more urgent and acute. It is extremely hard to accurately detect manipulated videos using content analysis, especially in the face of subtle, yet effective, manipulations, such as frame rate changes or skin tone adjustments. One prominent alternative to content analysis is to securely embed provenance information into videos. However, prior approaches have poor performance and/or granularity that is too coarse. To this end, we construct Vronicle -- a video provenance system that offers fine-grained provenance information and substantially better performance. It allows a video consumer to authenticate the camera that originated the video and the exact sequence of video filters that were subsequently applied to it. Vronicle exploits the increasing popularity and availability of Trusted Execution Environments (TEEs) on many types of computing platforms. One contribution of Vronicle is the design of provenance information that allows the consumer to verify various aspects of the video, thereby defeating numerous fake-video creation methods. Vronicle's adversarial model allows for a powerful adversary that can manipulate the video (e.g., in transit) and the software state outside the TEE. Another contribution is the use of fixed-function Intel SGX enclaves to post-process videos. This design facilitates verification of provenance information. We present a prototype implementation of Vronicle (to be open sourced), which relies on current technologies, making it readily deployable. Our evaluation demonstrates that Vronicle's performance is well-suited for offline use-cases. △ Less

Submitted 26 September, 2021; originally announced September 2021.

Language Model Priming for Cross-Lingual Event Extraction

Authors: Steven Fincke, Shantanu Agarwal, Scott Miller, Elizabeth Boschee

Abstract: We present a novel, language-agnostic approach to "priming" language models for the task of event extraction, providing particularly effective performance in low-resource and zero-shot cross-lingual settings. With priming, we augment the input to the transformer stack's language model differently depending on the question(s) being asked of the model at runtime. For instance, if the model is being… ▽ More We present a novel, language-agnostic approach to "priming" language models for the task of event extraction, providing particularly effective performance in low-resource and zero-shot cross-lingual settings. With priming, we augment the input to the transformer stack's language model differently depending on the question(s) being asked of the model at runtime. For instance, if the model is being asked to identify arguments for the trigger "protested", we will provide that trigger as part of the input to the language model, allowing it to produce different representations for candidate arguments than when it is asked about arguments for the trigger "arrest" elsewhere in the same sentence. We show that by enabling the language model to better compensate for the deficits of sparse and noisy training data, our approach improves both trigger and argument detection and classification significantly over the state of the art in a zero-shot cross-lingual setting. △ Less

Submitted 25 September, 2021; originally announced September 2021.

Visually Connecting Historical Figures Through Event Knowledge Graphs

Authors: Shahid Latif, Shivam Agarwal, Simon Gottschalk, Carina Chrosch, Felix Feit, Johannes Jahn, Tobias Braun, Yanick Christian Tchenko, Elena Demidova, Fabian Beck

Abstract: Knowledge graphs store information about historical figures and their relationships indirectly through shared events. We developed a visualization system, VisKonnect, for analyzing the intertwined lives of historical figures based on the events they participated in. A user's query is parsed for identifying named entities, and related data is retrieved from an event knowledge graph. While a short t… ▽ More Knowledge graphs store information about historical figures and their relationships indirectly through shared events. We developed a visualization system, VisKonnect, for analyzing the intertwined lives of historical figures based on the events they participated in. A user's query is parsed for identifying named entities, and related data is retrieved from an event knowledge graph. While a short textual answer to the query is generated using the GPT-3 language model, various linked visualizations provide context, display additional information related to the query, and allow exploration. △ Less

Submitted 20 September, 2021; originally announced September 2021.

Comments: 5 pages, 5 figures, short paper at VIS 2021

Grou** Search Results with Product Graphs in E-commerce Platforms

Authors: Suhas Ranganath, Shibsankar Das, Sanjay Thilaivasan, Shipra Agarwal, Varun Shrivastava

Abstract: Showing relevant search results to the user is the primary challenge for any search system. Walmart e-commerce provides an omnichannel search platform to its customers to search from millions of products. This search platform takes a textual query as input and shows relevant items from the catalog. One of the primary challenges is that this queries are complex to understand as it contains multiple… ▽ More Showing relevant search results to the user is the primary challenge for any search system. Walmart e-commerce provides an omnichannel search platform to its customers to search from millions of products. This search platform takes a textual query as input and shows relevant items from the catalog. One of the primary challenges is that this queries are complex to understand as it contains multiple intent in many cases. This paper proposes a framework to group search results into multiple ranked lists intending to provide better user intent. The framework is to create a product graph having relations between product entities and utilize it to group search results into a series of stacks where each stack provides a group of items based on a precise intent. As an example, for a query "milk," the results can be grouped into multiple stacks of "white milk", "low-fat milk", "almond milk", "flavored milk". We measure the impact of our algorithm by evaluating how it improves the user experience both in terms of search quality relevance and user behavioral signals like Add-To-Cart. △ Less

Submitted 20 September, 2021; originally announced September 2021.

Journal ref: ACM Web Conference 2021,Knowledge Management in e-Commerce Workshop

Uncovering Quasi-periodic Nature of Physical Systems: A Case Study of Signalized Intersections

Authors: Suddhasattwa Das, Shakib Mustavee, Shaurya Agarwal

Abstract: This paper presents a novel approach to analyze quasiperiodically driven dynamical systems. It aims to develop a complete data-driven framework for modeling such unknown dynamics. To achieve this, we characterize Koopman eigenfrequencies as generating frequencies of the quasiperiodic driver of the system. We compute true eigenfrequencies of Koopman operators by applying the theory of Reproducing K… ▽ More This paper presents a novel approach to analyze quasiperiodically driven dynamical systems. It aims to develop a complete data-driven framework for modeling such unknown dynamics. To achieve this, we characterize Koopman eigenfrequencies as generating frequencies of the quasiperiodic driver of the system. We compute true eigenfrequencies of Koopman operators by applying the theory of Reproducing Kernel Hibert Space (RKHS) and results from ergodic theory. We also demonstrate the decomposition of quasiperiodically driven dynamics into two components, i) the quasiperiodic driving source with generating frequencies and ii) the driven nonlinear dynamics. A unique aspect of the proposed framework is that it applies to the analysis of systems where the periodic component is either non-dominant or even absent. As a case study, we analyze a system of nine traffic signalized intersections. The proposed framework accurately reconstructs the measured queue lengths of the signalized intersections and makes stable long-term predictions. △ Less

Submitted 15 September, 2021; originally announced September 2021.

MSC Class: 37M25; 47A35; 37M10