-
DIPPER: Direct Preference Optimization to Accelerate Primitive-Enabled Hierarchical Reinforcement Learning
Authors:
Utsav Singh,
Souradip Chakraborty,
Wesley A. Suttle,
Brian M. Sadler,
Vinay P Namboodiri,
Amrit Singh Bedi
Abstract:
Learning control policies to perform complex robotics tasks from human preference data presents significant challenges. On the one hand, the complexity of such tasks typically requires learning policies to perform a variety of subtasks, then combining them to achieve the overall goal. At the same time, comprehensive, well-engineered reward functions are typically unavailable in such problems, whil…
▽ More
Learning control policies to perform complex robotics tasks from human preference data presents significant challenges. On the one hand, the complexity of such tasks typically requires learning policies to perform a variety of subtasks, then combining them to achieve the overall goal. At the same time, comprehensive, well-engineered reward functions are typically unavailable in such problems, while limited human preference data often is; making efficient use of such data to guide learning is therefore essential. Methods for learning to perform complex robotics tasks from human preference data must overcome both these challenges simultaneously. In this work, we introduce DIPPER: Direct Preference Optimization to Accelerate Primitive-Enabled Hierarchical Reinforcement Learning, an efficient hierarchical approach that leverages direct preference optimization to learn a higher-level policy and reinforcement learning to learn a lower-level policy. DIPPER enjoys improved computational efficiency due to its use of direct preference optimization instead of standard preference-based approaches such as reinforcement learning from human feedback, while it also mitigates the well-known hierarchical reinforcement learning issues of non-stationarity and infeasible subgoal generation due to our use of primitive-informed regularization inspired by a novel bi-level optimization formulation of the hierarchical reinforcement learning problem. To validate our approach, we perform extensive experimental analysis on a variety of challenging robotics tasks, demonstrating that DIPPER outperforms hierarchical and non-hierarchical baselines, while ameliorating the non-stationarity and infeasible subgoal generation issues of hierarchical reinforcement learning.
△ Less
Submitted 16 June, 2024;
originally announced June 2024.
-
LGR2: Language Guided Reward Relabeling for Accelerating Hierarchical Reinforcement Learning
Authors:
Utsav Singh,
Pramit Bhattacharyya,
Vinay P. Namboodiri
Abstract:
Develo** interactive systems that leverage natural language instructions to solve complex robotic control tasks has been a long-desired goal in the robotics community. Large Language Models (LLMs) have demonstrated exceptional abilities in handling complex tasks, including logical reasoning, in-context learning, and code generation. However, predicting low-level robotic actions using LLMs poses…
▽ More
Develo** interactive systems that leverage natural language instructions to solve complex robotic control tasks has been a long-desired goal in the robotics community. Large Language Models (LLMs) have demonstrated exceptional abilities in handling complex tasks, including logical reasoning, in-context learning, and code generation. However, predicting low-level robotic actions using LLMs poses significant challenges. Additionally, the complexity of such tasks usually demands the acquisition of policies to execute diverse subtasks and combine them to attain the ultimate objective. Hierarchical Reinforcement Learning (HRL) is an elegant approach for solving such tasks, which provides the intuitive benefits of temporal abstraction and improved exploration. However, HRL faces the recurring issue of non-stationarity due to unstable lower primitive behaviour. In this work, we propose LGR2, a novel HRL framework that leverages language instructions to generate a stationary reward function for the higher-level policy. Since the language-guided reward is unaffected by the lower primitive behaviour, LGR2 mitigates non-stationarity and is thus an elegant method for leveraging language instructions to solve robotic control tasks. To analyze the efficacy of our approach, we perform empirical analysis and demonstrate that LGR2 effectively alleviates non-stationarity in HRL. Our approach attains success rates exceeding 70$\%$ in challenging, sparse-reward robotic navigation and manipulation environments where the baselines fail to achieve any significant progress. Additionally, we conduct real-world robotic manipulation experiments and demonstrate that CRISP shows impressive generalization in real-world scenarios.
△ Less
Submitted 16 June, 2024; v1 submitted 9 June, 2024;
originally announced June 2024.
-
PIPER: Primitive-Informed Preference-based Hierarchical Reinforcement Learning via Hindsight Relabeling
Authors:
Utsav Singh,
Wesley A. Suttle,
Brian M. Sadler,
Vinay P. Namboodiri,
Amrit Singh Bedi
Abstract:
In this work, we introduce PIPER: Primitive-Informed Preference-based Hierarchical reinforcement learning via Hindsight Relabeling, a novel approach that leverages preference-based learning to learn a reward model, and subsequently uses this reward model to relabel higher-level replay buffers. Since this reward is unaffected by lower primitive behavior, our relabeling-based approach is able to mit…
▽ More
In this work, we introduce PIPER: Primitive-Informed Preference-based Hierarchical reinforcement learning via Hindsight Relabeling, a novel approach that leverages preference-based learning to learn a reward model, and subsequently uses this reward model to relabel higher-level replay buffers. Since this reward is unaffected by lower primitive behavior, our relabeling-based approach is able to mitigate non-stationarity, which is common in existing hierarchical approaches, and demonstrates impressive performance across a range of challenging sparse-reward tasks. Since obtaining human feedback is typically impractical, we propose to replace the human-in-the-loop approach with our primitive-in-the-loop approach, which generates feedback using sparse rewards provided by the environment. Moreover, in order to prevent infeasible subgoal prediction and avoid degenerate solutions, we propose primitive-informed regularization that conditions higher-level policies to generate feasible subgoals for lower-level policies. We perform extensive experiments to show that PIPER mitigates non-stationarity in hierarchical reinforcement learning and achieves greater than 50$\%$ success rates in challenging, sparse-reward robotic environments, where most other baselines fail to achieve any significant progress.
△ Less
Submitted 16 June, 2024; v1 submitted 20 April, 2024;
originally announced April 2024.
-
On the Performance of IRS-Assisted SSK and RPM over Rician Fading Channels
Authors:
Harsh Raj,
Ugrasen Singh,
B. R. Manoj
Abstract:
This paper presents the index modulation, that is, the space-shift keying (SSK) and reflection phase modulation (RPM) schemes for intelligent reflecting surface (IRS)-assisted wireless network. IRS simultaneously reflects the incoming information signal from the base station and explicitly encodes the local information bits in the reflection phase shift of IRS elements. The phase shift of the IRS…
▽ More
This paper presents the index modulation, that is, the space-shift keying (SSK) and reflection phase modulation (RPM) schemes for intelligent reflecting surface (IRS)-assisted wireless network. IRS simultaneously reflects the incoming information signal from the base station and explicitly encodes the local information bits in the reflection phase shift of IRS elements. The phase shift of the IRS elements is employed according to local data from the RPM constellation. A joint detection using a maximum-likelihood (ML) decoder is performed for the SSK and RPM symbols over a realistic fading scenario modeled as the Rician fading channel. The pairwise error probability over Rician fading channels is derived and utilized to determine the average bit error rate. In addition, the ergodic capacity of the presented system is derived. The derived analytical results are verified and are in exact agreement with Monte-Carlo simulations.
△ Less
Submitted 10 April, 2024;
originally announced April 2024.
-
Coherent Feed Forward Quantum Neural Network
Authors:
Utkarsh Singh,
Aaron Z. Goldberg,
Khabat Heshami
Abstract:
Quantum machine learning, focusing on quantum neural networks (QNNs), remains a vastly uncharted field of study. Current QNN models primarily employ variational circuits on an ansatz or a quantum feature map, often requiring multiple entanglement layers. This methodology not only increases the computational cost of the circuit beyond what is practical on near-term quantum devices but also misleadi…
▽ More
Quantum machine learning, focusing on quantum neural networks (QNNs), remains a vastly uncharted field of study. Current QNN models primarily employ variational circuits on an ansatz or a quantum feature map, often requiring multiple entanglement layers. This methodology not only increases the computational cost of the circuit beyond what is practical on near-term quantum devices but also misleadingly labels these models as neural networks, given their divergence from the structure of a typical feed-forward neural network (FFNN). Moreover, the circuit depth and qubit needs of these models scale poorly with the number of data features, resulting in an efficiency challenge for real-world machine-learning tasks. We introduce a bona fide QNN model, which seamlessly aligns with the versatility of a traditional FFNN in terms of its adaptable intermediate layers and nodes, absent from intermediate measurements such that our entire model is coherent. This model stands out with its reduced circuit depth and number of requisite C-NOT gates to outperform prevailing QNN models. Furthermore, the qubit count in our model remains unaffected by the data's feature quantity. We test our proposed model on various benchmarking datasets such as the diagnostic breast cancer (Wisconsin) and credit card fraud detection datasets. We compare the outcomes of our model with the existing QNN methods to showcase the advantageous efficacy of our approach, even with a reduced requirement on quantum resources. Our model paves the way for application of quantum neural networks to real relevant machine learning problems.
△ Less
Submitted 1 February, 2024;
originally announced February 2024.
-
memorAIs: an Optical Character Recognition and Rule-Based Medication Intake Reminder-Generating Solution
Authors:
Eden Shaveet,
Utkarsh Singh,
Nicholas Assaderaghi,
Maximo Librandi
Abstract:
Memory-based medication non-adherence is an unsolved problem that is responsible for considerable disease burden in the United States. Digital medication intake reminder solutions with minimal onboarding requirements that are usable at the point of medication acquisition may help to alleviate this problem by offering a low barrier way to help people remember to take their medications. In this pape…
▽ More
Memory-based medication non-adherence is an unsolved problem that is responsible for considerable disease burden in the United States. Digital medication intake reminder solutions with minimal onboarding requirements that are usable at the point of medication acquisition may help to alleviate this problem by offering a low barrier way to help people remember to take their medications. In this paper, we propose memorAIs, a digital medication intake reminder solution that mitigates onboarding friction by leveraging optical character recognition strategies for text extraction from medication bottles and rule based expressions for text processing to create configured medication reminders as local device calendar invitations. We describe our ideation and development process, as well as limitations of the current implementation. memorAIs was the winner of the Patient Safety award at the 2023 Columbia University DivHacks Hackathon, presented by the Patient Safety Technology Challenge, sponsored by the Pittsburgh Regional Health Initiative.
△ Less
Submitted 11 December, 2023;
originally announced December 2023.
-
Comparative Analysis of Transformers for Modeling Tabular Data: A Casestudy using Industry Scale Dataset
Authors:
Usneek Singh,
Piyush Arora,
Shamika Ganesan,
Mohit Kumar,
Siddhant Kulkarni,
Salil R. Joshi
Abstract:
We perform a comparative analysis of transformer-based models designed for modeling tabular data, specifically on an industry-scale dataset. While earlier studies demonstrated promising outcomes on smaller public or synthetic datasets, the effectiveness did not extend to larger industry-scale datasets. The challenges identified include handling high-dimensional data, the necessity for efficient pr…
▽ More
We perform a comparative analysis of transformer-based models designed for modeling tabular data, specifically on an industry-scale dataset. While earlier studies demonstrated promising outcomes on smaller public or synthetic datasets, the effectiveness did not extend to larger industry-scale datasets. The challenges identified include handling high-dimensional data, the necessity for efficient pre-processing of categorical and numerical features, and addressing substantial computational requirements.
To overcome the identified challenges, the study conducts an extensive examination of various transformer-based models using both synthetic datasets and the default prediction Kaggle dataset (2022) from American Express. The paper presents crucial insights into optimal data pre-processing, compares pre-training and direct supervised learning methods, discusses strategies for managing categorical and numerical features, and highlights trade-offs between computational resources and performance. Focusing on temporal financial data modeling, the research aims to facilitate the systematic development and deployment of transformer-based models in real-world scenarios, emphasizing scalability.
△ Less
Submitted 24 November, 2023;
originally announced November 2023.
-
pyParaOcean: A System for Visual Analysis of Ocean Data
Authors:
Toshit Jain,
Varun Singh,
Vijay Kumar Boda,
Upkar Singh,
Ingrid Hotz,
P. N. Vinayachandran,
Vijay Natarajan
Abstract:
Visual analysis is well adopted within the field of oceanography for the analysis of model simulations, detection of different phenomena and events, and tracking of dynamic processes. With increasing data sizes and the availability of multivariate dynamic data, there is a growing need for scalable and extensible tools for visualization and interactive exploration. We describe pyParaOcean, a visual…
▽ More
Visual analysis is well adopted within the field of oceanography for the analysis of model simulations, detection of different phenomena and events, and tracking of dynamic processes. With increasing data sizes and the availability of multivariate dynamic data, there is a growing need for scalable and extensible tools for visualization and interactive exploration. We describe pyParaOcean, a visualization system that supports several tasks routinely used in the visual analysis of ocean data. The system is available as a plugin to Paraview and is hence able to leverage its distributed computing capabilities and its rich set of generic analysis and visualization functionalities. pyParaOcean provides modules to support different visual analysis tasks specific to ocean data, such as eddy identification and salinity movement tracking. These modules are available as Paraview filters and this seamless integration results in a system that is easy to install and use. A case study on the Bay of Bengal illustrates the utility of the system for the study of ocean phenomena and processes.
△ Less
Submitted 25 September, 2023;
originally announced September 2023.
-
PEAR: Primitive enabled Adaptive Relabeling for boosting Hierarchical Reinforcement Learning
Authors:
Utsav Singh,
Vinay P. Namboodiri
Abstract:
Hierarchical reinforcement learning (HRL) has the potential to solve complex long horizon tasks using temporal abstraction and increased exploration. However, hierarchical agents are difficult to train due to inherent non-stationarity. We present primitive enabled adaptive relabeling (PEAR), a two-phase approach where we first perform adaptive relabeling on a few expert demonstrations to generate…
▽ More
Hierarchical reinforcement learning (HRL) has the potential to solve complex long horizon tasks using temporal abstraction and increased exploration. However, hierarchical agents are difficult to train due to inherent non-stationarity. We present primitive enabled adaptive relabeling (PEAR), a two-phase approach where we first perform adaptive relabeling on a few expert demonstrations to generate efficient subgoal supervision, and then jointly optimize HRL agents by employing reinforcement learning (RL) and imitation learning (IL). We perform theoretical analysis to $(i)$ bound the sub-optimality of our approach, and $(ii)$ derive a generalized plug-and-play framework for joint optimization using RL and IL. Since PEAR utilizes only a handful of expert demonstrations and considers minimal limiting assumptions on the task structure, it can be easily integrated with typical off-policy RL algorithms to produce a practical HRL approach. We perform extensive experiments on challenging environments and show that PEAR is able to outperform various hierarchical and non-hierarchical baselines on complex tasks that require long term decision making. We also perform ablations to thoroughly analyse the importance of our various design choices. Finally, we perform real world robotic experiments on complex tasks and demonstrate that PEAR consistently outperforms the baselines.
△ Less
Submitted 21 April, 2024; v1 submitted 10 June, 2023;
originally announced June 2023.
-
CRISP: Curriculum inducing Primitive Informed Subgoal Prediction
Authors:
Utsav Singh,
Vinay P. Namboodiri
Abstract:
Hierarchical reinforcement learning (HRL) is a promising approach that uses temporal abstraction to solve complex long horizon problems. However, simultaneously learning a hierarchy of policies is unstable as it is challenging to train higher-level policy when the lower-level primitive is non-stationary. In this paper, we present CRISP, a novel HRL algorithm that effectively generates a curriculum…
▽ More
Hierarchical reinforcement learning (HRL) is a promising approach that uses temporal abstraction to solve complex long horizon problems. However, simultaneously learning a hierarchy of policies is unstable as it is challenging to train higher-level policy when the lower-level primitive is non-stationary. In this paper, we present CRISP, a novel HRL algorithm that effectively generates a curriculum of achievable subgoals for evolving lower-level primitives using reinforcement learning and imitation learning. CRISP uses the lower level primitive to periodically perform data relabeling on a handful of expert demonstrations, using a novel primitive informed parsing (PIP) approach, thereby mitigating non-stationarity. Since our approach only assumes access to a handful of expert demonstrations, it is suitable for most robotic control tasks. Experimental evaluations on complex robotic maze navigation and robotic manipulation tasks demonstrate that inducing hierarchical curriculum learning significantly improves sample efficiency, and results in efficient goal conditioned policies for solving temporally extended tasks. Additionally, we perform real world robotic experiments on complex manipulation tasks and demonstrate that CRISP demonstrates impressive generalization in real world scenarios.
△ Less
Submitted 21 April, 2024; v1 submitted 7 April, 2023;
originally announced April 2023.
-
Quantum Depth in the Random Oracle Model
Authors:
Atul Singh Arora,
Andrea Coladangelo,
Matthew Coudron,
Alexandru Gheorghiu,
Uttam Singh,
Hendrik Waldner
Abstract:
We give a comprehensive characterization of the computational power of shallow quantum circuits combined with classical computation. Specifically, for classes of search problems, we show that the following statements hold, relative to a random oracle:
(a) $\mathsf{BPP}^{\mathsf{QNC}^{\mathsf{BPP}}} \neq \mathsf{BQP}$. This refutes Jozsa's conjecture [QIP 05] in the random oracle model. As a resu…
▽ More
We give a comprehensive characterization of the computational power of shallow quantum circuits combined with classical computation. Specifically, for classes of search problems, we show that the following statements hold, relative to a random oracle:
(a) $\mathsf{BPP}^{\mathsf{QNC}^{\mathsf{BPP}}} \neq \mathsf{BQP}$. This refutes Jozsa's conjecture [QIP 05] in the random oracle model. As a result, this gives the first instantiatable separation between the classes by replacing the oracle with a cryptographic hash function, yielding a resolution to one of Aaronson's ten semi-grand challenges in quantum computing.
(b) $\mathsf{BPP}^{\mathsf{QNC}} \nsubseteq \mathsf{QNC}^{\mathsf{BPP}}$ and $\mathsf{QNC}^{\mathsf{BPP}} \nsubseteq \mathsf{BPP}^{\mathsf{QNC}}$. This shows that there is a subtle interplay between classical computation and shallow quantum computation. In fact, for the second separation, we establish that, for some problems, the ability to perform adaptive measurements in a single shallow quantum circuit, is more useful than the ability to perform polynomially many shallow quantum circuits without adaptive measurements.
(c) There exists a 2-message proof of quantum depth protocol. Such a protocol allows a classical verifier to efficiently certify that a prover must be performing a computation of some minimum quantum depth. Our proof of quantum depth can be instantiated using the recent proof of quantumness construction by Yamakawa and Zhandry [STOC 22].
△ Less
Submitted 12 October, 2022;
originally announced October 2022.
-
NFDLM: A Lightweight Network Flow based Deep Learning Model for DDoS Attack Detection in IoT Domains
Authors:
Kumar Saurabh,
Tanuj Kumar,
Uphar Singh,
O. P. Vyas,
Rahamatullah Khondoker
Abstract:
In the recent years, Distributed Denial of Service (DDoS) attacks on Internet of Things (IoT) devices have become one of the prime concerns to Internet users around the world. One of the sources of the attacks on IoT ecosystems are botnets. Intruders force IoT devices to become unavailable for its legitimate users by sending large number of messages within a short interval. This study proposes NFD…
▽ More
In the recent years, Distributed Denial of Service (DDoS) attacks on Internet of Things (IoT) devices have become one of the prime concerns to Internet users around the world. One of the sources of the attacks on IoT ecosystems are botnets. Intruders force IoT devices to become unavailable for its legitimate users by sending large number of messages within a short interval. This study proposes NFDLM, a lightweight and optimised Artificial Neural Network (ANN) based Distributed Denial of Services (DDoS) attack detection framework with mutual correlation as feature selection method which produces a superior result when compared with Long Short Term Memory (LSTM) and simple ANN. Overall, the detection performance achieves approximately 99\% accuracy for the detection of attacks from botnets. In this work, we have designed and compared four different models where two are based on ANN and the other two are based on LSTM to detect the attack types of DDoS.
△ Less
Submitted 15 July, 2022;
originally announced July 2022.
-
LBDMIDS: LSTM Based Deep Learning Model for Intrusion Detection Systems for IoT Networks
Authors:
Kumar Saurabh,
Saksham Sood,
P. Aditya Kumar,
Uphar Singh,
Ranjana Vyas,
O. P. Vyas,
Rahamatullah Khondoker
Abstract:
In the recent years, we have witnessed a huge growth in the number of Internet of Things (IoT) and edge devices being used in our everyday activities. This demands the security of these devices from cyber attacks to be improved to protect its users. For years, Machine Learning (ML) techniques have been used to develop Network Intrusion Detection Systems (NIDS) with the aim of increasing their reli…
▽ More
In the recent years, we have witnessed a huge growth in the number of Internet of Things (IoT) and edge devices being used in our everyday activities. This demands the security of these devices from cyber attacks to be improved to protect its users. For years, Machine Learning (ML) techniques have been used to develop Network Intrusion Detection Systems (NIDS) with the aim of increasing their reliability/robustness. Among the earlier ML techniques DT performed well. In the recent years, Deep Learning (DL) techniques have been used in an attempt to build more reliable systems. In this paper, a Deep Learning enabled Long Short Term Memory (LSTM) Autoencoder and a 13-feature Deep Neural Network (DNN) models were developed which performed a lot better in terms of accuracy on UNSW-NB15 and Bot-IoT datsets. Hence we proposed LBDMIDS, where we developed NIDS models based on variants of LSTMs namely, stacked LSTM and bidirectional LSTM and validated their performance on the UNSW\_NB15 and BoT\-IoT datasets. This paper concludes that these variants in LBDMIDS outperform classic ML techniques and perform similarly to the DNN models that have been suggested in the past.
△ Less
Submitted 23 June, 2022;
originally announced July 2022.
-
Agricultural Plantation Classification using Transfer Learning Approach based on CNN
Authors:
Uphar Singh,
Tushar Musale,
Ranjana Vyas,
O. P. Vyas
Abstract:
Hyper-spectral images are images captured from a satellite that gives spatial and spectral information of specific region.A Hyper-spectral image contains much more number of channels as compared to a RGB image, hence containing more information about entities within the image. It makes them well suited for the classification of objects in a snap. In the past years, the efficiency of hyper-spectral…
▽ More
Hyper-spectral images are images captured from a satellite that gives spatial and spectral information of specific region.A Hyper-spectral image contains much more number of channels as compared to a RGB image, hence containing more information about entities within the image. It makes them well suited for the classification of objects in a snap. In the past years, the efficiency of hyper-spectral image recognition has increased significantly with deep learning. The Convolution Neural Network(CNN) and Multi-Layer Perceptron(MLP) has demonstrated to be an excellent process of classifying images. However, they suffer from the issues of long training time and requirement of large amounts of the labeled data, to achieve the expected outcome. These issues become more complex while dealing with hyper-spectral images. To decrease the training time and reduce the dependence on large labeled data-set, we propose using the method of transfer learning.The features learned by CNN and MLP models are then used by the transfer learning model to solve a new classification problem on an unseen dataset. A detailed comparison of CNN and multiple MLP architectural models is performed, to determine an optimum architecture that suits best the objective. The results show that the scaling of layers not always leads to increase in accuracy but often leads to over-fitting, and also an increase in the training time.The training time is reduced to greater extent by applying the transfer learning approach rather than just approaching the problem by directly training a new model on large data-sets, without much affecting the accuracy.
△ Less
Submitted 19 June, 2022;
originally announced June 2022.
-
Terrain Classification using Transfer Learning on Hyperspectral Images: A Comparative study
Authors:
Uphar Singh,
Kumar Saurabh,
Neelaksh Trehan,
Ranjana Vyas,
O. P. Vyas
Abstract:
A Hyperspectral image contains much more number of channels as compared to a RGB image, hence containing more information about entities within the image. The convolutional neural network (CNN) and the Multi-Layer Perceptron (MLP) have been proven to be an effective method of image classification. However, they suffer from the issues of long training time and requirement of large amounts of the la…
▽ More
A Hyperspectral image contains much more number of channels as compared to a RGB image, hence containing more information about entities within the image. The convolutional neural network (CNN) and the Multi-Layer Perceptron (MLP) have been proven to be an effective method of image classification. However, they suffer from the issues of long training time and requirement of large amounts of the labeled data, to achieve the expected outcome. These issues become more complex while dealing with hyperspectral images. To decrease the training time and reduce the dependence on large labeled dataset, we propose using the method of transfer learning. The hyperspectral dataset is preprocessed to a lower dimension using PCA, then deep learning models are applied to it for the purpose of classification. The features learned by this model are then used by the transfer learning model to solve a new classification problem on an unseen dataset. A detailed comparison of CNN and multiple MLP architectural models is performed, to determine an optimum architecture that suits best the objective. The results show that the scaling of layers not always leads to increase in accuracy but often leads to overfitting, and also an increase in the training time.The training time is reduced to greater extent by applying the transfer learning approach rather than just approaching the problem by directly training a new model on large datasets, without much affecting the accuracy.
△ Less
Submitted 19 June, 2022;
originally announced June 2022.
-
Oracle separations of hybrid quantum-classical circuits
Authors:
Atul Singh Arora,
Alexandru Gheorghiu,
Uttam Singh
Abstract:
An important theoretical problem in the study of quantum computation, that is also practically relevant in the context of near-term quantum devices, is to understand the computational power of hybrid models, that combine poly-time classical computation with short-depth quantum computation. Here, we consider two such models: CQ_d which captures the scenario of a polynomial-time classical algorithm…
▽ More
An important theoretical problem in the study of quantum computation, that is also practically relevant in the context of near-term quantum devices, is to understand the computational power of hybrid models, that combine poly-time classical computation with short-depth quantum computation. Here, we consider two such models: CQ_d which captures the scenario of a polynomial-time classical algorithm that queries a d-depth quantum computer many times; and QC_d which is more analogous to measurement-based quantum computation and captures the scenario of a d-depth quantum computer with the ability to change the sequence of gates being applied depending on measurement outcomes processed by a classical computation. Chia, Chung & Lai (STOC 2020) and Coudron & Menda (STOC 2020) showed that these models (with d=log^O(1) (n)) are strictly weaker than BQP (the class of problems solvable by poly-time quantum computation), relative to an oracle, disproving a conjecture of Jozsa in the relativised world. We show that, despite the similarities between CQ_d and QC_d, the two models are incomparable, i.e. CQ_d $\nsubseteq$ QC_d and QC_d $\nsubseteq$ CQ_d relative to an oracle. In other words, there exist problems that one model can solve but not the other and vice versa. We do this by considering new oracle problems that capture the distinctions between the two models and by introducing the notion of an intrinsically stochastic oracle, an oracle whose responses are inherently randomised, which is used for our second result. While we leave showing the second separation relative to a standard oracle as an open problem, we believe the notion of stochastic oracles could be of independent interest for studying complexity classes which have resisted separation in the standard oracle model. Our constructions also yield simpler oracle separations between the hybrid models and BQP, compared to earlier works.
△ Less
Submitted 5 January, 2022;
originally announced January 2022.
-
RoRD: Rotation-Robust Descriptors and Orthographic Views for Local Feature Matching
Authors:
Udit Singh Parihar,
Aniket Gujarathi,
Kinal Mehta,
Satyajit Tourani,
Sourav Garg,
Michael Milford,
K. Madhava Krishna
Abstract:
The use of local detectors and descriptors in typical computer vision pipelines work well until variations in viewpoint and appearance change become extreme. Past research in this area has typically focused on one of two approaches to this challenge: the use of projections into spaces more suitable for feature matching under extreme viewpoint changes, and attempting to learn features that are inhe…
▽ More
The use of local detectors and descriptors in typical computer vision pipelines work well until variations in viewpoint and appearance change become extreme. Past research in this area has typically focused on one of two approaches to this challenge: the use of projections into spaces more suitable for feature matching under extreme viewpoint changes, and attempting to learn features that are inherently more robust to viewpoint change. In this paper, we present a novel framework that combines learning of invariant descriptors through data augmentation and orthographic viewpoint projection. We propose rotation-robust local descriptors, learnt through training data augmentation based on rotation homographies, and a correspondence ensemble technique that combines vanilla feature correspondences with those obtained through rotation-robust features. Using a range of benchmark datasets as well as contributing a new bespoke dataset for this research domain, we evaluate the effectiveness of the proposed approach on key tasks including pose estimation and visual place recognition. Our system outperforms a range of baseline and state-of-the-art techniques, including enabling higher levels of place recognition precision across opposing place viewpoints and achieves practically-useful performance levels even under extreme viewpoint changes.
△ Less
Submitted 24 March, 2022; v1 submitted 15 March, 2021;
originally announced March 2021.
-
Monitoring Large Crowds With WiFi: A Privacy-Preserving Approach
Authors:
Jean-François Determe,
Sophia Azzagnuni,
Utkarsh Singh,
François Horlin,
Philippe De Doncker
Abstract:
This paper presents a crowd monitoring system based on the passive detection of probe requests. The system meets strict privacy requirements and is suited to monitoring events or buildings with a least a few hundreds of attendees. We present our counting process and an associated mathematical model. From this model, we derive a concentration inequality that highlights the accuracy of our crowd cou…
▽ More
This paper presents a crowd monitoring system based on the passive detection of probe requests. The system meets strict privacy requirements and is suited to monitoring events or buildings with a least a few hundreds of attendees. We present our counting process and an associated mathematical model. From this model, we derive a concentration inequality that highlights the accuracy of our crowd count estimator. Then, we describe our system. We present and discuss our sensor hardware, our computing system architecture, and an efficient implementation of our counting algorithm -- as well as its space and time complexity. We also show how our system ensures the privacy of people in the monitored area. Finally, we validate our system using nine weeks of data from a public library endowed with a camera-based counting system, which generates counts against which we compare those of our counting system. This comparison empirically quantifies the accuracy of our counting system, thereby showing it to be suitable for monitoring public areas. Similarly, the concentration inequality provides a theoretical validation of the system.
△ Less
Submitted 18 February, 2022; v1 submitted 20 October, 2020;
originally announced October 2020.
-
Early Bird: Loop Closures from Opposing Viewpoints for Perceptually-Aliased Indoor Environments
Authors:
Satyajit Tourani,
Dhagash Desai,
Udit Singh Parihar,
Sourav Garg,
Ravi Kiran Sarvadevabhatla,
Michael Milford,
K. Madhava Krishna
Abstract:
Significant advances have been made recently in Visual Place Recognition (VPR), feature correspondence, and localization due to the proliferation of deep-learning-based methods. However, existing approaches tend to address, partially or fully, only one of two key challenges: viewpoint change and perceptual aliasing. In this paper, we present novel research that simultaneously addresses both challe…
▽ More
Significant advances have been made recently in Visual Place Recognition (VPR), feature correspondence, and localization due to the proliferation of deep-learning-based methods. However, existing approaches tend to address, partially or fully, only one of two key challenges: viewpoint change and perceptual aliasing. In this paper, we present novel research that simultaneously addresses both challenges by combining deep-learned features with geometric transformations based on reasonable domain assumptions about navigation on a ground-plane, whilst also removing the requirement for specialized hardware setup (e.g. lighting, downwards facing cameras). In particular, our integration of VPR with SLAM by leveraging the robustness of deep-learned features and our homography-based extreme viewpoint invariance significantly boosts the performance of VPR, feature correspondence, and pose graph submodules of the SLAM pipeline. For the first time, we demonstrate a localization system capable of state-of-the-art performance despite perceptual aliasing and extreme 180-degree-rotated viewpoint change in a range of real-world and simulated experiments. Our system is able to achieve early loop closures that prevent significant drifts in SLAM trajectories. We also compare extensively several deep architectures for VPR and descriptor matching. We also show that superior place recognition and descriptor matching across opposite views results in a similar performance gain in back-end pose graph optimization.
△ Less
Submitted 20 December, 2020; v1 submitted 3 October, 2020;
originally announced October 2020.
-
A new operational matrix technique to solve linear boundary value problems
Authors:
Udaya Pratap Singh
Abstract:
A new technique is presented to solve a class of linear boundary value problems (BVP). Technique is primarily based on an operational matrix developed from a set of modified Bernoulli polynomials. The new set of polynomials is an orthonormal set obtained with Gram-Schmidt orthogonalization applied to classical Bernoulli polynomials. The presented method changes a given linear BVP into a system of…
▽ More
A new technique is presented to solve a class of linear boundary value problems (BVP). Technique is primarily based on an operational matrix developed from a set of modified Bernoulli polynomials. The new set of polynomials is an orthonormal set obtained with Gram-Schmidt orthogonalization applied to classical Bernoulli polynomials. The presented method changes a given linear BVP into a system of algebraic equations which is solved to find an approximate solution of BVP in form of a polynomial of required degree. The technique is applied to four problems and obtained approximate solutions are graphically compared to available exact and other numerical solutions. The method is simpler than many existing methods and provides a high degree of accuracy.
△ Less
Submitted 9 August, 2020;
originally announced August 2020.
-
Topological Map** for Manhattan-like Repetitive Environments
Authors:
Sai Shubodh Puligilla,
Satyajit Tourani,
Tushar Vaidya,
Udit Singh Parihar,
Ravi Kiran Sarvadevabhatla,
K. Madhava Krishna
Abstract:
We showcase a topological map** framework for a challenging indoor warehouse setting. At the most abstract level, the warehouse is represented as a Topological Graph where the nodes of the graph represent a particular warehouse topological construct (e.g. rackspace, corridor) and the edges denote the existence of a path between two neighbouring nodes or topologies. At the intermediate level, the…
▽ More
We showcase a topological map** framework for a challenging indoor warehouse setting. At the most abstract level, the warehouse is represented as a Topological Graph where the nodes of the graph represent a particular warehouse topological construct (e.g. rackspace, corridor) and the edges denote the existence of a path between two neighbouring nodes or topologies. At the intermediate level, the map is represented as a Manhattan Graph where the nodes and edges are characterized by Manhattan properties and as a Pose Graph at the lower-most level of detail. The topological constructs are learned via a Deep Convolutional Network while the relational properties between topological instances are learnt via a Siamese-style Neural Network. In the paper, we show that maintaining abstractions such as Topological Graph and Manhattan Graph help in recovering an accurate Pose Graph starting from a highly erroneous and unoptimized Pose Graph. We show how this is achieved by embedding topological and Manhattan relations as well as Manhattan Graph aided loop closure relations as constraints in the backend Pose Graph optimization framework. The recovery of near ground-truth Pose Graph on real-world indoor warehouse scenes vindicate the efficacy of the proposed framework.
△ Less
Submitted 10 March, 2020; v1 submitted 16 February, 2020;
originally announced February 2020.
-
Comparing domain wall synapse with other Non Volatile Memory devices for on-chip learning in Analog Hardware Neural Network
Authors:
Divya Kaushik,
Utkarsh Singh,
Upasana Sahu,
Indu Sreedevi,
Debanjan Bhowmik
Abstract:
Resistive Random Access Memory (RRAM) and Phase Change Memory (PCM) devices have been popularly used as synapses in crossbar array based analog Neural Network (NN) circuit to achieve more energy and time efficient data classification compared to conventional computers. Here we demonstrate the advantages of recently proposed spin orbit torque driven Domain Wall (DW) device as synapse compared to th…
▽ More
Resistive Random Access Memory (RRAM) and Phase Change Memory (PCM) devices have been popularly used as synapses in crossbar array based analog Neural Network (NN) circuit to achieve more energy and time efficient data classification compared to conventional computers. Here we demonstrate the advantages of recently proposed spin orbit torque driven Domain Wall (DW) device as synapse compared to the RRAM and PCM devices with respect to on-chip learning (training in hardware) in such NN. Synaptic characteristic of DW synapse, obtained by us from micromagnetic modeling, turns out to be much more linear and symmetric (between positive and negative update) than that of RRAM and PCM synapse. This makes design of peripheral analog circuits for on-chip learning much easier in DW synapse based NN compared to that for RRAM and PCM synapses. We next incorporate the DW synapse as a Verilog-A model in the crossbar array based NN circuit we design on SPICE circuit simulator. Successful on-chip learning is demonstrated through SPICE simulations on the popular Fisher's Iris dataset. Time and energy required for learning turn out to be orders of magnitude lower for DW synapse based NN circuit compared to that for RRAM and PCM synapse based NN circuits.
△ Less
Submitted 28 October, 2019;
originally announced October 2019.
-
Neural Program Synthesis By Self-Learning
Authors:
Yifan Xu,
Lu Dai,
Udaikaran Singh,
Kening Zhang,
Zhuowen Tu
Abstract:
Neural inductive program synthesis is a task generating instructions that can produce desired outputs from given inputs. In this paper, we focus on the generation of a chunk of assembly code that can be executed to match a state change inside the CPU and RAM. We develop a neural program synthesis algorithm, AutoAssemblet, learned via self-learning reinforcement learning that explores the large cod…
▽ More
Neural inductive program synthesis is a task generating instructions that can produce desired outputs from given inputs. In this paper, we focus on the generation of a chunk of assembly code that can be executed to match a state change inside the CPU and RAM. We develop a neural program synthesis algorithm, AutoAssemblet, learned via self-learning reinforcement learning that explores the large code space efficiently. Policy networks and value networks are learned to reduce the breadth and depth of the Monte Carlo Tree Search, resulting in better synthesis performance. We also propose an effective multi-entropy policy sampling technique to alleviate online update correlations. We apply AutoAssemblet to basic programming tasks and show significant higher success rates compared to several competing baselines.
△ Less
Submitted 13 October, 2019;
originally announced October 2019.
-
On-chip learning in a conventional silicon MOSFET based Analog Hardware Neural Network
Authors:
Nilabjo Dey,
Janak Sharda,
Utkarsh Saxena,
Divya Kaushik,
Utkarsh Singh,
Debanjan Bhowmik
Abstract:
On-chip learning in a crossbar array based analog hardware Neural Network (NN) has been shown to have major advantages in terms of speed and energy compared to training NN on a traditional computer. However analog hardware NN proposals and implementations thus far have mostly involved Non Volatile Memory (NVM) devices like Resistive Random Access Memory (RRAM), Phase Change Memory (PCM), spintroni…
▽ More
On-chip learning in a crossbar array based analog hardware Neural Network (NN) has been shown to have major advantages in terms of speed and energy compared to training NN on a traditional computer. However analog hardware NN proposals and implementations thus far have mostly involved Non Volatile Memory (NVM) devices like Resistive Random Access Memory (RRAM), Phase Change Memory (PCM), spintronic devices or floating gate transistors as synapses. Fabricating systems based on RRAM, PCM or spintronic devices need in-house laboratory facilities and cannot be done through merchant foundries, unlike conventional silicon based CMOS chips. Floating gate transistors need large voltage pulses for weight update, making on-chip learning in such systems energy inefficient. This paper proposes and implements through SPICE simulations on-chip learning in analog hardware NN using only conventional silicon based MOSFETs (without any floating gate) as synapses since they are easy to fabricate. We first model the synaptic characteristic of our single transistor synapse using SPICE circuit simulator and benchmark it against experimentally obtained current-voltage characteristics of a transistor. Next we design a Fully Connected Neural Network (FCNN) crossbar array using such transistor synapses. We also design analog peripheral circuits for neuron and synaptic weight update calculation, needed for on-chip learning, again using conventional transistors. Simulating the entire system on SPICE simulator, we obtain high training and test accuracy on the standard Fisher's Iris dataset, widely used in machine learning. We also compare the speed and energy performance of our transistor based implementation of analog hardware NN with some previous implementations of NN with NVM devices and show comparable performance with respect to on-chip learning.
△ Less
Submitted 1 July, 2019;
originally announced July 2019.
-
InfoRL: Interpretable Reinforcement Learning using Information Maximization
Authors:
Aadil Hayat,
Utsav Singh,
Vinay P. Namboodiri
Abstract:
Recent advances in reinforcement learning have proved that given an environment we can learn to perform a task in that environment if we have access to some form of a reward function (dense, sparse or derived from IRL). But most of the algorithms focus on learning a single best policy to perform a given set of tasks. In this paper, we focus on an algorithm that learns to not just perform a task bu…
▽ More
Recent advances in reinforcement learning have proved that given an environment we can learn to perform a task in that environment if we have access to some form of a reward function (dense, sparse or derived from IRL). But most of the algorithms focus on learning a single best policy to perform a given set of tasks. In this paper, we focus on an algorithm that learns to not just perform a task but different ways to perform the same task. As we know when the environment is complex enough there always exists multiple ways to perform a task. We show that using the concept of information maximization it is possible to learn latent codes for discovering multiple ways to perform any given task in an environment.
△ Less
Submitted 24 May, 2019;
originally announced May 2019.
-
K-Dominant Skyline Join Queries: Extending the Join Paradigm to K-Dominant Skylines
Authors:
Anuradha Awasthi,
Arnab Bhattacharya,
Sanchit Gupta,
Ujjwal Kumar Singh
Abstract:
Skyline queries enable multi-criteria optimization by filtering objects that are worse in all the attributes of interest than another object. To handle the large answer set of skyline queries in high-dimensional datasets, the concept of k-dominance was proposed where an object is said to dominate another object if it is better (or equal) in at least k attributes. This relaxes the full domination c…
▽ More
Skyline queries enable multi-criteria optimization by filtering objects that are worse in all the attributes of interest than another object. To handle the large answer set of skyline queries in high-dimensional datasets, the concept of k-dominance was proposed where an object is said to dominate another object if it is better (or equal) in at least k attributes. This relaxes the full domination criterion of normal skyline queries and, therefore, produces lesser number of skyline objects. This is called the k-dominant skyline set. Many practical applications, however, require that the preferences are applied on a joined relation. Common examples include flights having one or multiple stops, a combination of product price and ship** costs, etc. In this paper, we extend the k-dominant skyline queries to the join paradigm by enabling such queries to be asked on joined relations. We call such queries KSJQ (k-dominant skyline join queries). The number of skyline attributes, k, that an object must dominate is from the combined set of skyline attributes of the joined relation. We show how pre-processing the base relations helps in reducing the time of answering such queries over the naive method of joining the relations first and then running the k-dominant skyline computation. We also extend the query to handle cases where the skyline preference is on aggregated values in the joined relation (such as total cost of the multiple legs of the flight) which are available only after the join is performed. In addition to these problems, we devise efficient algorithms to choose the value of k based on the desired cardinality of the final skyline set. Experiments on both real and synthetic datasets demonstrate the efficiency, scalability and practicality of our algorithms.
△ Less
Submitted 11 February, 2017;
originally announced February 2017.
-
Content Base Image Retrieval Using Phong Shading
Authors:
Uday Pratap Singh,
Sanjeev Jain,
Gulfishan Firdose Ahmed
Abstract:
The digital image data is rapidly expanding in quantity and heterogeneity. The traditional information retrieval techniques does not meet the user's demand, so there is need to develop an efficient system for content based image retrieval. Content based image retrieval means retrieval of images from database on the basis of visual features of image like as color, texture etc. In our proposed metho…
▽ More
The digital image data is rapidly expanding in quantity and heterogeneity. The traditional information retrieval techniques does not meet the user's demand, so there is need to develop an efficient system for content based image retrieval. Content based image retrieval means retrieval of images from database on the basis of visual features of image like as color, texture etc. In our proposed method feature are extracted after applying Phong shading on input image. Phong shading, flattering out the dull surfaces of the image The features are extracted using color, texture & edge density methods. Feature extracted values are used to find the similarity between input query image and the data base image. It can be measure by the Euclidean distance formula. The experimental result shows that the proposed approach has a better retrieval results with phong shading.
△ Less
Submitted 24 May, 2010;
originally announced May 2010.
-
Implementing New-age Authentication Techniques using OpenID for Security Automation
Authors:
Dharmendra Choukse,
Umesh Kumar Singh,
Deepak Sukheja,
Rekha Shahapurkar
Abstract:
Security of any software can be enhanced manifolds if multiple factors for authorization and authentication are used .The main aim of this work was to design and implement an Academy Automation Software for IPS Academy which uses OpenID and Windows CardSpace as Authentication Techniques in addition to Role Based Authentication (RBA) System to ensure that only authentic users can access the predefi…
▽ More
Security of any software can be enhanced manifolds if multiple factors for authorization and authentication are used .The main aim of this work was to design and implement an Academy Automation Software for IPS Academy which uses OpenID and Windows CardSpace as Authentication Techniques in addition to Role Based Authentication (RBA) System to ensure that only authentic users can access the predefined roles as per their Authorization level. The Automation covers different computing hardware and software that can be used to digitally create, manipulate, collect, store, and relay Academy information needed for accomplishing basic Operation like admissions and registration, student and faculty interaction, online library, medical and business development. Raw data storage, electronic transfer, and the management of electronic business information comprise the basic activities of the Academy automation system. Further Transport Layer Security (TLS) protocol has been implemented to provide security and data integrity for communications over networks. TLS encrypts the segments of network connections at the Transport
△ Less
Submitted 7 March, 2010;
originally announced March 2010.
