-
Cross-Embodiment Robot Manipulation Skill Transfer using Latent Space Alignment
Authors:
Tianyu Wang,
Dwait Bhatt,
Xiaolong Wang,
Nikolay Atanasov
Abstract:
This paper focuses on transferring control policies between robot manipulators with different morphology. While reinforcement learning (RL) methods have shown successful results in robot manipulation tasks, transferring a trained policy from simulation to a real robot or deploying it on a robot with different states, actions, or kinematics is challenging. To achieve cross-embodiment policy transfe…
▽ More
This paper focuses on transferring control policies between robot manipulators with different morphology. While reinforcement learning (RL) methods have shown successful results in robot manipulation tasks, transferring a trained policy from simulation to a real robot or deploying it on a robot with different states, actions, or kinematics is challenging. To achieve cross-embodiment policy transfer, our key insight is to project the state and action spaces of the source and target robots to a common latent space representation. We first introduce encoders and decoders to associate the states and actions of the source robot with a latent space. The encoders, decoders, and a latent space control policy are trained simultaneously using loss functions measuring task performance, latent dynamics consistency, and encoder-decoder ability to reconstruct the original states and actions. To transfer the learned control policy, we only need to train target encoders and decoders that align a new target domain to the latent space. We use generative adversarial training with cycle consistency and latent dynamics losses without access to the task reward or reward tuning in the target domain. We demonstrate sim-to-sim and sim-to-real manipulation policy transfer with source and target robots of different states, actions, and embodiments. The source code is available at \url{https://github.com/ExistentialRobotics/cross_embodiment_transfer}.
△ Less
Submitted 4 June, 2024;
originally announced June 2024.
-
Practical Bias Mitigation through Proxy Sensitive Attribute Label Generation
Authors:
Bhushan Chaudhary,
Anubha Pandey,
Deepak Bhatt,
Darshika Tiwari
Abstract:
Addressing bias in the trained machine learning system often requires access to sensitive attributes. In practice, these attributes are not available either due to legal and policy regulations or data unavailability for a given demographic. Existing bias mitigation algorithms are limited in their applicability to real-world scenarios as they require access to sensitive attributes to achieve fairne…
▽ More
Addressing bias in the trained machine learning system often requires access to sensitive attributes. In practice, these attributes are not available either due to legal and policy regulations or data unavailability for a given demographic. Existing bias mitigation algorithms are limited in their applicability to real-world scenarios as they require access to sensitive attributes to achieve fairness. In this research work, we aim to address this bottleneck through our proposed unsupervised proxy-sensitive attribute label generation technique. Towards this end, we propose a two-stage approach of unsupervised embedding generation followed by clustering to obtain proxy-sensitive labels. The efficacy of our work relies on the assumption that bias propagates through non-sensitive attributes that are correlated to the sensitive attributes and, when mapped to the high dimensional latent space, produces clusters of different demographic groups that exist in the data. Experimental results demonstrate that bias mitigation using existing algorithms such as Fair Mixup and Adversarial Debiasing yields comparable results on derived proxy labels when compared against using true sensitive attributes.
△ Less
Submitted 26 December, 2023;
originally announced December 2023.
-
GroupMixNorm Layer for Learning Fair Models
Authors:
Anubha Pandey,
Aditi Rai,
Maneet Singh,
Deepak Bhatt,
Tanmoy Bhowmik
Abstract:
Recent research has identified discriminatory behavior of automated prediction algorithms towards groups identified on specific protected attributes (e.g., gender, ethnicity, age group, etc.). When deployed in real-world scenarios, such techniques may demonstrate biased predictions resulting in unfair outcomes. Recent literature has witnessed algorithms for mitigating such biased behavior mostly b…
▽ More
Recent research has identified discriminatory behavior of automated prediction algorithms towards groups identified on specific protected attributes (e.g., gender, ethnicity, age group, etc.). When deployed in real-world scenarios, such techniques may demonstrate biased predictions resulting in unfair outcomes. Recent literature has witnessed algorithms for mitigating such biased behavior mostly by adding convex surrogates of fairness metrics such as demographic parity or equalized odds in the loss function, which are often not easy to estimate. This research proposes a novel in-processing based GroupMixNorm layer for mitigating bias from deep learning models. The GroupMixNorm layer probabilistically mixes group-level feature statistics of samples across different groups based on the protected attribute. The proposed method improves upon several fairness metrics with minimal impact on overall accuracy. Analysis on benchmark tabular and image datasets demonstrates the efficacy of the proposed method in achieving state-of-the-art performance. Further, the experimental analysis also suggests the robustness of the GroupMixNorm layer against new protected attributes during inference and its utility in eliminating bias from a pre-trained network.
△ Less
Submitted 19 December, 2023;
originally announced December 2023.
-
Visual Semantic Parsing: From Images to Abstract Meaning Representation
Authors:
Mohamed Ashraf Abdelsalam,
Zhan Shi,
Federico Fancellu,
Kalliopi Basioti,
Dhaivat J. Bhatt,
Vladimir Pavlovic,
Afsaneh Fazly
Abstract:
The success of scene graphs for visual scene understanding has brought attention to the benefits of abstracting a visual input (e.g., image) into a structured representation, where entities (people and objects) are nodes connected by edges specifying their relations. Building these representations, however, requires expensive manual annotation in the form of images paired with their scene graphs o…
▽ More
The success of scene graphs for visual scene understanding has brought attention to the benefits of abstracting a visual input (e.g., image) into a structured representation, where entities (people and objects) are nodes connected by edges specifying their relations. Building these representations, however, requires expensive manual annotation in the form of images paired with their scene graphs or frames. These formalisms remain limited in the nature of entities and relations they can capture. In this paper, we propose to leverage a widely-used meaning representation in the field of natural language processing, the Abstract Meaning Representation (AMR), to address these shortcomings. Compared to scene graphs, which largely emphasize spatial relationships, our visual AMR graphs are more linguistically informed, with a focus on higher-level semantic concepts extrapolated from visual input. Moreover, they allow us to generate meta-AMR graphs to unify information contained in multiple image descriptions under one representation. Through extensive experimentation and analysis, we demonstrate that we can re-purpose an existing text-to-AMR parser to parse images into AMRs. Our findings point to important future research directions for improved scene understanding.
△ Less
Submitted 27 October, 2022; v1 submitted 26 October, 2022;
originally announced October 2022.
-
Graph2Vid: Flow graph to Video Grounding for Weakly-supervised Multi-Step Localization
Authors:
Nikita Dvornik,
Isma Hadji,
Hai Pham,
Dhaivat Bhatt,
Brais Martinez,
Afsaneh Fazly,
Allan D. Jepson
Abstract:
In this work, we consider the problem of weakly-supervised multi-step localization in instructional videos. An established approach to this problem is to rely on a given list of steps. However, in reality, there is often more than one way to execute a procedure successfully, by following the set of steps in slightly varying orders. Thus, for successful localization in a given video, recent works r…
▽ More
In this work, we consider the problem of weakly-supervised multi-step localization in instructional videos. An established approach to this problem is to rely on a given list of steps. However, in reality, there is often more than one way to execute a procedure successfully, by following the set of steps in slightly varying orders. Thus, for successful localization in a given video, recent works require the actual order of procedure steps in the video, to be provided by human annotators at both training and test times. Instead, here, we only rely on generic procedural text that is not tied to a specific video. We represent the various ways to complete the procedure by transforming the list of instructions into a procedure flow graph which captures the partial order of steps. Using the flow graphs reduces both training and test time annotation requirements. To this end, we introduce the new problem of flow graph to video grounding. In this setup, we seek the optimal step ordering consistent with the procedure flow graph and a given video. To solve this problem, we propose a new algorithm - Graph2Vid - that infers the actual ordering of steps in the video and simultaneously localizes them. To show the advantage of our proposed formulation, we extend the CrossTask dataset with procedure flow graph information. Our experiments show that Graph2Vid is both more efficient than the baselines and yields strong step localization results, without the need for step order annotation.
△ Less
Submitted 31 October, 2022; v1 submitted 10 October, 2022;
originally announced October 2022.
-
Adversarial synthesis based data-augmentation for code-switched spoken language identification
Authors:
Parth Shastri,
Chirag Patil,
Poorval Wanere,
Dr. Shrinivas Mahajan,
Dr. Abhishek Bhatt,
Dr. Hardik Sailor
Abstract:
Spoken Language Identification (LID) is an important sub-task of Automatic Speech Recognition(ASR) that is used to classify the language(s) in an audio segment. Automatic LID plays an useful role in multilingual countries. In various countries, identifying a language becomes hard, due to the multilingual scenario where two or more than two languages are mixed together during conversation. Such phe…
▽ More
Spoken Language Identification (LID) is an important sub-task of Automatic Speech Recognition(ASR) that is used to classify the language(s) in an audio segment. Automatic LID plays an useful role in multilingual countries. In various countries, identifying a language becomes hard, due to the multilingual scenario where two or more than two languages are mixed together during conversation. Such phenomenon of speech is called as code-mixing or code-switching. This nature is followed not only in India but also in many Asian countries. Such code-mixed data is hard to find, which further reduces the capabilities of the spoken LID. Hence, this work primarily addresses this problem using data augmentation as a solution on the on the data scarcity of the code-switched class. This study focuses on Indic language code-mixed with English. Spoken LID is performed on Hindi, code-mixed with English. This research proposes Generative Adversarial Network (GAN) based data augmentation technique performed using Mel spectrograms for audio data. GANs have already been proven to be accurate in representing the real data distribution in the image domain. Proposed research exploits these capabilities of GANs in speech domains such as speech classification, automatic speech recognition, etc. GANs are trained to generate Mel spectrograms of the minority code-mixed class which are then used to augment data for the classifier. Utilizing GANs give an overall improvement on Unweighted Average Recall by an amount of 3.5% as compared to a Convolutional Recurrent Neural Network (CRNN) classifier used as the baseline reference.
△ Less
Submitted 1 June, 2022; v1 submitted 30 May, 2022;
originally announced May 2022.
-
DesCert: Design for Certification
Authors:
Natarajan Shankar,
Devesh Bhatt,
Michael Ernst,
Minyoung Kim,
Srivatsan Varadarajan,
Suzanne Millstein,
Jorge Navas,
Jason Biatek,
Huascar Sanchez,
Anitha Murugesan,
Hao Ren
Abstract:
The goal of the DARPA Automated Rapid Certification Of Software (ARCOS) program is to "automate the evaluation of software assurance evidence to enable certifiers to determine rapidly that system risk is acceptable." As part of this program, the DesCert project focuses on the assurance-driven development of new software. The DesCert team consists of SRI International, Honeywell Research, and the U…
▽ More
The goal of the DARPA Automated Rapid Certification Of Software (ARCOS) program is to "automate the evaluation of software assurance evidence to enable certifiers to determine rapidly that system risk is acceptable." As part of this program, the DesCert project focuses on the assurance-driven development of new software. The DesCert team consists of SRI International, Honeywell Research, and the University of Washington. We have adopted a formal, tool-based approach to the construction of software artifacts that are supported by rigorous evidence. The DesCert workflow integrates evidence generation into a design process that goes from requirements capture and analysis to the decomposition of the high-level software requirements into architecture properties and software components with assertional contracts, and on to software that can be analyzed both dynamically and statically. The generated evidence is organized by means of an assurance ontology and integrated into the RACK knowledge base.
△ Less
Submitted 28 March, 2022;
originally announced March 2022.
-
Cascading Adaptors to Leverage English Data to Improve Performance of Question Answering for Low-Resource Languages
Authors:
Hariom A. Pandya,
Bhavik Ardeshna,
Dr. Brijesh S. Bhatt
Abstract:
Transformer based architectures have shown notable results on many down streaming tasks including question answering. The availability of data, on the other hand, impedes obtaining legitimate performance for low-resource languages. In this paper, we investigate the applicability of pre-trained multilingual models to improve the performance of question answering in low-resource languages. We tested…
▽ More
Transformer based architectures have shown notable results on many down streaming tasks including question answering. The availability of data, on the other hand, impedes obtaining legitimate performance for low-resource languages. In this paper, we investigate the applicability of pre-trained multilingual models to improve the performance of question answering in low-resource languages. We tested four combinations of language and task adapters using multilingual transformer architectures on seven languages similar to MLQA dataset. Additionally, we have also proposed zero-shot transfer learning of low-resource question answering using language and task adapters. We observed that stacking the language and the task adapters improves the multilingual transformer models' performance significantly for low-resource languages.
△ Less
Submitted 18 December, 2021;
originally announced December 2021.
-
Morpheme Boundary Detection & Grammatical Feature Prediction for Gujarati : Dataset & Model
Authors:
Jatayu Baxi,
Dr. Brijesh Bhatt
Abstract:
Develo** Natural Language Processing resources for a low resource language is a challenging but essential task. In this paper, we present a Morphological Analyzer for Gujarati. We have used a Bi-Directional LSTM based approach to perform morpheme boundary detection and grammatical feature tagging. We have created a data set of Gujarati words with lemma and grammatical features. The Bi-LSTM based…
▽ More
Develo** Natural Language Processing resources for a low resource language is a challenging but essential task. In this paper, we present a Morphological Analyzer for Gujarati. We have used a Bi-Directional LSTM based approach to perform morpheme boundary detection and grammatical feature tagging. We have created a data set of Gujarati words with lemma and grammatical features. The Bi-LSTM based model of Morph Analyzer discussed in the paper handles the language morphology effectively without the knowledge of any hand-crafted suffix rules. To the best of our knowledge, this is the first dataset and morph analyzer model for the Gujarati language which performs both grammatical feature tagging and morpheme boundary detection tasks.
△ Less
Submitted 18 December, 2021;
originally announced December 2021.
-
$f$-Cal: Calibrated aleatoric uncertainty estimation from neural networks for robot perception
Authors:
Dhaivat Bhatt,
Kaustubh Mani,
Dishank Bansal,
Krishna Murthy,
Hanju Lee,
Liam Paull
Abstract:
While modern deep neural networks are performant perception modules, performance (accuracy) alone is insufficient, particularly for safety-critical robotic applications such as self-driving vehicles. Robot autonomy stacks also require these otherwise blackbox models to produce reliable and calibrated measures of confidence on their predictions. Existing approaches estimate uncertainty from these n…
▽ More
While modern deep neural networks are performant perception modules, performance (accuracy) alone is insufficient, particularly for safety-critical robotic applications such as self-driving vehicles. Robot autonomy stacks also require these otherwise blackbox models to produce reliable and calibrated measures of confidence on their predictions. Existing approaches estimate uncertainty from these neural network perception stacks by modifying network architectures, inference procedure, or loss functions. However, in general, these methods lack calibration, meaning that the predictive uncertainties do not faithfully represent the true underlying uncertainties (process noise). Our key insight is that calibration is only achieved by imposing constraints across multiple examples, such as those in a mini-batch; as opposed to existing approaches which only impose constraints per-sample, often leading to overconfident (thus miscalibrated) uncertainty estimates. By enforcing the distribution of outputs of a neural network to resemble a target distribution by minimizing an $f$-divergence, we obtain significantly better-calibrated models compared to prior approaches. Our approach, $f$-Cal, outperforms existing uncertainty calibration approaches on robot perception tasks such as object detection and monocular depth estimation over multiple real-world benchmarks.
△ Less
Submitted 28 September, 2021;
originally announced September 2021.
-
Knowledge-Assisted Reasoning of Model-Augmented System Requirements with Event Calculus and Goal-Directed Answer Set Programming
Authors:
Brendan Hall,
Sarat Chandra Varanasi,
Jan Fiedor,
Joaquín Arias,
Kinjal Basu,
Fang Li,
Devesh Bhatt,
Kevin Driscoll,
Elmer Salazar,
Gopal Gupta
Abstract:
We consider requirements for cyber-physical systems represented in constrained natural language. We present novel automated techniques for aiding in the development of these requirements so that they are consistent and can withstand perceived failures. We show how cyber-physical systems' requirements can be modeled using the event calculus (EC), a formalism used in AI for representing actions and…
▽ More
We consider requirements for cyber-physical systems represented in constrained natural language. We present novel automated techniques for aiding in the development of these requirements so that they are consistent and can withstand perceived failures. We show how cyber-physical systems' requirements can be modeled using the event calculus (EC), a formalism used in AI for representing actions and change. We also show how answer set programming (ASP) and its query-driven implementation s(CASP) can be used to directly realize the event calculus model of the requirements. This event calculus model can be used to automatically validate the requirements. Since ASP is an expressive knowledge representation language, it can also be used to represent contextual knowledge about cyber-physical systems, which, in turn, can be used to find gaps in their requirements specifications. We illustrate our approach through an altitude alerting system from the avionics domain.
△ Less
Submitted 9 September, 2021;
originally announced September 2021.
-
Multi-Modal Image Captioning for the Visually Impaired
Authors:
Hiba Ahsan,
Nikita Bhalla,
Daivat Bhatt,
Kaivankumar Shah
Abstract:
One of the ways blind people understand their surroundings is by clicking images and relying on descriptions generated by image captioning systems. Current work on captioning images for the visually impaired do not use the textual data present in the image when generating captions. This problem is critical as many visual scenes contain text. Moreover, up to 21% of the questions asked by blind peop…
▽ More
One of the ways blind people understand their surroundings is by clicking images and relying on descriptions generated by image captioning systems. Current work on captioning images for the visually impaired do not use the textual data present in the image when generating captions. This problem is critical as many visual scenes contain text. Moreover, up to 21% of the questions asked by blind people about the images they click pertain to the text present in them. In this work, we propose altering AoANet, a state-of-the-art image captioning model, to leverage the text detected in the image as an input feature. In addition, we use a pointer-generator mechanism to copy the detected text to the caption when tokens need to be reproduced accurately. Our model outperforms AoANet on the benchmark dataset VizWiz, giving a 35% and 16.2% performance improvement on CIDEr and SPICE scores, respectively.
△ Less
Submitted 17 May, 2021;
originally announced May 2021.
-
Transitioning from Real to Synthetic data: Quantifying the bias in model
Authors:
Aman Gupta,
Deepak Bhatt,
Anubha Pandey
Abstract:
With the advent of generative modeling techniques, synthetic data and its use has penetrated across various domains from unstructured data such as image, text to structured dataset modeling healthcare outcome, risk decisioning in financial domain, and many more. It overcomes various challenges such as limited training data, class imbalance, restricted access to dataset owing to privacy issues. To…
▽ More
With the advent of generative modeling techniques, synthetic data and its use has penetrated across various domains from unstructured data such as image, text to structured dataset modeling healthcare outcome, risk decisioning in financial domain, and many more. It overcomes various challenges such as limited training data, class imbalance, restricted access to dataset owing to privacy issues. To ensure the trained model used for automated decisioning purposes makes a fair decision there exist prior work to quantify and mitigate those issues. This study aims to establish a trade-off between bias and fairness in the models trained using synthetic data. Variants of synthetic data generation techniques were studied to understand bias amplification including differentially private generation schemes. Through experiments on a tabular dataset, we demonstrate there exist a varying levels of bias impact on models trained using synthetic data. Techniques generating less correlated feature performs well as evident through fairness metrics with 94\%, 82\%, and 88\% relative drop in DPD (demographic parity difference), EoD (equality of odds) and EoP (equality of opportunity) respectively, and 24\% relative improvement in DRP (demographic parity ratio) with respect to the real dataset. We believe the outcome of our research study will help data science practitioners understand the bias in the use of synthetic data.
△ Less
Submitted 10 May, 2021;
originally announced May 2021.
-
Volterra model-based control for nonlinear systems via Carleman linearization
Authors:
Dhruvi Bhatt,
Shambhu Nath Sharma
Abstract:
This paper presents detailed insights of embedding Carleman linearization into nonlinear systems for designing Volterra model-based control technique. Volterra series method is a competent mathematical tool, which extends the convolution integral for linear systems to nonlinear systems. First, we utilize the Carleman linearization technique to arrive at the bilinear approximation of the nonlinear…
▽ More
This paper presents detailed insights of embedding Carleman linearization into nonlinear systems for designing Volterra model-based control technique. Volterra series method is a competent mathematical tool, which extends the convolution integral for linear systems to nonlinear systems. First, we utilize the Carleman linearization technique to arrive at the bilinear approximation of the nonlinear system. Secondly, the third-order Volterra model representation is computed from the Carleman bilinearized model. Then, Volterra model-based control strategy is developed. The proposed method is effectuated on the benchmark van de Vusse reactor exhibiting non-minimum phase response. The open and closed-loop simulation results are presented, demonstrating the superiority and practical utility of the proposed method.
△ Less
Submitted 2 January, 2021;
originally announced January 2021.
-
Abstractive Information Extraction from Scanned Invoices (AIESI) using End-to-end Sequential Approach
Authors:
Shreeshiv Patel,
Dvijesh Bhatt
Abstract:
Recent proliferation in the field of Machine Learning and Deep Learning allows us to generate OCR models with higher accuracy. Optical Character Recognition(OCR) is the process of extracting text from documents and scanned images. For document data streamlining, we are interested in data like, Payee name, total amount, address, and etc. Extracted information helps to get complete insight of data,…
▽ More
Recent proliferation in the field of Machine Learning and Deep Learning allows us to generate OCR models with higher accuracy. Optical Character Recognition(OCR) is the process of extracting text from documents and scanned images. For document data streamlining, we are interested in data like, Payee name, total amount, address, and etc. Extracted information helps to get complete insight of data, which can be helpful for fast document searching, efficient indexing in databases, data analytics, and etc. Using AIESI we can eliminate human effort for key parameters extraction from scanned documents. Abstract Information Extraction from Scanned Invoices (AIESI) is a process of extracting information like, date, total amount, payee name, and etc from scanned receipts. In this paper we proposed an improved method to ensemble all visual and textual features from invoices to extract key invoice parameters using Word wise BiLSTM.
△ Less
Submitted 12 September, 2020;
originally announced September 2020.
-
Filtering theory for a weakly coloured noise process
Authors:
Shaival H. Nagarsheth,
Dhruvi S. Bhatt,
Shambhu N. Sharma
Abstract:
The problem of analyzing the Ito stochastic differential system and its filtering has received attention. The classical approach to accomplish filtering for the Ito SDE is the Kushner equation. In contrast to the classical filtering approach, this paper presents filtering for the stochastic differential system affected by weakly coloured noise. As a special case, the process can be regarded as the…
▽ More
The problem of analyzing the Ito stochastic differential system and its filtering has received attention. The classical approach to accomplish filtering for the Ito SDE is the Kushner equation. In contrast to the classical filtering approach, this paper presents filtering for the stochastic differential system affected by weakly coloured noise. As a special case, the process can be regarded as the Ornstein-Uhlenbeck (OU) process. The theory of this paper is based on a pioneering contribution of Stratonovich involving the perturbation-theoretic approach to noisy dynamical systems in combination with the notion of the filtering density evolution. Making the use of the filtering density evolution equation, the stochastic evolution of condition moment is derived. A scalar Duffing system driven by the OU process is employed to test the effectiveness of the filtering theory of the paper. Numerical simulations involving four different sets of initial conditions and system parameters are utilized to examine the efficacy of the filtering algorithm of this paper.
△ Less
Submitted 12 October, 2019;
originally announced October 2019.
-
Estimation of the van de Vusse reactor via Carleman embedding
Authors:
Dhruvi S. Bhatt,
Shambhu N. Sharma
Abstract:
The van de Vusse reactor is an appealing benchmark problem in industrial control, since it has a non-minimum phase response. The van de Vusse stochasticity is attributed to the fluctuating input flow rate. The novelties of the paper are two. First, we utilize the surprising power of Ito stochastic calculus for applications to account for the van de Vusse stochasticity. Secondly, the Carleman embed…
▽ More
The van de Vusse reactor is an appealing benchmark problem in industrial control, since it has a non-minimum phase response. The van de Vusse stochasticity is attributed to the fluctuating input flow rate. The novelties of the paper are two. First, we utilize the surprising power of Ito stochastic calculus for applications to account for the van de Vusse stochasticity. Secondly, the Carleman embedding is unified with the Fokker-Planck equation for finding the estimation of the van de Vusse reactor. The revelation of the paper is that the Carleman linearized estimate of the van de Vusse reactor is more refined in contrast to the EKF predicted estimate. This paper will be useful to practitioners aspiring for formal methods for stochastically perturbed nonlinear reactors as well as system theorists aspiring for applications of their theoretical results to practical problems.
△ Less
Submitted 28 September, 2019;
originally announced September 2019.
-
Deep Active Localization
Authors:
Sai Krishna,
Keehong Seo,
Dhaivat Bhatt,
Vincent Mai,
Krishna Murthy,
Liam Paull
Abstract:
Active localization is the problem of generating robot actions that allow it to maximally disambiguate its pose within a reference map. Traditional approaches to this use an information-theoretic criterion for action selection and hand-crafted perceptual models. In this work we propose an end-to-end differentiable method for learning to take informative actions that is trainable entirely in simula…
▽ More
Active localization is the problem of generating robot actions that allow it to maximally disambiguate its pose within a reference map. Traditional approaches to this use an information-theoretic criterion for action selection and hand-crafted perceptual models. In this work we propose an end-to-end differentiable method for learning to take informative actions that is trainable entirely in simulation and then transferable to real robot hardware with zero refinement. The system is composed of two modules: a convolutional neural network for perception, and a deep reinforcement learned planning module. We introduce a multi-scale approach to the learned perceptual model since the accuracy needed to perform action selection with reinforcement learning is much less than the accuracy needed for robot control. We demonstrate that the resulting system outperforms using the traditional approach for either perception or planning. We also demonstrate our approaches robustness to different map configurations and other nuisance parameters through the use of domain randomization in training. The code is also compatible with the OpenAI gym framework, as well as the Gazebo simulator.
△ Less
Submitted 5 March, 2019;
originally announced March 2019.
-
Detection of self-generated nanowaves on the interface of an evaporating sessile water droplet
Authors:
Dhanush Bhatt,
Rahul Vaippully,
Bhavesh Kharbanda,
Anand Dev Ranjan,
Sulochana R.,
Viraj Dharod,
Basudev Roy
Abstract:
Evaporating sessile droplets have been known to exhibit oscillations on the air-liquid interface. These are generally over millimeter scales. Using a novel approach, we are able to measure surface height changes of 500 nm amplitude using optical trap** of a set of microscopic particles at the interface, particularly when the vertical thickness of the droplet reduces to less than 50 $μ$m. We find…
▽ More
Evaporating sessile droplets have been known to exhibit oscillations on the air-liquid interface. These are generally over millimeter scales. Using a novel approach, we are able to measure surface height changes of 500 nm amplitude using optical trap** of a set of microscopic particles at the interface, particularly when the vertical thickness of the droplet reduces to less than 50 $μ$m. We find that at the later stages of the droplet evaporation, particularly when the convection currents become large, the top air-water interface starts to spontaneously oscillate vertically as a function of time in consistency with predictions. We also detect travelling wave trains moving in the azimuthal direction of the drop surface which are consistent with hydrothermal waves at a different combination of Reynolds, Prandtl and Evaporation than previously observed. This is the first time that wave-trains have been observed in water, being extremely challenging to detect both interferometrically and with infra-red cameras. We also find that such waves apply a force parallel to the interface along the propagation direction.
△ Less
Submitted 5 February, 2019;
originally announced February 2019.
-
Study of adhesivity of surfaces using rotational optical tweezers
Authors:
Rahul Vaipully,
Dhanush Bhatt,
Anand Dev Ranjan,
Basudev Roy
Abstract:
Optical tweezers are powerful tools for high resolution study of surface properties. Such experiments are traditionally performed by studying the active or the brownian fluctuation of trapped particles in the X, Y, Z direction. Here we find that employing the fourth dimension, rotation, allows for sensitive and fast probing of the surface. Optical tweezers are capable of rotating trapped birefring…
▽ More
Optical tweezers are powerful tools for high resolution study of surface properties. Such experiments are traditionally performed by studying the active or the brownian fluctuation of trapped particles in the X, Y, Z direction. Here we find that employing the fourth dimension, rotation, allows for sensitive and fast probing of the surface. Optical tweezers are capable of rotating trapped birefringent microparticles when applied with circularly polarized light, thus called the Rotational Optical Tweezers. When the trapped birefringent microparticle is far enough away from the surface, the rotation rate is dependent only on the laser power. However, we find that if one traps close to a surface, the rotation rate goes to zero even at finite tweezers laser powers for some specific type of substrates. We suspect this to be due to interaction between the substrate and the birefringent particle, kee** in mind that the hydrodynamic drag for this mode of rotation cannot increase beyond 1.2 times the drag away from the surface. We use this to probe some surfaces and find that there is no binding for hydrophobic ones but hydrophilic ones particularly tend to show a power threshold beyond which the birefringent particle starts rotating. We calculate that the threshold energy of the tweezers is consistent with the Van der Waals potential energy, when the mode of interaction with the surface is purely physical. We also find that for chitosan, the mode of interaction is possibly different from Van der Waals. We place the particle on the threshold and observe stick-slip kind of rotational behaviour.
△ Less
Submitted 10 November, 2018;
originally announced November 2018.
-
Chance Constraints Integrated MPC Navigation in Uncertainty amongst Dynamic Obstacles: An overlap of Gaussians approach
Authors:
Dhaivat Bhatt,
Akash Garg,
Bharath Gopalakrishnan,
K. Madhava Krishna
Abstract:
In this paper, we formulate a novel trajectory optimization scheme that takes into consideration the state uncertainty of the robot and obstacle into its collision avoidance routine. The collision avoidance under uncertainty is modeled here as an overlap between two distributions that represent the state of the robot and obstacle respectively. We adopt the minmax procedure to characterize the area…
▽ More
In this paper, we formulate a novel trajectory optimization scheme that takes into consideration the state uncertainty of the robot and obstacle into its collision avoidance routine. The collision avoidance under uncertainty is modeled here as an overlap between two distributions that represent the state of the robot and obstacle respectively. We adopt the minmax procedure to characterize the area of overlap between two Gaussian distributions, and compare it with the method of Bhattacharyya distance. We provide closed form expressions that can characterize the overlap as a function of control. Our proposed algorithm can avoid overlap** uncertainty distributions in two possible ways. Firstly when a prescribed overlap** area that needs to be avoided is posed as a confidence contour lower bound, control commands are accordingly realized through a MPC framework such that these bounds are respected. Secondly in tight spaces control commands are computed such that the overlap** distribution respects a prescribed range of overlap characterized by lower and upper bounds of the confidence contours. We test our proposal with extensive set of simulations carried out under various constrained environmental configurations. We show usefulness of proposal under tight spaces where finding control maneuvers with minimal risk behavior becomes an inevitable task.
△ Less
Submitted 26 June, 2018;
originally announced June 2018.
-
A Study of Local Binary Pattern Method for Facial Expression Detection
Authors:
Ms. Drashti H. Bhatt,
Mr. Kirit R. Rathod,
Mr. Shardul J. Agravat
Abstract:
Face detection is a basic task for expression recognition. The reliability of face detection & face recognition approach has a major role on the performance and usability of the entire system. There are several ways to undergo face detection & recognition. We can use Image Processing Operations, various classifiers, filters or virtual machines for the former. Various strategies are being available…
▽ More
Face detection is a basic task for expression recognition. The reliability of face detection & face recognition approach has a major role on the performance and usability of the entire system. There are several ways to undergo face detection & recognition. We can use Image Processing Operations, various classifiers, filters or virtual machines for the former. Various strategies are being available for Facial Expression Detection. The field of facial expression detection can have various applications along with its importance & can be interacted between human being & computer. Many few options are available to identify a face in an image in accurate & efficient manner. Local Binary Pattern (LBP) based texture algorithms have gained popularity in these years. LBP is an effective approach to have facial expression recognition & is a feature-based approach.
△ Less
Submitted 4 February, 2014;
originally announced May 2014.
-
Biomolecular transitions: efficient computation of pathways, free energies, and rates
Authors:
Divesh Bhatt,
Ivet Bahar
Abstract:
We present an efficient method to compute transition rates between states for a two-state system. The method utilizes the equivalence between steady-state flux and mean first passage rate for such systems. More specifically, the procedure divides the configurational space into smaller regions and equilibrates trajectories within each region efficiently. The equilibrated conditional probabilities b…
▽ More
We present an efficient method to compute transition rates between states for a two-state system. The method utilizes the equivalence between steady-state flux and mean first passage rate for such systems. More specifically, the procedure divides the configurational space into smaller regions and equilibrates trajectories within each region efficiently. The equilibrated conditional probabilities between each pair of regions lead to transition rates between the two states. We apply the procedure to a non-trivial coarse-grained model of a 70 residue section of the calcium binding protein, calmodulin. The procedure yields a significant increase in efficiency compared to brute-force simulations, and this efficiency increases dramatically with a decrease in temperature.
△ Less
Submitted 4 September, 2011;
originally announced September 2011.
-
Stochastic modeling of p53-regulated apoptosis upon radiation damage
Authors:
Divesh Bhatt,
Zoltan Oltvai,
Ivet Bahar
Abstract:
We develop and study the evolution of a model of radiation induced apoptosis in cells using stochastic simulations, and identified key protein targets for effective mitigation of radiation damage. We identified several key proteins associated with cellular apoptosis using an extensive literature survey. In particular, we focus on the p53 transcription dependent and p53 transcription independent pa…
▽ More
We develop and study the evolution of a model of radiation induced apoptosis in cells using stochastic simulations, and identified key protein targets for effective mitigation of radiation damage. We identified several key proteins associated with cellular apoptosis using an extensive literature survey. In particular, we focus on the p53 transcription dependent and p53 transcription independent pathways for mitochondrial apoptosis. Our model reproduces known p53 oscillations following radiation damage. The key, experimentally testable hypotheses that we generate are - inhibition of PUMA is an effective strategy for mitigation of radiation damage if the treatment is administered immediately, at later stages following radiation damage, inhibition of tBid is more effective.
△ Less
Submitted 4 September, 2011;
originally announced September 2011.
-
Automated sampling assessment for molecular simulations using the effective sample size
Authors:
Xin Zhang,
Divesh Bhatt,
Daniel M. Zuckerman
Abstract:
To quantify the progress in development of algorithms and forcefields used in molecular simulations, a method for the assessment of the sampling quality is needed. We propose a general method to assess the sampling quality through the estimation of the number of independent samples obtained from molecular simulations. This method is applicable to both dynamic and nondynamic methods and utilizes…
▽ More
To quantify the progress in development of algorithms and forcefields used in molecular simulations, a method for the assessment of the sampling quality is needed. We propose a general method to assess the sampling quality through the estimation of the number of independent samples obtained from molecular simulations. This method is applicable to both dynamic and nondynamic methods and utilizes the variance in the populations of physical states to determine the ESS. We test the correctness and robustness of our procedure in a variety of systems--two-state toy model, all-atom butane, coarse-grained calmodulin, all-atom dileucine and Met-enkaphalin.
We also introduce an automated procedure to obtain approximate physical states from dynamic trajectories: this procedure allows for sample--size estimation for systems for which physical states are not known in advance.
△ Less
Submitted 19 February, 2010;
originally announced February 2010.
-
Symmetry of forward and reverse path populations
Authors:
Divesh Bhatt,
Daniel M. Zuckerman
Abstract:
In this note, we address formally the issue of symmetry for probabilities of different dynamical pathways in the forward and reverse directions of a conformational transition. Our discussion is based on a decomposition of equilibrium into opposing steady states, and makes clear the conditions necessary for symmetry to apply. From a practical point of view, we also discuss when approximate symmet…
▽ More
In this note, we address formally the issue of symmetry for probabilities of different dynamical pathways in the forward and reverse directions of a conformational transition. Our discussion is based on a decomposition of equilibrium into opposing steady states, and makes clear the conditions necessary for symmetry to apply. From a practical point of view, we also discuss when approximate symmetry is to be expected.
△ Less
Submitted 11 February, 2010; v1 submitted 11 February, 2010;
originally announced February 2010.
-
Steady-state simulations using weighted ensemble path sampling
Authors:
Divesh Bhatt,
Bin W. Zhang,
Daniel M. Zuckerman
Abstract:
We extend the weighted ensemble (WE) path sampling method to perform rigorous statistical sampling for systems at steady state. The straightforward steady-state implementation of WE is directly practical for simple landscapes, but not when significant metastable intermediates states are present. We therefore develop an enhanced WE scheme, building on existing ideas, which accelerates attainment…
▽ More
We extend the weighted ensemble (WE) path sampling method to perform rigorous statistical sampling for systems at steady state. The straightforward steady-state implementation of WE is directly practical for simple landscapes, but not when significant metastable intermediates states are present. We therefore develop an enhanced WE scheme, building on existing ideas, which accelerates attainment of steady state in complex systems. We apply both WE approaches to several model systems confirming their correctness and efficiency by comparison with brute-force results. The enhanced version is significantly faster than the brute force and straightforward WE for systems with WE bins that accurately reflect the reaction coordinate(s). The new WE methods can also be applied to equilibrium sampling, since equilibrium is a steady state.
△ Less
Submitted 28 February, 2010; v1 submitted 27 October, 2009;
originally announced October 2009.
-
Thermal Motions of the E. Coli Glucose-Galactose Binding Protein Studied Using Well-Sampled Semi-Atomistic Simulations
Authors:
Derek J. Cashman,
Artem B. Mamonov,
Divesh Bhatt,
Daniel M. Zuckerman
Abstract:
The E. coli glucose-galactose chemosensory receptor is a 309 residue, 32 kDa protein consisting of two distinct structural domains. In this computational study, we studied the protein's thermal fluctuations, including both the large scale interdomain movements that contribute to the receptor's mechanism of action, as well as smaller scale motions, using two different computational methods. We em…
▽ More
The E. coli glucose-galactose chemosensory receptor is a 309 residue, 32 kDa protein consisting of two distinct structural domains. In this computational study, we studied the protein's thermal fluctuations, including both the large scale interdomain movements that contribute to the receptor's mechanism of action, as well as smaller scale motions, using two different computational methods. We employ extremely fast, "semi-atomistic" Library-Based Monte Carlo (LBMC) simulations, which include all backbone atoms but "implicit" side chains. Our results were compared with previous experiments and an all-atom Langevin dynamics simulation. Both LBMC and Langevin dynamics simulations were performed using both the apo and glucose-bound form of the protein, with LBMC exhibiting significantly larger fluctuations. The LBMC simulations are also in general agreement with the disulfide trap** experiments of Careaga & Falke (JMB, 1992; Biophys. J., 1992), which indicate that distant residues in the crystal structure (i.e. beta carbons separated by 10 to 20 angstroms) form spontaneous transient contacts in solution. Our simulations illustrate several possible "mechanisms" (configurational pathways) for these fluctuations. We also observe several discrepancies between our calculations and experiment. Nevertheless, we believe that our semi-atomistic approach could be used to study the fluctuations in other proteins, perhaps for ensemble docking, or other analyses of protein flexibility in virtual screening studies.
△ Less
Submitted 27 October, 2009;
originally announced October 2009.
-
Heterogeneous path ensembles for conformational transitions in semi-atomistic models of adenylate kinase
Authors:
Divesh Bhatt,
Daniel M. Zuckerman
Abstract:
We performed "weighted ensemble" path-sampling simulations of adenylate kinase, using several semi-atomistic protein models. Our study investigated both the biophysics of conformational transitions as well as the possibility of increasing model accuracy without sacrificing good sampling. Biophysically, the path ensembles show significant heterogeneity and the explicit possibility of two principl…
▽ More
We performed "weighted ensemble" path-sampling simulations of adenylate kinase, using several semi-atomistic protein models. Our study investigated both the biophysics of conformational transitions as well as the possibility of increasing model accuracy without sacrificing good sampling. Biophysically, the path ensembles show significant heterogeneity and the explicit possibility of two principle pathways in the Open-Closed transition. We recently showed, under certain conditions, a "symmetry of hetereogeneity" is expected between the forward and the reverse transitions: the fraction of transitions taking a specific pathway/channel will be the same in both the directions. Our path ensembles are analyzed in the light of the symmetry relation and its conditions. In the realm of modeling, we employed an all-atom backbone with various levels of residue interactions. Because reasonable path sampling required only a few weeks of single-processor computing time with these models, the addition of further chemical detail should be feasible.
△ Less
Submitted 25 February, 2010; v1 submitted 8 October, 2009;
originally announced October 2009.
-
A library-based Monte Carlo technique enables rapid equilibrium sampling of a protein model with atomistic components
Authors:
Artem B. Mamonov,
Divesh Bhatt,
Derek J. Cashman,
Daniel M. Zuckerman
Abstract:
There is significant interest in rapid protein simulations because of the time-scale limitations of all-atom methods. Exploiting the low cost and great availability of computer memory, we report a Monte Carlo technique for incorporating fully flexible atomistic protein components (e.g., peptide planes) into protein models without compromising sampling speed or statistical rigor. Building on exis…
▽ More
There is significant interest in rapid protein simulations because of the time-scale limitations of all-atom methods. Exploiting the low cost and great availability of computer memory, we report a Monte Carlo technique for incorporating fully flexible atomistic protein components (e.g., peptide planes) into protein models without compromising sampling speed or statistical rigor. Building on existing approximate methods (e.g., Rosetta), the technique uses pre-generated statistical libraries of all-atom components which are swapped with the corresponding protein components during a simulation. The simple model we study consists of the three all-atom backbone residues -- Ala, Gly, and Pro -- with structure-based (Go-like) interactions. For the five different proteins considered in this study, LBMC can generate at least 30 statistically independent configurations in about a month of single CPU time. Minimal additional cost is required to add residue-specific interactions.
△ Less
Submitted 4 December, 2008; v1 submitted 22 September, 2008;
originally announced September 2008.
