-
Reinforcement Learning-Based Model Matching to Reduce the Sim-Real Gap in COBRA
Authors:
Adarsh Salagame,
Harin Kumar Nallaguntla,
Eric Sihite,
Gunar Schirner,
Alireza Ramezani
Abstract:
This paper employs a reinforcement learning-based model identification method aimed at enhancing the accuracy of the dynamics for our snake robot, called COBRA. Leveraging gradient information and iterative optimization, the proposed approach refines the parameters of COBRA's dynamical model such as coefficient of friction and actuator parameters using experimental and simulated data. Experimental…
▽ More
This paper employs a reinforcement learning-based model identification method aimed at enhancing the accuracy of the dynamics for our snake robot, called COBRA. Leveraging gradient information and iterative optimization, the proposed approach refines the parameters of COBRA's dynamical model such as coefficient of friction and actuator parameters using experimental and simulated data. Experimental validation on the hardware platform demonstrates the efficacy of the proposed approach, highlighting its potential to address sim-to-real gap in robot implementation.
△ Less
Submitted 19 June, 2024;
originally announced June 2024.
-
Dynamic Posture Manipulation During Tumbling for Closed-Loop Heading Angle Control
Authors:
Adarsh Salagame,
Eric Sihite,
Gunar Schirner,
Alireza Ramezani
Abstract:
Passive tumbling uses natural forces like gravity for efficient travel. But without an active means of control, passive tumblers must rely entirely on external forces. Northeastern University's COBRA is a snake robot that can morph into a ring, which employs passive tumbling to traverse down slopes. However, due to its articulated joints, it is also capable of dynamically altering its posture to m…
▽ More
Passive tumbling uses natural forces like gravity for efficient travel. But without an active means of control, passive tumblers must rely entirely on external forces. Northeastern University's COBRA is a snake robot that can morph into a ring, which employs passive tumbling to traverse down slopes. However, due to its articulated joints, it is also capable of dynamically altering its posture to manipulate the dynamics of the tumbling locomotion for active steering. This paper presents a modelling and control strategy based on collocation optimization for real-time steering of COBRA's tumbling locomotion. We validate our approach using Matlab simulations.
△ Less
Submitted 8 May, 2024;
originally announced May 2024.
-
Non-impulsive Contact-Implicit Motion Planning for Morpho-functional Loco-manipulation
Authors:
Adarsh Salagame,
Kruthika Gangaraju,
Harin Kumar Nallaguntla,
Eric Sihite,
Gunar Schirner,
Alireza Ramezani
Abstract:
Object manipulation has been extensively studied in the context of fixed base and mobile manipulators. However, the overactuated locomotion modality employed by snake robots allows for a unique blend of object manipulation through locomotion, referred to as loco-manipulation. The following work presents an optimization approach to solving the loco-manipulation problem based on non-impulsive implic…
▽ More
Object manipulation has been extensively studied in the context of fixed base and mobile manipulators. However, the overactuated locomotion modality employed by snake robots allows for a unique blend of object manipulation through locomotion, referred to as loco-manipulation. The following work presents an optimization approach to solving the loco-manipulation problem based on non-impulsive implicit contact path planning for our snake robot COBRA. We present the mathematical framework and show high fidelity simulation results for fixed-shape lateral rolling trajectories that demonstrate the object manipulation.
△ Less
Submitted 11 April, 2024;
originally announced April 2024.
-
Loco-Manipulation with Nonimpulsive Contact-Implicit Planning in a Slithering Robot
Authors:
Adarsh Salagame,
Kruthika Gangaraju,
Harin Kumar Nallaguntla,
Eric Sihite,
Gunar Schirner,
Alireza Ramezani
Abstract:
Object manipulation has been extensively studied in the context of fixed base and mobile manipulators. However, the overactuated locomotion modality employed by snake robots allows for a unique blend of object manipulation through locomotion, referred to as loco-manipulation. The following work presents an optimization approach to solving the loco-manipulation problem based on non-impulsive implic…
▽ More
Object manipulation has been extensively studied in the context of fixed base and mobile manipulators. However, the overactuated locomotion modality employed by snake robots allows for a unique blend of object manipulation through locomotion, referred to as loco-manipulation. The following work presents an optimization approach to solving the loco-manipulation problem based on non-impulsive implicit contact path planning for our snake robot COBRA. We present the mathematical framework and show high-fidelity simulation results and experiments to demonstrate the effectiveness of our approach.
△ Less
Submitted 11 April, 2024;
originally announced April 2024.
-
Enhancing Automatic Modulation Recognition for IoT Applications Using Transformers
Authors:
Narges Rashvand,
Kenneth Witham,
Gabriel Maldonado,
Vinit Katariya,
Nishanth Marer Prabhu,
Gunar Schirner,
Hamed Tabkhi
Abstract:
Automatic modulation recognition (AMR) is vital for accurately identifying modulation types within incoming signals, a critical task for optimizing operations within edge devices in IoT ecosystems. This paper presents an innovative approach that leverages Transformer networks, initially designed for natural language processing, to address the challenges of efficient AMR. Our transformer network ar…
▽ More
Automatic modulation recognition (AMR) is vital for accurately identifying modulation types within incoming signals, a critical task for optimizing operations within edge devices in IoT ecosystems. This paper presents an innovative approach that leverages Transformer networks, initially designed for natural language processing, to address the challenges of efficient AMR. Our transformer network architecture is designed with the mindset of real-time edge computing on IoT devices. Four tokenization techniques are proposed and explored for creating proper embeddings of RF signals, specifically focusing on overcoming the limitations related to the model size often encountered in IoT scenarios. Extensive experiments reveal that our proposed method outperformed advanced deep learning techniques, achieving the highest recognition accuracy. Notably, our model achieves an accuracy of 65.75 on the RML2016 and 65.80 on the CSPB.ML.2018+ dataset.
△ Less
Submitted 5 April, 2024; v1 submitted 8 March, 2024;
originally announced March 2024.
-
Multistatic-Radar RCS-Signature Recognition of Aerial Vehicles: A Bayesian Fusion Approach
Authors:
Michael Potter,
Murat Akcakaya,
Marius Necsoiu,
Gunar Schirner,
Deniz Erdogmus,
Tales Imbiriba
Abstract:
Radar Automated Target Recognition (RATR) for Unmanned Aerial Vehicles (UAVs) involves transmitting Electromagnetic Waves (EMWs) and performing target type recognition on the received radar echo, crucial for defense and aerospace applications. Previous studies highlighted the advantages of multistatic radar configurations over monostatic ones in RATR. However, fusion methods in multistatic radar c…
▽ More
Radar Automated Target Recognition (RATR) for Unmanned Aerial Vehicles (UAVs) involves transmitting Electromagnetic Waves (EMWs) and performing target type recognition on the received radar echo, crucial for defense and aerospace applications. Previous studies highlighted the advantages of multistatic radar configurations over monostatic ones in RATR. However, fusion methods in multistatic radar configurations often suboptimally combine classification vectors from individual radars probabilistically. To address this, we propose a fully Bayesian RATR framework employing Optimal Bayesian Fusion (OBF) to aggregate classification probability vectors from multiple radars. OBF, based on expected 0-1 loss, updates a Recursive Bayesian Classification (RBC) posterior distribution for target UAV type, conditioned on historical observations across multiple time steps. We evaluate the approach using simulated random walk trajectories for seven drones, correlating target aspect angles to Radar Cross Section (RCS) measurements in an anechoic chamber. Comparing against single radar Automated Target Recognition (ATR) systems and suboptimal fusion methods, our empirical results demonstrate that the OBF method integrated with RBC significantly enhances classification accuracy compared to other fusion methods and single radar configurations.
△ Less
Submitted 8 March, 2024; v1 submitted 27 February, 2024;
originally announced February 2024.
-
Inference of Upcoming Human Grasp Using EMG During Reach-to-Grasp Movement
Authors:
Mo Han,
Mehrshad Zandigohar,
Sezen Yagmur Gunay,
Gunar Schirner,
Deniz Erdogmus
Abstract:
Electromyography (EMG) data has been extensively adopted as an intuitive interface for instructing human-robot collaboration. A major challenge of the real-time detection of human grasp intent is the identification of dynamic EMG from hand movements. Previous studies mainly implemented steady-state EMG classification with a small number of grasp patterns on dynamic situations, which are insufficie…
▽ More
Electromyography (EMG) data has been extensively adopted as an intuitive interface for instructing human-robot collaboration. A major challenge of the real-time detection of human grasp intent is the identification of dynamic EMG from hand movements. Previous studies mainly implemented steady-state EMG classification with a small number of grasp patterns on dynamic situations, which are insufficient to generate differentiated control regarding the muscular activity variation in practice. In order to better detect dynamic movements, more EMG variability could be integrated into the model. However, only limited research were concentrated on such detection of dynamic grasp motions, and most existing assessments on non-static EMG classification either require supervised ground-truth timestamps of the movement status, or only contain limited kinematic variations. In this study, we propose a framework for classifying dynamic EMG signals into gestures, and examine the impact of different movement phases, using an unsupervised method to segment and label the action transitions. We collected and utilized data from large gesture vocabularies with multiple dynamic actions to encode the transitions from one grasp intent to another based on common sequences of the grasp movements. The classifier for identifying the gesture label was constructed afterwards based on the dynamic EMG signal, with no supervised annotation of kinematic movements required. Finally, we evaluated the performances of several training strategies using EMG data from different movement phases, and explored the information revealed from each phase. All experiments were evaluated in a real-time style with the performance transitions over time presented.
△ Less
Submitted 26 September, 2022; v1 submitted 19 April, 2021;
originally announced April 2021.
-
Multimodal Fusion of EMG and Vision for Human Grasp Intent Inference in Prosthetic Hand Control
Authors:
Mehrshad Zandigohar,
Mo Han,
Mohammadreza Sharif,
Sezen Yagmur Gunay,
Mariusz P. Furmanek,
Mathew Yarossi,
Paolo Bonato,
Cagdas Onal,
Taskin Padir,
Deniz Erdogmus,
Gunar Schirner
Abstract:
Objective: For transradial amputees, robotic prosthetic hands promise to regain the capability to perform daily living activities. Current control methods based on physiological signals such as electromyography (EMG) are prone to yielding poor inference outcomes due to motion artifacts, muscle fatigue, and many more. Vision sensors are a major source of information about the environment state and…
▽ More
Objective: For transradial amputees, robotic prosthetic hands promise to regain the capability to perform daily living activities. Current control methods based on physiological signals such as electromyography (EMG) are prone to yielding poor inference outcomes due to motion artifacts, muscle fatigue, and many more. Vision sensors are a major source of information about the environment state and can play a vital role in inferring feasible and intended gestures. However, visual evidence is also susceptible to its own artifacts, most often due to object occlusion, lighting changes, etc. Multimodal evidence fusion using physiological and vision sensor measurements is a natural approach due to the complementary strengths of these modalities. Methods: In this paper, we present a Bayesian evidence fusion framework for grasp intent inference using eye-view video, eye-gaze, and EMG from the forearm processed by neural network models. We analyze individual and fused performance as a function of time as the hand approaches the object to grasp it. For this purpose, we have also developed novel data processing and augmentation techniques to train neural network components. Results: Our results indicate that, on average, fusion improves the instantaneous upcoming grasp type classification accuracy while in the reaching phase by 13.66% and 14.8%, relative to EMG (81.64% non-fused) and visual evidence (80.5% non-fused) individually, resulting in an overall fusion accuracy of 95.3%. Conclusion: Our experimental data analyses demonstrate that EMG and visual evidence show complementary strengths, and as a consequence, fusion of multimodal evidence can outperform each individual evidence modality at any given time.
△ Less
Submitted 27 February, 2024; v1 submitted 8 April, 2021;
originally announced April 2021.
-
From Hand-Perspective Visual Information to Grasp Type Probabilities: Deep Learning via Ranking Labels
Authors:
Mo Han,
Sezen Ya{ğ}mur Günay,
İlkay Yıldız,
Paolo Bonato,
Cagdas D. Onal,
Taşkın Padır,
Gunar Schirner,
Deniz Erdo{ğ}muş
Abstract:
Limb deficiency severely affects the daily lives of amputees and drives efforts to provide functional robotic prosthetic hands to compensate this deprivation. Convolutional neural network-based computer vision control of the prosthetic hand has received increased attention as a method to replace or complement physiological signals due to its reliability by training visual information to predict th…
▽ More
Limb deficiency severely affects the daily lives of amputees and drives efforts to provide functional robotic prosthetic hands to compensate this deprivation. Convolutional neural network-based computer vision control of the prosthetic hand has received increased attention as a method to replace or complement physiological signals due to its reliability by training visual information to predict the hand gesture. Mounting a camera into the palm of a prosthetic hand is proved to be a promising approach to collect visual data. However, the grasp type labelled from the eye and hand perspective may differ as object shapes are not always symmetric. Thus, to represent this difference in a realistic way, we employed a dataset containing synchronous images from eye- and hand- view, where the hand-perspective images are used for training while the eye-view images are only for manual labelling. Electromyogram (EMG) activity and movement kinematics data from the upper arm are also collected for multi-modal information fusion in future work. Moreover, in order to include human-in-the-loop control and combine the computer vision with physiological signal inputs, instead of making absolute positive or negative predictions, we build a novel probabilistic classifier according to the Plackett-Luce model. To predict the probability distribution over grasps, we exploit the statistical model over label rankings to solve the permutation domain problems via a maximum likelihood estimation, utilizing the manually ranked lists of grasps as a new form of label. We indicate that the proposed model is applicable to the most popular and productive convolutional neural network frameworks.
△ Less
Submitted 8 March, 2021;
originally announced March 2021.
-
HANDS: A Multimodal Dataset for Modeling Towards Human Grasp Intent Inference in Prosthetic Hands
Authors:
Mo Han,
Sezen Ya{ğ}mur Günay,
Gunar Schirner,
Taşkın Padır,
Deniz Erdo{ğ}muş
Abstract:
Upper limb and hand functionality is critical to many activities of daily living and the amputation of one can lead to significant functionality loss for individuals. From this perspective, advanced prosthetic hands of the future are anticipated to benefit from improved shared control between a robotic hand and its human user, but more importantly from the improved capability to infer human intent…
▽ More
Upper limb and hand functionality is critical to many activities of daily living and the amputation of one can lead to significant functionality loss for individuals. From this perspective, advanced prosthetic hands of the future are anticipated to benefit from improved shared control between a robotic hand and its human user, but more importantly from the improved capability to infer human intent from multimodal sensor data to provide the robotic hand perception abilities regarding the operational context. Such multimodal sensor data may include various environment sensors including vision, as well as human physiology and behavior sensors including electromyography and inertial measurement units. A fusion methodology for environmental state and human intent estimation can combine these sources of evidence in order to help prosthetic hand motion planning and control.
In this paper, we present a dataset of this type that was gathered with the anticipation of cameras being built into prosthetic hands, and computer vision methods will need to assess this hand-view visual evidence in order to estimate human intent. Specifically, paired images from human eye-view and hand-view of various objects placed at different orientations have been captured at the initial state of gras** trials, followed by paired video, EMG and IMU from the arm of the human during a grasp, lift, put-down, and retract style trial structure. For each trial, based on eye-view images of the scene showing the hand and object on a table, multiple humans were asked to sort in decreasing order of preference, five grasp types appropriate for the object in its given configuration relative to the hand. The potential utility of paired eye-view and hand-view images was illustrated by training a convolutional neural network to process hand-view images in order to predict eye-view labels assigned by humans.
△ Less
Submitted 8 March, 2021;
originally announced March 2021.
-
NetCut: Real-Time DNN Inference Using Layer Removal
Authors:
Mehrshad Zandigohar,
Deniz Erdogmus,
Gunar Schirner
Abstract:
Deep Learning plays a significant role in assisting humans in many aspects of their lives. As these networks tend to get deeper over time, they extract more features to increase accuracy at the cost of additional inference latency. This accuracy-performance trade-off makes it more challenging for Embedded Systems, as resource-constrained processors with strict deadlines, to deploy them efficiently…
▽ More
Deep Learning plays a significant role in assisting humans in many aspects of their lives. As these networks tend to get deeper over time, they extract more features to increase accuracy at the cost of additional inference latency. This accuracy-performance trade-off makes it more challenging for Embedded Systems, as resource-constrained processors with strict deadlines, to deploy them efficiently. This can lead to selection of networks that can prematurely meet a specified deadline with excess slack time that could have potentially contributed to increased accuracy.
In this work, we propose: (i) the concept of layer removal as a means of constructing TRimmed Networks (TRNs) that are based on removing problem-specific features of a pretrained network used in transfer learning, and (ii) NetCut, a methodology based on an empirical or an analytical latency estimator, which only proposes and retrains TRNs that can meet the application's deadline, hence reducing the exploration time significantly.
We demonstrate that TRNs can expand the Pareto frontier that trades off latency and accuracy to provide networks that can meet arbitrary deadlines with potential accuracy improvement over off-the-shelf networks. Our experimental results show that such utilization of TRNs, while transferring to a simpler dataset, in combination with NetCut, can lead to the proposal of networks that can achieve relative accuracy improvement of up to 10.43% among existing off-the-shelf neural architectures while meeting a specific deadline, and 27x speedup in exploration time.
△ Less
Submitted 13 January, 2021;
originally announced January 2021.
-
Towards Creating a Deployable Grasp Type Probability Estimator for a Prosthetic Hand
Authors:
Mehrshad Zandigohar,
Mo Han,
Deniz Erdogmus,
Gunar Schirner
Abstract:
For lower arm amputees, prosthetic hands promise to restore most of physical interaction capabilities. This requires to accurately predict hand gestures capable of grabbing varying objects and execute them timely as intended by the user. Current approaches often rely on physiological signal inputs such as Electromyography (EMG) signal from residual limb muscles to infer the intended motion. Howeve…
▽ More
For lower arm amputees, prosthetic hands promise to restore most of physical interaction capabilities. This requires to accurately predict hand gestures capable of grabbing varying objects and execute them timely as intended by the user. Current approaches often rely on physiological signal inputs such as Electromyography (EMG) signal from residual limb muscles to infer the intended motion. However, limited signal quality, user diversity and high variability adversely affect the system robustness. Instead of solely relying on EMG signals, our work enables augmenting EMG intent inference with physical state probability through machine learning and computer vision method. To this end, we: (1) study state-of-the-art deep neural network architectures to select a performant source of knowledge transfer for the prosthetic hand, (2) use a dataset containing object images and probability distribution of grasp types as a new form of labeling where instead of using absolute values of zero and one as the conventional classification labels, our labels are a set of probabilities whose sum is 1. The proposed method generates probabilistic predictions which could be fused with EMG prediction of probabilities over grasps by using the visual information from the palm camera of a prosthetic hand. Our results demonstrate that InceptionV3 achieves highest accuracy with 0.95 angular similarity followed by 1.4 MobileNetV2 with 0.93 at ~20% the amount of operations.
△ Less
Submitted 13 January, 2021;
originally announced January 2021.
-
Multi-path Model and Sensitivity Analysis for Galvanic Coupled Intra-body Communication through Layered Tissue
Authors:
Meenupriya Swaminathan,
Ferran Simon Cabrera,
Joan Sebastia Pujol,
Ufuk Muncuk,
Gunar Schirner,
Kaushik R. Chowdhury
Abstract:
New medical procedures promise continuous patient monitoring and drug delivery through implanted sensors and actuators. When over the air wireless radio frequency (OTA-RF) links are used for intra-body implant communication, the network incurs heavy energy costs owing to absorption within the human tissue. With this motivation, we explore an alternate form of intra-body communication that relies o…
▽ More
New medical procedures promise continuous patient monitoring and drug delivery through implanted sensors and actuators. When over the air wireless radio frequency (OTA-RF) links are used for intra-body implant communication, the network incurs heavy energy costs owing to absorption within the human tissue. With this motivation, we explore an alternate form of intra-body communication that relies on weak electrical signals, instead of OTA-RF. To demonstrate the feasibility of this new paradigm for enabling communication between sensors and actuators embedded within the tissue, or placed on the surface of the skin, we develop a rigorous analytical model based on galvanic coupling of low energy signals. The main contributions in this paper are: (i) develo** a suite of analytical expressions for modeling the resulting communication channel for weak electrical signals in a three dimensional multi-layered tissue structure, (ii) validating and verifying the model through extensive finite element simulations, published measurements in existing literature, and experiments conducted with porcine tissue, (iii) designing the communication framework with safety considerations, and analyzing the influence of different network and hardware parameters such as transmission frequency and electrode placements. Our results reveal a close agreement between theory, simulation, literature and experimental findings, pointing to the suitability of the model for quick and accurate channel characterization and parameter estimation for networked and implanted sensors.
△ Less
Submitted 8 December, 2015;
originally announced December 2015.
