-
A wearable sensor vest for social humanoid robots with GPGPU, IoT, and modular software architecture
Authors:
Mohsen Jafarzadeh,
Stephen Brooks,
Shimeng Yu,
Balakrishnan Prabhakaran,
Yonas Tadesse
Abstract:
Currently, most social robots interact with their surroundings and humans through sensors that are integral parts of the robots, which limits the usability of the sensors, human-robot interaction, and interchangeability. A wearable sensor garment that fits many robots is needed in many applications. This article presents an affordable wearable sensor vest, and an open-source software architecture…
▽ More
Currently, most social robots interact with their surroundings and humans through sensors that are integral parts of the robots, which limits the usability of the sensors, human-robot interaction, and interchangeability. A wearable sensor garment that fits many robots is needed in many applications. This article presents an affordable wearable sensor vest, and an open-source software architecture with the Internet of Things (IoT) for social humanoid robots. The vest consists of touch, temperature, gesture, distance, vision sensors, and a wireless communication module. The IoT feature allows the robot to interact with humans locally and over the Internet. The designed architecture works for any social robot that has a general-purpose graphics processing unit (GPGPU), I2C/SPI buses, Internet connection, and the Robotics Operating System (ROS). The modular design of this architecture enables developers to easily add/remove/update complex behaviors. The proposed software architecture provides IoT technology, GPGPU nodes, I2C and SPI bus mangers, audio-visual interaction nodes (speech to text, text to speech, and image understanding), and isolation between behavior nodes and other nodes. The proposed IoT solution consists of related nodes in the robot, a RESTful web service, and user interfaces. We used the HTTP protocol as a means of two-way communication with the social robot over the Internet. Developers can easily edit or add nodes in C, C++, and Python programming languages. Our architecture can be used for designing more sophisticated behaviors for social humanoid robots.
△ Less
Submitted 6 January, 2022;
originally announced January 2022.
-
On-Policy Robust Adaptive Discrete-Time Regulator for Passive Unidirectional System using Stochastic Hill-climbing Algorithm and Associated Search Element
Authors:
Mohsen Jafarzadeh,
Nicholas Gans,
Yonas Tadesse
Abstract:
Non-linear discrete-time state-feedback regulators are widely used in passive unidirectional systems. Offline system identification is required for tuning parameters of these regulators. However, offline system identification is challenging in some applications. Furthermore, the parameters of a system may be slowly changing over time, which makes the system identification less effective. Many adap…
▽ More
Non-linear discrete-time state-feedback regulators are widely used in passive unidirectional systems. Offline system identification is required for tuning parameters of these regulators. However, offline system identification is challenging in some applications. Furthermore, the parameters of a system may be slowly changing over time, which makes the system identification less effective. Many adaptive regulators have been proposed to tune the parameters online when the offline information is neither accessible nor time-invariant. Stability and convergence of these adaptive regulators are challenging, especially in unidirectional systems. In this paper, a novel adaptive regulator is proposed for first-order unidirectional passive systems. In this method, an associated search element checks the eligibility of the update law. Then, a stochastic hill-climbing algorithm updates the parameters of the discrete-time state-feedback regulator. Simulation results demonstrate the effectiveness of the proposed method. The experiments on regulating of two passive systems show the ability of the method in regulating of passive unidirectional system in the presence of noise and disturbance.
△ Less
Submitted 29 December, 2021;
originally announced December 2021.
-
Haptic-enabled Mixed Reality System for Mixed-initiative Remote Robot Control
Authors:
Yuan Tian,
Lianjun Li,
Andrea Fumagalli,
Yonas Tadesse,
Balakrishnan Prabhakaran
Abstract:
Robots assist in many areas that are considered unsafe for humans to operate. For instance, in handling pandemic diseases such as the recent Covid-19 outbreak and other outbreaks like Ebola, robots can assist in reaching areas dangerous for humans and do simple tasks such as pick up the correct medicine (among a set of bottles prescribed) and deliver to patients. In such cases, it might not be goo…
▽ More
Robots assist in many areas that are considered unsafe for humans to operate. For instance, in handling pandemic diseases such as the recent Covid-19 outbreak and other outbreaks like Ebola, robots can assist in reaching areas dangerous for humans and do simple tasks such as pick up the correct medicine (among a set of bottles prescribed) and deliver to patients. In such cases, it might not be good to rely on the fully autonomous operation of robots. Since many mobile robots are fully functional with low-level tasks such as grabbing and moving, we consider the mixed-initiative control where the user can guide the robot remotely to finish such tasks. For this mixed-initiative control, the user controlling the robot needs to visualize a 3D scene as seen by the robot and guide it. Mixed reality can virtualize reality and immerse users in the 3D scene that is reconstructed from the real-world environment. This technique provides the user more freedom such as choosing viewpoints at view time. In recent years, benefiting from the high-quality data from Light Detection and Ranging (LIDAR) and RGBD cameras, mixed reality is widely used to build networked platforms to improve the performance of robot teleoperations and robot-human collaboration, and enhanced feedback for mixed-initiative control. In this paper, we proposed a novel haptic-enabled mixed reality system, that provides haptic interfaces to interact with the virtualized environments and give remote guidance for mobile robots towards high-level tasks. The experimental results show the effectiveness and flexibility of the proposed haptic enabled mixed reality system.
△ Less
Submitted 7 June, 2021; v1 submitted 6 February, 2021;
originally announced February 2021.
-
End-to-End Learning of Speech 2D Feature-Trajectory for Prosthetic Hands
Authors:
Mohsen Jafarzadeh,
Yonas Tadesse
Abstract:
Speech is one of the most common forms of communication in humans. Speech commands are essential parts of multimodal controlling of prosthetic hands. In the past decades, researchers used automatic speech recognition systems for controlling prosthetic hands by using speech commands. Automatic speech recognition systems learn how to map human speech to text. Then, they used natural language process…
▽ More
Speech is one of the most common forms of communication in humans. Speech commands are essential parts of multimodal controlling of prosthetic hands. In the past decades, researchers used automatic speech recognition systems for controlling prosthetic hands by using speech commands. Automatic speech recognition systems learn how to map human speech to text. Then, they used natural language processing or a look-up table to map the estimated text to a trajectory. However, the performance of conventional speech-controlled prosthetic hands is still unsatisfactory. Recent advancements in general-purpose graphics processing units (GPGPUs) enable intelligent devices to run deep neural networks in real-time. Thus, architectures of intelligent systems have rapidly transformed from the paradigm of composite subsystems optimization to the paradigm of end-to-end optimization. In this paper, we propose an end-to-end convolutional neural network (CNN) that maps speech 2D features directly to trajectories for prosthetic hands. The proposed convolutional neural network is lightweight, and thus it runs in real-time in an embedded GPGPU. The proposed method can use any type of speech 2D feature that has local correlations in each dimension such as spectrogram, MFCC, or PNCC. We omit the speech to text step in controlling the prosthetic hand in this paper. The network is written in Python with Keras library that has a TensorFlow backend. We optimized the CNN for NVIDIA Jetson TX2 developer kit. Our experiment on this CNN demonstrates a root-mean-square error of 0.119 and 20ms running time to produce trajectory outputs corresponding to the voice input data. To achieve a lower error in real-time, we can optimize a similar CNN for a more powerful embedded GPGPU such as NVIDIA AGX Xavier.
△ Less
Submitted 21 September, 2020;
originally announced September 2020.
-
Convolutional Neural Networks for Speech Controlled Prosthetic Hands
Authors:
Mohsen Jafarzadeh,
Yonas Tadesse
Abstract:
Speech recognition is one of the key topics in artificial intelligence, as it is one of the most common forms of communication in humans. Researchers have developed many speech-controlled prosthetic hands in the past decades, utilizing conventional speech recognition systems that use a combination of neural network and hidden Markov model. Recent advancements in general-purpose graphics processing…
▽ More
Speech recognition is one of the key topics in artificial intelligence, as it is one of the most common forms of communication in humans. Researchers have developed many speech-controlled prosthetic hands in the past decades, utilizing conventional speech recognition systems that use a combination of neural network and hidden Markov model. Recent advancements in general-purpose graphics processing units (GPGPUs) enable intelligent devices to run deep neural networks in real-time. Thus, state-of-the-art speech recognition systems have rapidly shifted from the paradigm of composite subsystems optimization to the paradigm of end-to-end optimization. However, a low-power embedded GPGPU cannot run these speech recognition systems in real-time. In this paper, we show the development of deep convolutional neural networks (CNN) for speech control of prosthetic hands that run in real-time on a NVIDIA Jetson TX2 developer kit. First, the device captures and converts speech into 2D features (like spectrogram). The CNN receives the 2D features and classifies the hand gestures. Finally, the hand gesture classes are sent to the prosthetic hand motion control system. The whole system is written in Python with Keras, a deep learning library that has a TensorFlow backend. Our experiments on the CNN demonstrate the 91% accuracy and 2ms running time of hand gestures (text output) from speech commands, which can be used to control the prosthetic hands in real-time.
△ Less
Submitted 3 October, 2019;
originally announced October 2019.
-
Deep learning approach to control of prosthetic hands with electromyography signals
Authors:
Mohsen Jafarzadeh,
Daniel Curtiss Hussey,
Yonas Tadesse
Abstract:
Natural muscles provide mobility in response to nerve impulses. Electromyography (EMG) measures the electrical activity of muscles in response to a nerve's stimulation. In the past few decades, EMG signals have been used extensively in the identification of user intention to potentially control assistive devices such as smart wheelchairs, exoskeletons, and prosthetic devices. In the design of conv…
▽ More
Natural muscles provide mobility in response to nerve impulses. Electromyography (EMG) measures the electrical activity of muscles in response to a nerve's stimulation. In the past few decades, EMG signals have been used extensively in the identification of user intention to potentially control assistive devices such as smart wheelchairs, exoskeletons, and prosthetic devices. In the design of conventional assistive devices, developers optimize multiple subsystems independently. Feature extraction and feature description are essential subsystems of this approach. Therefore, researchers proposed various hand-crafted features to interpret EMG signals. However, the performance of conventional assistive devices is still unsatisfactory. In this paper, we propose a deep learning approach to control prosthetic hands with raw EMG signals. We use a novel deep convolutional neural network to eschew the feature-engineering step. Removing the feature extraction and feature description is an important step toward the paradigm of end-to-end optimization. Fine-tuning and personalization are additional advantages of our approach. The proposed approach is implemented in Python with TensorFlow deep learning library, and it runs in real-time in general-purpose graphics processing units of NVIDIA Jetson TX2 developer kit. Our results demonstrate the ability of our system to predict fingers position from raw EMG signals. We anticipate our EMG-based control system to be a starting point to design more sophisticated prosthetic hands. For example, a pressure measurement unit can be added to transfer the perception of the environment to the user. Furthermore, our system can be modified for other prosthetic devices.
△ Less
Submitted 14 January, 2020; v1 submitted 21 September, 2019;
originally announced September 2019.
-
Modeling and Simulation of Robotic Finger Powered by Nylon Artificial Muscles- Equations with Simulink model
Authors:
Lokesh Saharan,
Lianjun Wu,
Yonas Tadesse
Abstract:
This paper shows a detailed modeling of three-link robotic finger that is actuated by nylon artificial muscles and a simulink model that can be used for numerical study of a robotic finger. The robotic hand prototype was recently demonstrated in recent publication Wu, L., Jung de Andrade, M., Saharan, L.,Rome, R., Baughman, R., and Tadesse, Y., 2017, Compact and Low-cost Humanoid Hand Powered by N…
▽ More
This paper shows a detailed modeling of three-link robotic finger that is actuated by nylon artificial muscles and a simulink model that can be used for numerical study of a robotic finger. The robotic hand prototype was recently demonstrated in recent publication Wu, L., Jung de Andrade, M., Saharan, L.,Rome, R., Baughman, R., and Tadesse, Y., 2017, Compact and Low-cost Humanoid Hand Powered by Nylon Artificial Muscles, Bioinspiration & Biomimetics, 12 (2). The robotic hand is a 3D printed, lightweight and compact hand actuated by silver-coated nylon muscles, often called Twisted and coiled Polymer (TCP) muscles. TCP muscles are thermal actuators that contract when they are heated and they are getting attention for application in robotics. The purpose of this paper is to demonstrate the modeling equations that were derived based on Euler Lagrangian approach that is suitable for implementation in simulink model.
△ Less
Submitted 27 January, 2019;
originally announced January 2019.
-
Symmetric group and the Axelrod model for dissemination of cultures
Authors:
Nirina Maurice Hasina Tahiridimbisoa,
Yabebal Tadesse
Abstract:
We consider the model proposed by Axelrod for dissemination of cultures on a 2-dimensional squared lattice. We review this model from an analytic point of view. We define $\left\langle s(t)\right\rangle$ to quantify possible culture configurations at time $t$ in a society. Typical initial culture configurations of this model are characterised. Equation of motion in terms of…
▽ More
We consider the model proposed by Axelrod for dissemination of cultures on a 2-dimensional squared lattice. We review this model from an analytic point of view. We define $\left\langle s(t)\right\rangle$ to quantify possible culture configurations at time $t$ in a society. Typical initial culture configurations of this model are characterised. Equation of motion in terms of $\left\langle s(t)\right\rangle$ is derived. We study the graph of development of this Axelrod system toward to its culture configurations equilibrium. Generically, we observe that this model undergoes three phases of development. We give a quantitative explanation about these three different phases of development.
Kee** up with this Axelrod model, we characterize its culture configurations space at equilibrium point where $\left\langle s(t_{\text{eq}})\right\rangle = 1$. This space is called monoculture space. Understanding this space is equivalent to restrict to the space of culture configurations from one individual in the model. This individual culture space is identified to the space $V_N^{\otimes F}$ up to isomorphisms. Action of the permutation group $S_N$ on the space $V_{N}^{\otimes F}$ is considered. Under this action, the observable $\left\langle s(t)\right\rangle$ is an invariant of the Axelrod system. We explore this symmetry and classify the different inequivalent classes of culture configurations composing the monoculture space. To achieve this, we consider the case $N\geq F$. We propose techniques from group representation theory to perform this classification. The inequivalent classes of culture configurations are indexed by the Dickau diagrams which are associated to the Bell number $B_F$. A concrete example with $F=4$ and $N\geq 4$ is considered for a full illustration of our analysis.
△ Less
Submitted 22 March, 2018;
originally announced April 2018.
-
Soft-NeuroAdapt: A 3-DOF Neuro-Adaptive Patient Pose Correction System For Frameless and Maskless Cancer Radiotherapy
Authors:
Olalekan Ogunmolu,
Adwait Kulkarni,
Yonas Tadesse,
Xuejun Gu,
Steve Jiang,
Nicholas Gans
Abstract:
Precise patient positioning is fundamental to successful removal of malignant tumors during treatment of head and neck cancers. Errors in patient positioning have been known to damage critical organs and cause complications. To better address issues of patient positioning and motion, we introduce a 3-DOF neuro-adaptive soft-robot, called Soft-NeuroAdapt to correct deviations along 3 axes. The robo…
▽ More
Precise patient positioning is fundamental to successful removal of malignant tumors during treatment of head and neck cancers. Errors in patient positioning have been known to damage critical organs and cause complications. To better address issues of patient positioning and motion, we introduce a 3-DOF neuro-adaptive soft-robot, called Soft-NeuroAdapt to correct deviations along 3 axes. The robot consists of inflatable air bladders that adaptively control head deviations from target while ensuring patient safety and comfort. The adaptive-neuro controller combines a state feedback component, a feedforward regulator, and a neural network that ensures correct adaptation. States are measured by a 3D vision system. We validate Soft-NeuroAdapt on a 3D printed head-and-neck dummy, and demonstrate that the controller provides adaptive actuation that compensates for intrafractional deviations in patient positioning.
△ Less
Submitted 22 September, 2017; v1 submitted 10 March, 2017;
originally announced March 2017.
-
Using Edge-induced and Vertex-induced Subhypergraph Polynomials
Authors:
Yohannes Tadesse
Abstract:
For a hypergraph $\mathcal H$, we consider the edge-induced and vertex-induced subhypergraph polynomials and study their relation. We use this relation to prove that both polynomials are reconstructible, and to prove a theorem relating the Hilbert series of the Stanley-Reisner ring of the independent complex of $\mathcal H$ and the edge-induced subhypergraph polynomial. We also consider reconstruc…
▽ More
For a hypergraph $\mathcal H$, we consider the edge-induced and vertex-induced subhypergraph polynomials and study their relation. We use this relation to prove that both polynomials are reconstructible, and to prove a theorem relating the Hilbert series of the Stanley-Reisner ring of the independent complex of $\mathcal H$ and the edge-induced subhypergraph polynomial. We also consider reconstruction of some algebraic invariants of $\mathcal H$.
△ Less
Submitted 19 March, 2013;
originally announced March 2013.
-
Poincare Series of Monomial Rings with Minimal Taylor Resolution
Authors:
Yohannes Tadesse
Abstract:
We give a comparison between the Poincare series of two monomial rings: $R=A/I$ and $R_q=A/I_q$ where $I_q$ is a monomial ideal generated by the $q$'th power of monomial generators of $I$. We compute the Poincare series for a class of monomial rings with minimal Taylor resolution. The paper was produced during Pragmatic 2011.
We give a comparison between the Poincare series of two monomial rings: $R=A/I$ and $R_q=A/I_q$ where $I_q$ is a monomial ideal generated by the $q$'th power of monomial generators of $I$. We compute the Poincare series for a class of monomial rings with minimal Taylor resolution. The paper was produced during Pragmatic 2011.
△ Less
Submitted 13 October, 2011;
originally announced October 2011.
-
Hilbert series of modules over Lie algebroids
Authors:
Rolf Källström,
Yohannes Tadesse
Abstract:
We consider modules $M$ over Lie algebroids ${\mathfrak g}_A$ which are of finite type over a local noetherian ring $A$. Using ideals $J\subset A$ such that ${\mathfrak g}_A \cdot J\subset J $ and the length $\ell_{{\mathfrak g}_A}(M/JM)< \infty$ we can define in a natural way the Hilbert series of $M$ with respect to the defining ideal $J$. This notion is in particular studied for modules over th…
▽ More
We consider modules $M$ over Lie algebroids ${\mathfrak g}_A$ which are of finite type over a local noetherian ring $A$. Using ideals $J\subset A$ such that ${\mathfrak g}_A \cdot J\subset J $ and the length $\ell_{{\mathfrak g}_A}(M/JM)< \infty$ we can define in a natural way the Hilbert series of $M$ with respect to the defining ideal $J$. This notion is in particular studied for modules over the Lie algebroid of $k$-linear derivations ${\mathfrak g}_A=T_{A/k}(I)$ that preserve an ideal $I\subset A$, for example when $A={\mathcal O}_n$, the ring of convergent power series. Hilbert series over Stanley-Reisner rings are also considered.
△ Less
Submitted 23 December, 2015; v1 submitted 27 June, 2011;
originally announced June 2011.
