-
Casting manipulation of unknown string by robot arm
Authors:
Kenta Tabata,
Hiroaki Seki,
Tokuo Tsuji,
Tatsuhiro Hiramitsu
Abstract:
Casting manipulation has been studied to expand the robot's movable range. In this manipulation, the robot throws and reaches the end effector to a distant target. Usually, a special casting manipulator, which consists of rigid arm links and specific flexible linear objects, is constructed for an effective casting manipulation. However, the special manipulator cannot perform normal manipulations,…
▽ More
Casting manipulation has been studied to expand the robot's movable range. In this manipulation, the robot throws and reaches the end effector to a distant target. Usually, a special casting manipulator, which consists of rigid arm links and specific flexible linear objects, is constructed for an effective casting manipulation. However, the special manipulator cannot perform normal manipulations, such as picking and placing, gras**, and operating objects. We propose that the normal robot arm, which can perform normal tasks, picks up an unknown string in the surrounding environment and realizes casting manipulation with it. As the properties of the string are not provided in advance, it is crucial how to reflect it in casting manipulation. This is realized by the motion generation of the robot arm with the simulation of string movement, actual string manipulation by the robot arm, and string parameter estimation from the actual string movement. After repeating these three steps, the simulated string movement approximates the actual to realize casting manipulation with the unknown string. We confirmed the effectiveness of the proposed method through experiments. The try of this study will lead to enhancement of the performance of home service robot, exploration robot, rescue robot and entertainment robot.
△ Less
Submitted 30 January, 2024;
originally announced January 2024.
-
Imitation Learning Inputting Image Feature to Each Layer of Neural Network
Authors:
Koki Yamane,
Sho Sakaino,
Toshiaki Tsuji
Abstract:
Imitation learning enables robots to learn and replicate human behavior from training data. Recent advances in machine learning enable end-to-end learning approaches that directly process high-dimensional observation data, such as images. However, these approaches face a critical challenge when processing data from multiple modalities, inadvertently ignoring data with a lower correlation to the de…
▽ More
Imitation learning enables robots to learn and replicate human behavior from training data. Recent advances in machine learning enable end-to-end learning approaches that directly process high-dimensional observation data, such as images. However, these approaches face a critical challenge when processing data from multiple modalities, inadvertently ignoring data with a lower correlation to the desired output, especially when using short sampling periods. This paper presents a useful method to address this challenge, which amplifies the influence of data with a relatively low correlation to the output by inputting the data into each neural network layer. The proposed approach effectively incorporates diverse data sources into the learning process. Through experiments using a simple pick-and-place operation with raw images and joint information as input, significant improvements in success rates are demonstrated even when dealing with data from short sampling periods.
△ Less
Submitted 19 January, 2024; v1 submitted 17 January, 2024;
originally announced January 2024.
-
Soft and Rigid Object Gras** With Cross-Structure Hand Using Bilateral Control-Based Imitation Learning
Authors:
Koki Yamane,
Sho Sakaino,
Toshiaki Tsuji
Abstract:
Object gras** is an important ability required for various robot tasks. In particular, tasks that require precise force adjustments during operation, such as gras** an unknown object or using a grasped tool, are difficult for humans to program in advance. Recently, AI-based algorithms that can imitate human force skills have been actively explored as a solution. In particular, bilateral contro…
▽ More
Object gras** is an important ability required for various robot tasks. In particular, tasks that require precise force adjustments during operation, such as gras** an unknown object or using a grasped tool, are difficult for humans to program in advance. Recently, AI-based algorithms that can imitate human force skills have been actively explored as a solution. In particular, bilateral control-based imitation learning achieves human-level motion speeds with environmental adaptability, only requiring human demonstration and without programming. However, owing to hardware limitations, its gras** performance remains limited, and tasks that involves gras** various objects are yet to be achieved. Here, we developed a cross-structure hand to grasp various objects. We experimentally demonstrated that the integration of bilateral control-based imitation learning and the cross-structure hand is effective for gras** various objects and harnessing tools.
△ Less
Submitted 15 November, 2023;
originally announced November 2023.
-
Lightweight High-Speed and High-Force Gripper for Assembly
Authors:
Toshihiro Nishimura,
Takeshi Takaki,
Yosuke Suzuki,
Tokuo Tsuji,
Tetsuyou Watanabe
Abstract:
This paper presents a novel industrial robotic gripper with a high gras** speed (maximum: 1396 mm/s), high tip force (maximum: 80 N) for gras**, large motion range, and lightweight design (0.3 kg). To realize these features, the high-speed section of the quick-return mechanism and load-sensitive continuously variable transmission mechanism are installed in the gripper. The gripper is also equi…
▽ More
This paper presents a novel industrial robotic gripper with a high gras** speed (maximum: 1396 mm/s), high tip force (maximum: 80 N) for gras**, large motion range, and lightweight design (0.3 kg). To realize these features, the high-speed section of the quick-return mechanism and load-sensitive continuously variable transmission mechanism are installed in the gripper. The gripper is also equipped with a self-centering function. The high gras** speed and self-centering function improve the cycle time in robotic operations. In addition, the high tip force is advantageous for stably gras** and assembling heavy objects. Moreover, the design of the gripper reduce the gripper's proportion of the manipulator's payload, thus increasing the weight of the object that can be grasped. The gripper performance was validated through kinematic and static analyses as well as experimental evaluations. This paper also presents the analysis of the self-centering function of the developed gripper.
△ Less
Submitted 26 October, 2023;
originally announced October 2023.
-
Stability analysis of admittance control using asymmetric stiffness matrix
Authors:
Toshiaki Tsuji,
Yasuhiro Kato
Abstract:
In contact-rich tasks, setting the stiffness of the control system is a critical factor in its performance. Although the setting range can be extended by making the stiffness matrix asymmetric, its stability has not been proven. This study focuses on the stability of compliance control in a robot arm that deals with an asymmetric stiffness matrix. It discusses the convergence stability of the admi…
▽ More
In contact-rich tasks, setting the stiffness of the control system is a critical factor in its performance. Although the setting range can be extended by making the stiffness matrix asymmetric, its stability has not been proven. This study focuses on the stability of compliance control in a robot arm that deals with an asymmetric stiffness matrix. It discusses the convergence stability of the admittance control. The paper explains how to derive an asymmetric stiffness matrix and how to incorporate it into the admittance model. The authors also present simulation and experimental results that demonstrate the effectiveness of their proposed method.
△ Less
Submitted 20 June, 2023;
originally announced June 2023.
-
1-degree-of-freedom Robotic Gripper With Infinite Self-Twist Function
Authors:
Toshihiro Nishimura,
Yosuke Suzuki,
Tokuo Tsuji,
Tetsuyou Watanabe
Abstract:
This study proposed a novel robotic gripper that can achieve gras** and infinite wrist twisting motions using a single actuator. The gripper is equipped with a differential gear mechanism that allows switching between the gras** and twisting motions according to the magnitude of the tip force applied to the finger. The gras** motion is activated when the tip force is below a set value, and t…
▽ More
This study proposed a novel robotic gripper that can achieve gras** and infinite wrist twisting motions using a single actuator. The gripper is equipped with a differential gear mechanism that allows switching between the gras** and twisting motions according to the magnitude of the tip force applied to the finger. The gras** motion is activated when the tip force is below a set value, and the wrist twisting motion is activated when the tip force exceeds this value. "Twist gras**," a special gras** mode that allows the wrap** of a flexible thin object around the fingers of the gripper, can be achieved by the twisting motion. Twist gras** is effective for handling objects with flexible thin parts, such as laminated packaging pouches, that are difficult to grasp using conventional antipodal gras**. In this study, the gripper design is presented, and twist gras** is analyzed. The gripper performance is experimentally validated.
△ Less
Submitted 9 November, 2022;
originally announced November 2022.
-
Sensing and Control of Friction Mode for Contact Area Variable Surfaces (Friction-variable Surface Structure)
Authors:
Seita Nojiri,
Akihiko Yamaguchi,
Yosuke Suzuki,
Tokuo Tsuji,
Tetsuyou Watanabe
Abstract:
Robotic hands with soft surfaces can perform stable gras**, but the high friction of the soft surfaces makes it difficult to release objects, or to perform operations that require sliding. To solve this issue, we previously developed a contact area variable surface (CAVS), whose friction changed according to the load. However, only our fundamental results were previously presented, with detailed…
▽ More
Robotic hands with soft surfaces can perform stable gras**, but the high friction of the soft surfaces makes it difficult to release objects, or to perform operations that require sliding. To solve this issue, we previously developed a contact area variable surface (CAVS), whose friction changed according to the load. However, only our fundamental results were previously presented, with detailed analyses not provided. In this study, we first investigated the CAVS friction anisotropy, and demonstrated that the longitudinal direction exhibited a larger ratio of friction change. Next, we proposed a sensible CAVS, capable of providing a variable-friction mechanism, and tested its sensing and control systems in operations requiring switching between sliding and stable-gras** modes. Friction sensing was performed using an embedded camera, and we developed a gripper using the sensible CAVS, considering the CAVS friction anisotropy. In CAVS, the low-friction mode corresponds to a small gras** force, while the high-friction mode corresponds to a greater gras** force. Therefore, by controlling only the friction mode, the gripper mode can be set to either the sliding or stable-gras** mode. Based on this feature, a methodology for controlling the contact mode was constructed. We demonstrated a manipulation involving sliding and stable gras**, and thus verified the efficacy of the developed sensible CAVS.
△ Less
Submitted 9 November, 2022;
originally announced November 2022.
-
Design of non-diagonal stiffness matrix for assembly task
Authors:
Tsukasa Kusakabe,
Sho Sakaino,
Toshiaki Tsuji
Abstract:
Compliance control is an increasingly employed technique used in the robotic field. It is known that various mechanical properties can be reproduced depending on the design of the stiffness matrix, but the design theory that takes advantage of this high degree of design freedom has not been elucidated. This paper, therefore, discusses the non-diagonal elements of the stiffness matrix. We proposed…
▽ More
Compliance control is an increasingly employed technique used in the robotic field. It is known that various mechanical properties can be reproduced depending on the design of the stiffness matrix, but the design theory that takes advantage of this high degree of design freedom has not been elucidated. This paper, therefore, discusses the non-diagonal elements of the stiffness matrix. We proposed a design method according to the conditions required for achieving stable motion. Additionally, we analyzed the displacement induced by the non-diagonal elements in response to an external force and found that to obtain stable contact with a symmetric matrix, the matrix should be positive definite, i.e., all eigenvalues must be positive, however its parameter design is complicated. In this study, we focused on the use of asymmetric matrices in compliance control and showed that the design of eigenvalues can be simplified by using a triangular matrix. This approach expands the range of the stiffness design and enhances the ability of the compliance control to induce motion. We conducted experiments using the stiffness matrix and confirmed that assembly could be achieved without complicated trajectory planning.
△ Less
Submitted 29 October, 2022;
originally announced October 2022.
-
Automated Classification of General Movements in Infants Using a Two-stream Spatiotemporal Fusion Network
Authors:
Yuki Hashimoto,
Akira Furui,
Koji Shimatani,
Maura Casadio,
Paolo Moretti,
Pietro Morasso,
Toshio Tsuji
Abstract:
The assessment of general movements (GMs) in infants is a useful tool in the early diagnosis of neurodevelopmental disorders. However, its evaluation in clinical practice relies on visual inspection by experts, and an automated solution is eagerly awaited. Recently, video-based GMs classification has attracted attention, but this approach would be strongly affected by irrelevant information, such…
▽ More
The assessment of general movements (GMs) in infants is a useful tool in the early diagnosis of neurodevelopmental disorders. However, its evaluation in clinical practice relies on visual inspection by experts, and an automated solution is eagerly awaited. Recently, video-based GMs classification has attracted attention, but this approach would be strongly affected by irrelevant information, such as background clutter in the video. Furthermore, for reliability, it is necessary to properly extract the spatiotemporal features of infants during GMs. In this study, we propose an automated GMs classification method, which consists of preprocessing networks that remove unnecessary background information from GMs videos and adjust the infant's body position, and a subsequent motion classification network based on a two-stream structure. The proposed method can efficiently extract the essential spatiotemporal features for GMs classification while preventing overfitting to irrelevant information for different recording environments. We validated the proposed method using videos obtained from 100 infants. The experimental results demonstrate that the proposed method outperforms several baseline models and the existing methods.
△ Less
Submitted 4 July, 2022;
originally announced July 2022.
-
Imitation Learning for Nonprehensile Manipulation through Self-Supervised Learning Considering Motion Speed
Authors:
Yuki Saigusa,
Sho Sakaino,
Toshiaki Tsuji
Abstract:
Robots are expected to replace menial tasks such as housework. Some of these tasks include nonprehensile manipulation performed without gras** objects. Nonprehensile manipulation is very difficult because it requires considering the dynamics of environments and objects. Therefore imitating complex behaviors requires a large number of human demonstrations. In this study, a self-supervised learnin…
▽ More
Robots are expected to replace menial tasks such as housework. Some of these tasks include nonprehensile manipulation performed without gras** objects. Nonprehensile manipulation is very difficult because it requires considering the dynamics of environments and objects. Therefore imitating complex behaviors requires a large number of human demonstrations. In this study, a self-supervised learning that considers dynamics to achieve variable speed for nonprehensile manipulation is proposed. The proposed method collects and fine-tunes only successful action data obtained during autonomous operations. By fine-tuning the successful data, the robot learns the dynamics among itself, its environment, and objects. We experimented with the task of scoo** and transporting pancakes using the neural network model trained on 24 human-collected training data. The proposed method significantly improved the success rate from 40.2% to 85.7%, and succeeded the task more than 75% for other objects.
△ Less
Submitted 22 June, 2022; v1 submitted 21 June, 2022;
originally announced June 2022.
-
Soft robotic hand with finger-bending/friction-reduction switching mechanism through 1-degree-of-freedom flow control
Authors:
Toshihiro Nishimura,
Kensuke Shimizu,
Seita Nojiri,
Kenjiro Tadakuma,
Yosuke Suzuki,
Tokuo Tsuji,
Tetsuyou Watanabe
Abstract:
This paper proposes a novel pneumatic soft robotic hand that incorporates a mechanism that can switch the airflow path using a single airflow control. The developed hand can control the finger motion and operate the surface friction variable mechanism. In the friction variable mechanism, a lubricant is injected onto the high-friction finger surface to reduce surface friction. To inject the lubrica…
▽ More
This paper proposes a novel pneumatic soft robotic hand that incorporates a mechanism that can switch the airflow path using a single airflow control. The developed hand can control the finger motion and operate the surface friction variable mechanism. In the friction variable mechanism, a lubricant is injected onto the high-friction finger surface to reduce surface friction. To inject the lubrication using a positive-pressure airflow, the Venturi effect is applied. The design and evaluation of the airflow-path switching and friction variable mechanisms are described. Moreover, the entire design of a soft robotic hand equipped with these mechanisms is presented. The performance was validated through gras**, placing, and manipulation tests.
△ Less
Submitted 27 March, 2022;
originally announced March 2022.
-
An Independently Learnable Hierarchical Model for Bilateral Control-Based Imitation Learning Applications
Authors:
Kazuki Hayashi,
Sho Sakaino,
Toshiaki Tsuji
Abstract:
Recently, motion generation by machine learning has been actively researched to automate various tasks. Imitation learning is one such method that learns motions from data collected in advance. However, executing long-term tasks remains challenging. Therefore, a novel framework for imitation learning is proposed to solve this problem. The proposed framework comprises upper and lower layers, where…
▽ More
Recently, motion generation by machine learning has been actively researched to automate various tasks. Imitation learning is one such method that learns motions from data collected in advance. However, executing long-term tasks remains challenging. Therefore, a novel framework for imitation learning is proposed to solve this problem. The proposed framework comprises upper and lower layers, where the upper layer model, whose timescale is long, and lower layer model, whose timescale is short, can be independently trained. In this model, the upper layer learns long-term task planning, and the lower layer learns motion primitives. The proposed method was experimentally compared to hierarchical RNN-based methods to validate its effectiveness. Consequently, the proposed method showed a success rate equal to or greater than that of conventional methods. In addition, the proposed method required less than 1/20 of the training time compared to conventional methods. Moreover, it succeeded in executing unlearned tasks by reusing the trained lower layer.
△ Less
Submitted 16 March, 2022;
originally announced March 2022.
-
A Self-Tuning Impedance-based Interaction Planner for Robotic Haptic Exploration
Authors:
Yasuhiro Kato,
Pietro Balatti,
Juan M. Gandarias,
Mattia Leonori,
Toshiaki Tsuji,
Arash Ajoudani
Abstract:
This paper presents a novel interaction planning method that exploits impedance tuning techniques in response to environmental uncertainties and unpredictable conditions using haptic information only. The proposed algorithm plans the robot's trajectory based on the haptic interaction with the environment and adapts planning strategies as needed. Two approaches are considered: Exploration and Bounc…
▽ More
This paper presents a novel interaction planning method that exploits impedance tuning techniques in response to environmental uncertainties and unpredictable conditions using haptic information only. The proposed algorithm plans the robot's trajectory based on the haptic interaction with the environment and adapts planning strategies as needed. Two approaches are considered: Exploration and Bouncing strategies. The Exploration strategy takes the actual motion of the robot into account in planning, while the Bouncing strategy exploits the forces and the motion vector of the robot. Moreover, self-tuning impedance is performed according to the planned trajectory to ensure compliant contact and low contact forces. In order to show the performance of the proposed methodology, two experiments with a torque-controller robotic arm are carried out. The first considers a maze exploration without obstacles, whereas the second includes obstacles. The proposed method performance is analyzed and compared against previously proposed solutions in both cases. Experimental results demonstrate that: i) the robot can successfully plan its trajectory autonomously in the most feasible direction according to the interaction with the environment, and ii) a compliant interaction with an unknown environment despite the uncertainties is achieved. Finally, a scalability demonstration is carried out to show the potential of the proposed method under multiple scenarios.
△ Less
Submitted 2 September, 2022; v1 submitted 10 March, 2022;
originally announced March 2022.
-
Generation Drawing/Grinding Trajectoy Based on Hierarchical CVAE
Authors:
Masahiro Aita,
Keito Sugawara,
Sho Sakaino,
Toshiaki Tsuji
Abstract:
In this study, we propose a method to model the local and global features of the drawing/grinding trajectory with hierarchical Variational Autoencoders (VAEs). By combining two separately trained VAE models in a hierarchical structure, it is possible to generate trajectories with high reproducibility for both local and global features. The hierarchical generation network enables the generation of…
▽ More
In this study, we propose a method to model the local and global features of the drawing/grinding trajectory with hierarchical Variational Autoencoders (VAEs). By combining two separately trained VAE models in a hierarchical structure, it is possible to generate trajectories with high reproducibility for both local and global features. The hierarchical generation network enables the generation of higher-order trajectories with a relatively small amount of training data. The simulation and experimental results demonstrate the generalization performance of the proposed method. In addition, we confirmed that it is possible to generate new trajectories, which have never been learned in the past, by changing the combination of the learned models.
△ Less
Submitted 23 November, 2021; v1 submitted 21 November, 2021;
originally announced November 2021.
-
A Time-Series Scale Mixture Model of EEG with a Hidden Markov Structure for Epileptic Seizure Detection
Authors:
Akira Furui,
Tomoyuki Akiyama,
Toshio Tsuji
Abstract:
In this paper, we propose a time-series stochastic model based on a scale mixture distribution with Markov transitions to detect epileptic seizures in electroencephalography (EEG). In the proposed model, an EEG signal at each time point is assumed to be a random variable following a Gaussian distribution. The covariance matrix of the Gaussian distribution is weighted with a latent scale parameter,…
▽ More
In this paper, we propose a time-series stochastic model based on a scale mixture distribution with Markov transitions to detect epileptic seizures in electroencephalography (EEG). In the proposed model, an EEG signal at each time point is assumed to be a random variable following a Gaussian distribution. The covariance matrix of the Gaussian distribution is weighted with a latent scale parameter, which is also a random variable, resulting in the stochastic fluctuations of covariances. By introducing a latent state variable with a Markov chain in the background of this stochastic relationship, time-series changes in the distribution of latent scale parameters can be represented according to the state of epileptic seizures. In an experiment, we evaluated the performance of the proposed model for seizure detection using EEGs with multiple frequency bands decomposed from a clinical dataset. The results demonstrated that the proposed model can detect seizures with high sensitivity and outperformed several baselines.
△ Less
Submitted 11 November, 2021;
originally announced November 2021.
-
EMG Pattern Recognition via Bayesian Inference with Scale Mixture-Based Stochastic Generative Models
Authors:
Akira Furui,
Takuya Igaue,
Toshio Tsuji
Abstract:
Electromyogram (EMG) has been utilized to interface signals for prosthetic hands and information devices owing to its ability to reflect human motion intentions. Although various EMG classification methods have been introduced into EMG-based control systems, they do not fully consider the stochastic characteristics of EMG signals. This paper proposes an EMG pattern classification method incorporat…
▽ More
Electromyogram (EMG) has been utilized to interface signals for prosthetic hands and information devices owing to its ability to reflect human motion intentions. Although various EMG classification methods have been introduced into EMG-based control systems, they do not fully consider the stochastic characteristics of EMG signals. This paper proposes an EMG pattern classification method incorporating a scale mixture-based generative model. A scale mixture model is a stochastic EMG model in which the EMG variance is considered as a random variable, enabling the representation of uncertainty in the variance. This model is extended in this study and utilized for EMG pattern classification. The proposed method is trained by variational Bayesian learning, thereby allowing the automatic determination of the model complexity. Furthermore, to optimize the hyperparameters of the proposed method with a partial discriminative approach, a mutual information-based determination method is introduced. Simulation and EMG analysis experiments demonstrated the relationship between the hyperparameters and classification accuracy of the proposed method as well as the validity of the proposed method. The comparison using public EMG datasets revealed that the proposed method outperformed the various conventional classifiers. These results indicated the validity of the proposed method and its applicability to EMG-based control systems. In EMG pattern recognition, a classifier based on a generative model that reflects the stochastic characteristics of EMG signals can outperform the conventional general-purpose classifier.
△ Less
Submitted 20 July, 2021;
originally announced July 2021.
-
Biomimetic Control of Myoelectric Prosthetic Hand Based on a Lambda-type Muscle Model
Authors:
Akira Furui,
Kosuke Nakagaki,
Toshio Tsuji
Abstract:
Myoelectric prosthetic hands are intended to replace the function of the amputee's lost arm. Therefore, develo** robotic prosthetics that can mimic not only the appearance and functionality of humans but also characteristics unique to human movements is paramount. Although the impedance model was proposed to realize biomimetic control, this model cannot replicate the characteristics of human mov…
▽ More
Myoelectric prosthetic hands are intended to replace the function of the amputee's lost arm. Therefore, develo** robotic prosthetics that can mimic not only the appearance and functionality of humans but also characteristics unique to human movements is paramount. Although the impedance model was proposed to realize biomimetic control, this model cannot replicate the characteristics of human movements effectively because the joint angle always converges to the equilibrium position during muscle relaxation. This paper proposes a novel biomimetic control method for myoelectric prosthetic hands integrating the impedance model with the concept of the $λ$-type muscle model. The proposed method can dynamically control the joint equilibrium position, according to the state of the muscle, and can maintain the joint angle naturally during muscle relaxation. The effectiveness of the proposed method is evaluated through simulations and a series of experiments on non-amputee participants. The experimental results, based on comparison with the actual human joint angles, suggest that the proposed method has a better correlation with the actual human motion than the conventional methods. Additionally, the control experiments showed that the proposed method could achieve a natural prosthetic hand movement similar to that of a human, thereby allowing voluntary hand opening and closing movements.
△ Less
Submitted 29 May, 2021;
originally announced May 2021.
-
A New Autoregressive Neural Network Model with Command Compensation for Imitation Learning Based on Bilateral Control
Authors:
Kazuki Hayashi,
Ayumu Sasagawa,
Sho Sakaino,
Toshiaki Tsuji
Abstract:
In the near future, robots are expected to work with humans or operate alone and may replace human workers in various fields such as homes and factories. In a previous study, we proposed bilateral control-based imitation learning that enables robots to utilize force information and operate almost simultaneously with an expert's demonstration. In addition, we recently proposed an autoregressive neu…
▽ More
In the near future, robots are expected to work with humans or operate alone and may replace human workers in various fields such as homes and factories. In a previous study, we proposed bilateral control-based imitation learning that enables robots to utilize force information and operate almost simultaneously with an expert's demonstration. In addition, we recently proposed an autoregressive neural network model (SM2SM) for bilateral control-based imitation learning to obtain long-term inferences. In the SM2SM model, both master and slave states must be input, but the master states are obtained from the previous outputs of the SM2SM model, resulting in destabilized estimation under large environmental variations. Hence, a new autoregressive neural network model (S2SM) is proposed in this study. This model requires only the slave state as input and its outputs are the next slave and master states, thereby improving the task success rates. In addition, a new feedback controller that utilizes the error between the responses and estimates of the slave is proposed, which shows better reproducibility.
△ Less
Submitted 16 March, 2021;
originally announced March 2021.
-
Imitation learning for variable speed motion generation over multiple actions
Authors:
Yuki Saigusa,
Ayumu Sasagawa,
Sho Sakaino,
Toshiaki Tsuji
Abstract:
Robotic motion generation methods using machine learning have been studied in recent years. Bilateral control-based imitation learning can imitate human motions using force information. By means of this method, variable speed motion generation that considers physical phenomena such as the inertial force and friction can be achieved. However, the previous study only focused on a simple reciprocatin…
▽ More
Robotic motion generation methods using machine learning have been studied in recent years. Bilateral control-based imitation learning can imitate human motions using force information. By means of this method, variable speed motion generation that considers physical phenomena such as the inertial force and friction can be achieved. However, the previous study only focused on a simple reciprocating motion. To learn the complex relationship between the force and speed more accurately, it is necessary to learn multiple actions using many joints. In this paper, we propose a variable speed motion generation method for multiple motions. We considered four types of neural network models for the motion generation and determined the best model for multiple motions at variable speeds. Subsequently, we used the best model to evaluate the reproducibility of the task completion time for the input completion time command. The results revealed that the proposed method could change the task completion time according to the specified completion time command in multiple motions.
△ Less
Submitted 1 October, 2021; v1 submitted 11 March, 2021;
originally announced March 2021.
-
Imitation Learning for Variable Speed Contact Motion for Operation up to Control Bandwidth
Authors:
Sho Sakaino,
Kazuki Fujimoto,
Yuki Saigusa,
Toshiaki Tsuji
Abstract:
The generation of robot motions in the real world is difficult by using conventional controllers alone and requires highly intelligent processing. In this regard, learning-based motion generations are currently being investigated. However, the main issue has been improvements of the adaptability to spatially varying environments, but a variation of the operating speed has not been investigated in…
▽ More
The generation of robot motions in the real world is difficult by using conventional controllers alone and requires highly intelligent processing. In this regard, learning-based motion generations are currently being investigated. However, the main issue has been improvements of the adaptability to spatially varying environments, but a variation of the operating speed has not been investigated in detail. In contact-rich tasks, it is especially important to be able to adjust the operating speed because a nonlinear relationship occurs between the operating speed and force (e.g., inertial and frictional forces), and it affects the results of the tasks. Therefore, in this study, we propose a method for generating variable operating speeds while adapting to spatial perturbations in the environment. The proposed method can be adapted to nonlinearities by utilizing a small amount of motion data. We experimentally evaluated the proposed method by erasing a line using an eraser fixed to the tip of the robot as an example of a contact-rich task. Furthermore, the proposed method enables a robot to perform a task faster than a human operator and is capable of operating close to the control bandwidth.
△ Less
Submitted 12 February, 2022; v1 submitted 20 February, 2021;
originally announced February 2021.
-
Motion Generation Using Bilateral Control-Based Imitation Learning with Autoregressive Learning
Authors:
Ayumu Sasagawa,
Sho Sakaino,
Toshiaki Tsuji
Abstract:
Robots that can execute various tasks automatically on behalf of humans are becoming an increasingly important focus of research in the field of robotics. Imitation learning has been studied as an efficient and high-performance method, and imitation learning based on bilateral control has been proposed as a method that can realize fast motion. However, because this method cannot implement autoregr…
▽ More
Robots that can execute various tasks automatically on behalf of humans are becoming an increasingly important focus of research in the field of robotics. Imitation learning has been studied as an efficient and high-performance method, and imitation learning based on bilateral control has been proposed as a method that can realize fast motion. However, because this method cannot implement autoregressive learning, this method may not generate desirable long-term behavior. Therefore, in this paper, we propose a method of autoregressive learning for bilateral control-based imitation learning. A new neural network model for implementing autoregressive learning is proposed. In this study, three types of experiments are conducted to verify the effectiveness of the proposed method. The performance is improved compared to conventional approaches; the proposed method has the highest rate of success. Owing to the structure and autoregressive learning of the proposed model, the proposed method can generate the desirable motion for successful tasks and have a high generalization ability for environmental changes.
△ Less
Submitted 4 February, 2021; v1 submitted 11 November, 2020;
originally announced November 2020.
-
Assembly robots with optimized control stiffness through reinforcement learning
Authors:
Masahide Oikawa,
Kyo Kutsuzawa,
Sho Sakaino,
Toshiaki Tsuji
Abstract:
There is an increased demand for task automation in robots. Contact-rich tasks, wherein multiple contact transitions occur in a series of operations, are extensively being studied to realize high accuracy. In this study, we propose a methodology that uses reinforcement learning (RL) to achieve high performance in robots for the execution of assembly tasks that require precise contact with objects…
▽ More
There is an increased demand for task automation in robots. Contact-rich tasks, wherein multiple contact transitions occur in a series of operations, are extensively being studied to realize high accuracy. In this study, we propose a methodology that uses reinforcement learning (RL) to achieve high performance in robots for the execution of assembly tasks that require precise contact with objects without causing damage. The proposed method ensures the online generation of stiffness matrices that help improve the performance of local trajectory optimization. The method has an advantage of rapid response owing to short sampling time of the trajectory planning. The effectiveness of the method was verified via experiments involving two contact-rich tasks. The results indicate that the proposed method can be implemented in various contact-rich manipulations. A demonstration video shows the performance. (https://youtu.be/gxSCl7Tp4-0)
△ Less
Submitted 27 February, 2020;
originally announced February 2020.
-
A Neural Network Based on the Johnson $S_\mathrm{U}$ Translation System and Related Application to Electromyogram Classification
Authors:
Hideaki Hayashi,
Taro Shibanoki,
Toshio Tsuji
Abstract:
Electromyogram (EMG) classification is a key technique in EMG-based control systems. The existing EMG classification methods do not consider the characteristics of EMG features that the distribution has skewness and kurtosis, causing drawbacks such as the requirement of hyperparameter tuning. In this paper, we propose a neural network based on the Johnson $S_\mathrm{U}$ translation system that is…
▽ More
Electromyogram (EMG) classification is a key technique in EMG-based control systems. The existing EMG classification methods do not consider the characteristics of EMG features that the distribution has skewness and kurtosis, causing drawbacks such as the requirement of hyperparameter tuning. In this paper, we propose a neural network based on the Johnson $S_\mathrm{U}$ translation system that is capable of representing distributions with skewness and kurtosis. The Johnson system is a normalizing translation that transforms non-normal data to a normal distribution, thereby enabling the representation of a wide range of distributions. In this study, a discriminative model based on the multivariate Johnson $S_\mathrm{U}$ translation system is transformed into a linear combination of coefficients and input vectors using log-linearization. This is then incorporated into a neural network structure, thereby allowing the calculation of the posterior probability of the input vectors for each class and the determination of model parameters as weight coefficients of the network. The uniqueness of convergence of the network learning is theoretically guaranteed. In the experiments, the suitability of the proposed network for distributions including skewness and kurtosis is evaluated using artificially generated data. Its applicability for real biological data is also evaluated via an EMG classification experiment. The results show that the proposed network achieves high classification performance without the need for hyperparameter optimization.
△ Less
Submitted 14 November, 2019;
originally announced December 2019.
-
A Recurrent Probabilistic Neural Network with Dimensionality Reduction Based on Time-series Discriminant Component Analysis
Authors:
Hideaki Hayashi,
Taro Shibanoki,
Keisuke Shima,
Yuichi Kurita,
Toshio Tsuji
Abstract:
This paper proposes a probabilistic neural network developed on the basis of time-series discriminant component analysis (TSDCA) that can be used to classify high-dimensional time-series patterns. TSDCA involves the compression of high-dimensional time series into a lower-dimensional space using a set of orthogonal transformations and the calculation of posterior probabilities based on a continuou…
▽ More
This paper proposes a probabilistic neural network developed on the basis of time-series discriminant component analysis (TSDCA) that can be used to classify high-dimensional time-series patterns. TSDCA involves the compression of high-dimensional time series into a lower-dimensional space using a set of orthogonal transformations and the calculation of posterior probabilities based on a continuous-density hidden Markov model with a Gaussian mixture model expressed in the reduced-dimensional space. The analysis can be incorporated into a neural network, which is named a time-series discriminant component network (TSDCN), so that parameters of dimensionality reduction and classification can be obtained simultaneously as network coefficients according to a backpropagation through time-based learning algorithm with the Lagrange multiplier method. The TSDCN is considered to enable high-accuracy classification of high-dimensional time-series patterns and to reduce the computation time taken for network training. The validity of the TSDCN is demonstrated for high-dimensional artificial data and EEG signals in the experiments conducted during the study.
△ Less
Submitted 14 November, 2019;
originally announced November 2019.
-
Robots Assembling Machines: Learning from the World Robot Summit 2018 Assembly Challenge
Authors:
Felix von Drigalski,
Christian Schlette,
Martin Rudorfer,
Nikolaus Correll,
Joshua Triyonoputro,
Weiwei Wan,
Tokuo Tsuji,
Tetsuyou Watanabe
Abstract:
The Industrial Assembly Challenge at the World Robot Summit was held in 2018 to showcase the state-of-the-art of autonomous manufacturing systems. The challenge included various tasks, such as bin picking, kitting, and assembly of standard industrial parts into 2D and 3D assemblies. Some of the tasks were only revealed at the competition itself, representing the challenge of "level 5" automation,…
▽ More
The Industrial Assembly Challenge at the World Robot Summit was held in 2018 to showcase the state-of-the-art of autonomous manufacturing systems. The challenge included various tasks, such as bin picking, kitting, and assembly of standard industrial parts into 2D and 3D assemblies. Some of the tasks were only revealed at the competition itself, representing the challenge of "level 5" automation, i. e., programming and setting up an autonomous assembly system in less than one day. We conducted a survey among the teams that participated in the challenge and investigated aspects such as team composition, development costs, system setups as well as the teams' strategies and approaches. An analysis of the survey results reveals that the competitors have been in two camps: those constructing conventional robotic work cells with off-the-shelf tools, and teams who mostly relied on custom-made end effectors and novel software approaches in combination with collaborative robots. While both camps performed reasonably well, the winning team chose a middle ground in between, combining the efficiency of established play-back programming with the autonomy gained by CAD-based object detection and force control for assembly operations.
△ Less
Submitted 13 November, 2019;
originally announced November 2019.
-
Imitation Learning Based on Bilateral Control for Human-Robot Cooperation
Authors:
Ayumu Sasagawa,
Kazuki Fujimoto,
Sho Sakaino,
Toshiaki Tsuji
Abstract:
Robots are required to autonomously respond to changing situations. Imitation learning is a promising candidate for achieving generalization performance, and extensive results have been demonstrated in object manipulation. However, cooperative work between humans and robots is still a challenging issue because robots must control dynamic interactions among themselves, humans, and objects. Furtherm…
▽ More
Robots are required to autonomously respond to changing situations. Imitation learning is a promising candidate for achieving generalization performance, and extensive results have been demonstrated in object manipulation. However, cooperative work between humans and robots is still a challenging issue because robots must control dynamic interactions among themselves, humans, and objects. Furthermore, it is difficult to follow subtle perturbations that may occur among coworkers. In this study, we find that cooperative work can be accomplished by imitation learning using bilateral control. Thanks to bilateral control, which can extract response values and command values independently, human skills to control dynamic interactions can be extracted. Then, the task of serving food is considered. The experimental results clearly demonstrate the importance of force control, and the dynamic interactions can be controlled by the inferred action force.
△ Less
Submitted 19 January, 2021; v1 submitted 27 September, 2019;
originally announced September 2019.
-
Time Series Motion Generation Considering Long Short-Term Motion
Authors:
Kazuki Fujimoto,
Sho Sakaino,
Toshiaki Tsuji
Abstract:
Various adaptive abilities are required for robots interacting with humans in daily life. It is difficult to design adaptive algorithms manually; however, by using end-to-end machine learning, labor can be saved during the design process. In our previous research, a task requiring force adjustment was achieved through imitation learning that considered position and force information using a four-c…
▽ More
Various adaptive abilities are required for robots interacting with humans in daily life. It is difficult to design adaptive algorithms manually; however, by using end-to-end machine learning, labor can be saved during the design process. In our previous research, a task requiring force adjustment was achieved through imitation learning that considered position and force information using a four-channel bilateral control. Unfortunately, tasks that include long-term (slow) motion are still challenging. Furthermore, during system identification, there is a method known as the multi-decimation (MD) identification method. It separates lower and higher frequencies, and then identifies the parameters characterized at each frequency. Therefore, we proposed utilizing machine learning to take advantage of the MD method to infer short-term and long-term (high and low frequency, respectively) motion. In this paper, long-term motion tasks such as writing a letter using a pen fixed on a robot are discussed. We found differences in suitable sampling periods between position and force information. The validity of the proposed method was then experimentally verified, showing the importance of long-term inference with adequate sampling periods.
△ Less
Submitted 18 September, 2019; v1 submitted 9 September, 2019;
originally announced September 2019.
-
Imitation Learning for Object Manipulation Based on Position/Force Information Using Bilateral Control
Authors:
Tsuyoshi Adachi,
Kazuki Fujimoto,
Sho Sakaino,
Toshiaki Tsuji
Abstract:
This study proposes an imitation learning method based on force and position information. Force information is required for precise object manipulation but is difficult to obtain because the acting and reaction forces cannnot be separated. To separate the forces, we proposed to introduce bilateral control, in which the acting and reaction forces are divided using two robots. In the proposed method…
▽ More
This study proposes an imitation learning method based on force and position information. Force information is required for precise object manipulation but is difficult to obtain because the acting and reaction forces cannnot be separated. To separate the forces, we proposed to introduce bilateral control, in which the acting and reaction forces are divided using two robots. In the proposed method, two models of neural networks learn a task; to draw a line along a ruler. We verify the possibility that force information is essential to imitate the human skill of object manipulation.
△ Less
Submitted 8 November, 2018;
originally announced November 2018.
-
Estimation of Relationship between Stimulation Current and Force Exerted during Isometric Contraction
Authors:
Tomoya Kitamura,
Yuu Hasegawa,
Sho Sakaino,
Toshiaki Tsuji
Abstract:
In this study, we developed a method to estimate the relationship between stimulation current and volatility during isometric contraction. In functional electrical stimulation (FES), joints are driven by applying voltage to muscles. This technology has been used for a long time in the field of rehabilitation, and recently application oriented research has been reported. However, estimation of the…
▽ More
In this study, we developed a method to estimate the relationship between stimulation current and volatility during isometric contraction. In functional electrical stimulation (FES), joints are driven by applying voltage to muscles. This technology has been used for a long time in the field of rehabilitation, and recently application oriented research has been reported. However, estimation of the relationship between stimulus value and exercise capacity has not been discussed to a great extent. Therefore, in this study, a human muscle model was estimated using the transfer function estimation method with fast Fourier transform. It was found that the relationship between stimulation current and force exerted could be expressed by a first-order lag system. In verification of the force estimate, the ability of the proposed model to estimate the exerted force under steady state response was found to be good.
△ Less
Submitted 20 January, 2021; v1 submitted 7 November, 2018;
originally announced November 2018.
-
Experiments on Learning Based Industrial Bin-picking with Iterative Visual Recognition
Authors:
Kensuke Harada,
Weiwei Wan,
Tokuo Tsuji,
Kohei Kikuchi,
Kazuyuki Nagata,
Hiromu Onda
Abstract:
This paper shows experimental results on learning based randomized bin-picking combined with iterative visual recognition. We use the random forest to predict whether or not a robot will successfully pick an object for given depth images of the pile taking the collision between a finger and a neighboring object into account. For the discriminator to be accurate, we consider estimating objects' pos…
▽ More
This paper shows experimental results on learning based randomized bin-picking combined with iterative visual recognition. We use the random forest to predict whether or not a robot will successfully pick an object for given depth images of the pile taking the collision between a finger and a neighboring object into account. For the discriminator to be accurate, we consider estimating objects' poses by merging multiple depth images of the pile captured from different points of view by using a depth sensor attached at the wrist. We show that, even if a robot is predicted to fail in picking an object with a single depth image due to its large occluded area, it is finally predicted as success after merging multiple depth images. In addition, we show that the random forest can be trained with the small number of training data.
△ Less
Submitted 22 May, 2018;
originally announced May 2018.
-
Iterative Visual Recognition for Learning Based Randomized Bin-Picking
Authors:
Kensuke Harada,
Weiwei Wan,
Tokuo Tsuji,
Kohei Kikuchi,
Kazuyuki Nagata,
Hiromu Onda
Abstract:
This paper proposes a iterative visual recognition system for learning based randomized bin-picking. Since the configuration on randomly stacked objects while executing the current picking trial is just partially different from the configuration while executing the previous picking trial, we consider detecting the poses of objects just by using a part of visual image taken at the current picking t…
▽ More
This paper proposes a iterative visual recognition system for learning based randomized bin-picking. Since the configuration on randomly stacked objects while executing the current picking trial is just partially different from the configuration while executing the previous picking trial, we consider detecting the poses of objects just by using a part of visual image taken at the current picking trial where it is different from the visual image taken at the previous picking trial. By using this method, we do not need to try to detect the poses of all objects included in the pile at every picking trial. Assuming the 3D vision sensor attached at the wrist of a manipulator, we first explain a method to determine the pose of a 3D vision sensor maximizing the visibility of randomly stacked objects. Then, we explain a method for detecting the poses of randomly stacked objects. Effectiveness of our proposed approach is confirmed by experiments using a dual-arm manipulator where a 3D vision sensor and the two-fingered hand attached at the right and the left wrists, respectively.
△ Less
Submitted 1 August, 2016;
originally announced August 2016.
-
Initial Experiments on Learning-Based Randomized Bin-Picking Allowing Finger Contact with Neighboring Objects
Authors:
Kensuke Harada,
Weiwei Wan,
Tokuo Tsuji,
Kohei Kikuchi,
Kazuyuki Nagata,
Hiromu Onda
Abstract:
This paper proposes a novel method for randomized bin-picking based on learning. When a two-fingered gripper tries to pick an object from the pile, a finger often contacts a neighboring object. Even if a finger contacts a neighboring object, the target object will be successfully picked depending on the configuration of neighboring objects. In our proposed method, we use the visual information on…
▽ More
This paper proposes a novel method for randomized bin-picking based on learning. When a two-fingered gripper tries to pick an object from the pile, a finger often contacts a neighboring object. Even if a finger contacts a neighboring object, the target object will be successfully picked depending on the configuration of neighboring objects. In our proposed method, we use the visual information on neighboring objects to train the discriminator. Corresponding to a gras** posture of an object, the discriminator predicts whether or not the pick will be successful even if a finger contacts a neighboring object. We examine two learning algorithms, the linear support vector machine (SVM) and the random forest (RF) approaches. By using both methods, we demonstrate that the picking success rate is significantly higher than with conventional methods without learning.
△ Less
Submitted 11 July, 2016;
originally announced July 2016.
