-
Fourier Analysis on Robustness of Graph Convolutional Neural Networks for Skeleton-based Action Recognition
Authors:
Nariki Tanaka,
Hiroshi Kera,
Kazuhiko Kawamoto
Abstract:
Using Fourier analysis, we explore the robustness and vulnerability of graph convolutional neural networks (GCNs) for skeleton-based action recognition. We adopt a joint Fourier transform (JFT), a combination of the graph Fourier transform (GFT) and the discrete Fourier transform (DFT), to examine the robustness of adversarially-trained GCNs against adversarial attacks and common corruptions. Expe…
▽ More
Using Fourier analysis, we explore the robustness and vulnerability of graph convolutional neural networks (GCNs) for skeleton-based action recognition. We adopt a joint Fourier transform (JFT), a combination of the graph Fourier transform (GFT) and the discrete Fourier transform (DFT), to examine the robustness of adversarially-trained GCNs against adversarial attacks and common corruptions. Experimental results with the NTU RGB+D dataset reveal that adversarial training does not introduce a robustness trade-off between adversarial attacks and low-frequency perturbations, which typically occurs during image classification based on convolutional neural networks. This finding indicates that adversarial training is a practical approach to enhancing robustness against adversarial attacks and common corruptions in skeleton-based action recognition. Furthermore, we find that the Fourier approach cannot explain vulnerability against skeletal part occlusion corruption, which highlights its limitations. These findings extend our understanding of the robustness of GCNs, potentially guiding the development of more robust learning methods for skeleton-based action recognition.
△ Less
Submitted 30 December, 2023; v1 submitted 29 May, 2023;
originally announced May 2023.
-
Integrating a Manual Pipette into a Collaborative Robot Manipulator for Flexible Liquid Dispensing
Authors:
Junbo Zhang,
Weiwei Wan,
Nobuyuki Tanaka,
Miki Fujita,
Kensuke Harada
Abstract:
This paper presents a system integration approach for a 6-DoF (Degree of Freedom) collaborative robot to operate a pipette for liquid dispensing. Its technical development is threefold. First, we designed an end-effector for holding and triggering manual pipettes. Second, we took advantage of a collaborative robot to recognize labware poses and planned robotic motion based on the recognized poses.…
▽ More
This paper presents a system integration approach for a 6-DoF (Degree of Freedom) collaborative robot to operate a pipette for liquid dispensing. Its technical development is threefold. First, we designed an end-effector for holding and triggering manual pipettes. Second, we took advantage of a collaborative robot to recognize labware poses and planned robotic motion based on the recognized poses. Third, we developed vision-based classifiers to predict and correct positioning errors and thus precisely attached pipettes to disposable tips. Through experiments and analysis, we confirmed that the developed system, especially the planning and visual recognition methods, could help secure high-precision and flexible liquid dispensing. The developed system is suitable for low-frequency, high-repetition biochemical liquid dispensing tasks. We expect it to promote the deployment of collaborative robots for laboratory automation and thus improve the experimental efficiency without significantly customizing a laboratory environment.
△ Less
Submitted 4 July, 2022;
originally announced July 2022.
-
Adversarial Bone Length Attack on Action Recognition
Authors:
Nariki Tanaka,
Hiroshi Kera,
Kazuhiko Kawamoto
Abstract:
Skeleton-based action recognition models have recently been shown to be vulnerable to adversarial attacks. Compared to adversarial attacks on images, perturbations to skeletons are typically bounded to a lower dimension of approximately 100 per frame. This lower-dimensional setting makes it more difficult to generate imperceptible perturbations. Existing attacks resolve this by exploiting the temp…
▽ More
Skeleton-based action recognition models have recently been shown to be vulnerable to adversarial attacks. Compared to adversarial attacks on images, perturbations to skeletons are typically bounded to a lower dimension of approximately 100 per frame. This lower-dimensional setting makes it more difficult to generate imperceptible perturbations. Existing attacks resolve this by exploiting the temporal structure of the skeleton motion so that the perturbation dimension increases to thousands. In this paper, we show that adversarial attacks can be performed on skeleton-based action recognition models, even in a significantly low-dimensional setting without any temporal manipulation. Specifically, we restrict the perturbations to the lengths of the skeleton's bones, which allows an adversary to manipulate only approximately 30 effective dimensions. We conducted experiments on the NTU RGB+D and HDM05 datasets and demonstrate that the proposed attack successfully deceived models with sometimes greater than 90% success rate by small perturbations. Furthermore, we discovered an interesting phenomenon: in our low-dimensional setting, the adversarial training with the bone length attack shares a similar property with data augmentation, and it not only improves the adversarial robustness but also improves the classification accuracy on the original data. This is an interesting counterexample of the trade-off between adversarial robustness and clean accuracy, which has been widely observed in studies on adversarial training in the high-dimensional regime.
△ Less
Submitted 25 March, 2022; v1 submitted 13 September, 2021;
originally announced September 2021.
-
A Word Communication System with Caregiver Assist for Amyotrophic Lateral Sclerosis Patients in Completely and Almost Completely Locked-in State
Authors:
Kuniaki Ozawa,
Masayoshi Naito,
Naoki Tanaka,
Shiryu Wada
Abstract:
People with heavy physical impairment such as amyotrophic lateral sclerosis (ALS) in a completely locked-in state (CLIS) suffer from inability to express their thoughts to others. To solve this problem, many brain-computer interface (BCI) systems have been developed, but they have not proven sufficient for CLIS. In this paper, we propose a word communication system: a BCI with caregiver assist, in…
▽ More
People with heavy physical impairment such as amyotrophic lateral sclerosis (ALS) in a completely locked-in state (CLIS) suffer from inability to express their thoughts to others. To solve this problem, many brain-computer interface (BCI) systems have been developed, but they have not proven sufficient for CLIS. In this paper, we propose a word communication system: a BCI with caregiver assist, in which caregivers play an active role in hel** patients express a word. We report here that four ALS patients in almost CLIS and one in CLIS succeeded in expressing their own words (in Japanese) in response to wh-questions that could not be answered "yes/no." Each subject selected vowels (maximum three) contained in the word that he or she wanted to express in a sequential way, by using a "yes/no" communication aid based on near-infrared light. Then, a caregiver entered the selected vowels into a dictionary with vowel entries, which returned candidate words having those vowels. When there were no appropriate words, the caregiver changed one vowel and searched again or started over from the beginning. When an appropriate word was selected, it was confirmed by the subject via "yes/no" answers. Three subjects expressed "yes" for the selected word at least six times out of eight (reliability of 91.0% by a statistical measure), one subject (in CLIS) did so five times out of eight (74.6%), and one subject three times out of four (81.3%). We have thus taken the first step toward a practical word communication system for such patients.
△ Less
Submitted 22 April, 2020;
originally announced April 2020.
