-
Exploring the Effects of Shared Autonomy on Cognitive Load and Trust in Human-Robot Interaction
Authors:
Jiahe Pan,
Jonathan Eden,
Denny Oetomo,
Wafa Johal
Abstract:
Teleoperation is increasingly recognized as a viable solution for deploying robots in hazardous environments. Controlling a robot to perform a complex or demanding task may overload operators resulting in poor performance. To design a robot controller to assist the human in executing such challenging tasks, a comprehensive understanding of the interplay between the robot's autonomous behavior and…
▽ More
Teleoperation is increasingly recognized as a viable solution for deploying robots in hazardous environments. Controlling a robot to perform a complex or demanding task may overload operators resulting in poor performance. To design a robot controller to assist the human in executing such challenging tasks, a comprehensive understanding of the interplay between the robot's autonomous behavior and the operator's internal state is essential. In this paper, we investigate the relationships between robot autonomy and both the human user's cognitive load and trust levels, and the potential existence of three-way interactions in the robot-assisted execution of the task. Our user study (N=24) results indicate that while autonomy level influences the teleoperator's perceived cognitive load and trust, there is no clear interaction between these factors. Instead, these elements appear to operate independently, thus highlighting the need to consider both cognitive load and trust as distinct but interrelated factors in varying the robot autonomy level in shared-control settings. This insight is crucial for the development of more effective and adaptable assistive robotic systems.
△ Less
Submitted 5 February, 2024;
originally announced February 2024.
-
When To Grow? A Fitting Risk-Aware Policy for Layer Growing in Deep Neural Networks
Authors:
Haihang Wu,
Wei Wang,
Tamasha Malepathirana,
Damith Senanayake,
Denny Oetomo,
Saman Halgamuge
Abstract:
Neural growth is the process of growing a small neural network to a large network and has been utilized to accelerate the training of deep neural networks. One crucial aspect of neural growth is determining the optimal growth timing. However, few studies investigate this systematically. Our study reveals that neural growth inherently exhibits a regularization effect, whose intensity is influenced…
▽ More
Neural growth is the process of growing a small neural network to a large network and has been utilized to accelerate the training of deep neural networks. One crucial aspect of neural growth is determining the optimal growth timing. However, few studies investigate this systematically. Our study reveals that neural growth inherently exhibits a regularization effect, whose intensity is influenced by the chosen policy for growth timing. While this regularization effect may mitigate the overfitting risk of the model, it may lead to a notable accuracy drop when the model underfits. Yet, current approaches have not addressed this issue due to their lack of consideration of the regularization effect from neural growth. Motivated by these findings, we propose an under/over fitting risk-aware growth timing policy, which automatically adjusts the growth timing informed by the level of potential under/overfitting risks to address both risks. Comprehensive experiments conducted using CIFAR-10/100 and ImageNet datasets show that the proposed policy achieves accuracy improvements of up to 1.3% in models prone to underfitting while achieving similar accuracies in models suffering from overfitting compared to the existing methods.
△ Less
Submitted 5 January, 2024;
originally announced January 2024.
-
Data-Driven Goal Recognition in Transhumeral Prostheses Using Process Mining Techniques
Authors:
Zihang Su,
Tianshi Yu,
Nir Lipovetzky,
Alireza Mohammadi,
Denny Oetomo,
Artem Polyvyanyy,
Sebastian Sardina,
Ying Tan,
Nick van Beest
Abstract:
A transhumeral prosthesis restores missing anatomical segments below the shoulder, including the hand. Active prostheses utilize real-valued, continuous sensor data to recognize patient target poses, or goals, and proactively move the artificial limb. Previous studies have examined how well the data collected in stationary poses, without considering the time steps, can help discriminate the goals.…
▽ More
A transhumeral prosthesis restores missing anatomical segments below the shoulder, including the hand. Active prostheses utilize real-valued, continuous sensor data to recognize patient target poses, or goals, and proactively move the artificial limb. Previous studies have examined how well the data collected in stationary poses, without considering the time steps, can help discriminate the goals. In this case study paper, we focus on using time series data from surface electromyography electrodes and kinematic sensors to sequentially recognize patients' goals. Our approach involves transforming the data into discrete events and training an existing process mining-based goal recognition system. Results from data collected in a virtual reality setting with ten subjects demonstrate the effectiveness of our proposed goal recognition approach, which achieves significantly better precision and recall than the state-of-the-art machine learning techniques and is less confident when wrong, which is beneficial when approximating smoother movements of prostheses.
△ Less
Submitted 14 September, 2023;
originally announced September 2023.
-
Varying Joint Patterns and Compensatory Strategies Can Lead to the Same Functional Gait Outcomes: A Case Study
Authors:
T. Bacek,
M. Sun,
H. Liu,
Z. Chen,
D. Kulic,
D. Oetomo,
Y. Tan
Abstract:
This paper analyses joint-space walking mechanisms and redundancies in delivering functional gait outcomes. Multiple biomechanical measures are analysed for two healthy male adults who participated in a multi-factorial study and walked during three sessions. Both participants employed varying intra- and inter-personal compensatory strategies (e.g., vaulting, hip hiking) across walking conditions a…
▽ More
This paper analyses joint-space walking mechanisms and redundancies in delivering functional gait outcomes. Multiple biomechanical measures are analysed for two healthy male adults who participated in a multi-factorial study and walked during three sessions. Both participants employed varying intra- and inter-personal compensatory strategies (e.g., vaulting, hip hiking) across walking conditions and exhibited notable gait pattern alterations while kee** task-space (functional) gait parameters invariant. They also preferred various levels of asymmetric step length but kept their symmetric step time consistent and cadence-invariant during free walking. The results demonstrate the importance of an individualised approach and the need for a paradigm shift from functional (task-space) to joint-space gait analysis in attending to (a)typical gaits and delivering human-centred human-robot interaction.
△ Less
Submitted 26 June, 2022;
originally announced June 2022.
-
Frequency Superposition -- A Multi-Frequency Stimulation Method in SSVEP-based BCIs
Authors:
**g Mu,
David B. Grayden,
Ying Tan,
Denny Oetomo
Abstract:
The steady-state visual evoked potential (SSVEP) is one of the most widely used modalities in brain-computer interfaces (BCIs) due to its many advantages. However, the existence of harmonics and the limited range of responsive frequencies in SSVEP make it challenging to further expand the number of targets without sacrificing other aspects of the interface or putting additional constraints on the…
▽ More
The steady-state visual evoked potential (SSVEP) is one of the most widely used modalities in brain-computer interfaces (BCIs) due to its many advantages. However, the existence of harmonics and the limited range of responsive frequencies in SSVEP make it challenging to further expand the number of targets without sacrificing other aspects of the interface or putting additional constraints on the system. This paper introduces a novel multi-frequency stimulation method for SSVEP and investigates its potential to effectively and efficiently increase the number of targets presented. The proposed stimulation method, obtained by the superposition of the stimulation signals at different frequencies, is size-efficient, allows single-step target identification, puts no strict constraints on the usable frequency range, can be suited to self-paced BCIs, and does not require specific light sources. In addition to the stimulus frequencies and their harmonics, the evoked SSVEP waveforms include frequencies that are integer linear combinations of the stimulus frequencies. Results of decoding SSVEPs collected from nine subjects using canonical correlation analysis (CCA) with only the frequencies and harmonics as reference, also demonstrate the potential of using such a stimulation paradigm in SSVEP-based BCIs.
△ Less
Submitted 11 August, 2021; v1 submitted 25 April, 2021;
originally announced April 2021.
-
Multi-Frequency Canonical Correlation Analysis (MFCCA): A Generalised Decoding Algorithm for Multi-Frequency SSVEP
Authors:
**g Mu,
Ying Tan,
David B. Grayden,
Denny Oetomo
Abstract:
Stimulation methods that utilise more than one stimulation frequency have been developed for steady-state visual evoked potential (SSVEP) brain-computer interfaces (BCIs) with the purpose of increasing the number of targets that can be presented simultaneously. However, there is no unified decoding algorithm that can be used without training for each individual users or cases, and applied to a lar…
▽ More
Stimulation methods that utilise more than one stimulation frequency have been developed for steady-state visual evoked potential (SSVEP) brain-computer interfaces (BCIs) with the purpose of increasing the number of targets that can be presented simultaneously. However, there is no unified decoding algorithm that can be used without training for each individual users or cases, and applied to a large class of multi-frequency stimulated SSVEP settings. This paper extends the widely used canonical correlation analysis (CCA) decoder to explicitly accommodate multi-frequency SSVEP by exploiting the interactions between the multiple stimulation frequencies. A concept of order, defined as the sum of absolute value of the coefficients in the linear combination of the input frequencies, was introduced to assist the design of Multi-Frequency CCA (MFCCA). The probability distribution of the order in the resulting SSVEP response was then used to improve decoding accuracy. Results show that, compared to the standard CCA formulation, the proposed MFCCA has a 20% improvement in decoding accuracy on average at order 2, while kee** its generality and training-free characteristics.
△ Less
Submitted 11 August, 2021; v1 submitted 27 October, 2020;
originally announced November 2020.
-
The Use of Implicit Human Motor Behaviour in the Online Personalisation of Prosthetic Interfaces
Authors:
Ricardo Garcia-Rosas,
Tianshi Yu,
Denny Oetomo,
Chris Manzie,
Ying Tan,
Peter Choong
Abstract:
In previous work, the authors proposed a data-driven optimisation algorithm for the personalisation of human-prosthetic interfaces, demonstrating the possibility of adapting prosthesis behaviour to its user while the user performs tasks with it. This method requires that the human and the prosthesis personalisation algorithm have same pre-defined objective function. This was previously ensured by…
▽ More
In previous work, the authors proposed a data-driven optimisation algorithm for the personalisation of human-prosthetic interfaces, demonstrating the possibility of adapting prosthesis behaviour to its user while the user performs tasks with it. This method requires that the human and the prosthesis personalisation algorithm have same pre-defined objective function. This was previously ensured by providing the human with explicit feedback on what the objective function is. However, constantly displaying this information to the prosthesis user is impractical. Moreover, the method utilised task information in the objective function which may not be available from the wearable sensors typically used in prosthetic applications. In this work, the previous approach is extended to use a prosthesis objective function based on implicit human motor behaviour, which represents able-bodied human motor control and is measureable using wearable sensors. The approach is tested in a hardware implementation of the personalisation algorithm on a prosthetic elbow, where the prosthetic objective function is a function of upper-body compensation, and is measured using wearable IMUs. Experimental results on able-bodied subjects using a supernumerary prosthetic elbow mounted on an elbow orthosis suggest that it is possible to use a prosthesis objective function which is implicit in human behaviour to achieve collaboration without providing explicit feedback to the human, motivating further studies.
△ Less
Submitted 2 March, 2020;
originally announced March 2020.
-
Task-space Synergies for Reaching using Upper-limb Prostheses
Authors:
Ricardo Garcia-Rosas,
Denny Oetomo,
Chris Manzie,
Ying Tan,
Peter Choong
Abstract:
Synergistic prostheses enable the coordinated movement of the human-prosthetic arm, as required by activities of daily living. This is achieved by coupling the motion of the prosthesis to the human command, such as the residual limb movement in motion-based interfaces. Previous studies demonstrated that develo** human-prosthetic synergies in joint-space must consider individual motor behaviour a…
▽ More
Synergistic prostheses enable the coordinated movement of the human-prosthetic arm, as required by activities of daily living. This is achieved by coupling the motion of the prosthesis to the human command, such as the residual limb movement in motion-based interfaces. Previous studies demonstrated that develo** human-prosthetic synergies in joint-space must consider individual motor behaviour and the intended task to be performed, requiring personalisation and task calibration. In this work, an alternative synergy-based strategy, utilising a synergistic relationship expressed in task-space, is proposed. This task-space synergy has the potential to replace the need for personalisation and task calibration with a model-based approach requiring knowledge of the individual user's arm kinematics, the anticipated hand motion during the task and voluntary information from the prosthetic user. The proposed method is compared with surface electromyography-based and joint-space synergy-based prosthetic interfaces in a study of motor behaviour and task performance on able-bodied subjects using a VR-based transhumeral prosthesis. Experimental results showed that for a set of forward reaching tasks the proposed task-space synergy achieves comparable performance to joint-space synergies without the need to rely on time-consuming calibration processes or human motor learning. Case study results with an amputee subject motivate the further development of the proposed task-space synergy method.
△ Less
Submitted 3 November, 2020; v1 submitted 19 February, 2020;
originally announced February 2020.
-
Control Barrier Functions for Mechanical Systems: Theory and Application to Robotic Gras**
Authors:
Wenceslao Shaw Cortez,
Denny Oetomo,
Chris Manzie,
Peter Choong
Abstract:
Control barrier functions have been demonstrated to be a useful method of ensuring constraint satisfaction for a wide class of controllers, however existing results are mostly restricted to continuous time systems of relative degree one. Mechanical systems, including robots, are typically second-order systems in which the control occurs at the force/torque level. These systems have velocity and po…
▽ More
Control barrier functions have been demonstrated to be a useful method of ensuring constraint satisfaction for a wide class of controllers, however existing results are mostly restricted to continuous time systems of relative degree one. Mechanical systems, including robots, are typically second-order systems in which the control occurs at the force/torque level. These systems have velocity and position constraints (i.e. relative degree two) that are vital for safety and/or task execution. Additionally, mechanical systems are typically controlled digitally as sampled-data systems. The contribution of this work is two-fold. First, is the development of novel, robust control barrier functions that ensure constraint satisfaction for relative degree two, sampled-data systems in the presence of model uncertainty. Second, is the application of the proposed method to the challenging problem of robotic gras** in which a robotic hand must ensure an object remains inside the grasp while manipulating it to a desired reference trajectory. A grasp constraint satisfying controller is proposed that can admit existing nominal manipulation controllers from the literature, while simultaneously ensuring no slip, no over-extension (e.g. singular configurations), and no rolling off of the fingertips. Simulation and experimental results validate the proposed control for the robotic hand application.
△ Less
Submitted 23 March, 2019;
originally announced March 2019.
-
Personalized On-line Adaptation of Kinematic Synergies for Human-Prosthesis Interfaces
Authors:
Ricardo Garcia-Rosas,
Ying Tan,
Denny Oetomo,
Chris Manzie,
Peter Choong
Abstract:
Synergies have been adopted in prosthetic limb applications to reduce complexity of design, but typically involve a single synergy setting for a population and ignore individual preference or adaptation capacity. However, personalization of the synergy setting is necessary for the effective operation of the prosthetic device. Two major challenges hinder the personalization of synergies in human-pr…
▽ More
Synergies have been adopted in prosthetic limb applications to reduce complexity of design, but typically involve a single synergy setting for a population and ignore individual preference or adaptation capacity. However, personalization of the synergy setting is necessary for the effective operation of the prosthetic device. Two major challenges hinder the personalization of synergies in human-prosthesis interfaces. The first is related to the process of human motor adaptation and the second to the variation in motor learning dynamics of individuals. In this paper, a systematic personalization of kinematic synergies for human-prosthesis interfaces using on-line measurements from each individual is proposed. The task of reaching using the upper-limb is described by an objective function and the interface is parameterized by a kinematic synergy. Consequently, personalizing the interface for a given individual can be formulated as finding an optimal personalized parameter. A structure to model the observed motor behavior that allows for the personalized traits of motor preference and motor learning is proposed, and subsequently used in an on-line optimization scheme to identify the synergies for an individual. The knowledge of the common features contained in the model enables on-line adaptation of the human-prosthesis interface to happen concurrently to human motor adaptation without the need to re-tune the personalization algorithm for each individual. Human-in-the-loop experimental results with able-bodied subjects, performed in a virtual reality environment to emulate amputation and prosthesis use, show that the proposed personalization algorithm was effective in obtaining optimal synergies with a fast uniform convergence speed across a group of individuals.
△ Less
Submitted 19 February, 2020; v1 submitted 19 February, 2019;
originally announced February 2019.
-
Grasp Constraint Satisfaction for Object Manipulation using Robotic Hands
Authors:
Wenceslao Shaw Cortez,
Denny Oetomo,
Chris Manzie,
Peter Choong
Abstract:
For successful object manipulation with robotic hands, it is important to ensure that the object remains in the grasp at all times. In addition to grasp constraints associated with slip** and singular hand configurations, excessive rolling is an important grasp concern where the contact points roll off of the fingertip surface. Related literature focus only on a subset of grasp constraints, or a…
▽ More
For successful object manipulation with robotic hands, it is important to ensure that the object remains in the grasp at all times. In addition to grasp constraints associated with slip** and singular hand configurations, excessive rolling is an important grasp concern where the contact points roll off of the fingertip surface. Related literature focus only on a subset of grasp constraints, or assume grasp constraint satisfaction without providing guarantees of such a claim. In this paper, we propose a control approach that systematically handles all grasp constraints. The proposed controller ensures that the object does not slip, joints do not exceed joint angle constraints (e.g. reach singular configurations), and the contact points remain in the fingertip workspace. The proposed controller accepts a nominal manipulation control, and ensures the gras** constraints are satisfied to support the assumptions made in the literature. Simulation results validate the proposed approach.
△ Less
Submitted 26 November, 2018;
originally announced November 2018.
-
Robust Object Manipulation for Tactile-based Blind Gras**
Authors:
Wenceslao Shaw-Cortez,
Denny Oetomo,
Chris Manzie,
Peter Choong
Abstract:
Tactile-based blind gras** addresses realistic robotic gras** in which the hand only has access to proprioceptive and tactile sensors. The robotic hand has no prior knowledge of the object/grasp properties, such as object weight, inertia, and shape. There exists no manipulation controller that rigorously guarantees object manipulation in such a setting. Here, a robust control law is proposed f…
▽ More
Tactile-based blind gras** addresses realistic robotic gras** in which the hand only has access to proprioceptive and tactile sensors. The robotic hand has no prior knowledge of the object/grasp properties, such as object weight, inertia, and shape. There exists no manipulation controller that rigorously guarantees object manipulation in such a setting. Here, a robust control law is proposed for object manipulation in tactile-based blind gras**. The analysis ensures semi-global asymptotic and exponential stability in the presence of model uncertainties and external disturbances that are neglected in related work. Simulation and experimental results validate the effectiveness of the proposed approach.
△ Less
Submitted 7 February, 2019; v1 submitted 9 September, 2017;
originally announced September 2017.
