-
Ergodic imitation: Learning from what to do and what not to do
Authors:
Aleksandra Kalinowska,
Ahalya Prabhakar,
Kathleen Fitzsimons,
Todd Murphey
Abstract:
With growing access to versatile robotics, it is beneficial for end users to be able to teach robots tasks without needing to code a control policy. One possibility is to teach the robot through successful task executions. However, near-optimal demonstrations of a task can be difficult to provide and even successful demonstrations can fail to capture task aspects key to robust skill replication. H…
▽ More
With growing access to versatile robotics, it is beneficial for end users to be able to teach robots tasks without needing to code a control policy. One possibility is to teach the robot through successful task executions. However, near-optimal demonstrations of a task can be difficult to provide and even successful demonstrations can fail to capture task aspects key to robust skill replication. Here, we propose a learning from demonstration (LfD) approach that enables learning of robust task definitions without the need for near-optimal demonstrations. We present a novel algorithmic framework for learning tasks based on the ergodic metric -- a measure of information content in motion. Moreover, we make use of negative demonstrations -- demonstrations of what not to do -- and show that they can help compensate for imperfect demonstrations, reduce the number of demonstrations needed, and highlight crucial task elements improving robot performance. In a proof-of-concept example of cart-pole inversion, we show that negative demonstrations alone can be sufficient to successfully learn and recreate a skill. Through a human subject study with 24 participants, we show that consistently more information about a task can be captured from combined positive and negative (posneg) demonstrations than from the same amount of just positive demonstrations. Finally, we demonstrate our learning approach on simulated tasks of target reaching and table cleaning with a 7-DoF Franka arm. Our results point towards a future with robust, data-efficient LfD for novice users.
△ Less
Submitted 31 March, 2021;
originally announced March 2021.
-
Dynamical System Segmentation for Information Measures in Motion
Authors:
Thomas A. Berrueta,
Ana Pervan,
Kathleen Fitzsimons,
Todd D. Murphey
Abstract:
Motions carry information about the underlying task being executed. Previous work in human motion analysis suggests that complex motions may result from the composition of fundamental submovements called movemes. The existence of finite structure in motion motivates information-theoretic approaches to motion analysis and robotic assistance. We define task embodiment as the amount of task informati…
▽ More
Motions carry information about the underlying task being executed. Previous work in human motion analysis suggests that complex motions may result from the composition of fundamental submovements called movemes. The existence of finite structure in motion motivates information-theoretic approaches to motion analysis and robotic assistance. We define task embodiment as the amount of task information encoded in an agent's motions. By decoding task-specific information embedded in motion, we can use task embodiment to create detailed performance assessments. We extract an alphabet of behaviors comprising a motion without \textit{a priori} knowledge using a novel algorithm, which we call dynamical system segmentation. For a given task, we specify an optimal agent, and compute an alphabet of behaviors representative of the task. We identify these behaviors in data from agent executions, and compare their relative frequencies against that of the optimal agent using the Kullback-Leibler divergence. We validate this approach using a dataset of human subjects (n=53) performing a dynamic task, and under this measure find that individuals receiving assistance better embody the task. Moreover, we find that task embodiment is a better predictor of assistance than integrated mean-squared-error.
△ Less
Submitted 9 December, 2020;
originally announced December 2020.
-
Shoulder abduction loading affects motor coordination in individuals with chronic stroke, informing targeted rehabilitation
Authors:
Aleksandra Kalinowska,
Kyra Rudy,
Millicent Schlafly,
Kathleen Fitzsimons,
Julius P Dewald,
Todd D Murphey
Abstract:
Individuals post stroke experience motor impairments, such as loss of independent joint control, leading to an overall reduction in arm function. Their motion becomes slower and more discoordinated, making it difficult to complete timing-sensitive tasks, such as balancing a glass of water or carrying a bowl with a ball inside it. Understanding how the stroke-induced motor impairments interact with…
▽ More
Individuals post stroke experience motor impairments, such as loss of independent joint control, leading to an overall reduction in arm function. Their motion becomes slower and more discoordinated, making it difficult to complete timing-sensitive tasks, such as balancing a glass of water or carrying a bowl with a ball inside it. Understanding how the stroke-induced motor impairments interact with each other can help design assisted training regimens for improved recovery. In this study, we investigate the effects of abnormal joint coupling patterns induced by flexion synergy on timing-sensitive motor coordination in the paretic upper limb. We design a virtual ball-in-bowl task that requires fast movements for optimal performance and implement it on a robotic system, capable of providing varying levels of abduction loading at the shoulder. We recruit 12 participants (6 individuals with chronic stroke and 6 unimpaired controls) and assess their skill at the task at 3 levels of loading, defined by the vertical force applied at the robot end-effector. Our results show that, for individuals with stroke, loading has a significant effect on their ability to generate quick coordinated motion. With increases in loading, their overall task performance decreases and they are less able to compensate for ball dynamics---frequency analysis of their motion indicates that abduction loading weakens their ability to generate movements at the resonant frequency of the dynamic task. This effect is likely due to an increased reliance on lower resolution indirect motor pathways in individuals post stroke. Given the inter-dependency of loading and dynamic task performance, we can create targeted robot-aided training protocols focused on improving timing-sensitive motor control, similar to existing progressive loading therapies, which have shown efficacy for expanding reachable workspace post stroke.
△ Less
Submitted 5 June, 2020;
originally announced July 2020.
-
A note on blind contact tracing at scale with applications to the COVID-19 pandemic
Authors:
Jack K. Fitzsimons,
Atul Mantri,
Robert Pisarczyk,
Tom Rainforth,
Zhikuan Zhao
Abstract:
The current COVID-19 pandemic highlights the utility of contact tracing, when combined with case isolation and social distancing, as an important tool for mitigating the spread of a disease [1]. Contact tracing provides a mechanism of identifying individuals with a high likelihood of previous exposure to a contagious disease, allowing additional precautions to be put in place to prevent continued…
▽ More
The current COVID-19 pandemic highlights the utility of contact tracing, when combined with case isolation and social distancing, as an important tool for mitigating the spread of a disease [1]. Contact tracing provides a mechanism of identifying individuals with a high likelihood of previous exposure to a contagious disease, allowing additional precautions to be put in place to prevent continued transmission. Here we consider a cryptographic approach to contact tracing based on secure two-party computation (2PC). We begin by considering the problem of comparing a set of location histories held by two parties to determine whether they have come within some threshold distance while at the same time maintaining the privacy of the location histories. We propose a solution to this problem using pre-shared keys, adapted from an equality testing protocol due to Ishai et al [2]. We discuss how this protocol can be used to maintain privacy within practical contact tracing scenarios, including both app-based approaches and approaches which leverage location history held by telecoms and internet service providers. We examine the efficiency of this approach and show that existing infrastructure is sufficient to support anonymised contact tracing at a national level.
△ Less
Submitted 10 April, 2020;
originally announced April 2020.
-
Implicit Priors for Knowledge Sharing in Bayesian Neural Networks
Authors:
Jack K Fitzsimons,
Sebastian M Schmon,
Stephen J Roberts
Abstract:
Bayesian interpretations of neural network have a long history, dating back to early work in the 1990's and have recently regained attention because of their desirable properties like uncertainty estimation, model robustness and regularisation. We want to discuss here the application of Bayesian models to knowledge sharing between neural networks. Knowledge sharing comes in different facets, such…
▽ More
Bayesian interpretations of neural network have a long history, dating back to early work in the 1990's and have recently regained attention because of their desirable properties like uncertainty estimation, model robustness and regularisation. We want to discuss here the application of Bayesian models to knowledge sharing between neural networks. Knowledge sharing comes in different facets, such as transfer learning, model distillation and shared embeddings. All of these tasks have in common that learned "features" ought to be shared across different networks. Theoretically rooted in the concepts of Bayesian neural networks this work has widespread application to general deep learning.
△ Less
Submitted 2 December, 2019;
originally announced December 2019.
-
Task-Based Hybrid Shared Control for Training Through Forceful Interaction
Authors:
Kathleen Fitzsimons,
Aleksandra Kalinowska,
Julius P. A. Dewald,
Todd Murphey
Abstract:
Despite the fact that robotic platforms can provide both consistent practice and objective assessments of users over the course of their training, there are relatively few instances where physical human robot interaction has been significantly more effective than unassisted practice or human-mediated training. This paper describes a hybrid shared control robot, which enhances task learning through…
▽ More
Despite the fact that robotic platforms can provide both consistent practice and objective assessments of users over the course of their training, there are relatively few instances where physical human robot interaction has been significantly more effective than unassisted practice or human-mediated training. This paper describes a hybrid shared control robot, which enhances task learning through kinesthetic feedback. The assistance assesses user actions using a task-specific evaluation criterion and selectively accepts or rejects them at each time instant. Through two human subject studies (total n=68), we show that this hybrid approach of switching between full transparency and full rejection of user inputs leads to increased skill acquisition and short-term retention compared to unassisted practice. Moreover, we show that the shared control paradigm exhibits features previously shown to promote successful training. It avoids user passivity by only rejecting user actions and allowing failure at the task. It improves performance during assistance, providing meaningful task-specific feedback. It is sensitive to initial skill of the user and behaves as an `assist-as-needed' control scheme---adapting its engagement in real time based on the performance and needs of the user. Unlike other successful algorithms, it does not require explicit modulation of the level of impedance or error amplification during training and it is permissive to a range of strategies because of its evaluation criterion. We demonstrate that the proposed hybrid shared control paradigm with a task-based minimal intervention criterion significantly enhances task-specific training.
△ Less
Submitted 18 November, 2019;
originally announced November 2019.
-
Online User Assessment for Minimal Intervention During Task-Based Robotic Assistance
Authors:
Aleksandra Kalinowska,
Kathleen Fitzsimons,
Julius Dewald,
Todd D Murphey
Abstract:
We propose a novel criterion for evaluating user input for human-robot interfaces for known tasks. We use the mode insertion gradient (MIG)---a tool from hybrid control theory---as a filtering criterion that instantaneously assesses the impact of user actions on a dynamic system over a time window into the future. As a result, the filter is permissive to many chosen strategies, minimally engaging,…
▽ More
We propose a novel criterion for evaluating user input for human-robot interfaces for known tasks. We use the mode insertion gradient (MIG)---a tool from hybrid control theory---as a filtering criterion that instantaneously assesses the impact of user actions on a dynamic system over a time window into the future. As a result, the filter is permissive to many chosen strategies, minimally engaging, and skill-sensitive---qualities desired when evaluating human actions. Through a human study with 28 healthy volunteers, we show that the criterion exhibits a low, but significant, negative correlation between skill level, as estimated from task-specific measures in unassisted trials, and the rate of controller intervention during assistance. Moreover, a MIG-based filter can be utilized to create a shared control scheme for training or assistance. In the human study, we observe a substantial training effect when using a MIG-based filter to perform cart-pendulum inversion, particularly when comparing improvement via the RMS error measure. Using simulation of a controlled spring-loaded inverted pendulum (SLIP) as a test case, we observe that the MIG criterion could be used for assistance to guarantee either task completion or safety of a joint human-robot system, while maintaining the system's flexibility with respect to user-chosen strategies.
△ Less
Submitted 6 June, 2018;
originally announced June 2018.
-
Smooth input preparation for quantum and quantum-inspired machine learning
Authors:
Zhikuan Zhao,
Jack K. Fitzsimons,
Patrick Rebentrost,
Vedran Dunjko,
Joseph F. Fitzsimons
Abstract:
Machine learning has recently emerged as a fruitful area for finding potential quantum computational advantage. Many of the quantum enhanced machine learning algorithms critically hinge upon the ability to efficiently produce states proportional to high-dimensional data points stored in a quantum accessible memory. Even given query access to exponentially many entries stored in a database, the con…
▽ More
Machine learning has recently emerged as a fruitful area for finding potential quantum computational advantage. Many of the quantum enhanced machine learning algorithms critically hinge upon the ability to efficiently produce states proportional to high-dimensional data points stored in a quantum accessible memory. Even given query access to exponentially many entries stored in a database, the construction of which is considered a one-off overhead, it has been argued that the cost of preparing such amplitude-encoded states may offset any exponential quantum advantage. Here we prove using smoothed analysis, that if the data-analysis algorithm is robust against small entry-wise input perturbation, state preparation can always be achieved with constant queries. This criterion is typically satisfied in realistic machine learning applications, where input data is subjective to moderate noise. Our results are equally applicable to the recent seminal progress in quantum-inspired algorithms, where specially constructed databases suffice for polylogarithmic classical algorithm in low-rank cases. The consequence of our finding is that for the purpose of practical machine learning, polylogarithmic processing time is possible under a general and flexible input model with quantum algorithms or quantum-inspired classical algorithms in the low-rank cases.
△ Less
Submitted 2 August, 2019; v1 submitted 1 April, 2018;
originally announced April 2018.
-
Quantum algorithms for training Gaussian Processes
Authors:
Zhikuan Zhao,
Jack K. Fitzsimons,
Michael A. Osborne,
Stephen J. Roberts,
Joseph F. Fitzsimons
Abstract:
Gaussian processes (GPs) are important models in supervised machine learning. Training in Gaussian processes refers to selecting the covariance functions and the associated parameters in order to improve the outcome of predictions, the core of which amounts to evaluating the logarithm of the marginal likelihood (LML) of a given model. LML gives a concrete measure of the quality of prediction that…
▽ More
Gaussian processes (GPs) are important models in supervised machine learning. Training in Gaussian processes refers to selecting the covariance functions and the associated parameters in order to improve the outcome of predictions, the core of which amounts to evaluating the logarithm of the marginal likelihood (LML) of a given model. LML gives a concrete measure of the quality of prediction that a GP model is expected to achieve. The classical computation of LML typically carries a polynomial time overhead with respect to the input size. We propose a quantum algorithm that computes the logarithm of the determinant of a Hermitian matrix, which runs in logarithmic time for sparse matrices. This is applied in conjunction with a variant of the quantum linear system algorithm that allows for logarithmic time computation of the form $\mathbf{y}^TA^{-1}\mathbf{y}$, where $\mathbf{y}$ is a dense vector and $A$ is the covariance matrix. We hence show that quantum computing can be used to estimate the LML of a GP with exponentially improved efficiency under certain conditions.
△ Less
Submitted 28 March, 2018;
originally announced March 2018.
-
Improved stochastic trace estimation using mutually unbiased bases
Authors:
J. K. Fitzsimons,
M. A. Osborne,
S. J. Roberts,
J. F. Fitzsimons
Abstract:
We examine the problem of estimating the trace of a matrix $A$ when given access to an oracle which computes $x^\dagger A x$ for an input vector $x$. We make use of the basis vectors from a set of mutually unbiased bases, widely studied in the field of quantum information processing, in the selection of probing vectors $x$. This approach offers a new state of the art single shot sampling variance…
▽ More
We examine the problem of estimating the trace of a matrix $A$ when given access to an oracle which computes $x^\dagger A x$ for an input vector $x$. We make use of the basis vectors from a set of mutually unbiased bases, widely studied in the field of quantum information processing, in the selection of probing vectors $x$. This approach offers a new state of the art single shot sampling variance while requiring only $O(\log(n))$ random bits to generate each vector. This significantly improves on traditional methods such as Hutchinson's and Gaussian estimators in terms of the number of random bits required and worst case sample variance.
△ Less
Submitted 30 July, 2016;
originally announced August 2016.
-
Quantum assisted Gaussian process regression
Authors:
Zhikuan Zhao,
Jack K. Fitzsimons,
Joseph F. Fitzsimons
Abstract:
Gaussian processes (GP) are a widely used model for regression problems in supervised machine learning. Implementation of GP regression typically requires $O(n^3)$ logic gates. We show that the quantum linear systems algorithm [Harrow et al., Phys. Rev. Lett. 103, 150502 (2009)] can be applied to Gaussian process regression (GPR), leading to an exponential reduction in computation time in some ins…
▽ More
Gaussian processes (GP) are a widely used model for regression problems in supervised machine learning. Implementation of GP regression typically requires $O(n^3)$ logic gates. We show that the quantum linear systems algorithm [Harrow et al., Phys. Rev. Lett. 103, 150502 (2009)] can be applied to Gaussian process regression (GPR), leading to an exponential reduction in computation time in some instances. We show that even in some cases not ideally suited to the quantum linear systems algorithm, a polynomial increase in efficiency still occurs.
△ Less
Submitted 12 December, 2015;
originally announced December 2015.
