-
R-LGP: A Reachability-guided Logic-geometric Programming Framework for Optimal Task and Motion Planning on Mobile Manipulators
Authors:
Kim Tien Ly,
Valeriy Semenov,
Mattia Risiglione,
Wolfgang Merkt,
Ioannis Havoutis
Abstract:
This paper presents an optimization-based solution to task and motion planning (TAMP) on mobile manipulators. Logic-geometric programming (LGP) has shown promising capabilities for optimally dealing with hybrid TAMP problems that involve abstract and geometric constraints. However, LGP does not scale well to high-dimensional systems (e.g. mobile manipulators) and can suffer from obstacle avoidance…
▽ More
This paper presents an optimization-based solution to task and motion planning (TAMP) on mobile manipulators. Logic-geometric programming (LGP) has shown promising capabilities for optimally dealing with hybrid TAMP problems that involve abstract and geometric constraints. However, LGP does not scale well to high-dimensional systems (e.g. mobile manipulators) and can suffer from obstacle avoidance issues due to local minima. In this work, we extend LGP with a sampling-based reachability graph to enable solving optimal TAMP on high-DoF mobile manipulators. The proposed reachability graph can incorporate environmental information (obstacles) to provide the planner with sufficient geometric constraints. This reachability-aware heuristic efficiently prunes infeasible sequences of actions in the continuous domain, hence, it reduces replanning by securing feasibility at the final full path trajectory optimization. Our framework proves to be time-efficient in computing optimal and collision-free solutions, while outperforming the current state of the art on metrics of success rate, planning time, path length and number of steps. We validate our framework on the physical Toyota HSR robot and report comparisons on a series of mobile manipulation tasks of increasing difficulty. Videos of the experiments are available at https://youtu.be/NEVVHEhQnOQ.
△ Less
Submitted 5 March, 2024; v1 submitted 4 October, 2023;
originally announced October 2023.
-
Article Classification with Graph Neural Networks and Multigraphs
Authors:
Khang Ly,
Yury Kashnitsky,
Savvas Chamezopoulos,
Valeria Krzhizhanovskaya
Abstract:
Classifying research output into context-specific label taxonomies is a challenging and relevant downstream task, given the volume of existing and newly published articles. We propose a method to enhance the performance of article classification by enriching simple Graph Neural Network (GNN) pipelines with multi-graph representations that simultaneously encode multiple signals of article relatedne…
▽ More
Classifying research output into context-specific label taxonomies is a challenging and relevant downstream task, given the volume of existing and newly published articles. We propose a method to enhance the performance of article classification by enriching simple Graph Neural Network (GNN) pipelines with multi-graph representations that simultaneously encode multiple signals of article relatedness, e.g. references, co-authorship, shared publication source, shared subject headings, as distinct edge types. Fully supervised transductive node classification experiments are conducted on the Open Graph Benchmark OGBN-arXiv dataset and the PubMed diabetes dataset, augmented with additional metadata from Microsoft Academic Graph and PubMed Central, respectively. The results demonstrate that multi-graphs consistently improve the performance of a variety of GNN models compared to the default graphs. When deployed with SOTA textual node embedding methods, the transformed multi-graphs enable simple and shallow 2-layer GNN pipelines to achieve results on par with more complex architectures.
△ Less
Submitted 28 May, 2024; v1 submitted 20 September, 2023;
originally announced September 2023.
-
Toward smart composites: small-scale, untethered prediction and control for soft sensor/actuator systems
Authors:
Sarah Aguasvivas Manzano,
Vani Sundaram,
Artemis Xu,
Khoi Ly,
Mark Rentschler,
Robert Shepherd,
Nikolaus Correll
Abstract:
We present formulation and open-source tools to achieve in-material model predictive control of sensor/actuator systems using learned forward kinematics and on-device computation. Microcontroller units (MCUs) that compute the prediction and control task while colocated with the sensors and actuators enable in-material untethered behaviors. In this approach, small parameter size neural network mode…
▽ More
We present formulation and open-source tools to achieve in-material model predictive control of sensor/actuator systems using learned forward kinematics and on-device computation. Microcontroller units (MCUs) that compute the prediction and control task while colocated with the sensors and actuators enable in-material untethered behaviors. In this approach, small parameter size neural network models learn forward kinematics offline. Our open-source compiler, nn4mc, generates code to offload these predictions onto MCUs. A Newton-Raphson solver then computes the control input in real time. We first benchmark this nonlinear control approach against a PID controller on a mass-spring-damper simulation. We then study experimental results on two experimental rigs with different sensing, actuation and computational hardware: a tendon-based platform with embedded LightLace sensors and a HASEL-based platform with magnetic sensors. Experimental results indicate effective high-bandwidth tracking of reference paths (greater than or equal to 120 Hz) with a small memory footprint (less than or equal to 6.4% of flash memory). The measured path following error does not exceed 2mm in the tendon-based platform. The simulated path following error does not exceed 1mm in the HASEL-based platform. The mean power consumption of this approach in an ARM Cortex-M4f device is 45.4 mW. This control approach is also compatible with Tensorflow Lite models and equivalent on-device code. In-material intelligence enables a new class of composites that infuse autonomy into structures and systems with refined artificial proprioception.
△ Less
Submitted 22 August, 2022; v1 submitted 22 May, 2022;
originally announced May 2022.
-
Augmented Reality Appendages for Robots: Design Considerations and Recommendations for Maximizing Social and Functional Perception
Authors:
Ipek Goktan,
Karen Ly,
Thomas R. Groechel,
Maja J. Mataric
Abstract:
In order to address the limitations of gestural capabilities in physical robots, researchers in Virtual, Augmented, Mixed Reality Human-Robot Interaction (VAM-HRI) have been using augmented-reality visualizations that increase robot expressivity and improve user perception (e.g., social presence). While a multitude of virtual robot deictic gestures (e.g., pointing to an object) have been implement…
▽ More
In order to address the limitations of gestural capabilities in physical robots, researchers in Virtual, Augmented, Mixed Reality Human-Robot Interaction (VAM-HRI) have been using augmented-reality visualizations that increase robot expressivity and improve user perception (e.g., social presence). While a multitude of virtual robot deictic gestures (e.g., pointing to an object) have been implemented to improve interactions within VAM-HRI, such systems are often reported to have tradeoffs between functional and social user perceptions of robots, creating a need for a unified approach that considers both attributes. We performed a literature analysis that selected factors that were noted to significantly influence either user perception or task efficiency and propose a set of design considerations and recommendations that address those factors by combining anthropomorphic and non-anthropomorphic virtual gestures based on the motivation of the interaction, visibility of the target and robot, salience of the target, and distance between the target and robot. The proposed recommendations provide the VAM-HRI community with starting points for selecting appropriate gesture types for a multitude of interaction contexts.
△ Less
Submitted 13 May, 2022;
originally announced May 2022.
-
DINs: Deep Interactive Networks for Neurofibroma Segmentation in Neurofibromatosis Type 1 on Whole-Body MRI
Authors:
Jian-Wei Zhang,
Wei Chen,
K. Ina Ly,
Xubin Zhang,
Fan Yan,
Justin Jordan,
Gordon Harris,
Scott Plotkin,
Pengyi Hao,
Wenli Cai
Abstract:
Neurofibromatosis type 1 (NF1) is an autosomal dominant tumor predisposition syndrome that involves the central and peripheral nervous systems. Accurate detection and segmentation of neurofibromas are essential for assessing tumor burden and longitudinal tumor size changes. Automatic convolutional neural networks (CNNs) are sensitive and vulnerable as tumors' variable anatomical location and heter…
▽ More
Neurofibromatosis type 1 (NF1) is an autosomal dominant tumor predisposition syndrome that involves the central and peripheral nervous systems. Accurate detection and segmentation of neurofibromas are essential for assessing tumor burden and longitudinal tumor size changes. Automatic convolutional neural networks (CNNs) are sensitive and vulnerable as tumors' variable anatomical location and heterogeneous appearance on MRI. In this study, we propose deep interactive networks (DINs) to address the above limitations. User interactions guide the model to recognize complicated tumors and quickly adapt to heterogeneous tumors. We introduce a simple but effective Exponential Distance Transform (ExpDT) that converts user interactions into guide maps regarded as the spatial and appearance prior. Comparing with popular Euclidean and geodesic distances, ExpDT is more robust to various image sizes, which reserves the distribution of interactive inputs. Furthermore, to enhance the tumor-related features, we design a deep interactive module to propagate the guides into deeper layers. We train and evaluate DINs on three MRI data sets from NF1 patients. The experiment results yield significant improvements of 44% and 14% in DSC comparing with automated and other interactive methods, respectively. We also experimentally demonstrate the efficiency of DINs in reducing user burden when comparing with conventional interactive methods. The source code of our method is available at \url{https://github.com/Jarvis73/DINs}.
△ Less
Submitted 7 June, 2021;
originally announced June 2021.
-
High-bandwidth nonlinear control for soft actuators with recursive network models
Authors:
Sarah Aguasvivas Manzano,
Patricia Xu,
Khoi Ly,
Robert Shepherd,
Nikolaus Correll
Abstract:
We present a high-bandwidth, lightweight, and nonlinear output tracking technique for soft actuators that combines parsimonious recursive layers for forward output predictions and online optimization using Newton-Raphson. This technique allows for reduced model sizes and increased control loop frequencies when compared with conventional RNN models. Experimental results of this controller prototype…
▽ More
We present a high-bandwidth, lightweight, and nonlinear output tracking technique for soft actuators that combines parsimonious recursive layers for forward output predictions and online optimization using Newton-Raphson. This technique allows for reduced model sizes and increased control loop frequencies when compared with conventional RNN models. Experimental results of this controller prototype on a single soft actuator with soft positional sensors indicate effective tracking of referenced spatial trajectories and rejection of mechanical and electromagnetic disturbances. These are evidenced by root mean squared path tracking errors (RMSE) of 1.8mm using a fully connected (FC) substructure, 1.62mm using a gated recurrent unit (GRU) and 2.11mm using a long short term memory (LSTM) unit, all averaged over three tasks. Among these models, the highest flash memory requirement is 2.22kB enabling co-location of controller and actuator.
△ Less
Submitted 4 January, 2021;
originally announced January 2021.
-
Miniaturized Circuitry for Capacitive Self-sensing and Closed-loop Control of Soft Electrostatic Transducers
Authors:
Khoi Ly,
Nicholas Kellaris,
Dade McMorris,
Brian K. Johnson,
Eric Acome,
Vani Sundaram,
Mantas Naris,
J. Sean Humbert,
Mark E. Rentschler,
Christoph Keplinger,
Nikolaus Correll
Abstract:
Soft robotics is a field of robotic system design characterized by materials and structures that exhibit large-scale deformation, high compliance, and rich multifunctionality. The incorporation of soft and deformable structures endows soft robotic systems with the compliance and resiliency that makes them well-adapted for unstructured and dynamic environments. While actuation mechanisms for soft r…
▽ More
Soft robotics is a field of robotic system design characterized by materials and structures that exhibit large-scale deformation, high compliance, and rich multifunctionality. The incorporation of soft and deformable structures endows soft robotic systems with the compliance and resiliency that makes them well-adapted for unstructured and dynamic environments. While actuation mechanisms for soft robots vary widely, soft electrostatic transducers such as dielectric elastomer actuators (DEAs) and hydraulically amplified self-healing electrostatic (HASEL) actuators have demonstrated promise due to their muscle-like performance and capacitive self-sensing capabilities. Despite previous efforts to implement self-sensing in electrostatic transducers by overlaying sinusoidal low-voltage signals, these designs still require sensing high-voltage signals, requiring bulky components that prevent integration with miniature, untethered soft robots. We present a circuit design that eliminates the need for any high-voltage sensing components, thereby facilitating the design of simple, low cost circuits using off-the-shelf components. Using this circuit, we perform simultaneous sensing and actuation for a range of electrostatic transducers including circular DEAs and HASEL actuators and demonstrate accurate estimated displacements with errors under 4%. We further develop this circuit into a compact and portable system that couples HV actuation, sensing, and computation as a prototype towards untethered, multifunctional soft robotic systems. Finally, we demonstrate the capabilities of our self-sensing design through feedback-control of a robotic arm powered by Peano-HASEL actuators.
△ Less
Submitted 14 September, 2020;
originally announced September 2020.
-
Built Infrastructure Monitoring and Inspection Using UAVs and Vision-based Algorithms
Authors:
Khai Ky Ly,
Manh Duong Phung
Abstract:
This study presents an inspecting system using real-time control unmanned aerial vehicles (UAVs) to investigate structural surfaces. The system operates under favourable weather conditions to inspect a target structure, which is the Wentworth light rail base structure in this study. The system includes a drone, a GoPro HERO4 camera, a controller and a mobile phone. The drone takes off the ground m…
▽ More
This study presents an inspecting system using real-time control unmanned aerial vehicles (UAVs) to investigate structural surfaces. The system operates under favourable weather conditions to inspect a target structure, which is the Wentworth light rail base structure in this study. The system includes a drone, a GoPro HERO4 camera, a controller and a mobile phone. The drone takes off the ground manually in the testing field to collect the data requiring for later analysis. The images are taken through HERO 4 camera and then transferred in real time to the remote processing unit such as a ground control station by the wireless connection established by a Wi-Fi router. An image processing method has been proposed to detect defects or damages such as cracks. The method based on intensity histogram algorithms to exploit the pixel group related to the crack contained in the low intensity interval. Experiments, simulation and comparisons have been conducted to evaluate the performance and validity of the proposed system.
△ Less
Submitted 19 May, 2020;
originally announced May 2020.
-
Automated verification of termination certificates
Authors:
Frédéric Blanqui,
Kim Quyen Ly
Abstract:
In order to increase user confidence, many automated theorem provers provide certificates that can be independently verified. In this paper, we report on our progress in develo** a standalone tool for checking the correctness of certificates for the termination of term rewrite systems, and formally proving its correctness in the proof assistant Coq. To this end, we use the extraction mechanism o…
▽ More
In order to increase user confidence, many automated theorem provers provide certificates that can be independently verified. In this paper, we report on our progress in develo** a standalone tool for checking the correctness of certificates for the termination of term rewrite systems, and formally proving its correctness in the proof assistant Coq. To this end, we use the extraction mechanism of Coq and the library on rewriting theory and termination called CoLoR.
△ Less
Submitted 11 December, 2012;
originally announced December 2012.
-
Product Review Summarization based on Facet Identification and Sentence Clustering
Authors:
Duy Khang Ly,
Kazunari Sugiyama,
Ziheng Lin,
Min-Yen Kan
Abstract:
Product review nowadays has become an important source of information, not only for customers to find opinions about products easily and share their reviews with peers, but also for product manufacturers to get feedback on their products. As the number of product reviews grows, it becomes difficult for users to search and utilize these resources in an efficient way. In this work, we build a produc…
▽ More
Product review nowadays has become an important source of information, not only for customers to find opinions about products easily and share their reviews with peers, but also for product manufacturers to get feedback on their products. As the number of product reviews grows, it becomes difficult for users to search and utilize these resources in an efficient way. In this work, we build a product review summarization system that can automatically process a large collection of reviews and aggregate them to generate a concise summary. More importantly, the drawback of existing product summarization systems is that they cannot provide the underlying reasons to justify users' opinions. In our method, we solve this problem by applying clustering, prior to selecting representative candidates for summarization.
△ Less
Submitted 7 October, 2011;
originally announced October 2011.
