-
Bundling and Tumbling in Bacterial-inspired Bi-flagellated Soft Robots for Attitude Adjustment
Authors:
Zhuonan Hao,
Siddharth Zalavadia,
Mohammad Khalid Jawed
Abstract:
We create a mechanism inspired by bacterial swimmers, featuring two flexible flagella with individual control over rotation speed and direction in viscous fluid environments. Using readily available materials, we design and fabricate silicone-based helical flagella. To simulate the robot's motion, we develop a physics-based computational tool, drawing inspiration from computer graphics. The framew…
▽ More
We create a mechanism inspired by bacterial swimmers, featuring two flexible flagella with individual control over rotation speed and direction in viscous fluid environments. Using readily available materials, we design and fabricate silicone-based helical flagella. To simulate the robot's motion, we develop a physics-based computational tool, drawing inspiration from computer graphics. The framework incorporates the Discrete Elastic Rod method, modeling the flagella as Kirchhoff's elastic rods, and couples it with the Regularized Stokeslet Segments method for hydrodynamics, along with the Implicit Contact Model to handle contact. This approach effectively captures polymorphic phenomena like bundling and tumbling. Our study reveals how these emergent behaviors affect the robot's attitude angles, demonstrating its ability to self-reorient in both simulations and experiments. We anticipate that this framework will enhance our understanding of the directional change capabilities of flagellated robots, potentially stimulating further exploration on microscopic robot mobility.
△ Less
Submitted 19 January, 2024;
originally announced May 2024.
-
DisMech: A Discrete Differential Geometry-based Physical Simulator for Soft Robots and Structures
Authors:
Andrew Choi,
Ran **g,
Andrew Sabelhaus,
Mohammad Khalid Jawed
Abstract:
Fast, accurate, and generalizable simulations are a key enabler of modern advances in robot design and control. However, existing simulation frameworks in robotics either model rigid environments and mechanisms only, or if they include flexible or soft structures, suffer significantly in one or more of these performance areas. To close this "sim2real" gap, we introduce DisMech, a simulation enviro…
▽ More
Fast, accurate, and generalizable simulations are a key enabler of modern advances in robot design and control. However, existing simulation frameworks in robotics either model rigid environments and mechanisms only, or if they include flexible or soft structures, suffer significantly in one or more of these performance areas. To close this "sim2real" gap, we introduce DisMech, a simulation environment that models highly dynamic motions of rod-like soft continuum robots and structures, quickly and accurately, with arbitrary connections between them. Our methodology combines a fully implicit discrete differential geometry-based physics solver with fast and accurate contact handling, all in an intuitive software interface. Crucially, we propose a gradient descent approach to easily map the motions of hardware robot prototypes to control inputs in DisMech. We validate DisMech through several highly-nuanced soft robot simulations while demonstrating an order of magnitude speed increase over previous state of the art. Our real2sim validation shows high physical accuracy versus hardware, even with complicated soft actuation mechanisms such as shape memory alloy wires. With its low computational cost, physical accuracy, and ease of use, DisMech can accelerate translation of sim-based control for both soft robotics and deformable object manipulation.
△ Less
Submitted 19 February, 2024; v1 submitted 29 November, 2023;
originally announced November 2023.
-
Modeling, Characterization, and Control of Bacteria-inspired Bi-flagellated Mechanism with Tumbling
Authors:
Zhuonan Hao,
Sangmin Lim,
M. Khalid Jawed
Abstract:
Multi-flagellated bacteria utilize the hydrodynamic interaction between their filamentary tails, known as flagella, to swim and change their swimming direction in low Reynolds number flow. This interaction, referred to as bundling and tumbling, is often overlooked in simplified hydrodynamic models such as Resistive Force Theories (RFT). However, for the development of efficient and steerable robot…
▽ More
Multi-flagellated bacteria utilize the hydrodynamic interaction between their filamentary tails, known as flagella, to swim and change their swimming direction in low Reynolds number flow. This interaction, referred to as bundling and tumbling, is often overlooked in simplified hydrodynamic models such as Resistive Force Theories (RFT). However, for the development of efficient and steerable robots inspired by bacteria, it becomes crucial to exploit this interaction. In this paper, we present the construction of a macroscopic bio-inspired robot featuring two rigid flagella arranged as right-handed helices, along with a cylindrical head. By rotating the flagella in opposite directions, the robot's body can reorient itself through repeatable and controllable tumbling. To accurately model this bi-flagellated mechanism in low Reynolds flow, we employ a coupling of rigid body dynamics and the method of Regularized Stokeslet Segments (RSS). Unlike RFT, RSS takes into account the hydrodynamic interaction between distant filamentary structures. Furthermore, we delve into the exploration of the parameter space to optimize the propulsion and torque of the system. To achieve the desired reorientation of the robot, we propose a tumble control scheme that involves modulating the rotation direction and speed of the two flagella. By implementing this scheme, the robot can effectively reorient itself to attain the desired attitude. Notably, the overall scheme boasts a simplified design and control as it only requires two control inputs. With our macroscopic framework serving as a foundation, we envision the eventual miniaturization of this technology to construct mobile and controllable micro-scale bacterial robots.
△ Less
Submitted 30 June, 2023;
originally announced July 2023.
-
Agronav: Autonomous Navigation Framework for Agricultural Robots and Vehicles using Semantic Segmentation and Semantic Line Detection
Authors:
Shivam K Panda,
Yongkyu Lee,
M. Khalid Jawed
Abstract:
The successful implementation of vision-based navigation in agricultural fields hinges upon two critical components: 1) the accurate identification of key components within the scene, and 2) the identification of lanes through the detection of boundary lines that separate the crops from the traversable ground. We propose Agronav, an end-to-end vision-based autonomous navigation framework, which ou…
▽ More
The successful implementation of vision-based navigation in agricultural fields hinges upon two critical components: 1) the accurate identification of key components within the scene, and 2) the identification of lanes through the detection of boundary lines that separate the crops from the traversable ground. We propose Agronav, an end-to-end vision-based autonomous navigation framework, which outputs the centerline from the input image by sequentially processing it through semantic segmentation and semantic line detection models. We also present Agroscapes, a pixel-level annotated dataset collected across six different crops, captured from varying heights and angles. This ensures that the framework trained on Agroscapes is generalizable across both ground and aerial robotic platforms. Codes, models and dataset will be released at \href{https://github.com/shivamkumarpanda/agronav}{github.com/shivamkumarpanda/agronav}.
△ Less
Submitted 9 April, 2023;
originally announced April 2023.
-
Rapid design of fully soft deployable structures via kirigami cuts and active learning
Authors:
Leixin Ma,
Mrunmayi Mungekar,
Vwani Roychowdhury,
M. Khalid Jawed
Abstract:
Soft deployable structures - unlike conventional piecewise rigid deployables based on hinges and springs - can assume intricate 3-D shapes, thereby enabling transformative technologies in soft robotics, shape-morphing architecture, and pop-up manufacturing. Their virtually infinite degrees of freedom allow precise control over the final shape. The same enabling high dimensionality, however, poses…
▽ More
Soft deployable structures - unlike conventional piecewise rigid deployables based on hinges and springs - can assume intricate 3-D shapes, thereby enabling transformative technologies in soft robotics, shape-morphing architecture, and pop-up manufacturing. Their virtually infinite degrees of freedom allow precise control over the final shape. The same enabling high dimensionality, however, poses a challenge for solving the inverse design problem involving this class of structures: to achieve desired 3D structures it typically requires manufacturing technologies with extensive local actuation and control during fabrication, and a trial and error search over a large design space. We address both of these shortcomings by first develo** a simplified planar fabrication approach that combines two ingredients: strain mismatch between two layers of a composite shell and kirigami cuts that relieves localized stress. In principle, it is possible to generate targeted 3-D shapes by designing the appropriate kirigami cuts and selecting the right amount of prestretch, thereby eliminating the need for local control. Second, we formulate a data-driven physics-guided framework that reduces the dimensionality of the inverse design problem using autoencoders and efficiently searches through the ``latent" parameter space in an active learning approach. We demonstrate the effectiveness of the rapid design procedure via a range of target shapes, such as peanuts, pringles, flowers, and pyramids. Tabletop experiments are conducted to fabricate the target shapes. Experimental results and numerical predictions from our framework are found to be in good agreement.
△ Less
Submitted 4 March, 2023;
originally announced March 2023.
-
Sim2Real Neural Controllers for Physics-based Robotic Deployment of Deformable Linear Objects
Authors:
Dezhong Tong,
Andrew Choi,
Longhui Qin,
Weicheng Huang,
Jungseock Joo,
M. Khalid Jawed
Abstract:
Deformable linear objects (DLOs), such as rods, cables, and ropes, play important roles in daily life. However, manipulation of DLOs is challenging as large geometrically nonlinear deformations may occur during the manipulation process. This problem is made even more difficult as the different deformation modes (e.g., stretching, bending, and twisting) may result in elastic instabilities during ma…
▽ More
Deformable linear objects (DLOs), such as rods, cables, and ropes, play important roles in daily life. However, manipulation of DLOs is challenging as large geometrically nonlinear deformations may occur during the manipulation process. This problem is made even more difficult as the different deformation modes (e.g., stretching, bending, and twisting) may result in elastic instabilities during manipulation. In this paper, we formulate a physics-guided data-driven method to solve a challenging manipulation task -- accurately deploying a DLO (an elastic rod) onto a rigid substrate along various prescribed patterns. Our framework combines machine learning, scaling analysis, and physical simulations to develop a physics-based neural controller for deployment. We explore the complex interplay between the gravitational and elastic energies of the manipulated DLO and obtain a control method for DLO deployment that is robust against friction and material properties. Out of the numerous geometrical and material properties of the rod and substrate, we show that only three non-dimensional parameters are needed to describe the deployment process with physical analysis. Therefore, the essence of the controlling law for the manipulation task can be constructed with a low-dimensional model, drastically increasing the computation speed. The effectiveness of our optimal control scheme is shown through a comprehensive robotic case study comparing against a heuristic control method for deploying rods for a wide variety of patterns. In addition to this, we also showcase the practicality of our control scheme by having a robot accomplish challenging high-level tasks such as mimicking human handwriting, cable placement, and tying knots.
△ Less
Submitted 10 December, 2023; v1 submitted 4 March, 2023;
originally announced March 2023.
-
mBEST: Realtime Deformable Linear Object Detection Through Minimal Bending Energy Skeleton Pixel Traversals
Authors:
Andrew Choi,
Dezhong Tong,
Brian Park,
Demetri Terzopoulos,
Jungseock Joo,
Mohammad Khalid Jawed
Abstract:
Robotic manipulation of deformable materials is a challenging task that often requires realtime visual feedback. This is especially true for deformable linear objects (DLOs) or "rods", whose slender and flexible structures make proper tracking and detection nontrivial. To address this challenge, we present mBEST, a robust algorithm for the realtime detection of DLOs that is capable of producing an…
▽ More
Robotic manipulation of deformable materials is a challenging task that often requires realtime visual feedback. This is especially true for deformable linear objects (DLOs) or "rods", whose slender and flexible structures make proper tracking and detection nontrivial. To address this challenge, we present mBEST, a robust algorithm for the realtime detection of DLOs that is capable of producing an ordered pixel sequence of each DLO's centerline along with segmentation masks. Our algorithm obtains a binary mask of the DLOs and then thins it to produce a skeleton pixel representation. After refining the skeleton to ensure topological correctness, the pixels are traversed to generate paths along each unique DLO. At the core of our algorithm, we postulate that intersections can be robustly handled by choosing the combination of paths that minimizes the cumulative bending energy of the DLO(s). We show that this simple and intuitive formulation outperforms the state-of-the-art methods for detecting DLOs with large numbers of sporadic crossings ranging from curvatures with high variance to nearly-parallel configurations. Furthermore, our method achieves a significant performance improvement of approximately 50% faster runtime and better scaling over the state of the art.
△ Less
Submitted 19 February, 2024; v1 submitted 18 February, 2023;
originally announced February 2023.
-
Design of Bistable Soft Deployable Structures via a Kirigami-inspired Planar Fabrication Approach
Authors:
Mrunmayi Mungekar,
Leixin Ma,
Wenzhong Yan,
Vishal Kackar,
Shyan Shokrzadeh,
M. Khalid Jawed
Abstract:
Fully soft bistable mechanisms have shown extensive applications ranging from soft robotics, wearable devices, and medical tools, to energy harvesting. However, the lack of design and fabrication methods that are easy and potentially scalable limits their further adoption into mainstream applications. Here a top-down planar approach is presented by introducing Kirigami-inspired engineering combine…
▽ More
Fully soft bistable mechanisms have shown extensive applications ranging from soft robotics, wearable devices, and medical tools, to energy harvesting. However, the lack of design and fabrication methods that are easy and potentially scalable limits their further adoption into mainstream applications. Here a top-down planar approach is presented by introducing Kirigami-inspired engineering combined with a pre-stretching process. Using this method, Kirigami-Pre-stretched Substrate-Kirigami trilayered precursors are created in a planar manner; upon release, the strain mismatch -- due to the pre-stretching of substrate -- between layers would induce an out-of-plane buckling to achieve targeted three dimensional (3D) bistable structures. By combining experimental characterization, analytical modeling, and finite element simulation, the effect of the pattern size of Kirigami layers and pre-stretching on the geometry and stability of resulting 3D composites is explored. In addition, methods to realize soft bistable structures with arbitrary shapes and soft composites with multistable configurations are investigated, which could encourage further applications. Our method is demonstrated by using bistable soft Kirigami composites to construct two soft machines: (i) a bistable soft gripper that can gently grasp delicate objects with different shapes and sizes and (ii) a flytrap-inspired robot that can autonomously detect and capture objects.
△ Less
Submitted 14 February, 2023; v1 submitted 22 January, 2023;
originally announced January 2023.
-
Soft kirigami composites for form-finding of fully flexible deployables
Authors:
Jan Zavodnik,
Yunbo Wang,
Wenzhong Yan,
Miha Brojan,
M. Khalid Jawed
Abstract:
We introduce a new class of thin flexible structures that morph from a flat shape into prescribed 3D shapes without an external stimulus such as mechanical loads or heat. To achieve control over the target shape, two different concepts are coupled. First, motivated by biological growth, strain mismatch is applied between the flat composite layers to transform it into a 3D shape. Depending on the a…
▽ More
We introduce a new class of thin flexible structures that morph from a flat shape into prescribed 3D shapes without an external stimulus such as mechanical loads or heat. To achieve control over the target shape, two different concepts are coupled. First, motivated by biological growth, strain mismatch is applied between the flat composite layers to transform it into a 3D shape. Depending on the amount of the applied strain mismatch, the transformation involves buckling into one of the available finite number of mode shapes. Second, inspired by kirigami, portions of the material are removed from one of the layers according to a specific pattern. This dramatically increases the number of possible 3D shapes and allows us to attain specific topologies. We devise an experimental apparatus that allows precise control of the strain mismatch. An inverse problem is posed, where starting from a given target shape, the physical parameters that make these shapes possible are determined. To show how the concept works, we focus on circular composite plates and design a kirigami pattern that yields a hemispherical structure. Our analysis combines a theoretical approach with numerical simulations and physical experiments to understand and predict the transition from 2D to 3D shapes. The tools developed here can be extended to attain arbitrary 3D shapes. The initially flat shape suggests that conventional additive manufacturing techniques can be used to functionalize the soft kirigami composites to fabricate, for example, deployable 3D shapes, smart skins, and soft electromagnetic metasurfaces.
△ Less
Submitted 19 January, 2023; v1 submitted 16 January, 2023;
originally announced January 2023.
-
Learning Neural Force Manifolds for Sim2Real Robotic Symmetrical Paper Folding
Authors:
Andrew Choi,
Dezhong Tong,
Demetri Terzopoulos,
Jungseock Joo,
M. Khalid Jawed
Abstract:
Robotic manipulation of slender objects is challenging, especially when the induced deformations are large and nonlinear. Traditionally, learning-based control approaches, such as imitation learning, have been used to address deformable material manipulation. These approaches lack generality and often suffer critical failure from a simple switch of material, geometric, and/or environmental (e.g.,…
▽ More
Robotic manipulation of slender objects is challenging, especially when the induced deformations are large and nonlinear. Traditionally, learning-based control approaches, such as imitation learning, have been used to address deformable material manipulation. These approaches lack generality and often suffer critical failure from a simple switch of material, geometric, and/or environmental (e.g., friction) properties. This article tackles a fundamental but difficult deformable manipulation task: forming a predefined fold in paper with only a single manipulator. A sim2real framework combining physically-accurate simulation and machine learning is used to train a deep neural network capable of predicting the external forces induced on the manipulated paper given a grasp position. We frame the problem using scaling analysis, resulting in a control framework robust against material and geometric changes. Path planning is then carried out over the generated ``neural force manifold'' to produce robot manipulation trajectories optimized to prevent sliding, with offline trajectory generation finishing 15$\times$ faster than previous physics-based folding methods. The inference speed of the trained model enables the incorporation of real-time visual feedback to achieve closed-loop model-predictive control. Real-world experiments demonstrate that our framework can greatly improve robotic manipulation performance compared to state-of-the-art folding strategies, even when manipulating paper objects of various materials and shapes.
△ Less
Submitted 19 February, 2024; v1 submitted 5 January, 2023;
originally announced January 2023.
-
Metalearning generalizable dynamics from trajectories
Authors:
Qiaofeng Li,
Tianyi Wang,
Vwani Roychowdhury,
M. Khalid Jawed
Abstract:
We present the interpretable meta neural ordinary differential equation (iMODE) method to rapidly learn generalizable (i.e., not parameter-specific) dynamics from trajectories of multiple dynamical systems that vary in their physical parameters. The iMODE method learns meta-knowledge, the functional variations of the force field of dynamical system instances without knowing the physical parameters…
▽ More
We present the interpretable meta neural ordinary differential equation (iMODE) method to rapidly learn generalizable (i.e., not parameter-specific) dynamics from trajectories of multiple dynamical systems that vary in their physical parameters. The iMODE method learns meta-knowledge, the functional variations of the force field of dynamical system instances without knowing the physical parameters, by adopting a bi-level optimization framework: an outer level capturing the common force field form among studied dynamical system instances and an inner level adapting to individual system instances. A priori physical knowledge can be conveniently embedded in the neural network architecture as inductive bias, such as conservative force field and Euclidean symmetry. With the learned meta-knowledge, iMODE can model an unseen system within seconds, and inversely reveal knowledge on the physical parameters of a system, or as a Neural Gauge to "measure" the physical parameters of an unseen system with observed trajectories. We test the validity of the iMODE method on bistable, double pendulum, Van der Pol, Slinky, and reaction-diffusion systems.
△ Less
Submitted 27 September, 2023; v1 submitted 3 January, 2023;
originally announced January 2023.
-
Machine Learning Assisted Resistive Force Theory for Helical Structures at Low Reynolds Number
Authors:
Sangmin Lim,
Charbel Habchi,
Mohammad Khalid Jawed
Abstract:
The hydrodynamic forces on a slender rod in a fluid medium at low Reynolds number can be modeled using resistive force theories (RFTs) or slender body theories (SBTs). The former represent the forces by local drag coefficients and are computationally cheap; however, they are physically inaccurate when long-range hydrodynamic interaction is involved. The later are physically accurate but require so…
▽ More
The hydrodynamic forces on a slender rod in a fluid medium at low Reynolds number can be modeled using resistive force theories (RFTs) or slender body theories (SBTs). The former represent the forces by local drag coefficients and are computationally cheap; however, they are physically inaccurate when long-range hydrodynamic interaction is involved. The later are physically accurate but require solving integral equations and, therefore, are computationally expensive. This paper investigates RFTs in comparison with state-of-the art SBT methods. During the process, a neural network-based hydrodynamic model that -- similar to RFTs -- relies on local drag coefficients for computational efficiency was developed. However, the network is trained using data from an SBT (regularized stokeslet segments method). The $R^2$ value of the trained coefficients were $\sim 0.99$ with mean absolute error of $1.6\times10^{-2}$. The machine learning resistive force theory (MLRFT) accounts for local hydrodynamic forces distribution, the dependence on rotational and translational speeds and directions, and geometric parameters of the slender object. We show that, when classical RFT fails to accurately predict the forces, torques, and drags on slender rods under low Reynolds number flows, MLRFT exhibits good agreement with physically accurate SBT simulations. In terms of computational speed, MLRFT forgoes the need of solving an inverse problem and, therefore, requires negligible computation time in comparison with SBT. MLRFT presents a computationally inexpensive hydrodynamic model for flagellar propulsion can be used in the design and optimization of biomimetic flagellated robots and analysis of bacterial locomotion.
△ Less
Submitted 13 September, 2023; v1 submitted 19 July, 2022;
originally announced July 2022.
-
A Fully Implicit Method for Robust Frictional Contact Handling in Elastic Rods
Authors:
Dezhong Tong,
Andrew Choi,
Jungseock Joo,
M. Khalid Jawed
Abstract:
Accurate frictional contact is critical in simulating the assembly of rod-like structures in the practical world, such as knots, hairs, flagella, and more. Due to their high geometric nonlinearity and elasticity, rod-on-rod contact remains a challenging problem tackled by researchers in both computational mechanics and computer graphics. Typically, frictional contact is regarded as constraints for…
▽ More
Accurate frictional contact is critical in simulating the assembly of rod-like structures in the practical world, such as knots, hairs, flagella, and more. Due to their high geometric nonlinearity and elasticity, rod-on-rod contact remains a challenging problem tackled by researchers in both computational mechanics and computer graphics. Typically, frictional contact is regarded as constraints for the equations of motions of a system. Such constraints are often computed independently at every time step in a dynamic simulation, thus slowing down the simulation and possibly introducing numerical convergence issues. This paper proposes a fully implicit penalty-based frictional contact method, Implicit Contact Model (IMC), that efficiently and robustly captures accurate frictional contact responses. We showcase our algorithm's performance in achieving visually realistic results for the challenging and novel contact scenario of flagella bundling in fluid medium, a significant phenomenon in biology that motivates novel engineering applications in soft robotics. In addition to this, we offer a side-by-side comparison with Incremental Potential Contact (IPC), a state-of-the-art contact handling algorithm. We show that IMC possesses comparable performance to IPC while converging at a faster rate.
△ Less
Submitted 19 February, 2024; v1 submitted 20 May, 2022;
originally announced May 2022.
-
Rapidly Encoding Generalizable Dynamics in a Euclidean Symmetric Neural Network
Authors:
Qiaofeng Li,
Tianyi Wang,
Vwani Roychowdhury,
M. Khalid Jawed
Abstract:
Slinky, a helical elastic rod, is a seemingly simple structure with unusual mechanical behavior; for example, it can walk down a flight of stairs under its own weight. Taking Slinky as a test-case, we propose a physics-informed deep learning approach for building reduced-order models of physical systems. The approach introduces a Euclidean symmetric neural network (ESNN) architecture that is train…
▽ More
Slinky, a helical elastic rod, is a seemingly simple structure with unusual mechanical behavior; for example, it can walk down a flight of stairs under its own weight. Taking Slinky as a test-case, we propose a physics-informed deep learning approach for building reduced-order models of physical systems. The approach introduces a Euclidean symmetric neural network (ESNN) architecture that is trained under the neural ordinary differential equation framework to learn the 2D latent dynamics from the motion trajectory of a reduced-order representation of the 3D Slinky. The ESNN implements a physics-guided architecture that simultaneously preserves energy invariance and force equivariance under Euclidean transformations of the input, including translation, rotation, and reflection. The embedded Euclidean symmetry provides physics-guided interpretability and generalizability, while preserving the full expressive power of the neural network. We demonstrate that the ESNN approach is able to accelerate simulation by one to two orders of magnitude compared to traditional numerical methods and achieve a superior generalization performance while classic neural networks fail to learn the Slinky dynamics, i.e., the ESNN, trained on a single demonstration case, predicts the motions accurately for unseen cases of different Slinky configurations and boundary conditions. Further investigation into the ESNN reveals that it explicitly learns the nonlinear coupling between stretching and bending of the Slinky.
△ Less
Submitted 26 January, 2023; v1 submitted 22 March, 2022;
originally announced March 2022.
-
Preemptive Motion Planning for Human-to-Robot Indirect Placement Handovers
Authors:
Andrew Choi,
Mohammad Khalid Jawed,
Jungseock Joo
Abstract:
As technology advances, the need for safe, efficient, and collaborative human-robot-teams has become increasingly important. One of the most fundamental collaborative tasks in any setting is the object handover. Human-to-robot handovers can take either of two approaches: (1) direct hand-to-hand or (2) indirect hand-to-placement-to-pick-up. The latter approach ensures minimal contact between the hu…
▽ More
As technology advances, the need for safe, efficient, and collaborative human-robot-teams has become increasingly important. One of the most fundamental collaborative tasks in any setting is the object handover. Human-to-robot handovers can take either of two approaches: (1) direct hand-to-hand or (2) indirect hand-to-placement-to-pick-up. The latter approach ensures minimal contact between the human and robot but can also result in increased idle time due to having to wait for the object to first be placed down on a surface. To minimize such idle time, the robot must preemptively predict the human intent of where the object will be placed. Furthermore, for the robot to preemptively act in any sort of productive manner, predictions and motion planning must occur in real-time. We introduce a novel prediction-planning pipeline that allows the robot to preemptively move towards the human agent's intended placement location using gaze and gestures as model inputs. In this paper, we investigate the performance and drawbacks of our early intent predictor-planner as well as the practical benefits of using such a pipeline through a human-robot case study.
△ Less
Submitted 19 February, 2024; v1 submitted 28 February, 2022;
originally announced March 2022.
-
Deep-CNN based Robotic Multi-Class Under-Canopy Weed Control in Precision Farming
Authors:
Yayun Du,
Guofeng Zhang,
Darren Tsang,
M. Khalid Jawed
Abstract:
Smart weeding systems to perform plant-specific operations can contribute to the sustainability of agriculture and the environment. Despite monumental advances in autonomous robotic technologies for precision weed management in recent years, work on under-canopy weeding in fields is yet to be realized. A prerequisite of such systems is reliable detection and classification of weeds to avoid mistak…
▽ More
Smart weeding systems to perform plant-specific operations can contribute to the sustainability of agriculture and the environment. Despite monumental advances in autonomous robotic technologies for precision weed management in recent years, work on under-canopy weeding in fields is yet to be realized. A prerequisite of such systems is reliable detection and classification of weeds to avoid mistakenly spraying and, thus, damaging the surrounding plants. Real-time multi-class weed identification enables species-specific treatment of weeds and significantly reduces the amount of herbicide use. Here, our first contribution is the first adequately large realistic image dataset \textit{AIWeeds} (one/multiple kinds of weeds in one image), a library of about 10,000 annotated images of flax, and the 14 most common weeds in fields and gardens taken from 20 different locations in North Dakota, California, and Central China. Second, we provide a full pipeline from model training with maximum efficiency to deploying the TensorRT-optimized model onto a single board computer. Based on \textit{AIWeeds} and the pipeline, we present a baseline for classification performance using five benchmark CNN models. Among them, MobileNetV2, with both the shortest inference time and lowest memory consumption, is the qualified candidate for real-time applications. Finally, we deploy MobileNetV2 onto our own compact autonomous robot \textit{SAMBot} for real-time weed detection. The 90\% test accuracy realized in previously unseen scenes in flax fields (with a row spacing of 0.2-0.3 m), with crops and weeds, distortion, blur, and shadows, is a milestone towards precision weed control in the real world. We have publicly released the dataset and code to generate the results at \url{https://github.com/StructuresComp/Multi-class-Weed-Classification}.
△ Less
Submitted 27 December, 2021;
originally announced December 2021.
-
Mechanics-based Analysis on Flagellated Robots
Authors:
Yayun Du,
Andrew Miller,
M. Khalid Jawed
Abstract:
We explore the locomotion of soft robots in granular medium (GM) resulting from the elastic deformation of slender rods. A low-cost, rapidly fabricable robot inspired by the physiological structure of bacteria is presented. It consists of a rigid head, with a motor and batteries embedded, and multiple elastic rods (our model for flagella) to investigate locomotion in GM. The elastic flagella are r…
▽ More
We explore the locomotion of soft robots in granular medium (GM) resulting from the elastic deformation of slender rods. A low-cost, rapidly fabricable robot inspired by the physiological structure of bacteria is presented. It consists of a rigid head, with a motor and batteries embedded, and multiple elastic rods (our model for flagella) to investigate locomotion in GM. The elastic flagella are rotated at one end by the motor, and they deform due to the drag from GM, propelling the robot. The external drag is determined by the flagellar shape, while the latter changes due to the competition between external loading and elastic forces. In this coupled fluid-structure interaction problem, we observe that increasing the number of flagella can decrease or increase the propulsive speed of the robot, depending on the physical parameters of the system. This nonlinearity in the functional relation between propulsion and the parameters of this simple robot motivates us to fundamentally analyze its mechanics using theory, numerical simulation, and experiments. We present a simple Euler-Bernoulli beam theory-based analytical framework that is capable of qualitatively capturing both cases. Theoretical prediction quantitatively matches experiments when the flagellar deformation is small. To account for the geometrically nonlinear deformation often encountered in soft robots and microbes, we implement a simulation framework that incorporates discrete differential geometry-based simulations of elastic rods, a resistive force theory-based model for drag, and a modified Stokes law for the hydrodynamics of the robot head. Comparison with experimental data indicates that the simulations can quantitatively predict robotic motion. Overall, the theoretical and numerical tools presented in this paper can shed light on the design and control of this class of articulated robots in granular or fluid media.
△ Less
Submitted 27 December, 2021;
originally announced December 2021.
-
Ballooning in Spiders using Multiple Silk Threads
Authors:
Charbel Habchi,
Mohammad Khalid Jawed
Abstract:
In this paper, three-dimensional numerical simulations of ballooning in spiders using multiple silk threads are performed using the discrete elastic rods method. The ballooning of spiders is hypothesised to be caused by the presence of the negative electric charge of the spider silk threads and the positive electric potential field in the earth's atmosphere. The numerical model presented here is f…
▽ More
In this paper, three-dimensional numerical simulations of ballooning in spiders using multiple silk threads are performed using the discrete elastic rods method. The ballooning of spiders is hypothesised to be caused by the presence of the negative electric charge of the spider silk threads and the positive electric potential field in the earth's atmosphere. The numerical model presented here is first validated against experimental data from the open literature. After which, two cases are examined, in the first it is assumed that the electric charge is uniformly distributed along the threads while in the second, the electric charge is located at the thread tip. It is shown that the normalized terminal ballooning velocity, i.e. the velocity at which the spiders balloon after they reach steady-state, decrease linearly with the normalized lift force, especially for the tip located charge case. For the uniform electric charge case, this velocity shows a slightly weaker dependence on the normalized lift force. Moreover, it is shown in both cases that the normalized terminal ballooning velocity has no dependence on the normalized elastic bending stiffness of the threads and on the normalized viscous forces. Finally, the multi-thread bending process shows a three-dimensional conical sheet. Here we show that this behavior is caused by the Coulomb repelling forces owing to the threads electric charge which leads to dispersing the threads apart and thus avoid entanglement.
△ Less
Submitted 20 December, 2021;
originally announced December 2021.
-
Automated stability testing of elastic rods with helical centerlines using a robotic system
Authors:
Dezhong Tong,
Andy Borum,
M. Khalid Jawed
Abstract:
Experimental analysis of the mechanics of a deformable object, and particularly its stability, requires repetitive testing and, depending on the complexity of the object's shape, a testing setup that can manipulate many degrees of freedom at the object's boundary. Motivated by recent advancements in robotic manipulation of deformable objects, this paper addresses these challenges by constructing a…
▽ More
Experimental analysis of the mechanics of a deformable object, and particularly its stability, requires repetitive testing and, depending on the complexity of the object's shape, a testing setup that can manipulate many degrees of freedom at the object's boundary. Motivated by recent advancements in robotic manipulation of deformable objects, this paper addresses these challenges by constructing a method for automated stability testing of a slender elastic rod -- a canonical example of a deformable object -- using a robotic system. We focus on rod configurations with helical centerlines since the stability of a helical rod can be described using only three parameters, but experimentally determining the stability requires manipulation of both the position and orientation at one end of the rod, which is not possible using traditional experimental methods that only actuate a limited number of degrees of freedom. Using a recent geometric characterization of stability for helical rods, we construct and implement a manipulation scheme to explore the space of stable helices, and we use a vision system to detect the onset of instabilities within this space. The experimental results obtained by our automated testing system show good agreement with numerical simulations of elastic rods in helical configurations. The methods described in this paper lay the groundwork for automation to grow within the field of experimental mechanics.
△ Less
Submitted 16 December, 2021;
originally announced December 2021.
-
A Low-cost Robot with Autonomous Recharge and Navigation for Weed Control in Fields with Narrow Row Spacing
Authors:
Yayun Du,
Bhrugu Mallajosyula,
Deming Sun,
**gyi Chen,
Zihang Zhao,
Mukhlesur Rahman,
Mohiuddin Quadir,
Mohammad Khalid Jawed
Abstract:
Modern herbicide application in agricultural settings typically relies on either large scale sprayers that dispense herbicide over crops and weeds alike or portable sprayers that require labor intensive manual operation. The former method results in overuse of herbicide and reduction in crop yield while the latter is often untenable in large scale operations. This paper presents the first fully au…
▽ More
Modern herbicide application in agricultural settings typically relies on either large scale sprayers that dispense herbicide over crops and weeds alike or portable sprayers that require labor intensive manual operation. The former method results in overuse of herbicide and reduction in crop yield while the latter is often untenable in large scale operations. This paper presents the first fully autonomous robot for weed management for row crops capable of computer vision based navigation, weed detection, complete field coverage, and automatic recharge for under \$400. The target application is autonomous inter-row weed control in crop fields, e.g. flax and canola, where the spacing between croplines is as small as one foot. The proposed robot is small enough to pass between croplines at all stages of plant growth while detecting weeds and spraying herbicide. A recharging system incorporates newly designed robotic hardware, a ramp, a robotic charging arm, and a mobile charging station. An integrated vision algorithm is employed to assist with charger alignment effectively. Combined, they enable the robot to work continuously in the field without access to electricity. In addition, a color-based contour algorithm combined with preprocessing techniques is applied for robust navigation relying on the input from the onboard monocular camera. Incorporating such compact robots into farms could help automate weed control, even during late stages of growth, and reduce herbicide use by targeting weeds with precision. The robotic platform is field-tested in the flaxseed fields of North Dakota.
△ Less
Submitted 3 December, 2021;
originally announced December 2021.
-
Bacteria-inspired robotic propulsion from bundling of soft helical filaments at low Reynolds number
Authors:
Sangmin Lim,
Achyuta Yadunandan,
Mohammad Khalid Jawed
Abstract:
The bundling of flagella is known to create a "run" phase, where the bacteria moves in a nearly straight line rather than making changes in direction. Historically, mechanical explanations for the bundling phenomenon intrigued many researchers, and significant advances were made in physical models and experimental methods. Contributing to the field of research, we present a bacteria-inspired centi…
▽ More
The bundling of flagella is known to create a "run" phase, where the bacteria moves in a nearly straight line rather than making changes in direction. Historically, mechanical explanations for the bundling phenomenon intrigued many researchers, and significant advances were made in physical models and experimental methods. Contributing to the field of research, we present a bacteria-inspired centimeter-scale soft robotic hardware platform and a computational framework for a physically plausible simulation model of the multi-flagellated robot under low Reynolds number (~0.1). The fluid-structure interaction simulation couples the Discrete Elastic Rods algorithm with the method of Regularized Stokeslet Segments. Contact between two flagella is handled by a penalty-based method. We present a comparison between our experimental and simulation results and verify that the simulation tool can capture the essential physics of this problem. Preliminary findings on robustness to buckling provided by the bundling phenomenon and the efficiency of a multi-flagellated soft robot are compared with the single-flagellated counterparts. Observations were made on the coupling between geometry and elasticity, which manifests itself in the propulsion of the robot by nonlinear dependency on the rotational speed of the flagella.
△ Less
Submitted 16 October, 2023; v1 submitted 24 November, 2021;
originally announced November 2021.
-
Simple Flagellated Soft Robot for Locomotion near Air-Fluid Interface
Authors:
Yayun Du,
Andrew Miller,
Mohammad Khalid Jawed
Abstract:
A wide range of microorganisms, e.g. bacteria, propel themselves by rotation of soft helical tails, also known as flagella. Due to the small size of these organisms, viscous forces overwhelm inertial effects and the flow is at low Reynolds number. In this fluid-structure problem, a competition between elastic forces and hydrodynamic (viscous) forces leads to a net propulsive force forward. A thoro…
▽ More
A wide range of microorganisms, e.g. bacteria, propel themselves by rotation of soft helical tails, also known as flagella. Due to the small size of these organisms, viscous forces overwhelm inertial effects and the flow is at low Reynolds number. In this fluid-structure problem, a competition between elastic forces and hydrodynamic (viscous) forces leads to a net propulsive force forward. A thorough understanding of this highly coupled fluid-structure interaction problem can not only help us better understand biological propulsion but also help us design bio-inspired functional robots with applications in oil spill cleanup, water quality monitoring, and infrastructure inspection. Here, we introduce arguably the simplest soft robot with a single binary control signal, which is capable of moving along an arbitrary 2D trajectory near air-fluid interface and at the interface between two fluids. The robot exploits the variation in viscosity to move along the prescribed trajectory. Our analysis of this newly introduced soft robot consists of three main components. First, we fabricate this simple robot and use it as an experimental testbed. Second, a discrete differential geometry-based modeling framework is used for simulation of the robot. Upon validation of the simulation tool, the third part of this study employs the simulations to develop a control scheme with a single binary input to make the robot follow any prescribed path.
△ Less
Submitted 9 March, 2021;
originally announced March 2021.
-
Modeling the locomotion of articulated soft robots in granular medium
Authors:
Yayun Du,
Jacqueline Lam,
Karunesh Sachanandani,
Mohammad Khalid Jawed
Abstract:
Soft robots, in contrast to their rigid counter parts, have infinite degrees of freedom that are coupled with their interaction with the environment. We consider the locomotion of an untethered robot, in the granular medium, comprised of multiple flexible flagella that rotate about an axis by a motor. Drag from the grains causes the flagella to deform and the deformed shape generates a net forward…
▽ More
Soft robots, in contrast to their rigid counter parts, have infinite degrees of freedom that are coupled with their interaction with the environment. We consider the locomotion of an untethered robot, in the granular medium, comprised of multiple flexible flagella that rotate about an axis by a motor. Drag from the grains causes the flagella to deform and the deformed shape generates a net forward propulsion. This external drag force depends on the shape of the flagella, while the change in flagellar shape is the result of the competition between the external loading and elastic forces. We introduce a numerical tool that couples discrete differential geometry based simulation of elastic rods - our model for flagella - and a resistive force theory based model for the drag. In parallel with simulations, we conduct experiments to quantify the propulsive speed of this class of robots. We find reasonable quantitative agreement between experiments and simulations. Owing to a rod-based kinematic representation of the robot, the simulation runs faster than real-time, and, therefore, we can use it as a design tool for this class of soft robots. We find that there is an optimal rotational speed at which maximum efficiency is achieved. Moreover, both experiments and simulations show that increasing the number of flagella decreases the speed of the robot. We also gain insight into the mechanics of granular medium - while resistive force theory can successfully describe the propulsion at low number of flagella, it fails when more flagella are added to the robot.
△ Less
Submitted 5 March, 2021;
originally announced March 2021.
-
Control of uniflagellar soft robots at low Reynolds number using buckling instability
Authors:
Mojtaba Forghani,
Weicheng Huang,
M. Khalid Jawed
Abstract:
In this paper, we analyze the inverse dynamics and control of a bacteria-inspired uniflagellar robot in a fluid medium at low Reynolds number. Inspired by the mechanism behind the locomotion of flagellated bacteria, we consider a robot comprised of a flagellum -- a flexible helical filament -- attached to a spherical head. The flagellum rotates about the head at a controlled angular velocity and g…
▽ More
In this paper, we analyze the inverse dynamics and control of a bacteria-inspired uniflagellar robot in a fluid medium at low Reynolds number. Inspired by the mechanism behind the locomotion of flagellated bacteria, we consider a robot comprised of a flagellum -- a flexible helical filament -- attached to a spherical head. The flagellum rotates about the head at a controlled angular velocity and generates a propulsive force that moves the robot forward. When the angular velocity exceeds a threshold value, the hydrodynamic force exerted by the fluid can cause the soft flagellum to buckle, characterized by a dramatic change in shape. In this computational study, a fluid-structure interaction model that combines Discrete Elastic Rods (DER) algorithm with Lighthill's Slender Body Theory (LSBT) is employed to simulate the locomotion and deformation of the robot. We demonstrate that the robot can follow a prescribed path in three dimensional space by exploiting buckling of the flagellum. The control scheme involves only a single (binary) scalar input -- the angular velocity of the flagellum. By triggering the buckling instability at the right moment, the robot can follow an arbitrary path in three dimensional space. We also show that the complexity of the dynamics of the helical filament can be captured using a deep neural network, from which we identify the input-output functional relationship between the control inputs and the trajectory of the robot. Furthermore, our study underscores the potential role of buckling in the locomotion of natural bacteria.
△ Less
Submitted 18 December, 2020; v1 submitted 7 October, 2018;
originally announced October 2018.
-
Propulsion and instability of a flexible helical rod rotating in a viscous fluid
Authors:
M. K. Jawed,
N. K. Khouri,
F. Da,
E. Grinspun,
P. M. Reis
Abstract:
We combine experiments with simulations to investigate the fluid-structure interaction of a flexible helical rod rotating in a viscous fluid, under low Reynolds number conditions. Our analysis takes into account the coupling between the geometrically nonlinear behavior of the elastic rod with a non-local hydrodynamic model for the fluid loading. We quantify the resulting propulsive force, as well…
▽ More
We combine experiments with simulations to investigate the fluid-structure interaction of a flexible helical rod rotating in a viscous fluid, under low Reynolds number conditions. Our analysis takes into account the coupling between the geometrically nonlinear behavior of the elastic rod with a non-local hydrodynamic model for the fluid loading. We quantify the resulting propulsive force, as well as the buckling instability of the originally helical filament that occurs above a critical rotation velocity. A scaling analysis is performed to rationalize the onset of this instability. A universal phase diagram is constructed to map out the region of successful propulsion and the corresponding boundary of stability are established. Comparing our results with data for flagellated bacteria suggests that this instability may be exploited in nature for physiological purposes.
△ Less
Submitted 18 September, 2015; v1 submitted 17 September, 2015;
originally announced September 2015.
