Showing 1–14 of 14 results for author: Agboh, W

The Teenager's Problem: Efficient Garment Decluttering With Grasp Optimization

Authors: Aviv Adler, Ayah Ahmad, Shengyin Wang, Wisdom C. Agboh, Edith Llontop, Tianshuang Qiu, Jeffrey Ichnowski, Mehmet Dogar, Thomas Kollar, Richard Cheng, Ken Goldberg

Abstract: This paper addresses the ''Teenager's Problem'': efficiently removing scattered garments from a planar surface. As gras** and transporting individual garments is highly inefficient, we propose analytical policies to select grasp locations for multiple garments using an overhead camera. Two classes of methods are considered: depth-based, which use overhead depth data to find efficient grasps, and… ▽ More This paper addresses the ''Teenager's Problem'': efficiently removing scattered garments from a planar surface. As gras** and transporting individual garments is highly inefficient, we propose analytical policies to select grasp locations for multiple garments using an overhead camera. Two classes of methods are considered: depth-based, which use overhead depth data to find efficient grasps, and segment-based, which use segmentation on the RGB overhead image (without requiring any depth data); grasp efficiency is measured by Objects per Transport, which denotes the average number of objects removed per trip to the laundry basket. Experiments suggest that both depth- and segment-based methods easily reduce Objects per Transport (OpT) by $20\%$; furthermore, these approaches complement each other, with combined hybrid methods yielding improvements of $34\%$. Finally, a method employing consolidation (with segmentation) is considered, which manipulates the garments on the work surface to increase OpT; this yields an improvement of $67\%$ over the baseline, though at a cost of additional physical actions. △ Less

Submitted 25 October, 2023; originally announced October 2023.

The Busboy Problem: Efficient Tableware Decluttering Using Consolidation and Multi-Object Grasps

Authors: Kishore Srinivas, Shreya Ganti, Rishi Parikh, Ayah Ahmad, Wisdom Agboh, Mehmet Dogar, Ken Goldberg

Abstract: We present the "Busboy Problem": automating an efficient decluttering of cups, bowls, and silverware from a planar surface. As gras** and transporting individual items is highly inefficient, we propose policies to generate grasps for multiple items. We introduce the metric of Objects per Trip (OpT) carried by the robot to the collection bin to analyze the improvement seen as a result of our poli… ▽ More We present the "Busboy Problem": automating an efficient decluttering of cups, bowls, and silverware from a planar surface. As gras** and transporting individual items is highly inefficient, we propose policies to generate grasps for multiple items. We introduce the metric of Objects per Trip (OpT) carried by the robot to the collection bin to analyze the improvement seen as a result of our policies. In physical experiments with singulated items, we find that consolidation and multi-object grasps resulted in an 1.8x improvement in OpT, compared to methods without multi-object grasps. See https://sites.google.com/berkeley.edu/busboyproblem for code and supplemental materials. △ Less

Submitted 7 July, 2023; originally announced July 2023.

Push-MOG: Efficient Pushing to Consolidate Polygonal Objects for Multi-Object Gras**

Authors: Shrey Aeron, Edith LLontop, Aviv Adler, Wisdom C. Agboh, Mehmet R Dogar, Ken Goldberg

Abstract: Recently, robots have seen rapidly increasing use in homes and warehouses to declutter by collecting objects from a planar surface and placing them into a container. While current techniques grasp objects individually, Multi-Object Gras** (MOG) can improve efficiency by increasing the average number of objects grasped per trip (OpT). However, gras** multiple objects requires the objects to be… ▽ More Recently, robots have seen rapidly increasing use in homes and warehouses to declutter by collecting objects from a planar surface and placing them into a container. While current techniques grasp objects individually, Multi-Object Gras** (MOG) can improve efficiency by increasing the average number of objects grasped per trip (OpT). However, gras** multiple objects requires the objects to be aligned and in close proximity. In this work, we propose Push-MOG, an algorithm that computes "fork pushing" actions using a parallel-jaw gripper to create graspable object clusters. In physical decluttering experiments, we find that Push-MOG enables multi-object grasps, increasing the average OpT by 34%. Code and videos will be available at https://sites.google.com/berkeley.edu/push-mog. △ Less

Submitted 24 June, 2023; originally announced June 2023.

Comments: 6 pages, 4 figures, CASE 2023

Learning to Efficiently Plan Robust Frictional Multi-Object Grasps

Authors: Wisdom C. Agboh, Satvik Sharma, Kishore Srinivas, Mallika Parulekar, Gaurav Datta, Tianshuang Qiu, Jeffrey Ichnowski, Eugen Solowjow, Mehmet Dogar, Ken Goldberg

Abstract: We consider a decluttering problem where multiple rigid convex polygonal objects rest in randomly placed positions and orientations on a planar surface and must be efficiently transported to a packing box using both single and multi-object grasps. Prior work considered frictionless multi-object gras**. In this paper, we introduce friction to increase the number of potential grasps for a given gr… ▽ More We consider a decluttering problem where multiple rigid convex polygonal objects rest in randomly placed positions and orientations on a planar surface and must be efficiently transported to a packing box using both single and multi-object grasps. Prior work considered frictionless multi-object gras**. In this paper, we introduce friction to increase the number of potential grasps for a given group of objects, and thus increase picks per hour. We train a neural network using real examples to plan robust multi-object grasps. In physical experiments, we find a 13.7% increase in success rate, a 1.6x increase in picks per hour, and a 6.3x decrease in grasp planning time compared to prior work on multi-object gras**. Compared to single-object gras**, we find a 3.1x increase in picks per hour. △ Less

Submitted 2 August, 2023; v1 submitted 13 October, 2022; originally announced October 2022.

Comments: IEEE IROS 2023

Multi-Object Gras** in the Plane

Authors: Wisdom C. Agboh, Jeffrey Ichnowski, Ken Goldberg, Mehmet R. Dogar

Abstract: We consider a novel problem where multiple rigid convex polygonal objects rest in randomly placed positions and orientations on a planar surface visible from an overhead camera. The objective is to efficiently grasp and transport all objects into a bin using multi-object push-grasps, where multiple objects are pushed together to facilitate multi-object gras**. We provide necessary conditions for… ▽ More We consider a novel problem where multiple rigid convex polygonal objects rest in randomly placed positions and orientations on a planar surface visible from an overhead camera. The objective is to efficiently grasp and transport all objects into a bin using multi-object push-grasps, where multiple objects are pushed together to facilitate multi-object gras**. We provide necessary conditions for frictionless multi-object push-grasps and apply these to filter inadmissible grasps in a novel multi-object grasp planner. We find that our planner is 19 times faster than a Mujoco simulator baseline. We also propose a picking algorithm that uses both single- and multi-object grasps to pick objects. In physical gras** experiments comparing performance with a single-object picking baseline, we find that the frictionless multi-object gras** system achieves 13.6\% higher grasp success and is 59.9\% faster, from 212 PPH to 340 PPH. See \url{https://sites.google.com/view/multi-object-gras**} for videos and code. △ Less

Submitted 21 September, 2022; v1 submitted 1 June, 2022; originally announced June 2022.

Comments: Accepted to the International Symposium on Robotics Research (ISRR), 2022

Robust Physics-Based Manipulation by Interleaving Open and Closed-Loop Execution

Authors: Wisdom C. Agboh, Mehmet R. Dogar

Abstract: We present a planning and control framework for physics-based manipulation under uncertainty. The key idea is to interleave robust open-loop execution with closed-loop control. We derive robustness metrics through contraction theory. We use these metrics to plan trajectories that are robust to both state uncertainty and model inaccuracies. However, fully robust trajectories are extremely difficult… ▽ More We present a planning and control framework for physics-based manipulation under uncertainty. The key idea is to interleave robust open-loop execution with closed-loop control. We derive robustness metrics through contraction theory. We use these metrics to plan trajectories that are robust to both state uncertainty and model inaccuracies. However, fully robust trajectories are extremely difficult to find or may not exist for many multi-contact manipulation problems. We separate a trajectory into robust and non-robust segments through a minimum cost path search on a robustness graph. Robust segments are executed open-loop and non-robust segments are executed with model-predictive control. We conduct experiments on a real robotic system for reaching in clutter. Our results suggest that the open and closed-loop approach results in up to 35% more real-world success compared to open-loop baselines and a 40% reduction in execution time compared to model-predictive control. We show for the first time that partially open-loop manipulation plans generated with our approach reach similar success rates to model-predictive control, while achieving a more fluent/real-time execution. A video showing real-robot executions can be found at https://youtu.be/rPOPCwHfV4g. △ Less

Submitted 23 June, 2021; v1 submitted 18 May, 2021; originally announced May 2021.

Occlusion-Aware Search for Object Retrieval in Clutter

Authors: Wissam Bejjani, Wisdom C. Agboh, Mehmet R. Dogar, Matteo Leonetti

Abstract: We address the manipulation task of retrieving a target object from a cluttered shelf. When the target object is hidden, the robot must search through the clutter for retrieving it. Solving this task requires reasoning over the likely locations of the target object. It also requires physics reasoning over multi-object interactions and future occlusions. In this work, we present a data-driven hybri… ▽ More We address the manipulation task of retrieving a target object from a cluttered shelf. When the target object is hidden, the robot must search through the clutter for retrieving it. Solving this task requires reasoning over the likely locations of the target object. It also requires physics reasoning over multi-object interactions and future occlusions. In this work, we present a data-driven hybrid planner for generating occlusion-aware actions in closed-loop. The hybrid planner explores likely locations of the occluded target object as predicted by a learned distribution from the observation stream. The search is guided by a heuristic trained with reinforcement learning to act on observations with occlusions. We evaluate our approach in different simulation and real-world settings (video available on https://youtu.be/dY7YQ3LUVQg). The results validate that our approach can search and retrieve a target object in near real time in the real world while only being trained in simulation. △ Less

Submitted 31 August, 2021; v1 submitted 6 November, 2020; originally announced November 2020.

Human-like Planning for Reaching in Cluttered Environments

Authors: Mohamed Hasan, Matthew Warburton, Wisdom C. Agboh, Mehmet R. Dogar, Matteo Leonetti, He Wang, Faisal Mushtaq, Mark Mon-Williams, Anthony G. Cohn

Abstract: Humans, in comparison to robots, are remarkably adept at reaching for objects in cluttered environments. The best existing robot planners are based on random sampling of configuration space -- which becomes excessively high-dimensional with large number of objects. Consequently, most planners often fail to efficiently find object manipulation plans in such environments. We addressed this problem b… ▽ More Humans, in comparison to robots, are remarkably adept at reaching for objects in cluttered environments. The best existing robot planners are based on random sampling of configuration space -- which becomes excessively high-dimensional with large number of objects. Consequently, most planners often fail to efficiently find object manipulation plans in such environments. We addressed this problem by identifying high-level manipulation plans in humans, and transferring these skills to robot planners. We used virtual reality to capture human participants reaching for a target object on a tabletop cluttered with obstacles. From this, we devised a qualitative representation of the task space to abstract the decision making, irrespective of the number of obstacles. Based on this representation, human demonstrations were segmented and used to train decision classifiers. Using these classifiers, our planner produced a list of waypoints in task space. These waypoints provided a high-level plan, which could be transferred to an arbitrary robot model and used to initialise a local trajectory optimiser. We evaluated this approach through testing on unseen human VR data, a physics-based robot simulation, and a real robot (dataset and code are publicly available). We found that the human-like planner outperformed a state-of-the-art standard trajectory optimisation algorithm, and was able to generate effective strategies for rapid planning -- irrespective of the number of obstacles in the environment. △ Less

Submitted 3 March, 2020; v1 submitted 28 February, 2020; originally announced February 2020.

Comments: To be published in ICRA 2020

Parareal with a Learned Coarse Model for Robotic Manipulation

Authors: Wisdom Agboh, Oliver Grainger, Daniel Ruprecht, Mehmet Dogar

Abstract: A key component of many robotics model-based planning and control algorithms is physics predictions, that is, forecasting a sequence of states given an initial state and a sequence of controls. This process is slow and a major computational bottleneck for robotics planning algorithms. Parallel-in-time integration methods can help to leverage parallel computing to accelerate physics predictions and… ▽ More A key component of many robotics model-based planning and control algorithms is physics predictions, that is, forecasting a sequence of states given an initial state and a sequence of controls. This process is slow and a major computational bottleneck for robotics planning algorithms. Parallel-in-time integration methods can help to leverage parallel computing to accelerate physics predictions and thus planning. The Parareal algorithm iterates between a coarse serial integrator and a fine parallel integrator. A key challenge is to devise a coarse model that is computationally cheap but accurate enough for Parareal to converge quickly. Here, we investigate the use of a deep neural network physics model as a coarse model for Parareal in the context of robotic manipulation. In simulated experiments using the physics engine Mujoco as fine propagator we show that the learned coarse model leads to faster Parareal convergence than a coarse physics-based model. We further show that the learned coarse model allows to apply Parareal to scenarios with multiple objects, where the physics-based coarse model is not applicable. Finally, we conduct experiments on a real robot and show that Parareal predictions are close to real-world physics predictions for robotic pushing of multiple objects. Videos are at https://youtu.be/wCh2o1rf-gA. △ Less

Submitted 19 June, 2020; v1 submitted 12 December, 2019; originally announced December 2019.

Comments: Accepted to Computing and Visualization in Science (special issue on parallel-in-time)

Journal ref: Computing and Visualization in Science 23(8), 2020

Combining Coarse and Fine Physics for Manipulation using Parallel-in-Time Integration

Authors: Wisdom C. Agboh, Daniel Ruprecht, Mehmet R. Dogar

Abstract: We present a method for fast and accurate physics-based predictions during non-prehensile manipulation planning and control. Given an initial state and a sequence of controls, the problem of predicting the resulting sequence of states is a key component of a variety of model-based planning and control algorithms. We propose combining a coarse (i.e. computationally cheap but not very accurate) pred… ▽ More We present a method for fast and accurate physics-based predictions during non-prehensile manipulation planning and control. Given an initial state and a sequence of controls, the problem of predicting the resulting sequence of states is a key component of a variety of model-based planning and control algorithms. We propose combining a coarse (i.e. computationally cheap but not very accurate) predictive physics model, with a fine (i.e. computationally expensive but accurate) predictive physics model, to generate a hybrid model that is at the required speed and accuracy for a given manipulation task. Our approach is based on the Parareal algorithm, a parallel-in-time integration method used for computing numerical solutions for general systems of ordinary differential equations. We adapt Parareal to combine a coarse pushing model with an off-the-shelf physics engine to deliver physics-based predictions that are as accurate as the physics engine but run in substantially less wall-clock time, thanks to parallelization across time. We use these physics-based predictions in a model-predictive-control framework based on trajectory optimization, to plan pushing actions that avoid an obstacle and reach a goal location. We show that with hybrid physics models, we can achieve the same success rates as the planner that uses the off-the-shelf physics engine directly, but significantly faster. We present experiments in simulation and on a real robotic setup. Videos are available here: https://youtu.be/5e9oTeu4JOU △ Less

Submitted 30 August, 2019; v1 submitted 20 March, 2019; originally announced March 2019.

Comments: International Symposium on Robotics Research (ISRR), 2019

Journal ref: In: Asfour T., Yoshida E., Park J., Christensen H., Khatib O. (eds) Robotics Research. ISRR 2019. Springer Proceedings in Advanced Robotics, vol 20. Springer, Cham

O2A: One-shot Observational learning with Action vectors

Authors: Leo Pauly, Wisdom C. Agboh, David C. Hogg, Raul Fuentes

Abstract: We present O2A, a novel method for learning to perform robotic manipulation tasks from a single (one-shot) third-person demonstration video. To our knowledge, it is the first time this has been done for a single demonstration. The key novelty lies in pre-training a feature extractor for creating a perceptual representation for actions that we call 'action vectors'. The action vectors are extracted… ▽ More We present O2A, a novel method for learning to perform robotic manipulation tasks from a single (one-shot) third-person demonstration video. To our knowledge, it is the first time this has been done for a single demonstration. The key novelty lies in pre-training a feature extractor for creating a perceptual representation for actions that we call 'action vectors'. The action vectors are extracted using a 3D-CNN model pre-trained as an action classifier on a generic action dataset. The distance between the action vectors from the observed third-person demonstration and trial robot executions is used as a reward for reinforcement learning of the demonstrated task. We report on experiments in simulation and on a real robot, with changes in viewpoint of observation, properties of the objects involved, scene background and morphology of the manipulator between the demonstration and the learning domains. O2A outperforms baseline approaches under different domain shifts and has comparable performance with an oracle (that uses an ideal reward function). △ Less

Submitted 21 December, 2020; v1 submitted 17 October, 2018; originally announced October 2018.

Journal ref: Front. Robot. AI 8:686368 (2021)

Real-Time Online Re-Planning for Gras** Under Clutter and Uncertainty

Authors: Wisdom C. Agboh, Mehmet R. Dogar

Abstract: We consider the problem of gras** in clutter. While there have been motion planners developed to address this problem in recent years, these planners are mostly tailored for open-loop execution. Open-loop execution in this domain, however, is likely to fail, since it is not possible to model the dynamics of the multi-body multi-contact physical system with enough accuracy, neither is it reasonab… ▽ More We consider the problem of gras** in clutter. While there have been motion planners developed to address this problem in recent years, these planners are mostly tailored for open-loop execution. Open-loop execution in this domain, however, is likely to fail, since it is not possible to model the dynamics of the multi-body multi-contact physical system with enough accuracy, neither is it reasonable to expect robots to know the exact physical properties of objects, such as frictional, inertial, and geometrical. Therefore, we propose an online re-planning approach for gras** through clutter. The main challenge is the long planning times this domain requires, which makes fast re-planning and fluent execution difficult to realize. In order to address this, we propose an easily parallelizable stochastic trajectory optimization based algorithm that generates a sequence of optimal controls. We show that by running this optimizer only for a small number of iterations, it is possible to perform real time re-planning cycles to achieve reactive manipulation under clutter and uncertainty. △ Less

Submitted 9 October, 2018; v1 submitted 24 July, 2018; originally announced July 2018.

Comments: Published as a conference paper in IEEE Humanoids 2018

Pushing Fast and Slow: Task-Adaptive Planning for Non-prehensile Manipulation Under Uncertainty

Authors: Wisdom C. Agboh, Mehmet R. Dogar

Abstract: We propose a planning and control approach to physics-based manipulation. The key feature of the algorithm is that it can adapt to the accuracy requirements of a task, by slowing down and generating `careful' motion when the task requires high accuracy, and by speeding up and moving fast when the task tolerates inaccuracy. We formulate the problem as an MDP with action-dependent stochasticity and… ▽ More We propose a planning and control approach to physics-based manipulation. The key feature of the algorithm is that it can adapt to the accuracy requirements of a task, by slowing down and generating `careful' motion when the task requires high accuracy, and by speeding up and moving fast when the task tolerates inaccuracy. We formulate the problem as an MDP with action-dependent stochasticity and propose an approximate online solution to it. We use a trajectory optimizer with a deterministic model to suggest promising actions to the MDP, to reduce computation time spent on evaluating different actions. We conducted experiments in simulation and on a real robotic system. Our results show that with a task-adaptive planning and control approach, a robot can choose fast or slow actions depending on the task accuracy and uncertainty level. The robot makes these decisions online and is able to maintain high success rates while completing manipulation tasks as fast as possible. △ Less

Submitted 21 January, 2019; v1 submitted 8 May, 2018; originally announced May 2018.

Comments: Camera-ready manuscript for WAFR 2018

The Hbot : A Holonomic Spherical Haptic Interface Driven by Non-Holonomic Wheels

Authors: Wisdom C. Agboh

Abstract: We present the Hbot, a holonomic, singularity-free spherical robot designed for haptic simulations. The Hbot is made up of a caged sphere actuated by steered and driven non-holonomic wheels to produce continuous and unlimited spherical motions. We analyse the kinematic interface between a sphere and $n$ arbitrarily positioned, steered and driven non-holonomic wheels in the general case. We also pr… ▽ More We present the Hbot, a holonomic, singularity-free spherical robot designed for haptic simulations. The Hbot is made up of a caged sphere actuated by steered and driven non-holonomic wheels to produce continuous and unlimited spherical motions. We analyse the kinematic interface between a sphere and $n$ arbitrarily positioned, steered and driven non-holonomic wheels in the general case. We also present a detailed singularity analysis and show that workspace-boundary and workspace-interior singularities can both be avoided at the design stage. We implement a prototype using two steered and driven non-holonomic wheels and show experimental results for trajectory tracking and rendering of various rotational stiffness levels. △ Less

Submitted 5 October, 2018; v1 submitted 21 March, 2018; originally announced March 2018.

Comments: We need to conduct more experiments with the prototype