-
SAID-NeRF: Segmentation-AIDed NeRF for Depth Completion of Transparent Objects
Authors:
Avinash Ummadisingu,
Jongkeum Choi,
Koki Yamane,
Shimpei Masuda,
Naoki Fukaya,
Kuniyuki Takahashi
Abstract:
Acquiring accurate depth information of transparent objects using off-the-shelf RGB-D cameras is a well-known challenge in Computer Vision and Robotics. Depth estimation/completion methods are typically employed and trained on datasets with quality depth labels acquired from either simulation, additional sensors or specialized data collection setups and known 3d models. However, acquiring reliable…
▽ More
Acquiring accurate depth information of transparent objects using off-the-shelf RGB-D cameras is a well-known challenge in Computer Vision and Robotics. Depth estimation/completion methods are typically employed and trained on datasets with quality depth labels acquired from either simulation, additional sensors or specialized data collection setups and known 3d models. However, acquiring reliable depth information for datasets at scale is not straightforward, limiting training scalability and generalization. Neural Radiance Fields (NeRFs) are learning-free approaches and have demonstrated wide success in novel view synthesis and shape recovery. However, heuristics and controlled environments (lights, backgrounds, etc) are often required to accurately capture specular surfaces. In this paper, we propose using Visual Foundation Models (VFMs) for segmentation in a zero-shot, label-free way to guide the NeRF reconstruction process for these objects via the simultaneous reconstruction of semantic fields and extensions to increase robustness. Our proposed method Segmentation-AIDed NeRF (SAID-NeRF) shows significant performance on depth completion datasets for transparent objects and robotic gras**.
△ Less
Submitted 28 March, 2024;
originally announced March 2024.
-
Precise Well-plate Placing Utilizing Contact During Sliding with Tactile-based Pose Estimation for Laboratory Automation
Authors:
Sameer Pai,
Kuniyuki Takahashi,
Shimpei Masuda,
Naoki Fukaya,
Koki Yamane,
Avinash Ummadisingu
Abstract:
Micro well-plates are an apparatus commonly used in chemical and biological experiments that are a few centimeters thick and contain wells or divets. In this paper, we aim to solve the task of placing the well-plate onto a well-plate holder (referred to as holder). This task is challenging due to the holder's raised grooves being a few millimeters in height, with a clearance of less than 1 mm betw…
▽ More
Micro well-plates are an apparatus commonly used in chemical and biological experiments that are a few centimeters thick and contain wells or divets. In this paper, we aim to solve the task of placing the well-plate onto a well-plate holder (referred to as holder). This task is challenging due to the holder's raised grooves being a few millimeters in height, with a clearance of less than 1 mm between the well-plate and holder, thus requiring precise control during placing. Our placing task has the following challenges: 1) The holder's detected pose is uncertain; 2) the required accuracy is at the millimeter to sub-millimeter level due to the raised groove's shallow height and small clearance; 3) the holder is not fixed to a desk and is susceptible to movement from external forces. To address these challenges, we developed methods including a) using tactile sensors for accurate pose estimation of the grasped well-plate to handle issue (1); b) sliding the well-plate onto the target holder while maintaining contact with the holder's groove and estimating its orientation for accurate alignment. This allows for high precision control (addressing issue (2)) and prevents displacement of the holder during placement (addressing issue (3)). We demonstrate a high success rate for the well-plate placing task, even under noisy observation of the holder's pose.
△ Less
Submitted 31 March, 2024; v1 submitted 28 September, 2023;
originally announced September 2023.
-
Two-fingered Hand with Gear-type Synchronization Mechanism with Magnet for Improved Small and Offset Objects Gras**: F2 Hand
Authors:
Naoki Fukaya,
Avinash Ummadisingu,
Kuniyuki Takahashi,
Guilherme Maeda,
Shin-ichi Maeda
Abstract:
A problem that plagues robotic gras** is the misalignment of the object and gripper due to difficulties in precise localization, actuation, etc. Under-actuated robotic hands with compliant mechanisms are used to adapt and compensate for these inaccuracies. However, these mechanisms come at the cost of controllability and coordination. For instance, adaptive functions that let the fingers of a tw…
▽ More
A problem that plagues robotic gras** is the misalignment of the object and gripper due to difficulties in precise localization, actuation, etc. Under-actuated robotic hands with compliant mechanisms are used to adapt and compensate for these inaccuracies. However, these mechanisms come at the cost of controllability and coordination. For instance, adaptive functions that let the fingers of a two-fingered gripper adapt independently may affect the coordination necessary for gras** small objects. In this work, we develop a two-fingered robotic hand capable of gras** objects that are offset from the gripper's center, while still having the requisite coordination for gras** small objects via a novel gear-type synchronization mechanism with a magnet. This gear synchronization mechanism allows the adaptive finger's tips to be aligned enabling it to grasp objects as small as toothpicks and washers. The magnetic component allows this coordination to automatically turn off when needed, allowing for the gras** of objects that are offset/misaligned from the gripper. This equips the hand with the capability of gras** light, fragile objects (strawberries, creampuffs, etc) to heavy frying pan lids, all while maintaining their position and posture which is vital in numerous applications that require precise positioning or careful manipulation.
△ Less
Submitted 20 September, 2023; v1 submitted 15 September, 2023;
originally announced September 2023.
-
F3 Hand: A Versatile Robot Hand Inspired by Human Thumb and Index Fingers
Authors:
Naoki Fukaya,
Avinash Ummadisingu,
Guilherme Maeda,
Shin-ichi Maeda
Abstract:
It is challenging to grasp numerous objects with varying sizes and shapes with a single robot hand. To address this, we propose a new robot hand called the 'F3 hand' inspired by the complex movements of human index finger and thumb. The F3 hand attempts to realize complex human-like gras** movements by combining a parallel motion finger and a rotational motion finger with an adaptive function. I…
▽ More
It is challenging to grasp numerous objects with varying sizes and shapes with a single robot hand. To address this, we propose a new robot hand called the 'F3 hand' inspired by the complex movements of human index finger and thumb. The F3 hand attempts to realize complex human-like gras** movements by combining a parallel motion finger and a rotational motion finger with an adaptive function. In order to confirm the performance of our hand, we attached it to a mobile manipulator - the Toyota Human Support Robot (HSR) and conducted gras** experiments. In our results, we show that it is able to grasp all YCB objects (82 in total), including washers with outer diameters as small as 6.4mm. We also built a system for intuitive operation with a 3D mouse and grasp an additional 24 objects, including small toothpicks and paper clips and large pitchers and cracker boxes. The F3 hand is able to achieve a 98% success rate in gras** even under imprecise control and positional offsets. Furthermore, owing to the finger's adaptive function, we demonstrate characteristics of the F3 hand that facilitate the gras** of soft objects such as strawberries in a desirable posture.
△ Less
Submitted 16 June, 2022; v1 submitted 13 June, 2022;
originally announced June 2022.
-
F1 Hand: A Versatile Fixed-Finger Gripper for Delicate Teleoperation and Autonomous Gras**
Authors:
Guilherme Maeda,
Naoki Fukaya,
Shin-ichi Maeda
Abstract:
Teleoperation is often limited by the ability of an operator to react and predict the behavior of the robot as it interacts with the environment. For example, to grasp small objects on a table, the teleoperator needs to predict the position of the fingertips before the fingers are closed to avoid them hitting the table. For that reason, we developed the F1 hand, a single-motor gripper that facilit…
▽ More
Teleoperation is often limited by the ability of an operator to react and predict the behavior of the robot as it interacts with the environment. For example, to grasp small objects on a table, the teleoperator needs to predict the position of the fingertips before the fingers are closed to avoid them hitting the table. For that reason, we developed the F1 hand, a single-motor gripper that facilitates teleoperation with the use of a fixed finger. The hand is capable of gras** objects as thin as a paper clip, and as heavy and large as a cordless drill. The applicability of the hand can be expanded by replacing the fixed finger with different shapes. This flexibility makes the hand highly versatile while being easy and cheap to develop. However, due to the atypical asymmetric structure and actuation of the hand usual gras** strategies no longer apply. Thus, we propose a controller that approximates actuation symmetry by using the motion of the whole arm. The F1 hand and its controller are compared side-by-side with the original Toyota Human Support Robot (HSR) gripper in teleoperation using 22 objects from the YCB dataset in addition to small objects. The gras** time and peak contact forces could be decreased by 20% and 70%, respectively while increasing success rates by 5%. Using an off-the-shelf grasp pose estimator for autonomous gras**, the system achieved similar success rates to the original HSR gripper, at the order of 80%.
△ Less
Submitted 14 May, 2022;
originally announced May 2022.
-
Cluttered Food Gras** with Adaptive Fingers and Synthetic-Data Trained Object Detection
Authors:
Avinash Ummadisingu,
Kuniyuki Takahashi,
Naoki Fukaya
Abstract:
The food packaging industry handles an immense variety of food products with wide-ranging shapes and sizes, even within one kind of food. Menus are also diverse and change frequently, making automation of pick-and-place difficult. A popular approach to bin-picking is to first identify each piece of food in the tray by using an instance segmentation method. However, human annotations to train these…
▽ More
The food packaging industry handles an immense variety of food products with wide-ranging shapes and sizes, even within one kind of food. Menus are also diverse and change frequently, making automation of pick-and-place difficult. A popular approach to bin-picking is to first identify each piece of food in the tray by using an instance segmentation method. However, human annotations to train these methods are unreliable and error-prone since foods are packed close together with unclear boundaries and visual similarity making separation of pieces difficult. To address this problem, we propose a method that trains purely on synthetic data and successfully transfers to the real world using sim2real methods by creating datasets of filled food trays using high-quality 3d models of real pieces of food for the training instance segmentation models. Another concern is that foods are easily damaged during gras**. We address this by introducing two additional methods -- a novel adaptive finger mechanism to passively retract when a collision occurs, and a method to filter grasps that are likely to cause damage to neighbouring pieces of food during a grasp. We demonstrate the effectiveness of the proposed method on several kinds of real foods.
△ Less
Submitted 10 March, 2022;
originally announced March 2022.
-
Target-mass Gras** of Entangled Food using Pre-gras** & Post-gras**
Authors:
Kuniyuki Takahashi,
Naoki Fukaya,
Avinash Ummadisingu
Abstract:
Food packing industries typically use seasonal ingredients with immense variety that factory workers manually pack. For small pieces of food picked by volume or weight that tend to get entangled, stick or clump together, it is difficult to predict how intertwined they are from a visual examination, making it a challenge to grasp the requisite target mass accurately. Workers rely on a combination o…
▽ More
Food packing industries typically use seasonal ingredients with immense variety that factory workers manually pack. For small pieces of food picked by volume or weight that tend to get entangled, stick or clump together, it is difficult to predict how intertwined they are from a visual examination, making it a challenge to grasp the requisite target mass accurately. Workers rely on a combination of weighing scales and a sequence of complex maneuvers to separate out the food and reach the target mass. This makes automation of the process a non-trivial affair. In this study, we propose methods that combines 1) pre-gras** to reduce the degree of the entanglement, 2) post-gras** to adjust the grasped mass using a novel gripper mechanism to carefully discard excess food when the grasped amount is larger than the target mass, and 3) selecting the gras** point to grasp an amount likely to be reasonably higher than target gras** mass with confidence. We evaluate the methods on a variety of foods that entangle, stick and clump, each of which has a different size, shape, and material properties such as volumetric mass density. We show significant improvement in grasp accuracy of user-specified target masses using our proposed methods.
△ Less
Submitted 2 March, 2022; v1 submitted 3 January, 2022;
originally announced January 2022.
