-
ADL4D: Towards A Contextually Rich Dataset for 4D Activities of Daily Living
Authors:
Marsil Zakour,
Partha Pratim Nath,
Ludwig Lohmer,
Emre Faik Gökçe,
Martin Piccolrovazzi,
Constantin Patsch,
Yuankai Wu,
Rahul Chaudhari,
Eckehard Steinbach
Abstract:
Hand-Object Interactions (HOIs) are conditioned on spatial and temporal contexts like surrounding objects, previous actions, and future intents (for example, gras** and handover actions vary greatly based on objects proximity and trajectory obstruction). However, existing datasets for 4D HOI (3D HOI over time) are limited to one subject interacting with one object only. This restricts the genera…
▽ More
Hand-Object Interactions (HOIs) are conditioned on spatial and temporal contexts like surrounding objects, previous actions, and future intents (for example, gras** and handover actions vary greatly based on objects proximity and trajectory obstruction). However, existing datasets for 4D HOI (3D HOI over time) are limited to one subject interacting with one object only. This restricts the generalization of learning-based HOI methods trained on those datasets. We introduce ADL4D, a dataset of up to two subjects interacting with different sets of objects performing Activities of Daily Living (ADL) like breakfast or lunch preparation activities. The transition between multiple objects to complete a certain task over time introduces a unique context lacking in existing datasets. Our dataset consists of 75 sequences with a total of 1.1M RGB-D frames, hand and object poses, and per-hand fine-grained action annotations. We develop an automatic system for multi-view multi-hand 3D pose annotation capable of tracking hand poses over time. We integrate and test it against publicly available datasets. Finally, we evaluate our dataset on the tasks of Hand Mesh Recovery (HMR) and Hand Action Segmentation (HAS).
△ Less
Submitted 27 February, 2024;
originally announced February 2024.
-
Care3D: An Active 3D Object Detection Dataset of Real Robotic-Care Environments
Authors:
Michael G. Adam,
Sebastian Eger,
Martin Piccolrovazzi,
Maged Iskandar,
Joern Vogel,
Alexander Dietrich,
Seongjien Bien,
Jon Skerlj,
Abdeldjallil Naceri,
Eckehard Steinbach,
Alin Albu-Schaeffer,
Sami Haddadin,
Wolfram Burgard
Abstract:
As labor shortage increases in the health sector, the demand for assistive robotics grows. However, the needed test data to develop those robots is scarce, especially for the application of active 3D object detection, where no real data exists at all. This short paper counters this by introducing such an annotated dataset of real environments. The captured environments represent areas which are al…
▽ More
As labor shortage increases in the health sector, the demand for assistive robotics grows. However, the needed test data to develop those robots is scarce, especially for the application of active 3D object detection, where no real data exists at all. This short paper counters this by introducing such an annotated dataset of real environments. The captured environments represent areas which are already in use in the field of robotic health care research. We further provide ground truth data within one room, for assessing SLAM algorithms running directly on a health care robot.
△ Less
Submitted 9 October, 2023;
originally announced October 2023.
-
Bounding Box Disparity: 3D Metrics for Object Detection With Full Degree of Freedom
Authors:
Michael G. Adam,
Martin Piccolrovazzi,
Sebastian Eger,
Eckehard Steinbach
Abstract:
The most popular evaluation metric for object detection in 2D images is Intersection over Union (IoU). Existing implementations of the IoU metric for 3D object detection usually neglect one or more degrees of freedom. In this paper, we first derive the analytic solution for three dimensional bounding boxes. As a second contribution, a closed-form solution of the volume-to-volume distance is derive…
▽ More
The most popular evaluation metric for object detection in 2D images is Intersection over Union (IoU). Existing implementations of the IoU metric for 3D object detection usually neglect one or more degrees of freedom. In this paper, we first derive the analytic solution for three dimensional bounding boxes. As a second contribution, a closed-form solution of the volume-to-volume distance is derived. Finally, the Bounding Box Disparity is proposed as a combined positive continuous metric. We provide open source implementations of the three metrics as standalone python functions, as well as extensions to the Open3D library and as ROS nodes.
△ Less
Submitted 11 November, 2022; v1 submitted 8 July, 2022;
originally announced July 2022.
