-
Pragmatic auditing: a pilot-driven approach for auditing Machine Learning systems
Authors:
Djalel Benbouzid,
Christiane Plociennik,
Laura Lucaj,
Mihai Maftei,
Iris Merget,
Aljoscha Burchardt,
Marc P. Hauer,
Abdeldjallil Naceri,
Patrick van der Smagt
Abstract:
The growing adoption and deployment of Machine Learning (ML) systems came with its share of ethical incidents and societal concerns. It also unveiled the necessity to properly audit these systems in light of ethical principles. For such a novel type of algorithmic auditing to become standard practice, two main prerequisites need to be available: A lifecycle model that is tailored towards transpare…
▽ More
The growing adoption and deployment of Machine Learning (ML) systems came with its share of ethical incidents and societal concerns. It also unveiled the necessity to properly audit these systems in light of ethical principles. For such a novel type of algorithmic auditing to become standard practice, two main prerequisites need to be available: A lifecycle model that is tailored towards transparency and accountability, and a principled risk assessment procedure that allows the proper sco** of the audit. Aiming to make a pragmatic step towards a wider adoption of ML auditing, we present a respective procedure that extends the AI-HLEG guidelines published by the European Commission. Our audit procedure is based on an ML lifecycle model that explicitly focuses on documentation, accountability, and quality assurance; and serves as a common ground for alignment between the auditors and the audited organisation. We describe two pilots conducted on real-world use cases from two different organisations and discuss the shortcomings of ML algorithmic auditing as well as future directions thereof.
△ Less
Submitted 21 May, 2024;
originally announced May 2024.
-
Anthropomorphic Gras** with Neural Object Shape Completion
Authors:
Diego Hidalgo-Carvajal,
Hanzhi Chen,
Gemma C. Bettelani,
Jaesug Jung,
Melissa Zavaglia,
Laura Busse,
Abdeldjallil Naceri,
Stefan Leutenegger,
Sami Haddadin
Abstract:
The progressive prevalence of robots in human-suited environments has given rise to a myriad of object manipulation techniques, in which dexterity plays a paramount role. It is well-established that humans exhibit extraordinary dexterity when handling objects. Such dexterity seems to derive from a robust understanding of object properties (such as weight, size, and shape), as well as a remarkable…
▽ More
The progressive prevalence of robots in human-suited environments has given rise to a myriad of object manipulation techniques, in which dexterity plays a paramount role. It is well-established that humans exhibit extraordinary dexterity when handling objects. Such dexterity seems to derive from a robust understanding of object properties (such as weight, size, and shape), as well as a remarkable capacity to interact with them. Hand postures commonly demonstrate the influence of specific regions on objects that need to be grasped, especially when objects are partially visible. In this work, we leverage human-like object understanding by reconstructing and completing their full geometry from partial observations, and manipulating them using a 7-DoF anthropomorphic robot hand. Our approach has significantly improved the gras** success rates of baselines with only partial reconstruction by nearly 30% and achieved over 150 successful grasps with three different object categories. This demonstrates our approach's consistent ability to predict and execute gras** postures based on the completed object shapes from various directions and positions in real-world scenarios. Our work opens up new possibilities for enhancing robotic applications that require precise gras** and manipulation skills of real-world reconstructed objects.
△ Less
Submitted 9 November, 2023; v1 submitted 4 November, 2023;
originally announced November 2023.
-
LoHoRavens: A Long-Horizon Language-Conditioned Benchmark for Robotic Tabletop Manipulation
Authors:
Shengqiang Zhang,
Philipp Wicke,
Lütfi Kerem Şenel,
Luis Figueredo,
Abdeldjallil Naceri,
Sami Haddadin,
Barbara Plank,
Hinrich Schütze
Abstract:
The convergence of embodied agents and large language models (LLMs) has brought significant advancements to embodied instruction following. Particularly, the strong reasoning capabilities of LLMs make it possible for robots to perform long-horizon tasks without expensive annotated demonstrations. However, public benchmarks for testing the long-horizon reasoning capabilities of language-conditioned…
▽ More
The convergence of embodied agents and large language models (LLMs) has brought significant advancements to embodied instruction following. Particularly, the strong reasoning capabilities of LLMs make it possible for robots to perform long-horizon tasks without expensive annotated demonstrations. However, public benchmarks for testing the long-horizon reasoning capabilities of language-conditioned robots in various scenarios are still missing. To fill this gap, this work focuses on the tabletop manipulation task and releases a simulation benchmark, \textit{LoHoRavens}, which covers various long-horizon reasoning aspects spanning color, size, space, arithmetics and reference. Furthermore, there is a key modality bridging problem for long-horizon manipulation tasks with LLMs: how to incorporate the observation feedback during robot execution for the LLM's closed-loop planning, which is however less studied by prior work. We investigate two methods of bridging the modality gap: caption generation and learnable interface for incorporating explicit and implicit observation feedback to the LLM, respectively. These methods serve as the two baselines for our proposed benchmark. Experiments show that both methods struggle to solve some tasks, indicating long-horizon manipulation tasks are still challenging for current popular models. We expect the proposed public benchmark and baselines can help the community develop better models for long-horizon tabletop manipulation tasks.
△ Less
Submitted 23 October, 2023; v1 submitted 18 October, 2023;
originally announced October 2023.
-
Care3D: An Active 3D Object Detection Dataset of Real Robotic-Care Environments
Authors:
Michael G. Adam,
Sebastian Eger,
Martin Piccolrovazzi,
Maged Iskandar,
Joern Vogel,
Alexander Dietrich,
Seongjien Bien,
Jon Skerlj,
Abdeldjallil Naceri,
Eckehard Steinbach,
Alin Albu-Schaeffer,
Sami Haddadin,
Wolfram Burgard
Abstract:
As labor shortage increases in the health sector, the demand for assistive robotics grows. However, the needed test data to develop those robots is scarce, especially for the application of active 3D object detection, where no real data exists at all. This short paper counters this by introducing such an annotated dataset of real environments. The captured environments represent areas which are al…
▽ More
As labor shortage increases in the health sector, the demand for assistive robotics grows. However, the needed test data to develop those robots is scarce, especially for the application of active 3D object detection, where no real data exists at all. This short paper counters this by introducing such an annotated dataset of real environments. The captured environments represent areas which are already in use in the field of robotic health care research. We further provide ground truth data within one room, for assessing SLAM algorithms running directly on a health care robot.
△ Less
Submitted 9 October, 2023;
originally announced October 2023.
-
A Concise Overview of Safety Aspects in Human-Robot Interaction
Authors:
Mazin Hamad,
Simone Nertinger,
Robin J. Kirschner,
Luis Figueredo,
Abdeldjallil Naceri,
Sami Haddadin
Abstract:
As of today, robots exhibit impressive agility but also pose potential hazards to humans using/collaborating with them. Consequently, safety is considered the most paramount factor in human-robot interaction (HRI). This paper presents a multi-layered safety architecture, integrating both physical and cognitive aspects for effective HRI. We outline critical requirements for physical safety layers a…
▽ More
As of today, robots exhibit impressive agility but also pose potential hazards to humans using/collaborating with them. Consequently, safety is considered the most paramount factor in human-robot interaction (HRI). This paper presents a multi-layered safety architecture, integrating both physical and cognitive aspects for effective HRI. We outline critical requirements for physical safety layers as service modules that can be arbitrarily queried. Further, we showcase an HRI scheme that addresses human factors and perceived safety as high-level constraints on a validated impact safety paradigm. The aim is to enable safety certification of human-friendly robots across various HRI scenarios.
△ Less
Submitted 18 September, 2023;
originally announced September 2023.
-
Towards Language-Based Modulation of Assistive Robots through Multimodal Models
Authors:
Philipp Wicke,
Lüfti Kerem Şenel,
Shengqiang Zhang,
Luis Figueredo,
Abdeldjallil Naceri,
Sami Haddadin,
Hinrich Schütze
Abstract:
In the field of Geriatronics, enabling effective and transparent communication between humans and robots is crucial for enhancing the acceptance and performance of assistive robots. Our early-stage research project investigates the potential of language-based modulation as a means to improve human-robot interaction. We propose to explore real-time modulation during task execution, leveraging langu…
▽ More
In the field of Geriatronics, enabling effective and transparent communication between humans and robots is crucial for enhancing the acceptance and performance of assistive robots. Our early-stage research project investigates the potential of language-based modulation as a means to improve human-robot interaction. We propose to explore real-time modulation during task execution, leveraging language cues, visual references, and multimodal inputs. By develo** transparent and interpretable methods, we aim to enable robots to adapt and respond to language commands, enhancing their usability and flexibility. Through the exchange of insights and knowledge at the workshop, we seek to gather valuable feedback to advance our research and contribute to the development of interactive robotic systems for Geriatronics and beyond.
△ Less
Submitted 27 June, 2023; v1 submitted 26 June, 2023;
originally announced June 2023.
