-
Visual Imitation Learning of Task-Oriented Object Gras** and Rearrangement
Authors:
Yichen Cai,
Jianfeng Gao,
Christoph Pohl,
Tamim Asfour
Abstract:
Task-oriented object gras** and rearrangement are critical skills for robots to accomplish different real-world manipulation tasks. However, they remain challenging due to partial observations of the objects and shape variations in categorical objects. In this paper, we propose the Multi-feature Implicit Model (MIMO), a novel object representation that encodes multiple spatial features between a…
▽ More
Task-oriented object gras** and rearrangement are critical skills for robots to accomplish different real-world manipulation tasks. However, they remain challenging due to partial observations of the objects and shape variations in categorical objects. In this paper, we propose the Multi-feature Implicit Model (MIMO), a novel object representation that encodes multiple spatial features between a point and an object in an implicit neural field. Training such a model on multiple features ensures that it embeds the object shapes consistently in different aspects, thus improving its performance in object shape reconstruction from partial observation, shape similarity measure, and modeling spatial relations between objects. Based on MIMO, we propose a framework to learn task-oriented object gras** and rearrangement from single or multiple human demonstration videos. The evaluations in simulation show that our approach outperforms the state-of-the-art methods for multi- and single-view observations. Real-world experiments demonstrate the efficacy of our approach in one- and few-shot imitation learning of manipulation tasks.
△ Less
Submitted 20 March, 2024;
originally announced March 2024.
-
AutoGPT+P: Affordance-based Task Planning with Large Language Models
Authors:
Timo Birr,
Christoph Pohl,
Abdelrahman Younes,
Tamim Asfour
Abstract:
Recent advances in task planning leverage Large Language Models (LLMs) to improve generalizability by combining such models with classical planning algorithms to address their inherent limitations in reasoning capabilities. However, these approaches face the challenge of dynamically capturing the initial state of the task planning problem. To alleviate this issue, we propose AutoGPT+P, a system th…
▽ More
Recent advances in task planning leverage Large Language Models (LLMs) to improve generalizability by combining such models with classical planning algorithms to address their inherent limitations in reasoning capabilities. However, these approaches face the challenge of dynamically capturing the initial state of the task planning problem. To alleviate this issue, we propose AutoGPT+P, a system that combines an affordance-based scene representation with a planning system. Affordances encompass the action possibilities of an agent on the environment and objects present in it. Thus, deriving the planning domain from an affordance-based scene representation allows symbolic planning with arbitrary objects. AutoGPT+P leverages this representation to derive and execute a plan for a task specified by the user in natural language. In addition to solving planning tasks under a closed-world assumption, AutoGPT+P can also handle planning with incomplete information, e. g., tasks with missing objects by exploring the scene, suggesting alternatives, or providing a partial plan. The affordance-based scene representation combines object detection with an automatically generated object-affordance-map** using ChatGPT. The core planning tool extends existing work by automatically correcting semantic and syntactic errors. Our approach achieves a success rate of 98%, surpassing the current 81% success rate of the current state-of-the-art LLM-based planning method SayCan on the SayCan instruction set. Furthermore, we evaluated our approach on our newly created dataset with 150 scenarios covering a wide range of complex tasks with missing objects, achieving a success rate of 79% on our dataset. The dataset and the code are publicly available at https://git.h2t.iar.kit.edu/birr/autogpt-p-standalone.
△ Less
Submitted 16 February, 2024;
originally announced February 2024.
-
MAkEable: Memory-centered and Affordance-based Task Execution Framework for Transferable Mobile Manipulation Skills
Authors:
Christoph Pohl,
Fabian Reister,
Fabian Peller-Konrad,
Tamim Asfour
Abstract:
To perform versatile mobile manipulation tasks in human-centered environments, the ability to efficiently transfer learned tasks and experiences from one robot to another or across different environments is key. In this paper, we present MAkEable, a versatile uni- and multi-manual mobile manipulation framework that facilitates the transfer of capabilities and knowledge across different tasks, envi…
▽ More
To perform versatile mobile manipulation tasks in human-centered environments, the ability to efficiently transfer learned tasks and experiences from one robot to another or across different environments is key. In this paper, we present MAkEable, a versatile uni- and multi-manual mobile manipulation framework that facilitates the transfer of capabilities and knowledge across different tasks, environments, and robots. Our framework integrates an affordance-based task description into the memory-centric cognitive architecture of the ARMAR humanoid robot family, which supports the sharing of experiences and demonstrations for transfer learning. By representing mobile manipulation actions through affordances, i.e., interaction possibilities of the robot with its environment, we provide a unifying framework for the autonomous uni- and multi-manual manipulation of known and unknown objects in various environments. We demonstrate the applicability of the framework in real-world experiments for multiple robots, tasks, and environments. This includes gras** known and unknown objects, object placing, bimanual object gras**, memory-enabled skill transfer in a drawer opening scenario across two different humanoid robots, and a pouring task learned from human demonstration.
△ Less
Submitted 21 March, 2024; v1 submitted 30 January, 2024;
originally announced January 2024.
-
Apate - A Linux Kernel Module for High Interaction Honeypots
Authors:
Christoph Pohl,
Michael Meier,
Hans-Joachim Hof
Abstract:
Honeypots are used in IT Security to detect and gather information about ongoing intrusions, e.g., by documenting the approach of an attacker. Honeypots do so by presenting an interactive system that seems just like a valid application to an attacker. One of the main design goals of honeypots is to stay unnoticed by attackers as long as possible. The longer the intruder interacts with the honeypot…
▽ More
Honeypots are used in IT Security to detect and gather information about ongoing intrusions, e.g., by documenting the approach of an attacker. Honeypots do so by presenting an interactive system that seems just like a valid application to an attacker. One of the main design goals of honeypots is to stay unnoticed by attackers as long as possible. The longer the intruder interacts with the honeypot, the more valuable information about the attack can be collected. Of course, another main goal of honeypots is to not open new vulnerabilities that attackers can exploit. Thus, it is necessary to harden the honeypot and the surrounding environment. This paper presents Apate, a Linux Kernel Module (LKM) that is able to log, block and manipulate system calls based on preconfigurable conditions like Process ID (PID), User Id (UID), and many more. Apate can be used to build and harden High Interaction Honeypots. Apate can be configured using an integrated high level language. Thus, Apate is an important and easy to use building block for upcoming High Interaction Honeypots.
△ Less
Submitted 11 July, 2015;
originally announced July 2015.
-
Secure Scrum: Development of Secure Software with Scrum
Authors:
Christoph Pohl,
Hans-Joachim Hof
Abstract:
Nowadays, the use of agile software development methods like Scrum is common in industry and academia. Considering the current attacking landscape, it is clear that develo** secure software should be a main concern in all software development projects. In traditional software projects, security issues require detailed planning in an initial planning phase, typically resulting in a detailed secur…
▽ More
Nowadays, the use of agile software development methods like Scrum is common in industry and academia. Considering the current attacking landscape, it is clear that develo** secure software should be a main concern in all software development projects. In traditional software projects, security issues require detailed planning in an initial planning phase, typically resulting in a detailed security analysis (e.g., threat and risk analysis), a security architecture, and instructions for security implementation (e.g., specification of key sizes and cryptographic algorithms to use). Agile software development methods like Scrum are known for reducing the initial planning phases (e.g., sprint 0 in Scrum) and for focusing more on producing running code. Scrum is also known for allowing fast adaption of the emerging software to changes of customer wishes. For security, this means that it is likely that there are no detailed security architecture or security implementation instructions from the start of the project. It also means that a lot of design decisions will be made during the runtime of the project. Hence, to address security in Scrum, it is necessary to consider security issues throughout the whole software development process. Secure Scrum is a variation of the Scrum framework with special focus on the development of secure software throughout the whole software development process. It puts emphasis on implementation of security related issues without the need of changing the underlying Scrum process or influencing team dynamics. Secure Scrum allows even non- security experts to spot security issues, to implement security features, and to verify implementations. A field test of Secure Scrum shows that the security level of software developed using Secure Scrum is higher then the security level of software developed using standard Scrum.
△ Less
Submitted 10 July, 2015;
originally announced July 2015.
-
The All-Seeing Eye: A Massive-Multi-Sensor Zero-Configuration Intrusion Detection System for Web Applications
Authors:
Christoph Pohl,
Hans-Joachim Hof
Abstract:
Timing attacks are a challenge for current intrusion detection solutions. Timing attacks are dangerous for web applications because they may leak information about side channel vulnerabilities. This paper presents a massive-multi-sensor zero-configuration Intrusion Detection System that is especially good at detecting timing attacks. Unlike current solutions, the proposed Intrusion Detection Syste…
▽ More
Timing attacks are a challenge for current intrusion detection solutions. Timing attacks are dangerous for web applications because they may leak information about side channel vulnerabilities. This paper presents a massive-multi-sensor zero-configuration Intrusion Detection System that is especially good at detecting timing attacks. Unlike current solutions, the proposed Intrusion Detection System uses a huge number of sensors for attack detection. These sensors include sensors automatically inserted into web application or into the frameworks used to build web applications. With this approach the Intrusion Detection System is able to detect sophisticated attacks like timing attacks or other brute-force attacks with increased accuracy. The proposed massive-multi-sensor zero-configuration intrusion detection system does not need specific knowledge about the system to protect, hence it offers zero-configuration capability.
△ Less
Submitted 23 June, 2015;
originally announced June 2015.
-
BREW: A Breakable Web Application for IT-Security Classroom Use
Authors:
Christoph Pohl,
Kathrin Schlierkamp,
Hans-Joachim Hof
Abstract:
This paper presents BREW (Breakable Web Application), a tool for teaching IT Security. BREWs main teaching targets are identification and exploitation of vulnerabilities, using technologies and methodologies for software auditing and testing, and bug detection, fixation, and writing of secure code. Main advantages of BREW include that it is easy to apply in practice, it is a perfect tool to create…
▽ More
This paper presents BREW (Breakable Web Application), a tool for teaching IT Security. BREWs main teaching targets are identification and exploitation of vulnerabilities, using technologies and methodologies for software auditing and testing, and bug detection, fixation, and writing of secure code. Main advantages of BREW include that it is easy to apply in practice, it is a perfect tool to create and retain motivation, it corresponds to the demands of the psychology of learning, and it can be used for a heterogeneous group of students. BREW has been success- fully used for teaching IT Security in Germany as well as on an Erasmus Project with international student groups.
△ Less
Submitted 10 June, 2015;
originally announced June 2015.
