Showing 1–12 of 12 results for author: Pfrommer, J

Sorting multibay block stacking storage systems

Authors: Jakob Pfrommer, Thomas Bömer, Daniyar Akizhanov, Anne Meyer

Abstract: Autonomous mobile robots (AMRs) are increasingly used to automate operations in intralogistics. One crucial feature of AMRs is their availability, allowing them to operate 24/7. This work addresses the multibay unit load pre-marshalling problem, which extends pre-marshalling from a single bay to larger warehouse configurations with multiple bays. Pre-marshalling leverages off-peak time intervals t… ▽ More Autonomous mobile robots (AMRs) are increasingly used to automate operations in intralogistics. One crucial feature of AMRs is their availability, allowing them to operate 24/7. This work addresses the multibay unit load pre-marshalling problem, which extends pre-marshalling from a single bay to larger warehouse configurations with multiple bays. Pre-marshalling leverages off-peak time intervals to sort a block stacking warehouse in anticipation of future orders. These larger warehouse configurations require not only the minimization of the number of moves but also the consideration of distance or time when making sorting decisions. Our proposed solution for the multibay unit load pre-marshalling problem is based on our two-step approach that first determines the access direction for each stack and then finds a sequence of moves to sort the warehouse. In addition to adapting the existing approach that integrates a network flow model and an extended A* algorithm, we additionally present an exact constraint programming approach for the second stage of the problem-solving process. The results demonstrate that the presented solution approach effectively enhances the access time of unit loads and reduces the sorting effort for block stacking warehouses with multiple bays. △ Less

Submitted 8 May, 2024; originally announced May 2024.

Attention-based Part Assembly for 3D Volumetric Shape Modeling

Authors: Chengzhi Wu, Junwei Zheng, Julius Pfrommer, Jürgen Beyerer

Abstract: Modeling a 3D volumetric shape as an assembly of decomposed shape parts is much more challenging, but semantically more valuable than direct reconstruction from a full shape representation. The neural network needs to implicitly learn part relations coherently, which is typically performed by dedicated network layers that can generate transformation matrices for each part. In this paper, we propos… ▽ More Modeling a 3D volumetric shape as an assembly of decomposed shape parts is much more challenging, but semantically more valuable than direct reconstruction from a full shape representation. The neural network needs to implicitly learn part relations coherently, which is typically performed by dedicated network layers that can generate transformation matrices for each part. In this paper, we propose a VoxAttention network architecture for attention-based part assembly. We further propose a variant of using channel-wise part attention and show the advantages of this approach. Experimental results show that our method outperforms most state-of-the-art methods for the part relation-aware 3D shape modeling task. △ Less

Submitted 17 April, 2023; originally announced April 2023.

Attention-based Point Cloud Edge Sampling

Authors: Chengzhi Wu, Junwei Zheng, Julius Pfrommer, Jürgen Beyerer

Abstract: Point cloud sampling is a less explored research topic for this data representation. The most commonly used sampling methods are still classical random sampling and farthest point sampling. With the development of neural networks, various methods have been proposed to sample point clouds in a task-based learning manner. However, these methods are mostly generative-based, rather than selecting poin… ▽ More Point cloud sampling is a less explored research topic for this data representation. The most commonly used sampling methods are still classical random sampling and farthest point sampling. With the development of neural networks, various methods have been proposed to sample point clouds in a task-based learning manner. However, these methods are mostly generative-based, rather than selecting points directly using mathematical statistics. Inspired by the Canny edge detection algorithm for images and with the help of the attention mechanism, this paper proposes a non-generative Attention-based Point cloud Edge Sampling method (APES), which captures salient points in the point cloud outline. Both qualitative and quantitative experimental results show the superior performance of our sampling method on common benchmark tasks. △ Less

Submitted 26 March, 2023; v1 submitted 28 February, 2023; originally announced February 2023.

Sim2real Transfer Learning for Point Cloud Segmentation: An Industrial Application Case on Autonomous Disassembly

Authors: Chengzhi Wu, Xuelei Bi, Julius Pfrommer, Alexander Cebulla, Simon Mangold, Jürgen Beyerer

Abstract: On robotics computer vision tasks, generating and annotating large amounts of data from real-world for the use of deep learning-based approaches is often difficult or even impossible. A common strategy for solving this problem is to apply simulation-to-reality (sim2real) approaches with the help of simulated scenes. While the majority of current robotics vision sim2real work focuses on image data,… ▽ More On robotics computer vision tasks, generating and annotating large amounts of data from real-world for the use of deep learning-based approaches is often difficult or even impossible. A common strategy for solving this problem is to apply simulation-to-reality (sim2real) approaches with the help of simulated scenes. While the majority of current robotics vision sim2real work focuses on image data, we present an industrial application case that uses sim2real transfer learning for point cloud data. We provide insights on how to generate and process synthetic point cloud data in order to achieve better performance when the learned model is transferred to real-world data. The issue of imbalanced learning is investigated using multiple strategies. A novel patch-based attention network is proposed additionally to tackle this problem. △ Less

Submitted 12 January, 2023; originally announced January 2023.

MotorFactory: A Blender Add-on for Large Dataset Generation of Small Electric Motors

Authors: Chengzhi Wu, Kanran Zhou, Jan-Philipp Kaiser, Norbert Mitschke, Jan-Felix Klein, Julius Pfrommer, Jürgen Beyerer, Gisela Lanza, Michael Heizmann, Kai Furmans

Abstract: To enable automatic disassembly of different product types with uncertain conditions and degrees of wear in remanufacturing, agile production systems that can adapt dynamically to changing requirements are needed. Machine learning algorithms can be employed due to their generalization capabilities of learning from various types and variants of products. However, in reality, datasets with a diversi… ▽ More To enable automatic disassembly of different product types with uncertain conditions and degrees of wear in remanufacturing, agile production systems that can adapt dynamically to changing requirements are needed. Machine learning algorithms can be employed due to their generalization capabilities of learning from various types and variants of products. However, in reality, datasets with a diversity of samples that can be used to train models are difficult to obtain in the initial period. This may cause bad performances when the system tries to adapt to new unseen input data in the future. In order to generate large datasets for different learning purposes, in our project, we present a Blender add-on named MotorFactory to generate customized mesh models of various motor instances. MotorFactory allows to create mesh models which, complemented with additional add-ons, can be further used to create synthetic RGB images, depth images, normal images, segmentation ground truth masks, and 3D point cloud datasets with point-wise semantic labels. The created synthetic datasets may be used for various tasks including motor type classification, object detection for decentralized material transfer tasks, part segmentation for disassembly and handling tasks, or even reinforcement learning-based robotics control or view-planning. △ Less

Submitted 11 January, 2023; originally announced January 2023.

SynMotor: A Benchmark Suite for Object Attribute Regression and Multi-task Learning

Authors: Chengzhi Wu, Linxi Qiu, Kanran Zhou, Julius Pfrommer, Jürgen Beyerer

Abstract: In this paper, we develop a novel benchmark suite including both a 2D synthetic image dataset and a 3D synthetic point cloud dataset. Our work is a sub-task in the framework of a remanufacturing project, in which small electric motors are used as fundamental objects. Apart from the given detection, classification, and segmentation annotations, the key objects also have multiple learnable attribute… ▽ More In this paper, we develop a novel benchmark suite including both a 2D synthetic image dataset and a 3D synthetic point cloud dataset. Our work is a sub-task in the framework of a remanufacturing project, in which small electric motors are used as fundamental objects. Apart from the given detection, classification, and segmentation annotations, the key objects also have multiple learnable attributes with ground truth provided. This benchmark can be used for computer vision tasks including 2D/3D detection, classification, segmentation, and multi-attribute learning. It is worth mentioning that most attributes of the motors are quantified as continuously variable rather than binary, which makes our benchmark well-suited for the less explored regression tasks. In addition, appropriate evaluation metrics are adopted or developed for each task and promising baseline results are provided. We hope this benchmark can stimulate more research efforts on the sub-domain of object attribute learning and multi-task learning in the future. △ Less

Submitted 11 January, 2023; originally announced January 2023.

Object Detection in 3D Point Clouds via Local Correlation-Aware Point Embedding

Authors: Chengzhi Wu, Julius Pfrommer, Jürgen Beyerer, Kangning Li, Boris Neubert

Abstract: We present an improved approach for 3D object detection in point cloud data based on the Frustum PointNet (F-PointNet). Compared to the original F-PointNet, our newly proposed method considers the point neighborhood when computing point features. The newly introduced local neighborhood embedding operation mimics the convolutional operations in 2D neural networks. Thus features of each point are no… ▽ More We present an improved approach for 3D object detection in point cloud data based on the Frustum PointNet (F-PointNet). Compared to the original F-PointNet, our newly proposed method considers the point neighborhood when computing point features. The newly introduced local neighborhood embedding operation mimics the convolutional operations in 2D neural networks. Thus features of each point are not only computed with the features of its own or of the whole point cloud but also computed especially with respect to the features of its neighbors. Experiments show that our proposed method achieves better performance than the F-Pointnet baseline on 3D object detection tasks. △ Less

Submitted 11 January, 2023; originally announced January 2023.

Generative-Contrastive Learning for Self-Supervised Latent Representations of 3D Shapes from Multi-Modal Euclidean Input

Authors: Chengzhi Wu, Julius Pfrommer, Mingyuan Zhou, Jürgen Beyerer

Abstract: We propose a combined generative and contrastive neural architecture for learning latent representations of 3D volumetric shapes. The architecture uses two encoder branches for voxel grids and multi-view images from the same underlying shape. The main idea is to combine a contrastive loss between the resulting latent representations with an additional reconstruction loss. That helps to avoid colla… ▽ More We propose a combined generative and contrastive neural architecture for learning latent representations of 3D volumetric shapes. The architecture uses two encoder branches for voxel grids and multi-view images from the same underlying shape. The main idea is to combine a contrastive loss between the resulting latent representations with an additional reconstruction loss. That helps to avoid collapsing the latent representations as a trivial solution for minimizing the contrastive loss. A novel switching scheme is used to cross-train two encoders with a shared decoder. The switching scheme also enables the stop gradient operation on a random branch. Further classification experiments show that the latent representations learned with our self-supervised method integrate more useful information from the additional input data implicitly, thus leading to better reconstruction and classification performance. △ Less

Submitted 11 January, 2023; originally announced January 2023.

Solving the unit-load pre-marshalling problem in block stacking storage systems with multiple access directions

Authors: Jakob Pfrommer, Anne Meyer, Kevin Tierney

Abstract: Block stacking storage systems are highly adaptable warehouse systems with low investment costs. With multiple, deep lanes they can achieve high storage densities, but accessing some unit loads can be time-consuming. The unit-load pre-marshalling problem sorts the unit loads in a block stacking storage system in off-peak time periods to prepare for upcoming orders. The goal is to find a minimum nu… ▽ More Block stacking storage systems are highly adaptable warehouse systems with low investment costs. With multiple, deep lanes they can achieve high storage densities, but accessing some unit loads can be time-consuming. The unit-load pre-marshalling problem sorts the unit loads in a block stacking storage system in off-peak time periods to prepare for upcoming orders. The goal is to find a minimum number of unit-load moves needed to sequence a storage bay in ascending order based on the retrieval priority group of each unit load. In this paper, we present two solution approaches for determining the minimum number of unit-load moves. We show that for storage bays with one access direction, it is possible to adapt existing, optimal tree search procedures and lower bound heuristics from the container pre-marshalling problem. For multiple access directions, we develop a novel, two-step solution approach based on a network flow model and an A* algorithm with an adapted lower bound that is applicable in all scenarios. We further analyze the performance of the presented solutions in computational experiments for randomly generated problem instances and show that multiple access directions greatly reduce both the total access time of unit loads and the required sorting effort. △ Less

Submitted 19 July, 2022; originally announced July 2022.

On the quest of low temperature nitrogen infusion relevant for superconducting Nb based radio-frequency cavities

Authors: G. D. L. Semione, A. D. Pandey, S. Tober, J. Pfrommer, A. Pouilan, J. Drnec, G. Schütz, T. F. Keller, H. Noei, V. Vonk, B. Foster, A. Stierle

Abstract: A detailed study of the near-surface structure and composition of Nb, the material of choice for Superconducting Radio Frequency accelerator (SRF) cavities, is of great importance in order to understand the effects of different treatments applied during cavity production. By means of surface-sensitive techniques such as grazing incidence diffuse X-ray scattering, X-ray reflectivity and X-ray photo… ▽ More A detailed study of the near-surface structure and composition of Nb, the material of choice for Superconducting Radio Frequency accelerator (SRF) cavities, is of great importance in order to understand the effects of different treatments applied during cavity production. By means of surface-sensitive techniques such as grazing incidence diffuse X-ray scattering, X-ray reflectivity and X-ray photoelectron spectroscopy, single-crystalline Nb(100) samples were investigated in and ex-situ during annealing in UHV as well as in nitrogen atmospheres with temperatures and pressures similar to the ones employed in real Nb cavity treatments. Annealing of Nb specimens up to 800°C in vacuum promotes partial reduction of the natural surface oxides (Nb2O5, NbO2, NbO) into NbO. Upon cooling to 120°C, no evidence of nitrogen-rich layers was detected after nitrogen exposure times of up to 48 hours. Oxygen enrichment below the Nb/oxide interface and posterior diffusion of oxygen species towards the Nb matrix, along with a partial reduction of the natural surface oxides was observed upon a stepwise annealing up to 250°C. Nitrogen introduction to the system at 250°C neither promotes N diffusion into the Nb matrix nor the formation of new surface layers. Upon further heating to 500°C in a nitrogen atmosphere, the growth of a new subsurface Nb$_x$N$_y$ layer was detected. These results shed light on the composition of the near-surface region of Nb after low-temperature nitrogen treatments, which are reported to lead to a performance enhancement of SRF cavities. △ Less

Submitted 27 June, 2019; originally announced June 2019.

Comments: 15 pages, 8 figures, 1 table, to be submitted to PRB

Journal ref: Phys. Rev. Accel. Beams 22, 103102 (2019)

Informed Machine Learning -- A Taxonomy and Survey of Integrating Knowledge into Learning Systems

Authors: Laura von Rueden, Sebastian Mayer, Katharina Beckh, Bogdan Georgiev, Sven Giesselbach, Raoul Heese, Birgit Kirsch, Julius Pfrommer, Annika Pick, Rajkumar Ramamurthy, Michal Walczak, Jochen Garcke, Christian Bauckhage, Jannis Schuecker

Abstract: Despite its great success, machine learning can have its limits when dealing with insufficient training data. A potential solution is the additional integration of prior knowledge into the training process which leads to the notion of informed machine learning. In this paper, we present a structured overview of various approaches in this field. We provide a definition and propose a concept for inf… ▽ More Despite its great success, machine learning can have its limits when dealing with insufficient training data. A potential solution is the additional integration of prior knowledge into the training process which leads to the notion of informed machine learning. In this paper, we present a structured overview of various approaches in this field. We provide a definition and propose a concept for informed machine learning which illustrates its building blocks and distinguishes it from conventional machine learning. We introduce a taxonomy that serves as a classification framework for informed machine learning approaches. It considers the source of knowledge, its representation, and its integration into the machine learning pipeline. Based on this taxonomy, we survey related research and describe how different knowledge representations such as algebraic equations, logic rules, or simulation results can be used in learning systems. This evaluation of numerous papers on the basis of our taxonomy uncovers key methods in the field of informed machine learning. △ Less

Submitted 28 May, 2021; v1 submitted 29 March, 2019; originally announced March 2019.

Comments: Accepted at IEEE Transactions on Knowledge and Data Engineering: https://ieeexplore.ieee.org/document/9429985

Dynamic vehicle redistribution and online price incentives in shared mobility systems

Authors: Julius Pfrommer, Joseph Warrington, Georg Schildbach, Manfred Morari

Abstract: This paper considers a combination of intelligent repositioning decisions and dynamic pricing for the improved operation of shared mobility systems. The approach is applied to London's Barclays Cycle Hire scheme, which the authors have simulated based on historical data. Using model-based predictive control principles, dynamically varying rewards are computed and offered to customers carrying out… ▽ More This paper considers a combination of intelligent repositioning decisions and dynamic pricing for the improved operation of shared mobility systems. The approach is applied to London's Barclays Cycle Hire scheme, which the authors have simulated based on historical data. Using model-based predictive control principles, dynamically varying rewards are computed and offered to customers carrying out journeys. The aim is to encourage them to park bicycles at nearby under-used stations, thereby reducing the expected cost of repositioning them using dedicated staff. In parallel, the routes that repositioning staff should take are periodically recomputed using a model-based heuristic. It is shown that a trade-off between reward payouts to customers and the cost of hiring repositioning staff could be made, in order to minimize operating costs for a given desired service level. △ Less

Submitted 12 May, 2013; v1 submitted 14 April, 2013; originally announced April 2013.