Showing 1–4 of 4 results for author: Eltouny, K

KG-Planner: Knowledge-Informed Graph Neural Planning for Collaborative Manipulators

Authors: Wansong Liu, Kareem Eltouny, Sibo Tian, Xiao Liang, Minghui Zheng

Abstract: This paper presents a novel knowledge-informed graph neural planner (KG-Planner) to address the challenge of efficiently planning collision-free motions for robots in high-dimensional spaces, considering both static and dynamic environments involving humans. Unlike traditional motion planners that struggle with finding a balance between efficiency and optimality, the KG-Planner takes a different a… ▽ More This paper presents a novel knowledge-informed graph neural planner (KG-Planner) to address the challenge of efficiently planning collision-free motions for robots in high-dimensional spaces, considering both static and dynamic environments involving humans. Unlike traditional motion planners that struggle with finding a balance between efficiency and optimality, the KG-Planner takes a different approach. Instead of relying solely on a neural network or imitating the motions of an oracle planner, our KG-Planner integrates explicit physical knowledge from the workspace. The integration of knowledge has two key aspects: (1) we present an approach to design a graph that can comprehensively model the workspace's compositional structure. The designed graph explicitly incorporates critical elements such as robot joints, obstacles, and their interconnections. This representation allows us to capture the intricate relationships between these elements. (2) We train a Graph Neural Network (GNN) that excels at generating nearly optimal robot motions. In particular, the GNN employs a layer-wise propagation rule to facilitate the exchange and update of information among workspace elements based on their connections. This propagation emphasizes the influence of these elements throughout the planning process. To validate the efficacy and efficiency of our KG-Planner, we conduct extensive experiments in both static and dynamic environments. These experiments include scenarios with and without human workers. The results of our approach are compared against existing methods, showcasing the superior performance of the KG-Planner. A short video introduction of this work is available (video link provided in the paper). △ Less

Submitted 13 May, 2024; originally announced May 2024.

High-Resolution Vision Transformers for Pixel-Level Identification of Structural Components and Damage

Authors: Kareem Eltouny, Seyedomid Sajedi, Xiao Liang

Abstract: Visual inspection is predominantly used to evaluate the state of civil structures, but recent developments in unmanned aerial vehicles (UAVs) and artificial intelligence have increased the speed, safety, and reliability of the inspection process. In this study, we develop a semantic segmentation network based on vision transformers and Laplacian pyramids scaling networks for efficiently parsing hi… ▽ More Visual inspection is predominantly used to evaluate the state of civil structures, but recent developments in unmanned aerial vehicles (UAVs) and artificial intelligence have increased the speed, safety, and reliability of the inspection process. In this study, we develop a semantic segmentation network based on vision transformers and Laplacian pyramids scaling networks for efficiently parsing high-resolution visual inspection images. The massive amounts of collected high-resolution images during inspections can slow down the investigation efforts. And while there have been extensive studies dedicated to the use of deep learning models for damage segmentation, processing high-resolution visual data can pose major computational difficulties. Traditionally, images are either uniformly downsampled or partitioned to cope with computational demands. However, the input is at risk of losing local fine details, such as thin cracks, or global contextual information. Inspired by super-resolution architectures, our vision transformer model learns to resize high-resolution images and masks to retain both the valuable local features and the global semantics without sacrificing computational efficiency. The proposed framework has been evaluated through comprehensive experiments on a dataset of bridge inspection report images using multiple metrics for pixel-wise materials detection. △ Less

Submitted 5 August, 2023; originally announced August 2023.

High-Fidelity Visual Structural Inspections through Transformers and Learnable Resizers

Authors: Kareem Eltouny, Seyedomid Sajedi, Xiao Liang

Abstract: Visual inspection is the predominant technique for evaluating the condition of civil infrastructure. The recent advances in unmanned aerial vehicles (UAVs) and artificial intelligence have made the visual inspections faster, safer, and more reliable. Camera-equipped UAVs are becoming the new standard in the industry by collecting massive amounts of visual data for human inspectors. Meanwhile, ther… ▽ More Visual inspection is the predominant technique for evaluating the condition of civil infrastructure. The recent advances in unmanned aerial vehicles (UAVs) and artificial intelligence have made the visual inspections faster, safer, and more reliable. Camera-equipped UAVs are becoming the new standard in the industry by collecting massive amounts of visual data for human inspectors. Meanwhile, there has been significant research on autonomous visual inspections using deep learning algorithms, including semantic segmentation. While UAVs can capture high-resolution images of buildings' façades, high-resolution segmentation is extremely challenging due to the high computational memory demands. Typically, images are uniformly downsized at the price of losing fine local details. Contrarily, breaking the images into multiple smaller patches can cause a loss of global contextual in-formation. We propose a hybrid strategy that can adapt to different inspections tasks by managing the global and local semantics trade-off. The framework comprises a compound, high-resolution deep learning architecture equipped with an attention-based segmentation model and learnable downsampler-upsampler modules designed for optimal efficiency and in-formation retention. The framework also utilizes vision transformers on a grid of image crops aiming for high precision learning without downsizing. An augmented inference technique is used to boost the performance and re-duce the possible loss of context due to grid crop**. Comprehensive experiments have been performed on 3D physics-based graphics models synthetic environments in the Quake City dataset. The proposed framework is evaluated using several metrics on three segmentation tasks: component type, component damage state, and global damage (crack, rebar, spalling). △ Less

Submitted 21 October, 2022; originally announced October 2022.

A Nonparametric Unsupervised Learning Approach for Structural Damage Detection

Authors: Kareem Eltouny, Xiao Liang

Abstract: In a world of aging infrastructure, structural health monitoring (SHM) emerges as a major step towards resilient and sustainable societies. The current advancements in machine learning and sensor technology have made SHM a more promising damage detection method than the traditional non-destructive testing methods. SHM using unsupervised learning methods offers an attractive alternative to the more… ▽ More In a world of aging infrastructure, structural health monitoring (SHM) emerges as a major step towards resilient and sustainable societies. The current advancements in machine learning and sensor technology have made SHM a more promising damage detection method than the traditional non-destructive testing methods. SHM using unsupervised learning methods offers an attractive alternative to the more commonly used supervised learning since it only requires data of the structure in normal conditions for the training process. The density-based novelty detection method provides a statistical element to the damage detection process but it relies heavily on the accuracy of the estimated probability density function (PDF). In this study, a novel unsupervised learning approach for SHM is proposed. It is based on the Kernel Density Maximum Entropy method by leveraging Bayesian optimization for hyperparameter tuning and also by extending the method into the multivariate space by the use of independent components analysis. The proposed approach is evaluated on a numerically simulated three-story reinforced concrete moment frame, where 94% of accuracy is achieved in structural damage detection. △ Less

Submitted 28 September, 2020; originally announced September 2020.