-
Nadine: An LLM-driven Intelligent Social Robot with Affective Capabilities and Human-like Memory
Authors:
Hangyeol Kang,
Maher Ben Moussa,
Nadia Magnenat-Thalmann
Abstract:
In this work, we describe our approach to develo** an intelligent and robust social robotic system for the Nadine social robot platform. We achieve this by integrating Large Language Models (LLMs) and skilfully leveraging the powerful reasoning and instruction-following capabilities of these types of models to achieve advanced human-like affective and cognitive capabilities. This approach is nov…
▽ More
In this work, we describe our approach to develo** an intelligent and robust social robotic system for the Nadine social robot platform. We achieve this by integrating Large Language Models (LLMs) and skilfully leveraging the powerful reasoning and instruction-following capabilities of these types of models to achieve advanced human-like affective and cognitive capabilities. This approach is novel compared to the current state-of-the-art LLM-based agents which do not implement human-like long-term memory or sophisticated emotional appraisal. The naturalness of social robots, consisting of multiple modules, highly depends on the performance and capabilities of each component of the system and the seamless integration of the components. We built a social robot system that enables generating appropriate behaviours through multimodal input processing, bringing episodic memories accordingly to the recognised user, and simulating the emotional states of the robot induced by the interaction with the human partner. In particular, we introduce an LLM-agent frame for social robots, SoR-ReAct, serving as a core component for the interaction module in our system. This design has brought forth the advancement of social robots and aims to increase the quality of human-robot interaction.
△ Less
Submitted 30 May, 2024;
originally announced May 2024.
-
Implicit Sensing in Traffic Optimization: Advanced Deep Reinforcement Learning Techniques
Authors:
Emanuel Figetakis,
Yahuza Bello,
Ahmed Refaey,
Lei Lei,
Medhat Moussa
Abstract:
A sudden roadblock on highways due to many reasons such as road maintenance, accidents, and car repair is a common situation we encounter almost daily. Autonomous Vehicles (AVs) equipped with sensors that can acquire vehicle dynamics such as speed, acceleration, and location can make intelligent decisions to change lanes before reaching a roadblock. A number of literature studies have examined car…
▽ More
A sudden roadblock on highways due to many reasons such as road maintenance, accidents, and car repair is a common situation we encounter almost daily. Autonomous Vehicles (AVs) equipped with sensors that can acquire vehicle dynamics such as speed, acceleration, and location can make intelligent decisions to change lanes before reaching a roadblock. A number of literature studies have examined car-following models and lane-changing models. However, only a few studies proposed an integrated car-following and lane-changing model, which has the potential to model practical driving maneuvers. Hence, in this paper, we present an integrated car-following and lane-changing decision-control system based on Deep Reinforcement Learning (DRL) to address this issue. Specifically, we consider a scenario where sudden construction work will be carried out along a highway. We model the scenario as a Markov Decision Process (MDP) and employ the well-known DQN algorithm to train the RL agent to make the appropriate decision accordingly (i.e., either stay in the same lane or change lanes). To overcome the delay and computational requirement of DRL algorithms, we adopt an MEC-assisted architecture where the RL agents are trained on MEC servers. We utilize the highly reputable SUMO simulator and OPENAI GYM to evaluate the performance of the proposed model under two policies; ε-greedy policy and Boltzmann policy. The results unequivocally demonstrate that the DQN agent trained using the ε-greedy policy significantly outperforms the one trained with the Boltzmann policy.
△ Less
Submitted 25 September, 2023;
originally announced September 2023.
-
Towards develo** a realistic robotics simulation environment of an indoor vegetable greenhouse
Authors:
Brent Van De Walker,
Brendan Byrne,
Joshua Near,
Blake Purdie,
Matthew Whatman,
David Weales,
Cole Tarry,
Medhat Moussa
Abstract:
This article presents a method for develo** a realistic robotics simulation environment for application in vegetable greenhouses. The method pipeline starts with the construction of a 3D cloud images of the greenhouse rows. This data is then used to develop a robotics simulation environment using the CoppeliaSim simulation software. The method has been tested using images from a commercial green…
▽ More
This article presents a method for develo** a realistic robotics simulation environment for application in vegetable greenhouses. The method pipeline starts with the construction of a 3D cloud images of the greenhouse rows. This data is then used to develop a robotics simulation environment using the CoppeliaSim simulation software. The method has been tested using images from a commercial greenhouse.
△ Less
Submitted 29 July, 2021;
originally announced July 2021.
-
An Image Labeling Tool and Agricultural Dataset for Deep Learning
Authors:
Patrick Wspanialy,
Justin Brooks,
Medhat Moussa
Abstract:
We introduce a labeling tool and dataset aimed to facilitate computer vision research in agriculture. The annotation tool introduces novel methods for labeling with a variety of manual, semi-automatic, and fully-automatic tools. The dataset includes original images collected from commercial greenhouses, images from PlantVillage, and images from Google Images. Images were annotated with segmentatio…
▽ More
We introduce a labeling tool and dataset aimed to facilitate computer vision research in agriculture. The annotation tool introduces novel methods for labeling with a variety of manual, semi-automatic, and fully-automatic tools. The dataset includes original images collected from commercial greenhouses, images from PlantVillage, and images from Google Images. Images were annotated with segmentations for foreground leaf, fruit, and stem instances, and diseased leaf area. Labels were in an extended COCO format. In total the dataset contained 10k tomatoes, 7k leaves, 2k stems, and 2k diseased leaf annotations.
△ Less
Submitted 6 April, 2020;
originally announced April 2020.
-
Batch Normalization is a Cause of Adversarial Vulnerability
Authors:
Angus Galloway,
Anna Golubeva,
Thomas Tanay,
Medhat Moussa,
Graham W. Taylor
Abstract:
Batch normalization (batch norm) is often used in an attempt to stabilize and accelerate training in deep neural networks. In many cases it indeed decreases the number of parameter updates required to achieve low training error. However, it also reduces robustness to small adversarial input perturbations and noise by double-digit percentages, as we show on five standard datasets. Furthermore, subs…
▽ More
Batch normalization (batch norm) is often used in an attempt to stabilize and accelerate training in deep neural networks. In many cases it indeed decreases the number of parameter updates required to achieve low training error. However, it also reduces robustness to small adversarial input perturbations and noise by double-digit percentages, as we show on five standard datasets. Furthermore, substituting weight decay for batch norm is sufficient to nullify the relationship between adversarial vulnerability and the input dimension. Our work is consistent with a mean-field analysis that found that batch norm causes exploding gradients.
△ Less
Submitted 29 May, 2019; v1 submitted 6 May, 2019;
originally announced May 2019.
-
Beyond RAID 6 --- an Efficient Systematic Code Protecting Against Multiple Errors, Erasures, and Silent Data Corruption
Authors:
Mohamad Moussa,
Marek Rychlik
Abstract:
We describe a replacement for RAID 6, based on a new linear, systematic code, which detects and corrects any combination of $E$ errors (unknown location) and $Z$ erasures (known location) provided that $Z+2E \leq 4$. We investigate some scenarios for error correction beyond the code's minimum distance, using list decoding. We describe a decoding algorithm with quasi-logarithmic time complexity, wh…
▽ More
We describe a replacement for RAID 6, based on a new linear, systematic code, which detects and corrects any combination of $E$ errors (unknown location) and $Z$ erasures (known location) provided that $Z+2E \leq 4$. We investigate some scenarios for error correction beyond the code's minimum distance, using list decoding. We describe a decoding algorithm with quasi-logarithmic time complexity, when parallel processing is used: $\approx O(\log N)$ where $N$ is the number of disks in the array (similar to RAID 6).
By comparison, the error correcting code implemented by RAID 6 allows error detection and correction only when $(E,Z)=(1,0)$, $(0,1)$, or $(0,2)$. Hence, when in degraded mode (i.e., when $Z \geq 1$), RAID 6 loses its ability for detecting and correcting random errors (i.e., $E=0$), leading to data loss known as silent data corruption. In contrast, the proposed code does not experience silent data corruption unless $Z \geq 3$.
The aforementioned properties of our code, the relative simplicity of implementation, vastly improved data protection, and low computational complexity of the decoding algorithm, make our code a natural successor to RAID 6. As this code is based on the use of quintuple parity, this justifies the name PentaRAID for the RAID technology implementing the ideas of the current paper.
△ Less
Submitted 5 July, 2018; v1 submitted 21 June, 2018;
originally announced June 2018.
-
Predicting Adversarial Examples with High Confidence
Authors:
Angus Galloway,
Graham W. Taylor,
Medhat Moussa
Abstract:
It has been suggested that adversarial examples cause deep learning models to make incorrect predictions with high confidence. In this work, we take the opposite stance: an overly confident model is more likely to be vulnerable to adversarial examples. This work is one of the most proactive approaches taken to date, as we link robustness with non-calibrated model confidence on noisy images, provid…
▽ More
It has been suggested that adversarial examples cause deep learning models to make incorrect predictions with high confidence. In this work, we take the opposite stance: an overly confident model is more likely to be vulnerable to adversarial examples. This work is one of the most proactive approaches taken to date, as we link robustness with non-calibrated model confidence on noisy images, providing a data-augmentation-free path forward. The adversarial examples phenomenon is most easily explained by the trend of increasing non-regularized model capacity, while the diversity and number of samples in common datasets has remained flat. Test accuracy has incorrectly been associated with true generalization performance, ignoring that training and test splits are often extremely similar in terms of the overall representation space. The transferability property of adversarial examples was previously used as evidence against overfitting arguments, a perceived random effect, but overfitting is not always random.
△ Less
Submitted 12 February, 2018;
originally announced February 2018.
-
Attacking Binarized Neural Networks
Authors:
Angus Galloway,
Graham W. Taylor,
Medhat Moussa
Abstract:
Neural networks with low-precision weights and activations offer compelling efficiency advantages over their full-precision equivalents. The two most frequently discussed benefits of quantization are reduced memory consumption, and a faster forward pass when implemented with efficient bitwise operations. We propose a third benefit of very low-precision neural networks: improved robustness against…
▽ More
Neural networks with low-precision weights and activations offer compelling efficiency advantages over their full-precision equivalents. The two most frequently discussed benefits of quantization are reduced memory consumption, and a faster forward pass when implemented with efficient bitwise operations. We propose a third benefit of very low-precision neural networks: improved robustness against some adversarial attacks, and in the worst case, performance that is on par with full-precision models. We focus on the very low-precision case where weights and activations are both quantized to $\pm$1, and note that stochastically quantizing weights in just one layer can sharply reduce the impact of iterative attacks. We observe that non-scaled binary neural networks exhibit a similar effect to the original defensive distillation procedure that led to gradient masking, and a false notion of security. We address this by conducting both black-box and white-box experiments with binary models that do not artificially mask gradients.
△ Less
Submitted 31 January, 2018; v1 submitted 1 November, 2017;
originally announced November 2017.
-
The Ciona17 Dataset for Semantic Segmentation of Invasive Species in a Marine Aquaculture Environment
Authors:
Angus Galloway,
Graham W. Taylor,
Aaron Ramsay,
Medhat Moussa
Abstract:
An original dataset for semantic segmentation, Ciona17, is introduced, which to the best of the authors' knowledge, is the first dataset of its kind with pixel-level annotations pertaining to invasive species in a marine environment. Diverse outdoor illumination, a range of object shapes, colour, and severe occlusion provide a significant real world challenge for the computer vision community. An…
▽ More
An original dataset for semantic segmentation, Ciona17, is introduced, which to the best of the authors' knowledge, is the first dataset of its kind with pixel-level annotations pertaining to invasive species in a marine environment. Diverse outdoor illumination, a range of object shapes, colour, and severe occlusion provide a significant real world challenge for the computer vision community. An accompanying ground-truthing tool for superpixel labeling, Truth and Crop, is also introduced. Finally, we provide a baseline using a variant of Fully Convolutional Networks, and report results in terms of the standard mean intersection over union (mIoU) metric.
△ Less
Submitted 17 February, 2017;
originally announced February 2017.
-
An Integrated Simulator and Dataset that Combines Gras** and Vision for Deep Learning
Authors:
Matthew Veres,
Medhat Moussa,
Graham W. Taylor
Abstract:
Deep learning is an established framework for learning hierarchical data representations. While compute power is in abundance, one of the main challenges in applying this framework to robotic gras** has been obtaining the amount of data needed to learn these representations, and structuring the data to the task at hand. Among contemporary approaches in the literature, we highlight key properties…
▽ More
Deep learning is an established framework for learning hierarchical data representations. While compute power is in abundance, one of the main challenges in applying this framework to robotic gras** has been obtaining the amount of data needed to learn these representations, and structuring the data to the task at hand. Among contemporary approaches in the literature, we highlight key properties that have encouraged the use of deep learning techniques, and in this paper, detail our experience in develo** a simulator for collecting cylindrical precision grasps of a multi-fingered dexterous robotic hand.
△ Less
Submitted 17 April, 2017; v1 submitted 7 February, 2017;
originally announced February 2017.
-
Modeling Grasp Motor Imagery through Deep Conditional Generative Models
Authors:
Matthew Veres,
Medhat Moussa,
Graham W. Taylor
Abstract:
Gras** is a complex process involving knowledge of the object, the surroundings, and of oneself. While humans are able to integrate and process all of the sensory information required for performing this task, equip** machines with this capability is an extremely challenging endeavor. In this paper, we investigate how deep learning techniques can allow us to translate high-level concepts such…
▽ More
Gras** is a complex process involving knowledge of the object, the surroundings, and of oneself. While humans are able to integrate and process all of the sensory information required for performing this task, equip** machines with this capability is an extremely challenging endeavor. In this paper, we investigate how deep learning techniques can allow us to translate high-level concepts such as motor imagery to the problem of robotic grasp synthesis. We explore a paradigm based on generative models for learning integrated object-action representations, and demonstrate its capacity for capturing and generating multimodal, multi-finger grasp configurations on a simulated gras** dataset.
△ Less
Submitted 11 January, 2017;
originally announced January 2017.
-
Ladder and Subdivision of Ladder Graphs with Pendant Edges are Odd Graceful
Authors:
M. I. Moussa,
E. M. Badr
Abstract:
The ladder graph plays an important role in many applications as Electronics, Electrical and Wireless communication areas. The aim of this work is to present a new class of odd graceful labeling for the ladder graph. In particular, we show that the ladder graph Ln with m-pendant Ln + mk1 is odd graceful. We also show that the subdivision of ladder graph Ln with m-pendant S(Ln) + mk1 is odd gracefu…
▽ More
The ladder graph plays an important role in many applications as Electronics, Electrical and Wireless communication areas. The aim of this work is to present a new class of odd graceful labeling for the ladder graph. In particular, we show that the ladder graph Ln with m-pendant Ln + mk1 is odd graceful. We also show that the subdivision of ladder graph Ln with m-pendant S(Ln) + mk1 is odd graceful. Finally, we prove that all the subdivision of triangular snakes (delt-k snake) with pendant edges S(delt-k snake)+ mk are odd graceful.
△ Less
Submitted 8 April, 2016;
originally announced April 2016.
-
Interference Cancellation trough Interference Alignment for Downlink of Cognitive Cellular Networks
Authors:
May Moussa,
Fotis Foukalas,
Tamer Khattab
Abstract:
In this letter, we propose the interference cancellation through interference alignment at the downlink of cognitive cellular networks. Interference alignment helps the spatial resources to be shared among primary and secondary cells and thus, it can provide higher degrees of freedom through interference cancellation. We derive and depict the achievable degrees of freedom. We also analyse and calc…
▽ More
In this letter, we propose the interference cancellation through interference alignment at the downlink of cognitive cellular networks. Interference alignment helps the spatial resources to be shared among primary and secondary cells and thus, it can provide higher degrees of freedom through interference cancellation. We derive and depict the achievable degrees of freedom. We also analyse and calculate the achievable sum rates applying water-filling optimal power allocation.
△ Less
Submitted 24 November, 2014;
originally announced November 2014.
-
An Algorithm for Odd Gracefulness of the Tensor Product of Two Line Graphs
Authors:
M. Ibrahim Moussa
Abstract:
An odd graceful labeling of a graph G=(V,E) is a function f:V(G)->[0,1,2,...,2|E(G)|-1} such that |f(u)-f(v)| is odd value less than or equal to 2|E(G)-1| for any u, v in V(G). In spite of the large number of papers published on the subject of graph labeling, there are few algorithms to be used by researchers to gracefully label graphs. This work provides generalized odd graceful solutions to all…
▽ More
An odd graceful labeling of a graph G=(V,E) is a function f:V(G)->[0,1,2,...,2|E(G)|-1} such that |f(u)-f(v)| is odd value less than or equal to 2|E(G)-1| for any u, v in V(G). In spite of the large number of papers published on the subject of graph labeling, there are few algorithms to be used by researchers to gracefully label graphs. This work provides generalized odd graceful solutions to all the vertices and edges for the tensor product of the two paths P_n and P_m denoted P_n^P_m . Firstly, we describe an algorithm to label the vertices and the edges of the vertex set V(P_n^P_m) and the edge set E(P_n^P_m) respectively. Finally, we prove that the graph P_n^P_m is odd graceful for all integers n and m.
△ Less
Submitted 23 March, 2011;
originally announced March 2011.
-
An Algorithm for Odd Graceful Labeling of the Union of Paths and Cycles
Authors:
M. Ibrahim Moussa
Abstract:
In 1991, Gnanajothi [4] proved that the path graph P_n with n vertex and n-1 edge is odd graceful, and the cycle graph C_m with m vertex and m edges is odd graceful if and only if m even, she proved the cycle graph is not graceful if m odd. In this paper, firstly, we studied the graph C_m $\cup$ P_m when m = 4, 6,8,10 and then we proved that the graph C_ $\cup$ P_n is odd graceful if m is even. Fi…
▽ More
In 1991, Gnanajothi [4] proved that the path graph P_n with n vertex and n-1 edge is odd graceful, and the cycle graph C_m with m vertex and m edges is odd graceful if and only if m even, she proved the cycle graph is not graceful if m odd. In this paper, firstly, we studied the graph C_m $\cup$ P_m when m = 4, 6,8,10 and then we proved that the graph C_ $\cup$ P_n is odd graceful if m is even. Finally, we described an algorithm to label the vertices and the edges of the vertex set V(C_m $\cup$ P_n) and the edge set E(C_m $\cup$ P_n).
△ Less
Submitted 21 April, 2010;
originally announced April 2010.
-
An Algorithm for Odd Graceful Labeling of the Union of Paths and Cycles
Authors:
M. Ibrahim Moussa
Abstract:
In 1991, Gnanajothi [4] proved that the path graph P_n with n vertex and n-1 edge is odd graceful, and the cycle graph C_m with m vertex and m edges is odd graceful if and only if m even, she proved the cycle graph is not graceful if m odd. In this paper, firstly, we studied the graph C_m $\cup$ P_m when m = 4, 6,8,10 and then we proved that the graph C_ $\cup$ P_n is odd graceful if m is even. Fi…
▽ More
In 1991, Gnanajothi [4] proved that the path graph P_n with n vertex and n-1 edge is odd graceful, and the cycle graph C_m with m vertex and m edges is odd graceful if and only if m even, she proved the cycle graph is not graceful if m odd. In this paper, firstly, we studied the graph C_m $\cup$ P_m when m = 4, 6,8,10 and then we proved that the graph C_ $\cup$ P_n is odd graceful if m is even. Finally, we described an algorithm to label the vertices and the edges of the vertex set V(C_m $\cup$ P_n) and the edge set E(C_m $\cup$ P_n).
△ Less
Submitted 18 March, 2010;
originally announced March 2010.
