-
The COMMOTIONS Urban Interactions Driving Simulator Study Dataset
Authors:
Aravinda Ramakrishnan Srinivasan,
Julian Schumann,
Yueyang Wang,
Yi-Shin Lin,
Michael Daly,
Albert Solernou,
Arkady Zgonnikov,
Matteo Leonetti,
Jac Billington,
Gustav Markkula
Abstract:
Accurate modelling of road user interaction has received lot of attention in recent years due to the advent of increasingly automated vehicles. To support such modelling, there is a need to complement naturalistic datasets of road user interaction with targeted, controlled study data. This paper describes a dataset collected in a simulator study conducted in the project COMMOTIONS, addressing urba…
▽ More
Accurate modelling of road user interaction has received lot of attention in recent years due to the advent of increasingly automated vehicles. To support such modelling, there is a need to complement naturalistic datasets of road user interaction with targeted, controlled study data. This paper describes a dataset collected in a simulator study conducted in the project COMMOTIONS, addressing urban driving interactions, in a state of the art moving base driving simulator. The study focused on two types of near-crash situations that can arise in urban driving interactions, and also collected data on human driver gap acceptance across a range of controlled gap sequences.
△ Less
Submitted 2 July, 2024; v1 submitted 17 May, 2023;
originally announced May 2023.
-
Proceedings of the AI-HRI Symposium at AAAI-FSS 2022
Authors:
Zhao Han,
Emmanuel Senft,
Muneeb I. Ahmad,
Shelly Bagchi,
Amir Yazdani,
Jason R. Wilson,
Boyoung Kim,
Ruchen Wen,
Justin W. Hart,
Daniel Hernández García,
Matteo Leonetti,
Ross Mead,
Reuth Mirsky,
Ahalya Prabhakar,
Megan L. Zimmerman
Abstract:
The Artificial Intelligence (AI) for Human-Robot Interaction (HRI) Symposium has been a successful venue of discussion and collaboration on AI theory and methods aimed at HRI since 2014. This year, after a review of the achievements of the AI-HRI community over the last decade in 2021, we are focusing on a visionary theme: exploring the future of AI-HRI. Accordingly, we added a Blue Sky Ideas trac…
▽ More
The Artificial Intelligence (AI) for Human-Robot Interaction (HRI) Symposium has been a successful venue of discussion and collaboration on AI theory and methods aimed at HRI since 2014. This year, after a review of the achievements of the AI-HRI community over the last decade in 2021, we are focusing on a visionary theme: exploring the future of AI-HRI. Accordingly, we added a Blue Sky Ideas track to foster a forward-thinking discussion on future research at the intersection of AI and HRI. As always, we appreciate all contributions related to any topic on AI/HRI and welcome new researchers who wish to take part in this growing community.
With the success of past symposia, AI-HRI impacts a variety of communities and problems, and has pioneered the discussions in recent trends and interests. This year's AI-HRI Fall Symposium aims to bring together researchers and practitioners from around the globe, representing a number of university, government, and industry laboratories. In doing so, we hope to accelerate research in the field, support technology transition and user adoption, and determine future directions for our group and our research.
△ Less
Submitted 28 November, 2022; v1 submitted 28 September, 2022;
originally announced September 2022.
-
Beyond RMSE: Do machine-learned models of road user interaction produce human-like behavior?
Authors:
Aravinda Ramakrishnan Srinivasan,
Yi-Shin Lin,
Morris Antonello,
Anthony Knittel,
Mohamed Hasan,
Majd Hawasly,
John Redford,
Subramanian Ramamoorthy,
Matteo Leonetti,
Jac Billington,
Richard Romano,
Gustav Markkula
Abstract:
Autonomous vehicles use a variety of sensors and machine-learned models to predict the behavior of surrounding road users. Most of the machine-learned models in the literature focus on quantitative error metrics like the root mean square error (RMSE) to learn and report their models' capabilities. This focus on quantitative error metrics tends to ignore the more important behavioral aspect of the…
▽ More
Autonomous vehicles use a variety of sensors and machine-learned models to predict the behavior of surrounding road users. Most of the machine-learned models in the literature focus on quantitative error metrics like the root mean square error (RMSE) to learn and report their models' capabilities. This focus on quantitative error metrics tends to ignore the more important behavioral aspect of the models, raising the question of whether these models really predict human-like behavior. Thus, we propose to analyze the output of machine-learned models much like we would analyze human data in conventional behavioral research. We introduce quantitative metrics to demonstrate presence of three different behavioral phenomena in a naturalistic highway driving dataset: 1) The kinematics-dependence of who passes a merging point first 2) Lane change by an on-highway vehicle to accommodate an on-ramp vehicle 3) Lane changes by vehicles on the highway to avoid lead vehicle conflicts. Then, we analyze the behavior of three machine-learned models using the same metrics. Even though the models' RMSE value differed, all the models captured the kinematic-dependent merging behavior but struggled at varying degrees to capture the more nuanced courtesy lane change and highway lane change behavior. Additionally, the collision aversion analysis during lane changes showed that the models struggled to capture the physical aspect of human driving: leaving adequate gap between the vehicles. Thus, our analysis highlighted the inadequacy of simple quantitative metrics and the need to take a broader behavioral perspective when analyzing machine-learned models of human driving predictions.
△ Less
Submitted 28 March, 2023; v1 submitted 22 June, 2022;
originally announced June 2022.
-
A Utility Maximization Model of Pedestrian and Driver Interactions
Authors:
Yi-Shin Lin,
Aravinda Ramakrishnan Srinivasan,
Matteo Leonetti,
Jac Billington,
Gustav Markkula
Abstract:
Many models account for the traffic flow of road users but few take the details of local interactions into consideration and how they could deteriorate into safety-critical situations. Building on the concept of sensorimotor control, we develop a modeling framework applying the principles of utility maximization, motor primitives, and intermittent action decisions to account for the details of int…
▽ More
Many models account for the traffic flow of road users but few take the details of local interactions into consideration and how they could deteriorate into safety-critical situations. Building on the concept of sensorimotor control, we develop a modeling framework applying the principles of utility maximization, motor primitives, and intermittent action decisions to account for the details of interactive behaviors among road users. The framework connects these principles to the decision theory and is applied to determine whether such an approach can reproduce the following phenomena: When two pedestrians travel on crossing paths, (a) their interaction is sensitive to initial asymmetries, and (b) based on which, they rapidly resolve collision conflict by adapting their behaviors. When a pedestrian crosses the road while facing an approaching car, (c) either road user yields to the other to resolve their conflict, akin to the pedestrian interaction, and (d) the outcome reveals a specific situational kinematics, associated with the nature of vehicle acceleration. We show that these phenomena emerge naturally from our modeling framework when the model can evolve its parameters as a consequence of the situations. We believe that the modeling framework and phenomenon-centered analysis offer promising tools to understand road user interactions. We conclude with a discussion on how the model can be instrumental in studying the safety-critical situations when including other variables in road-user interactions.
△ Less
Submitted 21 October, 2021;
originally announced October 2021.
-
AI-HRI 2021 Proceedings
Authors:
Reuth Mirsky,
Megan Zimmerman,
Muneed Ahmad,
Shelly Bagchi,
Felix Gervits,
Zhao Han,
Justin Hart,
Daniel Hernández García,
Matteo Leonetti,
Ross Mead,
Emmanuel Senft,
Jivko Sinapov,
Jason Wilson
Abstract:
The Artificial Intelligence (AI) for Human-Robot Interaction (HRI) Symposium has been a successful venue of discussion and collaboration since 2014. During that time, these symposia provided a fertile ground for numerous collaborations and pioneered many discussions revolving trust in HRI, XAI for HRI, service robots, interactive learning, and more.
This year, we aim to review the achievements o…
▽ More
The Artificial Intelligence (AI) for Human-Robot Interaction (HRI) Symposium has been a successful venue of discussion and collaboration since 2014. During that time, these symposia provided a fertile ground for numerous collaborations and pioneered many discussions revolving trust in HRI, XAI for HRI, service robots, interactive learning, and more.
This year, we aim to review the achievements of the AI-HRI community in the last decade, identify the challenges facing ahead, and welcome new researchers who wish to take part in this growing community. Taking this wide perspective, this year there will be no single theme to lead the symposium and we encourage AI-HRI submissions from across disciplines and research interests. Moreover, with the rising interest in AR and VR as part of an interaction and following the difficulties in running physical experiments during the pandemic, this year we specifically encourage researchers to submit works that do not include a physical robot in their evaluation, but promote HRI research in general. In addition, acknowledging that ethics is an inherent part of the human-robot interaction, we encourage submissions of works on ethics for HRI. Over the course of the two-day meeting, we will host a collaborative forum for discussion of current efforts in AI-HRI, with additional talks focused on the topics of ethics in HRI and ubiquitous HRI.
△ Less
Submitted 23 September, 2021; v1 submitted 22 September, 2021;
originally announced September 2021.
-
Meta-Reinforcement Learning for Heuristic Planning
Authors:
Ricardo Luna Gutierrez,
Matteo Leonetti
Abstract:
In Meta-Reinforcement Learning (meta-RL) an agent is trained on a set of tasks to prepare for and learn faster in new, unseen, but related tasks. The training tasks are usually hand-crafted to be representative of the expected distribution of test tasks and hence all used in training. We show that given a set of training tasks, learning can be both faster and more effective (leading to better perf…
▽ More
In Meta-Reinforcement Learning (meta-RL) an agent is trained on a set of tasks to prepare for and learn faster in new, unseen, but related tasks. The training tasks are usually hand-crafted to be representative of the expected distribution of test tasks and hence all used in training. We show that given a set of training tasks, learning can be both faster and more effective (leading to better performance in the test tasks), if the training tasks are appropriately selected. We propose a task selection algorithm, Information-Theoretic Task Selection (ITTS), based on information theory, which optimizes the set of tasks used for training in meta-RL, irrespectively of how they are generated. The algorithm establishes which training tasks are both sufficiently relevant for the test tasks, and different enough from one another. We reproduce different meta-RL experiments from the literature and show that ITTS improves the final performance in all of them.
△ Less
Submitted 6 July, 2021;
originally announced July 2021.
-
Comparing merging behaviors observed in naturalistic data with behaviors generated by a machine learned model
Authors:
Aravinda Ramakrishnan Srinivasan,
Mohamed Hasan,
Yi-Shin Lin,
Matteo Leonetti,
Jac Billington,
Richard Romano,
Gustav Markkula
Abstract:
There is quickly growing literature on machine-learned models that predict human driving trajectories in road traffic. These models focus their learning on low-dimensional error metrics, for example average distance between model-generated and observed trajectories. Such metrics permit relative comparison of models, but do not provide clearly interpretable information on how close to human behavio…
▽ More
There is quickly growing literature on machine-learned models that predict human driving trajectories in road traffic. These models focus their learning on low-dimensional error metrics, for example average distance between model-generated and observed trajectories. Such metrics permit relative comparison of models, but do not provide clearly interpretable information on how close to human behavior the models actually come, for example in terms of higher-level behavior phenomena that are known to be present in human driving. We study highway driving as an example scenario, and introduce metrics to quantitatively demonstrate the presence, in a naturalistic dataset, of two familiar behavioral phenomena: (1) The kinematics-dependent contest, between on-highway and on-ramp vehicles, of who passes the merging point first. (2) Courtesy lane changes away from the outermost lane, to leave space for a merging vehicle. Applying the exact same metrics to the output of a state-of-the-art machine-learned model, we show that the model is capable of reproducing the former phenomenon, but not the latter. We argue that this type of behavioral analysis provides information that is not available from conventional model-fitting metrics, and that it may be useful to analyze (and possibly fit) models also based on these types of behavioral criteria.
△ Less
Submitted 21 April, 2021;
originally announced April 2021.
-
Occlusion-Aware Search for Object Retrieval in Clutter
Authors:
Wissam Bejjani,
Wisdom C. Agboh,
Mehmet R. Dogar,
Matteo Leonetti
Abstract:
We address the manipulation task of retrieving a target object from a cluttered shelf. When the target object is hidden, the robot must search through the clutter for retrieving it. Solving this task requires reasoning over the likely locations of the target object. It also requires physics reasoning over multi-object interactions and future occlusions. In this work, we present a data-driven hybri…
▽ More
We address the manipulation task of retrieving a target object from a cluttered shelf. When the target object is hidden, the robot must search through the clutter for retrieving it. Solving this task requires reasoning over the likely locations of the target object. It also requires physics reasoning over multi-object interactions and future occlusions. In this work, we present a data-driven hybrid planner for generating occlusion-aware actions in closed-loop. The hybrid planner explores likely locations of the occluded target object as predicted by a learned distribution from the observation stream. The search is guided by a heuristic trained with reinforcement learning to act on observations with occlusions. We evaluate our approach in different simulation and real-world settings (video available on https://youtu.be/dY7YQ3LUVQg). The results validate that our approach can search and retrieve a target object in near real time in the real world while only being trained in simulation.
△ Less
Submitted 31 August, 2021; v1 submitted 6 November, 2020;
originally announced November 2020.
-
Information-theoretic Task Selection for Meta-Reinforcement Learning
Authors:
Ricardo Luna Gutierrez,
Matteo Leonetti
Abstract:
In Meta-Reinforcement Learning (meta-RL) an agent is trained on a set of tasks to prepare for and learn faster in new, unseen, but related tasks. The training tasks are usually hand-crafted to be representative of the expected distribution of test tasks and hence all used in training. We show that given a set of training tasks, learning can be both faster and more effective (leading to better perf…
▽ More
In Meta-Reinforcement Learning (meta-RL) an agent is trained on a set of tasks to prepare for and learn faster in new, unseen, but related tasks. The training tasks are usually hand-crafted to be representative of the expected distribution of test tasks and hence all used in training. We show that given a set of training tasks, learning can be both faster and more effective (leading to better performance in the test tasks), if the training tasks are appropriately selected. We propose a task selection algorithm, Information-Theoretic Task Selection (ITTS), based on information theory, which optimizes the set of tasks used for training in meta-RL, irrespectively of how they are generated. The algorithm establishes which training tasks are both sufficiently relevant for the test tasks, and different enough from one another. We reproduce different meta-RL experiments from the literature and show that ITTS improves the final performance in all of them.
△ Less
Submitted 1 July, 2021; v1 submitted 2 November, 2020;
originally announced November 2020.
-
Proceedings of the AI-HRI Symposium at AAAI-FSS 2020
Authors:
Shelly Bagchi,
Jason R. Wilson,
Muneeb I. Ahmad,
Christian Dondrup,
Zhao Han,
Justin W. Hart,
Matteo Leonetti,
Katrin Lohan,
Ross Mead,
Emmanuel Senft,
Jivko Sinapov,
Megan L. Zimmerman
Abstract:
The Artificial Intelligence (AI) for Human-Robot Interaction (HRI) Symposium has been a successful venue of discussion and collaboration since 2014. In that time, the related topic of trust in robotics has been rapidly growing, with major research efforts at universities and laboratories across the world. Indeed, many of the past participants in AI-HRI have been or are now involved with research i…
▽ More
The Artificial Intelligence (AI) for Human-Robot Interaction (HRI) Symposium has been a successful venue of discussion and collaboration since 2014. In that time, the related topic of trust in robotics has been rapidly growing, with major research efforts at universities and laboratories across the world. Indeed, many of the past participants in AI-HRI have been or are now involved with research into trust in HRI. While trust has no consensus definition, it is regularly associated with predictability, reliability, inciting confidence, and meeting expectations. Furthermore, it is generally believed that trust is crucial for adoption of both AI and robotics, particularly when transitioning technologies from the lab to industrial, social, and consumer applications. However, how does trust apply to the specific situations we encounter in the AI-HRI sphere? Is the notion of trust in AI the same as that in HRI? We see a growing need for research that lives directly at the intersection of AI and HRI that is serviced by this symposium. Over the course of the two-day meeting, we propose to create a collaborative forum for discussion of current efforts in trust for AI-HRI, with a sub-session focused on the related topic of explainable AI (XAI) for HRI.
△ Less
Submitted 14 December, 2020; v1 submitted 26 October, 2020;
originally announced October 2020.
-
Curriculum Learning with a Progression Function
Authors:
Andrea Bassich,
Francesco Foglino,
Matteo Leonetti,
Daniel Kudenko
Abstract:
Curriculum Learning for Reinforcement Learning is an increasingly popular technique that involves training an agent on a sequence of intermediate tasks, called a Curriculum, to increase the agent's performance and learning speed. This paper introduces a novel paradigm for curriculum generation based on progression and map** functions. While progression functions specify the complexity of the env…
▽ More
Curriculum Learning for Reinforcement Learning is an increasingly popular technique that involves training an agent on a sequence of intermediate tasks, called a Curriculum, to increase the agent's performance and learning speed. This paper introduces a novel paradigm for curriculum generation based on progression and map** functions. While progression functions specify the complexity of the environment at any given time, map** functions generate environments of a specific complexity. Different progression functions are introduced, including an autonomous online task progression based on the agent's performance. Our approach's benefits and wide applicability are shown by empirically comparing its performance to two state-of-the-art Curriculum Learning algorithms on six domains.
△ Less
Submitted 31 October, 2021; v1 submitted 2 August, 2020;
originally announced August 2020.
-
Curriculum Learning for Reinforcement Learning Domains: A Framework and Survey
Authors:
Sanmit Narvekar,
Bei Peng,
Matteo Leonetti,
Jivko Sinapov,
Matthew E. Taylor,
Peter Stone
Abstract:
Reinforcement learning (RL) is a popular paradigm for addressing sequential decision tasks in which the agent has only limited environmental feedback. Despite many advances over the past three decades, learning in many domains still requires a large amount of interaction with the environment, which can be prohibitively expensive in realistic scenarios. To address this problem, transfer learning ha…
▽ More
Reinforcement learning (RL) is a popular paradigm for addressing sequential decision tasks in which the agent has only limited environmental feedback. Despite many advances over the past three decades, learning in many domains still requires a large amount of interaction with the environment, which can be prohibitively expensive in realistic scenarios. To address this problem, transfer learning has been applied to reinforcement learning such that experience gained in one task can be leveraged when starting to learn the next, harder task. More recently, several lines of research have explored how tasks, or data samples themselves, can be sequenced into a curriculum for the purpose of learning a problem that may otherwise be too difficult to learn from scratch. In this article, we present a framework for curriculum learning (CL) in reinforcement learning, and use it to survey and classify existing CL methods in terms of their assumptions, capabilities, and goals. Finally, we use our framework to find open problems and suggest directions for future RL curriculum learning research.
△ Less
Submitted 17 September, 2020; v1 submitted 10 March, 2020;
originally announced March 2020.
-
Human-like Planning for Reaching in Cluttered Environments
Authors:
Mohamed Hasan,
Matthew Warburton,
Wisdom C. Agboh,
Mehmet R. Dogar,
Matteo Leonetti,
He Wang,
Faisal Mushtaq,
Mark Mon-Williams,
Anthony G. Cohn
Abstract:
Humans, in comparison to robots, are remarkably adept at reaching for objects in cluttered environments. The best existing robot planners are based on random sampling of configuration space -- which becomes excessively high-dimensional with large number of objects. Consequently, most planners often fail to efficiently find object manipulation plans in such environments. We addressed this problem b…
▽ More
Humans, in comparison to robots, are remarkably adept at reaching for objects in cluttered environments. The best existing robot planners are based on random sampling of configuration space -- which becomes excessively high-dimensional with large number of objects. Consequently, most planners often fail to efficiently find object manipulation plans in such environments. We addressed this problem by identifying high-level manipulation plans in humans, and transferring these skills to robot planners. We used virtual reality to capture human participants reaching for a target object on a tabletop cluttered with obstacles. From this, we devised a qualitative representation of the task space to abstract the decision making, irrespective of the number of obstacles. Based on this representation, human demonstrations were segmented and used to train decision classifiers. Using these classifiers, our planner produced a list of waypoints in task space. These waypoints provided a high-level plan, which could be transferred to an arbitrary robot model and used to initialise a local trajectory optimiser. We evaluated this approach through testing on unseen human VR data, a physics-based robot simulation, and a real robot (dataset and code are publicly available). We found that the human-like planner outperformed a state-of-the-art standard trajectory optimisation algorithm, and was able to generate effective strategies for rapid planning -- irrespective of the number of obstacles in the environment.
△ Less
Submitted 3 March, 2020; v1 submitted 28 February, 2020;
originally announced February 2020.
-
Proceedings of the AI-HRI Symposium at AAAI-FSS 2019
Authors:
Justin W. Hart,
Nick DePalma,
Richard G. Freedman,
Luca Iocchi,
Matteo Leonetti,
Katrin Lohan,
Ross Mead,
Emmanuel Senft,
Jivko Sinapov,
Elin A. Topp,
Tom Williams
Abstract:
The past few years have seen rapid progress in the development of service robots. Universities and companies alike have launched major research efforts toward the deployment of ambitious systems designed to aid human operators performing a variety of tasks. These robots are intended to make those who may otherwise need to live in assisted care facilities more independent, to help workers perform t…
▽ More
The past few years have seen rapid progress in the development of service robots. Universities and companies alike have launched major research efforts toward the deployment of ambitious systems designed to aid human operators performing a variety of tasks. These robots are intended to make those who may otherwise need to live in assisted care facilities more independent, to help workers perform their jobs, or simply to make life more convenient. Service robots provide a powerful platform on which to study Artificial Intelligence (AI) and Human-Robot Interaction (HRI) in the real world. Research sitting at the intersection of AI and HRI is crucial to the success of service robots if they are to fulfill their mission.
This symposium seeks to highlight research enabling robots to effectively interact with people autonomously while modeling, planning, and reasoning about the environment that the robot operates in and the tasks that it must perform. AI-HRI deals with the challenge of interacting with humans in environments that are relatively unstructured or which are structured around people rather than machines, as well as the possibility that the robot may need to interact naturally with people rather than through teach pendants, programming, or similar interfaces.
△ Less
Submitted 19 September, 2019; v1 submitted 10 September, 2019;
originally announced September 2019.
-
A gray-box approach for curriculum learning
Authors:
Francesco Foglino,
Matteo Leonetti,
Simone Sagratella,
Ruggiero Seccia
Abstract:
Curriculum learning is often employed in deep reinforcement learning to let the agent progress more quickly towards better behaviors. Numerical methods for curriculum learning in the literature provides only initial heuristic solutions, with little to no guarantee on their quality. We define a new gray-box function that, including a suitable scheduling problem, can be effectively used to reformula…
▽ More
Curriculum learning is often employed in deep reinforcement learning to let the agent progress more quickly towards better behaviors. Numerical methods for curriculum learning in the literature provides only initial heuristic solutions, with little to no guarantee on their quality. We define a new gray-box function that, including a suitable scheduling problem, can be effectively used to reformulate the curriculum learning problem. We propose different efficient numerical methods to address this gray-box reformulation. Preliminary numerical results on a benchmark task in the curriculum learning literature show the viability of the proposed approach.
△ Less
Submitted 16 June, 2019;
originally announced June 2019.
-
Curriculum Learning for Cumulative Return Maximization
Authors:
Francesco Foglino,
Christiano Coletto Christakou,
Ricardo Luna Gutierrez,
Matteo Leonetti
Abstract:
Curriculum learning has been successfully used in reinforcement learning to accelerate the learning process, through knowledge transfer between tasks of increasing complexity. Critical tasks, in which suboptimal exploratory actions must be minimized, can benefit from curriculum learning, and its ability to shape exploration through transfer. We propose a task sequencing algorithm maximizing the cu…
▽ More
Curriculum learning has been successfully used in reinforcement learning to accelerate the learning process, through knowledge transfer between tasks of increasing complexity. Critical tasks, in which suboptimal exploratory actions must be minimized, can benefit from curriculum learning, and its ability to shape exploration through transfer. We propose a task sequencing algorithm maximizing the cumulative return, that is, the return obtained by the agent across all the learning episodes. By maximizing the cumulative return, the agent not only aims at achieving high rewards as fast as possible, but also at doing so while limiting suboptimal actions. We experimentally compare our task sequencing algorithm to several popular metaheuristic algorithms for combinatorial optimization, and show that it achieves significantly better performance on the problem of cumulative return maximization. Furthermore, we validate our algorithm on a critical task, optimizing a home controller for a micro energy grid.
△ Less
Submitted 13 June, 2019;
originally announced June 2019.
-
Learning Physics-Based Manipulation in Clutter: Combining Image-Based Generalization and Look-Ahead Planning
Authors:
Wissam Bejjani,
Mehmet R. Dogar,
Matteo Leonetti
Abstract:
Physics-based manipulation in clutter involves complex interaction between multiple objects. In this paper, we consider the problem of learning, from interaction in a physics simulator, manipulation skills to solve this multi-step sequential decision making problem in the real world. Our approach has two key properties: (i) the ability to generalize and transfer manipulation skills (over the type,…
▽ More
Physics-based manipulation in clutter involves complex interaction between multiple objects. In this paper, we consider the problem of learning, from interaction in a physics simulator, manipulation skills to solve this multi-step sequential decision making problem in the real world. Our approach has two key properties: (i) the ability to generalize and transfer manipulation skills (over the type, shape, and number of objects in the scene) using an abstract image-based representation that enables a neural network to learn useful features; and (ii) the ability to perform look-ahead planning in the image space using a physics simulator, which is essential for such multi-step problems. We show, in sets of simulated and real-world experiments (video available on https://youtu.be/EmkUQfyvwkY), that by learning to evaluate actions in an abstract image-based representation of the real world, the robot can generalize and adapt to the object shapes in challenging real-world environments.
△ Less
Submitted 26 July, 2019; v1 submitted 3 April, 2019;
originally announced April 2019.
-
An Optimization Framework for Task Sequencing in Curriculum Learning
Authors:
Francesco Foglino,
Christiano Coletto Christakou,
Matteo Leonetti
Abstract:
Curriculum learning in reinforcement learning is used to shape exploration by presenting the agent with increasingly complex tasks. The idea of curriculum learning has been largely applied in both animal training and pedagogy. In reinforcement learning, all previous task sequencing methods have shaped exploration with the objective of reducing the time to reach a given performance level. We propos…
▽ More
Curriculum learning in reinforcement learning is used to shape exploration by presenting the agent with increasingly complex tasks. The idea of curriculum learning has been largely applied in both animal training and pedagogy. In reinforcement learning, all previous task sequencing methods have shaped exploration with the objective of reducing the time to reach a given performance level. We propose novel uses of curriculum learning, which arise from choosing different objective functions. Furthermore, we define a general optimization framework for task sequencing and evaluate the performance of popular metaheuristic search methods on several tasks. We show that curriculum learning can be successfully used to: improve the initial performance, take fewer suboptimal actions during exploration, and discover better policies.
△ Less
Submitted 13 June, 2019; v1 submitted 31 January, 2019;
originally announced January 2019.
-
Effect of dilution in asymmetric recurrent neural networks
Authors:
Viola Folli,
Giorgio Gosti,
Marco Leonetti,
Giancarlo Ruocco
Abstract:
We study with numerical simulation the possible limit behaviors of synchronous discrete-time deterministic recurrent neural networks composed of N binary neurons as a function of a network's level of dilution and asymmetry. The network dilution measures the fraction of neuron couples that are connected, and the network asymmetry measures to what extent the underlying connectivity matrix is asymmet…
▽ More
We study with numerical simulation the possible limit behaviors of synchronous discrete-time deterministic recurrent neural networks composed of N binary neurons as a function of a network's level of dilution and asymmetry. The network dilution measures the fraction of neuron couples that are connected, and the network asymmetry measures to what extent the underlying connectivity matrix is asymmetric. For each given neural network, we study the dynamical evolution of all the different initial conditions, thus characterizing the full dynamical landscape without imposing any learning rule. Because of the deterministic dynamics, each trajectory converges to an attractor, that can be either a fixed point or a limit cycle. These attractors form the set of all the possible limit behaviors of the neural network. For each network, we then determine the convergence times, the limit cycles' length, the number of attractors, and the sizes of the attractors' basin. We show that there are two network structures that maximize the number of possible limit behaviors. The first optimal network structure is fully-connected and symmetric. On the contrary, the second optimal network structure is highly sparse and asymmetric. The latter optimal is similar to what observed in different biological neuronal circuits. These observations lead us to hypothesize that independently from any given learning model, an efficient and effective biologic network that stores a number of limit behaviors close to its maximum capacity tends to develop a connectivity structure similar to one of the optimal networks we found.
△ Less
Submitted 10 May, 2018;
originally announced May 2018.
-
Planning with a Receding Horizon for Manipulation in Clutter using a Learned Value Function
Authors:
Wissam Bejjani,
Rafael Papallas,
Matteo Leonetti,
Mehmet R. Dogar
Abstract:
Manipulation in clutter requires solving complex sequential decision making problems in an environment rich with physical interactions. The transfer of motion planning solutions from simulation to the real world, in open-loop, suffers from the inherent uncertainty in modelling real world physics. We propose interleaving planning and execution in real-time, in a closed-loop setting, using a Recedin…
▽ More
Manipulation in clutter requires solving complex sequential decision making problems in an environment rich with physical interactions. The transfer of motion planning solutions from simulation to the real world, in open-loop, suffers from the inherent uncertainty in modelling real world physics. We propose interleaving planning and execution in real-time, in a closed-loop setting, using a Receding Horizon Planner (RHP) for pushing manipulation in clutter. In this context, we address the problem of finding a suitable value function based heuristic for efficient planning, and for estimating the cost-to-go from the horizon to the goal. We estimate such a value function first by using plans generated by an existing sampling-based planner. Then, we further optimize the value function through reinforcement learning. We evaluate our approach and compare it to state-of-the-art planning techniques for manipulation in clutter. We conduct experiments in simulation with artificially injected uncertainty on the physics parameters, as well as in real world tasks of manipulation in clutter. We show that this approach enables the robot to react to the uncertain dynamics of the real world effectively.
△ Less
Submitted 27 July, 2018; v1 submitted 21 March, 2018;
originally announced March 2018.
-
Design and Implementation of an IP based authentication mechanism for Open Source Proxy Servers in Interception Mode
Authors:
Tejaswi Agarwal,
Mike A. Leonetti
Abstract:
Proxy servers are being increasingly deployed at organizations for performance benefits; however, there still exists drawbacks in ease of client authentication in interception proxy mode mainly for Open Source Proxy Servers.
Technically, an interception mode is not designed for client authentication, but implementation in certain organizations does require this feature. In this paper, we focus o…
▽ More
Proxy servers are being increasingly deployed at organizations for performance benefits; however, there still exists drawbacks in ease of client authentication in interception proxy mode mainly for Open Source Proxy Servers.
Technically, an interception mode is not designed for client authentication, but implementation in certain organizations does require this feature. In this paper, we focus on the World Wide Web, highlight the existing transparent proxy authentication mechanisms, its drawbacks and propose an authentication scheme for transparent proxy users by using external scripts based on the clients Internet Protocol Address. This authentication mechanism has been implemented and verified on Squid-one of the most widely used HTTP Open Source Proxy Server.
△ Less
Submitted 17 February, 2013;
originally announced February 2013.
