-
DEAR: Disentangled Environment and Agent Representations for Reinforcement Learning without Reconstruction
Authors:
Ameya Pore,
Riccardo Muradore,
Diego Dall'Alba
Abstract:
Reinforcement Learning (RL) algorithms can learn robotic control tasks from visual observations, but they often require a large amount of data, especially when the visual scene is complex and unstructured. In this paper, we explore how the agent's knowledge of its shape can improve the sample efficiency of visual RL methods. We propose a novel method, Disentangled Environment and Agent Representat…
▽ More
Reinforcement Learning (RL) algorithms can learn robotic control tasks from visual observations, but they often require a large amount of data, especially when the visual scene is complex and unstructured. In this paper, we explore how the agent's knowledge of its shape can improve the sample efficiency of visual RL methods. We propose a novel method, Disentangled Environment and Agent Representations (DEAR), that uses the segmentation mask of the agent as supervision to learn disentangled representations of the environment and the agent through feature separation constraints. Unlike previous approaches, DEAR does not require reconstruction of visual observations. These representations are then used as an auxiliary loss to the RL objective, encouraging the agent to focus on the relevant features of the environment. We evaluate DEAR on two challenging benchmarks: Distracting DeepMind control suite and Franka Kitchen manipulation tasks. Our findings demonstrate that DEAR surpasses state-of-the-art methods in sample efficiency, achieving comparable or superior performance with reduced parameters. Our results indicate that integrating agent knowledge into visual RL methods has the potential to enhance their learning efficiency and robustness.
△ Less
Submitted 30 June, 2024;
originally announced July 2024.
-
Contactless actuators and pyramid wavefront sensor, the SPLATT concept for space active optics: an overview of the project and the last laboratory results
Authors:
Runa Briguglio,
Marco Xompero,
Marcello Scalera,
Marco Riva,
Ciro Del Vecchio,
Luca Carbonaro,
Carmelo Arcidiacono,
Guido Agapito,
Enrico Pinna,
Alessandro Terreri,
Fernando Pedichini,
Riccardo Muradore,
Matteo Tintori,
Daniele Gallieni Roberto Biasi,
Christian Patauner,
Alessandro Zuccaro Marchi
Abstract:
In the last few years the concept of an active space telescope has been greatly developed, to meet demanding requirements with a substantial reduction of tolerances, risks and costs. This is the frame of the LATT project (an ESA TRP) and its follow-up SPLATT (an INAF funded R&D project). Within the SPLATT activities, we outline a novel approach and investigate, both via simulations and in the opti…
▽ More
In the last few years the concept of an active space telescope has been greatly developed, to meet demanding requirements with a substantial reduction of tolerances, risks and costs. This is the frame of the LATT project (an ESA TRP) and its follow-up SPLATT (an INAF funded R&D project). Within the SPLATT activities, we outline a novel approach and investigate, both via simulations and in the optical laboratory, two main elements: an active segmented primary with contactless actuators and a pyramid wavefront sensor (PWFS) to drive the correction chain. The key point is the synergy between them: the sensitivity of the PWFS and the intrinsic stability of a contactless-actuated mirror segment.
Voice-coil, contactless actuators are in facts a natural decoupling layer between the payload and the optical surface and can suppress the high frequency vibration as we verified in the lab. We subjected a 40 cm diameter prototype with 19 actuators to an externally injected vibration spectrum; we then measured optically the reduction of vibrations when the optical surface is floating controlled by the actuators, thus validating the concept at the first stage of the design.
The PWFS, which is largely adopted on ground-based telescope, is a pupil-conjugated sensor and offers a user-selectable sampling and capture range, in order to match different use cases; it is also more sensitive than Shack-Hartmann sensor especially at the low-mid spatial scales. We run a set of numerical simulations with the PWFS measuring the misalignment and phase steps of a JWST-like primary mirrors: we investigated the PWFS sensitivity in the sub-nanometer regime in presence of photon and detector noise, and with guide star magnitudes in the range 8 to 14.
In the paper we discuss the outcomes of the project and present a possible roadmap for further developments.
△ Less
Submitted 1 February, 2023;
originally announced February 2023.
-
Weak energy sha** for stochastic controlled port-Hamiltonian systems
Authors:
Francesco G. Cordoni,
Luca Di Persio,
Riccardo Muradore
Abstract:
The present work address the problem of energy sha** for stochastic port-Hamiltonian system. Energy sha** is a powerful technique that allows to systematically find feedback law to shape the Hamiltonian of a controlled system so that, under a general passivity condition, it converges or tracks a desired configuration. Energy sha** has been recently generalized to consider stochastic port-Ham…
▽ More
The present work address the problem of energy sha** for stochastic port-Hamiltonian system. Energy sha** is a powerful technique that allows to systematically find feedback law to shape the Hamiltonian of a controlled system so that, under a general passivity condition, it converges or tracks a desired configuration. Energy sha** has been recently generalized to consider stochastic port-Hamiltonian system. Nonetheless the resulting theory presents several limitation in the application so that relevant examples, such as the additive noise case, are immediately ruled out from the possible application of energy sha**. The current paper continues the investigation of the properties of a weak notion of passivity for a stochastic system and a consequent weak notion of convergence for the shaped system considered recently by the authors. Such weak notion of passivity is strictly related to the existence and uniqueness of an invariant measure for the system so that the theory developed has a purely probabilistic flavour. We will show how all the relevant results of energy sha** can be recover under the weak setting developed. We will also show how the weak passivity setting considered draw an insightful connection between stochastic port-Hamiltonian systems and infinite-dimensional port-Hamiltonian system.
△ Less
Submitted 17 February, 2022;
originally announced February 2022.
-
The SARAS Endoscopic Surgeon Action Detection (ESAD) dataset: Challenges and methods
Authors:
Vivek Singh Bawa,
Gurkirt Singh,
Francis Ka**A,
Inna Skarga-Bandurova,
Elettra Oleari,
Alice Leporini,
Carmela Landolfo,
Pengfei Zhao,
Xi Xiang,
Gongning Luo,
Kuanquan Wang,
Liangzhi Li,
Bowen Wang,
Shang Zhao,
Li Li,
Armando Stabile,
Francesco Setti,
Riccardo Muradore,
Fabio Cuzzolin
Abstract:
For an autonomous robotic system, monitoring surgeon actions and assisting the main surgeon during a procedure can be very challenging. The challenges come from the peculiar structure of the surgical scene, the greater similarity in appearance of actions performed via tools in a cavity compared to, say, human actions in unconstrained environments, as well as from the motion of the endoscopic camer…
▽ More
For an autonomous robotic system, monitoring surgeon actions and assisting the main surgeon during a procedure can be very challenging. The challenges come from the peculiar structure of the surgical scene, the greater similarity in appearance of actions performed via tools in a cavity compared to, say, human actions in unconstrained environments, as well as from the motion of the endoscopic camera. This paper presents ESAD, the first large-scale dataset designed to tackle the problem of surgeon action detection in endoscopic minimally invasive surgery. ESAD aims at contributing to increase the effectiveness and reliability of surgical assistant robots by realistically testing their awareness of the actions performed by a surgeon. The dataset provides bounding box annotation for 21 action classes on real endoscopic video frames captured during prostatectomy, and was used as the basis of a recent MIDL 2020 challenge. We also present an analysis of the dataset conducted using the baseline model which was released as part of the challenge, and a description of the top performing models submitted to the challenge together with the results they obtained. This study provides significant insight into what approaches can be effective and can be extended further. We believe that ESAD will serve in the future as a useful benchmark for all researchers active in surgeon action detection and assistive robotics at large.
△ Less
Submitted 7 April, 2021;
originally announced April 2021.
-
Minimal controllability time for systems with nonlinear drift under a compact convex state constraint
Authors:
Viktor Bezborodov,
Luca Di Persio,
Riccardo Muradore
Abstract:
In this paper we estimate the minimal controllability time for a class of non-linear control systems with a bounded convex state constraint. An explicit expression is given for the controllability time if the image of the control matrix is of co-dimension one. A lower bound for the controllability time is given in the general case. The technique is based on finding a lower dimension system with th…
▽ More
In this paper we estimate the minimal controllability time for a class of non-linear control systems with a bounded convex state constraint. An explicit expression is given for the controllability time if the image of the control matrix is of co-dimension one. A lower bound for the controllability time is given in the general case. The technique is based on finding a lower dimension system with the similar controllability properties as the original system. The controls corresponding to the minimal time, or time close to the minimal one, are discussed and computed analytically. The effectiveness of the proposed approach is illustrated by a few examples.
△ Less
Submitted 11 June, 2023; v1 submitted 23 December, 2020;
originally announced December 2020.
-
Improving rigid 3D calibration for robotic surgery
Authors:
Andrea Roberti,
Nicola Piccinelli,
Daniele Meli,
Riccardo Muradore,
Paolo Fiorini
Abstract:
Autonomy is the frontier of research in robotic surgery and its aim is to improve the quality of surgical procedures in the next future. One fundamental requirement for autonomy is advanced perception capability through vision sensors. In this paper, we propose a novel calibration technique for a surgical scenario with da Vinci robot. Calibration of the camera and the robot is necessary for precis…
▽ More
Autonomy is the frontier of research in robotic surgery and its aim is to improve the quality of surgical procedures in the next future. One fundamental requirement for autonomy is advanced perception capability through vision sensors. In this paper, we propose a novel calibration technique for a surgical scenario with da Vinci robot. Calibration of the camera and the robot is necessary for precise positioning of the tools in order to emulate the high performance surgeons. Our calibration technique is tailored for RGB-D camera. Different tests performed on relevant use cases for surgery prove that we significantly improve precision and accuracy with respect to the state of the art solutions for similar devices on a surgical-size setup. Moreover, our calibration method can be easily extended to standard surgical endoscope to prompt its use in real surgical scenario.
△ Less
Submitted 16 July, 2020;
originally announced July 2020.
-
ESAD: Endoscopic Surgeon Action Detection Dataset
Authors:
Vivek Singh Bawa,
Gurkirt Singh,
Francis Ka**A,
Inna Skarga-Bandurova,
Alice Leporini,
Carmela Landolfo,
Armando Stabile,
Francesco Setti,
Riccardo Muradore,
Elettra Oleari,
Fabio Cuzzolin
Abstract:
In this work, we take aim towards increasing the effectiveness of surgical assistant robots. We intended to make assistant robots safer by making them aware about the actions of surgeon, so it can take appropriate assisting actions. In other words, we aim to solve the problem of surgeon action detection in endoscopic videos. To this, we introduce a challenging dataset for surgeon action detection…
▽ More
In this work, we take aim towards increasing the effectiveness of surgical assistant robots. We intended to make assistant robots safer by making them aware about the actions of surgeon, so it can take appropriate assisting actions. In other words, we aim to solve the problem of surgeon action detection in endoscopic videos. To this, we introduce a challenging dataset for surgeon action detection in real-world endoscopic videos. Action classes are picked based on the feedback of surgeons and annotated by medical professional. Given a video frame, we draw bounding box around surgical tool which is performing action and label it with action label. Finally, we presenta frame-level action detection baseline model based on recent advances in ob-ject detection. Results on our new dataset show that our presented dataset provides enough interesting challenges for future method and it can serveas strong benchmark corresponding research in surgeon action detection in endoscopic videos.
△ Less
Submitted 12 June, 2020;
originally announced June 2020.
-
Stochastic port--Hamiltonian systems
Authors:
Francesco Cordoni,
Luca Di Persio,
Riccardo Muradore
Abstract:
In the present work we formally extend the theory of port-Hamiltonian systems to include random perturbations. In particular, suitably choosing the space of flow and effort variables we will show how several elements coming from possibly different physical domains can be interconnected in order to describe a dynamic system perturbed by general continuous semimartingale. Relevant enough, the noise…
▽ More
In the present work we formally extend the theory of port-Hamiltonian systems to include random perturbations. In particular, suitably choosing the space of flow and effort variables we will show how several elements coming from possibly different physical domains can be interconnected in order to describe a dynamic system perturbed by general continuous semimartingale. Relevant enough, the noise does not enter into the system solely as an external random perturbation, since each port is itself intrinsically stochastic. Coherently to the classical deterministic setting, we will show how such an approach extends existing literature of stochastic Hamiltonian systems on pseudo-Poisson and pre-symplectic manifolds. Moreover, we will prove that a power-preserving interconnection of stochastic port-Hamiltonian systems is a stochastic port-Hamiltonian system as well.
△ Less
Submitted 11 May, 2022; v1 submitted 4 October, 2019;
originally announced October 2019.
-
A Formal Approach to Cyber-Physical Attacks
Authors:
Ruggero Lanotte,
Massimo Merro,
Riccardo Muradore,
Luca Viganò
Abstract:
We apply formal methods to lay and streamline theoretical foundations to reason about Cyber-Physical Systems (CPSs) and cyber-physical attacks. We focus on %a formal treatment of both integrity and DoS attacks to sensors and actuators of CPSs, and on the timing aspects of these attacks. Our contributions are threefold: (1) we define a hybrid process calculus to model both CPSs and cyber-physical a…
▽ More
We apply formal methods to lay and streamline theoretical foundations to reason about Cyber-Physical Systems (CPSs) and cyber-physical attacks. We focus on %a formal treatment of both integrity and DoS attacks to sensors and actuators of CPSs, and on the timing aspects of these attacks. Our contributions are threefold: (1) we define a hybrid process calculus to model both CPSs and cyber-physical attacks; (2) we define a threat model of cyber-physical attacks and provide the means to assess attack tolerance/vulnerability with respect to a given attack; (3) we formalise how to estimate the impact of a successful attack on a CPS and investigate possible quantifications of the success chances of an attack. We illustrate definitions and results by means of a non-trivial engineering application.
△ Less
Submitted 21 April, 2017; v1 submitted 4 November, 2016;
originally announced November 2016.
-
Inertial Parameter Identification Including Friction and Motor Dynamics
Authors:
Silvio Traversaro,
Andrea Del Prete,
Riccardo Muradore,
Lorenzo Natale,
Francesco Nori
Abstract:
Identification of inertial parameters is fundamental for the implementation of torque-based control in humanoids. At the same time, good models of friction and actuator dynamics are critical for the low-level control of joint torques. We propose a novel method to identify inertial, friction and motor parameters in a single procedure. The identification exploits the measurements of the PWM of the D…
▽ More
Identification of inertial parameters is fundamental for the implementation of torque-based control in humanoids. At the same time, good models of friction and actuator dynamics are critical for the low-level control of joint torques. We propose a novel method to identify inertial, friction and motor parameters in a single procedure. The identification exploits the measurements of the PWM of the DC motors and a 6-axis force/torque sensor mounted inside the kinematic chain. The partial least-square (PLS) method is used to perform the regression. We identified the inertial, friction and motor parameters of the right arm of the iCub humanoid robot. We verified that the identified model can accurately predict the force/torque sensor measurements and the motor voltages. Moreover, we compared the identified parameters against the CAD parameters, in the prediction of the force/torque sensor measurements. Finally, we showed that the estimated model can effectively detect external contacts, comparing it against a tactile-based contact detection. The presented approach offers some advantages with respect to other state-of-the-art methods, because of its completeness (i.e. it identifies inertial, friction and motor parameters) and simplicity (only one data collection, with no particular requirements).
△ Less
Submitted 16 October, 2014;
originally announced October 2014.
