-
Sharp inequalities for discrete and continuous multi-tiling, using the Bombieri-Siegel approach
Authors:
Michel Faleiros Martins,
Sinai Robins
Abstract:
Given a finite subset $F$ of integer points in $\mathbb Z^d$, it is of interest to seek conditions on $F$ that allow it to multi-tile $\mathbb Z^d$ by translations. In addition to the continuous multi-tiling results presented here, we also give analogous discrete applications to arithmetic combinatorics. Namely we give a discretized version of the Bombieri-Siegel formula, namely a finite sum of di…
▽ More
Given a finite subset $F$ of integer points in $\mathbb Z^d$, it is of interest to seek conditions on $F$ that allow it to multi-tile $\mathbb Z^d$ by translations. In addition to the continuous multi-tiling results presented here, we also give analogous discrete applications to arithmetic combinatorics. Namely we give a discretized version of the Bombieri-Siegel formula, namely a finite sum of discrete covariograms, taken over any finite set of integer points in $\mathbb R^d$. As a consequence, we arrive at a new equivalent condition for multi-tiling $\mathbb Z^d$ by translating $F$ with a fixed integer sublattice.
Similar questions related to convex bodies have already been investigated extensively. In order to develop lattice sums of the cross covariogram for any two bounded sets $A, B\subset \mathbb R^d$, we prove a refined continuous version of the classical Bombieri-Siegel formula from the geometry of numbers. To achieve this goal, we use a variant of the Poisson Summation formula, adapted for continuous functions of compact support.
As an application of this refined Bombieri-Siegel formula, a new characterization of multi-tilings of Euclidean space by translations of a compact set by using a lattice is given. A further consequence is a spectral formula for the volume of any bounded measurable set.
△ Less
Submitted 15 December, 2023; v1 submitted 18 April, 2022;
originally announced April 2022.
-
A Distributional View on Multi-Objective Policy Optimization
Authors:
Abbas Abdolmaleki,
Sandy H. Huang,
Leonard Hasenclever,
Michael Neunert,
H. Francis Song,
Martina Zambelli,
Murilo F. Martins,
Nicolas Heess,
Raia Hadsell,
Martin Riedmiller
Abstract:
Many real-world problems require trading off multiple competing objectives. However, these objectives are often in different units and/or scales, which can make it challenging for practitioners to express numerical preferences over objectives in their native units. In this paper we propose a novel algorithm for multi-objective reinforcement learning that enables setting desired preferences for obj…
▽ More
Many real-world problems require trading off multiple competing objectives. However, these objectives are often in different units and/or scales, which can make it challenging for practitioners to express numerical preferences over objectives in their native units. In this paper we propose a novel algorithm for multi-objective reinforcement learning that enables setting desired preferences for objectives in a scale-invariant way. We propose to learn an action distribution for each objective, and we use supervised learning to fit a parametric policy to a combination of these distributions. We demonstrate the effectiveness of our approach on challenging high-dimensional real and simulated robotics tasks, and show that setting different preferences in our framework allows us to trace out the space of nondominated solutions.
△ Less
Submitted 15 May, 2020;
originally announced May 2020.
-
Learning Gentle Object Manipulation with Curiosity-Driven Deep Reinforcement Learning
Authors:
Sandy H. Huang,
Martina Zambelli,
Jackie Kay,
Murilo F. Martins,
Yuval Tassa,
Patrick M. Pilarski,
Raia Hadsell
Abstract:
Robots must know how to be gentle when they need to interact with fragile objects, or when the robot itself is prone to wear and tear. We propose an approach that enables deep reinforcement learning to train policies that are gentle, both during exploration and task execution. In a reward-based learning environment, a natural approach involves augmenting the (task) reward with a penalty for non-ge…
▽ More
Robots must know how to be gentle when they need to interact with fragile objects, or when the robot itself is prone to wear and tear. We propose an approach that enables deep reinforcement learning to train policies that are gentle, both during exploration and task execution. In a reward-based learning environment, a natural approach involves augmenting the (task) reward with a penalty for non-gentleness, which can be defined as excessive impact force. However, augmenting with only this penalty impairs learning: policies get stuck in a local optimum which avoids all contact with the environment. Prior research has shown that combining auxiliary tasks or intrinsic rewards can be beneficial for stabilizing and accelerating learning in sparse-reward domains, and indeed we find that introducing a surprise-based intrinsic reward does avoid the no-contact failure case. However, we show that a simple dynamics-based surprise is not as effective as penalty-based surprise. Penalty-based surprise, based on predicting forceful contacts, has a further benefit: it encourages exploration which is contact-rich yet gentle. We demonstrate the effectiveness of the approach using a complex, tendon-powered robot hand with tactile sensors. Videos are available at http://sites.google.com/view/gentlemanipulation.
△ Less
Submitted 20 March, 2019;
originally announced March 2019.
-
Simultaneously Learning Vision and Feature-based Control Policies for Real-world Ball-in-a-Cup
Authors:
Devin Schwab,
Tobias Springenberg,
Murilo F. Martins,
Thomas Lampe,
Michael Neunert,
Abbas Abdolmaleki,
Tim Hertweck,
Roland Hafner,
Francesco Nori,
Martin Riedmiller
Abstract:
We present a method for fast training of vision based control policies on real robots. The key idea behind our method is to perform multi-task Reinforcement Learning with auxiliary tasks that differ not only in the reward to be optimized but also in the state-space in which they operate. In particular, we allow auxiliary task policies to utilize task features that are available only at training-ti…
▽ More
We present a method for fast training of vision based control policies on real robots. The key idea behind our method is to perform multi-task Reinforcement Learning with auxiliary tasks that differ not only in the reward to be optimized but also in the state-space in which they operate. In particular, we allow auxiliary task policies to utilize task features that are available only at training-time. This allows for fast learning of auxiliary policies, which subsequently generate good data for training the main, vision-based control policies. This method can be seen as an extension of the Scheduled Auxiliary Control (SAC-X) framework. We demonstrate the efficacy of our method by using both a simulated and real-world Ball-in-a-Cup game controlled by a robot arm. In simulation, our approach leads to significant learning speed-ups when compared to standard SAC-X. On the real robot we show that the task can be learned from-scratch, i.e., with no transfer from simulation and no imitation learning. Videos of our learned policies running on the real robot can be found at https://sites.google.com/view/rss-2019-sawyer-bic/.
△ Less
Submitted 18 February, 2019; v1 submitted 12 February, 2019;
originally announced February 2019.
