-
Tournesol: A quest for a large, secure and trustworthy database of reliable human judgments
Authors:
Lê-Nguyên Hoang,
Louis Faucon,
Aidan Jungo,
Sergei Volodin,
Dalia Papuc,
Orfeas Liossatos,
Ben Crulis,
Mariame Tighanimine,
Isabela Constantin,
Anastasiia Kucherenko,
Alexandre Maurer,
Felix Grimberg,
Vlad Nitu,
Chris Vossen,
Sébastien Rouault,
El-Mahdi El-Mhamdi
Abstract:
Today's large-scale algorithms have become immensely influential, as they recommend and moderate the content that billions of humans are exposed to on a daily basis. They are the de-facto regulators of our societies' information diet, from sha** opinions on public health to organizing groups for social movements. This creates serious concerns, but also great opportunities to promote quality info…
▽ More
Today's large-scale algorithms have become immensely influential, as they recommend and moderate the content that billions of humans are exposed to on a daily basis. They are the de-facto regulators of our societies' information diet, from sha** opinions on public health to organizing groups for social movements. This creates serious concerns, but also great opportunities to promote quality information. Addressing the concerns and seizing the opportunities is a challenging, enormous and fabulous endeavor, as intuitively appealing ideas often come with unwanted {\it side effects}, and as it requires us to think about what we deeply prefer.
Understanding how today's large-scale algorithms are built is critical to determine what interventions will be most effective. Given that these algorithms rely heavily on {\it machine learning}, we make the following key observation: \emph{any algorithm trained on uncontrolled data must not be trusted}. Indeed, a malicious entity could take control over the data, poison it with dangerously manipulative fabricated inputs, and thereby make the trained algorithm extremely unsafe. We thus argue that the first step towards safe and ethical large-scale algorithms must be the collection of a large, secure and trustworthy dataset of reliable human judgments.
To achieve this, we introduce \emph{Tournesol}, an open source platform available at \url{https://tournesol.app}. Tournesol aims to collect a large database of human judgments on what algorithms ought to widely recommend (and what they ought to stop widely recommending). We outline the structure of the Tournesol database, the key features of the Tournesol platform and the main hurdles that must be overcome to make it a successful project. Most importantly, we argue that, if successful, Tournesol may then serve as the essential foundation for any safe and ethical large-scale algorithm.
△ Less
Submitted 29 May, 2021;
originally announced July 2021.
-
Resolving Spurious Correlations in Causal Models of Environments via Interventions
Authors:
Sergei Volodin,
Nevan Wichers,
Jeremy Nixon
Abstract:
Causal models bring many benefits to decision-making systems (or agents) by making them interpretable, sample-efficient, and robust to changes in the input distribution. However, spurious correlations can lead to wrong causal models and predictions. We consider the problem of inferring a causal model of a reinforcement learning environment and we propose a method to deal with spurious correlations…
▽ More
Causal models bring many benefits to decision-making systems (or agents) by making them interpretable, sample-efficient, and robust to changes in the input distribution. However, spurious correlations can lead to wrong causal models and predictions. We consider the problem of inferring a causal model of a reinforcement learning environment and we propose a method to deal with spurious correlations. Specifically, our method designs a reward function that incentivizes an agent to do an intervention to find errors in the causal model. The data obtained from doing the intervention is used to improve the causal model. We propose several intervention design methods and compare them. The experimental results in a grid-world environment show that our approach leads to better causal models compared to baselines: learning the model on data from a random policy or a policy trained on the environment's reward. The main contribution consists of methods to design interventions to resolve spurious correlations.
△ Less
Submitted 7 December, 2020; v1 submitted 12 February, 2020;
originally announced February 2020.
-
The Probabilistic Fault Tolerance of Neural Networks in the Continuous Limit
Authors:
El-Mahdi El-Mhamdi,
Rachid Guerraoui,
Andrei Kucharavy,
Sergei Volodin
Abstract:
The loss of a few neurons in a brain rarely results in any visible loss of function. However, the insight into what "few" means in this context is unclear. How many random neuron failures will it take to lead to a visible loss of function? In this paper, we address the fundamental question of the impact of the crash of a random subset of neurons on the overall computation of a neural network and t…
▽ More
The loss of a few neurons in a brain rarely results in any visible loss of function. However, the insight into what "few" means in this context is unclear. How many random neuron failures will it take to lead to a visible loss of function? In this paper, we address the fundamental question of the impact of the crash of a random subset of neurons on the overall computation of a neural network and the error in the output it produces. We study fault tolerance of neural networks subject to small random neuron/weight crash failures in a probabilistic setting. We give provable guarantees on the robustness of the network to these crashes. Our main contribution is a bound on the error in the output of a network under small random Bernoulli crashes proved by using a Taylor expansion in the continuous limit, where close-by neurons at a layer are similar. The failure mode we adopt in our model is characteristic of neuromorphic hardware, a promising technology to speed up artificial neural networks, as well as of biological networks. We show that our theoretical bounds can be used to compare the fault tolerance of different architectures and to design a regularizer improving the fault tolerance of a given architecture. We design an algorithm achieving fault tolerance using a reasonable number of neurons. In addition to the theoretical proof, we also provide experimental validation of our results and suggest a connection to the generalization capacity problem.
△ Less
Submitted 25 September, 2019; v1 submitted 5 February, 2019;
originally announced February 2019.
-
Geometry of quadratic maps via convex relaxation
Authors:
Anatoly Dymarsky,
Elena Gryazina,
Sergei Volodin,
Boris Polyak
Abstract:
We consider several basic questions pertaining to the geometry of image of a general quadratic map. In general the image of a quadratic map is non-convex, although there are several known classes of quadratic maps when the image is convex. Remarkably, even when the image is not convex it often exhibits hidden convexity: a surprising efficiency of convex relaxation to address various geometric ques…
▽ More
We consider several basic questions pertaining to the geometry of image of a general quadratic map. In general the image of a quadratic map is non-convex, although there are several known classes of quadratic maps when the image is convex. Remarkably, even when the image is not convex it often exhibits hidden convexity: a surprising efficiency of convex relaxation to address various geometric questions by reformulating them in terms of convex optimization problems. In this paper we employ this strategy and put forward several algorithms that solve the following problems pertaining to the image: verify if a given point does not belong to the image; find the boundary point of the image lying in a particular direction; stochastically check if the image is convex, and if it is not, find a maximal convex subset of the image. Proposed algorithms are implemented in the form of an open-source MATLAB library CAQM, which accompanies the paper. Our results can be used for various problems of discrete optimization, uncertainty analysis, physical applications, and study of power flow equations.
△ Less
Submitted 1 October, 2018;
originally announced October 2018.
-
Simulation of atomic structure in the neighbourhood of nanovoids in bcc Fe
Authors:
A. V. Nazarov,
I. V. Ershova,
Y. S. Volodin
Abstract:
Generally displacement fields in the vicinity of voids were determined by the equations of theory of elasticity. Such a description has its disadvantages as it does not take into account the discrete atomic structure of materials and it should be expected that atom displacements in the vicinity of nanovoids should significantly differ from the predictions of mentioned theory. In our recent works a…
▽ More
Generally displacement fields in the vicinity of voids were determined by the equations of theory of elasticity. Such a description has its disadvantages as it does not take into account the discrete atomic structure of materials and it should be expected that atom displacements in the vicinity of nanovoids should significantly differ from the predictions of mentioned theory. In our recent works a new variant of Molecular Static method was developed. In particular in this model an iterative procedure is used in which the atomic structure in the vicinity of defect and constant, determining the displacement of atoms embedded into an elastic continuum, are obtained in a self-consistent manner. In this work we use our model for investigation of the atomic structure in the vicinity of nanovoids of different sizes. Results show that the displacements are significantly different for variant crystallographic directions and these differences are particularly large in bcc metals.
△ Less
Submitted 15 April, 2017;
originally announced April 2017.
