-
LLaMandement: Large Language Models for Summarization of French Legislative Proposals
Authors:
Joseph Gesnouin,
Yannis Tannier,
Christophe Gomes Da Silva,
Hatim Tapory,
Camille Brier,
Hugo Simon,
Raphael Rozenberg,
Hermann Woehrel,
Mehdi El Yakaabi,
Thomas Binder,
Guillaume Marie,
Emilie Caron,
Mathile Nogueira,
Thomas Fontas,
Laure Puydebois,
Marie Theophile,
Stephane Morandi,
Mael Petit,
David Creissac,
Pauline Ennouchy,
Elise Valetoux,
Celine Visade,
Severine Balloux,
Emmanuel Cortes,
Pierre-Etienne Devineau
, et al. (3 additional authors not shown)
Abstract:
This report introduces LLaMandement, a state-of-the-art Large Language Model, fine-tuned by the French government and designed to enhance the efficiency and efficacy of processing parliamentary sessions (including the production of bench memoranda and documents required for interministerial meetings) by generating neutral summaries of legislative proposals. Addressing the administrative challenges…
▽ More
This report introduces LLaMandement, a state-of-the-art Large Language Model, fine-tuned by the French government and designed to enhance the efficiency and efficacy of processing parliamentary sessions (including the production of bench memoranda and documents required for interministerial meetings) by generating neutral summaries of legislative proposals. Addressing the administrative challenges of manually processing a growing volume of legislative amendments, LLaMandement stands as a significant legal technological milestone, providing a solution that exceeds the scalability of traditional human efforts while matching the robustness of a specialized legal drafter. We release all our fine-tuned models and training data to the community.
△ Less
Submitted 29 January, 2024;
originally announced January 2024.
-
A finite sample analysis of the benign overfitting phenomenon for ridge function estimation
Authors:
Emmanuel Caron,
Stephane Chretien
Abstract:
Recent extensive numerical experiments in high scale machine learning have allowed to uncover a quite counterintuitive phase transition, as a function of the ratio between the sample size and the number of parameters in the model. As the number of parameters $p$ approaches the sample size $n$, the generalisation error increases, but surprisingly, it starts decreasing again past the threshold…
▽ More
Recent extensive numerical experiments in high scale machine learning have allowed to uncover a quite counterintuitive phase transition, as a function of the ratio between the sample size and the number of parameters in the model. As the number of parameters $p$ approaches the sample size $n$, the generalisation error increases, but surprisingly, it starts decreasing again past the threshold $p=n$. This phenomenon, brought to the theoretical community attention in \cite{belkin2019reconciling}, has been thoroughly investigated lately, more specifically for simpler models than deep neural networks, such as the linear model when the parameter is taken to be the minimum norm solution to the least-squares problem, firstly in the asymptotic regime when $p$ and $n$ tend to infinity, see e.g. \cite{hastie2019surprises}, and recently in the finite dimensional regime and more specifically for linear models \cite{bartlett2020benign}, \cite{tsigler2020benign}, \cite{lecue2022geometrical}. In the present paper, we propose a finite sample analysis of non-linear models of \textit{ridge} type, where we investigate the \textit{overparametrised regime} of the double descent phenomenon for both the \textit{estimation problem} and the \textit{prediction} problem. Our results provide a precise analysis of the distance of the best estimator from the true parameter as well as a generalisation bound which complements recent works of \cite{bartlett2020benign} and \cite{chinot2020benign}. Our analysis is based on tools closely related to the continuous Newton method \cite{neuberger2007continuous} and a refined quantitative analysis of the performance in prediction of the minimum $\ell_2$-norm solution.
△ Less
Submitted 12 January, 2024; v1 submitted 25 July, 2020;
originally announced July 2020.
-
Gaussian linear model selection in a dependent context
Authors:
Emmanuel Caron,
Jérôme Dedecker,
Bertrand Michel
Abstract:
In this paper, we study the nonparametric linear model, when the error process is a dependent Gaussian process. We focus on the estimation of the mean vector via a model selection approach. We first give the general theoretical form of the penalty function, ensuring that the penalized estimator among a collection of models satisfies an oracle inequality. Then we derive a penalty shape involving th…
▽ More
In this paper, we study the nonparametric linear model, when the error process is a dependent Gaussian process. We focus on the estimation of the mean vector via a model selection approach. We first give the general theoretical form of the penalty function, ensuring that the penalized estimator among a collection of models satisfies an oracle inequality. Then we derive a penalty shape involving the spectral radius of the covariance matrix of the errors, which can be chosen proportional to the dimension when the error process is stationary and short range dependent. However, this penalty can be too rough in some cases, in particular when the error process is long range dependent. In a second part, we focus on the fixed-design regression model assuming that the error process is a stationary Gaussian process. We propose a model selection procedure in order to estimate the mean function via piecewise polynomials on a regular partition, when the error process is either short range dependent, long range dependent or anti-persistent. We present different kinds of penalties, depending on the memory of the process. For each case, an adaptive estimator is built, and the rates of convergence are computed. Thanks to several sets of simulations, we study the performance of these different penalties for all types of errors (short memory, long memory and anti-persistent errors). Finally, we give an application of our method to the well-known Nile data, which clearly shows that the type of dependence of the error process must be taken into account.
△ Less
Submitted 3 May, 2020;
originally announced May 2020.
-
Linear regression with stationary errors : the R package slm
Authors:
Emmanuel Caron,
Jérôme Dedecker,
Bertrand Michel
Abstract:
This paper introduces the R package slm which stands for Stationary Linear Models. The package contains a set of statistical procedures for linear regression in the general context where the error process is strictly stationary with short memory. We work in the setting of Hannan (1973), who proved the asymptotic normality of the (normalized) least squares estimators (LSE) under very mild condition…
▽ More
This paper introduces the R package slm which stands for Stationary Linear Models. The package contains a set of statistical procedures for linear regression in the general context where the error process is strictly stationary with short memory. We work in the setting of Hannan (1973), who proved the asymptotic normality of the (normalized) least squares estimators (LSE) under very mild conditions on the error process. We propose different ways to estimate the asymptotic covariance matrix of the LSE, and then to correct the type I error rates of the usual tests on the parameters (as well as confidence intervals). The procedures are evaluated through different sets of simulations, and two examples of real datasets are studied.
△ Less
Submitted 22 October, 2019; v1 submitted 15 June, 2019;
originally announced June 2019.
-
Asymptotic distribution of least square estimators for linear models with dependent errors
Authors:
Emmanuel Caron
Abstract:
In this paper, we consider the usual linear regression model in the case where the error process is assumed strictly stationary. We use a result from Hannan (1973), who proved a Central Limit Theorem for the usual least square estimator under general conditions on the design and on the error process. Whatever the design satisfying Hannan's conditions, we define an estimator of the covariance matri…
▽ More
In this paper, we consider the usual linear regression model in the case where the error process is assumed strictly stationary. We use a result from Hannan (1973), who proved a Central Limit Theorem for the usual least square estimator under general conditions on the design and on the error process. Whatever the design satisfying Hannan's conditions, we define an estimator of the covariance matrix and we prove its consistency under very mild conditions. As an application, we show how to modify the usual tests on the linear model in this dependent context, in such a way that the type-I error rate remains asymptotically correct, and we illustrate the performance of this procedure through different sets of simulations.
△ Less
Submitted 15 June, 2019; v1 submitted 13 June, 2018;
originally announced June 2018.
-
Asymptotic distribution of least squares estimators for linear models with dependent errors : regular designs
Authors:
Emmanuel Caron,
Sophie Dede
Abstract:
In this paper, we consider the usual linear regression model in the case where the error process is assumed strictly stationary. We use a result from Hannan, who proved a Central Limit Theorem for the usual least squares estimator under general conditions on the design and on the error process. We show that for a large class of designs, the asymptotic covariance matrix is as simple as the independ…
▽ More
In this paper, we consider the usual linear regression model in the case where the error process is assumed strictly stationary. We use a result from Hannan, who proved a Central Limit Theorem for the usual least squares estimator under general conditions on the design and on the error process. We show that for a large class of designs, the asymptotic covariance matrix is as simple as the independent and identically distributed case. We then estimate the covariance matrix using an estimator of the spectral density whose consistency is proved under very mild conditions.
△ Less
Submitted 15 June, 2019; v1 submitted 16 October, 2017;
originally announced October 2017.
-
Mixture model for designs in high dimensional regression and the LASSO
Authors:
Mohamed Ibrahim Assoweh,
Emmanuel Caron,
Stéphane Chrétien
Abstract:
The LASSO is a recent technique for variable selection in the regression model \bean y & = & Xβ+ z, \eean where $X\in \R^{n\times p}$ and $z$ is a centered gaussian i.i.d. noise vector $\mathcal N(0,σ^2I)$. The LASSO has been proved to achieve remarkable properties such as exact support recovery of sparse vectors when the columns are sufficently incoherent and low prediction error under even less…
▽ More
The LASSO is a recent technique for variable selection in the regression model \bean y & = & Xβ+ z, \eean where $X\in \R^{n\times p}$ and $z$ is a centered gaussian i.i.d. noise vector $\mathcal N(0,σ^2I)$. The LASSO has been proved to achieve remarkable properties such as exact support recovery of sparse vectors when the columns are sufficently incoherent and low prediction error under even less stringent conditions. However, many matrices do not satisfy small coherence in practical applications and the LASSO estimator may thus suffer from what is known as the slow rate regime.
The goal of the present paper is to study the LASSO from a slightly different perspective by proposing a mixture model for the design matrix which is able to capture in a natural way the potentially clustered nature of the columns in many practical situations. In this model, the columns of the design matrix are drawn from a Gaussian mixture model. Instead of requiring incoherence for the design matrix $X$, we only require incoherence of the much smaller matrix of the mixture's centers.
Our main result states that $Xβ$ can be estimated with the same precision as for incoherent designs except for a correction term depending on the maximal variance in the mixture model.
△ Less
Submitted 19 December, 2023; v1 submitted 17 October, 2012;
originally announced October 2012.
-
Optimization in a Self-Stabilizing Service Discovery Framework for Large Scale Systems
Authors:
Eddy Caron,
Florent Chuffart,
Anissa Lamani,
Franck Petit
Abstract:
Ability to find and get services is a key requirement in the development of large-scale distributed sys- tems. We consider dynamic and unstable environments, namely Peer-to-Peer (P2P) systems. In previous work, we designed a service discovery solution called Distributed Lexicographic Placement Table (DLPT), based on a hierar- chical overlay structure. A self-stabilizing version was given using the…
▽ More
Ability to find and get services is a key requirement in the development of large-scale distributed sys- tems. We consider dynamic and unstable environments, namely Peer-to-Peer (P2P) systems. In previous work, we designed a service discovery solution called Distributed Lexicographic Placement Table (DLPT), based on a hierar- chical overlay structure. A self-stabilizing version was given using the Propagation of Information with Feedback (PIF) paradigm. In this paper, we introduce the self-stabilizing COPIF (for Collaborative PIF) scheme. An algo- rithm is provided with its correctness proof. We use this approach to improve a distributed P2P framework designed for the services discovery. Significantly efficient experimental results are presented.
△ Less
Submitted 5 July, 2012;
originally announced July 2012.
-
Cosmological Simulations on a Grid of Computers
Authors:
Benjamin Depardon,
Eddy Caron,
Frédéric Desprez,
Jérémy Blaizot,
Hélène M. Courtois
Abstract:
The work presented in this paper aims at restricting the input parameter values of the semi-analytical model used in GALICS and MOMAF, so as to derive which parameters influence the most the results, e.g., star formation, feedback and halo recycling efficiencies, etc. Our approach is to proceed empirically: we run lots of simulations and derive the correct ranges of values. The computation time ne…
▽ More
The work presented in this paper aims at restricting the input parameter values of the semi-analytical model used in GALICS and MOMAF, so as to derive which parameters influence the most the results, e.g., star formation, feedback and halo recycling efficiencies, etc. Our approach is to proceed empirically: we run lots of simulations and derive the correct ranges of values. The computation time needed is so large, that we need to run on a grid of computers. Hence, we model GALICS and MOMAF execution time and output files size, and run the simulation using a grid middleware: DIET. All the complexity of accessing resources, scheduling simulations and managing data is harnessed by DIET and hidden behind a web portal accessible to the users.
△ Less
Submitted 21 May, 2010;
originally announced May 2010.
-
An Intuitive Automated Modelling Interface for Systems Biology
Authors:
Ozan Kahramanoğullari,
Luca Cardelli,
Emmanuelle Caron
Abstract:
We introduce a natural language interface for building stochastic pi calculus models of biological systems. In this language, complex constructs describing biochemical events are built from basic primitives of association, dissociation and transformation. This language thus allows us to model biochemical systems modularly by describing their dynamics in a narrative-style language, while making a…
▽ More
We introduce a natural language interface for building stochastic pi calculus models of biological systems. In this language, complex constructs describing biochemical events are built from basic primitives of association, dissociation and transformation. This language thus allows us to model biochemical systems modularly by describing their dynamics in a narrative-style language, while making amendments, refinements and extensions on the models easy. We demonstrate the language on a model of Fc-gamma receptor phosphorylation during phagocytosis. We provide a tool implementation of the translation into a stochastic pi calculus language, Microsoft Research's SPiM.
△ Less
Submitted 12 November, 2009;
originally announced November 2009.
-
Cosmological Simulations using Grid Middleware
Authors:
Y. Caniou,
E. Caron,
B. Depardon,
H. Courtois,
R. Teyssier
Abstract:
One way to access the aggregated power of a collection of heterogeneous machines is to use a grid middleware, such as DIET, GridSolve or NINF. It addresses the problem of monitoring the resources, of handling the submissions of jobs and as an example the inherent transfer of input and output data, in place of the user.
In this paper we present how to run cosmological simulations using the RAMS…
▽ More
One way to access the aggregated power of a collection of heterogeneous machines is to use a grid middleware, such as DIET, GridSolve or NINF. It addresses the problem of monitoring the resources, of handling the submissions of jobs and as an example the inherent transfer of input and output data, in place of the user.
In this paper we present how to run cosmological simulations using the RAMSES application along with the DIET middleware. We will describe how to write the corresponding DIET client and server. The remainder of the paper is organized as follows: Section 2 presents the DIET middleware. Section 3 describes the RAMSES cosmological software and simulations, and how to interface it with DIET. We show how to write a client and a server in Section 4. Finally, Section 5 presents the experiments realized on Grid'5000, the French Research Grid, and we conclude in Section 6.
△ Less
Submitted 27 November, 2006;
originally announced November 2006.
