Showing 1–9 of 9 results for author: Garatti, S

Compression, Generalization and Learning

Authors: Marco C. Campi, Simone Garatti

Abstract: A compression function is a map that slims down an observational set into a subset of reduced size, while preserving its informational content. In multiple applications, the condition that one new observation makes the compressed set change is interpreted that this observation brings in extra information and, in learning theory, this corresponds to misclassification, or misprediction. In this pape… ▽ More A compression function is a map that slims down an observational set into a subset of reduced size, while preserving its informational content. In multiple applications, the condition that one new observation makes the compressed set change is interpreted that this observation brings in extra information and, in learning theory, this corresponds to misclassification, or misprediction. In this paper, we lay the foundations of a new theory that allows one to keep control on the probability of change of compression (which maps into the statistical "risk" in learning applications). Under suitable conditions, the cardinality of the compressed set is shown to be a consistent estimator of the probability of change of compression (without any upper limit on the size of the compressed set); moreover, unprecedentedly tight finite-sample bounds to evaluate the probability of change of compression are obtained under a generally applicable condition of preference. All results are usable in a fully agnostic setup, i.e., without requiring any a priori knowledge on the probability distribution of the observations. Not only these results offer a valid support to develop trust in observation-driven methodologies, they also play a fundamental role in learning techniques as a tool for hyper-parameter tuning. △ Less

Submitted 8 January, 2024; v1 submitted 30 January, 2023; originally announced January 2023.

Comments: https://www.jmlr.org/papers/v24/22-0605.html

Journal ref: Journal of Machine Learning Research, 24(339):1-74, 2023

On the sensitivity of linear resource sharing problems to the arrival of new agents

Authors: Alessandro Falsone, Kostas Margellos, Jacopo Zizzo, Maria Prandini, Simone Garatti

Abstract: We consider a multi-agent optimal resource sharing problem that is represented by a linear program. The amount of resource to be shared is fixed, and agents belong to a population that is characterized probabilistically so as to allow heterogeneity among the agents. In this paper, we provide a characterization of the probability that the arrival of a new agent affects the resource share of other a… ▽ More We consider a multi-agent optimal resource sharing problem that is represented by a linear program. The amount of resource to be shared is fixed, and agents belong to a population that is characterized probabilistically so as to allow heterogeneity among the agents. In this paper, we provide a characterization of the probability that the arrival of a new agent affects the resource share of other agents, which means that accommodating the new agent request at the detriment of the other agents allocation provides some payoff. This probability represents a sensitivity index for the optimal solution of a linear programming resource sharing problem when a new agent shows up, and it is of fundamental importance for a correct and profitable operation of the multi-agent system. Our developments build on the equivalence between the resource sharing problem and certain dual reformulations which can be interpreted as scenario programs with the number of scenarios corresponding to the number of agents in the primal problem. The recent "wait-and-judge" scenario approach is then used to obtain the sought sensitivity index. Our theoretical findings are demonstrated through a numerical example on optimal cargo aircraft loading. △ Less

Submitted 28 September, 2021; originally announced September 2021.

Comments: 14 pages, 3 figures

A Theory of the Risk for Optimization with Relaxation and its Application to Support Vector Machines

Authors: Marco C. Campi, Simone Garatti

Abstract: In this paper we consider optimization with relaxation, an ample paradigm to make data-driven designs. This approach was previously considered by the same authors of this work in Garatti and Campi (2019), a study that revealed a deep-seated connection between two concepts: risk (probability of not satisfying a new, out-of-sample, constraint) and complexity (according to a definition introduced in… ▽ More In this paper we consider optimization with relaxation, an ample paradigm to make data-driven designs. This approach was previously considered by the same authors of this work in Garatti and Campi (2019), a study that revealed a deep-seated connection between two concepts: risk (probability of not satisfying a new, out-of-sample, constraint) and complexity (according to a definition introduced in paper Garatti and Campi (2019)). This connection was shown to have profound implications in applications because it implied that the risk can be estimated from the complexity, a quantity that can be measured from the data without any knowledge of the data-generation mechanism. In the present work we establish new results. First, we expand the scope of Garatti and Campi (2019) so as to embrace a more general setup that covers various algorithms in machine learning. Then, we study classical support vector methods - including SVM (Support Vector Machine), SVR (Support Vector Regression) and SVDD (Support Vector Data Description) - and derive new results for the ability of these methods to generalize. All results are valid for any finite size of the data set. When the sample size tends to infinity, we establish the unprecedented result that the risk approaches the ratio between the complexity and the cardinality of the data sample, regardless of the value of the complexity. △ Less

Submitted 8 January, 2024; v1 submitted 13 April, 2020; originally announced April 2020.

Comments: https://www.jmlr.org/papers/v22/21-0641.html

Journal ref: Journal of Machine Learning Research 22(288):1-38, 2021

An incremental scenario approach for building energy management with uncertain occupancy

Authors: Arman Karshenas, Kostas Margellos, Simone Garatti

Abstract: We deal with the problem of energy management in buildings subject to uncertain occupancy. To this end, we formulate this as a finite horizon optimization program and optimize with respect to the windows' blinds position, radiator and cooling flux. Aiming at a schedule which is robust with respect to uncertain occupancy levels while avoiding imposing arbitrary assumptions on the underlying probabi… ▽ More We deal with the problem of energy management in buildings subject to uncertain occupancy. To this end, we formulate this as a finite horizon optimization program and optimize with respect to the windows' blinds position, radiator and cooling flux. Aiming at a schedule which is robust with respect to uncertain occupancy levels while avoiding imposing arbitrary assumptions on the underlying probability distribution of the uncertainty, we follow a data driven paradigm. In particular, we apply an incremental scenario approach methodology that has been recently proposed in the literature to our energy management formulation. To demonstrate the efficacy of the proposed implementation we provide a detailed numerical analysis on a stylized building and compare it with respect to a deterministic design and the standard scenario approach typically encountered in the literature. We show that our schedule is not agnostic with respect to uncertainty as deterministic approaches, while it requires fewer scenarios with respect to the standard scenario approach, thus resulting in a less conservative performance. △ Less

Submitted 15 November, 2019; originally announced November 2019.

Energy management for building district cooling: a distributed approach to resource sharing

Authors: Fabio Belluschi, Alessandro Falsone, Daniele Ioli, Kostas Margellos, Simone Garatti, Maria Prandini

Abstract: This paper deals with energy management in a district where multiple buildings can communicate over a time-varying network and aim at optimizing the use of shared resources like storage systems. We focus on building cooling, and propose an iterative, distributed algorithm that accounts for information privacy, since buildings are not required to disclose information about their individual utility… ▽ More This paper deals with energy management in a district where multiple buildings can communicate over a time-varying network and aim at optimizing the use of shared resources like storage systems. We focus on building cooling, and propose an iterative, distributed algorithm that accounts for information privacy, since buildings are not required to disclose information about their individual utility functions and constraint sets encoding, e.g., their consumption profiles, and overcomes the communication and computational challenges imposed by centralized management paradigms. Our approach relies on a methodology based on proximal minimization that has recently appeared in the literature. Motivated by the structure of the considered energy management optimization program, we provide a theoretical extension of this novel methodology, that is applicable to any problem that exhibits such structural properties. The efficacy of the resulting energy management algorithm is illustrated by means of a detailed simulation based study, considering different network topologies. △ Less

Submitted 20 October, 2016; originally announced October 2016.

Comments: 13 pages, 10 figures, submitted to IEEE Transactions on Control Systems Technology

A relaxation technique to ensure feasibility in stochastic control with input and state constraints

Authors: Luca Deori, Simone Garatti, Maria Prandini

Abstract: We consider a stochastic linear system and address the design of a finite horizon control policy that is optimal according to some average cost criterion and accounts also for probabilistic constraints on both the input and state variables. This finite horizon control problem formulation is quite common in the literature and has potential for being implemented in a receding horizon fashion accordi… ▽ More We consider a stochastic linear system and address the design of a finite horizon control policy that is optimal according to some average cost criterion and accounts also for probabilistic constraints on both the input and state variables. This finite horizon control problem formulation is quite common in the literature and has potential for being implemented in a receding horizon fashion according to the model predictive control strategy. Such a possibility, however, is hampered by the fact that, if the disturbance has unbounded support, a feasibility issue may arise. In this paper, we address this issue by introducing a constraint relaxation that is effective only when the original problem turns out to be unfeasible and, in that case, recovers feasibility as quickly as possible. This is obtained via a cascade of two probabilistically-constrained optimization problems, which are solved here through a computationally tractable scenario-based scheme, providing an approximate solution that satisfies the original probabilistic constraints of the cascade, with high confidence. A simulation example showing the effectiveness of the proposed approach concludes the paper. △ Less

Submitted 20 October, 2016; originally announced October 2016.

Finite time distributed averaging over ring networks

Authors: Alessandro Falsone, Kostas Margellos, Simone Garatti, Maria Prandini

Abstract: We consider a multi-agent system where each agent has its own estimate of a given quantity and the goal is to reach consensus on the average. To this purpose, we propose a distributed consensus algorithm that guarantees convergence to the average in a finite number of iterations. The algorithm is tailored to ring networks with bidirectional pairwise communications. If the number of agents $m$ is e… ▽ More We consider a multi-agent system where each agent has its own estimate of a given quantity and the goal is to reach consensus on the average. To this purpose, we propose a distributed consensus algorithm that guarantees convergence to the average in a finite number of iterations. The algorithm is tailored to ring networks with bidirectional pairwise communications. If the number of agents $m$ is even, say $m=2n$, then, the number of iterations needed is equal to $n$, which in this case is the diameter of the network, whereas the number of iterations grows to $3n$ if the number of agents is odd and equal to $m=2n+1$. △ Less

Submitted 30 August, 2016; originally announced August 2016.

Comments: 6 pages, 3 figures, submitted to IEEE Transactions on Control of Network Systems

Dual decomposition for multi-agent distributed optimization with coupling constraints

Authors: Alessandro Falsone, Kostas Margellos, Simone Garatti, Maria Prandini

Abstract: We study distributed optimization in a cooperative multi-agent setting, where agents have to agree on the usage of shared resources and can communicate via a time-varying network to this purpose. Each agent has its own decision variables that should be set so as to minimize its individual objective function subject to local constraints. Resource sharing is modeled via coupling constraints that inv… ▽ More We study distributed optimization in a cooperative multi-agent setting, where agents have to agree on the usage of shared resources and can communicate via a time-varying network to this purpose. Each agent has its own decision variables that should be set so as to minimize its individual objective function subject to local constraints. Resource sharing is modeled via coupling constraints that involve the non-positivity of the sum of agents' individual functions, each one depending on the decision variables of one single agent. We propose a novel distributed algorithm to minimize the sum of the agents' objective functions subject to both local and coupling constraints, where dual decomposition and proximal minimization are combined in an iterative scheme. Notably, privacy of information is guaranteed since only the dual optimization variables associated with the coupling constraints are exchanged by the agents. Under convexity assumptions, jointly with suitable connectivity properties of the communication network, we are able to prove that agents reach consensus to some optimal solution of the centralized dual problem counterpart, while primal variables converge to the set of optimizers of the centralized primal problem. The efficacy of the proposed approach is demonstrated on a plug-in electric vehicles charging problem. △ Less

Submitted 19 April, 2017; v1 submitted 3 July, 2016; originally announced July 2016.

Comments: 13 pages, 3 figures, submitted to Automatica

Distributed constrained optimization and consensus in uncertain networks via proximal minimization

Authors: Kostas Margellos, Alessandro Falsone, Simone Garatti, Maria Prandini

Abstract: We provide a unifying framework for distributed convex optimization over time-varying networks, in the presence of constraints and uncertainty, features that are typically treated separately in the literature. We adopt a proximal minimization perspective and show that this set-up allows us to bypass the difficulties of existing algorithms while simplifying the underlying mathematical analysis. We… ▽ More We provide a unifying framework for distributed convex optimization over time-varying networks, in the presence of constraints and uncertainty, features that are typically treated separately in the literature. We adopt a proximal minimization perspective and show that this set-up allows us to bypass the difficulties of existing algorithms while simplifying the underlying mathematical analysis. We develop an iterative algorithm and show convergence of the resulting scheme to some optimizer of the centralized problem. To deal with the case where the agents' constraint sets are affected by a possibly common uncertainty vector, we follow a scenario-based methodology and offer probabilistic guarantees regarding the feasibility properties of the resulting solution. To this end, we provide a distributed implementation of the scenario approach, allowing agents to use a different set of uncertainty scenarios in their local optimization programs. The efficacy of our algorithm is demonstrated by means of a numerical example related to a regression problem subject to regularization. △ Less

Submitted 23 May, 2017; v1 submitted 7 March, 2016; originally announced March 2016.

Comments: 15 pages, 4 figures, submitted to IEEE Transactions on Automatic Control