-
FedCBO: Reaching Group Consensus in Clustered Federated Learning through Consensus-based Optimization
Authors:
Jose A. Carrillo,
Nicolas Garcia Trillos,
Sixu Li,
Yuhua Zhu
Abstract:
Federated learning is an important framework in modern machine learning that seeks to integrate the training of learning models from multiple users, each user having their own local data set, in a way that is sensitive to data privacy and to communication loss constraints. In clustered federated learning, one assumes an additional unknown group structure among users, and the goal is to train model…
▽ More
Federated learning is an important framework in modern machine learning that seeks to integrate the training of learning models from multiple users, each user having their own local data set, in a way that is sensitive to data privacy and to communication loss constraints. In clustered federated learning, one assumes an additional unknown group structure among users, and the goal is to train models that are useful for each group, rather than simply training a single global model for all users. In this paper, we propose a novel solution to the problem of clustered federated learning that is inspired by ideas in consensus-based optimization (CBO). Our new CBO-type method is based on a system of interacting particles that is oblivious to group memberships. Our model is motivated by rigorous mathematical reasoning, including a mean field analysis describing the large number of particles limit of our particle system, as well as convergence guarantees for the simultaneous global optimization of general non-convex objective functions (corresponding to the loss functions of each cluster of users) in the mean-field regime. Experimental results demonstrate the efficacy of our FedCBO algorithm compared to other state-of-the-art methods and help validate our methodological and theoretical work.
△ Less
Submitted 4 May, 2023;
originally announced May 2023.
-
Individual Explanations in Machine Learning Models: A Case Study on Poverty Estimation
Authors:
Alfredo Carrillo,
Luis F. Cantú,
Luis Tejerina,
Alejandro Noriega
Abstract:
Machine learning methods are being increasingly applied in sensitive societal contexts, where decisions impact human lives. Hence it has become necessary to build capabilities for providing easily-interpretable explanations of models' predictions. Recently in academic literature, a vast number of explanations methods have been proposed. Unfortunately, to our knowledge, little has been documented a…
▽ More
Machine learning methods are being increasingly applied in sensitive societal contexts, where decisions impact human lives. Hence it has become necessary to build capabilities for providing easily-interpretable explanations of models' predictions. Recently in academic literature, a vast number of explanations methods have been proposed. Unfortunately, to our knowledge, little has been documented about the challenges machine learning practitioners most often face when applying them in real-world scenarios. For example, a typical procedure such as feature engineering can make some methodologies no longer applicable. The present case study has two main objectives. First, to expose these challenges and how they affect the use of relevant and novel explanations methods. And second, to present a set of strategies that mitigate such challenges, as faced when implementing explanation methods in a relevant application domain -- poverty estimation and its use for prioritizing access to social policies.
△ Less
Submitted 11 April, 2021; v1 submitted 8 April, 2021;
originally announced April 2021.
-
Individual Explanations in Machine Learning Models: A Survey for Practitioners
Authors:
Alfredo Carrillo,
Luis F. Cantú,
Alejandro Noriega
Abstract:
In recent years, the use of sophisticated statistical models that influence decisions in domains of high societal relevance is on the rise. Although these models can often bring substantial improvements in the accuracy and efficiency of organizations, many governments, institutions, and companies are reluctant to their adoption as their output is often difficult to explain in human-interpretable w…
▽ More
In recent years, the use of sophisticated statistical models that influence decisions in domains of high societal relevance is on the rise. Although these models can often bring substantial improvements in the accuracy and efficiency of organizations, many governments, institutions, and companies are reluctant to their adoption as their output is often difficult to explain in human-interpretable ways. Hence, these models are often regarded as black-boxes, in the sense that their internal mechanisms can be opaque to human audit. In real-world applications, particularly in domains where decisions can have a sensitive impact--e.g., criminal justice, estimating credit scores, insurance risk, health risks, etc.--model interpretability is desired. Recently, the academic literature has proposed a substantial amount of methods for providing interpretable explanations to machine learning models. This survey reviews the most relevant and novel methods that form the state-of-the-art for addressing the particular problem of explaining individual instances in machine learning. It seeks to provide a succinct review that can guide data science and machine learning practitioners in the search for appropriate methods to their problem domain.
△ Less
Submitted 11 April, 2021; v1 submitted 8 April, 2021;
originally announced April 2021.
-
Enhancement of damaged-image prediction through Cahn-Hilliard Image Inpainting
Authors:
José A. Carrillo,
Serafim Kalliadasis,
Fuyue Liang,
Sergio P. Perez
Abstract:
We assess the benefit of including an image inpainting filter before passing damaged images into a classification neural network. For this we employ a modified Cahn-Hilliard equation as an image inpainting filter, which is solved via a finite volume scheme with reduced computational cost and adequate properties for energy stability and boundedness. The benchmark dataset employed here is MNIST, whi…
▽ More
We assess the benefit of including an image inpainting filter before passing damaged images into a classification neural network. For this we employ a modified Cahn-Hilliard equation as an image inpainting filter, which is solved via a finite volume scheme with reduced computational cost and adequate properties for energy stability and boundedness. The benchmark dataset employed here is MNIST, which consists of binary images of handwritten digits and is a standard dataset to validate image-processing methodologies. We train a neural network based of dense layers with the training set of MNIST, and subsequently we contaminate the test set with damage of different types and intensities. We then compare the prediction accuracy of the neural network with and without applying the Cahn-Hilliard filter to the damaged images test. Our results quantify the significant improvement of damaged-image prediction due to applying the Cahn-Hilliard filter, which for specific damages can increase up to 50% and is in general advantageous for low to moderate damage.
△ Less
Submitted 15 March, 2021; v1 submitted 21 July, 2020;
originally announced July 2020.
-
Pedestrian Models based on Rational Behaviour
Authors:
Rafael Bailo,
José A. Carrillo,
Pierre Degond
Abstract:
Following the paradigm set by attraction-repulsion-alignment schemes, a myriad of individual based models have been proposed to calculate the evolution of abstract agents. While the emergent features of many agent systems have been described astonishingly well with force-based models, this is not the case for pedestrians. Many of the classical schemes have failed to capture the fine detail of crow…
▽ More
Following the paradigm set by attraction-repulsion-alignment schemes, a myriad of individual based models have been proposed to calculate the evolution of abstract agents. While the emergent features of many agent systems have been described astonishingly well with force-based models, this is not the case for pedestrians. Many of the classical schemes have failed to capture the fine detail of crowd dynamics, and it is unlikely that a purely mechanical model will succeed. As a response to the mechanistic literature, we will consider a model for pedestrian dynamics that attempts to reproduce the rational behaviour of individual agents through the means of anticipation. Each pedestrian undergoes a two-step time evolution based on a perception stage and a decision stage. We will discuss the validity of this game theoretical based model in regimes with varying degrees of congestion, ultimately presenting a correction to the mechanistic model in order to achieve realistic high-density dynamics.
△ Less
Submitted 1 May, 2019; v1 submitted 21 August, 2018;
originally announced August 2018.
-
BPM in the cloud: A systematic literature review
Authors:
Ageo Carrillo,
Marco Sobrevilla
Abstract:
Business Process Management (BPM) in the cloud is focused on how to provide Business Process as a Service (BPaaS) and also implementing the Elastic Business Process Management (eBPM). These approaches are finding models, techniques and methodologies to increase the BPM adoption in the cloud. The objective on this research is to carry out a literature review about the state of the art of Business P…
▽ More
Business Process Management (BPM) in the cloud is focused on how to provide Business Process as a Service (BPaaS) and also implementing the Elastic Business Process Management (eBPM). These approaches are finding models, techniques and methodologies to increase the BPM adoption in the cloud. The objective on this research is to carry out a literature review about the state of the art of Business Process Management in the cloud, identify categories and analyze what the approach on eBPM researches and BPaaS researches is. The method used is the systematic literature review and the researches evolution over the time will be categorized and analyzed using a timeline and cumulative charts. The results show that the Business Process enactment is the main approach focused on eBPM researches, whereas the researches on BPaaS are focused on a greater number of categories. The conclusion of this research is that the authors could not identify any systems supporting the eBPM and BPaaS approaches at the same time and they introduce the definition of a new approach called Elastic Business Process Management as a Service (eBPMaaS)
△ Less
Submitted 23 September, 2017;
originally announced September 2017.
-
Bayesian Inference of Recursive Sequences of Group Activities from Tracks
Authors:
Ernesto Brau,
Colin Dawson,
Alfredo Carrillo,
David Sidi,
Clayton T. Morrison
Abstract:
We present a probabilistic generative model for inferring a description of coordinated, recursively structured group activities at multiple levels of temporal granularity based on observations of individuals' trajectories. The model accommodates: (1) hierarchically structured groups, (2) activities that are temporally and compositionally recursive, (3) component roles assigning different subactivi…
▽ More
We present a probabilistic generative model for inferring a description of coordinated, recursively structured group activities at multiple levels of temporal granularity based on observations of individuals' trajectories. The model accommodates: (1) hierarchically structured groups, (2) activities that are temporally and compositionally recursive, (3) component roles assigning different subactivity dynamics to subgroups of participants, and (4) a nonparametric Gaussian Process model of trajectories. We present an MCMC sampling framework for performing joint inference over recursive activity descriptions and assignment of trajectories to groups, integrating out continuous parameters. We demonstrate the model's expressive power in several simulated and complex real-world scenarios from the VIRAT and UCLA Aerial Event video data sets.
△ Less
Submitted 23 April, 2016;
originally announced April 2016.
-
Sign Language Tutoring Tool
Authors:
Oya Aran,
Ismail Ari,
Alexandre Benoit,
Ana Huerta Carrillo,
François-Xavier Fanard,
Pavel Campr,
Lale Akarun,
Alice Caplier,
Michele Rombaut,
Bulent Sankur
Abstract:
In this project, we have developed a sign language tutor that lets users learn isolated signs by watching recorded videos and by trying the same signs. The system records the user's video and analyses it. If the sign is recognized, both verbal and animated feedback is given to the user. The system is able to recognize complex signs that involve both hand gestures and head movements and expressio…
▽ More
In this project, we have developed a sign language tutor that lets users learn isolated signs by watching recorded videos and by trying the same signs. The system records the user's video and analyses it. If the sign is recognized, both verbal and animated feedback is given to the user. The system is able to recognize complex signs that involve both hand gestures and head movements and expressions. Our performance tests yield a 99% recognition rate on signs involving only manual gestures and 85% recognition rate on signs that involve both manual and non manual components, such as head movement and facial expressions.
△ Less
Submitted 18 February, 2008;
originally announced February 2008.
