-
Artificial Inductive Bias for Synthetic Tabular Data Generation in Data-Scarce Scenarios
Authors:
Patricia A. Apellániz,
Ana Jiménez,
Borja Arroyo Galende,
Juan Parras,
Santiago Zazo
Abstract:
While synthetic tabular data generation using Deep Generative Models (DGMs) offers a compelling solution to data scarcity and privacy concerns, their effectiveness relies on substantial training data, often unavailable in real-world applications. This paper addresses this challenge by proposing a novel methodology for generating realistic and reliable synthetic tabular data with DGMs in limited re…
▽ More
While synthetic tabular data generation using Deep Generative Models (DGMs) offers a compelling solution to data scarcity and privacy concerns, their effectiveness relies on substantial training data, often unavailable in real-world applications. This paper addresses this challenge by proposing a novel methodology for generating realistic and reliable synthetic tabular data with DGMs in limited real-data environments. Our approach proposes several ways to generate an artificial inductive bias in a DGM through transfer learning and meta-learning techniques. We explore and compare four different methods within this framework, demonstrating that transfer learning strategies like pre-training and model averaging outperform meta-learning approaches, like Model-Agnostic Meta-Learning, and Domain Randomized Search. We validate our approach using two state-of-the-art DGMs, namely, a Variational Autoencoder and a Generative Adversarial Network, to show that our artificial inductive bias fuels superior synthetic data quality, as measured by Jensen-Shannon divergence, achieving relative gains of up to 50\% when using our proposed approach. This methodology has broad applicability in various DGMs and machine learning tasks, particularly in areas like healthcare and finance, where data scarcity is often a critical issue.
△ Less
Submitted 3 July, 2024;
originally announced July 2024.
-
Synthetic Tabular Data Validation: A Divergence-Based Approach
Authors:
Patricia A. Apellániz,
Ana Jiménez,
Borja Arroyo Galende,
Juan Parras,
Santiago Zazo
Abstract:
The ever-increasing use of generative models in various fields where tabular data is used highlights the need for robust and standardized validation metrics to assess the similarity between real and synthetic data. Current methods lack a unified framework and rely on diverse and often inconclusive statistical measures. Divergences, which quantify discrepancies between data distributions, offer a p…
▽ More
The ever-increasing use of generative models in various fields where tabular data is used highlights the need for robust and standardized validation metrics to assess the similarity between real and synthetic data. Current methods lack a unified framework and rely on diverse and often inconclusive statistical measures. Divergences, which quantify discrepancies between data distributions, offer a promising avenue for validation. However, traditional approaches calculate divergences independently for each feature due to the complexity of joint distribution modeling. This paper addresses this challenge by proposing a novel approach that uses divergence estimation to overcome the limitations of marginal comparisons. Our core contribution lies in applying a divergence estimator to build a validation metric considering the joint distribution of real and synthetic data. We leverage a probabilistic classifier to approximate the density ratio between datasets, allowing the capture of complex relationships. We specifically calculate two divergences: the well-known Kullback-Leibler (KL) divergence and the Jensen-Shannon (JS) divergence. KL divergence offers an established use in the field, while JS divergence is symmetric and bounded, providing a reliable metric. The efficacy of this approach is demonstrated through a series of experiments with varying distribution complexities. The initial phase involves comparing estimated divergences with analytical solutions for simple distributions, setting a benchmark for accuracy. Finally, we validate our method on a real-world dataset and its corresponding synthetic counterpart, showcasing its effectiveness in practical applications. This research offers a significant contribution with applicability beyond tabular data and the potential to improve synthetic data validation in various fields.
△ Less
Submitted 13 May, 2024;
originally announced May 2024.
-
An improved tabular data generator with VAE-GMM integration
Authors:
Patricia A. Apellániz,
Juan Parras,
Santiago Zazo
Abstract:
The rising use of machine learning in various fields requires robust methods to create synthetic tabular data. Data should preserve key characteristics while addressing data scarcity challenges. Current approaches based on Generative Adversarial Networks, such as the state-of-the-art CTGAN model, struggle with the complex structures inherent in tabular data. These data often contain both continuou…
▽ More
The rising use of machine learning in various fields requires robust methods to create synthetic tabular data. Data should preserve key characteristics while addressing data scarcity challenges. Current approaches based on Generative Adversarial Networks, such as the state-of-the-art CTGAN model, struggle with the complex structures inherent in tabular data. These data often contain both continuous and discrete features with non-Gaussian distributions. Therefore, we propose a novel Variational Autoencoder (VAE)-based model that addresses these limitations. Inspired by the TVAE model, our approach incorporates a Bayesian Gaussian Mixture model (BGM) within the VAE architecture. This avoids the limitations imposed by assuming a strictly Gaussian latent space, allowing for a more accurate representation of the underlying data distribution during data generation. Furthermore, our model offers enhanced flexibility by allowing the use of various differentiable distributions for individual features, making it possible to handle both continuous and discrete data types. We thoroughly validate our model on three real-world datasets with mixed data types, including two medically relevant ones, based on their resemblance and utility. This evaluation demonstrates significant outperformance against CTGAN and TVAE, establishing its potential as a valuable tool for generating synthetic tabular data in various domains, particularly in healthcare.
△ Less
Submitted 12 April, 2024;
originally announced April 2024.
-
SAVAE: Leveraging the variational Bayes autoencoder for survival analysis
Authors:
Patricia A. Apellániz,
Juan Parras,
Santiago Zazo
Abstract:
As in many fields of medical research, survival analysis has witnessed a growing interest in the application of deep learning techniques to model complex, high-dimensional, heterogeneous, incomplete, and censored medical data. Current methods often make assumptions about the relations between data that may not be valid in practice. In response, we introduce SAVAE (Survival Analysis Variational Aut…
▽ More
As in many fields of medical research, survival analysis has witnessed a growing interest in the application of deep learning techniques to model complex, high-dimensional, heterogeneous, incomplete, and censored medical data. Current methods often make assumptions about the relations between data that may not be valid in practice. In response, we introduce SAVAE (Survival Analysis Variational Autoencoder), a novel approach based on Variational Autoencoders. SAVAE contributes significantly to the field by introducing a tailored ELBO formulation for survival analysis, supporting various parametric distributions for covariates and survival time (as long as the log-likelihood is differentiable). It offers a general method that consistently performs well on various metrics, demonstrating robustness and stability through different experiments. Our proposal effectively estimates time-to-event, accounting for censoring, covariate interactions, and time-varying risk associations. We validate our model in diverse datasets, including genomic, clinical, and demographic data, with varying levels of censoring. This approach demonstrates competitive performance compared to state-of-the-art techniques, as assessed by the Concordance Index and the Integrated Brier Score. SAVAE also offers an interpretable model that parametrically models covariates and time. Moreover, its generative architecture facilitates further applications such as clustering, data imputation, and the generation of synthetic patient data through latent space inference from survival data.
△ Less
Submitted 22 December, 2023;
originally announced December 2023.
-
Strong Duality in Nonconvex Quadratic Problems with Separable Quadratic Constraints
Authors:
Javier Zazo,
Santiago Zazo
Abstract:
We study nonconvex quadratic problems (QPs) with quadratic separable constraints, where these constraints can be defined both as inequalities or equalities. We derive sufficient conditions for these types of problems to present the S-property, which ultimately guarantees strong duality between the primal and dual problems of the QP. We study the existence of solutions and propose a novel distribut…
▽ More
We study nonconvex quadratic problems (QPs) with quadratic separable constraints, where these constraints can be defined both as inequalities or equalities. We derive sufficient conditions for these types of problems to present the S-property, which ultimately guarantees strong duality between the primal and dual problems of the QP. We study the existence of solutions and propose a novel distributed algorithm to solve the problem optimally when the S-property is satisfied. Finally, we illustrate our theoretical results proving that the robust least squares problem with multiple constraints has the strong duality property.
△ Less
Submitted 11 November, 2021;
originally announced November 2021.
-
Bifurcation control for a ship maneuvering model with nonsmooth nonlinearities
Authors:
Miriam Steinherr Zazo,
Jens D. M. Rademacher
Abstract:
We consider a widely used form of models for ship maneuvering, whose nonlinearities entail continuous but nonsmooth second-order modulus terms. For such models bifurcations of straight motion are not amenable to standard center manifold reduction and normal forms. Based on a recently developed analytical approach, we nevertheless determine the character of local bifurcations when stabilizing the s…
▽ More
We consider a widely used form of models for ship maneuvering, whose nonlinearities entail continuous but nonsmooth second-order modulus terms. For such models bifurcations of straight motion are not amenable to standard center manifold reduction and normal forms. Based on a recently developed analytical approach, we nevertheless determine the character of local bifurcations when stabilizing the straight motion course with standard proportional control. For a specific model class we perform a detailed analysis of the linearization to determine the location of these bifurcations in the control parameter space and its dependence on selected design parameters. By computing the analytically derived characteristic parameters, we find that `safe' supercritical Andronov--Hopf bifurcations are typical. Through numerical continuation we provide a more global bifurcation analysis, which identifies the arrangement and relative location of stable and unstable equilibria and periodic orbits.
△ Less
Submitted 26 June, 2022; v1 submitted 21 April, 2021;
originally announced April 2021.
-
Lyapunov coefficients for Hopf bifurcations in systems with piecewise smooth nonlinearity
Authors:
Miriam Steinherr Zazo,
Jens D. M. Rademacher
Abstract:
Motivated by models that arise in controlled ship maneuvering, we analyze Hopf bifurcations in systems with piecewise smooth nonlinear part. In particular, we derive explicit formulas for the generalization of the first Lyapunov coefficient to this setting. This generically determines the direction of branching (super- versus sub-criticality), but in general this differs from any fixed smoothing o…
▽ More
Motivated by models that arise in controlled ship maneuvering, we analyze Hopf bifurcations in systems with piecewise smooth nonlinear part. In particular, we derive explicit formulas for the generalization of the first Lyapunov coefficient to this setting. This generically determines the direction of branching (super- versus sub-criticality), but in general this differs from any fixed smoothing of the vector field. We focus on nonsmooth nonlinearities of the form $u_i|u_j|$, but our results are formulated in broader generality for systems in any dimension with piecewise smooth nonlinear part. In addition, we discuss some codimension-one degeneracies and apply the results to a model of a shimmying wheel.
△ Less
Submitted 19 January, 2023; v1 submitted 12 June, 2020;
originally announced June 2020.
-
Learning Parametric Closed-Loop Policies for Markov Potential Games
Authors:
Sergio Valcarcel Macua,
Javier Zazo,
Santiago Zazo
Abstract:
Multiagent systems where agents interact among themselves and with a stochastic environment can be formalized as stochastic games. We study a subclass named Markov potential games (MPGs) that appear often in economic and engineering applications when the agents share a common resource. We consider MPGs with continuous state-action variables, coupled constraints and nonconvex rewards. Previous anal…
▽ More
Multiagent systems where agents interact among themselves and with a stochastic environment can be formalized as stochastic games. We study a subclass named Markov potential games (MPGs) that appear often in economic and engineering applications when the agents share a common resource. We consider MPGs with continuous state-action variables, coupled constraints and nonconvex rewards. Previous analysis followed a variational approach that is only valid for very simple cases (convex rewards, invertible dynamics, and no coupled constraints); or considered deterministic dynamics and provided open-loop (OL) analysis, studying strategies that consist in predefined action sequences, which are not optimal for stochastic environments. We present a closed-loop (CL) analysis for MPGs and consider parametric policies that depend on the current state. We provide easily verifiable, sufficient and necessary conditions for a stochastic game to be an MPG, even for complex parametric functions (e.g., deep neural networks); and show that a closed-loop Nash equilibrium (NE) can be found (or at least approximated) by solving a related optimal control problem (OCP). This is useful since solving an OCP--which is a single-objective problem--is usually much simpler than solving the original set of coupled OCPs that form the game--which is a multiobjective control problem. This is a considerable improvement over the previously standard approach for the CL analysis of MPGs, which gives no approximate solution if no NE belongs to the chosen parametric family, and which is practical only for simple parametric forms. We illustrate the theoretical contributions with an example by applying our approach to a noncooperative communications engineering game. We then solve the game with a deep reinforcement learning algorithm that learns policies that closely approximates an exact variational NE of the game.
△ Less
Submitted 22 May, 2018; v1 submitted 2 February, 2018;
originally announced February 2018.
-
Diff-DAC: Distributed Actor-Critic for Average Multitask Deep Reinforcement Learning
Authors:
Sergio Valcarcel Macua,
Aleksi Tukiainen,
Daniel García-Ocaña Hernández,
David Baldazo,
Enrique Munoz de Cote,
Santiago Zazo
Abstract:
We propose a fully distributed actor-critic algorithm approximated by deep neural networks, named \textit{Diff-DAC}, with application to single-task and to average multitask reinforcement learning (MRL). Each agent has access to data from its local task only, but it aims to learn a policy that performs well on average for the whole set of tasks. During the learning process, agents communicate thei…
▽ More
We propose a fully distributed actor-critic algorithm approximated by deep neural networks, named \textit{Diff-DAC}, with application to single-task and to average multitask reinforcement learning (MRL). Each agent has access to data from its local task only, but it aims to learn a policy that performs well on average for the whole set of tasks. During the learning process, agents communicate their value-policy parameters to their neighbors, diffusing the information across the network, so that they converge to a common policy, with no need for a central node. The method is scalable, since the computational and communication costs per agent grow with its number of neighbors. We derive Diff-DAC's from duality theory and provide novel insights into the standard actor-critic framework, showing that it is actually an instance of the dual ascent method that approximates the solution of a linear program. Experiments suggest that Diff-DAC can outperform the single previous distributed MRL approach (i.e., Dist-MTLPS) and even the centralized architecture.
△ Less
Submitted 25 October, 2020; v1 submitted 27 October, 2017;
originally announced October 2017.
-
Robust Worst-Case Analysis of Demand-Side Management in Smart Grids
Authors:
Javier Zazo,
Santiago Zazo,
Sergio Valcarcel Macua
Abstract:
Demand-side management presents significant benefits in reducing the energy load in smart grids by balancing consumption demands or including energy generation and/or storage devices in the user's side. These techniques coordinate the energy load so that users minimize their monetary expenditure. However, these methods require accurate predictions in the energy consumption profiles, which make the…
▽ More
Demand-side management presents significant benefits in reducing the energy load in smart grids by balancing consumption demands or including energy generation and/or storage devices in the user's side. These techniques coordinate the energy load so that users minimize their monetary expenditure. However, these methods require accurate predictions in the energy consumption profiles, which make them inflexible to real demand variations. In this paper we propose a realistic model that accounts for uncertainty in these variations and calculates a robust price for all users in the smart grid. We analyze the existence of solutions for this novel scenario, propose convergent distributed algorithms to find them, and perform simulations considering energy expenditure. We show that this model can effectively reduce the monetary expenses for all users in a real-time market, while at the same time it provides a reliable production cost estimate to the energy supplier.
△ Less
Submitted 5 May, 2016; v1 submitted 26 April, 2016;
originally announced April 2016.
-
Experimental testbed for seawater channel characterization
Authors:
Pablo Mena,
Pablo Dorta-Naranjo,
Gara Quintana,
Iván Pérez-Álvarez,
Eugenio Jiménez,
Santiago Zazo,
Marina Pérez,
Laura Cardona,
J. Joaquin Hernández
Abstract:
Shallow seawaters are problematic for acoustic and optical communications. Sensor networks based on electromagnetic (EM) communications are evaluated in this environment. In order to characterize the subaquatic channel, several measurement systems have been designed, built and tested in the sea obtaining very reliable results. Experiments carried out with dipoles and loop antennas showed serious d…
▽ More
Shallow seawaters are problematic for acoustic and optical communications. Sensor networks based on electromagnetic (EM) communications are evaluated in this environment. In order to characterize the subaquatic channel, several measurement systems have been designed, built and tested in the sea obtaining very reliable results. Experiments carried out with dipoles and loop antennas showed serious disagreement with the state of the art, especially when dipole antennas are used. Dipoles performance was poor while magnetic loops showed relevant results. Measurement system is described in detail and real attenuation of the subaquatic channel is obtained for several distances and antennas. Finally, measured and simulated results are compared with good agreement.
△ Less
Submitted 13 April, 2016;
originally announced April 2016.
-
Investigation on Radio Wave Propagation in Shallow Seawater: Simulations and Measurements
Authors:
Eugenio Jimenez,
Gara Quintana,
Pablo Mena,
Pablo Dorta,
Ivan Perez-Alvarez,
Santiago Zazo,
Marina Perez,
Eduardo Quevedo
Abstract:
The authors present full wave simulations and experimental results of propagation of electromagnetic waves in shallow seawaters. Transmitter and receiver antennas are ten-turns loops placed on the seabed. Some propagation frameworks are presented and simulated. Finally, simulation results are compared with experimental ones.
The authors present full wave simulations and experimental results of propagation of electromagnetic waves in shallow seawaters. Transmitter and receiver antennas are ten-turns loops placed on the seabed. Some propagation frameworks are presented and simulated. Finally, simulation results are compared with experimental ones.
△ Less
Submitted 13 April, 2016;
originally announced April 2016.
-
Cooperative Network Node Positioning Techniques Using Underwater Radio Communications
Authors:
Javier Zazo,
Santiago Zazo,
Sergio Valcarcel Macua,
Marina Pérez,
Iván Pérez-Álvarez,
Laura Cardona,
Eduardo Quevedo
Abstract:
We analyze the problem of localization algorithms for underwater sensor networks. We first characterize the underwater channel for radio communications and adjust a linear model with measurements of real transmissions. We propose an algorithm where the sensor nodes collaboratively estimate their unknown positions in the network. In this setting, we assume low connectivity of the nodes, low data ra…
▽ More
We analyze the problem of localization algorithms for underwater sensor networks. We first characterize the underwater channel for radio communications and adjust a linear model with measurements of real transmissions. We propose an algorithm where the sensor nodes collaboratively estimate their unknown positions in the network. In this setting, we assume low connectivity of the nodes, low data rates, and nonzero probability of lost packets in the transmission. Finally, we consider the problem of a node estimating it's position in underwater navigation. We also provide simulations illustrating the previous proposals.
△ Less
Submitted 12 April, 2016;
originally announced April 2016.
-
Simulation of Underwater RF Wireless Sensor Networks using Castalia
Authors:
Sergio Valcarcel Macua,
Santiago Zazo,
Javier Zazo,
Marina Pérez Jiménez,
Iván Pérez-Álvarez,
Eugenio Jiménez,
Joaquín Hernández Brito
Abstract:
We use real measurements of the underwater channel to simulate a whole underwater RF wireless sensor networks, including propagation impairments (e.g., noise, interference), radio hardware (e.g., modulation scheme, bandwidth, transmit power), hardware limitations (e.g., clock drift, transmission buffer) and complete MAC and routing protocols. The results should be useful for designing centralized…
▽ More
We use real measurements of the underwater channel to simulate a whole underwater RF wireless sensor networks, including propagation impairments (e.g., noise, interference), radio hardware (e.g., modulation scheme, bandwidth, transmit power), hardware limitations (e.g., clock drift, transmission buffer) and complete MAC and routing protocols. The results should be useful for designing centralized and distributed algorithms for applications like monitoring, event detection, localization and aid to navigation. We also explain the changes that have to be done to Castalia in order to perform the simulations.
△ Less
Submitted 12 April, 2016;
originally announced April 2016.
-
Dynamic Potential Games in Communications: Fundamentals and Applications
Authors:
Santiago Zazo,
Sergio Valcarcel Macua,
Matilde Sánchez-Fernández,
Javier Zazo
Abstract:
In a noncooperative dynamic game, multiple agents operating in a changing environment aim to optimize their utilities over an infinite time horizon. Time-varying environments allow to model more realistic scenarios (e.g., mobile devices equipped with batteries, wireless communications over a fading channel, etc.). However, solving a dynamic game is a difficult task that requires dealing with multi…
▽ More
In a noncooperative dynamic game, multiple agents operating in a changing environment aim to optimize their utilities over an infinite time horizon. Time-varying environments allow to model more realistic scenarios (e.g., mobile devices equipped with batteries, wireless communications over a fading channel, etc.). However, solving a dynamic game is a difficult task that requires dealing with multiple coupled optimal control problems. We focus our analysis on a class of problems, named \textit{dynamic potential games}, whose solution can be found through a single multivariate optimal control problem. Our analysis generalizes previous studies by considering that the set of environment's states and the set of players' actions are constrained, as it is required by most of the applications. And the theoretical results are the natural extension of the analysis for static potential games. We apply the analysis and provide numerical methods to solve four key example problems, with different features each: energy demand control in a smart-grid network, network flow optimization in which the relays have bounded link capacity and limited battery life, uplink multiple access communication with users that have to optimize the use of their batteries, and two optimal scheduling games with nonstationary channels.
△ Less
Submitted 28 December, 2015; v1 submitted 3 September, 2015;
originally announced September 2015.
-
Distributed Policy Evaluation Under Multiple Behavior Strategies
Authors:
Sergio Valcarcel Macua,
Jianshu Chen,
Santiago Zazo,
Ali H. Sayed
Abstract:
We apply diffusion strategies to develop a fully-distributed cooperative reinforcement learning algorithm in which agents in a network communicate only with their immediate neighbors to improve predictions about their environment. The algorithm can also be applied to off-policy learning, meaning that the agents can predict the response to a behavior different from the actual policies they are foll…
▽ More
We apply diffusion strategies to develop a fully-distributed cooperative reinforcement learning algorithm in which agents in a network communicate only with their immediate neighbors to improve predictions about their environment. The algorithm can also be applied to off-policy learning, meaning that the agents can predict the response to a behavior different from the actual policies they are following. The proposed distributed strategy is efficient, with linear complexity in both computation time and memory footprint. We provide a mean-square-error performance analysis and establish convergence under constant step-size updates, which endow the network with continuous learning capabilities. The results show a clear gain from cooperation: when the individual agents can estimate the solution, cooperation increases stability and reduces bias and variance of the prediction error; but, more importantly, the network is able to approach the optimal solution even when none of the individual agents can (e.g., when the individual behavior policies restrict each agent to sample a small portion of the state space).
△ Less
Submitted 5 November, 2014; v1 submitted 29 December, 2013;
originally announced December 2013.
-
Belief Consensus Algorithms for Fast Distributed Target Tracking in Wireless Sensor Networks
Authors:
Vladimir Savic,
Henk Wymeersch,
Santiago Zazo
Abstract:
In distributed target tracking for wireless sensor networks, agreement on the target state can be achieved by the construction and maintenance of a communication path, in order to exchange information regarding local likelihood functions. Such an approach lacks robustness to failures and is not easily applicable to ad-hoc networks. To address this, several methods have been proposed that allow agr…
▽ More
In distributed target tracking for wireless sensor networks, agreement on the target state can be achieved by the construction and maintenance of a communication path, in order to exchange information regarding local likelihood functions. Such an approach lacks robustness to failures and is not easily applicable to ad-hoc networks. To address this, several methods have been proposed that allow agreement on the global likelihood through fully distributed belief consensus (BC) algorithms, operating on local likelihoods in distributed particle filtering (DPF). However, a unified comparison of the convergence speed and communication cost has not been performed. In this paper, we provide such a comparison and propose a novel BC algorithm based on belief propagation (BP). According to our study, DPF based on metropolis belief consensus (MBC) is the fastest in loopy graphs, while DPF based on BP consensus is the fastest in tree graphs. Moreover, we found that BC-based DPF methods have lower communication overhead than data flooding when the network is sufficiently sparse.
△ Less
Submitted 30 July, 2013; v1 submitted 23 February, 2012;
originally announced February 2012.
-
Location-aided Distributed Primary User Identification in a Cognitive Radio Scenario
Authors:
Pavle Belanovic,
Sergio Valcarcel Macua,
Santiago Zazo
Abstract:
We address a cognitive radio scenario, where a number of secondary users performs identification of which primary user, if any, is transmitting, in a distributed way and using limited location information. We propose two fully distributed algorithms: the first is a direct identification scheme, and in the other a distributed sub-optimal detection based on a simplified Neyman-Pearson energy detecto…
▽ More
We address a cognitive radio scenario, where a number of secondary users performs identification of which primary user, if any, is transmitting, in a distributed way and using limited location information. We propose two fully distributed algorithms: the first is a direct identification scheme, and in the other a distributed sub-optimal detection based on a simplified Neyman-Pearson energy detector precedes the identification scheme. Both algorithms are studied analytically in a realistic transmission scenario, and the advantage obtained by detection pre-processing is also verified via simulation. Finally, we give details of their fully distributed implementation via consensus averaging algorithms.
△ Less
Submitted 26 October, 2011;
originally announced October 2011.
