-
A Survey of Temporal Credit Assignment in Deep Reinforcement Learning
Authors:
Eduardo Pignatelli,
Johan Ferret,
Matthieu Geist,
Thomas Mesnard,
Hado van Hasselt,
Olivier Pietquin,
Laura Toni
Abstract:
The Credit Assignment Problem (CAP) refers to the longstanding challenge of Reinforcement Learning (RL) agents to associate actions with their long-term consequences. Solving the CAP is a crucial step towards the successful deployment of RL in the real world since most decision problems provide feedback that is noisy, delayed, and with little or no information about the causes. These conditions ma…
▽ More
The Credit Assignment Problem (CAP) refers to the longstanding challenge of Reinforcement Learning (RL) agents to associate actions with their long-term consequences. Solving the CAP is a crucial step towards the successful deployment of RL in the real world since most decision problems provide feedback that is noisy, delayed, and with little or no information about the causes. These conditions make it hard to distinguish serendipitous outcomes from those caused by informed decision-making. However, the mathematical nature of credit and the CAP remains poorly understood and defined. In this survey, we review the state of the art of Temporal Credit Assignment (CA) in deep RL. We propose a unifying formalism for credit that enables equitable comparisons of state-of-the-art algorithms and improves our understanding of the trade-offs between the various methods. We cast the CAP as the problem of learning the influence of an action over an outcome from a finite amount of experience. We discuss the challenges posed by delayed effects, transpositions, and a lack of action influence, and analyse how existing methods aim to address them. Finally, we survey the protocols to evaluate a credit assignment method and suggest ways to diagnose the sources of struggle for different methods. Overall, this survey provides an overview of the field for new-entry practitioners and researchers, it offers a coherent perspective for scholars looking to expedite the starting stages of a new study on the CAP, and it suggests potential directions for future research.
△ Less
Submitted 4 July, 2024; v1 submitted 2 December, 2023;
originally announced December 2023.
-
AGAR: Attention Graph-RNN for Adaptative Motion Prediction of Point Clouds of Deformable Objects
Authors:
Pedro Gomes,
Silvia Rossi,
Laura Toni
Abstract:
This paper focuses on motion prediction for point cloud sequences in the challenging case of deformable 3D objects, such as human body motion. First, we investigate the challenges caused by deformable shapes and complex motions present in this type of representation, with the ultimate goal of understanding the technical limitations of state-of-the-art models. From this understanding, we propose an…
▽ More
This paper focuses on motion prediction for point cloud sequences in the challenging case of deformable 3D objects, such as human body motion. First, we investigate the challenges caused by deformable shapes and complex motions present in this type of representation, with the ultimate goal of understanding the technical limitations of state-of-the-art models. From this understanding, we propose an improved architecture for point cloud prediction of deformable 3D objects. Specifically, to handle deformable shapes, we propose a graph-based approach that learns and exploits the spatial structure of point clouds to extract more representative features. Then we propose a module able to combine the learned features in an adaptative manner according to the point cloud movements. The proposed adaptative module controls the composition of local and global motions for each point, enabling the network to model complex motions in deformable 3D objects more effectively. We tested the proposed method on the following datasets: MNIST moving digits, the Mixamo human bodies motions, JPEG and CWIPC-SXR real-world dynamic bodies. Simulation results demonstrate that our method outperforms the current baseline methods given its improved ability to model complex movements as well as preserve point cloud shape. Furthermore, we demonstrate the generalizability of the proposed framework for dynamic feature learning, by testing the framework for action recognition on the MSRAction3D dataset and achieving results on-par with state-of-the-art methods
△ Less
Submitted 19 July, 2023;
originally announced July 2023.
-
Online Network Source Optimization with Graph-Kernel MAB
Authors:
Laura Toni,
Pascal Frossard
Abstract:
We propose Grab-UCB, a graph-kernel multi-arms bandit algorithm to learn online the optimal source placement in large scale networks, such that the reward obtained from a priori unknown network processes is maximized. The uncertainty calls for online learning, which suffers however from the curse of dimensionality. To achieve sample efficiency, we describe the network processes with an adaptive gr…
▽ More
We propose Grab-UCB, a graph-kernel multi-arms bandit algorithm to learn online the optimal source placement in large scale networks, such that the reward obtained from a priori unknown network processes is maximized. The uncertainty calls for online learning, which suffers however from the curse of dimensionality. To achieve sample efficiency, we describe the network processes with an adaptive graph dictionary model, which typically leads to sparse spectral representations. This enables a data-efficient learning framework, whose learning rate scales with the dimension of the spectral representation model instead of the one of the network. We then propose Grab-UCB, an online sequential decision strategy that learns the parameters of the spectral representation while optimizing the action strategy. We derive the performance guarantees that depend on network parameters, which further influence the learning curve of the sequential decision strategy We introduce a computationally simplified solving method, Grab-arm-Light, an algorithm that walks along the edges of the polytope representing the objective function. Simulations results show that the proposed online learning algorithm outperforms baseline offline methods that typically separate the learning phase from the testing one. The results confirm the theoretical findings, and further highlight the gain of the proposed online learning strategy in terms of cumulative regret, sample efficiency and computational complexity.
△ Less
Submitted 7 July, 2023;
originally announced July 2023.
-
MiDi: Mixed Graph and 3D Denoising Diffusion for Molecule Generation
Authors:
Clement Vignac,
Nagham Osman,
Laura Toni,
Pascal Frossard
Abstract:
This work introduces MiDi, a novel diffusion model for jointly generating molecular graphs and their corresponding 3D arrangement of atoms. Unlike existing methods that rely on predefined rules to determine molecular bonds based on the 3D conformation, MiDi offers an end-to-end differentiable approach that streamlines the molecule generation process. Our experimental results demonstrate the effect…
▽ More
This work introduces MiDi, a novel diffusion model for jointly generating molecular graphs and their corresponding 3D arrangement of atoms. Unlike existing methods that rely on predefined rules to determine molecular bonds based on the 3D conformation, MiDi offers an end-to-end differentiable approach that streamlines the molecule generation process. Our experimental results demonstrate the effectiveness of this approach. On the challenging GEOM-DRUGS dataset, MiDi generates 92% of stable molecules, against 6% for the previous EDM model that uses interatomic distances for bond prediction, and 40% using EDM followed by an algorithm that directly optimize bond orders for validity. Our code is available at github.com/cvignac/MiDi.
△ Less
Submitted 5 June, 2023; v1 submitted 17 February, 2023;
originally announced February 2023.
-
Information-theoretic Characterizations of Generalization Error for the Gibbs Algorithm
Authors:
Gholamali Aminian,
Yuheng Bu,
Laura Toni,
Miguel R. D. Rodrigues,
Gregory W. Wornell
Abstract:
Various approaches have been developed to upper bound the generalization error of a supervised learning algorithm. However, existing bounds are often loose and even vacuous when evaluated in practice. As a result, they may fail to characterize the exact generalization ability of a learning algorithm. Our main contributions are exact characterizations of the expected generalization error of the wel…
▽ More
Various approaches have been developed to upper bound the generalization error of a supervised learning algorithm. However, existing bounds are often loose and even vacuous when evaluated in practice. As a result, they may fail to characterize the exact generalization ability of a learning algorithm. Our main contributions are exact characterizations of the expected generalization error of the well-known Gibbs algorithm (a.k.a. Gibbs posterior) using different information measures, in particular, the symmetrized KL information between the input training samples and the output hypothesis. Our result can be applied to tighten existing expected generalization error and PAC-Bayesian bounds. Our information-theoretic approach is versatile, as it also characterizes the generalization error of the Gibbs algorithm with a data-dependent regularizer and that of the Gibbs algorithm in the asymptotic regime, where it converges to the standard empirical risk minimization algorithm. Of particular relevance, our results highlight the role the symmetrized KL information plays in controlling the generalization error of the Gibbs algorithm.
△ Less
Submitted 18 October, 2022;
originally announced October 2022.
-
Learning Algorithm Generalization Error Bounds via Auxiliary Distributions
Authors:
Gholamali Aminian,
Saeed Masiha,
Laura Toni,
Miguel R. D. Rodrigues
Abstract:
Generalization error bounds are essential for comprehending how well machine learning models work. In this work, we suggest a novel method, i.e., the Auxiliary Distribution Method, that leads to new upper bounds on expected generalization errors that are appropriate for supervised learning scenarios. We show that our general upper bounds can be specialized under some conditions to new bounds invol…
▽ More
Generalization error bounds are essential for comprehending how well machine learning models work. In this work, we suggest a novel method, i.e., the Auxiliary Distribution Method, that leads to new upper bounds on expected generalization errors that are appropriate for supervised learning scenarios. We show that our general upper bounds can be specialized under some conditions to new bounds involving the $α$-Jensen-Shannon, $α$-Rényi ($0< α< 1$) information between a random variable modeling the set of training samples and another random variable modeling the set of hypotheses. Our upper bounds based on $α$-Jensen-Shannon information are also finite. Additionally, we demonstrate how our auxiliary distribution method can be used to derive the upper bounds on excess risk of some learning algorithms in the supervised learning context {\blue and the generalization error under the distribution mismatch scenario in supervised learning algorithms, where the distribution mismatch is modeled as $α$-Jensen-Shannon or $α$-Rényi divergence between the distribution of test and training data samples distributions.} We also outline the conditions for which our proposed upper bounds might be tighter than other earlier upper bounds.
△ Less
Submitted 16 April, 2024; v1 submitted 2 October, 2022;
originally announced October 2022.
-
Explaining Hierarchical Features in Dynamic Point Cloud Processing
Authors:
Pedro Gomes,
Silvia Rossi,
Laura Toni
Abstract:
This paper aims at bringing some light and understanding to the field of deep learning for dynamic point cloud processing. Specifically, we focus on the hierarchical features learning aspect, with the ultimate goal of understanding which features are learned at the different stages of the process and what their meaning is. Last, we bring clarity on how hierarchical components of the network affect…
▽ More
This paper aims at bringing some light and understanding to the field of deep learning for dynamic point cloud processing. Specifically, we focus on the hierarchical features learning aspect, with the ultimate goal of understanding which features are learned at the different stages of the process and what their meaning is. Last, we bring clarity on how hierarchical components of the network affect the learned features and their importance for a successful learning model. This study is conducted for point cloud prediction tasks, useful for predicting coding applications.
△ Less
Submitted 30 September, 2022;
originally announced September 2022.
-
Semi-supervised Batch Learning From Logged Data
Authors:
Gholamali Aminian,
Armin Behnamnia,
Roberto Vega,
Laura Toni,
Chengchun Shi,
Hamid R. Rabiee,
Omar Rivasplata,
Miguel R. D. Rodrigues
Abstract:
Off-policy learning methods are intended to learn a policy from logged data, which includes context, action, and feedback (cost or reward) for each sample point. In this work, we build on the counterfactual risk minimization framework, which also assumes access to propensity scores. We propose learning methods for problems where feedback is missing for some samples, so there are samples with feedb…
▽ More
Off-policy learning methods are intended to learn a policy from logged data, which includes context, action, and feedback (cost or reward) for each sample point. In this work, we build on the counterfactual risk minimization framework, which also assumes access to propensity scores. We propose learning methods for problems where feedback is missing for some samples, so there are samples with feedback and samples missing-feedback in the logged data. We refer to this type of learning as semi-supervised batch learning from logged data, which arises in a wide range of application domains. We derive a novel upper bound for the true risk under the inverse propensity score estimator to address this kind of learning problem. Using this bound, we propose a regularized semi-supervised batch learning method with logged data where the regularization term is feedback-independent and, as a result, can be evaluated using the logged missing-feedback data. Consequently, even though feedback is only present for some samples, a learning policy can be learned by leveraging the missing-feedback samples. The results of experiments derived from benchmark datasets indicate that these algorithms achieve policies with better performance in comparison with logging policies.
△ Less
Submitted 18 February, 2024; v1 submitted 15 September, 2022;
originally announced September 2022.
-
Group velocity of obliquely propagating Alfvén waves in a magnetized dusty plasma
Authors:
L. B. De Toni,
R. Gaelzer,
L. F. Ziebell
Abstract:
In this work we investigate the characteristics of the group velocity of obliquely propagating Alfvén waves in a dusty plasma typical of a stellar wind. The dispersion relation is derived with the aid of the kinetic theory for a magnetized dusty plasma consisting of electrons and ions, with distribution of momenta described by a Maxwellian function. The dust particles are considered to be immobile…
▽ More
In this work we investigate the characteristics of the group velocity of obliquely propagating Alfvén waves in a dusty plasma typical of a stellar wind. The dispersion relation is derived with the aid of the kinetic theory for a magnetized dusty plasma consisting of electrons and ions, with distribution of momenta described by a Maxwellian function. The dust particles are considered to be immobile and have all the same size; they are electrically charged by absorption of plasma particles via inelastic collisions and by photoionization. We numerically solve the dispersion relation and calculate the components of group velocity (along and transverse to the magnetic field) for the normal modes, namely the compressional and shear Alfvén waves (CAW and SAW). The results show that the direction of the group velocity of CAWs is greatly modified with the wave-vector direction. On the other hand, SAWs will present group velocity propagating practically along the magnetic field. The changes in dust parameters, such as number density and equilibrium electrical charge, may significantly change the waves' characteristics. It is seen that for sufficiently high dust to ion number density ratio, the SAWs may present perpendicular group velocity propagating in opposite direction to the perpendicular phase velocity, in a small interval of wavenumber values; we also notice that this interval may change, or even vanish, when the flux of radiation incident on the dust is altered, changing the equilibrium electrical charge of the grains.
△ Less
Submitted 8 September, 2022;
originally announced September 2022.
-
Oblique Alfvén waves in a stellar wind environment with dust particles charged by inelastic collisions and by photoionization
Authors:
L. B. De Toni,
R. Gaelzer,
L. F. Ziebell
Abstract:
The characteristics of Alfvén waves propagating in a direction oblique to the ambient magnetic field in a stellar wind environment are discussed. A kinetic formulation for a magnetized dusty plasma is adopted considering Maxwellian distributions of electrons and ions, and immobile dust particles electrically charged by absorption of plasma particles and by photoionization. The dispersion relation…
▽ More
The characteristics of Alfvén waves propagating in a direction oblique to the ambient magnetic field in a stellar wind environment are discussed. A kinetic formulation for a magnetized dusty plasma is adopted considering Maxwellian distributions of electrons and ions, and immobile dust particles electrically charged by absorption of plasma particles and by photoionization. The dispersion relation is numerically solved and the results are compared with situations previously studied where dust particles were not charged by photoionization, which is an important process in a stellar wind of a relatively hot star. We show that the presence of dust causes the shear Alfvén waves to present a region of wavenumber values with zero frequency and that the minimum wavelength for which the mode becomes dispersive again is roughly proportional to the radiation intensity to which the dust grains are exposed. The dam** rates of both shear and compressional Alfvén waves are observed to decrease with increasing radiation flux, for the parameters considered. For the particular case where both modes present a region with null real frequency when the radiation flux is absent or weak, it is shown that when the radiation flux is sufficiently strong, the photoionization mechanism may cause this region to get smaller or even to vanish, for compressional Alfvén waves. In that case, the compressional Alfvén waves present non zero frequency for all wavenumber values, while the shear Alfvén waves still present null frequency in a certain interval of wavenumber values, which gets smaller with the presence of radiation.
△ Less
Submitted 20 March, 2022;
originally announced March 2022.
-
An Information-theoretical Approach to Semi-supervised Learning under Covariate-shift
Authors:
Gholamali Aminian,
Mahed Abroshan,
Mohammad Mahdi Khalili,
Laura Toni,
Miguel R. D. Rodrigues
Abstract:
A common assumption in semi-supervised learning is that the labeled, unlabeled, and test data are drawn from the same distribution. However, this assumption is not satisfied in many applications. In many scenarios, the data is collected sequentially (e.g., healthcare) and the distribution of the data may change over time often exhibiting so-called covariate shifts. In this paper, we propose an app…
▽ More
A common assumption in semi-supervised learning is that the labeled, unlabeled, and test data are drawn from the same distribution. However, this assumption is not satisfied in many applications. In many scenarios, the data is collected sequentially (e.g., healthcare) and the distribution of the data may change over time often exhibiting so-called covariate shifts. In this paper, we propose an approach for semi-supervised learning algorithms that is capable of addressing this issue. Our framework also recovers some popular methods, including entropy minimization and pseudo-labeling. We provide new information-theoretical based generalization error upper bounds inspired by our novel framework. Our bounds are applicable to both general semi-supervised learning and the covariate-shift scenario. Finally, we show numerically that our method outperforms previous approaches proposed for semi-supervised learning under the covariate shift.
△ Less
Submitted 24 February, 2022;
originally announced February 2022.
-
Extending 3-DoF Metrics to Model User Behaviour Similarity in 6-DoF Immersive Applications
Authors:
Silvia Rossi,
Irene Viola,
Laura Toni,
Pablo Cesar
Abstract:
Immersive reality technologies, such as Virtual and Augmented Reality, have ushered a new era of user-centric systems, in which every aspect of the coding--delivery--rendering chain is tailored to the interaction of the users. Understanding the actual interactivity and behaviour of the users is still an open challenge and a key step to enabling such a user-centric system. Our main goal is to exten…
▽ More
Immersive reality technologies, such as Virtual and Augmented Reality, have ushered a new era of user-centric systems, in which every aspect of the coding--delivery--rendering chain is tailored to the interaction of the users. Understanding the actual interactivity and behaviour of the users is still an open challenge and a key step to enabling such a user-centric system. Our main goal is to extend the applicability of existing behavioural methodologies for studying user navigation in the case of 6 Degree-of-Freedom (DoF). Specifically, we first compare the navigation in 6-DoF with its 3-DoF counterpart highlighting the main differences and novelties. Then, we define new metrics aimed at better modelling behavioural similarities between users in a 6-DoF system. We validate and test our solutions on real navigation paths of users interacting with dynamic volumetric media in 6-DoF Virtual Reality conditions. Our results show that metrics that consider both user position and viewing direction better perform in detecting user similarity while navigating in a 6-DoF system. Having easy-to-use but robust metrics that underpin multiple tools and answer the question ``how do we detect if two users look at the same content?" open the gate to new solutions for a user-centric system.
△ Less
Submitted 20 June, 2023; v1 submitted 17 December, 2021;
originally announced December 2021.
-
Characterizing and Understanding the Generalization Error of Transfer Learning with Gibbs Algorithm
Authors:
Yuheng Bu,
Gholamali Aminian,
Laura Toni,
Miguel Rodrigues,
Gregory Wornell
Abstract:
We provide an information-theoretic analysis of the generalization ability of Gibbs-based transfer learning algorithms by focusing on two popular transfer learning approaches, $α$-weighted-ERM and two-stage-ERM. Our key result is an exact characterization of the generalization behaviour using the conditional symmetrized KL information between the output hypothesis and the target training samples g…
▽ More
We provide an information-theoretic analysis of the generalization ability of Gibbs-based transfer learning algorithms by focusing on two popular transfer learning approaches, $α$-weighted-ERM and two-stage-ERM. Our key result is an exact characterization of the generalization behaviour using the conditional symmetrized KL information between the output hypothesis and the target training samples given the source samples. Our results can also be applied to provide novel distribution-free generalization error upper bounds on these two aforementioned Gibbs algorithms. Our approach is versatile, as it also characterizes the generalization errors and excess risks of these two Gibbs algorithms in the asymptotic regime, where they converge to the $α$-weighted-ERM and two-stage-ERM, respectively. Based on our theoretical results, we show that the benefits of transfer learning can be viewed as a bias-variance trade-off, with the bias induced by the source distribution and the variance induced by the lack of target samples. We believe this viewpoint can guide the choice of transfer learning algorithms in practice.
△ Less
Submitted 2 November, 2021;
originally announced November 2021.
-
Effects of dust particles charged by inelastic collisions and by photoionization on Alfvén waves in a stellar wind
Authors:
L. B. De Toni,
R. Gaelzer
Abstract:
Using a kinetic description of a homogeneous magnetized dusty plasma with Maxwellian distribution of electrons and protons and dust particles charged by inelastic collisions and by photoionization, we analyse the dispersion relation considering the case where waves and radiation propagate exactly parallel to the ambient magnetic field. The investigation emphasizes the changes that the photoionizat…
▽ More
Using a kinetic description of a homogeneous magnetized dusty plasma with Maxwellian distribution of electrons and protons and dust particles charged by inelastic collisions and by photoionization, we analyse the dispersion relation considering the case where waves and radiation propagate exactly parallel to the ambient magnetic field. The investigation emphasizes the changes that the photoionization process brings to the propagation and dam** of the waves in a stellar wind environment, since Alfvén waves are believed to play a significant role in the heating and acceleration processes that take place in the wind. The results show that, in the presence of dust with negative equilibrium electrical charge, the Alfvén mode decouples into the whistler and ion cyclotron modes for all values of wavenumber, but when dust particles acquire neutral or positive values of electrical charge, these modes may couple for certain values of wavenumber. It is also seen that the whistler and ion cyclotron modes present null group velocity in a interval of small wavenumber, and that the maximum value of wavenumber for which the waves are non-propagating is reduced in the presence of the photoionization process. For very small values of wavenumber, the dam** rates of the modes could change significantly from very small to very high values if the sign of the dust electrical charge is changed.
△ Less
Submitted 13 September, 2021;
originally announced September 2021.
-
Characterizing the Generalization Error of Gibbs Algorithm with Symmetrized KL information
Authors:
Gholamali Aminian,
Yuheng Bu,
Laura Toni,
Miguel R. D. Rodrigues,
Gregory Wornell
Abstract:
Bounding the generalization error of a supervised learning algorithm is one of the most important problems in learning theory, and various approaches have been developed. However, existing bounds are often loose and lack of guarantees. As a result, they may fail to characterize the exact generalization ability of a learning algorithm. Our main contribution is an exact characterization of the expec…
▽ More
Bounding the generalization error of a supervised learning algorithm is one of the most important problems in learning theory, and various approaches have been developed. However, existing bounds are often loose and lack of guarantees. As a result, they may fail to characterize the exact generalization ability of a learning algorithm. Our main contribution is an exact characterization of the expected generalization error of the well-known Gibbs algorithm in terms of symmetrized KL information between the input training samples and the output hypothesis. Such a result can be applied to tighten existing expected generalization error bound. Our analysis provides more insight on the fundamental role the symmetrized KL information plays in controlling the generalization error of the Gibbs algorithm.
△ Less
Submitted 28 July, 2021;
originally announced July 2021.
-
Spatio-temporal Graph-RNN for Point Cloud Prediction
Authors:
Pedro Gomes,
Silvia Rossi,
Laura Toni
Abstract:
In this paper, we propose an end-to-end learning network to predict future frames in a point cloud sequence. As main novelty, an initial layer learns topological information of point clouds as geometric features, to form representative spatio-temporal neighborhoods. This module is followed by multiple Graph-RNN cells. Each cell learns points dynamics (i.e., RNN states) by processing each point joi…
▽ More
In this paper, we propose an end-to-end learning network to predict future frames in a point cloud sequence. As main novelty, an initial layer learns topological information of point clouds as geometric features, to form representative spatio-temporal neighborhoods. This module is followed by multiple Graph-RNN cells. Each cell learns points dynamics (i.e., RNN states) by processing each point jointly with the spatio-temporal neighbouring points. We tested the network performance with a MINST dataset of moving digits, a synthetic human bodies motions and JPEG dynamic bodies datasets. Simulation results demonstrate that our method outperforms baseline ones that neglect geometry features information.
△ Less
Submitted 22 February, 2021; v1 submitted 15 February, 2021;
originally announced February 2021.
-
Information-Theoretic Bounds on the Moments of the Generalization Error of Learning Algorithms
Authors:
Gholamali Aminian,
Laura Toni,
Miguel R. D. Rodrigues
Abstract:
Generalization error bounds are critical to understanding the performance of machine learning models. In this work, building upon a new bound of the expected value of an arbitrary function of the population and empirical risk of a learning algorithm, we offer a more refined analysis of the generalization behaviour of a machine learning models based on a characterization of (bounds) to their genera…
▽ More
Generalization error bounds are critical to understanding the performance of machine learning models. In this work, building upon a new bound of the expected value of an arbitrary function of the population and empirical risk of a learning algorithm, we offer a more refined analysis of the generalization behaviour of a machine learning models based on a characterization of (bounds) to their generalization error moments. We discuss how the proposed bounds -- which also encompass new bounds to the expected generalization error -- relate to existing bounds in the literature. We also discuss how the proposed generalization error moment bounds can be used to construct new generalization error high-probability bounds.
△ Less
Submitted 5 May, 2021; v1 submitted 3 February, 2021;
originally announced February 2021.
-
Jensen-Shannon Information Based Characterization of the Generalization Error of Learning Algorithms
Authors:
Gholamali Aminian,
Laura Toni,
Miguel R. D. Rodrigues
Abstract:
Generalization error bounds are critical to understanding the performance of machine learning models. In this work, we propose a new information-theoretic based generalization error upper bound applicable to supervised learning scenarios. We show that our general bound can specialize in various previous bounds. We also show that our general bound can be specialized under some conditions to a new b…
▽ More
Generalization error bounds are critical to understanding the performance of machine learning models. In this work, we propose a new information-theoretic based generalization error upper bound applicable to supervised learning scenarios. We show that our general bound can specialize in various previous bounds. We also show that our general bound can be specialized under some conditions to a new bound involving the Jensen-Shannon information between a random variable modelling the set of training samples and another random variable modelling the hypothesis. We also prove that our bound can be tighter than mutual information-based bounds under some conditions.
△ Less
Submitted 8 January, 2021; v1 submitted 23 October, 2020;
originally announced October 2020.
-
Graph signal processing for machine learning: A review and new perspectives
Authors:
Xiaowen Dong,
Dorina Thanou,
Laura Toni,
Michael Bronstein,
Pascal Frossard
Abstract:
The effective representation, processing, analysis, and visualization of large-scale structured data, especially those related to complex domains such as networks and graphs, are one of the key questions in modern machine learning. Graph signal processing (GSP), a vibrant branch of signal processing models and algorithms that aims at handling data supported on graphs, opens new paths of research t…
▽ More
The effective representation, processing, analysis, and visualization of large-scale structured data, especially those related to complex domains such as networks and graphs, are one of the key questions in modern machine learning. Graph signal processing (GSP), a vibrant branch of signal processing models and algorithms that aims at handling data supported on graphs, opens new paths of research to address this challenge. In this article, we review a few important contributions made by GSP concepts and tools, such as graph filters and transforms, to the development of novel machine learning algorithms. In particular, our discussion focuses on the following three aspects: exploiting data structure and relational priors, improving data and computational efficiency, and enhancing model interpretability. Furthermore, we provide new perspectives on future development of GSP techniques that may serve as a bridge between applied mathematics and signal processing on one side, and machine learning and network science on the other. Cross-fertilization across these different disciplines may help unlock the numerous challenges of complex data analysis in the modern age.
△ Less
Submitted 31 July, 2020;
originally announced July 2020.
-
Differentiable Linear Bandit Algorithm
Authors:
Kaige Yang,
Laura Toni
Abstract:
Upper Confidence Bound (UCB) is arguably the most commonly used method for linear multi-arm bandit problems. While conceptually and computationally simple, this method highly relies on the confidence bounds, failing to strike the optimal exploration-exploitation if these bounds are not properly set. In the literature, confidence bounds are typically derived from concentration inequalities based on…
▽ More
Upper Confidence Bound (UCB) is arguably the most commonly used method for linear multi-arm bandit problems. While conceptually and computationally simple, this method highly relies on the confidence bounds, failing to strike the optimal exploration-exploitation if these bounds are not properly set. In the literature, confidence bounds are typically derived from concentration inequalities based on assumptions on the reward distribution, e.g., sub-Gaussianity. The validity of these assumptions however is unknown in practice. In this work, we aim at learning the confidence bound in a data-driven fashion, making it adaptive to the actual problem structure. Specifically, noting that existing UCB-typed algorithms are not differentiable with respect to confidence bound, we first propose a novel differentiable linear bandit algorithm. Then, we introduce a gradient estimator, which allows the confidence bound to be learned via gradient ascent. Theoretically, we show that the proposed algorithm achieves a $\tilde{\mathcal{O}}(\hatβ\sqrt{dT})$ upper bound of $T$-round regret, where $d$ is the dimension of arm features and $\hatβ$ is the learned size of confidence bound. Empirical results show that $\hatβ$ is significantly smaller than its theoretical upper bound and proposed algorithms outperforms baseline ones on both simulated and real-world datasets.
△ Less
Submitted 4 June, 2020;
originally announced June 2020.
-
State2vec: Off-Policy Successor Features Approximators
Authors:
Sephora Madjiheurem,
Laura Toni
Abstract:
A major challenge in reinforcement learning (RL) is the design of agents that are able to generalize across tasks that share common dynamics. A viable solution is meta-reinforcement learning, which identifies common structures among past tasks to be then generalized to new tasks (meta-test). In meta-training, the RL agent learns state representations that encode prior information from a set of tas…
▽ More
A major challenge in reinforcement learning (RL) is the design of agents that are able to generalize across tasks that share common dynamics. A viable solution is meta-reinforcement learning, which identifies common structures among past tasks to be then generalized to new tasks (meta-test). In meta-training, the RL agent learns state representations that encode prior information from a set of tasks, used to generalize the value function approximation. This has been proposed in the literature as successor representation approximators. While promising, these methods do not generalize well across optimal policies, leading to sampling-inefficiency during meta-test phases. In this paper, we propose state2vec, an efficient and low-complexity framework for learning successor features which (i) generalize across policies, (ii) ensure sample-efficiency during meta-test. We extend the well known node2vec framework to learn state embeddings that account for the discounted future state transitions in RL. The proposed off-policy state2vec captures the geometry of the underlying state space, making good basis functions for linear value function approximation.
△ Less
Submitted 22 October, 2019;
originally announced October 2019.
-
Laplacian-regularized graph bandits: Algorithms and theoretical analysis
Authors:
Kaige Yang,
Xiaowen Dong,
Laura Toni
Abstract:
We consider a stochastic linear bandit problem with multiple users, where the relationship between users is captured by an underlying graph and user preferences are represented as smooth signals on the graph. We introduce a novel bandit algorithm where the smoothness prior is imposed via the random-walk graph Laplacian, which leads to a single-user cumulative regret scaling as…
▽ More
We consider a stochastic linear bandit problem with multiple users, where the relationship between users is captured by an underlying graph and user preferences are represented as smooth signals on the graph. We introduce a novel bandit algorithm where the smoothness prior is imposed via the random-walk graph Laplacian, which leads to a single-user cumulative regret scaling as $\tilde{\mathcal{O}}(Ψd \sqrt{T})$ with time horizon $T$, feature dimensionality $d$, and the scalar parameter $Ψ\in (0,1)$ that depends on the graph connectivity. This is an improvement over $\tilde{\mathcal{O}}(d \sqrt{T})$ in \algo{LinUCB}~\Ccite{li2010contextual}, where user relationship is not taken into account. In terms of network regret (sum of cumulative regret over $n$ users), the proposed algorithm leads to a scaling as $\tilde{\mathcal{O}}(Ψd\sqrt{nT})$, which is a significant improvement over $\tilde{\mathcal{O}}(nd\sqrt{T})$ in the state-of-the-art algorithm \algo{Gob.Lin} \Ccite{cesa2013gang}. To improve scalability, we further propose a simplified algorithm with a linear computational complexity with respect to the number of users, while maintaining the same regret. Finally, we present a finite-time analysis on the proposed algorithms, and demonstrate their advantage in comparison with state-of-the-art graph-based bandit algorithms on both synthetic and real-world data.
△ Less
Submitted 10 February, 2020; v1 submitted 12 July, 2019;
originally announced July 2019.
-
Error Analysis on Graph Laplacian Regularized Estimator
Authors:
Kaige Yang,
Xiaowen Dong,
Laura Toni
Abstract:
We provide a theoretical analysis of the representation learning problem aimed at learning the latent variables (design matrix) $Θ$ of observations $Y$ with the knowledge of the coefficient matrix $X$. The design matrix is learned under the assumption that the latent variables $Θ$ are smooth with respect to a (known) topological structure $\mathcal{G}$. To learn such latent variables, we study a g…
▽ More
We provide a theoretical analysis of the representation learning problem aimed at learning the latent variables (design matrix) $Θ$ of observations $Y$ with the knowledge of the coefficient matrix $X$. The design matrix is learned under the assumption that the latent variables $Θ$ are smooth with respect to a (known) topological structure $\mathcal{G}$. To learn such latent variables, we study a graph Laplacian regularized estimator, which is the penalized least squares estimator with penalty term proportional to a Laplacian quadratic form. This type of estimators has recently received considerable attention due to its capability in incorporating underlying topological graph structure of variables into the learning process. While the estimation problem can be solved efficiently by state-of-the-art optimization techniques, its statistical consistency properties have been largely overlooked. In this work, we develop a non-asymptotic bound of estimation error under the classical statistical setting, where sample size is larger than the ambient dimension of the latent variables. This bound illustrates theoretically the impact of the alignment between the data and the graph structure as well as the graph spectrum on the estimation accuracy. It also provides theoretical evidence of the advantage, in terms of convergence rate, of the graph Laplacian regularized estimator over classical ones (that ignore the graph structure) in case of a smoothness prior. Finally, we provide empirical results of the estimation error to corroborate the theoretical analysis.
△ Less
Submitted 10 February, 2019;
originally announced February 2019.
-
Representation Learning on Graphs: A Reinforcement Learning Application
Authors:
Sephora Madjiheurem,
Laura Toni
Abstract:
In this work, we study value function approximation in reinforcement learning (RL) problems with high dimensional state or action spaces via a generalized version of representation policy iteration (RPI). We consider the limitations of proto-value functions (PVFs) at accurately approximating the value function in low dimensions and we highlight the importance of features learning for an improved l…
▽ More
In this work, we study value function approximation in reinforcement learning (RL) problems with high dimensional state or action spaces via a generalized version of representation policy iteration (RPI). We consider the limitations of proto-value functions (PVFs) at accurately approximating the value function in low dimensions and we highlight the importance of features learning for an improved low-dimensional value function approximation. Then, we adopt different representation learning algorithm on graphs to learn the basis functions that best represent the value function. We empirically show that node2vec, an algorithm for scalable feature learning in networks, and the Variational Graph Auto-Encoder constantly outperform the commonly used smooth proto-value functions in low-dimensional feature space.
△ Less
Submitted 17 January, 2019; v1 submitted 16 January, 2019;
originally announced January 2019.
-
Spherical clustering of users navigating 360° content
Authors:
Silvia Rossi,
Francesca De Simone,
Pascal Frossard,
Laura Toni
Abstract:
In Virtual Reality (VR) applications, understanding how users explore the omnidirectional content is important to optimize content creation, to develop user-centric services, or even to detect disorders in medical applications. Clustering users based on their common navigation patterns is a first direction to understand users behaviour. However, classical clustering techniques fail in identifying…
▽ More
In Virtual Reality (VR) applications, understanding how users explore the omnidirectional content is important to optimize content creation, to develop user-centric services, or even to detect disorders in medical applications. Clustering users based on their common navigation patterns is a first direction to understand users behaviour. However, classical clustering techniques fail in identifying these common paths, since they are usually focused on minimizing a simple distance metric. In this paper, we argue that minimizing the distance metric does not necessarily guarantee to identify users that experience similar navigation path in the VR domain. Therefore, we propose a graph-based method to identify clusters of users who are attending the same portion of the spherical content over time. The proposed solution takes into account the spherical geometry of the content and aims at clustering users based on the actual overlap of displayed content among users. Our method is tested on real VR user navigation patterns. Results show that our solution leads to clusters in which at least 85% of the content displayed by one user is shared among the other users belonging to the same cluster.
△ Less
Submitted 5 May, 2020; v1 submitted 13 November, 2018;
originally announced November 2018.
-
Graph-Based Recommendation System
Authors:
Kaige Yang,
Laura Toni
Abstract:
In this work, we study recommendation systems modelled as contextual multi-armed bandit (MAB) problems. We propose a graph-based recommendation system that learns and exploits the geometry of the user space to create meaningful clusters in the user domain. This reduces the dimensionality of the recommendation problem while preserving the accuracy of MAB. We then study the effect of graph sparsity…
▽ More
In this work, we study recommendation systems modelled as contextual multi-armed bandit (MAB) problems. We propose a graph-based recommendation system that learns and exploits the geometry of the user space to create meaningful clusters in the user domain. This reduces the dimensionality of the recommendation problem while preserving the accuracy of MAB. We then study the effect of graph sparsity and clusters size on the MAB performance and provide exhaustive simulation results both in synthetic and in real-case datasets. Simulation results show improvements with respect to state-of-the-art MAB algorithms.
△ Less
Submitted 31 July, 2018;
originally announced August 2018.
-
The Sum-Rate-Distortion Region of Correlated Gauss-Markov Sources
Authors:
Giuseppe Cocco,
Laura Toni
Abstract:
Efficient low-delay video encoders are of fundamental importance to provide timely feedback in remotely controlled platforms such as drones. In order to fully understand the theoretical limits of low-delay video encoders, we consider an ideal differential predictive coded modulation (DPCM) encoder and provide the explicit derivation of the sum-rate-distortion region for a generic number of success…
▽ More
Efficient low-delay video encoders are of fundamental importance to provide timely feedback in remotely controlled platforms such as drones. In order to fully understand the theoretical limits of low-delay video encoders, we consider an ideal differential predictive coded modulation (DPCM) encoder and provide the explicit derivation of the sum-rate-distortion region for a generic number of successive correlated Gauss-Markov sources along the line of the work by Ma and Ishwar. Furthermore, we provide an upper bound on the minimum distortion achievable in case an arbitrary number of sources are not available at the decoder.
△ Less
Submitted 14 November, 2018; v1 submitted 10 April, 2018;
originally announced April 2018.
-
Adaptive Streaming in Interactive Multiview Video Systems
Authors:
Xue Zhang,
Laura Toni,
Pascal Frossard,
Yao Zhao,
Chunyu Lin
Abstract:
Multiview applications endow final users with the possibility to freely navigate within 3D scenes with minimum-delay. A real feeling of scene navigation is enabled by transmitting multiple high-quality camera views, which can be used to synthesize additional virtual views to offer a smooth navigation. However, when network resources are limited, not all camera views can be sent at high quality. It…
▽ More
Multiview applications endow final users with the possibility to freely navigate within 3D scenes with minimum-delay. A real feeling of scene navigation is enabled by transmitting multiple high-quality camera views, which can be used to synthesize additional virtual views to offer a smooth navigation. However, when network resources are limited, not all camera views can be sent at high quality. It is therefore important, yet challenging, to find the right tradeoff between coding artifacts (reducing the quality of camera views) and virtual synthesis artifacts (reducing the number of camera views sent to users). To this aim, we propose an optimal transmission strategy for interactive multiview HTTP adaptive streaming (HAS). We propose a problem formulation to select the optimal set of camera views that the client requests for downloading, such that the navigation quality experienced by the user is optimized while the bandwidth constraints are satisfied. We show that our optimization problem is NP-hard, and we therefore develop an optimal solution based on the dynamic programming algorithm with polynomial time complexity. To further simplify the deployment, we present a suboptimal greedy algorithm with effective performance and lower complexity. The proposed controller is evaluated in theoretical and realistic settings characterized by realistic network statistics estimation, buffer management and server-side representation optimization. Simulation results show significant improvement in terms of navigation quality compared with alternative baseline multiview adaptation logic solutions.
△ Less
Submitted 25 March, 2018; v1 submitted 25 February, 2018;
originally announced February 2018.
-
IRSA Transmission Optimization via Online Learning
Authors:
Laura Toni,
Pascal Frossard
Abstract:
In this work, we propose a new learning framework for optimising transmission strategies when irregular repetition slotted ALOHA (IRSA) MAC protocol is considered. We cast the online optimisation of the MAC protocol design as a multi-arm bandit problem that exploits the IRSA structure in the learning framework. Our learning algorithm quickly learns the optimal transmission strategy, leading to hig…
▽ More
In this work, we propose a new learning framework for optimising transmission strategies when irregular repetition slotted ALOHA (IRSA) MAC protocol is considered. We cast the online optimisation of the MAC protocol design as a multi-arm bandit problem that exploits the IRSA structure in the learning framework. Our learning algorithm quickly learns the optimal transmission strategy, leading to higher rate of successfully received packets with respect to baseline transmission optimizations.
△ Less
Submitted 27 January, 2018;
originally announced January 2018.
-
Price-based Controller for Quality-Fair HTTP Adaptive Streaming (Extended Version)
Authors:
Stefano D'Aronco,
Laura Toni,
Pascal Frossard
Abstract:
HTTP adaptive streaming (HAS) has become the universal technology for video streaming over the Internet. Many HAS system designs aim at sharing the network bandwidth in a rate-fair manner. However, rate fairness is in general not equivalent to quality fairness as different video sequences might have different characteristics and resource requirements. In this work, we focus on this limitation and…
▽ More
HTTP adaptive streaming (HAS) has become the universal technology for video streaming over the Internet. Many HAS system designs aim at sharing the network bandwidth in a rate-fair manner. However, rate fairness is in general not equivalent to quality fairness as different video sequences might have different characteristics and resource requirements. In this work, we focus on this limitation and propose a novel controller for HAS clients that is able to reach quality fairness while preserving the main characteristics of HAS systems and with a limited support from the network devices. In particular, we adopt a price-based mechanism in order to build a controller that maximizes the aggregate video quality for a set of HAS clients that share a common bottleneck. When network resources are scarce, the clients with simple video sequences reduce the requested bitrate in favor of users that subscribe to more complex video sequences, leading to a more efficient network usage. The proposed controller has been implemented in a network simulator, and the simulation results demonstrate its ability to share the available bandwidth among the HAS users in a quality-fair manner.
△ Less
Submitted 5 January, 2017;
originally announced January 2017.
-
Joint Source, Channel and Space-time Coding of Progressive Sources in MIMO Systems
Authors:
Meesue Shin,
Laura Toni,
Sang-Hyo Kim,
Seok-Ho Chang
Abstract:
The optimization of joint source and channel coding for a sequence of numerous progressive packets is a challenging problem. Further, the problem becomes more complicated if the space-time coding is also involved with the optimization in a multiple-input multiple-output (MIMO) system. This is because the number of ways of jointly assigning channels codes and space-time codes to progressive packets…
▽ More
The optimization of joint source and channel coding for a sequence of numerous progressive packets is a challenging problem. Further, the problem becomes more complicated if the space-time coding is also involved with the optimization in a multiple-input multiple-output (MIMO) system. This is because the number of ways of jointly assigning channels codes and space-time codes to progressive packets is much larger than that of solely assigning channel codes to the packets. We are unaware of any feasible and complete solution for such optimization of joint source, channel, and space-time coding of progressive packets. This paper applies a parametric approach to address that complex joint optimization problem in a MIMO system. We use the parametric methodology to derive some useful theoretical results, and then exploit those results to propose an optimization method where the joint assignment of channel codes and space-time codes to the packets can be optimized in a packet-by-packet manner. As a result, the computational complexity of the optimization is exponentially reduced, compared to the conventional exhaustive search. The numerical results show that the proposed method significantly improves the peak-signal-to-noise ratio performance of the rate-based optimal solution in a MIMO system.
△ Less
Submitted 14 September, 2017; v1 submitted 14 October, 2016;
originally announced October 2016.
-
Optimal Representations for Adaptive Streaming in Interactive Multi-View Video Systems
Authors:
Laura Toni,
Pascal Frossard
Abstract:
Interactive multi-view video streaming (IMVS) services permit to remotely immerse within a 3D scene. This is possible by transmitting a set of reference camera views (anchor views), which are used by the clients to freely navigate in the scene and possibly synthesize additional viewpoints of interest. From a networking perspective, the big challenge in IMVS systems is to deliver to each client the…
▽ More
Interactive multi-view video streaming (IMVS) services permit to remotely immerse within a 3D scene. This is possible by transmitting a set of reference camera views (anchor views), which are used by the clients to freely navigate in the scene and possibly synthesize additional viewpoints of interest. From a networking perspective, the big challenge in IMVS systems is to deliver to each client the best set of anchor views that maximizes the navigation quality, minimizes the view-switching delay and yet satisfies the network constraints. Integrating adaptive streaming solutions in free-viewpoint systems offers a promising solution to deploy IMVS in large and heterogeneous scenarios, as long as the multi-view video representations on the server are properly selected. We therefore propose to optimize the multi-view data at the server by minimizing the overall resource requirements, yet offering a good navigation quality to the different users. We propose a video representation set optimization for multiview adaptive streaming systems and we show that it is NP-hard. We therefore introduce the concept of multi-view navigation segment that permits to cast the video representation set selection as an integer linear programming problem with a bounded computational complexity. We then show that the proposed solution reduces the computational complexity while preserving optimality in most of the 3D scenes. We then provide simulation results for different classes of users and show the gain offered by an optimal multi-view video representation selection compared to recommended representation sets (e.g., Netflix and Apple ones) or to a baseline representation selection algorithm where the encoding parameters are decided a priori for all the views.
△ Less
Submitted 14 September, 2016;
originally announced September 2016.
-
Finite Length Performance of Random Slotted ALOHA Strategies
Authors:
Konstantinos Dovelos,
Laura Toni,
Pascal Frossard
Abstract:
Multiple connected devices sharing common wireless resources might create interference if they access the channel simultaneously. Medium access control (MAC) protocols gener- ally regulate the access of the devices to the shared channel to limit signal interference. In particular, irregular repetition slotted ALOHA (IRSA) techniques can achieve high-throughput performance when interference cancell…
▽ More
Multiple connected devices sharing common wireless resources might create interference if they access the channel simultaneously. Medium access control (MAC) protocols gener- ally regulate the access of the devices to the shared channel to limit signal interference. In particular, irregular repetition slotted ALOHA (IRSA) techniques can achieve high-throughput performance when interference cancellation methods are adopted to recover from collisions. In this work, we study the finite length performance for IRSA schemes by building on the analogy between successive interference cancellation and iterative belief- propagation on erasure channels. We use a novel combinatorial derivation based on the matrix-occupancy theory to compute the error probability and we validate our method with simulation results.
△ Less
Submitted 8 July, 2016;
originally announced July 2016.
-
In-Network View Synthesis for Interactive Multiview Video Systems
Authors:
Laura Toni,
Gene Cheung,
Pascal Frossard
Abstract:
To enable Interactive multiview video systems with a minimum view-switching delay, multiple camera views are sent to the users, which are used as reference images to synthesize additional virtual views via depth-image-based rendering. In practice, bandwidth constraints may however restrict the number of reference views sent to clients per time unit, which may in turn limit the quality of the synth…
▽ More
To enable Interactive multiview video systems with a minimum view-switching delay, multiple camera views are sent to the users, which are used as reference images to synthesize additional virtual views via depth-image-based rendering. In practice, bandwidth constraints may however restrict the number of reference views sent to clients per time unit, which may in turn limit the quality of the synthesized viewpoints. We argue that the reference view selection should ideally be performed close to the users, and we study the problem of in-network reference view synthesis such that the navigation quality is maximized at the clients. We consider a distributed cloud network architecture where data stored in a main cloud is delivered to end users with the help of cloudlets, i.e., resource-rich proxies close to the users. In order to satisfy last-hop bandwidth constraints from the cloudlet to the users, a cloudlet re-samples viewpoints of the 3D scene into a discrete set of views (combination of received camera views and virtual views synthesized) to be used as reference for the synthesis of additional virtual views at the client. This in-network synthesis leads to better viewpoint sampling given a bandwidth constraint compared to simple selection of camera views, but it may however carry a distortion penalty in the cloudlet-synthesized reference views. We therefore cast a new reference view selection problem where the best subset of views is defined as the one minimizing the distortion over a view navigation window defined by the user under some transmission bandwidth constraints. We show that the view selection problem is NP-hard, and propose an effective polynomial time algorithm using dynamic programming to solve the optimization problem. Simulation results finally confirm the performance gain offered by virtual view synthesis in the network.
△ Less
Submitted 1 September, 2015;
originally announced September 2015.
-
Optimal Layered Representation for Adaptive Interactive Multiview Video Streaming
Authors:
Ana De Abreu,
Laura Toni,
Nikolaos Thomos,
Thomas Maugey,
Fernando Pereira,
Pascal Frossard
Abstract:
We consider an interactive multiview video streaming (IMVS) system where clients select their preferred viewpoint in a given navigation window. To provide high quality IMVS, many high quality views should be transmitted to the clients. However, this is not always possible due to the limited and heterogeneous capabilities of the clients. In this paper, we propose a novel adaptive IMVS solution base…
▽ More
We consider an interactive multiview video streaming (IMVS) system where clients select their preferred viewpoint in a given navigation window. To provide high quality IMVS, many high quality views should be transmitted to the clients. However, this is not always possible due to the limited and heterogeneous capabilities of the clients. In this paper, we propose a novel adaptive IMVS solution based on a layered multiview representation where camera views are organized into layered subsets to match the different clients constraints. We formulate an optimization problem for the joint selection of the views subsets and their encoding rates. Then, we propose an optimal and a reduced computational complexity greedy algorithms, both based on dynamic-programming. Simulation results show the good performance of our novel algorithms compared to a baseline algorithm, proving that an effective IMVS adaptive solution should consider the scene content and the client capabilities and their preferences in navigation.
△ Less
Submitted 25 June, 2015;
originally announced June 2015.
-
Improved Utility-based Congestion Control for Delay-Constrained Communication
Authors:
Stefano D'Aronco,
Laura Toni,
Sergio Mena,
Xiaoqing Zhu,
Pascal Frossard
Abstract:
Due to the presence of buffers in the inner network nodes, each congestion event leads to buffer queueing and thus to an increasing end-to-end delay. In the case of delay sensitive applications, a large delay might not be acceptable and a solution to properly manage congestion events while maintaining a low end-to-end delay is required. Delay-based congestion algorithms are a viable solution as th…
▽ More
Due to the presence of buffers in the inner network nodes, each congestion event leads to buffer queueing and thus to an increasing end-to-end delay. In the case of delay sensitive applications, a large delay might not be acceptable and a solution to properly manage congestion events while maintaining a low end-to-end delay is required. Delay-based congestion algorithms are a viable solution as they target to limit the experienced end-to-end delay. Unfortunately, they do not perform well when sharing the bandwidth with congestion control algorithms not regulated by delay constraints (e.g., loss-based algorithms). Our target is to fill this gap, proposing a novel congestion control algorithm for delay-constrained communication over best effort packet switched networks. The proposed algorithm is able to maintain a bounded queueing delay when competing with other delay-based flows, and avoid starvation when competing with loss-based flows. We adopt the well-known price-based distributed mechanism as congestion control, but: 1) we introduce a novel non-linear map** between the experienced delay and the price function and 2) we combine both delay and loss information into a single price term based on packet interarrival measurements. We then provide a stability analysis for our novel algorithm and we show its performance in the simulation results carried out in the NS3 framework. Simulation results demonstrate that the proposed algorithm is able to: achieve good intra-protocol fairness properties, control efficiently the end-to-end delay, and finally, protect the flow from starvation when other flows cause the queuing delay to grow excessively.
△ Less
Submitted 20 January, 2017; v1 submitted 9 June, 2015;
originally announced June 2015.
-
Prioritized Random MAC Optimization via Graph-based Analysis
Authors:
Laura Toni,
Pascal Frossard
Abstract:
Motivated by the analogy between successive interference cancellation and iterative belief-propagation on erasure channels, irregular repetition slotted ALOHA (IRSA) strategies have received a lot of attention in the design of medium access control protocols. The IRSA schemes have been mostly analyzed for theoretical scenarios for homogenous sources, where they are shown to substantially improve t…
▽ More
Motivated by the analogy between successive interference cancellation and iterative belief-propagation on erasure channels, irregular repetition slotted ALOHA (IRSA) strategies have received a lot of attention in the design of medium access control protocols. The IRSA schemes have been mostly analyzed for theoretical scenarios for homogenous sources, where they are shown to substantially improve the system performance compared to classical slotted ALOHA protocols. In this work, we consider generic systems where sources in different importance classes compete for a common channel. We propose a new prioritized IRSA algorithm and derive the probability to correctly resolve collisions for data from each source class. We then make use of our theoretical analysis to formulate a new optimization problem for selecting the transmission strategies of heterogenous sources. We optimize both the replication probability per class and the source rate per class, in such a way that the overall system utility is maximized. We then propose a heuristic-based algorithm for the selection of the transmission strategy, which is built on intrinsic characteristics of the iterative decoding methods adopted for recovering from collisions. Experimental results validate the accuracy of the theoretical study and show the gain of well-chosen prioritized transmission strategies for transmission of data from heterogenous classes over shared wireless channels.
△ Less
Submitted 3 January, 2015;
originally announced January 2015.
-
Optimized Packet Scheduling in Multiview Video Navigation Systems
Authors:
Laura Toni,
Thomas Maugey,
Pascal Frossard
Abstract:
In multiview video systems, multiple cameras generally acquire the same scene from different perspectives, such that users have the possibility to select their preferred viewpoint. This results in large amounts of highly redundant data, which needs to be properly handled during encoding and transmission over resource-constrained channels. In this work, we study coding and transmission strategies i…
▽ More
In multiview video systems, multiple cameras generally acquire the same scene from different perspectives, such that users have the possibility to select their preferred viewpoint. This results in large amounts of highly redundant data, which needs to be properly handled during encoding and transmission over resource-constrained channels. In this work, we study coding and transmission strategies in multicamera systems, where correlated sources send data through a bottleneck channel to a central server, which eventually transmits views to different interactive users. We propose a dynamic correlation-aware packet scheduling optimization under delay, bandwidth, and interactivity constraints. The optimization relies both on a novel rate-distortion model, which captures the importance of each view in the 3D scene reconstruction, and on an objective function that optimizes resources based on a client navigation model. The latter takes into account the distortion experienced by interactive clients as well as the distortion variations that might be observed by clients during multiview navigation. We solve the scheduling problem with a novel trellis-based solution, which permits to formally decompose the multivariate optimization problem thereby significantly reducing the computation complexity. Simulation results show the gain of the proposed algorithm compared to baseline scheduling policies. More in details, we show the gain offered by our dynamic scheduling policy compared to static camera allocation strategies and to schemes with constant coding strategies. Finally, we show that the best scheduling policy consistently adapts to the most likely user navigation path and that it minimizes distortion variations that can be very disturbing for users in traditional navigation systems.
△ Less
Submitted 2 December, 2014;
originally announced December 2014.
-
Optimized Adaptive Streaming Representations based on System Dynamics
Authors:
Laura Toni,
Ramon Aparicio-Pardo,
Karine Pires,
Gwendal Simon,
Alberto Blanc,
Pascal Frossard
Abstract:
Adaptive streaming addresses the increasing and heterogenous demand of multimedia content over the Internet by offering several encoded versions for each video sequence. Each version (or representation) has a different resolution and bit rate, aimed at a specific set of users, like TV or mobile phone clients. While most existing works on adaptive streaming deal with effective playout-control strat…
▽ More
Adaptive streaming addresses the increasing and heterogenous demand of multimedia content over the Internet by offering several encoded versions for each video sequence. Each version (or representation) has a different resolution and bit rate, aimed at a specific set of users, like TV or mobile phone clients. While most existing works on adaptive streaming deal with effective playout-control strategies at the client side, we take in this paper a providers' perspective and propose solutions to improve user satisfaction by optimizing the encoding rates of the video sequences. We formulate an integer linear program that maximizes users' average satisfaction, taking into account the network dynamics, the video content information, and the user population characteristics. The solution of the optimization is a set of encoding parameters that permit to create different streams to robustly satisfy users' requests over time. We simulate multiple adaptive streaming sessions characterized by realistic network connections models, where the proposed solution outperforms commonly used vendor recommendations, in terms of user satisfaction but also in terms of fairness and outage probability. The simulation results further show that video content information as well as network constraints and users' statistics play a crucial role in selecting proper encoding parameters to provide fairness a mong users and to reduce network resource usage. We finally propose a few practical guidelines that can be used to choose the encoding parameters based on the user base characteristics, the network capacity and the type of video content.
△ Less
Submitted 16 October, 2014; v1 submitted 12 June, 2014;
originally announced June 2014.
-
Multi-View Video Packet Scheduling
Authors:
Laura Toni,
Thomas Maugey,
Pascal Frossard
Abstract:
In multiview applications, multiple cameras acquire the same scene from different viewpoints and generally produce correlated video streams. This results in large amounts of highly redundant data. In order to save resources, it is critical to handle properly this correlation during encoding and transmission of the multiview data. In this work, we propose a correlation-aware packet scheduling algor…
▽ More
In multiview applications, multiple cameras acquire the same scene from different viewpoints and generally produce correlated video streams. This results in large amounts of highly redundant data. In order to save resources, it is critical to handle properly this correlation during encoding and transmission of the multiview data. In this work, we propose a correlation-aware packet scheduling algorithm for multi-camera networks, where information from all cameras are transmitted over a bottleneck channel to clients that reconstruct the multiview images. The scheduling algorithm relies on a new rate-distortion model that captures the importance of each view in the scene reconstruction. We propose a problem formulation for the optimization of the packet scheduling policies, which adapt to variations in the scene content. Then, we design a low complexity scheduling algorithm based on a trellis search that selects the subset of candidate packets to be transmitted towards effective multiview reconstruction at clients. Extensive simulation results confirm the gain of our scheduling algorithm when inter-source correlation information is used in the scheduler, compared to scheduling policies with no information about the correlation or non-adaptive scheduling policies. We finally show that increasing the optimization horizon in the packet scheduling algorithm improves the transmission performance, especially in scenarios where the level of correlation rapidly varies with time.
△ Less
Submitted 28 May, 2013; v1 submitted 18 December, 2012;
originally announced December 2012.
-
Experimental evidence of delocalized states in random dimer superlattices
Authors:
V. Bellani,
E. Diez,
R. Hey,
L. Toni,
L. Tarricone,
G. P. Parravicini,
F. Dominguez-Adame,
R. Gomez-Alcala
Abstract:
We study the electronic properties of GaAs-AlGaAs superlattices with intentional correlated disorder by means of photoluminescence and vertical dc resistance. The results are compared to those obtained in ordered and uncorrelated disordered superlattices. We report the first experimental evidence that spatial correlations inhibit localization of states in disordered low-dimensional systems, as o…
▽ More
We study the electronic properties of GaAs-AlGaAs superlattices with intentional correlated disorder by means of photoluminescence and vertical dc resistance. The results are compared to those obtained in ordered and uncorrelated disordered superlattices. We report the first experimental evidence that spatial correlations inhibit localization of states in disordered low-dimensional systems, as our previous theoretical calculations suggested, in contrast to the earlier belief that all eigenstates are localized.
△ Less
Submitted 4 February, 1999;
originally announced February 1999.
