-
Nonsmooth convex optimization to estimate the Covid-19 reproduction number space-time evolution with robustness against low quality data
Authors:
Barbara Pascal,
Patrice Abry,
Nelly Pustelnik,
Stéphane G. Roux,
Rémi Gribonval,
Patrick Flandrin
Abstract:
Daily pandemic surveillance, often achieved through the estimation of the reproduction number, constitutes a critical challenge for national health authorities to design countermeasures. In an earlier work, we proposed to formulate the estimation of the reproduction number as an optimization problem, combining data-model fidelity and space-time regularity constraints, solved by nonsmooth convex pr…
▽ More
Daily pandemic surveillance, often achieved through the estimation of the reproduction number, constitutes a critical challenge for national health authorities to design countermeasures. In an earlier work, we proposed to formulate the estimation of the reproduction number as an optimization problem, combining data-model fidelity and space-time regularity constraints, solved by nonsmooth convex proximal minimizations. Though promising, that first formulation significantly lacks robustness against the Covid-19 data low quality (irrelevant or missing counts, pseudo-seasonalities,.. .) stemming from the emergency and crisis context, which significantly impairs accurate pandemic evolution assessments. The present work aims to overcome these limitations by carefully crafting a functional permitting to estimate jointly, in a single step, the reproduction number and outliers defined to model low quality data. This functional also enforces epidemiology-driven regularity properties for the reproduction number estimates, while preserving convexity, thus permitting the design of efficient minimization algorithms, based on proximity operators that are derived analytically. The explicit convergence of the proposed algorithm is proven theoretically. Its relevance is quantified on real Covid-19 data, consisting of daily new infection counts for 200+ countries and for the 96 metropolitan France counties, publicly available at Johns Hopkins University and Sant{é}-Publique-France. The procedure permits automated daily updates of these estimates, reported via animated and interactive maps. Open-source estimation procedures will be made publicly available.
△ Less
Submitted 20 September, 2021;
originally announced September 2021.
-
Exploring and comparing temporal clustering methods
Authors:
Jordan Cambe,
Sebastian Grauwin,
Patrick Flandrin,
Pablo Jensen
Abstract:
Description of temporal networks and detection of dynamic communities have been hot topics of research for the last decade. However, no consensual answers to these challenges have been found due to the complexity of the task. Static communities are not well defined objects, and adding a temporal dimension renders the description even more difficult. In this article, we propose a coherent temporal…
▽ More
Description of temporal networks and detection of dynamic communities have been hot topics of research for the last decade. However, no consensual answers to these challenges have been found due to the complexity of the task. Static communities are not well defined objects, and adding a temporal dimension renders the description even more difficult. In this article, we propose a coherent temporal clustering method: the Best Combination of Local Communities (BCLC). Our method aims at finding a good balance between two conflicting objectives : closely following the short time evolution by finding optimal partitions at each time step and temporal smoothness, which privileges historical continuity.
△ Less
Submitted 2 December, 2020;
originally announced December 2020.
-
Revealing evolutions in dynamical networks
Authors:
Matteo Morini,
Patrick Flandrin,
Eric Fleury,
Tommaso Venturini,
Pablo Jensen
Abstract:
The description of large temporal graphs requires effective methods giving an appropriate mesoscopic partition. Many approaches exist today to detect communities in static graphs. However, many networks are intrinsically dynamical, and need a dynamic mesoscale description, as interpreting them as static networks would cause loss of important information. For example, dynamic processes such as the…
▽ More
The description of large temporal graphs requires effective methods giving an appropriate mesoscopic partition. Many approaches exist today to detect communities in static graphs. However, many networks are intrinsically dynamical, and need a dynamic mesoscale description, as interpreting them as static networks would cause loss of important information. For example, dynamic processes such as the emergence of new scientific disciplines, their fusion, split or death need a mesoscopic description of the evolving network of scientific articles. There are two straightforward approaches to describe an evolving network using methods developed for static networks.
The first finds the community structure of the aggregated network; however, this approach discards most temporal information, and may lead to inappropriate descriptions, as very different dynamic data can give rise to the identical static graphs.
The opposite approach closely follows the evolutions and builds networks for successive time slices by selecting the relevant nodes and edges, the mesoscopic structure of each of these slices is found independently and the structures are connected to obtain a temporal description. By using an optimal structural description at each time slice, this method avoids the inertia of the aggregated approach. The inherent fuzziness of the communities leads to noise and artifacts.
Here, we present an approach that distinguishes real trends and noise in the mesoscopic description of data using the continuity of social evolutions. To be follow the dynamics, we compute partitions for each time slice, but to avoid transients generated by noise, we modify the community description at time t using the structures found at times t-1 and t+1. We show the relevance of our method on the analysis of a scientific network showing the birth of a new subfield, wavelet analysis.
△ Less
Submitted 7 July, 2017;
originally announced July 2017.
-
When interpolation-induced reflection artifact meets time-frequency analysis
Authors:
Yu-Ting Lin,
Patrick Flandrin,
Hau-tieng Wu
Abstract:
While extracting the temporal dynamical features based on the time-frequency analyses, like the reassignment and synchrosqueezing transform, attracts more and more interest in bio-medical data analysis, we should be careful about artifacts generated by interpolation schemes, in particular when the sampling rate is not significantly higher than the frequency of the oscillatory component we are inte…
▽ More
While extracting the temporal dynamical features based on the time-frequency analyses, like the reassignment and synchrosqueezing transform, attracts more and more interest in bio-medical data analysis, we should be careful about artifacts generated by interpolation schemes, in particular when the sampling rate is not significantly higher than the frequency of the oscillatory component we are interested in. In this study, we formulate the problem called the reflection effect and provide a theoretical justification of the statement. We also show examples in the anesthetic depth analysis with clear but undesirable artifacts. The results show that the artifact associated with the reflection effect exists not only theoretically but practically. Its influence is pronounced when we apply the time-frequency analyses to extract the time-varying dynamics hidden inside the signal. In conclusion, we have to carefully deal with the artifact associated with the reflection effect by choosing a proper interpolation scheme.
△ Less
Submitted 14 December, 2015; v1 submitted 19 July, 2015;
originally announced July 2015.
-
Duality between Temporal Networks and Signals: Extraction of the Temporal Network Structures
Authors:
Ronan Hamon,
Pierre Borgnat,
Patrick Flandrin,
Céline Robardet
Abstract:
We develop a framework to track the structure of temporal networks with a signal processing approach. The method is based on the duality between networks and signals using a multidimensional scaling technique. This enables a study of the network structure using frequency patterns of the corresponding signals. An extension is proposed for temporal networks, thereby enabling a tracking of the networ…
▽ More
We develop a framework to track the structure of temporal networks with a signal processing approach. The method is based on the duality between networks and signals using a multidimensional scaling technique. This enables a study of the network structure using frequency patterns of the corresponding signals. An extension is proposed for temporal networks, thereby enabling a tracking of the network structure over time. A method to automatically extract the most significant frequency patterns and their activation coefficients over time is then introduced, using nonnegative matrix factorization of the temporal spectra. The framework, inspired by audio decomposition, allows transforming back these frequency patterns into networks, to highlight the evolution of the underlying structure of the network over time. The effectiveness of the method is first evidenced on a toy example, prior being used to study a temporal network of face-to-face contacts. The extraction of sub-networks highlights significant structures decomposed on time intervals.
△ Less
Submitted 12 May, 2015;
originally announced May 2015.
-
From graphs to signals and back: Identification of network structures using spectral analysis
Authors:
Ronan Hamon,
Pierre Borgnat,
Patrick Flandrin,
Céline Robardet
Abstract:
Many systems comprising entities in interactions can be represented as graphs, whose structure gives significant insights about how these systems work. Network theory has undergone further developments, in particular in relation to detection of communities in graphs, to catch this structure. Recently, an approach has been proposed to transform a graph into a collection of signals: Using a multidim…
▽ More
Many systems comprising entities in interactions can be represented as graphs, whose structure gives significant insights about how these systems work. Network theory has undergone further developments, in particular in relation to detection of communities in graphs, to catch this structure. Recently, an approach has been proposed to transform a graph into a collection of signals: Using a multidimensional scaling technique on a distance matrix representing relations between vertices of the graph, points in a Euclidean space are obtained and interpreted as signals, indexed by the vertices. In this article, we propose several extensions to this approach, develo** a framework to study graph structures using signal processing tools. We first extend the current methodology, enabling us to highlight connections between properties of signals and graph structures, such as communities, regularity or randomness, as well as combinations of those. A robust inverse transformation method is next described, taking into account possible changes in the signals compared to original ones. This technique uses, in addition to the relationships between the points in the Euclidean space, the energy of each signal, coding the different scales of the graph structure. These contributions open up new perspectives in the study of graphs, by enabling processing of graphs through the processing of the corresponding collection of signals, using reliable tools from signal processing. A technique of denoising of a graph by filtering of the corresponding signals is then described, suggesting considerable potential of the approach.
△ Less
Submitted 10 June, 2016; v1 submitted 16 February, 2015;
originally announced February 2015.
-
Discovering the structure of complex networks by minimizing cyclic bandwidth sum
Authors:
Ronan Hamon,
Pierre Borgnat,
Patrick Flandrin,
Céline Robardet
Abstract:
Getting a labeling of vertices close to the structure of the graph has been proved to be of interest in many applications e.g., to follow smooth signals indexed by the vertices of the network. This question can be related to a graph labeling problem known as the cyclic bandwidth sum problem. It consists in finding a labeling of the vertices of an undirected and unweighted graph with distinct integ…
▽ More
Getting a labeling of vertices close to the structure of the graph has been proved to be of interest in many applications e.g., to follow smooth signals indexed by the vertices of the network. This question can be related to a graph labeling problem known as the cyclic bandwidth sum problem. It consists in finding a labeling of the vertices of an undirected and unweighted graph with distinct integers such that the sum of (cyclic) difference of labels of adjacent vertices is minimized. Although theoretical results exist that give optimal value of cyclic bandwidth sum for standard graphs, there are neither results in the general case, nor explicit methods to reach this optimal result. In addition to this lack of theoretical knowledge, only a few methods have been proposed to approximately solve this problem. In this paper, we introduce a new heuristic to find an approximate solution for the cyclic bandwidth sum problem, by following the structure of the graph. The heuristic is a two-step algorithm: the first step consists of traversing the graph to find a set of paths which follow the structure of the graph, using a similarity criterion based on the Jaccard index to jump from one vertex to the next one. The second step is the merging of all obtained paths, based on a greedy approach that extends a partial solution by inserting a new path at the position that minimizes the cyclic bandwidth sum. The effectiveness of the proposed heuristic, both in terms of performance and time execution, is shown through experiments on graphs whose optimal value of CBS is known as well as on real-world networks, where the consistence between labeling and topology is highlighted. An extension to weighted graphs is also proposed.
△ Less
Submitted 16 February, 2015; v1 submitted 22 October, 2014;
originally announced October 2014.
-
2-D Prony-Huang Transform: A New Tool for 2-D Spectral Analysis
Authors:
Jérémy Schmitt,
Nelly Pustelnik,
Pierre Borgnat,
Patrick Flandrin,
Laurent Condat
Abstract:
This work proposes an extension of the 1-D Hilbert Huang transform for the analysis of images. The proposed method consists in (i) adaptively decomposing an image into oscillating parts called intrinsic mode functions (IMFs) using a mode decomposition procedure, and (ii) providing a local spectral analysis of the obtained IMFs in order to get the local amplitudes, frequencies, and orientations. Fo…
▽ More
This work proposes an extension of the 1-D Hilbert Huang transform for the analysis of images. The proposed method consists in (i) adaptively decomposing an image into oscillating parts called intrinsic mode functions (IMFs) using a mode decomposition procedure, and (ii) providing a local spectral analysis of the obtained IMFs in order to get the local amplitudes, frequencies, and orientations. For the decomposition step, we propose two robust 2-D mode decompositions based on non-smooth convex optimization: a "Genuine 2-D" approach, that constrains the local extrema of the IMFs, and a "Pseudo 2-D" approach, which constrains separately the extrema of lines, columns, and diagonals. The spectral analysis step is based on Prony annihilation property that is applied on small square patches of the IMFs. The resulting 2-D Prony-Huang transform is validated on simulated and real data.
△ Less
Submitted 30 April, 2014;
originally announced April 2014.
-
French Roadmap for complex Systems 2008-2009
Authors:
Paul Bourgine,
David Chavalarias,
Edith Perrier,
Frederic Amblard,
Francois Arlabosse,
Pierre Auger,
Jean-Bernard Baillon,
Olivier Barreteau,
Pierre Baudot,
Elisabeth Bouchaud,
Soufian Ben Amor,
Hugues Berry,
Cyrille Bertelle,
Marc Berthod,
Guillaume Beslon,
Giulio Biroli,
Daniel Bonamy,
Daniele Bourcier,
Nicolas Brodu,
Marc Bui,
Yves Burnod,
Bertrand Chapron,
Catherine Christophe,
Bruno Clement,
Jean-Louis Coatrieux
, et al. (56 additional authors not shown)
Abstract:
This second issue of the French Complex Systems Roadmap is the outcome of the Entretiens de Cargese 2008, an interdisciplinary brainstorming session organized over one week in 2008, jointly by RNSC, ISC-PIF and IXXI. It capitalizes on the first roadmap and gathers contributions of more than 70 scientists from major French institutions. The aim of this roadmap is to foster the coordination of the…
▽ More
This second issue of the French Complex Systems Roadmap is the outcome of the Entretiens de Cargese 2008, an interdisciplinary brainstorming session organized over one week in 2008, jointly by RNSC, ISC-PIF and IXXI. It capitalizes on the first roadmap and gathers contributions of more than 70 scientists from major French institutions. The aim of this roadmap is to foster the coordination of the complex systems community on focused topics and questions, as well as to present contributions and challenges in the complex systems sciences and complexity science to the public, political and industrial spheres.
△ Less
Submitted 13 July, 2009;
originally announced July 2009.