-
Risk-aware Scheduling and Dispatch of Flexibility Events in Buildings
Authors:
Paul Scharnhorst,
Baptiste Schubnel,
Rafael E. Carrillo,
Pierre-Jean Alet,
Colin N. Jones
Abstract:
Residential and commercial buildings, equipped with systems such as heat pumps, hot water tanks, or stationary energy storage, have a large potential to offer their consumption flexibility as grid services. In this work, we leverage this flexibility to react to consumption requests related to maximizing self-consumption and reducing peak loads. We present a general characterization of consumption…
▽ More
Residential and commercial buildings, equipped with systems such as heat pumps, hot water tanks, or stationary energy storage, have a large potential to offer their consumption flexibility as grid services. In this work, we leverage this flexibility to react to consumption requests related to maximizing self-consumption and reducing peak loads. We present a general characterization of consumption flexibility in the form of flexibility envelopes and discuss a data-driven battery model formulation for modeling individual buildings. These models are used to predict the available consumption flexibility while incorporating a description of uncertainty and being risk-aware with a pre-defined risk level. A Mixed-integer Linear Program (MILP) is formulated to schedule the activation of the buildings in order to best respond to an external consumption request. An aggregated consumption request is dispatched to the active individual buildings by an algorithm, based on the previously determined schedule. The effectiveness of the approach is demonstrated by coordinating up to 500 simulated buildings using the Energym Python library and observing about 1.5 times peak power reduction in comparison with a baseline approach while maintaining comfort more robustly. We demonstrate the scalability of the approach, with solving times being approximately linear in the number of considered assets in the scheduling problem.
△ Less
Submitted 9 November, 2023;
originally announced November 2023.
-
Measuring Cause-Effect with the Variability of the Largest Eigenvalue
Authors:
Alejandro Rodriguez Dominguez,
Irving Ramirez Carrillo,
David Parraga Riquelme
Abstract:
We present a method to test and monitor structural relationships between time variables. The distribution of the first eigenvalue for lagged correlation matrices (Tracy-Widom distribution) is used to test structural time relationships between variables against the alternative hypothesis (Independence). This distribution studies the asymptotic dynamics of the largest eigenvalue as a function of the…
▽ More
We present a method to test and monitor structural relationships between time variables. The distribution of the first eigenvalue for lagged correlation matrices (Tracy-Widom distribution) is used to test structural time relationships between variables against the alternative hypothesis (Independence). This distribution studies the asymptotic dynamics of the largest eigenvalue as a function of the lag in lagged correlation matrices. By analyzing the time series of the standard deviation of the greatest eigenvalue for $2\times 2$ correlation matrices with different lags we can analyze deviations from the Tracy-Widom distribution to test structural relationships between these two time variables. These relationships can be related to causality. We use the standard deviation of the explanatory power of the first eigenvalue at different lags as a proxy for testing and monitoring structural causal relationships. The method is applied to analyse causal dependencies between daily monetary flows in a retail brokerage business allowing to control for liquidity risks.
△ Less
Submitted 16 July, 2023; v1 submitted 10 July, 2023;
originally announced July 2023.
-
Uncertainty-aware Flexibility Envelope Prediction in Buildings with Controller-agnostic Battery Models
Authors:
Paul Scharnhorst,
Baptiste Schubnel,
Rafael E. Carrillo,
Pierre-Jean Alet,
Colin N. Jones
Abstract:
Buildings are a promising source of flexibility for the application of demand response. In this work, we introduce a novel battery model formulation to capture the state evolution of a single building. Being fully data-driven, the battery model identification requires one dataset from a period of nominal controller operation, and one from a period with flexibility requests, without making any assu…
▽ More
Buildings are a promising source of flexibility for the application of demand response. In this work, we introduce a novel battery model formulation to capture the state evolution of a single building. Being fully data-driven, the battery model identification requires one dataset from a period of nominal controller operation, and one from a period with flexibility requests, without making any assumptions on the underlying controller structure. We consider parameter uncertainty in the model formulation and show how to use risk measures to encode risk preferences of the user in robust uncertainty sets. Finally, we demonstrate the uncertainty-aware prediction of flexibility envelopes for a building simulation model from the Python library Energym.
△ Less
Submitted 7 October, 2022;
originally announced October 2022.
-
DRT-based modelling framework for Li-ion cells
Authors:
Pietro Iurilli,
Claudio Brivio,
Rafael E. Carrillo,
Vanessa Wood
Abstract:
The correct assessment of battery states is essential to maximize battery pack performances while ensuring reliable and safe operation. This work introduces EIS2MOD, a novel modelling framework for Li-ion cells based on Distribution of Relaxation Time (DRT). A physically based Electric Circuit Model (ECM) is developed starting from Electrochemical Impedance Spectroscopy (EIS) and Open Circuit Volt…
▽ More
The correct assessment of battery states is essential to maximize battery pack performances while ensuring reliable and safe operation. This work introduces EIS2MOD, a novel modelling framework for Li-ion cells based on Distribution of Relaxation Time (DRT). A physically based Electric Circuit Model (ECM) is developed starting from Electrochemical Impedance Spectroscopy (EIS) and Open Circuit Voltage (OCV) measurements. DRT is applied to deconvolve the electrochemical phenomena from the EIS. The presented methodology is based on: i) DRT calculation from EIS, ii) DRT analysis for ECM configuration and iii) Model parameters extraction and fitting. The proposed framework is applied to large format Li-ion pouch cells, which are tested over the whole State of Charge (SoC) range and a wide temperature range (-10°C to 35°C). Different current profiles have been tested to validate the model, showing its high accuracy in reproducing the battery cell behavior (e.g. RMSE on the battery terminals voltage lower than 1.50% for driving cycle simulations at variable temperature and SoC). An additional advantage of EIS2MOD is its light computational load thus offering an attractive framework for battery management system implementation.
△ Less
Submitted 16 March, 2022;
originally announced March 2022.
-
Smart sensors network for accurate indirect heat accounting in apartment buildings
Authors:
Yves Stauffer,
Fabio Saba,
Rafael E. Carrillo,
Max Boegli,
Andrea Malengo,
Andreas Hutter
Abstract:
A new method for accurate indirect heat accounting in apartment buildings has been recently developed by the Centre Suisse d'Electronique et de Microtechnique (CSEM). It is based on a data driven approach aimed to the smart networking of any type of indirect heat allocation devices, which can provide, for each heat delivery point of an apartment building, measurements or estimations of the tempera…
▽ More
A new method for accurate indirect heat accounting in apartment buildings has been recently developed by the Centre Suisse d'Electronique et de Microtechnique (CSEM). It is based on a data driven approach aimed to the smart networking of any type of indirect heat allocation devices, which can provide, for each heat delivery point of an apartment building, measurements or estimations of the temperature difference between the heat transfer fluid and the indoor environment. The analysis of the data gathered from the devices installed on the heating bodies, together with the measurements of the overall building heat consumption provided by direct heat metering, allows the evaluation of the characteristic thermal model parameters of heating bodies at actual installation and working conditions. Thus overcoming the negative impact on accuracy of conventional indirect heat accounting due to off-design operation, in which these measurement systems normally operate. The method has been tested on conventional heat cost allocators (HCA), and on innovative smart radiator thermostatic valves developed by CSEM. The evaluations were carried out at the centralized heating system mock-up of the Istituto Nazionale di Ricerca Metrologica (INRIM), and also in a real building in Neuchatel, Switzerland. The method has proven to be an effective tool to improve the accuracy of indirect heat metering systems; compared to conventional HCA systems, the error on the individual heating bill is reduced by 20% to 50%.
△ Less
Submitted 3 November, 2021;
originally announced November 2021.
-
Spatio-temporal graph neural networks for multi-site PV power forecasting
Authors:
Jelena Simeunović,
Baptiste Schubnel,
Pierre-Jean Alet,
Rafael E. Carrillo
Abstract:
Accurate forecasting of solar power generation with fine temporal and spatial resolution is vital for the operation of the power grid. However, state-of-the-art approaches that combine machine learning with numerical weather predictions (NWP) have coarse resolution. In this paper, we take a graph signal processing perspective and model multi-site photovoltaic (PV) production time series as signals…
▽ More
Accurate forecasting of solar power generation with fine temporal and spatial resolution is vital for the operation of the power grid. However, state-of-the-art approaches that combine machine learning with numerical weather predictions (NWP) have coarse resolution. In this paper, we take a graph signal processing perspective and model multi-site photovoltaic (PV) production time series as signals on a graph to capture their spatio-temporal dependencies and achieve higher spatial and temporal resolution forecasts. We present two novel graph neural network models for deterministic multi-site PV forecasting dubbed the graph-convolutional long short term memory (GCLSTM) and the graph-convolutional transformer (GCTrafo) models. These methods rely solely on production data and exploit the intuition that PV systems provide a dense network of virtual weather stations. The proposed methods were evaluated in two data sets for an entire year: 1) production data from 304 real PV systems, and 2) simulated production of 1000 PV systems, both distributed over Switzerland. The proposed models outperform state-of-the-art multi-site forecasting methods for prediction horizons of six hours ahead. Furthermore, the proposed models outperform state-of-the-art single-site methods with NWP as inputs on horizons up to four hours ahead.
△ Less
Submitted 1 November, 2021; v1 submitted 29 July, 2021;
originally announced July 2021.
-
Reconfigurable Cyber-Physical System for Critical Infrastructure Protection in Smart Cities via Smart Video-Surveillance
Authors:
Juan Isern,
Francisco Barranco,
Daniel Deniz,
Juho Lesonen,
Jari Hannuksela,
Richard R. Carrillo
Abstract:
Automated surveillance is essential for the protection of Critical Infrastructures (CIs) in future Smart Cities. The dynamic environments and bandwidth requirements demand systems that adapt themselves to react when events of interest occur. We present a reconfigurable Cyber Physical System for the protection of CIs using distributed cloud-edge smart video surveillance. Our local edge nodes perfor…
▽ More
Automated surveillance is essential for the protection of Critical Infrastructures (CIs) in future Smart Cities. The dynamic environments and bandwidth requirements demand systems that adapt themselves to react when events of interest occur. We present a reconfigurable Cyber Physical System for the protection of CIs using distributed cloud-edge smart video surveillance. Our local edge nodes perform people detection via Deep Learning. Processing is embedded in high performance SoCs (System-on-Chip) achieving real-time performance ($\approx$ 100 fps - frames per second) which enables efficiently managing video streams of more cameras source at lower frame rate. Cloud server gathers results from nodes to carry out biometric facial identification, tracking, and perimeter monitoring. A Quality and Resource Management module monitors data bandwidth and triggers reconfiguration adapting the transmitted video resolution. This also enables a flexible use of the network by multiple cameras while maintaining the accuracy of biometric identification. A real-world example shows a reduction of $\approx$ 75\% bandwidth use with respect to the no-reconfiguration scenario.
△ Less
Submitted 29 November, 2020;
originally announced November 2020.
-
State space models for building control: how deep should you go?
Authors:
Baptiste Schubnel,
Rafael E. Carrillo,
Paolo Taddeo,
Lluc Canals Casals,
Jaume Salom,
Yves Stauffer,
Pierre-Jean Alet
Abstract:
Power consumption in buildings show non-linear behaviors that linear models cannot capture whereas recurrent neural networks (RNNs) can. This ability makes RNNs attractive alternatives for the model-predictive control (MPC) of buildings. However RNN models lack mathematical regularity which makes their use challenging in optimization problems. This work therefore systematically investigates whethe…
▽ More
Power consumption in buildings show non-linear behaviors that linear models cannot capture whereas recurrent neural networks (RNNs) can. This ability makes RNNs attractive alternatives for the model-predictive control (MPC) of buildings. However RNN models lack mathematical regularity which makes their use challenging in optimization problems. This work therefore systematically investigates whether using RNNs for building control provides net gains in an MPC framework. It compares the representation power and control performance of two architectures: a fully non-linear RNN architecture and a linear state-space model with non-linear regressor. The comparison covers five instances of each architecture over two months of simulated operation in identical conditions. The error on the one-hour forecast of temperature is 69% lower with the RNN model than with the linear one. In control the linear state-space model outperforms by 10% on the objective function, shows 2.8 times higher average temperature violations, and needs a third of the computation time the RNN model requires. This work therefore demonstrates that in their current form RNNs do improve accuracy but on balance well-designed linear state-space models with non-linear regressors are best in most cases of MPC.
△ Less
Submitted 23 October, 2020;
originally announced October 2020.
-
Semantic Workflows and Machine Learning for the Assessment of Carbon Storage by Urban Trees
Authors:
Juan Carrillo,
Daniel Garijo,
Mark Crowley,
Rober Carrillo,
Yolanda Gil,
Katherine Borda
Abstract:
Climate science is critical for understanding both the causes and consequences of changes in global temperatures and has become imperative for decisive policy-making. However, climate science studies commonly require addressing complex interoperability issues between data, software, and experimental approaches from multiple fields. Scientific workflow systems provide unparalleled advantages to add…
▽ More
Climate science is critical for understanding both the causes and consequences of changes in global temperatures and has become imperative for decisive policy-making. However, climate science studies commonly require addressing complex interoperability issues between data, software, and experimental approaches from multiple fields. Scientific workflow systems provide unparalleled advantages to address these issues, including reproducibility of experiments, provenance capture, software reusability and knowledge sharing. In this paper, we introduce a novel workflow with a series of connected components to perform spatial data preparation, classification of satellite imagery with machine learning algorithms, and assessment of carbon stored by urban trees. To the best of our knowledge, this is the first study that estimates carbon storage for a region in Africa following the guidelines from the Intergovernmental Panel on Climate Change (IPCC).
△ Less
Submitted 21 September, 2020;
originally announced September 2020.
-
A hybrid learning method for system identification and optimal control
Authors:
Baptiste Schubnel,
Rafael E. Carrillo,
Pierre-Jean Alet,
Andreas Hutter
Abstract:
We present a three-step method to perform system identification and optimal control of non-linear systems. Our approach is mainly data driven and does not require active excitation of the system to perform system identification. In particular, it is designed for systems for which only historical data under closed-loop control are available and where historical control commands exhibit low variabil…
▽ More
We present a three-step method to perform system identification and optimal control of non-linear systems. Our approach is mainly data driven and does not require active excitation of the system to perform system identification. In particular, it is designed for systems for which only historical data under closed-loop control are available and where historical control commands exhibit low variability. In a first step, simple simulation models of the system are built and run under various conditions. In a second step, a neural network architecture is extensively trained on the simulation outputs to learn the system physics, and retrained with historical data from the real system with stop** rules. These constraints avoid overfitting that arise by fitting closed-loop controlled systems. By doing so, we obtain one (or many) system model(s), represented by this architecture, and whose behaviour can be chosen to match more or less the real system. Finally, state-of-the-art reinforcement learning with a variant of domain randomization and distributed learning is used for optimal control of the system. We first illustrate the model identification strategy with a simple example, the pendulum with external torque. We then apply our method to model and optimize the control of a large building facility located in Switzerland. Simulation results demonstrate that this approach generates stable functional controllers which outperform on comfort and energy benchmark rule-based controllers.
△ Less
Submitted 13 August, 2020; v1 submitted 18 March, 2020;
originally announced March 2020.
-
A Metric for Evaluating Neural Input Representation in Supervised Learning Networks
Authors:
Richard R Carrillo,
Francisco Naveros,
Eduardo Ros,
Niceto R Luque
Abstract:
Supervised learning has long been attributed to several feed-forward neural circuits within the brain, with attention being paid to the cerebellar granular layer. The focus of this study is to evaluate the input activity representation of these feed-forward neural networks. The activity of cerebellar granule cells is conveyed by parallel fibers and translated into Purkinje cell activity; the sole…
▽ More
Supervised learning has long been attributed to several feed-forward neural circuits within the brain, with attention being paid to the cerebellar granular layer. The focus of this study is to evaluate the input activity representation of these feed-forward neural networks. The activity of cerebellar granule cells is conveyed by parallel fibers and translated into Purkinje cell activity; the sole output of the cerebellar cortex. The learning process at this parallel-fiber-to-Purkinje-cell connection makes each Purkinje cell sensitive to a set of specific cerebellar states, determined by the granule-cell activity during a certain time window. A Purkinje cell becomes sensitive to each neural input state and, consequently, the network operates as a function able to generate a desired output for each provided input by means of supervised learning. However, not all sets of Purkinje cell responses can be assigned to any set of input states due to the network's own limitations (inherent to the network neurobiological substrate), that is, not all input-output map** can be learned. A limiting factor is the representation of the input states through granule-cell activity. The quality of this representation will determine the capacity of the network to learn a varied set of outputs. In this study we present an algorithm for evaluating quantitatively the level of compatibility/interference amongst a set of given cerebellar states according to their representation (granule-cell activation patterns) without the need for actually conducting simulations and network training. The algorithm input consists of a real-number matrix that codifies the activity level of every considered granule-cell in each state. The capability of this representation to generate a varied set of outputs is evaluated geometrically, thus resulting in a real number that assesses the goodness of the representation
△ Less
Submitted 3 March, 2020;
originally announced March 2020.
-
Some error estimates for the DEC method in the plane
Authors:
Ruben Carrillo,
Miguel Angel Moreles,
Rafael Herrera
Abstract:
We show that the Discrete Exterior Calculus (DEC) method can be cast as the earlier box method for the Poisson problem in the plane. Consequently, error estimates are established, proving that the DEC method is comparable to the Finite Element Method with linear elements. We also discuss some virtues, others than convergence, of the DEC method.
We show that the Discrete Exterior Calculus (DEC) method can be cast as the earlier box method for the Poisson problem in the plane. Consequently, error estimates are established, proving that the DEC method is comparable to the Finite Element Method with linear elements. We also discuss some virtues, others than convergence, of the DEC method.
△ Less
Submitted 15 July, 2019;
originally announced July 2019.
-
The formation of compact dwarf ellipticals through merging star clusters
Authors:
F. Urrutia Zapata,
M. Fellhauer,
A. G. Alarcon Jara,
D. R. Matus Carrillo,
C. A. Aravena
Abstract:
In the last decades, extended old stellar clusters have been observed. These extended objects cover a large range in masses, from extended clusters or faint fuzzies to ultra compact dwarf galaxies. It has been demonstrated that these extended objects can be the result of the merging of star clusters in cluster complexes (small regions in which dozens to hundreds of star clusters form). This format…
▽ More
In the last decades, extended old stellar clusters have been observed. These extended objects cover a large range in masses, from extended clusters or faint fuzzies to ultra compact dwarf galaxies. It has been demonstrated that these extended objects can be the result of the merging of star clusters in cluster complexes (small regions in which dozens to hundreds of star clusters form). This formation channel is called the `Merging Star Cluster Scenario'. This work tries to explain the formation of compact ellipticals in the same theoretical framework. Compact ellipticals are a comparatively rare class of spheroidal galaxies, possessing very small effective radii and high central surface brightnesses. With the use of numerical simulations we show that the merging star cluster scenario, adopted for higher masses, as found with those galaxies, can reproduce all major characteristics and the dynamics of these objects. This opens up a new formation channel to explain the existence of compact elliptical galaxies.
△ Less
Submitted 6 March, 2019;
originally announced March 2019.
-
Dynamical Ejections of Stars due to an Accelerating Gas Filament
Authors:
Tjarda C. N. Boekholt,
Amelia M. Stutz,
Michael Fellhauer,
Dominik R. G. Schleicher,
Diego R. Matus Carrillo
Abstract:
Observations of the Orion-A integral shaped filament (ISF) have shown indications of an oscillatory motion of the gas filament. This evidence is based on both the wave-like morphology of the filament as well as the kinematics of the gas and stars, where the characteristic velocities of the stars require a dynamical heating mechanism. As proposed by Stutz and Gould (2016), such a heating mechanism…
▽ More
Observations of the Orion-A integral shaped filament (ISF) have shown indications of an oscillatory motion of the gas filament. This evidence is based on both the wave-like morphology of the filament as well as the kinematics of the gas and stars, where the characteristic velocities of the stars require a dynamical heating mechanism. As proposed by Stutz and Gould (2016), such a heating mechanism (the "Slingshot") may be the result of an oscillating gas filament in a gas-dominated (as opposed to stellar-mass dominated) system. Here we test this hypothesis with the first stellar-dynamical simulations in which the stars are subjected to the influence of an oscillating cylindrical potential. The accelerating, cylindrical background potential is populated with a narrow distribution of stars. By coupling the potential to N-body dynamics, we are able to measure the influence of the potential on the stellar distribution. The simulations provide evidence that the slingshot mechanism can successfully reproduce several stringent observational constraints. These include the stellar spread (both in projected position and in velocity) around the filament, the symmetry in these distributions, and a bulk motion of the stars with respect to the filament. Using simple considerations we show that star-star interactions are incapable of reproducing these spreads on their own when properly accounting for the gas potential. Thus, properly accounting for the gas potential is essential for understanding the dynamical evolution of star forming filamentary systems in the era of Gaia.
△ Less
Submitted 17 July, 2017; v1 submitted 3 April, 2017;
originally announced April 2017.
-
PURIFYing real radio interferometric observations
Authors:
Luke Pratley,
Jason D. McEwen,
Mayeul d'Avezac,
Rafael E. Carrillo,
Alexandru Onose,
Yves Wiaux
Abstract:
Next-generation radio interferometers, such as the Square Kilometre Array (SKA), will revolutionise our understanding of the universe through their unprecedented sensitivity and resolution. However, standard methods in radio interferometry produce reconstructed interferometric images that are limited in quality and they are not scalable for big data. In this work we apply and evaluate alternative…
▽ More
Next-generation radio interferometers, such as the Square Kilometre Array (SKA), will revolutionise our understanding of the universe through their unprecedented sensitivity and resolution. However, standard methods in radio interferometry produce reconstructed interferometric images that are limited in quality and they are not scalable for big data. In this work we apply and evaluate alternative interferometric reconstruction methods that make use of state-of-the-art sparse image reconstruction algorithms motivated by compressive sensing, which have been implemented in the PURIFY software package. In particular, we implement and apply the proximal alternating direction method of multipliers (P-ADMM) algorithm presented in a recent article. We apply PURIFY to real interferometric observations. For all observations PURIFY outperforms the standard CLEAN, where in some cases PURIFY provides an improvement in dynamic range by over an order of magnitude. The latest version of PURIFY, which includes the developments presented in this work, is made publicly available.
△ Less
Submitted 22 February, 2017;
originally announced February 2017.
-
A randomised primal-dual algorithm for distributed radio-interferometric imaging
Authors:
Alexandru Onose,
Rafael E. Carrillo,
Jason D. McEwen,
Yves Wiaux
Abstract:
Next generation radio telescopes, like the Square Kilometre Array, will acquire an unprecedented amount of data for radio astronomy. The development of fast, parallelisable or distributed algorithms for handling such large-scale data sets is of prime importance. Motivated by this, we investigate herein a convex optimisation algorithmic structure, based on primal-dual forward-backward iterations, f…
▽ More
Next generation radio telescopes, like the Square Kilometre Array, will acquire an unprecedented amount of data for radio astronomy. The development of fast, parallelisable or distributed algorithms for handling such large-scale data sets is of prime importance. Motivated by this, we investigate herein a convex optimisation algorithmic structure, based on primal-dual forward-backward iterations, for solving the radio interferometric imaging problem. It can encompass any convex prior of interest. It allows for the distributed processing of the measured data and introduces further flexibility by employing a probabilistic approach for the selection of the data blocks used at a given iteration. We study the reconstruction performance with respect to the data distribution and we propose the use of nonuniform probabilities for the randomised updates. Our simulations show the feasibility of the randomisation given a limited computing infrastructure as well as important computational advantages when compared to state-of-the-art algorithmic structures.
△ Less
Submitted 27 October, 2016;
originally announced October 2016.
-
Robust sparse image reconstruction of radio interferometric observations with purify
Authors:
Luke Pratley,
Jason D. McEwen,
Mayeul d'Avezac,
Rafael E. Carrillo,
Alexandru Onose,
Yves Wiaux
Abstract:
Next-generation radio interferometers, such as the Square Kilometre Array (SKA), will revolutionise our understanding of the universe through their unprecedented sensitivity and resolution. However, to realise these goals significant challenges in image and data processing need to be overcome. The standard methods in radio interferometry for reconstructing images, such as CLEAN, have served the co…
▽ More
Next-generation radio interferometers, such as the Square Kilometre Array (SKA), will revolutionise our understanding of the universe through their unprecedented sensitivity and resolution. However, to realise these goals significant challenges in image and data processing need to be overcome. The standard methods in radio interferometry for reconstructing images, such as CLEAN, have served the community well over the last few decades and have survived largely because they are pragmatic. However, they produce reconstructed inter\-ferometric images that are limited in quality and scalability for big data. In this work we apply and evaluate alternative interferometric reconstruction methods that make use of state-of-the-art sparse image reconstruction algorithms motivated by compressive sensing, which have been implemented in the PURIFY software package. In particular, we implement and apply the proximal alternating direction method of multipliers (P-ADMM) algorithm presented in a recent article. First, we assess the impact of the interpolation kernel used to perform gridding and degridding on sparse image reconstruction. We find that the Kaiser-Bessel interpolation kernel performs as well as prolate spheroidal wave functions, while providing a computational saving and an analytic form. Second, we apply PURIFY to real interferometric observations from the Very Large Array (VLA) and the Australia Telescope Compact Array (ATCA) and find images recovered by PURIFY are higher quality than those recovered by CLEAN. Third, we discuss how PURIFY reconstructions exhibit additional advantages over those recovered by CLEAN. The latest version of PURIFY, with developments presented in this work, is made publicly available.
△ Less
Submitted 16 November, 2017; v1 submitted 7 October, 2016;
originally announced October 2016.
-
A Fourier dimensionality reduction model for big data interferometric imaging
Authors:
S. Vijay Kartik,
Rafael E. Carrillo,
Jean-Philippe Thiran,
Yves Wiaux
Abstract:
Data dimensionality reduction in radio interferometry can provide savings of computational resources for image reconstruction through reduced memory footprints and lighter computations per iteration, which is important for the scalability of imaging methods to the big data setting of the next-generation telescopes. This article sheds new light on dimensionality reduction from the perspective of co…
▽ More
Data dimensionality reduction in radio interferometry can provide savings of computational resources for image reconstruction through reduced memory footprints and lighter computations per iteration, which is important for the scalability of imaging methods to the big data setting of the next-generation telescopes. This article sheds new light on dimensionality reduction from the perspective of compressed sensing theory and studies its interplay with imaging algorithms designed in the context of convex optimization. We propose a post-gridding linear data embedding to the space spanned by the left singular vectors of the measurement operator, providing a dimensionality reduction below image size. This embedding preserves the null space of the measurement operator and hence also its sampling properties as per compressed sensing theory. We show that this can be approximated by first computing the dirty image and then applying a weighted subsampled discrete Fourier transform to obtain the final reduced data vector. This Fourier dimensionality reduction model ensures a fast implementation of the full measurement operator, essential for any iterative image reconstruction method. The proposed reduction also preserves the i.i.d. Gaussian properties of the original measurement noise. For convex optimization-based imaging algorithms, this is key to justify the use of the standard L2-norm as the data fidelity term. Our simulations confirm that this dimensionality reduction approach can be leveraged by convex optimization algorithms with no loss in imaging quality relative to reconstructing the image from the complete visibility data set. Reconstruction results in simulation settings with no direction dependent effects or calibration errors show promising performance of the proposed dimensionality reduction. MATLAB code implementing the proposed reduction method is available on GitHub.
△ Less
Submitted 28 February, 2017; v1 submitted 7 September, 2016;
originally announced September 2016.
-
Scalable splitting algorithms for big-data interferometric imaging in the SKA era
Authors:
Alexandru Onose,
Rafael E. Carrillo,
Audrey Repetti,
Jason D. McEwen,
Jean-Philippe Thiran,
Jean-Christophe Pesquet,
Yves Wiaux
Abstract:
In the context of next generation radio telescopes, like the Square Kilometre Array, the efficient processing of large-scale datasets is extremely important. Convex optimisation tasks under the compressive sensing framework have recently emerged and provide both enhanced image reconstruction quality and scalability to increasingly larger data sets. We focus herein mainly on scalability and propose…
▽ More
In the context of next generation radio telescopes, like the Square Kilometre Array, the efficient processing of large-scale datasets is extremely important. Convex optimisation tasks under the compressive sensing framework have recently emerged and provide both enhanced image reconstruction quality and scalability to increasingly larger data sets. We focus herein mainly on scalability and propose two new convex optimisation algorithmic structures able to solve the convex optimisation tasks arising in radio-interferometric imaging. They rely on proximal splitting and forward-backward iterations and can be seen, by analogy with the CLEAN major-minor cycle, as running sophisticated CLEAN-like iterations in parallel in multiple data, prior, and image spaces. Both methods support any convex regularisation function, in particular the well studied l1 priors promoting image sparsity in an adequate domain. Tailored for big-data, they employ parallel and distributed computations to achieve scalability, in terms of memory and computational requirements. One of them also exploits randomisation, over data blocks at each iteration, offering further flexibility. We present simulation results showing the feasibility of the proposed methods as well as their advantages compared to state-of-the-art algorithmic solvers. Our Matlab code is available online on GitHub.
△ Less
Submitted 9 August, 2016; v1 submitted 15 January, 2016;
originally announced January 2016.
-
Why CLEAN when you can PURIFY? A new approach for next-generation radio-interferometric imaging
Authors:
Rafael E. Carrillo,
Jason D. McEwen,
Yves Wiaux
Abstract:
In recent works, sparse models and convex optimization techniques have been applied to radio-interferometric (RI) imaging showing the potential to outperform state-of-the-art imaging algorithms in the field. In this talk, I will review our latest contributions in RI imaging, which leverage the versatility of convex optimization to both handle realistic continuous visibilities and offer a highly pa…
▽ More
In recent works, sparse models and convex optimization techniques have been applied to radio-interferometric (RI) imaging showing the potential to outperform state-of-the-art imaging algorithms in the field. In this talk, I will review our latest contributions in RI imaging, which leverage the versatility of convex optimization to both handle realistic continuous visibilities and offer a highly parallelizable structure paving the way to high-dimensional data scalability. Firstly, I will review our recently proposed average sparsity approach, SARA, which relies on the observation that natural images exhibit strong average sparsity over multiple coherent bases. Secondly, I will discuss efficient implementations of SARA, and sparse regularization problems in general, for large-scale imaging problems in a new toolbox dubbed
△ Less
Submitted 17 February, 2015;
originally announced February 2015.
-
PURIFY: a new algorithmic framework for next-generation radio-interferometric imaging
Authors:
Rafael E. Carrillo,
Jason D. McEwen,
Yves Wiaux
Abstract:
In recent works, compressed sensing (CS) and convex optimization techniques have been applied to radio-interferometric imaging showing the potential to outperform state-of-the-art imaging algorithms in the field. We review our latest contributions, which leverage the versatility of convex optimization to both handle realistic continuous visibilities and offer a highly parallelizable structure pavi…
▽ More
In recent works, compressed sensing (CS) and convex optimization techniques have been applied to radio-interferometric imaging showing the potential to outperform state-of-the-art imaging algorithms in the field. We review our latest contributions, which leverage the versatility of convex optimization to both handle realistic continuous visibilities and offer a highly parallelizable structure paving the way to significant acceleration of the reconstruction and high-dimensional data scalability. The new algorithmic structure promoted in a new software PURIFY (beta version) relies on the simultaneous-direction method of multipliers (SDMM). The performance of various sparsity priors is evaluated through simulations in the continuous visibility setting, confirming the superiority of our recent average sparsity approach SARA.
△ Less
Submitted 2 June, 2014;
originally announced June 2014.
-
Exploiting Prior Knowledge in Compressed Sensing Wireless ECG Systems
Authors:
Luisa F. Polania,
Rafael E. Carrillo,
Manuel Blanco-Velasco,
Kenneth E. Barner
Abstract:
Recent results in telecardiology show that compressed sensing (CS) is a promising tool to lower energy consumption in wireless body area networks for electrocardiogram (ECG) monitoring. However, the performance of current CS-based algorithms, in terms of compression rate and reconstruction quality of the ECG, still falls short of the performance attained by state-of-the-art wavelet based algorithm…
▽ More
Recent results in telecardiology show that compressed sensing (CS) is a promising tool to lower energy consumption in wireless body area networks for electrocardiogram (ECG) monitoring. However, the performance of current CS-based algorithms, in terms of compression rate and reconstruction quality of the ECG, still falls short of the performance attained by state-of-the-art wavelet based algorithms. In this paper, we propose to exploit the structure of the wavelet representation of the ECG signal to boost the performance of CS-based methods for compression and reconstruction of ECG signals. More precisely, we incorporate prior information about the wavelet dependencies across scales into the reconstruction algorithms and exploit the high fraction of common support of the wavelet coefficients of consecutive ECG segments. Experimental results utilizing the MIT-BIH Arrhythmia Database show that significant performance gains, in terms of compression rate and reconstruction quality, can be obtained by the proposed algorithms compared to current CS-based methods.
△ Less
Submitted 29 May, 2014; v1 submitted 16 May, 2014;
originally announced May 2014.
-
Sparsity averaging for radio-interferometric imaging
Authors:
Rafael E. Carrillo,
Jason D. McEwen,
Yves Wiaux
Abstract:
We propose a novel regularization method for compressive imaging in the context of the compressed sensing (CS) theory with coherent and redundant dictionaries. Natural images are often complicated and several types of structures can be present at once. It is well known that piecewise smooth images exhibit gradient sparsity, and that images with extended structures are better encapsulated in wavele…
▽ More
We propose a novel regularization method for compressive imaging in the context of the compressed sensing (CS) theory with coherent and redundant dictionaries. Natural images are often complicated and several types of structures can be present at once. It is well known that piecewise smooth images exhibit gradient sparsity, and that images with extended structures are better encapsulated in wavelet frames. Therefore, we here conjecture that promoting average sparsity or compressibility over multiple frames rather than single frames is an extremely powerful regularization prior.
△ Less
Submitted 10 February, 2014;
originally announced February 2014.
-
Lorentzian Iterative Hard Thresholding: Robust Compressed Sensing with Prior Information
Authors:
Rafael E. Carrillo,
Kenneth E. Barner
Abstract:
Commonly employed reconstruction algorithms in compressed sensing (CS) use the $L_2$ norm as the metric for the residual error. However, it is well-known that least squares (LS) based estimators are highly sensitive to outliers present in the measurement vector leading to a poor performance when the noise no longer follows the Gaussian assumption but, instead, is better characterized by heavier-th…
▽ More
Commonly employed reconstruction algorithms in compressed sensing (CS) use the $L_2$ norm as the metric for the residual error. However, it is well-known that least squares (LS) based estimators are highly sensitive to outliers present in the measurement vector leading to a poor performance when the noise no longer follows the Gaussian assumption but, instead, is better characterized by heavier-than-Gaussian tailed distributions. In this paper, we propose a robust iterative hard Thresholding (IHT) algorithm for reconstructing sparse signals in the presence of impulsive noise. To address this problem, we use a Lorentzian cost function instead of the $L_2$ cost function employed by the traditional IHT algorithm. We also modify the algorithm to incorporate prior signal information in the recovery process. Specifically, we study the case of CS with partially known support. The proposed algorithm is a fast method with computational load comparable to the LS based IHT, whilst having the advantage of robustness against heavy-tailed impulsive noise. Sufficient conditions for stability are studied and a reconstruction error bound is derived. We also derive sufficient conditions for stable sparse signal recovery with partially known support. Theoretical analysis shows that including prior support information relaxes the conditions for successful reconstruction. Simulation results demonstrate that the Lorentzian-based IHT algorithm significantly outperform commonly employed sparse reconstruction techniques in impulsive environments, while providing comparable performance in less demanding, light-tailed environments. Numerical results also demonstrate that the partially known support inclusion improves the performance of the proposed algorithm, thereby requiring fewer samples to yield an approximate reconstruction.
△ Less
Submitted 17 July, 2013;
originally announced July 2013.
-
PURIFY: a new approach to radio-interferometric imaging
Authors:
Rafael E. Carrillo,
Jason D. McEwen,
Yves Wiaux
Abstract:
In a recent article series, the authors have promoted convex optimization algorithms for radio-interferometric imaging in the framework of compressed sensing, which leverages sparsity regularization priors for the associated inverse problem and defines a minimization problem for image reconstruction. This approach was shown, in theory and through simulations in a simple discrete visibility setting…
▽ More
In a recent article series, the authors have promoted convex optimization algorithms for radio-interferometric imaging in the framework of compressed sensing, which leverages sparsity regularization priors for the associated inverse problem and defines a minimization problem for image reconstruction. This approach was shown, in theory and through simulations in a simple discrete visibility setting, to have the potential to outperform significantly CLEAN and its evolutions. In this work, we leverage the versatility of convex optimization in solving minimization problems to both handle realistic continuous visibilities and offer a highly parallelizable structure paving the way to significant acceleration of the reconstruction and high-dimensional data scalability. The new algorithmic structure promoted relies on the simultaneous-direction method of multipliers (SDMM), and contrasts with the current major-minor cycle structure of CLEAN and its evolutions, which in particular cannot handle the state-of-the-art minimization problems under consideration where neither the regularization term nor the data term are differentiable functions. We release a beta version of an SDMM-based imaging software written in C and dubbed PURIFY (http://basp-group.github.io/purify/) that handles various sparsity priors, including our recent average sparsity approach SARA. We evaluate the performance of different priors through simulations in the continuous visibility setting, confirming the superiority of SARA.
△ Less
Submitted 2 February, 2014; v1 submitted 16 July, 2013;
originally announced July 2013.
-
Revisiting the spread spectrum effect in radio interferometric imaging: a sparse variant of the w-projection algorithm
Authors:
L. Wolz,
J. D. McEwen,
F. B. Abdalla,
R. E. Carrillo,
Y. Wiaux
Abstract:
Next-generation radio interferometric telescopes will exhibit non-coplanar baseline configurations and wide field-of-views, inducing a w-modulation of the sky image, which in turn induces the spread spectrum effect. We revisit the impact of this effect on imaging quality and study a new algorithmic strategy to deal with the associated operator in the image reconstruction process. In previous studi…
▽ More
Next-generation radio interferometric telescopes will exhibit non-coplanar baseline configurations and wide field-of-views, inducing a w-modulation of the sky image, which in turn induces the spread spectrum effect. We revisit the impact of this effect on imaging quality and study a new algorithmic strategy to deal with the associated operator in the image reconstruction process. In previous studies it has been shown that image recovery in the framework of compressed sensing is improved due to the spread spectrum effect, where the w-modulation can act to increase the incoherence between measurement and sparsifying signal representations. For the purpose of computational efficiency, idealised experiments were performed, where only a constant baseline component w in the pointing direction of the telescope was considered. We extend this analysis to the more realistic setting where the w-component varies for each visibility measurement. Firstly, incorporating varying w-components into imaging algorithms is a computational demanding task. We propose a variant of the w-projection algorithm for this purpose, which is based on an adaptive sparsification procedure, and incorporate it in compressed sensing imaging methods. This sparse matrix variant of the w-projection algorithm is generic and adapts to the support of each kernel. Consequently, it is applicable for all types of direction-dependent effects. Secondly, we show that for w-modulation with varying w-components, reconstruction quality is significantly improved compared to the setting where there is no w-modulation (i.e. w=0), reaching levels comparable to the quality of a constant, maximal w-component. This finding confirms that one may seek to optimise future telescope configurations to promote large w-components, thus enhancing the spread spectrum effect and consequently the fidelity of image reconstruction.
△ Less
Submitted 26 November, 2013; v1 submitted 12 July, 2013;
originally announced July 2013.
-
On sparsity averaging
Authors:
Rafael E. Carrillo,
Jason D. McEwen,
Yves Wiaux
Abstract:
Recent developments in Carrillo et al. (2012) and Carrillo et al. (2013) introduced a novel regularization method for compressive imaging in the context of compressed sensing with coherent redundant dictionaries. The approach relies on the observation that natural images exhibit strong average sparsity over multiple coherent frames. The associated reconstruction algorithm, based on an analysis pri…
▽ More
Recent developments in Carrillo et al. (2012) and Carrillo et al. (2013) introduced a novel regularization method for compressive imaging in the context of compressed sensing with coherent redundant dictionaries. The approach relies on the observation that natural images exhibit strong average sparsity over multiple coherent frames. The associated reconstruction algorithm, based on an analysis prior and a reweighted $\ell_1$ scheme, is dubbed Sparsity Averaging Reweighted Analysis (SARA). We review these advances and extend associated simulations establishing the superiority of SARA to regularization methods based on sparsity in a single frame, for a generic spread spectrum acquisition and for a Fourier acquisition of particular interest in radio astronomy.
△ Less
Submitted 4 July, 2013;
originally announced July 2013.
-
Tensor optimisation for optical-interferometric imaging
Authors:
Anna Auria,
Rafael Carrillo,
Jean-Philippe Thiran,
Yves Wiaux
Abstract:
Image recovery in optical interferometry is an ill-posed nonlinear inverse problem arising from incomplete power spectrum and bispectrum measurements. We reformulate this nonlin- ear problem as a linear problem for the supersymmetric rank-1 order-3 tensor formed by the tensor product of the vector representing the image under scrutiny with itself. On one hand, we propose a linear convex approach f…
▽ More
Image recovery in optical interferometry is an ill-posed nonlinear inverse problem arising from incomplete power spectrum and bispectrum measurements. We reformulate this nonlin- ear problem as a linear problem for the supersymmetric rank-1 order-3 tensor formed by the tensor product of the vector representing the image under scrutiny with itself. On one hand, we propose a linear convex approach for tensor recovery with built-in supersymmetry, and regularising the inverse problem through a nuclear norm relaxation of a low-rank constraint. On the other hand, we also study a nonlinear nonconvex approach with built-in rank-1 con- straint but where supersymmetry is relaxed, formulating the problem for the tensor product of 3 vectors. In this second approach, only linear convex minimisation subproblems are how- ever solved, alternately and iteratively for the 3 vectors. We provide a comparative analysis of these two novel approaches through numerical simulations on small-size images.
△ Less
Submitted 18 October, 2013; v1 submitted 28 June, 2013;
originally announced June 2013.
-
The varying w spread spectrum effect for radio interferometric imaging
Authors:
Laura Wolz,
Filipe B. Abdalla,
Rafael E. Carrillo,
Yves Wiaux,
Jason D. McEwen
Abstract:
We study the impact of the spread spectrum effect in radio interferometry on the quality of image reconstruction. This spread spectrum effect will be induced by the wide field-of-view of forthcoming radio interferometric telescopes. The resulting chirp modulation improves the quality of reconstructed interferometric images by increasing the incoherence of the measurement and sparsity dictionaries.…
▽ More
We study the impact of the spread spectrum effect in radio interferometry on the quality of image reconstruction. This spread spectrum effect will be induced by the wide field-of-view of forthcoming radio interferometric telescopes. The resulting chirp modulation improves the quality of reconstructed interferometric images by increasing the incoherence of the measurement and sparsity dictionaries. We extend previous studies of this effect to consider the more realistic setting where the chirp modulation varies for each visibility measurement made by the telescope. In these first preliminary results, we show that for this setting the quality of reconstruction improves significantly over the case without chirp modulation and achieves almost the reconstruction quality of the case of maximal, constant chirp modulation.
△ Less
Submitted 30 January, 2013;
originally announced January 2013.
-
Sparsity Averaging for Compressive Imaging
Authors:
Rafael E. Carrillo,
Jason D. McEwen,
Dimitri Van De Ville,
Jean-Philippe Thiran,
Yves Wiaux
Abstract:
We discuss a novel sparsity prior for compressive imaging in the context of the theory of compressed sensing with coherent redundant dictionaries, based on the observation that natural images exhibit strong average sparsity over multiple coherent frames. We test our prior and the associated algorithm, based on an analysis reweighted $\ell_1$ formulation, through extensive numerical simulations on…
▽ More
We discuss a novel sparsity prior for compressive imaging in the context of the theory of compressed sensing with coherent redundant dictionaries, based on the observation that natural images exhibit strong average sparsity over multiple coherent frames. We test our prior and the associated algorithm, based on an analysis reweighted $\ell_1$ formulation, through extensive numerical simulations on natural images for spread spectrum and random Gaussian acquisition schemes. Our results show that average sparsity outperforms state-of-the-art priors that promote sparsity in a single orthonormal basis or redundant frame, or that promote gradient sparsity. Code and test data are available at https://github.com/basp-group/sopt.
△ Less
Submitted 16 April, 2013; v1 submitted 11 August, 2012;
originally announced August 2012.
-
Sparsity Averaging Reweighted Analysis (SARA): a novel algorithm for radio-interferometric imaging
Authors:
R. E. Carrillo,
J. D. McEwen,
Y. Wiaux
Abstract:
We propose a novel algorithm for image reconstruction in radio interferometry. The ill-posed inverse problem associated with the incomplete Fourier sampling identified by the visibility measurements is regularized by the assumption of average signal sparsity over representations in multiple wavelet bases. The algorithm, defined in the versatile framework of convex optimization, is dubbed Sparsity…
▽ More
We propose a novel algorithm for image reconstruction in radio interferometry. The ill-posed inverse problem associated with the incomplete Fourier sampling identified by the visibility measurements is regularized by the assumption of average signal sparsity over representations in multiple wavelet bases. The algorithm, defined in the versatile framework of convex optimization, is dubbed Sparsity Averaging Reweighted Analysis (SARA). We show through simulations that the proposed approach outperforms state-of-the-art imaging methods in the field, which are based on the assumption of signal sparsity in a single basis only.
△ Less
Submitted 30 July, 2012; v1 submitted 14 May, 2012;
originally announced May 2012.
-
Companions of Bright Barred Shapley Ames Galaxies
Authors:
J. Antonio Garcia-Barreto,
Rene Carrillo,
Nelson Vera-Villamizar
Abstract:
Companion galaxy environment for a subset of 78 bright and nearby barred galaxies from the Shapley Ames Catalog is presented. Among spiral barred galaxies there are Seyfert galaxies, galaxies with circumnuclear structures, galaxies not associated with any large scale galaxy cloud structure, galaxies with peculiar disk morphology (crooked arms) and galaxies with normal disk morphology; the list i…
▽ More
Companion galaxy environment for a subset of 78 bright and nearby barred galaxies from the Shapley Ames Catalog is presented. Among spiral barred galaxies there are Seyfert galaxies, galaxies with circumnuclear structures, galaxies not associated with any large scale galaxy cloud structure, galaxies with peculiar disk morphology (crooked arms) and galaxies with normal disk morphology; the list includes all Hubble types. The companion galaxy list includes number of companion galaxies within 20 diameters (D25), their Hubble type and projected separation distance. Additionally, companion environment was searched for four known active spiral galaxies, three of them are Seyfert galaxies, namely, NGC 1068, NGC 1097, NGC 5548 and one is a starburst galaxy, M82. Among the results obtained it is noted that the only spiral barred galaxy classified as Sy 1 in our list has no companions within a projected distance of 20 diameters; 6 out of 10 Sy 2 bar galaxies have no companions within 10 diamters, 6 out of 10 Sy 2 galaxies have one or more companions at projected separation distances between 10 and 20 diameters; 6 out of 12 galaxies with circumnuclear structures have 2 or more companions within 20 diameters
△ Less
Submitted 9 July, 2003;
originally announced July 2003.
-
Optical Spectroscopy of the Central Regions of Bright Barred Spiral Galaxies
Authors:
J. A. Garcia-Barreto,
H. Aceves,
O. Kuhn,
G. Canalizo,
R. Carrillo,
J. Franco
Abstract:
Optical red spectra of a set of 18 bright barred spiral galaxies are presented. The study is aimed at determining the local kinematics, and the physical conditions of ionized gas in the compact nucleus (inside a diameter of 5") and in the circumnuclear regions (inside a diameter of 20"). Only 8 galaxies showed bright emission from their east and west side of the nucleus. The spectrum of each reg…
▽ More
Optical red spectra of a set of 18 bright barred spiral galaxies are presented. The study is aimed at determining the local kinematics, and the physical conditions of ionized gas in the compact nucleus (inside a diameter of 5") and in the circumnuclear regions (inside a diameter of 20"). Only 8 galaxies showed bright emission from their east and west side of the nucleus. The spectrum of each region was analized separately. In other 10 galaxies the line emission was so weak that we were only able to obtain an average spectrum of the central emission. No emission was detected in the remaining 8 galaxies. An estimate of the dynamical mass is presented based on the observed velocities in the circumnuclear regions.
In NGC 4314 and NGC 6951, that show H_alpha emission distributed in circumnuclear ring structures, we determine the [NII]/H_alpha and [SII]/H_alpha ratios for the eastern and western regions of the rings. The velocity difference for the two sides is used to derive the rotation velocity of the gas around the compact nucleus. The ratio [NII]6583/H_alpha is a factor of 2 larger in the compact nucleus of NGC 6951 than in its western side. The electron gas densities have been estimated from the [SII] lines ratio.
△ Less
Submitted 5 August, 1999;
originally announced August 1999.