-
On the Use of Autoregressive Methods for Audio Inpainting
Authors:
Ondřej Mokrý,
Pavel Rajmic
Abstract:
The paper presents an evaluation of popular audio inpainting methods based on autoregressive modelling, namely, extrapolation-based and Janssen methods. A novel variant of the Janssen method suitable for gap inpainting is also proposed. The main differences between the particular popular approaches are pointed out, and a mid-scale computational experiment is presented. The results demonstrate the…
▽ More
The paper presents an evaluation of popular audio inpainting methods based on autoregressive modelling, namely, extrapolation-based and Janssen methods. A novel variant of the Janssen method suitable for gap inpainting is also proposed. The main differences between the particular popular approaches are pointed out, and a mid-scale computational experiment is presented. The results demonstrate the importance of the choice of the AR model estimator and the suitability of the new gap-wise Janssen method.
△ Less
Submitted 7 March, 2024;
originally announced March 2024.
-
Audio Declip** with (Weighted) Analysis Social Sparsity
Authors:
Pavel Záviška,
Pavel Rajmic
Abstract:
We develop the analysis (cosparse) variant of the popular audio declip** algorithm of Siedenburg et al. (2014). Furthermore, we extend both the old and the new variants by the possibility of weighting the time-frequency coefficients. We examine the audio reconstruction performance of several combinations of weights and shrinkage operators. The weights are shown to improve the reconstruction qual…
▽ More
We develop the analysis (cosparse) variant of the popular audio declip** algorithm of Siedenburg et al. (2014). Furthermore, we extend both the old and the new variants by the possibility of weighting the time-frequency coefficients. We examine the audio reconstruction performance of several combinations of weights and shrinkage operators. The weights are shown to improve the reconstruction quality in some cases; however, the best scores achieved by the non-weighted methods are not surpassed with the help of weights. Yet, the analysis Empirical Wiener (EW) shrinkage was able to reach the quality of a computationally more expensive competitor, the Persistent Empirical Wiener (PEW). Moreover, the proposed analysis variant incorporating PEW slightly outperforms the synthesis counterpart in terms of an auditorily motivated metric.
△ Less
Submitted 27 June, 2022; v1 submitted 20 May, 2022;
originally announced May 2022.
-
Audio declip** performance enhancement via crossfading
Authors:
Pavel Záviška,
Pavel Rajmic,
Ondřej Mokrý
Abstract:
Some audio declip** methods produce waveforms that do not fully respect the physical process of clip**, which is why we refer to them as inconsistent. This letter reports what effect on perception it has if the solution by inconsistent methods is forced consistent by postprocessing. We first propose a simple sample replacement method, then we identify its main weaknesses and propose an improve…
▽ More
Some audio declip** methods produce waveforms that do not fully respect the physical process of clip**, which is why we refer to them as inconsistent. This letter reports what effect on perception it has if the solution by inconsistent methods is forced consistent by postprocessing. We first propose a simple sample replacement method, then we identify its main weaknesses and propose an improved variant. The experiments show that the vast majority of inconsistent declip** methods significantly benefit from the proposed approach in terms of objective perceptual metrics. In particular, we show that the SS PEW method based on social sparsity combined with the proposed method performs comparable to top methods from the consistent class, but at a computational cost of one order of magnitude lower.
△ Less
Submitted 7 April, 2021;
originally announced April 2021.
-
Audio Dequantization Using (Co)Sparse (Non)Convex Methods
Authors:
Pavel Záviška,
Pavel Rajmic,
Ondřej Mokrý
Abstract:
The paper deals with the hitherto neglected topic of audio dequantization. It reviews the state-of-the-art sparsity-based approaches and proposes several new methods. Convex as well as non-convex approaches are included, and all the presented formulations come in both the synthesis and analysis variants. In the experiments the methods are evaluated using the signal-to-distortion ratio (SDR) and PE…
▽ More
The paper deals with the hitherto neglected topic of audio dequantization. It reviews the state-of-the-art sparsity-based approaches and proposes several new methods. Convex as well as non-convex approaches are included, and all the presented formulations come in both the synthesis and analysis variants. In the experiments the methods are evaluated using the signal-to-distortion ratio (SDR) and PEMO-Q, a perceptually motivated metric.
△ Less
Submitted 10 February, 2021; v1 submitted 30 October, 2020;
originally announced October 2020.
-
A survey and an extensive evaluation of popular audio declip** methods
Authors:
Pavel Záviška,
Pavel Rajmic,
Alexey Ozerov,
Lucas Rencker
Abstract:
Dynamic range limitations in signal processing often lead to clip**, or saturation, in signals. The task of audio declip** is estimating the original audio signal, given its clipped measurements, and has attracted much interest in recent years. Audio declip** algorithms often make assumptions about the underlying signal, such as sparsity or low-rankness, and about the measurement system. In…
▽ More
Dynamic range limitations in signal processing often lead to clip**, or saturation, in signals. The task of audio declip** is estimating the original audio signal, given its clipped measurements, and has attracted much interest in recent years. Audio declip** algorithms often make assumptions about the underlying signal, such as sparsity or low-rankness, and about the measurement system. In this paper, we provide an extensive review of audio declip** algorithms proposed in the literature. For each algorithm, we present assumptions that are made about the audio signal, the modeling domain, and the optimization algorithm. Furthermore, we provide an extensive numerical evaluation of popular declip** algorithms, on real audio data. We evaluate each algorithm in terms of the Signal-to-Distortion Ratio, and also using perceptual metrics of sound quality. The article is accompanied by a repository containing the evaluated methods.
△ Less
Submitted 4 January, 2021; v1 submitted 15 July, 2020;
originally announced July 2020.
-
Flexible framework for audio reconstruction
Authors:
Ondřej Mokrý,
Pavel Rajmic,
Pavel Záviška
Abstract:
The paper presents a unified, flexible framework for the tasks of audio inpainting, declip**, and dequantization. The concept is further extended to cover analogous degradation models in a transformed domain, e.g. quantization of the signal's time-frequency coefficients. The task of reconstructing an audio signal from degraded observations in two different domains is formulated as an inverse pro…
▽ More
The paper presents a unified, flexible framework for the tasks of audio inpainting, declip**, and dequantization. The concept is further extended to cover analogous degradation models in a transformed domain, e.g. quantization of the signal's time-frequency coefficients. The task of reconstructing an audio signal from degraded observations in two different domains is formulated as an inverse problem, and several algorithmic solutions are developed. The viability of the presented concept is demonstrated on an example where audio reconstruction from partial and quantized observations of both the time-domain signal and its time-frequency coefficients is carried out.
△ Less
Submitted 29 July, 2020; v1 submitted 23 April, 2020;
originally announced April 2020.
-
Sparse and Cosparse Audio Dequantization Using Convex Optimization
Authors:
Pavel Záviška,
Pavel Rajmic
Abstract:
The paper shows the potential of sparsity-based methods in restoring quantized signals. Following up on the study of Brauer et al. (IEEE ICASSP 2016), we significantly extend the range of the evaluation scenarios: we introduce the analysis (cosparse) model, we use more effective algorithms, we experiment with another time-frequency transform. The paper shows that the analysis-based model performs…
▽ More
The paper shows the potential of sparsity-based methods in restoring quantized signals. Following up on the study of Brauer et al. (IEEE ICASSP 2016), we significantly extend the range of the evaluation scenarios: we introduce the analysis (cosparse) model, we use more effective algorithms, we experiment with another time-frequency transform. The paper shows that the analysis-based model performs comparably to the synthesis-model, but the Gabor transform produces better results than the originally used cosine transform. Last but not least, we provide codes and data in a reproducible way.
△ Less
Submitted 20 May, 2020; v1 submitted 5 March, 2020;
originally announced March 2020.
-
Audio Inpainting: Revisited and Reweighted
Authors:
Ondřej Mokrý,
Pavel Rajmic
Abstract:
We deal with the problem of sparsity-based audio inpainting, i.e. filling in the missing segments of audio. A consequence of the approaches based on mathematical optimization is the insufficient amplitude of the signal in the filled gaps. Remaining in the framework based on sparsity and convex optimization, we propose improvements to audio inpainting, aiming at compensating for such an energy loss…
▽ More
We deal with the problem of sparsity-based audio inpainting, i.e. filling in the missing segments of audio. A consequence of the approaches based on mathematical optimization is the insufficient amplitude of the signal in the filled gaps. Remaining in the framework based on sparsity and convex optimization, we propose improvements to audio inpainting, aiming at compensating for such an energy loss. The new ideas are based on different types of weighting, both in the coefficient and the time domains. We show that our propositions improve the inpainting performance in terms of both the SNR and ODG.
△ Less
Submitted 21 August, 2020; v1 submitted 8 January, 2020;
originally announced January 2020.
-
Psychoacoustically Motivated Audio Declip** Based on Weighted l1 Minimization
Authors:
Pavel Záviška,
Pavel Rajmic,
Jíří Schimmel
Abstract:
A novel method for audio declip** based on sparsity is presented. The method incorporates psychoacoustic information by weighting the transform coefficients in the $\ell_1$ minimization. Weighting leads to an improved quality of restoration while retaining a low complexity of the algorithm. Three possible constructions of the weights are proposed, based on the absolute threshold of hearing, the…
▽ More
A novel method for audio declip** based on sparsity is presented. The method incorporates psychoacoustic information by weighting the transform coefficients in the $\ell_1$ minimization. Weighting leads to an improved quality of restoration while retaining a low complexity of the algorithm. Three possible constructions of the weights are proposed, based on the absolute threshold of hearing, the global masking threshold and on a quadratic curve. Experiments compare the restoration quality according to the signal-to-distortion ratio (SDR) and PEMO-Q objective difference grade (ODG) and indicate that with correctly chosen weights, the presented method is able to compete, or even outperform, the current state of the art.
△ Less
Submitted 1 July, 2020; v1 submitted 2 May, 2019;
originally announced May 2019.
-
Introducing SPAIN (SParse Audio INpainter)
Authors:
Ondřej Mokrý,
Pavel Záviška,
Pavel Rajmic,
Vítězslav Veselý
Abstract:
A novel sparsity-based algorithm for audio inpainting is proposed. It is an adaptation of the SPADE algorithm by Kitić et al., originally developed for audio declip**, to the task of audio inpainting. The new SPAIN (SParse Audio INpainter) comes in synthesis and analysis variants. Experiments show that both A-SPAIN and S-SPAIN outperform other sparsity-based inpainting algorithms. Moreover, A-SP…
▽ More
A novel sparsity-based algorithm for audio inpainting is proposed. It is an adaptation of the SPADE algorithm by Kitić et al., originally developed for audio declip**, to the task of audio inpainting. The new SPAIN (SParse Audio INpainter) comes in synthesis and analysis variants. Experiments show that both A-SPAIN and S-SPAIN outperform other sparsity-based inpainting algorithms. Moreover, A-SPAIN performs on a par with the state-of-the-art method based on linear prediction in terms of the SNR, and, for larger gaps, SPAIN is even slightly better in terms of the PEMO-Q psychoacoustic criterion.
△ Less
Submitted 18 June, 2019; v1 submitted 31 October, 2018;
originally announced October 2018.
-
Proceedings of the third "international Traveling Workshop on Interactions between Sparse models and Technology" (iTWIST'16)
Authors:
V. Abrol,
O. Absil,
P. -A. Absil,
S. Anthoine,
P. Antoine,
T. Arildsen,
N. Bertin,
F. Bleichrodt,
J. Bobin,
A. Bol,
A. Bonnefoy,
F. Caltagirone,
V. Cambareri,
C. Chenot,
V. Crnojević,
M. Daňková,
K. Degraux,
J. Eisert,
J. M. Fadili,
M. Gabrié,
N. Gac,
D. Giacobello,
A. Gonzalez,
C. A. Gomez Gonzalez,
A. González
, et al. (36 additional authors not shown)
Abstract:
The third edition of the "international - Traveling Workshop on Interactions between Sparse models and Technology" (iTWIST) took place in Aalborg, the 4th largest city in Denmark situated beautifully in the northern part of the country, from the 24th to 26th of August 2016. The workshop venue was at the Aalborg University campus. One implicit objective of this biennial workshop is to foster collab…
▽ More
The third edition of the "international - Traveling Workshop on Interactions between Sparse models and Technology" (iTWIST) took place in Aalborg, the 4th largest city in Denmark situated beautifully in the northern part of the country, from the 24th to 26th of August 2016. The workshop venue was at the Aalborg University campus. One implicit objective of this biennial workshop is to foster collaboration between international scientific teams by disseminating ideas through both specific oral/poster presentations and free discussions. For this third edition, iTWIST'16 gathered about 50 international participants and features 8 invited talks, 12 oral presentations, and 12 posters on the following themes, all related to the theory, application and generalization of the "sparsity paradigm": Sparsity-driven data sensing and processing (e.g., optics, computer vision, genomics, biomedical, digital communication, channel estimation, astronomy); Application of sparse models in non-convex/non-linear inverse problems (e.g., phase retrieval, blind deconvolution, self calibration); Approximate probabilistic inference for sparse problems; Sparse machine learning and inference; "Blind" inverse problems and dictionary learning; Optimization for sparse modelling; Information theory, geometry and randomness; Sparsity? What's next? (Discrete-valued signals; Union of low-dimensional spaces, Cosparsity, mixed/group norm, model-based, low-complexity models, ...); Matrix/manifold sensing/processing (graph, low-rank approximation, ...); Complexity/accuracy tradeoffs in numerical methods/optimization; Electronic/optical compressive sensors (hardware).
△ Less
Submitted 14 September, 2016;
originally announced September 2016.
-
Proceedings of the second "international Traveling Workshop on Interactions between Sparse models and Technology" (iTWIST'14)
Authors:
L. Jacques,
C. De Vleeschouwer,
Y. Boursier,
P. Sudhakar,
C. De Mol,
A. Pizurica,
S. Anthoine,
P. Vandergheynst,
P. Frossard,
C. Bilen,
S. Kitic,
N. Bertin,
R. Gribonval,
N. Boumal,
B. Mishra,
P. -A. Absil,
R. Sepulchre,
S. Bundervoet,
C. Schretter,
A. Dooms,
P. Schelkens,
O. Chabiron,
F. Malgouyres,
J. -Y. Tourneret,
N. Dobigeon
, et al. (42 additional authors not shown)
Abstract:
The implicit objective of the biennial "international - Traveling Workshop on Interactions between Sparse models and Technology" (iTWIST) is to foster collaboration between international scientific teams by disseminating ideas through both specific oral/poster presentations and free discussions. For its second edition, the iTWIST workshop took place in the medieval and picturesque town of Namur in…
▽ More
The implicit objective of the biennial "international - Traveling Workshop on Interactions between Sparse models and Technology" (iTWIST) is to foster collaboration between international scientific teams by disseminating ideas through both specific oral/poster presentations and free discussions. For its second edition, the iTWIST workshop took place in the medieval and picturesque town of Namur in Belgium, from Wednesday August 27th till Friday August 29th, 2014. The workshop was conveniently located in "The Arsenal" building within walking distance of both hotels and town center. iTWIST'14 has gathered about 70 international participants and has featured 9 invited talks, 10 oral presentations, and 14 posters on the following themes, all related to the theory, application and generalization of the "sparsity paradigm": Sparsity-driven data sensing and processing; Union of low dimensional subspaces; Beyond linear and convex inverse problem; Matrix/manifold/graph sensing/processing; Blind inverse problems and dictionary learning; Sparsity and computational neuroscience; Information theory, geometry and randomness; Complexity/accuracy tradeoffs in numerical methods; Sparsity? What's next?; Sparse machine learning and inference.
△ Less
Submitted 9 October, 2014; v1 submitted 2 October, 2014;
originally announced October 2014.
