-
Neural empirical interpolation method for nonlinear model reduction
Authors:
Max Hirsch,
Federico Pichi,
Jan S. Hesthaven
Abstract:
In this paper, we introduce the neural empirical interpolation method (NEIM), a neural network-based alternative to the discrete empirical interpolation method for reducing the time complexity of computing the nonlinear term in a reduced order model (ROM) for a parameterized nonlinear partial differential equation. NEIM is a greedy algorithm which accomplishes this reduction by approximating an af…
▽ More
In this paper, we introduce the neural empirical interpolation method (NEIM), a neural network-based alternative to the discrete empirical interpolation method for reducing the time complexity of computing the nonlinear term in a reduced order model (ROM) for a parameterized nonlinear partial differential equation. NEIM is a greedy algorithm which accomplishes this reduction by approximating an affine decomposition of the nonlinear term of the ROM, where the vector terms of the expansion are given by neural networks depending on the ROM solution, and the coefficients are given by an interpolation of some "optimal" coefficients. Because NEIM is based on a greedy strategy, we are able to provide a basic error analysis to investigate its performance. NEIM has the advantages of being easy to implement in models with automatic differentiation, of being a nonlinear projection of the ROM nonlinearity, of being efficient for both nonlocal and local nonlinearities, and of relying solely on data and not the explicit form of the ROM nonlinearity. We demonstrate the effectiveness of the methodology on solution-dependent and solution-independent nonlinearities, a nonlinear elliptic problem, and a nonlinear parabolic model of liquid crystals.
△ Less
Submitted 5 June, 2024;
originally announced June 2024.
-
Training one model to detect heart and lung sound events from single point auscultations
Authors:
Leander Melms,
Robert R. Ilesan,
Ulrich Köhler,
Olaf Hildebrandt,
Regina Conradt,
Jens Eckstein,
Cihan Atila,
Sami Matrood,
Bernhard Schieffer,
Jürgen R. Schaefer,
Tobias Müller,
Julius Obergassel,
Nadine Schlicker,
Martin C. Hirsch
Abstract:
Objective: This work proposes a semi-supervised training approach for detecting lung and heart sounds simultaneously with only one trained model and in invariance to the auscultation point. Methods: We use open-access data from the 2016 Physionet/CinC Challenge, the 2022 George Moody Challenge, and from the lung sound database HF_V1. We first train specialist single-task models using foreground gr…
▽ More
Objective: This work proposes a semi-supervised training approach for detecting lung and heart sounds simultaneously with only one trained model and in invariance to the auscultation point. Methods: We use open-access data from the 2016 Physionet/CinC Challenge, the 2022 George Moody Challenge, and from the lung sound database HF_V1. We first train specialist single-task models using foreground ground truth (GT) labels from different auscultation databases to identify background sound events in the respective lung and heart auscultation databases. The pseudo-labels generated in this way were combined with the ground truth labels in a new training iteration, such that a new model was subsequently trained to detect foreground and background signals. Benchmark tests ensured that the newly trained model could detect both, lung, and heart sound events in different auscultation sites without regressing on the original task. We also established hand-validated labels for the respective background signal in heart and lung sound auscultations to evaluate the models. Results: In this work, we report for the first time results for i) a multi-class prediction for lung sound events and ii) for simultaneous detection of heart and lung sound events and achieve competitive results using only one model. The combined multi-task model regressed slightly in heart sound detection and gained significantly in lung sound detection accuracy with an overall macro F1 score of 39.2% over six classes, representing a 6.7% improvement over the single-task baseline models. Conclusion/Significance: To the best of our knowledge, this is the first approach developed to date for measuring heart and lung sound events invariant to both, the auscultation site and capturing device. Hence, our model is capable of performing lung and heart sound detection from any auscultation location.
△ Less
Submitted 15 January, 2023;
originally announced January 2023.
-
The State of Fortran
Authors:
Laurence Kedward,
Balint Aradi,
Ondrej Certik,
Milan Curcic,
Sebastian Ehlert,
Philipp Engel,
Rohit Goswami,
Michael Hirsch,
Asdrubal Lozada-Blanco,
Vincent Magnin,
Arjen Markus,
Emanuele Pagone,
Ivan Pribec,
Brad Richardson,
Harris Snyder,
John Urban,
Jeremie Vandenplas
Abstract:
A community of developers has formed to modernize the Fortran ecosystem. In this article, we describe the high-level features of Fortran that continue to make it a good choice for scientists and engineers in the 21st century. Ongoing efforts include the development of a Fortran standard library and package manager, the fostering of a friendly and welcoming online community, improved compiler suppo…
▽ More
A community of developers has formed to modernize the Fortran ecosystem. In this article, we describe the high-level features of Fortran that continue to make it a good choice for scientists and engineers in the 21st century. Ongoing efforts include the development of a Fortran standard library and package manager, the fostering of a friendly and welcoming online community, improved compiler support, and language feature development. The lessons learned are common across contemporary programming languages and help reduce the learning curve and increase adoption of Fortran.
△ Less
Submitted 30 March, 2022; v1 submitted 28 March, 2022;
originally announced March 2022.
-
The Unreasonable Effectiveness of Texture Transfer for Single Image Super-resolution
Authors:
Muhammad Waleed Gondal,
Bernhard Schölkopf,
Michael Hirsch
Abstract:
While implicit generative models such as GANs have shown impressive results in high quality image reconstruction and manipulation using a combination of various losses, we consider a simpler approach leading to surprisingly strong results. We show that texture loss alone allows the generation of perceptually high quality images. We provide a better understanding of texture constraining mechanism a…
▽ More
While implicit generative models such as GANs have shown impressive results in high quality image reconstruction and manipulation using a combination of various losses, we consider a simpler approach leading to surprisingly strong results. We show that texture loss alone allows the generation of perceptually high quality images. We provide a better understanding of texture constraining mechanism and develop a novel semantically guided texture constraining method for further improvement. Using a recently developed perceptual metric employing "deep features" and termed LPIPS, the method obtains state-of-the-art results. Moreover, we show that a texture representation of those deep features better capture the perceptual quality of an image than the original deep features. Using texture information, off-the-shelf deep classification networks (without training) perform as well as the best performing (tuned and calibrated) LPIPS metrics. The code is publicly available.
△ Less
Submitted 31 July, 2018;
originally announced August 2018.
-
Perceptual Video Super Resolution with Enhanced Temporal Consistency
Authors:
Eduardo Pérez-Pellitero,
Mehdi S. M. Sajjadi,
Michael Hirsch,
Bernhard Schölkopf
Abstract:
With the advent of perceptual loss functions, new possibilities in super-resolution have emerged, and we currently have models that successfully generate near-photorealistic high-resolution images from their low-resolution observations. Up to now, however, such approaches have been exclusively limited to single image super-resolution. The application of perceptual loss functions on video processin…
▽ More
With the advent of perceptual loss functions, new possibilities in super-resolution have emerged, and we currently have models that successfully generate near-photorealistic high-resolution images from their low-resolution observations. Up to now, however, such approaches have been exclusively limited to single image super-resolution. The application of perceptual loss functions on video processing still entails several challenges, mostly related to the lack of temporal consistency of the generated images, i.e., flickering artifacts. In this work, we present a novel adversarial recurrent network for video upscaling that is able to produce realistic textures in a temporally consistent way. The proposed architecture naturally leverages information from previous frames due to its recurrent architecture, i.e. the input to the generator is composed of the low-resolution image and, additionally, the warped output of the network at the previous step. Together with a video discriminator, we also propose additional loss functions to further reinforce temporal consistency in the generated sequences. The experimental validation of our algorithm shows the effectiveness of our approach which obtains images with high perceptual quality and improved temporal consistency.
△ Less
Submitted 2 May, 2019; v1 submitted 20 July, 2018;
originally announced July 2018.
-
Automatic Estimation of Modulation Transfer Functions
Authors:
Matthias Bauer,
Valentin Volchkov,
Michael Hirsch,
Bernhard Schölkopf
Abstract:
The modulation transfer function (MTF) is widely used to characterise the performance of optical systems. Measuring it is costly and it is thus rarely available for a given lens specimen. Instead, MTFs based on simulations or, at best, MTFs measured on other specimens of the same lens are used. Fortunately, images recorded through an optical system contain ample information about its MTF, only tha…
▽ More
The modulation transfer function (MTF) is widely used to characterise the performance of optical systems. Measuring it is costly and it is thus rarely available for a given lens specimen. Instead, MTFs based on simulations or, at best, MTFs measured on other specimens of the same lens are used. Fortunately, images recorded through an optical system contain ample information about its MTF, only that it is confounded with the statistics of the images. This work presents a method to estimate the MTF of camera lens systems directly from photographs, without the need for expensive equipment. We use a custom grid display to accurately measure the point response of lenses to acquire ground truth training data. We then use the same lenses to record natural images and employ a data-driven supervised learning approach using a convolutional neural network to estimate the MTF on small image patches, aggregating the information into MTF charts over the entire field of view. It generalises to unseen lenses and can be applied for single photographs, with the performance improving if multiple photographs are available.
△ Less
Submitted 4 May, 2018;
originally announced May 2018.
-
Learning Blind Motion Deblurring
Authors:
Patrick Wieschollek,
Michael Hirsch,
Bernhard Schölkopf,
Hendrik P. A. Lensch
Abstract:
As handheld video cameras are now commonplace and available in every smartphone, images and videos can be recorded almost everywhere at anytime. However, taking a quick shot frequently yields a blurry result due to unwanted camera shake during recording or moving objects in the scene. Removing these artifacts from the blurry recordings is a highly ill-posed problem as neither the sharp image nor t…
▽ More
As handheld video cameras are now commonplace and available in every smartphone, images and videos can be recorded almost everywhere at anytime. However, taking a quick shot frequently yields a blurry result due to unwanted camera shake during recording or moving objects in the scene. Removing these artifacts from the blurry recordings is a highly ill-posed problem as neither the sharp image nor the motion blur kernel is known. Propagating information between multiple consecutive blurry observations can help restore the desired sharp image or video. Solutions for blind deconvolution based on neural networks rely on a massive amount of ground-truth data which is hard to acquire. In this work, we propose an efficient approach to produce a significant amount of realistic training data and introduce a novel recurrent network architecture to deblur frames taking temporal information into account, which can efficiently handle arbitrary spatial and temporal input sizes. We demonstrate the versatility of our approach in a comprehensive comparison on a number of challening real-world examples.
△ Less
Submitted 14 August, 2017;
originally announced August 2017.
-
Weakly-supervised localization of diabetic retinopathy lesions in retinal fundus images
Authors:
Waleed M. Gondal,
Jan M. Köhler,
René Grzeszick,
Gernot A. Fink,
Michael Hirsch
Abstract:
Convolutional neural networks (CNNs) show impressive performance for image classification and detection, extending heavily to the medical image domain. Nevertheless, medical experts are sceptical in these predictions as the nonlinear multilayer structure resulting in a classification outcome is not directly graspable. Recently, approaches have been shown which help the user to understand the discr…
▽ More
Convolutional neural networks (CNNs) show impressive performance for image classification and detection, extending heavily to the medical image domain. Nevertheless, medical experts are sceptical in these predictions as the nonlinear multilayer structure resulting in a classification outcome is not directly graspable. Recently, approaches have been shown which help the user to understand the discriminative regions within an image which are decisive for the CNN to conclude to a certain class. Although these approaches could help to build trust in the CNNs predictions, they are only slightly shown to work with medical image data which often poses a challenge as the decision for a class relies on different lesion areas scattered around the entire image. Using the DiaretDB1 dataset, we show that on retina images different lesion areas fundamental for diabetic retinopathy are detected on an image level with high accuracy, comparable or exceeding supervised methods. On lesion level, we achieve few false positives with high sensitivity, though, the network is solely trained on image-level labels which do not include information about existing lesions. Classifying between diseased and healthy images, we achieve an AUC of 0.954 on the DiaretDB1.
△ Less
Submitted 29 June, 2017;
originally announced June 2017.
-
Online Video Deblurring via Dynamic Temporal Blending Network
Authors:
Tae Hyun Kim,
Kyoung Mu Lee,
Bernhard Schölkopf,
Michael Hirsch
Abstract:
State-of-the-art video deblurring methods are capable of removing non-uniform blur caused by unwanted camera shake and/or object motion in dynamic scenes. However, most existing methods are based on batch processing and thus need access to all recorded frames, rendering them computationally demanding and time consuming and thus limiting their practical use. In contrast, we propose an online (seque…
▽ More
State-of-the-art video deblurring methods are capable of removing non-uniform blur caused by unwanted camera shake and/or object motion in dynamic scenes. However, most existing methods are based on batch processing and thus need access to all recorded frames, rendering them computationally demanding and time consuming and thus limiting their practical use. In contrast, we propose an online (sequential) video deblurring method based on a spatio-temporal recurrent network that allows for real-time performance. In particular, we introduce a novel architecture which extends the receptive field while kee** the overall size of the network small to enable fast execution. In doing so, our network is able to remove even large blur caused by strong camera shake and/or fast moving objects. Furthermore, we propose a novel network layer that enforces temporal consistency between consecutive frames by dynamic temporal blending which compares and adaptively (at test time) shares features obtained at different time steps. We show the superiority of the proposed method in an extensive experimental evaluation.
△ Less
Submitted 11 April, 2017;
originally announced April 2017.
-
Discriminative Transfer Learning for General Image Restoration
Authors:
Lei Xiao,
Felix Heide,
Wolfgang Heidrich,
Bernhard Schölkopf,
Michael Hirsch
Abstract:
Recently, several discriminative learning approaches have been proposed for effective image restoration, achieving convincing trade-off between image quality and computational efficiency. However, these methods require separate training for each restoration task (e.g., denoising, deblurring, demosaicing) and problem condition (e.g., noise level of input images). This makes it time-consuming and di…
▽ More
Recently, several discriminative learning approaches have been proposed for effective image restoration, achieving convincing trade-off between image quality and computational efficiency. However, these methods require separate training for each restoration task (e.g., denoising, deblurring, demosaicing) and problem condition (e.g., noise level of input images). This makes it time-consuming and difficult to encompass all tasks and conditions during training. In this paper, we propose a discriminative transfer learning method that incorporates formal proximal optimization and discriminative learning for general image restoration. The method requires a single-pass training and allows for reuse across various problems and conditions while achieving an efficiency comparable to previous discriminative approaches. Furthermore, after being trained, our model can be easily transferred to new likelihood terms to solve untrained tasks, or be combined with existing priors to further improve image restoration quality.
△ Less
Submitted 27 March, 2017;
originally announced March 2017.
-
EnhanceNet: Single Image Super-Resolution Through Automated Texture Synthesis
Authors:
Mehdi S. M. Sajjadi,
Bernhard Schölkopf,
Michael Hirsch
Abstract:
Single image super-resolution is the task of inferring a high-resolution image from a single low-resolution input. Traditionally, the performance of algorithms for this task is measured using pixel-wise reconstruction measures such as peak signal-to-noise ratio (PSNR) which have been shown to correlate poorly with the human perception of image quality. As a result, algorithms minimizing these metr…
▽ More
Single image super-resolution is the task of inferring a high-resolution image from a single low-resolution input. Traditionally, the performance of algorithms for this task is measured using pixel-wise reconstruction measures such as peak signal-to-noise ratio (PSNR) which have been shown to correlate poorly with the human perception of image quality. As a result, algorithms minimizing these metrics tend to produce over-smoothed images that lack high-frequency textures and do not look natural despite yielding high PSNR values.
We propose a novel application of automated texture synthesis in combination with a perceptual loss focusing on creating realistic textures rather than optimizing for a pixel-accurate reproduction of ground truth images during training. By using feed-forward fully convolutional neural networks in an adversarial training setting, we achieve a significant boost in image quality at high magnification ratios. Extensive experiments on a number of datasets show the effectiveness of our approach, yielding state-of-the-art results in both quantitative and qualitative benchmarks.
△ Less
Submitted 30 July, 2017; v1 submitted 23 December, 2016;
originally announced December 2016.
-
Depth Estimation Through a Generative Model of Light Field Synthesis
Authors:
Mehdi S. M. Sajjadi,
Rolf Köhler,
Bernhard Schölkopf,
Michael Hirsch
Abstract:
Light field photography captures rich structural information that may facilitate a number of traditional image processing and computer vision tasks. A crucial ingredient in such endeavors is accurate depth recovery. We present a novel framework that allows the recovery of a high quality continuous depth map from light field data. To this end we propose a generative model of a light field that is f…
▽ More
Light field photography captures rich structural information that may facilitate a number of traditional image processing and computer vision tasks. A crucial ingredient in such endeavors is accurate depth recovery. We present a novel framework that allows the recovery of a high quality continuous depth map from light field data. To this end we propose a generative model of a light field that is fully parametrized by its corresponding depth map. The model allows for the integration of powerful regularization techniques such as a non-local means prior, facilitating accurate depth map estimation.
△ Less
Submitted 6 September, 2016;
originally announced September 2016.
-
End-to-End Learning for Image Burst Deblurring
Authors:
Patrick Wieschollek,
Bernhard Schölkopf,
Hendrik P. A. Lensch,
Michael Hirsch
Abstract:
We present a neural network model approach for multi-frame blind deconvolution. The discriminative approach adopts and combines two recent techniques for image deblurring into a single neural network architecture. Our proposed hybrid-architecture combines the explicit prediction of a deconvolution filter and non-trivial averaging of Fourier coefficients in the frequency domain. In order to make fu…
▽ More
We present a neural network model approach for multi-frame blind deconvolution. The discriminative approach adopts and combines two recent techniques for image deblurring into a single neural network architecture. Our proposed hybrid-architecture combines the explicit prediction of a deconvolution filter and non-trivial averaging of Fourier coefficients in the frequency domain. In order to make full use of the information contained in all images in one burst, the proposed network embeds smaller networks, which explicitly allow the model to transfer information between images in early layers. Our system is trained end-to-end using standard backpropagation on a set of artificially generated training examples, enabling competitive performance in multi-frame blind deconvolution, both with respect to quality and runtime.
△ Less
Submitted 6 September, 2016; v1 submitted 15 July, 2016;
originally announced July 2016.
-
Objektorientierte Graphendarstellung von Simulink-Modellen zur einfachen Analyse und Transformation
Authors:
Carsten Kolassa,
David Dieckow,
Michael Hirsch,
Uwe Creutzburg,
Christian Siemers,
Bernhard Rumpe
Abstract:
In software and hardware development MATLAB and Simulink are used to model cyber physical systems for many years, , especially in automation technology and the automotive industry. Compliance with the required product quality and project efficiency is facilitated by analyzing and transforming Simulink models. The existing API, provided by MATLAB is only suitable for programmatic changing of Simuli…
▽ More
In software and hardware development MATLAB and Simulink are used to model cyber physical systems for many years, , especially in automation technology and the automotive industry. Compliance with the required product quality and project efficiency is facilitated by analyzing and transforming Simulink models. The existing API, provided by MATLAB is only suitable for programmatic changing of Simulink models. We show using our own tool which is used in industry, how such as a Simulink model can be edited more easily. For this purpose the model, is converted to an object-oriented class structure that provides convenient access and editing APIs and allows applying well-known algorithms and analyses from graph theory directly. It is also designed as a bi-directional tool, so it transforms a Simulink model into a graph representation and vice versa.
-----
In der Software- und Hardwareentwicklung wird seit Jahren verstärkt MATLAB und Simulink für die Modellierung von cyberphysikalischen Systemen, insbesondere in der Automatisierungstechnik und der Automobilindustrie eingesetzt. Die Einhaltung der notwendigen Produktqualität und Projekteffizienz wird durch Analysen und Transformationen auf Simulink-Modellen erleichtert. Die bestehende, von MATLAB bereitgestellte, API ist für die programmatische Veränderung von Simulink-Modellen nur bedingt geeignet. Wir zeigen deshalb anhand eines eigenen, im industriellen Einsatz befindlichen Werkzeugs, wie ein Simulink-Modell leichter bearbeitet werden kann. Dazu wird es in eine objektorientierte Klassenstruktur überführt, die einen komfortablen Zugang und Bearbeitungs-APIs bietet und es erlaubt bekannte Algorithmen und Analysen aus der Graphentheorie direkt anzuwenden. Das Werkzeug ist bidirektional entworfen, es transformiert also ein Simulink-Modell in eine Graphenrepresentation und umgekehrt.
△ Less
Submitted 21 August, 2014;
originally announced August 2014.
-
Learning to Deblur
Authors:
Christian J. Schuler,
Michael Hirsch,
Stefan Harmeling,
Bernhard Schölkopf
Abstract:
We describe a learning-based approach to blind image deconvolution. It uses a deep layered architecture, parts of which are borrowed from recent work on neural network learning, and parts of which incorporate computations that are specific to image deconvolution. The system is trained end-to-end on a set of artificially generated training examples, enabling competitive performance in blind deconvo…
▽ More
We describe a learning-based approach to blind image deconvolution. It uses a deep layered architecture, parts of which are borrowed from recent work on neural network learning, and parts of which incorporate computations that are specific to image deconvolution. The system is trained end-to-end on a set of artificially generated training examples, enabling competitive performance in blind deconvolution, both with respect to quality and runtime.
△ Less
Submitted 28 June, 2014;
originally announced June 2014.
