Showing 1–6 of 6 results for author: Prangemeier, T

Deep Video Codec Control for Vision Models

Authors: Christoph Reich, Biplob Debnath, Deep Patel, Tim Prangemeier, Daniel Cremers, Srimat Chakradhar

Abstract: Standardized lossy video coding is at the core of almost all real-world video processing pipelines. Rate control is used to enable standard codecs to adapt to different network bandwidth conditions or storage constraints. However, standard video codecs (e.g., H.264) and their rate control modules aim to minimize video distortion w.r.t. human quality assessment. We demonstrate empirically that stan… ▽ More Standardized lossy video coding is at the core of almost all real-world video processing pipelines. Rate control is used to enable standard codecs to adapt to different network bandwidth conditions or storage constraints. However, standard video codecs (e.g., H.264) and their rate control modules aim to minimize video distortion w.r.t. human quality assessment. We demonstrate empirically that standard-coded videos vastly deteriorate the performance of deep vision models. To overcome the deterioration of vision performance, this paper presents the first end-to-end learnable deep video codec control that considers both bandwidth constraints and downstream deep vision performance, while adhering to existing standardization. We demonstrate that our approach better preserves downstream deep vision performance than traditional standard video coding. △ Less

Submitted 16 April, 2024; v1 submitted 30 August, 2023; originally announced August 2023.

Comments: Accepted at CVPR 2024 Workshop on AI for Streaming (AIS)

OSS-Net: Memory Efficient High Resolution Semantic Segmentation of 3D Medical Data

Authors: Christoph Reich, Tim Prangemeier, Özdemir Cetin, Heinz Koeppl

Abstract: Convolutional neural networks (CNNs) are the current state-of-the-art meta-algorithm for volumetric segmentation of medical data, for example, to localize COVID-19 infected tissue on computer tomography scans or the detection of tumour volumes in magnetic resonance imaging. A key limitation of 3D CNNs on voxelised data is that the memory consumption grows cubically with the training data resolutio… ▽ More Convolutional neural networks (CNNs) are the current state-of-the-art meta-algorithm for volumetric segmentation of medical data, for example, to localize COVID-19 infected tissue on computer tomography scans or the detection of tumour volumes in magnetic resonance imaging. A key limitation of 3D CNNs on voxelised data is that the memory consumption grows cubically with the training data resolution. Occupancy networks (O-Nets) are an alternative for which the data is represented continuously in a function space and 3D shapes are learned as a continuous decision boundary. While O-Nets are significantly more memory efficient than 3D CNNs, they are limited to simple shapes, are relatively slow at inference, and have not yet been adapted for 3D semantic segmentation of medical data. Here, we propose Occupancy Networks for Semantic Segmentation (OSS-Nets) to accurately and memory-efficiently segment 3D medical data. We build upon the original O-Net with modifications for increased expressiveness leading to improved segmentation performance comparable to 3D CNNs, as well as modifications for faster inference. We leverage local observations to represent complex shapes and prior encoder predictions to expedite inference. We showcase OSS-Net's performance on 3D brain tumour and liver segmentation against a function space baseline (O-Net), a performance baseline (3D residual U-Net), and an efficiency baseline (2D residual U-Net). OSS-Net yields segmentation results similar to the performance baseline and superior to the function space and efficiency baselines. In terms of memory efficiency, OSS-Net consumes comparable amounts of memory as the function space baseline, somewhat more memory than the efficiency baseline and significantly less than the performance baseline. As such, OSS-Net enables memory-efficient and accurate 3D semantic segmentation that can scale to high resolutions. △ Less

Submitted 20 October, 2021; originally announced October 2021.

Comments: BMVC 2021 (accepted), https://github.com/ChristophReich1996/OSS-Net (code)

Multi-StyleGAN: Towards Image-Based Simulation of Time-Lapse Live-Cell Microscopy

Authors: Christoph Reich, Tim Prangemeier, Christian Wildner, Heinz Koeppl

Abstract: Time-lapse fluorescent microscopy (TLFM) combined with predictive mathematical modelling is a powerful tool to study the inherently dynamic processes of life on the single-cell level. Such experiments are costly, complex and labour intensive. A complimentary approach and a step towards in silico experimentation, is to synthesise the imagery itself. Here, we propose Multi-StyleGAN as a descriptive… ▽ More Time-lapse fluorescent microscopy (TLFM) combined with predictive mathematical modelling is a powerful tool to study the inherently dynamic processes of life on the single-cell level. Such experiments are costly, complex and labour intensive. A complimentary approach and a step towards in silico experimentation, is to synthesise the imagery itself. Here, we propose Multi-StyleGAN as a descriptive approach to simulate time-lapse fluorescence microscopy imagery of living cells, based on a past experiment. This novel generative adversarial network synthesises a multi-domain sequence of consecutive timesteps. We showcase Multi-StyleGAN on imagery of multiple live yeast cells in microstructured environments and train on a dataset recorded in our laboratory. The simulation captures underlying biophysical factors and time dependencies, such as cell morphology, growth, physical interactions, as well as the intensity of a fluorescent reporter protein. An immediate application is to generate additional training and validation data for feature extraction algorithms or to aid and expedite development of advanced experimental techniques such as online monitoring or control of cells. Code and dataset is available at https://git.rwth-aachen.de/bcs/projects/tp/multi-stylegan. △ Less

Submitted 24 September, 2021; v1 submitted 15 June, 2021; originally announced June 2021.

Comments: revised -- accepted to MICCAI 2021 (doi.org/10.1007/978-3-030-87237-3_46) (Tim Prangemeier and Christoph Reich --- both authors contributed equally)

Maximizing Information Gain for the Characterization of Biomolecular Circuits

Authors: Tim Prangemeier, Christian Wildner, Maleen Hanst, Heinz Koeppl

Abstract: Quantitatively predictive models of biomolecular circuits are important tools for the design of synthetic biology and molecular communication circuits. The information content of typical time-lapse single-cell data for the inference of kinetic parameters is not only limited by measurement uncertainty and intrinsic stochasticity, but also by the employed perturbations. Novel microfluidic devices en… ▽ More Quantitatively predictive models of biomolecular circuits are important tools for the design of synthetic biology and molecular communication circuits. The information content of typical time-lapse single-cell data for the inference of kinetic parameters is not only limited by measurement uncertainty and intrinsic stochasticity, but also by the employed perturbations. Novel microfluidic devices enable the synthesis of temporal chemical concentration profiles. The informativeness of a perturbation can be quantified based on mutual information. We propose an approximate method to perform optimal experimental design of such perturbation profiles. To estimate the mutual information we perform a multivariate log-normal approximation of the joint distribution over parameters and observations and scan the design space using Metropolis-Hastings sampling. The method is demonstrated by finding optimal perturbation sequences for synthetic case studies on a gene expression model with varying reporter characteristics. △ Less

Submitted 8 January, 2021; originally announced January 2021.

Comments: NanoCom 2018, accepted

Attention-Based Transformers for Instance Segmentation of Cells in Microstructures

Authors: Tim Prangemeier, Christoph Reich, Heinz Koeppl

Abstract: Detecting and segmenting object instances is a common task in biomedical applications. Examples range from detecting lesions on functional magnetic resonance images, to the detection of tumours in histopathological images and extracting quantitative single-cell information from microscopy imagery, where cell segmentation is a major bottleneck. Attention-based transformers are state-of-the-art in a… ▽ More Detecting and segmenting object instances is a common task in biomedical applications. Examples range from detecting lesions on functional magnetic resonance images, to the detection of tumours in histopathological images and extracting quantitative single-cell information from microscopy imagery, where cell segmentation is a major bottleneck. Attention-based transformers are state-of-the-art in a range of deep learning fields. They have recently been proposed for segmentation tasks where they are beginning to outperforming other methods. We present a novel attention-based cell detection transformer (Cell-DETR) for direct end-to-end instance segmentation. While the segmentation performance is on par with a state-of-the-art instance segmentation method, Cell-DETR is simpler and faster. We showcase the method's contribution in a the typical use case of segmenting yeast in microstructured environments, commonly employed in systems or synthetic biology. For the specific use case, the proposed method surpasses the state-of-the-art tools for semantic segmentation and additionally predicts the individual object instances. The fast and accurate instance segmentation performance increases the experimental information yield for a posteriori data processing and makes online monitoring of experiments and closed-loop optimal experimental design feasible. △ Less

Submitted 20 November, 2020; v1 submitted 19 November, 2020; originally announced November 2020.

Comments: IEEE BIBM 2020 (accepted)

Multiclass Yeast Segmentation in Microstructured Environments with Deep Learning

Authors: Tim Prangemeier, Christian Wildner, André O. Françani, Christoph Reich, Heinz Koeppl

Abstract: Cell segmentation is a major bottleneck in extracting quantitative single-cell information from microscopy data. The challenge is exasperated in the setting of microstructured environments. While deep learning approaches have proven useful for general cell segmentation tasks, existing segmentation tools for the yeast-microstructure setting rely on traditional machine learning approaches. Here we p… ▽ More Cell segmentation is a major bottleneck in extracting quantitative single-cell information from microscopy data. The challenge is exasperated in the setting of microstructured environments. While deep learning approaches have proven useful for general cell segmentation tasks, existing segmentation tools for the yeast-microstructure setting rely on traditional machine learning approaches. Here we present convolutional neural networks trained for multiclass segmenting of individual yeast cells and discerning these from cell-similar microstructures. We give an overview of the datasets recorded for training, validating and testing the networks, as well as a typical use-case. We showcase the method's contribution to segmenting yeast in microstructured environments with a typical synthetic biology application in mind. The models achieve robust segmentation results, outperforming the previous state-of-the-art in both accuracy and speed. The combination of fast and accurate segmentation is not only beneficial for a posteriori data processing, it also makes online monitoring of thousands of trapped cells or closed-loop optimal experimental design feasible from an image processing perspective. △ Less

Submitted 19 November, 2020; v1 submitted 16 November, 2020; originally announced November 2020.

Comments: IEEE CIBCB 2020 (accepted)