-
Towards Deep Active Learning in Avian Bioacoustics
Authors:
Lukas Rauch,
Denis Huseljic,
Moritz Wirth,
Jens Decke,
Bernhard Sick,
Christoph Scholz
Abstract:
Passive acoustic monitoring (PAM) in avian bioacoustics enables cost-effective and extensive data collection with minimal disruption to natural habitats. Despite advancements in computational avian bioacoustics, deep learning models continue to encounter challenges in adapting to diverse environments in practical PAM scenarios. This is primarily due to the scarcity of annotations, which requires l…
▽ More
Passive acoustic monitoring (PAM) in avian bioacoustics enables cost-effective and extensive data collection with minimal disruption to natural habitats. Despite advancements in computational avian bioacoustics, deep learning models continue to encounter challenges in adapting to diverse environments in practical PAM scenarios. This is primarily due to the scarcity of annotations, which requires labor-intensive efforts from human experts. Active learning (AL) reduces annotation cost and speed ups adaption to diverse scenarios by querying the most informative instances for labeling. This paper outlines a deep AL approach, introduces key challenges, and conducts a small-scale pilot study.
△ Less
Submitted 26 June, 2024;
originally announced June 2024.
-
BirdSet: A Dataset and Benchmark for Classification in Avian Bioacoustics
Authors:
Lukas Rauch,
Raphael Schwinger,
Moritz Wirth,
René Heinrich,
Denis Huseljic,
Jonas Lange,
Stefan Kahl,
Bernhard Sick,
Sven Tomforde,
Christoph Scholz
Abstract:
Deep learning (DL) models have emerged as a powerful tool in avian bioacoustics to assess environmental health. To maximize the potential of cost-effective and minimal-invasive passive acoustic monitoring (PAM), DL models must analyze bird vocalizations across a wide range of species and environmental conditions. However, data fragmentation challenges a comprehensive evaluation of generalization p…
▽ More
Deep learning (DL) models have emerged as a powerful tool in avian bioacoustics to assess environmental health. To maximize the potential of cost-effective and minimal-invasive passive acoustic monitoring (PAM), DL models must analyze bird vocalizations across a wide range of species and environmental conditions. However, data fragmentation challenges a comprehensive evaluation of generalization performance. Therefore, we introduce the BirdSet dataset, comprising approximately 520,000 global bird recordings for training and over 400 hours of PAM recordings for testing. Our benchmark offers baselines for several DL models to enhance comparability and consolidate research across studies, along with code implementations that include comprehensive training and evaluation protocols.
△ Less
Submitted 17 June, 2024; v1 submitted 15 March, 2024;
originally announced March 2024.
-
Active Bird2Vec: Towards End-to-End Bird Sound Monitoring with Transformers
Authors:
Lukas Rauch,
Raphael Schwinger,
Moritz Wirth,
Bernhard Sick,
Sven Tomforde,
Christoph Scholz
Abstract:
We propose a shift towards end-to-end learning in bird sound monitoring by combining self-supervised (SSL) and deep active learning (DAL). Leveraging transformer models, we aim to bypass traditional spectrogram conversions, enabling direct raw audio processing. ActiveBird2Vec is set to generate high-quality bird sound representations through SSL, potentially accelerating the assessment of environm…
▽ More
We propose a shift towards end-to-end learning in bird sound monitoring by combining self-supervised (SSL) and deep active learning (DAL). Leveraging transformer models, we aim to bypass traditional spectrogram conversions, enabling direct raw audio processing. ActiveBird2Vec is set to generate high-quality bird sound representations through SSL, potentially accelerating the assessment of environmental changes and decision-making processes for wind farms. Additionally, we seek to utilize the wide variety of bird vocalizations through DAL, reducing the reliance on extensively labeled datasets by human experts. We plan to curate a comprehensive set of tasks through Huggingface Datasets, enhancing future comparability and reproducibility of bioacoustic research. A comparative analysis between various transformer models will be conducted to evaluate their proficiency in bird sound recognition tasks. We aim to accelerate the progression of avian bioacoustic research and contribute to more effective conservation strategies.
△ Less
Submitted 21 November, 2023; v1 submitted 14 August, 2023;
originally announced August 2023.
-
Sensor Equivariance by LiDAR Projection Images
Authors:
Hannes Reichert,
Manuel Hetzel,
Steven Schreck,
Konrad Doll,
Bernhard Sick
Abstract:
In this work, we propose an extension of conventional image data by an additional channel in which the associated projection properties are encoded. This addresses the issue of sensor-dependent object representation in projection-based sensors, such as LiDAR, which can lead to distorted physical and geometric properties due to variations in sensor resolution and field of view. To that end, we prop…
▽ More
In this work, we propose an extension of conventional image data by an additional channel in which the associated projection properties are encoded. This addresses the issue of sensor-dependent object representation in projection-based sensors, such as LiDAR, which can lead to distorted physical and geometric properties due to variations in sensor resolution and field of view. To that end, we propose an architecture for processing this data in an instance segmentation framework. We focus specifically on LiDAR as a key sensor modality for machine vision tasks and highly automated driving (HAD). Through an experimental setup in a controlled synthetic environment, we identify a bias on sensor resolution and field of view and demonstrate that our proposed method can reduce said bias for the task of LiDAR instance segmentation. Furthermore, we define our method such that it can be applied to other projection-based sensors, such as cameras. To promote transparency, we make our code and dataset publicly available. This method shows the potential to improve performance and robustness in various machine vision tasks that utilize projection-based sensors.
△ Less
Submitted 29 April, 2023;
originally announced May 2023.
-
Synthetic Photovoltaic and Wind Power Forecasting Data
Authors:
Stephan Vogt,
Jens Schreiber,
Bernhard Sick
Abstract:
Photovoltaic and wind power forecasts in power systems with a high share of renewable energy are essential in several applications. These include stable grid operation, profitable power trading, and forward-looking system planning. However, there is a lack of publicly available datasets for research on machine learning based prediction methods. This paper provides an openly accessible time series…
▽ More
Photovoltaic and wind power forecasts in power systems with a high share of renewable energy are essential in several applications. These include stable grid operation, profitable power trading, and forward-looking system planning. However, there is a lack of publicly available datasets for research on machine learning based prediction methods. This paper provides an openly accessible time series dataset with realistic synthetic power data. Other publicly and non-publicly available datasets often lack precise geographic coordinates, timestamps, or static power plant information, e.g., to protect business secrets. On the opposite, this dataset provides these. The dataset comprises 120 photovoltaic and 273 wind power plants with distinct sides all over Germany from 500 days in hourly resolution. This large number of available sides allows forecasting experiments to include spatial correlations and run experiments in transfer and multi-task learning. It includes side-specific, power source-dependent, non-synthetic input features from the ICON-EU weather model. A simulation of virtual power plants with physical models and actual meteorological measurements provides realistic synthetic power measurement time series. These time series correspond to the power output of virtual power plants at the location of the respective weather measurements. Since the synthetic time series are based exclusively on weather measurements, possible errors in the weather forecast are comparable to those in actual power data. In addition to the data description, we evaluate the quality of weather-prediction-based power forecasts by comparing simplified physical models and a machine learning model. This experiment shows that forecasts errors on the synthetic power data are comparable to real-world historical power measurements.
△ Less
Submitted 1 April, 2022;
originally announced April 2022.
-
Towards Organic Distribution Systems -- The Vision of Self-Configuring, Self-Organising, Self-Healing, and Self-Optimising Power Distribution Management
Authors:
Inga Loeser,
Martin Braun,
Christian Gruhl,
Jan-Hendrik Menke,
Bernhard Sick,
Sven Tomforde
Abstract:
Due to the decarbonisation of energy use, the power system is expected to become the backbone of all energy sectors and thus the basic critical infrastructure. High penetration with distributed energy resources demands the coordination of a large number of prosumers, partly controlled by home energy management systems (HEMS), to be designed in such a way that the power system's operational limits…
▽ More
Due to the decarbonisation of energy use, the power system is expected to become the backbone of all energy sectors and thus the basic critical infrastructure. High penetration with distributed energy resources demands the coordination of a large number of prosumers, partly controlled by home energy management systems (HEMS), to be designed in such a way that the power system's operational limits are not violated. On the grid level, distribution management systems (DMS) try to keep the power system in the normal operational state. On the prosumer level, distributed HEMS optimise the internal power flows by using batteries, photovoltaic generators, or flexible loads optimally. The vision of the ODiS (Organic Distribution System) initiative is to develop an architecture to operate a distribution grid reliably, with high resiliency, and fully autonomously by develo** "organic" HEMS and DMS which possess multiple self-* capabilities. Thus, ODiS seeks answers to the following question: How can we create the most appropriate models, techniques, and algorithms to develop novel kinds of self-configuring, self-organising, self-healing, and self-optimising DMS that are integrally coupled with the distributed HEMS? In this article, the vision of ODiS is presented in detail based on a thorough review of the state of the art.
△ Less
Submitted 14 December, 2021;
originally announced December 2021.
-
An Application-Driven Conceptualization of Corner Cases for Perception in Highly Automated Driving
Authors:
Florian Heidecker,
Jasmin Breitenstein,
Kevin Rösch,
Jonas Löhdefink,
Maarten Bieshaar,
Christoph Stiller,
Tim Fingscheidt,
Bernhard Sick
Abstract:
Systems and functions that rely on machine learning (ML) are the basis of highly automated driving. An essential task of such ML models is to reliably detect and interpret unusual, new, and potentially dangerous situations. The detection of those situations, which we refer to as corner cases, is highly relevant for successfully develo**, applying, and validating automotive perception functions i…
▽ More
Systems and functions that rely on machine learning (ML) are the basis of highly automated driving. An essential task of such ML models is to reliably detect and interpret unusual, new, and potentially dangerous situations. The detection of those situations, which we refer to as corner cases, is highly relevant for successfully develo**, applying, and validating automotive perception functions in future vehicles where multiple sensor modalities will be used. A complication for the development of corner case detectors is the lack of consistent definitions, terms, and corner case descriptions, especially when taking into account various automotive sensors. In this work, we provide an application-driven view of corner cases in highly automated driving. To achieve this goal, we first consider existing definitions from the general outlier, novelty, anomaly, and out-of-distribution detection to show relations and differences to corner cases. Moreover, we extend an existing camera-focused systematization of corner cases by adding RADAR (radio detection and ranging) and LiDAR (light detection and ranging) sensors. For this, we describe an exemplary toolchain for data acquisition and processing, highlighting the interfaces of the corner case detection. We also define a novel level of corner cases, the method layer corner cases, which appear due to uncertainty inherent in the methodology or the data distribution.
△ Less
Submitted 5 March, 2021;
originally announced March 2021.
-
Integrating uncertainty in deep neural networks for MRI based stroke analysis
Authors:
Lisa Herzog,
Elvis Murina,
Oliver Dürr,
Susanne Wegener,
Beate Sick
Abstract:
At present, the majority of the proposed Deep Learning (DL) methods provide point predictions without quantifying the models uncertainty. However, a quantification of the reliability of automated image analysis is essential, in particular in medicine when physicians rely on the results for making critical treatment decisions. In this work, we provide an entire framework to diagnose ischemic stroke…
▽ More
At present, the majority of the proposed Deep Learning (DL) methods provide point predictions without quantifying the models uncertainty. However, a quantification of the reliability of automated image analysis is essential, in particular in medicine when physicians rely on the results for making critical treatment decisions. In this work, we provide an entire framework to diagnose ischemic stroke patients incorporating Bayesian uncertainty into the analysis procedure. We present a Bayesian Convolutional Neural Network (CNN) yielding a probability for a stroke lesion on 2D Magnetic Resonance (MR) images with corresponding uncertainty information about the reliability of the prediction. For patient-level diagnoses, different aggregation methods are proposed and evaluated, which combine the single image-level predictions. Those methods take advantage of the uncertainty in image predictions and report model uncertainty at the patient-level. In a cohort of 511 patients, our Bayesian CNN achieved an accuracy of 95.33% at the image-level representing a significant improvement of 2% over a non-Bayesian counterpart. The best patient aggregation method yielded 95.89% of accuracy. Integrating uncertainty information about image predictions in aggregation models resulted in higher uncertainty measures to false patient classifications, which enabled to filter critical patient diagnoses that are supposed to be closer examined by a medical doctor. We therefore recommend using Bayesian approaches not only for improved image-level prediction and uncertainty estimation but also for the detection of uncertain aggregations at the patient-level.
△ Less
Submitted 13 August, 2020;
originally announced August 2020.
-
Off-the-shelf sensor vs. experimental radar -- How much resolution is necessary in automotive radar classification?
Authors:
Nicolas Scheiner,
Ole Schumann,
Florian Kraus,
Nils Appenrodt,
Jürgen Dickmann,
Bernhard Sick
Abstract:
Radar-based road user detection is an important topic in the context of autonomous driving applications. The resolution of conventional automotive radar sensors results in a sparse data representation which is tough to refine during subsequent signal processing. On the other hand, a new sensor generation is waiting in the wings for its application in this challenging field. In this article, two se…
▽ More
Radar-based road user detection is an important topic in the context of autonomous driving applications. The resolution of conventional automotive radar sensors results in a sparse data representation which is tough to refine during subsequent signal processing. On the other hand, a new sensor generation is waiting in the wings for its application in this challenging field. In this article, two sensors of different radar generations are evaluated against each other. The evaluation criterion is the performance on moving road user object detection and classification tasks. To this end, two data sets originating from an off-the-shelf radar and a high resolution next generation radar are compared. Special attention is given on how the two data sets are assembled in order to make them comparable. The utilized object detector consists of a clustering algorithm, a feature extraction module, and a recurrent neural network ensemble for classification. For the assessment, all components are evaluated both individually and, for the first time, as a whole. This allows for indicating where overall performance improvements have their origin in the pipeline. Furthermore, the generalization capabilities of both data sets are evaluated and important comparison metrics for automotive radar object detection are discussed. Results show clear benefits of the next generation radar. Interestingly, those benefits do not actually occur due to better performance at the classification stage, but rather because of the vast improvements at the clustering stage.
△ Less
Submitted 9 June, 2020;
originally announced June 2020.
-
Seeing Around Street Corners: Non-Line-of-Sight Detection and Tracking In-the-Wild Using Doppler Radar
Authors:
Nicolas Scheiner,
Florian Kraus,
Fangyin Wei,
Buu Phan,
Fahim Mannan,
Nils Appenrodt,
Werner Ritter,
Jürgen Dickmann,
Klaus Dietmayer,
Bernhard Sick,
Felix Heide
Abstract:
Conventional sensor systems record information about directly visible objects, whereas occluded scene components are considered lost in the measurement process. Non-line-of-sight (NLOS) methods try to recover such hidden objects from their indirect reflections - faint signal components, traditionally treated as measurement noise. Existing NLOS approaches struggle to record these low-signal compone…
▽ More
Conventional sensor systems record information about directly visible objects, whereas occluded scene components are considered lost in the measurement process. Non-line-of-sight (NLOS) methods try to recover such hidden objects from their indirect reflections - faint signal components, traditionally treated as measurement noise. Existing NLOS approaches struggle to record these low-signal components outside the lab, and do not scale to large-scale outdoor scenes and high-speed motion, typical in automotive scenarios. In particular, optical NLOS capture is fundamentally limited by the quartic intensity falloff of diffuse indirect reflections. In this work, we depart from visible-wavelength approaches and demonstrate detection, classification, and tracking of hidden objects in large-scale dynamic environments using Doppler radars that can be manufactured at low-cost in series production. To untangle noisy indirect and direct reflections, we learn from temporal sequences of Doppler velocity and position measurements, which we fuse in a joint NLOS detection and tracking network over time. We validate the approach on in-the-wild automotive scenes, including sequences of parked cars or house facades as relay surfaces, and demonstrate low-cost, real-time NLOS in dynamic automotive environments.
△ Less
Submitted 31 March, 2020; v1 submitted 13 December, 2019;
originally announced December 2019.
-
A Multi-Stage Clustering Framework for Automotive Radar Data
Authors:
Nicolas Scheiner,
Nils Appenrodt,
Jürgen Dickmann,
Bernhard Sick
Abstract:
Radar sensors provide a unique method for executing environmental perception tasks towards autonomous driving. Especially their capability to perform well in adverse weather conditions often makes them superior to other sensors such as cameras or lidar. Nevertheless, the high sparsity and low dimensionality of the commonly used detection data level is a major challenge for subsequent signal proces…
▽ More
Radar sensors provide a unique method for executing environmental perception tasks towards autonomous driving. Especially their capability to perform well in adverse weather conditions often makes them superior to other sensors such as cameras or lidar. Nevertheless, the high sparsity and low dimensionality of the commonly used detection data level is a major challenge for subsequent signal processing. Therefore, the data points are often merged in order to form larger entities from which more information can be gathered. The merging process is often implemented in form of a clustering algorithm. This article describes a novel approach for first filtering out static background data before applying a twostage clustering approach. The two-stage clustering follows the same paradigm as the idea for data association itself: First, clustering what is ought to belong together in a low dimensional parameter space, then, extracting additional features from the newly created clusters in order to perform a final clustering step. Parameters are optimized for filtering and both clustering steps. All techniques are assessed both individually and as a whole in order to demonstrate their effectiveness. Final results indicate clear benefits of the first two methods and also the cluster merging process under specific circumstances.
△ Less
Submitted 8 July, 2019;
originally announced July 2019.
-
Generative Adversarial Networks for Operational Scenario Planning of Renewable Energy Farms: A Study on Wind and Photovoltaic
Authors:
Jens Schreiber,
Maik Jessulat,
Bernhard Sick
Abstract:
For the integration of renewable energy sources, power grid operators need realistic information about the effects of energy production and consumption to assess grid stability.
Recently, research in scenario planning benefits from utilizing generative adversarial networks (GANs) as generative models for operational scenario planning.
In these scenarios, operators examine temporal as well as s…
▽ More
For the integration of renewable energy sources, power grid operators need realistic information about the effects of energy production and consumption to assess grid stability.
Recently, research in scenario planning benefits from utilizing generative adversarial networks (GANs) as generative models for operational scenario planning.
In these scenarios, operators examine temporal as well as spatial influences of different energy sources on the grid.
The analysis of how renewable energy resources affect the grid enables the operators to evaluate the stability and to identify potential weak points such as a limiting transformer.
However, due to their novelty, there are limited studies on how well GANs model the underlying power distribution.
This analysis is essential because, e.g., especially extreme situations with low or high power generation are required to evaluate grid stability.
We conduct a comparative study of the Wasserstein distance, binary-cross-entropy loss, and a Gaussian copula as the baseline applied on two wind and two solar datasets with limited data compared to previous studies.
Both GANs achieve good results considering the limited amount of data, but the Wasserstein GAN is superior in modeling temporal and spatial relations, and the power distribution.
Besides evaluating the generated power distribution over all farms, it is essential to assess terrain specific distributions for wind scenarios.
These terrain specific power distributions affect the grid by their differences in their generating power magnitude.
Therefore, in a second study, we show that even when simultaneously learning distributions from wind parks with terrain specific patterns, GANs are capable of modeling these individualities also when faced with limited data.
△ Less
Submitted 3 June, 2019;
originally announced June 2019.
-
Automated Ground Truth Estimation For Automotive Radar Tracking Applications With Portable GNSS And IMU Devices
Authors:
Nicolas Scheiner,
Stefan Haag,
Nils Appenrodt,
Bharanidhar Duraisamy,
Jürgen Dickmann,
Martin Fritzsche,
Bernhard Sick
Abstract:
Baseline generation for tracking applications is a difficult task when working with real world radar data. Data sparsity usually only allows an indirect way of estimating the original tracks as most objects' centers are not represented in the data. This article proposes an automated way of acquiring reference trajectories by using a highly accurate hand-held global navigation satellite system (GNS…
▽ More
Baseline generation for tracking applications is a difficult task when working with real world radar data. Data sparsity usually only allows an indirect way of estimating the original tracks as most objects' centers are not represented in the data. This article proposes an automated way of acquiring reference trajectories by using a highly accurate hand-held global navigation satellite system (GNSS). An embedded inertial measurement unit (IMU) is used for estimating orientation and motion behavior. This article contains two major contributions. A method for associating radar data to vulnerable road user (VRU) tracks is described. It is evaluated how accurate the system performs under different GNSS reception conditions and how carrying a reference system alters radar measurements. Second, the system is used to track pedestrians and cyclists over many measurement cycles in order to generate object centered occupancy grid maps. The reference system allows to much more precisely generate real world radar data distributions of VRUs than compared to conventional methods. Hereby, an important step towards radar-based VRU tracking is accomplished.
△ Less
Submitted 3 June, 2019; v1 submitted 28 May, 2019;
originally announced May 2019.
-
Radar-based Road User Classification and Novelty Detection with Recurrent Neural Network Ensembles
Authors:
Nicolas Scheiner,
Nils Appenrodt,
Jürgen Dickmann,
Bernhard Sick
Abstract:
Radar-based road user classification is an important yet still challenging task towards autonomous driving applications. The resolution of conventional automotive radar sensors results in a sparse data representation which is tough to recover by subsequent signal processing. In this article, classifier ensembles originating from a one-vs-one binarization paradigm are enriched by one-vs-all correct…
▽ More
Radar-based road user classification is an important yet still challenging task towards autonomous driving applications. The resolution of conventional automotive radar sensors results in a sparse data representation which is tough to recover by subsequent signal processing. In this article, classifier ensembles originating from a one-vs-one binarization paradigm are enriched by one-vs-all correction classifiers. They are utilized to efficiently classify individual traffic participants and also identify hidden object classes which have not been presented to the classifiers during training. For each classifier of the ensemble an individual feature set is determined from a total set of 98 features. Thereby, the overall classification performance can be improved when compared to previous methods and, additionally, novel classes can be identified much more accurately. Furthermore, the proposed structure allows to give new insights in the importance of features for the recognition of individual classes which is crucial for the development of new algorithms and sensor requirements.
△ Less
Submitted 28 May, 2019;
originally announced May 2019.
-
Automated Ground Truth Estimation of Vulnerable Road Users in Automotive Radar Data Using GNSS
Authors:
Nicolas Scheiner,
Nils Appenrodt,
Jürgen Dickmann,
Bernhard Sick
Abstract:
Annotating automotive radar data is a difficult task. This article presents an automated way of acquiring data labels which uses a highly accurate and portable global navigation satellite system (GNSS). The proposed system is discussed besides a revision of other label acquisitions techniques and a problem description of manual data annotation. The article concludes with a systematic comparison of…
▽ More
Annotating automotive radar data is a difficult task. This article presents an automated way of acquiring data labels which uses a highly accurate and portable global navigation satellite system (GNSS). The proposed system is discussed besides a revision of other label acquisitions techniques and a problem description of manual data annotation. The article concludes with a systematic comparison of conventional hand labeling and automatic data acquisition. The results show clear advantages of the proposed method without a relevant loss in labeling accuracy. Minor changes can be observed in the measured radar data, but the so introduced bias of the GNSS reference is clearly outweighed by the indisputable time savings. Beside data annotation, the proposed system can also provide a ground truth for validating object tracking or other automated driving system applications.
△ Less
Submitted 27 May, 2019;
originally announced May 2019.
