-
Depth Supervised Neural Surface Reconstruction from Airborne Imagery
Authors:
Vincent Hackstein,
Paul Fauth-Mayer,
Matthias Rothermel,
Norbert Haala
Abstract:
While originally developed for novel view synthesis, Neural Radiance Fields (NeRFs) have recently emerged as an alternative to multi-view stereo (MVS). Triggered by a manifold of research activities, promising results have been gained especially for texture-less, transparent, and reflecting surfaces, while such scenarios remain challenging for traditional MVS-based approaches. However, most of the…
▽ More
While originally developed for novel view synthesis, Neural Radiance Fields (NeRFs) have recently emerged as an alternative to multi-view stereo (MVS). Triggered by a manifold of research activities, promising results have been gained especially for texture-less, transparent, and reflecting surfaces, while such scenarios remain challenging for traditional MVS-based approaches. However, most of these investigations focus on close-range scenarios, with studies for airborne scenarios still missing. For this task, NeRFs face potential difficulties at areas of low image redundancy and weak data evidence, as often found in street canyons, facades or building shadows. Furthermore, training such networks is computationally expensive. Thus, the aim of our work is twofold: First, we investigate the applicability of NeRFs for aerial image blocks representing different characteristics like nadir-only, oblique and high-resolution imagery. Second, during these investigations we demonstrate the benefit of integrating depth priors from tie-point measures, which are provided during presupposed Bundle Block Adjustment. Our work is based on the state-of-the-art framework VolSDF, which models 3D scenes by signed distance functions (SDFs), since this is more applicable for surface reconstruction compared to the standard volumetric representation in vanilla NeRFs. For evaluation, the NeRF-based reconstructions are compared to results of a publicly available benchmark dataset for airborne images.
△ Less
Submitted 25 April, 2024;
originally announced April 2024.
-
Interpretable Concept Bottlenecks to Align Reinforcement Learning Agents
Authors:
Quentin Delfosse,
Sebastian Sztwiertnia,
Mark Rothermel,
Wolfgang Stammer,
Kristian Kersting
Abstract:
Goal misalignment, reward sparsity and difficult credit assignment are only a few of the many issues that make it difficult for deep reinforcement learning (RL) agents to learn optimal policies. Unfortunately, the black-box nature of deep neural networks impedes the inclusion of domain experts for inspecting the model and revising suboptimal policies. To this end, we introduce *Successive Concept…
▽ More
Goal misalignment, reward sparsity and difficult credit assignment are only a few of the many issues that make it difficult for deep reinforcement learning (RL) agents to learn optimal policies. Unfortunately, the black-box nature of deep neural networks impedes the inclusion of domain experts for inspecting the model and revising suboptimal policies. To this end, we introduce *Successive Concept Bottleneck Agents* (SCoBots), that integrate consecutive concept bottleneck (CB) layers. In contrast to current CB models, SCoBots do not just represent concepts as properties of individual objects, but also as relations between objects which is crucial for many RL tasks. Our experimental results provide evidence of SCoBots' competitive performances, but also of their potential for domain experts to understand and regularize their behavior. Among other things, SCoBots enabled us to identify a previously unknown misalignment problem in the iconic video game, Pong, and resolve it. Overall, SCoBots thus result in more human-aligned RL agents. Our code is available at https://github.com/k4ntz/SCoBots .
△ Less
Submitted 24 May, 2024; v1 submitted 11 January, 2024;
originally announced January 2024.
-
Cosmological Parameter Forecasts for a CMB-HD Survey
Authors:
Amanda MacInnis,
Neelima Sehgal,
Miriam Rothermel
Abstract:
We present forecasts on cosmological parameters for a CMB-HD survey. For a $Λ$CDM + $N_{eff}$ + $\sum m_ν$ model, we find $σ(n_s) = 0.0013$ and $σ(N_{eff}) = 0.014$ using CMB and CMB lensing multipoles in the range of $\ell \in [30, 20000]$, after adding anticipated residual foregrounds, delensing the acoustic peaks, and adding DESI BAO data. This is about a factor of two improvement in ability to…
▽ More
We present forecasts on cosmological parameters for a CMB-HD survey. For a $Λ$CDM + $N_{eff}$ + $\sum m_ν$ model, we find $σ(n_s) = 0.0013$ and $σ(N_{eff}) = 0.014$ using CMB and CMB lensing multipoles in the range of $\ell \in [30, 20000]$, after adding anticipated residual foregrounds, delensing the acoustic peaks, and adding DESI BAO data. This is about a factor of two improvement in ability to probe inflation via $n_s$ compared to precursor CMB surveys. The $N_{eff}$ constraint can rule out light thermal particles back to the end of inflation with 95% CL; for example, it can rule out the QCD axion in a model-independent way assuming the Universe's reheating temperature was high enough that the axion thermalized. We find that delensing the acoustic peaks and adding DESI BAO tightens parameter constraints. We also find that baryonic effects can bias parameters if not marginalized over, and that uncertainties in baryonic effects can increase parameter error bars; however, the latter can be mitigated by including information about baryonic effects from kinetic and thermal Sunyaev-Zel'dovich measurements by CMB-HD. The CMB-HD likelihood and Fisher estimation codes used here are publicly available; the likelihood is integrated with Cobaya to facilitate parameter forecasting.
△ Less
Submitted 5 February, 2024; v1 submitted 6 September, 2023;
originally announced September 2023.
-
Integrated monitoring of ice in selected Swiss lakes. Final project report
Authors:
Manu Tom,
Melanie Suetterlin,
Damien Bouffard,
Mathias Rothermel,
Stefan Wunderle,
Emmanuel Baltsavias
Abstract:
Various lake observables, including lake ice, are related to climate and climate change and provide a good opportunity for long-term monitoring. Lakes (and as part of them lake ice) is therefore considered an Essential Climate Variable (ECV) of the Global Climate Observing System (GCOS). Following the need for an integrated multi-temporal monitoring of lake ice in Switzerland, MeteoSwiss in the fr…
▽ More
Various lake observables, including lake ice, are related to climate and climate change and provide a good opportunity for long-term monitoring. Lakes (and as part of them lake ice) is therefore considered an Essential Climate Variable (ECV) of the Global Climate Observing System (GCOS). Following the need for an integrated multi-temporal monitoring of lake ice in Switzerland, MeteoSwiss in the framework of GCOS Switzerland supported this 2-year project to explore not only the use of satellite images but also the possibilities of Webcams and in-situ measurements. The aim of this project is to monitor some target lakes and detect the extent of ice and especially the ice-on/off dates, with focus on the integration of various input data and processing methods. The target lakes are: St. Moritz, Silvaplana, Sils, Sihl, Greifen and Aegeri, whereby only the first four were mainly frozen during the observation period and thus processed. The observation period was mainly the winter 2016-17. During the project, various approaches were developed, implemented, tested and compared. Firstly, low spatial resolution (250 - 1000 m) but high temporal resolution (1 day) satellite images from the optical sensors MODIS and VIIRS were used. Secondly, and as a pilot project, the use of existing public Webcams was investigated for (a) validation of results from satellite data, and (b) independent estimation of lake ice, especially for small lakes like St. Moritz, that could not be possibly monitored in the satellite images. Thirdly, in-situ measurements were made in order to characterize the development of the temperature profiles and partly pressure before freezing and under the ice-cover until melting. This report presents the results of the project work.
△ Less
Submitted 2 August, 2020;
originally announced August 2020.
-
Photometric Multi-View Mesh Refinement for High-Resolution Satellite Images
Authors:
Mathias Rothermel,
Ke Gong,
Dieter Fritsch,
Konrad Schindler,
Norbert Haala
Abstract:
Modern high-resolution satellite sensors collect optical imagery with ground sampling distances (GSDs) of 30-50cm, which has sparked a renewed interest in photogrammetric 3D surface reconstruction from satellite data. State-of-the-art reconstruction methods typically generate 2.5D elevation data. Here, we present an approach to recover full 3D surface meshes from multi-view satellite imagery. The…
▽ More
Modern high-resolution satellite sensors collect optical imagery with ground sampling distances (GSDs) of 30-50cm, which has sparked a renewed interest in photogrammetric 3D surface reconstruction from satellite data. State-of-the-art reconstruction methods typically generate 2.5D elevation data. Here, we present an approach to recover full 3D surface meshes from multi-view satellite imagery. The proposed method takes as input a coarse initial mesh and refines it by iteratively updating all vertex positions to maximize the photo-consistency between images. Photo-consistency is measured in image space, by transferring texture from one image to another via the surface. We derive the equations to propagate changes in texture similarity through the rational function model (RFM), often also referred to as rational polynomial coefficient (RPC) model. Furthermore, we devise a hierarchical scheme to optimize the surface with gradient descent. In experiments with two different datasets, we show that the refinement improves the initial digital elevation models (DEMs) generated with conventional dense image matching. Moreover, we demonstrate that our method is able to reconstruct true 3D geometry, such as facade structures, if off-nadir views are available.
△ Less
Submitted 12 May, 2020; v1 submitted 10 May, 2020;
originally announced May 2020.
-
Lake Ice Monitoring with Webcams and Crowd-Sourced Images
Authors:
Rajanie Prabha,
Manu Tom,
Mathias Rothermel,
Emmanuel Baltsavias,
Laura Leal-Taixe,
Konrad Schindler
Abstract:
Lake ice is a strong climate indicator and has been recognised as part of the Essential Climate Variables (ECV) by the Global Climate Observing System (GCOS). The dynamics of freezing and thawing, and possible shifts of freezing patterns over time, can help in understanding the local and global climate systems. One way to acquire the spatio-temporal information about lake ice formation, independen…
▽ More
Lake ice is a strong climate indicator and has been recognised as part of the Essential Climate Variables (ECV) by the Global Climate Observing System (GCOS). The dynamics of freezing and thawing, and possible shifts of freezing patterns over time, can help in understanding the local and global climate systems. One way to acquire the spatio-temporal information about lake ice formation, independent of clouds, is to analyse webcam images. This paper intends to move towards a universal model for monitoring lake ice with freely available webcam data. We demonstrate good performance, including the ability to generalise across different winters and different lakes, with a state-of-the-art Convolutional Neural Network (CNN) model for semantic image segmentation, Deeplab v3+. Moreover, we design a variant of that model, termed Deep-U-Lab, which predicts sharper, more correct segmentation boundaries. We have tested the model's ability to generalise with data from multiple camera views and two different winters. On average, it achieves intersection-over-union (IoU) values of ~71% across different cameras and ~69% across different winters, greatly outperforming prior work. Going even further, we show that the model even achieves 60% IoU on arbitrary images scraped from photo-sharing web sites. As part of the work, we introduce a new benchmark dataset of webcam images, Photi-LakeIce, from multiple cameras and two different winters, along with pixel-wise ground truth annotations.
△ Less
Submitted 7 May, 2020; v1 submitted 18 February, 2020;
originally announced February 2020.
-
Semantically Informed Multiview Surface Refinement
Authors:
Maros Blaha,
Mathias Rothermel,
Martin R. Oswald,
Torsten Sattler,
Audrey Richard,
Jan D. Wegner,
Marc Pollefeys,
Konrad Schindler
Abstract:
We present a method to jointly refine the geometry and semantic segmentation of 3D surface meshes. Our method alternates between updating the shape and the semantic labels. In the geometry refinement step, the mesh is deformed with variational energy minimization, such that it simultaneously maximizes photo-consistency and the compatibility of the semantic segmentations across a set of calibrated…
▽ More
We present a method to jointly refine the geometry and semantic segmentation of 3D surface meshes. Our method alternates between updating the shape and the semantic labels. In the geometry refinement step, the mesh is deformed with variational energy minimization, such that it simultaneously maximizes photo-consistency and the compatibility of the semantic segmentations across a set of calibrated images. Label-specific shape priors account for interactions between the geometry and the semantic labels in 3D. In the semantic segmentation step, the labels on the mesh are updated with MRF inference, such that they are compatible with the semantic segmentations in the input images. Also, this step includes prior assumptions about the surface shape of different semantic classes. The priors induce a tight coupling, where semantic information influences the shape update and vice versa. Specifically, we introduce priors that favor (i) adaptive smoothing, depending on the class label; (ii) straightness of class boundaries; and (iii) semantic labels that are consistent with the surface orientation. The novel mesh-based reconstruction is evaluated in a series of experiments with real and synthetic data. We compare both to state-of-the-art, voxel-based semantic 3D reconstruction, and to purely geometric mesh refinement, and demonstrate that the proposed scheme yields improved 3D geometry as well as an improved semantic segmentation.
△ Less
Submitted 26 June, 2017;
originally announced June 2017.
-
Comment on: "Revealing common artifacts due to ferromagnetic inclusions in highly oriented pyrolytic graphite", by M. Sepioni, R.R. Nair, I.-Ling Tsai, A.K. Geim and I.V. Grigorieva, EPL 97 (2012) 47001
Authors:
D. Spemann,
M. Rothermel,
P. Esquinazi,
M. Ramos,
Y. Kopelevich,
H. Ohldag
Abstract:
This comment addresses several issues in the paper by Sepioni et al., where it is stated that the ferromagnetism in pristine highly oriented pyrolytic graphite (HOPG) reported by several groups in the previous years is most likely due to impurity contamination. In this comment, clear arguments are given why this statement is not justified. Furthermore, it is pointed out, that there are already mea…
▽ More
This comment addresses several issues in the paper by Sepioni et al., where it is stated that the ferromagnetism in pristine highly oriented pyrolytic graphite (HOPG) reported by several groups in the previous years is most likely due to impurity contamination. In this comment, clear arguments are given why this statement is not justified. Furthermore, it is pointed out, that there are already measurements using element-sensitive microscopic techniques, e.g. X-ray Magnetic Circular Dichroism (XMCD) that directly proved the intrinsic origin of the ferromagnetism in graphite, also in pristine HOPG.
△ Less
Submitted 13 April, 2012;
originally announced April 2012.
-
The role of hydrogen in room-temperature ferromagnetism at graphite surfaces
Authors:
H. Ohldag,
P. Esquinazi,
E. Arenholz,
D. Spemann,
M. Rothermel,
A. Setzer,
T. Butz
Abstract:
We present a x-ray dichroism study of graphite surfaces that addresses the origin and magnitude of ferromagnetism in metal-free carbon. We find that, in addition to carbon $π$ states, also hydrogen-mediated electronic states exhibit a net spin polarization with significant magnetic remanence at room temperature. The observed magnetism is restricted to the top $\approx$10 nm of the irradiated samp…
▽ More
We present a x-ray dichroism study of graphite surfaces that addresses the origin and magnitude of ferromagnetism in metal-free carbon. We find that, in addition to carbon $π$ states, also hydrogen-mediated electronic states exhibit a net spin polarization with significant magnetic remanence at room temperature. The observed magnetism is restricted to the top $\approx$10 nm of the irradiated sample where the actual magnetization reaches $ \simeq 15$ emu/g at room temperature. We prove that the ferromagnetism found in metal-free untreated graphite is intrinsic and has a similar origin as the one found in proton bombarded graphite.
△ Less
Submitted 17 May, 2010; v1 submitted 26 May, 2009;
originally announced May 2009.
-
Magnetic order in Graphite: Experimental evidence, intrinsic and extrinsic difficulties
Authors:
P. Esquinazi,
J. Barzola-Quiquia,
D. Spemann,
M. Rothermel,
H. Ohldag,
N. García,
A. Setzer,
T. Butz
Abstract:
We discuss recently obtained data using different experimental methods including magnetoresistance measurements that indicate the existence of metal-free high-temperature magnetic order in graphite. Intrinsic as well as extrinsic difficulties to trigger magnetic order by irradiation of graphite are discussed in view of recently published theoretical work.
We discuss recently obtained data using different experimental methods including magnetoresistance measurements that indicate the existence of metal-free high-temperature magnetic order in graphite. Intrinsic as well as extrinsic difficulties to trigger magnetic order by irradiation of graphite are discussed in view of recently published theoretical work.
△ Less
Submitted 10 February, 2009;
originally announced February 2009.
-
A comparison of the magnetic properties of Proton- and Iron-implanted graphite
Authors:
J. Barzola-Quiquia,
R. Höhne,
M. Rothermel,
A. Setzer,
P. Esquinazi,
V. Heera
Abstract:
In this work we have investigated the changes of the magnetic properties of highly oriented pyrolytic graphite samples after irradiation either with $\sim 3 \times 10^{14}$ protons or $ 3.5 \times 10^{13} ... 3.5 \times 10^{14}$ iron ions with energies in the MeV range. Our results show that iron and proton irradiations can produce similar paramagnetic contributions depending on the implantation…
▽ More
In this work we have investigated the changes of the magnetic properties of highly oriented pyrolytic graphite samples after irradiation either with $\sim 3 \times 10^{14}$ protons or $ 3.5 \times 10^{13} ... 3.5 \times 10^{14}$ iron ions with energies in the MeV range. Our results show that iron and proton irradiations can produce similar paramagnetic contributions depending on the implantation temperature. However, only protons induce a ferromagnetic effect.
△ Less
Submitted 6 January, 2008;
originally announced January 2008.
-
Experimental Evidence for Two-Dimensional Magnetic Order in Proton Bombarded Graphite
Authors:
J. Barzola-Quiquia,
P. Esquinazi,
M. Rothermel,
D. Spemann,
T. Butz,
N. García
Abstract:
We have prepared magnetic graphite samples bombarded by protons at low temperatures and low fluences to attenuate the large thermal annealing produced during irradiation. An overall optimization of sample handling allowed us to find Curie temperatures $ T_c \gtrsim 350$ K at the used fluences. The magnetization versus temperature shows unequivocally a linear dependence, which can be interpreted…
▽ More
We have prepared magnetic graphite samples bombarded by protons at low temperatures and low fluences to attenuate the large thermal annealing produced during irradiation. An overall optimization of sample handling allowed us to find Curie temperatures $ T_c \gtrsim 350$ K at the used fluences. The magnetization versus temperature shows unequivocally a linear dependence, which can be interpreted as due to excitations of spin waves in a two dimensional Heisenberg model with a weak uniaxial anisotropy.
△ Less
Submitted 16 June, 2007;
originally announced June 2007.
-
Proton-induced magnetic order in carbon: SQUID measurements
Authors:
J. Barzola-Quiquia,
S. Petriconi,
P. Esquinazi,
M. Rothermel,
D. Spemann,
A. Setzer,
T. Butz
Abstract:
In this work we have studied systematically the changes in the magnetic behavior of highly oriented pyrolytic graphite (HOPG) samples after proton irradiation in the MeV energy range. Superconducting quantum interferometer device (SQUID) results obtained from samples with thousands of localized spots of micrometer size as well on samples irradiated with a broad beam confirm previously reported r…
▽ More
In this work we have studied systematically the changes in the magnetic behavior of highly oriented pyrolytic graphite (HOPG) samples after proton irradiation in the MeV energy range. Superconducting quantum interferometer device (SQUID) results obtained from samples with thousands of localized spots of micrometer size as well on samples irradiated with a broad beam confirm previously reported results. Both, the para- and ferromagnetic contributions depend strongly on the irradiation details. The results indicate that the magnetic moment at saturation of spots of micrometer size is of the order of $10^{-10}$ emu.
△ Less
Submitted 11 October, 2006;
originally announced October 2006.
