Showing 1–4 of 4 results for author: Barnes, R J

Adding Uncertainty to Neural Network Regression Tasks in the Geosciences

Authors: Elizabeth A. Barnes, Randal J. Barnes, Nicolas Gordillo

Abstract: A simple method for adding uncertainty to neural network regression tasks via estimation of a general probability distribution is described. The methodology supports estimation of heteroscedastic, asymmetric uncertainties by a simple modification of the network output and loss function. Method performance is demonstrated with a simple one dimensional data set and then applied to a more complex reg… ▽ More A simple method for adding uncertainty to neural network regression tasks via estimation of a general probability distribution is described. The methodology supports estimation of heteroscedastic, asymmetric uncertainties by a simple modification of the network output and loss function. Method performance is demonstrated with a simple one dimensional data set and then applied to a more complex regression task using synthetic climate data. △ Less

Submitted 15 September, 2021; originally announced September 2021.

Controlled abstention neural networks for identifying skillful predictions for classification problems

Authors: Elizabeth A. Barnes, Randal J. Barnes

Abstract: The earth system is exceedingly complex and often chaotic in nature, making prediction incredibly challenging: we cannot expect to make perfect predictions all of the time. Instead, we look for specific states of the system that lead to more predictable behavior than others, often termed "forecasts of opportunity." When these opportunities are not present, scientists need prediction systems that a… ▽ More The earth system is exceedingly complex and often chaotic in nature, making prediction incredibly challenging: we cannot expect to make perfect predictions all of the time. Instead, we look for specific states of the system that lead to more predictable behavior than others, often termed "forecasts of opportunity." When these opportunities are not present, scientists need prediction systems that are capable of saying "I don't know." We introduce a novel loss function, termed the "NotWrong loss", that allows neural networks to identify forecasts of opportunity for classification problems. The NotWrong loss introduces an abstention class that allows the network to identify the more confident samples and abstain (say "I don't know") on the less confident samples. The abstention loss is designed to abstain on a user-defined fraction of the samples via a PID controller. Unlike many machine learning methods used to reject samples post-training, the NotWrong loss is applied during training to preferentially learn from the more confident samples. We show that the NotWrong loss outperforms other existing loss functions for multiple climate use cases. The implementation of the proposed loss function is straightforward in most network architectures designed for classification as it only requires the addition of an abstention class to the output layer and modification of the loss function. △ Less

Submitted 16 April, 2021; originally announced April 2021.

Comments: submitted to the Journal of Advances in Earth System Modeling. arXiv admin note: substantial text overlap with arXiv:2104.08236

Controlled abstention neural networks for identifying skillful predictions for regression problems

Authors: Elizabeth A. Barnes, Randal J. Barnes

Abstract: The earth system is exceedingly complex and often chaotic in nature, making prediction incredibly challenging: we cannot expect to make perfect predictions all of the time. Instead, we look for specific states of the system that lead to more predictable behavior than others, often termed "forecasts of opportunity". When these opportunities are not present, scientists need prediction systems that a… ▽ More The earth system is exceedingly complex and often chaotic in nature, making prediction incredibly challenging: we cannot expect to make perfect predictions all of the time. Instead, we look for specific states of the system that lead to more predictable behavior than others, often termed "forecasts of opportunity". When these opportunities are not present, scientists need prediction systems that are capable of saying "I don't know." We introduce a novel loss function, termed "abstention loss", that allows neural networks to identify forecasts of opportunity for regression problems. The abstention loss works by incorporating uncertainty in the network's prediction to identify the more confident samples and abstain (say "I don't know") on the less confident samples. The abstention loss is designed to determine the optimal abstention fraction, or abstain on a user-defined fraction via a PID controller. Unlike many methods for attaching uncertainty to neural network predictions post-training, the abstention loss is applied during training to preferentially learn from the more confident samples. The abstention loss is built upon a standard computer science method. While the standard approach is itself a simple yet powerful tool for incorporating uncertainty in regression problems, we demonstrate that the abstention loss outperforms this more standard method for the synthetic climate use cases explored here. The implementation of proposed loss function is straightforward in most network architectures designed for regression, as it only requires modification of the output layer and loss function. △ Less

Submitted 16 April, 2021; originally announced April 2021.

Comments: submitted to the Journal of Advances of Earth System Modeling

SuperDARN Observations of Semidiurnal Tidal Variability in the MLT and the Response to Sudden Stratospheric Warming Events

Authors: R. E. Hibbins, P. J. Espy, Y. J. Orsolini, V. Limpasuvan, R. J. Barnes

Abstract: Using meteor wind data from the Super Dual Auroral Radar Network (SuperDARN) in the Northern Hemisphere, we (1) demonstrate that the migrating (Sun-synchronous) tides can be separated from the nonmigrating components in the mesosphere and lower thermosphere (MLT) region and (2) use this to determine the response of the different components of the semidiurnal tide (SDT) to sudden stratospheric warm… ▽ More Using meteor wind data from the Super Dual Auroral Radar Network (SuperDARN) in the Northern Hemisphere, we (1) demonstrate that the migrating (Sun-synchronous) tides can be separated from the nonmigrating components in the mesosphere and lower thermosphere (MLT) region and (2) use this to determine the response of the different components of the semidiurnal tide (SDT) to sudden stratospheric warming (SSW) conditions. The radars span a limited range of latitudes around 60$^{\circ}$ N and are located over nearly 180$^{\circ}$ of longitude. The migrating tide is extracted from the nonmigrating components observed in the meridional wind recorded from meteor ablation drift velocities around 95-km altitude, and a 20-year climatology of the different components is presented. The well-documented late summer and wintertime maxima in the semidiurnal winds are shown to be due primarily to the migrating SDT, whereas during late autumn and spring the nonmigrating components are at least as strong as the migrating SDT. The robust behavior of the SDT components during SSWs is then examined by compositing 13 SSW events associated with an elevated stratopause recorded between 1995 and 2013. The migrating SDT is seen to reduce in amplitude immediately after SSW onset and then return anomalously strongly around 10-17 days after the SSW onset. We conclude that changes in the underlying wind direction play a role in modulating the tidal amplitude during the evolution of SSWs and that the enhancement in the midlatitude migrating SDT (previously reported in modeling studies) is observed in the MLT at least up to 60$^{\circ}$ N. △ Less

Submitted 28 June, 2019; originally announced June 2019.