-
scores: A Python package for verifying and evaluating models and predictions with xarray
Authors:
Tennessee Leeuwenburg,
Nicholas Loveday,
Elizabeth E. Ebert,
Harrison Cook,
Mohammadreza Khanarmuei,
Robert J. Taggart,
Nikeeth Ramanathan,
Maree Carroll,
Stephanie Chong,
Aidan Griffiths,
John Sharples
Abstract:
`scores` is a Python package containing mathematical functions for the verification, evaluation and optimisation of forecasts, predictions or models. It supports labelled n-dimensional (multidimensional) data, which is used in many scientific fields and in machine learning. At present, `scores` primarily supports the geoscience communities; in particular, the meteorological, climatological and oce…
▽ More
`scores` is a Python package containing mathematical functions for the verification, evaluation and optimisation of forecasts, predictions or models. It supports labelled n-dimensional (multidimensional) data, which is used in many scientific fields and in machine learning. At present, `scores` primarily supports the geoscience communities; in particular, the meteorological, climatological and oceanographic communities. `scores` not only includes common scores (e.g., Mean Absolute Error), it also includes novel scores not commonly found elsewhere (e.g., FIxed Risk Multicategorical (FIRM) score, Flip-Flop Index), complex scores (e.g., threshold-weighted continuous ranked probability score), and statistical tests (such as the Diebold Mariano test). It also contains isotonic regression which is becoming an increasingly important tool in forecast verification and can be used to generate stable reliability diagrams. Additionally, it provides pre-processing tools for preparing data for scores in a variety of formats including cumulative distribution functions (CDF). At the time of writing, `scores` includes over 50 metrics, statistical techniques and data processing tools. All of the scores and statistical techniques in this package have undergone a thorough scientific and software review. Every score has a companion Jupyter Notebook tutorial that demonstrates its use in practice. `scores` supports `xarray` datatypes, allowing it to work with Earth system data in a range of formats including NetCDF4, HDF5, Zarr and GRIB among others. `scores` uses Dask for scaling and performance. Support for `pandas` is being introduced. The `scores` software repository can be found at https://github.com/nci/scores/
△ Less
Submitted 3 July, 2024; v1 submitted 11 June, 2024;
originally announced June 2024.
-
Machine Learning based Parameter Sensitivity of Regional Climate Models -- A Case Study of the WRF Model for Heat Extremes over Southeast Australia
Authors:
P. Jyoteeshkumar Reddy,
Sandeep Chinta,
Richard Matear,
John Taylor,
Harish Baki,
Marcus Thatcher,
Jatin Kala,
Jason Sharples
Abstract:
Heatwaves and bushfires cause substantial impacts on society and ecosystems across the globe. Accurate information of heat extremes is needed to support the development of actionable mitigation and adaptation strategies. Regional climate models are commonly used to better understand the dynamics of these events. These models have very large input parameter sets, and the parameters within the physi…
▽ More
Heatwaves and bushfires cause substantial impacts on society and ecosystems across the globe. Accurate information of heat extremes is needed to support the development of actionable mitigation and adaptation strategies. Regional climate models are commonly used to better understand the dynamics of these events. These models have very large input parameter sets, and the parameters within the physics schemes substantially influence the model's performance. However, parameter sensitivity analysis (SA) of regional models for heat extremes is largely unexplored. Here, we focus on the southeast Australian region, one of the global hotspots of heat extremes. In southeast Australia Weather Research and Forecasting (WRF) model is the widely used regional model to simulate extreme weather events across the region. Hence in this study, we focus on the sensitivity of WRF model parameters to surface meteorological variables such as temperature, relative humidity, and wind speed during two extreme heat events over southeast Australia. Due to the presence of multiple parameters and their complex relationship with output variables, a machine learning (ML) surrogate-based global sensitivity analysis method is considered for the SA. The ML surrogate-based Sobol SA is used to identify the sensitivity of 24 adjustable parameters in seven different physics schemes of the WRF model. Results show that out of these 24, only three parameters, namely the scattering tuning parameter, multiplier of saturated soil water content, and profile shape exponent in the momentum diffusivity coefficient, are important for the considered meteorological variables. These SA results are consistent for the two different extreme heat events. Further, we investigated the physical significance of sensitive parameters. This study's results will help in further optimising WRF parameters to improve model simulation.
△ Less
Submitted 27 July, 2023;
originally announced July 2023.
-
Anthropogenic Mixing of Seasonally Stratified Shelf Seas by Offshore Wind Farm Infrastructure
Authors:
Robert Dorrell,
Charlie Lloyd,
Ben Lincoln,
Tom Rippeth,
John Taylor,
Colm-cille Caulfield,
Jonathan Sharples,
Jeff Polton,
Brian Scannell,
Deborah Greaves,
Rob Hall,
John Simpson
Abstract:
The offshore wind energy sector has rapidly expanded over the past two decades, providing a renewable energy solution for coastal nations. Sector development has been led in Europe, but is growing globally. Most developments to date have been in well-mixed, i.e. unstratified, shallow-waters near to shore. Sector growth is, for the first time, pushing developments to deep water, into a brand new en…
▽ More
The offshore wind energy sector has rapidly expanded over the past two decades, providing a renewable energy solution for coastal nations. Sector development has been led in Europe, but is growing globally. Most developments to date have been in well-mixed, i.e. unstratified, shallow-waters near to shore. Sector growth is, for the first time, pushing developments to deep water, into a brand new environment: seasonally stratified shelf seas. Seasonally stratified shelf seas, where water density varies with depth, have a disproportionately key role in primary production, marine ecosystem and biochemically cycles. Infrastructure will directly mix stratified shelf seas. The magnitude of this mixing, additional to natural background processes, has yet to be fully quantified. If large enough it may erode shelf sea stratification. Therefore, offshore wind growth may destabilize and fundamentally change shelf sea systems. However, enhanced mixing may also positively impact some marine ecosystems. This paper sets the scene for sector development into this new environment, reviews the potential physical and environmental benefits and impacts of large scale industrialization of seasonally stratified shelf seas and identifies areas where research is required to best utilise, manage and mitigate environmental change.
△ Less
Submitted 18 December, 2021;
originally announced December 2021.
