-
Objective discovery of dominant dynamical processes with intelligible machine learning
Authors:
Bryan E. Kaiser,
Juan A. Saenz,
Maike Sonnewald,
Daniel Livescu
Abstract:
The advent of big data has vast potential for discovery in natural phenomena ranging from climate science to medicine, but overwhelming complexity stymies insight. Existing theory is often not able to succinctly describe salient phenomena, and progress has largely relied on ad hoc definitions of dynamical regimes to guide and focus exploration. We present a formal definition in which the identific…
▽ More
The advent of big data has vast potential for discovery in natural phenomena ranging from climate science to medicine, but overwhelming complexity stymies insight. Existing theory is often not able to succinctly describe salient phenomena, and progress has largely relied on ad hoc definitions of dynamical regimes to guide and focus exploration. We present a formal definition in which the identification of dynamical regimes is formulated as an optimization problem, and we propose an intelligible objective function. Furthermore, we propose an unsupervised learning framework which eliminates the need for a priori knowledge and ad hoc definitions; instead, the user need only choose appropriate clustering and dimensionality reduction algorithms, and this choice can be guided using our proposed objective function. We illustrate its applicability with example problems drawn from ocean dynamics, tumor angiogenesis, and turbulent boundary layers. Our method is a step towards unbiased data exploration that allows serendipitous discovery within dynamical systems, with the potential to propel the physical sciences forward.
△ Less
Submitted 21 June, 2021;
originally announced June 2021.
-
Towards interval uncertainty propagation control in bivariate aggregation processes and the introduction of width-limited interval-valued overlap functions
Authors:
Tiago da Cruz Asmus,
Graçaliz Pereira Dimuro,
Benjamín Bedregal,
José Antonio Sanz,
Radko Mesiar,
Humberto Bustince
Abstract:
Overlap functions are a class of aggregation functions that measure the overlap** degree between two values. Interval-valued overlap functions were defined as an extension to express the overlap** of interval-valued data, and they have been usually applied when there is uncertainty regarding the assignment of membership degrees. The choice of a total order for intervals can be significant, whi…
▽ More
Overlap functions are a class of aggregation functions that measure the overlap** degree between two values. Interval-valued overlap functions were defined as an extension to express the overlap** of interval-valued data, and they have been usually applied when there is uncertainty regarding the assignment of membership degrees. The choice of a total order for intervals can be significant, which motivated the recent developments on interval-valued aggregation functions and interval-valued overlap functions that are increasing to a given admissible order, that is, a total order that refines the usual partial order for intervals. Also, width preservation has been considered on these recent works, in an intent to avoid the uncertainty increase and guarantee the information quality, but no deeper study was made regarding the relation between the widths of the input intervals and the output interval, when applying interval-valued functions, or how one can control such uncertainty propagation based on this relation. Thus, in this paper we: (i) introduce and develop the concepts of width-limited interval-valued functions and width limiting functions, presenting a theoretical approach to analyze the relation between the widths of the input and output intervals of bivariate interval-valued functions, with special attention to interval-valued aggregation functions; (ii) introduce the concept of $(a,b)$-ultramodular aggregation functions, a less restrictive extension of one-dimension convexity for bivariate aggregation functions, which have an important predictable behaviour with respect to the width when extended to the interval-valued context; (iii) define width-limited interval-valued overlap functions, taking into account a function that controls the width of the output interval; (iv) present and compare three construction methods for these width-limited interval-valued overlap functions.
△ Less
Submitted 8 June, 2021;
originally announced June 2021.
-
Motor-Imagery-Based Brain Computer Interface using Signal Derivation and Aggregation Functions
Authors:
Javier Fumanal-Idocin,
Yu-Kai Wang,
Chin-Teng Lin,
Javier Fernández,
Jose Antonio Sanz,
Humberto Bustince
Abstract:
Brain Computer Interface technologies are popular methods of communication between the human brain and external devices. One of the most popular approaches to BCI is Motor Imagery. In BCI applications, the ElectroEncephaloGraphy is a very popular measurement for brain dynamics because of its non-invasive nature. Although there is a high interest in the BCI topic, the performance of existing system…
▽ More
Brain Computer Interface technologies are popular methods of communication between the human brain and external devices. One of the most popular approaches to BCI is Motor Imagery. In BCI applications, the ElectroEncephaloGraphy is a very popular measurement for brain dynamics because of its non-invasive nature. Although there is a high interest in the BCI topic, the performance of existing systems is still far from ideal, due to the difficulty of performing pattern recognition tasks in EEG signals. BCI systems are composed of a wide range of components that perform signal pre-processing, feature extraction and decision making. In this paper, we define a BCI Framework, named Enhanced Fusion Framework, where we propose three different ideas to improve the existing MI-based BCI frameworks. Firstly, we include aan additional pre-processing step of the signal: a differentiation of the EEG signal that makes it time-invariant. Secondly, we add an additional frequency band as feature for the system and we show its effect on the performance of the system. Finally, we make a profound study of how to make the final decision in the system. We propose the usage of both up to six types of different classifiers and a wide range of aggregation functions (including classical aggregations, Choquet and Sugeno integrals and their extensions and overlap functions) to fuse the information given by the considered classifiers. We have tested this new system on a dataset of 20 volunteers performing motor imagery-based brain-computer interface experiments. On this dataset, the new system achieved a 88.80% of accuracy. We also propose an optimized version of our system that is able to obtain up to 90,76%. Furthermore, we find that the pair Choquet/Sugeno integrals and overlap functions are the ones providing the best results.
△ Less
Submitted 2 June, 2021; v1 submitted 18 January, 2021;
originally announced January 2021.
-
A Prognostic, One-Equation Model of Meso-Scale Eddy Momentum Fluxes
Authors:
J. A. Saenz,
T. D. Ringler
Abstract:
We present a prognostic, one-equation model for eddy-mean flow interactions to parameterize the divergence of the Eliassen-Palm flux tensor (EPFT) that arises from thickness-weighted averaging (TWA) the hydrostatic Boussinesq equations. The TWA system of equations does not invoke approximations beyond those for which the hydrostatic Boussinesq equations are valid, constituting a mathematically con…
▽ More
We present a prognostic, one-equation model for eddy-mean flow interactions to parameterize the divergence of the Eliassen-Palm flux tensor (EPFT) that arises from thickness-weighted averaging (TWA) the hydrostatic Boussinesq equations. The TWA system of equations does not invoke approximations beyond those for which the hydrostatic Boussinesq equations are valid, constituting a mathematically consistent framework with clear physical interpretations. This model is intended for the adiabatic interior of zonally symmetric flows, in the absence of topographic features, where terms corresponding to eddy interfacial form drag in the EPFT dominate forces. We model eddy interfacial form drag terms for vertical flux of horizontal momentum using the gradient hypothesis, as the product of an eddy viscosity and the vertical gradient of horizontal momentum. We use mixing length theory to relate viscosity to an eddy length scale and an eddy velocity, which is proportional to the eddy energy in the TWA system. The eddy length scale is modeled as the first Rossby radius of deformation, which we calculate as a function of the mean flow. We use a prognostic equation for vertically integrated eddy energy at each horizontal location, which we derive from the TWA framework, and then simplify to the flows of interest by ignoring transport, redistribution and diabatic terms. The prognostic vertically integrated eddy energy is projected onto the water column using the eigenvalue of the first baroclinic mode to obtain the eddy energy at each vertical position. The eddy viscosity has horizontal as well as vertical structure. We diagnosed the model equations in an eddy resolving numerical simulation of a zonally re-entrant channel representative of the Southern Ocean. We have implemented the model parameterization in an ocean model and tested it to simulate a parameterized simulation of this flow.
△ Less
Submitted 18 December, 2020;
originally announced December 2020.
-
Filtering, averaging and scale dependency in homogeneous variable density turbulence
Authors:
J. A. Saenz,
D. Aslangil,
D. Livescu
Abstract:
We investigate relationships between statistics obtained from filtering and from ensemble or Reynolds-averaging turbulence flow fields as a function of length scale. Generalized central moments in the filtering approach are expressed as inner products of generalized fluctuating quantities, $q'(ξ,x)=q(ξ)-\overline q(x)$, representing fluctuations of a field $q(ξ)$, at any point $ξ$, with respect to…
▽ More
We investigate relationships between statistics obtained from filtering and from ensemble or Reynolds-averaging turbulence flow fields as a function of length scale. Generalized central moments in the filtering approach are expressed as inner products of generalized fluctuating quantities, $q'(ξ,x)=q(ξ)-\overline q(x)$, representing fluctuations of a field $q(ξ)$, at any point $ξ$, with respect to its filtered value at $x$. For positive-definite filter kernels, these expressions provide a scale-resolving framework, with statistics and realizability conditions at any length scale. In the small-scale limit, scale-resolving statistics become zero. In the large-scale limit, scale-resolving statistics and realizability conditions are the same as in the Reynolds-averaged description. Using direct numerical simulations (DNS) of homogeneous variable density turbulence, we diagnose Reynolds stresses, $\mathcal{T}_{ij}$, resolved kinetic energy, $k_r$, turbulent mass-flux velocity, $a_i$, and density-specific volume covariance, $b$, defined in the scale-resolving framework. These variables, and terms in their governing equations, vary smoothly between zero and their Reynolds-averaged definitions at the small and large scale limits, respectively. At intermediate scales, the governing equations exhibit interactions between terms that are not active in the Reynolds-averaged limit. For example, in the Reynolds-averaged limit, $b$ follows a decaying process driven by a destruction term; at intermediate length scales it is a balance between production, redistribution, destruction, and transport, where $b$ grows as the density spectrum develops, and then decays when mixing becomes strong enough. This work supports the notion of a generalized, length-scale adaptive model that converges to DNS at high resolutions, and to Reynolds-averaged statistics at coarse resolutions.
△ Less
Submitted 12 December, 2020;
originally announced December 2020.
-
Interval-valued aggregation functions based on moderate deviations applied to Motor-Imagery-Based Brain Computer Interface
Authors:
Javier Fumanal-Idocin,
Zdenko Takáč,
Javier Fernández Jose Antonio Sanz,
Harkaitz Goyena,
Ching-Teng Lin,
Yu-Kai Wang,
Humberto Bustince
Abstract:
In this work we study the use of moderate deviation functions to measure similarity and dissimilarity among a set of given interval-valued data. To do so, we introduce the notion of interval-valued moderate deviation function and we study in particular those interval-valued moderate deviation functions which preserve the width of the input intervals. Then, we study how to apply these functions to…
▽ More
In this work we study the use of moderate deviation functions to measure similarity and dissimilarity among a set of given interval-valued data. To do so, we introduce the notion of interval-valued moderate deviation function and we study in particular those interval-valued moderate deviation functions which preserve the width of the input intervals. Then, we study how to apply these functions to construct interval-valued aggregation functions. We have applied them in the decision making phase of two Motor-Imagery Brain Computer Interface frameworks, obtaining better results than those obtained using other numerical and intervalar aggregations.
△ Less
Submitted 1 July, 2021; v1 submitted 19 November, 2020;
originally announced November 2020.
-
Interpreting neural network models of residual scalar flux
Authors:
Gavin D. Portwood,
Balasubramanya T. Nadiga,
Juan A. Saenz,
Daniel Livescu
Abstract:
We show that in addition to providing effective and competitive closures, when analysed in terms of dynamics and physically-relevant diagnostics, artificial neural networks (ANNs) can be both interpretable and provide useful insights in the on-going task of develo** and improving turbulence closures. In the context of large-eddy simulations (LES) of a passive scalar in homogeneous isotropic turb…
▽ More
We show that in addition to providing effective and competitive closures, when analysed in terms of dynamics and physically-relevant diagnostics, artificial neural networks (ANNs) can be both interpretable and provide useful insights in the on-going task of develo** and improving turbulence closures. In the context of large-eddy simulations (LES) of a passive scalar in homogeneous isotropic turbulence, exact subfilter fluxes obtained by filtering direct numerical simulations (DNS) are used both to train deep ANN models as a function of filtered variables, and to optimise the coefficients of a turbulent Prandtl number LES closure. \textit{A-priori} analysis of the subfilter scalar variance transfer rate demonstrates that learnt ANN models out-perform optimised turbulent Prandtl number closures and Clark-type gradient models. Next, \textit{a-posteriori} solutions are obtained with each model over several integral timescales. These experiments reveal, with single- and multi-point diagnostics, that ANN models temporally track exact resolved scalar variance with greater accuracy compared to other subfilter flux models for a given filter length scale. Finally, we interpret the artificial neural networks statistically with differential sensitivity analysis to show that the ANN models feature dynamics reminiscent of so-called "mixed models", where mixed models are understood as comprising both a structural and functional component. Besides enabling enhanced-accuracy LES of passive scalars henceforth, we anticipate this work to contribute to utilising neural network models as a tool in interpretability, robustness and model discovery.
△ Less
Submitted 5 October, 2020; v1 submitted 15 April, 2020;
originally announced April 2020.
-
Turbulence forecasting via Neural ODE
Authors:
Gavin D. Portwood,
Peetak P. Mitra,
Mateus Dias Ribeiro,
Tan Minh Nguyen,
Balasubramanya T. Nadiga,
Juan A. Saenz,
Michael Chertkov,
Animesh Garg,
Anima Anandkumar,
Andreas Dengel,
Richard Baraniuk,
David P. Schmidt
Abstract:
Fluid turbulence is characterized by strong coupling across a broad range of scales. Furthermore, besides the usual local cascades, such coupling may extend to interactions that are non-local in scale-space. As such the computational demands associated with explicitly resolving the full set of scales and their interactions, as in the Direct Numerical Simulation (DNS) of the Navier-Stokes equations…
▽ More
Fluid turbulence is characterized by strong coupling across a broad range of scales. Furthermore, besides the usual local cascades, such coupling may extend to interactions that are non-local in scale-space. As such the computational demands associated with explicitly resolving the full set of scales and their interactions, as in the Direct Numerical Simulation (DNS) of the Navier-Stokes equations, in most problems of practical interest are so high that reduced modeling of scales and interactions is required before further progress can be made. While popular reduced models are typically based on phenomenological modeling of relevant turbulent processes, recent advances in machine learning techniques have energized efforts to further improve the accuracy of such reduced models. In contrast to such efforts that seek to improve an existing turbulence model, we propose a machine learning(ML) methodology that captures, de novo, underlying turbulence phenomenology without a pre-specified model form. To illustrate the approach, we consider transient modeling of the dissipation of turbulent kinetic energy, a fundamental turbulent process that is central to a wide range of turbulence models using a Neural ODE approach. After presenting details of the methodology, we show that this approach outperforms state-of-the-art approaches.
△ Less
Submitted 12 November, 2019;
originally announced November 2019.
-
Dimensionality-Reduction of Climate Data using Deep Autoencoders
Authors:
J. A. Saenz,
N. Lubbers,
N. M. Urban
Abstract:
We explore the use of deep neural networks for nonlinear dimensionality reduction in climate applications. We train convolutional autoencoders (CAEs) to encode two temperature field datasets from pre-industrial control runs in the CMIP5 first ensemble, obtained with the CCSM4 model and the IPSL-CM5A-LR model, respectively. With the later dataset, consisting of 36500 96$\times$96 surface temperatur…
▽ More
We explore the use of deep neural networks for nonlinear dimensionality reduction in climate applications. We train convolutional autoencoders (CAEs) to encode two temperature field datasets from pre-industrial control runs in the CMIP5 first ensemble, obtained with the CCSM4 model and the IPSL-CM5A-LR model, respectively. With the later dataset, consisting of 36500 96$\times$96 surface temperature fields, the CAE out-performs PCA in terms of mean squared error of the reconstruction from a 40 dimensional encoding. Moreover, the noise in the filters of the convolutional layers in the autoencoders suggests that the CAE can be trained to produce better results. Our results indicate that convolutional autoencoders may provide an effective platform for the construction of surrogate climate models.
△ Less
Submitted 27 August, 2018;
originally announced September 2018.
