Showing 1–37 of 37 results for author: Rasheed, A

Exploring Urban Mobility Trends using Cellular Network Data

Authors: Oluwaleke Yusuf, Adil Rasheed, Frank Lindseth

Abstract: The growth of urban areas intensifies the need for sustainable, efficient transportation infrastructure and mobility systems, driving initiatives to enhance infrastructure and public transport while reducing congestion and emissions. By utilizing real-world mobility data, a data-driven approach can provide crucial insights for planning and decision-making. This study explores the efficacy of lev… ▽ More The growth of urban areas intensifies the need for sustainable, efficient transportation infrastructure and mobility systems, driving initiatives to enhance infrastructure and public transport while reducing congestion and emissions. By utilizing real-world mobility data, a data-driven approach can provide crucial insights for planning and decision-making. This study explores the efficacy of leveraging telecoms data from cellular network signals for studying crowd movement patterns, focusing on Trondheim, Norway. It examines routing reports to understand the spatiotemporal dynamics of various transportation routes and modes. A data preprocessing and feature engineering framework was developed to process raw routing reports for historical analysis. This enabled the examination of geospatial trends and temporal patterns, including a comparative analysis of various transportation modes, along with public transit usage. Specific routes and areas were analyzed in-depth to compare their mobility patterns with the broader city context. The study highlights the potential of cellular network data as a resource for sha** urban transportation and mobility systems. By identifying deficiencies and potential improvements, city planners and stakeholders can foster more sustainable and effective transportation solutions. △ Less

Submitted 31 March, 2024; originally announced April 2024.

Unveiling Urban Mobility Patterns: A Data-Driven Analysis of Public Transit

Authors: Oluwaleke Yusuf, Adil Rasheed, Frank Lindseth

Abstract: The expansion of urban centers necessitates enhanced efficiency and sustainability in their transportation infrastructure and mobility systems. The big data obtainable from various transportation modes potentially offers critical insights for urban planning. This study presents analysis of detailed historical public transit data, enriched with relevant temporal and geospatial metadata, as a precur… ▽ More The expansion of urban centers necessitates enhanced efficiency and sustainability in their transportation infrastructure and mobility systems. The big data obtainable from various transportation modes potentially offers critical insights for urban planning. This study presents analysis of detailed historical public transit data, enriched with relevant temporal and geospatial metadata, as a precursor to injecting dynamism into digital twins of mobility systems via ML/DL-based predictive modeling. A data preprocessing framework was implemented to refine the raw data for effective historical analysis and predictive modeling. This paper examines public transit data for patterns and trends -- incorporating factors such as time, geospatial elements, external influences, and operational aspects. From a technical standpoint, this research helps to assess the quality of the available transit data and identify important information for use in digital twins. Such digital twins foster educated decisions for efficient, sustainable urban mobility systems by anticipating infrastructure demand, identifying service gaps, and understanding mobility dynamics. △ Less

Submitted 31 March, 2024; originally announced April 2024.

Insights into the mechanics of pure and bacteria-laden sessile whole blood droplet evaporation

Authors: Durbar Roy, Sophia M, Kush K Dewangan, Abdur Rasheed, Siddhant Jain, Anmol Singh, Dipshikha Chakravortty, Saptarshi Basu

Abstract: We study the mechanics of sessile blood drop evaporation using optical diagnostics, theoretical analysis, and micro/nano-characterization. The transient evaporation process has three major phases (A, B, and C) based on the evaporation rate. Phase A is the fastest, where edge evaporation dominates and forms a gelated three-phase contact line. Gelation results from sol-gel phase transition that occu… ▽ More We study the mechanics of sessile blood drop evaporation using optical diagnostics, theoretical analysis, and micro/nano-characterization. The transient evaporation process has three major phases (A, B, and C) based on the evaporation rate. Phase A is the fastest, where edge evaporation dominates and forms a gelated three-phase contact line. Gelation results from sol-gel phase transition that occurs due to the accumulation of red blood cells, as they get transported due to outward capillary flow generated during drop evaporation. The intermediate phase B consists of a gelation front propagating radially inwards due to the combined effect of outward capillary flow and drop height reduction evaporating in pinned mode, forming a wet gel phase. We unearthed that the gelation of the entire droplet occurs in Phase B, and the gel formed contains trace amounts of water that are detectable in our experiments. Phase C is the final slowest stage of evaporation, where the wet gel transforms into a dry gel and leads to desiccation induced stress, forming diverse crack patterns in the dried blood drop precipitate. We observe radial and orthoradial cracks in the precipitate's thicker region, mud-flat cracks in the drop center, and the outer contact line where thickness and curvature are relatively small. We also study the evaporation of bacteria-laden droplets to simulate bacterial infection in human blood and show that the drop evaporation rate and final dried residue pattern do not change appreciably within the parameter variation of the bacterial concentration typically found in bacterial infection of living organisms. △ Less

Submitted 17 April, 2024; v1 submitted 19 February, 2024; originally announced February 2024.

Enhancing wind field resolution in complex terrain through a knowledge-driven machine learning approach

Authors: Jacob Wulff Wold, Florian Stadtmann, Adil Rasheed, Mandar Tabib, Omer San, Jan-Tore Horn

Abstract: Atmospheric flows are governed by a broad variety of spatio-temporal scales, thus making real-time numerical modeling of such turbulent flows in complex terrain at high resolution computationally intractable. In this study, we demonstrate a neural network approach motivated by Enhanced Super-Resolution Generative Adversarial Networks to upscale low-resolution wind fields to generate high-resolutio… ▽ More Atmospheric flows are governed by a broad variety of spatio-temporal scales, thus making real-time numerical modeling of such turbulent flows in complex terrain at high resolution computationally intractable. In this study, we demonstrate a neural network approach motivated by Enhanced Super-Resolution Generative Adversarial Networks to upscale low-resolution wind fields to generate high-resolution wind fields in an actual wind farm in Bessaker, Norway. The neural network-based model is shown to successfully reconstruct fully resolved 3D velocity fields from a coarser scale while respecting the local terrain and that it easily outperforms trilinear interpolation. We also demonstrate that by using appropriate cost function based on domain knowledge, we can alleviate the use of adversarial training. △ Less

Submitted 2 April, 2024; v1 submitted 18 September, 2023; originally announced September 2023.

PoroTwin: A digital twin for a FluidFlower rig

Authors: Eirik Keilegavlen, Eivind Fonn, Kjetil Johannessen, Kristoffer Eikehaug, Jakub Both, Martin Fernø, Trond Kvamsdal, Adil Rasheed, Jan M. Nordbotten

Abstract: We present a framework for integrated experiments and simulations of tracer transport in heterogeneous porous media using digital twin technology. The physical asset in our setup is a meter-scale FluidFlower rig. The digital twin consists of a traditional physics-based forward simulation tool and a correction technique which compensates for mismatches between simulation results and observations. T… ▽ More We present a framework for integrated experiments and simulations of tracer transport in heterogeneous porous media using digital twin technology. The physical asset in our setup is a meter-scale FluidFlower rig. The digital twin consists of a traditional physics-based forward simulation tool and a correction technique which compensates for mismatches between simulation results and observations. The latter augments the range of the physics-based simulation and allows us to bridge the gap between simulation and experiments in a quantitative sense. We describe the setup of the physical and digital twin, including data transfer protocols using cloud technology. The accuracy of the digital twin is demonstrated on a case with artificially high diffusion that must be compensated by the correction approach, as well as by simulations in geologically complex media. The digital twin is then applied to control tracer transport by manipulating fluid injection and production in the experimental rig, thereby enabling two-way coupling between the physical and digital twins. △ Less

Submitted 6 January, 2023; v1 submitted 1 December, 2022; originally announced December 2022.

Sparse deep neural networks for modeling aluminum electrolysis dynamics

Authors: Erlend Torje Berg Lundby, Adil Rasheed, Ivar Johan Halvorsen, Jan Tommy Gravdahl

Abstract: Deep neural networks have become very popular in modeling complex nonlinear processes due to their extraordinary ability to fit arbitrary nonlinear functions from data with minimal expert intervention. However, they are almost always overparameterized and challenging to interpret due to their internal complexity. Furthermore, the optimization process to find the learned model parameters can be uns… ▽ More Deep neural networks have become very popular in modeling complex nonlinear processes due to their extraordinary ability to fit arbitrary nonlinear functions from data with minimal expert intervention. However, they are almost always overparameterized and challenging to interpret due to their internal complexity. Furthermore, the optimization process to find the learned model parameters can be unstable due to the process getting stuck in local minima. In this work, we demonstrate the value of sparse regularization techniques to significantly reduce the model complexity. We demonstrate this for the case of an aluminium extraction process, which is highly nonlinear system with many interrelated subprocesses. We trained a densely connected deep neural network to model the process and then compared the effects of sparsity promoting l1 regularization on generalizability, interpretability, and training stability. We found that the regularization significantly reduces model complexity compared to a corresponding dense neural network. We argue that this makes the model more interpretable, and show that training an ensemble of sparse neural networks with different parameter initializations often converges to similar model structures with similar learned input features. Furthermore, the empirical study shows that the resulting sparse models generalize better from small training sets than their dense counterparts. △ Less

Submitted 13 January, 2023; v1 submitted 13 September, 2022; originally announced September 2022.

Comments: 32 pages, 18 figures

Prospects of federated machine learning in fluid dynamics

Authors: Omer San, Suraj Pawar, Adil Rasheed

Abstract: Physics-based models have been mainstream in fluid dynamics for develo** predictive models. In recent years, machine learning has offered a renaissance to the fluid community due to the rapid developments in data science, processing units, neural network based technologies, and sensor adaptations. So far in many applications in fluid dynamics, machine learning approaches have been mostly focused… ▽ More Physics-based models have been mainstream in fluid dynamics for develo** predictive models. In recent years, machine learning has offered a renaissance to the fluid community due to the rapid developments in data science, processing units, neural network based technologies, and sensor adaptations. So far in many applications in fluid dynamics, machine learning approaches have been mostly focused on a standard process that requires centralizing the training data on a designated machine or in a data center. In this letter, we present a federated machine learning approach that enables localized clients to collaboratively learn an aggregated and shared predictive model while kee** all the training data on each edge device. We demonstrate the feasibility and prospects of such decentralized learning approach with an effort to forge a deep learning surrogate model for reconstructing spatiotemporal fields. Our results indicate that federated machine learning might be a viable tool for designing highly accurate predictive decentralized digital twins relevant to fluid dynamics. △ Less

Submitted 15 August, 2022; originally announced August 2022.

Comments: arXiv admin note: substantial text overlap with arXiv:2207.12245

Variational multiscale reinforcement learning for discovering reduced order closure models of nonlinear spatiotemporal transport systems

Authors: Omer San, Suraj Pawar, Adil Rasheed

Abstract: A central challenge in the computational modeling and simulation of a multitude of science applications is to achieve robust and accurate closures for their coarse-grained representations due to underlying highly nonlinear multiscale interactions. These closure models are common in many nonlinear spatiotemporal systems to account for losses due to reduced order representations, including many tran… ▽ More A central challenge in the computational modeling and simulation of a multitude of science applications is to achieve robust and accurate closures for their coarse-grained representations due to underlying highly nonlinear multiscale interactions. These closure models are common in many nonlinear spatiotemporal systems to account for losses due to reduced order representations, including many transport phenomena in fluids. Previous data-driven closure modeling efforts have mostly focused on supervised learning approaches using high fidelity simulation data. On the other hand, reinforcement learning (RL) is a powerful yet relatively uncharted method in spatiotemporally extended systems. In this study, we put forth a modular dynamic closure modeling and discovery framework to stabilize the Galerkin projection based reduced order models that may arise in many nonlinear spatiotemporal dynamical systems with quadratic nonlinearity. However, a key element in creating a robust RL agent is to introduce a feasible reward function, which can be constituted of any difference metrics between the RL model and high fidelity simulation data. First, we introduce a multi-modal RL (MMRL) to discover mode-dependant closure policies that utilize the high fidelity data in rewarding our RL agent. We then formulate a variational multiscale RL (VMRL) approach to discover closure models without requiring access to the high fidelity data in designing the reward function. Specifically, our chief innovation is to leverage variational multiscale formalism to quantify the difference between modal interactions in Galerkin systems. Our results in simulating the viscous Burgers equation indicate that the proposed VMRL method leads to robust and accurate closure parameterizations, and it may potentially be used to discover scale-aware closure models for complex dynamical systems. △ Less

Submitted 7 July, 2022; originally announced July 2022.

Decentralized digital twins of complex dynamical systems

Authors: Omer San, Suraj Pawar, Adil Rasheed

Abstract: In this paper, we introduce a decentralized digital twin (DDT) framework for dynamical systems and discuss the prospects of the DDT modeling paradigm in computational science and engineering applications. The DDT approach is built on a federated learning concept, a branch of machine learning that encourages knowledge sharing without sharing the actual data. This approach enables clients to collabo… ▽ More In this paper, we introduce a decentralized digital twin (DDT) framework for dynamical systems and discuss the prospects of the DDT modeling paradigm in computational science and engineering applications. The DDT approach is built on a federated learning concept, a branch of machine learning that encourages knowledge sharing without sharing the actual data. This approach enables clients to collaboratively learn an aggregated model while kee** all the training data on each client. We demonstrate the feasibility of the DDT framework with various dynamical systems, which are often considered prototypes for modeling complex transport phenomena in spatiotemporally extended systems. Our results indicate that federated machine learning might be a key enabler for designing highly accurate decentralized digital twins in complex nonlinear spatiotemporal systems. △ Less

Submitted 7 July, 2022; originally announced July 2022.

Physics Guided Machine Learning for Variational Multiscale Reduced Order Modeling

Authors: Shady E. Ahmed, Omer San, Adil Rasheed, Traian Iliescu, Alessandro Veneziani

Abstract: We propose a new physics guided machine learning (PGML) paradigm that leverages the variational multiscale (VMS) framework and available data to dramatically increase the accuracy of reduced order models (ROMs) at a modest computational cost. The hierarchical structure of the ROM basis and the VMS framework enable a natural separation of the resolved and unresolved ROM spatial scales. Modern PGML… ▽ More We propose a new physics guided machine learning (PGML) paradigm that leverages the variational multiscale (VMS) framework and available data to dramatically increase the accuracy of reduced order models (ROMs) at a modest computational cost. The hierarchical structure of the ROM basis and the VMS framework enable a natural separation of the resolved and unresolved ROM spatial scales. Modern PGML algorithms are used to construct novel models for the interaction among the resolved and unresolved ROM scales. Specifically, the new framework builds ROM operators that are closest to the true interaction terms in the VMS framework. Finally, machine learning is used to reduce the projection error and further increase the ROM accuracy. Our numerical experiments for a two-dimensional vorticity transport problem show that the novel PGML-VMS-ROM paradigm maintains the low computational cost of current ROMs, while significantly increasing the ROM accuracy. △ Less

Submitted 24 May, 2022; originally announced May 2022.

Frame invariant neural network closures for Kraichnan turbulence

Authors: Suraj Pawar, Omer San, Adil Rasheed, Prakash Vedula

Abstract: Numerical simulations of geophysical and atmospheric flows have to rely on parameterizations of subgrid scale processes due to their limited spatial resolution. Despite substantial progress in develo** parameterization (or closure) models for subgrid scale (SGS) processes using physical insights and mathematical approximations, they remain imperfect and can lead to inaccurate predictions. In rec… ▽ More Numerical simulations of geophysical and atmospheric flows have to rely on parameterizations of subgrid scale processes due to their limited spatial resolution. Despite substantial progress in develo** parameterization (or closure) models for subgrid scale (SGS) processes using physical insights and mathematical approximations, they remain imperfect and can lead to inaccurate predictions. In recent years, machine learning has been successful in extracting complex patterns from high-resolution spatio-temporal data, leading to improved parameterization models, and ultimately better coarse grid prediction. However, the inability to satisfy known physics and poor generalization hinders the application of these models for real-world problems. In this work, we propose a frame invariant closure approach to improve the accuracy and generalizability of deep learning-based subgrid scale closure models by embedding physical symmetries directly into the structure of the neural network. Specifically, we utilized specialized layers within the convolutional neural network in such a way that desired constraints are theoretically guaranteed without the need for any regularization terms. We demonstrate our framework for a two-dimensional decaying turbulence test case mostly characterized by the forward enstrophy cascade. We show that our frame invariant SGS model (i) accurately predicts the subgrid scale source term, (ii) respects the physical symmetries such as translation, Galilean, and rotation invariance, and (iii) is numerically stable when implemented in coarse-grid simulation with generalization to different initial conditions and Reynolds number. This work builds a bridge between extensive physics-based theories and data-driven modeling paradigms, and thus represents a promising step towards the development of physically consistent data-driven turbulence closure models. △ Less

Submitted 9 January, 2022; originally announced January 2022.

Nonlinear proper orthogonal decomposition for convection-dominated flows

Authors: Shady E. Ahmed, Omer San, Adil Rasheed, Traian Iliescu

Abstract: Autoencoder techniques find increasingly common use in reduced order modeling as a means to create a latent space. This reduced order representation offers a modular data-driven modeling approach for nonlinear dynamical systems when integrated with a time series predictive model. In this letter, we put forth a nonlinear proper orthogonal decomposition (POD) framework, which is an end-to-end Galerk… ▽ More Autoencoder techniques find increasingly common use in reduced order modeling as a means to create a latent space. This reduced order representation offers a modular data-driven modeling approach for nonlinear dynamical systems when integrated with a time series predictive model. In this letter, we put forth a nonlinear proper orthogonal decomposition (POD) framework, which is an end-to-end Galerkin-free model combining autoencoders with long short-term memory networks for dynamics. By eliminating the projection error due to the truncation of Galerkin models, a key enabler of the proposed nonintrusive approach is the kinematic construction of a nonlinear map** between the full-rank expansion of the POD coefficients and the latent space where the dynamics evolve. We test our framework for model reduction of a convection-dominated system, which is generally challenging for reduced order models. Our approach not only improves the accuracy, but also significantly reduces the computational cost of training and testing. △ Less

Submitted 5 November, 2021; v1 submitted 15 October, 2021; originally announced October 2021.

Multi-fidelity information fusion with concatenated neural networks

Authors: Suraj Pawar, Omer San, Prakash Vedula, Adil Rasheed, Trond Kvamsdal

Abstract: Recently, computational modeling has shifted towards the use of deep learning, and other data-driven modeling frameworks. Although this shift in modeling holds promise in many applications like design optimization and real-time control by lowering the computational burden, training deep learning models needs a huge amount of data. This big data is not always available for scientific problems and l… ▽ More Recently, computational modeling has shifted towards the use of deep learning, and other data-driven modeling frameworks. Although this shift in modeling holds promise in many applications like design optimization and real-time control by lowering the computational burden, training deep learning models needs a huge amount of data. This big data is not always available for scientific problems and leads to poorly generalizable data-driven models. This gap can be furnished by leveraging information from physics-based models. Exploiting prior knowledge about the problem at hand, this study puts forth a concatenated neural network approach to build more tailored, effective, and efficient machine learning models. For our analysis, without losing its generalizability and modularity, we focus on the development of predictive models for laminar and turbulent boundary layer flows. In particular, we combine the self-similarity solution and power-law velocity profile (low-fidelity models) with the noisy data obtained either from experiments or computational fluid dynamics simulations (high-fidelity models) through a concatenated neural network. We illustrate how the knowledge from these simplified models results in reducing uncertainties associated with deep learning models. The proposed framework produces physically consistent models that attempt to achieve better generalization than data-driven models obtained purely based on data. While we demonstrate our framework for a problem relevant to fluid mechanics, its workflow and principles can be adopted for many scientific problems where empirical models are prevalent. In line with grand demands in novel physics-guided machine learning principles, this work builds a bridge between extensive physics-based theories and data-driven modeling paradigms and paves the way for using hybrid modeling approaches for next-generation digital twin technologies. △ Less

Submitted 8 October, 2021; originally announced October 2021.

On closures for reduced order models $-$ A spectrum of first-principle to machine-learned avenues

Authors: Shady E. Ahmed, Suraj Pawar, Omer San, Adil Rasheed, Traian Iliescu, Bernd R. Noack

Abstract: For over a century, reduced order models (ROMs) have been a fundamental discipline of theoretical fluid mechanics. Early examples include Galerkin models inspired by the Orr-Sommerfeld stability equation and numerous vortex models, of which the von Kármán vortex street is one of the most prominent. Subsequent ROMs typically relied on first principles, like mathematical Galerkin models, weakly nonl… ▽ More For over a century, reduced order models (ROMs) have been a fundamental discipline of theoretical fluid mechanics. Early examples include Galerkin models inspired by the Orr-Sommerfeld stability equation and numerous vortex models, of which the von Kármán vortex street is one of the most prominent. Subsequent ROMs typically relied on first principles, like mathematical Galerkin models, weakly nonlinear stability theory, and two- and three-dimensional vortex models. Aubry et al. [N. Aubry, P. Holmes, J. Lumley, and E. Stone, Journal of Fluid Mechanics, 192, 115-173 (1988)] pioneered data-driven proper orthogonal decomposition (POD) modeling. In early POD modeling, available data was used to build an optimal basis, which was then utilized in a classical Galerkin procedure to construct the ROM. But data has made a profound impact on ROMs beyond the Galerkin expansion. In this paper, we take a modest step and illustrate the impact of data-driven modeling on one significant ROM area. Specifically, we focus on ROM closures, which are correction terms that are added to classical ROMs in order to model the effect of the discarded ROM modes in under-resolved simulations. Through simple examples, we illustrate the main modeling principles used to construct classical ROMs, motivate and introduce modern ROM closures, and show how data-driven modeling, artificial intelligence, and machine learning have changed the standard ROM methodology over the last two decades. Finally, we outline our vision on how state-of-the-art data-driven modeling can continue to reshape the field of reduced order modeling. △ Less

Submitted 23 August, 2021; v1 submitted 28 June, 2021; originally announced June 2021.

Deep neural network enabled corrective source term approach to hybrid analysis and modeling

Authors: Sindre Stenen Blakseth, Adil Rasheed, Trond Kvamsdal, Omer San

Abstract: In this work, we introduce, justify and demonstrate the Corrective Source Term Approach (CoSTA) -- a novel approach to Hybrid Analysis and Modeling (HAM). The objective of HAM is to combine physics-based modeling (PBM) and data-driven modeling (DDM) to create generalizable, trustworthy, accurate, computationally efficient and self-evolving models. CoSTA achieves this objective by augmenting the go… ▽ More In this work, we introduce, justify and demonstrate the Corrective Source Term Approach (CoSTA) -- a novel approach to Hybrid Analysis and Modeling (HAM). The objective of HAM is to combine physics-based modeling (PBM) and data-driven modeling (DDM) to create generalizable, trustworthy, accurate, computationally efficient and self-evolving models. CoSTA achieves this objective by augmenting the governing equation of a PBM model with a corrective source term generated using a deep neural network. In a series of numerical experiments on one-dimensional heat diffusion, CoSTA is found to outperform comparable DDM and PBM models in terms of accuracy -- often reducing predictive errors by several orders of magnitude -- while also generalizing better than pure DDM. Due to its flexible but solid theoretical foundation, CoSTA provides a modular framework for leveraging novel developments within both PBM and DDM. Its theoretical foundation also ensures that CoSTA can be used to model any system governed by (deterministic) partial differential equations. Moreover, CoSTA facilitates interpretation of the DNN-generated source term within the context of PBM, which results in improved explainability of the DNN. These factors make CoSTA a potential door-opener for data-driven techniques to enter high-stakes applications previously reserved for pure PBM. △ Less

Submitted 30 November, 2021; v1 submitted 24 May, 2021; originally announced May 2021.

Model fusion with physics-guided machine learning

Authors: Suraj Pawar, Omer San, Aditya Nair, Adil Rasheed, Trond Kvamsdal

Abstract: The unprecedented amount of data generated from experiments, field observations, and large-scale numerical simulations at a wide range of spatio-temporal scales have enabled the rapid advancement of data-driven and especially deep learning models in the field of fluid mechanics. Although these methods are proven successful for many applications, there is a grand challenge of improving their genera… ▽ More The unprecedented amount of data generated from experiments, field observations, and large-scale numerical simulations at a wide range of spatio-temporal scales have enabled the rapid advancement of data-driven and especially deep learning models in the field of fluid mechanics. Although these methods are proven successful for many applications, there is a grand challenge of improving their generalizability. This is particularly essential when data-driven models are employed within outer-loop applications like optimization. In this work, we put forth a physics-guided machine learning (PGML) framework that leverages the interpretable physics-based model with a deep learning model. The PGML framework is capable of enhancing the generalizability of data-driven models and effectively protect against or inform about the inaccurate predictions resulting from extrapolation. We apply the PGML framework as a novel model fusion approach combining the physics-based Galerkin projection model and long-short term memory (LSTM) network for parametric model order reduction of fluid flows. We demonstrate the improved generalizability of the PGML framework against a purely data-driven approach through the injection of physics features into intermediate LSTM layers. Our quantitative analysis shows that the overall model uncertainty can be reduced through the PGML approach especially for test data coming from a distribution different than the training data. Moreover, we demonstrate that our approach can be used as an inverse diagnostic tool providing a confidence score associated with models and observations. The proposed framework also allows for multi-fidelity computing by making use of low-fidelity models in the online deployment of quantified data-driven models. △ Less

Submitted 9 April, 2021; originally announced April 2021.

A nonintrusive hybrid neural-physics modeling of incomplete dynamical systems: Lorenz equations

Authors: Suraj Pawar, Omer San, Adil Rasheed, Ionel M. Navon

Abstract: This work presents a hybrid modeling approach to data-driven learning and representation of unknown physical processes and closure parameterizations. These hybrid models are suitable for situations where the mechanistic description of dynamics of some variables is unknown, but reasonably accurate observational data can be obtained for the evolution of the state of the system. In this work, we prop… ▽ More This work presents a hybrid modeling approach to data-driven learning and representation of unknown physical processes and closure parameterizations. These hybrid models are suitable for situations where the mechanistic description of dynamics of some variables is unknown, but reasonably accurate observational data can be obtained for the evolution of the state of the system. In this work, we propose machine learning to account for missing physics and then data assimilation to correct the prediction. In particular, we devise an effective methodology based on a recurrent neural network to model the unknown dynamics. A long short-term memory (LSTM) based correction term is added to the predictive model in order to take into account hidden physics. Since LSTM introduces a black-box approach for the unknown part of the model, we investigate whether the proposed hybrid neural-physical model can be further corrected through a sequential data assimilation step. We apply this framework to the weakly nonlinear Lorenz model that displays quasiperiodic oscillations, the highly nonlinear Lorenz model, and two-scale Lorenz model. The hybrid neural-physics model yields accurate results for the weakly nonlinear Lorenz model with the predicted state close to the true Lorenz model trajectory. For the highly nonlinear Lorenz model and the two-scale Lorenz model, the hybrid neural-physics model deviates from the true state due to the accumulation of prediction error from one time step to the next time step. The ensemble Kalman filter approach takes into account the prediction error and updates the diverged prediction using available observations in order to provide a more accurate state estimate. The successful synergistic integration of neural network and data assimilation for low-dimensional system shows the potential benefits of the proposed hybrid-neural physics model for complex systems. △ Less

Submitted 13 August, 2021; v1 submitted 31 March, 2021; originally announced April 2021.

Hybrid analysis and modeling, eclecticism, and multifidelity computing toward digital twin revolution

Authors: Omer San, Adil Rasheed, Trond Kvamsdal

Abstract: Most modeling approaches lie in either of the two categories: physics-based or data-driven. Recently, a third approach which is a combination of these deterministic and statistical models is emerging for scientific applications. To leverage these developments, our aim in this perspective paper is centered around exploring numerous principle concepts to address the challenges of (i) trustworthiness… ▽ More Most modeling approaches lie in either of the two categories: physics-based or data-driven. Recently, a third approach which is a combination of these deterministic and statistical models is emerging for scientific applications. To leverage these developments, our aim in this perspective paper is centered around exploring numerous principle concepts to address the challenges of (i) trustworthiness and generalizability in develo** data-driven models to shed light on understanding the fundamental trade-offs in their accuracy and efficiency, and (ii) seamless integration of interface learning and multifidelity coupling approaches that transfer and represent information between different entities, particularly when different scales are governed by different physics, each operating on a different level of abstraction. Addressing these challenges could enable the revolution of digital twin technologies for scientific and engineering applications. △ Less

Submitted 26 March, 2021; originally announced March 2021.

Hybrid analysis and modeling for next generation of digital twins

Authors: Suraj Pawar, Shady E. Ahmed, Omer San, Adil Rasheed

Abstract: The physics-based modeling has been the workhorse for many decades in many scientific and engineering applications ranging from wind power, weather forecasting, and aircraft design. Recently, data-driven models are increasingly becoming popular in many branches of science and engineering due to their non-intrusive nature and online learning capability. Despite the robust performance of data-driven… ▽ More The physics-based modeling has been the workhorse for many decades in many scientific and engineering applications ranging from wind power, weather forecasting, and aircraft design. Recently, data-driven models are increasingly becoming popular in many branches of science and engineering due to their non-intrusive nature and online learning capability. Despite the robust performance of data-driven models, they are faced with challenges of poor generalizability and difficulty in interpretation. These challenges have encouraged the integration of physics-based models with data-driven models, herein denoted hybrid analysis and modeling (HAM). We propose two different frameworks under the HAM paradigm for applications relevant to wind energy in order to bring the physical realism within emerging digital twin technologies. The physics-guided machine learning (PGML) framework reduces the uncertainty of neural network predictions by embedding physics-based features from a simplified model at intermediate layers and its performance is demonstrated for the aerodynamic force prediction task. Our results show that the proposed PGML framework achieves approximately 75\% reduction in uncertainty for smaller angle of attacks. The interface learning (IL) framework illustrates how different solvers can be coupled to produce a multi-fidelity model and is successfully applied for the Boussinesq equations that govern a broad class of transport processes. The IL approach paves the way for seamless integration of multi-scale, multi-physics and multi-fidelity models (M^3 models). △ Less

Submitted 14 January, 2021; originally announced January 2021.

Comments: arXiv admin note: text overlap with arXiv:2012.13343

Physics guided machine learning using simplified theories

Authors: Suraj Pawar, Omer San, Burak Aksoylu, Adil Rasheed, Trond Kvamsdal

Abstract: Recent applications of machine learning, in particular deep learning, motivate the need to address the generalizability of the statistical inference approaches in physical sciences. In this letter, we introduce a modular physics guided machine learning framework to improve the accuracy of such data-driven predictive engines. The chief idea in our approach is to augment the knowledge of the simplif… ▽ More Recent applications of machine learning, in particular deep learning, motivate the need to address the generalizability of the statistical inference approaches in physical sciences. In this letter, we introduce a modular physics guided machine learning framework to improve the accuracy of such data-driven predictive engines. The chief idea in our approach is to augment the knowledge of the simplified theories with the underlying learning process. To emphasise on their physical importance, our architecture consists of adding certain features at intermediate layers rather than in the input layer. To demonstrate our approach, we select a canonical airfoil aerodynamic problem with the enhancement of the potential flow theory. We include features obtained by a panel method that can be computed efficiently for an unseen configuration in our training procedure. By addressing the generalizability concerns, our results suggest that the proposed feature enhancement approach can be effectively used in many scientific machine learning applications, especially for the systems where we can use a theoretical, empirical, or simplified model to guide the learning module. △ Less

Submitted 18 December, 2020; originally announced December 2020.

Dynamics of Droplet Generation from Corneal Tear Film during Non-contact Eye Procedure in the Context of COVID-19

Authors: Durbar Roy, Sophia M, Abdur Rasheed, Prasenjit Kabi, Abhijit Sinha Roy, Rohit Shetty, Saptarshi Basu

Abstract: Non-invasive medical diagnostics demonstrate a propensity for droplet generation and should be studied to devise risk mitigation strategies against the spread of the SARS-CoV-2 virus. We investigate the air-puff tonometry, which uses a short-timed air-puff to applanate the human eye in a bid to detect the early onset of glaucoma by measuring the intraocular pressure. The air-puff consists of a vor… ▽ More Non-invasive medical diagnostics demonstrate a propensity for droplet generation and should be studied to devise risk mitigation strategies against the spread of the SARS-CoV-2 virus. We investigate the air-puff tonometry, which uses a short-timed air-puff to applanate the human eye in a bid to detect the early onset of glaucoma by measuring the intraocular pressure. The air-puff consists of a vortex trailed by a high-speed jet. High-speed imaging of the eye during a typical tonometry measurement reveals a sequence of events starting with the interaction between the tear layer and the air puff leading to an initial sheet ejection. It is immediately followed by the trailing jet applanating the central corneal section, causing capillary waves to form and interact with the highly 3D transient expanding sheet. Such interaction with the capillary waves and the surrounding airfield due to the trailing jet causes the expanding sheet to undergo bag breakup, finger formation by Rayleigh Taylor instability and further break up into subsequent droplets by Rayleigh Plateau instability, which eventually splashes onto nearby objects, potentially forming fomites or aerosols which can lead to infections. The complex spatiotemporal phenomenon is carefully documented by rigorous experiments and corroborated using comprehensive theoretical analyses. △ Less

Submitted 10 November, 2020; v1 submitted 2 November, 2020; originally announced November 2020.

A nudged hybrid analysis and modeling approach for realtime wake-vortex transport and decay prediction

Authors: Shady Ahmed, Suraj Pawar, Omer San, Adil Rasheed, Mandar Tabib

Abstract: We put forth a long short-term memory (LSTM) nudging framework for the enhancement of reduced order models (ROMs) of fluid flows utilizing noisy measurements for air traffic improvements. Toward emerging applications of digital twins in aviation, the proposed approach allows for constructing a realtime predictive tool for wake-vortex transport and decay systems. We build on the fact that in realis… ▽ More We put forth a long short-term memory (LSTM) nudging framework for the enhancement of reduced order models (ROMs) of fluid flows utilizing noisy measurements for air traffic improvements. Toward emerging applications of digital twins in aviation, the proposed approach allows for constructing a realtime predictive tool for wake-vortex transport and decay systems. We build on the fact that in realistic application, there are uncertainties in initial and boundary conditions, model parameters, as well as measurements. Moreover, conventional nonlinear ROMs based on Galerkin projection (GROMs) suffer from imperfection and solution instabilities, especially for advection-dominated flows with slow decay in the Kolmogorov width. In the presented LSTM nudging (LSTM-N) approach, we fuse forecasts from a combination of imperfect GROM and uncertain state estimates, with sparse Eulerian sensor measurements to provide more reliable predictions in a dynamical data assimilation framework. We illustrate our concept by solving a two-dimensional vorticity transport equation. We investigate the effects of measurements noise and state estimate uncertainty on the performance of the LSTM-N behavior. We also demonstrate that it can sufficiently handle different levels of temporal and spatial measurement sparsity, and offer a huge potential in develo** next-generation digital twin technologies. △ Less

Submitted 5 March, 2021; v1 submitted 5 August, 2020; originally announced August 2020.

Comments: arXiv admin note: substantial text overlap with arXiv:2005.14246

Multifidelity Computing for Coupling Full and Reduced Order Models

Authors: Shady E. Ahmed, Omer San, Kursat Kara, Rami Younis, Adil Rasheed

Abstract: Hybrid physics-machine learning models are increasingly being used in simulations of transport processes. Many complex multiphysics systems relevant to scientific and engineering applications include multiple spatiotemporal scales and comprise a multifidelity problem sharing an interface between various formulations or heterogeneous computational entities. To this end, we present a robust hybrid a… ▽ More Hybrid physics-machine learning models are increasingly being used in simulations of transport processes. Many complex multiphysics systems relevant to scientific and engineering applications include multiple spatiotemporal scales and comprise a multifidelity problem sharing an interface between various formulations or heterogeneous computational entities. To this end, we present a robust hybrid analysis and modeling approach combining a physics-based full order model (FOM) and a data-driven reduced order model (ROM) to form the building blocks of an integrated approach among mixed fidelity descriptions toward predictive digital twin technologies. At the interface, we introduce a long short-term memory network to bridge these high and low-fidelity models in various forms of interfacial error correction or prolongation. The proposed interface learning approaches are tested as a new way to address ROM-FOM coupling problems solving nonlinear advection-diffusion flow situations with a bifidelity setup that captures the essence of a broad class of transport processes. △ Less

Submitted 12 February, 2021; v1 submitted 13 July, 2020; originally announced July 2020.

Interface learning of multiphysics and multiscale systems

Authors: Shady E. Ahmed, Omer San, Kursat Kara, Rami Younis, Adil Rasheed

Abstract: Complex natural or engineered systems comprise multiple characteristic scales, multiple spatiotemporal domains, and even multiple physical closure laws. To address such challenges, we introduce an interface learning paradigm and put forth a data-driven closure approach based on memory embedding to provide physically correct boundary conditions at the interface. To enable the interface learning for… ▽ More Complex natural or engineered systems comprise multiple characteristic scales, multiple spatiotemporal domains, and even multiple physical closure laws. To address such challenges, we introduce an interface learning paradigm and put forth a data-driven closure approach based on memory embedding to provide physically correct boundary conditions at the interface. To enable the interface learning for hyperbolic systems by considering the domain of influence and wave structures into account, we put forth the concept of upwind learning towards a physics-informed domain decomposition. The promise of the proposed approach is shown for a set of canonical illustrative problems. We highlight that high-performance computing environments can benefit from this methodology to reduce communication costs among processing units in emerging machine learning ready heterogeneous platforms toward exascale era. △ Less

Submitted 31 October, 2020; v1 submitted 17 June, 2020; originally announced June 2020.

Journal ref: Phys. Rev. E 102, 053304 (2020)

Reduced order modeling of fluid flows: Machine learning, Kolmogorov barrier, closure modeling, and partitioning

Authors: Shady Ahmed, Suraj Pawar, Omer San, Adil Rasheed

Abstract: In this paper, we put forth a long short-term memory (LSTM) nudging framework for the enhancement of reduced order models (ROMs) of fluid flows utilizing noisy measurements. We build on the fact that in a realistic application, there are uncertainties in initial conditions, boundary conditions, model parameters, and/or field measurements. Moreover, conventional nonlinear ROMs based on Galerkin pro… ▽ More In this paper, we put forth a long short-term memory (LSTM) nudging framework for the enhancement of reduced order models (ROMs) of fluid flows utilizing noisy measurements. We build on the fact that in a realistic application, there are uncertainties in initial conditions, boundary conditions, model parameters, and/or field measurements. Moreover, conventional nonlinear ROMs based on Galerkin projection (GROMs) suffer from imperfection and solution instabilities due to the modal truncation, especially for advection-dominated flows with slow decay in the Kolmogorov width. In the presented LSTM-Nudge approach, we fuse forecasts from a combination of imperfect GROM and uncertain state estimates, with sparse Eulerian sensor measurements to provide more reliable predictions in a dynamical data assimilation framework. We illustrate the idea with the viscous Burgers problem, as a benchmark test bed with quadratic nonlinearity and Laplacian dissipation. We investigate the effects of measurements noise and state estimate uncertainty on the performance of the LSTM-Nudge behavior. We also demonstrate that it can sufficiently handle different levels of temporal and spatial measurement sparsity. This first step in our assessment of the proposed model shows that the LSTM nudging could represent a viable realtime predictive tool in emerging digital twin systems. △ Less

Submitted 28 May, 2020; originally announced May 2020.

Long short-term memory embedded nudging schemes for nonlinear data assimilation of geophysical flows

Authors: Suraj Pawar, Shady E. Ahmed, Omer San, Adil Rasheed, Ionel M. Navon

Abstract: Reduced rank nonlinear filters are increasingly utilized in data assimilation of geophysical flows, but often require a set of ensemble forward simulations to estimate forecast covariance. On the other hand, predictor-corrector type nudging approaches are still attractive due to their simplicity of implementation when more complex methods need to be avoided. However, optimal estimate of nudging ga… ▽ More Reduced rank nonlinear filters are increasingly utilized in data assimilation of geophysical flows, but often require a set of ensemble forward simulations to estimate forecast covariance. On the other hand, predictor-corrector type nudging approaches are still attractive due to their simplicity of implementation when more complex methods need to be avoided. However, optimal estimate of nudging gain matrix might be cumbersome. In this paper, we put forth a fully nonintrusive recurrent neural network approach based on a long short-term memory (LSTM) embedding architecture to estimate the nudging term, which plays a role not only to force the state trajectories to the observations but also acts as a stabilizer. Furthermore, our approach relies on the power of archival data and the trained model can be retrained effectively due to power of transfer learning in any neural network applications. In order to verify the feasibility of the proposed approach, we perform twin experiments using Lorenz 96 system. Our results demonstrate that the proposed LSTM nudging approach yields more accurate estimates than both extended Kalman filter (EKF) and ensemble Kalman filter (EnKF) when only sparse observations are available. With the availability of emerging AI-friendly and modular hardware technologies and heterogeneous computing platforms, we articulate that our simplistic nudging framework turns out to be computationally more efficient than either the EKF or EnKF approaches. △ Less

Submitted 22 May, 2020; originally announced May 2020.

Forward sensitivity approach for estimating eddy viscosity closures in nonlinear model reduction

Authors: Shady E. Ahmed, Kinjal Bhar, Omer San, Adil Rasheed

Abstract: In this paper, we propose a variational approach to estimate eddy viscosity using forward sensitivity method (FSM) for closure modeling in nonlinear reduced order models. FSM is a data assimilation technique that blends model's predictions with noisy observations to correct initial state and/or model parameters. We apply this approach on a projection based reduced order model (ROM) of the one-dime… ▽ More In this paper, we propose a variational approach to estimate eddy viscosity using forward sensitivity method (FSM) for closure modeling in nonlinear reduced order models. FSM is a data assimilation technique that blends model's predictions with noisy observations to correct initial state and/or model parameters. We apply this approach on a projection based reduced order model (ROM) of the one-dimensional viscous Burgers equation with a square wave defining a moving shock, and the two-dimensional vorticity transport equation formulating a decay of Kraichnan turbulence. We investigate the capability of the approach to approximate an optimal value for eddy viscosity with different measurement configurations. Specifically, we show that our approach can sufficiently assimilate information either through full field or sparse noisy measurements to estimate eddy viscosity closure to cure standard Galerkin reduced order model (GROM) predictions. Therefore, our approach provides a modular framework to correct forecasting error from a sparse observational network on a latent space. We highlight that the proposed GROM-FSM framework is promising for emerging digital twin applications, where real-time sensor measurements can be used to update and optimize surrogate model's parameters. △ Less

Submitted 4 November, 2020; v1 submitted 21 May, 2020; originally announced May 2020.

Journal ref: Phys. Rev. E 102, 043302 (2020)

A long short-term memory embedding for hybrid uplifted reduced order models

Authors: Shady E. Ahmed, Omer San, Adil Rasheed, Traian Iliescu

Abstract: In this paper, we introduce an uplifted reduced order modeling (UROM) approach through the integration of standard projection based methods with long short-term memory (LSTM) embedding. Our approach has three modeling layers or components. In the first layer, we utilize an intrusive projection approach to model dynamics represented by the largest modes. The second layer consists of an LSTM model t… ▽ More In this paper, we introduce an uplifted reduced order modeling (UROM) approach through the integration of standard projection based methods with long short-term memory (LSTM) embedding. Our approach has three modeling layers or components. In the first layer, we utilize an intrusive projection approach to model dynamics represented by the largest modes. The second layer consists of an LSTM model to account for residuals beyond this truncation. This closure layer refers to the process of including the residual effect of the discarded modes into the dynamics of the largest scales. However, the feasibility of generating a low rank approximation tails off for higher Kolmogorov $n$-width systems due to the underlying nonlinear processes. The third uplifting layer, called super-resolution, addresses this limited representation issue by expanding the span into a larger number of modes utilizing the versatility of LSTM. Therefore, our model integrates a physics-based projection model with a memory embedded LSTM closure and an LSTM based super-resolution model. In several applications, we exploit the use of Grassmann manifold to construct UROM for unseen conditions. We performed numerical experiments by using the Burgers and Navier-Stokes equations with quadratic nonlinearity. Our results show robustness of the proposed approach in building reduced order models for parameterized systems and confirm the improved trade-off between accuracy and efficiency. △ Less

Submitted 13 December, 2019; originally announced December 2019.

Feature engineering and symbolic regression methods for detecting hidden physics from sparse sensors

Authors: Harsha Vaddireddy, Adil Rasheed, Anne E Staples, Omer San

Abstract: In this study we put forth a modular approach for distilling hidden flow physics in discrete and sparse observations. To address functional expressiblity, a key limitation of the black-box machine learning methods, we have exploited the use of symbolic regression as a principle for identifying relations and operators that are related to the underlying processes. This approach combines evolutionary… ▽ More In this study we put forth a modular approach for distilling hidden flow physics in discrete and sparse observations. To address functional expressiblity, a key limitation of the black-box machine learning methods, we have exploited the use of symbolic regression as a principle for identifying relations and operators that are related to the underlying processes. This approach combines evolutionary computation with feature engineering to provide a tool to discover hidden parameterizations embedded in the trajectory of fluid flows in the Eulerian frame of reference. Our approach in this study mainly involves gene expression programming (GEP) and sequential threshold ridge regression (STRidge) algorithms. We demonstrate our results in three different applications: (i) equation discovery, (ii) truncation error analysis, and (iii) hidden physics discovery, for which we include both predicting unknown source terms from a set of sparse observations and discovering subgrid scale closure models. We illustrate that both GEP and STRidge algorithms are able to distill the Smagorinsky model from an array of tailored features in solving the Kraichnan turbulence problem. Our results demonstrate the huge potential of these techniques in complex physics problems, and reveal the importance of feature selection and feature engineering in model discovery approaches. △ Less

Submitted 9 January, 2020; v1 submitted 12 November, 2019; originally announced November 2019.

An evolve-then-correct reduced order model for hidden fluid dynamics

Authors: Suraj Pawar, Shady E. Ahmed, O. San, A. Rasheed

Abstract: In this paper, we put forth an evolve-then-correct reduced order modeling approach that combines intrusive and nonintrusive models to take hidden physical processes into account. Specifically, we split the underlying dynamics into known and unknown components. In the known part, we first utilize an intrusive Galerkin method projected on a set of basis functions obtained by proper orthogonal decomp… ▽ More In this paper, we put forth an evolve-then-correct reduced order modeling approach that combines intrusive and nonintrusive models to take hidden physical processes into account. Specifically, we split the underlying dynamics into known and unknown components. In the known part, we first utilize an intrusive Galerkin method projected on a set of basis functions obtained by proper orthogonal decomposition. We then formulate a recurrent neural network emulator based on the assumption that observed data is a manifestation of all relevant processes. We further enhance our approach by using an orthonormality conforming basis interpolation approach on a Grassmannian manifold to address off-design conditions. The proposed framework is illustrated here with the application of two-dimensional co-rotating vortex simulations under modeling uncertainty. The results demonstrate highly accurate predictions underlining the effectiveness of the evolve-then-correct approach toward realtime simulations, where the full process model is not known a priori. △ Less

Submitted 5 November, 2019; originally announced November 2019.

Data-driven recovery of hidden physics in reduced order modeling of fluid flows

Authors: Suraj Pawar, Shady E. Ahmed, Omer San, Adil Rasheed

Abstract: In this article, we introduce a modular hybrid analysis and modeling (HAM) approach to account for hidden physics in reduced order modeling (ROM) of parameterized systems relevant to fluid dynamics. The hybrid ROM framework is based on using the first principles to model the known physics in conjunction with utilizing the data-driven machine learning tools to model remaining residual that is hidde… ▽ More In this article, we introduce a modular hybrid analysis and modeling (HAM) approach to account for hidden physics in reduced order modeling (ROM) of parameterized systems relevant to fluid dynamics. The hybrid ROM framework is based on using the first principles to model the known physics in conjunction with utilizing the data-driven machine learning tools to model remaining residual that is hidden in data. This framework employs proper orthogonal decomposition as a compression tool to construct orthonormal bases and Galerkin projection (GP) as a model to built the dynamical core of the system. Our proposed methodology hence compensates structural or epistemic uncertainties in models and utilizes the observed data snapshots to compute true modal coefficients spanned by these bases. The GP model is then corrected at every time step with a data-driven rectification using a long short-term memory (LSTM) neural network architecture to incorporate hidden physics. A Grassmannian manifold approach is also adapted for interpolating basis functions to unseen parametric conditions. The control parameter governing the system's behavior is thus implicitly considered through true modal coefficients as input features to the LSTM network. The effectiveness of the HAM approach is discussed through illustrative examples that are generated synthetically to take hidden physics into account. Our approach thus provides insights addressing a fundamental limitation of the physics-based models when the governing equations are incomplete to represent underlying physical processes. △ Less

Submitted 30 October, 2019; originally announced October 2019.

Memory embedded non-intrusive reduced order modeling of non-ergodic flows

Authors: Shady E. Ahmed, Sk. Mashfiqur Rahman, Omer San, Adil Rasheed, Ionel M. Navon

Abstract: Generating a digital twin of any complex system requires modeling and computational approaches that are efficient, accurate, and modular. Traditional reduced order modeling techniques are targeted at only the first two but the novel non-intrusive approach presented in this study is an attempt at taking all three into account effectively compared to their traditional counterparts. Based on dimensio… ▽ More Generating a digital twin of any complex system requires modeling and computational approaches that are efficient, accurate, and modular. Traditional reduced order modeling techniques are targeted at only the first two but the novel non-intrusive approach presented in this study is an attempt at taking all three into account effectively compared to their traditional counterparts. Based on dimensionality reduction using proper orthogonal decomposition (POD), we introduce a long short-term memory (LSTM) neural network architecture together with a principal interval decomposition (PID) framework as an enabler to account for localized modal deformation, which is a key element in accurate reduced order modeling of convective flows. Our applications for convection dominated systems governed by Burgers, Navier-Stokes, and Boussinesq equations demonstrate that the proposed approach yields significantly more accurate predictions than the POD-Galerkin method, and could be a key enabler towards near real-time predictions of unsteady flows. △ Less

Submitted 16 October, 2019; originally announced October 2019.

A priori analysis on deep learning of subgrid-scale parameterizations for Kraichnan turbulence

Authors: Suraj Pawar, Omer San, Adil Rasheed, Prakash Vedula

Abstract: In the present study, we investigate different data-driven parameterizations for large eddy simulation of two-dimensional turbulence in the \emph{a priori} settings. These models utilize resolved flow field variables on the coarser grid to estimate the subgrid-scale stresses. We use data-driven closure models based on localized learning that employs multilayer feedforward artificial neural network… ▽ More In the present study, we investigate different data-driven parameterizations for large eddy simulation of two-dimensional turbulence in the \emph{a priori} settings. These models utilize resolved flow field variables on the coarser grid to estimate the subgrid-scale stresses. We use data-driven closure models based on localized learning that employs multilayer feedforward artificial neural network (ANN) with point-to-point map** and neighboring stencil data map**, and convolutional neural network (CNN) fed by data snapshots of the whole domain. The performance of these data-driven closure models is measured through a probability density function and is compared with the dynamic Smagorinksy model (DSM). The quantitative performance is evaluated using the cross-correlation coefficient between the true and predicted stresses. We analyze different frameworks in terms of the amount of training data, selection of input and output features, their characteristics in modeling with accuracy, and training and deployment computational time. We also demonstrate computational gain that can be achieved using the intelligent eddy viscosity model that learns eddy viscosity computed by the DSM instead of subgrid-scale stresses. We detail the hyperparameters optimization of these models using the grid search algorithm. △ Less

Submitted 15 October, 2019; originally announced October 2019.

A deep learning enabler for non-intrusive reduced order modeling of fluid flows

Authors: S. Pawar, S. M. Rahman, H. Vaddireddy, O. San, A. Rasheed, P. Vedula

Abstract: In this paper, we introduce a modular deep neural network (DNN) framework for data-driven reduced order modeling of dynamical systems relevant to fluid flows. We propose various deep neural network architectures which numerically predict evolution of dynamical systems by learning from either using discrete state or slope information of the system. Our approach has been demonstrated using both resi… ▽ More In this paper, we introduce a modular deep neural network (DNN) framework for data-driven reduced order modeling of dynamical systems relevant to fluid flows. We propose various deep neural network architectures which numerically predict evolution of dynamical systems by learning from either using discrete state or slope information of the system. Our approach has been demonstrated using both residual formula and backward difference scheme formulas. However, it can be easily generalized into many different numerical schemes as well. We give a demonstration of our framework for three examples: (i) Kraichnan-Orszag system, an illustrative coupled nonlinear ordinary differential equations, (ii) Lorenz system exhibiting chaotic behavior, and (iii) a non-intrusive model order reduction framework for the two-dimensional Boussinesq equations with a differentially heated cavity flow setup at various Rayleigh numbers. Using only snapshots of state variables at discrete time instances, our data-driven approach can be considered truly non-intrusive, since any prior information about the underlying governing equations is not required for generating the reduced order model. Our \textit{a posteriori} analysis shows that the proposed data-driven approach is remarkably accurate, and can be used as a robust predictive tool for non-intrusive model order reduction of complex fluid flows. △ Less

Submitted 10 July, 2019; originally announced July 2019.

Comments: 36 pages, 21 figures

A non-intrusive reduced order modeling framework for quasi-geostrophic turbulence

Authors: Sk. Mashfiqur Rahman, Suraj Pawar, Omer San, Adil Rasheed, Traian Iliescu

Abstract: In this study, we present a non-intrusive reduced order modeling (ROM) framework for large-scale quasi-stationary systems. The framework proposed herein exploits the time series prediction capability of long short-term memory (LSTM) recurrent neural network such that: (i) in the training phase, the LSTM model is trained on the modal coefficients extracted from the high-resolution data using proper… ▽ More In this study, we present a non-intrusive reduced order modeling (ROM) framework for large-scale quasi-stationary systems. The framework proposed herein exploits the time series prediction capability of long short-term memory (LSTM) recurrent neural network such that: (i) in the training phase, the LSTM model is trained on the modal coefficients extracted from the high-resolution data using proper orthogonal decomposition (POD) transform, and (ii) in the testing phase, the trained model predicts the modal coefficients for the total time recursively based on the initial time history. To illustrate the predictive performance of the proposed framework, the mean flow fields and time series response of the field values are reconstructed from the predicted modal coefficients by using an inverse POD transform. As a representative benchmark test case, we consider a two-dimensional quasi-geostrophic (QG) ocean circulation model which, in general, displays an enormous range of fluctuating spatial and temporal scales. We first illustrate that the conventional Galerkin projection based ROM of such systems requires a high number of POD modes to obtain a stable flow physics. In addition, ROM-GP does not seem to capture the intermittent bursts appearing in the dynamics of the first few most energetic modes. However, the proposed non-intrusive ROM framework based on LSTM (ROM-LSTM) yields a stable solution even for a small number of POD modes. We also observe that the ROM-LSTM model is able to capture quasi-periodic intermittent bursts accurately, and yields a stable and accurate mean flow dynamics using the time history of a few previous time states, denoted as the lookback time-window in this paper. Our findings suggest that the proposed ROM framework is capable of predicting noisy nonlinear fluid flows in an extremely efficient way compared to the conventional projection based ROM. △ Less

Submitted 23 June, 2019; originally announced June 2019.

Journal ref: Phys. Rev. E 100, 053306 (2019)

Data-driven deconvolution for large eddy simulations of Kraichnan turbulence

Authors: Romit Maulik, Omer San, Adil Rasheed, Prakash Vedula

Abstract: In this article, we demonstrate the use of artificial neural networks as optimal maps which are utilized for convolution and deconvolution of coarse-grained fields to account for sub-grid scale turbulence effects. We demonstrate that an effective eddy-viscosity is predicted by our purely data-driven large eddy simulation framework without explicit utilization of phenomenological arguments. In addi… ▽ More In this article, we demonstrate the use of artificial neural networks as optimal maps which are utilized for convolution and deconvolution of coarse-grained fields to account for sub-grid scale turbulence effects. We demonstrate that an effective eddy-viscosity is predicted by our purely data-driven large eddy simulation framework without explicit utilization of phenomenological arguments. In addition, our data-driven framework precludes the knowledge of true sub-grid stress information during the training phase due to its focus on estimating an effective filter and its inverse so that grid-resolved variables may be related to direct numerical simulation data statistically. The proposed predictive framework is also combined with a statistical truncation mechanism for ensuring numerical realizability in an explicit formulation. Through this we seek to unite structural and functional modeling strategies for modeling non-linear partial differential equations using reduced degrees of freedom. Both a priori and a posteriori results are shown for a two-dimensional decaying turbulence case in addition to a detailed description of validation and testing. A hyperparameter sensitivity study also shows that the proposed dual network framework simplifies learning complexity and is viable with exceedingly simple network architectures. Our findings indicate that the proposed framework approximates a robust and stable sub-grid closure which compares favorably to the Smagorinsky and Leith hypotheses for capturing the theoretical $k^{-3}$ scaling in Kraichnan turbulence. △ Less

Submitted 5 December, 2018; originally announced December 2018.

Sub-grid modelling for two-dimensional turbulence using neural networks

Authors: Romit Maulik, Omer San, Adil Rasheed, Prakash Vedula

Abstract: In this investigation, a data-driven turbulence closure framework is introduced and deployed for the sub-grid modelling of Kraichnan turbulence. The novelty of the proposed method lies in the fact that snapshots from high-fidelity numerical data are used to inform artificial neural networks for predicting the turbulence source term through localized grid-resolved information. In particular, our pr… ▽ More In this investigation, a data-driven turbulence closure framework is introduced and deployed for the sub-grid modelling of Kraichnan turbulence. The novelty of the proposed method lies in the fact that snapshots from high-fidelity numerical data are used to inform artificial neural networks for predicting the turbulence source term through localized grid-resolved information. In particular, our proposed methodology successfully establishes a map between inputs given by stencils of the vorticity and the streamfunction along with information from two well-known eddy-viscosity kernels. Through this we predict the sub-grid vorticity forcing in a temporally and spatially dynamic fashion. Our study is both a-priori and a-posteriori in nature. In the former, we present an extensive hyper-parameter optimization analysis in addition to learning quantification through probability density function based validation of sub-grid predictions. In the latter, we analyse the performance of our framework for flow evolution in a classical decaying two-dimensional turbulence test case in the presence of errors related to temporal and spatial discretization. Statistical assessments in the form of angle-averaged kinetic energy spectra demonstrate the promise of the proposed methodology for sub-grid quantity inference. In addition, it is also observed that some measure of a-posteriori error must be considered during optimal model selection for greater accuracy. The results in this article thus represent a promising development in the formalization of a framework for generation of heuristic-free turbulence closures from data. △ Less

Submitted 8 August, 2018; originally announced August 2018.