Showing 1–9 of 9 results for author: Bettencourt, J

Forecasting Black Sigatoka Infection Risks with Latent Neural ODEs

Authors: Yuchen Wang, Matthieu Chan Chee, Ziyad Edher, Minh Duc Hoang, Shion Fujimori, Sornnujah Kathirgamanathan, Jesse Bettencourt

Abstract: Black Sigatoka disease severely decreases global banana production, and climate change aggravates the problem by altering fungal species distributions. Due to the heavy financial burden of managing this infectious disease, farmers in develo** countries face significant banana crop losses. Though scientists have produced mathematical models of infectious diseases, adapting these models to incorpo… ▽ More Black Sigatoka disease severely decreases global banana production, and climate change aggravates the problem by altering fungal species distributions. Due to the heavy financial burden of managing this infectious disease, farmers in develo** countries face significant banana crop losses. Though scientists have produced mathematical models of infectious diseases, adapting these models to incorporate climate effects is difficult. We present MR. NODE (Multiple predictoR Neural ODE), a neural network that models the dynamics of black Sigatoka infection learnt directly from data via Neural Ordinary Differential Equations. Our method encodes external predictor factors into the latent space in addition to the variable that we infer, and it can also predict the infection risk at an arbitrary point in time. Empirically, we demonstrate on historical climate data that our method has superior generalization performance on time points up to one month in the future and unseen irregularities. We believe that our method can be a useful tool to control the spread of black Sigatoka. △ Less

Submitted 10 January, 2021; v1 submitted 1 December, 2020; originally announced December 2020.

Learning Differential Equations that are Easy to Solve

Authors: Jacob Kelly, Jesse Bettencourt, Matthew James Johnson, David Duvenaud

Abstract: Differential equations parameterized by neural networks become expensive to solve numerically as training progresses. We propose a remedy that encourages learned dynamics to be easier to solve. Specifically, we introduce a differentiable surrogate for the time cost of standard numerical solvers, using higher-order derivatives of solution trajectories. These derivatives are efficient to compute wit… ▽ More Differential equations parameterized by neural networks become expensive to solve numerically as training progresses. We propose a remedy that encourages learned dynamics to be easier to solve. Specifically, we introduce a differentiable surrogate for the time cost of standard numerical solvers, using higher-order derivatives of solution trajectories. These derivatives are efficient to compute with Taylor-mode automatic differentiation. Optimizing this additional objective trades model performance against the time cost of solving the learned dynamics. We demonstrate our approach by training substantially faster, while nearly as accurate, models in supervised classification, density estimation, and time-series modelling tasks. △ Less

Submitted 22 October, 2020; v1 submitted 8 July, 2020; originally announced July 2020.

DiffEqFlux.jl - A Julia Library for Neural Differential Equations

Authors: Chris Rackauckas, Mike Innes, Yingbo Ma, Jesse Bettencourt, Lyndon White, Vaibhav Dixit

Abstract: DiffEqFlux.jl is a library for fusing neural networks and differential equations. In this work we describe differential equations from the viewpoint of data science and discuss the complementary nature between machine learning models and differential equations. We demonstrate the ability to incorporate DifferentialEquations.jl-defined differential equation problems into a Flux-defined neural netwo… ▽ More DiffEqFlux.jl is a library for fusing neural networks and differential equations. In this work we describe differential equations from the viewpoint of data science and discuss the complementary nature between machine learning models and differential equations. We demonstrate the ability to incorporate DifferentialEquations.jl-defined differential equation problems into a Flux-defined neural network, and vice versa. The advantages of being able to use the entire DifferentialEquations.jl suite for this purpose is demonstrated by counter examples where simple integration strategies fail, but the sophisticated integration strategies provided by the DifferentialEquations.jl library succeed. This is followed by a demonstration of delay differential equations and stochastic differential equations inside of neural networks. We show high-level functionality for defining neural ordinary differential equations (neural networks embedded into the differential equation) and describe the extra models in the Flux model zoo which includes neural stochastic differential equations. We conclude by discussing the various adjoint methods used for backpropogation of the differential equation solvers. DiffEqFlux.jl is an important contribution to the area, as it allows the full weight of the differential equation solvers developed from decades of research in the scientific computing field to be readily applied to the challenges posed by machine learning and data science. △ Less

Submitted 6 February, 2019; originally announced February 2019.

Comments: Julialang Blog post, DiffEqFlux.jl

FFJORD: Free-form Continuous Dynamics for Scalable Reversible Generative Models

Authors: Will Grathwohl, Ricky T. Q. Chen, Jesse Bettencourt, Ilya Sutskever, David Duvenaud

Abstract: A promising class of generative models maps points from a simple distribution to a complex distribution through an invertible neural network. Likelihood-based training of these models requires restricting their architectures to allow cheap computation of Jacobian determinants. Alternatively, the Jacobian trace can be used if the transformation is specified by an ordinary differential equation. In… ▽ More A promising class of generative models maps points from a simple distribution to a complex distribution through an invertible neural network. Likelihood-based training of these models requires restricting their architectures to allow cheap computation of Jacobian determinants. Alternatively, the Jacobian trace can be used if the transformation is specified by an ordinary differential equation. In this paper, we use Hutchinson's trace estimator to give a scalable unbiased estimate of the log-density. The result is a continuous-time invertible generative model with unbiased density estimation and one-pass sampling, while allowing unrestricted neural network architectures. We demonstrate our approach on high-dimensional density estimation, image generation, and variational inference, achieving the state-of-the-art among exact likelihood methods with efficient sampling. △ Less

Submitted 22 October, 2018; v1 submitted 2 October, 2018; originally announced October 2018.

Comments: 8 Pages, 6 figures

Neural Ordinary Differential Equations

Authors: Ricky T. Q. Chen, Yulia Rubanova, Jesse Bettencourt, David Duvenaud

Abstract: We introduce a new family of deep neural network models. Instead of specifying a discrete sequence of hidden layers, we parameterize the derivative of the hidden state using a neural network. The output of the network is computed using a black-box differential equation solver. These continuous-depth models have constant memory cost, adapt their evaluation strategy to each input, and can explicitly… ▽ More We introduce a new family of deep neural network models. Instead of specifying a discrete sequence of hidden layers, we parameterize the derivative of the hidden state using a neural network. The output of the network is computed using a black-box differential equation solver. These continuous-depth models have constant memory cost, adapt their evaluation strategy to each input, and can explicitly trade numerical precision for speed. We demonstrate these properties in continuous-depth residual networks and continuous-time latent variable models. We also construct continuous normalizing flows, a generative model that can train by maximum likelihood, without partitioning or ordering the data dimensions. For training, we show how to scalably backpropagate through any ODE solver, without access to its internal operations. This allows end-to-end training of ODEs within larger models. △ Less

Submitted 13 December, 2019; v1 submitted 19 June, 2018; originally announced June 2018.

Characterization of the structure and cross-shore transport properties of a coastal upwelling filament using three-dimensional finite-size Lyapunov exponents

Authors: Joao H. Bettencourt, Vincent Rossi, Emilio Hernandez-Garcia, Martinho Marta-Almeida, Cristobal Lopez

Abstract: The three dimensional structure, dynamics and dispersion characteristics of a simulated upwelling filament in the Iberian upwelling system are analyzed using Lagrangian tools. We used a realistic regional simulation of the western Iberian shelf which is concomitant with an in-situ oceanographic campaign that surveyed the area. We compute 3d fields of finite--size Lyapunov exponents (FSLE) from 3d… ▽ More The three dimensional structure, dynamics and dispersion characteristics of a simulated upwelling filament in the Iberian upwelling system are analyzed using Lagrangian tools. We used a realistic regional simulation of the western Iberian shelf which is concomitant with an in-situ oceanographic campaign that surveyed the area. We compute 3d fields of finite--size Lyapunov exponents (FSLE) from 3d velocity fields and extract the field's ridges to study the spatial distribution and temporal evolution of the Lagrangian Coherent Structures (LCSs) evolving around the filament. We find that the most intense curtain-like LCSs delimit the boundaries of the whole filamentary structure whose general properties match well the observations. The filament interior is characterized by small dispersion of fluid elements. Furthermore, we identify a weak LCS separating the filament into a warmer vein and a colder filament associated with the interaction of a mesoscale eddy with the upwelling front. The cold upwelled water parcels move along the filament conserving their density. The filament itself is characterized by small dispersion of fluid elements in its interior. The comparison of LCSs with potential temperature and salinity gradient fields shows that the outer limits of the filament coincide with regions of large hydrographic gradients, similar to those observed, explaining the isolation of the interior of the filament with the surrounding waters. We conclude that the Lagrangian analysis used in this work is useful in explaining the dynamics of cross-shore exchanges of materials between coastal regions and the open ocean due to mesoscale processes. △ Less

Submitted 31 July, 2017; originally announced July 2017.

Comments: 24 pages, 11 figures

Journal ref: Journal of Geophysical Research: Oceans 122, 7433-7448 (2017)

Boundaries of the Peruvian Oxygen Minimum Zone shaped by coherent mesoscale dynamics

Authors: João H. Bettencourt, Cristóbal López, Emilio Hernández García, Ivonne Montes, Joël Sudre, Boris Dewitte, Aurélien Paulmier, Véronique Garçon

Abstract: Dissolved oxygen in sea water is a major factor affecting marine habitats and biogeochemical cycles. Oceanic zones with oxygen deficits represent significant portions of the area and volume of the oceans and are thought to be expanding. The Peruvian oxygen minimum zone is one of the most pronounced and lies in a region of strong mesoscale activity in the form of vortices and frontal regions, whose… ▽ More Dissolved oxygen in sea water is a major factor affecting marine habitats and biogeochemical cycles. Oceanic zones with oxygen deficits represent significant portions of the area and volume of the oceans and are thought to be expanding. The Peruvian oxygen minimum zone is one of the most pronounced and lies in a region of strong mesoscale activity in the form of vortices and frontal regions, whose effect in the dynamics of the oxygen minimum zone is largely unknown. Here, we study this issue from a modeling approach and a Lagrangian point of view, using a coupled physical-biogeochemical simulation of the Peruvian oxygen minimum zone and finite-size Lyapunov exponent fields to understand the link between mesoscale dynamics and oxygen variations. Our results show that, at depths between 380 and 600 meters, mesoscale structures have a relevant dual role. First, their mean positions and paths delimit and maintain the oxygen minimum zone boundaries. Second, their high frequency fluctuations entrain oxygen across these boundaries as eddy fluxes that point towards the interior of the oxygen minimum zone and are one order of magnitude larger than mean fluxes. We conclude that these eddy fluxes contribute to the ventilation of the oxygen minimum zone. △ Less

Submitted 13 June, 2015; originally announced June 2015.

Characterization of coherent structures in three-dimensional flows using the finite-size Lyapunov exponent

Authors: João H Bettencourt, Cristóbal López, Emilio Hernández-García

Abstract: In this paper we use the finite size Lyapunov Exponent (FSLE) to characterize Lagrangian coherent structures in three-dimensional (3d) turbulent flows. Lagrangian coherent structures act as the organizers of transport in fluid flows and are crucial to understand their stirring and mixing properties. Generalized maxima (ridges) of the FSLE fields are used to locate these coherent structures. Thre… ▽ More In this paper we use the finite size Lyapunov Exponent (FSLE) to characterize Lagrangian coherent structures in three-dimensional (3d) turbulent flows. Lagrangian coherent structures act as the organizers of transport in fluid flows and are crucial to understand their stirring and mixing properties. Generalized maxima (ridges) of the FSLE fields are used to locate these coherent structures. Three-dimensional FSLE fields are calculated in two phenomenologically distinct turbulent flows: a wall-bounded flow (channel flow) and a regional oceanic flow obtained by numerical solution of the primitive equations where two-dimensional turbulence dominates. In the channel flow, autocorrelations of the FSLE field show that the structure is substantially different from the near wall to the mid-channel region and relates well to the more widely studied Eulerian coherent structure of the turbulent channel flow. The ridges of the FSLE field have complex shapes due to the 3d character of the turbulent fluctuations. In the oceanic flow, strong horizontal stirring is present and the flow regime is similar to that of 2d turbulence where the domain is populated by coherent eddies that interact strongly. This in turn results in the presence of high FSLE lines throughout the domain leading to strong non-local mixing. The ridges of the FSLE field are quasi-vertical surfaces, indicating that the horizontal dynamics dominates the flow. Indeed, due to rotation and stratification, vertical motions in the ocean are much less intense than horizontal ones. This suppression is absent in the channel flow, as the 3d character of the FSLE ridges shows. △ Less

Submitted 11 June, 2013; v1 submitted 9 July, 2012; originally announced July 2012.

Journal ref: João H Bettencourt et al 2013 J. Phys. A: Math. Theor. 46 254022

Oceanic three-dimensional Lagrangian Coherent Structures: A study of a mesoscale eddy in the Benguela ocean region

Authors: João H. Bettencourt, Cristóbal López, Emilio Hernández-García

Abstract: We study three dimensional oceanic Lagrangian Coherent Structures (LCSs) in the Benguela region, as obtained from an output of the ROMS model. To do that we first compute Finite-Size Lyapunov exponent (FSLE) fields in the region volume, characterizing mesoscale stirring and mixing. Average FSLE values show a general decreasing trend with depth, but there is a local maximum at about 100 m depth. LC… ▽ More We study three dimensional oceanic Lagrangian Coherent Structures (LCSs) in the Benguela region, as obtained from an output of the ROMS model. To do that we first compute Finite-Size Lyapunov exponent (FSLE) fields in the region volume, characterizing mesoscale stirring and mixing. Average FSLE values show a general decreasing trend with depth, but there is a local maximum at about 100 m depth. LCSs are extracted as ridges of the calculated FSLE fields. They present a "curtain-like" geometry in which the strongest attracting and repelling structures appear as quasivertical surfaces. LCSs around a particular cyclonic eddy, pinched off from the upwelling front are also calculated. The LCSs are confirmed to provide pathways and barriers to transport in and out of the eddy. △ Less

Submitted 9 July, 2012; v1 submitted 16 November, 2011; originally announced November 2011.

Journal ref: Ocean Modelling 51, 73-83 (2012)