-
Uniform Transformation: Refining Latent Representation in Variational Autoencoders
Authors:
Ye Shi,
C. S. George Lee
Abstract:
Irregular distribution in latent space causes posterior collapse, misalignment between posterior and prior, and ill-sampling problem in Variational Autoencoders (VAEs). In this paper, we introduce a novel adaptable three-stage Uniform Transformation (UT) module -- Gaussian Kernel Density Estimation (G-KDE) clustering, non-parametric Gaussian Mixture (GM) Modeling, and Probability Integral Transfor…
▽ More
Irregular distribution in latent space causes posterior collapse, misalignment between posterior and prior, and ill-sampling problem in Variational Autoencoders (VAEs). In this paper, we introduce a novel adaptable three-stage Uniform Transformation (UT) module -- Gaussian Kernel Density Estimation (G-KDE) clustering, non-parametric Gaussian Mixture (GM) Modeling, and Probability Integral Transform (PIT) -- to address irregular latent distributions. By reconfiguring irregular distributions into a uniform distribution in the latent space, our approach significantly enhances the disentanglement and interpretability of latent representations, overcoming the limitation of traditional VAE models in capturing complex data structures. Empirical evaluations demonstrated the efficacy of our proposed UT module in improving disentanglement metrics across benchmark datasets -- dSprites and MNIST. Our findings suggest a promising direction for advancing representation learning techniques, with implication for future research in extending this framework to more sophisticated datasets and downstream tasks.
△ Less
Submitted 2 July, 2024;
originally announced July 2024.
-
Multilevel well modeling in aggregation-based nonlinear multigrid for multiphase flow in porous media
Authors:
Chak Shing Lee,
François P. Hamon,
Nicola Castelletto,
Panayot S. Vassilevski,
Joshua A. White
Abstract:
A full approximation scheme (FAS) nonlinear multigrid solver for two-phase flow and transport problems driven by wells with multiple perforations is developed. It is an extension to our previous work on FAS solvers for diffusion and transport problems. The solver is applicable to discrete problems defined on unstructured grids as the coarsening algorithm is aggregation-based and algebraic. To cons…
▽ More
A full approximation scheme (FAS) nonlinear multigrid solver for two-phase flow and transport problems driven by wells with multiple perforations is developed. It is an extension to our previous work on FAS solvers for diffusion and transport problems. The solver is applicable to discrete problems defined on unstructured grids as the coarsening algorithm is aggregation-based and algebraic. To construct coarse basis that can better capture the radial flow near wells, coarse grids in which perforated well cells are not near the coarse-element interface are desired. This is achieved by an aggregation algorithm proposed in this paper that makes use of the location of well cells in the cell-connectivity graph. Numerical examples in which the FAS solver is compared against Newton's method on benchmark problems are given. In particular, for a refined version of the SAIGUP model, the FAS solver is at least 35% faster than Newton's method for time steps with a CFL number greater than 10.
△ Less
Submitted 31 July, 2023;
originally announced August 2023.
-
Regularization of the inverse Laplace transform by Mollification
Authors:
Pierre Maréchal,
Faouzi Triki,
Walter C. Simo Tao Lee
Abstract:
In this paper we study the inverse Laplace transform. We first derive a new global logarithmic stability estimate that shows that the inversion is severely ill-posed. Then we propose a regularization method to compute the inverse Laplace transform using the concept of mollification. Taking into account the exponential instability we derive a criterion for selection of the regularization parameter.…
▽ More
In this paper we study the inverse Laplace transform. We first derive a new global logarithmic stability estimate that shows that the inversion is severely ill-posed. Then we propose a regularization method to compute the inverse Laplace transform using the concept of mollification. Taking into account the exponential instability we derive a criterion for selection of the regularization parameter. We show that by taking the optimal value of this parameter we improve significantly the convergence of the method. Finally, making use of the holomorphic extension of the Laplace transform, we suggest a new PDEs based numerical method for the computation of the solution. The effectiveness of the proposed regularization method is demonstrated through several numerical examples.
△ Less
Submitted 17 April, 2023;
originally announced April 2023.
-
A variational technique of mollification applied to backward heat conduction problems
Authors:
Walter C. Simo Tao Lee
Abstract:
This paper addresses a backward heat conduction problem with fractional Laplacian and time-dependent coefficient in an unbounded domain. The problem models generalized diffusion processes and is well-known to be severely ill-posed. We investigate a simple and powerful variational regularization technique based on mollification. Under classical Sobolev smoothness conditions, we derive order-optimal…
▽ More
This paper addresses a backward heat conduction problem with fractional Laplacian and time-dependent coefficient in an unbounded domain. The problem models generalized diffusion processes and is well-known to be severely ill-posed. We investigate a simple and powerful variational regularization technique based on mollification. Under classical Sobolev smoothness conditions, we derive order-optimal convergence rates between the exact solution and regularized approximation in the practical case where both the data and the operator are noisy. Moreover, we propose an order-optimal a-posteriori parameter choice rule based on the Morozov principle. Finally, we illustrate the robustness and efficiency of the regularization technique by some numerical examples including image deblurring.
△ Less
Submitted 21 February, 2022;
originally announced February 2022.
-
An Aggregation-based Nonlinear Multigrid Solver for Two-phase Flow and Transport in Porous Media
Authors:
Chak Shing Lee,
François P. Hamon,
Nicola Castelletto,
Panayot S. Vassilevski,
Joshua A. White
Abstract:
A nonlinear multigrid solver for two-phase flow and transport in a mixed fractional-flow velocity-pressure-saturation formulation is proposed. The solver, which is under the framework of the full approximation scheme (FAS), extends our previous work on nonlinear multigrid for heterogeneous diffusion problems. The coarse spaces in the multigrid hierarchy are constructed by first aggregating degrees…
▽ More
A nonlinear multigrid solver for two-phase flow and transport in a mixed fractional-flow velocity-pressure-saturation formulation is proposed. The solver, which is under the framework of the full approximation scheme (FAS), extends our previous work on nonlinear multigrid for heterogeneous diffusion problems. The coarse spaces in the multigrid hierarchy are constructed by first aggregating degrees of freedom, and then solving some local flow problems. The mixed formulation and the choice of coarse spaces allow us to assemble the coarse problems without visiting finer levels during the solving phase, which is crucial for the scalability of multigrid methods. Specifically, a natural generalization of the upwind flux can be evaluated directly on coarse levels using the precomputed coarse flux basis vectors. The resulting solver is applicable to problems discretized on general unstructured grids. The performance of the proposed nonlinear multigrid solver in comparison with the standard single level Newton's method is demonstrated through challenging numerical examples. It is observed that the proposed solver is robust for highly nonlinear problems and clearly outperforms Newton's method in the case of high Courant-Friedrichs-Lewy (CFL) numbers.
△ Less
Submitted 15 September, 2021;
originally announced September 2021.
-
Nonlinear multigrid based on local spectral coarsening for heterogeneous diffusion problems
Authors:
Chak Shing Lee,
François Hamon,
Nicola Castelletto,
Panayot S. Vassilevski,
Joshua White
Abstract:
This work develops a nonlinear multigrid method for diffusion problems discretized by cell-centered finite volume methods on general unstructured grids. The multigrid hierarchy is constructed algebraically using aggregation of degrees of freedom and spectral decomposition of reference linear operators associated with the aggregates. For rapid convergence, it is important that the resulting coarse…
▽ More
This work develops a nonlinear multigrid method for diffusion problems discretized by cell-centered finite volume methods on general unstructured grids. The multigrid hierarchy is constructed algebraically using aggregation of degrees of freedom and spectral decomposition of reference linear operators associated with the aggregates. For rapid convergence, it is important that the resulting coarse spaces have good approximation properties. In our approach, the approximation quality can be directly improved by including more spectral degrees of freedom in the coarsening process. Further, by exploiting local coarsening and a piecewise-constant approximation when evaluating the nonlinear component, the coarse level problems are assembled and solved without ever re-visiting the fine level, an essential element for multigrid algorithms to achieve optimal scalability. Numerical examples comparing relative performance of the proposed nonlinear multigrid solvers with standard single-level approaches -- Picard's and Newton's methods -- are presented. Results show that the proposed solver consistently outperforms the single-level methods, both in efficiency and robustness.
△ Less
Submitted 4 May, 2020;
originally announced May 2020.
-
Regularization of linear and nonlinear ill-posed problems by mollification
Authors:
Walter Cedric Simo Tao Lee
Abstract:
In this paper, we address the problem of approximating solutions of ill-posed problems using mollification. We quickly review existing mollification regularization methods and provide two new approximate solutions to a general ill-posed equation $T(f) =g$ where $T$ can be nonlinear. The regularized solutions we define extend the work of Bonnefond and Maréchal \cite{xapi}, and trace their origins i…
▽ More
In this paper, we address the problem of approximating solutions of ill-posed problems using mollification. We quickly review existing mollification regularization methods and provide two new approximate solutions to a general ill-posed equation $T(f) =g$ where $T$ can be nonlinear. The regularized solutions we define extend the work of Bonnefond and Maréchal \cite{xapi}, and trace their origins in the variational formulation of mollification, which to the best of our knowledge, was first introduced by Lannes et al. \cite{lannes}. In addition to consistency results, for the first time, we provide some convergence rates for a mollification method defined through a variational formulation.
△ Less
Submitted 2 May, 2020; v1 submitted 17 March, 2020;
originally announced March 2020.
-
Multilevel spectral coarsening for graph Laplacian problems with application to reservoir simulation
Authors:
Andrew T. Barker,
Stephan V. Gelever,
Chak S. Lee,
Sarah V. Osborn,
Panayot S. Vassilevski
Abstract:
We extend previously developed two-level coarsening procedures for graph Laplacian problems written in a mixed saddle point form to the fully recursive multilevel case. The resulting hierarchy of discretizations gives rise to a hierarchy of upscaled models, in the sense that they provide approximation in the natural norms (in the mixed setting). This property enables us to utilize them in three ap…
▽ More
We extend previously developed two-level coarsening procedures for graph Laplacian problems written in a mixed saddle point form to the fully recursive multilevel case. The resulting hierarchy of discretizations gives rise to a hierarchy of upscaled models, in the sense that they provide approximation in the natural norms (in the mixed setting). This property enables us to utilize them in three applications: (i) as an accurate reduced model, (ii) as a tool in multilevel Monte Carlo simulations (in application to finite volume discretizations), and (iii) for providing a sequence of nonlinear operators in FAS (full approximation scheme) for solving nonlinear pressure equations discretized by the conservative two-point flux approximation. We illustrate the potential of the proposed multilevel technique in all three applications on a number of popular benchmark problems used in reservoir simulation.
△ Less
Submitted 9 March, 2020;
originally announced March 2020.
-
A new regularization method for linear exponentially ill-posed problems
Authors:
Walter Cedric Simo Tao Lee
Abstract:
This paper provides a new regularization method which is particularly suitable for linear exponentially ill-posed problems. Under logarithmic source conditions (which have a natural interpretation in terms of Sobolev spaces in the aforementioned context), concepts of qualifications as well as order optimal rates of convergence are presented. Optimality results under general source conditions expre…
▽ More
This paper provides a new regularization method which is particularly suitable for linear exponentially ill-posed problems. Under logarithmic source conditions (which have a natural interpretation in terms of Sobolev spaces in the aforementioned context), concepts of qualifications as well as order optimal rates of convergence are presented. Optimality results under general source conditions expressed in terms of index functions are also studied.
Finally, numerical experiments on three test problems attest the better performance of the new method compared to the well known Tikhonov method in instances of exponentially ill-posed problems.
△ Less
Submitted 7 July, 2020; v1 submitted 26 September, 2019;
originally announced September 2019.
-
Enhancement of Energy-Based Swing-Up Controller via Entropy Search
Authors:
Chang Sik Lee,
Dong Eui Chang
Abstract:
An energy based approach for stabilizing a mechanical system has offered a simple yet powerful control scheme. However, since it does not impose such strong constraints on parameter space of the controller, finding appropriate parameter values for an optimal controller is known to be hard. This paper intends to generate an optimal energy-based controller for swinging up a rotary inverted pendulum,…
▽ More
An energy based approach for stabilizing a mechanical system has offered a simple yet powerful control scheme. However, since it does not impose such strong constraints on parameter space of the controller, finding appropriate parameter values for an optimal controller is known to be hard. This paper intends to generate an optimal energy-based controller for swinging up a rotary inverted pendulum, also known as the Furuta pendulum, by applying the Bayesian optimization called Entropy Search. Simulations and experiments show that the optimal controller has an improved performance compared to a nominal controller for various initial conditions.
△ Less
Submitted 3 April, 2019; v1 submitted 2 April, 2019;
originally announced April 2019.
-
Algebraic Hybridization and Static Condensation with Application to Scalable H(div) Preconditioning
Authors:
Veselin A. Dobrev,
Tzanio V. Kolev,
Chak S. Lee,
Vladimir Z. Tomov,
Panayot S. Vassilevski
Abstract:
We propose an unified algebraic approach for static condensation and hybridization, two popular techniques in finite element discretizations. The algebraic approach is supported by the construction of scalable solvers for problems involving H(div)-spaces discretized by conforming (Raviart-Thomas) elements of arbitrary order. We illustrate through numerical experiments the relative performance of t…
▽ More
We propose an unified algebraic approach for static condensation and hybridization, two popular techniques in finite element discretizations. The algebraic approach is supported by the construction of scalable solvers for problems involving H(div)-spaces discretized by conforming (Raviart-Thomas) elements of arbitrary order. We illustrate through numerical experiments the relative performance of the two (in some sense dual) techniques in comparison with a state-of-the-art parallel solver, ADS, available in the software hypre and MFEM. The superior performance of the hybridization technique is clearly demonstrated with increased benefit for higher order elements.
△ Less
Submitted 26 January, 2018;
originally announced January 2018.
-
Mixed Generalized Multiscale Finite Element Methods and Applications
Authors:
Eric T. Chung,
Yalchin Efendiev,
Chak Shing Lee
Abstract:
In this paper, we present a mixed Generalized Multiscale Finite Element Method (GMsFEM) for solving flow in heterogeneous media. Our approach constructs multiscale basis functions following a GMsFEM framework and couples these basis functions using a mixed finite element method, which allows us to obtain a mass conservative velocity field. To construct multiscale basis functions for each coarse ed…
▽ More
In this paper, we present a mixed Generalized Multiscale Finite Element Method (GMsFEM) for solving flow in heterogeneous media. Our approach constructs multiscale basis functions following a GMsFEM framework and couples these basis functions using a mixed finite element method, which allows us to obtain a mass conservative velocity field. To construct multiscale basis functions for each coarse edge, we design a snapshot space that consists of fine-scale velocity fields supported in a union of two coarse regions that share the common interface. The snapshot vectors have zero Neumann boundary conditions on the outer boundaries and we prescribe their values on the common interface. We describe several spectral decompositions in the snapshot space motivated by the analysis. In the paper, we also study oversampling approaches that enhance the accuracy of mixed GMsFEM. A main idea of oversampling techniques is to introduce a small dimensional snapshot space. We present numerical results for two-phase flow and transport, without updating basis functions in time. Our numerical results show that one can achieve good accuracy with a few basis functions per coarse edge if one selects appropriate offline spaces.
△ Less
Submitted 4 June, 2014;
originally announced June 2014.
