-
The Flare Likelihood and Region Eruption Forecasting (FLARECAST) Project: Flare forecasting in the big data & machine learning era
Authors:
M. K. Georgoulis,
D. S. Bloomfield,
M. Piana,
A. M. Massone,
M. Soldati,
P. T. Gallagher,
E. Pariat,
N. Vilmer,
E. Buchlin,
F. Baudin,
A. Csillaghy,
H. Sathiapal,
D. R. Jackson,
P. Alingery,
F. Benvenuto,
C. Campi,
K. Florios,
C. Gontikakis,
C. Guennou,
J. A. Guerra,
I. Kontogiannis,
V. Latorre,
S. A. Murray,
S. -H. Park,
S. von Stachelski
, et al. (3 additional authors not shown)
Abstract:
The EU funded the FLARECAST project, that ran from Jan 2015 until Feb 2018. FLARECAST had a R2O focus, and introduced several innovations into the discipline of solar flare forecasting. FLARECAST innovations were: first, the treatment of hundreds of physical properties viewed as promising flare predictors on equal footing, extending multiple previous works; second, the use of fourteen (14) differe…
▽ More
The EU funded the FLARECAST project, that ran from Jan 2015 until Feb 2018. FLARECAST had a R2O focus, and introduced several innovations into the discipline of solar flare forecasting. FLARECAST innovations were: first, the treatment of hundreds of physical properties viewed as promising flare predictors on equal footing, extending multiple previous works; second, the use of fourteen (14) different ML techniques, also on equal footing, to optimize the immense Big Data parameter space created by these many predictors; third, the establishment of a robust, three-pronged communication effort oriented toward policy makers, space-weather stakeholders and the wider public. FLARECAST pledged to make all its data, codes and infrastructure openly available worldwide. The combined use of 170+ properties (a total of 209 predictors are now available) in multiple ML algorithms, some of which were designed exclusively for the project, gave rise to changing sets of best-performing predictors for the forecasting of different flaring levels. At the same time, FLARECAST reaffirmed the importance of rigorous training and testing practices to avoid overly optimistic pre-operational prediction performance. In addition, the project has (a) tested new and revisited physically intuitive flare predictors and (b) provided meaningful clues toward the transition from flares to eruptive flares, namely, events associated with coronal mass ejections (CMEs). These leads, along with the FLARECAST data, algorithms and infrastructure, could help facilitate integrated space-weather forecasting efforts that take steps to avoid effort duplication. In spite of being one of the most intensive and systematic flare forecasting efforts to-date, FLARECAST has not managed to convincingly lift the barrier of stochasticity in solar flare occurrence and forecasting: solar flare prediction thus remains inherently probabilistic.
△ Less
Submitted 12 May, 2021;
originally announced May 2021.
-
Topology Optimization with Bilevel Knapsack: An Efficient 51 Lines MATLAB Code
Authors:
Vittorio Latorre
Abstract:
This paper presents an efficient 51 lines Matlab code to solve topology optimization problems. By the fact that the presented code is based on an hard 0-1 optimization method that handles the integer part of the optimization in a simple fashion and in sub-polynomial time, it has been possible to not only reduce the number of lines to 51 without sacrificing any readability, but also to obtain void-…
▽ More
This paper presents an efficient 51 lines Matlab code to solve topology optimization problems. By the fact that the presented code is based on an hard 0-1 optimization method that handles the integer part of the optimization in a simple fashion and in sub-polynomial time, it has been possible to not only reduce the number of lines to 51 without sacrificing any readability, but also to obtain void-solid designs with close to none checkerboard patterns with improved efficiency. The numerical results in the paper show that the proposed method has the best average times compared to several codes available in literature. We also present extensions to different boundary conditions and to three dimensional designs. The code can be used by students and the newcomers in topology optimization because of its simplicity and readability. The 51 lines Matlab code and the presented extensions can be downloaded on the webpage https://github.com/vlatorre847/TOSSE.
△ Less
Submitted 3 February, 2019;
originally announced February 2019.
-
Global Optimal Trajectory in Chaos and NP-Hardness
Authors:
Vittorio Latorre,
David Yang Gao
Abstract:
This paper presents a new canonical duality methodology for solving general nonlinear dynamical systems. Instead of the conventional iterative methods, the discretized nonlinear system is first formulated as a global optimization problem via the least squares method. The canonical duality theory shows that this nonconvex minimization problem can be solved deterministically in polynomial time if a…
▽ More
This paper presents a new canonical duality methodology for solving general nonlinear dynamical systems. Instead of the conventional iterative methods, the discretized nonlinear system is first formulated as a global optimization problem via the least squares method. The canonical duality theory shows that this nonconvex minimization problem can be solved deterministically in polynomial time if a global optimality condition is satisfied. The so-called pseudo-chaos produced by Runge-Kutta type of linear iterations are mainly due to the intrinsic numerical error accumulations. Otherwise, the global optimization problem could be NP-hard and the nonlinear system can be really chaotic.
A conjecture is proposed, which reveals the connection between chaos in nonlinear dynamics and NP-hardness in computer science. The methodology and the conjecture are verified by applications to the well-known logistic equation, a forced memristive circuit and the Lorenz system. Computational results show that the canonical duality theory can be used to identify chaotic systems and to obtain realistic global optimal solutions in nonlinear dynamical systems.
△ Less
Submitted 28 December, 2015;
originally announced December 2015.
-
Canonical Duality-Triality Theory: Bridge Between Nonconvex Analysis/Mechanics and Global Optimization in Complex Systems
Authors:
David Y Gao,
Ning Ruan,
Vittorio Latorre
Abstract:
Canonical duality-triality is a breakthrough methodological theory, which can be used not only for modeling complex systems within a unified framework, but also for solving a wide class of challenging problems from real-world applications. This paper presents a brief review on this theory, its philosophical origin, physics foundation, and mathematical statements in both finite and infinite dimensi…
▽ More
Canonical duality-triality is a breakthrough methodological theory, which can be used not only for modeling complex systems within a unified framework, but also for solving a wide class of challenging problems from real-world applications. This paper presents a brief review on this theory, its philosophical origin, physics foundation, and mathematical statements in both finite and infinite dimensional spaces, with emphasizing on its role for bridging the gap between nonconvex analysis/mechanics and global optimization. Special attentions are paid on unified understanding the fundamental difficulties in large deformation mechanics, bifurcation/chaos in nonlinear science, and the NP-hard problems in global optimization, as well as the theorems, methods, and algorithms for solving these challenging problems. Misunderstandings and confusions on some basic concepts, such as objectivity, nonlinearity, Lagrangian, and generalized convexities are discussed and classified. Breakthrough from recent challenges and conceptual mistakes by M. Voisei, C. Zalinescu and his co-worker are addressed. Some open problems and future works in global optimization and nonconvex mechanics are proposed.
△ Less
Submitted 26 November, 2014; v1 submitted 9 October, 2014;
originally announced October 2014.
-
A Potential Reduction Method for Canonical Duality, with an Application to the Sensor Network Localization Problem
Authors:
Vittorio Latorre
Abstract:
We propose to solve large instances of the non-convex optimization problems reformulated with canonical duality theory. To this aim we propose an interior point potential reduction algorithm based on the solution of the primal-dual total complementarity (Lagrange) function. We establish the global convergence result for the algorithm under mild assumptions and demonstrate the method on instances o…
▽ More
We propose to solve large instances of the non-convex optimization problems reformulated with canonical duality theory. To this aim we propose an interior point potential reduction algorithm based on the solution of the primal-dual total complementarity (Lagrange) function. We establish the global convergence result for the algorithm under mild assumptions and demonstrate the method on instances of the Sensor Network Localization problem. Our numerical results are promising and show the possibility of devising efficient interior points methods for non-convex duality.
△ Less
Submitted 24 October, 2014; v1 submitted 24 March, 2014;
originally announced March 2014.
-
Canonical Dual Approach for Contact Mechanics Problems with Friction
Authors:
Vittorio Latorre,
Simone Sagratella,
David Yang Gao
Abstract:
This paper presents an application of Canonical duality theory to the solution of contact problems with Coulomb friction. The contact problem is formulated as a quasi-variational inequality which solution is found by solving its Karush-Kunt-Tucker system of equations. The complementarity conditions are reformulated by using the Fischer-Burmeister complementarity function, obtaining a non-convex gl…
▽ More
This paper presents an application of Canonical duality theory to the solution of contact problems with Coulomb friction. The contact problem is formulated as a quasi-variational inequality which solution is found by solving its Karush-Kunt-Tucker system of equations. The complementarity conditions are reformulated by using the Fischer-Burmeister complementarity function, obtaining a non-convex global optimization problem. Then canonical duality theory is applied to reformulate the non-convex global optimization problem and define its optimality conditions, finding a solution of the original quasi-variational inequality. We also propose a methodology for finding the solutions of the new formulation, and report the results on well known instances from literature.
△ Less
Submitted 27 February, 2014;
originally announced February 2014.
-
Canonical duality for solving general nonconvex constrained problems
Authors:
Vittorio Latorre,
David Y. Gao
Abstract:
This paper presents a canonical duality theory for solving a general nonconvex constrained optimization problem within a unified framework to cover Lagrange multiplier method and KKT theory. It is proved that if both target function and constraints possess certain patterns necessary for modeling real systems, a perfect dual problem (without duality gap)can be obtained in a unified form with global…
▽ More
This paper presents a canonical duality theory for solving a general nonconvex constrained optimization problem within a unified framework to cover Lagrange multiplier method and KKT theory. It is proved that if both target function and constraints possess certain patterns necessary for modeling real systems, a perfect dual problem (without duality gap)can be obtained in a unified form with global optimality conditions provided. While the popular augmented Lagrangian method may produce more difficult nonconvex problems due to the nonlinearity of constraints.
△ Less
Submitted 8 October, 2013;
originally announced October 2013.
-
Canonical dual solutions to nonconvex radial basis neural network optimization problem
Authors:
Vittorio Latorre,
David Yang Gao
Abstract:
Radial Basis Functions Neural Networks (RBFNNs) are tools widely used in regression problems. One of their principal drawbacks is that the formulation corresponding to the training with the supervision of both the centers and the weights is a highly non-convex optimization problem, which leads to some fundamentally difficulties for traditional optimization theory and methods.
This paper presents…
▽ More
Radial Basis Functions Neural Networks (RBFNNs) are tools widely used in regression problems. One of their principal drawbacks is that the formulation corresponding to the training with the supervision of both the centers and the weights is a highly non-convex optimization problem, which leads to some fundamentally difficulties for traditional optimization theory and methods.
This paper presents a generalized canonical duality theory for solving this challenging problem. We demonstrate that by sequential canonical dual transformations, the nonconvex optimization problem of the RBFNN can be reformulated as a canonical dual problem (without duality gap). Both global optimal solution and local extrema can be classified. Several applications to one of the most used Radial Basis Functions, the Gaussian function, are illustrated. Our results show that even for one-dimensional case, the global minimizer of the nonconvex problem may not be the best solution to the RBFNNs, and the canonical dual theory is a promising tool for solving general neural networks training problems.
△ Less
Submitted 17 February, 2013;
originally announced February 2013.
