-
Convergence of ADAM for Lipschitz Objective Functions
Authors:
Juan Ferrera,
Javier Gómez Gil
Abstract:
The aim of this paper is to prove the exponential convergence, local
and global, of Adam algorithm under precise conditions on the
parameters, when the objective function lacks
differentiability. More precisely, we require Lipschitz continuity,
and control on the gradient whenever it exists. We provide also
examples of interesting functions that satisfies the required
restrictions.
The aim of this paper is to prove the exponential convergence, local
and global, of Adam algorithm under precise conditions on the
parameters, when the objective function lacks
differentiability. More precisely, we require Lipschitz continuity,
and control on the gradient whenever it exists. We provide also
examples of interesting functions that satisfies the required
restrictions.
△ Less
Submitted 13 March, 2024;
originally announced March 2024.
-
Surfing the modeling of PoS taggers in low-resource scenarios
Authors:
Manuel Vilares Ferro,
Víctor M. Darriba Bilbao,
Francisco J. Ribadas-Pena,
Jorge Graña Gil
Abstract:
The recent trend towards the application of deep structured techniques has revealed the limits of huge models in natural language processing. This has reawakened the interest in traditional machine learning algorithms, which have proved still to be competitive in certain contexts, in particular low-resource settings. In parallel, model selection has become an essential task to boost performance at…
▽ More
The recent trend towards the application of deep structured techniques has revealed the limits of huge models in natural language processing. This has reawakened the interest in traditional machine learning algorithms, which have proved still to be competitive in certain contexts, in particular low-resource settings. In parallel, model selection has become an essential task to boost performance at reasonable cost, even more so when we talk about processes involving domains where the training and/or computational resources are scarce. Against this backdrop, we evaluate the early estimation of learning curves as a practical mechanism for selecting the most appropriate model in scenarios characterized by the use of non-deep learners in resource-lean settings. On the basis of a formal approximation model previously evaluated under conditions of wide availability of training and validation resources, we study the reliability of such an approach in a different and much more demanding operationalenvironment. Using as case study the generation of PoS taggers for Galician, a language belonging to the Western Ibero-Romance group, the experimental results are consistent with our expectations.
△ Less
Submitted 4 February, 2024;
originally announced February 2024.
-
A characterization of the nowhere differentiable functions of the Generalized Takagi Class
Authors:
Juan Ferrera,
Javier Gómez Gil,
Jesús Llorente
Abstract:
In this paper, we prove that for some Generalized Takagi Classes, in particular for the Takagi-Van der Waerden Class, the functions are nowhere differentiable if, and only if, the sequence of weights does not belong to $c_0$.
In this paper, we prove that for some Generalized Takagi Classes, in particular for the Takagi-Van der Waerden Class, the functions are nowhere differentiable if, and only if, the sequence of weights does not belong to $c_0$.
△ Less
Submitted 12 September, 2019;
originally announced September 2019.
-
Superdifferential of the Takagi function
Authors:
Juan Ferrera,
Javier Gómez Gil
Abstract:
The Takagi function is a classical example of a continuous nowhere differentiable function. It has empty subdifferential except in a countable set where its subdifferential is $\mathbb{R}$. In this paper we characterize its superdifferential.
The Takagi function is a classical example of a continuous nowhere differentiable function. It has empty subdifferential except in a countable set where its subdifferential is $\mathbb{R}$. In this paper we characterize its superdifferential.
△ Less
Submitted 24 June, 2019;
originally announced June 2019.
-
Infinite derivatives of the Takagi-Van der Waerden functions
Authors:
Juan Ferrera,
Javier Gómez Gil,
Jesús Llorente
Abstract:
In this paper we characterize the set of points where the lateral derivatives of the Takagi-Van der Waerden functions are infinite. We also prove that the set of points with infinite derivative has Hausdorff dimension one and Lebesgue measure zero.
In this paper we characterize the set of points where the lateral derivatives of the Takagi-Van der Waerden functions are infinite. We also prove that the set of points with infinite derivative has Hausdorff dimension one and Lebesgue measure zero.
△ Less
Submitted 29 March, 2019; v1 submitted 27 March, 2019;
originally announced March 2019.
-
Non approximate derivability of the Takagi function
Authors:
Juan Ferrera,
Javier Gómez Gil
Abstract:
The Takagi function is a classical example of a continuous nowhere differentiable function. In this paper we prove that it is nowhere approximately derivable.
The Takagi function is a classical example of a continuous nowhere differentiable function. In this paper we prove that it is nowhere approximately derivable.
△ Less
Submitted 24 June, 2019; v1 submitted 23 July, 2018;
originally announced July 2018.
-
Uncovering Universal Wave Fluctuations In a Scaled Ray-Chaotic Cavity With Remote Injection
Authors:
Bo Xiao,
Thomas M. Antonsen,
Edward Ott,
Zachary B. Drikas,
Jesus Gil Gil,
Steven M. Anlage
Abstract:
The Random Coupling Model (RCM), introduced by Zheng, Antonsen and Ott, predicts the statistical properties of waves inside a ray-chaotic enclosure in the semi-classical regime by using Random Matrix Theory, combined with system-specific information. Experiments on single cavities are in general agreement with the predictions of the RCM. It is now desired to test the RCM on more complex structures…
▽ More
The Random Coupling Model (RCM), introduced by Zheng, Antonsen and Ott, predicts the statistical properties of waves inside a ray-chaotic enclosure in the semi-classical regime by using Random Matrix Theory, combined with system-specific information. Experiments on single cavities are in general agreement with the predictions of the RCM. It is now desired to test the RCM on more complex structures, such as a cascade or network of coupled cavities, that represent realistic situations, but which are difficult to test due to the large size of the structures of interest. This paper presents a novel experimental setup that replaces a cubic-meter-scale microwave cavity with a miniaturized cavity, scaled down by a factor of 20 in each dimension, operated at a frequency scaled up by a factor of 20 and having wall conductivity appropriately scaled up by a factor of 20. We demonstrate experimentally that the miniaturized cavity maintains the statistical wave properties of the larger cavity. This scaled setup opens the opportunity to study wave properties in large structures such as the floor of an office building, a ship, or an aircraft, in a controlled laboratory setting.
△ Less
Submitted 15 February, 2018;
originally announced February 2018.
-
Impedance and Scattering Variance Ratios of Complicated Wave Scattering Systems in the Low Loss Regime
Authors:
Jen-Hao Yeh,
Zachary Drikas,
Jesus Gil Gil,
Sun Hong,
Biniyam T. Taddese,
Edward Ott,
Thomas M. Antonsen,
Tim Andreadis,
Steven M. Anlage
Abstract:
Random matrix theory (RMT) successfully predicts universal statistical properties of complicated wave scattering systems in the semiclassical limit, while the random coupling model offers a complete statistical model with a simple additive formula in terms of impedance to combine the predictions of RMT and nonuniversal system-specific features. The statistics of measured wave properties generally…
▽ More
Random matrix theory (RMT) successfully predicts universal statistical properties of complicated wave scattering systems in the semiclassical limit, while the random coupling model offers a complete statistical model with a simple additive formula in terms of impedance to combine the predictions of RMT and nonuniversal system-specific features. The statistics of measured wave properties generally have nonuniversal features. However, ratios of the variances of elements of the impedance matrix are predicted to be independent of such nonuniversal features and thus should be universal functions of the overall system loss. In contrast with impedance variance ratios, scattering variance ratios depends on nonuniversal features unless the system is in the high loss regime. In this paper, we present numerical tests of the predicted universal impedance variance ratios and show that an insufficient sample size can lead to apparent deviation from the theory, particularly in the low loss regime. Experimental tests are carried out in three two-port microwave cavities with varied loss parameters, including a novel experimental system with a superconducting microwave billiard, to test the variance-ratio predictions in the low loss time-reversal-invariant regime. It is found that the experimental results agree with the theoretical predictions to the extent permitted by the finite sample size.
△ Less
Submitted 13 November, 2013; v1 submitted 12 September, 2013;
originally announced September 2013.
