-
Furthering a Comprehensive SETI Bibliography
Authors:
Julia LaFond,
Jason T. Wright,
Macy J. Huston
Abstract:
In 2019, Reyes & Wright used the NASA Astrophysics Data System (ADS) to initiate a comprehensive bibliography for SETI accessible to the public. Since then, updates to the library have been incomplete, partly due to the difficulty in managing the large number of false positive publications generated by searching ADS using simple search terms. In preparation for a recent update, the scope of the li…
▽ More
In 2019, Reyes & Wright used the NASA Astrophysics Data System (ADS) to initiate a comprehensive bibliography for SETI accessible to the public. Since then, updates to the library have been incomplete, partly due to the difficulty in managing the large number of false positive publications generated by searching ADS using simple search terms. In preparation for a recent update, the scope of the library was revised and reexamined. The scope now includes social sciences and commensal SETI. Results were curated based on five SETI keyword searches: "SETI", "technosignature", "Fermi Paradox," "Drake Equation", and "extraterrestrial intelligence." These keywords returned 553 publications that merited inclusion in the bibliography that were not previously present. A curated library of false positive results is now concurrently maintained to facilitate their exclusion from future searches. A search query and workflow was developed to capture nearly all SETI-related papers indexed by ADS while minimizing false positives. These tools will enable efficient, consistent updates of the SETI library by future curators, and could be adopted for other bibliography projects as well.
△ Less
Submitted 6 July, 2021;
originally announced July 2021.
-
Diagonal Rescaling For Neural Networks
Authors:
Jean Lafond,
Nicolas Vasilache,
Léon Bottou
Abstract:
We define a second-order neural network stochastic gradient training algorithm whose block-diagonal structure effectively amounts to normalizing the unit activations. Investigating why this algorithm lacks in robustness then reveals two interesting insights. The first insight suggests a new way to scale the stepsizes, clarifying popular algorithms such as RMSProp as well as old neural network tric…
▽ More
We define a second-order neural network stochastic gradient training algorithm whose block-diagonal structure effectively amounts to normalizing the unit activations. Investigating why this algorithm lacks in robustness then reveals two interesting insights. The first insight suggests a new way to scale the stepsizes, clarifying popular algorithms such as RMSProp as well as old neural network tricks such as fanin stepsize scaling. The second insight stresses the practical importance of dealing with fast changes of the curvature of the cost.
△ Less
Submitted 25 May, 2017;
originally announced May 2017.
-
On the Online Frank-Wolfe Algorithms for Convex and Non-convex Optimizations
Authors:
Jean Lafond,
Hoi-To Wai,
Eric Moulines
Abstract:
In this paper, the online variants of the classical Frank-Wolfe algorithm are considered. We consider minimizing the regret with a stochastic cost. The online algorithms only require simple iterative updates and a non-adaptive step size rule, in contrast to the hybrid schemes commonly considered in the literature. Several new results are derived for convex and non-convex losses. With a strongly co…
▽ More
In this paper, the online variants of the classical Frank-Wolfe algorithm are considered. We consider minimizing the regret with a stochastic cost. The online algorithms only require simple iterative updates and a non-adaptive step size rule, in contrast to the hybrid schemes commonly considered in the literature. Several new results are derived for convex and non-convex losses. With a strongly convex stochastic cost and when the optimal solution lies in the interior of the constraint set or the constraint set is a polytope, the regret bound and anytime optimality are shown to be ${\cal O}( \log^3 T / T )$ and ${\cal O}( \log^2 T / T)$, respectively, where $T$ is the number of rounds played. These results are based on an improved analysis on the stochastic Frank-Wolfe algorithms. Moreover, the online algorithms are shown to converge even when the loss is non-convex, i.e., the algorithms find a stationary point to the time-varying/stochastic loss at a rate of ${\cal O}(\sqrt{1/T})$. Numerical experiments on realistic data sets are presented to support our theoretical claims.
△ Less
Submitted 15 August, 2016; v1 submitted 5 October, 2015;
originally announced October 2015.
