-
Learning to (Learn at Test Time)
Authors:
Yu Sun,
Xinhao Li,
Karan Dalal,
Chloe Hsu,
Sanmi Koyejo,
Carlos Guestrin,
Xiaolong Wang,
Tatsunori Hashimoto,
Xinlei Chen
Abstract:
We reformulate the problem of supervised learning as learning to learn with two nested loops (i.e. learning problems). The inner loop learns on each individual instance with self-supervision before final prediction. The outer loop learns the self-supervised task used by the inner loop, such that its final prediction improves. Our inner loop turns out to be equivalent to linear attention when the i…
▽ More
We reformulate the problem of supervised learning as learning to learn with two nested loops (i.e. learning problems). The inner loop learns on each individual instance with self-supervision before final prediction. The outer loop learns the self-supervised task used by the inner loop, such that its final prediction improves. Our inner loop turns out to be equivalent to linear attention when the inner-loop learner is only a linear model, and to self-attention when it is a kernel estimator. For practical comparison with linear or self-attention layers, we replace each of them in a transformer with an inner loop, so our outer loop is equivalent to training the architecture. When each inner-loop learner is a neural network, our approach vastly outperforms transformers with linear attention on ImageNet from 224 x 224 raw pixels in both accuracy and FLOPs, while (regular) transformers cannot run.
△ Less
Submitted 7 January, 2024; v1 submitted 20 October, 2023;
originally announced October 2023.
-
Two-way Linear Probing Revisited
Authors:
Ketan Dalal,
Luc Devroye,
Ebrahim Malalla
Abstract:
We introduce linear probing hashing schemes that construct a hash table of size $n$, with constant load factor $α$, on which the worst-case unsuccessful search time is asymptotically almost surely $O(\log \log n)$. The schemes employ two linear probe sequences to find empty cells for the keys. Matching lower bounds on the maximum cluster size produced by any algorithm that uses two linear probe se…
▽ More
We introduce linear probing hashing schemes that construct a hash table of size $n$, with constant load factor $α$, on which the worst-case unsuccessful search time is asymptotically almost surely $O(\log \log n)$. The schemes employ two linear probe sequences to find empty cells for the keys. Matching lower bounds on the maximum cluster size produced by any algorithm that uses two linear probe sequences are obtained as well.
△ Less
Submitted 18 September, 2023; v1 submitted 11 September, 2023;
originally announced September 2023.
-
Detection of the Splashback Radius and Halo Assembly bias of Massive Galaxy Clusters
Authors:
Surhud More,
Hironao Miyatake,
Masahiro Takada,
Benedikt Diemer,
Andrey V. Kravtsov,
Neal K. Dalal,
Anupreeta More,
Ryoma Murata,
Rachel Mandelbaum,
Eduardo Rozo,
Eli S. Rykoff,
Masamune Oguri,
David N. Spergel
Abstract:
We show that the projected number density profiles of SDSS photometric galaxies around galaxy clusters displays strong evidence for the splashback radius, a sharp halo edge corresponding to the location of the first orbital apocenter of satellite galaxies after their infall. We split the clusters into two subsamples with different mean projected radial distances of their members,…
▽ More
We show that the projected number density profiles of SDSS photometric galaxies around galaxy clusters displays strong evidence for the splashback radius, a sharp halo edge corresponding to the location of the first orbital apocenter of satellite galaxies after their infall. We split the clusters into two subsamples with different mean projected radial distances of their members, $\langle R_{\rm mem}\rangle$, at fixed richness and redshift, and show that the sample with smaller $\langle R_{\rm mem}\rangle$ has a smaller ratio of the splashback radius to the traditional halo boundary $R_{\rm 200m}$, than the subsample with larger $\langle R_{\rm mem}\rangle$, indicative of different mass accretion rates for the two subsamples. The same cluster samples were recently used by Miyatake et al. to show that their large-scale clustering differs despite their similar weak lensing masses, demonstrating strong evidence for halo assembly bias. We expand on this result by presenting a 6.6-$σ$ detection of halo assembly bias using the cluster-photometric galaxy cross-correlations. Our measured splashback radii are smaller, while the strength of the assembly bias signal is stronger, than expectations from N-body simulations based on the $Λ$-dominated, cold dark matter structure formation model. Dynamical friction or cluster-finding systematics such as miscentering or projection effects are not likely to be the sole source of these discrepancies.
△ Less
Submitted 22 January, 2016;
originally announced January 2016.