-
Scaling and evaluating sparse autoencoders
Authors:
Leo Gao,
Tom Dupré la Tour,
Henk Tillman,
Gabriel Goh,
Rajan Troll,
Alec Radford,
Ilya Sutskever,
Jan Leike,
Jeffrey Wu
Abstract:
Sparse autoencoders provide a promising unsupervised approach for extracting interpretable features from a language model by reconstructing activations from a sparse bottleneck layer. Since language models learn many concepts, autoencoders need to be very large to recover all relevant features. However, studying the properties of autoencoder scaling is difficult due to the need to balance reconstr…
▽ More
Sparse autoencoders provide a promising unsupervised approach for extracting interpretable features from a language model by reconstructing activations from a sparse bottleneck layer. Since language models learn many concepts, autoencoders need to be very large to recover all relevant features. However, studying the properties of autoencoder scaling is difficult due to the need to balance reconstruction and sparsity objectives and the presence of dead latents. We propose using k-sparse autoencoders [Makhzani and Frey, 2013] to directly control sparsity, simplifying tuning and improving the reconstruction-sparsity frontier. Additionally, we find modifications that result in few dead latents, even at the largest scales we tried. Using these techniques, we find clean scaling laws with respect to autoencoder size and sparsity. We also introduce several new metrics for evaluating feature quality based on the recovery of hypothesized features, the explainability of activation patterns, and the sparsity of downstream effects. These metrics all generally improve with autoencoder size. To demonstrate the scalability of our approach, we train a 16 million latent autoencoder on GPT-4 activations for 40 billion tokens. We release training code and autoencoders for open-source models, as well as a visualizer.
△ Less
Submitted 6 June, 2024;
originally announced June 2024.
-
SegNBDT: Visual Decision Rules for Segmentation
Authors:
Alvin Wan,
Daniel Ho,
Youn** Song,
Henk Tillman,
Sarah Adel Bargal,
Joseph E. Gonzalez
Abstract:
The black-box nature of neural networks limits model decision interpretability, in particular for high-dimensional inputs in computer vision and for dense pixel prediction tasks like segmentation. To address this, prior work combines neural networks with decision trees. However, such models (1) perform poorly when compared to state-of-the-art segmentation models or (2) fail to produce decision rul…
▽ More
The black-box nature of neural networks limits model decision interpretability, in particular for high-dimensional inputs in computer vision and for dense pixel prediction tasks like segmentation. To address this, prior work combines neural networks with decision trees. However, such models (1) perform poorly when compared to state-of-the-art segmentation models or (2) fail to produce decision rules with spatially-grounded semantic meaning. In this work, we build a hybrid neural-network and decision-tree model for segmentation that (1) attains neural network segmentation accuracy and (2) provides semi-automatically constructed visual decision rules such as "Is there a window?". We obtain semantic visual meaning by extending saliency methods to segmentation and attain accuracy by leveraging insights from neural-backed decision trees, a deep learning analog of decision trees for image classification. Our model SegNBDT attains accuracy within ~2-4% of the state-of-the-art HRNetV2 segmentation model while also retaining explainability; we achieve state-of-the-art performance for explainable models on three benchmark datasets -- Pascal-Context (49.12%), Cityscapes (79.01%), and Look Into Person (51.64%). Furthermore, user studies suggest visual decision rules are more interpretable, particularly for incorrect predictions. Code and pretrained models can be found at https://github.com/daniel-ho/SegNBDT.
△ Less
Submitted 11 June, 2020;
originally announced June 2020.
-
Practical Limits in the Sensitivity-Linearity Trade-off for Radio Telescope Front Ends in the HF and VHF-low Bands
Authors:
R. H. Tillman,
S. W. Ellingson,
J. Brendler
Abstract:
Radio telescope front ends must have simultaneously low noise and sufficiently-high linearity to accommodate interfering signals. Typically these are opposing design goals. For modern radio telescopes operating in the HF (3-30 MHz) and VHF-low (30-88 MHz) bands, the problem is more nuanced in that front end noise temperature may be a relatively small component of the system temperature, and increa…
▽ More
Radio telescope front ends must have simultaneously low noise and sufficiently-high linearity to accommodate interfering signals. Typically these are opposing design goals. For modern radio telescopes operating in the HF (3-30 MHz) and VHF-low (30-88 MHz) bands, the problem is more nuanced in that front end noise temperature may be a relatively small component of the system temperature, and increased linearity may be required due to the particular interference problems associated with this spectrum. In this paper we present an analysis of the sensitivity-linearity trade off at these frequencies, applicable to existing commercially-available monolithic microwave integrated circuit (MMIC) amplifiers in single-ended, differential, and parallelized configurations. This analysis and associated findings should be useful in the design and upgrade of front ends for low frequency radio telescopes. The analysis is demonstrated explicitly for one of the better-performing amplifiers encountered in this study, the Mini-Circuits PGA-103, and is confirmed by hardware measurements. We also present a design based on the Mini-Circuits HELA-10 amplifier, which is better-suited for applications where linearity is a primary concern.
△ Less
Submitted 9 March, 2016;
originally announced March 2016.
