-
Who's asking? User personas and the mechanics of latent misalignment
Authors:
Asma Ghandeharioun,
Ann Yuan,
Marius Guerard,
Emily Reif,
Michael A. Lepori,
Lucas Dixon
Abstract:
Despite investments in improving model safety, studies show that misaligned capabilities remain latent in safety-tuned models. In this work, we shed light on the mechanics of this phenomenon. First, we show that even when model generations are safe, harmful content can persist in hidden representations and can be extracted by decoding from earlier layers. Then, we show that whether the model divul…
▽ More
Despite investments in improving model safety, studies show that misaligned capabilities remain latent in safety-tuned models. In this work, we shed light on the mechanics of this phenomenon. First, we show that even when model generations are safe, harmful content can persist in hidden representations and can be extracted by decoding from earlier layers. Then, we show that whether the model divulges such content depends significantly on its perception of who it is talking to, which we refer to as user persona. In fact, we find manipulating user persona to be even more effective for eliciting harmful content than direct attempts to control model refusal. We study both natural language prompting and activation steering as control methods and show that activation steering is significantly more effective at bypassing safety filters. We investigate why certain personas break model safeguards and find that they enable the model to form more charitable interpretations of otherwise dangerous queries. Finally, we show we can predict a persona's effect on refusal given only the geometry of its steering vector.
△ Less
Submitted 17 June, 2024;
originally announced June 2024.
-
MOSPAT: AutoML based Model Selection and Parameter Tuning for Time Series Anomaly Detection
Authors:
Sourav Chatterjee,
Rohan Bopardikar,
Marius Guerard,
Uttam Thakore,
Xiaodong Jiang
Abstract:
Organizations leverage anomaly and changepoint detection algorithms to detect changes in user behavior or service availability and performance. Many off-the-shelf detection algorithms, though effective, cannot readily be used in large organizations where thousands of users monitor millions of use cases and metrics with varied time series characteristics and anomaly patterns. The selection of algor…
▽ More
Organizations leverage anomaly and changepoint detection algorithms to detect changes in user behavior or service availability and performance. Many off-the-shelf detection algorithms, though effective, cannot readily be used in large organizations where thousands of users monitor millions of use cases and metrics with varied time series characteristics and anomaly patterns. The selection of algorithm and parameters needs to be precise for each use case: manual tuning does not scale, and automated tuning requires ground truth, which is rarely available.
In this paper, we explore MOSPAT, an end-to-end automated machine learning based approach for model and parameter selection, combined with a generative model to produce labeled data. Our scalable end-to-end system allows individual users in large organizations to tailor time-series monitoring to their specific use case and data characteristics, without expert knowledge of anomaly detection algorithms or laborious manual labeling. Our extensive experiments on real and synthetic data demonstrate that this method consistently outperforms using any single algorithm.
△ Less
Submitted 23 May, 2022;
originally announced May 2022.
-
ICLR 2021 Challenge for Computational Geometry & Topology: Design and Results
Authors:
Nina Miolane,
Matteo Caorsi,
Umberto Lupo,
Marius Guerard,
Nicolas Guigui,
Johan Mathe,
Yann Cabanes,
Wojciech Reise,
Thomas Davies,
António Leitão,
Somesh Mohapatra,
Saiteja Utpala,
Shailja Shailja,
Gabriele Corso,
Guoxi Liu,
Federico Iuricich,
Andrei Manolache,
Mihaela Nistor,
Matei Bejan,
Armand Mihai Nicolicioiu,
Bogdan-Alexandru Luchian,
Mihai-Sorin Stupariu,
Florent Michel,
Khanh Dao Duc,
Bilal Abdulrahman
, et al. (8 additional authors not shown)
Abstract:
This paper presents the computational challenge on differential geometry and topology that happened within the ICLR 2021 workshop "Geometric and Topological Representation Learning". The competition asked participants to provide creative contributions to the fields of computational geometry and topology through the open-source repositories Geomstats and Giotto-TDA. The challenge attracted 16 teams…
▽ More
This paper presents the computational challenge on differential geometry and topology that happened within the ICLR 2021 workshop "Geometric and Topological Representation Learning". The competition asked participants to provide creative contributions to the fields of computational geometry and topology through the open-source repositories Geomstats and Giotto-TDA. The challenge attracted 16 teams in its two month duration. This paper describes the design of the challenge and summarizes its main findings.
△ Less
Submitted 25 August, 2021; v1 submitted 22 August, 2021;
originally announced August 2021.
-
Anatomy of a blazar in the (sub-)millimeter: ALMA Observations of PKS 0521-365
Authors:
S. Leon,
P. C. Cortes,
M. Guerard,
E. Villard,
T. Hidayat,
B. Ocana Flaquer,
B. Vila-Vilaro
Abstract:
We aim at analyzing the (sub-)millimeter emission in a nearby blazar, PKS 0521-365 , to study the synchrotron and thermal emission in the different components detected at low frequency. We analyze the archive public data of the ALMA Cycle 0 where PKS 0521-365 is used as a calibrator. A total of 13 projects with 23 dataset is analyzed in band 3, 6 and 7 and combined. The whole set of data is combin…
▽ More
We aim at analyzing the (sub-)millimeter emission in a nearby blazar, PKS 0521-365 , to study the synchrotron and thermal emission in the different components detected at low frequency. We analyze the archive public data of the ALMA Cycle 0 where PKS 0521-365 is used as a calibrator. A total of 13 projects with 23 dataset is analyzed in band 3, 6 and 7 and combined. The whole set of data is combined and wavelet filtered to obtain a deep image reaching a dynamic range of 47000. The individual emission flux is measured at different date over a period of 11 months in various components. Finally we analyze the Spectral Energy Distribution (SED) in each different component, including the radio jet and counter jet. The point sources detected in the field follow a similar distribution to previous studies. The blazar flux shows large variation especially in band 3. Different components are observed: core, radio jet and newly detected counter jet, Hot Spot (HS) and a disky structure roughly perpendicular to the jet. The HS emission is formed by a point source surrounded by an extended emission. The viewing angle of the jet is about 30 with a Doppler factor of 1.6$. The HS is at a distance of 19 kpc from the center. The SED analysis shows a strong variation of the core spectral index, especially in band 3. The two components in the radio jet have roughly a flat spectral index in band 6 and 7. Using these ALMA data the different weak and extended components are detected. The analysis of both jets constrains the geometrical distance of the HS to the center. The SED presents a different shape in time and frequency for each component. Finally a new structure is detected roughly perpendicular to the radio jet and a thermal emission origin is currently favoured. Further observations at higher spatial resolution are needed to confirm that hypothesis.
△ Less
Submitted 26 October, 2015;
originally announced October 2015.
