-
Towards Understanding Sycophancy in Language Models
Authors:
Mrinank Sharma,
Meg Tong,
Tomasz Korbak,
David Duvenaud,
Amanda Askell,
Samuel R. Bowman,
Newton Cheng,
Esin Durmus,
Zac Hatfield-Dodds,
Scott R. Johnston,
Shauna Kravec,
Timothy Maxwell,
Sam McCandlish,
Kamal Ndousse,
Oliver Rausch,
Nicholas Schiefer,
Da Yan,
Miranda Zhang,
Ethan Perez
Abstract:
Human feedback is commonly utilized to finetune AI assistants. But human feedback may also encourage model responses that match user beliefs over truthful ones, a behaviour known as sycophancy. We investigate the prevalence of sycophancy in models whose finetuning procedure made use of human feedback, and the potential role of human preference judgments in such behavior. We first demonstrate that…
▽ More
Human feedback is commonly utilized to finetune AI assistants. But human feedback may also encourage model responses that match user beliefs over truthful ones, a behaviour known as sycophancy. We investigate the prevalence of sycophancy in models whose finetuning procedure made use of human feedback, and the potential role of human preference judgments in such behavior. We first demonstrate that five state-of-the-art AI assistants consistently exhibit sycophancy across four varied free-form text-generation tasks. To understand if human preferences drive this broadly observed behavior, we analyze existing human preference data. We find that when a response matches a user's views, it is more likely to be preferred. Moreover, both humans and preference models (PMs) prefer convincingly-written sycophantic responses over correct ones a non-negligible fraction of the time. Optimizing model outputs against PMs also sometimes sacrifices truthfulness in favor of sycophancy. Overall, our results indicate that sycophancy is a general behavior of state-of-the-art AI assistants, likely driven in part by human preference judgments favoring sycophantic responses.
△ Less
Submitted 27 October, 2023; v1 submitted 20 October, 2023;
originally announced October 2023.
-
Optically Coupled Methods for Microwave Impedance Microscopy
Authors:
Scott R. Johnston,
Eric Yue Ma,
Zhi-Xun Shen
Abstract:
Scanning Microwave Impedance Microscopy (MIM) measurement of photoconductivity with 50 nm resolution is demonstrated using a modulated optical source. The use of a modulated source allows for measurement of photoconductivity in a single scan without a reference region on the sample, as well as removing most topographical artifacts and enhancing signal to noise as compared with unmodulated measurem…
▽ More
Scanning Microwave Impedance Microscopy (MIM) measurement of photoconductivity with 50 nm resolution is demonstrated using a modulated optical source. The use of a modulated source allows for measurement of photoconductivity in a single scan without a reference region on the sample, as well as removing most topographical artifacts and enhancing signal to noise as compared with unmodulated measurement. A broadband light source with tunable monochrometer is then used to measure energy resolved photoconductivity with the same methodology. Finally, a pulsed optical source is used to measure local photo-carrier lifetimes via MIM, using the same 50 nm resolution tip.
△ Less
Submitted 26 January, 2018; v1 submitted 27 October, 2017;
originally announced October 2017.
-
Measurement of Surface Acoustic Wave Resonances in Ferroelectric Domains by Microwave Microscopy
Authors:
Scott R. Johnston,
Yongliang Yang,
Yong-Tao Cui,
Eric Yue Ma,
Thomas Kämpfe,
Lukas M. Eng,
Jian Zhou,
Yan-Feng Chen,
Minghui Lu,
Zhi-Xun Shen
Abstract:
Surface Acoustic Wave (SAW) resonances were imaged within a closed domain in the ferroelectric LiTaO$_3$ via scanning Microwave Impedance Microscopy (MIM). The MIM probe is used for both SAW generation and measurement, allowing contact-less measurement within a mesoscopic structure. Measurements taken over a range of microwave frequencies are consistent with a constant acoustic velocity, demonstra…
▽ More
Surface Acoustic Wave (SAW) resonances were imaged within a closed domain in the ferroelectric LiTaO$_3$ via scanning Microwave Impedance Microscopy (MIM). The MIM probe is used for both SAW generation and measurement, allowing contact-less measurement within a mesoscopic structure. Measurements taken over a range of microwave frequencies are consistent with a constant acoustic velocity, demonstrating the acoustic nature of the measurement.
△ Less
Submitted 1 May, 2017;
originally announced May 2017.
-
Submicrosecond-timescale readout of carbon nanotube mechanical motion
Authors:
H. B. Meerwaldt,
S. R. Johnston,
H. S. J. van der Zant,
G. A. Steele
Abstract:
We report fast readout of the motion of a carbon nanotube mechanical resonator. A close-proximity high electron mobility transistor amplifier is used to increase the bandwidth of the measurement of nanotube displacements from the kHz to the MHz regime. Using an electrical detection scheme with the nanotube acting as a mixer, we detect the amplitude of its mechanical motion at room temperature with…
▽ More
We report fast readout of the motion of a carbon nanotube mechanical resonator. A close-proximity high electron mobility transistor amplifier is used to increase the bandwidth of the measurement of nanotube displacements from the kHz to the MHz regime. Using an electrical detection scheme with the nanotube acting as a mixer, we detect the amplitude of its mechanical motion at room temperature with an intermediate frequency of 6 MHz and a timeconstant of 780 ns, both up to five orders of magnitude faster than achieved before. The transient response of the mechanical motion indicates a ring-down time faster than our enhanced time resolution, placing an upper bound on the contribution of energy relaxation processes to the room temperature mechanical quality factor.
△ Less
Submitted 5 October, 2013;
originally announced October 2013.
