-
Position: Stop Making Unscientific AGI Performance Claims
Authors:
Patrick Altmeyer,
Andrew M. Demetriou,
Antony Bartlett,
Cynthia C. S. Liem
Abstract:
Developments in the field of Artificial Intelligence (AI), and particularly large language models (LLMs), have created a 'perfect storm' for observing 'sparks' of Artificial General Intelligence (AGI) that are spurious. Like simpler models, LLMs distill meaningful representations in their latent embeddings that have been shown to correlate with external variables. Nonetheless, the correlation of s…
▽ More
Developments in the field of Artificial Intelligence (AI), and particularly large language models (LLMs), have created a 'perfect storm' for observing 'sparks' of Artificial General Intelligence (AGI) that are spurious. Like simpler models, LLMs distill meaningful representations in their latent embeddings that have been shown to correlate with external variables. Nonetheless, the correlation of such representations has often been linked to human-like intelligence in the latter but not the former. We probe models of varying complexity including random projections, matrix decompositions, deep autoencoders and transformers: all of them successfully distill information that can be used to predict latent or external variables and yet none of them have previously been linked to AGI. We argue and empirically demonstrate that the finding of meaningful patterns in latent spaces of models cannot be seen as evidence in favor of AGI. Additionally, we review literature from the social sciences that shows that humans are prone to seek such patterns and anthropomorphize. We conclude that both the methodological setup and common public image of AI are ideal for the misinterpretation that correlations between model representations and some variables of interest are 'caused' by the model's understanding of underlying 'ground truth' relationships. We, therefore, call for the academic community to exercise extra caution, and to be keenly aware of principles of academic integrity, in interpreting and communicating about AI research outcomes.
△ Less
Submitted 31 May, 2024; v1 submitted 6 February, 2024;
originally announced February 2024.
-
Procoli: Profiles of cosmological likelihoods
Authors:
Tanvi Karwal,
Yashvi Patel,
Alexa Bartlett,
Vivian Poulin,
Tristan L. Smith,
Daniel N. Pfeffer
Abstract:
Frequentist profile likelihoods have seen a resurgence in cosmology, offering an alternative to Bayesian methods as they can circumvent the impact of prior-volume effects. This paper presents Procoli, a fast and accessible package to obtain profile likelihoods in cosmology, available on GitHub and PyPI. Procoli seamlessly integrates with MontePython, incorporating all its available data likelihood…
▽ More
Frequentist profile likelihoods have seen a resurgence in cosmology, offering an alternative to Bayesian methods as they can circumvent the impact of prior-volume effects. This paper presents Procoli, a fast and accessible package to obtain profile likelihoods in cosmology, available on GitHub and PyPI. Procoli seamlessly integrates with MontePython, incorporating all its available data likelihoods, as well as any modified versions of CLASS. This paper provides a comprehensive overview of the Procoli code, detailing the simulated-annealing optimizer at its core and the sequential computation of the profile. An an example, we use the early dark energy model which is afflicted by prior-volume effects to illustrate the code's features. We validate its optimizer with mock data, and compare optimization techniques for both the global minimum and the profile. Procoli further enables splitting profiles into their component contributions from individual experiments, offering nuanced insights into the data and model. As a valuable addition to the cosmologist's toolkit, Procoli supplements existing Bayesian codes, contributing to more robust parameter constraints in cosmological studies.
△ Less
Submitted 25 January, 2024;
originally announced January 2024.
-
The weak, the strong and the ugly -- A comparative analysis of interacting stepped dark radiation
Authors:
Nils Schöneberg,
Guillermo Franco Abellán,
Théo Simon,
Alexa Bartlett,
Yashvi Patel,
Tristan L. Smith
Abstract:
Models which address both the Hubble and $S_8$ tensions with the same mechanism generically cause a pre-recombination suppression of the small scale matter power spectrum. Here we focus on two such models. Both models introduce a self-interacting dark radiation fluid scattering with dark matter, which has a step in its abundance around some transition redshift. In one model, the interaction is wea…
▽ More
Models which address both the Hubble and $S_8$ tensions with the same mechanism generically cause a pre-recombination suppression of the small scale matter power spectrum. Here we focus on two such models. Both models introduce a self-interacting dark radiation fluid scattering with dark matter, which has a step in its abundance around some transition redshift. In one model, the interaction is weak and with all of the dark matter whereas in the other it is strong but with only a fraction of the dark matter. The weakly interacting case is able to address both tensions simultaneously and provide a good fit to a the Planck measurements of the cosmic microwave background (CMB), the Pantheon Type Ia supernovae, and a combination of low and high redshift baryon acoustic oscillation data, whereas the strongly interacting model cannot significantly ease both tensions simultaneously. The addition of high-resolution cosmic microwave background (CMB) measurements (ACT DR4 and SPT-3G) slightly limits both model's ability to address the Hubble tension. The use of the effective field theory of large-scale structures analysis of BOSS DR12 LRG and eBOSS DR16 QSO data additionally limits their ability to address the $S_8$ tension. We explore how these models respond to these data sets in detail in order to draw general conclusions about what is required for a mechanism to address both tensions. We find that in order to fit the CMB data the time dependence of the suppression of the matter power spectrum plays a central role.
△ Less
Submitted 12 March, 2024; v1 submitted 21 June, 2023;
originally announced June 2023.
-
Dark Energy at early times and ACT: a larger Hubble constant without late-time priors
Authors:
Vivian Poulin,
Tristan L. Smith,
Alexa Bartlett
Abstract:
In this paper we fit two models of Early Dark Energy (EDE) (an increase in the expansion rate before recombination) to the combination of Atacama Cosmology Telescope (ACT) measurements of the Cosmic Microwave Background (CMB) with data from either the WMAP or the Planck satellite, along with measurements of the baryon acoustic oscillations and uncalibrated supernovae luminosity distance. We study…
▽ More
In this paper we fit two models of Early Dark Energy (EDE) (an increase in the expansion rate before recombination) to the combination of Atacama Cosmology Telescope (ACT) measurements of the Cosmic Microwave Background (CMB) with data from either the WMAP or the Planck satellite, along with measurements of the baryon acoustic oscillations and uncalibrated supernovae luminosity distance. We study a phenomenological axion-like potential ('axEDE') and a scalar field experiencing a first-order phase-transition ('NEDE'). We find that for both models the 'Planck-free' analysis yields non-zero EDE at > 2 sigma and an increased value for $H_0 \sim 70-74$ km/s/Mpc, compatible with local measurements, without the inclusion of any prior on $H_0$. On the other hand, the inclusion of Planck data restricts the EDE contribution to an upper-limit only at 95% C.L. For axEDE, the combination of Planck and ACT leads to constraints 30% weaker than with Planck alone, and there is no residual Hubble tension. On the other hand, NEDE is more strongly constrained in a Planck+ACT analysis, and the Hubble tension remains at $\sim 3σ$, illustrating the ability for CMB data to distinguish between EDE models. We explore the apparent inconsistency between the Planck and ACT data and find that it comes (mostly) from a slight tension between the temperature power spectrum at multipoles around $\sim 1000$ and $\sim 1500$. Finally, through a mock analysis of ACT data, we demonstrate that the preference for EDE is not driven by a lack of information at high-$\ell$ when removing Planck data, and that a LCDM fit to the fiducial EDE cosmology results in a significant bias on $\{H_0,ω_{\rm cdm}\}$. More accurate measurements of the TT power spectra above $\ell\sim 2500$ and EE between $\ell \sim 300-500$ will play a crucial role in differentiating EDE models.
△ Less
Submitted 24 September, 2021; v1 submitted 13 September, 2021;
originally announced September 2021.
-
Don't answer the question! How on-line Moodle-based 'Data Retrieval Tests' encourage good record-kee** and a divergent experimental mindset for undergraduate physics students
Authors:
Paul Andrew Bartlett
Abstract:
The use of Data Retrieval Tests (DRTs), as an alternative to physics laboratory notebook marking, is discussed. The implementation of a Moodle-based, on-line DRT for 1st year physics students is described. The advantages of using such a methodology are highlighted and student comments shown. The paper also describes how students change their behaviour as a consequence of having an end of module DR…
▽ More
The use of Data Retrieval Tests (DRTs), as an alternative to physics laboratory notebook marking, is discussed. The implementation of a Moodle-based, on-line DRT for 1st year physics students is described. The advantages of using such a methodology are highlighted and student comments shown. The paper also describes how students change their behaviour as a consequence of having an end of module DRT via 'bootstrap**', both singly and in peer groups.
△ Less
Submitted 30 May, 2019;
originally announced May 2019.
-
Secret objectives: promoting inquiry and tackling preconceptions in teaching laboratories
Authors:
P. A. Bartlett,
K. Dunnett
Abstract:
In its most general form, a `secret objective' is any inconsistency between the experimental reality and the information provided to students prior to starting work on an experiment. Students are challenged to identify the secret objectives and then given freedom to explore and understand the experiment, thus encouraging and facilitating genuine inquiry elements in introductory laboratory courses.…
▽ More
In its most general form, a `secret objective' is any inconsistency between the experimental reality and the information provided to students prior to starting work on an experiment. Students are challenged to identify the secret objectives and then given freedom to explore and understand the experiment, thus encouraging and facilitating genuine inquiry elements in introductory laboratory courses. Dam** of a simple pendulum is used as a concrete example to demonstrate how secret objectives can be included. We also discuss the implications of the secret objectives method and how this can provide a link between the concepts of problem based learning and inquiry style labs.
△ Less
Submitted 17 May, 2019;
originally announced May 2019.
-
\txtit{In situ} determination of the anisotropy field in ferromagnetic films using magnetic susceptibility measurements by MOKE
Authors:
A. Bartlett,
R. Belanger,
M. Amman,
D. Venus
Abstract:
An alternate method of measuring anisotropy fields in thin film ferromagnets is demonstrated. The method relies on the magnetic susceptibility in a small a.c. magnetic field, measured \txtit{in situ} using the magneto-optic Kerr effect (MOKE), and will be useful in situations where more specialized apparatus are not available, or constraints discourage the use of a large, static magnetic field. Th…
▽ More
An alternate method of measuring anisotropy fields in thin film ferromagnets is demonstrated. The method relies on the magnetic susceptibility in a small a.c. magnetic field, measured \txtit{in situ} using the magneto-optic Kerr effect (MOKE), and will be useful in situations where more specialized apparatus are not available, or constraints discourage the use of a large, static magnetic field. The method is demonstrated for Co/W(110) films, where it yields anisotropy fields in agreement with previous studies using more conventional torque magnetometry. The sensitivity of the method is demonstrated using CoO/Co/W(110) bilayer films, where the anisotropy due to interfacial exchange coupling is detected and used to find the Néel temperature of the thin CoO layer.
△ Less
Submitted 9 April, 2019;
originally announced April 2019.
-
Control of Intramolecular Electron Transfer in Perylene Dihydrazides and Perylene Diimides: A Comparative Study by Time-Resolved Spectroscopy
Authors:
Robin C. Döring,
Eduard Baal,
Malcolm A. Bartlett,
Christian Prinzisky,
Remco W. A. Havenith,
Jörg Sundermeyer,
Sangam Chatterjee
Abstract:
Electron transfer (ET) in molecular donor-acceptor dye systems is crucial for charge transport in organic semiconductors. Classically, ET rates should decrease with increasing donor-acceptor distance while the microscopic mechanism is more complex and shows intricate dependencies on the excitation conditions. In this paper, we introduce highly soluble N,N'-dialkyl perylene dihydrazides (PDH) - per…
▽ More
Electron transfer (ET) in molecular donor-acceptor dye systems is crucial for charge transport in organic semiconductors. Classically, ET rates should decrease with increasing donor-acceptor distance while the microscopic mechanism is more complex and shows intricate dependencies on the excitation conditions. In this paper, we introduce highly soluble N,N'-dialkyl perylene dihydrazides (PDH) - perylene dyes with a dialkylamino -NR$_2$ donor functionality directly bonded to both of their imide nitrogen atoms. We compare the PDH electron-transfer dynamics with a group of classical N,N'-bisalkylperylene diimides (PDI) equipped with a -NR$_2$ donor linked to the PDI acceptor core via a varying number of alkylene -(CH$_2$)- spacer groups, thus at distinctively different distance. Special physicochemical design features of our study objects include: i) amine moieties as donor group to minimize spin-orbit coupling; ii) substitution solely at both imide positions to avoid major impact on HOMO and LUMO levels and distortions of the PDI backbone; iii) control of donor-acceptor separation by non-conjugated alkylene groups to exclude any additional effects due to delocalized $π$ electron systems. All materials show non-single-exponential photoluminescence decay dynamics. A rate equation analysis supported by electrochemical and absolute photoluminescence efficiency measurements yields evidence for efficient intersystem-crossing without heavy elements and reveals that the charge-transfer efficiency across the intramolecular interface strongly depends on the surplus excitation energy.
△ Less
Submitted 15 December, 2016;
originally announced December 2016.
-
The Ecology of Fringe Science and its Bearing on Policy
Authors:
HM Collins,
A Bartlett,
LI Reyes-Galindo
Abstract:
In this paper we illustrate the tension between mainstream 'normal', 'unorthodox' and 'fringe' science that is the focus of two ongoing projects that are analysing the full ecology of physics knowledge. The first project concentrates on empirically understanding the notion of consensus in physics by investigating the policing of boundaries that is carried out at the arXiv preprint server, a fundam…
▽ More
In this paper we illustrate the tension between mainstream 'normal', 'unorthodox' and 'fringe' science that is the focus of two ongoing projects that are analysing the full ecology of physics knowledge. The first project concentrates on empirically understanding the notion of consensus in physics by investigating the policing of boundaries that is carried out at the arXiv preprint server, a fundamental element of the contemporary physics publishing landscape. The second project looks at physics outside the mainstream and focuses on the set of organisations and publishing outlets that have mushroomed outside of mainstream physics to cover the needs of 'alternative', 'independent' and 'unorthodox' scientists. Consolidating both projects into the different images of science that characterise the mainstream (based on consensus) and the fringe (based on dissent), we draw out an explanation of why today's social scientists ought to make the case that, for policy-making purposes, the mainstream's consensus should be our main source of technical knowledge.
△ Less
Submitted 18 June, 2016;
originally announced June 2016.
-
Spectral Theory of Substitutions in $\mathbb{Z}^d$
Authors:
Alan Bartlett
Abstract:
In this paper, we generalize and develop results of Queffelec allowing us to characterize the spectrum of an aperiodic substitution in $\mathbb{Z}^d$ by describing the Fourier coefficients of mutually singular measures of pure type giving rise to the maximal spectral type of its Koopman representation. We note that this is done without the assumptions of primitivity or trivial height, and provides…
▽ More
In this paper, we generalize and develop results of Queffelec allowing us to characterize the spectrum of an aperiodic substitution in $\mathbb{Z}^d$ by describing the Fourier coefficients of mutually singular measures of pure type giving rise to the maximal spectral type of its Koopman representation. We note that this is done without the assumptions of primitivity or trivial height, and provides a simple algorithm for determining singularity to Lebesgue spectrum for such substitutions. This is used to show that the spectrum of any aperiodic bijective and commutative $\mathbb{Z}^d$ substitution on a finite alphabet is purely singular. Additionally, we use the algorithm to show singularity of the spectrum for Queffelec's noncommutative bijective substitution, as well as the Table tiling, answering an open question of Solomyak.
△ Less
Submitted 18 July, 2016; v1 submitted 29 October, 2014;
originally announced October 2014.