-
Extracting Reward Functions from Diffusion Models
Authors:
Felipe Nuti,
Tim Franzmeyer,
João F. Henriques
Abstract:
Diffusion models have achieved remarkable results in image generation, and have similarly been used to learn high-performing policies in sequential decision-making tasks. Decision-making diffusion models can be trained on lower-quality data, and then be steered with a reward function to generate near-optimal trajectories. We consider the problem of extracting a reward function by comparing a decis…
▽ More
Diffusion models have achieved remarkable results in image generation, and have similarly been used to learn high-performing policies in sequential decision-making tasks. Decision-making diffusion models can be trained on lower-quality data, and then be steered with a reward function to generate near-optimal trajectories. We consider the problem of extracting a reward function by comparing a decision-making diffusion model that models low-reward behavior and one that models high-reward behavior; a setting related to inverse reinforcement learning. We first define the notion of a relative reward function of two diffusion models and show conditions under which it exists and is unique. We then devise a practical learning algorithm for extracting it by aligning the gradients of a reward function -- parametrized by a neural network -- to the difference in outputs of both diffusion models. Our method finds correct reward functions in navigation environments, and we demonstrate that steering the base model with the learned reward functions results in significantly increased performance in standard locomotion benchmarks. Finally, we demonstrate that our approach generalizes beyond sequential decision-making by learning a reward-like function from two large-scale image generation diffusion models. The extracted reward function successfully assigns lower rewards to harmful images.
△ Less
Submitted 9 December, 2023; v1 submitted 1 June, 2023;
originally announced June 2023.
-
Simulation and background characterisation of the SABRE South experiment
Authors:
E. Barberio,
T. Baroncelli,
L. J. Bignell,
I. Bolognino,
G. Brooks,
F. Dastgiri,
G. D'Imperio,
A. Di Giacinto,
A. R. Duffy,
M. Froehlich,
G. Fu,
M. S. M. Gerathy,
G. C. Hill,
S. Krishnan,
G. J. Lane,
G. Lawrence,
K. T. Leaver,
I. Mahmood,
A. Mariani,
P. McGee,
L. J. McKie,
P. C. McNamara,
M. Mews,
W. J. D. Melbourne,
G. Milana
, et al. (16 additional authors not shown)
Abstract:
SABRE (Sodium iodide with Active Background REjection) is a direct detection dark matter experiment based on arrays of radio-pure NaI(Tl) crystals. The experiment aims at achieving an ultra-low background rate and its primary goal is to confirm or refute the results from the DAMA/LIBRA experiment. The SABRE Proof-of-Principle phase was carried out in 2020-2021 at the Gran Sasso National Laboratory…
▽ More
SABRE (Sodium iodide with Active Background REjection) is a direct detection dark matter experiment based on arrays of radio-pure NaI(Tl) crystals. The experiment aims at achieving an ultra-low background rate and its primary goal is to confirm or refute the results from the DAMA/LIBRA experiment. The SABRE Proof-of-Principle phase was carried out in 2020-2021 at the Gran Sasso National Laboratory (LNGS), in Italy. The next phase consists of two full-scale experiments: SABRE South at the Stawell Underground Physics Laboratory, in Australia, and SABRE North at LNGS. This paper focuses on SABRE South and presents a detailed simulation of the detector, which is used to characterise the background for dark matter searches including DAMA/LIBRA-like modulation. We estimate an overall background of 0.72 cpd/kg/keV$_{ee}$ in the energy range 1$-$6 keV$_{ee}$ primarily due to radioactive contamination in the crystals. Given this level of background and considering that the SABRE South has a target mass of 50 kg, we expect to exclude (confirm) DAMA/LIBRA modulation at $4~(5)σ$ within 2.5 years of data taking.
△ Less
Submitted 12 May, 2023; v1 submitted 27 May, 2022;
originally announced May 2022.
-
Pulse Shape Discrimination of low-energy nuclear and electron recoils for improved particle identification in NaI:Tl
Authors:
N. J. Spinks,
L. J. Bignell,
G. J. Lane,
A. Akber,
E. Barberio,
T. Baroncelli,
B. J. Coombes,
J. T. H. Dowie,
T. K. Eriksen,
M. S. M. Gerathy,
T. J. Gray,
I. Mahmood,
B. P. McCormick,
W. J. D. Melbourne,
A. J. Mitchell,
F. Nuti,
M. S. Rahman,
F. Scutti,
A. E. Stuchbery,
H. Timmers,
P. Urquijo,
Y. Y. Zhong,
M. J. Zurowski
Abstract:
The scintillation mechanism in NaI:Tl crystals produces different pulse shapes that are dependent on the incoming particle type. The time distribution of scintillation light from nuclear recoil events decays faster than for electron recoil events and this difference can be categorised using various Pulse Shape Discrimination (PSD) techniques. In this study, we measured nuclear and electron recoils…
▽ More
The scintillation mechanism in NaI:Tl crystals produces different pulse shapes that are dependent on the incoming particle type. The time distribution of scintillation light from nuclear recoil events decays faster than for electron recoil events and this difference can be categorised using various Pulse Shape Discrimination (PSD) techniques. In this study, we measured nuclear and electron recoils in a NaI:Tl crystal, with electron equivalent energies between 2 and 40 keV. We report on a new PSD approach, based on an event-type likelihood; this outperforms the charge-weighted mean-time, which is the conventional metric for PSD in NaI:Tl. Furthermore, we show that a linear combination of the two methods improves the discrimination power at these energies.
△ Less
Submitted 18 May, 2022;
originally announced May 2022.
-
High sensitivity characterization of an ultra-high purity NaI(Tl) crystal scintillator with the SABRE proof-of-principle detector
Authors:
F. Calaprice,
S. Copello,
I. Dafinei,
D. D'Angelo,
G. D'Imperio,
G. Di Carlo,
M. Diemoz,
A. Di Giacinto,
A. Di Ludovico,
A. Ianni,
M. Iannone,
F. Marchegiani,
A. Mariani,
S. Milana,
S. Nisi,
F. Nuti,
D. Orlandi,
V. Pettinacci,
L. Pietrofaccia,
S. Rahatlou,
M. Souza,
B. Suerfu,
C. Tomei,
C. Vignoli,
M. Wada
, et al. (1 additional authors not shown)
Abstract:
We present new results on the radiopurity of a 3.4-kg NaI(Tl) crystal scintillator operated in the SABRE proof-of-principle detector setup. The amount of potassium contamination, determined by the direct counting of radioactive $^{40}$K, is found to be $2.2\pm1.5$ ppb, lowest ever achieved for NaI(Tl) crystals. With the active veto, the average background rate in the crystal in the 1-6 keV energy…
▽ More
We present new results on the radiopurity of a 3.4-kg NaI(Tl) crystal scintillator operated in the SABRE proof-of-principle detector setup. The amount of potassium contamination, determined by the direct counting of radioactive $^{40}$K, is found to be $2.2\pm1.5$ ppb, lowest ever achieved for NaI(Tl) crystals. With the active veto, the average background rate in the crystal in the 1-6 keV energy region-of-interest (ROI) is $1.20\pm0.05$ counts/day/kg/keV, which is a breakthrough since the DAMA/LIBRA experiment. Our background model indicates that the rate is dominated by $^{210}$Pb and that about half of this contamination is located in the PTFE reflector. We discuss ongoing developments of the crystal manufacture aimed at the further reduction of the background, including data from purification by zone refining. A projected background rate lower than $\sim$0.2 counts/day/kg/keV in the ROI is within reach. These results represent a benchmark for the development of next-generation NaI(Tl) detector arrays for the direct detection of dark matter particles.
△ Less
Submitted 19 May, 2021;
originally announced May 2021.
-
Quenching factor measurements of sodium nuclear recoils in NaI:Tl determined by spectrum fitting
Authors:
L. J. Bignell,
I. Mahmood,
F. Nuti,
G. J. Lane,
A. Akber,
E. Barberio,
T. Baroncelli,
B. Coombes,
W. Dix,
J. T. H. Dowie,
T. Eriksen,
M. S. M. Gerathy,
T. J. Gray,
B. P. McCormick,
A. J. Mitchell,
M. S. Rahman,
F. Scutti,
N. J. Spinks,
A. E. Stuchbery,
H. Timmers,
P. Urquijo,
L. Wang,
Y. Y. Zhong,
M. Zurowski
Abstract:
We have performed measurements of sodium nuclear recoils in NaI:Tl crystals, following scattering by neutrons produced in a $^{7}$Li(p,n)$^{7}$Be reaction. Understanding the light output from such recoils, which is reduced relative to electrons of equivalent energy by the quenching factor, is critical to interpret dark matter experiments that search for nuclear scattering interactions. We have dev…
▽ More
We have performed measurements of sodium nuclear recoils in NaI:Tl crystals, following scattering by neutrons produced in a $^{7}$Li(p,n)$^{7}$Be reaction. Understanding the light output from such recoils, which is reduced relative to electrons of equivalent energy by the quenching factor, is critical to interpret dark matter experiments that search for nuclear scattering interactions. We have developed a spectrum-fitting methodology to extract the quenching factor from our measurements, and report quenching factors for nuclear recoil energies between 36 and 401 keV. Our results agree with other recent quenching factor measurements that use quasi-monoenergetic neutron sources. The new method will be applied in the future to the NaI:Tl crystals used in the SABRE experiment.
△ Less
Submitted 15 April, 2021; v1 submitted 4 February, 2021;
originally announced February 2021.
-
Characterization of SABRE crystal NaI-33 with direct underground counting
Authors:
M. Antonello,
I. J. Arnquist,
E. Barberio,
T. Baroncelli,
J. Benziger,
L. J. Bignell,
I. Bolognino,
F. Calaprice,
S. Copello,
I. Dafinei,
D. D'Angelo,
G. D'Imperio,
M. D'Incecco,
G. Di Carlo,
M. Diemoz,
A. Di Giacinto,
A. Di Ludovico,
W. Dix,
A. R. Duffy,
E. Hoppe,
A. Ianni,
M. Iannone,
L. Ioannucci,
S. Krishnan,
G. J. Lane
, et al. (21 additional authors not shown)
Abstract:
Ultra-pure NaI(Tl) crystals are the key element for a model-independent verification of the long standing DAMA result and a powerful means to search for the annual modulation signature of dark matter interactions. The SABRE collaboration has been develo** cutting-edge techniques for the reduction of intrinsic backgrounds over several years. In this paper we report the first characterization of a…
▽ More
Ultra-pure NaI(Tl) crystals are the key element for a model-independent verification of the long standing DAMA result and a powerful means to search for the annual modulation signature of dark matter interactions. The SABRE collaboration has been develo** cutting-edge techniques for the reduction of intrinsic backgrounds over several years. In this paper we report the first characterization of a 3.4 kg crystal, named NaI-33, performed in an underground passive shielding setup at LNGS. NaI-33 has a record low $^{39}$K contamination of 4.3$\pm$0.2 ppb as determined by mass spectrometry. We measured a light yield of 11.1$\pm$0.2 photoelectrons/keV and an energy resolution of 13.2% (FWHM/E) at 59.5 keV. We evaluated the activities of $^{226}$Ra and $^{228}$Th inside the crystal to be $5.9\pm0.6 μ$Bq/kg and $1.6\pm0.3 μ$Bq/kg, respectively, which would indicate a contamination from $^{238}$U and $^{232}$Th at part-per-trillion level. We measured an activity of 0.51$\pm$0.02 mBq/kg due to $^{210}$Pb out of equilibrium and a $α$ quenching factor of 0.63$\pm$0.01 at 5304 keV. We illustrate the analyses techniques developed to reject electronic noise in the lower part of the energy spectrum. A cut-based strategy and a multivariate approach indicated a rate, attributed to the intrinsic radioactivity of the crystal, of $\sim$1 count/day/kg/keV in the [5-20] keV region.
△ Less
Submitted 12 April, 2021; v1 submitted 4 December, 2020;
originally announced December 2020.
-
SABRE and the Stawell Underground Physics Laboratory: Dark Matter Research at the Australian National University
Authors:
Lindsey J. Bignell,
Elisabetta Barberio,
Michaela B. Froehlich,
Gregory J. Lane,
Owen Lennon,
Ibtihal Mahmood,
Francesco Nuti,
Md. Shahinur Rahman,
Cederic Simenel,
Nathan J. Spinks,
Andrew E. Stuchbery,
Heiko Timmers,
Anton Wallner,
Lei Wang,
Jo Wu,
Yi Yi Zhong
Abstract:
The direct detection of dark matter is a key problem in astroparticle physics that generally requires the use of deep-underground laboratories for a low-background environment where the rare signals from dark matter interactions can be observed. This work reports on the Stawell Underground Physics Laboratory - currently under construction and the first such laboratory in the Southern Hemisphere -…
▽ More
The direct detection of dark matter is a key problem in astroparticle physics that generally requires the use of deep-underground laboratories for a low-background environment where the rare signals from dark matter interactions can be observed. This work reports on the Stawell Underground Physics Laboratory - currently under construction and the first such laboratory in the Southern Hemisphere - and the associated research program. A particular focus will be given to ANU's contribution to SABRE, a NaI:Tl dark matter direct detection experiment that aims to confirm or refute the long-standing DAMA result. Preliminary measurements of the NaI:Tl quenching factor and characterisation of the SABRE liquid scintillator veto are reported.
△ Less
Submitted 4 August, 2020;
originally announced August 2020.
-
Monte Carlo simulation of the SABRE PoP background
Authors:
M. Antonello,
E. Barberio,
T. Baroncelli,
J. Benziger,
L. J. Bignell,
I. Bolognino,
F. Calaprice,
S. Copello,
D. D'Angelo,
G. D'Imperio,
I. Dafinei,
G. Di Carlo,
M. Diemoz,
A. Di Ludovico,
A. R. Duffy,
F. Froborg,
G. K. Giovanetti,
E. Hoppe,
A. Ianni,
L. Ioannucci,
S. Krishnan,
G. J. Lane,
I. Mahmood,
A. Mariani,
P. McGee
, et al. (23 additional authors not shown)
Abstract:
SABRE (Sodium-iodide with Active Background REjection) is a direct dark matter search experiment based on an array of radio-pure NaI(Tl) crystals surrounded by a liquid scintillator veto. Twin SABRE experiments in the Northern and Southern Hemispheres will differentiate a dark matter signal from seasonal and local effects. The experiment is currently in a Proof-of-Principle (PoP) phase, whose goal…
▽ More
SABRE (Sodium-iodide with Active Background REjection) is a direct dark matter search experiment based on an array of radio-pure NaI(Tl) crystals surrounded by a liquid scintillator veto. Twin SABRE experiments in the Northern and Southern Hemispheres will differentiate a dark matter signal from seasonal and local effects. The experiment is currently in a Proof-of-Principle (PoP) phase, whose goal is to demonstrate that the background rate is low enough to carry out an independent search for a dark matter signal, with sufficient sensitivity to confirm or refute the DAMA result during the following full-scale experimental phase. The impact of background radiation from the detector materials and the experimental site needs to be carefully investigated, including both intrinsic and cosmogenically activated radioactivity. Based on the best knowledge of the most relevant sources of background, we have performed a detailed Monte Carlo study evaluating the expected background in the dark matter search spectral region. The simulation model described in this paper guides the design of the full-scale experiment and will be fundamental for the interpretation of the measured background and hence for the extraction of a possible dark matter signal.
△ Less
Submitted 14 November, 2018; v1 submitted 25 June, 2018;
originally announced June 2018.
-
The SABRE project and the SABRE PoP
Authors:
M. Antonello,
E. Barberio,
T. Baroncelli,
J. Benziger,
L. J. Bignell,
I. Bolognino,
F. Calaprice,
S. Copello,
D. D'Angelo,
G. D'Imperio,
I. Dafinei,
G. Di Carlo,
M. Diemoz,
A. Di Ludovico,
W. Dix,
A. R. Duffy,
F. Froborg,
G. K. Giovanetti,
E. Hoppe,
A. Ianni,
L. Ioannucci,
S. Krishnan,
G. J. Lane,
I. Mahmood,
A. Mariani
, et al. (24 additional authors not shown)
Abstract:
SABRE aims to directly measure the annual modulation of the dark matter interaction rate with NaI(Tl) crystals. A modulation compatible with the standard hypothesis in which our Galaxy is embedded in a dark matter halo has been measured by the DAMA experiment in the same target material. Other direct detection experiments, using different target materials, seem to exclude the interpretation of suc…
▽ More
SABRE aims to directly measure the annual modulation of the dark matter interaction rate with NaI(Tl) crystals. A modulation compatible with the standard hypothesis in which our Galaxy is embedded in a dark matter halo has been measured by the DAMA experiment in the same target material. Other direct detection experiments, using different target materials, seem to exclude the interpretation of such modulation in the simplest scenario of WIMP-nucleon elastic scattering. The SABRE experiment aims to carry out an independent search with sufficient sensitivity to confirm or refute the DAMA claim. The SABRE concept and goal is to obtain a background rate of the order of 0.1 cpd/kg/keVee in the energy region of interest. This challenging goal is achievable by operating high-purity crystals inside a liquid scintillator veto for active background rejection. In addition, twin detectors will be located in the northern and southern hemispheres to identify possible contributions to the modulation from seasonal or site-related effects. The SABRE project includes an initial Proof-of-Principle phase at LNGS (Italy), to assess the radio-purity of the crystals and the efficiency of the liquid scintillator veto. This paper describes the general concept of SABRE and the expected sensitivity to WIMP annual modulation.
△ Less
Submitted 19 November, 2018; v1 submitted 25 June, 2018;
originally announced June 2018.
