-
Integrating the PanDA Workload Management System with the Vera C. Rubin Observatory
Authors:
Edward Karavakis,
Wen Guan,
Zhaoyu Yang,
Tadashi Maeno,
Torre Wenaus,
Jennifer Adelman-McCarthy,
Fernando Barreiro Megino,
Kaushik De,
Richard Dubois,
Michelle Gower,
Tim Jenness,
Alexei Klimentov,
Tatiana Korchuganova,
Mikolaj Kowalik,
Fa-Hui Lin,
Paul Nilsson,
Sergey Padolski,
Wei Yang,
Shuwei Ye
Abstract:
The Vera C. Rubin Observatory will produce an unprecedented astronomical data set for studies of the deep and dynamic universe. Its Legacy Survey of Space and Time (LSST) will image the entire southern sky every three to four days and produce tens of petabytes of raw image data and associated calibration data over the course of the experiment's run. More than 20 terabytes of data must be stored ev…
▽ More
The Vera C. Rubin Observatory will produce an unprecedented astronomical data set for studies of the deep and dynamic universe. Its Legacy Survey of Space and Time (LSST) will image the entire southern sky every three to four days and produce tens of petabytes of raw image data and associated calibration data over the course of the experiment's run. More than 20 terabytes of data must be stored every night, and annual campaigns to reprocess the entire dataset since the beginning of the survey will be conducted over ten years. The Production and Distributed Analysis (PanDA) system was evaluated by the Rubin Observatory Data Management team and selected to serve the Observatory's needs due to its demonstrated scalability and flexibility over the years, for its Directed Acyclic Graph (DAG) support, its support for multi-site processing, and its highly scalable complex workflows via the intelligent Data Delivery Service (iDDS). PanDA is also being evaluated for prompt processing where data must be processed within 60 seconds after image capture. This paper will briefly describe the Rubin Data Management system and its Data Facilities (DFs). Finally, it will describe in depth the work performed in order to integrate the PanDA system with the Rubin Observatory to be able to run the Rubin Science Pipelines using PanDA.
△ Less
Submitted 8 December, 2023;
originally announced December 2023.
-
The Sloan Digital Sky Survey-II Supernova Survey: Technical Summary
Authors:
Joshua A. Frieman,
B. Bassett,
A. Becker,
C. Choi,
D. Cinabro,
F. DeJongh,
D. L. Depoy,
B. Dilday,
M. Doi,
P. M. Garnavich,
C. J. Hogan,
J. Holtzman,
M. Im,
S. Jha,
R. Kessler,
K. Konishi,
H. Lampeitl,
J. Marriner,
J. L. Marshall,
D. McGinnis,
G. Miknaitis,
R. C. Nichol,
J. L. Prieto,
A. G. Riess,
M. W. Richmond
, et al. (76 additional authors not shown)
Abstract:
The Sloan Digital Sky Survey-II (SDSS-II) has embarked on a multi-year project to identify and measure light curves for intermediate-redshift (0.05 < z < 0.35) Type Ia supernovae (SNe Ia) using repeated five-band (ugriz) imaging over an area of 300 sq. deg. The survey region is a stripe 2.5 degrees wide centered on the celestial equator in the Southern Galactic Cap that has been imaged numerous…
▽ More
The Sloan Digital Sky Survey-II (SDSS-II) has embarked on a multi-year project to identify and measure light curves for intermediate-redshift (0.05 < z < 0.35) Type Ia supernovae (SNe Ia) using repeated five-band (ugriz) imaging over an area of 300 sq. deg. The survey region is a stripe 2.5 degrees wide centered on the celestial equator in the Southern Galactic Cap that has been imaged numerous times in earlier years, enabling construction of a deep reference image for discovery of new objects. Supernova imaging observations are being acquired between 1 September and 30 November of 2005-7. During the first two seasons, each region was imaged on average every five nights. Spectroscopic follow-up observations to determine supernova type and redshift are carried out on a large number of telescopes. In its first two three-month seasons, the survey has discovered and measured light curves for 327 spectroscopically confirmed SNe Ia, 30 probable SNe Ia, 14 confirmed SNe Ib/c, 32 confirmed SNe II, plus a large number of photometrically identified SNe Ia, 94 of which have host-galaxy spectra taken so far. This paper provides an overview of the project and briefly describes the observations completed during the first two seasons of operation.
△ Less
Submitted 20 August, 2007;
originally announced August 2007.
-
The Sixth Data Release of the Sloan Digital Sky Survey
Authors:
Jennifer K. Adelman-McCarthy
Abstract:
With the Sixth Data Release of the Sloan Digital Sky Survey, the imaging of the Northern Galactic Cap is now complete. The survey contains images and parameters of roughly 287 million objects over 9583 deg^2, and 1.27 million spectra of stars, galaxies, quasars and blank sky (for sky subtraction) selected over 7425 deg^2. This release includes much more extensive stellar spectroscopy than previo…
▽ More
With the Sixth Data Release of the Sloan Digital Sky Survey, the imaging of the Northern Galactic Cap is now complete. The survey contains images and parameters of roughly 287 million objects over 9583 deg^2, and 1.27 million spectra of stars, galaxies, quasars and blank sky (for sky subtraction) selected over 7425 deg^2. This release includes much more extensive stellar spectroscopy than previously, and also includes detailed estimates of stellar temperatures, gravities, and metallicities. The results of improved photometric calibration are now available, with uncertainties of roughly 1% in g, r, i, and z, and 2% in u, substantially better than the uncertainties in previous data releases. The spectra in this data release have improved wavelength and flux calibration, especially in the extreme blue and extreme red, leading to the qualitatively better determination of stellar types and radial velocities. The spectrophotometric fluxes are now tied to point spread function magnitudes of stars rather than fiber magnitudes, giving a 0.35 mag change in the spectrophotometric flux scale. Systematic errors in the velocity dispersions of galaxies have been fixed, and the results of two independent codes for determining spectral classifications and redshifts are made available. (Abridged)
△ Less
Submitted 19 October, 2007; v1 submitted 23 July, 2007;
originally announced July 2007.
-
The Fifth Data Release of the Sloan Digital Sky Survey
Authors:
Jennifer K. Adelman-McCarthy
Abstract:
This paper describes the Fifth Data Release (DR5) of the Sloan Digital Sky Survey (SDSS). DR5 includes all survey quality data taken through June 2005 and represents the completion of the SDSS-I project (whose successor, SDSS-II will continue through mid-2008). It includes five-band photometric data for 217 million objects selected over 8000 square degrees, and 1,048,960 spectra of galaxies, qua…
▽ More
This paper describes the Fifth Data Release (DR5) of the Sloan Digital Sky Survey (SDSS). DR5 includes all survey quality data taken through June 2005 and represents the completion of the SDSS-I project (whose successor, SDSS-II will continue through mid-2008). It includes five-band photometric data for 217 million objects selected over 8000 square degrees, and 1,048,960 spectra of galaxies, quasars, and stars selected from 5713 square degrees of that imaging data. These numbers represent a roughly 20% increment over those of the Fourth Data Release; all the data from previous data releases are included in the present release. In addition to "standard" SDSS observations, DR5 includes repeat scans of the southern equatorial stripe, imaging scans across M31 and the core of the Perseus cluster of galaxies, and the first spectroscopic data from SEGUE, a survey to explore the kinematics and chemical evolution of the Galaxy. The catalog database incorporates several new features, including photometric redshifts of galaxies, tables of matched objects in overlap regions of the imaging survey, and tools that allow precise computations of survey geometry for statistical investigations.
△ Less
Submitted 23 July, 2007;
originally announced July 2007.
-
The Sloan Digital Sky Survey Monitor Telescope Pipeline
Authors:
D. L. Tucker,
S. Kent,
M. W. Richmond,
J. Annis,
J. A. Smith,
S. S. Allam,
C. T. Rodgers,
J. L. Stute,
J. K. Adelman-McCarthy,
J. Brinkmann,
M. Doi,
D. Finkbeiner,
M. Fukugita,
J. Goldston,
B. Greenway,
J. E. Gunn,
J. S. Hendry,
D. W. Hogg,
S. -I. Ichikawa,
Z. Ivezic,
G. R. Knapp,
H. Lampeitl,
B. C. Lee,
H. Lin,
T. A. McKay
, et al. (9 additional authors not shown)
Abstract:
The photometric calibration of the Sloan Digital Sky Survey (SDSS) is a multi-step process which involves data from three different telescopes: the 1.0-m telescope at the US Naval Observatory (USNO), Flagstaff Station, Arizona (which was used to establish the SDSS standard star network); the SDSS 0.5-m Photometric Telescope (PT) at the Apache Point Observatory (APO), New Mexico (which calculates…
▽ More
The photometric calibration of the Sloan Digital Sky Survey (SDSS) is a multi-step process which involves data from three different telescopes: the 1.0-m telescope at the US Naval Observatory (USNO), Flagstaff Station, Arizona (which was used to establish the SDSS standard star network); the SDSS 0.5-m Photometric Telescope (PT) at the Apache Point Observatory (APO), New Mexico (which calculates nightly extinctions and calibrates secondary patch transfer fields); and the SDSS 2.5-m telescope at APO (which obtains the imaging data for the SDSS proper).
In this paper, we describe the Monitor Telescope Pipeline, MTPIPE, the software pipeline used in processing the data from the single-CCD telescopes used in the photometric calibration of the SDSS (i.e., the USNO 1.0-m and the PT). We also describe transformation equations that convert photometry on the USNO-1.0m u'g'r'i'z' system to photometry the SDSS 2.5m ugriz system and the results of various validation tests of the MTPIPE software. Further, we discuss the semi-automated PT factory, which runs MTPIPE in the day-to-day standard SDSS operations at Fermilab. Finally, we discuss the use of MTPIPE in current SDSS-related projects, including the Southern u'g'r'i'z' Standard Star project, the u'g'r'i'z' Open Star Clusters project, and the SDSS extension (SDSS-II).
△ Less
Submitted 26 August, 2006;
originally announced August 2006.
-
The Fall 2004 SDSS Supernova Survey
Authors:
Masao Sako,
Roger Romani,
Josh Frieman,
Jen Adelman-McCarthy,
Andrew Becker,
Fritz DeJongh,
Ben Dilday,
Juan Estrada,
John Hendry,
Jon Holtzman,
Jared Kaplan,
Rick Kessler,
Hubert Lampeitl,
John Marriner,
Gajus Miknaitis,
Adam Riess,
Douglas Tucker,
J. Barentine,
R. Blandford,
H. Brewington,
J. Dembicky,
M. Harvanek,
S. Hawley,
C. Hogan,
D. Johnston
, et al. (16 additional authors not shown)
Abstract:
In preparation for the Supernova Survey of the Sloan Digital Sky Survey (SDSS) II, a proposed 3-year extension to the SDSS, we have conducted an early engineering and science run during the fall of 2004, which consisted of approximately 20 scheduled nights of repeated imaging of half of the southern equatorial stripe. Transient supernova-like events were detected in near real-time and photometri…
▽ More
In preparation for the Supernova Survey of the Sloan Digital Sky Survey (SDSS) II, a proposed 3-year extension to the SDSS, we have conducted an early engineering and science run during the fall of 2004, which consisted of approximately 20 scheduled nights of repeated imaging of half of the southern equatorial stripe. Transient supernova-like events were detected in near real-time and photometric measurements were made in the five SDSS filter bandpasses with a cadence of ~2 days. Candidate type Ia supernovae (SNe) were pre-selected based on their colors, light curve shape, and the properties of the host galaxy. Follow-up spectroscopic observations were performed with the Astrophysical Research Consortium 3.5m telescope and the 9.2m Hobby-Eberly Telescope to confirm their types and measure the redshifts. The 2004 campaign resulted in 22 spectroscopically confirmed SNe, which includes 16 type Ia, 5 type II, and 1 type Ib/c. These SN Ia will help fill in the sparsely sampled redshift interval of z = 0.05 - 0.35, the so-called 'redshift desert', in the Hubble diagram. Detailed investigation of the spectral properties of these moderate-redshift SNe Ia will also provide a bridge between local SNe and high-redshift objects, and will help us understand the systematics for future cosmological applications that require high photometric precision. Finally, the large survey volume also provides the opportunity to select unusual supernovae for spectroscopic study that are poorly sampled in other surveys. We report on some of the early results from this program and discuss potential future applications.
△ Less
Submitted 25 April, 2005; v1 submitted 20 April, 2005;
originally announced April 2005.
-
SDSS Data Management and Photometric Quality Assessment
Authors:
Z. Ivezic,
R. H. Lupton,
D. Schlegel,
B. Boroski,
J. Adelman-McCarthy,
B. Yanny,
S. Kent,
C. Stoughton,
D. Finkbeiner,
N. Padmanabhan,
C. M. Rockosi,
J. E. Gunn,
G. R. Knapp,
M. A. Strauss,
G. T. Richards,
D. Eisenstein,
T. Nicinski,
S. J. Kleinman,
J. Krzesinski,
P. R. Newman,
S. Snedden,
A. R. Thakar,
A. Szalay,
J. A. Munn,
J. A. Smith
, et al. (2 additional authors not shown)
Abstract:
We summarize the Sloan Digital Sky Survey data acquisition and processing steps, and describe runQA, a pipeline designed for automated data quality assessment. In particular, we show how the position of the stellar locus in color-color diagrams can be used to estimate the accuracy of photometric zeropoint calibration to better than 0.01 mag in 0.03 deg2 patches. Using this method, we estimate th…
▽ More
We summarize the Sloan Digital Sky Survey data acquisition and processing steps, and describe runQA, a pipeline designed for automated data quality assessment. In particular, we show how the position of the stellar locus in color-color diagrams can be used to estimate the accuracy of photometric zeropoint calibration to better than 0.01 mag in 0.03 deg2 patches. Using this method, we estimate that typical photometric zeropoint calibration errors for SDSS imaging data are not larger than ~0.01 mag in the g, r, and i bands, 0.02 mag in the z band, and 0.03 mag in the u band (root-mean-scatter for zeropoint offsets).
△ Less
Submitted 7 October, 2004;
originally announced October 2004.
