-
Lessons Learned from Sloan Digital Sky Survey Operations
Authors:
S. J. Kleinman,
J. E. Gunn,
B. Boroski,
D. Long,
S. Snedden,
A. Nitta,
J. Krzesiński,
M. Harvanek,
E. Neilsen,
B. Gillespie,
J. C. Barentine,
A. Uomoto,
D. Tucker,
D. York,
S. Jester
Abstract:
Astronomy is changing. Large projects, large collaborations, and large budgets are becoming the norm. The Sloan Digital Sky Survey (SDSS) is one example of this new astronomy, and in operating the original survey, we put in place and learned many valuable operating principles. Scientists sometimes have the tendency to invent everything themselves but when budgets are large, deadlines are many, a…
▽ More
Astronomy is changing. Large projects, large collaborations, and large budgets are becoming the norm. The Sloan Digital Sky Survey (SDSS) is one example of this new astronomy, and in operating the original survey, we put in place and learned many valuable operating principles. Scientists sometimes have the tendency to invent everything themselves but when budgets are large, deadlines are many, and both are tight, learning from others and applying it appropriately can make the difference between success and failure. We offer here our experiences well as our thoughts, opinions, and beliefs on what we learned in operating the SDSS.
△ Less
Submitted 15 October, 2008;
originally announced October 2008.
-
SkyServer Traffic Report - The First Five Years
Authors:
Vik Singh,
Jim Gray,
Ani Thakar,
Alexander S. Szalay,
Jordan Raddick,
Bill Boroski,
Svetlana Lebedeva,
Brian Yanny
Abstract:
The SkyServer is an Internet portal to the Sloan Digital Sky Survey Catalog Archive Server. From 2001 to 2006, there were a million visitors in 3 million sessions generating 170 million Web hits, 16 million ad-hoc SQL queries, and 62 million page views. The site currently averages 35 thousand visitors and 400 thousand sessions per month. The Web and SQL logs are public. We analyzed traffic and s…
▽ More
The SkyServer is an Internet portal to the Sloan Digital Sky Survey Catalog Archive Server. From 2001 to 2006, there were a million visitors in 3 million sessions generating 170 million Web hits, 16 million ad-hoc SQL queries, and 62 million page views. The site currently averages 35 thousand visitors and 400 thousand sessions per month. The Web and SQL logs are public. We analyzed traffic and sessions by duration, usage pattern, data product, and client type (mortal or bot) over time. The analysis shows (1) the site's popularity, (2) the educational website that delivered nearly fifty thousand hours of interactive instruction, (3) the relative use of interactive, programmatic, and batch-local access, (4) the success of offering ad-hoc SQL, personal database, and batch job access to scientists as part of the data publication, (5) the continuing interest in "old" datasets, (6) the usage of SQL constructs, and (7) a novel approach of using the corpus of correct SQL queries to suggest similar but correct statements when a user presents an incorrect SQL statement.
△ Less
Submitted 26 January, 2007;
originally announced January 2007.
-
SDSS Data Management and Photometric Quality Assessment
Authors:
Z. Ivezic,
R. H. Lupton,
D. Schlegel,
B. Boroski,
J. Adelman-McCarthy,
B. Yanny,
S. Kent,
C. Stoughton,
D. Finkbeiner,
N. Padmanabhan,
C. M. Rockosi,
J. E. Gunn,
G. R. Knapp,
M. A. Strauss,
G. T. Richards,
D. Eisenstein,
T. Nicinski,
S. J. Kleinman,
J. Krzesinski,
P. R. Newman,
S. Snedden,
A. R. Thakar,
A. Szalay,
J. A. Munn,
J. A. Smith
, et al. (2 additional authors not shown)
Abstract:
We summarize the Sloan Digital Sky Survey data acquisition and processing steps, and describe runQA, a pipeline designed for automated data quality assessment. In particular, we show how the position of the stellar locus in color-color diagrams can be used to estimate the accuracy of photometric zeropoint calibration to better than 0.01 mag in 0.03 deg2 patches. Using this method, we estimate th…
▽ More
We summarize the Sloan Digital Sky Survey data acquisition and processing steps, and describe runQA, a pipeline designed for automated data quality assessment. In particular, we show how the position of the stellar locus in color-color diagrams can be used to estimate the accuracy of photometric zeropoint calibration to better than 0.01 mag in 0.03 deg2 patches. Using this method, we estimate that typical photometric zeropoint calibration errors for SDSS imaging data are not larger than ~0.01 mag in the g, r, and i bands, 0.02 mag in the z band, and 0.03 mag in the u band (root-mean-scatter for zeropoint offsets).
△ Less
Submitted 7 October, 2004;
originally announced October 2004.
