-
The Future of Astronomical Data Infrastructure: Meeting Report
Authors:
Michael R. Blanton,
Janet D. Evans,
Dara Norman,
William O'Mullane,
Adrian Price-Whelan,
Luca Rizzi,
Alberto Accomazzi,
Megan Ansdell,
Stephen Bailey,
Paul Barrett,
Steven Berukoff,
Adam Bolton,
Julian Borrill,
Kelle Cruz,
Julianne Dalcanton,
Vandana Desai,
Gregory P. Dubois-Felsmann,
Frossie Economou,
Henry Ferguson,
Bryan Field,
Dan Foreman-Mackey,
Jaime Forero-Romero,
Niall Gaffney,
Kim Gillies,
Matthew J. Graham
, et al. (47 additional authors not shown)
Abstract:
The astronomical community is grappling with the increasing volume and complexity of data produced by modern telescopes, due to difficulties in reducing, accessing, analyzing, and combining archives of data. To address this challenge, we propose the establishment of a coordinating body, an "entity," with the specific mission of enhancing the interoperability, archiving, distribution, and productio…
▽ More
The astronomical community is grappling with the increasing volume and complexity of data produced by modern telescopes, due to difficulties in reducing, accessing, analyzing, and combining archives of data. To address this challenge, we propose the establishment of a coordinating body, an "entity," with the specific mission of enhancing the interoperability, archiving, distribution, and production of both astronomical data and software. This report is the culmination of a workshop held in February 2023 on the Future of Astronomical Data Infrastructure. Attended by 70 scientists and software professionals from ground-based and space-based missions and archives spanning the entire spectrum of astronomical research, the group deliberated on the prevailing state of software and data infrastructure in astronomy, identified pressing issues, and explored potential solutions. In this report, we describe the ecosystem of astronomical data, its existing flaws, and the many gaps, duplication, inconsistencies, barriers to access, drags on productivity, missed opportunities, and risks to the long-term integrity of essential data sets. We also highlight the successes and failures in a set of deep dives into several different illustrative components of the ecosystem, included as an appendix.
△ Less
Submitted 7 November, 2023;
originally announced November 2023.
-
Software Architecture and System Design of Rubin Observatory
Authors:
William O'Mullane,
Frossie Economou,
Kian-Tat Lim,
Fritz Mueller,
Tim Jenness,
Gregory P. Dubois-Felsmann,
Leanne P. Guy,
Ian S. Sullivan,
Yusra AlSayyad,
John D. Swinbank,
K. Simon Krughoff
Abstract:
Starting from a description of the Rubin Observatory Data Management System Architecture, and drawing on our experience with and involvement in a range of other projects including Gaia, SDSS, UKIRT, and JCMT, we derive a series of generic design patterns and lessons learned.
Starting from a description of the Rubin Observatory Data Management System Architecture, and drawing on our experience with and involvement in a range of other projects including Gaia, SDSS, UKIRT, and JCMT, we derive a series of generic design patterns and lessons learned.
△ Less
Submitted 24 November, 2022;
originally announced November 2022.
-
Rubin Science Platform on Google: the story so far
Authors:
William O'Mullane,
Frossie Economou,
Flora Huang,
Dan Speck,
Hsin-Fang Chiang,
Melissa L. Graham,
Russ Allbery,
Christine Banek,
Jonathan Sick,
Adam J. Thornton,
Jess Masciarelli,
Kian-Tat Lim,
Fritz Mueller,
Sergey Padolski,
Tim Jenness,
K. Simon Krughoff,
Michelle Gower,
Leanne P. Guy,
Gregory P. Dubois-Felsmann
Abstract:
We describe Rubin Observatory's experience with offering a data access facility (and associated services including our Science Platform) deployed on Google Cloud infrastructure as part of our pre-Operations Data Preview program.
We describe Rubin Observatory's experience with offering a data access facility (and associated services including our Science Platform) deployed on Google Cloud infrastructure as part of our pre-Operations Data Preview program.
△ Less
Submitted 29 November, 2021;
originally announced November 2021.
-
Why is the LSST Science Platform built on Kubernetes?
Authors:
Christine Banek,
Adam Thornton,
Frossie Economou,
Angelo Fausti,
K. Simon Krughoff,
Jonathan Sick
Abstract:
LSST has chosen Kubernetes as the platform for deploying and operating the LSST Science Platform. We first present the background reasoning behind this decision, including both instrument-agnostic as well as LSST-specific requirements. We then discuss the basic principles of Kubernetes and Helm, and how they are used as the deployment base for the LSST Science Platform. Furthermore, we provide an…
▽ More
LSST has chosen Kubernetes as the platform for deploying and operating the LSST Science Platform. We first present the background reasoning behind this decision, including both instrument-agnostic as well as LSST-specific requirements. We then discuss the basic principles of Kubernetes and Helm, and how they are used as the deployment base for the LSST Science Platform. Furthermore, we provide an example of how an external group may use these publicly available software resources to deploy their own instance of the LSST Science Platform, and customize it to their needs. Finally, we discuss how more astronomy software can follow these patterns to gain similar benefits.
△ Less
Submitted 14 November, 2019;
originally announced November 2019.
-
The Growing Importance of a Tech Savvy Astronomy and Astrophysics Workforce
Authors:
Dara Norman,
Kelle Cruz,
Vandana Desai,
Britt Lundgren,
Eric Bellm,
Frossie Economou,
Arfon Smith,
Amanda Bauer,
Brian Nord,
Chad Schafer,
Gautham Narayan,
Ting Li,
Erik Tollerud,
Brigitta Sipocz,
Heloise Stevance,
Timothy Pickering,
Manodeep Sinha,
Joseph Harrington,
Jeyhan Kartaltepe,
Dany Vohl,
Adrian Price-Whelan,
Brian Cherinka,
Chi-kwan Chan,
Benjamin Weiner,
Maryam Modjaz
, et al. (4 additional authors not shown)
Abstract:
Fundamental coding and software development skills are increasingly necessary for success in nearly every aspect of astronomical and astrophysical research as large surveys and high resolution simulations become the norm. However, professional training in these skills is inaccessible or impractical for many members of our community. Students and professionals alike have been expected to acquire th…
▽ More
Fundamental coding and software development skills are increasingly necessary for success in nearly every aspect of astronomical and astrophysical research as large surveys and high resolution simulations become the norm. However, professional training in these skills is inaccessible or impractical for many members of our community. Students and professionals alike have been expected to acquire these skills on their own, apart from formal classroom curriculum or on-the-job training. Despite the recognized importance of these skills, there is little opportunity to develop them - even for interested researchers. To ensure a workforce capable of taking advantage of the computational resources and the large volumes of data coming in the next decade, we must identify and support ways to make software development training widely accessible to community members, regardless of affiliation or career level. To develop and sustain a technology capable astronomical and astrophysical workforce, we recommend that agencies make funding and other resources available in order to encourage, support and, in some cases, require progress on necessary training, infrastructure and policies. In this white paper, we focus on recommendations for how funding agencies can lead in the promotion of activities to support the astronomy and astrophysical workforce in the 2020s.
△ Less
Submitted 17 October, 2019;
originally announced October 2019.
-
The demise of the filesystem and multi level service architecture
Authors:
William O'Mullane,
Niall Gaffney,
Frossie Economou,
Arfon M. Smith,
J. Ross Thomson,
Tim Jenness
Abstract:
Many astronomy data centres still work on filesystems. Industry has moved on; current practice in computing infrastructure is to achieve Big Data scalability using object stores rather than POSIX file systems. This presents us with opportunities for portability and reuse of software underlying processing and archive systems but it also causes problems for legacy implementations in current data cen…
▽ More
Many astronomy data centres still work on filesystems. Industry has moved on; current practice in computing infrastructure is to achieve Big Data scalability using object stores rather than POSIX file systems. This presents us with opportunities for portability and reuse of software underlying processing and archive systems but it also causes problems for legacy implementations in current data centers.
△ Less
Submitted 31 July, 2019; v1 submitted 26 July, 2019;
originally announced July 2019.
-
Astro2020 APC White Paper: Elevating the Role of Software as a Product of the Research Enterprise
Authors:
Arfon M. Smith,
Dara Norman,
Kelle Cruz,
Vandana Desai,
Eric Bellm,
Britt Lundgren,
Frossie Economou,
Brian D. Nord,
Chad Schafer,
Gautham Narayan,
Joseph Harrington,
Erik Tollerud,
Brigitta Sipőcz,
Timothy Pickering,
Molly S. Peeples,
Bruce Berriman,
Peter Teuben,
David Rodriguez,
Andre Gradvohl,
Lior Shamir,
Alice Allen,
Joel R. Brownstein,
Adam Ginsburg,
Manodeep Sinha,
Cameron Hummels
, et al. (20 additional authors not shown)
Abstract:
Software is a critical part of modern research, and yet there are insufficient mechanisms in the scholarly ecosystem to acknowledge, cite, and measure the impact of research software. The majority of academic fields rely on a one-dimensional credit model whereby academic articles (and their associated citations) are the dominant factor in the success of a researcher's career. In the petabyte era o…
▽ More
Software is a critical part of modern research, and yet there are insufficient mechanisms in the scholarly ecosystem to acknowledge, cite, and measure the impact of research software. The majority of academic fields rely on a one-dimensional credit model whereby academic articles (and their associated citations) are the dominant factor in the success of a researcher's career. In the petabyte era of astronomical science, citing software and measuring its impact enables academia to retain and reward researchers that make significant software contributions. These highly skilled researchers must be retained to maximize the scientific return from petabyte-scale datasets. Evolving beyond the one-dimensional credit model requires overcoming several key challenges, including the current scholarly ecosystem and scientific culture issues. This white paper will present these challenges and suggest practical solutions for elevating the role of software as a product of the research enterprise.
△ Less
Submitted 14 July, 2019;
originally announced July 2019.
-
Astro2020 APC White Paper: Astronomy should be in the clouds
Authors:
Arfon M. Smith,
Rob Pike,
William O'Mullane,
Frossie Economou,
Adam Bolton,
Ivelina Momcheva,
Amanda E Bauer,
Bruce Becker,
Eric Bellm,
Andrew Connolly,
Steven M. Crawford,
Nimish Hathi,
Peter Melchior,
Joshua Peek,
Arif Solmaz,
Ross Thomson,
Erik TollerudI,
David W. Liska
Abstract:
Commodity cloud computing, as provided by commercial vendors such as Amazon, Google, and Microsoft, has revolutionized computing in many sectors. With the advent of a new class of big data, public access astronomical facility such as LSST, DKIST, and WFIRST, there exists a real opportunity to combine these missions with cloud computing platforms and fundamentally change the way astronomical data i…
▽ More
Commodity cloud computing, as provided by commercial vendors such as Amazon, Google, and Microsoft, has revolutionized computing in many sectors. With the advent of a new class of big data, public access astronomical facility such as LSST, DKIST, and WFIRST, there exists a real opportunity to combine these missions with cloud computing platforms and fundamentally change the way astronomical data is collected, processed, archived, and curated. Making these changes in a cross-mission, coordinated way can provide unprecedented economies of scale in personnel, data collection and management, archiving, algorithm and software development and, most importantly, science.
△ Less
Submitted 14 July, 2019;
originally announced July 2019.
-
Petabytes to Science
Authors:
Amanda E. Bauer,
Eric C. Bellm,
Adam S. Bolton,
Surajit Chaudhuri,
A. J. Connolly,
Kelle L. Cruz,
Vandana Desai,
Alex Drlica-Wagner,
Frossie Economou,
Niall Gaffney,
J. Kavelaars,
J. Kinney,
Ting S. Li,
B. Lundgren,
R. Margutti,
G. Narayan,
B. Nord,
Dara J. Norman,
W. O'Mullane,
S. Padhi,
J. E. G. Peek,
C. Schafer,
Megan E. Schwamb,
Arfon M. Smith,
Erik J. Tollerud
, et al. (2 additional authors not shown)
Abstract:
A Kavli foundation sponsored workshop on the theme \emph{Petabytes to Science} was held 12$^{th}$ to 14$^{th}$ of February 2019 in Las Vegas. The aim of the this workshop was to discuss important trends and technologies which may support astronomy. We also tackled how to better shape the workforce for the new trends and how we should approach education and public outreach. This document was coauth…
▽ More
A Kavli foundation sponsored workshop on the theme \emph{Petabytes to Science} was held 12$^{th}$ to 14$^{th}$ of February 2019 in Las Vegas. The aim of the this workshop was to discuss important trends and technologies which may support astronomy. We also tackled how to better shape the workforce for the new trends and how we should approach education and public outreach. This document was coauthored during the workshop and edited in the weeks after. It comprises the discussions and highlights many recommendations which came out of the workshop.
We shall distill parts of this document and formulate potential white papers for the decadal survey.
△ Less
Submitted 17 November, 2019; v1 submitted 13 May, 2019;
originally announced May 2019.
-
The LSST Data Management System
Authors:
Mario Jurić,
Jeffrey Kantor,
K-T Lim,
Robert H. Lupton,
Gregory Dubois-Felsmann,
Tim Jenness,
Tim S. Axelrod,
Jovan Aleksić,
Roberta A. Allsman,
Yusra AlSayyad,
Jason Alt,
Robert Armstrong,
Jim Basney,
Andrew C. Becker,
Jacek Becla,
Steven J. Bickerton,
Rahul Biswas,
James Bosch,
Dominique Boutigny,
Matias Carrasco Kind,
David R. Ciardi,
Andrew J. Connolly,
Scott F. Daniel,
Gregory E. Daues,
Frossie Economou
, et al. (40 additional authors not shown)
Abstract:
The Large Synoptic Survey Telescope (LSST) is a large-aperture, wide-field, ground-based survey system that will image the sky in six optical bands from 320 to 1050 nm, uniformly covering approximately $18,000$deg$^2$ of the sky over 800 times. The LSST is currently under construction on Cerro Pachón in Chile, and expected to enter operations in 2022. Once operational, the LSST will explore a wide…
▽ More
The Large Synoptic Survey Telescope (LSST) is a large-aperture, wide-field, ground-based survey system that will image the sky in six optical bands from 320 to 1050 nm, uniformly covering approximately $18,000$deg$^2$ of the sky over 800 times. The LSST is currently under construction on Cerro Pachón in Chile, and expected to enter operations in 2022. Once operational, the LSST will explore a wide range of astrophysical questions, from discovering "killer" asteroids to examining the nature of Dark Energy.
The LSST will generate on average 15 TB of data per night, and will require a comprehensive Data Management system to reduce the raw data to scientifically useful catalogs and images with minimum human intervention. These reductions will result in a real-time alert stream, and eleven data releases over the 10-year duration of LSST operations. To enable this processing, the LSST project is develo** a new, general-purpose, high-performance, scalable, well documented, open source data processing software stack for O/IR surveys. Prototypes of this stack are already capable of processing data from existing cameras (e.g., SDSS, DECam, MegaCam), and form the basis of the Hyper-Suprime Cam (HSC) Survey data reduction pipeline.
△ Less
Submitted 24 December, 2015;
originally announced December 2015.
-
Significant problems in FITS limit its use in modern astronomical research
Authors:
Brian Thomas,
Tim Jenness,
Frossie Economou,
Perry Greenfield,
Paul Hirst,
David S. Berry,
Erik Bray,
Norman Gray,
Demitri Muna James Turner,
Miguel de Val-Borro,
Juande Santander Vela,
David Shupe,
John Good,
G. Bruce Berriman
Abstract:
The Flexible Image Transport System (FITS) standard has been a great boon to astronomy, allowing observatories, scientists and the public to exchange astronomical information easily. The FITS standard is, however, showing its age. Developed in the late 1970s the FITS authors made a number of implementation choices for the format that, while common at the time, are now seen to limit its utility wit…
▽ More
The Flexible Image Transport System (FITS) standard has been a great boon to astronomy, allowing observatories, scientists and the public to exchange astronomical information easily. The FITS standard is, however, showing its age. Developed in the late 1970s the FITS authors made a number of implementation choices for the format that, while common at the time, are now seen to limit its utility with modern data. The authors of the FITS standard could not appreciate the challenges which we would be facing today in astronomical computing. Difficulties we now face include, but are not limited to, having to address the need to handle an expanded range of specialized data product types (data models), being more conducive to the networked exchange and storage of data, handling very large datasets and the need to capture significantly more complex metadata and data relationships.
There are members of the community today who find some (or all) of these limitations unworkable, and have decided to move ahead with storing data in other formats. This reaction should be taken as a wakeup call to the FITS community to make changes in the FITS standard, or to see its usage fall. In this paper we detail some selected important problems which exist within the FITS standard today. It is not our intention to prescribe specific remedies to these issues; rather, we hope to call attention of the FITS and greater astronomical computing communities to these issues in the hopes that it will spur action to address them.
△ Less
Submitted 20 February, 2015;
originally announced February 2015.
-
Learning from FITS: Limitations in use in modern astronomical research
Authors:
Brian Thomas,
Tim Jenness,
Frossie Economou,
Perry Greenfield,
Paul Hirst,
David S. Berry,
Erik Bray,
Norman Gray,
Demitri Muna,
James Turner,
Miguel de Val-Borro,
Juande Santander-Vela,
David Shupe,
John Good,
G. Bruce Berriman,
Slava Kitaeff,
Jonathan Fay,
Omar Laurino,
Anastasia Alexov,
Walter Landry,
Joe Masters,
Adam Brazier,
Reinhold Schaaf,
Kevin Edwards,
Russell O. Redman
, et al. (13 additional authors not shown)
Abstract:
The Flexible Image Transport System (FITS) standard has been a great boon to astronomy, allowing observatories, scientists and the public to exchange astronomical information easily. The FITS standard, however, is showing its age. Developed in the late 1970s, the FITS authors made a number of implementation choices that, while common at the time, are now seen to limit its utility with modern data.…
▽ More
The Flexible Image Transport System (FITS) standard has been a great boon to astronomy, allowing observatories, scientists and the public to exchange astronomical information easily. The FITS standard, however, is showing its age. Developed in the late 1970s, the FITS authors made a number of implementation choices that, while common at the time, are now seen to limit its utility with modern data. The authors of the FITS standard could not anticipate the challenges which we are facing today in astronomical computing. Difficulties we now face include, but are not limited to, addressing the need to handle an expanded range of specialized data product types (data models), being more conducive to the networked exchange and storage of data, handling very large datasets, and capturing significantly more complex metadata and data relationships.
There are members of the community today who find some or all of these limitations unworkable, and have decided to move ahead with storing data in other formats. If this fragmentation continues, we risk abandoning the advantages of broad interoperability, and ready archivability, that the FITS format provides for astronomy. In this paper we detail some selected important problems which exist within the FITS standard today. These problems may provide insight into deeper underlying issues which reside in the format and we provide a discussion of some lessons learned. It is not our intention here to prescribe specific remedies to these issues; rather, it is to call attention of the FITS and greater astronomical computing communities to these problems in the hope that it will spur action to address them.
△ Less
Submitted 10 February, 2015; v1 submitted 3 February, 2015;
originally announced February 2015.
-
Observatory/data centre partnerships and the VO-centric archive: The JCMT Science Archive experience
Authors:
Frossie Economou,
Severin Gaudet,
Tim Jenness,
Russell O. Redman,
Sharon Goliath,
Patrick Dowler,
Malcolm J. Currie,
Graham S. Bell,
Sarah F. Graves,
John Ouellette,
Doug Johnstone,
David Schade,
Antonio Chrysostomou
Abstract:
We present, as a case study, a description of the partnership between an observatory (JCMT) and a data centre (CADC) that led to the development of the JCMT Science Archive (JSA). The JSA is a successful example of a service designed to use Virtual Observatory (VO) technologies from the start. We describe the motivation, process and lessons learned from this approach.
We present, as a case study, a description of the partnership between an observatory (JCMT) and a data centre (CADC) that led to the development of the JCMT Science Archive (JSA). The JSA is a successful example of a service designed to use Virtual Observatory (VO) technologies from the start. We describe the motivation, process and lessons learned from this approach.
△ Less
Submitted 14 December, 2014;
originally announced December 2014.
-
Learning from 25 years of the extensible N-Dimensional Data Format
Authors:
Tim Jenness,
David S. Berry,
Malcolm J. Currie,
Peter W. Draper,
Frossie Economou,
Norman Gray,
Brian McIlwrath,
Keith Shortridge,
Mark B. Taylor,
Patrick T. Wallace,
Rodney F. Warren-Smith
Abstract:
The extensible N-Dimensional Data Format (NDF) was designed and developed in the late 1980s to provide a data model suitable for use in a variety of astronomy data processing applications supported by the UK Starlink Project. Starlink applications were used extensively, primarily in the UK astronomical community, and form the basis of a number of advanced data reduction pipelines today. This paper…
▽ More
The extensible N-Dimensional Data Format (NDF) was designed and developed in the late 1980s to provide a data model suitable for use in a variety of astronomy data processing applications supported by the UK Starlink Project. Starlink applications were used extensively, primarily in the UK astronomical community, and form the basis of a number of advanced data reduction pipelines today. This paper provides an overview of the historical drivers for the development of NDF and the lessons learned from using a defined hierarchical data model for many years in data reduction software, data pipelines and in data acquisition systems.
△ Less
Submitted 28 October, 2014;
originally announced October 2014.
-
ORAC-DR: A generic data reduction pipeline infrastructure
Authors:
Tim Jenness,
Frossie Economou
Abstract:
ORAC-DR is a general purpose data reduction pipeline system designed to be instrument and observatory agnostic. The pipeline works with instruments as varied as infrared integral field units, imaging arrays and spectrographs, and sub-millimeter heterodyne arrays & continuum cameras. This paper describes the architecture of the pipeline system and the implementation of the core infrastructure. We f…
▽ More
ORAC-DR is a general purpose data reduction pipeline system designed to be instrument and observatory agnostic. The pipeline works with instruments as varied as infrared integral field units, imaging arrays and spectrographs, and sub-millimeter heterodyne arrays & continuum cameras. This paper describes the architecture of the pipeline system and the implementation of the core infrastructure. We finish by discussing the lessons learned since the initial deployment of the pipeline system in the late 1990s.
△ Less
Submitted 28 October, 2014;
originally announced October 2014.
-
Your data is your dogfood: DevOps in the astronomical observatory
Authors:
Frossie Economou,
Joshua C. Hoblitt,
Pat Norris
Abstract:
DevOps is the contemporary term for a software development culture that purposefully blurs distinction between software development and IT operations by treating "infrastructure as code." DevOps teams typically implement practices summarised by the colloquial directive to "eat your own dogfood;" meaning that software tools developed by a team should be used internally rather thrown over the fence…
▽ More
DevOps is the contemporary term for a software development culture that purposefully blurs distinction between software development and IT operations by treating "infrastructure as code." DevOps teams typically implement practices summarised by the colloquial directive to "eat your own dogfood;" meaning that software tools developed by a team should be used internally rather thrown over the fence to operations or users. We present a brief overview of how DevOps techniques bring proven software engineering practices to IT operations. We then discuss the application of these practices to astronomical observatories.
△ Less
Submitted 24 July, 2014;
originally announced July 2014.
-
SCUBA-2: iterative map-making with the Sub-Millimetre User Reduction Facility
Authors:
Edward L. Chapin,
David S. Berry,
Andrew G. Gibb,
Tim Jenness,
Douglas Scott,
Remo P. J. Tilanus,
Frossie Economou,
Wayne S. Holland
Abstract:
The Submillimetre Common User Bolometer Array 2 (SCUBA-2) is an instrument operating on the 15-m James Clerk Maxwell Telescope, nominally consisting of 5120 bolometers in each of two simultaneous imaging bands centred over 450 and 850 um. The camera is operated by scanning across the sky and recording data at a rate of 200 Hz. As the largest of a new generation of multiplexed kilopixel bolometer c…
▽ More
The Submillimetre Common User Bolometer Array 2 (SCUBA-2) is an instrument operating on the 15-m James Clerk Maxwell Telescope, nominally consisting of 5120 bolometers in each of two simultaneous imaging bands centred over 450 and 850 um. The camera is operated by scanning across the sky and recording data at a rate of 200 Hz. As the largest of a new generation of multiplexed kilopixel bolometer cameras operating in the (sub)millimetre, SCUBA-2 data analysis represents a significant challenge. We describe the production of maps using the Sub-Millimetre User Reduction Facility (SMURF) in which we have adopted a fast, iterative approach to map-making that enables data reduction on single, modern, high-end desktop computers, with execution times that are typically shorter than the observing times. SMURF is used in an automated setting, both at the telescope for real-time feedback to observers, as well as for the production of science products for the JCMT Science Archive at the Canadian Astronomy Data Centre. Three detailed case studies are used to: (i) explore convergence properties of the map-maker using simple prior constraints (Uranus -- a point source); (ii) achieve the white-noise limit for faint point-source studies (extragalactic blank-field survey of the Lockman Hole); and (iii) demonstrate that our strategy is capable of recovering angular scales comparable to the size of the array footprint (approximately 5 arcmin) for bright extended sources (star-forming region M17).
△ Less
Submitted 25 March, 2013; v1 submitted 16 January, 2013;
originally announced January 2013.
-
SCUBA-2: The 10000 pixel bolometer camera on the James Clerk Maxwell Telescope
Authors:
W. S. Holland,
D. Bintley,
E. L. Chapin,
A. Chrysostomou,
G. R. Davis,
J. T. Dempsey,
W. D. Duncan,
M. Fich,
P. Friberg,
M. Halpern,
K. D. Irwin,
T. Jenness,
B. D. Kelly,
M. J. MacIntosh,
E. I. Robson,
D. Scott,
P. A. R. Ade,
E. Atad-Ettedgui,
D. S. Berry,
S. C. Craig,
X. Gao,
A. G. Gibb,
G. C. Hilton,
M. I. Hollister,
J. B. Kycia
, et al. (24 additional authors not shown)
Abstract:
SCUBA-2 is an innovative 10000 pixel bolometer camera operating at submillimetre wavelengths on the James Clerk Maxwell Telescope (JCMT). The camera has the capability to carry out wide-field surveys to unprecedented depths, addressing key questions relating to the origins of galaxies, stars and planets. With two imaging arrays working simultaneously in the atmospheric windows at 450 and 850 micro…
▽ More
SCUBA-2 is an innovative 10000 pixel bolometer camera operating at submillimetre wavelengths on the James Clerk Maxwell Telescope (JCMT). The camera has the capability to carry out wide-field surveys to unprecedented depths, addressing key questions relating to the origins of galaxies, stars and planets. With two imaging arrays working simultaneously in the atmospheric windows at 450 and 850 microns, the vast increase in pixel count means that SCUBA-2 maps the sky 100-150 times faster than the previous SCUBA instrument. In this paper we present an overview of the instrument, discuss the physical characteristics of the superconducting detector arrays, outline the observing modes and data acquisition, and present the early performance figures on the telescope. We also showcase the capabilities of the instrument via some early examples of the science SCUBA-2 has already undertaken. In February 2012, SCUBA-2 began a series of unique legacy surveys for the JCMT community. These surveys will take 2.5 years and the results are already providing complementary data to the shorter wavelength, shallower, larger-area surveys from Herschel. The SCUBA-2 surveys will also provide a wealth of information for further study with new facilities such as ALMA, and future telescopes such as CCAT and SPICA.
△ Less
Submitted 16 January, 2013;
originally announced January 2013.
-
Data Management At the UKIRT and JCMT
Authors:
Timothy Jenness,
Frossie Economou
Abstract:
For more than a decade the Joint Astronomy Centre has been develo** software tools to simplify observing and make it possible to use the telescopes in many different operational modes. In order to support remote operations the data handling systems need to be in place to allow observation preparation, flexible queue scheduling, data quality pipelines and science archives all to be connected in a…
▽ More
For more than a decade the Joint Astronomy Centre has been develo** software tools to simplify observing and make it possible to use the telescopes in many different operational modes. In order to support remote operations the data handling systems need to be in place to allow observation preparation, flexible queue scheduling, data quality pipelines and science archives all to be connected in a data-driven environment. We discuss the history of these developments at UKIRT and JCMT and how the decision to combine software development at both telescopes led each to get features that they could not have justified if they were treated independently.
△ Less
Submitted 24 November, 2011;
originally announced November 2011.
-
A Pilot Study for the SCUBA-2 'All-Sky' Survey
Authors:
Todd Mackenzie,
Filiberto G. Braglia,
Andy G. Gibb,
Douglas Scott,
Tim Jenness,
Stephen Serjeant,
Mark Thompson,
David Berry,
Christopher M. Brunt,
Edward Chapin,
Antonio Chrysostomou,
Dave Clements,
Kristen Coppin,
Frossie Economou,
A. Evans,
Per Friberg,
Jane Greaves,
T. Hill,
Wayne Holland,
R. J. Ivison,
Johan H. Knapen,
Neal Jackson,
Gilles Joncas,
Larry Morgan,
Chris Pearson
, et al. (8 additional authors not shown)
Abstract:
We have carried out a pilot study for the SCUBA-2 'All-Sky' Survey, SASSy, a wide and shallow map** project at 850 microns, designed to find rare objects, both Galactic and extragalactic. Two distinct sets of exploratory observations were undertaken, and used to test the SASSy approach and data reduction pipeline. The first was a 0.5 by 0.5 degrees map around the nearby galaxy NGC 2559. The gala…
▽ More
We have carried out a pilot study for the SCUBA-2 'All-Sky' Survey, SASSy, a wide and shallow map** project at 850 microns, designed to find rare objects, both Galactic and extragalactic. Two distinct sets of exploratory observations were undertaken, and used to test the SASSy approach and data reduction pipeline. The first was a 0.5 by 0.5 degrees map around the nearby galaxy NGC 2559. The galaxy was easily detected at 156 mJy, but no other convincing sources are present in the map. Comparison with other galaxies with similar wavelength coverage indicates that NGC 2559 has relatively warm dust. The second observations cover 1 square degree around the W5-E HII region. As well as diffuse structure in the map, a filtering approach was able to extract 27 compact sources with signal-to-noise greater than 6. By matching with data at other wavelengths we can see that the SCUBA-2 data can be used to discriminate the colder cores. Together these observations show that the SASSy project will be able to meet its original goals of detecting new bright sources which will be ideal for follow-up observations with other facilities.
△ Less
Submitted 10 December, 2010; v1 submitted 7 December, 2010;
originally announced December 2010.
-
SCUBA-2 Data Processing
Authors:
Tim Jenness,
David Berry,
Ed Chapin,
Frossie Economou,
Andy Gibb,
Douglas Scott
Abstract:
SCUBA-2 is the largest submillimetre array camera in the world and was commissioned on the James Clerk Maxwell Telescope (JCMT) with two arrays towards the end of 2009. A period of shared-risks observing was then completed and the full planned complement of 8 arrays, 4 at 850 microns and 4 at 450 microns, are now installed and ready to be commissioned. SCUBA-2 has 10,240 bolometers, corresponding…
▽ More
SCUBA-2 is the largest submillimetre array camera in the world and was commissioned on the James Clerk Maxwell Telescope (JCMT) with two arrays towards the end of 2009. A period of shared-risks observing was then completed and the full planned complement of 8 arrays, 4 at 850 microns and 4 at 450 microns, are now installed and ready to be commissioned. SCUBA-2 has 10,240 bolometers, corresponding to a data rate of 8 MB/s when sampled at the nominal rate of 200 Hz. The pipeline produces useful maps in near real time at the telescope and often publication quality maps in the JCMT Science Archive (JSA) hosted at the Canadian Astronomy Data Centre (CADC).
△ Less
Submitted 26 November, 2010;
originally announced November 2010.
-
LSST: from Science Drivers to Reference Design and Anticipated Data Products
Authors:
Željko Ivezić,
Steven M. Kahn,
J. Anthony Tyson,
Bob Abel,
Emily Acosta,
Robyn Allsman,
David Alonso,
Yusra AlSayyad,
Scott F. Anderson,
John Andrew,
James Roger P. Angel,
George Z. Angeli,
Reza Ansari,
Pierre Antilogus,
Constanza Araujo,
Robert Armstrong,
Kirk T. Arndt,
Pierre Astier,
Éric Aubourg,
Nicole Auza,
Tim S. Axelrod,
Deborah J. Bard,
Jeff D. Barr,
Aurelian Barrau,
James G. Bartlett
, et al. (288 additional authors not shown)
Abstract:
(Abridged) We describe here the most ambitious survey currently planned in the optical, the Large Synoptic Survey Telescope (LSST). A vast array of science will be enabled by a single wide-deep-fast sky survey, and LSST will have unique survey capability in the faint time domain. The LSST design is driven by four main science themes: probing dark energy and dark matter, taking an inventory of the…
▽ More
(Abridged) We describe here the most ambitious survey currently planned in the optical, the Large Synoptic Survey Telescope (LSST). A vast array of science will be enabled by a single wide-deep-fast sky survey, and LSST will have unique survey capability in the faint time domain. The LSST design is driven by four main science themes: probing dark energy and dark matter, taking an inventory of the Solar System, exploring the transient optical sky, and map** the Milky Way. LSST will be a wide-field ground-based system sited at Cerro Pachón in northern Chile. The telescope will have an 8.4 m (6.5 m effective) primary mirror, a 9.6 deg$^2$ field of view, and a 3.2 Gigapixel camera. The standard observing sequence will consist of pairs of 15-second exposures in a given field, with two such visits in each pointing in a given night. With these repeats, the LSST system is capable of imaging about 10,000 square degrees of sky in a single filter in three nights. The typical 5$σ$ point-source depth in a single visit in $r$ will be $\sim 24.5$ (AB). The project is in the construction phase and will begin regular survey operations by 2022. The survey area will be contained within 30,000 deg$^2$ with $δ<+34.5^\circ$, and will be imaged multiple times in six bands, $ugrizy$, covering the wavelength range 320--1050 nm. About 90\% of the observing time will be devoted to a deep-wide-fast survey mode which will uniformly observe a 18,000 deg$^2$ region about 800 times (summed over all six bands) during the anticipated 10 years of operations, and yield a coadded map to $r\sim27.5$. The remaining 10\% of the observing time will be allocated to projects such as a Very Deep and Fast time domain survey. The goal is to make LSST data products, including a relational database of about 32 trillion observations of 40 billion objects, available to the public and scientists around the world.
△ Less
Submitted 23 May, 2018; v1 submitted 15 May, 2008;
originally announced May 2008.
-
The SCUBA-2 "All-Sky" Survey
Authors:
M. A. Thompson,
S. Serjeant,
T. Jenness,
D. Scott,
M. Ashdown,
C. Brunt,
H. Butner,
E. Chapin,
A. C. Chrysostomou,
J. S. Clark,
D. Clements,
J. L. Collett,
K. Coppin,
I. M. Coulson,
W. R. F. Dent,
F. Economou,
A. Evans,
P. Friberg,
G. A. Fuller,
A. G. Gibb,
J. Greaves,
J. Hatchell,
W. S. Holland,
M. Hudson,
R. J. Ivison
, et al. (23 additional authors not shown)
Abstract:
The sub-millimetre wavelength regime is perhaps the most poorly explored over large areas of the sky, despite the considerable effort that has been expended in making deep maps over small regions. As a consequence the properties of the sub-millimetre sky as a whole, and of rare bright objects in particular, remains largely unknown. Here we describe a forthcoming survey (the SCUBA-2 ``All-Sky'' S…
▽ More
The sub-millimetre wavelength regime is perhaps the most poorly explored over large areas of the sky, despite the considerable effort that has been expended in making deep maps over small regions. As a consequence the properties of the sub-millimetre sky as a whole, and of rare bright objects in particular, remains largely unknown. Here we describe a forthcoming survey (the SCUBA-2 ``All-Sky'' Survey, or SASSy) designed to address this issue by making a large-area map of approximately one-fifth of the sky visible from the JCMT (4800 square degrees) down to a 1 sigma noise level of 30 mJy/beam. This map forms the pilot for a much larger survey, which will potentially map the remaining sky visible from the JCMT, with the region also visible to ALMA as a priority. SASSy has been awarded 500 hours for the 4800 square degree pilot phase and will commence after the commissioning of SCUBA-2, expected in early 2008.
△ Less
Submitted 2 May, 2007; v1 submitted 24 April, 2007;
originally announced April 2007.
-
SCUBA-2: a 10,000 pixel submillimeter camera for the James Clerk Maxwell Telescope
Authors:
Wayne Holland,
Michael MacIntosh,
Alasdair Fairley,
Dennis Kelly,
David Montgomery,
David Gostick,
Eli Atad-Ettedgui,
Maureen Ellis,
Ian Robson,
Matthew Hollister,
Adam Woodcraft,
Peter Ade,
Ian Walker,
Kent Irwin,
Gene Hilton,
William Duncan,
Carl Reintsema,
Anthony Walton,
William Parkes,
Camelia Dunare,
Michel Fich,
Jan Kycia,
Mark Halpern,
Douglas Scott,
Andy Gibb
, et al. (8 additional authors not shown)
Abstract:
SCUBA-2 is an innovative 10,000 pixel submillimeter camera due to be delivered to the James Clerk Maxwell Telescope in late 2006. The camera is expected to revolutionize submillimeter astronomy in terms of the ability to carry out wide-field surveys to unprecedented depths addressing key questions relating to the origins of galaxies, stars and planets. This paper presents an update on the projec…
▽ More
SCUBA-2 is an innovative 10,000 pixel submillimeter camera due to be delivered to the James Clerk Maxwell Telescope in late 2006. The camera is expected to revolutionize submillimeter astronomy in terms of the ability to carry out wide-field surveys to unprecedented depths addressing key questions relating to the origins of galaxies, stars and planets. This paper presents an update on the project with particular emphasis on the laboratory commissioning of the instrument. The assembly and integration will be described as well as the measured thermal performance of the instrument. A summary of the performance results will be presented from the TES bolometer arrays, which come complete with in-focal plane SQUID amplifiers and multiplexed readouts, and are cooled to 100mK by a liquid cryogen-free dilution refrigerator. Considerable emphasis has also been placed on the operating modes of the instrument and the common-user aspect of the user interface and data reduction pipeline. These areas will also be described in the paper.
△ Less
Submitted 14 June, 2006;
originally announced June 2006.
-
Is There a Disc in the Superluminal Quasars?
Authors:
E. Rokaki,
A. Lawrence,
F. Economou,
A. Mastichiadis
Abstract:
We present a new evidence of the accretion disc in active galactic nuclei, by examining the properties of the Ha emission line versus viewing angle, in a sample of superluminal (SL) quasars. Both line velocity width (FWHM) and rest equivalent width (EW) correlate with viewing angle. These correlations are quantitatively consistent with a disc geometry for both the line and the underlying continu…
▽ More
We present a new evidence of the accretion disc in active galactic nuclei, by examining the properties of the Ha emission line versus viewing angle, in a sample of superluminal (SL) quasars. Both line velocity width (FWHM) and rest equivalent width (EW) correlate with viewing angle. These correlations are quantitatively consistent with a disc geometry for both the line and the underlying continuum source.
△ Less
Submitted 21 January, 2003;
originally announced January 2003.
-
Towards the automated reduction and calibration of SCUBA data from the James Clerk Maxwell Telescope
Authors:
Tim Jenness,
Jason A. Stevens,
Elese N. Archibald,
Frossie Economou,
N. E. Jessop,
E. I. Robson
Abstract:
The Submillimetre Common User Bolometer Array (SCUBA) instrument has been operating on the James Clerk Maxwell Telescope (JCMT) since 1997. The data archive is now sufficiently large that it can be used to investigate instrumental properties and the variability of astronomical sources. This paper describes the automated calibration and reduction scheme used to process the archive data with parti…
▽ More
The Submillimetre Common User Bolometer Array (SCUBA) instrument has been operating on the James Clerk Maxwell Telescope (JCMT) since 1997. The data archive is now sufficiently large that it can be used to investigate instrumental properties and the variability of astronomical sources. This paper describes the automated calibration and reduction scheme used to process the archive data with particular emphasis on `jiggle-map' observations of compact sources. We demonstrate the validity of our automated approach at both 850- and 450-microns and apply it to several of the JCMT secondary flux calibrators. We determine light curves for the variable sources IRC+10216 and OH231.8. This automation is made possible by using the ORAC-DR data reduction pipeline, a flexible and extensible data reduction pipeline that is used on UKIRT and the JCMT.
△ Less
Submitted 25 April, 2002;
originally announced April 2002.
-
Measurement of Star-Formation Rate from H-alpha in field galaxies at z=1
Authors:
Karl Glazebrook,
Chris Blake,
Frossie Economou,
Simon Lilly,
Matthew Colless
Abstract:
We report the results of J-band infrared spectroscopy of a sample of 13 z=1 field galaxies drawn from the Canada-France Redshift Survey, targeting galaxies whose redshifts place the rest frame H-alpha line emission from HII regions in between the bright night sky OH lines. As a result we detect emission down to a flux limit of ~10^{-16} ergs cm^{-2} s^{-1} corresponding to a luminosity limit of…
▽ More
We report the results of J-band infrared spectroscopy of a sample of 13 z=1 field galaxies drawn from the Canada-France Redshift Survey, targeting galaxies whose redshifts place the rest frame H-alpha line emission from HII regions in between the bright night sky OH lines. As a result we detect emission down to a flux limit of ~10^{-16} ergs cm^{-2} s^{-1} corresponding to a luminosity limit of ~10^{41} ergs at this redshift for a H_0=50 km s^{-1} Mpc,^{-1} q_0=0.5 cosmology. From these luminosities we derive estimates of the star-formation rates in these galaxies which are independent of previous estimates based upon their rest-frame ultraviolet (2800A) luminosity. The mean star-formation rate at z=1, from this sample, is found to be at least three times as high as the ultraviolet estimates. The standard dust extinction in these galaxies is inferred to be A_V=0.5-1.0 mags, comparable to local field galaxies, suggesting that the bulk of star-formation is not heavily obscured unless one uses greyer extinction laws. Star-forming galaxies have the bluest colours and a preponderance of disturbed/interacting morphologies. We also investigate the effects of particular star-formation histories, in particular the role of bursts vs continuous star-formation in changing the detailed distribution of UV to H-alpha emission. Generally we find that models dominated by short, overlap**, bursts at typically 0.2 Gyr intervals provide a better model for the data than a constant rate of star-formation. The star-formation history of the Universe from Balmer lines is compiled and found to be typically 2--3\times higher than that inferred from the UV {\em at all redshifts}. It can not yet be clearly established whether the star-formation rate falls off or remains constant at high-redshift.
△ Less
Submitted 10 February, 1999; v1 submitted 24 August, 1998;
originally announced August 1998.
