Hostname: page-component-cd9895bd7-dk4vv Total loading time: 0 Render date: 2024-12-25T04:28:28.297Z Has data issue: false hasContentIssue false
  • English
  • Français

What will the future of cloud-based astronomical data processing look like?

Published online by Cambridge University Press:  30 May 2017

Andrew W. Green Open the ORCID record for Andrew W. Green [Opens in a new window] ,
Elizabeth Mannering ,
Lloyd Harischandra ,
Minh Vuong ,
Simon O’Toole ,
Katrina Sealey  and
Andrew M. Hopkins
Andrew W. Green
Affiliation:
Australian Astronomical Observatory, PO Box 915, North Ryde, NSW, 1670Australia email: [email protected]
Elizabeth Mannering
Affiliation:
Australian Astronomical Observatory, PO Box 915, North Ryde, NSW, 1670Australia email: [email protected]
Lloyd Harischandra
Affiliation:
Australian Astronomical Observatory, PO Box 915, North Ryde, NSW, 1670Australia email: [email protected]
Minh Vuong
Affiliation:
Australian Astronomical Observatory, PO Box 915, North Ryde, NSW, 1670Australia email: [email protected]
Simon O’Toole
Affiliation:
Australian Astronomical Observatory, PO Box 915, North Ryde, NSW, 1670Australia email: [email protected]
Katrina Sealey
Affiliation:
Australian Astronomical Observatory, PO Box 915, North Ryde, NSW, 1670Australia email: [email protected]
Andrew M. Hopkins
Affiliation:
Australian Astronomical Observatory, PO Box 915, North Ryde, NSW, 1670Australia email: [email protected]
Rights & Permissions [Opens in a new window]

Abstract

Core share and HTML view are not available for this content. However, as you have access to this content, a full PDF is available via the ‘Save PDF’ action button.

Astronomy is rapidly approaching an impasse: very large datasets require remote or cloud-based parallel processing, yet many astronomers still try to download the data and develop serial code locally. Astronomers understand the need for change, but the hurdles remain high. We are developing a data archive designed from the ground up to simplify and encourage cloud-based parallel processing. While the volume of data we host remains modest by some standards, it is still large enough that download and processing times are measured in days and even weeks. We plan to implement a python based, notebook-like interface that automatically parallelises execution. Our goal is to provide an interface sufficiently familiar and user-friendly that it encourages the astronomer to run their analysis on our system in the cloud—astroinformatics as a service. We describe how our system addresses the approaching impasse in astronomy using the SAMI Galaxy Survey as an example.

Keywords

astronomical data bases: miscellaneousmethods: data analysis
Type
Contributed Papers
Information
Proceedings of the International Astronomical Union , Volume 12 , Symposium S325: Astroinformatics , October 2016 , pp. 27 - 31
Copyright
Copyright © International Astronomical Union 2017 

References

Momcheva, I. & Tollerud, E.. Software Use in Astronomy: an Informal Survey. ArXiv e-prints, July 2015.Google Scholar