ann20020 — Announcement

Rubin Observatory Partners with Google Cloud to Build Interim Data Facility

Cloud-based data platform will expand observatory’s capabilities and help scientific community unlock greater insights about our Universe

4 December 2020

Google Cloud today announced that Vera C. Rubin Observatory has entered into a three-year agreement to host its Interim Data Facility (IDF) on the Google Cloud Platform. This collaboration ushers in a new era in large-scale scientific computing of astronomy workloads in the cloud. Until the telescope becomes operational in 2023, the IDF will collect astronomical data, or “pre-survey data,” to prepare the Rubin Operations team and science community for the advent of the full survey. The observatory is located on Cerro Pachón in the Chilean Andes at the Southern end of the great Atacama Desert in Chile, taking advantage of its prime geographic location for astronomical observations and discovery.

The Rubin IDF will process astronomical data collected by Rubin Observatory in its commissioning phase and make the data available to hundreds of users in the scientific community in advance of Rubin Observatory’s Legacy Survey of Space and Time (LSST), a planned 10-year survey of the southern sky. A 500-petabyte set of data products and images, the LSST will be used to address the most critical questions about the Universe’s structure and evolution, including the nature of dark matter and dark energy.

“We’re extremely pleased to work with Google Cloud on this project,” said Acting Rubin Observatory Operations Director Bob Blum. “By using an established and trusted cloud infrastructure, we’ve been able to stay on track at this point in our project to ensure we’ll be ready to deliver high-impact science to our community LSST is a single experiment that gathers data on the Solar System, variable and exploding stars, the stars that make up our Milky Way galaxy, and the expansion of the Universe itself.”

“The advancements we’re seeing in astronomy point to the growing appetite for data that can only be supported by the cloud’s scale and speed,” said Mike Daniels, vice president, Global Public Sector, Google Cloud. “By collaborating with Google Cloud, Rubin Observatory can build in more flexibility for the rising demand of astronomical data, while taking advantage of low-cost cloud data storage. This means Rubin Observatory can use more funds toward discoveries, instead of IT.”

Because the LSST dataset will be too large to be downloaded and stored locally, astronomers will access the data via the browser-based Rubin Science Platform. The same platform will be deployed on the Google Cloud-hosted IDF. With this level of access, the platform will provide the Rubin Observatory user community with astronomy data previews and the ability to experience LSST-like data in a real-time analysis environment. The IDF will also be used to deploy the Rubin Science pipelines, which will convert pixel data into a database of stars and galaxies. This will provide a first test of the pipelines at scale.

Rubin Observatory will use Google Cloud’s Storage, Kubernetes Engine and Google Cloud Compute as the foundation for the IDF project. Google Workspace will enable productivity and collaboration among its scientists and team members. The Google Cloud IDF is also scalable in response to usage, which will be critical when members of the Rubin Science Community begin accessing the data facility in late 2021.

Financial support for Rubin Observatory comes from the US National Science Foundation, the US Department of Energy, and private funding raised by the LSST Corporation.

About Rubin Observatory

Rubin Observatory will employ the 8.4-meter Simonyi Survey Telescope and the 3200-megapixel LSST Camera to capture about 1,000 images of the sky every night for ten years. Each image will cover a 9.6-square-degree field of view, or about 40 times the area of the full Moon. LSST survey images will contain data for about 20 billion galaxies and a similar number of stars — more celestial objects than there are humans on Earth. Rubin Observatory data will be used for scientific investigations ranging from cosmological studies of the Universe to searches for potentially dangerous Earth-impacting asteroids. Additionally a public engagement program will enable anyone to explore the Universe and be part of the discovery process.

This federal project is jointly funded by the National Science Foundation (NSF) and the Department of Energy (DOE) Office of Science, with early construction funding received through private donations through the LSST Corporation. The NSF-funded LSST Project Office for construction was established as an operating center under management of the Association of Universities for Research in Astronomy (AURA). The DOE-funded effort to build the LSST camera is managed by the SLAC National Accelerator Laboratory (SLAC).

NSF and DOE will continue to support the Rubin Observatory in its Operations phase to carry out the Legacy Survey of Space and Time. They will also provide support for scientific research with the data. During operations NSF funding is managed by the Association of Universities for Research in Astronomy (AURA) under a cooperative agreement with NSF and DOE funding is managed by SLAC under contract by DOE. Rubin Observatory is operated by NSF’s NOIRLab with support from the NSF and the Department of Energy’s Office of Science.

NSF’s NOIRLab (National Optical-Infrared Astronomy Research Laboratory), the US center for ground-based optical-infrared astronomy, operates the international Gemini Observatory (a facility of NSF, NRC–Canada, ANID–Chile, MCTIC–Brazil, MINCyT–Argentina, and KASI–Republic of Korea), Kitt Peak National Observatory (KPNO), Cerro Tololo Inter-American Observatory (CTIO), the Community Science and Data Center (CSDC), and Vera C. Rubin Observatory. It is managed by the Association of Universities for Research in Astronomy (AURA) under a cooperative agreement with NSF and is headquartered in Tucson, Arizona. The astronomical community is honored to have the opportunity to conduct astronomical research on Iolkam Du’ag (Kitt Peak) in Arizona, on Maunakea in Hawaiʻi, and on Cerro Tololo and Cerro Pachón in Chile. We recognize and acknowledge the very significant cultural role and reverence that these sites have to the Tohono O'odham Nation, to the Native Hawaiian community, and to the local communities in Chile, respectively.