Designing a multi-petabyte database for LSST

Jacek Becla; Andrew Hanushevsky; Sergei Nikolaev; Ghaleb Abdulla; Alex Szalay; Maria Nieto-Santisteban; Ani Thakar; Jim Gray

doi:10.1117/12.671721

29 June 2006 Designing a multi-petabyte database for LSST

Jacek Becla, Andrew Hanushevsky, Sergei Nikolaev, Ghaleb Abdulla, Alex Szalay, Maria Nieto-Santisteban, Ani Thakar, Jim Gray

Author Affiliations +

Proceedings Volume 6270, Observatory Operations: Strategies, Processes, and Systems; 62700R (2006) https://doi.org/10.1117/12.671721
Event: SPIE Astronomical Telescopes + Instrumentation, 2006, Orlando, Florida , United States

Abstract

The 3.2 giga-pixel LSST camera will produce approximately half a petabyte of archive images every month. These data need to be reduced in under a minute to produce real-time transient alerts, and then added to the cumulative catalog for further analysis. The catalog is expected to grow about three hundred terabytes per year. The data volume, the real-time transient alerting requirements of the LSST, and its spatio-temporal aspects require innovative techniques to build an efficient data access system at reasonable cost. As currently envisioned, the system will rely on a database for catalogs and metadata. Several database systems are being evaluated to understand how they perform at these data rates, data volumes, and access patterns. This paper describes the LSST requirements, the challenges they impose, the data access philosophy, results to date from evaluating available database technologies against LSST requirements, and the proposed database architecture to meet the data challenges.

Citation Download Citation

Jacek Becla, Andrew Hanushevsky, Sergei Nikolaev, Ghaleb Abdulla, Alex Szalay, Maria Nieto-Santisteban, Ani Thakar, and Jim Gray "Designing a multi-petabyte database for LSST", Proc. SPIE 6270, Observatory Operations: Strategies, Processes, and Systems, 62700R (29 June 2006); https://doi.org/10.1117/12.671721

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available