Metadata and data management for the Keck Observatory Archive

H. D. Tran; J. Holt; R. W. Goodrich; J. A. Mader; M. Swain; A. C. Laity; M. Kong; C. R. Gelino; G. B. Berriman

doi:10.1117/12.2054830

18 July 2014 Metadata and data management for the Keck Observatory Archive

H. D. Tran, J. Holt, R. W. Goodrich, J. A. Mader, M. Swain, A. C. Laity, M. Kong, C. R. Gelino, G. B. Berriman

Proceedings Volume 9152, Software and Cyberinfrastructure for Astronomy III; 91522I (2014) https://doi.org/10.1117/12.2054830
Event: SPIE Astronomical Telescopes + Instrumentation, 2014, Montréal, Quebec, Canada

Abstract

A collaboration between the W. M. Keck Observatory (WMKO) in Hawaii and the NASA Exoplanet Science Institute (NExScI) in California, the Keck Observatory Archive (KOA) was commissioned in 2004 to archive observing data from WMKO, which operates two classically scheduled 10 m ground-based telescopes. The observing data from Keck is not suitable for direct ingestion into the archive since the metadata contained in the original FITS headers lack the information necessary for proper archiving. Coupled with different standards among instrument builders and the heterogeneous nature of the data inherent in classical observing, in which observers have complete control of the instruments and their observations, the data pose a number of technical challenges for KOA. For example, it is often difficult to determine if an observation is a science target, a sky frame, or a sky flat. It is also necessary to assign the data to the correct owners and observing programs, which can be a challenge for time-domain and target-of-opportunity observations, or on split nights, during which two or more principle investigators share a given night. In addition, having uniform and adequate calibrations are important for the proper reduction of data. Therefore, KOA needs to distinguish science files from calibration files, identify the type of calibrations available, and associate the appropriate calibration files with each science frame. We describe the methodologies and tools that we have developed to successfully address these difficulties, adding content to the FITS headers and "retrofitting" the metadata in order to support archiving Keck data, especially those obtained before the archive was designed. With the expertise gained from having successfully archived observations taken with all eight currently active instruments at WMKO, we have developed lessons learned from handling this complex array of heterogeneous metadata that help ensure a smooth ingestion of data not only for current but also future instruments, as well as a better experience for the archive user.

Citation Download Citation

H. D. Tran, J. Holt, R. W. Goodrich, J. A. Mader, M. Swain, A. C. Laity, M. Kong, C. R. Gelino, and G. B. Berriman "Metadata and data management for the Keck Observatory Archive", Proc. SPIE 9152, Software and Cyberinfrastructure for Astronomy III, 91522I (18 July 2014); https://doi.org/10.1117/12.2054830

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available