KEYWORDS: Data archive systems, Information technology, Observatories, Databases, Control systems, Data centers, Binary data, Visibility, Standards development, Java
ALMA provides a wide range of web applications. Their main purposes are to support the work of its end users, be it staff astronomers or the external scientific community which uses these to propose and track their observation projects, including the download of their scientific data. These web applications -internally known as Offline Software, in contrast to the Online Software which corresponds mainly to the Control Software- are separated in two groups. One group of applications, which requires to modify data contained in the ALMA Archive, is deployed at JAO Offices in Chile, and a second group of applications, which doesn't modify data in the ALMA Archive, is deployed at each ALMA Regional Center (ARC), to improve the application response time by running in a location closer to the final user. Based on previous improvements done to the deployment of these web applications used by the Joint ALMA Observatory (JAO), recently ALMA has achieved a unified way of deploying the applications that run at each ARC. This has been achieved by implementing an infrastructure/configuration as code approach. The corresponding code base that holds the configuration and infrastructure definitions to achieve this are kept under configuration control, following a set of DevOps best practices to handle the day-by-day operations of all these applications, in a unified way, across all ARCs and the JAO. To manage these tools at the different ARCs a maintenance group for this deployment framework has been recently established. In this paper we detail the framework implemented in this process. We also explain the characteristics of the globally distributed maintenance group, the process by which we manage the deployment of the applications at each ARC and the successes we have enjoyed thanks to this collaboration within ALMA's partner institutions.
Elasticsearch is one of solutions to monitor and analyze logs. Even with ALMA∗, observation logs are stored and anyone can look into it according to their purpose. For example, Hastings, which is a tool discovers the root cause of the defect, is utilized for ACA Correlator subsystem†. It queries logs to an ALMA Elasticsearch storing operational logs, analyzes specific messages which infer troubles, then outputs a result. Before the ALMA Elasticsearch was deployed, logs should have been collected manually in advance. Now the ALMA Elasticsearch has become available and we’ve known: 1) Elasticsearch can directly configure and access features by using REST API, 2) Logs taken even years ago can also be retrieved easily, 3) Elasticsearch’s major update didn’t cause much loss of time to change Hastings, 4) Python has several methods to manage Elasticsearch so that we can choose a favorite one. Therefore, we thought to apply Elasticsearch to the Subaru telescope‡. Size of Subaru logs are quite large but they are not stored in any database yet and just archived. We created a cluster system with Elasticsearch for the evaluation purpose and found ways to store data in a short time. We estimated the total ingestion time for 20 years of telescope status data to be at most 5 months. Our goal is to find a feasible cause of any defects in near real time, to predict any errors that may occur in near future, and to analyze communication between the telescope and observational equipment to optimize observations.
Access to the requested content is limited to institutions that have purchased or subscribe to SPIE eBooks.
You are receiving this notice because your organization may not have SPIE eBooks access.*
*Shibboleth/Open Athens users─please
sign in
to access your institution's subscriptions.
To obtain this item, you may purchase the complete book in print or electronic format on
SPIE.org.
INSTITUTIONAL Select your institution to access the SPIE Digital Library.
PERSONAL Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.