The diverse and dynamic landscape of services provided by data archives that has recently emerged
is in stark contrast with the classical idea of astronomical archives as static, passive repositories whose
only goal is to capture, record and preserve forever the assets produced by their observatories. In this
new scenario, archives occupy a central role as engines and enablers of the success of the astronomical
facilities they support in multiple ways. More specifically, in the case of mature missions and established observatories
that have collected large amount of data, archives can be considered new instruments in the own right as they, by favoring re-use
and new uses of single or aggregated archival datasets, promote the investigation of regions of the observational
parameter space that can otherwise be impractical to access or inaccessible altogether. In this contribution, I
will describe how the Chandra Data Archive (CDA) contributes to the final science return of the Chandra mission by
focusing on four different areas: maximization of the observational efficiency through contributions to
smooth operations of the observatory; collection and curation of a comprehensive mission bibliography;
assessment of the scientific impact of the mission by the development of specific metrics; promotion of use of
archival data across different astronomical communities. Finally, using the Chandra archive as an example,
I will briefly discuss the changes to the roles and priorities of an astronomical archive that are necessary to
adjust to the evolving needs of the mission and its constituencies.
This work has been supported by NASA under contract NAS 8-03060 to the Smithsonian Astrophysical
Observatory for operation of the Chandra X-ray Center.
'What is the science impact of your observatory?' is the dreaded question most observatories face in one way or another. Classically, the number of science papers and citation rates of those papers are presented to show the science impact of an observatory with perhaps some download statistics thrown in for good measure. But a more expansive notion of science impact is needed to assess the most in uential types of science coming from the observatory. This necessarily leads to a deeper and broader look at the science being done with the obser- vatory's data. It encompasses notions of what kinds of observing programs will have long-lasting contributions to astronomy; what serendipitous science is being, or can be, performed with the observatory's archive; what unexpected science topics are being investigated with the observatory's data; or even, which sorts of targets need more exposure time. Observatory bibliographies can provide insight to these questions. In this presentation I will provide examples of how the extensive metadata connected to the Chandra Bibliography is being used to uncover the science impact of the Chandra X-ray Observatory (CXO) within the astronomical community. This work has been supported by NASA under contract NAS 8-03060 to the Smithsonian Astrophysical Observatory for operation of the Chandra X-ray Center (CXC).
The Chandra Data Archive (CDA) maintains an extensive observatory bibliography. By linking the published articles with the individual datasets analyzed in the paper, we have the opportunity to join the bibliographic metadata (including keywords, subjects, objects, data references from other observatories, etc.) with the meta- data associated with the observational datasets. This rich body of information is ripe for far more sophisticated data mining than the two repositories (publications and data) would afford individually. Throughout the course of the mission the CDA has investigated numerous questions regarding the impact of specific types of Chandra programs such as the relative science impact of GTO, GO, and DDT programs or observing, archive, and theory programs. Most recently the Chandra bibliography was used to assess the impact of programs based on the size of the program to examine whether the dividing line between standard and large projects should be changed and whether another round of X-ray Visionary Programs should be offered. Traditionally we have grouped observations by proposal when assessing the impact of programs. For this investigation we aggregated observations by pointing and instrument configuration such that objects observed multiple times in the mission were considered single observing programs. This change in perspective has given us new ideas for assessing the science impact of Chandra and for presenting data to our users. In this paper we present the methodologies used in the recent study, some of its results, and most importantly some unexpected insights into assessing the science impact of an observatory.
KEYWORDS: Observatories, Data archive systems, Calibration, Sensors, Spectroscopy, Keck Observatory, Data processing, Spectrographs, Telescopes, Signal to noise ratio
The purpose of this paper is to present a quantitative assessment of how well grant and/or program acknowledgments reflect the science impact of Chandra observing, archive, and theory programs and to assess whether observatory acknowledgments alone are a good indicator for inclusion in an observatory bibliography. For grant citations we find that curators will often need to determine the correct grant being cited and they will need to assess relationship between the content of a paper and the grant proposal being cited for statistics to be meaningful. We also find a significant number of papers can be attributed to observing programs through grant links only and that performing full-text searches against the ADS for grant numbers can lead to additional articles for inclusion in the bibliography. When looking at acknowledgment sections as a whole, we find that using an observatory acknowledgment as the sole source for determining inclusion in a bibliography will greatly underestimate the number of science papers attributable to the observatory.
Observatories have wrestled for decades with the questions how to measure their importance to the astronomical
community, what their scientific impact is, and how their performance in that respect compares to that of other
observatories. There is a general sense that the answer is to be found in the publication record - specifically, in the
refereed journal articles. However, simple parameters (such as the number of papers) are not helpful, because in isolation
(applied to a single observatory) they are meaningless, while in comparison between observatories they are subject to
external influences that all but invalidate the comparisons.
We were fortunate in having the Chandra X-ray Observatory's bibliographic database with its rich variety of metadata
available as a resource for experimenting with more sophisticated metrics. Out of this project we propose a modest set
that contains meaningful information when viewed in the isolation of a single observatory as well as in comparison with
other observatories. Even so, we urge users not to draw conclusions on the basis of the face value of the comparisons,
but only after a serious analysis of potential causes for any differences or similarities.
We have designed our metrics to provide useful information in three main areas of interest: speed of publication; fraction
of observing time published; and archival usage. The basic measured parameters are the percentage of available
observing time published as a function of the data's age, at a few specific age values; the median time it takes to publish
observations; and similar parameters for multiple publications of the same observations.
Citation of results is a fourth category, but it does not lend itself well to comparisons and defies the search for definite
statements.
Assessing the impact of astronomical facilities rests upon an evaluation of the scientic discoveries which their
data have enabled. Telescope bibliographies, which link data products with the literature, provide a way to use
bibliometrics as an impact measure for the underlying observations. In this paper we argue that the creation
and maintenance of telescope bibliographies should be considered an integral part of an observatory's operations.
We review the existing tools, services, and work
ows which support these curation activities, giving an estimate
of the eort and expertise required to maintain an archive-based telescope bibliography.
Creating and maintaining an observatory bibliography is labor intensive, but the results can be used for more
than accounting purposes. The information gathered during the curating process can be used by data discovery
and research tools as well; the key is to collect sufficient metadata during the publication classification phase.
The Chandra X-ray Center has taken this approach from the inception of its bibliography and we now have an
interconnected web of links which lead researchers to the Chandra Data Archive from many sources.
We provide links between datasets and astronomical publications to the Astrophysics Data System (ADS) so
users of the ADS can directly access Chandra data associated with a publication. Those same links are used by
WebChaser, the Chandra data access tool, so users can directly access articles associated with the data they are
reviewing. We are expanding our exchange with the ADS to include details about the observations, proposals
and bibliographic classifications related to the data in publications. This information will be used by the ADS
to provide new semantic literature search capabilities. These interactions with the ADS and WebChaser have
improved scientists’ ability to discover Chandra data in meaningful ways.
In this paper we will cover how the Chandra bibliography has grown over the years and the many ways we
have used our bibliography metadata for statistics, user services, and data discovery aids.
The Chandra Source Catalog, presently being developed by the Chandra X-ray Center, will be the definitive catalog of
all X-ray sources detected by the Chandra X-ray Observatory. The catalog interface will provide users with a simple
mechanism to perform advanced queries on the data content of the archival holdings on a source-by-source basis for
X-ray sources matching user-specified search criteria, and is intended to satisfy the needs of a broad-based group of
scientists, including those who may be less familiar with astronomical data analysis in the X-ray regime.
For each detected X-ray source, the catalog will record commonly tabulated quantities that can be queried, including
source position, dimensions, multi-band fluxes, hardness ratios, and variability statistics, derived from all of the
observations that include the source within the field of view. However, in addition to these traditional catalog elements,
for each X-ray source the catalog will include an extensive set of file-based data products that can be manipulated
interactively by the catalog user, including source images, event lists, light curves, and spectra from each observation in
which a source is detected.
In this paper, we emphasize the design and development of the Chandra Source Catalog. We describe the evaluation
process used to plan the data content of the catalog, and the selection of the tabular properties and file-based data
products to be provided to the user. We discuss our approach for managing catalog updates derived from either
additional data from new observations or from improvements to calibrations and/or analysis algorithms.
KEYWORDS: Calibration, Error analysis, Monte Carlo methods, Principal component analysis, Data modeling, Sensors, Statistical analysis, Data analysis, Point spread functions, Astronomy
Unlike statistical errors, whose importance has been well established in astronomical applications, uncertainties
in instrument calibration are generally ignored. Despite wide recognition that uncertainties in calibration can
cause large systematic errors, robust and principled methods to account for them have not been developed, and
consequently there is no mechanism by which they can be incorporated into standard astronomical data analysis.
Here we present a framework where they can be encoded such that they can be brought within the scope of
analysis. We describe this framework, which is based on a modified MCMC algorithm, and propose a format
standard derived from experience with effective area measurements of the ACIS-S detector on Chandra that can
be applied to any instrument or method of codifying systematic errors. Calibration uncertainties can then be
propagated into model parameter estimates to produce error bars that include systematic error information.
The calibration database implemented for the Chandra X-ray Observatory is the most detailed and extensive CalDB of
its kind to date. Built according to the NASA High Energy Astrophysics Science Archive Research Center (HEASARC)
CalDB prescription, the Chandra CalDB provides indexed, selectable calibration data for detector responses, mirror
effective areas, grating efficiencies, instrument geometries, default source aim points, CCD characteristics, and quantum
efficiencies, among many others. The combined index comprises approximately 500 entries. A standard FTOOLS
parametric interface allows users and tools to access the index. Unique dataset selection requires certain input
calibration parameters such as mission, instrument, detector, UTC date and time, and certain ranged parameter values.
The goals of the HEASARC CalDB design are (1) to separate software upgrades from calibration upgrades, (2) to allow
multi-mission use of analysis software (for missions with a compliant CalDB) and (3) to facilitate the use of multiple
software packages for the same data. While we have been able to meet the multivariate needs of Chandra with the
current CalDB implementation from HEASARC, certain requirements and desirable enhancements have been identified
that raise the prospect of a developmental rewrite of the CalDB system. The explicit goal is to meet Chandra's specific
needs better, but such upgrades may also provide significant advantages to CalDB planning for future missions. In
particular we believe we will introduce important features aiding in the development of mission-independent analysis
software. We report our current plans and progress.
KEYWORDS: Databases, Data archive systems, Observatories, Space operations, Data storage, X-rays, Human-machine interfaces, Data processing, Astronomy, Interfaces
We present a discussion of the lessons learned from establishing and operating the Chandra Data Archive (CDA). We offer an overview of the archive, what preparations were done before launch, the transition to operations, actual operations, and some of the unexpected developments that had to be addressed in running the archive.
From this experience we highlight some of the important issues that need to be addressed in the creation and running of an archive for a major project. Among these are the importance of data format standards; the integration of the archive with the rest of the mission; requirements throughout all phases of the mission; operational requirements; what to expect at launch; the user interfaces; how to anticipate new tasks; and overall importance of team management and organization.
KEYWORDS: Databases, Data archive systems, Observatories, X-rays, Control systems, Space operations, Interfaces, X-ray telescopes, Data processing, X-ray imaging
The Chandra Data Archive plays a central role in the Chandra X-ray Center (CXC) that manages the operations of the Chandra X-ray Observatory. We shall give an overview of two salient aspects of the CDA's operations, as they are pertinent to the operation of any large observatory.
First, in the database design it was decided to have a single observation catalog database that controls the entire life cycle of Chandra observations (as opposed to separate databases for uplink and downlink, as is common for many scientific space missions). We will discuss the pros and cons of this design choice and present some lessons learnt.
Second, we shall review the complicated network that consists of Automated (pipeline) Processing, archive ingest, Verification & Validation, reprocessing, data distribution, and public release of observations. The CXC is required to deliver high-level products to its users. This is achieved through a sophisticated system of processing pipelines. However, occasional failures as well as the need to reprocess observations complicate this seemingly simple series of actions. In addition, we need to keep track of allotted and used observing time and of proprietary periods. Central to the solution is the Processing Status Database which is described in more detail in a related poster presentation.
KEYWORDS: Databases, Data archive systems, Data processing, X-rays, Observatories, Data centers, Data acquisition, Target detection, Astrophysics, Calibration
The Chandra Data Archive has been archiving and distributing data for the Chandra X-ray Observatory and keeping observers informed of the status of their observations since shortly after launch in July 1999. Due to the complicated processing history of Chandra data, it became apparent that a database was needed to track this history on an observation by observation basis. The result is the Processing Status Database and the Chandra Observations Processing Status tool. In this paper, a description of the database design is given, followed by details of the tools which populate and display the database.
Access to the requested content is limited to institutions that have purchased or subscribe to SPIE eBooks.
You are receiving this notice because your organization may not have SPIE eBooks access.*
*Shibboleth/Open Athens users─please
sign in
to access your institution's subscriptions.
To obtain this item, you may purchase the complete book in print or electronic format on
SPIE.org.
INSTITUTIONAL Select your institution to access the SPIE Digital Library.
PERSONAL Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.