Previous
abstract 
Next
abstract
Session 64 - New Digital Surveys in the Optical and Near IR: Technical Challenges and Scientific Opportunities - II.
Topical, Oral session, Wednesday, June 10
Friars,
We will discuss the novel database technologies and techniques under development within the Digital Sky project which are required to seamlessly integrate and mine large datasets. The astronomical field is entering a new era as multiple, large area, digital sky surveys are coming on-line. These new datasets are not only interesting in their own right, but a truly revolutionary leap forward arises in the aggregation of complimentary, multi-wavelength surveys (i.e. the cross-identification of billions of sources). Federating these large, complex datasets, however, is not an easy task and represents a serious technical challenge. It is the solution to this challenge, as well as the development of the technology necessary to effectively mine the aggregate data, which is the goal of the Digital Sky project.
These technologies include both hardware, to optimize the retrieval and analysis of geographically distributed datasets, and software, to simplify the ingestion of archives into the digital sky federation. For the former, we are utilizing our NPACI (National Partnership for Advanced Computational Infrastructure) relationship to identify new techniques for parallelizing both the retrieval and computational requirements, as well as innovative approaches, such as network weather, to improve the response between the distributed system components. In order to grow and develop, the digital sky will need to incorporate new surveys and datasets in addition to its original tenants: the 2MASS (2 Micron All Sky Survey) and DPOSS (Digitized Palomar Observatory Sky Survey) projects. This requirement necessitates a standardized, modular interface for each archive that allows for an exchange of both the types of data and the types of services provided by the constituent archives. This interface is built on a common sky partitioning scheme: the Hierarchical Triangulated Mesh, and a standardized metadata for astronomical surveys.
