[Previous] | [Session 60] | [Next]

[60.04] Low-Storage, Sequential, Simultaneous Estimation of Multiple Quantiles for Massive Datasets

J. P. McDermott, G. J. Babu (Dept of Statistics, Penn State University), E. D. Feigelson (Dept of Astronomy & Astrophysics, Penn State University)

Due to the massive amounts of data to be stored in a Virtual Observatory, new non-traditional statistical methods are required to process the data efficiently. We propose a low-storage, single-pass, sequential method for simultaneous estimation of multiple quantiles for massive datasets. The proposed method uses estimated ranks, assigned weights, and a scoring function that determines the most attractive candidate data points for estimates of the quantiles. The method uses a small fixed amount of storage and its computation time is O(n). Asymptotically the proposed estimates are as accurate as the sample quantiles. We compare the proposed method's performance with that of the empirical distribution function through simulation study. This work is produced by an interdisciplinary collaborative effort supported by the NSF FRG grant DMS-0101360.

The author(s) of this abstract have provided an email address for comments about the abstract: mcdermott@stat.psu.edu

[Previous] | [Session 60] | [Next]

Bulletin of the American Astronomical Society, 34
© 2002. The American Astronomical Soceity.