article

Free access

The P² algorithm for dynamic calculation of quantiles and histograms without storing observations

Authors:

Raj Jain and

Imrich ChlamtacAuthors Info & Claims

Communications of the ACM, Volume 28, Issue 10

Pages 1076 - 1085

https://doi.org/10.1145/4372.4378

Published: 01 October 1985 Publication History

PDF eReader

Abstract

A heuristic algorithm is proposed for dynamic calculation of the median and other quantiles. The estimates are produced dynamically as the observations are generated. The observations are not stored; therefore, the algorithm has a very small and fixed storage requirement regardless of the number of observations. This makes it ideal for implementing in a quantile chip that can be used in industrial controllers and recorders. The algorithm is further extended to histogram plotting. The accuracy of the algorithm is analyzed.

References

[1]

Buchholz, W. File organization and addressing. IBM Syst. 1. 2, (196% 66-111.

Google Scholar

[2]

Knott, G.D. Hashing functions. Cornput. 1. 18. 3 (1975). 265-276.

Google Scholar

[3]

Knuth. D.E. The Art of Computer Programming. Vol. 3, Sorting and Searching. Addison-Wesley, Reading, Mass., 1973.

Digital Library

Google Scholar

[4]

Pearson, ES. and Hartley, H.O. Biometrikn, Tables for Stotisticinns. Cambridge University Press, Cambridge, 1972.

Google Scholar

[5]

Peterson, W.W. Addressing for random access storage. IBM I. Res. Develop. 1. 2 (Apr. 1957). 130-146.

Digital Library

Google Scholar

[6]

Van der Pool, J.A. Optimum storage allocation for initial loading of a file. IBM J. Res. Develop. 16 (Nov. 1972), 579-586.

Digital Library

Google Scholar

[7]

Vitter, J.S. Implementations for coalesced hashing. Commun. ACM 25, 12 (Dec. 1962). 911-926.

Digital Library

Google Scholar

Cited By

View all

Pugh SFosdick BNehring MGallichotte EVandeWoude SWilson A(2024)Estimating cutoff values for diagnostic tests to achieve target specificity using extreme value theoryBMC Medical Research Methodology10.1186/s12874-023-02139-524:1Online publication date: 8-Feb-2024
https://doi.org/10.1186/s12874-023-02139-5
Girault HBonniot LNeumann C(2024)Inferring Video Streaming Quality of Experience at Scale using Incremental Statistics from CDN LogsProceedings of the 3rd Mile-High Video Conference10.1145/3638036.3640803(34-40)Online publication date: 11-Feb-2024
https://dl.acm.org/doi/10.1145/3638036.3640803
Sukhoplyuev DNazarov A(2024)Methods of Descriptive Statistics in Telemetry Tasks2024 Systems of Signals Generating and Processing in the Field of on Board Communications10.1109/IEEECONF60226.2024.10496798(1-5)Online publication date: 12-Mar-2024
https://doi.org/10.1109/IEEECONF60226.2024.10496798
Show More Cited By

Index Terms

The P² algorithm for dynamic calculation of quantiles and histograms without storing observations

Recommendations

Performance analysis of triple-frequency ambiguity resolution with BeiDou observations

We investigate triple-frequency ambiguity resolution performance using real BeiDou data. We test four ambiguity resolution (AR) methods which are applicable to triple-frequency observations. These are least squares ambiguity decorrelation adjustment (...
Read More
Ionospheric tomography based on GNSS observations of the CMONOC: performance in the topside ionosphere

This study carries out a quantitative analysis of the performance of ionospheric tomography in the topside ionosphere, utilizing data of October 2011 collected from 260 Global Navigation Satellite System (GNSS) stations in the Crustal Movement ...
Read More
A satellite-based method for modeling ionospheric slant TEC from GNSS observations: algorithm and validation
Abstract
Total Electron Content (TEC) modeling is critical for Global Navigation Satellite System (GNSS) users to mitigate ionospheric delay errors. The mapping function is usually used for Vertical TEC ionospheric correction models for slant and vertical ...
Read More

Reviews

Reviewer: Leonard Francis Zettel

.abstract A heuristic algorithm is proposed for dynamic calculation of the median and other quantiles. . . . [T]he algorithm has a very small and fixed storage requirement regardless of the number of observations. . . . The algorithm is further extended to histogram plotting. . . . — From the Authors' Abstract The algorithm looks interesting and is presented in sufficient detail so that one could implement it. It requires the storage of ten numbers for the calculation of a quantile; thus, it would probably save storage on large samples of observations, since conventional algorithms require the equivalent to access the data sorted from lowest to highest. The analysis that accompanies the algorithm is best described as a lick, a promise, and a bit of arm waving. There is no formal analysis of execution behavior. Nothing is proven about the statistical behavior of the estimators that the algorithm produces. Results of Monte Carlo runs of 50 samples are given for various cases. Similar work presented in the Journal of the American Statistical Association would typically have sample sizes of 10,000.

Access critical reviews of Computing literature here

Become a reviewer for Computing Reviews.

Comments

Information & Contributors

Information

Published In

Communications of the ACM Volume 28, Issue 10

Oct. 1985

64 pages

ISSN:0001-0782

EISSN:1557-7317

DOI:10.1145/4372

Editor:
Peter J. Denning
NASA Ames Research Center, Moffett Field, CA

Issue’s Table of Contents

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 01 October 1985

Published in CACM Volume 28, Issue 10

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Qualifiers

Article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

155
Total Citations
View Citations
1,422
Total Downloads

Downloads (Last 12 months)215
Downloads (Last 6 weeks)21

Other Metrics

View Author Metrics

Citations

Cited By

View all

Pugh SFosdick BNehring MGallichotte EVandeWoude SWilson A(2024)Estimating cutoff values for diagnostic tests to achieve target specificity using extreme value theoryBMC Medical Research Methodology10.1186/s12874-023-02139-524:1Online publication date: 8-Feb-2024
https://doi.org/10.1186/s12874-023-02139-5
Girault HBonniot LNeumann C(2024)Inferring Video Streaming Quality of Experience at Scale using Incremental Statistics from CDN LogsProceedings of the 3rd Mile-High Video Conference10.1145/3638036.3640803(34-40)Online publication date: 11-Feb-2024
https://dl.acm.org/doi/10.1145/3638036.3640803
Sukhoplyuev DNazarov A(2024)Methods of Descriptive Statistics in Telemetry Tasks2024 Systems of Signals Generating and Processing in the Field of on Board Communications10.1109/IEEECONF60226.2024.10496798(1-5)Online publication date: 12-Mar-2024
https://doi.org/10.1109/IEEECONF60226.2024.10496798
Stephanou MVarughese M(2024)hermiter: R package for sequential nonparametric estimationComputational Statistics10.1007/s00180-023-01382-039:3(1127-1163)Online publication date: 1-May-2024
https://dl.acm.org/doi/10.1007/s00180-023-01382-0
Wang BChen RTang L(2023)EasyQuantile: Efficient Quantile Tracking in the Data PlaneProceedings of the 7th Asia-Pacific Workshop on Networking10.1145/3600061.3600084(123-129)Online publication date: 29-Jun-2023
https://dl.acm.org/doi/10.1145/3600061.3600084
Hariat MManzanera AFilliat D(2023)Rebalancing gradient to improve self-supervised co-training of depth, odometry and optical flow predictions2023 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV)10.1109/WACV56688.2023.00132(1267-1276)Online publication date: Jan-2023
https://doi.org/10.1109/WACV56688.2023.00132
Wang MBarbu A(2023)Online Feature Screening for Data Streams With Concept DriftIEEE Transactions on Knowledge and Data Engineering10.1109/TKDE.2022.323275235:11(11693-11707)Online publication date: 1-Nov-2023
https://dl.acm.org/doi/10.1109/TKDE.2022.3232752
Liang ZGai JChen XMa TLiu Q(2023)A Method for Threshold Setting and False Alarm Probability Evaluation for Radar DetectorsSignal Processing10.1016/j.sigpro.2023.108930207:COnline publication date: 1-Jun-2023
https://dl.acm.org/doi/10.1016/j.sigpro.2023.108930
Rapp TPeters CDachsbacher C(2022)Image-based Visualization of Large Volumetric Data Using MomentsIEEE Transactions on Visualization and Computer Graphics10.1109/TVCG.2022.316534628:6(2314-2325)Online publication date: 1-Jun-2022
https://dl.acm.org/doi/10.1109/TVCG.2022.3165346
Ben Ameur FBalis JVandenhoeck RLani APoedts S(2022)r-adaptive algorithms for supersonic flows with high-order Flux Reconstruction methodsComputer Physics Communications10.1016/j.cpc.2022.108373276(108373)Online publication date: Jul-2022
https://doi.org/10.1016/j.cpc.2022.108373
Show More Cited By

View Options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Abstract

References

Cited By

Index Terms

Recommendations

Performance analysis of triple-frequency ambiguity resolution with BeiDou observations

Ionospheric tomography based on GNSS observations of the CMONOC: performance in the topside ionosphere

A satellite-based method for modeling ionospheric slant TEC from GNSS observations: algorithm and validation

Reviews

Access critical reviews of Computing literature here

Comments

Information

Published In

Publisher

Publication History

Permissions

Check for updates

Qualifiers

Contributors

Other Metrics

Bibliometrics

Article Metrics

Other Metrics

Citations

Cited By

View options

PDF

eReader

Get Access

Login options

Full Access

Figures

Other

Share

Share this Publication link

Share on social media

Affiliations