poster

Stochastic gradient boosted distributed decision trees

Authors:

Jerry Ye,

Jyh-Herng Chow,

Jiang Chen,

Zhaohui ZhengAuthors Info & Claims

CIKM '09: Proceedings of the 18th ACM conference on Information and knowledge management

Pages 2061 - 2064

https://doi.org/10.1145/1645953.1646301

Published: 02 November 2009 Publication History

Get Access

Abstract

Stochastic Gradient Boosted Decision Trees (GBDT) is one of the most widely used learning algorithms in machine learning today. It is adaptable, easy to interpret, and produces highly accurate models. However, most implementations today are computationally expensive and require all training data to be in main memory. As training data becomes ever larger, there is motivation for us to parallelize the GBDT algorithm. Parallelizing decision tree training is intuitive and various approaches have been explored in existing literature. Stochastic boosting on the other hand is inherently a sequential process and have not been applied to distributed decision trees. In this work, we present two different distributed methods that generates exact stochastic GBDT models, the first is a MapReduce implementation and the second utilizes MPI on the Hadoop grid environment.

References

[1]

CARAGEA, D., SILVESCU, A., AND HONAVAR, V. A framework for learning from distributed data using sufficient statistics and its application to learning decision trees. International Journal of Hybrid Intelligent Systems 1, 2 (2004).

Digital Library

Google Scholar

[2]

CHEN, K., LU, R., WONG, C. K., SUN, G., HECK, L., AND TSENG, B. L. Trada: tree based ranking function adaptation. In CIKM (2008), pp. 1143--1152.

Digital Library

Google Scholar

[3]

DEAN, J., AND GHEMAWAT, S. Mapreduce: simplified data processing on large clusters. Commun. ACM 51, 1 (2008), 107--113.

Digital Library

Google Scholar

[4]

FOUNDATION, A. Apache hadoop project. lucene.apache.org/hadoop.

Google Scholar

[5]

FRIEDMAN, J. H. Greedy function approximation: A gradient boosting machine. Annals of Statistics 29 (2001), 1189--1232.

Crossref

Google Scholar

[6]

FRIEDMAN, J. H. Stochastic gradient boosting. Comput. Stat. Data Anal. 38, 4(February 2002), 367--378.

Digital Library

Google Scholar

[7]

GEHRKE, J., RAMAKRISHNAN, R., AND GANTI, V. Rainforest - a framework for fast decision tree construction of large datasets. In VLDB'98, Proceedings of 24rd International Conference on Very Large Data Bases, August 24-27, 1998, New York City, New York, USA (1998), A. Gupta, O. Shmueli, and J. Widom, Eds., Morgan Kaufmann, pp. 416--427.

Digital Library

Google Scholar

[8]

PROVOST, F., KOLLURI, V., AND FAYYAD, U. A survey of methods for scaling up inductive algorithms. Data Mining and Knowledge Discovery 3 (1999), 131--169.

Digital Library

Google Scholar

[9]

QUINLAN, J. R. Induction of decision trees. In Machine Learning (1986), pp. 81--106.

Digital Library

Google Scholar

[10]

SHAFER, J. C., AGRAWAL, R., AND 0002, M. M. Sprint: A scalable parallel classifier for data mining. In VLDB'96, Proceedings of 22th International Conference on Very Large Data Bases, September 3-6, 1996, Mumbai (Bombay), India (1996), T. M. Vijayaraman, A. P. Buchmann, C. Mohan, and N. L. Sarda, Eds., Morgan Kaufmann, pp. 544--555.

Digital Library

Google Scholar

[11]

STATISTICS, L. B., AND BREIMAN, L. Random forests. In Machine Learning (2001), pp. 5--32.

Digital Library

Google Scholar

[12]

SU, J., AND ZHANG, H. A fast decision tree learning algorithm. In AAAI (2006).

Digital Library

Google Scholar

[13]

ZHENG, Z., CHEN, K., SUN, G., AND ZHA, H. A regression framework for learning ranking functions using relative relevance judgments. Proceedings of the 30th annual international ACM SIGIR conference on Research and development in information retrieval (2007), 287--294.

Digital Library

Google Scholar

Cited By

View all

Thottempudi P(2024)Leveraging IoT and Machine Learning for Improved Health Prediction SystemsSustainable Science and Intelligent Technologies for Societal Development10.4018/979-8-3693-1186-8.ch016(278-306)Online publication date: 5-Jan-2024
https://doi.org/10.4018/979-8-3693-1186-8.ch016
Baser OSamayoa GYapar NBaser E(2024)Artificial Intelligence in Identifying Patients With Undiagnosed Nonalcoholic SteatohepatitisJournal of Health Economics and Outcomes Research10.36469/jheor.2024.123645(86-94)Online publication date: 25-Sep-2024
https://doi.org/10.36469/jheor.2024.123645
Baser OSamayoa GYapar NBaser E(2024)Artificial Intelligence in Identifying Patients With Undiagnosed Nonalcoholic SteatohepatitisJournal of Health Economics and Outcomes Research10.36469/001c.12364511:2Online publication date: 25-Sep-2024
https://doi.org/10.36469/001c.123645
Show More Cited By

Index Terms

Stochastic gradient boosted distributed decision trees
1. Computing methodologies
  1. Machine learning
2. Information systems
  1. Information retrieval
    1. Retrieval models and ranking

Recommendations

Adversarial Training of Gradient-Boosted Decision Trees
CIKM '19: Proceedings of the 28th ACM International Conference on Information and Knowledge Management

Adversarial training is a prominent approach to make machine learning (ML) models resilient to adversarial examples. Unfortunately, such approach assumes the use of differentiable learning models, hence it cannot be applied to relevant ML techniques, ...
Ensembles of decision trees for imbalanced data
MCS'11: Proceedings of the 10th international conference on Multiple classifier systems

Ensembles of decision trees are considered for imbalanced datasets. Conventional decision trees (C4.5) and trees for imbalanced data (CCPDT: Class Confidence Proportion Decision Tree) are used as base classifiers. Ensemble methods, based on ...
Building boosted classification tree ensemble with genetic programming
GECCO '18: Proceedings of the Genetic and Evolutionary Computation Conference Companion

Adaptive boosting (AdaBoost) is a method for building classification ensemble, which combines multiple classifiers built in an iterative process of reweighting instances. This method proves to be a very effective classification method, therefore it was ...

Comments

Information & Contributors

Information

Published In

CIKM '09: Proceedings of the 18th ACM conference on Information and knowledge management

November 2009

2162 pages

ISBN:9781605585123

DOI:10.1145/1645953

General Chairs:
David Cheung
University of Hong Kong, Hong Kong
,
Il-Yeol Song
Drexel University, USA
,
Program Chairs:
Wesley Chu
UCLA, USA
,
Xiaohua Hu
Drexel University, USA
,
Jimmy Lin
University of Maryland, USA

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 02 November 2009

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Poster

Conference

CIKM '09

Sponsor:

CIKM '09: Conference on Information and Knowledge Management

November 2 - 6, 2009

Hong Kong, China

Acceptance Rates

Overall Acceptance Rate 1,861 of 8,427 submissions, 22%

Upcoming Conference

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

205
Total Citations
View Citations
3,172
Total Downloads

Downloads (Last 12 months)138
Downloads (Last 6 weeks)11

Reflects downloads up to 23 Sep 2024

Other Metrics

View Author Metrics

Citations

Cited By

View all

Thottempudi P(2024)Leveraging IoT and Machine Learning for Improved Health Prediction SystemsSustainable Science and Intelligent Technologies for Societal Development10.4018/979-8-3693-1186-8.ch016(278-306)Online publication date: 5-Jan-2024
https://doi.org/10.4018/979-8-3693-1186-8.ch016
Baser OSamayoa GYapar NBaser E(2024)Artificial Intelligence in Identifying Patients With Undiagnosed Nonalcoholic SteatohepatitisJournal of Health Economics and Outcomes Research10.36469/jheor.2024.123645(86-94)Online publication date: 25-Sep-2024
https://doi.org/10.36469/jheor.2024.123645
Baser OSamayoa GYapar NBaser E(2024)Artificial Intelligence in Identifying Patients With Undiagnosed Nonalcoholic SteatohepatitisJournal of Health Economics and Outcomes Research10.36469/001c.12364511:2Online publication date: 25-Sep-2024
https://doi.org/10.36469/001c.123645
Liu SMa RZhao CLi ZXiao JLi Q(2024)BPCoach: Exploring Hero Drafting in Professional MOBA Tournaments via Visual AnalyticsProceedings of the ACM on Human-Computer Interaction10.1145/36373038:CSCW1(1-31)Online publication date: 26-Apr-2024
https://dl.acm.org/doi/10.1145/3637303
Song WEo JUm TJeon MChun B(2024)Blaze: Holistic Caching for Iterative Data ProcessingProceedings of the Nineteenth European Conference on Computer Systems10.1145/3627703.3629558(370-386)Online publication date: 22-Apr-2024
https://dl.acm.org/doi/10.1145/3627703.3629558
Zhang LLi ZChandra SNasrallah F(2024)Multi-Modal Traumatic Brain Injury Prognosis via Structure-Aware Field-Wise LearningIEEE Transactions on Knowledge and Data Engineering10.1109/TKDE.2024.336438536:8(4089-4100)Online publication date: Aug-2024
https://doi.org/10.1109/TKDE.2024.3364385
Grigorev AMihăiţă ASaleh KChen F(2024)Automatic Accident Detection, Segmentation and Duration Prediction Using Machine LearningIEEE Transactions on Intelligent Transportation Systems10.1109/TITS.2023.332363625:2(1547-1568)Online publication date: Feb-2024
https://doi.org/10.1109/TITS.2023.3323636
Yin XZhu YChen C(2024)Reinforcement Learning with Neighborhood Search and Self-learned Rules2024 IEEE 13th Data Driven Control and Learning Systems Conference (DDCLS)10.1109/DDCLS61622.2024.10606856(1474-1481)Online publication date: 17-May-2024
https://doi.org/10.1109/DDCLS61622.2024.10606856
Cai YShi YWang SChen Y(2024)A Method for Predicting the Film Thickness of IC Deposited Films Based on FCBF-CATBOOST2024 Conference of Science and Technology for Integrated Circuits (CSTIC)10.1109/CSTIC61820.2024.10531926(1-3)Online publication date: 17-Mar-2024
https://doi.org/10.1109/CSTIC61820.2024.10531926
Castro RAndrade DFraguela B(2024)STuning-DL: Model-Driven Autotuning of Sparse GPU Kernels for Deep LearningIEEE Access10.1109/ACCESS.2024.340232612(70581-70599)Online publication date: 2024
https://doi.org/10.1109/ACCESS.2024.3402326
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Cited By

Index Terms

Recommendations

Adversarial Training of Gradient-Boosted Decision Trees

Ensembles of decision trees for imbalanced data

Building boosted classification tree ensemble with genetic programming

Comments

Published In

Sponsors

Publisher

Publication History

Permissions

Check for updates

Author Tags

Qualifiers

Conference

Acceptance Rates

Upcoming Conference

Other Metrics

Article Metrics

Other Metrics

Cited By

Login options

Full Access

PDF

eReader

Abstract

References

Cited By

Index Terms

Recommendations

Adversarial Training of Gradient-Boosted Decision Trees

Ensembles of decision trees for imbalanced data

Building boosted classification tree ensemble with genetic programming

Comments

Information

Published In

Sponsors

Publisher

Publication History

Permissions

Check for updates

Author Tags

Qualifiers

Conference

Acceptance Rates

Upcoming Conference

Contributors

Other Metrics

Bibliometrics

Article Metrics

Other Metrics

Citations

Cited By

Get Access

Login options

Full Access

View options

PDF

eReader

Figures

Other

Share

Share this Publication link

Share on social media

Affiliations