poster

Accurate intelligible models with pairwise interactions

Authors:

Johannes Gehrke,

Giles HookerAuthors Info & Claims

KDD '13: Proceedings of the 19th ACM SIGKDD international conference on Knowledge discovery and data mining

Pages 623 - 631

https://doi.org/10.1145/2487575.2487579

Published: 11 August 2013 Publication History

Abstract

Standard generalized additive models (GAMs) usually model the dependent variable as a sum of univariate models. Although previous studies have shown that standard GAMs can be interpreted by users, their accuracy is significantly less than more complex models that permit interactions.

In this paper, we suggest adding selected terms of interacting pairs of features to standard GAMs. The resulting models, which we call GA²{M}$-models, for Generalized Additive Models plus Interactions, consist of univariate terms and a small number of pairwise interaction terms. Since these models only include one- and two-dimensional components, the components of GA²M-models can be visualized and interpreted by users. To explore the huge (quadratic) number of pairs of features, we develop a novel, computationally efficient method called FAST for ranking all possible pairs of features as candidates for inclusion into the model.

In a large-scale empirical study, we show the effectiveness of FAST in ranking candidate pairs of features. In addition, we show the surprising result that GA²M-models have almost the same performance as the best full-complexity models on a number of real datasets. Thus this paper postulates that for many problems, GA²M-models can yield models that are both intelligible and accurate.

References

[1]

http://www.liaad.up.pt/~ltorgo/Regression/DataSets.html.

[2]

http://www.cs.toronto.edu/~delve/data/datasets.html.

[3]

http://research.microsoft.com/en-us/projects/mslr/.

[4]

http://archive.ics.uci.edu/ml/.

[5]

http://www.nipsfsc.ecs.soton.ac.uk/.

[6]

http://osmot.cs.cornell.edu/kddcup/.

[7]

http://www-stat.stanford.edu/~jhf/R-RuleFit.html.

[8]

http://additivegroves.net.

[9]

E. Bauer and R. Kohavi. An empirical comparison of voting classification algorithms: Bagging, boosting, and variants. Machine learning, 36(1):105--139, 1999.

Digital Library

[10]

J. Friedman. Greedy function approximation: a gradient boosting machine. Annals of Statistics, 29:1189--1232, 2001.

[11]

J. Friedman and B. Popescu. Predictive learning via rule ensembles. The Annals of Applied Statistics, pages 916--954, 2008.

[12]

I. Guyon and A. Elisseeff. An introduction to variable and feature selection. The Journal of Machine Learning Research, 3:1157--1182, 2003.

Digital Library

[13]

T. Hastie and R. Tibshirani. Generalized additive models. Chapman & Hall/CRC, 1990.

[14]

G. Hooker. Discovering additive structure in black box functions. In KDD, 2004.

Digital Library

[15]

G. Hooker. Generalized functional anova diagnostics for high-dimensional functions of dependent variables. Journal of Computational and Graphical Statistics, 16(3):709--732, 2007.

[16]

R. Kelley Pace and R. Barry. Sparse spatial autoregressions. Statistics & Probability Letters, 33(3):291--297, 1997.

[17]

P. Li, C. Burges, and Q. Wu. Mcrank: Learning to rank using multiple classification and gradient boosting. In NIPS, 2007.

Digital Library

[18]

W. Loh. Regression trees with unbiased variable selection and interaction detection. Statistica Sinica, 12(2):361--386, 2002.

[19]

Y. Lou, R. Caruana, and J. Gehrke. Intelligible models for classification and regression. In KDD, 2012.

Digital Library

[20]

C. D. Manning, P. Raghavan, and H. Schütze. Introduction to information retrieval. Cambridge University Press Cambridge, 2008.

[21]

D. Sorokina, R. Caruana, and M. Riedewald. Additive groves of regression trees. In ECML, 2007.

Digital Library

[22]

D. Sorokina, R. Caruana, M. Riedewald, and D. Fink. Detecting statistical interactions with additive groves of trees. In ICML, 2008.

Digital Library

[23]

S. M. Weiss and N. Indurkhya. Rule-based machine learning methods for functional prediction. Journal of Artificial Intelligence Research, 3:383--403, 1995.

Digital Library

[24]

S. Wood. Thin plate regression splines. Journal of the Royal Statistical Society: Series B (Statistical Methodology), 65(1):95--114, 2003.

[25]

S. Wood. Generalized additive models: an introduction with R. CRC Press, 2006.

Digital Library

Cited By

Filom KMiroshnikov AKotsiopoulos KKannan A(2024)On marginal feature attributions of tree-based modelsFoundations of Data Science10.3934/fods.2024021(0-0)Online publication date: 2024
https://doi.org/10.3934/fods.2024021
Servati MVaccaro CDiller EPellegrino Da Silva RMafra FCao SStanley KCohen-Gadol AParker J(2024)Metabolic Insight into Glioma Heterogeneity: Mapping Whole Exome Sequencing to In Vivo Imaging with Stereotactic Localization and Deep LearningMetabolites10.3390/metabo1406033714:6(337)Online publication date: 16-Jun-2024
https://doi.org/10.3390/metabo14060337
Arslan AYagin FAlgarni AAL-Hashem FArdigò L(2024)Combining the Strengths of the Explainable Boosting Machine and Metabolomics Approaches for Biomarker Discovery in Acute Myocardial InfarctionDiagnostics10.3390/diagnostics1413135314:13(1353)Online publication date: 26-Jun-2024
https://doi.org/10.3390/diagnostics14131353
Show More Cited By

Index Terms

Accurate intelligible models with pairwise interactions
1. Computing methodologies
  1. Machine learning
    1. Machine learning approaches
      1. Logical and relational learning
        Inductive logic learning

Recommendations

Intelligible models for classification and regression
KDD '12: Proceedings of the 18th ACM SIGKDD international conference on Knowledge discovery and data mining

Complex models for regression and classification have high accuracy, but are unfortunately no longer interpretable by users. We study the performance of generalized additive models (GAMs), which combine single-feature models called shape functions ...
Efficiently Training Intelligible Models for Global Explanations
CIKM '20: Proceedings of the 29th ACM International Conference on Information & Knowledge Management

Generalized additive models (GAMs) are one of the popular methods of building intelligible models on classification and regression problems. Fitting the most accurate GAMs is usually done via gradient boosting with bagged shallow trees. However, such ...
Models and selection criteria for regression and classification
UAI'97: Proceedings of the Thirteenth conference on Uncertainty in artificial intelligence

When performing regression or classification, we are interested in the conditional probability distribution for an outcome or class variable Y given a set of explanatory or input variables X. We consider Bayesian models for this task. In particular, we ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

KDD '13: Proceedings of the 19th ACM SIGKDD international conference on Knowledge discovery and data mining

August 2013

1534 pages

ISBN:9781450321747

DOI:10.1145/2487575

Editors:
Rayid Ghani
University of Chicago
,
Ted E. Senator
SAIC
,
Paul Bradley
MethodCare, Inc.
,
Rajesh Parekh
Groupon
,
Jingrui He
Stevens Institute of Technology
,
General Chairs:
Robert L. Grossman
University of Chicago and Open Data Group
,
Ramasamy Uthurusamy
General Motors Corporation (retired)
,
Program Chairs:
Inderjit S. Dhillon
University of Texas
,
Yehuda Koren
Google

Copyright © 2013 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 11 August 2013

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Poster

Conference

KDD' 13

Sponsor:

KDD' 13: The 19th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining

August 11 - 14, 2013

Illinois, Chicago, USA

Acceptance Rates

KDD '13 Paper Acceptance Rate 125 of 726 submissions, 17%;

Overall Acceptance Rate 1,133 of 8,635 submissions, 13%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

247
Total Citations
View Citations
1,770
Total Downloads

Downloads (Last 12 months)253
Downloads (Last 6 weeks)29

Reflects downloads up to 23 Sep 2024

Other Metrics

View Author Metrics

Citations

Cited By

Filom KMiroshnikov AKotsiopoulos KKannan A(2024)On marginal feature attributions of tree-based modelsFoundations of Data Science10.3934/fods.2024021(0-0)Online publication date: 2024
https://doi.org/10.3934/fods.2024021
Servati MVaccaro CDiller EPellegrino Da Silva RMafra FCao SStanley KCohen-Gadol AParker J(2024)Metabolic Insight into Glioma Heterogeneity: Mapping Whole Exome Sequencing to In Vivo Imaging with Stereotactic Localization and Deep LearningMetabolites10.3390/metabo1406033714:6(337)Online publication date: 16-Jun-2024
https://doi.org/10.3390/metabo14060337
Arslan AYagin FAlgarni AAL-Hashem FArdigò L(2024)Combining the Strengths of the Explainable Boosting Machine and Metabolomics Approaches for Biomarker Discovery in Acute Myocardial InfarctionDiagnostics10.3390/diagnostics1413135314:13(1353)Online publication date: 26-Jun-2024
https://doi.org/10.3390/diagnostics14131353
Chang TXia HMahajan SMahajan RMaisog JVattikuti SChow CChang J(2024)Interpretable (not just posthoc-explainable) medical claims modeling for discharge placement to reduce preventable all-cause readmissions or deathPLOS ONE10.1371/journal.pone.030287119:5(e0302871)Online publication date: 9-May-2024
https://doi.org/10.1371/journal.pone.0302871
Pisu FWilliamson BNardi VParaskevas KPuig JVagal Ade Rubeis GPorcu MCau RBenson JBalestrieri ALanzino GSuri JMahammedi ASaba L(2024)Machine Learning Detects Symptomatic Plaques in Patients With Carotid Atherosclerosis on CT AngiographyCirculation: Cardiovascular Imaging10.1161/CIRCIMAGING.123.01627417:6Online publication date: Jun-2024
https://doi.org/10.1161/CIRCIMAGING.123.016274
Youngmann BCafarella MGilad ARoy S(2024)Summarized Causal Explanations For Aggregate ViewsProceedings of the ACM on Management of Data10.1145/36393282:1(1-27)Online publication date: 26-Mar-2024
https://dl.acm.org/doi/10.1145/3639328
Kairgeldin RCarreira-Perpiñán MBaeza-Yates RBonchi F(2024)Bivariate Decision Trees: Smaller, Interpretable, More AccurateProceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining10.1145/3637528.3671903(1336-1347)Online publication date: 25-Aug-2024
https://dl.acm.org/doi/10.1145/3637528.3671903
Vengroff D(2024)Impact Charts: A Tool for Identifying Systematic Bias in Social Systems and DataProceedings of the 2024 ACM Conference on Fairness, Accountability, and Transparency10.1145/3630106.3658965(1187-1198)Online publication date: 3-Jun-2024
https://dl.acm.org/doi/10.1145/3630106.3658965
Nilofer ASasikala S(2024)A Comparative Study of Machine Learning Algorithms Using Explainable Artificial Intelligence System for Predicting Liver DiseaseComputing Open10.1142/S2972370123500034Online publication date: 16-Feb-2024
https://doi.org/10.1142/S2972370123500034
Nasir MSummerfield NSimsek SOztekin A(2024)An interpretable machine learning methodology to generate interaction effect hypotheses from complex datasetsDecision Sciences10.1111/deci.12642Online publication date: 13-Aug-2024
https://doi.org/10.1111/deci.12642
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents