Article

Rule Reduction over Numerical Attributes in Decision Tree Using Multilayer Perceptron

Authors:

Jaeho LeeAuthors Info & Claims

PAKDD '01: Proceedings of the 5th Pacific-Asia Conference on Knowledge Discovery and Data Mining

Pages 538 - 549

Published: 16 April 2001 Publication History

Abstract

Many data sets show significant correlations between input variables, and much useful information is hidden in the data in a non-linear format. It has been shown that a neural network is better than a direct application of induction trees in modeling nonlinear characteristics of sample data. We have extracted a compact set of rules to support data with input variable relations over continuous-valued attributes. Those relations as a set of linear classifiers can be obtained from neural network modeling based on back-propagation. It is shown in this paper that variable thresholds play an important role in constructing linear classifier rules when we use a decision tree over linear classifiers extracted from a multilayer perceptron. We have tested this scheme over several data sets to compare it with the decision tree results.

References

[1]

R. Andrews, J. Diederich, and A.B. Tickle. A survey and critique of techniques for extracting rules from trained artificial neural networks. Knowledge-Based Systems, 8(6), 1996.

[2]

C. Blake, E. Keogh, and C.J. Merz. UCI repository of machine learning databases. In Preceedings of the Fifth International Conference on Machine Learning, http://www.ics.uci.edu/~mlearn, 1998.

[3]

G. Cybenko. Continuous valued neural networks with two hidden layers are sufficient. Technical report, Technical Report, Department of Computer Science, Tufts University, Medford, MA, 1988.

[4]

T.G. Dietterich, H. Hild, and G. Bakiri. A comparative study of id3 and backpropagation for english text-to-speech mapping. In Proceedings of the 1990 Machine Learning Conference, pages 24-31. Austin, TX, 1990.

[5]

U.M. Fayyad and K.B. Irani. Multi-interval discretization of continuous-valued attributes for classification learning. In Proceedings of IJCAI'93, pages 1022-1027. Morgan Kaufmann, 1993.

[6]

D.H. Fisher and K.B. McKusick. An empirical comparison of id3 and backpropagation. In Proceedings of 11th International Joint Conference on AI, pages 788-793, 1989.

[7]

L. Fu. Rule learning by searching on adaptive nets. In Preceedings of the 9th National Conference on Artificial Intelligence, pages 590-595, 1991.

[8]

L. Fu. Neural Networks in Computer Intelligence. McGraw-Hill, New York, 1994.

[9]

S.S. Haykin. Neural networks : a comprehensive foundation. Prentice Hall, Upper Saddle River, N.J., 2nd edition, 1999.

[10]

J. Hertz, R.G. Palmer, and A.S. Krogh. Introduction to the Theory of Neural Computation. Addision Wesley, Redwood City, Calif., 1991.

[11]

K. Hornik, M. Stinchrombe, and H. White. Multilayer feedforward networks are universal approximators. Neural Networks, 2(5):359-366, 1989.

[12]

K.B. Irani and Qian Z. Karsm : A combined response surface / knowledge acquisition approach for deriving rules for expert system. In TECHCON'90 Conference, pages 209-212, 1990.

[13]

D. Kim. Knowledge acquisition based on neural network modeling. Technical report, Directed Study, The University of Michigan, Ann Arbor, 1991.

[14]

D. Kim and J. Lee. Handling continuous-valued attributes in decision tree using neural network modeling. In European Conference on Machine Learning, Lecture Notes in Artificial Intelligence 1810, pages 211-219. Springer Verlag, 2000.

[15]

W. Kweldo and M. Kretowski. An evolutionary algorithm using multivariate discretization for decision rule induction. In Proceedings of the Third European Conference on Principles of Data Mining and Knowledge Discovery, pages 392-397. Springer, 1999.

[16]

J.R. Quinlan. Comparing connectionist and symbolic learning methods. In Computational Learning Theory and Natural Learning Systems, pages 445-456. MIT Press, 1994.

[17]

J.R. Quinlan. Improved use of continuous attributes in c4.5. Journal of Artificial Intelligence Approach, (4):77-90, 1996.

[18]

R. Setiono and Huan Lie. Symbolic representation of neural networks. Computer, 29(3):71-77, 1996.

[19]

R. Setiono and H. Liu. Neurolinear: A system for extracting oblique decision rules from neural networks. In European Conference on Machine Learning, pages 221- 233. Springer Verlag, 1997.

[20]

J.W. Shavlik, R.J. Mooney, and G.G. Towell. Symbolic and neural learning algorithms: An experimental comparison. Machine Learning, 6(2):111-143, 1991.

[21]

Ismail A. Taha and Joydeep Ghosh. Symbolic interpretation of artificial neural networks. IEEE Transactions on Knowledge and Data Engineering, 11(3):448- 463, 1999.

[22]

G.G. Towell and J.W. Shavlik. Extracting refined rules from knowledge-based neural networks. Machine Learning, 13(1):71-101, Oct. 1993.

Cited By

Matsak E(2009)Representing Logical Inference Steps with Digital CircuitsProceedings of the Symposium on Human Interface 2009 on Human Interface and the Management of Information. Information and Interaction. Part II: Held as part of HCI International 200910.1007/978-3-642-02559-4_20(178-184)Online publication date: 15-Jul-2009
https://dl.acm.org/doi/10.1007/978-3-642-02559-4_20

Index Terms

Rule Reduction over Numerical Attributes in Decision Tree Using Multilayer Perceptron
1. Computing methodologies
  1. Machine learning
    1. Learning paradigms
      1. Supervised learning
        Supervised learning by classification
    2. Machine learning approaches
      1. Classification and regression trees
      2. Neural networks

Index terms have been assigned to the content through auto-classification.

Recommendations

Cochannel Interference Minimization Using Wilcoxon Multilayer Perceptron Neural Network
ITC '10: Proceedings of the 2010 International Conference on Recent Trends in Information, Telecommunication and Computing

This paper presents the application of wilcoxon machines learning technique for artificial neural network in channel equalization application. The equalizer mitigates the effect of co-channel interference, inter-symbol interference in the presence of ...
Improved Classification Performance for Multiple Multilayer Perceptron (MMLP) Network Using Voting Technique
AMS '10: Proceedings of the 2010 Fourth Asia International Conference on Mathematical/Analytical Modelling and Computer Simulation

This project investigates the capability of multiple multilayer perceptron (MMLP) system with majority voting technique. It is a system which consists of all the best-performed MLPs and a single final output from these MLPs is selected by the voting ...
Centroid based Multilayer Perceptron Networks

In this study we investigate a hybrid neural network architecture for modelling purposes. The proposed network is based on the multilayer perceptron (MLP) network. However, in addition to the usual hidden layers the first hidden layer is selected to be a ...

Comments

Information & Contributors

Information

Published In

cover image Guide Proceedings

PAKDD '01: Proceedings of the 5th Pacific-Asia Conference on Knowledge Discovery and Data Mining

April 2001

592 pages

ISBN:3540419101

Publisher

Springer-Verlag

Berlin, Heidelberg

Publication History

Published: 16 April 2001

Qualifiers

Article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

1
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 23 Sep 2024

Other Metrics

View Author Metrics

Citations

Cited By

Matsak E(2009)Representing Logical Inference Steps with Digital CircuitsProceedings of the Symposium on Human Interface 2009 on Human Interface and the Management of Information. Information and Interaction. Part II: Held as part of HCI International 200910.1007/978-3-642-02559-4_20(178-184)Online publication date: 15-Jul-2009
https://dl.acm.org/doi/10.1007/978-3-642-02559-4_20

View Options

View options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Media

Figures

Other

Tables

View Table of Contents