Effective pruning method for a multiple classifier system based on self-generating neural networks

ABSTRACT

Recently, multiple classifier systems (MCS) have been used for practical applications to improve classification accuracy. Self-generating neural networks (SGNN) are one of the suitable base-classifiers for MCS because of their simple setting and fast learning. However, the computational cost of the MCS increases in proportion to the number of SGNN. In this paper, we propose a novel pruning method for the structure of the SGNN in the MCS. Experiments have been conducted to compare the pruned MCS with an unpruned MCS, the MCS based on C4.5, and k-nearest neighbor method. The results show that the pruned MCS can improve its classification accuracy as well as reducing the computational cost.

References

1. J. Han and M. Kamber. Data Mining: Concepts and Techniques. Morgan Kaufmann Publishers, San Francisco, CA, 2000.
2. J. R. Quinlan. Bagging, Boosting, and C4.5. In Proceedings of the Thirteenth National Conference on Artificial Intelligence, pages 725-730, Portland, OR, 1996.
3. G. Rätsch, T. Onoda, and K.-R. Müller. Soft margins for AdaBoost. Machine Learning, 42(3):287-320, 2001.
4. C. M. Bishop. Neural Networks for Pattern Recognition. Oxford University Press, New York, 1995.
5. R. O. Duda, P. E. Hart, and D. G. Stork. Pattern Classification. John Wiley & Sons Inc., New York, 2nd ed., 2000.
6. W. X. Wen, A. Jennings, and H. Liu. Learning a neural tree. In the International Joint Conference on Neural Networks, Beijing, China, 1992. This paper is available at ftp://ftp.cis.ohio-state.edu/pub/neuroprose/wen.sgnt-learn.ps.Z.
7. T. Kohonen. Self-Organizing Maps. Springer-Verlag, Berlin, 1995.
8. H. Inoue and H. Narihisa. Improving generalization ability of self-generating neural networks through ensemble averaging. In T. Terano, H. Liu, and A. L P. Chen, eds, The Fourth Pacific-Asia Conference on Knowledge Discovery and Data Mining, vol. 1805 of LNAI, pages 177-180, Springer-Verlag, 2000.
9. M. Stone. Cross-validation: A review. Math. Operationsforsch. Statist., Ser. Statistics , 9(1):127-139, 1978.
10. L. Breiman. Bagging predictors. Machine Learning, 24:123-140, 1996.
11. J. R. Quinlan. C4.5: Programs for Machine Learning. Morgan Kaufmann, San Mateo, CA, USA, 1993.
12. C.L. Blake and C.J. Merz. UCI repository of machine learning databases, University of California, Irvine, Dept of Information and Computer Science, 1998. Datasets is available at http://www.ics.uci.edu/~mlearn/MLRepository.html.
13. E. A. Patrick and F. P. Fischer. A generalized k-nearest neighbor rule. Information and Control, 16(2):128-152, 1970.

Index Terms

• Computing methodologies

  • Machine learning

    • Learning paradigms

      • Supervised learning

        • Supervised learning by classification

    • Machine learning approaches

      • Classification and regression trees

      • Neural networks