Learning and decision-making in the framework of fuzzy lattices

ABSTRACT

A novel theoretical framework is delineated for supervised and unsupervised learning. It is called framework of fuzzy lattices, or FL-framework for short, and it suggests mathematically sound tools for dealing separately and/or jointly with disparate types of data including vectors of numbers, fuzzy sets, symbols, etc. Specific schemes are proposed for clustering and classification having the capacity to deal with both missing and don't care data values; the schemes in question can be implemented as neural networks. The proposed learning schemes are employed here for pattern recognition on seven data sets including benchmark data sets, and the results are compared with those ones by various learning techniques from the literature. Finally, aiming at a mutual cross-fertilization, the FL-framework is associated with established theories for learning and/or decision-making including probability theory, fuzzy set theory, Bayesian decision-making, theory of evidence, and adaptive resonance theory.

References

1. Aba, D.W. (1998), "The omnipresence of case-based reasoning in science and application," Knowledge-Based Systems, vol. 11, no. 5-6, pp. 261-273.
2. Amari, S. and Wu, S. (1999), "Improving Support Vector Machine Classifiers by Modifying Kernel Functions," Neural Networks, vol. 12, no. 6, pp. 783-789.
3. Baker, D., Brett, P.N., Griffiths, MV., and Reyes, L. (1996), "A Mechatronic Tool for Ear Surgery A Case Study of Some Design Characteristics." Mechatronics, vol. 6, no. 4, pp. 461477.
4. Baralis, E., Ceri, S., and Paraboschi, S. (1998), "Compile-Time and Runtime Analysis of Active Behaviors," IEEE Trans. Knowledge Data Engineering, vol. 10, no. 3, pp. 353-370.
5. Bianchini, M., Frasconi, P., and Gori, M. (1995), "Learning Without Local Minima in Radial Basis Function Networks," IEEE Trans. Neural Networks, vol. 6, no. 3, pp. 749-756.
6. Birkhoff, 0. (1967), Lattice Theory, American Mathematical Society, Colloquium Publications, vol. 25, Providence, RI.
7. Bishop, C. (1991), "Improving the generalization properties of radial basis function neural networks," Neural Computation, vol. 3, no. 4, pp. 579-588.
8. Blumer, A., Ehrenfeucht, A., Haussler, D. and Warmuth, M.K. (1989), "Learnability and the Vapnik-Chervonenkis dimension," Journal of the ACM, vol. 36, no. 4, pp. 929-965.
9. Bugler, P.L. (1987) "Shafer-Dempster reasoning with applications to multisensor target identification systems," IEEE Trans. Systems, Man, and Cybernetics, vol. 17, no. 6, pp. 968977.
10. Bryson, N. and Mobolurin, A. (1998), "A qualitative discriminant approach for generating quantitative belief functions," IEEE Trans. Knowledge Data Engineering, vol. 10, no. 2, pp. 345-348.
11. Buede, D.M. and Girardi, P. (1997), "A target identification comparison of Bayesian and Dempster-Shafer multisensor fusion," IEEE Trans. Sys., Man, and Cybernetics - A, vol. 27, no. 5, pp. 569-577.
12. Card, H.C., Rosendahl, O.K., McNeill, D.K., and McLeod, R.D. (1998), "Competitive learning algorithms and neurocomputer architecture." IEEE Trans. Computers, vol. 47, no. 8, pp. 847-858
13. Carpenter, GA. and Grossberg, S. (1987), "A massively parallel architecture for selforganizing neural pattern recognition machine,' Computer Vision, Graphics and Image Understanding, vol. 37, pp. 54-115.
14. Carpenter, GA., Grossberg, S., and Rosen, D.B. (199!), "Fuzzy ART: Fast stable learning and categorization of analog patterns by an adaptive resonance system," Neural Networks, vol. 4, pp. 759-771.
15. Cercone, N., An, A. and Chan, C. (1999), "Rule-induction and case-based reasoning: hybrid architectures appear advantageous," IEEE Trans. Knowledge and Data Engineering, vol. II, no. I, pp. 166-174.
16. Chen, J-Q. and Xi, Y-G. (1998), "Nonlinear system modeling by competitive learning and adaptive fuzzy inference system," IEEE Trans. Systems, Man, and Cybernetics - C, vol. 28, no. 2, pp. 231-238.
17. Chen, S., Cowan, C.F.N., and Grant, P.M. (1991), "Orthogonal least squares learning for radial basis function networks," IEEE Trans. Neural Networks, vol. 2, no. 2, pp. 302-309.
18. Davey, BA. and Priestley, HA. (1990), Introduction to Lattices and Order Cambridge University Press, Cambridge, Great Britain.
19. Dempster, A.P. (1967), "Upper and lower probabilities induced by multivalued mappings," Annals of Math. Stat., vol. 38, pp. 325-329.
20. Dunyak, 3., Saad, 1W., and Wunsch, D. (1999), "A theory of independent fuzzy probability for system reliability," IEEE Trans. Fuzzy Systems, vol. 7, no. 2, pp. 286-294.
21. Evans, B. and Fisher, D. (1994), "Overcoming process delays with decision tree induction," IEEE Expert, pp. 60-66, February.
22. Fahlman, SE. and White, M., Carnegie Mellon University (CMU) repository of neural net benchmarks. Available at ht tp:I /www. . boltz - cs. cmu - edo /.
23. Feller, W. (1968), An introduction to Probability Theory and Its Applications, third edition. John Wiley & Sons, New York, NY.
24. Fixsen, D. and Mahler, R.P.S. (1997), 'The modified Dempster-Shafer approach to classification," IEEE Trans. Systems, Man, and Cybernetics - A, vol. 27, no. 1, pp. 96-104.
25. Fritzke, B. (1994), "Growing cell structures - A self-organizing network for unsupervised learning," Neural Networks, vol. 7, no. 9, pp. 1441-1460.
26. Gaines, BR. (1978). "Fuzzy and probability uncertainty logics," Information and Control, vol. 38, pp. 154-169.
27. Georgiopoulos, M., Dagher, I., Heileman, G.L., and Behis, G. (1999), Properties of learning of a fuzzy ART variant," Neural Networks, vol. 12, no. 6, pp. 837-850.
28. Giarratano, 3. and Riley G. (1994), Expert Systems - Principles and Programming, second edition, PWS Publishing Company, Boston, MA.
29. Giles, CL., Miller, C.B., Chen, D., Chen, H.H., Sun, G.Z,, and Lee. Y.C. (1992), "Learning and extracting finite state automata with second-order recurrent neural networks," Neural Computation, vol. 4, no. 3, pp. 393-405.
30. Gratzer, G. (1971), Lattice theory, WI-I. Freeman, San Francisco, CA.
31. Healy, M.J. (1999), 'A topological semantics for rule extraction with neural networks,'' Connection Science, vol. 11, no. I , pp. 91-113
32. Healy, MT and Caudell, T.P. (1997), 'Acquiring rule sets as a product of learning in a logical neural architecture," IEEE Trans. Neural Networks, vol. 8, no. 3, pp. 461-474.
33. Hong, L. and Jam, A. (1998), "Integrating faces and fingerprints for personal identification." IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 20, no. 12, pp. 1295-1307.
34. Horiuchi, T. (1998), "Decision rule for pattern classification by integrating interval feature values," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 20, no. 4, pp. 440448.
35. Hummel, R. and Manevitz, L. (1996), "A statistical approach to the representation of uncertainty in beliefs using spread of opinion," IEEE Trans Sys., Man, and Cybernetics - A, vol. 26, no. 3, pp. 378-384.
36. Ishibuchi, H., Fujioka, R., and Tanaka, H. (1993), "Neural networks that team from fuzzy if-then rules," IEEE Trans. on Fuzzy Systems, vol. I, no. 2, pp. S. 97.
37. John, M.F.St. and McClelland, J.L. (1990), "Learning and applying contextual constraints in sentence comprehension," Artificial Intelligence, vol. 46, pp. 5-46
38. Joshi, A., Ramakrishman, N., Houstis, E.N., and Rice, J.R. (1997), "On neurobiological, neuro-fuzzy, machine learning, and statistical pauci recognition techniques," IEEE Trans. Neural Networks, vol. 8, no. I, pp. 18-3
39. Kaburlasos, V.G. (1992), Adaptive Resonance Theory with Supervised Learning and Large Database Applications. Ph.D. Thesis, Dept. Electrical Engineering University of Nevada, Reno.
40. Kaburlasos, V.G. and Petridis, V. (1997), "Fuzzy lattice neurocomputing (FLN A novel connectionist scheme for versatile learning and decision making by clustering," International Journal of Computers and Their Applications, vol. .1 no. 2, pp. 31-43.
41. Kaburlasos, V.G. and Petridis, V. (1998), "A unifying framework for hybtrJ information processing," Proceedings of the JSCA 7th Intl. Conf on Intellig Systems (IC/S'98), Paris, France, pp. 68-7 1.
42. Kaburlasos, V.G., Petridis, V., Brett, P., and Baker, D. (1998), "Learning linear association of drilling profiles in stapedotomy surgery," Proceeding., the IEEE /998 Intl. Conf on Robotics & Automation (ICRA '98), Leoveri Belgium, pp. 705-7 10.
43. Kearns, M.J. and Vazirani, U.V. (1994), An Introduction to ComputatiooH Learning Theory, The MIT Press, Cambridge, MA.
44. Kim, I. and Vachtsevanos, 0. (1998), "Overlapping object recognition .1 paradigm for multiple sensor fusion," IEEE Robotics and Automation Magazoi pp. 37-44, September.
45. Klir, G.J. and Folger, TA. (3988), Fuzzy Sets, Uncertainty, and Information, PrenticeHall International Editions, London, UK.
46. Klir, G.J. and yuan, B. (1997), Fuzzy Sets, and Fuzzy Logic, Prentice-Hall of India, New Delhi, India.
47. Kohonen, T. (1995), Self-Organizing Maps, Springer-Verlag, Berlin Heidelberg
48. Kosko, B. (1998), "Global stability of generalized additive fuzzy systems." IEEE Trans. Systems, Man, and Cybernetics - C, vol. 28, no. 3, pp. 441-452.
49. Kuo, Y-H., Hsu, J-P., and Wang, C-W. (1998), "A parallel fuzzy inference model with distributed prediction scheme for reinforcement learning, IEEE Trans. Systems, Man, and Cybernetics - B, vol. 28, no. 2, pp. 160-172.
50. Lin, C-T., Lin, C-J., and Lee, C.S.G. (1995), 'Fuzzy adaptive learning control network with on-line neural learning," Fuzzy Sets and Systems, vol. 7!, pp. 2545.
51. Lucas, C. and Araabi, B,N. (1999), "Generalization of the Dempster-Shafer theory: A fuzzy-valued measure," IEEE Trans. Fuzzy Systems, vol. 7, no. 3, pp. 255-270.
52. Luo, R.C. and Kay, MG. (1989), "Multisensor integration and fusion in intelligent systems," IEEE Trans. Systems, Man, and Cybernetics, vol. 19, no. 5, pp. 901-93 I. -
53. Mamdani, E.H. and Assilian, S. (1975), "An experiment in linguistic synthesis with a fuzzy logic controller," Intl. I of Man-Machine Studies, vol. 7, pp. 1-13.
54. Martha, Ky., Kent, J.T., and Bibby, J.M. (1979), Multivariate Analysis, Academic Press, London, UK.
55. Margaris, A. (1990), First Order Mathematical Logic, Dover Publications, Inc., New York, NY.
56. McLachlan, G.J. (1992), Discriminant Analysis and Statistical Pattern Recognition, John Wiley & Sons, New York, NY.
57. Merz, C.J. and Murphy, P.M. (1996), UCI Repository of Machine Learning Databases, Dept. Inform. Comput. Sci., University of California, Irvine.
58. Murphy, R.R. (1998), "Dempster-Shafer theory for sensor fusion in autonomous mobile robots," IEEE Trans. Robotics and Automation, vol. 14, no. 2, pp. 97206.
59. Nguyen, H.T. and Walker, L.A. (1997), Fuzzy Logic, CRC Press Inc., New York, NY.
60. Papoulis, A. (1991), Probability, Random Variables, and Stochastic Processes, third edition, McGraw-Hill, Inc., New York, NY.
61. Park, JS., Chen, M-S., and Yu, P.S. (1997), "Using a hash-based method with transaction trimming for mining association rules," IEEE Trans. Knowledge and Data Engring, vol.9, no. 5, pp. 813-825.
62. Petridis, V., Kaburlasos, V.0., Brett, P., Parker, T., and Day, i.C.C. (1996), "Two level fuzzy lattice (2L-FL) supervised clustering: A new method for soft tissue identification in surgery," Proceedings of the CESA 96 IMACS Multiconference, Lille, France, pp. 232-237.
63. Petridis, V. and Kaburlasos, V.0. (1998), "Fuzzy lattice neural network (FLNN) A hybrid model for learning," IEEE Trans. Neural Networks, vol. 9, no. 5, pp. 877-890.
64. Petridis, V. and Kaburlasos, V.0. (1999), "Learning in the framework of fuzzy lattices," IEEE Trans. Fuzzy Systems, vol. 7, no. 4, pp. 422-440.
65. Pontil, M. and Verri, A. (1998), "Support vector machines for 3D object recognition," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 20, no. 6, pp. 637-646.
66. Pindyck, R.S. and Rubinfeld, DL. (1991), Econometric Models & ononik Forecasts, McGraw-Hill, Inc., New York, NY.
67. Rutherford, D.E. (1965), Introduction to Lattice Theory, Oliver and Boyd Ltd., Edinburgh, Great Britain.
68. Saridis, G.N. (1983), 'Intelligent robotic control," IEEE Trans. Automatic Control, vol. 28, no. 5, pp. 547-557.
69. Sarkar, S., Chakrabarti, PP., and Ghose, 5. (1998), "A framework for learning in searchbased systems," IEEE Trans. Knowledge and Data Engineering, vol. 10, no. 4, pp. 563-575.
70. Seines, M., Babuka, R., Kayrnak, U., and van Nauta Lemke, HR. (1998), "Similarity measures in fuzzy rule base simplification," IEEE Trans. Systems, Man, and Cybernetics - B, vol. 28, no. 3, pp. 376-386.
71. Shafer, G. (1976), A Mathematical Theory of Evidence, Princeton University Press.
72. Simpson, P.K. (1992), "Fuzzy mm-max neural networks - partl: classification", IEEE Trans. Neural Networks, vol. 3, no, 5, pp. 776-786.
73. Simpson, P.K. (1993), "Fuzzy mm-max neural networks - part2: clustering," IEEE Trans. Fuzzy Systems, vol. I, no. 1, pp. 32-45.
74. Smets, P. (1990), "The combination of evidence in the transferable belief model," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 12, no. 5, pp. 447-458.
75. Tan, A-H. (1995), "Adaptive resonance associative map," Neural Networks, vol. 8, no. 3, pp. 437-446.
76. Valiant, L.G. (1984), "A theory of the learnable," Communications of the ACM, vol. 27, no. 11. pp. 1134-1142.
77. Vapnik, V.N. and Chervonenkis, Y.A. (1971), "On the uniform convergence of relative frequencies of events to their probabilities," Theory of Probability and its Applications, vol. 16, no. 2, pp. 264-280.
78. Vidyasagar, M. (1997), A Theory of Learning and Generalization, SpringerVerlag, London, England.
79. Waugh, S. (1995), "Extending and benchmarking Cascade-Correlation," Ph.D. Thesis, Dept. Computer Science, University of Tasmania.
80. Williamson, JR. (1996), "Gaussian ARTMAP: A neural network for fast incremental learning of noisy multidimensional maps," Neural Networks, vol. 9, no. 5, pp. 881-897.
81. Xu, H., Hsia, Y-T., and Smets, P. (1996), "Transferable belief model for decision making in the valuation-based systems," IEEE Trans. Systems, Man, and Cybernetics - A, vol. 26, no. 6, pp. 698-707.
82. Yager, R.R. (1996), "On the aggregation of prioritized belief structured belief structures," IEEE Trans. Systems, Man, and Cybernetics - A, vol. 26. no. 6, pp. 708-717.
83. Zadeh, L.A. (1965), "Fuzzy sets," Information and Control, vol. 8, pp. 338-353.
84. Zadeh, L.A. (1984), "Review of books: A mathematical theory of evidence," TheAlMagazine, pp. 81-83.
85. Zadeh, L.A. (1996), "Fuzzy logic = Computing with words," IEEE Trans. Fuzzy Systems, vol. 4, no. 2, pp. 103-111.
86. Zeng, X-J. and Singh, MG. (1997), Fuzzy bounded least-squares method for the identification of linear systems," IEEE Trans. Systems, Man, Cybernetics - A, vol. 27, no. 5, pp. 624-635.
87. Ziminermann, H.J. (1991), Fuzzy Set Theory and Its Applications, Kluwcr Academic Publishers, Norwell, MA.
88. Zouhal, L.M, and Denoeux, T. (1998), "An evidence-theoretic k-NN Rule with parameter optimization," IEEE Trans. Systems, Man, and Cybernetics . C, vol. 28, no. 2, pp. 263-271.
89. Zurada, J.M., Marks II, R.J., and Robinson, C.J. (Eds.) (1994), Computational Intelligence - Imitating Life, IEEE Press, New York, NY.

Index Terms

• Computing methodologies

  • Artificial intelligence

    • Knowledge representation and reasoning

      • Probabilistic reasoning

      • Vagueness and fuzzy logic

  • Machine learning

• Theory of computation

  • Logic

  • Models of computation

    • Computability