Two classes of algorithms for data clustering

ABSTRACT

The two classes of agglomerative hierarchical clustering algorithms and K-means algorithms are overviewed. Moreover recent topics of kernel functions and semi-supervised clustering in the two classes are discussed. This paper reviews traditional methods as well as new techniques.

References

1. Akaike, H.: A Bayesian Analysis of the Minimum AIC Procedure. Annals of the Institute of Statistical Mathematics 30(1), 9-14 (1978).
2. Anderberg, M.R.: Cluster Analysis for Applications. Academic Press, New York (1973).
3. Basu, S., Bilenko, M., Mooney, R.J.: A Probabilistic Framework for Semi-Supervised Clustering. In: Proc. of the Tenth ACM SIGKDD (KDD 2004), pp. 59-68 (2004).
4. Basu, S., Banerjee, A., Mooney, R.J.: Active Semi-Supervision for Pairwise Constrained Clustering. In: Proc. of the SIAM International Conference on Data Mining (SDM 2004), pp. 333-344 (2004).
5. Basu, S., Davidson, I., Wagstaff, K.L. (eds.): Constrained Clustering: Advances in Algorithms, Theory, and Applications. Chapman & Hall/CRC, Boca Raton (2009).
6. Bezdek, J.C.: Pattern Recognition with Fuzzy Objective Function Algorithms. Plenum Press (1981).
7. Bezdek, J.C., Keller, J., Krishnapuram, R., Pal, N.R.: Fuzzy Models and Algorithms for Pattern Recognition and Image Processing. Kluwer, Boston (1999).
8. Bouchachia, A., Pedrycz, W.: A Semi-supervised Clustering Algorithm for Data Exploration. In: De Baets, B., Kaynak, O., Bilgiç, T. (eds.) IFSA 2003. LNCS (LNAI), vol. 2715, pp. 328-337. Springer, Heidelberg (2003).
9. Chapelle, O., Schölkopf, B., Zien, A. (eds.): Semi-Supervised Learning. MIT Press, Cambridge (2006).
10. Davé, R.N., Krishnapuram, R.: Robust Clustering Methods: A Unified View. IEEE Trans. on Fuzzy Systems 5(2), 270-293 (1997).
11. Dempster, A.P., Laird, N.M., Rubin, D.B.: Maximum Likelihood from Incomplete Data via the EM Algorithm. J. R. Stat. Soc. B39, 1-38 (1977).
12. Dumitrescu, D., Lazzerini, B., Jain, L.C.: Fuzzy Sets and Their Application to Clustering and Training. CRC Press, Boca Raton (2000).
13. Duda, R.O., Hart, P.E.: Pattern Classification and Scene Analysis. John Wiley & Sons (1973).
14. Duda, R.O., Hart, P.E., Stork, D.G.: Pattern Classification, 2nd edn. Wiley, New York (2001).
15. Dunn, J.C.: A Fuzzy Relative of the ISODATA Process and Its Use in Detecting Compact Well-separated Clusters. J. of Cybernetics 3, 32-57 (1974).
16. Dunn, J.C.: Well-separated Clusters and Optimal Fuzzy Partitions. J. of Cybernetics 4, 95-104 (1974).
17. Endo, Y., Haruyama, H., Okubo, T.: On Some Hierarchical Clustering Algorithms Using Kernel Functions. In: Proc. of FUZZ-IEEE 2004, CD-ROM Proc., Budapest, Hungary, July 25-29, pp. 1-6 (2004).
18. Everitt, B.S.: Cluster Analysis, 3rd edn. Arnold, London (1993).
19. Girolami, M.: Mercer Kernel Based Clustering in Feature Space. IEEE Trans. on Neural Networks 13(3), 780-784 (2002).
20. Hashimoto, W., Nakamura, T., Miyamoto, S.: Comparison and Evaluation of Different Cluster Validity Measures Including Their Kernelization. Journal of Advanced Computational Intelligence and Intelligent Informatics 13(3), 204-209 (2009).
21. Hathaway, R.J., Bezdek, J.C.: Switching Regression Models and Fuzzy Clustering. IEEE Trans. on Fuzzy Systems 1, 195-204 (1993).
22. Höppner, F., Klawonn, F., Kruse, R., Runkler, T.: Fuzzy Cluster Analysis. Wiley, Chichester (1999).
23. Hwang, J., Miyamoto, S.: Kernel Functions Derived from Fuzzy Clustering and Their Application to Kernel Fuzzy c-Means. Journal of Advanced Computational Intelligence and Intelligent Informatics 15(1), 90-94 (2011).
24. Ichihashi, H., Honda, K., Tani, N.: Gaussian Mixture PDF Approximation and Fuzzy c-Means Clustering with Entropy Regularization. In: Proc. of Fourth Asian Fuzzy Systems Symposium, vol. 1, pp. 217-221 (2000).
25. Kaufman, L., Rousseeuw, P.J.: Finding Groups in Data: An Introduction to Cluster Analysis. Wiley, New York (1990).
26. Klein, D., Kamvar, S.D., Manning, C.: From Instance-level Constraints to Spacelevel Constraints: Making the Most of Prior Knowledge in Data Clustering. In: Proc. of the Intern. Conf. on Machine Learning, Sydney, Australia, pp. 307-314 (2002).
27. Kohonen, T.: Self-Organizing Maps, 2nd edn. Springer, Berlin (1997).
28. Krishnapuram, R., Keller, J.M.: A Possibilistic Approach to Clustering. IEEE Trans. on Fuzzy Systems 1, 98-110 (1993).
29. Kulis, B., Basu, S., Dhillon, I., Mooney, R.: Semi-supervised Graph Clustering: A Kernel Approach. Mach. Learn. 74, 1-22 (2009).
30. Li, R.P., Mukaidono, M.: A Maximum Entropy Approach to Fuzzy Clustering. In: Proc. of the 4th IEEE Intern. Conf. on Fuzzy Systems (FUZZ-IEEE/IFES 1995), Yokohama, Japan, March 20-24, pp. 2227-2232 (1995).
31. Li, R.P., Mukaidono, M.: Gaussian Clustering Method Based on Maximum-fuzzyentropy Interpretation. Fuzzy Sets and Systems 102, 253-258 (1999).
32. MacQueen, J.B.: Some Methods of Classification and Analysis of Multivariate Observations. In: Proc. of 5th Berkeley Symposium on Math. Stat. and Prob., pp. 281-297 (1967).
33. McLachlan, G., Peel, D.: Finite Mixture Models. Wiley, New York (2000)
34. Miyamoto, S.: Fuzzy Sets in Information Retrieval and Cluster Analysis. Kluwer, Dordrecht (1990).
35. Miyamoto, S., Mukaidono, M.: Fuzzy c-means as a Regularization and Maximum Entropy Approach. In: Proc. of the 7th International Fuzzy Systems Association World Congress (IFSA 1997), Prague, Czech, June 25-30, vol. II, pp. 86-92 (1997)
36. Miyamoto, S.: Introduction to Cluster Analysis, Morikita-Shuppan, Tokyo (1999) (in Japanese)
37. Miyamoto, S., Nakayama, Y.: Algorithms of Hard c-means Clustering Using Kernel Functions in Support Vector Machines. Journal of Advanced Computational Intelligence and Intelligent Informatics 7(1), 19-24 (2003)
38. Miyamoto, S., Suizu, D.: Fuzzy c-means Clustering Using Kernel Functions in Support Vector Machines. Journal of Advanced Computational Intelligence and Intelligent Informatics 7(1), 25-30 (2003)
39. Miyamoto, S., Suizu, D., Takata, O.: Methods of Fuzzy c-means and Possibilistic Clustering Using a Quadratic Term. Scientiae Mathematicae Japonicae 60(2), 217- 233 (2004)
40. Miyamoto, S., Ichihashi, H., Honda, K.: Algorithms for Fuzzy Clustering. Springer, Heidelberg (2008)
41. Miyamoto, S., Terami, A.: Semi-Supervised Agglomerative Hierarchical Clustering Algorithms with Pairwise Constraints. In: Proc. of WCCI 2010 IEEE World Congress on Computational Intelligence, CCIB, Barcelona, Spain, July 18-23, pp. 2796-2801 (2010)
42. Miyamoto, S., Terami, A.: Constrained Agglomerative Hierarchical Clustering Algorithms with Penalties. In: Proc. of 2011 IEEE International Conference on Fuzzy Systems, Taipei, Taiwan, June 27-30, pp. 422-427 (2011)
43. Redner, R.A., Walker, H.F.: Mixture Densities, Maximum Likelihood and the EM Algorithm. SIAM Review 26(2), 195-239 (1984)
44. Schölkopf, B., Smola, A.: Learning with Kernels. MIT Press (2002)
45. Schönberg, I.J.: Metric Spaces and Completely Monotone Functions. Annals of Mathematics 39(4), 811-841 (1938)
46. Shental, N., Bar-Hillel, A., Hertz, T., Weinshall, D.: Computing Gaussian Mixture Models with EM Using Equivalence Constraints. In: Advances in Neural Information Processing Systems, vol. 16 (2004)
47. Vapnik, V.N.: Statistical Learning Theory. Wiley, New York (1998)
48. Vapnik, V.N.: The Nature of Statistical Learning Theory, 2nd edn. Springer, New York (2000)
49. Vapnik, V.N.: Transductive Inference and Semi-supervised Learning. In: Chapelle, O., et al. (eds.) Semi-Supervised Learning, pp. 453-472. MIT Press, Cambridge (2006)
50. Wagstaff, K., Cardie, C., Rogers, S., Schroedl, S.: Constrained K-means Clustering with Background Knowledge. In: Proc. of the 9th ICML, pp. 577-584 (2001)
51. Wang, N., Li, X., Luo, X.: Semi-supervised Kernel-based Fuzzy c-Means with Pairwise Constraints. In: Proc. of WCCI 2008, pp. 1099-1103 (2008)
52. Zhu, X., Goldberg, A.B.: Introduction to Semi-Supervised Learning. Morgan and Claypool (2009).