Splog detection using self-similarity analysis on blog temporal dynamics

ABSTRACT

This paper focuses on spam blog (splog) detection. Blogs are highly popular, new media social communication mechanisms. The presence of splogs degrades blog search results as well as wastes network resources. In our approach we exploit unique blog temporal dynamics to detect splogs.

There are three key ideas in our splog detection framework. We first represent the blog temporal dynamics using self-similarity matrices defined on the histogram intersection similarity measure of the time, content, and link attributes of posts. Second, we show via a novel visualization that the blog temporal characteristics reveal attribute correlation, depending on type of the blog (normal blogs and splogs). Third, we propose the use of temporal structural properties computed from self-similarity matrices across different attributes. In a splog detector, these novel features are combined with content based features. We extract a content based feature vector from different parts of the blog -- URLs, post content, etc. The dimensionality of the feature vector is reduced by Fisher linear discriminant analysis. We have tested an SVM based splog detector using proposed features on real world datasets, with excellent results (90% accuracy).

References

1. Wikipedia, Spam blog http://en.wikipedia.org/wiki/Splog.
2. C.-C. Chang and C.-J. Lin (2001). LIBSVM: a library for support vector machines.
3. J. Eckmann, S. O. Kamphorst and D. Ruelle (1987). Recurrence plots of dynamical systems. Europhysics Letters(4): 973--977.
4. J. Foote, M. Cooper and U. Nam (2002). Audio retrieval by rhythmic similarity, Proceedings of the International Conference on Music Information Retrieval, 265--266.
5. Z. Gyöngyi, H. Garcia-Molina and J. Pedersen (2004). Combating web spam with TrustRank, Proceedings of the 30th International Conference on Very Large Data Bases (VLDB) 2004, Toronto, Canada,
6. Z. Gyöngyi, P. Berkhin, Hector Garcia-Molina and J. Pedersen (2006). Link Spam Detection Based on Mass Estimation, 32nd International Conference on Very Large Data Bases (VLDB), Seoul, Korea,
7. P. Kolari (2005) Welcome to the Splogosphere: 75% of new pings are spings (splogs) permalink: http://ebiquity.umbc.edu/blogger/2005/12/15/welcome-to-the-splogosphere-75-of-new-blog-posts-are-spam/.
8. P. Kolari, T. Finin and A. Joshi (2006). SVMs for the blogosphere: Blog identification and splog detection, AAAI Spring Symposium on Computational Approaches to Analyzing Weblogs,
9. P. Kolari, A. Java and T. Finin (2006). Characterizing the Splogosphere, Proceedings of the 3rd Annual Workshop on Weblogging Ecosystem: Aggregation, Analysis and Dynamics, 15th World Wid Web Conference,
10. P. Kolari, A. Java, T. Finin, T. Oates and A. Joshi (2006). Detecting Spam Blogs: A Machine Learning Approach, Proceedings of the 21st National Conference on Artificial Intelligence (AAAI 2006), Boston, MA, July 2006.
11. A. Ntoulas, M. Najork, M. Manasse and D. Fetterly (2006). Detecting spam web pages through content analysis, Proceedings of the 15th international conference on World Wide Web, Edinburgh, Scotland, May 2006.
12. G. Shen, B. Gao, T.-Y. Liu, G. Feng, S. Song and H. Li (2006). Detecting Link Spam using Temporal Information, Proc. of ICDM-2006, to appear, 2006,
13. Umbria (2006) Spam in the blogosphere http://www.umbrialistens.com/files/uploads/umbria_splog.pdf.

Index Terms

• Applied computing

  • Document management and text processing

    • Document preparation

      • Multi / mixed media creation

• Human-centered computing

  • Collaborative and social computing

    • Collaborative and social computing systems and tools

  • Human computer interaction (HCI)

    • Interaction paradigms

      • Hypertext / hypermedia

• Information systems

  • Information retrieval

  • World Wide Web