Abstract
Social media platforms are turning into important news sources since they provide real-time information from different perspectives. However, high volume, dynamism, noise, and redundancy exhibited by social media data make it difficult to comprehend the entire content. Recent works emphasize on summarizing the content of either a single social media platform or of a single modality (either textual or visual). However, each platform has its own unique characteristics and user base, which brings to light different aspects of real-world events. This makes it critical as well as challenging to combine textual and visual data from different platforms. In this article, we propose summarization of real-world events with data stemming from different platforms and multiple modalities. We present the use of a Markov Random Fields based similarity measure to link content across multiple platforms. This measure also enables the linking of content across time, which is useful for tracking the evolution of long-running events. For the final content selection, summarization is modeled as a subset selection problem. To handle the complexity of the optimal subset selection, we propose the use of submodular objectives. Facets such as coverage, novelty, and significance are modeled as submodular objectives in a multimodal social media setting. We conduct a series of quantitative and qualitative experiments to illustrate the effectiveness of our approach compared to alternative methods.
- Majed AlRubaian, Muhammad Al-Qurishi, Mabrook Al-Rakhami, Sk Md Mizanur Rahman, and Atif Alamri. 2015. A multistage credibility analysis model for microblogs. In Proceedings of the IEEE/ACM International Conference on Advances in Social Network Analysis and Mining (ASONAM). ACM, New York, NY, 1434--1440. Google Scholar
Digital Library
- Matthew A. Baum and Yuri M. Zhukov. 2015. Filtering revolution: Reporting bias in international newspaper coverage of the Libyan civil war. Journal of Peace Research 52, 3 (2015), 384–400.Google Scholar
Cross Ref
- Jingwen Bian, Yang Yang, and Tat-Seng Chua. 2013. Multimedia summarization for trending topics in microblogs. In Proceedings of the International Conference on Information and Knowledge Management (CIKM). ACM, 1807--1812. Google Scholar
Digital Library
- Jingwen Bian, Yang Yang, Hanwang Zhang, and Tat-Seng Chua. 2015. Multimedia summarization for social events in microblog stream. IEEE Transactions on Multimedia 17, 2 (2015), 216--228.Google Scholar
Digital Library
- David M. Blei and Michael I. Jordan. 2003. Modeling annotated data. In Proceedings of the ACM SIGIR Conference on Research and Development in Information Retrieval. ACM, 127--134. Google Scholar
Digital Library
- David M. Blei, Andrew Y. Ng, and Michael I. Jordan. 2003. Latent dirichlet allocation. Journal of Machine Learning Research 3 (2003), 993--1022. Google Scholar
Digital Library
- Zhiyuan Chen and Bing Liu. 2014. Topic modeling using topics from many domains, lifelong learning and big data. In Proceedings of the International Conference on Machine Learning (ICML). Google Scholar
Digital Library
- Lingyang Chu, Yanyan Zhang, Guorong Li, Shuhui Wang, Weigang Zhang, and Qingming Huang. 2016. Effective multimodality fusion framework for cross-media topic detection. IEEE Transactions on Circuits and Systems for Video Technology 26, 3 (2016), 556--569. Google Scholar
Digital Library
- Charles L. A. Clarke, Maheedhar Kolla, Gordon V. Cormack, Olga Vechtomova, Azin Ashkan, Stefan Büttcher, and Ian MacKinnon. 2008. Novelty and diversity in information retrieval evaluation. In Proceedings of the ACM SIGIR Conference on Research and Development in Information Retrieval. ACM, New York, NY, 659--666. Google Scholar
Digital Library
- Bin Cui, Anthony K. H. Tung, Ce Zhang, and Zhe Zhao. 2010. Multiple feature fusion for social media applications. In Proceedings of the ACM SIGMOD Conference of the Special Interest Group on Management of Data. ACM, 435--446. Google Scholar
Digital Library
- Peng Cui, Wenwu Zhu, Tat-Seng Chua, and Ramesh Jain. 2016. Social-sensed multimedia computing. IEEE MultiMedia 23, 1 (2016), 92--96. Google Scholar
Digital Library
- Manfred Del Fabro, Anita Sobe, and Laszlo Böszörmenyi. 2012. Summarization of real-life events based on community-contributed content. In Proceedings of the 4th International Conferences on Advances in Multimedia, MMEDIA. Google Scholar
Digital Library
- J. L. Fleiss and others. 1971. Measuring nominal scale agreement among many raters. Psychological Bulletin 76, 5 (1971), 378--382.Google Scholar
Cross Ref
- Michael Gygli, Helmut Grabner, and Luc Van Gool. 2015. Video summarization by learning submodular mixtures of objectives. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 3090--3098.Google Scholar
Cross Ref
- Aria Haghighi and Lucy Vanderwende. 2009. Exploring content models for multi-document summarization. In Proceedings of Human Language Technologies: The 2009 Conference of the North American Chapter of the Association for Computational Linguistics. Association for Computational Linguistics, 362--370. Google Scholar
Digital Library
- Kalervo Järvelin and Jaana Kekäläinen. 2002. Cumulated gain-based evaluation of IR techniques. ACM Transactions on Information Systems 20, 4 (Oct. 2002), 422--446. Google Scholar
Digital Library
- Gunhee Kim, Seungwhan Moon, and Leonid Sigal. 2015. Joint photo stream and blog post summarization and exploration. In Proceedings of the 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, 3081--3089.Google Scholar
Cross Ref
- Ross Kindermann and Laurie Shell. 1980. Markov random fields and their application. American Mathematical Society, volume 1.Google Scholar
Cross Ref
- Katrin Kirchhoff and Jeff Bilmes. 2014. Submodularity for data selection in machine translation. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP). Association for Computational Linguistics, Doha, Qatar, 131--141.Google Scholar
Cross Ref
- Jure Leskovec, Andreas Krause, Carlos Guestrin, Christos Faloutsos, Jeanne VanBriesen, and Natalie Glance. 2007. Cost-effective outbreak detection in networks. In ACM SIGKDD’07. ACM, 420--429. Google Scholar
Digital Library
- Bang Hui Lim, Dongyuan Lu, Tao Chen, and Min-Yen Kan. 2015. # mytweet via instagram: Exploring user behaviour across multiple social networks. In Proceedings of the IEEE/ACM International Conference on Advances in Social Network Analysis and Mining (ASONAM). ACM, 113--120. Google Scholar
Digital Library
- Chin-Yew Lin. 2004. Rouge: A package for automatic evaluation of summaries. Text Summarization Branches Out: Proceedings of the ACL-04 Workshop 8 (2004).Google Scholar
- Hui Lin and Jeff Bilmes. 2011. A class of submodular functions for document summarization. In Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies-Volume 1. Association for Computational Linguistics, 510--520. Google Scholar
Digital Library
- X. Lu, Z. Lin, H. Jin, J. Yang, and J. Z. Wang. 2015. Rating image aesthetics using deep learning. IEEE Transactions on Multimedia 17, 11 (Nov. 2015), 2021--2034.Google Scholar
Cross Ref
- Philip J. McParlane, Andrew James McMinn, and Joemon M. Jose. 2014. “Picture the scene...”; Visually summarising social media events. In Proceedings of the International Conference on Information and Knowledge Management (CIKM). ACM, 1459--1468. Google Scholar
Digital Library
- Donald Metzler and W. Bruce Croft. 2005. A Markov random field model for term dependencies. In Proceedings of the ACM SIGIR Conference on Research and Development in Information Retrieval. ACM, 472--479. Google Scholar
Digital Library
- Michel Minoux. 1978. Accelerated greedy algorithms for maximizing submodular set functions. In Optimization Techniques. Springer, 234--243.Google Scholar
- Baharan Mirzasoleiman, Amin Karbasi, Rik Sarkar, and Andreas Krause. 2013. Distributed submodular maximization: Identifying representative elements in massive data. In Advances in Neural Information Processing Systems. 2049--2057. Google Scholar
Digital Library
- George L. Nemhauser, Laurence A. Wolsey, and Marshall L. Fisher. 1978. An analysis of approximations for maximizing submodular set functions I. Mathematical Programming 14, 1 (1978), 265--294. Google Scholar
Digital Library
- Liqiang Nie, Xuemeng Song, and Tat-Seng Chua. 2016. Learning from multiple social networks. Synthesis Lectures on Information Concepts, Retrieval, and Services 8, 2 (2016), 1--118.Google Scholar
Cross Ref
- Liqiang Nie, Luming Zhang, Yi Yang, Meng Wang, Richang Hong, and Tat-Seng Chua. 2015. Beyond doctors: Future health prediction from multimedia and multimodal observations. In Proceedings of the 23rd ACM International Conference on Multimedia. ACM, 591--600. Google Scholar
Digital Library
- Brendan O’Connor, Michel Krieger, and David Ahn. 2010. TweetMotif: Exploratory search and topic summarization for Twitter. In Proceedings of the International Conference on Weblogs and Social Media (ICWSM).Google Scholar
- Shengsheng Qian, Tianzhu Zhang, Richang Hong, and Changsheng Xu. 2015a. Cross-domain collaborative learning in social multimedia. In Proceedings of the 23rd ACM International Conference on Multimedia. ACM, 99--108. Google Scholar
Digital Library
- Shengsheng Qian, Tianzhu Zhang, and Changsheng Xu. 2016. Multi-modal multi-view topic-opinion mining for social event analysis. In Proceedings of the 2016 ACM on Multimedia Conference. ACM, 2--11. Google Scholar
Digital Library
- Shengsheng Qian, Tianzhu Zhang, Changsheng Xu, and M. Shamim Hossain. 2015b. Social event classification via boosted multimodal supervised latent dirichlet allocation. ACM Transactions on Multimedia Computing, Communications, and Applications (TOMM) 11, 2 (2015), 27. Google Scholar
Digital Library
- Manos Schinas, Symeon Papadopoulos, Yiannis Kompatsiaris, and Pericles A. Mitkas. 2015. Visual event summarization on social media using topic modelling and graph-based ranking algorithms. In Proceedings of the 5th ACM on International Conference on Multimedia Retrieval. ACM, 203--210. Google Scholar
Digital Library
- Rajiv Ratn Shah, Anwar Dilawar Shaikh, Yi Yu, Wenjing Geng, Roger Zimmermann, and Gangshan Wu. 2015. EventBuilder: Real-time multimedia event summarization by visualizing social media. In ACM Multimedia. ACM, 185--188. Google Scholar
Digital Library
- Beaux Sharifi, Mark-Anthony Hutton, and Jugal Kalita. 2010. Summarizing microblogs automatically. In Human Language Technologies: The 2010 Annual Conference of the North American Chapter of the Association for Computational Linguistics. Association for Computational Linguistics, 685--688. Google Scholar
Digital Library
- Pinaki Sinha, Sharad Mehrotra, and Ramesh Jain. 2011. Summarization of personal photologs using multidimensional content and context. In Proceedings of the ACM International Conference on Multimedia Retrieval. ACM. Google Scholar
Digital Library
- Ruben Sipos, Adith Swaminathan, Pannaga Shivaswamy, and Thorsten Joachims. 2012. Temporal corpus summarization using submodular word coverage. In Proceedings of the International Conference on Information and Knowledge Management (CIKM). ACM, 754--763. Google Scholar
Digital Library
- Xuemeng Song, Liqiang Nie, Luming Zhang, Mohammad Akbari, and Tat-Seng Chua. 2015. Multiple social network learning and its application in volunteerism tendency prediction. In Proceedings of the ACM SIGIR Conference on Research and Development in Information Retrieval. ACM, 213--222. Google Scholar
Digital Library
- X. Tang, W. Luo, and X. Wang. 2013. Content-based photo quality assessment. IEEE Transactions on Multimedia 15, 8 (Dec. 2013), 1930--1943. Google Scholar
Digital Library
- Sebastian Tschiatschek, Rishabh K. Iyer, Haochen Wei, and Jeff A. Bilmes. 2014. Learning mixtures of submodular functions for image collection summarization. In Advances in Neural Information Processing Systems. 1413--1421. Google Scholar
Digital Library
- Zhiyu Wang, Peng Cui, Lexing Xie, Hao Chen, Wenwu Zhu, and Shiqiang Yang. 2012. Analyzing social media via event facets. In Proceedings of the 20th ACM International Conference on Multimedia. ACM, 1359--1360. Google Scholar
Digital Library
- Zhiyu Wang, Peng Cui, Lexing Xie, Wenwu Zhu, Yong Rui, and Shiqiang Yang. 2014. Bilateral correspondence model for words-and-pictures association in multimedia-rich microblogs. ACM Transactions on Multimedia Computing, Communications, and Applications (TOMM) 10, 4 (2014), 34. Google Scholar
Digital Library
- Kai Wei, Yuzong Liu, Katrin Kirchhoff, and Jeff A Bilmes. 2013. Using document summarization techniques for speech data subset selection. In Proceedings of the Conference North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL HLT). 721--726.Google Scholar
- Ming Yan, Jitao Sang, Changsheng Xu, and M. Shamim Hossain. 2016. A unified video recommendation by cross-network user modeling. ACM Transactions on Multimedia Computing, Communications, and Applications (TOMM) 12, 4 (2016), 53. Google Scholar
Digital Library
- M. Zaharieva, M. D. Fabro, and M. Zeppelzauer. 2015. Cross-platform social event detection. IEEE MultiMedia 22, 3 (July 2015), 14--25.Google Scholar
Cross Ref
- Tianzhu Zhang, Shengsheng Qian, Changsheng Xu, and Jie Shao. 2016. Multi-modal event topic model for social event analysis. IEEE Transactions on Multimedia 18, 2 (2016), 233--246.Google Scholar
Cross Ref
- Tianzhu Zhang and Changsheng Xu. 2014. Cross-domain multi-event tracking via CO-PMHT. ACM Transactions on Multimedia Computing, Communications, and Applications (TOMM) 10, 4 (2014), 31. Google Scholar
Digital Library
Index Terms
Multimodal Multiplatform Social Media Event Summarization
Recommendations
On Analyzing User Topic-Specific Platform Preferences Across Multiple Social Media Sites
WWW '17: Proceedings of the 26th International Conference on World Wide WebTopic modeling has traditionally been studied for single text collections and applied to social media data represented in the form of text documents. With the emergence of many social media platforms, users find themselves using different social media ...
Social Media Text Summarisation Techniques and Approaches: A Literature Review
ACAI '22: Proceedings of the 2022 5th International Conference on Algorithms, Computing and Artificial IntelligenceThe great explosion in the abundance of information in the Social Media sector urgently demands proficient text summarization so that people are amply offered concise knowledge avoiding time-consuming procedures. Various text summarization techniques ...






Comments