Abstract
Sporting events generate a massive amount of traffic on social media with live moment-to-moment accounts as any given situation unfolds. The generated data are intensified by fans feelings, reactions, and subjective opinions towards what happens during the event, all of which are based on their individual points of view. Analyzing and summarizing this data will generate a comprehensive overview of the event in terms of how the event evolves and how fans react and view the event based on their perspectives. Previously, most of the summarization works ignore fan reactions and subjective opinions, and focus primarily on generating an objective-view summary. We believe that an effective and useful summary should consider human reactions, sentiment, and point of view, as opposed to simply describing what happens during the event. Accordingly, in this work, we propose MMSUM Digital Twins: a summarization framework that is capable of generating a multi-view multi-modal summary for sporting events in real-time. The proposed digital twins-based framework consists of four main components: sub-event recognition which detects the event’s key moments, tweet categorization, which determines which team the tweets’ writers support and assigns tweets to their teams, sentiment analysis to track fans’ state of mind, and image popularity prediction for selecting representative images. Furthermore, the MMSUM employs a visual-filtering model to address the issue of noisy images that inundate social media, compromising the summarization quality. We leverage the knowledge of sport fans to evaluate the generated multi-view summarization through an online user study. The experiment results confirm the effectiveness of our proposed approach for summarizing sporting events by considering multimedia data, sentiment, and subjective views of the event.
- [1] . 2018. Sentiment identification in football-specific tweets. IEEE Access 6 (2018), 78609–78621.Google Scholar
Cross Ref
- [2] . 2017. On the prediction of Flickr image popularity by analyzing heterogeneous social sensory data. Sensors 17, 3 (2017), 631.Google Scholar
- [3] . 2015. Multimedia summarization for social events in microblog stream. IEEE Transactions on Multimedia 17, 2 (
Feb 2015), 216–228. https://doi.org/10.1109/TMM.2014.2384912Google ScholarDigital Library
- [4] . 2019. Large-scale social multimedia analysis. In Big Data Analytics for Large-Scale Multimedia Search. Wiley, 157–178.Google Scholar
Cross Ref
- [5] . 2013. Large-scale visual sentiment ontology and detectors using adjective noun pairs. In Proceedings of the 21st ACM International Conference on Multimedia. 223–232. Google Scholar
Digital Library
- [6] . 2011. Event summarization using tweets. In Proceedings of the 5th International AAAI Conference on Weblogs and Social Media.Google Scholar
- [7] . 2013. Automatic summarization of events from social media. In Proceedings of the 7th International AAAI Conference on Weblogs and Social Media.Google Scholar
- [8] . 2014. Two sides to every story: Subjective event summarization of sports events using Twitter. In ICMR 2014, 1st International Workshop on Social Multimedia and Storytelling (SoMuS 2014), CEUR Proceedings.Google Scholar
- [9] . 2018. Digital twins: The convergence of multimedia technologies. IEEE MultiMedia 25, 2 (2018), 87–92.Google Scholar
Digital Library
- [10] . 2019. Dtwins: A digital twins ecosystem for health and well-being. MMTC Communications-Frontiers 14, 2 (2019), 39–46.Google Scholar
- [11] . 2015. GOAALLL!: Using sentiment in the World Cup to explore theories of emotion. In Proceedings of the 2015 International Conference on Affective Computing and Intelligent Interaction (ACII). 898–903. https://doi.org/10.1109/ACII.2015.7344681 Google Scholar
Digital Library
- [12] . 2017. CrowdStory: Fine-grained event storyline generation by fusion of multi-modal crowdsourced data. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 1, 3, Article
55 (Sept. 2017), 19 pages. https://doi.org/10.1145/3130920 Google ScholarDigital Library
- [13] . 2012. Live semantic sport highlight detection based on analyzing tweets of Twitter. In Proceedings of the 2012 IEEE International Conference on Multimedia and Expo. 949–954. https://doi.org/10.1109/ICME.2012.135 Google Scholar
Digital Library
- [14] . 2011. Comparing Twitter summarization algorithms for multiple post summaries. In 2011 IEEE Third International Conference on Privacy, Security, Risk and Trust and 2011 IEEE Third International Conference on Social Computing. 298–306. https://doi.org/10.1109/PASSAT/SocialCom.2011.31Google Scholar
Cross Ref
- [15] . 2014. Caffe: Convolutional architecture for fast feature embedding. In Proceedings of the 22nd ACM International Conference on Multimedia (MM’14). 675–678. https://doi.org/10.1145/2647868.2654889 Google Scholar
Digital Library
- [16] . 2012. ImageNet classification with deep convolutional neural networks. In Advances in Neural Information Processing Systems. 1097–1105. Google Scholar
Digital Library
- [17] . 2013. Generating live sports updates from Twitter by finding good reporters. In Proceedings of the 2013 IEEE/WIC/ACM International Joint Conferences on Web Intelligence (WI) and Intelligent Agent Technologies (IAT), Vol. 1. 527–534. https://doi.org/10.1109/WI-IAT.2013.74 Google Scholar
Digital Library
- [18] . 2017. NBA memes: The role of fan image macros within the online NBA fan community. Ph.D. Dissertation. Queensland University of Technology.Google Scholar
- [19] . 2002. Classifying images on the web automatically. Journal of Electronic Imaging 11, 4 (2002), 445–454. https://doi.org/10.1117/1.1502259Google Scholar
Cross Ref
- [20] . 2004. ROUGE: A package for automatic evaluation of summaries. In Text Summarization Branches Out. 74–81. https://www.aclweb.org/anthology/W04-1013Google Scholar
- [21] . 2018. Ranking news-quality multimedia. In Proceedings of the 2018 ACM on International Conference on Multimedia Retrieval (Yokohama, Japan) (
ICMR’18 ). 10–18. https://doi.org/10.1145/3206025.3206053 Google ScholarDigital Library
- [22] . 2011. Twitinfo: Aggregating and visualizing microblogs for event exploration. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (Vancouver, BC, Canada) (
CHI’11 ). 227–236. https://doi.org/10.1145/1978942.1978975 Google ScholarDigital Library
- [23] . 2014. ‘Picture the Scene...’: Visually summarising social media events. In Proceedings of the 23rd ACM International Conference on Information and Knowledge Management (Shanghai, China) (
CIKM’14 ). 1459–1468. https://doi.org/10.1145/2661829.2661923 Google ScholarDigital Library
- [24] . 2015. Degeneracy-based real-time sub-event detection in twitter stream. In Proceedings of the 9th International AAAI Conference on Web and Social Media.Google Scholar
- [25] . 2018. An optimization approach for sub-event detection and summarization in twitter. In Advances in Information Retrieval. 481–493.Google Scholar
- [26] . 2012. Summarizing sporting events using twitter. In Proceedings of the 2012 ACM International Conference on Intelligent User Interfaces (Lisbon, Portugal) (
IUI’12 ). 189–198. https://doi.org/10.1145/2166966.2166999 Google ScholarDigital Library
- [27] . 2010. Breaking news detection and tracking in twitter. In Proceedings of the 2010 IEEE/WIC/ACM International Conference on Web Intelligence and Intelligent Agent Technology, Vol. 3. 120–123. https://doi.org/10.1109/WI-IAT.2010.205 Google Scholar
Digital Library
- [28] . 2018. Social media based event summarization by user-text-image co-clustering. Knowledge-Based Systems (2018). https://doi.org/10.1016/j.knosys.2018.10.028Google Scholar
- [29] . 2016. MGraph: Multimodal event summarization in social media using topic models and graph-based ranking. International Journal of Multimedia Information Retrieval 5, 1 (
01 Mar 2016), 51–69. https://doi.org/10.1007/s13735-015-0089-9Google ScholarCross Ref
- [30] . 2010. Summarizing microblogs automatically. In Human Language Technologies: The 2010 Annual Conference of the North American Chapter of the Association for Computational Linguistics. 685–688. Google Scholar
Digital Library
- [31] . 2018. A survey of content-aware video analysis for sports. IEEE Transactions on Circuits and Systems for Video Technology 28, 5 (2018), 1212–1231. https://doi.org/10.1109/TCSVT.2017.2655624Google Scholar
Cross Ref
- [32] . 2018. Sports Game Summarization Based on Sub-events and Game-Changing Phrases. Springer International Publishing, 65–80. https://doi.org/10.1007/978-3-319-70636-8_5Google Scholar
- [33] . 2012. # EpicPlay: Crowd-sourcing sports video highlights. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. 1569–1572. Google Scholar
Digital Library
- [34] . 2017. PASS: A Dutch data-to-text system for soccer, targeted towards specific audiences. In Proceedings of the 10th International Conference on Natural Language Generation. 95–104. https://doi.org/10.18653/v1/W17-3513Google Scholar
- [35] . 2012. Automatic extraction of soccer game events from twitter. In Proceedings of Detection, Representation and Exploitation of Events in the Semantic Web (DeRiVE 2012).Google Scholar
- [36] . 2006. NPIC: Hierarchical synthetic image classification using image search and generic features. In Proceedings of the 5th International Conference on Image and Video Retrieval. 473–482. Google Scholar
Digital Library
- [37] . 2014. Multi-view video summarization algorithm for WMSN. In 2014 International Conference on Wireless Communication and Sensor Network. IEEE, 213–216. Google Scholar
Digital Library
- [38] . 2012. Towards real-time summarization of scheduled events from twitter streams. In Proceedings of the 23rd ACM Conference on Hypertext and Social Media (HT’12). 319–320. https://doi.org/10.1145/2309996.2310053 Google Scholar
Digital Library
Index Terms
MMSUM Digital Twins: A Multi-view Multi-modality Summarization Framework for Sporting Events
Recommendations
Brand-Related Events Detection, Classification and Summarization on Twitter
WI-IAT '12: Proceedings of the The 2012 IEEE/WIC/ACM International Joint Conferences on Web Intelligence and Intelligent Agent Technology - Volume 01The huge and ever increasing amount of text generated by Twitter users everyday embeds a wealth of information, in particular, about themes that become suddenly relevant to many users as well as about the sentiment polarity that users tend to associate ...
Multi-modal Multi-view Topic-opinion Mining for Social Event Analysis
MM '16: Proceedings of the 24th ACM international conference on MultimediaIn this paper, we propose a novel multi-modal multi-view topic-opinion mining (MMTOM) model for social event analysis in multiple collection sources. Compared with existing topic-opinion mining methods, our proposed model has several advantages: (1) The ...
Using topic themes for multi-document summarization
The problem of using topic representations for multidocument summarization (MDS) has received considerable attention recently. Several topic representations have been employed for producing informative and coherent summaries. In this article, we ...






Comments