Abstract
Due to various applications, research on personal traits using information on social media has become an important area. In this paper, a new method for the classification of behavior-oriented social images uploaded on various social media platforms is presented. The proposed method introduces a multimodality concept using skin of different parts of human body and background information, such as indoor and outdoor environments. For each image, the proposed method detects skin candidate components based on R, G, B color spaces and entropy features. The iterative mutual nearest neighbor approach is proposed to detect accurate skin candidate components, which result in foreground components. Next, the proposed method detects the remaining part (other than skin components) as background components based on structure tensor of R, G, B color spaces, and Maximally Stable Extremal Regions (MSER) concept in the wavelet domain. We then explore Hanman Transform for extracting context features from foreground and background components through clustering and fusion operation. These features are then fed to an SVM classifier for the classification of behavior-oriented images. Comprehensive experiments on 10-class datasets of Normal Behavior-Oriented Social media Image (NBSI) and Abnormal Behavior-Oriented Social media Image (ABSI) show that the proposed method is effective and outperforms the existing methods in terms of average classification rate. Also, the results on the benchmark dataset of five classes of personality traits and two classes of emotions of different facial expressions (FERPlus dataset) demonstrated the robustness of the proposed method over the existing methods.
- [1] . 2015. Connection discovery using big data of user shared images in social media. IEEE Trans, MM 17, (2015), 1417–1428.Google Scholar
Digital Library
- [2] . 2014. What we Instagram: A first analysis of Instagram photo content and user types. In Proc. AAAI, 595–598.Google Scholar
- [3] . 2016. Analyzing personality through social media profile picture choice. In Proc. ICWSM, 2016.Google Scholar
- [4] . 2018. Multimodal multiplatform social media event summarization. ACM. TOMM 14, 38 (2018).Google Scholar
- [5] . 2017. Sentiment analysis for social media images. In Proc. ICDAMW, 1584–1591.Google Scholar
- [6] . 2017. Face search at scale. IEEE Trans. PAMI 39, 1122–1136, 2017.Google Scholar
Digital Library
- [7] . 2015. Natural language processing for social media. Synthesis Lectures on Human Language Techniques, Morgan and Claypool Publishers, 2015.Google Scholar
Cross Ref
- [8] . 2019. Multi-level similarity perception network for person re-identification. ACM. TOMM 15, 32 (2019).Google Scholar
- [9] . 2015. Demographic estimation from face images: Human vs. machine performance. IEEE Trans. PAMI 37, (2015), 1148–1161.Google Scholar
Digital Library
- [10] . 2018. Abnormal behavior recognition for intelligent video surveillance systems: A review. Expert Systems with Applications 91, (2018), 480–491.Google Scholar
Digital Library
- [11] . 2018. Score level fusion for cancelable multi-biometric verification. Pattern Recognition Letters, 2018.Google Scholar
- [12] . 2017. Collective first person for automatic gaze analysis in multiparty conversations. IEEE. Trans. MM 1, 107–122.Google Scholar
Digital Library
- [13] . 2017. Multi-modal framework for analyzing the affect of a group of people. IEEE. Trans. MM 8, 1–16, 2017.Google Scholar
- [14] . 2019. Deep patch representation with shared codebook for scene classification. ACM. TOMM 15, 5 (2019).Google Scholar
- [15] . 2019. Structure-aware deep learning for product image classification. ACM. TOMM 15, 4, (2019).Google Scholar
- [16] . 2018. Structure-aware multimodal features fusion for RGB-D scene classification and beyond. ACM. TOMM 14, 39 (2018).Google Scholar
- [17] . 2017. Mobilenets: Efficient convolutional neural networks for mobile vision applications. arXiv preprint arXiv:1704.04861 (2017).Google Scholar
- [18] . 2009. Abnormal event detection based on spatio-temporal co-occurences. In Proc. CVPR, 2458–2465, 2009.Google Scholar
Cross Ref
- [19] . 2017. Abnormal behavior detection in LWIR surveillance of railway platforms. In Proc. AVSS 2017.Google Scholar
- [20] . 2011. Robust video surveillance for fall detection based on human shape deformation. IEEE Trans. CSVT 21, (2011), 611–622.Google Scholar
Digital Library
- [21] . 2017. Toward abnormal trajectory and event detection in video surveillance. IEEE Trans. CSVT 27, (2017), 683–695.Google Scholar
Digital Library
- [22] . 2017. Accelerating vanishing point based line sampling scheme for real time people localization. IEEE Trans. CSVT 27, (2017), 409–420.Google Scholar
Digital Library
- [23] . 2015. Social attribute aware force model: Exploiting richness of interaction for abnormal crowd detection. IEEE Trans. CSVT 25, (2015), 1231–1245.Google Scholar
- [24] . 2014. Detection of abnormal visual events via global optical flow orientation histogram. IEEE Trans. IFS 9, (2014), 988–998.Google Scholar
- [25] . 2015. Suspicious face detection based on eye and other facial features movement monitoring. In Proc. AIPR, 1–8.Google Scholar
Cross Ref
- [26] . 2018. A video based abnormal human behavior detection for psychiatric patient monitoring. In Proc. IWAIT, 1–4, 2018.Google Scholar
Cross Ref
- [27] . 2016. Training deep networks for facial expression recognition with crowd-sourced label distribution. In Proc. ACMMM, 279–283, 2016.Google Scholar
Digital Library
- [28] . 2019. Emotion recognition using facial expression by using key points descriptor and texture features. Multimedia Tools and Applications 78, (2019), 16195–16219.Google Scholar
Digital Library
- [29] . 2020. PRATIT: A CNN-based emotion recognition system using histogram equalization and data augmentation. Multimedia Tools and Applications, 2285–2307.Google Scholar
Cross Ref
- [30] . 2019. Structure function based transform features for behavior-oriented social media image classification. In Proc. ACPR, 594–608.Google Scholar
- [31] . 2018. The fusion of multispectral palmprints using the information set based features and classifier. Engineering Applications of Artificial Intelligence 67, (2018), 111–125.Google Scholar
Digital Library
- [32] . 2017. Universal skin detection without color information. In Proc. WCACV, 20–28.Google Scholar
Cross Ref
- [33] . 2018. Multi-script-oriented text detection and recognition in video/scene/born digital images. IEEE Trans. CSVT, 1–16.Google Scholar
- [34] . 1999. Fast training of support vector machines using sequential minimal optimization. Advances in Kernel Methods, 185–208.Google Scholar
Digital Library
- [35] . 2016. Joint face detection and alignment using multi-task cascaded convolutional networks. IEEE Signal Processing Letters 23, (2016), 1499–1503.Google Scholar
- [36] . 2014. Blurred image region detection and segmentation. In Proc. ICIP 4427–4431, 2014.Google Scholar
Cross Ref
- [37] . 2016. Medical image fusion using discrete fractional wavelet transform. Biomedical Signal Processing and Control 27, 103–111, 2016.Google Scholar
Cross Ref
- [38] . 2012. ImageNet classification with deep convolutional neural networks. In Proc. NIPS, 1–9, 2012.Google Scholar
Digital Library
- [39] . 2015. Very deep convolutional networks for large scale image recognition. In Proc. ICLR, 1–14.Google Scholar
- [40] . 2015. Going deeper with convolutions. In Proc. CVPR, 1–9.Google Scholar
- [41] . 2016. Deep residual learning for image recognition. In Proc. CVPR, 770–778, 2016.Google Scholar
Cross Ref
Index Terms
A New Foreground-Background based Method for Behavior-Oriented Social Media Image Classification
Recommendations
Uses and gratifications of social networking sites for bridging and bonding social capital
Applying uses and gratifications theory (UGT) and social capital theory, our study examined users of four social networking sites (SNSs) (Facebook, Twitter, Instagram, and Snapchat), and their influence on online bridging and bonding social capital. ...
Social media user classification: based on social capital expectation, susceptibility, and compulsion loop
ICEC '17: Proceedings of the International Conference on Electronic CommerceSocial media such as Facebook, Instagram and Twitter are originally developed as communication tools among individuals for private conversations. Through the platforms, people share photos, stories and news with their social media friends to interact ...
Structure Function Based Transform Features for Behavior-Oriented Social Media Image Classification
Pattern RecognitionAbstractSocial media has become an essential part of people to reflect their day to day activities including emotions, feelings, threatening and so on. This paper presents a new method for the automatic classification of behavior-oriented images like ...






Comments