Abstract
Nowadays, broadcasting news on social media and websites has grown at a swifter pace, which has had negative impacts on both the general public and governments; hence, this has urged us to build a fake news detection system. Contextualized word embeddings have achieved great success in recent years due to their power to embed both syntactic and semantic features of textual contents. In this article, we aim to address the problem of the lack of fake news datasets in Persian by introducing a new dataset crawled from different news agencies, and propose two deep models based on the Bidirectional Encoder Representations from Transformers model (BERT), which is a deep contextualized pre-trained model for extracting valuable features. In our proposed models, we benefit from two different settings of BERT, namely pool-based representation, which provides a representation for the whole document, and sequence representation, which provides a representation for each token of the document. In the former one, we connect a Single Layer Perceptron (SLP) to the BERT to use the embedding directly for detecting fake news. The latter one uses Convolutional Neural Network (CNN) after the BERT’s embedding layer to extract extra features based on the collocation of words in a corpus. Furthermore, we present the TAJ dataset, which is a new Persian fake news dataset crawled from news agencies’ websites. We evaluate our proposed models on the newly provided TAJ dataset as well as the two different Persian rumor datasets as baselines. The results indicate the effectiveness of using deep contextualized embedding approaches for the fake news detection task. We also show that both BERT-SLP and BERT-CNN models achieve superior performance to the previous baselines and traditional machine learning models, with 15.58% and 17.1% improvement compared to the reported results by Zamani et al. [30], and 11.29% and 11.18% improvement compared to the reported results by Jahanbakhsh-Nagadeh et al. [9].
- [1] . 2017. Detection of online fake news using n-gram analysis and machine learning techniques. In International Conference on Intelligent, Secure, and Dependable Systems in Distributed and Cloud Environments. Springer, 127–138.Google Scholar
- [2] . 2018. Detecting opinion spams and fake news using text classification. Security and Privacy 1, 1 (2018), e9.Google Scholar
Cross Ref
- [3] . 2002. SMOTE: Synthetic minority over-sampling technique. Journal of Artificial Intelligence Research 16 (2002), 321–357.Google Scholar
Digital Library
- [4] . 2019. BERT: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers).
Association for Computational Linguistics ,Minneapolis, MN , 4171–4186.DOI: DOI: https://doi.org/10.18653/v1/N19-1423Google Scholar - [5] . 2021. Detecting fake news with capsule neural networks. Applied Soft Computing 101 (
2021), 106991. DOI: DOI: https://doi.org/10.1016/j.asoc.2020.106991Google ScholarCross Ref
- [6] . 2021. Convolutional neural network with margin loss for fake news detection. Information Processing & Management 58, 1 (2021), 102418.Google Scholar
Cross Ref
- [7] . 2017. Fake news detection using naive Bayes classifier. In 2017 IEEE 1st Ukraine Conference on Electrical and Computer Engineering (UKRCON’17). IEEE, 900–903.Google Scholar
- [8] . 1994. Speech recognition using a stochastic language model integrating local and global constraints. In Human Language Technology: Proceedings of a Workshop held at Plainsboro, New Jersey. https://www.aclweb.org/anthology/H94-1015.Google Scholar
Digital Library
- [9] . 2020. A model to measure the spread power of rumors. arXiv preprint arXiv:2002.07563 (2020).Google Scholar
- [10] . 2019. Mvae: Multimodal variational autoencoder for fake news detection. In The World Wide Web Conference. 2915–2921.Google Scholar
Digital Library
- [11] . 2019. Text summarization with pretrained encoders. arXiv preprint arXiv:1908.08345 (2019).Google Scholar
- [12] . 2019. Detection of satiric news on social media: Analysis of the phenomenon with a French dataset. In 2019 28th International Conference on Computer Communication and Networks (ICCCN’19). 1–6.Google Scholar
- [13] . 2018. Persian rumor detection on twitter. In 2018 9th International Symposium on Telecommunications (IST’18). IEEE, 597–602.Google Scholar
- [14] . 2013. Distributed representations of words and phrases and their compositionality. In Advances in Neural Information Processing Systems. 3111–3119.Google Scholar
Digital Library
- [15] . 2010. Hierarchical Pitman-Yor language model for information retrieval. In Proceedings of the 33rd International ACM SIGIR Conference on Research and Development in Information Retrieval
(SIGIR’10) .Association for Computing Machinery ,New York, NY , 793–794.DOI: DOI: https://doi.org/10.1145/1835449.1835619Google ScholarDigital Library
- [16] . 2019. Fine-grained sentiment classification using BERT. In 2019 Artificial Intelligence for Transforming Business and Society
(AITB’19) , Vol. 1. 1–5.Google Scholar - [17] . 2014. Glove: Global vectors for word representation. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing
(EMNLP’14) . 1532–1543.Google ScholarCross Ref
- [18] . 2018. Deep contextualized word representations. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers).
Association for Computational Linguistics ,New Orleans, LA , 2227–2237.DOI: DOI: https://doi.org/10.18653/v1/N18-1202Google ScholarCross Ref
- [19] . 1998. A language modeling approach to information retrieval. In Proceedings of the 21st Annual International ACM SIGIR Conference on Research and Development in Information Retrieval
(SIGIR’98) .Association for Computing Machinery ,New York, NY , 275–281.DOI: DOI: https://doi.org/10.1145/290941.291008Google ScholarCross Ref
- [20] . 2019. Detection of fake news based on readability. In Proceedings of Reunión Internacional de Inteligencia Artificial y sus Aplicaciones.Google Scholar
- [21] . 2015. Deception detection for news: Three types of fakes. Proceedings of the Association for Information Science and Technology 52, 1 (2015), 1–4.Google Scholar
Cross Ref
- [22] . 2016. Hyperpartisan Facebook pages are publishing false and misleading information at an alarming rate. Buzzfeed News 20 (2016).Google Scholar
- [23] . 2019. Beheshti-NER: Persian named entity recognition using BERT. In Proceedings of International Conference on Natural Language and Speech Processing, Trento, Italy.Google Scholar
- [24] . 2008. Adaptive language modeling for word prediction. In Proceedings of the ACL-08: HLT Student Research Workshop.
Association for Computational Linguistics ,Columbus, OH , 61–66. https://www.aclweb.org/anthology/P08-3011.Google ScholarCross Ref
- [25] . 2019. Polarization and fake news: Early warning of potential misinformation targets. ACM Transactions on the Web (TWEB) 13, 2 (2019), 1–22.Google Scholar
Digital Library
- [26] . 2019. Fake news detection with the new German dataset “GermanFakeNC.” In International Conference on Theory and Practice of Digital Libraries. Springer, 288–295.Google Scholar
- [27] . 2017. “Liar, liar pants on fire”: A new benchmark dataset for fake news detection. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers).
Association for Computational Linguistics ,Vancouver, Canada , 422–426.DOI: DOI: https://doi.org/10.18653/v1/P17-2067Google ScholarCross Ref
- [28] . 2019. Multi-passage BERT: A globally normalized BERT model for open-domain question answering. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP’19).
Association for Computational Linguistics ,Hong Kong, China , 5878–5882.DOI: DOI: https://doi.org/10.18653/v1/D19-1599Google ScholarCross Ref
- [29] . 2020. Towards making the most of BERT in neural machine translation. Proceedings of the AAAI Conference on Artificial Intelligence 34 (
2020), 9378–9385. DOI: DOI: https://doi.org/10.1609/aaai.v34i05.6479Google ScholarCross Ref
- [30] . 2017. Rumor detection for persian tweets. In 2017 Iranian Conference on Electrical Engineering (ICEE’17).
IEEE , 1532–1536.Google Scholar - [31] . 2019. Persian stance classification data set. In Conference For Truth and Trust Online.
DOI: DOI: https://doi.org/10.36370/tto.2019.30Google Scholar
Index Terms
Persian Fake News Detection: Neural Representation and Classification at Word and Text Levels
Recommendations
Fake News Early Detection: A Theory-driven Model
Field NotesMassive dissemination of fake news and its potential to erode democracy has increased the demand for accurate fake news detection. Recent advancements in this area have proposed novel techniques that aim to detect fake news by exploring how it ...
Deep contextualized text representation and learning for fake news detection
AbstractIn recent years, due to the widespread use of social media and broadcasting agencies around the world, people are extremely exposed to being affected by false information and fake news, all of which have negative impacts on both ...
Highlights- Using different deep contextualized text representation models for fake news detection.
Fake News Research: Theories, Detection Strategies, and Open Problems
KDD '19: Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data MiningFake news has become a global phenomenon due its explosive growth, particularly on social media. The goal of this tutorial is to (1) clearly introduce the concept and characteristics of fake news and how it can be formally differentiated from other ...






Comments