Abstract
Social media provides the means by which extremist social movements, such as white supremacy and anti-LGBTQ, thrive online. Yet, we know little about the roles played by the participants of such movements. In this paper, we investigate these participants to characterize their roles, their role dynamics, and their influence in spreading online extremism. Our participants-online extremist accounts-are 4,876 public Facebook pages or groups that have shared information from the websites of 289 Southern Poverty LawCenter (SPLC) designated extremist groups. Guided by theories of participatory activism, we map the information sharing features of these extremists accounts. By clustering the quantitative features followed by qualitative expert validation, we identify five roles surrounding extremist activism-educators, solicitors, flamers, motivators, sympathizers. For example, solicitors use links from extremist websites to attract donations and participation in extremist issues, whereas flamers share inflammatory extremist content inciting anger. We further investigate role dynamics such as, how stable these roles are over time and how likely will extremist accounts transition from one role into another. We find that roles core to the movement-educators and solicitors-are more stable, while flamers and motivators can transition to sympathizers with high probability.
Finally, using a Hawkes process model, we test which roles are more influential in spreading various types of information. We find that educators and solicitors exert the most influence in triggering extremist link posts, whereas flamers are influential in triggering the spread of information from fake news sources. Our results help in situating various roles on the trajectory of deeper engagement into the extremist movements and understanding the potential effect of various counter-extremism interventions. Our findings have implications for understanding how online extremist movements flourish through participatory activism and how they gain a spectrum of allies for mobilizing extremism online.
- ADL. 2020 a. BitChute: A Hotbed of Hate | Anti-Defamation League. https://www.adl.org/blog/bitchute-a-hotbed-of-hate. (Accessed on 10/14/2020).Google Scholar
- ADL. 2020 b. Who We Are | About Anti-Defamation League | ADL. https://www.adl.org/who-we-are. (Accessed on 10/15/2020).Google Scholar
- Ofer Arazy, Hila Liifshitz-Assaf, Oded Nov, Johannes Daxenberger, Martina Balestra, and Coye Cheshire. 2017. On the" how" and" why" of emergent role behaviors in Wikipedia. In Proceedings of the 2017 ACM Conference on Computer Supported Cooperative Work and Social Computing. 2039--2051.Google Scholar
Digital Library
- Balca Arda. 2015. The construction of a new sociality through social media: The case of the Gezi uprising in Turkey. Conjunctions. Transdisciplinary Journal of Cultural Participation, Vol. 2, 1 (2015), 72--99.Google Scholar
- Prakhar Biyani, Cornelia Caragea, Prasenjit Mitra, and John Yen. 2014. Identifying emotional and informational support in online health communities. In Proceedings of COLING 2014, the 25th International Conference on Computational Linguistics: Technical Papers. 827--836.Google Scholar
- David G Bromley and Anson D Shupe Jr. 1980. Financing the new religions: A resource mobilization approach. Journal for the Scientific Study of Religion (1980), 227--239.Google Scholar
- William K Carroll and Robert A Hackett. 2006. Democratic media activism through the lens of social movement theory. Media, culture & society, Vol. 28, 1 (2006), 83--104.Google Scholar
- Stevie Chancellor, Jessica Annette Pater, Trustin Clear, Eric Gilbert, and Munmun De Choudhury. 2016. # thyghgapp: Instagram content moderation and lexical variation in pro-eating disorder communities. In Proceedings of the 19th ACM Conference on Computer-Supported Cooperative Work & Social Computing. 1201--1213.Google Scholar
Digital Library
- Xi Chen, Shen Zhao, and Wei Li. 2019. Opinion dynamics model based on cognitive styles: field-dependence and field-independence. Complexity, Vol. 2019 (2019).Google Scholar
- Catherine Corrigall-Brown. 2011. Patterns of protest: Trajectories of participation in social movements. Stanford University Press.Google Scholar
- Jacob Davey and Julia Ebner. 2017. The Fringe Insurgency. Connectivity, Convergence and Mainstreaming of the Extreme Right. Institute for Strategic Dialogue (http://www.isdglobal.org/wp-content/uploads/2017/10/The-Fringe-Insurgency-221017.pdf) (2017).Google Scholar
- Claes H De Vreese and Hajo Boomgaarden. 2006. News, political knowledge and participation: The differential effects of news media exposure on political knowledge and participation. Acta Politica, Vol. 41, 4 (2006), 317--341.Google Scholar
Cross Ref
- Mario Diani. 1992. The concept of social movement. The sociological review, Vol. 40, 1 (1992), 1--25.Google Scholar
- Joan Donovan. 2019. Extremists Understand What Tech Platforms Have Built - The Atlantic. https://www.theatlantic.com/ideas/archive/2019/03/extremists-understand-what-tech-platforms-have-built/585136/. (Accessed on 09/17/2020).Google Scholar
- Bob Edwards and John D McCarthy. 2004. Resources and social movement mobilization. The Blackwell companion to social movements (2004), 116--152.Google Scholar
Cross Ref
- Paul Embrechts, Thomas Liniger, and Lu Lin. 2011. Multivariate Hawkes processes: an application to financial data. Journal of Applied Probability, Vol. 48, A (2011), 367--378.Google Scholar
Cross Ref
- Myra Marx Ferree and Frederick D Miller. 1985. Mobilization and meaning: Toward an integration of social psychological and resource perspectives on social movements. Sociological Inquiry, Vol. 55, 1 (1985), 38--61.Google Scholar
Cross Ref
- Institute for Economics and Peace. 2020. GTI-2020-web-1.pdf. https://www.visionofhumanity.org/wp-content/uploads/2020/11/GTI-2020-web-1.pdf. (Accessed on 01/11/2021).Google Scholar
- Tony Formica. 2020. A Social (Media) Contract: Reconciling American Freedom and Security in an Age of Online Radicalization and Extremism. Yale J. Int'l Aff., Vol. 15 (2020), 131.Google Scholar
- JA Frimer, R Boghrati, J Haidt, J Graham, and M Dehgani. 2019. Moral foundations dictionary for linguistic analyses 2.0. Unpublished manuscript (2019).Google Scholar
- William A Gamson, William Anthony Gamson Gamson, William Anthony Gamson, and William A Gamson. 1992. Talking politics .Cambridge university press.Google Scholar
- Malcolm Gladwell. 2010. Small change. The New Yorker, Vol. 4, 2010 (2010), 42--49.Google Scholar
- Rahul Goel, Sandeep Soni, Naman Goyal, John Paparrizos, Hanna Wallach, Fernando Diaz, and Jacob Eisenstein. 2016. The social dynamics of language change in online networks. In International Conference on Social Informatics. Springer, 41--57.Google Scholar
Cross Ref
- Max Halupka. 2018. The legitimisation of clicktivism. Australian Journal of Political Science, Vol. 53, 1 (2018), 130--141.Google Scholar
Cross Ref
- William L. Hamilton, Justine Zhang, Cristian Danescu-Niculescu-Mizil, Dan Jurafsky, and Jure Leskovec. 2017. Loyalty in Online Communities. (2017). http://arxiv.org/abs/1703.03386Google Scholar
- Benjamin D Horne, Sibel Adali, and Sujoy Sikdar. 2017. Identifying the social signals that drive online discussions: A case study of reddit communities. In 2017 26th International Conference on Computer Communication and Networks (ICCCN) .Google Scholar
Cross Ref
- IACP. 2020. Facebook and Violent Extremism Awareness Brief. https://www.theiacp.org/resources/document/facebook-and-violent-extremism-awareness-brief. (Accessed on 10/15/2020).Google Scholar
- Kirk Job-Sluder and Sasha A Barab. 2004. Shared" we" and shared" they" indicators of group identity in online teacher professional development. Designing for virtual communities in the service of learning (2004).Google Scholar
- NF Johnson, R Leahy, N Johnson Restrepo, N Velasquez, M Zheng, P Manrique, P Devkota, and Stefan Wuchty. 2019. Hidden resilience and adaptive dynamics of the global online hate ecology. Nature, Vol. 573, 7773 (2019), 261--265.Google Scholar
- Seth G Jones, Catrina Doxsee, and Nicholas Harrington. 2020. The escalating terrorism problem in the United States. (2020).Google Scholar
- Sebastian Jungkunz. 2019. Towards a measurement of extreme left-wing attitudes. German Politics, Vol. 28, 1 (2019), 101--122.Google Scholar
Cross Ref
- Alex Kaplan. 2018. The head of an anti-immigration PAC runs Facebook pages that share fake news from plagiarized sites | Media Matters for America. https://www.mediamatters.org/facebook/head-anti-immigration-pac-runs-facebook-pages-share-fake-news-plagiarized-sites. (Accessed on 10/15/2020).Google Scholar
- Jessalynn Marie Keller. 2012. Virtual feminisms: Girls' blogging communities, feminist activism, and participatory politics. Information, Communication & Society, Vol. 15, 3 (2012), 429--447.Google Scholar
Cross Ref
- Makena Kelly. 2020 a. Facebook still hosts boogaloo extremist groups, report finds - The Verge. https://www.theverge.com/2020/8/12/21365278/facebook-boogaloo-tech-transparency-right-wing-extremist-platform. (Accessed on 09/13/2020).Google Scholar
- Makena Kelly. 2020 b. Facebook still hosts boogaloo extremist groups, report finds - The Verge. https://www.theverge.com/2020/8/12/21365278/facebook-boogaloo-tech-transparency-right-wing-extremist-platform. (Accessed on 10/15/2020).Google Scholar
- Bert Klandermans. 1984. Mobilization and participation: Social-psychological expansisons of resource mobilization theory. American sociological review (1984), 583--600.Google Scholar
- Bert Klandermans and Dirk Oegema. 1987. Potentials, networks, motivations, and barriers: Steps towards participation in social movements. American sociological review (1987), 519--531.Google Scholar
- Michael Krona. 2019. 5 ISIS's Media Ecology and Participatory Activism Tactics. The Media World of ISIS (2019), 101.Google Scholar
- Srijan Kumar, Xikun Zhang, and Jure Leskovec. 2019. Predicting dynamic embedding trajectory in temporal interaction networks. In Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. 1269--1278.Google Scholar
Digital Library
- Ryan Lenz. 2013. Following the White Rabbit | Southern Poverty Law Center. https://www.splcenter.org/fighting-hate/intelligence-report/2013/following-white-rabbit. (Accessed on 10/15/2020).Google Scholar
- Erik Lewis and George Mohler. 2011. A nonparametric EM algorithm for multiscale Hawkes processes. Journal of Nonparametric Statistics, Vol. 1, 1 (2011), 1--20.Google Scholar
- Scott Linderman and Ryan Adams. 2014. Discovering latent network structure in point process data. In International Conference on Machine Learning. 1413--1421.Google Scholar
- Jenni Marsh and Tara Mulholland. [n.d.]. How the Christchurch terrorist attack was made for social media - CNN. https://www.cnn.com/2019/03/15/tech/christchurch-internet-radicalization-intl/index.html. (Accessed on 09/23/2020).Google Scholar
- Gary T Marx and James L Wood. 1975. Strands of theory and research in collective behavior. Annual review of sociology, Vol. 1, 1 (1975), 363--428.Google Scholar
- John McCarthy and Mayer N Zald. 2003. Social movement organizations. The social movements reader: Cases and concepts (2003), 169--186.Google Scholar
- John D Mccarthy, Mayer N Zald, Gary Long, Anthony Oberschall, Anthony Orum, Kathy Pearce, Jack Seidman, and Benjamin Walter. [n.d.]. Resource Mobilization and Social Movements: A Partial Theory'. Technical Report. http://www.journals.uchicago.edu/t-and-cGoogle Scholar
- Evgeny Morozov. 2009. The brave new world of slacktivism. Foreign policy, Vol. 19, 05 (2009).Google Scholar
- Aldon Morris. 2002. Leadership in Social Movements Aldon Morris and Suzanne Staggenborg. (2002).Google Scholar
- Karsten Müller and Carlo Schwarz. 2020. Fanning the Flames of Hate: Social Media and Hate Crime. SSRN (2020).Google Scholar
- Edward Newell, David Jurgens, Haji Mohammad Saleem, Hardik Vala, Jad Sassine, Caitrin Armstrong, and Derek Ruths. 2016. User Migration in Online Social Networks: A Case Study on Reddit During a Period of Community Unrest.. In ICWSM. 279--288.Google Scholar
- Permanent Culture Now. 2018. Introduction to activism. Recuperado dehttp://www. permanentculturenow. com/what-is-activism (2018).Google Scholar
- Sonia Nú nez Puente, Diana Fernández Romero, and Susana Vázquez Cupeiro. 2017. Online feminist practice, participatory activism and public policies against gender-based violence in Spain. Feminist Theory, Vol. 18, 3 (2017), 299--321.Google Scholar
Cross Ref
- Jonathan A Obar, Paul Zube, and Clifford Lampe. 2012. Advocacy 2.0: An analysis of how advocacy groups in the United States perceive and use social media as tools for facilitating civic engagement and collective action. Journal of information policy, Vol. 2 (2012), 1--25.Google Scholar
- Anthony Oberschall. 1973. Social conflict and social movements. Prentice hall.Google Scholar
- Nicholas Owen. [n.d.]. The conscience constituent reconsidered. In Other People's Struggles. Oxford University Press.Google Scholar
- Umashanthi Pavalanathan and Munmun De Choudhury. 2015. Identity management and mental health discourse in social media. In WWW. ACM.Google Scholar
- Tsveta Petrova and Sidney Tarrow. 2007. Transactional and participatory activism in the emerging European polity: The puzzle of East-Central Europe. Comparative political studies, Vol. 40, 1 (2007), 74--94.Google Scholar
Cross Ref
- Shruti Phadke and Tanushree Mitra. 2020. Many Faced Hate: A Cross Platform Study of Content Framing and Information Sharing by Online Hate Groups. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems (Honolulu, HI, USA) (CHI '20). Association for Computing Machinery, New York, NY, USA, 1--13. https://doi.org/10.1145/3313831.3376456Google Scholar
Digital Library
- Whitney Phillips. 2018. Data & Society - The Oxygen of Amplification. https://datasociety.net/library/oxygen-of-amplification. (Accessed on 09/23/2020).Google Scholar
- Counter Extremism Project. 2020 a. Germany: Extremism & Counter-Extremism | Counter Extremism Project. https://www.counterextremism.com/countries/germany. (Accessed on 01/14/2021).Google Scholar
- The Tech Transparency Project. 2020 b. Facebook's Boogaloo Problem: A Record of Failure | Tech Transparency Project. https://www.techtransparencyproject.org/articles/facebooks-boogaloo-problem-record-failure. (Accessed on 10/15/2020).Google Scholar
- Babak Rahimi. 2016. Vahid Online: Post-2009 Iran and the Politics of Citizen Media Convergence. Social Sciences, Vol. 5, 4 (2016), 77.Google Scholar
Cross Ref
- ADL Reports. 2019. Right-Wing Extremism Linked to Every 2018 Extremist Murder in the U.S., ADL Finds | Anti-Defamation League. https://www.adl.org/news/press-releases/right-wing-extremism-linked-to-every-2018-extremist-murder-in-the-us-adl-finds. (Accessed on 09/23/2020).Google Scholar
- Howard Rheingold. 2007. Smart mobs: The next social revolution .Basic books.Google Scholar
- Marian-Andrei Rizoiu, Young Lee, Swapnil Mishra, and Lexing Xie. 2017. A tutorial on hawkes processes for events in social media. arXiv preprint arXiv:1708.06401 (2017).Google Scholar
- Debbie Rodan and Jane Mummery. 2017. Activism and digital culture in Australia .Rowman & Littlefield.Google Scholar
- Dana Rotman, Sarah Vieweg, Sarita Yardi, Ed Chi, Jenny Preece, Ben Shneiderman, Peter Pirolli, and Tom Glaisyer. 2011. From slacktivism to activism: participatory culture in the age of social media. In CHI'11 Extended Abstracts on Human Factors in Computing Systems. 819--822.Google Scholar
Digital Library
- Roser Sauri. 2008. A factuality profiler for eventualities in text. Ph.D. Dissertation. Ph. D. thesis, Brandeis University.Google Scholar
Digital Library
- Swathi Shanmugasundaram. 2018. Anti-immigrant roundup: 7/6/18 | Southern Poverty Law Center. https://www.splcenter.org/hatewatch/2018/07/06/anti-immigrant-roundup-7618. (Accessed on 10/15/2020).Google Scholar
- Simon Sheather. 2009. A modern approach to regression with R. Springer Science & Business Media.Google Scholar
- Alexandra A Siegel and Vivienne Badaan. 2020. # No2Sectarianism: Experimental Approaches to Reducing Sectarian Hate Speech Online. American Political Science Review, Vol. 114, 3 (2020), 837--855.Google Scholar
Cross Ref
- Bernd Simon and PG Klandermans. 2001. Toward a social psychological analysis of politicized collective identity: Conceptualization, antecedents and consequences. American Psychologist, Vol. 56 (2001), 319--331.Google Scholar
Cross Ref
- SPLC. 2020 a. Alliance Defending Freedom | Southern Poverty Law Center. https://www.splcenter.org/fighting-hate/extremist-files/group/alliance-defending-freedom. (Accessed on 10/15/2020).Google Scholar
- SPLC. 2020 b. National Vanguard | Southern Poverty Law Center. https://www.splcenter.org/fighting-hate/extremist-files/group/national-vanguard. (Accessed on 10/15/2020).Google Scholar
- Hatewatch Staff. 2020. Facebook's Strategy for Taking Down Hate Groups is Spotty and Ineffective | Southern Poverty Law Center. https://www.splcenter.org/hatewatch/2020/04/07/facebooks-strategy-taking-down-hate-groups-spotty-and-ineffective. (Accessed on 09/15/2020).Google Scholar
- Kate Starbird, Ahmer Arif, and Tom Wilson. 2019. Disinformation as collaborative work: Surfacing the participatory nature of strategic information operations. Proceedings of the ACM on Human-Computer Interaction, Vol. 3, CSCW (2019), 1--26.Google Scholar
Digital Library
- Stefan Stürmer, Bernd Simon, Michael Loewy, and Heike Jörger. 2003. The dual-pathway model of social movement participation: The case of the fat acceptance movement. Social Psychology Quarterly (2003), 71--82.Google Scholar
- Hiroki Takikawa and Takuto Sakamoto. 2019. The moral--emotional foundations of political discourse: a comparative analysis of the speech records of the US and the Japanese legislatures. Quality & Quantity (2019), 1--20.Google Scholar
- Yla R Tausczik and James W Pennebaker. 2010. The psychological meaning of words: LIWC and computerized text analysis methods. Journal of language and social psychology, Vol. 29, 1 (2010), 24--54.Google Scholar
- Ralph H Turner. 1969. The public perception of protest. American Sociological Review (1969), 815--831.Google Scholar
- Sebastián Valenzuela. 2013. Unpacking the use of social media for protest behavior: The roles of information, opinion expression, and activism. American behavioral scientist, Vol. 57, 7 (2013), 920--942.Google Scholar
Cross Ref
- Jacquelien van Stekelenburg and Bert Klandermans. 2013. Social psychology of movement participation. The Wiley-Blackwell Encyclopedia of Social and Political Movements (2013).Google Scholar
- Jacquelien Van Stekelenburg and Bert Klandermans. 2013. The social psychology of protest. Current Sociology, Vol. 61, 5--6 (2013), 886--905.Google Scholar
- Jacquelien Van Stekelenburg and Bert Klandermans. 2017. Individuals in movements: A social psychology of contention. In Handbook of social movements across disciplines. Springer, 103--139.Google Scholar
- Ariadne Vromen. 2017. Digital citizenship and political engagement. In Digital Citizenship and Political Engagement. Springer, 9--49.Google Scholar
- Mattias Wahlström, Abby Peterson, and Magnus Wennerhag. 2018. "CONSCIENCE ADHERENTS" REVISITED: NON-LGBT PRIDE PARADE PARTICIPANTS. Mobilization: An International Quarterly, Vol. 23, 1 (2018), 83--100.Google Scholar
- Yi-Chia Wang, Robert Kraut, and John M Levine. 2012. To stay or leave? The relationship of emotional and informational support to commitment in online health support groups. In Proceedings of the ACM 2012 conference on computer supported cooperative work. 833--842.Google Scholar
Digital Library
- Micah White. 2010. Clicktivism is ruining leftist activism. The Guardian, Vol. 12, August (2010), 2010.Google Scholar
- Janyce Wiebe, Theresa Wilson, and Claire Cardie. 2005. Annotating expressions of opinions and emotions in language. Language resources and evaluation, Vol. 39, 2--3 (2005), 165--210.Google Scholar
- Mayer N Zald and Roberta Ash. 1966. Social movement organizations: Growth, decay and change. Social forces, Vol. 44, 3 (1966), 327--341.Google Scholar
- Savvas Zannettou, Tristan Caulfield, Emiliano De Cristofaro, Nicolas Kourtelris, Ilias Leontiadis, Michael Sirivianos, Gianluca Stringhini, and Jeremy Blackburn. 2017. The web centipede: understanding how web communities influence each other through the lens of mainstream and alternative news sources. In Proceedings of the 2017 Internet Measurement Conference. 405--417.Google Scholar
Digital Library
Index Terms
Educators, Solicitors, Flamers, Motivators, Sympathizers: Characterizing Roles in Online Extremist Movements
Recommendations
Many Faced Hate: A Cross Platform Study of Content Framing and Information Sharing by Online Hate Groups
CHI '20: Proceedings of the 2020 CHI Conference on Human Factors in Computing SystemsHate groups are increasingly using multiple social media platforms to promote extremist ideologies. Yet we know little about their communication practices across platforms. How do hate groups (or "in-groups"), frame their hateful agenda against the ...
Contagion dynamics of extremist propaganda in social networks
Recent terrorist attacks carried out on behalf of ISIS on American and European soil by lone wolf attackers or sleeper cells remind us of the importance of understanding the dynamics of radicalization mediated by social media communication channels. In ...
Characterizing Reddit Participation of Users Who Engage in the QAnon Conspiracy Theories
CSCW1Widespread conspiracy theories may significantly impact our society. This paper focuses on the QAnon conspiracy theory, a consequential conspiracy theory that started on and disseminated successfully through social media. Our work characterizes how ...






Comments