skip to main content
research-article

Neural Topic Model Training with the REBAR Gradient Estimator

Published:15 November 2022Publication History
Skip Abstract Section

Abstract

Topic modelling is an important approach of unsupervised machine learning that allows automatically extracting the main “topics” from large collections of documents. In addition, topic modelling is able to identify the topic proportions of each individual document, which can be helpful for organizing the collections. Many topic modelling algorithms have been proposed to date, including several that leverage advanced techniques such as variational inference and deep autoencoders. However, to date topic modelling has made limited use of reinforcement learning, a framework that has obtained vast success in many other unsupervised learning tasks. For this reason, in this article we propose training a neural topic model using a reinforcement learning objective and minimizing the objective with the recently-proposed REBAR gradient estimator. Experiments performed over two probing datasets have shown that the proposed model has achieved improvements over all the compared models in terms of both model perplexity and topic coherence, and produced topics that appear qualitatively informative and consistent.

REFERENCES

  1. [1] Martín Abadi, Ashish Agarwal, Paul Barham, Eugene Brevdo, Zhifeng Chen, Craig Citro, Greg Corrado, Andy Davis, Jeffrey Dean, Matthieu Devin, Sanjay Ghemawat, Ian Goodfellow, Andrew Harp, Geoffrey Irving, Michael Isard, Yangqing Jia, Rafal Jozefowicz, Lukasz Kaiser, Manjunath Kudlur, Josh Levenberg, Dan Mané, Rajat Monga, Sherry Moore, Derek Murray, Chris Olah, Mike Schuster, Jonathon Shlens, Benoit Steiner, Ilya Sutskever, Kunal Talwar, Paul Tucker, Vincent Vanhoucke, Vijay Vasudevan, Fernanda Viégas, Oriol Vinyals, Pete Warden, Martin Wattenberg, Martin Wicke, Yuan Yu, and Xiaoqiang Zheng. 2015. TensorFlow: Large-Scale machine learning on heterogeneous distributed systems. http://download.tensorflow.org/paper/whitepaper2015.pdf.Google ScholarGoogle Scholar
  2. [2] Alvarez-Melis David and Saveski Martin. 2016. Topic modeling in Twitter: Aggregating tweets by conversations. In Proceedings of the 10th International Conference on Web and Social Media. 519522.Google ScholarGoogle Scholar
  3. [3] Arnold Corey, El-Saden Suzie, Bui Alex, and Taira Ricky. 2010. Clinical case-based retrieval using latent topic analysis. AMIA Annual Symposium Proceedings 2010 (2010), 2630.Google ScholarGoogle Scholar
  4. [4] Blei David M., Ng Andrew Y., and Jordan Michael I.. 2003. Latent Dirichlet allocation. Journal of Machine Learning Research 3 (2003), 9931022.Google ScholarGoogle ScholarDigital LibraryDigital Library
  5. [5] Bradbury James, Frostig Roy, Hawkins Peter, Johnson Matthew James, Leary Chris, Maclaurin Dougal, and Wanderman-Milne Skye. 2018. JAX: composable transformations of Python+NumPy programs. Retrieved from http://github.com/google/jax.Google ScholarGoogle Scholar
  6. [6] Cecchini Mark, Aytug Haldun, Koehler Gary J., and Pathak Praveen. 2010. Making words work: Using financial text as a predictor of financial events. Decision Support Systems 50, 1 (2010), 164175.Google ScholarGoogle ScholarDigital LibraryDigital Library
  7. [7] Cheng Xueqi, Yan Xiaohui, Lan Yanyan, and Guo Jiafeng. 2014. BTM: Topic modeling over short texts. IEEE Transactions on Knowledge and Data Engineering 26, 12 (2014), 29282941.Google ScholarGoogle ScholarCross RefCross Ref
  8. [8] Das Rajarshi, Zaheer Manzil, and Dyer Chris. 2015. Gaussian LDA for topic models with word embeddings. In Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing. 795804.Google ScholarGoogle ScholarCross RefCross Ref
  9. [9] Deerwester Scott C., Dumais Susan T., Landauer Thomas K., Furnas George W., and Harshman Richard A.. 1990. Indexing by latent semantic analysis. Journal of the American Society for Information Science 41, 6 (1990), 391407.Google ScholarGoogle ScholarDigital LibraryDigital Library
  10. [10] Devyatkin Dmitry, Nechaeva Elena, Suvorov Roman, and Tikhomirov Ilya. 2018. Mapping the research landscape of agricultural sciences. Foresight and STI Governance1 (2018), 6978.Google ScholarGoogle ScholarCross RefCross Ref
  11. [11] Du Lan, Buntine Wray, Jin Huidong, and Chen Changyou. 2012. Sequential latent Dirichlet allocation. Knowledge and Information Systems 31 (2012), 475503.Google ScholarGoogle ScholarDigital LibraryDigital Library
  12. [12] Glover John. 2016. Modeling documents with Generative Adversarial Networks. In Proceedings of the NIPS 2016 Workshop on Adversarial Training. 17.Google ScholarGoogle Scholar
  13. [13] Grathwohl Will, Choi Dami, Wu Yuhuai, Roeder Geoffrey, and Duvenaud David. 2018. Backpropagation through the Void: Optimizing control variates for black-box gradient estimation. In Proceedings of the 6th International Conference on Learning Representations.Google ScholarGoogle Scholar
  14. [14] Grover Aditya, Dhar Manik, and Ermon Stefano. 2018. Flow-GAN: Combining maximum likelihood and adversarial learning in generative models. In Proceedings of the 32nd AAAI Conference on Artificial Intelligence. AAAI, 30693076.Google ScholarGoogle ScholarCross RefCross Ref
  15. [15] Gui Lin, Leng Jia, Pergola Gabriele, Zhou Yu, Xu Ruifeng, and He Yulan. 2019. Neural topic model with reinforcement learning. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing. Inui Kentaro, Jiang Jing, Ng Vincent, and Wan Xiaojun (Eds.), Association for Computational Linguistics, 34763481.Google ScholarGoogle ScholarCross RefCross Ref
  16. [16] Gumbel Emil J.. 1954. Statistical Theory of Extreme Values and Some Practical Applications: A Series of Lectures. US Government Printing Office.Google ScholarGoogle Scholar
  17. [17] Hand David J.. 2010. Text Mining: Classification, clustering, and applications. International Statistical Review 78 (2010), 134135.Google ScholarGoogle ScholarCross RefCross Ref
  18. [18] Hofmann Thomas. 1999. Probabilistic latent semantic analysis. In Proceedings of the 15th Conference on Uncertainty in Artificial Intelligence. 289296.Google ScholarGoogle ScholarDigital LibraryDigital Library
  19. [19] Jang Eric, Gu Shixiang, and Poole Ben. 2017. Categorical reparameterization with Gumbel-Softmax. In Proceedings of the 5th International Conference on Learning Representations. 112.Google ScholarGoogle Scholar
  20. [20] Kim Hannah, Drake Barry, Endert Alex, and Park Haesun. 2021. ArchiText: Interactive hierarchical topic modeling. IEEE Transactions on Visualization and Computer Graphics 17, 9 (2021), 36443655.Google ScholarGoogle ScholarDigital LibraryDigital Library
  21. [21] Kingma Diederik P. and Welling Max. 2014. Auto-Encoding variational Bayes. In Proceedings of the 2nd International Conference on Learning Representations. 114.Google ScholarGoogle Scholar
  22. [22] Kingma Diederik P. and Welling Max. 2019. An introduction to variational autoencoders. Foundations and Trends in Machine Learning 12, 4 (2019), 307392.Google ScholarGoogle ScholarDigital LibraryDigital Library
  23. [23] Konda Vijay R. and Tsitsiklis John N.. 1999. Actor-Critic algorithms. In Proceedings of the Advances in Neural Information Processing Systems 12. 10081014.Google ScholarGoogle Scholar
  24. [24] Kumar Amit, Esmaili Nazanin, and Piccardi Massimo. 2021. Topic-Document inference with the Gumbel-Softmax distribution. IEEE Access 9 (2021), 13131320. DOI:Google ScholarGoogle ScholarCross RefCross Ref
  25. [25] Lang Ken. 1995. Newsweeder: Learning to filter netnews. In Proceedings of the 12th International Conference on Machine Learning. 331339.Google ScholarGoogle ScholarCross RefCross Ref
  26. [26] Lau Jey Han, Newman David, and Baldwin Timothy. 2014. Machine reading tea leaves: Automatically evaluating topic coherence and topic model quality. In Proceedings of the 14th Conference of the European Chapter of the Association for Computational Linguistics. 530539.Google ScholarGoogle ScholarCross RefCross Ref
  27. [27] Li Weifeng, Yin Junming, and Chen Hsinchun. 2018. Supervised topic modeling using hierarchical Dirichlet process-based inverse regression: Experiments on E-Commerce applications. IEEE Transactions on Knowledge and Data Engineering 30, 6 (2018), 11921205.Google ScholarGoogle ScholarCross RefCross Ref
  28. [28] Lian Qiusheng, Yan Wenfeng, Zhang Xiaohua, and Chen Shuzhen. 2019. Single image rain removal using image decomposition and a dense network. IEEE/CAA Journal of Automatica Sinica 6 (2019), 14281437.Google ScholarGoogle Scholar
  29. [29] Liu Tengfei, Zhang Nevin Lianwen, and Chen Peixian. 2014. Hierarchical latent tree analysis for topic detection. CoRR 8725 (2014), 256272.Google ScholarGoogle Scholar
  30. [30] Maddison Chris J., Tarlow Daniel, and Minka Tom. 2014. A* sampling. In Proceedings of the Advances in Neural Information Processing Systems. 30863094.Google ScholarGoogle Scholar
  31. [31] McAuley Julian and Leskovec Jure. 2013. From amateurs to connoisseurs: Modeling the evolution of user expertise through online reviews. In Proceedings of the 22nd International Conference on World Wide Web. 897908.Google ScholarGoogle ScholarDigital LibraryDigital Library
  32. [32] Miao Yishu, Grefenstette Edward, and Blunsom Phil. 2017. Discovering discrete latent topics with neural variational inference. In Proceedings of the 34th International Conference on Machine Learning. 24102419.Google ScholarGoogle ScholarDigital LibraryDigital Library
  33. [33] Nguyen Thien Hai and Shirai Kiyoaki. 2015. Topic modeling based sentiment analysis on social media for stock market prediction. In Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing of the Asian Federation of Natural Language Processing. 13541364.Google ScholarGoogle ScholarCross RefCross Ref
  34. [34] Paszke Adam, Gross Sam, Chintala Soumith, Chanan Gregory, Yang Edward, DeVito Zachary, Lin Zeming, Desmaison Alban, Antiga Luca, and Lerer Adam. 2017. Automatic differentiation in PyTorch. In Proceedings of the NIPS 2017 Workshop on Autodiff Submission.Google ScholarGoogle Scholar
  35. [35] Peng Min, Xie Qianqian, Zhang Yanchun, Wang Hua, Zhang Xiuzhen, Huang Jimin, and Tian Gang. 2018. Neural sparse topical coding. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics. 23322340.Google ScholarGoogle ScholarCross RefCross Ref
  36. [36] Řehůřek Radim and Sojka Petr. 2010. Software framework for topic modelling with large corpora. In Proceedings of the LREC 2010 Workshop on New Challenges for NLP Frameworks. 4550.Google ScholarGoogle Scholar
  37. [37] Röder Michael, Both Andreas, and Hinneburg Alexander. 2015. Exploring the space of topic coherence measures. In Proceedings of the 8th ACM International Conference on Web Search and Data Mining. 399408.Google ScholarGoogle ScholarDigital LibraryDigital Library
  38. [38] Rodrigues Filipe, Lourenco Mariana, Ribeiro Bernardete, and Pereira Francisco C.. 2017. Learning supervised topic models for classification and regression from crowds. IEEE Transactions on Pattern Analysis and Machine Intelligence 39, 12 (2017), 24092422.Google ScholarGoogle ScholarCross RefCross Ref
  39. [39] Ruthotto Lars and Haber Eldad. 2021. An introduction to deep generative modeling.GAMM-Mitteilungen 44, 2 (2021), 1–24.Google ScholarGoogle Scholar
  40. [40] Sarioglu Efsun, Choi Hyeong-Ah, and Yadav Kabir. 2012. Clinical report classification using natural language processing and topic modeling. In Proceedings of the 11th International Conference on Machine Learning and Applications. 204209.Google ScholarGoogle ScholarDigital LibraryDigital Library
  41. [41] Schofield Alexandra, Magnusson Måns, Thompson Laure, and Mimno David. 2017. Understanding text pre-processing for latent Dirichlet allocation. In Proceedings of the First Women and Underrepresented Minorities in NLP Workshop. 14.Google ScholarGoogle Scholar
  42. [42] Seifollahi Sattar, Piccardi Massimo, and Jolfaei Alireza. 2021. An embedding-based topic model for document classification. ACM Transactions on Asian and Low-Resource Language Information Processing 20, 3 (2021), 113.Google ScholarGoogle ScholarDigital LibraryDigital Library
  43. [43] Srivastava Akash and Sutton Charles A.. 2017. Autoencoding variational inference for topic models. In Proceedings of the 5th International Conference on Learning Representations. 112.Google ScholarGoogle Scholar
  44. [44] Sutton Richard S. and Barto Andrew G.. 2018. Reinforcement Learning: An Introduction (second ed.). MIT Press.Google ScholarGoogle Scholar
  45. [45] Tucker George, Mnih Andriy, Maddison Chris J., Lawson Dieterich, and Sohl-Dickstein Jascha. 2017. REBAR: Low-variance, unbiased gradient estimates for discrete latent variable models. In Proceedings of the Advances in Neural Information Processing Systems.26272636.Google ScholarGoogle Scholar
  46. [46] Wang Chong, Paisley John W., and Blei David M.. 2011. Online variational inference for the hierarchical dirichlet process. In Proceedings of the 14th International Conference on Artificial Intelligence and Statistics. JMLR.org, 752760.Google ScholarGoogle Scholar
  47. [47] Williams Ronald J.. 1992. Simple statistical gradient-following algorithms for connectionist reinforcement learning. Machine learning 8 (1992), 229256.Google ScholarGoogle ScholarDigital LibraryDigital Library
  48. [48] Xu Guixian, Meng Yueting, Chen Zhan, Qiu Xiaoyu, Wang Changzhi, and Yao Haishen. 2019. Research on topic detection and tracking for online news texts. IEEE Access 7 (2019), 5840758418.Google ScholarGoogle ScholarCross RefCross Ref
  49. [49] Zhang Aonan, Zhu Jun, and Zhang Bo. 2013. Sparse online topic models. In Proceedings of the 22nd International World Wide Web Conference. 14891500.Google ScholarGoogle ScholarDigital LibraryDigital Library
  50. [50] Zhang Hao, Chen Bo, Cong Yulai, Guo Dandan, Liu Hongwei, and Zhou Mingyuan. 2020. Deep autoencoding topic model with scalable hybrid Bayesian inference. IEEE Transactions on Pattern Analysis and Machine Intelligence 43, 12 (2020), 122.Google ScholarGoogle Scholar
  51. [51] Zhang Rui, Pakhomov Serguei, Gladding Sophia, Aylward Michael, Borman-Shoap Emily, and Melton Genevieve. 2012. Automated assessment of medical training evaluation text. AMIA Annual Symposium Proceedings 2012 (2012), 1459–68.Google ScholarGoogle Scholar
  52. [52] Zhang Yaojie, Xu Bing, and Zhao Tiejun. 2020. Convolutional multi-head self-attention on memory for aspect sentiment classification. IEEE/CAA Journal of Automatica Sinica 7 (2020), 10381044.Google ScholarGoogle ScholarCross RefCross Ref
  53. [53] Zhu Jun and Xing Eric P.. 2011. Sparse topical coding. In Proceedings of the 27th Conference on Uncertainty in Artificial Intelligence. 831838.Google ScholarGoogle Scholar
  54. [54] Zuo Yuan, Li Congrui, Lin Hao, and Wu Junjie. 2021. Topic modeling of short texts: A pseudo-document view with word embedding enhancement. IEEE Transactions on Knowledge and Data Engineering Early Access (2021), 114.Google ScholarGoogle ScholarCross RefCross Ref

Index Terms

  1. Neural Topic Model Training with the REBAR Gradient Estimator

      Recommendations

      Comments

      Login options

      Check if you have access through your login credentials or your institution to get full access on this article.

      Sign in

      Full Access

      • Published in

        cover image ACM Transactions on Asian and Low-Resource Language Information Processing
        ACM Transactions on Asian and Low-Resource Language Information Processing  Volume 21, Issue 5
        September 2022
        486 pages
        ISSN:2375-4699
        EISSN:2375-4702
        DOI:10.1145/3533669
        Issue’s Table of Contents

        Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

        Publisher

        Association for Computing Machinery

        New York, NY, United States

        Publication History

        • Published: 15 November 2022
        • Online AM: 23 February 2022
        • Accepted: 9 February 2022
        • Revised: 14 November 2021
        • Received: 30 June 2021
        Published in tallip Volume 21, Issue 5

        Permissions

        Request permissions about this article.

        Request Permissions

        Check for updates

        Qualifiers

        • research-article
        • Refereed
      • Article Metrics

        • Downloads (Last 12 months)104
        • Downloads (Last 6 weeks)4

        Other Metrics

      PDF Format

      View or Download as a PDF file.

      PDF

      eReader

      View online with eReader.

      eReader

      Full Text

      View this article in Full Text.

      View Full Text

      HTML Format

      View this article in HTML Format .

      View HTML Format
      About Cookies On This Site

      We use cookies to ensure that we give you the best experience on our website.

      Learn more

      Got it!