skip to main content
research-article

Characterizing Subtle Facial Movements via Riemannian Manifold

Published:07 December 2019Publication History
Skip Abstract Section

Abstract

Characterizing subtle facial movements from videos is one of the most intensive topics in computer vision research. It is, however, challenging, since (1) the intensity of subtle facial muscle movement is usually low, (2) the duration may be transient, and (3) datasets containing spontaneous subtle movements with reliable annotations are painful to obtain and often of small sizes.

This article is targeted at addressing these problems for characterizing subtle facial movements from both the aspects of motion elucidation and description. First, we propose an efficient method for elucidating hidden and repressed movements to make them easier to get noticed. We explore the feasibility of linearizing motion magnification and temporal interpolation, which is obscured by the architecture of existing methods. On this basis, we propose a consolidated framework, termed MOTEL, to expand temporal duration and amplify subtle facial movements simultaneously. Second, we make our contribution to dynamic description. One major challenge is to capture the intrinsic temporal variations caused by movements and omit extrinsic ones caused by different individuals and various environments. To diminish the influences of such extrinsic diversity, we propose the tangent delta descriptor to characterize the dynamics of short-term movements using the differences between points on the tangent spaces to the manifolds, rather than the points themselves. We then relax the trajectory-smooth assumption of the conventional manifold-based trajectory modeling methods and incorporate the tangent delta descriptor with the sequential inference approaches to cover the period of facial movements. The proposed motion modeling approach is validated by a series of experiments on publicly available datasets in the tasks of micro-expression recognition and visual speech recognition.

References

  1. J. K. Aggarwal and M. S. Ryoo. 2011. Human activity analysis: A review. ACM Comput. Surv. 43, 3, Article 16 (Apr. 2011), 16:1--16:43.Google ScholarGoogle Scholar
  2. Stephan Bloehdorn and Andreas Hotho. 2004. Text classification by boosting weak learners based on terms and concepts. In Proceedings of the 4th IEEE International Conference on Data Mining (ICDM’04). IEEE, 331--334.Google ScholarGoogle ScholarCross RefCross Ref
  3. Christopher J. C. Burges. 1998. A tutorial on support vector machines for pattern recognition. Data Min. Knowl. Discov. 2, 2 (1998), 121--167.Google ScholarGoogle ScholarDigital LibraryDigital Library
  4. Peter J. Burt and Edward H. Adelson. 1987. The laplacian pyramid as a compact image code. In Readings in Computer Vision. Elsevier, 671--679.Google ScholarGoogle Scholar
  5. Yasuko Chikuse. 2006. State space models on special manifolds. J. Multivar. Anal. 97, 6 (2006), 1284--1294.Google ScholarGoogle ScholarDigital LibraryDigital Library
  6. Joon Son Chung, Andrew W. Senior, Oriol Vinyals, and Andrew Zisserman. 2017. Lip reading sentences in the wild. In Proceedings of the Conference on Computer Vision and Pattern Recognition (CVPR’17). 3444--3453.Google ScholarGoogle ScholarCross RefCross Ref
  7. Joon Son Chung and Andrew Zisserman. 2016. Lip reading in the wild. In Proceedings of the Asian Conference on Computer Vision. Springer, 87--103.Google ScholarGoogle Scholar
  8. Navneet Dalal and Bill Triggs. 2005. Histograms of oriented gradients for human detection. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. IEEE.Google ScholarGoogle ScholarDigital LibraryDigital Library
  9. Navneet Dalal, Bill Triggs, and Cordelia Schmid. 2006. Human detection using oriented histograms of flow and appearance. In Proceedings of the European Conference on Computer Vision (ECCV’06).Google ScholarGoogle ScholarDigital LibraryDigital Library
  10. Adrian K. Davison, Cliff Lansley, Nicholas Costen, Kevin Tan, and Moi Hoon Yap. 2018. Samm: A spontaneous micro-facial movement dataset. IEEE Trans. Affect. Comput. 9, 1 (2018), 116--129.Google ScholarGoogle ScholarDigital LibraryDigital Library
  11. P. Dollar, V. Rabaud, G. Cottrell, and S. Belongie. 2005. Behavior recognition via sparse spatio-temporal features. In Proceedings of the International Conference on Computer Communications and Networks (ICCCN’05).Google ScholarGoogle Scholar
  12. Paul Ekman and Erika L. Rosenberg. 1997. What the Face Reveals: Basic and Applied Studies of Spontaneous Expression Using the Facial Action Coding System. Oxford University Press.Google ScholarGoogle Scholar
  13. Mario Figueiredo and Anil Jain. 2002. Unsupervised learning of finite mixture models. IEEE Trans. Pattern Anal. Mach. Intell. 24, 3 (2002), 381--396.Google ScholarGoogle ScholarDigital LibraryDigital Library
  14. M. G. Frank, Malgorzata Herbasz, Kang Sinuk, A. Keller, and Courtney Nolan. 2009. I see how you feel: Training laypeople and professionals to recognize fleeting emotions. In Proceedings of the Annual Meeting of the International Communication Association.Google ScholarGoogle Scholar
  15. Gene H. Golub and Charles F. Van Loan. 2012. Matrix Computations. Vol. 3. JHU Press.Google ScholarGoogle Scholar
  16. Yusuf. Hakan Habiboglu, Osman Günay, and A. Enis Çetin. 2012. Covariance matrix-based fire and flame detection method in video.Mach. Vis. Appl. 23, 6 (2012), 1103--1113.Google ScholarGoogle ScholarCross RefCross Ref
  17. S. L. Happy and Aurobinda Routray. 2019. Fuzzy histogram of optical flow orientations for micro-expression recognition. IEEE Trans. Affect. Comput. 10, 3 (2019), 394--406.Google ScholarGoogle ScholarDigital LibraryDigital Library
  18. Mehrtash Harandi, Richard Hartley, Chunhua Shen, Brian Lovell, and Conrad Sanderson. 2015. Extrinsic methods for coding and dictionary learning on grassmann manifolds. Int. J. Comput. Vision 114, 2–3 (2015), 113--136.Google ScholarGoogle ScholarDigital LibraryDigital Library
  19. Mehrtash T. Harandi, Richard Hartley, Brian Lovell, and Conrad Sanderson. 2015. Sparse coding on symmetric positive definite manifolds using bregman divergences. IEEE Trans. Neural Netw. Learn. Syst. 27, 6 (2015), 1294--1306.Google ScholarGoogle ScholarCross RefCross Ref
  20. Xiaopeng Hong, Yingyue Xu, and Guoying Zhao. 2016. LBP-TOP: A tensor unfolding revisit. In Proceedings of the Asian Conference on Computer Vision. Springer, 513--527.Google ScholarGoogle Scholar
  21. X. Hong, G. Zhao, M. Pietikäinen, and X. Chen. 2014. Combining LBP difference and feature correlation for texture description. IEEE Trans. Image Process. 23, 6 (June 2014), 2557--2568.Google ScholarGoogle Scholar
  22. Xiaopeng Hong, Guoying Zhao, Stefanos Zafeiriou, Maja Pantic, and Matti Pietikäinen. 2016. Capturing correlations of local features for image representation. Neurocomputing 184 (2016), 99--106.Google ScholarGoogle ScholarDigital LibraryDigital Library
  23. Xiaohua Huang, Su-Jing Wang, Xin Liu, Guoying Zhao, Xiaoyi Feng, and Matti Pietikäinen. 2019. Discriminative spatiotemporal local binary pattern with revisited integral projection for spontaneous facial micro-expression recognition. IEEE Trans. Affect. Comput. 10, 1 (2019), 32--47.Google ScholarGoogle ScholarDigital LibraryDigital Library
  24. Xiaohua Huang and Guoying Zhao. 2017. Spontaneous facial micro-expression analysis using spatiotemporal local radon-based binary pattern. In Proceedings of the International Conference on the Frontiers and Advances in Data Science (FADS’17). IEEE.Google ScholarGoogle ScholarCross RefCross Ref
  25. X. Huang, G. Zhao, X. Hong, W. Zheng, and M. Pietikäinen. 2016. Spontaneous facial micro-expression analysis using spatiotemporal completed local quantized patterns. Neurocomputing 175 (2016), 564--578.Google ScholarGoogle ScholarDigital LibraryDigital Library
  26. Yan Ke, Rahul Sukthankar, and Martial Hebert. 2005. Efficient visual event detection using volumetric features. In Proceedings of the International Conference on Computer Vision (ICCV’05).Google ScholarGoogle Scholar
  27. Huai-Qian Khor, John See, Raphael C.-W. Phan, and Weiyao Lin. 2018. Enriched long-term recurrent convolutional network for facial micro-expression recognition. In Proceedings of the International Conference on Automatic Face and Gesture Recognition.Google ScholarGoogle Scholar
  28. Alexander Kläser, Marcin Marszałek, and Cordelia Schmid. 2008. A spatio-temporal descriptor based on 3D-gradients. In Proceedings of the British Machine Vision Conference (BMVC’08).Google ScholarGoogle ScholarCross RefCross Ref
  29. X. Lan, M. Ye, R. Shao, B. Zhong, P. C. Yuen, and H. Zhou. 2019. Learning modality-consistency feature templates: A robust RGB-infrared tracking system. IEEE Trans. Industr. Electron. (2019), 1--1.Google ScholarGoogle Scholar
  30. X. Lan, S. Zhang, P. C. Yuen, and R. Chellappa. 2018. Learning common and feature-specific patterns: A novel multiple-sparse-representation-based tracker. IEEE Trans. Image Process. 27, 4 (Apr. 2018), 2022--2037.Google ScholarGoogle ScholarCross RefCross Ref
  31. Ivan Laptev. 2005. On space-time interest points. Int. J. Comput. Vision 64, 2–3 (2005), 107--123.Google ScholarGoogle ScholarDigital LibraryDigital Library
  32. Ivan Laptev and Patrick Pérez. 2007. Retrieving actions in movies. Proceedings of the International Conference on Computer Vision (ICCV’07).Google ScholarGoogle ScholarCross RefCross Ref
  33. Christopher J. Leggetter and Philip C. Woodland. 1995. Maximum likelihood linear regression for speaker adaptation of continuous density hidden Markov models. Comput. Speech Lang. 9, 2 (1995), 171--185.Google ScholarGoogle ScholarCross RefCross Ref
  34. Shan Li and Weihong Deng. 2019. Reliable crowdsourcing and deep locality-preserving learning for unconstrained facial expression recognition. IEEE Trans. Image Process. 28, 1 (2019), 356--370.Google ScholarGoogle ScholarDigital LibraryDigital Library
  35. Xiaobai Li, Xiaopeng Hong, Antti Moilanen, Xiaohua Huang, Tomas Pfister, Guoying Zhao, and Matti Pietikäinen. 2015. Reading hidden emotions: Spontaneous micro-expression spotting and recognition. Arxiv:1511.00423 (2015).Google ScholarGoogle Scholar
  36. X. Li, X. Hong, A. Moilanen, X. Huang, T. Pfister, G. Zhao, and M. Pietikäinen. 2018. Towards reading hidden emotions: A comparative study of spontaneous micro-expression spotting and recognition methods. IEEE Trans. Affect. Comput. 9, 4 (Oct. 2018), 563--577.Google ScholarGoogle ScholarDigital LibraryDigital Library
  37. Xiaobai Li, Tomas Pfister, Xiaohua Huang, Guoying Zhao, and Matti Pietikäinen. 2013. A spontaneous micro-expression database: Inducement, collection and baseline. In Proceedings of the IEEE International Conference on Automatic Face and Gesture Recognition.Google ScholarGoogle ScholarCross RefCross Ref
  38. Yante Li, Xiaohua Huang, and Guoying Zhao. 2018. Can micro-expression be recognized based on single apex frame?. In Proceedings of the 25th IEEE International Conference on Image Processing (ICIP’18). IEEE, 3094--3098.Google ScholarGoogle ScholarCross RefCross Ref
  39. Ming Liang and Xiaolin Hu. 2015. Recurrent convolutional neural network for object recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 3367--3375.Google ScholarGoogle ScholarCross RefCross Ref
  40. Sze-Teng Liong, Raphael C.-W. Phan, John See, Yee-Hui Oh, and KokSheik Wong. 2014. Optical strain-based recognition of subtle emotions. In Proceedings of the International Symposium on Intelligent Signal Processing and Communication Systems. IEEE.Google ScholarGoogle Scholar
  41. Sze-Teng Liong, John See, Raphael C.-W. Phan, Anh Cat Le Ngo, Yee-Hui Oh, and KokSheik Wong. 2014. Subtle expression recognition using optical strain weighted features. In Proceedings of the Asian Conference on Computer Vision. Springer.Google ScholarGoogle Scholar
  42. Sze-Teng Liong, John See, KokSheik Wong, and Raphael C.-W. Phan. 2018. Less is more: Micro-expression recognition from video using apex frame. Signal Process.: Image Commun. 62 (2018), 82--92.Google ScholarGoogle ScholarCross RefCross Ref
  43. Ce Liu, Antonio Torralba, William T. Freeman, Frédo Durand, and Edward H. Adelson. 2005. Motion magnification. ACM Trans. Graph. 24, 3 (2005), 519--526.Google ScholarGoogle ScholarDigital LibraryDigital Library
  44. M. Liu, S. Shan, R. Wang, and X. Chen. 2016. Learning expressionlets via universal manifold model for dynamic facial expression recognition. IEEE Trans. Image Process. 25, 12 (Dec. 2016), 5920--5932.Google ScholarGoogle ScholarDigital LibraryDigital Library
  45. M. Liu, R. Wang, S. Li, S. Shan, Z. Huang, and X. Chen. 2014. Combining multiple kernel methods on riemannian manifold for emotion recognition in the wild. In Proceedings of the International Conference on Multimodal Interaction. ACM.Google ScholarGoogle Scholar
  46. Y.-J. Liu, J.-K. Zhang, W.-J. Yan, S.-J. Wang, G. Zhao, and X. Fu. 2016. A main directional mean optical flow feature for spontaneous micro-expression recognition. IEEE Trans. Affect. Comput. 7, 4 (2016), 299--310.Google ScholarGoogle ScholarDigital LibraryDigital Library
  47. Y. Lui. 2012. Advances in matrix manifolds for computer vision.Image Vision Comput. 30, 6–7 (2012), 380--388.Google ScholarGoogle Scholar
  48. K. Mase and A. Pentland. 1989. Automatic Lipreading by Optical-flow Analysis. Wiley.Google ScholarGoogle Scholar
  49. Iain Matthews, Tim Cootes, J. Andrew Bangham, Stephen Cox, and Richard Harvey. 2002. Extraction of visual features for lipreading. IEEE Trans. Pattern Anal. Mach. Intell. 24 (2002), 198--213.Google ScholarGoogle ScholarDigital LibraryDigital Library
  50. Veena Mayya, Radhika Pai, and Manohara Pai. 2016. Combining temporal interpolation and DCNN for faster recognition of micro-expressions in video sequences. In Proceedings of the International Conference on Advances in Computing, Communications and Informatics. IEEE.Google ScholarGoogle ScholarCross RefCross Ref
  51. AraV. Nefian, Luhong Liang, Xiaobo Pi, Xiaoxing Liu, and Kevin Murphy. 2002. Dynamic Bayesian networks for audio-visual speech recognition. EURASIP J. Appl. Signal Process. 2002, 1 (2002), 1274--1288.Google ScholarGoogle ScholarDigital LibraryDigital Library
  52. Jacob L. Newman and Stephen J. Cox. 2012. Language identification using visual features. IEEE Audio, Speech, Language Process. 20, 7 (2012), 1936--1947.Google ScholarGoogle ScholarDigital LibraryDigital Library
  53. JuanCarlos Niebles, Hongcheng Wang, and Li Fei-Fei. 2008. Unsupervised learning of human action categories using spatial-temporal words. Int. J. Comput. Vision 79, 3 (Sept. 2008), 299--318.Google ScholarGoogle ScholarDigital LibraryDigital Library
  54. Eng-Jon Ong and Richard Bowden. 2011. Learning temporal signatures for lip reading. In Proceedings of the International Conference on Computer Vision (ICCV’11). IEEE, 958--965.Google ScholarGoogle ScholarCross RefCross Ref
  55. Sungsoo Park and Daijin Kim. 2009. Subtle facial expression recognition using motion magnification. Pattern Recogn. Lett. 30, 7 (2009), 708--716.Google ScholarGoogle ScholarDigital LibraryDigital Library
  56. Sung Yeong Park, Seung Ho Lee, and Yong Man Ro. 2015. Subtle facial expression recognition using adaptive magnification of discriminative facial motion. In Proceedings of the 23rd ACM International Conference on Multimedia. ACM, 911--914.Google ScholarGoogle ScholarDigital LibraryDigital Library
  57. Devangini Patel, Xiaopeng Hong, and Guoying Zhao. 2016. Selective deep features for micro-expression recognition. In Proceedings of the 23rd International Conference on Pattern Recognition (ICPR’16). IEEE, 2258--2263.Google ScholarGoogle Scholar
  58. Yuru Pei, Tae-Kyun Kim, and Hongbin Zha. 2013. Unsupervised random forest manifold alignment for lipreading. In Proceedings of the IEEE International Conference on Computer Vision. 129--136.Google ScholarGoogle ScholarDigital LibraryDigital Library
  59. Min Peng, Chongyang Wang, Tong Chen, Guangyuan Liu, and Xiaolan Fu. 2017. Dual temporal scale convolutional neural network for micro-expression recognition. Front. Psychol. 8 (2017), 1745.Google ScholarGoogle ScholarCross RefCross Ref
  60. Wei Peng, Xiaopeng Hong, Yingyue Xu, and Guoying Zhao. 2019. A boost in revealing subtle facial expressions: A consolidated eulerian framework. In Proceedings of the IEEE International Conference on Automatic face and Gesture Recognition. IEEE.Google ScholarGoogle ScholarCross RefCross Ref
  61. Xavier Pennec. 2006. Intrinsic statistics on Riemannian manifolds: Basic tools for geometric measurements. J. Math. Imag. Vis. 25, 1 (2006), 127--154.Google ScholarGoogle ScholarDigital LibraryDigital Library
  62. Xavier Pennec, Pierre Fillard, and Nicholas Ayache. 2006. A riemannian framework for tensor computing. Int. J. Comput. Vision 66 (2006), 41--66.Google ScholarGoogle ScholarDigital LibraryDigital Library
  63. Tomas Pfister, Xiaobai Li, Guoying Zhao, and Matti Pietikäinen. 2011. Recognising spontaneous facial micro-expressions. In Proceedings of the IEEE International Conference on Computer Vision (ICCV’11). IEEE, 1449--1456.Google ScholarGoogle ScholarDigital LibraryDigital Library
  64. Senya Polikovsky, Yoshinari Kameda, and Yuichi Ohta. 2009. Facial micro-expressions recognition using high speed camera and 3D-gradient descriptor. In Proceedings of the 3rd International Conference on Imaging for Crime Detection and Prevention (ICDP’09). 1--6. DOI:10.1049/ic.2009.0244Google ScholarGoogle ScholarCross RefCross Ref
  65. Gerasimos Potamianos, Hans Peter Graf, and Eric Cosatto. 1998. An image transform approach for HMM-based automatic lipreading. In Proceedings of the IEEE International Conference on Image Processing (ICIP’98).Google ScholarGoogle ScholarCross RefCross Ref
  66. Gerasimos Potamianos, Chalapathy Neti, Guillaume Gravier, Ashutosh Garg, and Andrew W. Senior. 2003. Recent advances in the automatic recognition of audio-visual speech. In Proceedings of IEEE.Google ScholarGoogle Scholar
  67. Kate Saenko, Karen Livescu, James R. Glass, and Trevor Darrell. 2009. Multistream articulatory feature-based models for visual speech recognition. IEEE Trans. Pattern Anal. Mach. Intell. 31, 9 (2009), 1700--1707.Google ScholarGoogle ScholarDigital LibraryDigital Library
  68. Jingyong Su, Anuj Srivastava, Fillipe DM de Souza, and Sudeep Sarkar. 2014. Rate-invariant analysis of trajectories on riemannian manifolds with application in visual speech recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 620--627.Google ScholarGoogle ScholarDigital LibraryDigital Library
  69. Madhumita A. Takalkar and Min Xu. 2017. Image-based facial micro-expression recognition using deep learning on small datasets. In Proceedings of the International Conference on Digital Image Computing: Techniques and Applications (DICTA’17). IEEE.Google ScholarGoogle Scholar
  70. Oncel Tuzel, Fatih Porikli, and Peter Meer. 2006. Region covariance: A fast descriptor for detection and classification. In Proceedings of the of European Conference on Computer Vision (ECCV’06).Google ScholarGoogle ScholarDigital LibraryDigital Library
  71. Oncel Tuzel, Fatih Porikli, and Peter Meer. 2008. Learning on lie groups for invariant detection and tracking. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR’08).Google ScholarGoogle ScholarCross RefCross Ref
  72. Oncel Tuzel, Fatih Porikli, and Peter Meer. 2008. Pedestrian detection via classification on Riemannian manifolds. IEEE Trans. Pattern Anal. Mach. Intell. 30, 10 (2008), 1713--1727.Google ScholarGoogle ScholarDigital LibraryDigital Library
  73. Hanjie Wang, Xiujuan Chai, Xiaopeng Hong, Guoying Zhao, and Xilin Chen. 2016. Isolated sign language recognition with grassmann covariance matrices. ACM Trans. Access. Comput. 8, 4, Article 14 (May 2016).Google ScholarGoogle ScholarDigital LibraryDigital Library
  74. Heng Wang, Muhammad Muneeb Ullah, Alexander Kläser, Ivan Laptev, and Cordelia Schmid. 2009. Evaluation of local spatio-temporal features for action recognition. In Proceedings of the British Machine Vision Conference (BMVC’09).Google ScholarGoogle ScholarCross RefCross Ref
  75. S. L. Wang, A. W. C. Liew, W. H. Lau, and S. H. Leung. 2008. An automatic lipreading system for spoken digits with limited training data. IEEE Trans. Circ. Syst. Video Techn. 18, 12 (2008), 1760--1765.Google ScholarGoogle ScholarDigital LibraryDigital Library
  76. Su-Jing Wang, Wen-Jing Yan, Xiaobai Li, Guoying Zhao, Chun-Guang Zhou, Xiaolan Fu, Minghao Yang, and Jianhua Tao. 2015. Micro-expression recognition using color spaces. IEEE Trans. Image Process. 24, 12 (2015), 6034--6047.Google ScholarGoogle ScholarDigital LibraryDigital Library
  77. Xiaofan Wei, Huibin Li, Jian Sun, and Liming Chen. 2018. Unsupervised domain adaptation with regularized optimal transport for multimodal 2D+ 3D facial expression recognition. In Proceedings of the IEEE International Conference on Automatic Face and Gesture Recognition. IEEE, 31--37.Google ScholarGoogle ScholarCross RefCross Ref
  78. Xing Wei, Yue Zhang, Yihong Gong, Jiawei Zhang, and Nanning Zheng. 2018. Grassmann pooling as compact homogeneous bilinear pooling for fine-grained visual classification. In Proceedings of the European Conference on Computer Vision.Google ScholarGoogle ScholarCross RefCross Ref
  79. Xing Wei, Yue Zhang, Yihong Gong, and Nanning Zheng. 2018. Kernelized subspace pooling for deep local descriptors. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR’18).Google ScholarGoogle ScholarCross RefCross Ref
  80. HaoYu Wu, Michael Rubinstein, Eugene Shih, John Guttag, Frédo Durand, and William T. Freeman. 2012. Eulerian video magnification for revealing subtle changes in the world. ACM Trans. Graph. 31, 4 (2012).Google ScholarGoogle ScholarDigital LibraryDigital Library
  81. Zhaoqiang Xia, Xiaoyi Feng, Xiaopeng Hong, and Guoying Zhao. 2018. Spontaneous facial micro-expression recognition via deep convolutional network. In Proceedings of the International Conference on Image Processing Theory, Tools, and Applications.Google ScholarGoogle ScholarCross RefCross Ref
  82. Xiaopeng Hong, Hong Chang, Shiguang Shan, Xilin Chen, and Wen Gao. 2009. Sigma set: A small second order statistical region descriptor. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 1802--1809.Google ScholarGoogle Scholar
  83. Wen-Jing Yan, Xiaobai Li, Su-Jing Wang, Guoying Zhao, Yong-Jin Liu, Yu-Hsin Chen, and Xiaolan Fu. 2014. CASME II: An improved spontaneous micro-expression database and the baseline evaluation. PloS One 9, 1 (2014).Google ScholarGoogle Scholar
  84. Yichao Zhang, Silvia L. Pintea, and Jan C. van Gemert. 2017. Video acceleration magnification. Arxiv Preprint Arxiv:1704.04186 (2017).Google ScholarGoogle Scholar
  85. Zhengwu Zhang, Jingyong Su, Eric Klassen, Huiling Le, and Anuj Srivastava. 2015. Video-based action recognition using rate-invariant analysis of covariance trajectories. Arxiv Preprint Arxiv:1503.06699 (2015).Google ScholarGoogle Scholar
  86. Guoying Zhao, Mark Barnard, and Matti Pietikäinen. 2009. Lipreading with local spatiotemporal descriptors. IEEE Trans. Multimedia 11, 7 (2009), 1254--1265.Google ScholarGoogle ScholarDigital LibraryDigital Library
  87. Guoying Zhao and Matti Pietikäinen. 2007. Dynamic texture recognition using local binary patterns with an application to facial expressions. IEEE Trans. Pattern Anal. Mach. Intell. 29, 6 (2007), 915--928.Google ScholarGoogle ScholarDigital LibraryDigital Library
  88. S. Zhao, G. Ding, Y. Gao, and J. Han. 2017. Approximating discrete probability distribution of image emotions by multi-modal features fusion. In Proceedings of the International Joint Conference on Artificial Intelligence.Google ScholarGoogle Scholar
  89. S. Zhao, G. Ding, Y. Gao, X. Zhao, Y. Tang, J. Han, H. Yao, and Q. Huang. 2018. Discrete probability distribution prediction of image emotions with shared sparse learning. IEEE Trans. Affect. Comput. Early Access. DOI:10.1109/TAFFC.2018.2818685Google ScholarGoogle ScholarDigital LibraryDigital Library
  90. Sicheng Zhao, Guiguang Ding, Qingming Huang, Tat-Seng Chua, Björn W Schuller, and Kurt Keutzer. 2018. Affective image content analysis: A comprehensive survey. In Proceedings of the International Joint Conferences on Artificial Intelligence (IJCAI’18). 5534--5541.Google ScholarGoogle ScholarCross RefCross Ref
  91. Sicheng Zhao, Hongxun Yao, Yue Gao, Rongrong Ji, and Guiguang Ding. 2017. Continuous probability distribution prediction of image emotions via multitask shared sparse regression. IEEE Trans. Multimedia 19, 3 (2017), 632--645.Google ScholarGoogle ScholarDigital LibraryDigital Library
  92. Z. Zhou, X. Hong, G. Zhao, and M. Pietikäinen. 2014. A compact representation of visual speech data using latent variables. IEEE Trans. Pattern Anal. Mach. Intell. 36, 1 (Jan. 2014), 1--1.Google ScholarGoogle ScholarDigital LibraryDigital Library
  93. Ziheng Zhou, Guoying Zhao, Xiaopeng Hong, and Matti Pietikäinen. 2014. A review of recent advances in visual speech decoding. Image Vision Comput. 32, 9 (2014), 590--605.Google ScholarGoogle ScholarCross RefCross Ref
  94. Z. Zhou, G. Zhao, and M. Pietikäinen. 2011. Towards a practical lipreading system. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR’11).Google ScholarGoogle Scholar
  95. Ziheng Zhou, Guoying Zhao, and Matti Pietikäinen. 2011. Towards a practical lipreading system. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR’11). IEEE, 137--144.Google ScholarGoogle ScholarDigital LibraryDigital Library
  96. Yuan Zong, Xiaohua Huang, Wenming Zheng, Zhen Cui, and Guoying Zhao. 2018. Learning from hierarchical spatiotemporal descriptors for micro-expression recognition. IEEE Trans. Multimedia 20, 11 (2018), 3160--3172. DOI:10.1109/TMM.2018.2820321Google ScholarGoogle ScholarDigital LibraryDigital Library

Index Terms

  1. Characterizing Subtle Facial Movements via Riemannian Manifold

      Recommendations

      Comments

      Login options

      Check if you have access through your login credentials or your institution to get full access on this article.

      Sign in

      Full Access

      • Published in

        cover image ACM Transactions on Multimedia Computing, Communications, and Applications
        ACM Transactions on Multimedia Computing, Communications, and Applications  Volume 15, Issue 3s
        Special Issue on Face Analysis for Applications and Special Issue on Affective Computing for Large-Scale Heterogeneous Multimedia Data
        November 2019
        304 pages
        ISSN:1551-6857
        EISSN:1551-6865
        DOI:10.1145/3368027
        Issue’s Table of Contents

        Copyright © 2019 ACM

        Publisher

        Association for Computing Machinery

        New York, NY, United States

        Publication History

        • Published: 7 December 2019
        • Revised: 1 June 2019
        • Accepted: 1 June 2019
        • Received: 1 January 2019
        Published in tomm Volume 15, Issue 3s

        Permissions

        Request permissions about this article.

        Request Permissions

        Check for updates

        Qualifiers

        • research-article
        • Research
        • Refereed

      PDF Format

      View or Download as a PDF file.

      PDF

      eReader

      View online with eReader.

      eReader

      HTML Format

      View this article in HTML Format .

      View HTML Format
      About Cookies On This Site

      We use cookies to ensure that we give you the best experience on our website.

      Learn more

      Got it!