skip to main content
research-article

Distance and Direction Based Deep Discriminant Metric Learning for Kinship Verification

Published:23 January 2023Publication History
Skip Abstract Section

Abstract

Image-based kinship verification is an important task in computer vision and has many applications in practice, such as missing children search and family album construction, among others. Due to the differences in age, gender, expression and appearance, there usually exists a large discrepancy between the facial images of parent and child. This makes kinship verification a challenging task. In this article, we propose a Distance and Direction Based Deep Discriminant Metric Learning (D4ML) approach for kinship verification. The basic idea of D4ML is to make full use of the discriminant information contained in the facial images of parent and child such that the network can learn more a discriminating distance metric. Specifically, D4ML learns the metric by utilizing the discriminant information from two perspectives: distance-based perspective and direction-based perspective. From the distance-based perspective, the designed loss function is used to minimize the distance between images having kinship and maximize the distance between images without kinship. In practice, the gender difference and large age gap may significantly increase the distance between facial images of parent and child. Therefore, learning the metric only from a distance-based perspective is insufficient. Considering that two vectors with a large distance may appear with high similarity in direction, D4ML also employs the direction-based loss function in the training process. Both kinds of loss function work together to improve the discriminability of the learned metric. Experimental results on four small size publicly available datasets demonstrate the effectiveness of our approach. Source code of our approach can be found at https://github.com/lclhenu/D4ML.

REFERENCES

  1. [1] Chen Xiaopan, Li Changlong, Zhu Xiaoke, Zheng Liang, Chen Ya, Zheng Shanshan, and Yuan Caihong. 2022. Deep discriminant generation-shared feature learning for image-based kinship verification. Signal Processing: Image Communication 101 (2022), 116543.Google ScholarGoogle ScholarDigital LibraryDigital Library
  2. [2] Chen Xiaopan, Zhu Xiaoke, Zheng Shanshan, Zheng Taihao, and Zhang Fan. 2021. Semi-coupled synthesis and analysis dictionary pair learning for kinship verification. IEEE Transactions on Circuits and Systems for Video Technology 31, 5 (2021), 19391952.Google ScholarGoogle ScholarDigital LibraryDigital Library
  3. [3] Dahan Eran, Keller Yosi, and Mahpod Shahar. 2017. Kin-verification model on FIW dataset using multi-set learning and local features. In Proceedings of the 2017 Workshop on Recognizing Families in the Wild (RFIW’17). 31–35.Google ScholarGoogle ScholarDigital LibraryDigital Library
  4. [4] Dawson Mitchell, Zisserman Andrew, and Nellåker Christoffer. 2018. From same photo: Cheating on visual kinship challenges. In Proceedings of the Asian Conference on Computer Vision (ACCV’18), Vol. 11363. 654668.Google ScholarGoogle Scholar
  5. [5] Debruine Lisa M., Jones Benedict C., Little Anthony C., and Perrett David I.. 2008. Social perception of facial resemblance in humans. Archives of Sexual Behavior 37, 1 (2008), 6477.Google ScholarGoogle ScholarCross RefCross Ref
  6. [6] Dehghan Afshin, Ortiz Enrique G., Villegas Ruben, and Shah Mubarak. 2014. Who do I look like? Determining parent-offspring resemblance via gated autoencoders. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR’14). 17571764.Google ScholarGoogle ScholarDigital LibraryDigital Library
  7. [7] Dehshibi Mohammad Mahdi and Shanbehzadeh Jamshid. 2019. Cubic norm and kernel-based bi-directional PCA: Toward age-aware facial kinship verification. Visual Computer 35, 1 (2019), 2340.Google ScholarGoogle ScholarDigital LibraryDigital Library
  8. [8] Dibeklioglu Hamdi. 2017. Visual transformation aided contrastive learning for video-based kinship verification. In Proceedings of the IEEE International Conference on Computer Vision (ICCV’17). 24782487.Google ScholarGoogle ScholarCross RefCross Ref
  9. [9] Dibeklioglu Hamdi, Salah Albert Ali, and Gevers Theo. 2013. Like father, like son: Facial expression dynamics for kinship verification. In Proceedings of the IEEE International Conference on Computer Vision (ICCV’13). 14971504.Google ScholarGoogle ScholarDigital LibraryDigital Library
  10. [10] Ding Zhengming and Fu Yun. 2017. Robust transfer metric learning for image classification. IEEE Transactions on Image Processing 26, 2 (2017), 660670.Google ScholarGoogle ScholarDigital LibraryDigital Library
  11. [11] Fang Ruogu, Tang Kevin D., Snavely Noah, and Chen Tsuhan. 2010. Towards computational models of kinship verification. In Proceedings of the International Conference on Image Processing (ICIP’10). 15771580.Google ScholarGoogle ScholarCross RefCross Ref
  12. [12] Guo Guodong and Wang Xiaolong. 2012. Kinship measurement on salient facial features. IEEE Transactions on Instrumentation and Measurement 61, 8 (2012), 23222325.Google ScholarGoogle ScholarCross RefCross Ref
  13. [13] Jain Apoorv, Bhagat Naman, Srivastava Varun, Tyagi Priyanshu, and Jain Pragya. 2020. A feature-based kinship verification technique using convolutional neural network. In Advances in Data Sciences, Security and Applications. 353362.Google ScholarGoogle Scholar
  14. [14] Kohli Naman, Vatsa Mayank, Singh Richa, Noore Afzel, and Majumdar Angshul. 2017. Hierarchical representation learning for kinship verification. IEEE Transactions on Image Processing 26, 1 (2017), 289302.Google ScholarGoogle ScholarDigital LibraryDigital Library
  15. [15] Kou Lu, Zhou Xiuzhuang, Xu Min, and Shang Yuanyuan. 2015. Learning a genetic measure for kinship verification using facial images. Mathematical Problems in Engineering 2015, Pt. 1 (2015), 15.Google ScholarGoogle Scholar
  16. [16] Laiadi Oualid, Ouamane Abdelmalik, Benakcha Abdelhamid, Taleb-Ahmed Abdelmalik, and Hadid Abdenour. 2019. Kinship verification based deep and tensor features through extreme learning machine. In Proceedings of the IEEE International Conference on Automatic Face and Gesture Recognition. 14.Google ScholarGoogle ScholarDigital LibraryDigital Library
  17. [17] Li Lei, Feng Xiaoyi, Wu Xiaoting, Xia Zhaoqiang, and Hadid Abdenour. 2016. Kinship verification from faces via similarity metric based convolutional neural network. In Proceedings of the International Conference on Image Analysis and Recognition, Vol. 9730. 539548.Google ScholarGoogle ScholarCross RefCross Ref
  18. [18] Li Lei, Feng Xiaoyi, Wu Xiaoting, Xia Zhaoqiang, and Hadid Abdenour. 2016. Kinship verification from faces via similarity metric based convolutional neural network. In Proceedings of the International Conference on Image Analysis and Recognition (ICIAR’16), Vol. 9730. 539548.Google ScholarGoogle ScholarCross RefCross Ref
  19. [19] Li Wanhua, Lu Jiwen, Wuerkaixi Abudukelimu, Feng Jianjiang, and Zhou Jie. 2021. Reasoning graph networks for kinship verification: From star-shaped to hierarchical. IEEE Transactions on Image Processing 30 (2021), 49474961.Google ScholarGoogle ScholarCross RefCross Ref
  20. [20] Li Wanhua, Wang Shiwei, Lu Jiwen, Feng Jianjiang, and Zhou Jie. 2021. Meta-mining discriminative samples for kinship verification. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR’21). 1613516144.Google ScholarGoogle ScholarCross RefCross Ref
  21. [21] Liang Jingyun, Guo Jinlin, Lao Songyang, and Li Jue. 2017. Using deep relational features to verify kinship. In Proceedings of the CCF Chinese Conference on Computer Vision, Vol. 771. 563573.Google ScholarGoogle ScholarCross RefCross Ref
  22. [22] Liang Jianqing, Hu Qinghua, Dang Chuangyin, and Zuo Wangmeng. 2019. Weighted graph embedding-based metric learning for kinship verification. IEEE Transactions on Image Processing 28, 3 (2019), 11491162.Google ScholarGoogle ScholarDigital LibraryDigital Library
  23. [23] Liu Qingfeng, Puthenputhussery Ajit, and Liu Chengjun. 2015. Inheritable Fisher vector feature for kinship verification. In Proceedings of the IEEE International Conference on Biometrics Theory, Applications, and Systems (BTAS’15). 16.Google ScholarGoogle ScholarDigital LibraryDigital Library
  24. [24] Lu Jiwen, Hu Junlin, and Tan Yap-Peng. 2017. Discriminative deep metric learning for face and kinship verification. IEEE Transactions on Image Processing 26, 9 (2017), 42694282.Google ScholarGoogle ScholarDigital LibraryDigital Library
  25. [25] Lu Jiwen, Zhou Xiuzhuang, Tan Yap-Peng, Shang Yuanyuan, and Zhou Jie. 2014. Neighborhood repulsed metric learning for kinship verification. IEEE Transactions on Pattern Analysis and Machine Intelligence 36, 2 (2014), 331345.Google ScholarGoogle ScholarDigital LibraryDigital Library
  26. [26] Mahalanobis P. C.. 1936. On the generalised distance in statistics. Proceedings of the National Institute of Sciences of India 2 (1936), 4955.Google ScholarGoogle Scholar
  27. [27] Nandy Abhilash and Mondal Shanka Subhra. 2019. Kinship verification using deep Siamese convolutional neural network. In Proceedings of the IEEE International Conference on Automatic Face and Gesture Recognition. 15.Google ScholarGoogle ScholarDigital LibraryDigital Library
  28. [28] Nguyen Hieu V. and Bai Li. 2010. Cosine similarity metric learning for face verification. In Proceedings of the Asian Conference on Computer Vision (ACCV’10), Vol. 6493. 709720.Google ScholarGoogle Scholar
  29. [29] Parkhi Omkar M., Vedaldi Andrea, and Zisserman Andrew. 2015. Deep face recognition. In Proceedings of the British Machine Vision Conference (BMVC’15). 41.1–41.12.Google ScholarGoogle ScholarCross RefCross Ref
  30. [30] Rachmadi Reza Fuad, Purnama I. Ketut Eddy, Nugroho Supeno Mardi Susiki, and Suprapto Yoyon Kusnendar. 2019. Image-based kinship verification using fusion convolutional neural network. In Proceedings of the IEEE International Workshop on Computational Intelligence and Applications (IWCIA’19). 5965.Google ScholarGoogle ScholarCross RefCross Ref
  31. [31] Robinson Joseph P., Shao Ming, Wu Yue, and Fu Yun. 2016. Families in the wild (FIW): Large-scale kinship image database and benchmarks. In Proceedings of the ACM Conference on Multimedia (MM’16). 242246.Google ScholarGoogle ScholarDigital LibraryDigital Library
  32. [32] Robinson Joseph P., Shao Ming, Wu Yue, and Fu Yun. 2016. Family in the wild (FIW): A large-scale kinship recognition database. CoRR abs/1604.02182 (2016).Google ScholarGoogle Scholar
  33. [33] Robinson Joseph P., Shao Ming, Wu Yue, Liu Hongfu, Gillis Timothy, and Fu Yun. 2018. Visual kinship recognition of families in the wild. IEEE Transactions on Pattern Analysis and Machine Intelligence 40, 11 (2018), 26242637.Google ScholarGoogle ScholarDigital LibraryDigital Library
  34. [34] Sellam Abdellah and Hamid Azzoune. 2018. Linear feature learning for kinship verification in the wild. In Proceedings of the International Conference on Applied Smart Systems (ICASS’18).Google ScholarGoogle ScholarCross RefCross Ref
  35. [35] Shao Ming, Xia Si-Yu, and Fu Yun. 2011. Genealogical face recognition based on UB KinFace database. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR’11). 6065.Google ScholarGoogle ScholarCross RefCross Ref
  36. [36] Sharma Abhishek, Kumar Abhishek, III Hal Daumé, and Jacobs David W.. 2012. Generalized multiview analysis: A discriminative latent space. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR’12). 21602167.Google ScholarGoogle ScholarCross RefCross Ref
  37. [37] Sun Yifan, Xu Qin, Li Yali, Zhang Chi, Li Yikang, Wang Shengjin, and Sun Jian. 2019. Perceive where to focus: Learning visibility-aware part-level features for partial person re-identification. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR’19). 393402.Google ScholarGoogle ScholarCross RefCross Ref
  38. [38] Sun Yifan, Zheng Liang, Yang Yi, Tian Qi, and Wang Shengjin. 2018. Beyond part models: Person retrieval with refined part pooling (and a strong convolutional baseline). In Computer Vision—(ECCV’18). Lecture Notes in Computer Science, Vol. 11208. Springer, 501518.Google ScholarGoogle Scholar
  39. [39] Tidjani Amina, Taleb-Ahmed Abdelmalik, Samai Djamel, and Aiadi Kamal Eddine. 2018. Deep learning features for robust facial kinship verification. IET Image Processing 12, 12 (2018), 23362345.Google ScholarGoogle ScholarCross RefCross Ref
  40. [40] Vieira Tiago F., Bottino Andrea, Laurentini Aldo, and Simone Matteo De. 2014. Detecting siblings in image pairs. Visual Computer 30, 12 (2014), 13331345.Google ScholarGoogle ScholarDigital LibraryDigital Library
  41. [41] Wang Liwei, Zhang Yan, and Feng Jufu. 2005. On the Euclidean distance of images. IEEE Transactions on Pattern Analysis and Machine Intelligence 27, 8 (2005), 13341339.Google ScholarGoogle ScholarDigital LibraryDigital Library
  42. [42] Wang Mengyin, Li Zechao, Shu Xiangbo, Wang Jingdong, and Tang Jinhui. 2015. Deep kinship verification. In Proceedings of the IEEE International Workshop on Multimedia Signal Processing (MMSP’15). 16.Google ScholarGoogle ScholarCross RefCross Ref
  43. [43] Wang Shuyang, Ding Zhengming, and Fu Yun. 2019. Cross-generation kinship verification with sparse discriminative metric. IEEE Transactions on Pattern Analysis and Machine Intelligence 41, 11 (2019), 27832790.Google ScholarGoogle ScholarCross RefCross Ref
  44. [44] Wang Shijun and Jin Rong. 2009. An information geometry approach for distance metric learning. In Proceedings of the International Conference on Artificial Intelligence and Statistics (AISTATS’09), Vol. 5. 591598.Google ScholarGoogle Scholar
  45. [45] Wang Shuyang, Robinson Joseph P., and Fu Yun. 2017. Kinship verification on families in the wild with marginalized denoising metric learning. In Proceedings of the IEEE International Conference on Automatic Face and Gesture Recognition. 216221.Google ScholarGoogle ScholarDigital LibraryDigital Library
  46. [46] Wang Shiwei and Yan Haibin. 2020. Discriminative sampling via deep reinforcement learning for kinship verification. Pattern Recognition Letters 138 (2020), 3843.Google ScholarGoogle ScholarCross RefCross Ref
  47. [47] Xia Siyu, Shao Ming, and Fu Yun. 2011. Kinship verification through transfer learning. In Proceedings of the International Joint Conference on Artificial Intelligence (IJCAI’11). 25392544.Google ScholarGoogle Scholar
  48. [48] Xia Siyu, Shao Ming, Luo Jiebo, and Fu Yun. 2012. Understanding kin relationships in a photo. IEEE Transactions on Multimedia 14, 4 (2012), 10461056.Google ScholarGoogle ScholarDigital LibraryDigital Library
  49. [49] Yan Haibin. 2019. Learning discriminative compact binary face descriptor for kinship verification. Pattern Recognition Letters 117 (2019), 146152.Google ScholarGoogle ScholarCross RefCross Ref
  50. [50] Yan Haibin, Lu Jiwen, Deng Weihong, and Zhou Xiuzhuang. 2014. Discriminative multimetric learning for kinship verification. IEEE Transactions on Information Forensics and Security 9, 7 (2014), 11691178.Google ScholarGoogle ScholarDigital LibraryDigital Library
  51. [51] Yan Haibin, Lu Jiwen, and Zhou Xiuzhuang. 2015. Prototype-based discriminative feature learning for kinship verification. IEEE Transactions on Cybernetics 45, 11 (2015), 25352545.Google ScholarGoogle ScholarCross RefCross Ref
  52. [52] Yan Haibin and Wang Shiwei. 2019. Learning part-aware attention networks for kinship verification. Pattern Recognition Letters 128 (2019), 169175.Google ScholarGoogle ScholarDigital LibraryDigital Library
  53. [53] Yan Haibin, Zhou Xiuzhuang, and Ge Yongxin. 2015. Neighborhood repulsed correlation metric learning for kinship verification. In Proceedings of the Conference on Visual Communications and Image Processing (VCIP’15). 14.Google ScholarGoogle ScholarCross RefCross Ref
  54. [54] Zhang Kaihao, Huang Yongzhen, Song Chunfeng, Wu Hong, and Wang Liang. 2015. Kinship verification with deep convolutional neural networks. In Proceedings of the British Machine Vision Conference (BMVC’15). 148.1–148.12.Google ScholarGoogle ScholarCross RefCross Ref
  55. [55] Zhang Lei, Duan Qingyan, Zhang David, Jia Wei, and Wang Xizhao. 2021. AdvKin: Adversarial convolutional network for kinship verification. IEEE Transactions on Cybernetics 51, 12 (2021), 5883–5896. DOI:Google ScholarGoogle ScholarCross RefCross Ref
  56. [56] Zhou Xiuzhuang, Hu Junlin, Lu Jiwen, Shang Yuanyuan, and Guan Yong. 2011. Kinship verification from facial images under uncontrolled conditions. In Proceedings of the International Conference on Multimedia. 953956.Google ScholarGoogle ScholarDigital LibraryDigital Library
  57. [57] Zhou Xiuzhuang, Jin Kai, Xu Min, and Guo Guodong. 2019. Learning deep compact similarity metric for kinship verification from face images. Information Fusion 48 (2019), 8494.Google ScholarGoogle ScholarDigital LibraryDigital Library
  58. [58] Zhou Xiuzhuang, Shang Yuanyuan, Yan Haibin, and Guo Guodong. 2016. Ensemble similarity learning for kinship verification from facial images in the wild. Information Fusion 32 (2016), 4048.Google ScholarGoogle ScholarDigital LibraryDigital Library

Index Terms

  1. Distance and Direction Based Deep Discriminant Metric Learning for Kinship Verification

    Recommendations

    Comments

    Login options

    Check if you have access through your login credentials or your institution to get full access on this article.

    Sign in

    Full Access

    • Published in

      cover image ACM Transactions on Multimedia Computing, Communications, and Applications
      ACM Transactions on Multimedia Computing, Communications, and Applications  Volume 19, Issue 1s
      February 2023
      504 pages
      ISSN:1551-6857
      EISSN:1551-6865
      DOI:10.1145/3572859
      • Editor:
      • Abdulmotaleb El Saddik
      Issue’s Table of Contents

      Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

      Publisher

      Association for Computing Machinery

      New York, NY, United States

      Publication History

      • Published: 23 January 2023
      • Online AM: 23 April 2022
      • Accepted: 10 April 2022
      • Revised: 15 November 2021
      • Received: 18 May 2021
      Published in tomm Volume 19, Issue 1s

      Permissions

      Request permissions about this article.

      Request Permissions

      Check for updates

      Qualifiers

      • research-article
      • Refereed

    PDF Format

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    Full Text

    View this article in Full Text.

    View Full Text

    HTML Format

    View this article in HTML Format .

    View HTML Format
    About Cookies On This Site

    We use cookies to ensure that we give you the best experience on our website.

    Learn more

    Got it!