Abstract
Image-based kinship verification is an important task in computer vision and has many applications in practice, such as missing children search and family album construction, among others. Due to the differences in age, gender, expression and appearance, there usually exists a large discrepancy between the facial images of parent and child. This makes kinship verification a challenging task. In this article, we propose a Distance and Direction Based Deep Discriminant Metric Learning (D4ML) approach for kinship verification. The basic idea of D4ML is to make full use of the discriminant information contained in the facial images of parent and child such that the network can learn more a discriminating distance metric. Specifically, D4ML learns the metric by utilizing the discriminant information from two perspectives: distance-based perspective and direction-based perspective. From the distance-based perspective, the designed loss function is used to minimize the distance between images having kinship and maximize the distance between images without kinship. In practice, the gender difference and large age gap may significantly increase the distance between facial images of parent and child. Therefore, learning the metric only from a distance-based perspective is insufficient. Considering that two vectors with a large distance may appear with high similarity in direction, D4ML also employs the direction-based loss function in the training process. Both kinds of loss function work together to improve the discriminability of the learned metric. Experimental results on four small size publicly available datasets demonstrate the effectiveness of our approach. Source code of our approach can be found at https://github.com/lclhenu/D4ML.
- [1] . 2022. Deep discriminant generation-shared feature learning for image-based kinship verification. Signal Processing: Image Communication 101 (2022), 116543.Google Scholar
Digital Library
- [2] . 2021. Semi-coupled synthesis and analysis dictionary pair learning for kinship verification. IEEE Transactions on Circuits and Systems for Video Technology 31, 5 (2021), 1939–1952.Google Scholar
Digital Library
- [3] . 2017. Kin-verification model on FIW dataset using multi-set learning and local features. In Proceedings of the 2017 Workshop on Recognizing Families in the Wild (RFIW’17). 31–35.Google Scholar
Digital Library
- [4] . 2018. From same photo: Cheating on visual kinship challenges. In Proceedings of the Asian Conference on Computer Vision (ACCV’18), Vol. 11363. 654–668.Google Scholar
- [5] . 2008. Social perception of facial resemblance in humans. Archives of Sexual Behavior 37, 1 (2008), 64–77.Google Scholar
Cross Ref
- [6] . 2014. Who do I look like? Determining parent-offspring resemblance via gated autoencoders. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR’14). 1757–1764.Google Scholar
Digital Library
- [7] . 2019. Cubic norm and kernel-based bi-directional PCA: Toward age-aware facial kinship verification. Visual Computer 35, 1 (2019), 23–40.Google Scholar
Digital Library
- [8] . 2017. Visual transformation aided contrastive learning for video-based kinship verification. In Proceedings of the IEEE International Conference on Computer Vision (ICCV’17). 2478–2487.Google Scholar
Cross Ref
- [9] . 2013. Like father, like son: Facial expression dynamics for kinship verification. In Proceedings of the IEEE International Conference on Computer Vision (ICCV’13). 1497–1504.Google Scholar
Digital Library
- [10] . 2017. Robust transfer metric learning for image classification. IEEE Transactions on Image Processing 26, 2 (2017), 660–670.Google Scholar
Digital Library
- [11] . 2010. Towards computational models of kinship verification. In Proceedings of the International Conference on Image Processing (ICIP’10). 1577–1580.Google Scholar
Cross Ref
- [12] . 2012. Kinship measurement on salient facial features. IEEE Transactions on Instrumentation and Measurement 61, 8 (2012), 2322–2325.Google Scholar
Cross Ref
- [13] . 2020. A feature-based kinship verification technique using convolutional neural network. In Advances in Data Sciences, Security and Applications. 353–362.Google Scholar
- [14] . 2017. Hierarchical representation learning for kinship verification. IEEE Transactions on Image Processing 26, 1 (2017), 289–302.Google Scholar
Digital Library
- [15] . 2015. Learning a genetic measure for kinship verification using facial images. Mathematical Problems in Engineering 2015, Pt. 1 (2015), 1–5.Google Scholar
- [16] . 2019. Kinship verification based deep and tensor features through extreme learning machine. In Proceedings of the IEEE International Conference on Automatic Face and Gesture Recognition. 1–4.Google Scholar
Digital Library
- [17] . 2016. Kinship verification from faces via similarity metric based convolutional neural network. In Proceedings of the International Conference on Image Analysis and Recognition, Vol. 9730. 539–548.Google Scholar
Cross Ref
- [18] . 2016. Kinship verification from faces via similarity metric based convolutional neural network. In Proceedings of the International Conference on Image Analysis and Recognition (ICIAR’16), Vol. 9730. 539–548.Google Scholar
Cross Ref
- [19] . 2021. Reasoning graph networks for kinship verification: From star-shaped to hierarchical. IEEE Transactions on Image Processing 30 (2021), 4947–4961.Google Scholar
Cross Ref
- [20] . 2021. Meta-mining discriminative samples for kinship verification. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR’21). 16135–16144.Google Scholar
Cross Ref
- [21] . 2017. Using deep relational features to verify kinship. In Proceedings of the CCF Chinese Conference on Computer Vision, Vol. 771. 563–573.Google Scholar
Cross Ref
- [22] . 2019. Weighted graph embedding-based metric learning for kinship verification. IEEE Transactions on Image Processing 28, 3 (2019), 1149–1162.Google Scholar
Digital Library
- [23] . 2015. Inheritable Fisher vector feature for kinship verification. In Proceedings of the IEEE International Conference on Biometrics Theory, Applications, and Systems (BTAS’15). 1–6.Google Scholar
Digital Library
- [24] . 2017. Discriminative deep metric learning for face and kinship verification. IEEE Transactions on Image Processing 26, 9 (2017), 4269–4282.Google Scholar
Digital Library
- [25] . 2014. Neighborhood repulsed metric learning for kinship verification. IEEE Transactions on Pattern Analysis and Machine Intelligence 36, 2 (2014), 331–345.Google Scholar
Digital Library
- [26] . 1936. On the generalised distance in statistics. Proceedings of the National Institute of Sciences of India 2 (1936), 49–55.Google Scholar
- [27] . 2019. Kinship verification using deep Siamese convolutional neural network. In Proceedings of the IEEE International Conference on Automatic Face and Gesture Recognition. 1–5.Google Scholar
Digital Library
- [28] . 2010. Cosine similarity metric learning for face verification. In Proceedings of the Asian Conference on Computer Vision (ACCV’10), Vol. 6493. 709–720.Google Scholar
- [29] . 2015. Deep face recognition. In Proceedings of the British Machine Vision Conference (BMVC’15). 41.1–41.12.Google Scholar
Cross Ref
- [30] . 2019. Image-based kinship verification using fusion convolutional neural network. In Proceedings of the IEEE International Workshop on Computational Intelligence and Applications (IWCIA’19). 59–65.Google Scholar
Cross Ref
- [31] . 2016. Families in the wild (FIW): Large-scale kinship image database and benchmarks. In Proceedings of the ACM Conference on Multimedia (MM’16). 242–246.Google Scholar
Digital Library
- [32] . 2016. Family in the wild (FIW): A large-scale kinship recognition database. CoRR abs/1604.02182 (2016).Google Scholar
- [33] . 2018. Visual kinship recognition of families in the wild. IEEE Transactions on Pattern Analysis and Machine Intelligence 40, 11 (2018), 2624–2637.Google Scholar
Digital Library
- [34] . 2018. Linear feature learning for kinship verification in the wild. In Proceedings of the International Conference on Applied Smart Systems (ICASS’18).Google Scholar
Cross Ref
- [35] . 2011. Genealogical face recognition based on UB KinFace database. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR’11). 60–65.Google Scholar
Cross Ref
- [36] . 2012. Generalized multiview analysis: A discriminative latent space. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR’12). 2160–2167.Google Scholar
Cross Ref
- [37] . 2019. Perceive where to focus: Learning visibility-aware part-level features for partial person re-identification. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR’19). 393–402.Google Scholar
Cross Ref
- [38] . 2018. Beyond part models: Person retrieval with refined part pooling (and a strong convolutional baseline). In Computer Vision—(ECCV’18). Lecture Notes in Computer Science, Vol. 11208. Springer, 501–518.Google Scholar
- [39] . 2018. Deep learning features for robust facial kinship verification. IET Image Processing 12, 12 (2018), 2336–2345.Google Scholar
Cross Ref
- [40] . 2014. Detecting siblings in image pairs. Visual Computer 30, 12 (2014), 1333–1345.Google Scholar
Digital Library
- [41] . 2005. On the Euclidean distance of images. IEEE Transactions on Pattern Analysis and Machine Intelligence 27, 8 (2005), 1334–1339.Google Scholar
Digital Library
- [42] . 2015. Deep kinship verification. In Proceedings of the IEEE International Workshop on Multimedia Signal Processing (MMSP’15). 1–6.Google Scholar
Cross Ref
- [43] . 2019. Cross-generation kinship verification with sparse discriminative metric. IEEE Transactions on Pattern Analysis and Machine Intelligence 41, 11 (2019), 2783–2790.Google Scholar
Cross Ref
- [44] . 2009. An information geometry approach for distance metric learning. In Proceedings of the International Conference on Artificial Intelligence and Statistics (AISTATS’09), Vol. 5. 591–598.Google Scholar
- [45] . 2017. Kinship verification on families in the wild with marginalized denoising metric learning. In Proceedings of the IEEE International Conference on Automatic Face and Gesture Recognition. 216–221.Google Scholar
Digital Library
- [46] . 2020. Discriminative sampling via deep reinforcement learning for kinship verification. Pattern Recognition Letters 138 (2020), 38–43.Google Scholar
Cross Ref
- [47] . 2011. Kinship verification through transfer learning. In Proceedings of the International Joint Conference on Artificial Intelligence (IJCAI’11). 2539–2544.Google Scholar
- [48] . 2012. Understanding kin relationships in a photo. IEEE Transactions on Multimedia 14, 4 (2012), 1046–1056.Google Scholar
Digital Library
- [49] . 2019. Learning discriminative compact binary face descriptor for kinship verification. Pattern Recognition Letters 117 (2019), 146–152.Google Scholar
Cross Ref
- [50] . 2014. Discriminative multimetric learning for kinship verification. IEEE Transactions on Information Forensics and Security 9, 7 (2014), 1169–1178.Google Scholar
Digital Library
- [51] . 2015. Prototype-based discriminative feature learning for kinship verification. IEEE Transactions on Cybernetics 45, 11 (2015), 2535–2545.Google Scholar
Cross Ref
- [52] . 2019. Learning part-aware attention networks for kinship verification. Pattern Recognition Letters 128 (2019), 169–175.Google Scholar
Digital Library
- [53] . 2015. Neighborhood repulsed correlation metric learning for kinship verification. In Proceedings of the Conference on Visual Communications and Image Processing (VCIP’15). 1–4.Google Scholar
Cross Ref
- [54] . 2015. Kinship verification with deep convolutional neural networks. In Proceedings of the British Machine Vision Conference (BMVC’15). 148.1–148.12.Google Scholar
Cross Ref
- [55] . 2021. AdvKin: Adversarial convolutional network for kinship verification. IEEE Transactions on Cybernetics 51, 12 (2021), 5883–5896.
DOI: Google ScholarCross Ref
- [56] . 2011. Kinship verification from facial images under uncontrolled conditions. In Proceedings of the International Conference on Multimedia. 953–956.Google Scholar
Digital Library
- [57] . 2019. Learning deep compact similarity metric for kinship verification from face images. Information Fusion 48 (2019), 84–94.Google Scholar
Digital Library
- [58] . 2016. Ensemble similarity learning for kinship verification from facial images in the wild. Information Fusion 32 (2016), 40–48.Google Scholar
Digital Library
Index Terms
Distance and Direction Based Deep Discriminant Metric Learning for Kinship Verification
Recommendations
Video-based kinship verification using distance metric learning
We investigate the problem of kinship verification from facial videos.We present a new video face dataset for the video-based kinship verification study.We develop a benchmark to evaluate state-of-the-art metric learning methods in video-based kinship ...
Kinship verification using neighborhood repulsed correlation metric learning
Kinship verification is an interesting and challenging problem in human face analysis, which has received increasing interests in computer vision and biometrics in recent years. This paper presents a neighborhood repulsed correlation metric learning (...
Component-based metric learning for fully automatic kinship verification
AbstractThis paper introduces a fully automatic method for kinship verification from facial images. Recently, a number of methods have been proposed to verify kinship from facial images, however, most of these methods are needed to exactly ...
Highlights- We present a fully automatic kinship verification method.
- We introduce three ...






Comments