skip to main content
research-article

Domain-invariant Graph for Adaptive Semi-supervised Domain Adaptation

Authors Info & Claims
Published:04 March 2022Publication History
Skip Abstract Section

Abstract

Domain adaptation aims to generalize a model from a source domain to tackle tasks in a related but different target domain. Traditional domain adaptation algorithms assume that enough labeled data, which are treated as the prior knowledge are available in the source domain. However, these algorithms will be infeasible when only a few labeled data exist in the source domain, thus the performance decreases significantly. To address this challenge, we propose a Domain-invariant Graph Learning (DGL) approach for domain adaptation with only a few labeled source samples. Firstly, DGL introduces the Nyström method to construct a plastic graph that shares similar geometric property with the target domain. Then, DGL flexibly employs the Nyström approximation error to measure the divergence between the plastic graph and source graph to formalize the distribution mismatch from the geometric perspective. Through minimizing the approximation error, DGL learns a domain-invariant geometric graph to bridge the source and target domains. Finally, we integrate the learned domain-invariant graph with the semi-supervised learning and further propose an adaptive semi-supervised model to handle the cross-domain problems. The results of extensive experiments on popular datasets verify the superiority of DGL, especially when only a few labeled source samples are available.

REFERENCES

  1. [1] Andersen M., Dahl J., Liu Z., and Vandenberghe L.. 2011. Interior-point Methods for Large-scale Cone Programming.Google ScholarGoogle Scholar
  2. [2] Belkin M., Niyogi P., and Sindhwani V.. 2006. Manifold regularization: A geometric framework for learning from labeled and unlabeled examples. Journal of Machine Learning Research 7 (2006), 23992434.Google ScholarGoogle ScholarDigital LibraryDigital Library
  3. [3] Chen S., Zhou F., and Liao Q.. 2016. Visual domain adaptation using weighted subspace alignment. In Proceedings of the Visual Communications and Image Processing (VCIP).Google ScholarGoogle ScholarCross RefCross Ref
  4. [4] Chu W., Torre F., and Cohn J. F.. 2017. Selective transfer machine for personalized facial expression analysis. IEEE Transactions on Pattern Analysis and Machine Intelligence 39, 3 (2017), 529545.Google ScholarGoogle ScholarDigital LibraryDigital Library
  5. [5] Courty N., Flamary R., Tuia D., and Rakotomamonjy A.. 2015. Optimal transport for domain adaptation. IEEE Transactions on Pattern Analysis and Machine Intelligence 39, 9 (2015).Google ScholarGoogle ScholarDigital LibraryDigital Library
  6. [6] Dai W., Yang Q., Xue G., and Yu Y.. 2007. Boosting for transfer learning. In Proceedings of the International Conference on Machine Learning (ICML). 193200.Google ScholarGoogle ScholarDigital LibraryDigital Library
  7. [7] Das D. and Lee C. S. G.. 2018. Graph matching and pseudo-label guided deep unsupervised domain adaptation. In Proceedings of the International Conference on Artificial Neural Networks (ICANN). 342352.Google ScholarGoogle ScholarCross RefCross Ref
  8. [8] Ding Z. and Fu Y.. 2017. Robust transfer metric learning for image classification. IEEE Transactions on Image Processing 26, 2 (2017), 660670.Google ScholarGoogle ScholarDigital LibraryDigital Library
  9. [9] Drineas P. and Mahoney M.. 2005. On the Nystr\( \ddot{o} \)m method for approximating a Gram matrix for improved kernel-based learning. Journal of Machine Learning Research 6 (2005), 21532175.Google ScholarGoogle ScholarDigital LibraryDigital Library
  10. [10] Fowlkes C., Belongie S., Chung F., and Malik J.. 2004. Spectral grouping using the Nyström method. IEEE Transactions on Pattern Analysis and Machine Intelligence 26, 2 (2004), 214225.Google ScholarGoogle ScholarDigital LibraryDigital Library
  11. [11] Gao J., Fan W., Jiang J., and Han J.. 2008. Knowledge transfer via multiple model local structure mapping. In The ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD). 283291.Google ScholarGoogle ScholarDigital LibraryDigital Library
  12. [12] Gong B., Shi Y., Sha F., and Grauman K.. 2012. Geodesic flow kernel for unsupervised domain adaptation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 20662073.Google ScholarGoogle Scholar
  13. [13] Gretton A., Borgwardt K. M., Rasch M. J., Schlkopf B., and Smola A. J.. 2006. A kernel method for the two-sample-problem. In Proceedings of the Advances in Neural Information Processing Systems (NIPS). 513520.Google ScholarGoogle Scholar
  14. [14] Griffin G., Holub A., and Perona. P.2007. Caltech-256 object category dataset. CalTech Report (2007).Google ScholarGoogle Scholar
  15. [15] Hao Y., Mu T., Hong R., Wang M., Liu X., and Goulermas J. Y.. 2020. Cross-domain sentiment encoding through stochastic word embedding. IEEE Transactions on Knowledge and Data Engineering 32, 10 (2020), 19091922.Google ScholarGoogle ScholarCross RefCross Ref
  16. [16] Jiang J., Wang X., Long M., and Wang J.. 2020. Resource efficient domain adaptation. In Proceedings of the 28th ACM International Conference on Multimedia. 22202228.Google ScholarGoogle ScholarDigital LibraryDigital Library
  17. [17] Jing T., Xia H., and Ding Z.. 2020. Adaptively-accumulated knowledge transfer for partial domain adaptation. In Proceedings of the ACM International Conference on Multimedia. 16061614.Google ScholarGoogle ScholarDigital LibraryDigital Library
  18. [18] Li M., Bi W., Kwok J., and Lu B.. 2015. Large-Scale Nystr\( \ddot{o} \)m Kernel Matrix Approximation Using Randomized SVD. IEEE Transactions on Neural Networks and Learning System 26, 1 (2015), 152164.Google ScholarGoogle ScholarCross RefCross Ref
  19. [19] Li Y., Wei B., Yao L., Chen H., and Li Z.. 2017. Knowledge-based document embedding for cross-domain text classification. In Proceedings of the International Joint Conference on Neural Networks (IJCNN). 13951402.Google ScholarGoogle ScholarCross RefCross Ref
  20. [20] Lin C. Chang.and C.. 2011. Libsvm: A library for support vector machines. ACM Transactions on Intelligent Systems and Technology 2, 3 (2011), 127.Google ScholarGoogle ScholarDigital LibraryDigital Library
  21. [21] Liu H., Shao M., Ding Z., and Fu Y.. 2019. Structure-preserved unsupervised domain adaptation. IEEE Transactions on Knowledge and Data Engineering 31, 4 (2019), 799812.Google ScholarGoogle ScholarDigital LibraryDigital Library
  22. [22] Liu J., Zha Z., Chen D., Hong R., and Wang M.. 2019. Adaptive transfer network for cross-domain person re-identification. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 72027211.Google ScholarGoogle ScholarCross RefCross Ref
  23. [23] Long M., Wang J., Ding G., Pan S. J., and Yu P. S.. 2014. Adaptation regularization: A general framework for transfer learning. IEEE Transactions on Knowledge and Data Engineering 26, 5 (2014), 10761089.Google ScholarGoogle ScholarDigital LibraryDigital Library
  24. [24] Long M., Wang J., Ding G., Sun J., and Yu P. S.. 2014. Transfer joint matching for unsupervised domain adaptation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 14101417.Google ScholarGoogle ScholarDigital LibraryDigital Library
  25. [25] Long M., Wang J., Sun J., and Yu P. S.. 2015. Domain invariant transfer kernel learning. IEEE Transactions on Knowledge and Data Engineering 27, 6 (2015), 15191532.Google ScholarGoogle ScholarDigital LibraryDigital Library
  26. [26] Pan S. J., Tsang I. W., Kwok J. T., and Yang Q.. 2011. Domain adaptation via transfer component analysis. IEEE Transactions on Neural Networks 22, 2 (2011), 199210.Google ScholarGoogle ScholarDigital LibraryDigital Library
  27. [27] Pan S. J. and Yang Q.. 2010. A survey on transfer learning. IEEE Transactions on Knowledge and Data Engineering 22, 10 (2010), 13451359.Google ScholarGoogle ScholarDigital LibraryDigital Library
  28. [28] Pilanci M. and Vural E.. 2020. Domain adaptation on graphs by learning aligned graph bases. IEEE Transactions on Knowledge and Data Engineering (2020).Google ScholarGoogle Scholar
  29. [29] Wang H., Nie F., Huang H., and Ding C.. 2011. Dyadic transfer learning for cross-domain image classification. In Proceedings of the IEEE International Conference on Computer Vision (ICCV). 551556.Google ScholarGoogle ScholarDigital LibraryDigital Library
  30. [30] Wang J., Chen Y., Hao S., Feng W., and Shen Z.. 2017. Balanced distribution adaptation for transfer learning. In Proceedings of the IEEE International Conference on Data Mining (ICDM). 11291134.Google ScholarGoogle ScholarCross RefCross Ref
  31. [31] Wang J., Feng W., Chen Y., Yu H., Huang M., and Yu P. S.. 2018. Visual domain adaptation with manifold embedded distribution alignment. In Proceedings of the ACM International Conference on Multimedia (ACMMM).Google ScholarGoogle ScholarDigital LibraryDigital Library
  32. [32] Wei P., Ke Y., and Goh C.. 2019. A general domain specific feature transfer framework for hybrid domain adaptation. IEEE Transactions on Knowledge and Data Engineering 31, 8 (2019), 14401451.Google ScholarGoogle ScholarDigital LibraryDigital Library
  33. [33] Williams C. and Seeger M.. 2000. Using the Nystr\( \ddot{o} \)m Method to speed up kernel machines. In Proceedings of Advances in Neural Information Processing Systems (NIPS). 682688.Google ScholarGoogle Scholar
  34. [34] Li W., Duan L., Xu D., and Tsang I. W.. 2014. Learning with augmented features for supervised and semi-supervised heterogeneous domain adaptation. IEEE Transactions on Pattern Analysis and Machine Intelligence 36, 6 (2014), 11341148.Google ScholarGoogle ScholarDigital LibraryDigital Library
  35. [35] Yan K., Kou L., and Zhang D.. 2018. Learning domain-invariant subspace using domain features and independence maximization. IEEE Transactions on Cybernetics 48, 1 (2018), 288299.Google ScholarGoogle ScholarCross RefCross Ref
  36. [36] Yang X., Zhang T., Xu C., and Yang M.. 2015. Boosted multifeature learning for cross-domain transfer. ACM Transactions on Multimedia Computing, Communications, and Applications 11, 3 (2015).Google ScholarGoogle ScholarDigital LibraryDigital Library
  37. [37] Yao T., Pan Y., Ngo C., Li H., and Mei T.. 2015. Semi-supervised domain adaptation with subspace learning for visual recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 21422150.Google ScholarGoogle ScholarCross RefCross Ref
  38. [38] Yu J., Rui Y., and Chen B.. 2014. Exploiting click constraints and multi-view features for image re-ranking. IEEE Transactions on Multimedia 16, 1 (2014), 159168.Google ScholarGoogle ScholarCross RefCross Ref
  39. [39] Yu J., Rui Y., and Tao D.. 2014. Click prediction for web image reranking using multimodal sparse coding. IEEE Transactions on Image Processing 23, 5 (2014), 20192032.Google ScholarGoogle ScholarCross RefCross Ref
  40. [40] Zhang J., Li W., and Ogunbona P.. 2017. Joint geometrical and statistical alignment for visual domain adaptation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 51505158.Google ScholarGoogle ScholarCross RefCross Ref
  41. [41] Zhang K., Tsang I. W., and Kwok J. T.. 2008. Improved Nystr\( \ddot{o} \)m Low-Rank Approximation and Error Analysis. In Proceedings of the International Conference on Machine Learning (ICML). 12321239.Google ScholarGoogle ScholarDigital LibraryDigital Library
  42. [42] Zhang L.. 2019. Transfer adaptation learning: A decade survey. Corr abs/1903.04687 (2019). arxiv:1903.04687 http://arxiv.org/abs/1903.04687Google ScholarGoogle Scholar
  43. [43] Zhang L., Fu J., Wang S., Zhang D., Dong Z., and Chen C. L. P.. 2019. Guide subspace learning for unsupervised domain adaptation. IEEE Transactions on Neural Networks and Learning Systems (2019), 115.Google ScholarGoogle Scholar
  44. [44] Zhang Y., Miao S., and Liao R.. 2018. Structural domain adaptation with latent graph alignment. In Proceedings of the IEEE International Conference on Image Processing (ICIP). 37533757.Google ScholarGoogle ScholarCross RefCross Ref
  45. [45] Zhuang F., Luo P., Shen Z., He Q., Xiong Y., Shi Z., and Xiong H.. 2012. Mining distinction and commonality across multiple domains using generative model for text classification. IEEE Transactions on Knowledge and Data Engineering 24, 11 (2012), 20252039.Google ScholarGoogle ScholarDigital LibraryDigital Library

Index Terms

  1. Domain-invariant Graph for Adaptive Semi-supervised Domain Adaptation

        Recommendations

        Comments

        Login options

        Check if you have access through your login credentials or your institution to get full access on this article.

        Sign in

        Full Access

        • Published in

          cover image ACM Transactions on Multimedia Computing, Communications, and Applications
          ACM Transactions on Multimedia Computing, Communications, and Applications  Volume 18, Issue 3
          August 2022
          478 pages
          ISSN:1551-6857
          EISSN:1551-6865
          DOI:10.1145/3505208
          Issue’s Table of Contents

          Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

          Publisher

          Association for Computing Machinery

          New York, NY, United States

          Publication History

          • Published: 4 March 2022
          • Accepted: 1 September 2021
          • Revised: 1 August 2021
          • Received: 1 December 2020
          Published in tomm Volume 18, Issue 3

          Permissions

          Request permissions about this article.

          Request Permissions

          Check for updates

          Qualifiers

          • research-article
          • Refereed

        PDF Format

        View or Download as a PDF file.

        PDF

        eReader

        View online with eReader.

        eReader

        Full Text

        View this article in Full Text.

        View Full Text

        HTML Format

        View this article in HTML Format .

        View HTML Format
        About Cookies On This Site

        We use cookies to ensure that we give you the best experience on our website.

        Learn more

        Got it!