skip to main content
research-article

Instance Correlation Graph for Unsupervised Domain Adaptation

Authors Info & Claims
Published:25 January 2022Publication History
Skip Abstract Section

Abstract

In recent years, deep neural networks have emerged as a dominant machine learning tool for a wide variety of application fields. Due to the expensive cost of manual labeling efforts, it is important to transfer knowledge from a label-rich source domain to an unlabeled target domain. The core problem is how to learn a domain-invariant representation to address the domain shift challenge, in which the training and test samples come from different distributions. First, considering the geometry of space probability distributions, we introduce an effective Hellinger Distance to match the source and target distributions on statistical manifold. Second, the data samples are not isolated individuals, and they are interrelated. The correlation information of data samples should not be neglected for domain adaptation. Distinguished from previous works, we pay attention to the correlation distributions over data samples. We design elaborately a Residual Graph Convolutional Network to construct the Instance Correlation Graph (ICG). The correlation information of data samples is exploited to reduce the domain shift. Therefore, a novel Instance Correlation Graph for Unsupervised Domain Adaptation is proposed, which is trained end-to-end by jointly optimizing three types of losses, i.e., Supervised Classification loss for source domain, Centroid Alignment loss to measure the centroid difference between source and target domain, ICG Alignment loss to match Instance Correlation Graph over two related domains. Extensive experiments are conducted on several hard transfer tasks to learn domain-invariant representations on three benchmarks: Office-31, Office-Home, and VisDA2017. Compared with other state-of-the-art techniques, our method achieves superior performance.

REFERENCES

  1. [1] Ajakan Hana, Germain Pascal, Larochelle Hugo, Laviolette François, and Marchand Mario. 2014. Domain-adversarial neural networks. CoRR abs/1412.4446 (2014).Google ScholarGoogle Scholar
  2. [2] Amini Massih-Reza and Gallinari Patrick. 2002. Semi supervised logistic regression. In ECAI, Harmelen Frank van (Ed.). 390394. Google ScholarGoogle ScholarDigital LibraryDigital Library
  3. [3] Amodei Dario, Olah Chris, Steinhardt Jacob, Christiano Paul F., Schulman John, and Mané Dan. 2016. Concrete problems in AI safety. CoRR abs/1606.06565 (2016).Google ScholarGoogle Scholar
  4. [4] Arjovsky Martín, Bottou Léon, Gulrajani Ishaan, and Lopez-Paz David. 2019. Invariant risk minimization. CoRR abs/1907.02893 (2019).Google ScholarGoogle Scholar
  5. [5] Ba Lei Jimmy, Kiros Jamie Ryan, and Hinton Geoffrey E.. 2016. Layer normalization. CoRR abs/1607.06450 (2016).Google ScholarGoogle Scholar
  6. [6] Baktashmotlagh Mahsa, Harandi Mehrtash Tafazzoli, and Salzmann Mathieu. 2016. Distribution-matching embedding for visual domain adaptation. J. Mach. Learn. Res. 17 (2016), 108:1–108:30. Google ScholarGoogle ScholarDigital LibraryDigital Library
  7. [7] Ben-David Shai, Blitzer John, Crammer Koby, Kulesza Alex, Pereira Fernando, and Vaughan Jennifer Wortman. 2010. A theory of learning from different domains. Mach. Learn. 79, 1–2 (2010), 151175. Google ScholarGoogle ScholarDigital LibraryDigital Library
  8. [8] Ben-David Shai, Blitzer John, Crammer Koby, and Pereira Fernando. 2006. Analysis of representations for domain adaptation. In NIPS. Google ScholarGoogle ScholarDigital LibraryDigital Library
  9. [9] Bouvier Victor, Very Philippe, Chastagnol Clément, Tami Myriam, and Hudelot Céline. 2020. Robust domain adaptation: Representations, weights and inductive bias. CoRR abs/2006.13629.Google ScholarGoogle Scholar
  10. [10] Cao Zhangjie, Long Mingsheng, Huang Chao, and Wang Jianmin. 2018. Transfer adversarial hashing for hamming space retrieval. In AAAI. Google ScholarGoogle ScholarDigital LibraryDigital Library
  11. [11] Carter Kevin. 2009. Dimensionality reduction on statistical manifolds. Dissertations & Theses - Gradworks.Google ScholarGoogle Scholar
  12. [12] Chen Xinyang, Wang Sinan, Long Mingsheng, and Wang Jianmin. 2019. Transferability vs. discriminability: Batch spectral penalization for adversarial domain adaptation. In ICML, Vol. 97. 10811090.Google ScholarGoogle Scholar
  13. [13] Cui Shuhao, Wang Shuhui, Zhuo Junbao, Li Liang, Huang Qingming, and Tian Qi. 2020. Towards discriminability and diversity: Batch nuclear-norm maximization under label insufficient situations. CoRR abs/2003.12237.Google ScholarGoogle Scholar
  14. [14] Cui Shuhao, Wang Shuhui, Zhuo Junbao, Su Chi, Huang Qingming, and Tian Qi. 2020. Gradually vanishing bridge for adversarial domain adaptation. In CVPR. 1245212461.Google ScholarGoogle Scholar
  15. [15] Das D. and Lee C. S. George. 2018. Unsupervised domain adaptation using regularized hyper-graph matching. In ICIP. 37583762.Google ScholarGoogle Scholar
  16. [16] Deng Jia, Dong Wei, Socher Richard, Li Li-Jia, Li Kai, and Li Fei-Fei. 2009. ImageNet: A large-scale hierarchical image database. In CVPR. 248255.Google ScholarGoogle Scholar
  17. [17] Ding Zhengming, Li Sheng, Shao Ming, and Fu Yun. 2018. Graph adaptive knowledge transfer for unsupervised domain adaptation. In ECCV.Google ScholarGoogle Scholar
  18. [18] Ganin Yaroslav and Lempitsky Victor S.. 2015. Unsupervised domain adaptation by backpropagation. In ICML. 11801189. Google ScholarGoogle ScholarDigital LibraryDigital Library
  19. [19] Ganin Yaroslav, Ustinova Evgeniya, Ajakan Hana, Germain Pascal, Larochelle Hugo, Laviolette François, Marchand Mario, and Lempitsky Victor S.. 2016. Domain-adversarial training of neural networks. J. Mach. Learn. Res. 17 (2016), 59:1–59:35. Google ScholarGoogle ScholarDigital LibraryDigital Library
  20. [20] Gebru Timnit, Hoffman Judy, and Fei-Fei Li. 2017. Fine-grained recognition in the wild: A multi-task domain adaptation approach. In ICCV.Google ScholarGoogle Scholar
  21. [21] Ghifary Muhammad, Kleijn W. Bastiaan, Zhang Mengjie, Balduzzi David, and Li Wen. 2016. Deep reconstruction-classification networks for unsupervised domain adaptation. In ECCV. 597613.Google ScholarGoogle Scholar
  22. [22] Gong Boqing, Shi Yuan, Sha Fei, and Grauman Kristen. 2012. Geodesic flow kernel for unsupervised domain adaptation. In CVPR.Google ScholarGoogle Scholar
  23. [23] Gong Rui, Li Wen, Chen Yuhua, and Gool Luc Van. 2019. DLOW: Domain flow for adaptation and generalization. In CVPR. 24772486.Google ScholarGoogle Scholar
  24. [24] Goodfellow Ian J., Pouget-Abadie Jean, Mirza Mehdi, Xu Bing, Warde-Farley David, Ozair Sherjil, Courville Aaron C., and Bengio Yoshua. 2014. Generative adversarial nets. In NIPS. Google ScholarGoogle ScholarDigital LibraryDigital Library
  25. [25] Goyal Priya, Dollár Piotr, Girshick Ross B., Noordhuis Pieter, Wesolowski Lukasz, Kyrola Aapo, Tulloch Andrew, Jia Yangqing, and He Kaiming. 2017. Accurate, large minibatch SGD: Training imagenet in 1 hour. CoRR abs/1706.02677.Google ScholarGoogle Scholar
  26. [26] Gretton A., Borgwardt K. M., Rasch Malte, B. Schölkopf, and Smola A.. 2012. A kernel two-sample test. J. Mach. Learn. Res. 13, 1 (2012), 723773. Google ScholarGoogle ScholarDigital LibraryDigital Library
  27. [27] Gretton Arthur, Sejdinovic Dino, Strathmann Heiko, Balakrishnan Sivaraman, Pontil Massimiliano, Fukumizu Kenji, and Sriperumbudur Bharath K.. 2012. Optimal kernel choice for large-scale two-sample tests. In NIPS. Google ScholarGoogle ScholarDigital LibraryDigital Library
  28. [28] Hartley Richard I., Trumpf Jochen, Dai Yuchao, and Li Hongdong. 2013. Rotation averaging. Int. J. Comput. Vis. 103, 3 (2013), 267305.Google ScholarGoogle ScholarCross RefCross Ref
  29. [29] He Kaiming, Zhang Xiangyu, Ren Shaoqing, and Sun Jian. 2016. Deep residual learning for image recognition. In CVPR. 770778.Google ScholarGoogle Scholar
  30. [30] Hoffman Judy, Tzeng Eric, Park Taesung, Zhu Jun-Yan, Isola Phillip, Saenko Kate, Efros Alexei A., and Darrell Trevor. 2018. CyCADA: Cycle-consistent adversarial domain adaptation. In ICML, Vol. 80. 19942003.Google ScholarGoogle Scholar
  31. [31] Huang Xun, Liu Ming-Yu, Belongie Serge J., and Kautz Jan. 2018. Multimodal unsupervised image-to-image translation. In ECCV, Vol. 11207. 179196.Google ScholarGoogle Scholar
  32. [32] Kang Guoliang, Jiang Lu, Yang Yi, and Hauptmann Alexander G.. 2019. Contrastive adaptation network for unsupervised domain adaptation. In CVPR. 48934902.Google ScholarGoogle Scholar
  33. [33] Kass Robert E. and Vos Paul W. 2011. Geometrical Foundations of Asymptotic Inference. Vol. 908.Google ScholarGoogle Scholar
  34. [34] Kipf Thomas N. and Welling Max. 2017. Semi-supervised classification with graph convolutional networks. In ICLR.Google ScholarGoogle Scholar
  35. [35] Krause Jonathan, Stark Michael, Deng Jia, and Fei-Fei Li. 2013. 3D object representations for fine-grained categorization. In ICCV Workshops. 554561. Google ScholarGoogle ScholarDigital LibraryDigital Library
  36. [36] Krizhevsky Alex, Sutskever Ilya, and Hinton Geoffrey E.. 2012. ImageNet classification with deep convolutional neural networks. In NIPS. 11061114. Google ScholarGoogle ScholarDigital LibraryDigital Library
  37. [37] LeCun Yann, Bottou Léon, Bengio Yoshua, Haffner Patrick, et al. 1998. Gradient-based learning applied to document recognition. Proc. IEEE 86, 11 (1998), 22782324.Google ScholarGoogle ScholarCross RefCross Ref
  38. [38] Li Jingjing, Jing Mengmeng, Lu Ke, Zhu Lei, and Shen Heng Tao. 2019. Locality preserving joint transfer for domain adaptation. IEEE Trans. Image Process. 28, 12 (2019), 6103–6115.Google ScholarGoogle Scholar
  39. [39] Lin Tsung-Yi, Dollár Piotr, Girshick Ross B., He Kaiming, Hariharan Bharath, and Belongie Serge J.. 2017. Feature pyramid networks for object detection. In CVPR. 936944.Google ScholarGoogle Scholar
  40. [40] Liu Xiaofeng, Li Site, Kong Lingsheng, Xie Wanqing, Jia Ping, You Jane, and Kumar B. V. K. Vijaya. 2019. Feature-level frankenstein: Eliminating variations for discriminative recognition. In CVPR. 637646.Google ScholarGoogle Scholar
  41. [41] Long Mingsheng, Cao Yue, Wang Jianmin, and Jordan Michael I.. 2015. Learning transferable features with deep adaptation networks. In ICML. Google ScholarGoogle ScholarDigital LibraryDigital Library
  42. [42] Long Mingsheng, Cao Zhangjie, Wang Jianmin, and Jordan Michael I.. 2018. Conditional adversarial domain adaptation. In NeurIPS. 16471657. Google ScholarGoogle ScholarDigital LibraryDigital Library
  43. [43] Long Mingsheng, Wang Jianmin, Ding Guiguang, Sun Jiaguang, and Yu Philip S.. 2013. Transfer feature learning with joint distribution adaptation. In ICCV. Google ScholarGoogle ScholarDigital LibraryDigital Library
  44. [44] Long Mingsheng, Zhu Han, Wang Jianmin, and Jordan Michael I.. 2016. Unsupervised domain adaptation with residual transfer networks. In NIPS. Google ScholarGoogle ScholarDigital LibraryDigital Library
  45. [45] Long Mingsheng, Zhu Han, Wang Jianmin, and Jordan Michael I.. 2017. Deep transfer learning with joint adaptation networks. In ICML. Google ScholarGoogle ScholarDigital LibraryDigital Library
  46. [46] Luo Zelun, Zou Yuliang, Hoffman Judy, and Li Fei-Fei. 2017. Label efficient learning of transferable representations acrosss domains and tasks. In NIPS. 165177. Google ScholarGoogle ScholarDigital LibraryDigital Library
  47. [47] Ma Xinhong, Zhang Tianzhu, and Xu Changsheng. 2019. GCAN: Graph convolutional adversarial network for unsupervised domain adaptation. In CVPR.Google ScholarGoogle Scholar
  48. [48] Pan Sinno Jialin, Tsang Ivor W., Kwok James T., and Yang Qiang. 2011. Domain adaptation via transfer component analysis. IEEE Trans. Neur. Netw. 22, 2 (2011), 199210. Google ScholarGoogle ScholarDigital LibraryDigital Library
  49. [49] Pei Zhongyi, Cao Zhangjie, Long Mingsheng, and Wang Jianmin. 2018. Multi-adversarial domain adaptation. In AAAI. Google ScholarGoogle ScholarDigital LibraryDigital Library
  50. [50] Peng Xingchao, Usman Ben, Kaushik Neela, Hoffman Judy, Wang Dequan, and Saenko Kate. 2017. Visda: The visual domain adaptation challenge. CoRR abs/1710.06924.Google ScholarGoogle Scholar
  51. [51] Pilanci M. and Vural E.. 2020. Domain adaptation on graphs by learning aligned graph bases. IEEE Trans. Knowl. Data Eng. (2020), 11.Google ScholarGoogle Scholar
  52. [52] Pinheiro Pedro O.. 2018. Unsupervised domain adaptation with similarity learning. In CVPR. 80048013.Google ScholarGoogle Scholar
  53. [53] Radenovic Filip, Tolias Giorgos, and Chum Ondrej. 2019. Fine-tuning CNN image retrieval with no human annotation. IEEE Trans. Pattern Anal. Mach. Intell. 41, 7 (2019), 16551668.Google ScholarGoogle ScholarCross RefCross Ref
  54. [54] Saenko Kate, Kulis Brian, Fritz Mario, and Darrell Trevor. 2010. Adapting visual category models to new domains. In ECCV. Google ScholarGoogle ScholarDigital LibraryDigital Library
  55. [55] Saito Kuniaki, Ushiku Yoshitaka, Harada Tatsuya, and Saenko Kate. 2018. Adversarial dropout regularization. In ICLR.Google ScholarGoogle Scholar
  56. [56] Sankaranarayanan Swami, Balaji Yogesh, Castillo Carlos Domingo, and Chellappa Rama. 2018. Generate to adapt: Aligning domains using generative adversarial networks. In CVPR. 85038512.Google ScholarGoogle Scholar
  57. [57] Shelhamer Evan, Long Jonathan, and Darrell Trevor. 2017. Fully convolutional networks for semantic segmentation. IEEE Trans. Pattern Anal. Mach. Intell. 39, 4 (2017), 640651. Google ScholarGoogle ScholarDigital LibraryDigital Library
  58. [58] Snell Jake, Swersky Kevin, and Zemel Richard S.. 2017. Prototypical networks for few-shot learning. In NIPS. 40774087. Google ScholarGoogle ScholarDigital LibraryDigital Library
  59. [59] Sun Baochen, Feng Jiashi, and Saenko Kate. 2016. Return of frustratingly easy domain adaptation. In AAAI. Google ScholarGoogle ScholarDigital LibraryDigital Library
  60. [60] Sun Baochen and Saenko Kate. 2016. Deep CORAL: Correlation alignment for deep domain adaptation. In ECCV Workshops.Google ScholarGoogle Scholar
  61. [61] Tran Luan, Liu Feng, and Liu Xiaoming. 2019. Towards high-fidelity nonlinear 3D face morphable model. In CVPR.Google ScholarGoogle Scholar
  62. [62] Tsai Yi-Hsuan, Hung Wei-Chih, Schulter Samuel, Sohn Kihyuk, Yang Ming-Hsuan, and Chandraker Manmohan. 2018. Learning to adapt structured output space for semantic segmentation. In CVPR. 74727481.Google ScholarGoogle Scholar
  63. [63] Tzeng Eric, Hoffman Judy, Darrell Trevor, and Saenko Kate. 2015. Simultaneous deep transfer across domains and tasks. In ICCV. Google ScholarGoogle ScholarDigital LibraryDigital Library
  64. [64] Tzeng Eric, Hoffman Judy, Saenko Kate, and Darrell Trevor. 2017. Adversarial discriminative domain adaptation. In CVPR.Google ScholarGoogle Scholar
  65. [65] Tzeng Eric, Hoffman Judy, Zhang Ning, Saenko Kate, and Darrell Trevor. 2014. Deep domain confusion: Maximizing for domain invariance. CoRR abs/1412.3474 (2014).Google ScholarGoogle Scholar
  66. [66] Vaswani Ashish, Shazeer Noam, Parmar Niki, Uszkoreit Jakob, Jones Llion, Gomez Aidan N., Kaiser Lukasz, and Polosukhin Illia. 2017. Attention is all you need. In NIPS. 59986008. Google ScholarGoogle ScholarDigital LibraryDigital Library
  67. [67] Venkateswara Hemanth, Eusebio Jose, Chakraborty Shayok, and Panchanathan Sethuraman. 2017. Deep hashing network for unsupervised domain adaptation. In CVPR.Google ScholarGoogle Scholar
  68. [68] Wang Xiaolong, Girshick Ross B., Gupta Abhinav, and He Kaiming. 2018. Non-local neural networks. In CVPR.Google ScholarGoogle Scholar
  69. [69] Wang Xiaolong and Gupta Abhinav. 2018. Videos as space-time region graphs. In ECCV. 413431.Google ScholarGoogle Scholar
  70. [70] Xie Shaoan, Zheng Zibin, Chen Liang, and Chen Chuan. 2018. Learning semantic representations for unsupervised domain adaptation. In ICML.Google ScholarGoogle Scholar
  71. [71] Xu Minghao, Zhang Jian, Ni Bingbing, Li Teng, Wang Chengjie, Tian Qi, and Zhang Wenjun. 2020. Adversarial domain adaptation with domain mixup. In AAAI. 65026509.Google ScholarGoogle Scholar
  72. [72] Xu Ruijia, Li Guanbin, Yang Jihan, and Lin Liang. 2019. Larger norm more transferable: An adaptive feature norm approach for unsupervised domain adaptation. In ICCV. 14261435.Google ScholarGoogle Scholar
  73. [73] Yan Hongliang, Ding Yukang, Li Peihua, Wang Qilong, Xu Yong, and Zuo Wangmeng. 2017. Mind the class weight bias: Weighted maximum mean discrepancy for unsupervised domain adaptation. In CVPR.Google ScholarGoogle Scholar
  74. [74] Yang Baoyao and Yuen Pong C.. 2019. Cross-domain visual representations via unsupervised graph alignment. In AAAI. 56135620. Google ScholarGoogle ScholarDigital LibraryDigital Library
  75. [75] Zellinger Werner, Grubinger Thomas, Lughofer Edwin, Natschläger Thomas, and Saminger-Platz Susanne. 2017. Central moment discrepancy (CMD) for domain-invariant representation learning. In ICLR.Google ScholarGoogle Scholar
  76. [76] Zhang Baiyan, Ling Hefei, Shen Jialie, Wang Qian, Lei Jie, Shi Yuxuan, Wu Lei, and Li Ping. 2021. Mixture distribution graph network for few shot learning. IEEE Trans. Cogn. Dev. Syst. (2021), 11. https://doi.org/10.1109/TCDS.2021.3075280Google ScholarGoogle Scholar
  77. [77] Zhang Yuchen, Liu Tianle, Long Mingsheng, and Jordan Michael I.. 2019. Bridging theory and algorithm for domain adaptation. In ICML. 74047413.Google ScholarGoogle Scholar
  78. [78] Zhang Yabin, Tang Hui, Jia Kui, and Tan Mingkui. 2019. Domain-symmetric networks for adversarial domain adaptation. In CVPR. 50315040.Google ScholarGoogle Scholar
  79. [79] Zhong Erheng, Fan Wei, Yang Qiang, Verscheure Olivier, and Ren Jiangtao. 2010. Cross validation framework to choose amongst models and datasets for transfer learning. In ECML. Google ScholarGoogle ScholarDigital LibraryDigital Library
  80. [80] Zhu Jun-Yan, Park Taesung, Isola Phillip, and Efros Alexei A.. 2017. Unpaired image-to-image translation using cycle-consistent adversarial networks. In ICCV. 22422251.Google ScholarGoogle Scholar
  81. [81] Zhuo Junbao, Wang Shuhui, Zhang Weigang, and Huang Qingming. 2017. Deep unsupervised convolutional domain adaptation. In ACM MM. 261269. Google ScholarGoogle ScholarDigital LibraryDigital Library

Index Terms

  1. Instance Correlation Graph for Unsupervised Domain Adaptation

    Recommendations

    Comments

    Login options

    Check if you have access through your login credentials or your institution to get full access on this article.

    Sign in

    Full Access

    • Published in

      cover image ACM Transactions on Multimedia Computing, Communications, and Applications
      ACM Transactions on Multimedia Computing, Communications, and Applications  Volume 18, Issue 1s
      February 2022
      352 pages
      ISSN:1551-6857
      EISSN:1551-6865
      DOI:10.1145/3505206
      Issue’s Table of Contents

      Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

      Publisher

      Association for Computing Machinery

      New York, NY, United States

      Publication History

      • Published: 25 January 2022
      • Accepted: 1 September 2021
      • Revised: 1 August 2021
      • Received: 1 January 2021
      Published in tomm Volume 18, Issue 1s

      Permissions

      Request permissions about this article.

      Request Permissions

      Check for updates

      Qualifiers

      • research-article
      • Refereed

    PDF Format

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    Full Text

    View this article in Full Text.

    View Full Text

    HTML Format

    View this article in HTML Format .

    View HTML Format
    About Cookies On This Site

    We use cookies to ensure that we give you the best experience on our website.

    Learn more

    Got it!