skip to main content
research-article

Unsupervised Domain Expansion for Visual Categorization

Authors Info & Claims
Published:12 November 2021Publication History
Skip Abstract Section

Abstract

Expanding visual categorization into a novel domain without the need of extra annotation has been a long-term interest for multimedia intelligence. Previously, this challenge has been approached by unsupervised domain adaptation (UDA). Given labeled data from a source domain and unlabeled data from a target domain, UDA seeks for a deep representation that is both discriminative and domain-invariant. While UDA focuses on the target domain, we argue that the performance on both source and target domains matters, as in practice which domain a test example comes from is unknown. In this article, we extend UDA by proposing a new task called unsupervised domain expansion (UDE), which aims to adapt a deep model for the target domain with its unlabeled data, meanwhile maintaining the model’s performance on the source domain. We propose Knowledge Distillation Domain Expansion (KDDE) as a general method for the UDE task. Its domain-adaptation module can be instantiated with any existing model. We develop a knowledge distillation-based learning mechanism, enabling KDDE to optimize a single objective wherein the source and target domains are equally treated. Extensive experiments on two major benchmarks, i.e., Office-Home and DomainNet, show that KDDE compares favorably against four competitive baselines, i.e., DDC, DANN, DAAN, and CDAN, for both UDA and UDE tasks. Our study also reveals that the current UDA models improve their performance on the target domain at the cost of noticeable performance loss on the source domain.

REFERENCES

  1. [1] Asami Taichi, Masumura Ryo, Yamaguchi Yoshikazu, Masataki Hirokazu, and Aono Yushi. 2017. Domain adaptation of DNN acoustic models using knowledge distillation. In Proceedings of the ICASSP.Google ScholarGoogle ScholarCross RefCross Ref
  2. [2] Ben-David Shai, Blitzer John, Crammer Koby, and Pereira Fernando. 2007. Analysis of representations for domain adaptation. In Proceedings of the NeurIPS. Google ScholarGoogle ScholarDigital LibraryDigital Library
  3. [3] Chen Guobin, Choi Wongun, Yu Xiang, Han Tony X., and Chandraker Manmohan. 2017. Learning efficient object detection models with knowledge distillation. In Proceedings of the NeurIPS. Google ScholarGoogle ScholarDigital LibraryDigital Library
  4. [4] Fauw Jeffrey De, Ledsam Joseph R., Romera-Paredes Bernardino, Nikolov Stanislav, Tomasev Nenad, Blackwell Sam, Askham Harry, Glorot Xavier, O’Donoghue Brendan, Visentin Daniel, et al. 2018. Clinically applicable deep learning for diagnosis and referral in retinal disease. Nature Med. 24, 9 (2018), 13421350.Google ScholarGoogle ScholarCross RefCross Ref
  5. [5] Duan Lixin, Tsang Ivor W., Xu Dong, and Maybank Stephen J.. 2009. Domain Transfer SVM for video concept detection. In Proceedings of the CVPR.Google ScholarGoogle Scholar
  6. [6] French, Michal Mackiewicz, Geoffrey and Fisher Mark. 2018. Self-ensembling for visual domain adaptation. In Proceedings of the ICLR.Google ScholarGoogle Scholar
  7. [7] Ganin Yaroslav, Ustinova Evgeniya, Ajakan Hana, Germain Pascal, Larochelle Hugo, Laviolette François, March Mario, and Lempitsky Victor. 2016. Domain-Adversarial Training of Neural Networks. J. Mach. Learn. Res. 17, 59 (2016), 135. Google ScholarGoogle ScholarDigital LibraryDigital Library
  8. [8] Goldblum Micah, Fowl Liam, Feizi Soheil, and Goldstein Tom. 2020. Adversarially robust distillation. In Proceedings of the AAAI.Google ScholarGoogle ScholarCross RefCross Ref
  9. [9] Hinton Geoffrey, Vinyals Oriol, and Dean Jeffrey. 2015. Distilling the Knowledge in a Neural Network. In Proceedings of the NIPS Deep Learning Workshop.Google ScholarGoogle Scholar
  10. [10] Jiang Wei, Zavesky Eric, Chang Shih-Fu, and Loui Alex. 2008. Cross-domain learning methods for high-level visual concept classification. In Proceedings of the ICIP.Google ScholarGoogle ScholarCross RefCross Ref
  11. [11] Kang Guoliang, Jiang Lu, Yang Yi, and Hauptmann Alexander G.. 2019. Contrastive Adaptation Network for Unsupervised Domain Adaptation. In Proceedings of the CVPR.Google ScholarGoogle ScholarCross RefCross Ref
  12. [12] Kermany Daniel S, Goldbaum Michael, Cai Wenjia, Valentim Carolina CS, Liang Huiying, Baxter Sally L, McKeown Alex, Yang Ge, Wu Xiaokang, Yan Fangbing, et al. 2018. Identifying medical diagnoses and treatable diseases by image-based deep learning. Cell 172, 5 (2018), 11221131.Google ScholarGoogle ScholarCross RefCross Ref
  13. [13] Li Shuang, Liu Chi Harold, Xie Binhui, Su Limin, Ding Zhengming, and Huang Gao. 2019. Joint Adversarial Domain Adaptation. In Proceedings of the ACMMM. Google ScholarGoogle ScholarDigital LibraryDigital Library
  14. [14] Liu Yifan, Chen Ke, Liu Chris, Qin Zengchang, Luo Zhenbo, and Wang Jingdong. 2019. Structured Knowledge Distillation for Semantic Segmentation. In Proceedings of the CVPR.Google ScholarGoogle ScholarCross RefCross Ref
  15. [15] Long Mingsheng, Cao Yue, Wang Jianmin, and Jordan Michael I.. 2015. Learning Transferable Features with Deep Adaptation Networks. In Proceedings of the ICML. Google ScholarGoogle ScholarDigital LibraryDigital Library
  16. [16] Long Miingsheng, Cao Zhangjie, Wang Jianmin, and Jordan Micheal I.. 2018. Conditional adversarial domain adaptation. In Proceedings of the NeurIPS. Google ScholarGoogle ScholarDigital LibraryDigital Library
  17. [17] Long Mingsheng, Zhu Han, Wang Jianmin, and Jordan Michael I.. 2017. Deep transfer learning with joint adaptation networks. In Proceedings of the ICML. Google ScholarGoogle ScholarDigital LibraryDigital Library
  18. [18] Maaten Laurens van der and Hinton Geoffrey. 2008. Visualizing data using t-SNE. J. Mach. Learn. Res. 9(Nov.2008), 25792605.Google ScholarGoogle Scholar
  19. [19] Meng Zhong, Li Jinyu, Gaur Yashesh, and Gong Yifan. 2019. Domain adaptation via teacher-student learning for end-to-end speech recognition. In Proceedings of the ASRU.Google ScholarGoogle ScholarCross RefCross Ref
  20. [20] Meng Zhong, Li Jinyu, Gong Yifan, and Juang Biing-Hwang. 2018. Adversarial teacher-student learning for unsupervised domain adaptation. In Proceedings of the ICASSP.Google ScholarGoogle ScholarCross RefCross Ref
  21. [21] Mirzadeh Seyed Iman, Farajtabar Mehrdad, Li Ang, Levine Nir, Matsukawa Akihiro, and Ghasemzadeh Hassan. 2020. Improved knowledge distillation via teacher assistant. In Proceedings of the AAAI.Google ScholarGoogle ScholarCross RefCross Ref
  22. [22] Paszke Adam, Gross Sam, Massa Francisco, Lerer Adam, Bradbury James, Chanan Gregory, Killeen Trevor, Lin Zeming, Gimelshein Natalia, Antiga Luca, Desmaison Alban, Kopf Andreas, Yang Edward, DeVito Zachary, Raison Martin, Tejani Alykhan, Chilamkurthy Sasank, Steiner Benoit, Fang Lu, Bai Junjie, and Chintala Soumith. 2019. PyTorch: An imperative style, high-performance deep learning library. In Proceedings of the NeurIPS. Google ScholarGoogle ScholarDigital LibraryDigital Library
  23. [23] Peng Xingchao, Bai Qinxun, Xia Xide, Huang Zijun, Saenko Kate, and Wang Bo. 2019. Moment matching for multi-source domain adaptation. In Proceedings of the ICCV.Google ScholarGoogle ScholarCross RefCross Ref
  24. [24] Pinheiro Pedro O.. 2018. Unsupervised domain adaptation with similarity learning. In Proceedings of the CVPR.Google ScholarGoogle ScholarCross RefCross Ref
  25. [25] Romero Adriana, Ballas Nicolas, Kahou Samira Ebrahimi, Chassang Antoine, Gatta Carlo, and Bengio Yoshua. 2015. FitNets: Hints for thin deep nets. Proceedings of the ICLR (2015).Google ScholarGoogle Scholar
  26. [26] Saito Kuniaki, Kim Donghyun, Sclaroff Stan, Darrell Trevor, and Saenko Kate. 2019. Semi-Supervised Domain Adaptation via Minimax Entropy. In Proceedings of the ICCV.Google ScholarGoogle ScholarCross RefCross Ref
  27. [27] Saito Kuniaki, Watanabe Kohei, Ushiku Yoshitaka, and Harada Tatsuya. 2018. Maximum classifier discrepancy for unsupervised domain adaptation. In Proceedings of the CVPR.Google ScholarGoogle ScholarCross RefCross Ref
  28. [28] Saputra Muhamad Risqi U., Gusmao Pedro P. B. De, Almalioglu Yasin, Markham Andrew, and Trigoni Niki. 2019. Distilling Knowledge From a Deep Pose Regressor Network. In Proceedings of the ICCV.Google ScholarGoogle ScholarCross RefCross Ref
  29. [29] Selvaraju Ramprasaath R., Cogswell Michael, Das Abhishek, Vedantam Ramakrishna, Parikh Devi, and Batra Dhruv. 2017. Grad-CAM: Visual Explanations From Deep Networks via Gradient-based Localization. In Proceedings of the ICCV.Google ScholarGoogle ScholarCross RefCross Ref
  30. [30] Shi Yangyang, Hwang Mei-Yuh, Lei Xin, and Sheng Haoyu. 2019. Knowledge distillation for recurrent neural network language modeling with trust regularization. In Proceedings of the ICASSP.Google ScholarGoogle ScholarCross RefCross Ref
  31. [31] Sun Baochen and Saenko Kate. 2016. Deep CORAL: Correlation Alignment for Deep Domain Adaptation. In Proceedings of the ECCV Workshop.Google ScholarGoogle ScholarCross RefCross Ref
  32. [32] Tian Yonglong, Krishnan Dilip, and Isola Phillip. 2020. Contrastive representation distillation. In Proceedings of the ICLR.Google ScholarGoogle Scholar
  33. [33] Toldo Marco, Maracani Andrea, Michieli Umberto, and Zanuttigh Pietro. 2020. Unsupervised Domain Adaptation in Semantic Segmentation: A Review. Technologies 8, 2 (2020), 35.Google ScholarGoogle ScholarCross RefCross Ref
  34. [34] Tzeng Eric, Hoffman Judy, Saenko Kate, and Darrell Trevor. 2017. Adversarial discriminative domain adaptation. In Proceedings of the CVPR.Google ScholarGoogle ScholarCross RefCross Ref
  35. [35] Tzeng Eric, Hoffman Judy, Zhang Ning, Saenko Kate, and Darrell Trevor. 2014. Deep Domain Confusion: Maximizing for Domain Invariance. Retrieved from https://ArXivabs/1412.3474.Google ScholarGoogle Scholar
  36. [36] Venkateswara Hemanth, Eusebio Jose, Chakraborty Shayok, and Panchanathan Sethuraman. 2017. Deep Hashing Network for Unsupervised Domain Adaptation. In Proceedings of the CVPR.Google ScholarGoogle ScholarCross RefCross Ref
  37. [37] Wang Weisen, Xu Zhiyan, Yu Weihong, Zhao Jianchun, Yang Jingyuan, He Feng, Yang Zhikun, Chen Di, Ding Dayong, Chen Youxin, and Li Xirong. 2019. Two-Stream CNN with Loose Pair Training for Multi-modal AMD Categorization. In Proceedings of the MICCAI.Google ScholarGoogle ScholarDigital LibraryDigital Library
  38. [38] Wang Ximei, Jin Ying, Long Mingsheng, Wang Jianmin, and Jordan Michael I.. 2019. Transferable Normalization: Towards Improving Transferability of Deep Neural Networks. In Proceedings of the NeurIPS. Google ScholarGoogle ScholarDigital LibraryDigital Library
  39. [39] Wang Ximei, Li Liang, Ye Weirui, Long Mingsheng, and Wang Jianmin. 2019. Transferable attention for domain adaptation. In Proceedings of the AAAI. Google ScholarGoogle ScholarDigital LibraryDigital Library
  40. [40] Yang Jun, Yan Rong, and Hauptmann Alexander G.. 2007. Cross-domain video concept detection using adaptive SVMs. In Proceedings of the ACMMM. Google ScholarGoogle ScholarDigital LibraryDigital Library
  41. [41] Yu Xiaofeng Liu, B. V. K. Vijaya, Kumar Jinsong, Wang Yang Zou, and Zhiding. 2019. Confidence regularized self-training. In Proceedings of the ICCV.Google ScholarGoogle Scholar
  42. [42] Yu Chaohui, Wang Jindong, Chen Yiqiang, and Huang Meiyu. 2019. Transfer Learning with Dynamic Adversarial Adaptation Network. In Proceedings of the ICDM.Google ScholarGoogle ScholarCross RefCross Ref
  43. [43] Zhang Linfeng, Song Jiebo, Gao Anni, Chen Jingwei, Bao Chenglong, and Ma Kaisheng. 2019. Be your own teacher: Improve the performance of convolutional neural networks via self distillation. In Proceedings of the ICCV.Google ScholarGoogle ScholarCross RefCross Ref
  44. [44] Zhang Peng, Su Li, Li Liang, Bao BingKun, Cosman Pamela, Li GuoRong, and Huang Qingming. 2019. Training Efficient Saliency Prediction Models with Knowledge Distillation. In Proceedings of the ACMMM. Google ScholarGoogle ScholarDigital LibraryDigital Library
  45. [45] Zhang Quanshi, Wang Wenguan, and Zhu Song-Chun. 2018. Examining CNN Representations with Respect to Dataset Bias. In Proceedings of the AAAI. Google ScholarGoogle ScholarDigital LibraryDigital Library
  46. [46] Zhang Ying, Xiang Tao, Hospedales Timothy M., and Lu Huchuan. 2018. Deep mutual learning. In Proceedings of the CVPR.Google ScholarGoogle ScholarCross RefCross Ref
  47. [47] Zou Yang, Yu Zhiding, Kumar B. V. K. Vijaya, and Wang Jinsong. 2018. Unsupervised domain adaptation for semantic segmentation via class-balanced self-training. In Proceedings of the ECCV.Google ScholarGoogle ScholarCross RefCross Ref

Index Terms

  1. Unsupervised Domain Expansion for Visual Categorization

        Recommendations

        Comments

        Login options

        Check if you have access through your login credentials or your institution to get full access on this article.

        Sign in

        Full Access

        • Published in

          cover image ACM Transactions on Multimedia Computing, Communications, and Applications
          ACM Transactions on Multimedia Computing, Communications, and Applications  Volume 17, Issue 4
          November 2021
          529 pages
          ISSN:1551-6857
          EISSN:1551-6865
          DOI:10.1145/3492437
          Issue’s Table of Contents

          Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

          Publisher

          Association for Computing Machinery

          New York, NY, United States

          Publication History

          • Published: 12 November 2021
          • Accepted: 1 January 2021
          • Revised: 1 December 2020
          • Received: 1 July 2020
          Published in tomm Volume 17, Issue 4

          Permissions

          Request permissions about this article.

          Request Permissions

          Check for updates

          Qualifiers

          • research-article
          • Refereed

        PDF Format

        View or Download as a PDF file.

        PDF

        eReader

        View online with eReader.

        eReader

        Full Text

        View this article in Full Text.

        View Full Text

        HTML Format

        View this article in HTML Format .

        View HTML Format
        About Cookies On This Site

        We use cookies to ensure that we give you the best experience on our website.

        Learn more

        Got it!