Abstract
Domain generalization aims at generalizing the network trained on multiple domains to unknown but related domains. Under the assumption that different domains share the same classes, previous works can build relationships across domains. However, in realistic scenarios, the change of domains is always followed by the change of categories, which raises a difficulty for collecting sufficient aligned categories across domains. Bearing this in mind, this article introduces union domain generalization (UDG) as a new domain generalization scenario, in which the label space varies across domains, and the categories in unknown domains belong to the union of all given domain categories. The absence of categories in given domains is the main obstacle to aligning different domain distributions and obtaining domain-invariant information. To address this problem, we propose category-stitch learning (CSL), which aims at jointly learning the domain-invariant information and completing missing categories in all domains through an improved variational autoencoder and generators. The domain-invariant information extraction and sample generation cross-promote each other to better generalizability. Additionally, we decouple category and domain information and propose explicitly regularizing the semantic information by the classification loss with transferred samples. Thus our method can breakthrough the category limit and generate samples of missing categories in each domain. Extensive experiments and visualizations are conducted on MNIST, VLCS, PACS, Office-Home, and DomainNet datasets to demonstrate the effectiveness of our proposed method.
- [1] . 2018. Metareg: Towards domain generalization using meta-regularization. In Proceedings of the Advances in Neural Information Processing Systems. 998–1008.Google Scholar
- [2] . 2012. Autoencoders, unsupervised learning, and deep architectures. In Proceedings of the ICML Workshop on Unsupervised and Transfer Learning. 37–49.Google Scholar
- [3] . 2007. Analysis of representations for domain adaptation. In Proceedings of the Advances in Neural Information Processing Systems. 137–144.Google Scholar
- [4] . 2018. Partial adversarial domain adaptation. In Proceedings of the European Conference on Computer Vision. 135–150.Google Scholar
Digital Library
- [5] . 2019. Learning to transfer examples for partial domain adaptation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.Google Scholar
Cross Ref
- [6] . 2019. Domain generalization by solving jigsaw puzzles. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2229–2238.Google Scholar
Cross Ref
- [7] . 2010. Exploiting hierarchical context on a large database of object categories. InProceedings of the 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.Google Scholar
- [8] . 2018. Stargan: Unified generative adversarial networks for multi-domain image-to-image translation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 8789–8797.Google Scholar
Cross Ref
- [9] . 2020. Adaptive exploration for unsupervised person re-identification. ACM Transactions on Multimedia Computing, Communications, and Applications 16, 1(2020), 19 pages.
DOI: Google ScholarDigital Library
- [10] . 2016. Tutorial on variational autoencoders. arXiv:1606.05908. Retrieved from https://arxiv.org/abs/1606.05908.Google Scholar
- [11] . 2010. The pascal visual object classes (voc) challenge. International Journal of Computer Vision 88, 2 (2010), 303–338.Google Scholar
Digital Library
- [12] . 2018. Self-ensembling for visual domain adaptation. In Proceedings of the International Conference on Learning Representations. Retrieved from https://openreview.net/forum?id=rkpoTaxA-.Google Scholar
- [13] . 2016. Domain-adversarial training of neural networks. The Journal of Machine Learning Research 17, 1 (2016), 2096–2030.Google Scholar
Cross Ref
- [14] . 2017. Scatter component analysis: A unified framework for domain adaptation and domain generalization. IEEE Transactions on Pattern Analysis and Machine Intelligence 39, 7 (2017), 1414–1430.Google Scholar
Digital Library
- [15] . 2015. Domain generalization for object recognition with multi-task autoencoders. In Proceedings of the IEEE International Conference on Computer Vision. 2551–2559.Google Scholar
Digital Library
- [16] . 2007. Caltech-256 Object Category Dataset. California Institute of Technology.Google Scholar
- [17] . 2012. Undoing the damage of dataset bias. In Proceedings of the European Conference on Computer Vision. Springer, 158–171.Google Scholar
Digital Library
- [18] . 2021. Bottom-up and layerwise domain adaptation for pedestrian detection in thermal images. ACM Transactions on Multimedia Computing, Communications, and Applications 17, 1(2021), 19 pages.
DOI: Google ScholarDigital Library
- [19] . 2013. Auto-encoding variational bayes. arXiv:1312.6114. Retrieved from https://arxiv.org/abs/1312.6114.Google Scholar
- [20] . 2018. Learning latent subspaces in variational autoencoders. In Proceedings of the Advances in Neural Information Processing Systems. 6444–6454.Google Scholar
- [21] . 2012. Imagenet classification with deep convolutional neural networks. In Proceedings of the Advances in Neural Information Processing Systems. 1097–1105.Google Scholar
Digital Library
- [22] . 1997. Information Theory and Statistics. Courier Corporation.Google Scholar
- [23] . 2017. Deeper, broader and artier domain generalization. In Proceedings of the 2017 IEEE International Conference on Computer Vision. IEEE, 5543–5551.Google Scholar
Cross Ref
- [24] . 2018. Learning to generalize: Meta-learning for domain generalization. In Proceedings of the 32nd AAAI Conference on Artificial Intelligence.Google Scholar
Cross Ref
- [25] . 2019. Episodic training for domain generalization. In Proceedings of the International Conference on Computer Vision. Institute of Electrical and Electronics Engineers (IEEE).Google Scholar
Cross Ref
- [26] . 2018. Deep domain generalization via conditional invariant adversarial networks. In Proceedings of the European Conference on Computer Vision. 624–639.Google Scholar
Digital Library
- [27] . 2019. Feature-critic networks for heterogeneous domain generalization. In Proceedings of the International Conference on Machine Learning. 3915–3924.Google Scholar
- [28] . 2019. Compact feature learning for multi-domain image classification. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.Google Scholar
Cross Ref
- [29] . 2021. Domain Generalization via Encoding and Resampling in a Unified Latent Space. In IEEE Transactions on Multimedia.
DOI: Google ScholarCross Ref
- [30] . 2021. Generalized domain adaptation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 1084–1093.Google Scholar
Cross Ref
- [31] . 2013. Domain generalization via invariant feature representation. In Proceedings of the International Conference on Machine Learning. 10–18.Google Scholar
- [32] . 2017. Conditional image synthesis with auxiliary classifier gans. In Proceedings of the International Conference on Machine Learning. PMLR, 2642–2651.Google Scholar
- [33] . 2009. A survey on transfer learning. IEEE Transactions on Knowledge and Data Engineering 22, 10 (2009), 1345–1359.Google Scholar
Digital Library
- [34] . 2020. Exploring category-agnostic clusters for open-set domain adaptation. In Proceedings of the 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition.13864–13872.
DOI: Google ScholarCross Ref
- [35] . 2019. Transferrable prototypical networks for unsupervised domain adaptation. In Proceedings of the 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2234–2242.
DOI: Google ScholarCross Ref
- [36] . 2020. Learning explanations that are hard to vary. (2020).Google Scholar
- [37] . 2019. Moment matching for multi-source domain adaptation. In Proceedings of the 2019 IEEE/CVF International Conference on Computer Vision. 1406–1415.
DOI: Google ScholarCross Ref
- [38] . 2019. Moment matching for multi-source domain adaptation. In Proceedings of the IEEE International Conference on Computer Vision. 1406–1415.Google Scholar
Cross Ref
- [39] . 2008. LabelMe: A database and web-based tool for image annotation. International Journal of Computer Vision 77, 1–3 (2008), 157–173.Google Scholar
Digital Library
- [40] . 2019. Gradient matching generative networks for zero-shot learning. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2168–2178.Google Scholar
Cross Ref
- [41] . 2018. Generalizing across domains via cross-gradient training. In Proceedings of the International Conference on Learning Representations. Retrieved from https://openreview.net/forum?id=r1Dx7fbCW.Google Scholar
- [42] . 2015. Learning structured output representation using deep conditional generative models. In Proceedings of the Advances in Neural Information Processing Systems. 3483–3491.Google Scholar
- [43] . 2016. Generalized deep transfer networks for knowledge propagation in heterogeneous domains. ACM Transactions on Multimedia Computing, Communications, and Applications 12, 4s(2016), 22 pages.
DOI: Google ScholarDigital Library
- [44] . 2018. Latent domain transfer: Crossing modalities with bridging autoencoders. In Proceedings of the ICLR 2019 Conference on Blind Submission.Google Scholar
- [45] . 2011. Unbiased look at dataset bias. In Proceedings of the 2011 IEEE Conference on Computer Vision and Pattern Recognition. IEEE, 1521–1528.Google Scholar
Digital Library
- [46] . 2017. Deep hashing network for unsupervised domain adaptation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 5018–5027.Google Scholar
Cross Ref
- [47] . 2019. SDIT: Scalable and diverse cross-domain image translation. In Proceedings of the 27th ACM International Conference on Multimedia. 1267–1276.Google Scholar
Digital Library
- [48] . 2018. Deep cocktail network: Multi-source unsupervised domain adaptation with category shift. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 3964–3973.Google Scholar
Cross Ref
- [49] . 2014. Exploiting low-rank structure from latent domains for domain generalization. In Proceedings of the European Conference on Computer Vision. Springer, 628–643.Google Scholar
Cross Ref
- [50] . 2019. Heterogeneous domain adaptation via soft transfer network. In Proceedings of the 27th ACM International Conference on Multimedia. 1578–1586.Google Scholar
Digital Library
- [51] . 2021. Equivariant adversarial network for image-to-image translation. ACM Transactions on Multimedia Computing, Communications, and Applications 17, 2s(2021), 14 pages.
DOI: Google ScholarDigital Library
- [52] . 2020. Domain generalization via entropy regularization. Advances in Neural Information Processing Systems 33 (2020).Google Scholar
Index Terms
Category-Stitch Learning for Union Domain Generalization
Recommendations
Learning to Learn with Variational Information Bottleneck for Domain Generalization
Computer Vision – ECCV 2020AbstractDomain generalization models learn to generalize to previously unseen domains, but suffer from prediction uncertainty and domain shift. In this paper, we address both problems. We introduce a probabilistic meta-learning model for domain ...
Sequential Learning for Domain Generalization
Computer Vision – ECCV 2020 WorkshopsAbstractIn this paper we propose a sequential learning framework for Domain Generalization (DG), the problem of training a model that is robust to domain shift by design. Various DG approaches have been proposed with different motivating intuitions, but ...
Learning to Balance Specificity and Invariance for In and Out of Domain Generalization
Computer Vision – ECCV 2020AbstractWe introduce Domain-specific Masks for Generalization, a model for improving both in-domain and out-of-domain generalization performance. For domain generalization, the goal is to learn from a set of source domains to produce a single model that ...






Comments