skip to main content
research-article

Balanced and Accurate Pseudo-Labels for Semi-Supervised Image Classification

Published:31 October 2022Publication History
Skip Abstract Section

Abstract

Image classification by semi-supervised learning has recently become a hot spot, and the Co-Training framework is an important method of semi-supervised image classification. In the traditional Co-Training structure, the sub-networks will generate pseudo-labels for each other, and these pseudo-labels will further be used as a supervisory signal for model training. However, the pseudo-labels will hurt classification performance because of their low accuracy and unbalanced distribution. In this article, we are trying to solve the preceding two problems by designing the Balanced Module (BM) and Gaussian Mixture Module (GMM), and propose BAPS (the Balanced and Accurate Pseudo-labels for Semi-supervised image classification). In BM, the two sub-networks jointly predict the unlabeled images, then select the pseudo-labels with a high-confidence threshold to perform the balancing operation to obtain the initial samples with balanced distribution of each category. In GMM, referring to the common practice of the Learning from Noise Labels task, we use GMM to fit the loss distribution of images with pseudo-labels output by BM, then clean samples and noise samples are divided based on the observation that the loss of correctly labeled images is generally smaller than that of wrongly labeled ones. Through BM and GMM, pseudo-labels with balanced distribution and high accuracy are obtained for the subsequent model training process. Our model has achieved better classification accuracy than most state-of-the-art semi-supervised image classification algorithms on the CIFAR-10/100 and SVHN datasets, and further ablation experiments demonstrate the effectiveness of our BAPS. The source code of BAPS will be available at https://github.com/zhaojianaaa.

REFERENCES

  1. [1] Sandler Mark, Howard Andrew, Zhu Menglong, Zhmoginov Andrey, and Chen Liang-Chieh. 2018. MobileNetV2: Inverted residuals and linear bottlenecks. arXiv preprint arXiv:1801.04381 (2018).Google ScholarGoogle Scholar
  2. [2] Hu Jie, Shen Li, Albanie Samuel, Sun Gang, and Wu Enhua. 2019. Squeeze-and-excitation networks. IEEE Transactions on Pattern Analysis and Machine Intelligence 42, 8 (2019), 20112023.Google ScholarGoogle ScholarDigital LibraryDigital Library
  3. [3] Zoph Barret, Vasudevan Vijay, Shlens Jonathon, and Le Quoc V.. 2017. Learning transferable architectures for scalable image recognition. arXiv preprint arXiv:1707.07012 (2017).Google ScholarGoogle Scholar
  4. [4] Chen Yunpeng, Li Jianan, Xiao Huaxin, Jin Xiaojie, Yan Shuicheng, and Feng Jiashi. 2017. Dual path networks. arXiv preprint arXiv:1707.01629 (2017).Google ScholarGoogle Scholar
  5. [5] Zhang Xingcheng, Li Zhizhong, Loy Chen Change, and Lin Dahua. 2017. PolyNet: A pursuit of structural diversity in very deep networks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR’17). 39003908.Google ScholarGoogle ScholarCross RefCross Ref
  6. [6] Bochkovskiy Alexey, Wang Chien-Yao, and Mark Liao Hong-Yuan. 2020. YOLOv4: Optimal speed and accuracy of object detection. arXiv preprint arXiv:2004.10934 (2020).Google ScholarGoogle Scholar
  7. [7] Pang Jiangmiao, Chen Kai, Shi Jianping, Feng Huajun, Quyang Wanli, and Lin Dahua. 2019. Libra R-CNN: Towards balanced learning for object detection. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR’19). 821830.Google ScholarGoogle ScholarCross RefCross Ref
  8. [8] Badrinarayanan Vijay, Kendall Alex, and Cipolla Roberto. 2017. SegNet: A deep convolutional encoder-decoder architecture for image segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence 39, 12 (2017), 24812495.Google ScholarGoogle Scholar
  9. [9] Li Yi, Qi Haozhi, Dai Jifeng, Ji Xiangyang, and Wei Yichen. 2017. Fully convolutional instance-aware semantic segmentation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR’17). 23592367.Google ScholarGoogle ScholarCross RefCross Ref
  10. [10] Reed Scott E., Lee Honglak, Anguelov Dragomir, Szegedy Christian, Erhan Dumitru, and Rabinovich Andrew. 2015. Training deep neural networks on noisy labels with bootstrapping. In Proceedings of the International Conference on Learning Representations (ICLR’15). 111.Google ScholarGoogle Scholar
  11. [11] Guo Hongyu and Viktor Herna L.. 2004. Learning from imbalanced data sets with boosting and data generation: The DataBoost-IM approach. ACM SIGKDD Explorations Newsletter 6, 1 (2004), 3039.Google ScholarGoogle ScholarDigital LibraryDigital Library
  12. [12] Krawczyk Bartosz, Galar Mikel, Jelen Lukasz, and Herrera Francisco. 2016. Evolutionary undersampling boosting for imbalanced classification of breast cancer malignancy. Applied Soft Computing 38 (2016), 714726.Google ScholarGoogle ScholarDigital LibraryDigital Library
  13. [13] Qian Yun, Liang Yanchun, Li Mu, Feng Guoxiang, and Sha Xiaohu. 2014. A resampling ensemble algorithm for classification of imbalance problems. Neurocomputing 143 (2014), 5767.Google ScholarGoogle ScholarDigital LibraryDigital Library
  14. [14] Hon Yun, Li Li, Li Bailin, and Liu Jiajia. 2019. An anti-noise ensemble algorithm for imbalance classification. Intelligent Data Analysis 23, 6 (2019), 12051217.Google ScholarGoogle ScholarCross RefCross Ref
  15. [15] Blum Avrim and Mitchell Tom. 1998. Combining labeled and unlabeled data with co-training. In Proceedings of the 11th Annual Conference on Computational Learning Theory. 92100.Google ScholarGoogle ScholarDigital LibraryDigital Library
  16. [16] Qiao Siyuan, Shen Wei, Zhang Zhishuai, Wang Bo, and Yuille Alan. 2018. Deep Co-Training for semi-supervised image recognition. In Proceedings of the European Conference on Computer Vision (ECCV’18). 117.Google ScholarGoogle ScholarDigital LibraryDigital Library
  17. [17] Chen Dongdong, Wang Wei, Gao Wei, and Zhou Zhihua. 2018. Tri-net for semi-supervised deep learning. In Proceedings of the 27th International Joint Conference on Artificial Intelligence (IJCAI’18). 20142020.Google ScholarGoogle ScholarDigital LibraryDigital Library
  18. [18] Mo Jianwen, Gan Yuwan, and Yuan Hua. 2021. Weighted pseudo labeled data and mutual learning for semi-supervised classification. IEEE Access 9 (2021), 3652236534.Google ScholarGoogle ScholarCross RefCross Ref
  19. [19] Laine Samuli and Aila Timo. 2017. Temporal ensembling for semi-supervised learning. In Proceedings of the International Conference on Learning Representations (ICLR’17). 113.Google ScholarGoogle Scholar
  20. [20] Tarvainen Antti and Valpola Harri. 2017. Mean teachers are better role models: Weight-averaged consistency targets improve semi-supervised deep learning results. In Proceedings of the Conference on Neural Information Processing Systems (NIPS’17). 116.Google ScholarGoogle Scholar
  21. [21] Xie Qizhe, Dai Zihang, Hovy Eduard, Luong Minhthang, and Le Quoc V.. 2020. Unsupervised data augmentation for consistency training. In Proceedings of the Conference on Neural Information Processing Systems (NIPS’20). 119.Google ScholarGoogle Scholar
  22. [22] Sohn Kihyuk, Berthelot David, Li Chun-Liang, Zhang Zizhao, Carlini Nicholas, Cubuk Ekin D., Kurakin Alex, Zhang Han, and Raffel Colin. 2020. FixMatch: Simplifying semi-supervised learning with consistency and confidence. In Proceedings of the Conference on Neural Information Processing Systems (NIPS’20). 121.Google ScholarGoogle Scholar
  23. [23] Zhang Hongyi, Cisse Moustapha, Dauphin Yann N., and Lopez-Paz David. 2018. Mixup: Beyond empirical risk minimization. In Proceedings of the International Conference on Learning Representations (ICLR’18). 113.Google ScholarGoogle Scholar
  24. [24] Berthelot David, Carlini Nicholas, Goodfellow Ian, Oliver Avital, Papernot Nicolas, and Raffel Colin. 2019. MixMatch: A holistic approach to semi-supervised learning. In Proceedings of the Conference on Neural Information Processing Systems (NIPS’19). 114.Google ScholarGoogle Scholar
  25. [25] Berthelot David, Carlini Nicholas, Cubuk Ekin D., Kurakin Alex, Zhang Han, Raffel Colin, and Sohn Kihyuk. 2020. ReMixMatch: Semi-supervised learning with distribution alignment and augmentation anchoring. In Proceedings of the International Conference on Learning Representations (ICLR’20). 113.Google ScholarGoogle Scholar
  26. [26] Goodfellow Ian J., Pouget-Abadie Jean, Mirza Mehdi, Xu Bing, Warde-Farley David, Ozair Sherjil, Courville Aaron, and Bengio Yoshua. 2014. Generative adversarial nets. In Advances in Neural Information Processing Systems 27, 2672-2680.Google ScholarGoogle ScholarDigital LibraryDigital Library
  27. [27] Salimans Tim, Goodfellow Ian, Zaremba Wojciech, Cheung Vicki, Radford Alec, and Chen Xi. 2016. Improved techniques for training GANs. In Proceedings of the Conference on Neural Information Processing Systems (NIPS’16). 111.Google ScholarGoogle Scholar
  28. [28] Odena Augustus. 2016. Semi-supervised learning with generative adversarial networks. In Proceedings of the International Conference on Machine Learning (ICML’16). 13.Google ScholarGoogle Scholar
  29. [29] Park Sungwon, Han Sungwon, Kim Sundong, Kim Danu, Park Sungkyu, Hong Seunghoon, and Cha Meeyoung. 2021. Improving unsupervised image clustering with robust learning. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR’21). 1227812287.Google ScholarGoogle ScholarCross RefCross Ref
  30. [30] Nguyen Duc Tam, Mummadi Chaithanya Kumar, Ngo Thi Phuong Nhung, Nguyen Thi Hoai Phuong, Beggel Laura, and Brox Thomas. 2020. SELF: Learning to filter noisy labels with self-ensembling. In Proceedings of the International Conference on Learning Representations (ICLR’20). 115.Google ScholarGoogle Scholar
  31. [31] Arazo Eric, Ortego Diego, Albert Paul, O'Connor Noel E., and McGuinnes Kevin. 2019. Unsupervised label noise modeling and loss correction. In Proceedings of the International Conference on Machine Learning (ICML’19). 112.Google ScholarGoogle Scholar
  32. [32] Li Junnan, Socher Richard, and Hoi Steven C. H.. 2020. DIVIDEMIX: Learning with noisy labels as semi-supervised learning. In Proceedings of the International Conference on Learning Representations (ICLR’20). 114.Google ScholarGoogle Scholar
  33. [33] Jiang Lu, Zhou Zhengyuan, Leung Thomas, Li Lijia, and Li Feifei. 2018. MentorNet: Learning data-driven curriculum for very deep neural networks on corrupted labels. In Proceedings of the International Conference on Machine Learning (ICML’18). 110.Google ScholarGoogle Scholar
  34. [34] Malach Eran and Shalev-Shwartz Shai. 2017. Decoupling “when to update” from “how to update.” In Proceedings of the Conference on Neural Information Processing Systems (NIPS’17). 119.Google ScholarGoogle Scholar
  35. [35] Han Bo, Yao Quanming, Yu Xingrui, Niu Gang, Xu Miao, Hu Weihua, Tsang Ivor W., and Sugiyama Masashi. 2018. Co-teaching: Robust training of deep neural networks with extremely noisy labels. In Proceedings of the Conference on Neural Information Processing Systems (NIPS’18). 111.Google ScholarGoogle Scholar
  36. [36] Zhong Zhisheng, Cui Jiequan, Liu Shu, and Jia Jiaya. 2021. Improving calibration for long-tailed recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR’21). 1648916498.Google ScholarGoogle ScholarCross RefCross Ref
  37. [37] Wei Chen, Sohn Kihyuk, Mellina Clayton, Yuille Alan, and Yang Fan. 2021. CReST: A class-rebalancing self-training framework for imbalanced semi-supervised learning. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR’21). 1085710866.Google ScholarGoogle ScholarCross RefCross Ref
  38. [38] Arpit Devansh, ebski Stanislaw Jastrz, Ballas Nicolas, Krueger David, Bengio Emmanuel, Kanwal Maxinder S., Maharaj Tegan, et al. 2017. A closer look at memorization in deep networks. In Proceedings of the International Conference on Machine Learning (ICML’17). 233242.Google ScholarGoogle Scholar
  39. [39] Chen Pengfei, Liao Benben, Chen Guangyong, and Zhang Shengyu. 2019. Understanding and utilizing deep neural networks trained with noisy labels. In Proceedings of the International Conference on Machine Learning (ICML’19). 10621070.Google ScholarGoogle Scholar
  40. [40] Krizhevsky A.. 2009. Learning Multiple Layers of Features from Tiny Images. Technical Report. University of Toronto.Google ScholarGoogle Scholar
  41. [41] Netzer Y., Wang T., Coates A., Bissacco A., Wu B., and Ng A. Y.. 2011. Reading digits in natural images with unsupervised feature learning. In Proceedings of the NIPS Workshop on Deep Learning and Unsupervised Feature Learning.Google ScholarGoogle Scholar
  42. [42] Zagoruyko Sergey and Komodakis Nikos. 2016. Wide residual networks. arXiv preprint arXiv:1605.07146 (2016).Google ScholarGoogle Scholar
  43. [43] He Kaiming, Zhang Xiangyu, Ren Shaoqing, and Sun Jian. 2016. Deep residual learning for image recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR’16). 770778.Google ScholarGoogle ScholarCross RefCross Ref
  44. [44] Cubuk E. D., Zoph B., Shlens J., and Le Q. V.. 2019. RandAugment: Practical automated data augmentation with a reduced search space. arXiv preprint arXiv:1909.13719 (2019).Google ScholarGoogle Scholar
  45. [45] Kim Byoungjip, Choo Jinho, Kwon Y.-D., Joe S., Min S., and Gwon Y.. 2021. SelfMatch: Combining contrastive self-supervision and consistency for semi-supervised learning. arXiv preprint arXiv:2101.06480 (2021).Google ScholarGoogle Scholar
  46. [46] Miyato Takeru, Maeda Shin-Ichi, Koyama Masanori, and Ishii Shin. 2019. Virtual adversarial training: A regularization method for supervised and semi-supervised learning. IEEE Transactions on Pattern Analysis and Machine Intelligence 41, 8 (2019), 19791993.Google ScholarGoogle ScholarCross RefCross Ref
  47. [47] Wang Lei, Sun Yu, and Wang Zheng. 2021. CCS-GAN: A semi-supervised generative adversarial network for image classification. Visual Computer. Published online, 2021.Google ScholarGoogle Scholar
  48. [48] Cevikalp Hakan, Benligiray Burak, and Nezih Gerek Omer. 2020. Semi-supervised robust deep neural networks for multi-label image classification. Pattern Recognition 100 (2020), 107164.Google ScholarGoogle ScholarDigital LibraryDigital Library
  49. [49] Chen Jiaming, Yang Meng, and Ling Jie. 2021. Attention-based label consistency for semi-supervised deep learning based image classification. Neurocomputing 453 (2021), 731741.Google ScholarGoogle ScholarDigital LibraryDigital Library

Index Terms

  1. Balanced and Accurate Pseudo-Labels for Semi-Supervised Image Classification

      Recommendations

      Comments

      Login options

      Check if you have access through your login credentials or your institution to get full access on this article.

      Sign in

      Full Access

      • Published in

        cover image ACM Transactions on Multimedia Computing, Communications, and Applications
        ACM Transactions on Multimedia Computing, Communications, and Applications  Volume 18, Issue 3s
        October 2022
        381 pages
        ISSN:1551-6857
        EISSN:1551-6865
        DOI:10.1145/3567476
        • Editor:
        • Abdulmotaleb El Saddik
        Issue’s Table of Contents

        Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

        Publisher

        Association for Computing Machinery

        New York, NY, United States

        Publication History

        • Published: 31 October 2022
        • Online AM: 2 May 2022
        • Accepted: 13 December 2021
        • Received: 27 September 2021
        Published in tomm Volume 18, Issue 3s

        Permissions

        Request permissions about this article.

        Request Permissions

        Check for updates

        Qualifiers

        • research-article
        • Refereed

      PDF Format

      View or Download as a PDF file.

      PDF

      eReader

      View online with eReader.

      eReader

      Full Text

      View this article in Full Text.

      View Full Text

      HTML Format

      View this article in HTML Format .

      View HTML Format
      About Cookies On This Site

      We use cookies to ensure that we give you the best experience on our website.

      Learn more

      Got it!