skip to main content
research-article

Optimizing Performance of Federated Person Re-identification: Benchmarking and Analysis

Published:23 January 2023Publication History
Skip Abstract Section

Abstract

Increasingly stringent data privacy regulations limit the development of person re-identification (ReID) because person ReID training requires centralizing an enormous amount of data that contains sensitive personal information. To address this problem, we introduce federated person re-identification (FedReID)—implementing federated learning, an emerging distributed training method, to person ReID. FedReID preserves data privacy by aggregating model updates, instead of raw data, from clients to a central server. Furthermore, we optimize the performance of FedReID under statistical heterogeneity via benchmark analysis. We first construct a benchmark with an enhanced algorithm, two architectures, and nine person ReID datasets with large variances to simulate the real-world statistical heterogeneity. The benchmark results present insights and bottlenecks of FedReID under statistical heterogeneity, including challenges in convergence and poor performance on datasets with large volumes. Based on these insights, we propose three optimization approaches: (1) we adopt knowledge distillation to facilitate the convergence of FedReID by better transferring knowledge from clients to the server, (2) we introduce client clustering to improve the performance of large datasets by aggregating clients with similar data distributions, and (3) we propose cosine distance weight to elevate performance by dynamically updating the weights for aggregation depending on how well models are trained in clients. Extensive experiments demonstrate that these approaches achieve satisfying convergence with much better performance on all datasets. We believe that FedReID will shed light on implementing and optimizing federated learning on more computer vision applications.

REFERENCES

  1. [1] Acar Durmus Alp Emre, Zhao Yue, Matas Ramon, Mattina Matthew, Whatmough Paul, and Saligrama Venkatesh. 2020. Federated learning based on dynamic regularization. In Proceedings of the International Conference on Learning Representations.Google ScholarGoogle Scholar
  2. [2] Baltieri Davide, Vezzani Roberto, and Cucchiara Rita. 2011. 3DPeS: 3D people dataset for surveillance and forensics. In Proceedings of the 2011 Joint ACM Workshop on Human Gesture and Behavior Understanding (J-HGBU’11). ACM, New York, NY, 5964. Google ScholarGoogle ScholarDigital LibraryDigital Library
  3. [3] Caldas Sebastian, Wu Peter, Li Tian, Konecný Jakub, McMahan H. Brendan, Smith Virginia, and Talwalkar Ameet. 2018. LEAF: A benchmark for federated settings. CoRR abs/1812.01097 (2018). http://arxiv.org/abs/1812.01097.Google ScholarGoogle Scholar
  4. [4] Chen Tianlong, Ding Shaojin, Xie Jingyi, Yuan Ye, Chen Wuyang, Yang Yang, Ren Zhou, and Wang Zhangyang. 2019. ABD-Net: Attentive but diverse person re-identification. In Proceedings of the IEEE/CVF International Conference on Computer Vision. 83518361.Google ScholarGoogle ScholarCross RefCross Ref
  5. [5] Chen Yiqiang, Qin Xin, Wang Jindong, Yu Chaohui, and Gao Wen. 2020. FedHealth: A federated transfer learning framework for wearable healthcare. IEEE Intelligent Systems 35 (2020), 83–93.Google ScholarGoogle ScholarCross RefCross Ref
  6. [6] Cheng D. S., Cristani M., Stoppa M., Bazzani L., and Murino V.. 2011. Custom pictorial structures for re-identification. In Proceedings of the British Machine Vision Conference (BMVC’11).Google ScholarGoogle ScholarCross RefCross Ref
  7. [7] Cohen Gregory, Afshar Saeed, Tapson Jonathan, and Schaik Andre Van. 2017. EMNIST: Extending MNIST to handwritten letters. In Proceedings of the 2017 International Joint Conference on Neural Networks (IJCNN’17). IEEE, Los Alamitos, CA, 29212926.Google ScholarGoogle ScholarCross RefCross Ref
  8. [8] Custers Bart, Sears Alan M., Dechesne Francien, Georgieva Ilina, Tani Tommaso, and Hof Simone van der. 2019. EU Personal Data Protection in Policy and Practice. Springer.Google ScholarGoogle ScholarCross RefCross Ref
  9. [9] Dean Jeffrey, Corrado Greg, Monga Rajat, Chen Kai, Devin Matthieu, Mao Mark, Ranzato Marc’ aurelio, et al. 2012. Large scale distributed deep networks. In Advances in Neural Information Processing Systems 25, Pereira F., Burges C. J. C., Bottou L., and Weinberger K. Q. (Eds.). Curran Associates, 1223–1231. http://papers.nips.cc/paper/4687-large-scale-distributed-deep-networks.pdf.Google ScholarGoogle Scholar
  10. [10] Deng J., Dong W., Socher R., Li L.-J., Li K., and Fei-Fei L.. 2009. ImageNet: A large-scale hierarchical image database. In Proceedings of the 2009 IEEE Conference on Computer Vision and Pattern Recognition.Google ScholarGoogle ScholarCross RefCross Ref
  11. [11] Gray Douglas and Tao Hai. 2008. Viewpoint invariant pedestrian recognition with an ensemble of localized features. In Proceedings of the European Conference on Computer Vision. 262275.Google ScholarGoogle ScholarDigital LibraryDigital Library
  12. [12] Hao Tianshu, Huang Yunyou, Wen Xu, Gao Wanling, Zhang Fan, Zheng Chen, Wang Lei, et al. 2018. Edge AIBench: Towards comprehensive end-to-end edge computing benchmarking. In Proceedings of the 2018 BenchCouncil International Symposium on Benchmarking, Measuring, and Optimizing.Google ScholarGoogle Scholar
  13. [13] He K., Zhang X., Ren S., and Sun J.. 2016. Deep residual learning for image recognition. In Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR’16). 770778.Google ScholarGoogle ScholarCross RefCross Ref
  14. [14] Hinton Geoffrey, Vinyals Oriol, and Dean Jeffrey. 2015. Distilling the knowledge in a neural network. In Proceedings of the NIPS Deep Learning and Representation Learning Workshop. http://arxiv.org/abs/1503.02531.Google ScholarGoogle Scholar
  15. [15] Hirzer Martin, Roth Peter, Beleznai Csaba, and Bischof Horst. 2011. Person re-identification by descriptive and discriminative classification. In Proceedings of the Scandinavian Conference on Image Analysis (SCIA’11). 91102.Google ScholarGoogle ScholarCross RefCross Ref
  16. [16] Hu Junlin, Lu Jiwen, and Tan Yap-Peng. 2015. Deep transfer metric learning. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 325333.Google ScholarGoogle ScholarCross RefCross Ref
  17. [17] Hu Sixu, Li Yuan, Liu Xu, Li Qinbin, Wu Zhaomin, and He Bingsheng. 2020. The OARF benchmark suite: Characterization and implications for federated learning systems. arXiv preprint arXiv:2006.07856 (2020).Google ScholarGoogle Scholar
  18. [18] Kairouz Peter, McMahan H. Brendan, Avent Brendan, Bellet Aurélien, Bennis Mehdi, Bhagoji Arjun Nitin, Bonawitz Keith, et al. 2019. Advances and open problems in federated learning. arXiv preprint arXiv:1912.04977 (2019).Google ScholarGoogle Scholar
  19. [19] Karimireddy Sai Praneeth, Kale Satyen, Mohri Mehryar, Reddi Sashank, Stich Sebastian, and Suresh Ananda Theertha. 2020. Scaffold: Stochastic controlled averaging for federated learning. In Proceedings of the International Conference on Machine Learning. 51325143.Google ScholarGoogle Scholar
  20. [20] Krizhevsky Alex and Geoffrey Hinton. 2009. Learning Multiple Layers of Features from Tiny Images. Technical Report. University of Toronto, Toronto, Ontario.Google ScholarGoogle Scholar
  21. [21] Krizhevsky Alex, Sutskever Ilya, and Hinton Geoffrey E.. 2012. ImageNet classification with deep convolutional neural networks. Advances in Neural Information Processing Systems 25 (2012), 10971105.Google ScholarGoogle ScholarDigital LibraryDigital Library
  22. [22] Leng Qingming, Ye Mang, and Tian Qi. 2019. A survey of open-world person re-identification. IEEE Transactions on Circuits and Systems for Video Technology 30, 4 (2019), 10921108.Google ScholarGoogle ScholarCross RefCross Ref
  23. [23] Li Da, Yang Yongxin, Song Yi-Zhe, and Hospedales Timothy M.. 2018. Learning to generalize: Meta-learning for domain generalization. In Proceedings of the 32nd AAAI Conference on Artificial Intelligence.Google ScholarGoogle ScholarCross RefCross Ref
  24. [24] Li Qinbin, He Bingsheng, and Song Dawn. 2021. Model-contrastive federated learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 1071310722.Google ScholarGoogle ScholarCross RefCross Ref
  25. [25] Li Tian, Sahu Anit Kumar, Talwalkar Ameet, and Smith Virginia. 2020. Federated learning: Challenges, methods, and future directions. IEEE Signal Processing Magazine 37 (2020), 5060.Google ScholarGoogle ScholarCross RefCross Ref
  26. [26] Li Tian, Sahu Anit Kumar, Zaheer Manzil, Sanjabi Maziar, Talwalkar Ameet, and Smith Virginia. 2020. Federated optimization in heterogeneous networks. In Proceedings of the 3rd Machine Learning and Systems Conference (MLSys’20). 429–450.Google ScholarGoogle Scholar
  27. [27] Li W. and Wang X.. 2013. Locally aligned feature transforms across views. In Proceedings of the 2013 IEEE Conference on Computer Vision and Pattern Recognition. 35943601.Google ScholarGoogle ScholarDigital LibraryDigital Library
  28. [28] Li Wei, Zhao Rui, and Wang Xiaogang. 2012. Human reidentification with transferred metric learning. In Computer Vision—ACCV 2012. Lecture Notes in Computer Science, Vol. 7724. Springer, 3144.Google ScholarGoogle Scholar
  29. [29] Li W., Zhao R., Xiao T., and Wang X.. 2014. DeepReID: Deep filter pairing neural network for person re-identification. In Proceedings of the 2014 IEEE Conference on Computer Vision and Pattern Recognition (CVPR’14). 152159.Google ScholarGoogle ScholarDigital LibraryDigital Library
  30. [30] Li Wei, Zhao Rui, Xiao Tong, and Wang Xiaogang. 2014. DeepReID: Deep filter pairing neural network for person re-identification. In Proceedings of the 2014 IEEE Conference on Computer Vision and Pattern Recognition (CVPR’14). 152–159.Google ScholarGoogle ScholarDigital LibraryDigital Library
  31. [31] Liu Jiawei, Zha Zheng-Jun, Tian Q. I., Liu Dong, Yao Ting, Ling Qiang, and Mei Tao. 2016. Multi-scale triplet CNN for person re-identification. In Proceedings of the 24th ACM International Conference on Multimedia (MM’16). ACM, New York, NY, 192196. Google ScholarGoogle ScholarDigital LibraryDigital Library
  32. [32] Liu Xiao, Song Mingli, Tao Dacheng, Zhou Xingchen, Chen Chun, and Bu Jiajun. 2014. Semi-supervised coupled dictionary learning for person re-identification. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 35503557.Google ScholarGoogle ScholarDigital LibraryDigital Library
  33. [33] Loy Chen Change, Liu Chunxiao, and Gong Shaogang. 2013. Person re-identification by manifold ranking. In Proceedings of the 2013 IEEE International Conference on Image Processing. IEEE, Los Alamitos, CA, 35673571.Google ScholarGoogle ScholarCross RefCross Ref
  34. [34] Luo Jiahuan, Wu Xueyang, Luo Yun, Huang Anbu, Huang Yunfeng, Liu Yang, and Yang Qiang. 2019. Real-world image datasets for federated learning. arxiv:1910.11089 [cs.CV] (2019).Google ScholarGoogle Scholar
  35. [35] McMahan Brendan, Moore Eider, Ramage Daniel, Hampson Seth, and Arcas Blaise Agüera y. 2017. Communication-efficient learning of deep networks from decentralized data. In Proceedings of the 20th International Conference on Artificial Intelligence and Statistics (AISTATS’17). 1273–1282. http://proceedings.mlr.press/v54/mcmahan17a.html.Google ScholarGoogle Scholar
  36. [36] Muhammad Khalil, Wang Qinqin, O’Reilly-Morgan Diarmuid, Tragos Elias, Smyth Barry, Hurley Neil, Geraci James, and Lawlor Aonghus. 2020. FedFast: Going beyond average for faster training of federated recommender systems. In Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. 12341242.Google ScholarGoogle ScholarDigital LibraryDigital Library
  37. [37] Niu Chaoyue, Wu Fan, Tang Shaojie, Hua Lifeng, Jia Rongfei, Lv Chengfei, Wu Zhihua, and Chen Guihai. 2020. Billion-scale federated learning on mobile clients: A submodel design with tunable privacy. In Proceedings of the 26th Annual International Conference on Mobile Computing and Networking. 114.Google ScholarGoogle ScholarDigital LibraryDigital Library
  38. [38] Peng Peixi, Xiang Tao, Wang Yaowei, Pontil Massimiliano, Gong Shaogang, Huang Tiejun, and Tian Yonghong. 2016. Unsupervised cross-dataset transfer learning for person re-identification. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 13061315.Google ScholarGoogle ScholarCross RefCross Ref
  39. [39] Sarfraz Saquib, Sharma Vivek, and Stiefelhagen Rainer. 2019. Efficient parameter-free clustering using first neighbor relations. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 89348943.Google ScholarGoogle ScholarCross RefCross Ref
  40. [40] Shankar Shiv, Piratla Vihari, Chakrabarti Soumen, Chaudhuri Siddhartha, Jyothi Preethi, and Sarawagi Sunita. 2018. Generalizing across domains via cross-gradient training. arXiv preprint arXiv:1804.10745 (2018).Google ScholarGoogle Scholar
  41. [41] Sheller Micah J., Reina G. Anthony, Edwards Brandon, Martin Jason, and Bakas Spyridon. 2018. Multi-institutional deep learning modeling without sharing patient data: A feasibility study on brain tumor segmentation. In Proceedings of the International MICCAI Brain Lesion Workshop (BrainLes’18). 92104.Google ScholarGoogle Scholar
  42. [42] Song Jifei, Yang Yongxin, Song Yi-Zhe, Xiang Tao, and Hospedales Timothy M.. 2019. Generalizable person re-identification by domain-invariant mapping network. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 719728.Google ScholarGoogle ScholarCross RefCross Ref
  43. [43] Su Chi, Zhang Shiliang, Xing Junliang, Gao Wen, and Tian Qi. 2016. Deep attributes driven multi-camera person re-identification. In Proceedings of the European Conference on Computer Vision. 475491.Google ScholarGoogle ScholarCross RefCross Ref
  44. [44] Sun P., Wen Y., Han R., Feng W., and Yan S.. 2022. GradientFlow: Optimizing network performance for large-scale distributed DNN training. IEEE Transactions on Big Data 8, 2 (2022), 495–507.Google ScholarGoogle Scholar
  45. [45] Sun Yifan, Zheng Liang, Yang Yi, Tian Qi, and Wang Shengjin. 2018. Beyond part models: Person retrieval with refined part pooling (and a strong convolutional baseline). In Proceedings of the European Conference on Computer Vision (ECCV’18). 480496.Google ScholarGoogle ScholarDigital LibraryDigital Library
  46. [46] Wang Guanshuo, Yuan Yufeng, Chen Xiong, Li Jiwei, and Zhou Xi. 2018. Learning discriminative features with multiple granularities for person re-identification. In Proceedings of the 26th ACM International Conference on Multimedia (MM’18). ACM, New York, NY, 274282. Google ScholarGoogle ScholarDigital LibraryDigital Library
  47. [47] Wang Hongyi, Yurochkin Mikhail, Sun Yuekai, Papailiopoulos Dimitris, and Khazaeni Yasaman. 2020. Federated learning with matched averaging. In Proceedings of the International Conference on Learning Representations. https://openreview.net/forum?id=BkluqlSFDS.Google ScholarGoogle Scholar
  48. [48] Wang Jianyu, Liu Qinghua, Liang Hao, Joshi Gauri, and Poor H. Vincent. 2020. Tackling the objective inconsistency problem in heterogeneous federated optimization. arXiv preprint arXiv:2007.07481 (2020).Google ScholarGoogle Scholar
  49. [49] Wang Taiqing, Gong Shaogang, Zhu Xiatian, and Wang Shengjin. 2014. Person re-identification by video ranking. In Computer Vision—ECCV 2014, Fleet David, Pajdla Tomas, Schiele Bernt, and Tuytelaars Tinne (Eds.). Springer International Publishing, Cham, Switzerland, 688703.Google ScholarGoogle Scholar
  50. [50] Wei Longhui, Zhang Shiliang, Gao Wen, and Tian Qi. 2018. Person transfer GAN to bridge domain gap for person re-identification. In Proceedings of the 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. 7988.Google ScholarGoogle ScholarCross RefCross Ref
  51. [51] Wu Guile and Gong Shaogang. 2021. Decentralised learning from independent multi-domain labels for person re-identification. Proceedings of the AAAI Conference on Artificial Intelligence 35, 4 (May2021), 28982906. https://ojs.aaai.org/index.php/AAAI/article/view/16396.Google ScholarGoogle ScholarCross RefCross Ref
  52. [52] Yang Jiwei, Shen Xu, Tian Xinmei, Li Houqiang, Huang Jianqiang, and Hua Xian-Sheng. 2018. Local convolutional neural networks for person re-identification. In Proceedings of the 26th ACM International Conference on Multimedia (MM’18). ACM, New York, NY, 10741082. Google ScholarGoogle ScholarDigital LibraryDigital Library
  53. [53] Yao Xin, Huang Tianchi, Zhang Rui-Xiao, Li Ruiyu, and Sun Lifeng. 2019. Federated learning with unbiased gradient aggregation and controllable meta updating. In Proceedings of the NIPS Federated Learning for Data Privacy and Confidentiality Workshop.Google ScholarGoogle Scholar
  54. [54] Ye Mang, Shen Jianbing, Lin Gaojie, Xiang Tao, Shao Ling, and Hoi Steven C. H.. 2021. Deep learning for person re-identification: A survey and outlook. arXiv:2001.04193 (2021).Google ScholarGoogle Scholar
  55. [55] Zhang Lin, Luo Yong, Bai Yan, Du Bo, and Duan Ling-Yu. 2021. Federated learning for non-IID data via unified feature learning and optimization objective alignment. In Proceedings of the IEEE/CVF International Conference on Computer Vision. 44204428.Google ScholarGoogle ScholarCross RefCross Ref
  56. [56] Zhao Yue, Li Meng, Lai Liangzhen, Suda Naveen, Civin Damon, and Chandra Vikas. 2018. Federated learning with non-IID data. CoRR abs/1806.00582 (2018). http://arxiv.org/abs/1806.00582.Google ScholarGoogle Scholar
  57. [57] Zheng Liang, Shen Liyue, Tian Lu, Wang Shengjin, Wang Jingdong, and Tian Qi. 2015. Scalable person re-identification: A benchmark. In Proceedings of the 2015 IEEE International Conference on Computer Vision (ICCV’15).11161124.Google ScholarGoogle ScholarCross RefCross Ref
  58. [58] Zheng Liang, Yang Yi, and Hauptmann Alexander G.. 2016. Person re-identification: Past, present and future. CoRR abs/1610.02984 (2016). http://arxiv.org/abs/1610.02984.Google ScholarGoogle Scholar
  59. [59] Zheng Zhedong, Zheng Liang, and Yang Yi. 2017. Unlabeled samples generated by GAN improve the person re-identification baseline in vitro. In Proceedings of the IEEE International Conference on Computer Vision.Google ScholarGoogle ScholarCross RefCross Ref
  60. [60] Zhuang Weiming, Gan Xin, Wen Yonggang, and Zhang Shuai. 2022. EasyFL: A low-code federated learning platform for dummies. IEEE Internet of Things Journal. Early access, January 20, 2022. Google ScholarGoogle ScholarCross RefCross Ref
  61. [61] Zhuang Weiming, Gan Xin, Wen Yonggang, Zhang Shuai, and Yi Shuai. 2021. Collaborative unsupervised visual representation learning from decentralized data. In Proceedings of the IEEE/CVF International Conference on Computer Vision. 49124921.Google ScholarGoogle ScholarCross RefCross Ref
  62. [62] Zhuang Weiming, Wen Yonggang, and Zhang Shuai. 2022. Divergence-aware federated self-supervised learning. In Proceedings of the International Conference on Learning Representations. https://openreview.net/forum?id=oVE1z8NlNe.Google ScholarGoogle Scholar
  63. [63] Zhuang Weiming, Wen Yonggang, Zhang Xuesen, Gan Xin, Yin Daiying, Zhou Dongzhan, Zhang Shuai, and Yi Shuai. 2020. Performance optimization of federated person re-identification via benchmark analysis. In Proceedings of the 28th ACM International Conference on Multimedia. 955963.Google ScholarGoogle ScholarDigital LibraryDigital Library

Index Terms

  1. Optimizing Performance of Federated Person Re-identification: Benchmarking and Analysis

          Recommendations

          Comments

          Login options

          Check if you have access through your login credentials or your institution to get full access on this article.

          Sign in

          Full Access

          • Published in

            cover image ACM Transactions on Multimedia Computing, Communications, and Applications
            ACM Transactions on Multimedia Computing, Communications, and Applications  Volume 19, Issue 1s
            February 2023
            504 pages
            ISSN:1551-6857
            EISSN:1551-6865
            DOI:10.1145/3572859
            • Editor:
            • Abdulmotaleb El Saddik
            Issue’s Table of Contents

            Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

            Publisher

            Association for Computing Machinery

            New York, NY, United States

            Publication History

            • Published: 23 January 2023
            • Online AM: 20 April 2022
            • Accepted: 10 April 2022
            • Revised: 11 March 2022
            • Received: 14 May 2021
            Published in tomm Volume 19, Issue 1s

            Permissions

            Request permissions about this article.

            Request Permissions

            Check for updates

            Qualifiers

            • research-article
            • Refereed
          • Article Metrics

            • Downloads (Last 12 months)277
            • Downloads (Last 6 weeks)23

            Other Metrics

          PDF Format

          View or Download as a PDF file.

          PDF

          eReader

          View online with eReader.

          eReader

          Full Text

          View this article in Full Text.

          View Full Text

          HTML Format

          View this article in HTML Format .

          View HTML Format
          About Cookies On This Site

          We use cookies to ensure that we give you the best experience on our website.

          Learn more

          Got it!