Abstract
Gait recognition has a rapid development in recent years. However, current gait recognition focuses primarily on ideal laboratory scenes, leaving the gait in the wild unexplored. One of the main reasons is the difficulty of collecting in-the-wild gait datasets, which must ensure diversity of both intrinsic and extrinsic human gait factors. To remedy this problem, we propose to construct a large-scale gait dataset with the help of controllable computer simulation. In detail, to diversify the intrinsic factors of gait, we generate numerous characters with diverse attributes and associate them with various types of walking styles. To diversify the extrinsic factors of gait, we build a complicated scene with a dense camera layout. Then we design an automatic generation toolkit under Unity3D for simulating the walking scenarios and capturing the gait data. As a result, we obtain a dataset simulating towards the in-the-wild scenario, called VersatileGait, which has more than one million silhouette sequences of 10,000 subjects with diverse scenarios. VersatileGait possesses several nice properties, including huge dataset size, diverse pedestrian attributes, complicated camera layout, high-quality annotations, small domain gap with the real one, good scalability for new demands, and no privacy issues. By conducting a series of experiments, we first explore the effects of different factors on gait recognition. We further illustrate the effectiveness of using our dataset to pre-train models, which obtain considerable performance gain on CASIA-B, OU-MVLP, and CASIA-E. Besides, we show the great potential of the fine-grained labels other than the ID label in improving the efficiency and effectiveness of models. Our dataset and its corresponding generation toolkit are available at https://github.com/peterzpy/VersatileGait.
- [1] 2021. CASIA-E. Retrieved May 29, 2021 from http://hid2020.iapr-tc4.org.
Google Scholar - [2] 2021. Mixamo. Retrieved May 29, 2021 from https://www.mixamo.com.Google Scholar
- [3] 2021. Unity3D. Retrieved May 29, 2021 from https://unity.com.
Google Scholar - [4] . 2011. Model-based 3D gait biometrics. In Proceedings of the IJCB. 1–7.Google Scholar
Digital Library
- [5] . 2018. Combining facial parts for learning gender, ethnicity, and emotional state based on RGB-D information. ACM Transactions on Multimedia Computing, Communications, and Applications 14, 1s (2018), 19:1–19:14.Google Scholar
Digital Library
- [6] . 2018. Domain adaptation through synthesis for unsupervised person re-identification. In Proceedings of the European Conference on Computer Vision.189–205.Google Scholar
- [7] . 2018. Gait recognition from motion capture data. ACM Transactions on Multimedia Computing, Communications, and Applications 14, 1s (2018), 22:1–22:18.Google Scholar
Digital Library
- [8] . 2009. Gait recognition using gait entropy image. In Proceedings of the ICDP. 1–6.Google Scholar
Cross Ref
- [9] . 2008. Ideas and methods for modeling 3D human figures: The principal algorithms used by MakeHuman and their implementation in a new approach to parametric modeling. In Proceedings of the Bangalore Compute Conference.1–6.Google Scholar
Digital Library
- [10] . 2009. View-independent human motion classification using image-based reconstruction. Image and Vision Computing 27, 8 (2009), 1194–1206.Google Scholar
Digital Library
- [11] . 2018. A survey of using biometrics for smart visual surveillance: Gait recognition. In Proceedings of the Surveillance in Action. 3–23.Google Scholar
Cross Ref
- [12] . 2007. Gait recognition based on human body components. In Proceedings of the IEEE International Conference on Image Processing.I–353.Google Scholar
Cross Ref
- [13] . 2019. Gaitset: Regarding gait as a set for cross-view gait recognition. In Proceedings of the AAAI. 8126–8133.Google Scholar
Digital Library
- [14] . 2014. Frontal gait recognition from incomplete sequences using RGB-D camera. IEEE Transactions on Information Forensics and Security 9, 11 (2014), 1843–1856.Google Scholar
Digital Library
- [15] Patrick Connor and Arun Ross. 2018. Biometric recognition by gait: A survey of modalities and features. Computer Vision and Image Understanding 167 (2018), 1–27.
DOI: Google ScholarDigital Library
- [16] . 2013. Investigating gait recognition in the short-wave infrared (SWIR) spectrum: Dataset and challenges. In Proceedings of the SPIE Biometric Surveillance Technology for Human and Activity Identification. 101–116.Google Scholar
Cross Ref
- [17] . 2018. Learning to detect and track visible and occluded body joints in a virtual world. In Proceedings of the European Conference on Computer Vision.430–446.Google Scholar
Digital Library
- [18] . 2020. Gaitpart: Temporal part-based model for gait recognition. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.14213–14221.Google Scholar
Cross Ref
- [19] . 2007. A survey of biometric gait recognition: Approaches, security and challenges. In Proceedings of the Annual Norwegian Computer Science Conference.19–21.Google Scholar
- [20] . 2009. Self-calibrating view-invariant gait biometrics. IEEE Transactions on Systems, Man, and Cybernetics 40, 4 (2009), 997–1008.Google Scholar
Digital Library
- [21] R. Gross and J. Shi. 2001. The CMU motion of body (MoBo) database. Technical Report CMU-RI-TR-01-18, Robotics Institute, Carnegie Mellon University, Pittsburgh, PA, June 2001.Google Scholar
- [22] . 2006. Individual recognition using gait energy image. IEEE Transactions on Pattern Analysis and Machine Intelligence 28, 2 (2006), 316–322.Google Scholar
Digital Library
- [23] . 2014. The TUM gait from audio, image and depth (GAID) database: Multimodal recognition of subjects and traits. Journal of Visual Communication and Image Representation 25, 1 (2014), 195–206.Google Scholar
Digital Library
- [24] . 2011. Gait recognition in the presence of occlusion: A new dataset and baseline algorithms. In Proceedings of the 19th International Conference in Central Europe on Computer Graphics, Visualization and Computer Vision.Google Scholar
- [25] . 2020. Gait lateral network: Learning discriminative and compact representations for gait recognition. In Proceedings of the European Conference on Computer Vision.382–398.Google Scholar
Digital Library
- [26] . 2021. Set residual network for silhouette-based gait recognition. IEEE Transactions on Biometrics, Behavior, and Identity Science 3, 3 (2021), 384–393.Google Scholar
Cross Ref
- [27] . 2019. Sail-vos: Semantic amodal instance level video object segmentation-a synthetic dataset and baselines. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.3105–3115.Google Scholar
Cross Ref
- [28] . 2021. Context-sensitive temporal feature learning for gait recognition. In Proceedings of the Int. Conf. Comput. Vis.12909–12918.Google Scholar
Cross Ref
- [29] . 2021. 3D Local convolutional neural networks for gait recognition. In Proceedings of the IEEE/CVF International Conference on Computer Vision.14920–14929.Google Scholar
Cross Ref
- [30] . 2012. The ou-isir gait database comprising the large population dataset and performance evaluation of gait recognition. IEEE Transactions on Information Forensics and Security 7, 5 (2012), 1511–1521.Google Scholar
Digital Library
- [31] . 2020. Feature extraction using an RNN autoencoder for skeleton-based abnormal gait recognition. IEEE Access 8 (2020), 19196–19207.
DOI: Google ScholarCross Ref
- [32] . 2013. Recognizing gaits across views through correlated motion co-clustering. IEEE Transactions on Image Processing 23, 2 (2013), 696–709.Google Scholar
Digital Library
- [33] . 2013. A new view-invariant feature for cross-view gait recognition. IEEE Transactions on Information Forensics and Security 8, 10 (2013), 1642–1653.Google Scholar
Digital Library
- [34] . 2020. Gait recognition via semi-supervised disentangled representation learning to identity and covariate features. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.13309–13319.Google Scholar
Cross Ref
- [35] . 2020. End-to-end model-based gait recognition. In Proceedings of the ACCV. 3–20.Google Scholar
- [36] . 2020. A model-based gait recognition method with body pose and human prior knowledge. Pattern Recognition 98 (2020), 107069.
DOI: Google ScholarDigital Library
- [37] . 2020. Gait recognition with multiple-temporal-scale 3D convolutional neural network. In Proceedings of the ACM International Conference on Multimedia. 3054–3062.Google Scholar
Digital Library
- [38] . 2021. Gait recognition via effective global-local feature representation and local temporal aggregation. In Proceedings of the International Conference on Computer Vision.14648–14656.Google Scholar
Cross Ref
- [39] Yu-Sheng Lin, Zhe-Yu Liu, Yu-An Chen, Yu-Siang Wang, Ya-Liang Chang, and Winston H. Hsu. 2021. XCos: An explainable cosine metric for face verification task. ACM Transactions on Multimedia Computing, Communications, and Applications 17, 3s (2021), 1–16.Google Scholar
- [40] . 2019. Taking a closer look at domain shift: Category-level adversaries for semantics consistent domain adaptation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.2507–2516.Google Scholar
Cross Ref
- [41] . 2006. Gait recognition using a view transformation model in the frequency domain. In Proceedings of the European Conference on Computer Vision.151–163.Google Scholar
Digital Library
- [42] . 2017. Joint intensity and spatial metric learning for robust gait recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.5705–5715.Google Scholar
Cross Ref
- [43] . 2021. Self-supervised real-to-sim scene generation. In Proceedings of the IEEE/CVF International Conference on Computer Vision. 16044–16054.Google Scholar
- [44] . 2012. A survey of gait recognition approaches using PCA and ICA. Glob. J. Comput. Sci. Tech. 12, 10 (2012), 975–980.Google Scholar
- [45] . 2021. GreyReID: A Novel two-stream deep framework with RGB-Grey information for person re-Identification. ACM Transactions on Multimedia Computing, Communications, and Applications 17, 1 (2021), 1–22.Google Scholar
Digital Library
- [46] . 2019. Robust gait recognition: A comprehensive survey. IET Biometrics 8, 1 (2019), 14–28.Google Scholar
Cross Ref
- [47] . 2017. An overview of multi-task learning in deep neural networks. arXiv:1706.05098. Retrieved from https://arxiv.org/abs/1706.05098.Google Scholar
- [48] . 2010. Performing content-based retrieval of humans using gait biometrics. Multimedia Tools and Applications 49, 1 (2010), 195–212.Google Scholar
Digital Library
- [49] . 2018. Learning from synthetic data: Addressing domain shift for semantic segmentation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.3752–3761.Google Scholar
Cross Ref
- [50] . 2005. The humanid gait challenge problem: Data sets, performance, and analysis. IEEE Transactions on Pattern Analysis and Machine Intelligence 27, 2 (2005), 162–177.Google Scholar
Digital Library
- [51] . 2022. Deep gait recognition: A survey. IEEE Transactions on Pattern Analysis and Machine Intelligence.
DOI: Google ScholarCross Ref
- [52] . 2019. Video retrieval with similarity-preserving deep temporal hashing. ACM Transactions on Multimedia Computing, Communications, and Applications 15, 4 (2019), 1–16.Google Scholar
Digital Library
- [53] . 2019. A Multi-Gait dataset for human recognition under occlusion scenario. In Proceedings of the International Conference on Issues and Challenges in Intelligent Computing Techniques. 1–6.Google Scholar
- [54] . 2021. Hypomimia recognition in parkinson’s disease with semantic features. ACM Transactions on Multimedia Computing, Communications, and Applications 17, 3s (2021), 1–20.Google Scholar
Digital Library
- [55] . 2017. Deep learning logo detection with data expansion by synthesising context. IEEE Winter Conference on Applications of Computer Vision (2017), 530–539.Google Scholar
- [56] . 2016. Feature extraction methods for human gait recognition–a survey. In Proceedings of the ICACDS. 377–385.Google Scholar
- [57] . 2018. Multi-view large population gait dataset and its performance evaluation for cross-view gait recognition. IPSJ Transactions on Computer Vision and Applications 10, 1 (2018), 1–14.Google Scholar
Cross Ref
- [58] . 2006. Efficient night gait recognition based on template matching. In Proceedings of the International Conference on Pattern Recognition. 1000–1003.Google Scholar
- [59] . 2014. Modern modelling techniques are data hungry: A simulation study for predicting dichotomous endpoints. BMC Medical Research Methodology 14, 1 (2014), 1–13.Google Scholar
- [60] . 2018. Training deep networks with synthetic data: Bridging the reality gap by domain randomization. In Proceedings of the IEEE Conference Comput. Vis. Pattern Recog. Worksh.969–977.Google Scholar
Cross Ref
- [61] . 2017. Learning from synthetic humans. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops.109–117.Google Scholar
Cross Ref
- [62] . 2018. A survey on gait recognition. ACM Computing Surveys 51, 5 (2018), 1–35.Google Scholar
Digital Library
- [63] . 2020. Faster person re-identification. In Proceedings of the European Conference on Computer Vision.275–292.Google Scholar
Digital Library
- [64] . 2002. Gait recognition based on procrustes shape analysis. In Proceedings of the IEEE International Conference on Image Processing.433–436.Google Scholar
Cross Ref
- [65] . 2019. Learning from synthetic data for crowd counting in the wild. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.8198–8207.Google Scholar
Cross Ref
- [66] . 2019. U-Net conditional GANs for photo-realistic and identity-preserving facial expression synthesis. ACM Transactions on Multimedia Computing, Communications, and Applications 15, 3s (2019), 1–23.Google Scholar
Digital Library
- [67] . 2020. Surpassing real-world source training data: Random 3d characters for generalizable person re-identification. In Proceedings of the ACM International Conference on Multimedia. 3422–3430.Google Scholar
Digital Library
- [68] . 2020. Unsupervised domain adaptation through synthesis for person re-identification. In Proceedings of the International Conference on Multimedia and Expo.1–6.Google Scholar
Cross Ref
- [69] . 2021. Cross-view gait recognition using pairwise spatial transformer networks. IEEE Transactions on Circuits and Systems for Video Technology 31, 1 (2021), 260–274.Google Scholar
Digital Library
- [70] . 2017. The ou-isir gait database comprising the large population dataset with age and performance evaluation of age estimation. IPSJ Transactions on Computer Vision and Applications 9, 1 (2017), 1–14.Google Scholar
Cross Ref
- [71] . 2006. A framework for evaluating the effect of view angle, clothing and carrying condition on gait recognition. In Proceedings of the 18th International Conference on Pattern Recognition.441–444.Google Scholar
- [72] . 2018. SKEPRID: Pose and illumination change-resistant skeleton-based person re-Identification. ACM Transactions on Multimedia Computing, Communications, and Applications 14, 4 (2018), 24.Google Scholar
Digital Library
- [73] . 2017. Curriculum domain adaptation for semantic segmentation of urban scenes. In Proceedings of the IEEE International Conference on Computer Vision.2020–2030.Google Scholar
Cross Ref
- [74] . 2019. Gait recognition via disentangled representation learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.4710–4719.Google Scholar
Cross Ref
- [75] Aite Zhao, Jianbo Li, Junyu Dong, Lin Qi, Qianni Zhang, Ning Li, Xin Wang, and Huiyu Zhou. 2021. IEEE Transactions on Cybernetics.
DOI: Google ScholarCross Ref
- [76] . 2006. 3D gait recognition using multiple cameras. In Proceedings of the FGR. 529–534.Google Scholar
- [77] . 2019. Multi-source domain adaptation for semantic segmentation. In Proceedings of the Advances in Neural Information Processing Systems.7285–7298.Google Scholar
- [78] . 2019. Structured3d: A large photo-realistic dataset for structured 3d modeling. In Proceedings of the European Conference on Computer Vision.519–535.Google Scholar
Index Terms
A Large-Scale Synthetic Gait Dataset Towards in-the-Wild Simulation and Comparison Study
Recommendations
A multi-modal dataset for gait recognition under occlusion
AbstractGait recognition aims to identify people by the way they walk. Currently available gait recognition datasets mainly contain single-person gait data in relatively simple walking conditions, which limits research of robust gait recognition methods. ...
Speed-Invariant Gait Recognition Using Single-Support Gait Energy Image
AbstractGait is one of the most popular behavioral biometrics because it can be authenticated at a distance from a camera without subject cooperation. Speed differences between matching pairs, however, cause significant performance drops in gait ...
Gait Recognition Using Acceleration Signal
ICEICE '12: Proceedings of the 2012 Second International Conference on Electric Information and Control Engineering - Volume 01Presents an approach on recognizing individuals based on 3D acceleration signal from walking, which are collected using the triaxial accelerometer MMA7260Q. Our approach is based on walking acceleration in three directions: backward-forward, right-left ...






Comments