Abstract
Future pedestrian trajectory prediction offers great prospects for many practical applications such as unmanned vehicles, building evacuation design and robotic path planning. Most existing methods focus on social interaction among pedestrians but ignore the fact that heterogeneous traffic objects (cars, dogs, bicycles, motorcycles, etc.) have significant influence on the future trajectory of a subject pedestrian. Also, the walking direction intention of a pedestrian may be referred by his/her facial keypoints. Considering this, this work proposes to predict a pedestrian's future trajectory by jointly using neighboring heterogeneous traffic information and his/her facial keypoints. To fulfill this, an end-to-end facial keypoints-based convolutional encoder-decoder network (FK-CEN) is designed, in which the heterogeneous traffic and facial keypoints are input. After training, FK-CEN is evaluated on 5 crowded video sequences collected from the public datasets MOT-16 and MOT-17. Experimental results demonstrate that it outperforms state-of-the-art approaches, in terms of prediction errors.
- [1] 2016. Social : Human trajectory prediction in crowded spaces. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 961–971.Google Scholar
- [2] . 2018. Social : Socially acceptable trajectories with generative adversarial networks. In CVPR, 2018.1, 2, 3, 4, 5, 6, 8, 9, 12.Google Scholar
- [3] 2012. Activity forecasting. European Conference on Computer Vision. Springer, Berlin, 201–214.Google Scholar
Digital Library
- [4] 2019. Peeking into the future: Predicting future person activities and locations in videos. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 5725–5734.Google Scholar
- [5] 2009. You'll never walk alone: Modeling social behavior for multi-target tracking. 2009 IEEE 12th International Conference on Computer Vision. IEEE, 261–268.Google Scholar
Cross Ref
- [6] . 2017. Social attention: Modeling attention in human crowds. arXiv:1710.04689 [cs].Google Scholar
- [7] . 2000. Simulating dynamical features of escape panic. Nature 407, 9 (2000), 487–491.Google Scholar
Cross Ref
- [8] . 2011. Trajectory learning for activity understanding: Unsupervised, multilevel, and long-term adaptive approach. IEEE Transactions on Pattern Analysis and Machine Intelligence 33, 11 (2011), 2287–2301.Google Scholar
Digital Library
- [9] 2012. Activity forecasting. Proceedings of the 2012 European Conference on Computer Vision, LNCS 7575. Berlin: Springer, 201–214.Google Scholar
- [10] . 2018. Encoding crowd interaction with deep neural network for pedestrian trajectory prediction. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 5275–5284.Google Scholar
Cross Ref
- [11] . 2016. Pedestrian behavior understanding and prediction with deep neural networks. European Conference on Computer Vision. Springer, Cham, 263–279.Google Scholar
- [12] 2018. Understanding human behaviors in crowds by imitating the decision-making process. Thirty-Second AAAI Conference on Artificial Intelligence.Google Scholar
Cross Ref
- [13] 2012. Activity forecasting. European Conference on Computer Vision. Springer, Berlin, 201–214.Google Scholar
- [14] 2017. Learning and inferring “dark matter” and predicting human intents and trajectories in videos. IEEE Transactions on Pattern Analysis and Machine Intelligence 40, 7 (2017), 1639–1652.Google Scholar
Cross Ref
- [15] . 2018. Scene-: A model for human trajectory prediction[J]. arXiv preprint arXiv:1808.04018.Google Scholar
- [16] . 2018. A transferable pedestrian motion prediction model for intersections with different geometries. arXiv preprint arXiv:1806.09444.Google Scholar
- [17] 2019. Sohie: An attentive for predicting paths compliant to social and physical constraints. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 1349–1358.Google Scholar
Cross Ref
- [18] . 2018. Convolutional neural network for trajectory prediction. Proceedings of the European Conference on Computer Vision (ECCV). 0-0.Google Scholar
- [19] 2018. Future person localization in first-person videos. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 7593–7602.Google Scholar
Cross Ref
- [20] 2016. Wider face: A face detection benchmark. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 5525–5533.Google Scholar
Cross Ref
- [21] 2014. Facial landmark detection by deep multi-task learning. European Conference on Computer Vision. Springer, Cham, 94–108.Google Scholar
- [22] 2015. Show and tell: A neural image caption generator. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 3156–3164.Google Scholar
Cross Ref
- [23] 2016. MOT16: A benchmark for multi-object tracking[J]. arXiv preprint arXiv:1603.00831.Google Scholar
- [24] . 2007. Crowds by example. Computer Graphics Forum 655–664.Google Scholar
Cross Ref
- [25] . 2009. You'll never walk alone: Modeling social behavior for multi-target tracking. In IEEE 12th International Conference on Computer Vision (ICCV). 261–268.Google Scholar
- [26] “PyTorch.” [Online]. Available: https://pytorch.org/. [Accessed: 23-Jun-2018].Google Scholar
- [27] . 2014. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980.Google Scholar
- [28] . 2018. A data-driven neural network approach to simulate pedestrian movement. Physica A-statistical Mechanics and Its Applications 509, 11 (2018), 827–844.Google Scholar
Cross Ref
- [29] . 2016. Selfishness- and selflessness-based models of pedestrian room evacuation. Physica A-statistical Mechanics and Its Applications 447, 4 (2016), 455–466.Google Scholar
Cross Ref
- [30] . 2019. Unpredictability in pedestrian flow: The impact of stochasticity and anxiety in the event of an emergency. Physica A: Statistical Mechanics and its Applications 531, 1 (2019), 121742.Google Scholar
Cross Ref
- [31] . 2018. Modeling handicapped pedestrians considering physical characteristics using cellular automaton. Physica A: Statistical Mechanics and its Applications 510, 15 (2018), 507–517.Google Scholar
Cross Ref
- [32] . 2014. Context-based pedestrian path prediction. In European Conference on Computer Vision. Springer, Cham, 618–633.Google Scholar
Cross Ref
- [33] 2020. Beyond artificial reality: Finding and monitoring live events from social sensors. ACM Transactions on Internet Technology (TOIT) 20, 1 (2020), 1–21.Google Scholar
Digital Library
- [34] . 2019. Incentive-based crowdsourcing of hotspot services. ACM Transactions on Internet Technology (TOIT) 19, 1 (2019), 1–24.Google Scholar
Digital Library
- [35] 2020. Pedestrian trajectory prediction based on deep convolutional LSTM network. IEEE Transactions on Intelligent Transportation Systems 3, (2020). .Google Scholar
Cross Ref
- [36] . 2019. Simulation of pedestrian rotation dynamics near crowded exits. IEEE Transactions on Intelligent Transportation Systems 20, 8 (2019), 3142–3155.Google Scholar
Cross Ref
- [37] . 2018. SS-LSTM: A hierarchical LSTM model for pedestrian trajectory prediction. In 2018 IEEE Winter Conference on Applications of Computer Vision (WACV). IEEE, 1186–1194.Google Scholar
- [38] . 2019. -Sr-lstm: State refinement for towards pedestrian trajectory prediction. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 12085–12094.Google Scholar
Cross Ref
Index Terms
Pedestrian Trajectory Prediction in Heterogeneous Traffic using Facial Keypoints-based Convolutional Encoder-decoder Network
Recommendations
Pedestrian Trajectory Prediction in Heterogeneous Traffic Using Pose Keypoints-Based Convolutional Encoder-Decoder Network
Future pedestrian trajectory prediction offers great prospects for many practical applications. Most existing methods focus on social interaction among pedestrians but ignore the factors that heterogeneous traffic objects (cars, dogs, bicycles, ...
PTP-STGCN: Pedestrian Trajectory Prediction Based on a Spatio-temporal Graph Convolutional Neural Network
AbstractIt is the prerequisite to ensure the safety of road users in traffic scenes for the application of autonomous vehicles. Pedestrians are the main participants in traffic scenes, and reasonable inference and prediction of their future trajectories ...
3D facial expression recognition using SIFT descriptors of automatically detected keypoints
Special Issue on 3DOR 2010Methods to recognize humans’ facial expressions have been proposed mainly focusing on 2D still images and videos. In this paper, the problem of person-independent facial expression recognition is addressed using the 3D geometry information extracted ...






Comments