Abstract
As one of the public transportation systems, metro is certainly an indispensable part in urban areas of a metropolis today. Several successful results have shown that deep learning technologies might provide an effective way to predict the number of passengers at a metro station. However, most information systems based on deep learning technologies are usually designed and tuned manually by using domain knowledge and trial-and-error; thus, how to find out a set of suitable hyperparameters for a deep neural network (DNN) has become a critical research issue. To deal with the problem of hyperparameter setting for a DNN in solving the prediction of passengers at a metro station, a novel metaheuristic algorithm called search economics for hyperparameter optimization is presented to improve the accuracy rate of such a prediction system. The basic idea of the proposed algorithm is to divide the solution space into a set of subspaces first and then assign a different number of search agents to each subspace based on the “potential of each subspace.” The potential is estimated based on the objective values of the searched solutions, the objective values of the probe solutions, and the computation time invested in each subspace. The proposed method is compared with Bayesian, random forest, support vector regression, DNN, and DNN with different hyperparameter search algorithms, namely, grid search, simulated annealing, and particle swarm optimization. The simulation results using the data provided by the government of Taipei city, Taiwan, indicate that the proposed method outperforms all the other forecasting methods compared in this article in terms of the mean absolute percentage error.
- James Bergstra and Yoshua Bengio. 2012. Random search for hyper-parameter optimization. J. Mach. Learn. Res. 13 (2012), 281–305. Google Scholar
Digital Library
- James S. Bergstra, Rémi Bardenet, Yoshua Bengio, and Balázs Kégl. 2011. Algorithms for hyper-parameter optimization. In Proceedings of the International Conference on Neural Information Processing. 2546–2554. Google Scholar
Digital Library
- Christian Blum and Andrea Roli. 2003. Metaheuristics in combinatorial optimization: Overview and conceptual comparison. ACM Comput. Surveys 35, 3 (2003), 268–308. Google Scholar
Digital Library
- Andrés Camero, Jamal Toutouh, Daniel H. Stolfi, and Enrique Alba. 2019. Evolutionary deep learning for car park occupancy prediction in smart cities. In Proceedings of the International Conference on Learning and Intelligent Optimization. 386–401.Google Scholar
Cross Ref
- Irina Ceapa, Chris Smith, and Licia Capra. 2012. Avoiding the crowds: Understanding tube station congestion patterns from trip data. In Proceedings of the ACM SIGKDD International Workshop on Urban Computing. 134–141. Google Scholar
Digital Library
- Taiwan Central Weather Bureau. 2020. CWB Observation Data Inquire System. Retrieved from https://e-service.cwb.gov.tw/HistoryDataQuery/index.jsp.Google Scholar
- Ronan Collobert and Jason Weston. 2008. A unified architecture for natural language processing: Deep neural networks with multitask learning. In Proceedings of the International Conference on Machine Learning. 160–167. Google Scholar
Digital Library
- Taipei Rapid Transit Corporation. 2020. The Passenger Flow Data of Taipei Mass Rapid Transit. Retrieved from http://163.29.157.32:8080/fi/dataset/98d67c29-464a-4003-9f78-b1cbb89bff59.Google Scholar
- Giovanni Lucca França da Silva, Thales Levi Azevedo Valente, Aristófanes Corrêa Silva, Anselmo Cardoso de Paiva, and Marcelo Gattass. 2018. Convolutional neural network-based PSO for lung nodule false positive reduction on CT images. Comput. Methods Programs Biomed. 162 (2018), 109–118.Google Scholar
Cross Ref
- Thomas Elsken, Jan Hendrik Metzen, and Frank Hutter. 2019. Neural architecture search: A survey. J. Mach. Learn. Res. 20 (2019), 1–21.Google Scholar
- Matthias Feurer and Frank Hutter. 2019. Hyperparameter Optimization. Springer, Cham, 3–33.Google Scholar
- Matthias Feurer, Jost Tobias Springenberg, and Frank Hutter. 2015. Initializing Bayesian hyperparameter optimization via meta-learning. In Proceedings of the AAAI Conference on Artificial Intelligence. 1128–1135. Google Scholar
Digital Library
- Simon Fong, Suash Deb, and Xin-she Yang. 2018. How meta-heuristic algorithms contribute to deep learning in the hype of big data analytics. In Proceedings of the Progress in Intelligent Computing Techniques: Theory, Practice, and Applications, Vol. 518. 3–25.Google Scholar
Cross Ref
- Fred Glover and Gary A. Kochenberger (Eds.). 1993. Handbook of Metaheuristics. Kluwer Academic Publishers, Boston, MA.Google Scholar
- Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2016. Deep residual learning for image recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 770–778.Google Scholar
Digital Library
- Xin He, Kaiyong Zhao, and Xiaowen Chu. 2019. AutoML: A survey of the state-of-the-art. Retrieved from https://arxiv:1908.00709.Google Scholar
- Erik Jenelius. 2019. Data-driven metro train crowding prediction based on real-time load data. IEEE Trans. Intell. Transport. Syst. (2019), 1–12, In Press.Google Scholar
Cross Ref
- Jaeyoung Jung and Keemin Sohn. 2017. Deep-learning architecture to forecast destinations of bus passengers from entry-only smart-card data. IET Intell. Transport. Syst. 11, 6 (2017), 334–339.Google Scholar
Cross Ref
- Tae-Young Kim and Sung-Bae Cho. 2019. Particle swarm optimization-based CNN-LSTM networks for forecasting energy consumption. In Proceedings of the IEEE Congress on Evolutionary Computation. 1510–1516.Google Scholar
Cross Ref
- Woo-Young Lee, Seung-Min Park, and Kwee-Bo Sim. 2018. Optimal hyperparameter tuning of convolutional neural networks based on the parameter-setting-free harmony search algorithm. Optik 172 (2018), 359–367.Google Scholar
Cross Ref
- Lisha Li, Kevin Jamieson, Giulia DeSalvo, Afshin Rostamizadeh, and Ameet Talwalkar. 2018. Hyperband: A novel bandit-based approach to hyperparameter optimization. J. Mach. Learn. Res. 18 (2018), 1–52. Google Scholar
Digital Library
- Victor C. Liang, Richard T. B. Ma, Wee Siong Ng, Li Wang, Marianne Winslett, Huayu Wu, Shanshan Ying, and Zhenjie Zhang. 2016. Mercury: Metro density prediction with recurrent neural network on streaming CDR data. In Proceedings of the IEEE International Conference on Data Engineering. 1374–1377.Google Scholar
Cross Ref
- Xiaoyuan Liang, Xunsheng Du, Guiling Wang, and Zhu Han. 2019. A deep reinforcement learning network for traffic light cycle control. IEEE Trans. Vehic. Technol. 68, 2 (2019), 1243–1253.Google Scholar
Cross Ref
- Pablo Ribalta Lorenzo, Jakub Nalepa, Michal Kawulok, Luciano Sanchez Ramos, and José Ranilla Pastor. 2017. Particle swarm optimization for hyper-parameter selection in deep neural networks. In Proceedings of the Genetic and Evolutionary Computation Conference. 481–488. Google Scholar
Digital Library
- Ilya Loshchilov and Frank Hutter. 2016. CMA-ES for hyperparameter optimization of deep neural networks. Retrieved from https://arxiv:1604.07269.Google Scholar
- Yisheng Lv, Yanjie Duann, Wenwen Kang, Zhengxi Li, and Fei-Yue Wang. 2015. Traffic flow prediction with big data: A deep learning approach. IEEE Trans. Intell. Transport. Syst. 16, 2 (2015), 865–873.Google Scholar
- Noboru Nakafuri and Yoshio Miki. 2017. Prediction model of number of bus users based on influence of sensible temperature and discomfort index. In Proceedings of the IEEE International Conference on Intelligent Transportation Engineering. 155–158.Google Scholar
Cross Ref
- A. J. Nebro, J. J. Durillo, J. García-Nieto, C. A. Coello Coello, F. Luna, and E. Alba. 2009. SMPSO: A new PSO-based metaheuristic for multi-objective optimization. In Proceedings of the IEEE Symposium on Computational Intelligence in Multi-Criteria Decision-Making. 66–73.Google Scholar
- Ming Ni, Qing He, and Jing Gao. 2017. Forecasting the subway passenger flow under event occurrences with social media. IEEE Trans. Intell. Transport. Syst. 18, 6 (2017), 1623–1632.Google Scholar
- Fernando Nogueira. 2014. Bayesian Optimization: Open source constrained global optimization tool for Python. Retrieved from https://github.com/fmfn/BayesianOptimization.Google Scholar
- Yoshihiko Ozaki, Masaki Yano, and Masaki Onishi. 2017. Effective hyperparameter optimization using Nelder-Mead method in deep learning. IPSJ Trans. Comput. Vision Appl. 9, 20 (2017), 1–12.Google Scholar
- Basheer Qolomany, Majdi Maabreh, Ala Al-Fuqaha, Ajay Gupta, and Driss Benhaddou. 2017. Parameters optimization of deep learning models using particle swarm optimization. In Proceedings of the International Wireless Communications and Mobile Computing Conference. 1285–1290.Google Scholar
Cross Ref
- Heath Rushing, Jim Wisnowski, and Andrew Karl. 2013. Design and Analysis of Experiments by Douglas Montgomery: A Supplement for Using JMP. SAS Institute, Cary, NC. Google Scholar
Digital Library
- scikit-learn 0.23.1. 2020. Retrieved from https://scikit-learn.org/stable/.Google Scholar
- Tatsuki Serizawa and Hamido Fujita. 2020. Optimization of convolutional neural network using the linearly decreasing weight particle swarm optimization. Retrieved from https://arxiv:2001.05670.Google Scholar
- Dhruv Srivastava, Yasha Singh, and Anita Sahoo. 2019. Auto tuning of RNN hyper-parameters using cuckoo search algorithm. In Proceedings of the International Conference on Contemporary Computing. 1–5.Google Scholar
Cross Ref
- Tetsuya Tamaki, Hiroki Nakamura, Hidemichi Fujii, and Shunsuke Managi. 2019. Efficiency and emissions from urban transport: Application to world city-level public transportation. Econ. Anal. Policy 61 (2019), 55–63.Google Scholar
Cross Ref
- Liyang Tang, Yang Zhao, Javier Cabrera, Jian Ma, and Kwok Leung Tsui. 2019. Forecasting short-term passenger flow: An empirical study on Shenzhen metro. IEEE Trans. Intell. Transport. Syst. 20, 10 (2019), 3613–3622.Google Scholar
Cross Ref
- Chris Thornton, Frank Hutter, Holger H. Hoos, and Kevin Leyton-Brown. 2013. Auto-WEKA: Combined selection and hyperparameter optimization of classification algorithms. In Proceedings of the ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. 847–855. Google Scholar
Digital Library
- Wichai Treethidtaphat, Wasan Pattara-Atikom, and Sippakorn Khaimook. 2017. Bus arrival time prediction at any distance of bus route using deep neural network model. In Proceedings of the IEEE International Conference on Intelligent Transportation Systems. 988–992.Google Scholar
Cross Ref
- Chun-Wei Tsai. 2016. An effective WSN deployment algorithm via search economics. Comput. Netw. 101 (2016), 178–191. Google Scholar
Digital Library
- Chun-Wei Tsai, Chien-Hui Hsia, Shuang-Jie Yang, Shih-Jui Liu, and Zhi-Yan Fang. 2020. Optimizing hyperparameters of deep learning in predicting bus passengers based on simulated annealing. Appl. Soft Comput. 88 (2020), 1–9.Google Scholar
Digital Library
- Chun-Wei Tsai and Shi-Jui Liu. 2018. An effective IoT service-to-interface assignment algorithm via search economics. IEEE Internet Things J. 5, 3 (2018), 1708–1718.Google Scholar
Cross Ref
- Chun-Wei Tsai and J. J. P. C. Rodrigues. 2014. Metaheuristic scheduling for cloud: A survey. IEEE Systems Journal 8, 1 (Mar. 2014), 279–291.Google Scholar
Cross Ref
- John Voelcker. 2014. 1.2 Billion Vehicles on World’s Roads Now, 2 Billion By 2035: Report. Technical Report. Green Car Report. Retrieved from https://www.greencarreports.com/news/1093560_1-2-billion-vehicles-on-worlds-roads-now-2-billion-by-2035-report.Google Scholar
- Jiazhuo Wang, Jason Xu, and Xuejun Wang. 2018. Combination of hyperband and Bayesian optimization for hyperparameter optimization in deep learning. Retrieved from https://arxiv:1801.01596.Google Scholar
- Quanming Yao, Mengshuo Wang, Hugo Jair Escalante, Isabelle Guyon, Yi-Qi Hu, Yu-Feng Li, Wei-Wei Tu, Qiang Yang, and Yang Yu. 2018. Taking human out of learning applications: A survey on automated machine learning. Retrieved from https://arxiv:1810.13306.Google Scholar
- Fei Ye. 2017. Particle swarm optimization-based automatic parameter selection for deep neural networks and its applications in large-scale and high-dimensional data. PLoS ONE 12, 12 (12 2017), 1–36.Google Scholar
- Steven R. Young, Derek C. Rose, Thomas P. Karnowski, Seung-Hwan Lim, and Robert M. Patton. 2015. Optimizing deep learning hyper-parameters through an evolutionary algorithm. In Proceedings of the Workshop on Machine Learning in High-Performance Computing Environments. 1–5. Google Scholar
Digital Library
Index Terms
An Effective Hyperparameter Optimization Algorithm for DNN to Predict Passengers at a Metro Station
Recommendations
An Effective Multi-Swarm Algorithm for Optimizing Hyperparameters of DNN
ACM ICEA '20: Proceedings of the 2020 ACM International Conference on Intelligent Computing and its Emerging ApplicationsDifferent hyperparameter settings for a deep neural network (DNN) algorithm will come up with different prediction results. One of the most important things is thus in selecting a set of suitable hyperparameters for a DNN so as to increase its accuracy. ...
An automatic hyperparameter optimization DNN model for precipitation prediction
AbstractDeep neural networks (DNN) have gained remarkable success on many rainfall predictions tasks in recent years. However, the performance of DNN highly relies upon the hyperparameter setting. In order to design DNNs with the best performance, ...
Improved particle swarm optimization algorithm based on grouping and its application in hyperparameter optimization
AbstractIn this article, an Improved Particle Swarm Optimization (IPSO) is proposed for solving global optimization and hyperparameter optimization. This improvement is proposed to reduce the probability of particles falling into local optimum and ...






Comments