skip to main content
research-article

An Effective Hyperparameter Optimization Algorithm for DNN to Predict Passengers at a Metro Station

Authors Info & Claims
Published:30 March 2021Publication History
Skip Abstract Section

Abstract

As one of the public transportation systems, metro is certainly an indispensable part in urban areas of a metropolis today. Several successful results have shown that deep learning technologies might provide an effective way to predict the number of passengers at a metro station. However, most information systems based on deep learning technologies are usually designed and tuned manually by using domain knowledge and trial-and-error; thus, how to find out a set of suitable hyperparameters for a deep neural network (DNN) has become a critical research issue. To deal with the problem of hyperparameter setting for a DNN in solving the prediction of passengers at a metro station, a novel metaheuristic algorithm called search economics for hyperparameter optimization is presented to improve the accuracy rate of such a prediction system. The basic idea of the proposed algorithm is to divide the solution space into a set of subspaces first and then assign a different number of search agents to each subspace based on the “potential of each subspace.” The potential is estimated based on the objective values of the searched solutions, the objective values of the probe solutions, and the computation time invested in each subspace. The proposed method is compared with Bayesian, random forest, support vector regression, DNN, and DNN with different hyperparameter search algorithms, namely, grid search, simulated annealing, and particle swarm optimization. The simulation results using the data provided by the government of Taipei city, Taiwan, indicate that the proposed method outperforms all the other forecasting methods compared in this article in terms of the mean absolute percentage error.

References

  1. James Bergstra and Yoshua Bengio. 2012. Random search for hyper-parameter optimization. J. Mach. Learn. Res. 13 (2012), 281–305. Google ScholarGoogle ScholarDigital LibraryDigital Library
  2. James S. Bergstra, Rémi Bardenet, Yoshua Bengio, and Balázs Kégl. 2011. Algorithms for hyper-parameter optimization. In Proceedings of the International Conference on Neural Information Processing. 2546–2554. Google ScholarGoogle ScholarDigital LibraryDigital Library
  3. Christian Blum and Andrea Roli. 2003. Metaheuristics in combinatorial optimization: Overview and conceptual comparison. ACM Comput. Surveys 35, 3 (2003), 268–308. Google ScholarGoogle ScholarDigital LibraryDigital Library
  4. Andrés Camero, Jamal Toutouh, Daniel H. Stolfi, and Enrique Alba. 2019. Evolutionary deep learning for car park occupancy prediction in smart cities. In Proceedings of the International Conference on Learning and Intelligent Optimization. 386–401.Google ScholarGoogle ScholarCross RefCross Ref
  5. Irina Ceapa, Chris Smith, and Licia Capra. 2012. Avoiding the crowds: Understanding tube station congestion patterns from trip data. In Proceedings of the ACM SIGKDD International Workshop on Urban Computing. 134–141. Google ScholarGoogle ScholarDigital LibraryDigital Library
  6. Taiwan Central Weather Bureau. 2020. CWB Observation Data Inquire System. Retrieved from https://e-service.cwb.gov.tw/HistoryDataQuery/index.jsp.Google ScholarGoogle Scholar
  7. Ronan Collobert and Jason Weston. 2008. A unified architecture for natural language processing: Deep neural networks with multitask learning. In Proceedings of the International Conference on Machine Learning. 160–167. Google ScholarGoogle ScholarDigital LibraryDigital Library
  8. Taipei Rapid Transit Corporation. 2020. The Passenger Flow Data of Taipei Mass Rapid Transit. Retrieved from http://163.29.157.32:8080/fi/dataset/98d67c29-464a-4003-9f78-b1cbb89bff59.Google ScholarGoogle Scholar
  9. Giovanni Lucca França da Silva, Thales Levi Azevedo Valente, Aristófanes Corrêa Silva, Anselmo Cardoso de Paiva, and Marcelo Gattass. 2018. Convolutional neural network-based PSO for lung nodule false positive reduction on CT images. Comput. Methods Programs Biomed. 162 (2018), 109–118.Google ScholarGoogle ScholarCross RefCross Ref
  10. Thomas Elsken, Jan Hendrik Metzen, and Frank Hutter. 2019. Neural architecture search: A survey. J. Mach. Learn. Res. 20 (2019), 1–21.Google ScholarGoogle Scholar
  11. Matthias Feurer and Frank Hutter. 2019. Hyperparameter Optimization. Springer, Cham, 3–33.Google ScholarGoogle Scholar
  12. Matthias Feurer, Jost Tobias Springenberg, and Frank Hutter. 2015. Initializing Bayesian hyperparameter optimization via meta-learning. In Proceedings of the AAAI Conference on Artificial Intelligence. 1128–1135. Google ScholarGoogle ScholarDigital LibraryDigital Library
  13. Simon Fong, Suash Deb, and Xin-she Yang. 2018. How meta-heuristic algorithms contribute to deep learning in the hype of big data analytics. In Proceedings of the Progress in Intelligent Computing Techniques: Theory, Practice, and Applications, Vol. 518. 3–25.Google ScholarGoogle ScholarCross RefCross Ref
  14. Fred Glover and Gary A. Kochenberger (Eds.). 1993. Handbook of Metaheuristics. Kluwer Academic Publishers, Boston, MA.Google ScholarGoogle Scholar
  15. Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2016. Deep residual learning for image recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 770–778.Google ScholarGoogle ScholarDigital LibraryDigital Library
  16. Xin He, Kaiyong Zhao, and Xiaowen Chu. 2019. AutoML: A survey of the state-of-the-art. Retrieved from https://arxiv:1908.00709.Google ScholarGoogle Scholar
  17. Erik Jenelius. 2019. Data-driven metro train crowding prediction based on real-time load data. IEEE Trans. Intell. Transport. Syst. (2019), 1–12, In Press.Google ScholarGoogle ScholarCross RefCross Ref
  18. Jaeyoung Jung and Keemin Sohn. 2017. Deep-learning architecture to forecast destinations of bus passengers from entry-only smart-card data. IET Intell. Transport. Syst. 11, 6 (2017), 334–339.Google ScholarGoogle ScholarCross RefCross Ref
  19. Tae-Young Kim and Sung-Bae Cho. 2019. Particle swarm optimization-based CNN-LSTM networks for forecasting energy consumption. In Proceedings of the IEEE Congress on Evolutionary Computation. 1510–1516.Google ScholarGoogle ScholarCross RefCross Ref
  20. Woo-Young Lee, Seung-Min Park, and Kwee-Bo Sim. 2018. Optimal hyperparameter tuning of convolutional neural networks based on the parameter-setting-free harmony search algorithm. Optik 172 (2018), 359–367.Google ScholarGoogle ScholarCross RefCross Ref
  21. Lisha Li, Kevin Jamieson, Giulia DeSalvo, Afshin Rostamizadeh, and Ameet Talwalkar. 2018. Hyperband: A novel bandit-based approach to hyperparameter optimization. J. Mach. Learn. Res. 18 (2018), 1–52. Google ScholarGoogle ScholarDigital LibraryDigital Library
  22. Victor C. Liang, Richard T. B. Ma, Wee Siong Ng, Li Wang, Marianne Winslett, Huayu Wu, Shanshan Ying, and Zhenjie Zhang. 2016. Mercury: Metro density prediction with recurrent neural network on streaming CDR data. In Proceedings of the IEEE International Conference on Data Engineering. 1374–1377.Google ScholarGoogle ScholarCross RefCross Ref
  23. Xiaoyuan Liang, Xunsheng Du, Guiling Wang, and Zhu Han. 2019. A deep reinforcement learning network for traffic light cycle control. IEEE Trans. Vehic. Technol. 68, 2 (2019), 1243–1253.Google ScholarGoogle ScholarCross RefCross Ref
  24. Pablo Ribalta Lorenzo, Jakub Nalepa, Michal Kawulok, Luciano Sanchez Ramos, and José Ranilla Pastor. 2017. Particle swarm optimization for hyper-parameter selection in deep neural networks. In Proceedings of the Genetic and Evolutionary Computation Conference. 481–488. Google ScholarGoogle ScholarDigital LibraryDigital Library
  25. Ilya Loshchilov and Frank Hutter. 2016. CMA-ES for hyperparameter optimization of deep neural networks. Retrieved from https://arxiv:1604.07269.Google ScholarGoogle Scholar
  26. Yisheng Lv, Yanjie Duann, Wenwen Kang, Zhengxi Li, and Fei-Yue Wang. 2015. Traffic flow prediction with big data: A deep learning approach. IEEE Trans. Intell. Transport. Syst. 16, 2 (2015), 865–873.Google ScholarGoogle Scholar
  27. Noboru Nakafuri and Yoshio Miki. 2017. Prediction model of number of bus users based on influence of sensible temperature and discomfort index. In Proceedings of the IEEE International Conference on Intelligent Transportation Engineering. 155–158.Google ScholarGoogle ScholarCross RefCross Ref
  28. A. J. Nebro, J. J. Durillo, J. García-Nieto, C. A. Coello Coello, F. Luna, and E. Alba. 2009. SMPSO: A new PSO-based metaheuristic for multi-objective optimization. In Proceedings of the IEEE Symposium on Computational Intelligence in Multi-Criteria Decision-Making. 66–73.Google ScholarGoogle Scholar
  29. Ming Ni, Qing He, and Jing Gao. 2017. Forecasting the subway passenger flow under event occurrences with social media. IEEE Trans. Intell. Transport. Syst. 18, 6 (2017), 1623–1632.Google ScholarGoogle Scholar
  30. Fernando Nogueira. 2014. Bayesian Optimization: Open source constrained global optimization tool for Python. Retrieved from https://github.com/fmfn/BayesianOptimization.Google ScholarGoogle Scholar
  31. Yoshihiko Ozaki, Masaki Yano, and Masaki Onishi. 2017. Effective hyperparameter optimization using Nelder-Mead method in deep learning. IPSJ Trans. Comput. Vision Appl. 9, 20 (2017), 1–12.Google ScholarGoogle Scholar
  32. Basheer Qolomany, Majdi Maabreh, Ala Al-Fuqaha, Ajay Gupta, and Driss Benhaddou. 2017. Parameters optimization of deep learning models using particle swarm optimization. In Proceedings of the International Wireless Communications and Mobile Computing Conference. 1285–1290.Google ScholarGoogle ScholarCross RefCross Ref
  33. Heath Rushing, Jim Wisnowski, and Andrew Karl. 2013. Design and Analysis of Experiments by Douglas Montgomery: A Supplement for Using JMP. SAS Institute, Cary, NC. Google ScholarGoogle ScholarDigital LibraryDigital Library
  34. scikit-learn 0.23.1. 2020. Retrieved from https://scikit-learn.org/stable/.Google ScholarGoogle Scholar
  35. Tatsuki Serizawa and Hamido Fujita. 2020. Optimization of convolutional neural network using the linearly decreasing weight particle swarm optimization. Retrieved from https://arxiv:2001.05670.Google ScholarGoogle Scholar
  36. Dhruv Srivastava, Yasha Singh, and Anita Sahoo. 2019. Auto tuning of RNN hyper-parameters using cuckoo search algorithm. In Proceedings of the International Conference on Contemporary Computing. 1–5.Google ScholarGoogle ScholarCross RefCross Ref
  37. Tetsuya Tamaki, Hiroki Nakamura, Hidemichi Fujii, and Shunsuke Managi. 2019. Efficiency and emissions from urban transport: Application to world city-level public transportation. Econ. Anal. Policy 61 (2019), 55–63.Google ScholarGoogle ScholarCross RefCross Ref
  38. Liyang Tang, Yang Zhao, Javier Cabrera, Jian Ma, and Kwok Leung Tsui. 2019. Forecasting short-term passenger flow: An empirical study on Shenzhen metro. IEEE Trans. Intell. Transport. Syst. 20, 10 (2019), 3613–3622.Google ScholarGoogle ScholarCross RefCross Ref
  39. Chris Thornton, Frank Hutter, Holger H. Hoos, and Kevin Leyton-Brown. 2013. Auto-WEKA: Combined selection and hyperparameter optimization of classification algorithms. In Proceedings of the ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. 847–855. Google ScholarGoogle ScholarDigital LibraryDigital Library
  40. Wichai Treethidtaphat, Wasan Pattara-Atikom, and Sippakorn Khaimook. 2017. Bus arrival time prediction at any distance of bus route using deep neural network model. In Proceedings of the IEEE International Conference on Intelligent Transportation Systems. 988–992.Google ScholarGoogle ScholarCross RefCross Ref
  41. Chun-Wei Tsai. 2016. An effective WSN deployment algorithm via search economics. Comput. Netw. 101 (2016), 178–191. Google ScholarGoogle ScholarDigital LibraryDigital Library
  42. Chun-Wei Tsai, Chien-Hui Hsia, Shuang-Jie Yang, Shih-Jui Liu, and Zhi-Yan Fang. 2020. Optimizing hyperparameters of deep learning in predicting bus passengers based on simulated annealing. Appl. Soft Comput. 88 (2020), 1–9.Google ScholarGoogle ScholarDigital LibraryDigital Library
  43. Chun-Wei Tsai and Shi-Jui Liu. 2018. An effective IoT service-to-interface assignment algorithm via search economics. IEEE Internet Things J. 5, 3 (2018), 1708–1718.Google ScholarGoogle ScholarCross RefCross Ref
  44. Chun-Wei Tsai and J. J. P. C. Rodrigues. 2014. Metaheuristic scheduling for cloud: A survey. IEEE Systems Journal 8, 1 (Mar. 2014), 279–291.Google ScholarGoogle ScholarCross RefCross Ref
  45. John Voelcker. 2014. 1.2 Billion Vehicles on World’s Roads Now, 2 Billion By 2035: Report. Technical Report. Green Car Report. Retrieved from https://www.greencarreports.com/news/1093560_1-2-billion-vehicles-on-worlds-roads-now-2-billion-by-2035-report.Google ScholarGoogle Scholar
  46. Jiazhuo Wang, Jason Xu, and Xuejun Wang. 2018. Combination of hyperband and Bayesian optimization for hyperparameter optimization in deep learning. Retrieved from https://arxiv:1801.01596.Google ScholarGoogle Scholar
  47. Quanming Yao, Mengshuo Wang, Hugo Jair Escalante, Isabelle Guyon, Yi-Qi Hu, Yu-Feng Li, Wei-Wei Tu, Qiang Yang, and Yang Yu. 2018. Taking human out of learning applications: A survey on automated machine learning. Retrieved from https://arxiv:1810.13306.Google ScholarGoogle Scholar
  48. Fei Ye. 2017. Particle swarm optimization-based automatic parameter selection for deep neural networks and its applications in large-scale and high-dimensional data. PLoS ONE 12, 12 (12 2017), 1–36.Google ScholarGoogle Scholar
  49. Steven R. Young, Derek C. Rose, Thomas P. Karnowski, Seung-Hwan Lim, and Robert M. Patton. 2015. Optimizing deep learning hyper-parameters through an evolutionary algorithm. In Proceedings of the Workshop on Machine Learning in High-Performance Computing Environments. 1–5. Google ScholarGoogle ScholarDigital LibraryDigital Library

Index Terms

  1. An Effective Hyperparameter Optimization Algorithm for DNN to Predict Passengers at a Metro Station

      Recommendations

      Comments

      Login options

      Check if you have access through your login credentials or your institution to get full access on this article.

      Sign in

      Full Access

      • Published in

        cover image ACM Transactions on Internet Technology
        ACM Transactions on Internet Technology  Volume 21, Issue 2
        June 2021
        599 pages
        ISSN:1533-5399
        EISSN:1557-6051
        DOI:10.1145/3453144
        • Editor:
        • Ling Liu
        Issue’s Table of Contents

        Copyright © 2021 Association for Computing Machinery.

        Publisher

        Association for Computing Machinery

        New York, NY, United States

        Publication History

        • Published: 30 March 2021
        • Accepted: 1 July 2020
        • Revised: 1 June 2020
        • Received: 1 March 2020
        Published in toit Volume 21, Issue 2

        Permissions

        Request permissions about this article.

        Request Permissions

        Check for updates

        Qualifiers

        • research-article
        • Refereed

      PDF Format

      View or Download as a PDF file.

      PDF

      eReader

      View online with eReader.

      eReader

      HTML Format

      View this article in HTML Format .

      View HTML Format
      About Cookies On This Site

      We use cookies to ensure that we give you the best experience on our website.

      Learn more

      Got it!