Abstract
Each year, an average of around 6 million car accidents occur in the United States. Road safety features (e.g., concrete barriers, metal crash barriers, rumble strips) play an important role in preventing or mitigating vehicle crashes. Accurate maps of road safety features is an important component of safety management systems for federal or state transportation agencies, helping traffic engineers identify locations to invest in safety infrastructure. In current practice, mapping road safety features is largely done manually (e.g., observations on the road or visual interpretation of streetview imagery), which is both expensive and time consuming. In this article, we propose a deep learning approach to automatically map road safety features from streetview imagery. Unlike existing convolutional neural networks that classify each image individually, we propose to further add a recurrent neural network (long short-term memory) to capture geographic context of images (spatial autocorrelation effect along linear road network paths). Evaluations on real-world streetview imagery show that our proposed model outperforms several baseline methods.
- Vahid Balali, Elizabeth Depwe, and Mani Golparvar-Fard. 2015. Multi-class traffic sign detection and classification using Google Street View images. In Proceedings of the Transportation Research Board 94th Annual Meeting.Google Scholar
- Vahid Balali, Armin Ashouri Rad, and Mani Golparvar-Fard. 2015. Detection, classification, and mapping of US traffic signs using Google Street View images for roadway inventory management. Visualization in Engineering 3, 1 (2015), 15.Google Scholar
Cross Ref
- Michael R. Bambach, R. J. Mitchell, and Raphael H. Grzebieta. 2013. The protective effect of roadside barriers for motorcyclists. Traffic Injury Prevention 14, 7 (2013), 756--765.Google Scholar
- James E. Bryden and Jan S. Fortuniewicz. 1986. Traffic barrier performance related to vehicle size and type. Transportation Research Record 1065 (1986), 69--78.Google Scholar
- Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. 2009. ImageNet: A large-scale hierarchical image database. In Proceedings of the 2009 IEEE Conference on Computer Vision and Pattern Recognition. IEEE, Los Alamitos, CA, 248--255.Google Scholar
Cross Ref
- Timnit Gebru, Jonathan Krause, Yilun Wang, Duyun Chen, Jia Deng, Erez Lieberman Aiden, and Li Fei-Fei. 2017. Using deep learning and Google Street View to estimate the demographic makeup of neighborhoods across the United States. Proceedings of the National Academy of Sciences 114, 50 (2017), 13108--13113.Google Scholar
Cross Ref
- Alex Graves and Jürgen Schmidhuber. 2005. Framewise phoneme classification with bidirectional LSTM and other neural network architectures. Neural Networks 18, 5--6 (2005), 602--610.Google Scholar
Digital Library
- Raphael Grzebieta, Mike Bambach, and Andrew McIntosh. 2013. Motorcyclist impacts into roadside barriers: Is the European crash test standard comprehensive enough? Transportation Research Record 2377, 1 (2013), 84--91.Google Scholar
- Gayle S. W. Hagler, Wei Tang, Matthew J. Freeman, David K. Heist, Steven G. Perry, and Alan F. Vette. 2011. Model evaluation of roadside barrier impact on near-road air pollution. Atmospheric Environment 45, 15 (2011), 2522--2530.Google Scholar
- Hussein H. Jama, Raphael H. Grzebieta, Rena Friswell, and Andrew S. McIntosh. 2011. Characteristics of fatal motorcycle crashes into roadside safety barriers in Australia and New Zealand. Accident Analysis 8 Prevention 43, 3 (2011), 652--660.Google Scholar
- Zhe Jiang. 2018. A survey on spatial prediction methods. IEEE Transactions on Knowledge and Data Engineering 31, 9 (2018), 1645--1664.Google Scholar
Digital Library
- Zhe Jiang, Michael Evans, Dev Oliver, and Shashi Shekhar. 2016. Identifying K primary corridors from urban bicycle GPS trajectories on a road network. Information Systems 57 (2016), 142--159.Google Scholar
Digital Library
- Zhe Jiang, Yan Li, Shashi Shekhar, Lian Rampi, and Joseph Knight. 2017. Spatial ensemble learning for heterogeneous geographic data with class ambiguity: A summary of results. In Proceedings of the 25th ACM SIGSPATIAL International Conference on Advances in Geographic Information Systems. ACM, New York, NY, 23.Google Scholar
Digital Library
- Zhe Jiang and Arpan Man Sainju. 2019. Hidden Markov contour tree: A spatial structured model for hydrological applications. In Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. ACM, New York, NY, 804--813.Google Scholar
Digital Library
- Zhe Jiang, Arpan Man Sainju, Yan Li, Shashi Shekhar, and Joseph Knight. 2019. Spatial ensemble learning for heterogeneous geographic data with class ambiguity. ACM Transactions on Intelligent Systems and Technology 10, 4 (2019), 43.Google Scholar
- Zhe Jiang and Shashi Shekhar. 2017. Spatial Big Data Science. Springer.Google Scholar
- Zhe Jiang, Shashi Shekhar, Pradeep Mohan, Joseph Knight, and Jennifer Corcoran. 2012. Learning spatial decision tree for geographical classification: A summary of results. In Proceedings of the 20th International Conference on Advances in Geographic Information Systems. ACM, New York, NY, 390--393.Google Scholar
Digital Library
- Zhe Jiang, Shashi Shekhar, Xun Zhou, Joseph Knight, and Jennifer Corcoran. 2013. Focal-test-based spatial decision tree learning: A summary of results. In Proceedings of the 2013 IEEE 13th International Conference on Data Mining. IEEE, Los Alamitos, CA, 320--329.Google Scholar
- Zhe Jiang, Shashi Shekhar, Xun Zhou, Joseph Knight, and Jennifer Corcoran. 2014. Focal-test-based spatial decision tree learning. IEEE Transactions on Knowledge and Data Engineering 27, 6 (2014), 1547--1559.Google Scholar
Digital Library
- Zhe Jiang, Miao Xie, and Arpan Man Sainju. 2019. Geographical hidden Markov tree. IEEE Transactions on Knowledge and Data Engineering. Early Access. July 23, 2019.Google Scholar
- Hawzheen Karim, Rolf Magnusson, and Mats Wiklund. 2012. Assessment of injury rates associated with road barrier collision. Procedia—Social and Behavioral Sciences 48 (2012), 52--63.Google Scholar
- Driver Knowledge. 2013. Car Accident Statistics in the U.S. Retrieved July 16, 2020 from https://www.driverknowledge.com/car-accident-statistics/.Google Scholar
- Xiaojiang Li, Chuanrong Zhang, Weidong Li, Robert Ricard, Qingyan Meng, and Weixing Zhang. 2015. Assessing street-level urban greenery using Google Street View and a modified green view index. Urban Forestry 8 Urban Greening 14, 3 (2015), 675--685.Google Scholar
- Yang Liu, Chengjie Sun, Lei Lin, and Xiaolong Wang. 2016. Learning natural language inference using bidirectional LSTM model and inner-attention. arXiv:1605.09090.Google Scholar
- Pradeep Mohan, Shashi Shekhar, James A. Shine, James P. Rogers, Zhe Jiang, and Nicole Wayant. 2011. A neighborhood graph based approach to regional co-location pattern discovery: A summary of results. In Proceedings of the 19th ACM SIGSPATIAL International Conference on Advances in Geographic Information Systems. ACM, New York, NY, 122--132.Google Scholar
- Aibek Musaev, Zhe Jiang, Steven Jones, Pezhman Sheinidashtegol, and Mirbek Dzhumaliev. 2018. Detection of damage and failure events of road infrastructure using social media. In Proceedings of the International Conference on Web Services. 134--148.Google Scholar
- Congressional Budget Office. 2018. Public Spending on Transportation and Water Infrastructure, 1956 to 2017. Retrieved July 16, 2020 from https://www.cbo.gov/system/files/2018-10/54539-Infrastructure.pdf.Google Scholar
- Benjamin Romano and Zhe Jiang. 2017. Visualizing traffic accident hotspots based on spatial-temporal network kernel density estimation. In Proceedings of the 25th ACM SIGSPATIAL International Conference on Advances in Geographic Information Systems. ACM, New York, NY, 98.Google Scholar
Digital Library
- Carlos Roque and João Lourenço Cardoso. 2013. Observations on the relationship between European standards for safety barrier impact severity and the degree of injury sustained. IATSS Research 37, 1 (2013), 21--29.Google Scholar
- Arpan Man Sainju, Danial Aghajarian, Zhe Jiang, and Sushil K. Prasad. 2018. Parallel grid-based colocation mining algorithms on GPUs for big spatial event data. IEEE Transactions on Big Data 6, 1 (2018), 107--118.Google Scholar
- Arpan Man Sainju and Zhe Jiang. 2017. Grid-based colocation mining algorithms on GPU for big spatial event data: A summary of results. In Proceedings of the International Symposium on Spatial and Temporal Databases. 263--280.Google Scholar
- Jennifer D. Schmidt, Ronald K. Faller, Dean L. Sicking, John D. Reid, Karla A. Lechtenberg, Robert W. Bielenberg, Scott K. Rosenbaugh, et al. 2013. Development of a New Energy-Absorbing Roadside/Median Barrier System with Restorable Elastomer Cartridges. Technical Report. Nebraska Department of Roads.Google Scholar
- Nico Schulte, Michelle Snyder, Vlad Isakov, David Heist, and Akula Venkatram. 2014. Effects of solid barriers on dispersion of roadway emissions. Atmospheric Environment 97 (2014), 286--295.Google Scholar
- Shashi Shekhar, Zhe Jiang, Reem Ali, Emre Eftelioglu, Xun Tang, Venkata Gunturi, and Xun Zhou. 2015. Spatiotemporal data mining: A computational perspective. ISPRS International Journal of Geo-Information 4, 4 (2015), 2306--2338.Google Scholar
Cross Ref
- Christian Szegedy, Sergey Ioffe, Vincent Vanhoucke, and Alexander A. Alemi. 2017. Inception-v4, Inception-ResNet and the impact of residual connections on learning. In Proceedings of the 31st AAAI Conference on Artificial Intelligence.Google Scholar
- Zheming Tong, Richard W. Baldauf, Vlad Isakov, Parikshit Deshmukh, and K. Max Zhang. 2016. Roadside vegetation barrier designs to mitigate near-road air pollution impacts. Science of the Total Environment 541 (2016), 920--927.Google Scholar
Cross Ref
- Victor J. D. Tsai, Jyun-Han Chen, and Hsun-Sheng Huang. 2016. Traffic sign inventory from Google Street View images. International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences 41 (2016), 243--246.Google Scholar
- Carlos M. Vieira, Henrique A. Almeida, Irene S. Ferreira, Joel O. Vasco, Paulo J. Bártolo, Rui B. Ruben, and Sérgio P. Santos. 2008. Development of an impact absorber for roadside barriers. In Proceedings of the 7th LS-DYNA Forum.Google Scholar
- Yuankai Wu and Huachun Tan. 2016. Short-term traffic flow forecasting with spatial-temporal correlation in a hybrid deep learning framework. arXiv:1612.01022.Google Scholar
- Miao Xie, Zhe Jiang, and Arpan Man Sainju. 2018. Geographical hidden Markov tree for flood extent mapping. In Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. ACM, New York, NY, 2545--2554.Google Scholar
Digital Library
- Huaxiu Yao, Xianfeng Tang, Hua Wei, Guanjie Zheng, Yanwei Yu, and Zhenhui Li. 2018. Modeling spatial-temporal dynamics for traffic prediction. arXiv:1803.01254.Google Scholar
- Huaxiu Yao, Fei Wu, Jintao Ke, Xianfeng Tang, Yitian Jia, Siyu Lu, Pinghua Gong, Jieping Ye, and Zhenhui Li. 2018. Deep multi-view spatial-temporal network for taxi demand prediction. In Proceedings of the 32nd AAAI Conference on Artificial Intelligence.Google Scholar
- Zhuoning Yuan, Xun Zhou, and Tianbao Yang. 2018. Hetero-ConvLSTM: A deep learning approach to traffic accident prediction on heterogeneous spatio-temporal data. In Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. ACM, New York, NY, 984--992.Google Scholar
Digital Library
- Zhuoning Yuan, Xun Zhou, Tianbao Yang, James Tamerius, and Ricardo Mantilla. 2017. Predicting traffic accidents through heterogeneous urban data: A case study. In Proceedings of the 6th International Workshop on Urban Computing (UrbComp’17), Vol. 14.Google Scholar
- Junbo Zhang, Yu Zheng, Dekang Qi, Ruiyuan Li, Xiuwen Yi, and Tianrui Li. 2018. Predicting citywide crowd flows using deep spatio-temporal residual networks. Artificial Intelligence 259 (2018), 147--166.Google Scholar
Cross Ref
- Shanghang Zhang, Guanhang Wu, Joao P. Costeira, and Jose M. F. Moura. 2017. FCN-rLSTM: Deep spatio-temporal neural networks for vehicle counting in city cameras. In Proceedings of the IEEE International Conference on Computer Vision (ICCV’17).Google Scholar
- Yaotian Zou, Andrew P. Tarko, Erdong Chen, and Mario A. Romero. 2014. Effectiveness of cable barriers, guardrails, and concrete barrier walls in reducing the risk of injury. Accident Analysis 8 Prevention 72 (2014), 55--65.Google Scholar
- A. Mehrara Molan, M. Rezapour, and K. Ksaibati. 2019. Investigating the relationship between the type of traffic barrier and crash severity in trucks and light vehicles. J. Traffic. Transp. Eng. (in press).Google Scholar
Index Terms
Mapping Road Safety Features from Streetview Imagery: A Deep Learning Approach
Recommendations
Mapping Road Safety Barriers Across Street View Image Sequences: A Hybrid Object Detection and Recurrent Model
GEOAI '21: Proceedings of the 4th ACM SIGSPATIAL International Workshop on AI for Geographic Knowledge DiscoveryRoad safety barriers (e.g., concrete barriers, metal crash barriers, rumble strips) play an important role in preventing or mitigating vehicle crashes. Accurate maps of road safety barriers are critical components of safety infrastructure management ...
Mechanism Analysis of Over-speeding Impact on the Road Safety
ICICTA '10: Proceedings of the 2010 International Conference on Intelligent Computation Technology and Automation - Volume 02Vehicle over-speeding is the most important reason of traffic accidents. This paper points at the speeding offense and expounds the relationship between over-speeding and traffic accidents. Besides, from the moving characteristic of vehicles, the ...
Safe Drive Map Concept for Road Curve Monitoring
DSD '15: Proceedings of the 2015 Euromicro Conference on Digital System DesignWe present a technique for dangerous curve monitoring relying on the innovative concept of Safe Driving Map (SDM), a geo-referenced database with data about safe vehicle behavior in the monitored area, also considering different weather conditions. A ...






Comments