Open the Black Box – Visualising CNN to Understand Its Decisions on Road Network Performance Level
Abstract
Visualisation helps explain the operating mechanisms of deep learning models, but its applications are rarely seen in traffic analysis. This paper employs a convolu-tional neural network (CNN) to evaluate road network performance level (NPL) and visualises the model to en-lighten how it works. A dataset of an urban road network covering a whole year is used to produce performance maps to train a CNN. In this process, a pretrained network is introduced to overcome the common issue of inadequa-cy of data in transportation research. Gradient weighted class activation mapping (Grad-CAM) is applied to vi-sualise the CNN, and four visualisation experiments are conducted. The results illustrate that the CNN focuses on different areas when it identifies the road network as dif-ferent NPLs, implying which region contributes the most to the deteriorating performance. There are particular visual patterns when the road network transits from one NPL to another, which may help performance prediction. Misclassified samples are analysed to determine how the CNN fails to make the right decisions, exposing the model’s deficiencies. The results indicate visualisation’s potential to contribute to comprehensive management strategies and effective model improvement.
References
Zhang J, et al. Data-driven intelligent transportation systems: A survey. IEEE Transactions on Intelligent Transportation Systems. 2011;12(4): 1624-1639. doi: 10.1109/TITS.2011.2158001.
Pamula T. Road traffic conditions classification based on multilevel filtering of image content using convolutional neural networks. IEEE Intelligent Transportation Systems Magazine. 2018;10(3): 11-21. doi: 10.1109/MITS.2018.2842040.
Duan Y, Lv Y, Liu Y, Wang F. An efficient realization of deep learning for traffic data imputation. Transportation Research Part C - Emerging Technologies. 2016;72: 168-181. doi: 10.1016/j.trc.2016.09.015.
Zhao Z, et al. LSTM network: A deep learning approach for short-term traffic forecast. IET Intelligent Transport Systems. 2017;11(3): 68-75. doi: 10.1049/iet-its.2016.0208.
Cheng Z, Wang W, Lu J, Xing X. Classifying the traffic state of urban expressways: A machine-learning approach. Transportation Research Part A - Policy and Practice. 2020;137: 411-428. doi: 10.1016/j.tra.2018.10.035.
Hoang N, Le-Minh K, Tao W, Chen C. Deep learning methods in transportation domain: A review. IET Intelligent Transport Systems. 2018;12(9): 998-1004. doi: 10.1049/iet-its.2018.0064.
Karlaftis MG, Vlahogianni EI. Statistical methods versus neural networks in transportation research: Differences, similarities and some insights. Transportation Research Part C - Emerging Technologies. 2011;19(3): 387-399. doi: 10.1016/j.trc.2010.10.004.
Selvaraju RR, et al. Grad-CAM: Visual explanations from deep networks via gradient-based localization. International Journal of Computer Vision. 2020;128(2): 336-359. doi: 10.1007/s11263-019-01228-7.
Du S, Li T, Gong X, Horng S. A hybrid method for traffic flow forecasting using multimodal deep learning. International Journal of Computational Intelligence Systems. 2020;13(1): 85-97. doi: 10.2991/ijcis.d.200120.001.
Bogaerts T, et al. A graph CNN-LSTM neural network for short and long-term traffic forecasting based on trajectory data. Transportation Research Part C - Emerging Technologies. 2020;112: 62-77. doi: 10.1016/j.trc.2020.01.010.
Zeiler MD, Fergus R. Visualizing and understanding convolutional networks. Proceedings of ECCV, 6-12 Sep. 2014, Zurich, Switzerland. Cham, Switzerland: Springer International Publishing; 2014. p. 818-833. doi: 10.1007/978-3-319-10590-1_53.
Springenberg JT, Dosovitskiy A, Brox T, Riedmiller M. Striving for simplicity: The all convolutional net. Proceedings of ICLR, 7-9 May 2015, San Diego, CA, USA. 2015. arXiv:1412.6806.
Mahendran A, Vedaldi A. Visualizing deep convolutional neural networks using natural pre-images. International Journal of Computer Vision. 2016;120(3): 233-255. doi: 10.1007/s11263-016-0911-8.
Dosovitskiy A, Brox T. Inverting visual representations with convolutional networks. Proceedings of IEEE Conf. on CVPR, 27-30 June 2016, Las Vegas, NV, USA. Los Alamitos, CA, USA: IEEE Computer Society; 2016. p. 4829-4837. doi: 10.1109/CVPR.2016.522.
Zhou B, et al. Learning deep features for discriminative localization. Proceedings of IEEE Conf. on CVPR, 27-30 June 2016, Las Vegas, NV, USA. Los Alamitos, CA, USA: IEEE Computer Society; 2016. p. 2921-2929. doi: 10.1109/CVPR.2016.319.
Lin M, Chen Q, Yan S. Network in network. Proceedings of ICLR, 14-16 Apr. 2014, Banff, Canada. 2014. arXiv:1312.4400, 2014.
Simonyan K, Zisserman A. Very deep convolutional networks for large scale image recognition. Proceedings of ICLR, 7-9 May 2015, San Diego, CA, USA. 2015. arXiv:1409.1556.
Jogin M, et al. Feature extraction using convolution neural networks (CNN) and deep learning. Proceedings of IEEE Int. Conf. on RTEICT, 18-19 May 2018, Bangalore, India. Piscataway, NJ, USA: IEEE; 2018. p. 2319-2323. doi: 10.1109/RTEICT42901.2018.9012507.
Agrawal P, Girshick R, Malik J. Analyzing the performance of multilayer neural networks for object recognition. Proceedings of ECCV, 6-12 Sept. 2014, Zurich, Switzerland. Cham, Switzerland: Springer International Publishing; 2014. p. 329-344. doi: 10.1007/978-3-319-10584-022.
Chollet F. Deep Learning with Python. Shelter Island, NY, USA: Manning Publications; 2017.
Hinton G. Coursera Course Lectures. 2012. http://www.cs.toronto.edu/~hinton/coursera_slides.html [Accessed 21st Dec. 2021].
Bach S, et al. On pixel-wise explanations for non-linear classifier decisions by layer-wise relevance propagation. PLoS ONE. 2015;10(7): e0130140. doi: 10.1371/journal.pone.0130140.
Shrikumar A, Greenside P, Kundaje A. Learning important features through propagating activation differences. 2019. arXiv:1704.02685v2.
Simonyan K, Vedaldi A, Zisserman A. Deep inside convolutional networks: Visualising image classification models and saliency maps. Proceedings of ICLR, 14-16 Apr. 2014, Banff, AB, Canada. 2015. arXiv: 1312.6034.
Copyright (c) 2022 Junxian LI, Zhizhou WU, Zhoubiao SHEN
This work is licensed under a Creative Commons Attribution 4.0 International License.
Authors who publish with this journal agree to the following terms:
- Authors retain copyright and grant the journal right of first publication with the work simultaneously licensed under a Creative Commons Attribution License that allows others to share the work with an acknowledgement of the work's authorship and initial publication in this journal.
- Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgement of its initial publication in this journal.
- Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work (See The Effect of Open Access).
