Abstract
Chart images exhibit significant variabilities that make each image different from others even though they belong to the same class or categories. Classification of charts is a major challenge because each chart class has variations in features, structure, and noises. However, due to the lack of affiliation between the dissimilar features and the structure of the chart, it is a challenging task to model these variations for automatic chart recognition. In this article, we present a novel dissimilarity-based learning model for similar structured but diverse chart classification. Our approach jointly learns the features of both dissimilar and similar regions. The model is trained by an improved loss function, which is fused by a structural variation-aware dissimilarity index and incorporated with regularization parameters, making the model more prone toward dissimilar regions. The dissimilarity index enhances the discriminative power of the learned features not only from dissimilar regions but also from similar regions. Extensive comparative evaluations demonstrate that our approach significantly outperforms other benchmark methods, including both traditional and deep learning models, over publicly available datasets.
- [1] . 2013. Boosting for learning from multiclass data sets via a regularized loss function. In Proceedings of the 2013 IEEE International Conference on Granular Computing (GrC’13). IEEE, Los Alamitos, CA, 4–9.Google Scholar
Cross Ref
- [2] . 2017. Convolutional neural network based chart image classification. In Proceedings of the 25th International Conference in Central Europe on Computer Graphics, Visualization, and Computer Vision.Google Scholar
- [3] . 2014. Study.com—Bar Graph Definition, Types and Examples. Retrieved September 16, 2021 from https://study.com/academy/lesson/bar-graph-definition-types-examples.html.Google Scholar
- [4] . 2019. Dynamic attention loss for small-sample image classification. In Proceedings of the 2019 Asia-Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA ASC’19). IEEE, Los Alamitos, CA, 75–79.Google Scholar
Cross Ref
- [5] . 2018. Evaluation of convolutional neural network architectures for chart image classification. In Proceedings of the 2018 International Joint Conference on Neural Networks (IJCNN’18). IEEE, Los Alamitos, CA, 1–8.Google Scholar
Cross Ref
- [6] . 2015. What may visualization processes optimize? IEEE Transactions on Visualization and Computer Graphics 22, 12 (2015), 2619–2632. Google Scholar
Digital Library
- [7] . 2013. Graphical figure classification using data fusion for integrating text and image features. In Proceedings of the 2013 12th International Conference on Document Analysis and Recognition. IEEE, Los Alamitos, CA, 693–697. Google Scholar
Digital Library
- [8] . 2018. Learning rotation-invariant and Fisher discriminative convolutional neural networks for object detection. IEEE Transactions on Image Processing 28, 1 (2018), 265–278. Google Scholar
Digital Library
- [9] . 2019. Visualizing for the non-visual: Enabling the visually impaired to use visualization. Computer Graphics Forum (2019), 249–260.Google Scholar
- [10] . 2005. Histograms of oriented gradients for human detection. In Proceedings of the 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR’05), Vol. 1. IEEE, Los Alamitos, CA, 886–893. Google Scholar
Digital Library
- [11] . 2019. ICDAR 2019 competition on harvesting raw tables from infographics (CHART-infographics). In Proceedings of the 2019 International Conference on Document Analysis and Recognition (ICDAR’19). IEEE, Los Alamitos, CA, 1594–1599.Google Scholar
Cross Ref
- [12] . 2020. Exploring the role of loss functions in multiclass classification. In Proceedings of the 2020 54th Annual Conference on Information Sciences and Systems (CISS’20). IEEE, Los Alamitos, CA, 1–5.Google Scholar
Cross Ref
- [13] . 2016. Discriminant correlation analysis: Real-time feature level fusion for multimodal biometric recognition. IEEE Transactions on Information Forensics and Security 11, 9 (2016), 1984–1996. Google Scholar
Digital Library
- [14] . 2018. Neural Networks for Machine Learning Online Course. Retrieved September 16, 2021 from https://www.coursera.org/learn/neural-networks/home/welcome.Google Scholar
- [15] . 2015. Multi-view ensemble manifold regularization for 3D object recognition. Information Sciences 320 (2015), 395–405. Google Scholar
Digital Library
- [16] . 2007. A system for understanding imaged infographics and its applications. In Proceedings of the 2007 ACM Symposium on Document Engineering. 9–18. Google Scholar
Digital Library
- [17] . 2004. Elliptic arc vectorization for 3D pie chart recognition. In Proceedings of the 2004 International Conference on Image Processing (ICIP’04), Vol. 5. IEEE, Los Alamitos, CA, 2889–2892.Google Scholar
- [18] . 2007. Chart image classification using multiple-instance learning. In Proceedings of the 2007 IEEE Workshop on Applications of Computer Vision (WACV’07). IEEE, Los Alamitos, CA, 27–27. Google Scholar
Digital Library
- [19] . 2018. Graph Laplacian regularized graph convolutional networks for semi-supervised learning. arXiv:1809.09839.Google Scholar
- [20] . 2017. ChartSense: Interactive data extraction from chart images. In Proceedings of the 2017 Chi Conference on Human Factors in Computing Systems. 6706–6717. Google Scholar
Digital Library
- [21] . 2014. New measures of homogeneity for image processing: An application to fingerprint segmentation. Soft Computing 18, 6 (2014), 1055–1066. Google Scholar
Digital Library
- [22] . 2017. Figureqa: An annotated figure dataset for visual reasoning. arXiv:1710.07300.Google Scholar
- [23] . 2012. Machine learning classification algorithms to recognize chart types in portable document format (PDF) files. International Journal of Computer Applications 39, 2 (2012), 1–5.Google Scholar
Cross Ref
- [24] . 2011. Automatic figure classification in bioscience literature. Journal of Biomedical Informatics 44, 5 (2011), 848–858. Google Scholar
Digital Library
- [25] . 2018. Data Science: Concepts and Practice. Morgan Kaufmann.Google Scholar
- [26] . 2012. ImageNet classification with deep convolutional neural networks. In Proceedings of the 25th International Conference on Neural Information Processing Systems—Volume 1(NIPS’12). 1097–1105. Google Scholar
Digital Library
- [27] . 1998. Gradient-based learning applied to document recognition. Proceedings of the IEEE 86, 11 (1998), 2278–2324.Google Scholar
Cross Ref
- [28] . 2018.
-Laplacian regularization for scene recognition. IEEE Transactions on Cybernetics 49, 8 (2018), 2927–2940.Google ScholarCross Ref
- [29] . 1999. Object recognition from local scale-invariant features. In Proceedings of the 7th IEEE International Conference on Computer Vision, Vol. 2. IEEE, Los Alamitos, CA, 1150–1157. Google Scholar
Digital Library
- [30] . 2011. Model-based chart image classification. In Proceedings of the International Symposium on Visual Computing. 476–485. Google Scholar
Digital Library
- [31] . 2016. Pie Chart and Donut Chart. https://code.tutsplus.comRetrieved November 14, 2016 from.Google Scholar
- [32] . 2006. A visual vocabulary for flower classification. In Proceedings of the 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR’06), Vol. 2. IEEE, Los Alamitos, CA, 1447–1454. Google Scholar
Digital Library
- [33] . 2002. Multiresolution gray-scale and rotation invariant texture classification with local binary patterns. IEEE Transactions on Pattern Analysis and Machine Intelligence 24, 7 (2002), 971–987. Google Scholar
Digital Library
- [34] . 2001. Modeling the shape of the scene: A holistic representation of the spatial envelope. International Journal of Computer Vision 42, 3 (2001), 145–175. Google Scholar
Digital Library
- [35] . 2019. Effects of modifying the input features and the loss function on improving emotion classification. In Proceedings of the 2019 IEEE Region 10 Conference (TENCON’19). IEEE, Los Alamitos, CA, 1159–1162.Google Scholar
Cross Ref
- [36] . 2015. Discriminative graph regularized extreme learning machine and its application to face recognition. Neurocomputing 149 (2015), 340–353. Google Scholar
Digital Library
- [37] . 2017. Reverse-engineering visualizations: Recovering visual encodings from chart images. Computer Graphics Forum 36 (2017), 353–363. Google Scholar
Digital Library
- [38] . 2007. Classifying computer generated charts. In Proceedings of the 2007 International Workshop on Content-Based Multimedia Indexing. IEEE, Los Alamitos, CA, 85–92.Google Scholar
Cross Ref
- [39] . 2014. Oracle Docs. Retrieved September 16, 2021 from docs.oracle.com.Google Scholar
- [40] . 2013. Classification via regularization on graphs. In Proceedings of the 2013 IEEE Global Conference on Signal and Information Processing. IEEE, Los Alamitos, CA, 495–498.Google Scholar
Cross Ref
- [41] . 2011. Revision: Automated classification, analysis and redesign of chart images. In Proceedings of the 24th Annual ACM Symposium on User Interface Software and Technology. 393–402. Google Scholar
Digital Library
- [42] . 2013. Indian movie face database: A benchmark for face recognition under wide variations. In Proceedings of the 2013 4th National Conference on Computer Vision, Pattern Recognition, Image Processing, and Graphics (NCVPRIPG’13). IEEE, Los Alamitos, CA, 1–5.Google Scholar
Cross Ref
- [43] . 2005. Graphics recognition in PDF documents. In Proceedings of the 6th International Conference on Graphics Recognition (GREC’05).Google Scholar
- [44] . 2008. Recognition and quality assessment of data charts in mixed-mode documents. International Journal of Document Analysis and Recognition 11, 3 (2008), 111. Google Scholar
Digital Library
- [45] . 2016. FigureSeer: Parsing result-figures in research papers. In Proceedings of the European Conference on Computer Vision. 664–680.Google Scholar
Cross Ref
- [46] . 2014. Very deep convolutional networks for large-scale image recognition. arXiv:1409.1556.Google Scholar
- [47] . 2018. Discriminative deep feature learning for semantic-based image retrieval. IEEE Access 6 (2018), 44268–44280.Google Scholar
Cross Ref
- [48] . 2015. How to Design Area Charts. https://visage.co/data-visualization-101-area-charts/Retrieved January 13, 2015 from.Google Scholar
- [49] . 2015. Going deeper with convolutions. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 1–9.Google Scholar
Cross Ref
- [50] . 2007. Graph based semi and unsupervised classification and segmentation of microscopic images. In Proceedings of the 2007 IEEE International Symposium on Signal Processing and Information Technology. IEEE, Los Alamitos, CA, 1160–1165.Google Scholar
Cross Ref
- [51] . 2016. DeepChart: Combining deep convolutional networks and deep belief networks in chart classification. Signal Processing 124 (2016), 156–161. Google Scholar
Digital Library
- [52] . 2020. Class balanced loss for image classification. IEEE Access 8 (2020), 81142–81153.Google Scholar
Cross Ref
- [53] . n.d. Lucid Charts. Retrieved September 16, 2021 from https://www.lucidchart.com/blog/how-to-make-a-bubble-chart-in-excel.Google Scholar
- [54] . 2016. A new discriminative sparse representation method for robust face recognition via
{2}
regularization. IEEE Transactions on Neural Networks and Learning Systems 28, 10 (2016), 2233–2242.Google ScholarCross Ref
- [55] . 2019. Deep graph regularized learning for binary classification. In Proceedings of the 2019 IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP’19). IEEE, Los Alamitos, CA, 3537–3541.Google Scholar
Cross Ref
- [56] . 2015. Scene recognition by manifold regularized deep learning architecture. IEEE Transactions on Neural Networks and Learning Systems 26, 10 (2015), 2222–2233.Google Scholar
Cross Ref
- [57] . 2019. Robust audio-visual speech recognition using bimodal DFSMN with multi-condition training and dropout regularization. In Proceedings of the 2019 IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP’19). IEEE, Los Alamitos, CA, 6570–6574.Google Scholar
Cross Ref
- [58] . 2001. Learning-based scientific chart recognition. In Proceedings of the 4th IAPR International Workshop on Graphics Recognition (GREC’01). 482–492.Google Scholar
Index Terms
Dissimilarity-Based Regularized Learning of Charts
Recommendations
Challenges in chart image classification: a comparative study of different deep learning methods
DocEng '21: Proceedings of the 21st ACM Symposium on Document EngineeringCharts are commonly used forms of visualizing scientific observations from research findings or commercial trends. They provide an abstraction of the underlying information in a more understandable way. Over time, different forms of charts are ...
Chart classification: an empirical comparative study of different learning models
ICVGIP '21: Proceedings of the Twelfth Indian Conference on Computer Vision, Graphics and Image ProcessingCharts are powerful tools for visualizing and comparing data. Representation of information through charts grows with time due to its easy and aesthetically attractive structure. With the increase in the number of documents with various chart types, ...
Classifying Chart Based on Structural Dissimilarities using Improved Regularized Loss Function
AbstractClassification of charts is a major challenge because each chart class has variations due to the styles, appearances, structure, and noises caused due to changing data values. These variations differ across all chart types and sub-types. Hence, it ...






Comments