skip to main content
research-article

GHOSM: Graph-based Hybrid Outline and Skeleton Modelling for Shape Recognition

Authors Info & Claims
Published:17 February 2023Publication History
Skip Abstract Section

Abstract

An efficient and accurate shape detection model plays a major role in many research areas. With the emergence of more complex shapes in real-life applications, shape recognition models need to capture the structure with more effective features to achieve high accuracy rates for shape recognition. This article presents a new method for 2D/3D shape recognition based on graph spectral domain handcrafted features, which are formulated by exploiting both an outline and a skeleton shape through the global outline and internal details. A fully connected graph is generated over the shape outline to capture the global outline representation while a hierarchically clustered graph with adaptive connectivity is formed on the skeleton to capture the structural descriptions of the shape. We demonstrate the ability of the Fiedler vector to provide the graph partitioning of the skeleton graph. The performance evaluation demonstrates the efficiency of the proposed method compared to state-of-the-art studies with increments of 4.09%, 2.2%, and 14.02% for 2D static hand gestures, 2D shapes, and 3D shapes, respectively.

REFERENCES

  1. [1] Aghasi A. and Romberg J.. 2018. Extracting the principal shape components via convex programming. IEEE Trans. Image Process. 27, 7 (2018), 35133528.Google ScholarGoogle ScholarCross RefCross Ref
  2. [2] Akimaliev M. and Demirci M.. 2015. Improving skeletal shape abstraction using multiple optimal solutions. Pattern Recogn. 48, 11 (2015), 35043515.Google ScholarGoogle ScholarDigital LibraryDigital Library
  3. [3] Alwaely B. and Abhayaratne C.. 2019. Graph spectral domain feature learning with application to in-air hand-drawn number and shape recognition. IEEE Access 7 (2019), 159661159673.Google ScholarGoogle ScholarCross RefCross Ref
  4. [4] Alwaely B. and Abhayaratne C.. 2019. Graph spectral domain features for static hand gesture recognition. In Proceedings of the European Signal Processing Conference (EUSIPCO’19). 15.Google ScholarGoogle ScholarCross RefCross Ref
  5. [5] Alwaely B. and Abhayaratne C.. 2020. AGSF: Adaptive graph formulation and handcrafted graph spectral features for shape representation. IEEE Access 8, 1 (2020), 113.Google ScholarGoogle Scholar
  6. [6] Aowal M. A., Zaman A. S., Rahman S. M., and Hatzinakos D.. 2014. Static hand gesture recognition using discriminative 2D Zernike moments. In Proceedings of the Region 10 Conference (TENCON’14). IEEE, 15.Google ScholarGoogle ScholarCross RefCross Ref
  7. [7] Asad M. and Abhayaratne C.. 2013. Kinect depth stream pre-processing for hand gesture recognition. In Proceedings of the International Conference on Image Processing. IEEE, 37353739. Google ScholarGoogle ScholarCross RefCross Ref
  8. [8] Aslan C., Erdem A., Erdem E., and Tari S.. 2008. Disconnected skeleton: Shape at its absolute scale. IEEE Trans. Pattern Anal. Mach. Intell. 30, 12 (2008), 21882203.Google ScholarGoogle ScholarDigital LibraryDigital Library
  9. [9] Ayzenberg V. and Lourenco S. F.. 2019. Skeletal descriptions of shape provide unique perceptual information for object recognition. Sci. Rep. 9 (2019), 9359.Google ScholarGoogle ScholarCross RefCross Ref
  10. [10] Bai X., Latecki L., and Liu W.. 2007. Skeleton pruning by contour partitioning with discrete curve evolution. IEEE Trans. Pattern Anal. Mach. Intell. 29, 3 (2007).Google ScholarGoogle ScholarDigital LibraryDigital Library
  11. [11] Bai X., Liu W., and Tu Z.. 2009. Integrating contour and skeleton for shape classification. In Proceedings of the International Conference on Computer Vision Workshops (ICCV’09). IEEE, 360367.Google ScholarGoogle ScholarCross RefCross Ref
  12. [12] Baker N. and Kellman P. J.. 2018. Abstract shape representation in human visual perception. J. Exper. Psychol.: Gen. 147, 9 (2018), 1295.Google ScholarGoogle ScholarCross RefCross Ref
  13. [13] Barczak A., Reyes N., Abastillas M., Piccio A., and Susnjak T.. 2011. A new 2D static hand gesture colour image dataset for ASL gestures. Res. Lett. Info. Math. Sci. 15 (2011), 1220.Google ScholarGoogle Scholar
  14. [14] Brincat S. L. and Connor C. E.. 2004. Underlying principles of visual shape selectivity in posterior inferotemporal cortex. Nature Neurosci. 7, 8 (2004), 880.Google ScholarGoogle ScholarCross RefCross Ref
  15. [15] Carcassoni M. and Hancock E.. 2002. Alignment using spectral clusters. In Proceedings of the British Machine Vision Conference (BMVC’02). 110.Google ScholarGoogle ScholarCross RefCross Ref
  16. [16] Verroust-Blondet A. Chaouch, and M.. 2007. A new descriptor for 2D depth image indexing and 3D model retrieval. In Proceedings of the International Conference on Image Processing, Vol. 6. IEEE, VI–373.Google ScholarGoogle Scholar
  17. [17] Chen X., Golovinskiy A., and Funkhouser T.. 2009. A benchmark for 3D mesh segmentation. ACM Trans. Graph. 28 (2009), 73.Google ScholarGoogle ScholarDigital LibraryDigital Library
  18. [18] Connor C. E., Brincat S. L., and Pasupathy A.. 2007. Transformation of shape information in the ventral pathway. Curr. Opin. Neurobiol. 17, 2 (2007), 140147.Google ScholarGoogle ScholarCross RefCross Ref
  19. [19] Cour T., Srinivasan P. N., and Shi J.. 2007. Balanced graph matching. Adv. Neural Info. Process. Syst. 19 (2007), 313.Google ScholarGoogle Scholar
  20. [20] Silva G. E. da and Backes A. R.. 2018. Characterizing 3D shapes: A complex network-based approach. In Proceedings of the European Signal Processing Conference (EUSIPCO’18). 608612.Google ScholarGoogle Scholar
  21. [21] Demirci M., Shokoufandeh A., and Dickinson S.. 2009. Skeletal shape abstraction from examples. IEEE Trans. Pattern Anal. Mach. Intell. 31, 5 (2009), 944952.Google ScholarGoogle ScholarDigital LibraryDigital Library
  22. [22] Dibra E., Melchior S., Balkis A., Wolf T., Oztireli C., and Gross M.. 2018. Monocular RGB hand pose inference from unsupervised refinable nets. In Proceedings of the the IEEE Conference on Computer Vision and Pattern Recognition Workshops. 10751085.Google ScholarGoogle ScholarCross RefCross Ref
  23. [23] Emmert-Streib F., Dehmer M., and Shi Y.. 2016. Fifty years of graph matching, network alignment and network comparison. Info. Sci. 346 (2016), 180197.Google ScholarGoogle ScholarDigital LibraryDigital Library
  24. [24] Felzenszwalb P. and Huttenlocher D.. 2004. Efficient graph-based image segmentation. J. Comput. Vision 59, 2 (2004), 167181.Google ScholarGoogle Scholar
  25. [25] Fiedler M.. 1975. A property of eigenvectors of nonnegative symmetric matrices and its application to graph theory. Czech. Math. J. 25, 4 (1975), 619633.Google ScholarGoogle ScholarCross RefCross Ref
  26. [26] Guo Y., Bennamoun M., Sohel F., Lu M., Wan J., and Kwok N. M.. 2016. A comprehensive performance evaluation of 3D local feature descriptors. Int. J. Comput. Vision 116, 1 (2016), 6689.Google ScholarGoogle ScholarDigital LibraryDigital Library
  27. [27] Han Z., Liu Z., Vong C., Liu Y., Bu S., Han J., and Chen C.. 2017. BoSCC: Bag of spatial context correlations for spatially enhanced 3D shape representation. IEEE Trans. Image Process. 26, 8 (2017), 37073720.Google ScholarGoogle ScholarDigital LibraryDigital Library
  28. [28] Horaud R. and Sossa H.. 1995. Polyhedral object recognition by indexing. Pattern Recogn. 28, 12 (1995), 18551870.Google ScholarGoogle ScholarDigital LibraryDigital Library
  29. [29] Jomma H. and Hussein A.. 2016. Circle views signature: A novel shape representation for shape recognition and retrieval. Can. J. Electric. Comput. Eng. 39, 4 (2016), 274282.Google ScholarGoogle ScholarCross RefCross Ref
  30. [30] Latecki L., Lakamper R., and Eckhardt T.. 2000. Shape descriptors for non-rigid shapes with a single closed contour. In Proceedings of the the Computer Vision and Pattern Recognition (CVPR’00), Vol. 1. IEEE, 424429.Google ScholarGoogle ScholarCross RefCross Ref
  31. [31] Lavoué Guillaume. 2012. Combination of bag-of-words descriptors for robust partial shape retrieval. Visual Computer 28, 9 (2012), 931942.Google ScholarGoogle ScholarDigital LibraryDigital Library
  32. [32] Leordeanu M. and Hebert M.. 2005. A spectral technique for correspondence problems using pairwise constraints. In Proceedings of the International Conference on Computer Vision (ICCV’05). 14821489.Google ScholarGoogle ScholarDigital LibraryDigital Library
  33. [33] Li S., Lu H., and Shao X.. 2014. Human body segmentation via data-driven graph cut. IEEE Trans. Cybernet. 44, 11 (2014), 20992108.Google ScholarGoogle ScholarCross RefCross Ref
  34. [34] Li Y., Bu R., Sun M., Wu W., Di X., and Chen B.. 2018. Pointcnn: Convolution on x-transformed points. In Advances in Neural Information Processing Systems. MIT Press, 820830.Google ScholarGoogle Scholar
  35. [35] Liu J., Song S., Liu C., Li Y., and Hu Y.. 2020. A benchmark dataset and comparison study for multi-modal human action analytics. ACM Trans. Multimedia Comput. Commun. Appl. 16, 2 (2020), 124.Google ScholarGoogle ScholarDigital LibraryDigital Library
  36. [36] Luo L., Shen C., Liu X., and Zhang C.. 2015. A computational model of the short-cut rule for 2D shape decomposition. IEEE Trans. Image Process. 24, 1 (2015), 273283.Google ScholarGoogle ScholarCross RefCross Ref
  37. [37] Luo X., Wong F., and Hu H.. 2020. FIN: Feature integrated network for object detection. ACM Trans. Multimedia Comput. Commun. Appl. 16, 2 (2020), 118.Google ScholarGoogle ScholarDigital LibraryDigital Library
  38. [38] Martinetz T. and Schulten K.. 1991. A neural-gas network learns topologies. In Proceedings of the International Conference on Artificial Neural Networks. 397–402.Google ScholarGoogle Scholar
  39. [39] Masoumi M. and Hamza A.. 2017. Spectral shape classification: A deep learning approach. J. Visual Commun. Image Represent. 43 (2017), 198–211.Google ScholarGoogle ScholarDigital LibraryDigital Library
  40. [40] Maturana D. and Scherer S.. 2015. Voxnet: A 3D convolutional neural network for real-time object recognition. In Proceedings of the international conference on Intelligent Robots and Systems (IROS’15). IEEE, 922928.Google ScholarGoogle ScholarDigital LibraryDigital Library
  41. [41] Minto A. Memo, L., and Zanuttigh P.. 2015. Exploiting silhouette descriptors and synthetic data for hand gesture recognition. In Proceedings of the Conference on Smart Tools & Apps for Graphics (STAG’15).Google ScholarGoogle Scholar
  42. [42] Murray S. O., Olshausen B. A., and Woods D. L.. 2003. Processing shape, motion and three-dimensional shape-from-motion in the human cortex. Cerebral Cortex 13, 5 (2003), 508516.Google ScholarGoogle ScholarCross RefCross Ref
  43. [43] Nanni L., Brahnam S., and Lumini A.. 2012. Local phase quantization descriptor for improving shape retrieval/classification. Pattern Recogn. Lett. 33, 16 (2012), 22542260.Google ScholarGoogle ScholarDigital LibraryDigital Library
  44. [44] Qi C. R., Yi L., Su H., and Guibas L. J.. 2017. Pointnet++: Deep hierarchical feature learning on point sets in a metric space. In Advances in Neural Information Processing Systems. MIT Press, 50995108.Google ScholarGoogle ScholarDigital LibraryDigital Library
  45. [45] Ren Z., Yuan J., Meng J., and Zhang Z.. 2013. Robust part-based hand gesture recognition using Kinect sensor. IEEE Trans. Multimedia 15, 5 (Aug.2013), 11101120. Google ScholarGoogle ScholarDigital LibraryDigital Library
  46. [46] Reppa I. and Leek E. C.. 2019. Surface diagnosticity predicts the high-level representation of regular and irregular object shape in human vision. Attent. Percept. Psychophys. 81 (2019), 15891608.Google ScholarGoogle ScholarCross RefCross Ref
  47. [47] Roberto P., Emanuele F., Primo Z., Adriano M., Jelena L., and Marina P.. 2019. Design, large-scale usage testing, and important metrics for augmented reality gaming applications. ACM Trans. Multimedia Comput. Commun. Appl. 15, 2 (2019), 118.Google ScholarGoogle ScholarDigital LibraryDigital Library
  48. [48] Rusu R. B., Blodow N., and Beetz M.. 2009. Fast point feature histograms (FPFH) for 3D registration. In Proceedings of the International Conference on Robotics and Automation. IEEE, 32123217.Google ScholarGoogle ScholarCross RefCross Ref
  49. [49] Rusu R. B., Blodow N., Marton Z. C., and Beetz M.. 2008. Aligning point cloud views using persistent feature histograms. In Proceedings of the the International Conference on Intelligent Robots and Systems. IEEE, 33843391.Google ScholarGoogle ScholarCross RefCross Ref
  50. [50] Sebastian T., Klein P., and Kimia B.. 2004. Recognition of shapes by editing their shock graphs. IEEE Trans. Pattern Anal. Mach. Intell. 26, 5 (2004), 550571.Google ScholarGoogle ScholarDigital LibraryDigital Library
  51. [51] Shamaie A. and Sutherland A.. 2001. Graph-based matching of occluded hand gestures. In Proceedings of the Applied Imagery Pattern Recognition Workshop. 6773. Google ScholarGoogle ScholarCross RefCross Ref
  52. [52] Shapiro S. and Brady J.. 1992. Feature-based correspondence: An eigenvector approach. Image Vision Comput. 10, 5 (1992), 283288.Google ScholarGoogle ScholarDigital LibraryDigital Library
  53. [53] Shuman D. I., Narang S. K., Frossard P., Ortega A., and Vandergheynst P.. 2013. The emerging field of signal processing on graphs: Extending high-dimensional data analysis to networks and other irregular domains. IEEE Signal Process. Mag. 30, 3 (May2013), 8398. Google ScholarGoogle ScholarCross RefCross Ref
  54. [54] Siddiqi K., Zhang J., Macrini D., Shokoufandeh A., Bouix S., and Dickinson S.. 2008. Retrieving articulated 3D models using medial surfaces. Mach. Vision Appl. 19, 4 (2008), 261275.Google ScholarGoogle ScholarDigital LibraryDigital Library
  55. [55] Sinha A., Bai J., and Ramani K.. 2016. Deep learning 3D shape surfaces using geometry images. In Proceedings of the European Conference on Computer Vision. Springer, 223240.Google ScholarGoogle ScholarCross RefCross Ref
  56. [56] Su H., Maji S., Kalogerakis E., and Miller E.. 2015. Multi-view convolutional neural networks for 3D shape recognition. In Proceedings of the International Conference on Computer Vision. 945953.Google ScholarGoogle ScholarDigital LibraryDigital Library
  57. [57] Taati B. and Greenspan Mi. 2011. Local shape descriptor selection for object recognition in range data. Comput. Vision Image Understand. 115, 5 (2011), 681694.Google ScholarGoogle ScholarDigital LibraryDigital Library
  58. [58] Tan Y. S., Lim K. M., Tee C., Lee C. P., and Low C. Y.. 2020. Convolutional neural network with spatial pyramid pooling for hand gesture recognition. Neural Comput. Appl. 33 (2021), 53395351.Google ScholarGoogle Scholar
  59. [59] Umeyama S.. 1988. An eigendecomposition approach to weighted graph matching problems. IEEE Trans. Pattern Anal. Mach. Intell. 10, 5 (Sept.1988), 695703. Google ScholarGoogle ScholarDigital LibraryDigital Library
  60. [60] Vranic D. and Saupe D.. 2002. Description of 3D-shape using a complex function on the sphere. In Proceedings of the International Conference on Multimedia and Expo, Vol. 1. IEEE, 177180.Google ScholarGoogle ScholarCross RefCross Ref
  61. [61] Wang B., Shen W., Liu W., and Bai X.. 2010. Shape classification using tree-unions. In Proceedings of the International Conference on of Pattern Recognition(ICPR’10). 983986.Google ScholarGoogle ScholarDigital LibraryDigital Library
  62. [62] Wang C., Liu Z., and Chan S. C.. 2015. Superpixel-based hand gesture recognition with Kinect depth camera. IEEE Trans. Multimedia 17, 1 (2015), 2939.Google ScholarGoogle ScholarCross RefCross Ref
  63. [63] Wang X., Tian Y., Zhao X., Yang T., Gelernter J., Wang J., Cheng G., and Hu W.. 2020. Improving multiperson pose estimation by mask-aware deep reinforcement learning. ACM Trans. Multimedia Comput. Commun. Appl. 16, 3 (2020), 118.Google ScholarGoogle ScholarDigital LibraryDigital Library
  64. [64] Watt R. and Andrews D.. 1982. Contour curvature analysis: Hyperacuities in the discrimination of detailed shape. Vision Res. 22, 4 (1982), 449460.Google ScholarGoogle ScholarCross RefCross Ref
  65. [65] Wu Z., Song S., Khosla A., Yu F., Zhang L., Tang X., and Xiao J.. 2015. 3D shapenets: A deep representation for volumetric shapes. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR’15). 19121920.Google ScholarGoogle Scholar
  66. [66] Xie J., Fang Y., Zhu F., and Wong E.. 2015. Deepshape: Deep learned shape descriptor for 3D shape matching and retrieval. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR’15). 12751283.Google ScholarGoogle Scholar
  67. [67] Yasseen Z., Verroust A., and Nasri A.. 2016. Shape matching by part alignment using extended chordal axis transform. Pattern Recogn. 57 (2016), 115135.Google ScholarGoogle ScholarDigital LibraryDigital Library
  68. [68] Zahn C. and Roskies R.. 1972. Fourier descriptors for plane closed curves. IEEE Trans. Comput. 100, 3 (1972), 269281.Google ScholarGoogle ScholarDigital LibraryDigital Library
  69. [69] Zheng Y., Guo B., Yan Y., and He W.. 2019. O2O method for fast 2D shape retrieval. IEEE Trans. Image Process. 28, 11 (2019), 53665378.Google ScholarGoogle ScholarDigital LibraryDigital Library
  70. [70] Zhi S., Liu Y., Li X., and Guo Y.. 2018. Toward real-time 3D object recognition: A lightweight volumetric CNN framework using multitask learning. Comput. Graph. 71 (2018), 199207.Google ScholarGoogle ScholarCross RefCross Ref
  71. [71] Zhou F. and Torre F. De La. 2015. Factorized graph matching. IEEE Trans. Pattern Anal. Mach. Intell. 38, 9 (2015), 17741789. Google ScholarGoogle ScholarDigital LibraryDigital Library
  72. [72] Zhou Y. and Zeng F.. 2017. 2D compressive sensing and multi-feature fusion for effective 3D shape retrieval. Info. Sci. 409 (2017), 101120.Google ScholarGoogle ScholarCross RefCross Ref

Index Terms

  1. GHOSM: Graph-based Hybrid Outline and Skeleton Modelling for Shape Recognition

          Recommendations

          Comments

          Login options

          Check if you have access through your login credentials or your institution to get full access on this article.

          Sign in

          Full Access

          • Published in

            cover image ACM Transactions on Multimedia Computing, Communications, and Applications
            ACM Transactions on Multimedia Computing, Communications, and Applications  Volume 19, Issue 2s
            April 2023
            545 pages
            ISSN:1551-6857
            EISSN:1551-6865
            DOI:10.1145/3572861
            • Editor:
            • Abdulmotaleb El Saddik
            Issue’s Table of Contents

            Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

            Publisher

            Association for Computing Machinery

            New York, NY, United States

            Publication History

            • Published: 17 February 2023
            • Online AM: 4 August 2022
            • Accepted: 27 July 2022
            • Revised: 25 April 2022
            • Received: 4 February 2021
            Published in tomm Volume 19, Issue 2s

            Permissions

            Request permissions about this article.

            Request Permissions

            Check for updates

            Qualifiers

            • research-article
          • Article Metrics

            • Downloads (Last 12 months)105
            • Downloads (Last 6 weeks)6

            Other Metrics

          PDF Format

          View or Download as a PDF file.

          PDF

          eReader

          View online with eReader.

          eReader

          Full Text

          View this article in Full Text.

          View Full Text

          HTML Format

          View this article in HTML Format .

          View HTML Format
          About Cookies On This Site

          We use cookies to ensure that we give you the best experience on our website.

          Learn more

          Got it!