Abstract
Full-reference (FR) point cloud quality assessment (PCQA) has achieved impressive progress in recent years. However, in many cases, obtaining the reference point clouds is difficult, so no-reference (NR) metrics have become a research hotspot. Few researches about NR-PCQA are carried out due to the lack of a large-scale PCQA dataset. In this article, we first build a large-scale PCQA dataset named LS-PCQA, which includes 104 reference point clouds and more than 22,000 distorted samples. In the dataset, each reference point cloud is augmented with 31 types of impairments (e.g., Gaussian noise, contrast distortion, local missing, and compression loss) at 7 distortion levels. Besides, each distorted point cloud is assigned with a pseudo-quality score as its substitute of Mean Opinion Score. Inspired by the hierarchical perception system and considering the intrinsic attributes of point clouds, we propose a NR metric ResSCNN based on sparse convolutional neural network (CNN) to accurately estimate the subjective quality of point clouds. We conduct several experiments to evaluate the performance of the proposed NR metric. The results demonstrate that ResSCNN exhibits the state-of-the-art performance among all the existing NR-PCQA metrics and even outperforms some FR metrics. The dataset presented in this work will be made publicly accessible at https://smt.sjtu.edu.cn. The source code for the proposed ResSCNN can be found at https://github.com/lyp22/ResSCNN.
- [1] Free3d. [n.d.]. Retrieved from https://free3d.com/3d-models.Google Scholar
- [2] Image and Video Quality Assessment Research at LIVE. [n.d.]. Retrieved from http://live.ece.utexas.edu/research/quality/.Google Scholar
- [3] JPEG Pleno Database. [n.d.]. Retrieved from http://uspaulopc.di.ubi.pt/.Google Scholar
- [4] MPEG Inanimate DataSets. [n.d.]. Retrieved from http://mpegfs.int-evry.fr/MPEG/PCC/DataSets/pointCloud/CfP/datasets/Static_Objects_and_Scenes/Inanimate_Objects/.Google Scholar
- [5] MPEG People DataSets. [n.d.]. Retrieved from http://mpegfs.int-evry.fr/MPEG/PCC/DataSets/pointCloud/CfP/decoded/Dynamic_Objects/People/8i/.Google Scholar
- [6] MPEG Reference Software. [n.d.]. Retrieved from http://mpegx.int-evry.fr/software/MPEG/PCC/TM/mpeg-pcc-dmetric.Google Scholar
- [7] MPEG Static Object and Scenes DataSets. [n.d.]. Retrieved from http://mpegfs.int-evry.fr/MPEG/PCC/DataSets/pointCloud/CfP/datasets/Static_Objects_and_Scenes/ULB_Unicorn/.Google Scholar
- [8] Sketchfab. [n.d.]. Retrieved from https://sketchfab.com.Google Scholar
- [9] . 2018. Point cloud quality assessment metric based on angular similarity. In Proceedings of the IEEE International Conference on Multimedia and Expo. 1–6.Google Scholar
Cross Ref
- [10] . 2020. Towards a point cloud structural similarity metric. In Proceedings of the IEEE International Conference on Multimedia and Expo Workshops. 1–3.Google Scholar
Cross Ref
- [11] . 2020. PointXR: A toolbox for visualization and subjective evaluation of point clouds in virtual reality. In Proceedings of the International Conference on Quality of Multimedia Experience. 1–6.Google Scholar
Cross Ref
- [12] . 2020. Multi-hierarchical independent correlation filters for visual tracking. In Proceedings of the IEEE International Conference on Multimedia and Expo. 1–6.Google Scholar
Cross Ref
- [13] . 2015. A review paper: Noise models in digital image processing. Signal Image Process.: Int. J. 6, 2 (2015).Google Scholar
- [14] . 2017. PointNet: Deep learning on point sets for 3D classification and segmentation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 77–85.Google Scholar
Cross Ref
- [15] . 2019. 4D spatio-temporal convnets: Minkowski convolutional neural networks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 3070–3079.Google Scholar
Cross Ref
- [16] . 1998. Metro: Measuring error on simplified surfaces. Comput. Graph. Forum 17, 2 (1998), 167–174.Google Scholar
Cross Ref
- [17] . 2018. Deformation-based 3D facial expression representation. ACM Trans. Multimedia Comput. Commun. Appl. 14, 1s (2018).Google Scholar
Digital Library
- [18] . 2018. PPF-FoldNet: Unsupervised learning of rotation invariant 3D local descriptors. In Proceedings of the European Conference on Computer Vision. 620–638.Google Scholar
Digital Library
- [19] . 2018. PPFNet: Global context aware local features for robust 3D point matching. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 195–205.Google Scholar
Cross Ref
- [20] . 2018. Perceptual quality assessment of omnidirectional images. In Proceedings of the IEEE International Symposium on Circuits and Systems. 1–5.Google Scholar
Cross Ref
- [21] . 2016. Biologically inspired image quality assessment. Signal Process. 124 (2016), 210–219.Google Scholar
Digital Library
- [22] . 2018. 3D semantic segmentation with submanifold sparse convolutional networks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 9224–9232.Google Scholar
Cross Ref
- [23] . 2017. Submanifold sparse convolutional networks. Retrieved from https://arXiv:1706.01307.Google Scholar
- [24] . 2016. A comprehensive performance evaluation of 3d local feature descriptors. In International Journal of Computer Vision, Vol. 116. 66–89.Google Scholar
- [25] . 2012. Methodology for the subjective assessment of the quality of television pictures. ITU-R Recommendation BT.500-13. https://www.itu.int/rec/R-REC-BT.500-14-201910-I.Google Scholar
- [26] . 2012. Methods, metrics and procedures for statistical evaluation, qualification and comparison of objective quality prediction models. ITU-T Recommendation, 1401. https://www.itu.int/rec/T-REC-P.1401-202001-I.Google Scholar
- [27] . 2021. A point-to-distribution joint geometry and color metric for point cloud quality assessment. In Proceedings of the IEEE International Workshop on Multimedia Signal Processing. 1–6.Google Scholar
Cross Ref
- [28] . 2020. A generalized Hausdorff distance based quality metric for point cloud geometry. In Proceedings of the International Conference on Quality of Multimedia Experience. 1–6.Google Scholar
Cross Ref
- [29] . 2021. Joint geometry and color projection-based point cloud quality metric. Retrieved from https://arXiv:2108.02481.Google Scholar
- [30] . 2021. Point cloud rendering after coding: Impacts on subjective and objective quality. IEEE Trans. Multimedia 23 (2021), 4049–4064.Google Scholar
Cross Ref
- [31] . 2020. Blind image quality measurement by exploiting high-order statistics with deep dictionary encoding network. IEEE Trans. Instrument. Measure. 69, 10 (2020), 7398–7410.Google Scholar
Cross Ref
- [32] . 2018. Optimizing multistage discriminative dictionaries for blind image quality assessment. IEEE Trans. Multimedia 20, 8 (2018), 2035–2048.Google Scholar
Cross Ref
- [33] . 2020. A full-reference stereoscopic image quality measurement via hierarchical deep feature degradation fusion. IEEE Trans. Instrument. Measure. 69, 12 (2020), 9784–9796.Google Scholar
Cross Ref
- [34] . 1999. Using spin images for efficient object recognition in cluttered 3D scenes. IEEE Trans. Pattern Anal. Mach. Intell. 21, 5 (1999), 433–449.Google Scholar
Digital Library
- [35] . 2006. Poisson surface reconstruction. In Proceedings of the 4th Eurographics Symposium on Geometry Processing. 61–70.Google Scholar
Digital Library
- [36] . 2017. Learning compact geometric features. In Proceedings of the IEEE International Conference on Computer Vision. 153–161.Google Scholar
Cross Ref
- [37] . 2022. Perceptual quality assessment of colored 3D point clouds. IEEE Trans. Visual. Comput. Graph. (2022), 1–1.Google Scholar
- [38] . 2021. Reduced reference perceptual quality model with application to rate control for video-based point cloud compression. IEEE Trans. Image Process. 30 (2021), 6623–6636.Google Scholar
Digital Library
- [39] . 2021. PQA-Net: Deep no reference point cloud quality assessment via multi-view projection. IEEE Trans. Circ. Syst. Video Technol. 31, 12 (2021), 4645–4660.Google Scholar
Cross Ref
- [40] . 2017. Waterloo exploration database: New challenges for image quality assessment models. IEEE Trans. Image Process. 26, 2 (2017), 1004–1016.Google Scholar
Digital Library
- [41] . 2016. Evaluation criteria for point cloud compression. ISO/IEC MPEG w16332, Geneva, Switzerland. https://dms.mpeg.expert/doc_end_user/current_document.php?id=55875.Google Scholar
- [42] . 2019. PC-MSDM: A quality metric for 3D point clouds. In Proceedings of the International Conference on Quality of Multimedia Experience. 1–3.Google Scholar
Cross Ref
- [43] . 2020. PCQM: A full-reference quality metric for colored 3D point clouds. In Proceedings of the International Conference on Quality of Multimedia Experience. 1–6.Google Scholar
Cross Ref
- [44] . 2021. Screen content quality assessment: Overview, benchmark, and beyond. ACM Comput. Surv. 54, 9 (2021), 1–36.Google Scholar
Digital Library
- [45] . 2017. Unified blind quality assessment of compressed natural, graphic, and screen content images. IEEE Trans. Image Process. 26, 11 (2017), 5462–5474.Google Scholar
Digital Library
- [46] . 2019. Objective quality evaluation of dehazed images. IEEE Trans. Intell. Transport. Syst. 20, 8 (2019), 2879–2892.Google Scholar
Cross Ref
- [47] . 2019. Quality evaluation of image dehazing methods using synthetic hazy images. IEEE Trans. Multimedia 21, 9 (2019), 2319–2333.Google Scholar
Cross Ref
- [48] . 2020. Study of subjective and objective quality assessment of audio-visual signals. IEEE Trans. Image Process. 29 (2020), 6054–6068.Google Scholar
Cross Ref
- [49] . 2020. A metric for light field reconstruction, compression, and display quality evaluation. IEEE Trans. Image Process. 29 (2020), 3790–3804.Google Scholar
Digital Library
- [50] . 2017. Greyc 3D colored mesh database. Technical Report, Normandie Université.Google Scholar
- [51] . 2015. Image database TID2013: Peculiarities, results and perspectives. Signal Process. Image Commun. 30 (2015), 57–77.Google Scholar
Digital Library
- [52] . 2009. Tid2008-a database for evaluation of full-reference visual quality assessment metrics. Adv. Modern Radioelectr. 10, 4 (2009), 30–45.Google Scholar
- [53] . 2009. Fast point feature histograms (FPFH) for 3D registration. In Proceedings of the IEEE International Conference on Robotics and Automation. 3212–3217.Google Scholar
Cross Ref
- [54] . 2008. Aligning point cloud views using persistent feature histograms. In Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems. 3384–3391.Google Scholar
- [55] . 2014. SHOT: Unique signatures of histograms for surface and texture description. Comput. Vision Image Understand. 125 (2014), 251–264.Google Scholar
Cross Ref
- [56] . 2006. Image information and visual quality. IEEE Trans. Image Process. 15, 2 (2006), 430–444.Google Scholar
Digital Library
- [57] . 2019. Perceptual quality assessment of 3D point clouds. In Proceedings of the IEEE International Conference on Image Processing. 3182–3186.Google Scholar
Cross Ref
- [58] . 2019. Expression robust 3D facial landmarking via progressive coarse-to-fine tuning. ACM Trans. Multimedia Comput. Commun. Appl. 15, 1 (2019), 1–23.Google Scholar
Digital Library
- [59] . 2020. MC360IQA: A multi-channel CNN for blind 360-degree image quality assessment. IEEE J. Select. Top. Signal Process. 14, 1, 64–77.Google Scholar
Cross Ref
- [60] . 2021. Point cloud projection and multi-scale feature fusion network based blind quality assessment for colored point clouds. In Proceedings of the ACM International Conference on Multimedia. 5266–5272.Google Scholar
Digital Library
- [61] . 2017. Evaluation metrics for point cloud compression. ISO/IEC JTC m74008, Geneva, Switzerland (2017).Google Scholar
- [62] . 2010. Unique shape context for 3d data description. In Proceedings of the ACM Workshop on 3D Object Retrieval. 57–62.Google Scholar
Digital Library
- [63] . 2018. A novel methodology for quality assessment of voxelized point clouds. Appl. Dig. Image Process. XLI 10752 (2018), 174–190.Google Scholar
- [64] . 2000. Final report from the video quality experts group on the validation of objective models of video quality assessment. In Proceedings of the VQEG Meeting.Google Scholar
- [65] . 2010. Report on the validation of video quality models for high definition video content. In Proceedings of the VQEG Meeting.Google Scholar
- [66] . 2020. A reduced reference metric for visual quality evaluation of point cloud contents. IEEE Signal Process. Lett. 27 (2020), 1660–1664.Google Scholar
Cross Ref
- [67] . 2020. A color-based objective quality metric for point cloud contents. In Proceedings of the International Conference on Quality of Multimedia Experience. 1–6.Google Scholar
Cross Ref
- [68] . 2004. Image quality assessment: From error visibility to structural similarity. IEEE Trans. Image Process. 13, 4 (2004), 600–612.Google Scholar
Digital Library
- [69] . 2020. End-to-end blind image quality prediction with cascaded deep neural network. IEEE Trans. Image Process. 29 (2020), 7414–7426.Google Scholar
Digital Library
- [70] . 2014. Gradient magnitude similarity deviation: A highly efficient perceptual image quality index. IEEE Trans. Image Process. 23, 2 (2014), 684–695.Google Scholar
Digital Library
- [71] . 2021. Predicting the perceptual quality of point cloud: A 3D-to-2D projection-based exploration. IEEE Trans. Multimedia 23 (2021), 3877–3891.Google Scholar
Digital Library
- [72] . 2021. Point cloud distortion quantification based on potential energy for human and machine perception. Retrieved from https://arXiv:2103.02850.Google Scholar
- [73] . 2022. No-reference point cloud quality assessment via domain adaptation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 1–1.Google Scholar
Cross Ref
- [74] . 2022. Inferring point cloud quality via graph similarity. IEEE Trans. Pattern Anal. Mach. Intell. 44, 6 (2022), 3015–3029.Google Scholar
Cross Ref
- [75] . 2017. 3DMatch: Learning local geometric descriptors from RGB-D reconstructions. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 199–208.Google Scholar
Cross Ref
- [76] . 2020. Perceptual image quality assessment: A survey. Sci. China Info. Sci. 63, 11 (2020), 211301.Google Scholar
Cross Ref
- [77] . 2021. MS-GraphSIM: Inferring point cloud quality via multiscale graph similarity. In Proceedings of the ACM International Conference on Multimedia. 1230–1238.Google Scholar
Digital Library
- [78] . 2021. No-reference quality assessment for 3D colored point cloud and mesh Models. Retrieved from https://arXiv:2107.02041.Google Scholar
- [79] . 2021. A no-reference visual quality metric for 3D color meshes. In Proceedings of the IEEE International Conference on Multimedia and Expo Workshops. 1–6.Google Scholar
Cross Ref
Index Terms
Point Cloud Quality Assessment: Dataset Construction and Learning-based No-reference Metric
Recommendations
Point Cloud Quality Assessment Using Cross-correlation of Deep Features
QoEVMA '22: Proceedings of the 2nd Workshop on Quality of Experience in Visual Multimedia Applications3D point clouds have emerged as a preferred format for recent immersive communication systems, due to the six degrees of freedom they offer. The huge data size of point clouds, which consists of both geometry and color information, has motivated the ...
Comparative Evaluation of Temporal Pooling Methods for No-Reference Quality Assessment of Dynamic Point Clouds
PIES-ME '22: Proceedings of the 1st Workshop on Photorealistic Image and Environment Synthesis for Multimedia ExperimentsPoint Cloud Quality Assessment (PCQA) has become an important task in immersive multimedia since it is fundamental for improving computer graphics applications and ensuring the best Quality of Experience(QoE) for the end user. In recent years, the field ...
Textured Mesh Quality Assessment: Large-scale Dataset and Deep Learning-based Quality Metric
Over the past decade, three-dimensional (3D) graphics have become highly detailed to mimic the real world, exploding their size and complexity. Certain applications and device constraints necessitate their simplification and/or lossy compression, which ...






Comments