Abstract
Objective object segmentation visual quality evaluation is an emergent member of the visual quality assessment family. It aims to develop an objective measure instead of a subjective survey to evaluate the object segmentation quality in agreement with human visual perception. It is an important benchmark for assessing and comparing the performances of object segmentation methods in terms of visual quality. Despite its essential role, sufficient study compared with other visual quality evaluation studies is still lacking. In this article, we propose a novel full-reference objective measure that includes a two-level single object segmentation visual quality measure and a pooling method for multiple object segmentation overall visual quality. The single object segmentation visual quality measure combines a pixel-level sub-measure and a region-level sub-measure for evaluating the similarity of area, shape, and object completeness between the segmentation result and the ground truth in terms of human visual perception. For the proposed multiple object segmentation overall visual quality pooling method, the rank of each object’s segmentation quality as a novel factor is integrated into the weighted harmonic mean to evaluate the overall quality. To evaluate the performance of our proposed measure, we tested it on an object segmentation subjective visual quality assessment database. The experimental results demonstrate that our proposed two-level measure and pooling method with good robustness perform better in matching subjective assessments compared with other state-of-the-art objective measures.
- [1] . 2007. Seam carving for content-aware image resizing. In ACM SIGGRAPH 2007 Papers. 10–es.Google Scholar
- [2] . 2016. Regions matching algorithms analysis to quantify the image segmentation results. In Proceedings of the 2016 11th International Scientific and Technical Conference on Computer Sciences and Information Technologies (CSIT’16). IEEE, Los Alamitos, CA, 33–36.Google Scholar
Cross Ref
- [3] . 2019. Visual quality evaluation for semantic segmentation: Subjective assessment database and objective assessment measure. IEEE Transactions on Image Processing 28, 12 (2019), 5785–5796.Google Scholar
Cross Ref
- [4] . 2015. Global contrast based salient region detection. IEEE Transactions on Pattern Analysis and Machine Intelligence 37, 3 (2015), 569–582.Google Scholar
Digital Library
- [5] . 2013. What is a good evaluation measure for semantic segmentation? In Proceedings of the British Machine Vision Conference, Vol. 27.Google Scholar
Cross Ref
- [6] . 2012. Evaluation of segmentation techniques using region area and boundary matching information. Journal of Visual Communication and Image Representation 23, 1 (2012), 150–160.Google Scholar
Digital Library
- [7] . 2017. Structure-measure: A new way to evaluate foreground maps. In Proceedings of the IEEE International Conference on Computer Vision.Google Scholar
Cross Ref
- [8] . 2018. Enhanced-alignment measure for binary foreground map evaluation. In Proceedings of the International Joint Conferences on Artificial Intelligence.Google Scholar
Cross Ref
- [9] . 2019. Attentive feedback network for boundary-aware salient object detection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 1623–1632.Google Scholar
Cross Ref
- [10] . 2021. Subjective and objective quality assessment for stereoscopic image retargeting. IEEE Transactions on Multimedia 23 (2021), 2100–2113.Google Scholar
Cross Ref
- [11] . 2007. New benchmark for image segmentation evaluation. Journal of Electronic Imaging 16, 3 (2007), 033011.Google Scholar
Cross Ref
- [12] . 2005. Full-Reference Objective Quality Metrics for Video Watermarking, Video Segmentation and 3D Model Watermarking. Ph.D. Dissertation. Lausanne, EPFL.Google Scholar
- [13] . 2009. On evaluating video object segmentation quality: A perceptually driven objective metric. IEEE Journal of Selected Topics in Signal Processing 3, 2 (2009), 319–335.Google Scholar
Cross Ref
- [14] . 2013. Fast saliency-aware multi-modality image fusion. Neurocomputing 111 (2013), 70–80.Google Scholar
Digital Library
- [15] . 2016. QualityNet: Segmentation quality evaluation with deep convolutional networks. In Proceedings of the 2016 Conference on Visual Communications and Image Processing (VCIP’16). IEEE, Los Alamitos, CA, 1–4.Google Scholar
Cross Ref
- [16] . 2021. Densely deformable efficient salient object detection network. Arxiv Preprint arXiv:2102.06407 (2021).Google Scholar
- [17] . 2009. Robust higher order potentials for enforcing label consistency. International Journal of Computer Vision 82, 3 (2009), 302–324.Google Scholar
Digital Library
- [18] . 2014. The secrets of salient object segmentation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 280–287.Google Scholar
Digital Library
- [19] . 2019. A simple pooling-based design for real-time salient object detection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 3917–3926.Google Scholar
Cross Ref
- [20] . 2014. How to evaluate foreground maps? In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 248–255.Google Scholar
Digital Library
- [21] . 2010. A comparative evaluation of interactive segmentation algorithms. Pattern Recognition 43, 2 (2010), 434–444.Google Scholar
Digital Library
- [22] . 2010. Design and perceptual validation of performance measures for salient object segmentation. In Proceedings of the Computer Vision and Pattern Recognition Workshops (CVPRW’10). 49–56.Google Scholar
Cross Ref
- [23] . 2018. Region-based image segmentation evaluation via perceptual pooling strategies. Machine Vision and Applications 29, 3 (2018), 477–488.Google Scholar
Digital Library
- [24] . 2015. Supervised evaluation of image segmentation and object proposal techniques. IEEE Transactions on Pattern Analysis and Machine Intelligence 38, 7 (2015), 1465–1478.Google Scholar
Cross Ref
- [25] . 2011. Evaluation: From precision, recall and F-measure to ROC, informedness, markedness and correlation. Journal of Machine Learning Technologies 2, 1 (2011), 37–63.Google Scholar
Cross Ref
- [26] . 2019. BASNet: Boundary-aware salient object detection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 7479–7489.Google Scholar
Cross Ref
- [27] . 2018. A perceptually weighted rank correlation indicator for objective image quality assessment. IEEE Transactions on Image Processing 27, 5 (2018), 2499–2513.Google Scholar
- [28] . 2020. Objective object segmentation visual quality evaluation based on pixel-level and region-level characteristics. In Proceedings of the 2nd ACM International Conference on Multimedia in Asia (MMAsia’20).Google Scholar
- [29] . 2004. GrabCut: Interactive foreground extraction using iterated graph cuts. ACM Transactions on Graphics 23 (2004), 309–314.Google Scholar
- [30] . 2015. Visual quality evaluation of image object segmentation: Subjective assessment and objective measure. IEEE Transactions on Image Processing 24, 12 (2015), 5033–5045.Google Scholar
Digital Library
- [31] . 2017. Segmentation quality evaluation based on multi-scale convolutional neural networks. In Proceedings of the 2017 IEEE Conference on Visual Communications and Image Processing (VCIP’17). IEEE, Los Alamitos, CA, 1–4.Google Scholar
Cross Ref
- [32] . 2020. MaD-DLS: Mean and deviation of deep and local similarity for image quality assessment. IEEE Transactions on Multimedia 23 (2020), 4037–4048.Google Scholar
- [33] . 2020. F-BRS: Rethinking backpropagating refinement for interactive segmentation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR’20).Google Scholar
Cross Ref
- [34] . 2004. Perceptually-weighted evaluation criteria for segmentation masks in video sequences. IEEE Transactions on Image Processing 13, 8 (2004), 1092–1103.Google Scholar
Digital Library
- [35] Final Report From the Video Quality Experts Group on the Validation of Objective Models of Video Quality Assessment II, Video Quality Expert Group (VQEG), 2003. [Online]. Available http://www.vqeg.org/.Google Scholar
- [36] . 2021. Salient object detection in the deep learning era: An in-depth survey. IEEE Transactions on Pattern Analysis and Machine Intelligence. Early access, January 12, 2021.Google Scholar
- [37] . 2019. An iterative and cooperative top-down and bottom-up inference network for salient object detection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 5968–5977.Google Scholar
Cross Ref
- [38] . 2004. Image quality assessment: From error visibility to structural similarity. IEEE Transactions on Image Processing 13, 4 (2004), 600–612.Google Scholar
Digital Library
- [39] . 2001. The psychometric function: I. Fitting, sampling, and goodness of fit. Perception & Psychophysics 63, 8 (2001), 1293–1313.Google Scholar
Cross Ref
- [40] . 2020. Subjective and objective de-raining quality assessment towards authentic rain image. IEEE Transactions on Circuits and Systems for Video Technology 30, 11 (2020), 3883–3897.Google Scholar
Cross Ref
- [41] . 2019. A mutual learning method for salient object detection with intertwined multi-supervision. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 8150–8159.Google Scholar
Cross Ref
- [42] . 2019. Cascaded partial decoder for fast and accurate salient object detection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 3907–3916.Google Scholar
Cross Ref
- [43] . 2019. Multi-source weak supervision for saliency detection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 6074–6083.Google Scholar
Cross Ref
- [44] . 2019. Synthesizing supervision for learning deep saliency network without human annotation. IEEE Transactions on Pattern Analysis and Machine Intelligence 42, 7 (2019), 1755–1769.Google Scholar
Cross Ref
- [45] . 2019. Capsal: Leveraging captioning to boost semantics for salient object detection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 6024–6033.Google Scholar
Cross Ref
- [46] . 2017. Fuzzy evaluations of image segmentations. IEEE Transactions on Fuzzy Systems 26, 4 (2017), 1789–1799.Google Scholar
Cross Ref
Index Terms
Objective Object Segmentation Visual Quality Evaluation: Quality Measure and Pooling Method
Recommendations
Objective object segmentation visual quality evaluation based on pixel-level and region-level characteristics
MMAsia '20: Proceedings of the 2nd ACM International Conference on Multimedia in AsiaObjective object segmentation visual quality evaluation is an emergent member of the visual quality assessment family. It aims at developing an objective measure instead of a subjective survey to evaluate the object segmentation quality in agreement ...
Visual Quality Evaluation of Image Object Segmentation: Subjective Assessment and Objective Measure
A visual quality evaluation of image object segmentation as one member of the visual quality evaluation family has been studied over the years. Researchers aim at developing the objective measures that can evaluate the visual quality of object ...
The Objective Evaluation of Image Object Segmentation Quality
ACIVS 2013: 15th International Conference on Advanced Concepts for Intelligent Vision Systems - Volume 8192In this paper, a novel objective quality metric is proposed for individual object segmentation in images. We analyze four types of segmentation errors, and verify experimentally that besides quantity, area and contour, the distortion of object content ...






Comments