Abstract
Image quality measures are becoming increasingly important in the field of computer graphics. For example, there is currently a major focus on generating photorealistic images in real time by combining path tracing with denoising, for which such quality assessment is integral. We present FLIP, which is a difference evaluator with a particular focus on the differences between rendered images and corresponding ground truths. Our algorithm produces a map that approximates the difference perceived by humans when alternating between two images. FLIP is a combination of modified existing building blocks, and the net result is surprisingly powerful. We have compared our work against a wide range of existing image difference algorithms and we have visually inspected over a thousand image pairs that were either retrieved from image databases or generated in-house. We also present results of a user study which indicate that our method performs substantially better, on average, than the other algorithms. To facilitate the use of FLIP, we provide source code in C++, MATLAB, NumPy/SciPy, and PyTorch.
Supplemental Material
Available for Download
Supplemental movie, appendix, image and software files for, FLIP: A Difference Evaluator for Alternating Images
- Saeedeh Abasi, Mohammad Amani Tehran, and Mark D. Fairchild. 2020. Distance Metrics for Very Large Color Differences. Color Research and Application 45, 2 (2020), 208--223.Google Scholar
Cross Ref
- Jyrki Alakuijala, Robert Obryk, Ostap Stoliarchuk, Zoltan Szabadka, Lode Vandevenne, and Jan Wassenberg. 2017. Guetzli: Perceptually Guided JPEG Encoder. CoRR abs/1703.04421 (2017). arXiv:1703.04421 http://arxiv.org/abs/1703.04421Google Scholar
- Steve Bako, Thijs Vogels, Brian McWilliams, Mark Meyer, Jan Novák, Alex Harvill, Pradeep Sen, Tony DeRose, and Fabrice Rousselle. 2017. Kernel-Predicting Convolutional Networks for Denoising Monte Carlo Renderings. ACM Transactions on Graphics 36, 4 (2017), 97:1--97:14.Google Scholar
Digital Library
- Benedikt Bitterli. 2016. Rendering Resources. https://benedikt-bitterli.me/resources/.Google Scholar
- Martin Čadík, Robert Herzog, Rafał Mantiuk, Karol Myszkowski, and Hans-Peter Seidel. 2012. New Measurements Reveal Weaknesses of Image Quality Metrics in Evaluating Graphics Artifacts. ACM Transactions of Graphics 31, 6 (2012), 147:1--147:10.Google Scholar
Digital Library
- Damon M Chandler. 2013. Seven Challenges in Image Quality Assessment: Past, Present, and Future Research. ISRN Signal Processing 2013 (2013), 1--53.Google Scholar
Cross Ref
- Comission Internationale de l'Eclairage CIE. 1978. Recommendations on Uniform Color Spaces, Color-Difference Equations, Psychometric Color Terms. Supplement No.2 to CIE publication No.15 (E.-1.3.1) 1971/(TC-1.3.) (1978).Google Scholar
- Oliver Deussen, Pat Hanrahan, Bernd Lintermann, Radomír Měch, Matt Pharr, and Przemyslaw Prusinkiewicz. 1998. Realistic Modeling and Rendering of Plant Ecosystems. In Proceedings of SIGGRAPH. 275--286.Google Scholar
Digital Library
- Russell L. DeValois and Karen K. DeValois. 1990. Spatial Vision. Oxford University Press.Google Scholar
- Mark D Fairchild. 1996. Refinement of the RLAB Color Space. Color Research & Application 21, 5 (1996), 338--346.Google Scholar
Cross Ref
- Mark D. Fairchild. 2013. Color Appearance Models (3rd ed.). John Wiley & Sons.Google Scholar
- Mark D. Fairchild and Garrett M. Johnson. 2004. The iCAM Framework for Image Appearance, Image Differences, and Image Quality. Journal of Electronic Imaging 13, 1 (2004), 126--138.Google Scholar
Cross Ref
- Thomas J Flohr, Bernd W Kolpatzik, Raja Balasubramanian, David A Carrara, Charles A Bouman, and Jan P Allebach. 1993. Model-Based Color Image Quantization. In Human Vision, Visual Processing, and Digital Display IV, Vol. 1913. 270--281.Google Scholar
- Robert Herzog, Martin Čadík, Tunc. O. Aydın, Kwawng In Kim, Karol Myszkowski, and Hans-Peter Seidel. 2012. NoRM: No-Reference Image Quality Metric for Realistic Image Synthesis. Computer Graphics Forum 31, 2 (2012), 545--554.Google Scholar
Digital Library
- Min Huang, Guihua Cui, Manuel Melgosa, Manuel Sánchez-Mara nón, Changjun Li, M. Ronnier Luo, and Haoxue Liu. 2015. Power Functions Improving the Performance of Color-Difference Formulas. Optics Express 23, 1 (Jan 2015), 597--610.Google Scholar
- Robert William Gainer Hunt. 1952. Light and Dark Adaptation and the Perception of Color. Journal of the Optical Society of America 42, 3 (1952), 190--199.Google Scholar
Cross Ref
- Garrett M Johnson and Mark D Fairchild. 2001. Darwinism of Color Image Difference Models. In Color and Imaging Conference, Vol. 2001. 108--112.Google Scholar
- Garrett M. Johnson and Mark D. Fairchild. 2003. A Top Down Description of S-CIELAB and CIEDE2000. Color Research & Application 28, 6 (2003), 425--435.Google Scholar
Cross Ref
- E. C. Larson and D. M. Chandler. 2010. Most Apparent Distortion: Full-Reference Image Quality Assessment and the Role of Strategy. Journal of Electronic Imaging 19, 1 (2010).Google Scholar
- Amazon Lumberyard. 2017. Amazon Lumberyard Bistro, Open Research Content Archive (ORCA). http://developer.nvidia.com/orca/amazon-lumberyard-bistroGoogle Scholar
- Rafał Mantiuk, Kil Joong Kim, Allan G. Rempel, and Wolfgang Heidrich. 2011. HDR-VDP-2: A Calibrated Visual Metric for Visibility and Quality Predictions in All Luminance Conditions. ACM Transactions on Graphics 30, 4 (2011), 40:1-40:14.Google Scholar
Digital Library
- William McIlhagga. 2018. Estimates of Edge Detection Filters in Human Vision. Vision Research 153 (2018), 30--36.Google Scholar
Cross Ref
- Vishal Monga, Wilson S Geisler, and Brian L Evans. 2003. Linear Color-Separable Human Visual System Models for Vector Error Diffusion Halftoning. IEEE Signal Processing Letters 10, 4 (2003), 93--97.Google Scholar
Cross Ref
- J. Anthony Movshon and Lynne Kiorpes. 1988. Analysis of the Development of Spatial Contrast Sensitivity in Monkey and Human Infants. Journal of the Optical Society of America 5, 12 (1988), 2166--2172.Google Scholar
Cross Ref
- Kathy T. Mullen. 1985. The Contrast Sensitivity of Human Colour Vision to Red-Green and Blue-Yellow Chromatic Gratings. The Journal of Physiology 359, 1 (1985), 381--400.Google Scholar
Cross Ref
- Jim Nilsson and Tomas Akenine-Möller. 2020. Understanding SSIM. arXiv:2006.13846 [eess.IV]Google Scholar
- Charles Poynton. 2003. Digital Video and HDTV -- Algorithms and Interfaces. Morgan Kaufmann.Google Scholar
- Ekta Prashnani, Hong Cai, Yasamin Mostofi, and Pradeep Sen. 2018. PieAPP: Perceptual Image-Error Assessment through Pairwise Preference. CoRR abs/1806.02067 (2018). arXiv:1806.02067 http://arxiv.org/abs/1806.02067Google Scholar
- Thijs Vogels, Fabrice Rousselle, Brian Mcwilliams, Gerhard Röthlin, Alex Harvill, David Adler, Mark Meyer, and Jan Novák. 2018. Denoising with Kernel Prediction and Asymmetric Loss Functions. ACM Transactions of Graphics 37, 4, Article 124 (2018).Google Scholar
Digital Library
- Z. Wang, A. C. Bovik, H. R. Sheikh, and E. P. Simoncelli. 2004. Image Quality Assessment: from Error Visibility to Structural Similarity. IEEE Transactions on Image Processing 13, 4 (2004), 600--612.Google Scholar
Digital Library
- Zhou Wang, Eero P Simoncelli, and Alan C Bovik. 2003. Multiscale Structural Similarity for Image Quality Assessment. In The Thrity-Seventh Asilomar Conference on Signals, Systems & Computers, Vol. 2. IEEE, 1398--1402.Google Scholar
- Krzysztof Wolski, Daniele Giunchi, Nanyang Ye, Piotr Didyk, Karol Myszkowski, Radosław Mantiuk, Hans-Peter Seidel, Anthony Steed, and Rafał K. Mantiuk. 2018. Dataset and Metrics for Predicting Local Visible Differences. ACM Transactions on Graphics 37, 5, Article 172 (2018).Google Scholar
Digital Library
- Gunter Wyszecki and Walter Stanley Stiles. 1982. Color Science. Vol. 8. Wiley New York.Google Scholar
- R. Zhang, P. Isola, A. A. Efros, E. Shechtman, and O. Wang. 2018. The Unreasonable Effectiveness of Deep Features as a Perceptual Metric. In IEEE Conference on Computer Vision and Pattern Recognition. 586--595.Google Scholar
- Xuemei Zhang and Brian A Wandell. 1997. A Spatial Extension of CIELAB for Digital Color-Image Reproduction. Journal of the Society for Information Display 5, 1 (1997), 61--63.Google Scholar
Cross Ref
- Matthias Zwicker, Wojciech Jarosz, Jaakko Lehtinen, Bochang Moon, Ravi Ramamoorthi, Fabrice Rousselle, Pradeep Sen, Cyril Soler, and Sung-Eui Yoon. 2015. Recent Advances in Adaptive Sampling and Reconstruction for Monte Carlo Rendering. Computer Graphics Forum (Proceedings of Eurographics - State of the Art Reports) 34, 2 (2015), 667--681.Google Scholar
Digital Library
Index Terms
FLIP: A Difference Evaluator for Alternating Images
Recommendations
A Systolic Image Difference Algorithm for RLE-Compressed Images
A new systolic algorithm which computes image differences in run-length encoded (RLE) format is described. The binary image difference operation is commonly used in many image processing applications including automated inspection systems, character ...
An adaptive image Euclidean distance
The image Euclidean distance (IMED) considers the spatial relationship between the pixels of different images and can easily be embedded in existing image recognition algorithms that are based on Euclidean distance. IMED uses the prior knowledge that ...
A psychophysical investigation of global illumination algorithms used in augmented reality
The overarching goal of this research was to compare different rendering solutions in order to understand why some yield better results specifically when applied to rendering synthetic objects into real photographs. A psychophysical experiment was ...






Comments