skip to main content
research-article

FLIP: A Difference Evaluator for Alternating Images

Published:26 August 2020Publication History
Skip Abstract Section

Abstract

Image quality measures are becoming increasingly important in the field of computer graphics. For example, there is currently a major focus on generating photorealistic images in real time by combining path tracing with denoising, for which such quality assessment is integral. We present FLIP, which is a difference evaluator with a particular focus on the differences between rendered images and corresponding ground truths. Our algorithm produces a map that approximates the difference perceived by humans when alternating between two images. FLIP is a combination of modified existing building blocks, and the net result is surprisingly powerful. We have compared our work against a wide range of existing image difference algorithms and we have visually inspected over a thousand image pairs that were either retrieved from image databases or generated in-house. We also present results of a user study which indicate that our method performs substantially better, on average, than the other algorithms. To facilitate the use of FLIP, we provide source code in C++, MATLAB, NumPy/SciPy, and PyTorch.

Skip Supplemental Material Section

Supplemental Material

3406183.mp4

Presentation Video

References

  1. Saeedeh Abasi, Mohammad Amani Tehran, and Mark D. Fairchild. 2020. Distance Metrics for Very Large Color Differences. Color Research and Application 45, 2 (2020), 208--223.Google ScholarGoogle ScholarCross RefCross Ref
  2. Jyrki Alakuijala, Robert Obryk, Ostap Stoliarchuk, Zoltan Szabadka, Lode Vandevenne, and Jan Wassenberg. 2017. Guetzli: Perceptually Guided JPEG Encoder. CoRR abs/1703.04421 (2017). arXiv:1703.04421 http://arxiv.org/abs/1703.04421Google ScholarGoogle Scholar
  3. Steve Bako, Thijs Vogels, Brian McWilliams, Mark Meyer, Jan Novák, Alex Harvill, Pradeep Sen, Tony DeRose, and Fabrice Rousselle. 2017. Kernel-Predicting Convolutional Networks for Denoising Monte Carlo Renderings. ACM Transactions on Graphics 36, 4 (2017), 97:1--97:14.Google ScholarGoogle ScholarDigital LibraryDigital Library
  4. Benedikt Bitterli. 2016. Rendering Resources. https://benedikt-bitterli.me/resources/.Google ScholarGoogle Scholar
  5. Martin Čadík, Robert Herzog, Rafał Mantiuk, Karol Myszkowski, and Hans-Peter Seidel. 2012. New Measurements Reveal Weaknesses of Image Quality Metrics in Evaluating Graphics Artifacts. ACM Transactions of Graphics 31, 6 (2012), 147:1--147:10.Google ScholarGoogle ScholarDigital LibraryDigital Library
  6. Damon M Chandler. 2013. Seven Challenges in Image Quality Assessment: Past, Present, and Future Research. ISRN Signal Processing 2013 (2013), 1--53.Google ScholarGoogle ScholarCross RefCross Ref
  7. Comission Internationale de l'Eclairage CIE. 1978. Recommendations on Uniform Color Spaces, Color-Difference Equations, Psychometric Color Terms. Supplement No.2 to CIE publication No.15 (E.-1.3.1) 1971/(TC-1.3.) (1978).Google ScholarGoogle Scholar
  8. Oliver Deussen, Pat Hanrahan, Bernd Lintermann, Radomír Měch, Matt Pharr, and Przemyslaw Prusinkiewicz. 1998. Realistic Modeling and Rendering of Plant Ecosystems. In Proceedings of SIGGRAPH. 275--286.Google ScholarGoogle ScholarDigital LibraryDigital Library
  9. Russell L. DeValois and Karen K. DeValois. 1990. Spatial Vision. Oxford University Press.Google ScholarGoogle Scholar
  10. Mark D Fairchild. 1996. Refinement of the RLAB Color Space. Color Research & Application 21, 5 (1996), 338--346.Google ScholarGoogle ScholarCross RefCross Ref
  11. Mark D. Fairchild. 2013. Color Appearance Models (3rd ed.). John Wiley & Sons.Google ScholarGoogle Scholar
  12. Mark D. Fairchild and Garrett M. Johnson. 2004. The iCAM Framework for Image Appearance, Image Differences, and Image Quality. Journal of Electronic Imaging 13, 1 (2004), 126--138.Google ScholarGoogle ScholarCross RefCross Ref
  13. Thomas J Flohr, Bernd W Kolpatzik, Raja Balasubramanian, David A Carrara, Charles A Bouman, and Jan P Allebach. 1993. Model-Based Color Image Quantization. In Human Vision, Visual Processing, and Digital Display IV, Vol. 1913. 270--281.Google ScholarGoogle Scholar
  14. Robert Herzog, Martin Čadík, Tunc. O. Aydın, Kwawng In Kim, Karol Myszkowski, and Hans-Peter Seidel. 2012. NoRM: No-Reference Image Quality Metric for Realistic Image Synthesis. Computer Graphics Forum 31, 2 (2012), 545--554.Google ScholarGoogle ScholarDigital LibraryDigital Library
  15. Min Huang, Guihua Cui, Manuel Melgosa, Manuel Sánchez-Mara nón, Changjun Li, M. Ronnier Luo, and Haoxue Liu. 2015. Power Functions Improving the Performance of Color-Difference Formulas. Optics Express 23, 1 (Jan 2015), 597--610.Google ScholarGoogle Scholar
  16. Robert William Gainer Hunt. 1952. Light and Dark Adaptation and the Perception of Color. Journal of the Optical Society of America 42, 3 (1952), 190--199.Google ScholarGoogle ScholarCross RefCross Ref
  17. Garrett M Johnson and Mark D Fairchild. 2001. Darwinism of Color Image Difference Models. In Color and Imaging Conference, Vol. 2001. 108--112.Google ScholarGoogle Scholar
  18. Garrett M. Johnson and Mark D. Fairchild. 2003. A Top Down Description of S-CIELAB and CIEDE2000. Color Research & Application 28, 6 (2003), 425--435.Google ScholarGoogle ScholarCross RefCross Ref
  19. E. C. Larson and D. M. Chandler. 2010. Most Apparent Distortion: Full-Reference Image Quality Assessment and the Role of Strategy. Journal of Electronic Imaging 19, 1 (2010).Google ScholarGoogle Scholar
  20. Amazon Lumberyard. 2017. Amazon Lumberyard Bistro, Open Research Content Archive (ORCA). http://developer.nvidia.com/orca/amazon-lumberyard-bistroGoogle ScholarGoogle Scholar
  21. Rafał Mantiuk, Kil Joong Kim, Allan G. Rempel, and Wolfgang Heidrich. 2011. HDR-VDP-2: A Calibrated Visual Metric for Visibility and Quality Predictions in All Luminance Conditions. ACM Transactions on Graphics 30, 4 (2011), 40:1-40:14.Google ScholarGoogle ScholarDigital LibraryDigital Library
  22. William McIlhagga. 2018. Estimates of Edge Detection Filters in Human Vision. Vision Research 153 (2018), 30--36.Google ScholarGoogle ScholarCross RefCross Ref
  23. Vishal Monga, Wilson S Geisler, and Brian L Evans. 2003. Linear Color-Separable Human Visual System Models for Vector Error Diffusion Halftoning. IEEE Signal Processing Letters 10, 4 (2003), 93--97.Google ScholarGoogle ScholarCross RefCross Ref
  24. J. Anthony Movshon and Lynne Kiorpes. 1988. Analysis of the Development of Spatial Contrast Sensitivity in Monkey and Human Infants. Journal of the Optical Society of America 5, 12 (1988), 2166--2172.Google ScholarGoogle ScholarCross RefCross Ref
  25. Kathy T. Mullen. 1985. The Contrast Sensitivity of Human Colour Vision to Red-Green and Blue-Yellow Chromatic Gratings. The Journal of Physiology 359, 1 (1985), 381--400.Google ScholarGoogle ScholarCross RefCross Ref
  26. Jim Nilsson and Tomas Akenine-Möller. 2020. Understanding SSIM. arXiv:2006.13846 [eess.IV]Google ScholarGoogle Scholar
  27. Charles Poynton. 2003. Digital Video and HDTV -- Algorithms and Interfaces. Morgan Kaufmann.Google ScholarGoogle Scholar
  28. Ekta Prashnani, Hong Cai, Yasamin Mostofi, and Pradeep Sen. 2018. PieAPP: Perceptual Image-Error Assessment through Pairwise Preference. CoRR abs/1806.02067 (2018). arXiv:1806.02067 http://arxiv.org/abs/1806.02067Google ScholarGoogle Scholar
  29. Thijs Vogels, Fabrice Rousselle, Brian Mcwilliams, Gerhard Röthlin, Alex Harvill, David Adler, Mark Meyer, and Jan Novák. 2018. Denoising with Kernel Prediction and Asymmetric Loss Functions. ACM Transactions of Graphics 37, 4, Article 124 (2018).Google ScholarGoogle ScholarDigital LibraryDigital Library
  30. Z. Wang, A. C. Bovik, H. R. Sheikh, and E. P. Simoncelli. 2004. Image Quality Assessment: from Error Visibility to Structural Similarity. IEEE Transactions on Image Processing 13, 4 (2004), 600--612.Google ScholarGoogle ScholarDigital LibraryDigital Library
  31. Zhou Wang, Eero P Simoncelli, and Alan C Bovik. 2003. Multiscale Structural Similarity for Image Quality Assessment. In The Thrity-Seventh Asilomar Conference on Signals, Systems & Computers, Vol. 2. IEEE, 1398--1402.Google ScholarGoogle Scholar
  32. Krzysztof Wolski, Daniele Giunchi, Nanyang Ye, Piotr Didyk, Karol Myszkowski, Radosław Mantiuk, Hans-Peter Seidel, Anthony Steed, and Rafał K. Mantiuk. 2018. Dataset and Metrics for Predicting Local Visible Differences. ACM Transactions on Graphics 37, 5, Article 172 (2018).Google ScholarGoogle ScholarDigital LibraryDigital Library
  33. Gunter Wyszecki and Walter Stanley Stiles. 1982. Color Science. Vol. 8. Wiley New York.Google ScholarGoogle Scholar
  34. R. Zhang, P. Isola, A. A. Efros, E. Shechtman, and O. Wang. 2018. The Unreasonable Effectiveness of Deep Features as a Perceptual Metric. In IEEE Conference on Computer Vision and Pattern Recognition. 586--595.Google ScholarGoogle Scholar
  35. Xuemei Zhang and Brian A Wandell. 1997. A Spatial Extension of CIELAB for Digital Color-Image Reproduction. Journal of the Society for Information Display 5, 1 (1997), 61--63.Google ScholarGoogle ScholarCross RefCross Ref
  36. Matthias Zwicker, Wojciech Jarosz, Jaakko Lehtinen, Bochang Moon, Ravi Ramamoorthi, Fabrice Rousselle, Pradeep Sen, Cyril Soler, and Sung-Eui Yoon. 2015. Recent Advances in Adaptive Sampling and Reconstruction for Monte Carlo Rendering. Computer Graphics Forum (Proceedings of Eurographics - State of the Art Reports) 34, 2 (2015), 667--681.Google ScholarGoogle ScholarDigital LibraryDigital Library

Index Terms

  1. FLIP: A Difference Evaluator for Alternating Images

      Recommendations

      Comments

      Login options

      Check if you have access through your login credentials or your institution to get full access on this article.

      Sign in

      Full Access

      • Published in

        cover image Proceedings of the ACM on Computer Graphics and Interactive Techniques
        Proceedings of the ACM on Computer Graphics and Interactive Techniques  Volume 3, Issue 2
        August 2020
        218 pages
        EISSN:2577-6193
        DOI:10.1145/3420254
        Issue’s Table of Contents

        Copyright © 2020 ACM

        Publisher

        Association for Computing Machinery

        New York, NY, United States

        Publication History

        • Published: 26 August 2020
        Published in pacmcgit Volume 3, Issue 2

        Permissions

        Request permissions about this article.

        Request Permissions

        Check for updates

        Qualifiers

        • research-article
        • Research
        • Refereed

      PDF Format

      View or Download as a PDF file.

      PDF

      eReader

      View online with eReader.

      eReader
      About Cookies On This Site

      We use cookies to ensure that we give you the best experience on our website.

      Learn more

      Got it!