skip to main content
research-article

High-Performance Image Filters via Sparse Approximations

Authors Info & Claims
Published:26 August 2020Publication History
Skip Abstract Section

Abstract

We present a numerical optimization method to find highly efficient (sparse) approximations for convolutional image filters. Using a modified parallel tempering approach, we solve a constrained optimization that maximizes approximation quality while strictly staying within a user-prescribed performance budget. The results are multi-pass filters where each pass computes a weighted sum of bilinearly interpolated sparse image samples, exploiting hardware acceleration on the GPU. We systematically decompose the target filter into a series of sparse convolutions, trying to find good trade-offs between approximation quality and performance. Since our sparse filters are linear and translation-invariant, they do not exhibit the aliasing and temporal coherence issues that often appear in filters working on image pyramids. We show several applications, ranging from simple Gaussian or box blurs to the emulation of sophisticated Bokeh effects with user-provided masks. Our filters achieve high performance as well as high quality, often providing significant speed-up at acceptable quality even for separable filters. The optimized filters can be baked into shaders and used as a drop-in replacement for filtering tasks in image processing or rendering pipelines.

Skip Supplemental Material Section

Supplemental Material

References

  1. Marius Bjørge. 2015. Bandwidth-efficient Graphics. In ACM SIGGRAPH 2015 Courses - Moving Mobile Graphics (Los Angeles, California) (SIGGRAPH '15). Association for Computing Machinery, New York, NY, USA, Article 18, 1 pages.Google ScholarGoogle Scholar
  2. John Canny. 1986. A computational approach to edge detection. IEEE Transactions on pattern analysis and machine intelligence 6 (1986), 679--698.Google ScholarGoogle ScholarDigital LibraryDigital Library
  3. Robert L Cook. 1986. Stochastic sampling in computer graphics. ACM Transactions on Graphics (TOG) 5, 1 (1986), 51--72.Google ScholarGoogle ScholarDigital LibraryDigital Library
  4. Phill Djonov. 2012. Bicubic Filtering in Fewer Taps. Shiny Pixels (2012). http://vec3.ca/bicubic-filtering-in-fewer-taps/Google ScholarGoogle Scholar
  5. Ondirej Fialka and Martin Cadik. 2006. FFT and convolution performance in image filtering on GPU. In Tenth International Conference on Information Visualisation (IV'06). IEEE, 609--614.Google ScholarGoogle ScholarDigital LibraryDigital Library
  6. Kleber Garcia. 2017. Circular separable convolution depth of field. In ACM SIGGRAPH 2017 Talks. 1--2.Google ScholarGoogle ScholarDigital LibraryDigital Library
  7. Rafael C Gonzales and Richard E Woods. 2018. Digital Image Processing (4 ed.). Pearson. 153--162 pages.Google ScholarGoogle Scholar
  8. Craig Gotsman. 1994. Constant-Time Filtering by Singular Value Decomposition. In Computer Graphics Forum, Vol. 13. Wiley Online Library, 153--163.Google ScholarGoogle Scholar
  9. Peter J Green. 1995. Reversible jump Markov chain Monte Carlo computation and Bayesian model determination. Biometrika 82, 4 (1995), 711--732.Google ScholarGoogle ScholarCross RefCross Ref
  10. Christopher G Harris, Mike Stephens, et al. 1988. A combined corner and edge detector.. In Alvey vision conference, Vol. 15. Citeseer, 10--5244.Google ScholarGoogle Scholar
  11. Justin Hensley, Thorsten Scheuermann, Greg Coombe, Montek Singh, and Anselmo Lastra. 2005. Fast summed-area table generation and its applications. In Computer Graphics Forum, Vol. 24. Wiley Online Library, 547--555.Google ScholarGoogle Scholar
  12. Hyuntae Joo, Soonhyeon Kwon, Sangmin Lee, Elmar Eisemann, and Sungkil Lee. 2016. Efficient ray tracing through aspheric lenses and imperfect bokeh synthesis. In Computer Graphics Forum, Vol. 35. Wiley Online Library, 99--105.Google ScholarGoogle Scholar
  13. Masaki Kawase. 2003. Frame Buffer Postprocessing Effects in DOUBLE-STEAL (Wrechless). In Game Developers Conference 2003, 3.Google ScholarGoogle Scholar
  14. Todd Jerome Kosloff, Justin Hensley, and Brian A Barsky. 2009. Fast filter spreading and its applications. EECS Department, University of California, Berkeley, Tech. Rep. UCB/EECS-2009-54 (2009).Google ScholarGoogle Scholar
  15. Sungkil Lee, Elmar Eisemann, and Hans-Peter Seidel. 2009a. Depth-of-field rendering with multiview synthesis. ACM Transactions on Graphics (TOG) 28, 5 (2009), 1--6.Google ScholarGoogle ScholarDigital LibraryDigital Library
  16. Sungkil Lee, Elmar Eisemann, and Hans-Peter Seidel. 2010. Real-time lens blur effects and focus control. ACM Transactions on Graphics (TOG) 29, 4 (2010), 1--7.Google ScholarGoogle ScholarDigital LibraryDigital Library
  17. Sungkil Lee, Gerard Jounghyun Kim, and Seungmoon Choi. 2008. Real-time depth-of-field rendering using point splatting on per-pixel layers. In Computer Graphics Forum, Vol. 27. Wiley Online Library, 1955--1962.Google ScholarGoogle Scholar
  18. Sungkil Lee, Gerard Jounghyun Kim, and Seungmoon Choi. 2009b. Real-time depth-of-field rendering using anisotropically filtered mipmap interpolation. IEEE Transactions on Visualization and Computer Graphics 15, 3 (2009), 453--464.Google ScholarGoogle ScholarDigital LibraryDigital Library
  19. Josiah Manson and Scott Schaefer. 2013. Cardinality-constrained texture filtering. ACM Transactions on Graphics (TOG) 32, 4 (2013), 1--8.Google ScholarGoogle ScholarDigital LibraryDigital Library
  20. Josiah Manson and Peter-Pike Sloan. 2016. Fast filtering of reflection probes. In Computer Graphics Forum, Vol. 35. Wiley Online Library, 119--127.Google ScholarGoogle Scholar
  21. Pavlos Mavridis and Georgios Papaioannou. 2011. High quality elliptical texture filtering on GPU. In Symposium on Interactive 3D Graphics and Games. 23--30.Google ScholarGoogle ScholarDigital LibraryDigital Library
  22. Joel McCormack, Ronald Perry, Keith I Farkas, and Norman P Jouppi. 1999. Feline: fast elliptical lines for anisotropic texture mapping. In Proceedings of the 26th annual conference on Computer graphics and interactive techniques. 243--250.Google ScholarGoogle ScholarDigital LibraryDigital Library
  23. Tim McGraw. 2015. Fast Bokeh effects using low-rank linear filters. The Visual Computer 31, 5 (2015), 601--611.Google ScholarGoogle ScholarDigital LibraryDigital Library
  24. L McIntosh, Bernhard E Riecke, and Steve DiPaola. 2012. Efficiently Simulating the Bokeh of Polygonal Apertures in a Post-Process Depth of Field Shader. In Computer Graphics Forum, Vol. 31. Wiley Online Library, 1810--1822.Google ScholarGoogle Scholar
  25. Johannes Moersch and Howard J Hamilton. 2014. Variable-sized, circular bokeh depth of field effects. In Proceedings of Graphics Interface 2014. 103--107.Google ScholarGoogle Scholar
  26. Diego Nehab and André Maximo. 2016. Parallel recursive filtering of infinite input extensions. ACM Transactions on Graphics (TOG) 35, 6 (2016), 1--13.Google ScholarGoogle ScholarDigital LibraryDigital Library
  27. Diego Nehab, André Maximo, Rodolfo S. Lima, and Hugues Hoppe. 2011. GPU-Efficient Recursive Filtering and Summed-Area Tables. ACM Trans. Graph. 30, 6 (Dec. 2011), 1--12.Google ScholarGoogle ScholarDigital LibraryDigital Library
  28. Dan Piponi. 2012. Fast and Exact Convolution with Polygonal Filters. (01 2012).Google ScholarGoogle Scholar
  29. Michael Potmesil and Indranil Chakravarty. 1982. Synthetic image generation with a lens and aperture camera model. ACM Transactions on Graphics (TOG) 1, 2 (1982), 85--108.Google ScholarGoogle ScholarDigital LibraryDigital Library
  30. Daniel Rákos. 2010. Efficient Gaussian blur with linear sampling. RasterGrid Blogosphere (2010). http://rastergrid.com/blog/2010/09/efficient-gaussian-blur-with-linear-sampling/Google ScholarGoogle Scholar
  31. Christian Sigg and Markus Hadwiger. 2005. Fast third-order texture filtering. GPU gems 2 (2005), 313--329.Google ScholarGoogle Scholar
  32. Tiago Sousa. 2013. CryEngine 3 Graphics Gems. ACM SIGGRAPH Advances in Real-Time Rendering Course (2013).Google ScholarGoogle Scholar
  33. Filip Strugar. 2014. An investigation of fast real-time GPU-based image blur algorithms. (2014). https://software.intel.com/enus/blogs/2014/07/15/an-investigation-of-fast-real-time-gpu-based-image-blur-algorithmsGoogle ScholarGoogle Scholar
  34. Swendsen and Wang. 1986. Replica Monte Carlo simulation of spin glasses. Physical review letters 57 21 (1986), 2607--2609.Google ScholarGoogle Scholar
  35. Zhou Wang, Alan C Bovik, Hamid R Sheikh, and Eero P Simoncelli. 2004. Image quality assessment: from error visibility to structural similarity. IEEE transactions on image processing 13, 4 (2004), 600--612.Google ScholarGoogle ScholarDigital LibraryDigital Library
  36. John White and Colin Barré-Brisebois. 2011. More Performance! Five Rendering Ideas from Battlefield 3 and Need For Speed: The Run. ACM SIGGRAPH 2011: Advances in the realtime rendering course (2011).Google ScholarGoogle Scholar

Index Terms

  1. High-Performance Image Filters via Sparse Approximations

      Recommendations

      Comments

      Login options

      Check if you have access through your login credentials or your institution to get full access on this article.

      Sign in

      Full Access

      • Published in

        cover image Proceedings of the ACM on Computer Graphics and Interactive Techniques
        Proceedings of the ACM on Computer Graphics and Interactive Techniques  Volume 3, Issue 2
        August 2020
        218 pages
        EISSN:2577-6193
        DOI:10.1145/3420254
        Issue’s Table of Contents

        Copyright © 2020 ACM

        Publisher

        Association for Computing Machinery

        New York, NY, United States

        Publication History

        • Published: 26 August 2020
        Published in pacmcgit Volume 3, Issue 2

        Permissions

        Request permissions about this article.

        Request Permissions

        Check for updates

        Qualifiers

        • research-article
        • Research
        • Refereed
      • Article Metrics

        • Downloads (Last 12 months)37
        • Downloads (Last 6 weeks)3

        Other Metrics

      PDF Format

      View or Download as a PDF file.

      PDF

      eReader

      View online with eReader.

      eReader
      About Cookies On This Site

      We use cookies to ensure that we give you the best experience on our website.

      Learn more

      Got it!