Abstract
We present a numerical optimization method to find highly efficient (sparse) approximations for convolutional image filters. Using a modified parallel tempering approach, we solve a constrained optimization that maximizes approximation quality while strictly staying within a user-prescribed performance budget. The results are multi-pass filters where each pass computes a weighted sum of bilinearly interpolated sparse image samples, exploiting hardware acceleration on the GPU. We systematically decompose the target filter into a series of sparse convolutions, trying to find good trade-offs between approximation quality and performance. Since our sparse filters are linear and translation-invariant, they do not exhibit the aliasing and temporal coherence issues that often appear in filters working on image pyramids. We show several applications, ranging from simple Gaussian or box blurs to the emulation of sophisticated Bokeh effects with user-provided masks. Our filters achieve high performance as well as high quality, often providing significant speed-up at acceptable quality even for separable filters. The optimized filters can be baked into shaders and used as a drop-in replacement for filtering tasks in image processing or rendering pipelines.
Supplemental Material
Available for Download
Supplemental movie, appendix, image and software files for, High-Performance Image Filters via Sparse Approximations
- Marius Bjørge. 2015. Bandwidth-efficient Graphics. In ACM SIGGRAPH 2015 Courses - Moving Mobile Graphics (Los Angeles, California) (SIGGRAPH '15). Association for Computing Machinery, New York, NY, USA, Article 18, 1 pages.Google Scholar
- John Canny. 1986. A computational approach to edge detection. IEEE Transactions on pattern analysis and machine intelligence 6 (1986), 679--698.Google Scholar
Digital Library
- Robert L Cook. 1986. Stochastic sampling in computer graphics. ACM Transactions on Graphics (TOG) 5, 1 (1986), 51--72.Google Scholar
Digital Library
- Phill Djonov. 2012. Bicubic Filtering in Fewer Taps. Shiny Pixels (2012). http://vec3.ca/bicubic-filtering-in-fewer-taps/Google Scholar
- Ondirej Fialka and Martin Cadik. 2006. FFT and convolution performance in image filtering on GPU. In Tenth International Conference on Information Visualisation (IV'06). IEEE, 609--614.Google Scholar
Digital Library
- Kleber Garcia. 2017. Circular separable convolution depth of field. In ACM SIGGRAPH 2017 Talks. 1--2.Google Scholar
Digital Library
- Rafael C Gonzales and Richard E Woods. 2018. Digital Image Processing (4 ed.). Pearson. 153--162 pages.Google Scholar
- Craig Gotsman. 1994. Constant-Time Filtering by Singular Value Decomposition. In Computer Graphics Forum, Vol. 13. Wiley Online Library, 153--163.Google Scholar
- Peter J Green. 1995. Reversible jump Markov chain Monte Carlo computation and Bayesian model determination. Biometrika 82, 4 (1995), 711--732.Google Scholar
Cross Ref
- Christopher G Harris, Mike Stephens, et al. 1988. A combined corner and edge detector.. In Alvey vision conference, Vol. 15. Citeseer, 10--5244.Google Scholar
- Justin Hensley, Thorsten Scheuermann, Greg Coombe, Montek Singh, and Anselmo Lastra. 2005. Fast summed-area table generation and its applications. In Computer Graphics Forum, Vol. 24. Wiley Online Library, 547--555.Google Scholar
- Hyuntae Joo, Soonhyeon Kwon, Sangmin Lee, Elmar Eisemann, and Sungkil Lee. 2016. Efficient ray tracing through aspheric lenses and imperfect bokeh synthesis. In Computer Graphics Forum, Vol. 35. Wiley Online Library, 99--105.Google Scholar
- Masaki Kawase. 2003. Frame Buffer Postprocessing Effects in DOUBLE-STEAL (Wrechless). In Game Developers Conference 2003, 3.Google Scholar
- Todd Jerome Kosloff, Justin Hensley, and Brian A Barsky. 2009. Fast filter spreading and its applications. EECS Department, University of California, Berkeley, Tech. Rep. UCB/EECS-2009-54 (2009).Google Scholar
- Sungkil Lee, Elmar Eisemann, and Hans-Peter Seidel. 2009a. Depth-of-field rendering with multiview synthesis. ACM Transactions on Graphics (TOG) 28, 5 (2009), 1--6.Google Scholar
Digital Library
- Sungkil Lee, Elmar Eisemann, and Hans-Peter Seidel. 2010. Real-time lens blur effects and focus control. ACM Transactions on Graphics (TOG) 29, 4 (2010), 1--7.Google Scholar
Digital Library
- Sungkil Lee, Gerard Jounghyun Kim, and Seungmoon Choi. 2008. Real-time depth-of-field rendering using point splatting on per-pixel layers. In Computer Graphics Forum, Vol. 27. Wiley Online Library, 1955--1962.Google Scholar
- Sungkil Lee, Gerard Jounghyun Kim, and Seungmoon Choi. 2009b. Real-time depth-of-field rendering using anisotropically filtered mipmap interpolation. IEEE Transactions on Visualization and Computer Graphics 15, 3 (2009), 453--464.Google Scholar
Digital Library
- Josiah Manson and Scott Schaefer. 2013. Cardinality-constrained texture filtering. ACM Transactions on Graphics (TOG) 32, 4 (2013), 1--8.Google Scholar
Digital Library
- Josiah Manson and Peter-Pike Sloan. 2016. Fast filtering of reflection probes. In Computer Graphics Forum, Vol. 35. Wiley Online Library, 119--127.Google Scholar
- Pavlos Mavridis and Georgios Papaioannou. 2011. High quality elliptical texture filtering on GPU. In Symposium on Interactive 3D Graphics and Games. 23--30.Google Scholar
Digital Library
- Joel McCormack, Ronald Perry, Keith I Farkas, and Norman P Jouppi. 1999. Feline: fast elliptical lines for anisotropic texture mapping. In Proceedings of the 26th annual conference on Computer graphics and interactive techniques. 243--250.Google Scholar
Digital Library
- Tim McGraw. 2015. Fast Bokeh effects using low-rank linear filters. The Visual Computer 31, 5 (2015), 601--611.Google Scholar
Digital Library
- L McIntosh, Bernhard E Riecke, and Steve DiPaola. 2012. Efficiently Simulating the Bokeh of Polygonal Apertures in a Post-Process Depth of Field Shader. In Computer Graphics Forum, Vol. 31. Wiley Online Library, 1810--1822.Google Scholar
- Johannes Moersch and Howard J Hamilton. 2014. Variable-sized, circular bokeh depth of field effects. In Proceedings of Graphics Interface 2014. 103--107.Google Scholar
- Diego Nehab and André Maximo. 2016. Parallel recursive filtering of infinite input extensions. ACM Transactions on Graphics (TOG) 35, 6 (2016), 1--13.Google Scholar
Digital Library
- Diego Nehab, André Maximo, Rodolfo S. Lima, and Hugues Hoppe. 2011. GPU-Efficient Recursive Filtering and Summed-Area Tables. ACM Trans. Graph. 30, 6 (Dec. 2011), 1--12.Google Scholar
Digital Library
- Dan Piponi. 2012. Fast and Exact Convolution with Polygonal Filters. (01 2012).Google Scholar
- Michael Potmesil and Indranil Chakravarty. 1982. Synthetic image generation with a lens and aperture camera model. ACM Transactions on Graphics (TOG) 1, 2 (1982), 85--108.Google Scholar
Digital Library
- Daniel Rákos. 2010. Efficient Gaussian blur with linear sampling. RasterGrid Blogosphere (2010). http://rastergrid.com/blog/2010/09/efficient-gaussian-blur-with-linear-sampling/Google Scholar
- Christian Sigg and Markus Hadwiger. 2005. Fast third-order texture filtering. GPU gems 2 (2005), 313--329.Google Scholar
- Tiago Sousa. 2013. CryEngine 3 Graphics Gems. ACM SIGGRAPH Advances in Real-Time Rendering Course (2013).Google Scholar
- Filip Strugar. 2014. An investigation of fast real-time GPU-based image blur algorithms. (2014). https://software.intel.com/enus/blogs/2014/07/15/an-investigation-of-fast-real-time-gpu-based-image-blur-algorithmsGoogle Scholar
- Swendsen and Wang. 1986. Replica Monte Carlo simulation of spin glasses. Physical review letters 57 21 (1986), 2607--2609.Google Scholar
- Zhou Wang, Alan C Bovik, Hamid R Sheikh, and Eero P Simoncelli. 2004. Image quality assessment: from error visibility to structural similarity. IEEE transactions on image processing 13, 4 (2004), 600--612.Google Scholar
Digital Library
- John White and Colin Barré-Brisebois. 2011. More Performance! Five Rendering Ideas from Battlefield 3 and Need For Speed: The Run. ACM SIGGRAPH 2011: Advances in the realtime rendering course (2011).Google Scholar
Index Terms
High-Performance Image Filters via Sparse Approximations
Recommendations
Performance Assessment of Edge Preserving Filters
Denoising is one of the important aspects in image processing applications. Denoising is the process of eliminating the noise from the noisy image. In most cases, noise accumulates at the edges. So that prevention of noise at edges is one of the most ...
Effects of Image Filters on Various Image Datasets
ICCTA '19: Proceedings of the 2019 5th International Conference on Computer and Technology ApplicationsImage classification is a very common research area, on which researchers work with various classification techniques. The aim of this study is to apply different filters on four different datasets and evaluate their performances in image ...






Comments