Abstract
As a new form of volumetric media, Light Field (LF) can provide users with a true six degrees of freedom immersive experience because LF captures the scene with photo-realism, including aperture-limited changes in viewpoint. But uncompressed LF data is too large for network transmission, which is the reason why LF compression has become an important research topic. One of the more recent approaches for LF compression is to reduce the angular resolution of the input LF during compression and to use LF reconstruction to recover the discarded viewpoints during decompression. Following this approach, we propose a new LF reconstruction algorithm based on Graph Neural Networks; we show that it can achieve higher compression and better quality compared to existing reconstruction methods, although suffering from the same problem as those methods—the inability to deal effectively with high-frequency image components. To solve this problem, we propose an adaptive two-layer compression architecture that separates high-frequency and low-frequency components and compresses each with a different strategy so that the performance can become robust and controllable. Experiments with multiple datasets1 show that our proposed scheme is capable of providing a decompression quality of above 40 dB, and can significantly improve compression efficiency compared with similar LF reconstruction schemes.
- Edward H. Adelson and James R. Bergen. 1991. The Plenoptic Function and the Elements of Early Vision. Vol. 2. Vision and Modeling Group, Media Laboratory, Massachusetts Institute of Technology.Google Scholar
- Amar Aggoun. 2011. Compression of 3D integral images using 3D wavelet transform. Journal of Display Technology 7, 11 (2011), 586--592.Google Scholar
Cross Ref
- H. Amirpour, M. Pereira, and A. Pinheiro. 2018. High efficient snake order pseudo-sequence based light field image compression. In 2018 Data Compression Conference. 397--397. DOI:https://doi.org/10.1109/DCC.2018.00050Google Scholar
Cross Ref
- Computer Graphics Laboratory, Stanford University 2008. Light Field Datasets. Retrieved from http://lightfield.stanford.edu/lfs.html.Google Scholar
- Caroline Conti, Luís Ducla Soares, and Paulo Nunes. 2016. HEVC-based 3D holoscopic video coding using self-similarity compensated prediction. Signal Processing Image Communication 42 (2016), 59--78.Google Scholar
Digital Library
- D. G. Dansereau, O. Pizarro, and S. B. Williams. 2013. Decoding, calibration and rectification for lenselet-based plenoptic cameras. In 2013 IEEE Conference on Computer Vision and Pattern Recognition. 1027--1034. DOI:https://doi.org/10.1109/CVPR.2013.137Google Scholar
Digital Library
- Steven J. Gortler, Radek Grzeszczuk, Richard Szeliski, and Michael F. Cohen. 1996. The lumigraph. In 23rd Annual Conference on Computer Graphics and Interactive Techniques. 43--54.Google Scholar
- Bichuan Guo, Yuxing Han, and Jiangtao Wen. 2018. Convex optimization based bit allocation for light field compression under weighting and consistency constraints. In Data Compression Conference. IEEE.Google Scholar
Cross Ref
- Will Hamilton, Zhitao Ying, and Jure Leskovec. 2017. Inductive representation learning on large graphs. In Advances in Neural Information Processing Systems. 1024--1034.Google Scholar
- Harini Priyadarshini Hariharan, Tobias Lange, and Thorsten Herfet. 2017. Low complexity light field compression based on pseudo-temporal circular sequencing. In IEEE International Symposium on Broadband Multimedia Systems and Broadcasting. 1--5.Google Scholar
Cross Ref
- Fatma Hawary, Christine Guillemot, Dominique Thoreau, and Guillaume Boisson. 2017. Scalable light field compression scheme using sparse reconstruction and restoration. In 2017 IEEE International Conference on Image Processing (ICIP). IEEE, 3250--3254.Google Scholar
Cross Ref
- Xinjue Hu, Jingming Shan, Yu Liu, and Lin Zhang. 2019. Adaptive two-layer light field compression scheme based on sparse reconstruction. In 10th ACM Multimedia Systems Conference. 74--85.Google Scholar
Digital Library
- Xiaoran Jiang, Mikaël Le Pendu, Reuben A. Farrugia, and Christine Guillemot. 2017. Light field compression with homography-based low rank approximation. IEEE Journal of Selected Topics in Signal Processing PP, 99 (2017), 1--1.Google Scholar
- Shinjini Kundu. 2012. Light field compression using homography and 2D warping. In IEEE International Conference on Acoustics, Speech and Signal Processing. 1349--1352.Google Scholar
Cross Ref
- Marc Levoy and Pat Hanrahan. 1996. Light field rendering. In 23rd Annual Conference on Computer Graphics and Interactive Techniques. 31--42.Google Scholar
Digital Library
- Li Li, Zhu Li, Bin Li, Dong Liu, and Houqiang Li. 2017. Pseudo-sequence-based 2-D hierarchical coding structure for light-field image compression. IEEE Journal of Selected Topics in Signal Processing 11, 7 (2017), 1107--1119.Google Scholar
Cross Ref
- Yun Li, Marten Sjostrom, Roger Olsson, and Ulf Jennehag. 2014. Efficient intra prediction scheme for light field image compression. In IEEE International Conference on Acoustics, Speech and Signal Processing. 539--543.Google Scholar
Cross Ref
- Dong Liu, Lizhi Wang, Li Li, Zhiwei Xiong, Feng Wu, and Wenjun Zeng. 2016. Pseudo-sequence-based light field image compression. In IEEE International Conference on Multimedia and Expo Workshops. 1--4.Google Scholar
Cross Ref
- Luís F. R. Lucas, Caroline Conti, Paulo Nunes, Luís Ducla Soares, Nuno M. M. Rodrigues, Carla L. Pagliari, Eduardo A. B. Da Silva, and Sérgio M. M. De Faria. 2014. Locally linear embedding-based prediction for 3D holoscopic image coding using HEVC. In Signal Processing Conference. 11--15.Google Scholar
- Ricardo Monteiro, Luis Lucas, Caroline Conti, Paulo Nunes, Nuno Rodrigues, Sergio Faria, Carla Pagliari, Eduardo Da Silva, and Luis Soares. 2016. Light field HEVC-based image coding using locally linear embedding and self-similarity compensated prediction. In IEEE International Conference on Multimedia and Expo Workshops. 1--4.Google Scholar
Cross Ref
- Ren Ng, Marc Levoy, Mathieu Brédif, Gene Duval, Mark Horowitz, Pat Hanrahan, et al. 2005. Light field photography with a hand-held plenoptic camera. Computer Science Technical Report CSTR 2, 11 (2005), 1--11.Google Scholar
- Mathias Niepert, Mohamed Ahmed, and Konstantin Kutzkov. 2016. Learning convolutional neural networks for graphs. In International Conference on Machine Learning.Google Scholar
- C. Perra and P. Assuncao. 2016. High efficiency coding of light field images based on tiling and pseudo-temporal data arrangement. In IEEE International Conference on Multimedia and Expo Workshops. 1--4.Google Scholar
- Cristian Perra and Daniele Giusto. 2017. JPEG 2000 compression of unfocused light field images based on lenslet array slicing. In IEEE International Conference on Consumer Electronics. 27--28.Google Scholar
- Reza Rassool. 2017. VMAF reproducibility: Validating a perceptual practical video quality metric. In IEEE International Symposium on Broadband Multimedia Systems and Broadcasting.Google Scholar
- Franco Scarselli, Marco Gori, Ah Chung Tsoi, Markus Hagenbuchner, and Gabriele Monfardini. 2008. The graph neural network model. IEEE Transactions on Neural Networks 20, 1 (2008), 61--80.Google Scholar
Digital Library
- Lixin Shi, Haitham Hassanieh, Abe Davis, Dina Katabi, and Fredo Durand. 2014. Light field reconstruction using sparsity in the continuous fourier domain. ACM Transactions on Graphics (TOG) 34, 1 (2014), 12.Google Scholar
Digital Library
- Irene Viola, Hermina Petric Maretic, Pascal Frossard, and Touradj Ebrahimi. 2018. A graph learning approach for light field image compression. Applications of Digital Image Processing XLISpie-Int Soc Optical Engineering (2018), 12.Google Scholar
- Gaochang Wu, Belen Masia, Adrian Jarabo, Yuchen Zhang, Liangyong Wang, Qionghai Dai, Tianyou Chai, and Yebin Liu. 2017. Light field image processing: An overview. IEEE Journal of Selected Topics in Signal Processing 11, 7 (2017), 926--954.Google Scholar
Cross Ref
- Shan Xu, Zhi-Liang Zhou, and Nicholas Devaney. 2014. Multi-view image restoration from plenoptic raw images. In Asian Conference on Computer Vision. Springer, 3--15.Google Scholar
- Wei Zhang, Dong Liu, Zhiwei Xiong, and Jizheng Xu. 2018. SIFT-based adaptive prediction structure for light field compression. In Visual Communications and Image Processing. 1--4.Google Scholar
- Xiang Zhang, Philip A. Chou, Ming Ting Sun, Maolong Tang, Shanshe Wang, Siwei Ma, and Wen Gao. 2018. Surface light field compression using a point cloud codec. IEEE Journal on Emerging and Selected Topics in Circuits and Systems 9, 1 (2018), 163--176.Google Scholar
Cross Ref
Index Terms
An Adaptive Two-Layer Light Field Compression Scheme Using GNN-Based Reconstruction
Recommendations
Adaptive two-layer light field compression scheme based on sparse reconstruction
MMSys '19: Proceedings of the 10th ACM Multimedia Systems ConferenceAs a new form of volumetric media, the technology of light field and its compression has gradually become the research hotspots in academia. The scheme of compressing using the sparsity of the light field is a very promising idea, which has the ...
Layered Light Field Reconstruction for Defocus Blur
We present a novel algorithm for reconstructing high-quality defocus blur from a sparsely sampled light field. Our algorithm builds upon recent developments in the area of sheared reconstruction filters and significantly improves reconstruction quality ...
Hybrid linear weighted prediction and intra block copy based light field image coding
Light field imaging can capture both spatial and angular information of a 3D scene and is considered as a prospective acquisition and display solution to supply a more natural and fatigue-free 3D visualization. However, one problem that occupies an ...






Comments