Abstract
In light field image compression, facilitating random access to individual views plays a significant role in decoding views quickly, reducing memory footprint, and decreasing the bandwidth requirement for transmission. Highly efficient light field image compression methods mainly use inter view prediction. Therefore, they typically do not provide random access to individual views. On the other hand, methods that provide full random access usually reduce compression efficiency. To address this trade-off, a light field image encoding method that favors random access is proposed in this paper. Light field image views are grouped into independent (3× 3) views, which are called Macro View Images (MVIs). To encode MVIs, the central view is used as a reference to compress its adjacent neighboring views using a hierarchical reference structure. To encode the central view of each MVI, the most central view along with the center of a maximum of three MVIs, are used as reference images for the disparity estimation. In addition, the proposed method allows the use of parallel processing to reduce the maximum encoding/decoding time-complexity in multi-core processors. Tile partitioning can also be used to randomly access different regions of the light field images. The simulation results show that the proposed method outperforms other state-of-the-art methods in terms of compression efficiency while providing random access to both views and regions of interest.
- [1] . 2011. Compression of 3D integral images using 3D wavelet transform. Journal of Display Technology 7, 11 (2011), 586–592.
DOI: Google ScholarCross Ref
- [2] . 2017. Interpreting plenoptic images as multi-view sequences for improved compression. In 2017 IEEE International Conference on Image Processing (ICIP). 4557–4561.Google Scholar
Digital Library
- [3] . 2018. Shearlet transform based prediction scheme for light field compression. In 2018 Data Compression Conference. 396–396.
DOI: Google ScholarCross Ref
- [4] . 2018. High efficient snake order pseudo-sequence based light field image compression. In 2018 Data Compression Conference. 397–397.
DOI: Google ScholarCross Ref
- [5] . 2019. Light field image compression with random access. In 2019 Data Compression Conference (DCC). 553–553.
DOI: Google ScholarCross Ref
- [6] . 2019. Fast and efficient lenslet image compression. CoRR abs/1901.11396 (2019).
arxiv:1901.11396 http://arxiv.org/abs/1901.11396.Google Scholar - [7] . 2019. Reliability of the most common objective metrics for light field quality assessment. In ICASSP 2019-2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). 2402–2406.
DOI: Google ScholarCross Ref
- [8] . 2018. WaSP: Hierarchical warping, merging, and sparse prediction for light field image compression. In The 7th European Workshop on Visual Information Processing (EUVIP). 435–439.
DOI: Google ScholarCross Ref
- [9] . 2020. Random access prediction structures for light field video coding with MV-HEVC. Multimedia Tools and Applications (2020), 1–21.Google Scholar
- [10] . 2001. Calculation of average PSNR differences between RD-curves (VCEG-M33). (
April 2001).Google Scholar - [11] . 2015. Parameter values for the HDTV standards for production and international programme exchange. International Telecommunication Union.Google Scholar
- [12] . 2000. Compression of lumigraph with multiple reference frame (MRF) prediction and just-in-time rendering. In Proceedings DCC 2000. Data Compression Conference. 253–262.
DOI: Google ScholarCross Ref
- [13] . 2020. LFGAN: 4D light field synthesis from a single RGB image. ACM Trans. Multimedia Comput. Commun. Appl. 16, 1, Article 2 (
Feb. 2020), 20 pages.DOI: Google ScholarDigital Library
- [14] . 2018. Light field coding with field-of-view scalability and exemplar-based interlayer prediction. IEEE Transactions on Multimedia 20, 11 (2018), 2905–2920.Google Scholar
Digital Library
- [15] . 2020. Dense light field coding: A survey. IEEE Access 8 (2020), 49244–49284.
DOI: Google ScholarCross Ref
- [16] . 2015. Lenslet image compression scheme based on subaperture images streaming. In 2015 IEEE International Conference on Image Processing (ICIP). 4733–4737.Google Scholar
Digital Library
- [17] . 2013. Decoding, calibration and rectification for lenselet-based plenoptic cameras. In 2013 IEEE Conference on Computer Vision and Pattern Recognition. 1027–1034.
DOI: Google ScholarDigital Library
- [18] . 2018. A 4D DCT-based lenslet light field codec. In 2018 25th IEEE International Conference on Image Processing (ICIP). 435–439.
DOI: Google ScholarCross Ref
- [19] . 2019. Light field compression using fourier disparity layers. In 2019 IEEE International Conference on Image Processing (ICIP). 3751–3755.
DOI: Google ScholarCross Ref
- [20] . 2016. JPEG pleno: Toward an efficient representation of visual reality. IEEE MultiMedia 23, 4 (2016), 14–20.
DOI: Google ScholarDigital Library
- [21] . 2018. On the use of graph fourier transform for light-field compression. Journal of Communication and Information Systems 33 (2018), 92–103.Google Scholar
Cross Ref
- [22] . 2013. Lytro camera technology: Theory, algorithms, performance analysis. In Multimedia Content and Mobile Devices(
Society of Photo-Optical Instrumentation Engineers (SPIE) Conference Series , Vol. 8667), , , , , , , , , and (Eds.). Article 86671J, 86671J pages.DOI: Google ScholarCross Ref
- [23] . 2017. A dataset and evaluation methodology for depth estimation on 4D light fields. In Computer Vision – ACCV 2016, , , , and (Eds.). Springer International Publishing, Cham, 19–34.Google Scholar
- [24] . 2020. An adaptive two-layer light field compression scheme using GNN-based reconstruction. ACM Trans. Multimedia Comput. Commun. Appl. 16, 2s, Article 72 (
June 2020), 23 pages.DOI: Google ScholarDigital Library
- [25] . 2016. Principles of light field imaging: Briefly revisiting 25 years of research. IEEE Signal Processing Magazine 33, 5 (
Sept 2016), 59–69.Google ScholarCross Ref
- [26] . 2021. Multi-volumetric refocusing of light fields. IEEE Signal Processing Letters 28 (2021), 31–35.
DOI: Google ScholarCross Ref
- [27] . [n.d.]. Verification model software version 2.1 on JPEG Pleno light field coding, ISO/IEC JTC1/SC29/WG1, Geneva, Switzerland, Tech. Rep. ISO/IEC JTC1/SC29/WG1N83034, 2019.Google Scholar
- [28] . 2016. Learning-based view synthesis for light field cameras. ACM Trans. Graph. 35, 6, Article 193 (
Nov. 2016), 10 pages.DOI: Google ScholarDigital Library
- [29] . 2008. Compression scheme of sub-images using Karhunen-Loeve transform in three-dimensional integral imaging. Optics Communications 281, 14 (2008), 3640–3647.
DOI: Google ScholarCross Ref
- [30] . 2017. Pseudo sequence based 2-D hierarchical coding structure for light-field image compression. In 2017 Data Compression Conference (DCC). 131–140.Google Scholar
Cross Ref
- [31] . 2020. Content-based light field image compression method with Gaussian process regression. IEEE Transactions on Multimedia 22, 4 (2020), 846–859.Google Scholar
Digital Library
- [32] . 2016. Pseudo-sequence-based light field image compression. In 2016 IEEE International Conference on Multimedia Expo Workshops (ICMEW). 1–4.Google Scholar
Cross Ref
- [33] . 2009. The focused plenoptic camera. In 2009 IEEE International Conference on Computational Photography (ICCP). 1–8.
DOI: Google ScholarCross Ref
- [34] . 2000. Data compression for light-field rendering. IEEE Transactions on Circuits and Systems for Video Technology 10, 3 (2000), 338–343.
DOI: Google ScholarDigital Library
- [35] . 2013. An overview of tiles in HEVC. IEEE Journal of Selected Topics in Signal Processing 7, 6 (2013), 969–977.Google Scholar
Cross Ref
- [36] . 2005. Light Field Photography with a Hand-held Plenoptic Camera.
Research Report CSTR 2005-02. Stanford University. Stanford University Computer Science Tech Report pages. https://hal.archives-ouvertes.fr/hal-02551481.Google Scholar - [37] . 2020. Zero-shot depth estimation from light field using a convolutional neural network. IEEE Transactions on Computational Imaging 6 (2020), 682–696.
DOI: Google ScholarCross Ref
- [38] . 2018. JPEG pleno light field coding common test conditions. ISO/IEC JTC 1/SC29/WG1N81022 (2018).Google Scholar
- [39] . 2016. High efficiency coding of light field images based on tiling and pseudo-temporal data arrangement. In 2016 IEEE International Conference on Multimedia Expo Workshops (ICMEW). 1–4.Google Scholar
Cross Ref
- [40] . 2018. RLFC: Random access light field compression using key views. CoRR abs/1805.06019 (2018).
arxiv:1805.06019 http://arxiv.org/abs/1805.06019Google Scholar - [41] . 2019. HMLFC: Hierarchical motion-compensated light field compression for interactive rendering. Computer Graphics Forum 38, 8 (2019), 1–12.
DOI: arXiv:https://onlinelibrary.wiley.com/doi/pdf/10.1111/cgf.13755 Google ScholarCross Ref
- [42] . 2015. EPFL light-field image dataset. (2015).Google Scholar
- [43] . 2017. Learning to synthesize a 4D RGBD light field from a single image. In 2017 IEEE International Conference on Computer Vision (ICCV). 2262–2270.
DOI: Google ScholarCross Ref
- [44] . 2012. Overview of the high efficiency video coding (HEVC) standard. IEEE Trans. Cir. and Sys. for Video Technol. 22, 12 (
Dec. 2012), 1649–1668.Google ScholarDigital Library
- [45] . 2016. Overview of the multiview and 3D extensions of high efficiency video coding. IEEE Transactions on Circuits and Systems for Video Technology 26, 1 (
Jan 2016), 35–49.DOI: Google ScholarDigital Library
- [46] . 2015. Data formats for high efficiency coding of Lytro-Illum light fields. In 2015 International Conference on Image Processing Theory, Tools and Applications (IPTA). 494–497.Google Scholar
Cross Ref
- [47] . 2018. VALID: Visual quality assessment for light field images dataset. (2018), 3.Google Scholar
- [48] . 2017. Light field image processing: An overview. IEEE Journal of Selected Topics in Signal Processing 11, 7 (
Oct 2017), 926–954.Google Scholar - [49] . 2017. Saliency detection on light field: A multi-cue approach. ACM Trans. Multimedia Comput. Commun. Appl. 13, 3, Article 32 (
July 2017), 22 pages.DOI: Google ScholarDigital Library
- [50] . 2017. Light field image coding via linear approximation prior. In 2017 IEEE International Conference on Image Processing (ICIP). 4562–4566.Google Scholar
Digital Library
- [51] Xiaoran Jiang, Mikaël Le Pendu, and Christine Guillemot. 2017. Light field compression using depth image based view synthesis. In 2017 IEEE International Conference on Multimedia Expo Workshops (ICMEW’17). 19–24.
DOI: Google ScholarCross Ref
- [52] Junhui Hou, Jie Chen, and Lap-Pui Chau. 2019. Light field image compression based on bi-level view compensation with rate-distortion Optimization. IEEE Transactions on Circuits and Systems for Video Technology 29, 2 (2019), 517–530.
DOI: Google ScholarDigital Library
- [53] Chuanmin Jia, Xinfeng Zhang, Shanshe Wang, Shiqi Wang, and Siwei Ma. 2019. Light field image compression using generative adversarial network-based view synthesis. IEEE Journal on Emerging and Selected Topics in Circuits and Systems 9, 1 (2019), 177–189.
DOI: Google ScholarCross Ref
Index Terms
(auto-classified)Efficient Light Field Image Compression with Enhanced Random Access
Recommendations
HEVC-based lossless intra coding for efficient still image compression
Latest advancements in capture and display technologies demand better compression techniques for the storage and transmission of still images and video. High efficiency video coding (HEVC) is the latest video compression standard developed by the joint ...
Conditional Entropy Coding of VQ Indexes for Image Compression
DCC '97: Proceedings of the Conference on Data CompressionVector quantization (VQ) is a source coding methodology with provable rate-distortion optimality. However, despite more than two decades of intensive research, VQ theoretical promise is yet to be fully realized in image compression practice. Restricted ...
Hierarchical lossless coding of light fields with improved random access
AbstractThe adoption of light field (LF) technology is often hindered by the large amount of data that is necessary to represent such information. To alleviate such burden, specific coding techniques have been under research in recent years. ...
Highlights- Light field lossless hierarchical codec with enhanced stream navigation.
- ...






Comments