skip to main content
research-article

Efficient Light Field Image Compression with Enhanced Random Access

Authors Info & Claims
Published:23 March 2022Publication History
Skip Abstract Section

Abstract

In light field image compression, facilitating random access to individual views plays a significant role in decoding views quickly, reducing memory footprint, and decreasing the bandwidth requirement for transmission. Highly efficient light field image compression methods mainly use inter view prediction. Therefore, they typically do not provide random access to individual views. On the other hand, methods that provide full random access usually reduce compression efficiency. To address this trade-off, a light field image encoding method that favors random access is proposed in this paper. Light field image views are grouped into independent (3× 3) views, which are called Macro View Images (MVIs). To encode MVIs, the central view is used as a reference to compress its adjacent neighboring views using a hierarchical reference structure. To encode the central view of each MVI, the most central view along with the center of a maximum of three MVIs, are used as reference images for the disparity estimation. In addition, the proposed method allows the use of parallel processing to reduce the maximum encoding/decoding time-complexity in multi-core processors. Tile partitioning can also be used to randomly access different regions of the light field images. The simulation results show that the proposed method outperforms other state-of-the-art methods in terms of compression efficiency while providing random access to both views and regions of interest.

REFERENCES

  1. [1] Aggoun A.. 2011. Compression of 3D integral images using 3D wavelet transform. Journal of Display Technology 7, 11 (2011), 586592. DOI:Google ScholarGoogle ScholarCross RefCross Ref
  2. [2] Ahmad W., Olsson R., and Sjostrom M.. 2017. Interpreting plenoptic images as multi-view sequences for improved compression. In 2017 IEEE International Conference on Image Processing (ICIP). 45574561.Google ScholarGoogle ScholarDigital LibraryDigital Library
  3. [3] Ahmad W., Vagharshakyan S., Sjostrom M., Gotchev A., Bregovic R., and Olsson R.. 2018. Shearlet transform based prediction scheme for light field compression. In 2018 Data Compression Conference. 396396. DOI:Google ScholarGoogle ScholarCross RefCross Ref
  4. [4] Amirpour H., Pereira M., and Pinheiro A.. 2018. High efficient snake order pseudo-sequence based light field image compression. In 2018 Data Compression Conference. 397397. DOI:Google ScholarGoogle ScholarCross RefCross Ref
  5. [5] Amirpour H., Pinheiro A., Pereira M., Lopes F., and Ghanbari M.. 2019. Light field image compression with random access. In 2019 Data Compression Conference (DCC). 553553. DOI:Google ScholarGoogle ScholarCross RefCross Ref
  6. [6] Amirpour Hadi, Pinheiro António M. G., Pereira Manuela, and Ghanbari Mohammad. 2019. Fast and efficient lenslet image compression. CoRR abs/1901.11396 (2019). arxiv:1901.11396 http://arxiv.org/abs/1901.11396.Google ScholarGoogle Scholar
  7. [7] Amirpour H., Pinheiro A. M. G., Pereira M., and Ghanbari M.. 2019. Reliability of the most common objective metrics for light field quality assessment. In ICASSP 2019-2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). 24022406. DOI:Google ScholarGoogle ScholarCross RefCross Ref
  8. [8] Astola P. and Tabus I.. 2018. WaSP: Hierarchical warping, merging, and sparse prediction for light field image compression. In The 7th European Workshop on Visual Information Processing (EUVIP). 435439. DOI:Google ScholarGoogle ScholarCross RefCross Ref
  9. [9] Avramelos Vasileios, Praeter Johan De, Wallendael Glenn Van, and Lambert Peter. 2020. Random access prediction structures for light field video coding with MV-HEVC. Multimedia Tools and Applications (2020), 121.Google ScholarGoogle Scholar
  10. [10] Bjontegaard G.. 2001. Calculation of average PSNR differences between RD-curves (VCEG-M33). (April 2001).Google ScholarGoogle Scholar
  11. [11] BT.709-6 ITU-R. 2015. Parameter values for the HDTV standards for production and international programme exchange. International Telecommunication Union.Google ScholarGoogle Scholar
  12. [12] Zhang Cha and Li Jin. 2000. Compression of lumigraph with multiple reference frame (MRF) prediction and just-in-time rendering. In Proceedings DCC 2000. Data Compression Conference. 253262. DOI:Google ScholarGoogle ScholarCross RefCross Ref
  13. [13] Chen Bin, Ruan Lingyan, and Lam Miu-Ling. 2020. LFGAN: 4D light field synthesis from a single RGB image. ACM Trans. Multimedia Comput. Commun. Appl. 16, 1, Article 2 (Feb. 2020), 20 pages. DOI:Google ScholarGoogle ScholarDigital LibraryDigital Library
  14. [14] Conti C., Soares L. D., and Nunes P.. 2018. Light field coding with field-of-view scalability and exemplar-based interlayer prediction. IEEE Transactions on Multimedia 20, 11 (2018), 29052920.Google ScholarGoogle ScholarDigital LibraryDigital Library
  15. [15] Conti C., Soares L. D., and Nunes P.. 2020. Dense light field coding: A survey. IEEE Access 8 (2020), 4924449284. DOI:Google ScholarGoogle ScholarCross RefCross Ref
  16. [16] Dai F., Zhang J., Ma Y., and Zhang Y.. 2015. Lenslet image compression scheme based on subaperture images streaming. In 2015 IEEE International Conference on Image Processing (ICIP). 47334737.Google ScholarGoogle ScholarDigital LibraryDigital Library
  17. [17] Dansereau D. G., Pizarro O., and Williams S. B.. 2013. Decoding, calibration and rectification for lenselet-based plenoptic cameras. In 2013 IEEE Conference on Computer Vision and Pattern Recognition. 10271034. DOI:Google ScholarGoogle ScholarDigital LibraryDigital Library
  18. [18] Carvalho M. B. de, Pereira M. P., Alves G., Silva E. A. B. da, Pagliari C. L., Pereira F., and Testoni V.. 2018. A 4D DCT-based lenslet light field codec. In 2018 25th IEEE International Conference on Image Processing (ICIP). 435439. DOI:Google ScholarGoogle ScholarCross RefCross Ref
  19. [19] Dib E., Pendu M. L., and Guillemot C.. 2019. Light field compression using fourier disparity layers. In 2019 IEEE International Conference on Image Processing (ICIP). 37513755. DOI:Google ScholarGoogle ScholarCross RefCross Ref
  20. [20] Ebrahimi T., Foessel S., Pereira F., and Schelkens P.. 2016. JPEG pleno: Toward an efficient representation of visual reality. IEEE MultiMedia 23, 4 (2016), 1420. DOI:Google ScholarGoogle ScholarDigital LibraryDigital Library
  21. [21] Elias Vitor R. M. and Martins W.. 2018. On the use of graph fourier transform for light-field compression. Journal of Communication and Information Systems 33 (2018), 92103.Google ScholarGoogle ScholarCross RefCross Ref
  22. [22] Georgiev Todor, Yu Zhan, Lumsdaine Andrew, and Goma Sergio. 2013. Lytro camera technology: Theory, algorithms, performance analysis. In Multimedia Content and Mobile Devices(Society of Photo-Optical Instrumentation Engineers (SPIE) Conference Series, Vol. 8667), Snoek Cees G. M., Wller Dietmar, Creutzburg Reiner, Georgiev Todor G., Lumsdaine Andrew, Akopian David, Matherson Kevin J., Sebe Nicu, and Kennedy Lyndon S. (Eds.). Article 86671J, 86671J pages. DOI:Google ScholarGoogle ScholarCross RefCross Ref
  23. [23] Honauer Katrin, Johannsen Ole, Kondermann Daniel, and Goldluecke Bastian. 2017. A dataset and evaluation methodology for depth estimation on 4D light fields. In Computer Vision – ACCV 2016, Lai Shang-Hong, Lepetit Vincent, Nishino Ko, and Sato Yoichi (Eds.). Springer International Publishing, Cham, 1934.Google ScholarGoogle Scholar
  24. [24] Hu Xinjue, Shan Jingming, Liu Yu, Zhang Lin, and Shirmohammadi Shervin. 2020. An adaptive two-layer light field compression scheme using GNN-based reconstruction. ACM Trans. Multimedia Comput. Commun. Appl. 16, 2s, Article 72 (June 2020), 23 pages. DOI:Google ScholarGoogle ScholarDigital LibraryDigital Library
  25. [25] Ihrke I., Restrepo J., and Mignard-Debise L.. 2016. Principles of light field imaging: Briefly revisiting 25 years of research. IEEE Signal Processing Magazine 33, 5 (Sept 2016), 5969.Google ScholarGoogle ScholarCross RefCross Ref
  26. [26] Jayaweera S. S., Edussooriya C. U. S., Wijenayake C., Agathoklis P., and Bruton L. T.. 2021. Multi-volumetric refocusing of light fields. IEEE Signal Processing Letters 28 (2021), 3135. DOI:Google ScholarGoogle ScholarCross RefCross Ref
  27. [27] JTC1/SC29/WG1 ISO/IEC. [n.d.]. Verification model software version 2.1 on JPEG Pleno light field coding, ISO/IEC JTC1/SC29/WG1, Geneva, Switzerland, Tech. Rep. ISO/IEC JTC1/SC29/WG1N83034, 2019.Google ScholarGoogle Scholar
  28. [28] Kalantari Nima Khademi, Wang Ting-Chun, and Ramamoorthi Ravi. 2016. Learning-based view synthesis for light field cameras. ACM Trans. Graph. 35, 6, Article 193 (Nov. 2016), 10 pages. DOI:Google ScholarGoogle ScholarDigital LibraryDigital Library
  29. [29] Kang Ho-Hyun, Shin Dong-Hak, and Kim Eun-Soo. 2008. Compression scheme of sub-images using Karhunen-Loeve transform in three-dimensional integral imaging. Optics Communications 281, 14 (2008), 36403647. DOI:Google ScholarGoogle ScholarCross RefCross Ref
  30. [30] Li L., Li Z., Li B., Liu D., and Li H.. 2017. Pseudo sequence based 2-D hierarchical coding structure for light-field image compression. In 2017 Data Compression Conference (DCC). 131140.Google ScholarGoogle ScholarCross RefCross Ref
  31. [31] Liu D., An P., Ma R., Zhan W., Huang X., and Yahya A. A.. 2020. Content-based light field image compression method with Gaussian process regression. IEEE Transactions on Multimedia 22, 4 (2020), 846859.Google ScholarGoogle ScholarDigital LibraryDigital Library
  32. [32] Liu D., Wang L., Li L., Xiong Zhiwei, Wu Feng, and Zeng Wenjun. 2016. Pseudo-sequence-based light field image compression. In 2016 IEEE International Conference on Multimedia Expo Workshops (ICMEW). 14.Google ScholarGoogle ScholarCross RefCross Ref
  33. [33] Lumsdaine A. and Georgiev T.. 2009. The focused plenoptic camera. In 2009 IEEE International Conference on Computational Photography (ICCP). 18. DOI:Google ScholarGoogle ScholarCross RefCross Ref
  34. [34] Magnor M. and Girod B.. 2000. Data compression for light-field rendering. IEEE Transactions on Circuits and Systems for Video Technology 10, 3 (2000), 338343. DOI:Google ScholarGoogle ScholarDigital LibraryDigital Library
  35. [35] Misra K., Segall A., Horowitz M., Xu S., Fuldseth A., and Zhou M.. 2013. An overview of tiles in HEVC. IEEE Journal of Selected Topics in Signal Processing 7, 6 (2013), 969977.Google ScholarGoogle ScholarCross RefCross Ref
  36. [36] Ng Ren, Levoy Marc, Brédif Mathieu, Duval Gene, Horowitz Mark, and Hanrahan Pat. 2005. Light Field Photography with a Hand-held Plenoptic Camera. Research Report CSTR 2005-02. Stanford University. Stanford University Computer Science Tech Report pages. https://hal.archives-ouvertes.fr/hal-02551481.Google ScholarGoogle Scholar
  37. [37] Peng J., Xiong Z., Wang Y., Zhang Y., and Liu D.. 2020. Zero-shot depth estimation from light field using a convolutional neural network. IEEE Transactions on Computational Imaging 6 (2020), 682696. DOI:Google ScholarGoogle ScholarCross RefCross Ref
  38. [38] Pereira F., Pagliari C., Silva E. da, Tabus I., Amirpour H., Bernardo M., and Pinheiro A.. 2018. JPEG pleno light field coding common test conditions. ISO/IEC JTC 1/SC29/WG1N81022 (2018).Google ScholarGoogle Scholar
  39. [39] Perra C. and Assuncao P.. 2016. High efficiency coding of light field images based on tiling and pseudo-temporal data arrangement. In 2016 IEEE International Conference on Multimedia Expo Workshops (ICMEW). 14.Google ScholarGoogle ScholarCross RefCross Ref
  40. [40] Pratapa Srihari and Manocha Dinesh. 2018. RLFC: Random access light field compression using key views. CoRR abs/1805.06019 (2018). arxiv:1805.06019 http://arxiv.org/abs/1805.06019Google ScholarGoogle Scholar
  41. [41] Pratapa S. and Manocha D.. 2019. HMLFC: Hierarchical motion-compensated light field compression for interactive rendering. Computer Graphics Forum 38, 8 (2019), 112. DOI:arXiv:https://onlinelibrary.wiley.com/doi/pdf/10.1111/cgf.13755Google ScholarGoogle ScholarCross RefCross Ref
  42. [42] Rerabek M., Yuan L., Authier L. A., and Ebrahimi T.. 2015. EPFL light-field image dataset. (2015).Google ScholarGoogle Scholar
  43. [43] Srinivasan P. P., Wang T., Sreelal A., Ramamoorthi R., and Ng R.. 2017. Learning to synthesize a 4D RGBD light field from a single image. In 2017 IEEE International Conference on Computer Vision (ICCV). 22622270. DOI:Google ScholarGoogle ScholarCross RefCross Ref
  44. [44] Sullivan Gary J., Ohm Jens-Rainer, Han Woo-Jin, and Wiegand Thomas. 2012. Overview of the high efficiency video coding (HEVC) standard. IEEE Trans. Cir. and Sys. for Video Technol. 22, 12 (Dec. 2012), 16491668.Google ScholarGoogle ScholarDigital LibraryDigital Library
  45. [45] Tech G., Chen Y., Müller K., Ohm J. R., Vetro A., and Wang Y. K.. 2016. Overview of the multiview and 3D extensions of high efficiency video coding. IEEE Transactions on Circuits and Systems for Video Technology 26, 1 (Jan 2016), 3549. DOI:Google ScholarGoogle ScholarDigital LibraryDigital Library
  46. [46] Vieira A., Duarte H., Perra C., Tavora L., and Assuncao P.. 2015. Data formats for high efficiency coding of Lytro-Illum light fields. In 2015 International Conference on Image Processing Theory, Tools and Applications (IPTA). 494497.Google ScholarGoogle ScholarCross RefCross Ref
  47. [47] Viola Irene and Ebrahimi Touradj. 2018. VALID: Visual quality assessment for light field images dataset. (2018), 3.Google ScholarGoogle Scholar
  48. [48] Wu G., Masia B., Jarabo A., Zhang Y., Wang L., Dai Q., Chai T., and Liu Y.. 2017. Light field image processing: An overview. IEEE Journal of Selected Topics in Signal Processing 11, 7 (Oct 2017), 926954.Google ScholarGoogle Scholar
  49. [49] Zhang Jun, Wang Meng, Lin Liang, Yang Xun, Gao Jun, and Rui Yong. 2017. Saliency detection on light field: A multi-cue approach. ACM Trans. Multimedia Comput. Commun. Appl. 13, 3, Article 32 (July 2017), 22 pages. DOI:Google ScholarGoogle ScholarDigital LibraryDigital Library
  50. [50] Zhao S. and Chen Z.. 2017. Light field image coding via linear approximation prior. In 2017 IEEE International Conference on Image Processing (ICIP). 45624566.Google ScholarGoogle ScholarDigital LibraryDigital Library
  51. [51] Xiaoran Jiang, Mikaël Le Pendu, and Christine Guillemot. 2017. Light field compression using depth image based view synthesis. In 2017 IEEE International Conference on Multimedia Expo Workshops (ICMEW’17). 19–24. DOI:Google ScholarGoogle ScholarCross RefCross Ref
  52. [52] Junhui Hou, Jie Chen, and Lap-Pui Chau. 2019. Light field image compression based on bi-level view compensation with rate-distortion Optimization. IEEE Transactions on Circuits and Systems for Video Technology 29, 2 (2019), 517–530. DOI:Google ScholarGoogle ScholarDigital LibraryDigital Library
  53. [53] Chuanmin Jia, Xinfeng Zhang, Shanshe Wang, Shiqi Wang, and Siwei Ma. 2019. Light field image compression using generative adversarial network-based view synthesis. IEEE Journal on Emerging and Selected Topics in Circuits and Systems 9, 1 (2019), 177–189. DOI:Google ScholarGoogle ScholarCross RefCross Ref

Index Terms

(auto-classified)
  1. Efficient Light Field Image Compression with Enhanced Random Access

        Recommendations

        Comments

        Login options

        Check if you have access through your login credentials or your institution to get full access on this article.

        Sign in

        Full Access

        • Published in

          cover image ACM Transactions on Multimedia Computing, Communications, and Applications
          ACM Transactions on Multimedia Computing, Communications, and Applications  Volume 18, Issue 2
          May 2022
          494 pages
          ISSN:1551-6857
          EISSN:1551-6865
          DOI:10.1145/3505207
          Issue’s Table of Contents

          Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

          Publisher

          Association for Computing Machinery

          New York, NY, United States

          Publication History

          • Published: 23 March 2022
          • Revised: 1 May 2021
          • Accepted: 1 May 2021
          • Received: 1 August 2020
          Published in tomm Volume 18, Issue 2

          Permissions

          Request permissions about this article.

          Request Permissions

          Check for updates

          Qualifiers

          • research-article
          • Refereed

        PDF Format

        View or Download as a PDF file.

        PDF

        eReader

        View online with eReader.

        eReader

        Full Text

        View this article in Full Text.

        View Full Text

        HTML Format

        View this article in HTML Format .

        View HTML Format
        About Cookies On This Site

        We use cookies to ensure that we give you the best experience on our website.

        Learn more

        Got it!