skip to main content
article

Light field microscopy

Published: 01 July 2006 Publication History

Abstract

By inserting a microlens array into the optical train of a conventional microscope, one can capture light fields of biological specimens in a single photograph. Although diffraction places a limit on the product of spatial and angular resolution in these light fields, we can nevertheless produce useful perspective views and focal stacks from them. Since microscopes are inherently orthographic devices, perspective views represent a new way to look at microscopic specimens. The ability to create focal stacks from a single photograph allows moving or light-sensitive specimens to be recorded. Applying 3D deconvolution to these focal stacks, we can produce a set of cross sections, which can be visualized using volume rendering. In this paper, we demonstrate a prototype light field microscope (LFM), analyze its optical performance, and show perspective views, focal stacks, and reconstructed volumes for a variety of biological specimens. We also show that synthetic focusing followed by 3D deconvolution is equivalent to applying limited-angle tomography directly to the 4D light field.

Supplementary Material

JPG File (p924-levoy-high.jpg)
JPG File (p924-levoy-low.jpg)
High Resolution (p924-levoy-high.mov)
Low Resolution (p924-levoy-low.mov)

References

[1]
Adelson, T., Wang, J. Y. A. 1992. Single lens stereo with a plenoptic camera. IEEE Transactions on Pattern Analysis and Machine Intelligence 14, 2, 99--106.
[2]
Agard, D. A. 1984. Optical sectioning microscopy: Cellular architecture in three dimensions. Ann. Rev. Biophys. Bioeng 13, 191--219.
[3]
Andersen, A. H., Kak, A. C., 1984. Simultaneous algebraic reconstruction technique (SART): A superior implementation of the ART algorithm. Ultrasonic Imaging 6, 81--94.
[4]
Arridge, S. R. 2001. Methods for the inverse problem in optical tomography. Proc. Waves and Imaging Through Complex Media. Kluwer, 307--329.
[5]
Castleman, K. R. 1979. Digital Image Processing. Prentice Hall.
[6]
Chamgoulov, R. O., Lane, P. M., Macaulay, C. E. 2004. Optical computed-tomography microscope using digital spatial light modulation. Proc. SPIE 5324, 182--190.
[7]
Colsher, J. G. 1980. Fully three-dimensional positron emission tomography. Phys. Med. Biol. 25, 1, 103--115.
[8]
Corle, T. R., Kino, G. S. 1996. Confocal Scanning Optical Microscopy and Related Imaging Systems. Academic Press.
[9]
Ellis, G. W. 1966. Holomicrography: transformation of image during reconstruction a posteriori. Science 143, 1195--1196.
[10]
Goldberg, N. 1992. Camera technology: the dark side of the lens. Academic Press.
[11]
Goodman, J. 1996. Introduction to Fourier optics. 2nd edition, McGraw-Hill.
[12]
Gustafsson, M. G. L. 2005. Nonlinear structured-illumination microscopy: Wide-field fluorescence imaging with theoretically unlimited resolution. Proc. National Academy of Sciences 102, 37.
[13]
Holmes, T. J., Bhattacharyya, S., et al. 1995. Light microscopic images reconstructed by maximum likelihood deconvolution. In Handbook of Biological Confocal Microscopy, ed. J. B. Pawley, Plenum Press, 389--402.
[14]
Inoue, S., Oldenbourg, R. 1995. Microscopes. In Handbook of Optics, 2nd edition, McGraw-Hill.
[15]
Inoue, S. and Spring, K. R. 1997. Video Microscopy. 2nd edition, Plenum Press.
[16]
Gabor, D. 1948. A new microscopic principle. Nature 161, 777--778.
[17]
Isaksen, A., Mcmillan, L., Gortler, S. J. 2000. Dynamically reparameterized light fields. Proc. SIGGRAPH 2000.
[18]
Javidi, B., Okano, F., eds. 2002. Three-Dimensional Television, Video and Display Technologies. Springer-Verlag.
[19]
Kak, A. C., Slaney, M. 1988. Principles of Computerized Tomographic Imaging. IEEE Press.
[20]
Kawata, S., Nakamura, 0., Minami, S. 1987. Optical microscope tomography. I. Support constraint. J. Opt. Soc. Am. A 4, 1, 292--297.
[21]
Kingslake, R. 1983. Optical system design. Academic Press.
[22]
Levoy, M., Hanrahan, P. 1996. Light field rendering. Proc. SIGGRAPH 1996.
[23]
Levoy, M., Chen, B., Vaish, V., Horowitz, M., Mcdowall, I., Bolas, M. 2004. Synthetic aperture confocal imaging. ACM Transactions on Graphics (Proc. SIGGRAPH) 23, 3, 825--834.
[24]
Markham, J., Conchello, J.-A. 2001. Artefacts in restored images due to intensity loss in three-dimensional fluorescence microscopy. J. Microscopy 204, 2, 93--98.
[25]
Mcnally, J. G., Preza, C., Conchello, J. A., Thomas, L. J. Jr. 1994. Artifacts in computational optical-sectioning microscopy. J. Opt. Soc. Am. A 11, 3, 1056--67.
[26]
Nayar, S. K., Nakagawa, Y. 1990. Shape from focus: An effective approach for rough surfaces. Proc. International Conference on Robotics and Automation (ICRA), Vol. 2, 218--225.
[27]
Nayar, S. K., Narasimhan, S. G. 2002. Assorted pixels: Multi-sampled imaging with structural models. Proc. ECCV.
[28]
Ng, R. 2005. Fourier slice photography. ACM Transactions on Graphics (Proc. SIGGRAPH) 24, 3, 735--744.
[29]
Ng, R., Levoy, M., Bredif, M., Duval, G., Horowitz, M., Hanrahan, P. 2005. Light Field Photography with a Hand-Held Plenoptic Camera. Stanford Tech Report CTSR 2005-02.
[30]
Ng, R. 2006. Digital Light Field Photography. PhD dissertation, Stanford University.
[31]
Noguchi, M., Nayar, S. 1994. Microscopic shape from focus using active illumination. Proc. IAPR International Conference on Pattern Recognition (ICPR), Vol. A, 147--152.
[32]
Okoshi, T. 1976. Three-Dimensional Imaging Techniques. Academic Press.
[33]
Piller, H. 1977. Microscope Photometry. Springer-Verlag.
[34]
Pluta, M. 1988. Advanced Light Microscopy (in 3 volumes). Elsevier.
[35]
Schechner, Y., Kiryati, N. 1999. The optimal axial interval for estimating depth from defocus. Proc. ICCV.
[36]
Schechner, Y., Kiryati, N. 2000. Depth from defocus vs. stereo: How different really are they? IJCV 39, 2, 141--162.
[37]
Schechner, Y. Y., Kiryati, N., Basri, R. 2000. Separation of transparent layers using focus. IJCV 39, 1, 25--39.
[38]
Schechner, Y., Nayar, S. 2001. Generalized Mosaicing. Proc. ICCV.
[39]
Shah, U. B., Nayar, S. K. 1992. Extracting 3-D structure and focused images using an optical microscope. Proc. IEEE Symposium on Computer-Based Medical Systems.
[40]
Streibl, N. 1984. Depth transfer by an imaging system. Optica Acta 31, 11, 1233--1241.
[41]
Streibl, N. 1985. Three-dimensional imaging by a microscope. J. Opt. Soc. Am. A 2, 2, 121--127.
[42]
Swedlow, J. R., Sedat, J. W., Agard, D. A. 1997. Deconvolution in optical microscopy. In Deconvolution of Images and Spectra, ed. P. A. Jansson, Academic Press, 284--309.
[43]
Vaish, V., Garg, G., Talvala, E., Antunez, E., Wilburn, B., Horowitz, M., Levoy, M., Synthetic aperture focusing using a shear-warp factorization of the viewing transform. Proc. Workshop on Advanced 3D Imaging for Safety and Security, in conjunction with CVPR 2005.
[44]
Weinstein, R. S., Descour, M. R., et al. 2004. An array microscope for ultrarapid virtual slide processing and telepathology. Design, fabrication, and validation study. Human Pathology 35, 11, 1303--1314.

Cited By

View all
  • (2025)Random Illumination Microscopy: faster, thicker, and aberration-insensitiveLight: Science & Applications10.1038/s41377-024-01687-914:1Online publication date: 2-Jan-2025
  • (2025)Filtered back projection-deconvolution method for light field reconstruction from the focal stackOptics and Lasers in Engineering10.1016/j.optlaseng.2024.108576184(108576)Online publication date: Jan-2025
  • (2024)中国光学十大进展:深度学习赋能的高通量荧光显微成像技术(特邀)‡Laser & Optoelectronics Progress10.3788/LOP23254961:16(1600001)Online publication date: 2024
  • Show More Cited By

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Transactions on Graphics
ACM Transactions on Graphics  Volume 25, Issue 3
July 2006
742 pages
ISSN:0730-0301
EISSN:1557-7368
DOI:10.1145/1141911
Issue’s Table of Contents
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 01 July 2006
Published in TOG Volume 25, Issue 3

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. deconvolution
  2. light fields
  3. microscopy
  4. synthetic aperture
  5. tomography
  6. volume rendering

Qualifiers

  • Article

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)265
  • Downloads (Last 6 weeks)33
Reflects downloads up to 09 Jan 2025

Other Metrics

Citations

Cited By

View all
  • (2025)Random Illumination Microscopy: faster, thicker, and aberration-insensitiveLight: Science & Applications10.1038/s41377-024-01687-914:1Online publication date: 2-Jan-2025
  • (2025)Filtered back projection-deconvolution method for light field reconstruction from the focal stackOptics and Lasers in Engineering10.1016/j.optlaseng.2024.108576184(108576)Online publication date: Jan-2025
  • (2024)中国光学十大进展:深度学习赋能的高通量荧光显微成像技术(特邀)‡Laser & Optoelectronics Progress10.3788/LOP23254961:16(1600001)Online publication date: 2024
  • (2024)Differential high-speed aperture-coding light field microscopy for dynamic sample observation with enhanced contrastAdvanced Imaging10.3788/AI.2024.100171:3(031002)Online publication date: 2024
  • (2024)Feature-enhanced fiber bundle imaging based on light field acquisitionAdvanced Imaging10.3788/AI.2024.100021:1(011002)Online publication date: 2024
  • (2024)3D Correlation Imaging for Localized Phase Disturbance MitigationPhotonics10.3390/photonics1108073311:8(733)Online publication date: 6-Aug-2024
  • (2024)Inertial algorithm with dry fraction and convolutional sparse coding for 3D localization with light field microscopyProceedings of the Thirty-Eighth AAAI Conference on Artificial Intelligence and Thirty-Sixth Conference on Innovative Applications of Artificial Intelligence and Fourteenth Symposium on Educational Advances in Artificial Intelligence10.1609/aaai.v38i18.30072(20830-20837)Online publication date: 20-Feb-2024
  • (2024)High-resolution 3D imaging in light-field microscopy through Stokes matrices and data fusionOptics Express10.1364/OE.510728Online publication date: 9-Jan-2024
  • (2024)Fast light-field 3D microscopy with out-of-distribution detection and adaptation through conditional normalizing flowsBiomedical Optics Express10.1364/BOE.50403915:2(1219)Online publication date: 30-Jan-2024
  • (2024)Multi-plane imaging technology with constant imaging qualityApplied Optics10.1364/AO.52843163:24(6456)Online publication date: 16-Aug-2024
  • Show More Cited By

View Options

Login options

Full Access

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Media

Figures

Other

Tables

Share

Share

Share this Publication link

Share on social media