Femto-photography: capturing and visualizing the propagation of light

Abstract

We present femto-photography, a novel imaging technique to capture and visualize the propagation of light. With an effective exposure time of 1.85 picoseconds (ps) per frame, we reconstruct movies of ultrafast events at an equivalent resolution of about one half trillion frames per second. Because cameras with this shutter speed do not exist, we re-purpose modern imaging hardware to record an ensemble average of repeatable events that are synchronized to a streak sensor, in which the time of arrival of light from the scene is coded in one of the sensor's spatial dimensions. We introduce reconstruction methods that allow us to visualize the propagation of femtosecond light pulses through macroscopic scenes; at such fast resolution, we must consider the notion of time-unwarping between the camera's and the world's space-time coordinate systems to take into account effects associated with the finite speed of light. We apply our femto-photography technique to visualizations of very different scenes, which allow us to observe the rich dynamics of time-resolved light transport effects, including scattering, specular reflections, diffuse interreflections, diffraction, caustics, and subsurface scattering. Our work has potential applications in artistic, educational, and scientific visualizations; industrial imaging to analyze material properties; and medical imaging to reconstruct subsurface elements. In addition, our time-resolved technique may motivate new forms of computational photography.

References

1. Abramson, N. 1978. Light-in-flight recording by holography. Optics Letters 3, 4, 121--123.
2. Busck, J., and Heiselberg, H. 2004. Gated viewing and high-accuracy three-dimensional laser radar. Applied optics 43, 24, 4705--4710.
3. Campillo, A., and Shapiro, S. 1987. Picosecond streak camera fluorometry: a review. IEEE Journal of Quantum Electronics 19, 4, 585--603.
4. Charbon, E. 2007. Will avalanche photodiode arrays ever reach 1 megapixel? In International Image Sensor Workshop, 246--249.
5. Colaço, A., Kirmani, A., Howland, G. A., Howell, J. C., and Goyal, V. K. 2012. Compressive depth map acquisition using a single photon-counting detector: Parametric signal processing meets sparsity. In IEEE Computer Vision and Pattern Recognition, CVPR 2012, 96--102.
6. Duguay, M. A., and Mattick, A. T. 1971. Pulsed-image generation and detection. Applied Optics 10, 2162--2170.
7. Faro, 2012. Faro Technologies Inc.: Measuring Arms. http://www.faro.com.
8. Gbur, G., 2012. A camera fast enough to watch light move? http://skullsinthestars.com/2012/01/04/a-camera-fast-enough-to-watch-light-move/.
9. Gelbart, A., Redman, B. C., Light, R. S., Schwartzlow, C. A., and Griffis, A. J. 2002. Flash lidar based on multiple-slit streak tube imaging lidar. SPIE, vol. 4723, 9--18.
10. Goda, K., Tsia, K. K., and Jalali, B. 2009. Serial time-encoded amplified imaging for real-time observation of fast dynamic phenomena. Nature 458, 1145--1149.
11. Gupta, O., Willwacher, T., Velten, A., Veeraraghavan, A., and Raskar, R. 2012. Reconstruction of hidden 3D shapes using diffuse reflections. Optics Express 20, 19096--19108.
12. Hamamatsu, 2012. Guide to Streak Cameras. http://sales.hamamatsu.com/assets/pdf/catsandguides/e_streakh.pdf.
13. Hebden, J. C. 1993. Line scan acquisition for time-resolved imaging through scattering media. Opt. Eng. 32, 3, 626--633.
14. Heide, F., Hullin, M., Gregson, J., and Heidrich, W. 2013. Low-budget transient imaging using photonic mixer devices. ACM Trans. Graph. 32, 4.
15. Huang, D., Swanson, E., Lin, C., Schuman, J., Stinson, W., Chang, W., Hee, M., Flotte, T., Gregory, K., and Puliafito, C. 1991. Optical coherence tomography. Science 254, 5035, 1178--1181.
16. Itatani, J., Quéré, F., Yudin, G. L., Ivanov, M. Y., Krausz, F., and Corkum, P. B. 2002. Attosecond streak camera. Phys. Rev. Lett. 88, 173903.
17. Jarabo, A., Masia, B., and Gutierrez, D. 2013. Transient rendering and relativistic visualization. Tech. Rep. TR-01-2013, Universidad de Zaragoza, April.
18. Kirmani, A., Hutchison, T., Davis, J., and Raskar, R. 2011. Looking around the corner using ultrafast transient imaging. International Journal of Computer Vision 95, 1, 13--28.
19. Kodama, R., Okada, K., and Kato, Y. 1999. Development of a two-dimensional space-resolved high speed sampling camera. Rev. Sci. Instrum. 70, 625.
20. Naik, N., Zhao, S., Velten, A., Raskar, R., and Bala, K. 2011. Single view reflectance capture using multiplexed scattering and time-of-flight imaging. ACM Trans. Graph. 30, 6, 171:1--171:10.
21. Pandharkar, R., Velten, A., Bardagjy, A., Bawendi, M., and Raskar, R. 2011. Estimating motion and size of moving non-line-of-sight objects in cluttered environments. In IEEE Computer Vision and Pattern Recognition, CVPR 2011, 265--272.
22. Qu, J., Liu, L., Chen, D., Lin, Z., Xu, G., Guo, B., and Niu, H. 2006. Temporally and spectrally resolved sampling imaging with a specially designed streak camera. Optics Letters 31, 368--370.
23. Raskar, R., and Davis, J. 2008. 5D time-light transport matrix: What can we reason about scene properties? Tech. rep., MIT.
24. Shiraga, H., Heya, M., Maegawa, O., Shimada, K., Kato, Y., Yamanaka, T., and Nakai, S. 1995. Laser-imploded core structure observed by using two-dimensional x-ray imaging with 10-ps temporal resolution. Rev. Sci. Instrum. 66, 1, 722--724.
25. Tou, T. Y. 1995. Multislit streak camera investigation of plasma focus in the steady-state rundown phase. IEEE Trans. Plasma Science 23, 870--873.
26. Velten, A., Fritz, A., Bawendi, M. G., and Raskar, R. 2012. Multibounce time-of-flight imaging for object reconstruction from indirect light. In Conference for Lasers and Electro-Optics, OSA, CM2F.5.
27. Velten, A., Willwacher, T., Gupta, O., Veeraraghavan, A., Bawendi, M. G., and Raskar, R. 2012. Recovering three-dimensional shape around a corner using ultrafast time-of-flight imaging. Nature Communications 3, 745.
28. Velten, A., Wu, D., Jarabo, A., Masia, B., Barsi, C., Lawson, E., Joshi, C., Gutierrez, D., Bawendi, M. G., and Raskar, R. 2012. Relativistic ultrafast rendering using time-of-flight imaging. In ACM SIGGRAPH Talks.
29. Wu, D., O'Toole, M., Velten, A., Agrawal, A., and Raskar, R. 2012. Decomposing global light transport using time of flight imaging. In IEEE Computer Vision and Pattern Recognition, CVPR 2012, IEEE, 366--373.
30. Wu, D., Wetzstein, G., Barsi, C., Willwacher, T., O'Toole, M., Naik, N., Dai, Q., Kutulakos, K., and Raskar, R. 2012. Frequency analysis of transient light transport with applications in bare sensor imaging. In European Conference on Computer Vision, ECCV 2012, Springer, 542--555.
31. Wyant, J. C. 2002. White light interferometry. In SPIE, vol. 4737, 98--107.
32. Xia, H., and Zhang, C. 2009. Ultrafast ranging lidar based on real-time Fourier transformation. Optics Letters 34, 2108--2110.

Index Terms

• Computing methodologies

  • Computer graphics

    • Image manipulation