3D imaging spectroscopy for measuring hyperspectral patterns on solid objects

Abstract

Sophisticated methods for true spectral rendering have been developed in computer graphics to produce highly accurate images. In addition to traditional applications in visualizing appearance, such methods have potential applications in many areas of scientific study. In particular, we are motivated by the application of studying avian vision and appearance. An obstacle to using graphics in this application is the lack of reliable input data. We introduce an end-to-end measurement system for capturing spectral data on 3D objects. We present the modification of a recently developed hyperspectral imager to make it suitable for acquiring such data in a wide spectral range at high spectral and spatial resolution. We capture four megapixel images, with data at each pixel from the near-ultraviolet (359 nm) to near-infrared (1,003 nm) at 12 nm spectral resolution. We fully characterize the imaging system, and document its accuracy. This imager is integrated into a 3D scanning system to enable the measurement of the diffuse spectral reflectance and fluorescence of specimens. We demonstrate the use of this measurement system in the study of the interplay between the visual capabilities and appearance of birds. We show further the use of the system in gaining insight into artifacts from geology and cultural heritage.

References

1. 3DCoForm, 2012. MeshLab. http://meshlab.sourceforge.net/.
2. Andersson, S., ornborg, J., and Andersson, M. 1998. Ultraviolet sexual dimorphism and assortative mating in blue tits. In Proc. R. Soc. Lond., vol. 265, 445--450.
3. Arnold, K. E., Owens, I. P. F., and Marshall, N. J. 2002. Fluorescent signaling in parrots. Science 295, 5552, 92.
4. Attas, M., Cloutis, E., Collins, C., Goltz, D., Majzels, C., Mansfield, J., and Mantsch, H. 2003. Near-infrared spectroscopic imaging in art conservation: investigation of drawing constituents. J. Cultural Heritage 4, 2, 127--136.
5. Battle, D. R. 1997. The measurement of colour. In Colour Physics for Industry, R. McDonald, Ed., 2nd ed. Soc. Dyers Col., Bradford, 57--80.
6. Bennett, A. T. D., Cuthill, I. C., Partridge, J. C., and Maier, E. J. 1996. Ultraviolet vision and mate choice in zebra finches. Nature 380, 433--435.
7. Bernardini, F., and Rushmeier, H. 2002. The 3D model acquisition pipeline. Comput. Graph. Forum 21, 2, 149--149.
8. Bernardini, F., Martin, I. M., and Rushmeier, H. E. 2001. High-quality texture reconstruction from multiple scans. IEEE Trans. Vis. Comput. Graph. 7, 4, 318--332.
9. Bioucas-Dias, J., and Figueiredo, M. 2007. A new twist: two-step iterative shrinkage/thresholding for image restoration. IEEE Trans. Image Processing 16, 12, 2992--3004.
10. Brady, D. J. 2008. Optical Imaging and Spectroscopy. Wiley-interscience, New Jersey.
11. Brusco, N., Capeleto, S., Fedel, M., Paviotti, A., Poletto, L., Cortelazzo, G. M., and Tondello, G. 2006. A system for 3D modeling frescoed historical buildings with multispectral texture information. Machine Vision and Appl. 17, 6, 373--393.
12. Burns, P. D. 2002. Slanted-edge MTF for digital camera and scanner analysis. In Proc. PICS Conf., IS&T, 135--138.
13. Chambolle, A. 2004. An algorithm for total variation minimization and applications. J. Mathematical Imaging and Vision 20, 1--2, 89--97.
14. Chen, D. M., and Goldsmith, T. H. 1986. Four spectral classes of cone in the retinas of birds. J. Comparative Physiology A 159, 4, 473--479.
15. Chen, D. M., Collins, J. S., and Goldsmith, T. H. 1984. The ultraviolet receptor of bird retinas. Science 225, 4659, 337--340.
16. CIE. 2001. Improvement to industrial colour difference equation. CIE Pub. 142, Commission Internationale de l'Eclairage, Vienna.
17. Devlin, K., Chalmers, A., Wilkie, A., and Purgathofer, W. 2002. Tone reproduction and physically based spectral rendering. Eurographics 2002: State of the Art Reports, 101--123.
18. Du, H., Tong, X., Cao, X., and Lin, S. 2009. A prism-based system for multispectral video acquisition. In Proc. Int. Conf. Comput. Vision (ICCV), IEEE, 175--182.
19. Elgazzar, S., Liscano, R., Blais, F., and Miles, A. 1997. 3-D data acquisition for indoor environment modeling using a compact active range sensor. In Proc. the IEEE Instrumentation, Measurement and Technology Conference, 1--8.
20. Farouk, M., Rifai, I. E., Tayar, S. E., Shishiny, H. E., Hosny, M., Rayes, M. E., Gomes, J., Giordano, F., Rushmeier, H. E., Bernardini, F., and Magerlein, K. 2003. Scanning and processing 3D objects for web display. In Proc. Int. Conf. 3D Digital Imaging and Modeling (3DIM), 310--317.
21. Figueiredo, M., Nowak, R., and Wright, S. 2007. Gradient projection for sparse reconstruction: Application to compressed sensing and other inverse problems. IEEE J. Selected Topics in Signal Processing 1, 4, 586--597.
22. Fischer, C., and Kakoulli, I. 2006. Multispectral and hyper-spectral imaging technologies in conservation: current research and potential applications. Reviews in Conservation 7, 3--16.
23. Habel, R., Kudenov, M., and Wimmer, M. 2012. Practical spectral photography. Comput. Graph. Forum 31, 2, 1--10.
24. Hart, N. S. 2001. The visual ecology of avian photoreceptors. Progress in Retinal and Eye Research 20, 5, 675--703.
25. Holroyd, M., Lawrence, J., and Zickler, T. 2010. A coaxial optical scanner for synchronous acquisition of 3D geometry and surface reflectance. ACM Trans. Graph. (Proc. SIGGRAPH 2010) 29, 3, 99:1--12.
26. ISO. 2000. Photography -- electronic still-picture cameras -- resolution measurements. Tech. Rep. ISO 12233:2000, International Organization for Standardization (ISO).
27. Kawakami, R., Wright, J., Tai, Y.-W., Matsushita, Y., Ben-Ezra, M., and Ikeuchi, K. 2011. High-resolution hyperspectral imaging via matrix factorization. In Proc. IEEE Conf. Comput. Vision and Pattern Recognition, 2329--2336.
28. Kittle, D., Choi, K., Wagadarikar, A., and Brady, D. J. 2010. Multiframe image estimation for coded aperture snapshot spectral imagers. Appl. Opt. 49, 36, 6824--6833.
29. Labshpere, 2011. Spectralon diffuse reflectance standards. http://www.labsphere.com/uploads/datasheets/diffuse-reflectance-standards-product-sheet.pdf.
30. Land, M. F., and Nilsson, D.-E., Eds. 2002. Animal Eyes. Oxford University Press, Oxford.
31. Lensch, H., Kautz, J., Goesele, M., Heidrich, W., and Seidel, H. 2003. Image-based reconstruction of spatial appearance and geometric detail. ACM Trans. Graph. (Proc. SIGGRAPH 2003) 22, 2, 234--257.
32. Levoy, M., Pulli, K., Curless, B., Rusinkiewicz, S., Koller, D., Pereira, L., Ginzton, M., Anderson, S. E., Davis, J., Ginsberg, J., Shade, J., and Fulk, D. 2000. The digital Michelangelo project: 3D scanning of large statues. In Proc. SIGGRAPH 2000, 131--144.
33. Mansouri, A., Lathuiliere, A., Marzani, F. S., Voisin, Y., and Gouton, P. 2007. Toward a 3D multispectral scanner: An application to multimedia. IEEE Multimedia 14, 1, 40--47.
34. Mcgraw, K. J. 2006. Mechanics of uncommon colors in birds: pterins, porphyrins, and psittacofulvins. In Bird coloration, Vol. I, Mechanisms and measurements, G. E. Hill and K. J. Mcgraw, Eds. Harvard University Press, Cambridge, Massachusetts.
35. Newsome, D., and Modreski, P. 1981. The colors and spectral distributions of fluorescent minerals. J. the Fluorescent Mineral Society 10, 7--56.
36. Next-Limit-Technologies, 2012. Maxwell Render. http://www.maxwellrender.com/.
37. nVidia, 2012. About iRay, physically correct GPU rendering technology. http://www.mentalimages.com/products/iray/about-iray.html.
38. Qin, J. 2010. Hyperspectral imaging instruments. In Hyperspectral Imaging for Food Quality Analysis and Control, D.-W. Sun, Ed. Elsevier, 129--175.
39. Rapantzikos, K., and Balas, C. 2005. Hyperspectral imaging: potential in non-destructive analysis of palimpsests. In Proc. Int. Conf. Image Processing (ICIP), vol. 2, 618--621.
40. Saunders, D., and Cupitt, J. 1993. Image processing at the National Gallery: The VASARI project. National Gallery Tech. Bull. 14, 72--86.
41. Stoddard, M. C., and Prum, R. O. 2008. Evolution of avian plumage color in a tetrahedral color space: a phylogenetic analysis of new world buntings. The American Naturalist 171, 6, 755--776.
42. Stoddard, M. C., and Prum, R. O. 2011. How colorful are birds? Evolution of the avian plumage color gamut. Behavioral Ecology 22, 5, 1042--1052.
43. Sugiura, H., Kuno, T., Watanabe, N., Matoba, N., Hayashi, J., and Miyata, Y. 2000. Development of a multi-spectral camera system. In Proc. SPIE 3965, 331--339.
44. Sun, X., and Pitsianis, N. 2008. Solving non-negative linear inverse problems with the NeAREst method. Proc. SPIE 7074, 707402.
45. Thoury, M., Elias, M., Frigerio, J. M., and Barthou, C. 2005. Non-destructive identification of varnishes by UV fluorescence spectroscopy. In Proc. SPIE 5857, 1--11.
46. Vos, J. J. 1978. Colorimetric and photometric properties of a 2-deg fundamental observer. Color Res. Appl. 3, 125--128.
47. Wagadarikar, A., John, R., Willett, R., and Brady, D. J. 2008. Single disperser design for coded aperture snapshot spectral imaging. Appl. Opt. 47, 10, B44--B51.
48. Wagadarikar, A. A., Pitsianis, N. P., Sun, X., and Brady, D. J. 2009. Video rate spectral imaging using a coded aperture snapshot spectral imager. Opt. Express 17, 8, 6368--6388.
49. Ware, G., Chabries, D., Christiansen, R., Brady, J., and Martin, C. 2000. Multispectral analysis of ancient Maya pigments: implications for the Naj Tunich Corpus. In Proc. IEEE Geoscience and Remote Sensing Symposium, vol. 6, 2489--2491.
50. Wright, S., Nowak, R., and Figueiredo, M. 2009. Sparse reconstruction by separable approximation. IEEE Trans. Signal Processing 57, 7, 2479--2493.
51. Zhao, Y., Berns, R. S., Taplin, L. A., and Coddington, J. 2008. An investigation of multispectral imaging for the mapping of pigments in paintings. In Proc. SPIE 6810, 1--9.

Index Terms

• Computing methodologies

  • Artificial intelligence

    • Computer vision

      • Computer vision tasks

  • Computer graphics

    • Image manipulation