Bokode: imperceptible visual tags for camera based interaction from a distance

Abstract

We show a new camera based interaction solution where an ordinary camera can detect small optical tags from a relatively large distance. Current optical tags, such as barcodes, must be read within a short range and the codes occupy valuable physical space on products. We present a new low-cost optical design so that the tags can be shrunk to 3mm visible diameter, and unmodified ordinary cameras several meters away can be set up to decode the identity plus the relative distance and angle. The design exploits the bokeh effect of ordinary cameras lenses, which maps rays exiting from an out of focus scene point into a disk like blur on the camera sensor. This bokeh-code or Bokode is a barcode design with a simple lenslet over the pattern. We show that a code with 15μm features can be read using an off-the-shelf camera from distances of up to 2 meters. We use intelligent binary coding to estimate the relative distance and angle to the camera, and show potential for applications in augmented reality and motion capture. We analyze the constraints and performance of the optical system, and discuss several plausible application scenarios.

References

1. Abowd, G. D., and Mynatt, E. D. 2000. Charting past, present, and future research in ubiquitous computing. ACM Trans. Comput.-Hum. Interact. 7, 1, 29--58. 2
2. Azuma, R., Baillot, Y., Behringer, R., Feiner, S., Julier, S., and MacIntyre, B. 2001. Recent advances in augmented reality. IEEE CG&A 21, 6, 34--47. 2
3. Claus, D., and Fitzgibbon, A. W. 2004. Reliable fiducial detection in natural scenes. In ECCV 2004, 469--480. 2
4. de Ipiña, D. L., Mendonça, P. R. S., and Hopper, A. 2002. TRIP: A low-cost vision-based location system for ubiquitous computing. Personal Ubiquitous Computing 6, 3, 206--219. 2
5. Fernald, R. D. 2006. Casting a genetic light on the evolution of eyes. Science 313, 5795, 1914--1918. 5
6. Fiala, M. 2005. ARTag, a fiducial marker system using digital techniques. In IEEE CVPR, vol. 2, 590--596. 2
7. ISO, 2006. Data Matrix bar code symbology specification. ISO/IEC 16022:2006. 2, 4
8. ISO, 2006. QR code 2005 bar code symbology specification. ISO/IEC 18004:2006. 2
9. Kato, H., and Billinghurst, M. 1999. Marker tracking and hmd calibration for a video-based augmented reality conferencing system. In IWAR 99, 85--94. 2, 6
10. Kato, H., and Tan, K. T. 2007. Pervasive 2D barcodes for camera phone applications. IEEE Pervasive Computing 6, 4, 76--85. 2
11. Kitamura, Y., Konishi, T., Yamamoto, S., and Kishino, F. 2001. Interactive stereoscopic display for three or more users. In SIGGRAPH 2001, ACM, 231--240. 2
12. Langlotz, T., and Bimber, O. 2007. Unsynchronized 4D barcodes. In ISVC, 363--374. 2
13. Levin, A., Fergus, R., Durand, F., and Freeman, W. T. 2007. Image and depth from a conventional camera with a coded aperture. In SIGGRAPH 2007, vol. 26. 2
14. Levoy, M., and Hanrahan, P. 1996. Light field rendering. In SIGGRAPH 1996, 31--42. 2
15. MacKay, D. J. C. 2003. Information Theory, Inference, and Learning Algorithms. Cambridge University Press. 2
16. Matsushita, N., Hihara, D., Ushiro, T., Yoshimura, S., Rekimoto, J., and Yamamoto, Y. 2003. ID CAM: A smart camera for scene capturing and id recognition. In International Symposium on Mixed and Augmented Reality, 227--236. 2
17. Mohan, A., Lanman, D., Hiura, S., and Raskar, R. 2009. Image Destabilization: Programmable defocus using lens and sensor motion. In IEEE ICCP. 5
18. Morton, A. Q. 1994. Packaging history: The emergence of the Uniform Product Code (UPC) in the United States. History and Technology 11, 1, 101--111. 2
19. Pavlidis, T., Swartz, J., and Wang, Y. P. 1990. Fundamentals of bar code information theory. IEEE Computer 23, 4, 74--86. 2
20. Pentland, A. P. 1987. A new sense for depth of field. IEEE PAMI 9, 4, 523--531. 6
21. Raskar, R., Beardsley, P., van Baar, J., Wang, Y., Dietz, P., Lee, J., Leigh, D., and Willwacher, T. 2004. RFIG Lamps: Interacting with a self-describing world via photosensing wireless tags and projectors. In SIGGRAPH, 406--415. 2
22. Sony, EyeToy. http://www.eyetoy.com. 2
23. Tateno, K., Kitahara, I., and Ohta, Y. 2007. A nested marker for augmented reality. In IEEE VR, 259--262. 2
24. Teller, S., Chen, K., and Balakrishnan, H. 2003. Pervasive pose-aware applications and infrastructure. IEEE Computer Graphics and Applications (July/August). 2
25. Tremblay, E. J., Stack, R. A., Morrison, R. L., and Ford, J. E. 2007. Ultrathin cameras using annular folded optics. Applied Optics 46, 4, 463--471. 7
26. Veeraraghavan, A., Raskar, R., Agrawal, A., Mohan, A., and Tumblin, J. 2007. Dappled Photography: Mask enhanced cameras for heterodyned light fields and coded aperture refocusing. In SIGGRAPH 2007, ACM, vol. 26, 69:1--69:12. 2
27. Want, R. 2003. RFID: A key to automating everything. Scientific American. 2
28. Weiser, M. 1993. Ubiquitous Computing. IEEE Computer 26, 10, 71--72. 2
29. Welch, G., Bishop, G., Vicci, L., Brumback, S., Keller, K., and Colucci, D. 1999. The HiBall Tracker: High-performance wide-area tracking for virtual and augmented environments. In ACM VRST. 2
30. Zhang, L., Curless, B., and Seitz, S. M. 2002. Rapid shape acquisition using color structured light and multi-pass dynamic programming. In IEEE 3DPVT, 24--36. 5
31. Zhang, X., Fronz, S., and Navab, N. 2002. Visual marker detection and decoding in AR systems: A comparative study. In ISMAR 2002, 97--106. 2
32. Zhang, L., Subramaniam, N., Lin, R., Raskar, R., and Nayar, S. 2008. Capturing images with sparse informational pixels using projected 3D tags. In IEEE Virtual Reality. 2

Index Terms

• Computing methodologies

  • Artificial intelligence

    • Computer vision

      • Computer vision problems

  • Computer graphics

    • Graphics systems and interfaces

    • Image manipulation

      • Texturing

  • Machine learning

• Human-centered computing

  • Human computer interaction (HCI)

    • Interaction devices

      • Graphics input devices

• Theory of computation

  • Design and analysis of algorithms

    • Data structures design and analysis

      • Pattern matching