Improving static and dynamic registration in an optical see-through HMD

ABSTRACT

In Augmented Reality, see-through HMDs superimpose virtual 3D objects on the real world. This technology has the potential to enhance a user's perception and interaction with the real world. However, many Augmented Reality applications will not be accepted until we can accurately register virtual objects with their real counterparts. In previous systems, such registration was achieved only from a limited range of viewpoints, when the user kept his head still. This paper offers improved registration in two areas. First, our system demonstrates accurate static registration across a wide variety of viewing angles and positions. An optoelectronic tracker provides the required range and accuracy. Three calibration steps determine the viewing parameters. Second, dynamic errors that occur when the user moves his head are reduced by predicting future head locations. Inertial sensors mounted on the HMD aid head-motion prediction. Accurate determination of prediction distances requires low-overhead operating systems and eliminating unpredictable sources of latency. On average, prediction with inertial sensors produces errors 2-3 times lower than prediction without inertial sensors and 5-10 times lower than using no prediction at all. Future steps that may further improve registration are outlined.

References

1. 1.Albrecht, R. E. An adaptive digital filter to predict pilot head look direction for helmet-mounted displays. MS Thesis, University of Dayton, Ohio (July 1989).]]
2. 2.Andres, Robert O., and Kenny J. Hartung. Prediction of Head Move-ment Time Using Fitts' Law. Human Factors 31, 6 (1989), 703-713.]]
3. 3.Azuma, Ronald. Tracking Requirements for Augmented Reality. CACM 36, 7 (July 1993), 50-51.]]
4. 4.Bajura, Michael, Henry Fuchs, and Ryutarou Ohbuchi. Merging Vir-tual Objects with the Real World: Seeing Ultrasound Imagery within the Patient. Proceedings of SIGGRAPH '92 (Chicago, IL, July 26- 31, 1992), 203-210.]]
5. 5.Beer, Ferdinand P. and E. Russell Johnston, Jr. Vector Mechanics for Engineers: Statics and Dynamics (5th ed). McGraw-Hill, 1988.]]
6. 6.Caudell, Thomas P. and David W. Mizell. Augmented Reality: An Application of Heads-Up Display Technology to Manual Manufac-turing Processes. Proceedings of Hawaii International Conference on System Sciences (Jan. 1992), 659-669.]]
7. 7.Chou, Jack C.K. Quaternion Kinematic and Dynamic Differential Equations. IEEE Trans Robotics and Automation 8, 1 (Feb. 1992), 53-64.]]
8. 8.Cohen, Jonathan, and Mark Olano. Low Latency Rendering on Pixel-Planes 5. UNC Chapel Hill Dept. of Computer Science technical report TR94-028 (1994).]]
9. 9.Cruz-Neira, Carolina, Daniel Sandin, and Thomas DeFanti. Surround-Screen Projection-Based Virtual Reality: The Design and Implemen-tation of the CAVE. Proceedings of SIGGRAPH '93 (Anaheim, CA, Aug. 1-6, 1993), 135-142.]]
10. 10.Deering, Michael. High Resolution Virtual Reality. Proceedings of SIGGRAPH '92 (Chicago, IL, July 26-31, 1992), 195-202.]]
11. 11.Feiner, Steven, Blair MacIntyre, and Dor~e Seligmann. Knowledge-Based Augmented Reality. CACM 36, 7 (July 1993), 53-62.]]
12. 12.Foley, James D., Andries van Dam, Steven K. Feiner, and John F. Hughes. Computer Graphics: Principles and Practice, 2nd edition. Addison-Wesley (1990), 238-239.]]
13. 13.Friedmann, Martin, Thad Starner, and Alex Pentland. Device Syn-chronization Using an Optimal Filter. Proceedings of 1992 Sympo-sium on Interactive 3D Graphics (Cambridge, MA, 29 March - 1 April 1992), 57-62.]]
14. 14.Fuchs, Henry, John Poulton, John Eyles, et al. Pixel-Planes 5: A Het-erogeneous Multiprocessor Graphics System Using Processor-En-hanced Memories. Proceedings of SIGGRAPH '89 (Boston, MA, July 31-Aug 4, 1989), 79-88.]]
15. 15.Gottschalk, Stefan and John F. Hughes. Autocalibration for Virtual Environments Tracking Hardware. Proceedings of SIGGRAPH '93 (Anaheim, CA, Aug 1-6, 1993), 65-72.]]
16. 16.Holmgren, Douglas E. Design and Construction of a 30-Degree See-Through Head-Mounted Display. UNC Chapel Hill Dept. of Com-puter Science technical report TR92-030 (July 1992).]]
17. 17.Jagacinski, Richard J., and Donald L. Monk. Fitts' Law in Two Di-mensions with Hand and Head Movements. Journal of Motor Be-havior 17, 1 (1985), 77-95.]]
18. 18.Janin, Adam L., David W. Mizell, and Thomas P. Caudell. Calibra-tion of Head-Mounted Displays for Augmented Reality Applications. Proceedings of IEEE VRAIS '93 (Seattle, WA, Sept. 18-22, 1993), 246-255.]]
19. 19.Kalman, R. E., and R. S. Bucy. New Results in Linear Filtering and Prediction Theory. Trans ASME, J. Basic Eng., Series 83D (Mar. 1961), 95-108.]]
20. 20.Krueger, Myron W. Simulation versus artificial reality. Proceedings of IMAGE VI Conference (Scottsdale, AZ, 14-17 July 1992), 147- 155.]]
21. 21.Lenz, Reimar K. and Roger Y. Tsai. Techniques for Calibration of the Scale Factor and Image Center for High Accuracy 3-D Machine Vi-sion Metrology. IEEE Transactions on Pattern Analysis and Ma-chine Intelligence 10, 5 (Sept. 1988), 713-720.]]
22. 22.Lewis, Frank L. Optimal Estimation. John Wiley & Sons, 1986.]]
23. 23.Liang, Jiandong, Chris Shaw, and Mark Green. On Temporal-Spatial Realism in the Virtual Reality Environment. Proceedings of the 4th annual ACM Symposium on User Interface Software & Technology (Hilton Head, SC, Nov 11-13, 1991), 19-25.]]
24. 24.List, Uwe H. Nonlinear Prediction of Head Movements for Helmet-Mounted Displays. Technical report AFHRL-TP-83-45 {AD-A136590}, Williams AFB, AZ: Operations Training Division (1984).]]
25. 25.Murray, P.M. and B. Barber. Visual Display Research Tool. AGARD Conference Proceedings No. 408 Flight Simulation (Cambridge, UK, 30 Sept. - 3 Oct. 1985).]]
26. 26.Paley, W. Bradford. Head-Tracking Stereo Display: Experiments and Applications. SPIE Vol. 1669 Stereoscopic Displays and Applica-tions III (San Jose, CA, Feb. 12-13, 1992), 84-89.]]
27. 27.Press, William H., et al. Numerical Recipes in C. Cambridge Uni-versity Press, 1988.]]
28. 28.Rebo, Robert. A Helmet-Mounted Virtual Environment Display Sys-tem. MS Thesis, Air Force Institute of Technology (Dec 1988).]]
29. 29.Robinett, Warren. Synthetic Experience: A Proposed Taxonomy. Presence 1, 2 (Spring 1992), 229-247.]]
30. 30.Robinett, Warren, and Richard Holloway. Implementation of Flying, Scaling and Grabbing in Virtual Worlds. Proceedings of 1992 Sym-posium on Interactive 3D Graphics (Cambridge, MA, 29 March - 1 April 1992), 189-192.]]
31. 31.Robinett, Warren, and Jannick P. Rolland. A Computational Model for the Stereoscopic Optics of a Head-Mounted Display. Presence 1, 1 (Winter 1992), 45-62.]]
32. 32.Shaw, Chris and Jiandong Liang. An Experiment to Characterize Head Motion in VR and RR Using MR. Proceedings of 1992 Western Com-puter Graphics Symposium (Banff, Alberta, Canada, April 6-8, 1992), 99-101.]]
33. 33.Sims, Dave. New Realities in Aircraft Design and Manufacture. IEEE CG&A 14, 2 (March 1994), 91.]]
34. 34.Smith Jr., B. R. Digital head tracking and position prediction for hel-met mounted visual display systems. Proceedings of AIAA 22nd Aerospace Sciences Meeting (Reno, NV, Jan. 9-12, 1984).]]
35. 35.So, Richard H. Y. and Michael J. Griffin. Compensating Lags in Head-Coupled Displays Using Head Position Prediction and Image Deflec-tion. Journal of Aircraft 29, 6 (Nov-Dec 1992), 1064-1068.]]
36. 36.Ward, Mark, Ronald Azuma, Robert Bennett, Stefan Gottschalk, and Henry Fuchs. A Demonstrated Optical Tracker With Scalable Work Area for Head-Mounted Display Systems. Proceedings of 1992 Sym-posium on Interactive 3D Graphics (Cambridge, MA, 29 March - 1 April 1992), 43-52.]]
37. 37.Welch, Brian L., Ron Kruk, et al. Flight Simulator Wide Field-of-View Helmet-Mounted Infinity Display System. Technical report AFHRL-TR-85-59, WilliamsAFB,AZ, Operations Training Division (May 1986).]]

Index Terms

• Computing methodologies

  • Artificial intelligence

    • Computer vision

      • Image and video acquisition

        • 3D imaging

  • Computer graphics

    • Animation

• Hardware

  • Communication hardware, interfaces and storage

    • Displays and imagers