Knowledge-based augmented reality

Reviewer: D. Charles Hair

Overall, this issue provides an interesting and informative look at current research in an emerging area. The papers discuss a variety of projects that reflect a shared view of how the future of computing could unfold. Although the directions being explored are generally not mature enough for widespread application, some promising early results are indicated. Wellner, Mackay, and Gold In the introduction, the guest editors of this special issue point to a difference in philosophical approaches between work in virtual reality and the new work they are introducing. Both approaches are aimed at developing computer environments, but where virtual reality takes the approach of putting the user in a completely artificial, computer-generated setting, the computer-augmented environment approach is to put the computer into a real-world setting. This basic distinction between the two approaches is pointed to in a number of the following papers. The introduction also introduces two terms that are used throughout the rest of the papers: “ubiquitous computing” and “augmented reality.” Ubiquitous computing means incorporating some form of computerization into many common facets of people's environments, operating in an essentially transparent fashion. Augmented reality connotes a similar idea, with more stress on the contrast between incorporating computing into the real environment and creating a new, artificial environment. The introduction briefly summarizes the papers to follow, and emphasizes that social issues are treated along with technical issues. Spreitzer and Theimer In this sidebar, the authors raise the ethical issue of how to protect privacy in a setting where a person's location information may be needed in order to facilitate mobile computing of some kind. This sidebar puts an immediate emphasis on the social issues mentioned in the introduction. Baudel and Beaudouin-Lafon A project that explores the use of freehand gestures to interact with computers is described. The authors point to a number of possible advantages of such interaction. This kind of interaction can be natural and easy to learn, gestures can economically convey a rich amount of information, and hand gestures can be a direct way to interact with a ubiquitous computing environment. The Charade work relies on the use of a dataglove, but the authors note that datagloves may not be desirable in general. They suggest that the use of video cameras to monitor gestures might be a better alternative. The discussion focuses on how the Charade project was used to explore solutions to typical problems in the use of hand gestures, such as fatigue, segmentation of hand gestures, and discomfort. A small but noticeable editing error occurs in this paper. The last paragraph refers to a scenario that the authors say was described at the beginning of the paper. No such scenario is described, however. Krueger This sidebar discusses some developments and applications made by the author's company to enable people to directly interact with computing environments through the use of gestures. Here, the focus is on the use of cameras to monitor the gestures. Fitzmaurice Chameleon is a prototype system used to explore the idea of spatially aware palmtop computers. The prototype was used in connection with an output display of a geographical wall map. The palmtop computer is used to manipulate information related to the map display in reaction to both spatial movements of the palmtop computer and button clicks on the palmtop. The paper includes some discussion of problems encountered, as well as indicating the promise this approach shows. In addition, it discusses three possible application domains: active maps and paper, computer-augmented libraries, and portable surrogate offices. Azuma Azuma contrasts augmented reality with virtual reality on the basis that in augmented reality settings, it is critically important to achieve a fidelity such that virtual objects fit in seamlessly with real objects. The main point is to describe progress on a project involving the use of head-mounted units that act as head and object trackers. Feiner, MacIntyre, and Seligmann This work also involves the use of head-mounted displays. The central idea is to explore ways to use knowledge-based systems in connection with such displays in augmented reality settings. The research involves the use of “see-through” displays, which merge computer-synthesized graphics with the user's real-world view in order to assist in three-dimensional design tasks. The authors' ideas were tested in the domain of maintenance and repair tasks for a laser printer. Their primary system is called KARMA (Knowledge-based Augmented Reality for Maintenance Assistance), and this work involved the development of a knowledge-based graphics component called IBIS (Intent-Based Illustration System). Resnick “Behavior Construction Kits” describes innovative work aimed at children. The work builds on prior research done at MIT with LEGO/Logo construction kits. The new idea described here is to go beyond letting children build structures and mechanisms, to allow the building of behaviors into robotic creations. This idea has led to the use of “electronic bricks” as building pieces. The author also describes possible new goals to be achieved through the use of individually programmable bricks. Gold The following sidebar, “This Is Not a Pipe,” briefly suggests some of the potential for ubiquitous computing. Weiser Weiser discusses research he has done that is aimed at defining and constructing new kinds of computing artifacts that can facilitate ubiquitous computing. He has approached this goal by beginning with three sizes of devices: a wall-sized interactive device, or board; a notepad device intended to be analogous to scrap paper, referred to as a pad; and a small device analogous to the Post-it note, referred to as a tab. The author describes the implementation of these devices, and then discusses some of the computer science issues related to the use of such devices. Those issues include hardware matters, networking considerations, interaction questions, and possible applications. Elrod, Hall, Costanza, Dixon, and Des Rivie`res Next is a sidebar on “Responsive Office Environments.” It gives a brief discussion of the development of a ubiquitous computing application for use in managing energy and controlling the environment in an office setting. Wellner In “Interacting with Paper on the DigitalDesk,” the idea is to implement a computer-augmented environment for paper. The DigitalDesk is a real physical desk that is augmented in various ways through computerization. For instance, computer displays can be projected on the desk, and user actions can be monitored with overhead cameras. Mackay, Velay, Carter, Ma, and Pagani The final sidebar takes a further look at ways to augment paper. It discusses work on the Digital Drawing Board and work on a project called Mosaic that combines paper storyboards with computer-controlled videos. Conclusion These papers are generally well written and informative. Taken together, they are a good introduction to the kinds of ideas and projects that are being explored in connection with ubiquitous computing and augmented reality. This area holds potential for exciting new applications and should be of interest to a wide audience.