Creating highly-interactive and graphical user interfaces by demonstration

ABSTRACT

It is very time-consuming and expensive to create the graphical, highly-interactive styles of user interfaces that are increasingly common. User Interface Management Systems (UIMSs) attempt to make the creation of user interfaces easier, but most existing UIMSs cannot create the low-level interaction techniques (pop-up pull-down and fixed menus, on-screen "light buttons", scroll-bars, elaborate feedback mechanisms and animations, etc.) that are frequently used. This paper describes Peridot, a system that automatically creates the code for these user interfaces while the designer demonstrates to the system how the interface should look and work. Peridot uses rule-based inferencing so no programming by the designer is required, and Direct Manipulation techniques are used to create Direct Manipulation interfaces, which can make full use of a mouse and other input devices. This allows extremely rapid prototyping of user interfaces.

References

1. Apple 85.Apple Computer, Inc. Inside Macintosh. Addison- Wesley, 1985.
2. Bauer 78.Michael Anthony Bauer. A Basis for the Acquisition of Procedures. PhD Thesis, Department of Computer Science, University of'Toronto. 1978. 310 pages.
3. Biermann 76a.Alan W. Biermann. '~Approaches to Automatic Programming," Advanc~ in Computers, Morris Rubinoff and Marshall C. Yovitz, eds. Vol. 15. New York: Academic Press, 1976. pp. 1-63.
4. Biermann 76b.Alan W. Biermann and Ramachandran Krishnaswamy. "Constructing Programs from Example Computations," IEEE Transactions on Software Engineering. Vol. SE-2, no. 3. Sept. 1976. pp. i41-153.
5. Borning 79.Alan Borning. ThingIab--A Constraint-Oriented Simulation Laboratory. Xerox Pale Alto Research Center Technical Report SSL-79-3. July, 1979. 100 pages.
6. Buxton 83.W. Buxton, M.R. Lamb, D. Sherman, and K.C~ Smith. "Towards a Comprehensive User Interface Management System," Computer Graphics: SIGGRAPH'83 Conference Proceedings. Detroit, Mich. Vol. 17, no. 3, July 25-29, 1983. pp. 35-42.
7. Buxton 86.William Buxton and Brad Myers. "A Study in Two- Handed Input," Proceedings SIGCHI'86: Human Factors in Computing Systems. Boston, MA. April 13-17, 1986.
8. Cardelli 85.Luca Cardelli and Rob Pike. "Squeak: A Language for Communicating with Mice," Computer Graphics: SIG- GRAPIT85 Conference Proceedings. San Francisco, CA. Vol. 19, no. 3. July 22-26, 1985, pp. 199-204.
9. Foley 84.James D. Foley. "Managing the Design of User- Computer Interfaces," Proceedings of the Fifth Annual NCGA Conference and Exposition. Anaheim, CA. Vol. II. May 13-17, 1984. pp. 436-451.
10. Gould 84.Laura Gould and William Finzer. Programming by Rehearsal. Xerox Pale Alto Research Center Technical Report SCL-84-1. May, 1984. 133 pages. A short version appears in Byte. Vol. 9, no. 6. June, 1984.
11. Halbert 81.Daniel C. Halbert. An Example of Programming by Example. Masters of Science Thesis. Computer Science Division, Dept. of EE&CS, University of California, Berkeley and Xerox Corporation Office Products Division, Pale Alto, CA. June, 1981.55 pages.
12. Halbert 84.Daniel C. Halhert. Programming by Example. PhD Thesis. Computer Science Division, Dept. of EE&CS, University of California, Berkeley. 1984. Also: Xerox Office Systems Division, Systems Development Department, TR OSD-T8402, December, 1984. 83 pages.
13. Hayes 85.Philip J. Hayes, Pedro A. Szekely, and Richard A. Lerner. "Design Alternatives for User Interface Management Systems Based on Experience with COUSIN," Proceedings SIGCHI'85: Human Factors in Computing Systems. San Francisco, CA. April 14-18, 1985. pp. 169-175.
14. Henderson 86.D. Austin Henderson, Jr. '~The Trillium User Interface Design Environment," Proceedings SIGCHl'86: Human Factors in Computing Systems. Boston, MA. April 13-17, 1986. pp. 221-227.
15. Hutchins 86.Edwin L. Hutchins, James D. Hollan, and Donald A. Norman. "Direct Manipulation Interfaces," User Centered System Design, Donald A. Norman and Stephen W. Draper, eds. Hillsdale, New Jersey: Lawrence Erlbaum Associates, 1986. pp. 87-124.
16. Jacob 85.Robert J.K. Jacob. "A State Transition Diagram Language for Visual Programming," IEEE Computer. VoI. 18, no. 8. Aug. 1985. pp. 51-59.
17. Lieberman 82.Henry Lieberman. "Constructing Graphical User Interfaces by Example," Graphics Interface, '82, Toronto, Ontario, March 17-21, 1982. pp. 295-302.
18. Myers 84.Brad A. Myers. "The User Interface for Sapphire," IEEE Computer Graphics and Applications. Vol. 4, no~ 12, December, 1984. pp. 13-23.
19. Myers 86.Brad A. Myers. "Visual Programming, Programming by Example, and Program Visualization; A Taxonomy," Proceedings SIGCHI'86: Human Factors in Computing Systems Boston, MA. April 13-17, 1986. pp. 59-66.
20. Myers prep.Brad A. Myers. Applying Visual Programming with Programming by Example and Constraints to User Interface Management Systems. PhD Thesis, Department of Computer Science, University of Toronto, Toronto, Ontario, Canada. In progress.
21. Nix 86.Robert P. Nix. ':Editing by Example," ACM Transac. tions on Programming Languages and Systems. Vol. 7, no. 4. Oct. 1985. pp. 600-621.
22. Olsen 83.Dan R. Olsen and Elizabeth P. Dempsey. "Syngraph: A Graphical User Interface Generator," Computer Graphics: SIGGRAPIT83 Conference Proceedings. Detroit, Mich. Vol. 17, no. 3. July 25-29, 1983. pp. 43-50.
23. Olsen 84.Dan R. Olsen, Jr., William Buxton, Roger Ehrich, David J. Kasik, James R. Rhyne, and John Sibert. "A Context for User Interface Management," IEEE Computer Graphics and Applications. Vol. 4, no. 2. Dec. 1984. pp. 33- 42.
24. Olsen 85.Dan R. Olsen, Jr., Elisabeth P. Dempsey, and Roy Rogge. "Input-Output Linkage in a User Interface Managemerit System," Computer Graphics: SIGGRAPI-I'83 Conference Proceedings~ San Francisco, CA. Vol. 19, no. 3. July 22-26, 1985. pp. 225-234.
25. Pavlidis 85.Theo Pavlidis and Christopher J, Van Wyk, "An Automatic Beautifier for Drawings and Illustrations," Computer Graphics: $IGGRAPH'85 Conference Proceedings. San Francisco, CA. Vol. 19, no. 3. July 22-26, 1985. pp. 225-234.
26. Pfaff 85.Gunther R. Pfaff, ed. User Interface Management Systems. Berlin: Springer-Verlag, 1985. 224 pages.
27. Shaw 75.David E. Shaw, William R. Swartout, and C. CordeI1 Green. "Inferring Lisp Programs from Examples," Fourth International Joint Conference on Artificial Intelligence. Tbilisi, USSR. Sept. 3-8, 1975. Vol. 1. pp. 260-267.
28. Shneiderman 83.Ben Shneiderman. "Direct Manipulation: A Step Beyond Programming Languages," IEEE Computer. Vol. 16, no. 8. Aug. 1983. pp. 57-69.
29. Shneiderman 86.Ben Shneiderman. "Seven Plus or Minus Two Central Issues in Human-Computer Interfaces," Proceedings SIGCHI'86: Human Factors in Computing Systems. (closing plenary address) Boston, MA. April 13-17, 1986. pp. 343-349.
30. Smith 77.David Canfield Smith. Pygmalion: A Computer Program to Model and Stimulate Creative Thought. Basel, Stuttgart: Birkhat~ser, 1977. 187 pages.
31. Smith 82.David Canfield Smith, Charles Irby, Ralph Kimball, Bill Verplank, and Erik Harslem. "Designing the Star User Interface," Byte Magazine, April 1982, pp. 242-282.
32. Tanner 85.Peter P. Tanner and William A.S. Buxton. "Some Issues in Future User Interface Management System (UIMS) Development," in User Interface Management Systems, Gunther R. Pfaff, ed. Berlin: Springer-Verlag, I985. pp. 67-79.
33. Tesler 81.Larry Tesler. 'Toe Smalltalk Environment," Byte Magazine. August 198I, pp. 90-147.
34. Thomas 83.James J. Thomas and Griffith Hamlin, eds. "Graphical Input interaction Technique (GIIT) Workshop Summary." ACM/SIGGRAPH, Seattle, WA. June 2-4, 1982. in Computer Graphics. Vol. 17, no. 1. Jan. 1983. pp. 5-30.
35. Williams 83.Gregg Williams. "The Lisa Computer System," Byte Magazine, February 1983, pp. 33-50.
36. Williams 84.Gregg Williams. "The Apple Macintosh Computer," Byte Magazine. February 1984. pp. 30-54.
37. Xerox 83.Xerox Corporation. Interlisp Reference Manual. Pasadena, CA. October, 1983.

Index Terms

• Applied computing

  • Document management and text processing

    • Document preparation

      • Hypertext / hypermedia creation

• Computing methodologies

  • Computer graphics

    • Graphics systems and interfaces

• Human-centered computing

  • Human computer interaction (HCI)

    • Interaction devices

      • Graphics input devices

    • Interaction paradigms

      • Hypertext / hypermedia

  • Visualization

    • Visualization techniques

• Software and its engineering

  • Software notations and tools

    • Development frameworks and environments

      • Integrated and visual development environments