skip to main content
research-article

Designers, the Stage Is Yours! Medium-Fidelity Prototyping of Augmented & Virtual Reality Interfaces with 360theater

Published:29 May 2021Publication History
Skip Abstract Section

Abstract

While augmented and virtual reality technologies are becoming mainstream, it is still technically challenging and time-consuming to create new applications. Many designers draw from traditional low-fidelity prototyping methods that do not lend themselves well to designing in 3D. Developers use high-end programming frameworks such as Unity and Unreal which require significant hardware/software setups and coding skills. We see a gap in the medium-fidelity range where there is an opportunity for new tools to leverage the advantages of 360° content for AR/VR prototyping. Existing tools, however, have only limited support for 3D geometry, spatial and proxemic interactions, puppeteering, and storytelling. We present 360theater, a new method and a tool for rapid prototyping of AR/VR experiences, which takes dioramas into the virtual realm by enhancing 360° video capture with 3D geometry and simulating spatial interactions via Wizard of Oz. Our comparative evaluation of techniques with novice and experienced AR/VR designers shows that 360theater can close the gap and achieve a higher fidelity and more realistic AR/VR prototypes than comparable methods.

Skip Supplemental Material Section

Supplemental Material

V5eics205VF.mp4

Supplemental video

References

  1. Narges Ashtari, Andrea Bunt, Joanna McGrenere, Michael Nebeling, and Parmit K. Chilana. 2020. Creating Augmented and Virtual Reality Applications: Current Practices, Challenges, and Opportunities (CHI '20). Association for Computing Machinery, New York, NY, USA, 1--13. https://doi.org/10.1145/3313831.3376722 Google ScholarGoogle ScholarDigital LibraryDigital Library
  2. Till Ballendat, Nicolai Marquardt, and Saul Greenberg. 2010. Proxemic Interaction: Designing for a Proximity and Orientation-Aware Environment. In Proc. ITS . Google ScholarGoogle ScholarDigital LibraryDigital Library
  3. Davide Bolchini, Diego Pulido, and Anthony Faiola. 2009. Feature -- “Paper in screen” prototyping: an agile technique to anticipate the mobile experience. Interactions , Vol. 16, 4 (2009), 29--33. https://doi.org/10.1145/1551986.1551992 Google ScholarGoogle ScholarDigital LibraryDigital Library
  4. Bill Buxton. 2010. Sketching user experiences: getting the design right and the right design .Morgan Kaufmann. Google ScholarGoogle ScholarDigital LibraryDigital Library
  5. Steven Dow, Jaemin Lee, Christopher Oezbek, Blair MacIntyre, Jay David Bolter, and Maribeth Gandy. 2005 a. Wizard of Oz interfaces for mixed reality applications. In Proc. CHI Extended Abstracts. 1339--1342. https://doi.org/10.1145/1056808.1056911 Google ScholarGoogle ScholarDigital LibraryDigital Library
  6. Steven Dow, Blair MacIntyre, Jaemin Lee, Christopher Oezbek, Jay David Bolter, and Maribeth Gandy. 2005 b. Wizard of Oz support throughout an iterative design process. IEEE Pervasive Computing , Vol. 4, 4 (2005), 18--26. https://doi.org/10.1109/MPRV.2005.93 Google ScholarGoogle ScholarDigital LibraryDigital Library
  7. Maribeth Gandy and Blair MacIntyre. 2014. Designer's augmented reality toolkit, ten years later: implications for new media authoring tools. In Proc. UIST . Google ScholarGoogle ScholarDigital LibraryDigital Library
  8. Terrell Glenn, Ananya Ipsita, Caleb Carithers, Kylie Peppler, and Karthik Ramani. 2020. StoryMakAR: Bringing Stories to Life With An Augmented Reality & Physical Prototyping Toolkit for Youth. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems (Honolulu, HI, USA) (CHI '20). Association for Computing Machinery, New York, NY, USA, 1--14. https://doi.org/10.1145/3313831.3376790 Google ScholarGoogle ScholarDigital LibraryDigital Library
  9. Andrew J. Hunsucker, Kelly McClinton, Jennifer Wang, and Erik Stolterman. 2017. Augmented Reality Prototyping For Interaction Design Students. In Proc. CHI Extended Abstracts. 1018--1023. https://doi.org/10.1145/3027063.3053684 Google ScholarGoogle ScholarDigital LibraryDigital Library
  10. Brett R. Jones, Rajinder Sodhi, Michael Murdock, Ravish Mehra, Hrvoje Benko, Andrew Wilson, Eyal Ofek, Blair MacIntyre, Nikunj Raghuvanshi, and Lior Shapira. 2014. RoomAlive: magical experiences enabled by scalable, adaptive projector-camera units. In Proc. UIST. 637--644. https://doi.org/10.1145/2642918.2647383 Google ScholarGoogle ScholarDigital LibraryDigital Library
  11. Annie Kelly, R. Benjamin Shapiro, Jonathan de Halleux, and Thomas Ball. 2018. ARcadia: A Rapid Prototyping Platform for Real-time Tangible Interfaces. In Proc. CHI , , Regan L. Mandryk, Mark Hancock, Mark Perry, and Anna L. Cox (Eds.). ACM, 409. https://doi.org/10.1145/3173574.3173983 Google ScholarGoogle ScholarDigital LibraryDigital Library
  12. Han-Jong Kim, Ju-Whan Kim, and Tek-Jin Nam. 2016. MiniStudio: Designers' Tool for Prototyping Ubicomp Space with Interactive Miniature. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems (San Jose, California, USA) (CHI '16). Association for Computing Machinery, New York, NY, USA, 213--224. https://doi.org/10.1145/2858036.2858180 Google ScholarGoogle ScholarDigital LibraryDigital Library
  13. Volodymyr Kurbatov. 2017. Draw Sketches for Virtual Reality Like a Pro. Available from Medium at https://medium.com/inborn-experience/vr-sketches-56599f99b357 .Google ScholarGoogle Scholar
  14. Germán Leiva, Cuong Nguyen, Rubaiat Habib Kazi, and Paul Asente. 2020. Pronto: Rapid Augmented Reality Video Prototyping Using Sketches and Enaction. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems (Honolulu, HI, USA) (CHI '20). Association for Computing Machinery, New York, NY, USA, 1--13. https://doi.org/10.1145/3313831.3376160 Google ScholarGoogle ScholarDigital LibraryDigital Library
  15. Blair MacIntyre, Maribeth Gandy, Steven Dow, and Jay David Bolter. 2004. DART: a toolkit for rapid design exploration of augmented reality experiences. In Proc. UIST . https://doi.org/10.1145/1029632.1029669 Google ScholarGoogle ScholarDigital LibraryDigital Library
  16. Brad A. Myers, Scott E. Hudson, and Randy F. Pausch. 2000. Past, present, and future of user interface software tools. TOCHI , Vol. 7, 1 (2000), 3--28. https://doi.org/10.1145/344949.344959 Google ScholarGoogle ScholarDigital LibraryDigital Library
  17. Michael Nebeling, Katy Lewis, Yu-Cheng Chang, Lihan Zhu, Michelle Chung, Piaoyang Wang, and Janet Nebeling. 2020. XRDirector: A Role-Based Collaborative Immersive Authoring System. In Proc. CHI . ACM, New York, NY, USA, 1--12. https://doi.org/10.1145/3313831.3376637 Google ScholarGoogle ScholarDigital LibraryDigital Library
  18. Michael Nebeling and Katy Madier. 2019. 360proto: Making Interactive Virtual Reality & Augmented Reality Prototypes from Paper (CHI '19). Association for Computing Machinery, New York, NY, USA, 1--13. https://doi.org/10.1145/3290605.3300826 Google ScholarGoogle ScholarDigital LibraryDigital Library
  19. Michael Nebeling, Janet Nebeling, Ao Yu, and Rob Rumble. 2018. ProtoAR: Rapid Physical-Digital Prototyping of Mobile Augmented Reality Applications. In Proc. CHI . Google ScholarGoogle ScholarDigital LibraryDigital Library
  20. M. Nebeling and M. Speicher. 2018. The Trouble with Augmented Reality/Virtual Reality Authoring Tools. In 2018 IEEE International Symposium on Mixed and Augmented Reality Adjunct (ISMAR-Adjunct) . 333--337.Google ScholarGoogle Scholar
  21. Gregory M Nielson and Dan R Olsen Jr. 1987. Direct manipulation techniques for 3D objects using 2D locator devices. In Proc. Workshop on Interactive 3D graphics . Google ScholarGoogle ScholarDigital LibraryDigital Library
  22. Ji-Young Oh and Wolfgang Stuerzlinger. 2005. Moving objects with 2D input devices in CAD systems and desktop virtual environments. In Proc. Graphics Interface . Google ScholarGoogle ScholarDigital LibraryDigital Library
  23. Marc Rettig. 1994. Prototyping for Tiny Fingers. Commun. ACM , Vol. 37, 4 (1994), 21--27. https://doi.org/10.1145/175276.175288 Google ScholarGoogle ScholarDigital LibraryDigital Library
  24. Carolyn Snyder. 2003. Paper prototyping: The fast and easy way to design and refine user interfaces .Morgan Kaufmann. Google ScholarGoogle ScholarDigital LibraryDigital Library
  25. Kihoon Son, Hwiwon Chun, Sojin Park, and Kyung Hoon Hyun. 2020. C-Space: An Interactive Prototyping Platform for Collaborative Spatial Design Exploration. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems (Honolulu, HI, USA) (CHI '20). Association for Computing Machinery, New York, NY, USA, 1--13. https://doi.org/10.1145/3313831.3376452 Google ScholarGoogle ScholarDigital LibraryDigital Library
  26. Maximilian Speicher and Michael Nebeling. 2018. GestureWiz: A Human-Powered Gesture Design Environment for User Interface Prototypes. In Proc. CHI . Google ScholarGoogle ScholarDigital LibraryDigital Library
  27. Pieter Jan Stappers and Elisa Giaccardi. [n.d.]. Research through Design. Available from https://www.interaction-design.org/literature/book/the-encyclopedia-of-human-computer-interaction-2nd-ed/research-through-design .Google ScholarGoogle Scholar
  28. Ryo Suzuki, Rubaiat Habib Kazi, Li-Yi Wei, Stephen DiVerdi, Wilmot Li, and Daniel Leithinger. 2020. RealitySketch: Embedding Responsive Graphics and Visualizations in AR through Dynamic Sketching. In Proc. UIST , , Shamsi T. Iqbal, Karon E. MacLean, Fanny Chevalier, and Stefanie Mueller (Eds.). ACM , 166--181. https://doi.org/10.1145/3379337.3415892 Google ScholarGoogle ScholarDigital LibraryDigital Library
  29. Nicolas Villar, Daniel Cletheroe, Greg Saul, Christian Holz, Tim Regan, Oscar Salandin, Misha Sra, Hui-Shyong Yeo, William Field, and Haiyan Zhang. 2018. Project Zanzibar: A Portable and Flexible Tangible Interaction Platform. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems (Montreal QC, Canada) (CHI '18). Association for Computing Machinery, New York, NY, USA, 1--13. https://doi.org/10.1145/3173574.3174089 Google ScholarGoogle ScholarDigital LibraryDigital Library
  30. Karl D.D. Willis, Ivan Poupyrev, and Takaaki Shiratori. 2011. Motionbeam: A Metaphor for Character Interaction with Handheld Projectors. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (Vancouver, BC, Canada) (CHI '11). Association for Computing Machinery, New York, NY, USA, 1031--1040. https://doi.org/10.1145/1978942.1979096 Google ScholarGoogle ScholarDigital LibraryDigital Library

Index Terms

  1. Designers, the Stage Is Yours! Medium-Fidelity Prototyping of Augmented & Virtual Reality Interfaces with 360theater

        Recommendations

        Comments

        Login options

        Check if you have access through your login credentials or your institution to get full access on this article.

        Sign in

        Full Access

        PDF Format

        View or Download as a PDF file.

        PDF

        eReader

        View online with eReader.

        eReader
        About Cookies On This Site

        We use cookies to ensure that we give you the best experience on our website.

        Learn more

        Got it!