Abstract
While augmented and virtual reality technologies are becoming mainstream, it is still technically challenging and time-consuming to create new applications. Many designers draw from traditional low-fidelity prototyping methods that do not lend themselves well to designing in 3D. Developers use high-end programming frameworks such as Unity and Unreal which require significant hardware/software setups and coding skills. We see a gap in the medium-fidelity range where there is an opportunity for new tools to leverage the advantages of 360° content for AR/VR prototyping. Existing tools, however, have only limited support for 3D geometry, spatial and proxemic interactions, puppeteering, and storytelling. We present 360theater, a new method and a tool for rapid prototyping of AR/VR experiences, which takes dioramas into the virtual realm by enhancing 360° video capture with 3D geometry and simulating spatial interactions via Wizard of Oz. Our comparative evaluation of techniques with novice and experienced AR/VR designers shows that 360theater can close the gap and achieve a higher fidelity and more realistic AR/VR prototypes than comparable methods.
Supplemental Material
- Narges Ashtari, Andrea Bunt, Joanna McGrenere, Michael Nebeling, and Parmit K. Chilana. 2020. Creating Augmented and Virtual Reality Applications: Current Practices, Challenges, and Opportunities (CHI '20). Association for Computing Machinery, New York, NY, USA, 1--13. https://doi.org/10.1145/3313831.3376722 Google Scholar
Digital Library
- Till Ballendat, Nicolai Marquardt, and Saul Greenberg. 2010. Proxemic Interaction: Designing for a Proximity and Orientation-Aware Environment. In Proc. ITS . Google Scholar
Digital Library
- Davide Bolchini, Diego Pulido, and Anthony Faiola. 2009. Feature -- “Paper in screen” prototyping: an agile technique to anticipate the mobile experience. Interactions , Vol. 16, 4 (2009), 29--33. https://doi.org/10.1145/1551986.1551992 Google Scholar
Digital Library
- Bill Buxton. 2010. Sketching user experiences: getting the design right and the right design .Morgan Kaufmann. Google Scholar
Digital Library
- Steven Dow, Jaemin Lee, Christopher Oezbek, Blair MacIntyre, Jay David Bolter, and Maribeth Gandy. 2005 a. Wizard of Oz interfaces for mixed reality applications. In Proc. CHI Extended Abstracts. 1339--1342. https://doi.org/10.1145/1056808.1056911 Google Scholar
Digital Library
- Steven Dow, Blair MacIntyre, Jaemin Lee, Christopher Oezbek, Jay David Bolter, and Maribeth Gandy. 2005 b. Wizard of Oz support throughout an iterative design process. IEEE Pervasive Computing , Vol. 4, 4 (2005), 18--26. https://doi.org/10.1109/MPRV.2005.93 Google Scholar
Digital Library
- Maribeth Gandy and Blair MacIntyre. 2014. Designer's augmented reality toolkit, ten years later: implications for new media authoring tools. In Proc. UIST . Google Scholar
Digital Library
- Terrell Glenn, Ananya Ipsita, Caleb Carithers, Kylie Peppler, and Karthik Ramani. 2020. StoryMakAR: Bringing Stories to Life With An Augmented Reality & Physical Prototyping Toolkit for Youth. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems (Honolulu, HI, USA) (CHI '20). Association for Computing Machinery, New York, NY, USA, 1--14. https://doi.org/10.1145/3313831.3376790 Google Scholar
Digital Library
- Andrew J. Hunsucker, Kelly McClinton, Jennifer Wang, and Erik Stolterman. 2017. Augmented Reality Prototyping For Interaction Design Students. In Proc. CHI Extended Abstracts. 1018--1023. https://doi.org/10.1145/3027063.3053684 Google Scholar
Digital Library
- Brett R. Jones, Rajinder Sodhi, Michael Murdock, Ravish Mehra, Hrvoje Benko, Andrew Wilson, Eyal Ofek, Blair MacIntyre, Nikunj Raghuvanshi, and Lior Shapira. 2014. RoomAlive: magical experiences enabled by scalable, adaptive projector-camera units. In Proc. UIST. 637--644. https://doi.org/10.1145/2642918.2647383 Google Scholar
Digital Library
- Annie Kelly, R. Benjamin Shapiro, Jonathan de Halleux, and Thomas Ball. 2018. ARcadia: A Rapid Prototyping Platform for Real-time Tangible Interfaces. In Proc. CHI , , Regan L. Mandryk, Mark Hancock, Mark Perry, and Anna L. Cox (Eds.). ACM, 409. https://doi.org/10.1145/3173574.3173983 Google Scholar
Digital Library
- Han-Jong Kim, Ju-Whan Kim, and Tek-Jin Nam. 2016. MiniStudio: Designers' Tool for Prototyping Ubicomp Space with Interactive Miniature. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems (San Jose, California, USA) (CHI '16). Association for Computing Machinery, New York, NY, USA, 213--224. https://doi.org/10.1145/2858036.2858180 Google Scholar
Digital Library
- Volodymyr Kurbatov. 2017. Draw Sketches for Virtual Reality Like a Pro. Available from Medium at https://medium.com/inborn-experience/vr-sketches-56599f99b357 .Google Scholar
- Germán Leiva, Cuong Nguyen, Rubaiat Habib Kazi, and Paul Asente. 2020. Pronto: Rapid Augmented Reality Video Prototyping Using Sketches and Enaction. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems (Honolulu, HI, USA) (CHI '20). Association for Computing Machinery, New York, NY, USA, 1--13. https://doi.org/10.1145/3313831.3376160 Google Scholar
Digital Library
- Blair MacIntyre, Maribeth Gandy, Steven Dow, and Jay David Bolter. 2004. DART: a toolkit for rapid design exploration of augmented reality experiences. In Proc. UIST . https://doi.org/10.1145/1029632.1029669 Google Scholar
Digital Library
- Brad A. Myers, Scott E. Hudson, and Randy F. Pausch. 2000. Past, present, and future of user interface software tools. TOCHI , Vol. 7, 1 (2000), 3--28. https://doi.org/10.1145/344949.344959 Google Scholar
Digital Library
- Michael Nebeling, Katy Lewis, Yu-Cheng Chang, Lihan Zhu, Michelle Chung, Piaoyang Wang, and Janet Nebeling. 2020. XRDirector: A Role-Based Collaborative Immersive Authoring System. In Proc. CHI . ACM, New York, NY, USA, 1--12. https://doi.org/10.1145/3313831.3376637 Google Scholar
Digital Library
- Michael Nebeling and Katy Madier. 2019. 360proto: Making Interactive Virtual Reality & Augmented Reality Prototypes from Paper (CHI '19). Association for Computing Machinery, New York, NY, USA, 1--13. https://doi.org/10.1145/3290605.3300826 Google Scholar
Digital Library
- Michael Nebeling, Janet Nebeling, Ao Yu, and Rob Rumble. 2018. ProtoAR: Rapid Physical-Digital Prototyping of Mobile Augmented Reality Applications. In Proc. CHI . Google Scholar
Digital Library
- M. Nebeling and M. Speicher. 2018. The Trouble with Augmented Reality/Virtual Reality Authoring Tools. In 2018 IEEE International Symposium on Mixed and Augmented Reality Adjunct (ISMAR-Adjunct) . 333--337.Google Scholar
- Gregory M Nielson and Dan R Olsen Jr. 1987. Direct manipulation techniques for 3D objects using 2D locator devices. In Proc. Workshop on Interactive 3D graphics . Google Scholar
Digital Library
- Ji-Young Oh and Wolfgang Stuerzlinger. 2005. Moving objects with 2D input devices in CAD systems and desktop virtual environments. In Proc. Graphics Interface . Google Scholar
Digital Library
- Marc Rettig. 1994. Prototyping for Tiny Fingers. Commun. ACM , Vol. 37, 4 (1994), 21--27. https://doi.org/10.1145/175276.175288 Google Scholar
Digital Library
- Carolyn Snyder. 2003. Paper prototyping: The fast and easy way to design and refine user interfaces .Morgan Kaufmann. Google Scholar
Digital Library
- Kihoon Son, Hwiwon Chun, Sojin Park, and Kyung Hoon Hyun. 2020. C-Space: An Interactive Prototyping Platform for Collaborative Spatial Design Exploration. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems (Honolulu, HI, USA) (CHI '20). Association for Computing Machinery, New York, NY, USA, 1--13. https://doi.org/10.1145/3313831.3376452 Google Scholar
Digital Library
- Maximilian Speicher and Michael Nebeling. 2018. GestureWiz: A Human-Powered Gesture Design Environment for User Interface Prototypes. In Proc. CHI . Google Scholar
Digital Library
- Pieter Jan Stappers and Elisa Giaccardi. [n.d.]. Research through Design. Available from https://www.interaction-design.org/literature/book/the-encyclopedia-of-human-computer-interaction-2nd-ed/research-through-design .Google Scholar
- Ryo Suzuki, Rubaiat Habib Kazi, Li-Yi Wei, Stephen DiVerdi, Wilmot Li, and Daniel Leithinger. 2020. RealitySketch: Embedding Responsive Graphics and Visualizations in AR through Dynamic Sketching. In Proc. UIST , , Shamsi T. Iqbal, Karon E. MacLean, Fanny Chevalier, and Stefanie Mueller (Eds.). ACM , 166--181. https://doi.org/10.1145/3379337.3415892 Google Scholar
Digital Library
- Nicolas Villar, Daniel Cletheroe, Greg Saul, Christian Holz, Tim Regan, Oscar Salandin, Misha Sra, Hui-Shyong Yeo, William Field, and Haiyan Zhang. 2018. Project Zanzibar: A Portable and Flexible Tangible Interaction Platform. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems (Montreal QC, Canada) (CHI '18). Association for Computing Machinery, New York, NY, USA, 1--13. https://doi.org/10.1145/3173574.3174089 Google Scholar
Digital Library
- Karl D.D. Willis, Ivan Poupyrev, and Takaaki Shiratori. 2011. Motionbeam: A Metaphor for Character Interaction with Handheld Projectors. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (Vancouver, BC, Canada) (CHI '11). Association for Computing Machinery, New York, NY, USA, 1031--1040. https://doi.org/10.1145/1978942.1979096 Google Scholar
Digital Library
Index Terms
Designers, the Stage Is Yours! Medium-Fidelity Prototyping of Augmented & Virtual Reality Interfaces with 360theater
Recommendations
Elements of XR Prototyping: Characterizing the Role and Use of Prototypes in Augmented and Virtual Reality Design
CHI '22: Proceedings of the 2022 CHI Conference on Human Factors in Computing SystemsCurrent research in augmented, virtual, and mixed reality (XR) reveals a lack of tool support for designing and, in particular, prototyping XR applications. While recent tools research is often motivated by studying the requirements of non-technical ...
An Exploration of Low-Fidelity Prototyping Methods for Augmented and Virtual Reality
Design, User Experience, and Usability. Design for Contemporary Interactive EnvironmentsAbstractLow-fidelity prototyping is well-established method of developing concepts for traditional two-dimensional user interfaces. This paper considers the techniques and methods that can be used for simulating and early testing of augmented and virtual ...
Augmented and Virtual Reality Object Repository for Rapid Prototyping
Human-Centered Software EngineeringAbstractAugmented Reality (AR) and Virtual Reality (VR) are technologies on the rise and as their market grows, speeding up the process of developing new ideas becomes even more important. In achieving rapid development, rapid prototyping plays a very ...






Comments