Abstract
Pervasive interfaces can present relevant information anywhere in our environment, and they are thus challenged by the non rectilinearity of the display surface (e.g. circular table) and by the presence of objects that can partially occlude the interface (e.g. a book or cup on the table). To tackle this problem, we propose a novel solution based on two core contributions: the decomposition of the interface into deformable graphical units, called Dynamic Decals, and the control of their position and behaviour by a constraint-based approach. Our approach dynamically deforms the interface when needed while minimizing the impact on its visibility and layout properties. To do so, we extend previous work on implicit deformations to propose and experimentally validate functions defining different decal shapes and new deformers modeling decal deformations when they collide. Then, we interactively optimize the decal placements according to the interface geometry and their interrelations. Relations are modeled as constraints and the interface evolution results from an easy and efficient to solve minimization problem. Our approach is validated by a user study showing that, compared to two baselines, Dynamic decals is an aesthetically pleasant interface that preserves visibility, layout and aesthetic properties.
Supplemental Material
- Anand Agarawala and Ravin Balakrishnan. 2006. Keepin' It Real: Pushing the Desktop Metaphor with Physics, Piles and the Pen. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (Montréal, Québec, Canada) (CHI '06). Association for Computing Machinery, New York, NY, USA, 1283--1292. https://doi.org/10.1145/1124772.1124965Google Scholar
Digital Library
- Greg J. Badros, Alan Borning, and Peter J. Stuckey. 2001. The Cassowary Linear Arithmetic Constraint Solving Algorithm. ACM Trans. Comput.-Hum. Interact. , Vol. 8, 4 (Dec. 2001), 267--306. https://doi.org/10.1145/504704.504705Google Scholar
Digital Library
- Xiaojun Bi, Tovi Grossman, Justin Matejka, and George Fitzmaurice. 2011. Magic Desk: Bringing Multi-Touch Surfaces into Desktop Work. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (Vancouver, BC, Canada) (CHI '11). Association for Computing Machinery, New York, NY, USA, 2511--2520. https://doi.org/10.1145/1978942.1979309Google Scholar
Digital Library
- James F. Blinn. 1982. A Generalization of Algebraic Surface Drawing. ACM Trans. Graph. , Vol. 1, 3 (July 1982), 235--256. https://doi.org/10.1145/357306.357310Google Scholar
Digital Library
- Alan Borning, Richard Lin, and Kim Marriott. 1997. Constraints for the Web. In Proceedings of the Fifth ACM International Conference on Multimedia (Seattle, Washington, USA) (MULTIMEDIA '97). Association for Computing Machinery, New York, NY, USA, 173--182. https://doi.org/10.1145/266180.266361Google Scholar
Digital Library
- Ralph Allan Bradley and Milton E. Terry. 1952. Rank Analysis of Incomplete Block Designs: I. The Method of Paired Comparisons. Biometrika , Vol. 39, 3/4 (1952), 324--345. http://www.jstor.org/stable/2334029Google Scholar
- Kuan-Ta Chen, Chen-Chi Wu, Yu-Chun Chang, and Chin-Laung Lei. 2009. A Crowdsourceable QoE Evaluation Framework for Multimedia Content. In Proceedings of the 17th ACM International Conference on Multimedia (Beijing, China) (MM '09). Association for Computing Machinery, New York, NY, USA, 491--500. https://doi.org/10.1145/1631272.1631339Google Scholar
Digital Library
- Daniel Cotting, Markus Gross, and Markus Gross. 2006. Interactive Environment-aware Display Bubbles. In Proceedings of the 19th Annual ACM Symposium on User Interface Software and Technology (Montreux, Switzerland) (UIST '06). ACM, New York, NY, USA, 245--254. https://doi.org/10.1145/1166253.1166291Google Scholar
Digital Library
- Niraj Ramesh Dayama, Kashyap Todi, Taru Saarelainen, and Antti Oulasvirta. 2020. GRIDS: Interactive Layout Design with Integer Programming. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems (Honolulu, HI, USA) (CHI '20). Association for Computing Machinery, New York, NY, USA, 1--13. https://doi.org/10.1145/3313831.3376553Google Scholar
Digital Library
- Erwin de Groot, Brian Wyvill, Lo"ic Barthe, Ahmad Nasri, and Paul Lalonde. 2014. Implicit Decals: Interactive Editing of Repetitive Patterns on Surfaces. Comput. Graph. Forum , Vol. 33, 1 (Feb. 2014), 141--151. https://doi.org/10.1111/cgf.12260Google Scholar
Digital Library
- Joost de Winter and Dimitra Dodou. 2010. Five-Point Likert Items: t Test Versus Mann--Whitney--Wilcoxon. Practical Assessment, Research and Evaluation , Vol. 15 (01 2010).Google Scholar
- Peitong Duan, Casimir Wierzynski, and Lama Nachman. 2020. Optimizing User Interface Layouts via Gradient Descent. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems (Honolulu, HI, USA) (CHI '20). Association for Computing Machinery, New York, NY, USA, 1--12. https://doi.org/10.1145/3313831.3376589Google Scholar
Digital Library
- Barrett Ens, Benjamin Bach, Maxime Cordeil, Ulrich Engelke, Marcos Serrano, Wesley Willett, Arnaud Prouzeau, Christoph Anthes, Wolfgang Büschel, Cody Dunne, Tim Dwyer, Jens Grubert, Jason H. Haga, Nurit Kirshenbaum, Dylan Kobayashi, Tica Lin, Monsurat Olaosebikan, Fabian Pointecker, David Saffo, Nazmus Saquib, Dieter Schmalstieg, Danielle Albers Szafir, Matt Whitlock, and Yalong Yang. 2021. Grand Challenges in Immersive Analytics. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems (Yokohama, Japan) (CHI '21). Association for Computing Machinery, New York, NY, USA, Article 459, bibinfonumpages17 pages. https://doi.org/10.1145/3411764.3446866Google Scholar
Digital Library
- Genki Furumi, Daisuke Sakamoto, and Takeo Igarashi. 2012. SnapRail: A Tabletop User Interface Widget for Addressing Occlusion by Physical Objects. In Proceedings of the 2012 ACM International Conference on Interactive Tabletops and Surfaces (Cambridge, Massachusetts, USA) (ITS '12). Association for Computing Machinery, New York, NY, USA, 193--196. https://doi.org/10.1145/2396636.2396666Google Scholar
Digital Library
- Wilbert O. Galitz. 2007. The Essential Guide to User Interface Design: An Introduction to GUI Design Principles and Techniques .John Wiley & Sons, Inc., USA.Google Scholar
Digital Library
- Michael Gleicher. 1992 a. Briar: A Constraint-Based Drawing Program. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (Monterey, California, USA) (CHI '92). Association for Computing Machinery, New York, NY, USA, 661--662. https://doi.org/10.1145/142750.143074Google Scholar
Digital Library
- Michael Gleicher. 1992 b. Integrating constraints and direct manipulation . In Proceedings of the 1992 symposium on Interactive 3D graphics - SI3D '92. ACM Press, New York, New York, USA, 171--174. https://doi.org/10.1145/147156.147194Google Scholar
Digital Library
- Gaël Guennebaud, Benoît Jacob, et almbox. 2010. Eigen v3. http://eigen.tuxfamily.org.Google Scholar
- Alina Hang, Enrico Rukzio, and Andrew Greaves. 2008. Projector Phone: A Study of Using Mobile Phones with Integrated Projector for Interaction with Maps. In Proceedings of the 10th International Conference on Human Computer Interaction with Mobile Devices and Services (Amsterdam, The Netherlands) (MobileHCI '08). Association for Computing Machinery, New York, NY, USA, 207--216. https://doi.org/10.1145/1409240.1409263Google Scholar
Digital Library
- Hiroshi Hosobe. 2001. A modular geometric constraint solver for user interface applications. In Proceedings of the 14th annual ACM symposium on User interface software and technology - UIST '01. ACM Press, New York, New York, USA, 91. https://doi.org/10.1145/502348.502362Google Scholar
Digital Library
- Scott E. Hudson and Ian Smith. 1996. Ultra-Lightweight Constraints. In Proceedings of the 9th Annual ACM Symposium on User Interface Software and Technology (Seattle, Washington, USA) (UIST '96). Association for Computing Machinery, New York, NY, USA, 147--155. https://doi.org/10.1145/237091.237112Google Scholar
Digital Library
- Waqas Javed, KyungTae Kim, Sohaib Ghani, and Niklas Elmqvist. 2011. Evaluating Physical/Virtual Occlusion Management Techniques for Horizontal Displays. In Human-Computer Interaction -- INTERACT 2011 , , Pedro Campos, Nicholas Graham, Joaquim Jorge, Nuno Nunes, Philippe Palanque, and Marco Winckler (Eds.). Springer Berlin Heidelberg, Berlin, Heidelberg, 391--408.Google Scholar
- Tyler Johnson and Henry Fuchs. 2007. Real-Time Projector Tracking on Complex Geometry Using Ordinary Imagery. In 2007 IEEE Conference on Computer Vision and Pattern Recognition. 1--8. https://doi.org/10.1109/CVPR.2007.383460Google Scholar
- Brett Jones, Hrvoje Benko, Eyal Ofek, and Andrew Wilson. 2013. IllumiRoom: peripheral projected illusions for interactive experiences. https://doi.org/10.1145/2503368.2503375Google Scholar
Digital Library
- Brett Jones, Rajinder Sodhi, Michael Murdock, Ravish Mehra, Hrvoje Benko, Andrew Wilson, Eyal Ofek, Blair MacIntyre, Nikunj Raghuvanshi, and Lior Shapira. 2014. RoomAlive: Magical Experiences Enabled by Scalable, Adaptive Projector-Camera Units. In Proceedings of the 27th Annual ACM Symposium on User Interface Software and Technology (Honolulu, Hawaii, USA) (UIST '14). Association for Computing Machinery, New York, NY, USA, 637--644. https://doi.org/10.1145/2642918.2647383Google Scholar
Digital Library
- Brett R. Jones, Rajinder Sodhi, Roy H. Campbell, Guy Garnett, and Brian P. Bailey. 2010. Build your world and play in it: Interacting with surface particles on complex objects. In 2010 IEEE International Symposium on Mixed and Augmented Reality . 165--174. https://doi.org/10.1109/ISMAR.2010.5643566Google Scholar
Cross Ref
- Mohammadreza Khalilbeigi, Jürgen Steimle, Jan Riemann, Niloofar Dezfuli, Max Mühlh"auser, and James D. Hollan. 2013. ObjecTop: Occlusion Awareness of Physical Objects on Interactive Tabletops. In Proceedings of the 2013 ACM International Conference on Interactive Tabletops and Surfaces (St. Andrews, Scotland, United Kingdom) (ITS '13). Association for Computing Machinery, New York, NY, USA, 255--264. https://doi.org/10.1145/2512349.2512806Google Scholar
Digital Library
- Masaaki Kurosu and Kaori Kashimura. 1995. Apparent Usability vs. Inherent Usability: Experimental Analysis on the Determinants of the Apparent Usability. In Conference Companion on Human Factors in Computing Systems (Denver, Colorado, USA) (CHI '95). Association for Computing Machinery, New York, NY, USA, 292--293. https://doi.org/10.1145/223355.223680Google Scholar
Digital Library
- Ricardo Langner, John Brosz, Raimund Dachselt, and Sheelagh Carpendale. 2010. PhysicsBox: Playful Educational Tabletop Games. In ACM International Conference on Interactive Tabletops and Surfaces (Saarbrücken, Germany) (ITS '10). Association for Computing Machinery, New York, NY, USA, 273--274. https://doi.org/10.1145/1936652.1936712Google Scholar
Digital Library
- Talia Lavie and Noam Tractinsky. 2004. Assessing dimensions of perceived visual aesthetics of web sites. International Journal of Human-Computer Studies , Vol. 60, 3 (2004), 269 -- 298. https://doi.org/10.1016/j.ijhcs.2003.09.002Google Scholar
Digital Library
- Johnny C. Lee, Scott E. Hudson, Jay W. Summet, and Paul H. Dietz. 2005. Moveable Interactive Projected Displays Using Projector Based Tracking. In Proceedings of the 18th Annual ACM Symposium on User Interface Software and Technology (Seattle, WA, USA) (UIST '05). Association for Computing Machinery, New York, NY, USA, 63--72. https://doi.org/10.1145/1095034.1095045Google Scholar
Digital Library
- Y. K. Leung and M. D. Apperley. 1994. A Review and Taxonomy of Distortion-Oriented Presentation Techniques. ACM Trans. Comput.-Hum. Interact. , Vol. 1, 2 (June 1994), 126--160. https://doi.org/10.1145/180171.180173Google Scholar
Digital Library
- Nicolas Mellado, David Vanderhaeghe, Charlotte Hoarau, Sidonie Christophe, Mathieu Brédif, and Loic Barthe. 2017. Constrained Palette-Space Exploration. ACM Trans. Graph. , Vol. 36, 4, Article 60 (July 2017), bibinfonumpages14 pages. https://doi.org/10.1145/3072959.3073650Google Scholar
Digital Library
- Jan Meskens, Jo Vermeulen, Kris Luyten, and Karin Coninx. 2008. Gummy for Multi-Platform User Interface Designs: Shape Me, Multiply Me, Fix Me, Use Me. In Proceedings of the Working Conference on Advanced Visual Interfaces (Napoli, Italy) (AVI '08). Association for Computing Machinery, New York, NY, USA, 233--240. https://doi.org/10.1145/1385569.1385607Google Scholar
Digital Library
- M. Müller. 2008. Hierarchical Position Based Dynamics. In VRIPHYS .Google Scholar
- Miguel A. Nacenta, Satoshi Sakurai, Tokuo Yamaguchi, Yohei Miki, Yuichi Itoh, Yoshifumi Kitamura, Sriram Subramanian, and Carl Gutwin. 2007. E-Conic: A Perspective-Aware Interface for Multi-Display Environments. In Proceedings of the 20th Annual ACM Symposium on User Interface Software and Technology (Newport, Rhode Island, USA) (UIST '07). Association for Computing Machinery, New York, NY, USA, 279--288. https://doi.org/10.1145/1294211.1294260Google Scholar
Digital Library
- David Chek Ling Ngo, Lian Seng Teo, and John G. Byrne. 2003. Modelling Interface Aesthetics. Inf. Sci. , Vol. 152, 1 (June 2003), 25--46. https://doi.org/10.1016/S0020-0255(02)00404--8Google Scholar
- Stephen Oney, Brad Myers, and Joel Brandt. 2012. ConstraintJS. In Proceedings of the 25th annual ACM symposium on User interface software and technology - UIST '12. ACM Press, New York, New York, USA, 229. https://doi.org/10.1145/2380116.2380146Google Scholar
Digital Library
- Ramesh Raskar, Michael S. Brown, Ruigang Yang, Wei-Chao Chen, Greg Welch, Herman Towles, Brent Seales, and Henry Fuchs. 1999. Multi-Projector Displays Using Camera-Based Registration. In Proceedings of the Conference on Visualization '99: Celebrating Ten Years (San Francisco, California, USA) (VIS '99). IEEE Computer Society Press, Washington, DC, USA, 161--168.Google Scholar
Digital Library
- Ramesh Raskar, Greg Welch, Matt Cutts, Adam Lake, Lev Stesin, and Henry Fuchs. 1998. The Office of the Future: A Unified Approach to Image-Based Modeling and Spatially Immersive Displays. In Proceedings of the 25th Annual Conference on Computer Graphics and Interactive Techniques (SIGGRAPH '98). Association for Computing Machinery, New York, NY, USA, 179--188. https://doi.org/10.1145/280814.280861Google Scholar
Digital Library
- Jan Riemann, Martin Schmitz, Alexander Hendrich, and Max Mühlh"auser. 2018. FlowPut: Environment-Aware Interactivity for Tangible 3D Objects. Proc. ACM Interact. Mob. Wearable Ubiquitous Technol. , Vol. 2, 1, Article 31 (March 2018), bibinfonumpages23 pages. https://doi.org/10.1145/3191763Google Scholar
Digital Library
- Marcos Serrano and Laurence Nigay. 2009. Temporal Aspects of CARE-Based Multimodal Fusion: From a Fusion Mechanism to Composition Components and WoZ Components. In Proceedings of the 2009 International Conference on Multimodal Interfaces (Cambridge, Massachusetts, USA) (ICMI-MLMI '09). Association for Computing Machinery, New York, NY, USA, 177--184. https://doi.org/10.1145/1647314.1647346Google Scholar
Digital Library
- Marcos Serrano and Laurence Nigay. 2010. A wizard of oz component-based approach for rapidly prototyping and testing input multimodal interfaces. Journal on Multimodal User Interfaces, Springer Publ. , Vol. 3, 3 (2010), 215--225. http://www.springerlink.com/content/f116502x844117t6Google Scholar
- Marcos Serrano, Anne Roudaut, and Pourang Irani. 2016. Investigating Text Legibility on Non-Rectangular Displays. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems (San Jose, California, USA) (CHI '16). ACM, New York, NY, USA, 498--508. https://doi.org/10.1145/2858036.2858057Google Scholar
Digital Library
- Marcos Serrano, Anne Roudaut, and Pourang Irani. 2017. Visual Composition of Graphical Elements on Non-Rectangular Displays. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems (Denver, Colorado, USA) (CHI '17). ACM, New York, NY, USA, 4405--4416. https://doi.org/10.1145/3025453.3025677Google Scholar
Digital Library
- Florine Simon, Anne Roudaut, Pourang Irani, and Marcos Serrano. 2019. Finding Information on Non-Rectangular Interfaces. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems (Glasgow, Scotland Uk) (CHI '19). ACM, New York, NY, USA, Article 102, bibinfonumpages8 pages. https://doi.org/10.1145/3290605.3300332Google Scholar
Digital Library
- S. Sistare. 1991. Graphical Interaction Techniques in Constraint-Based Geometric Modeling. In Proceedings of Graphics Interface '91 (Calgary, Alberta, Canada) (GI '91). Canadian Man-Computer Communications Society, Toronto, Ontario, Canada, 85--92. http://graphicsinterface.org/wp-content/uploads/gi1991--12.pdfGoogle Scholar
- Wolfgang Stuerzlinger, Olivier Chapuis, Dusty Phillips, and Nicolas Roussel. 2006. User Interface FaccAdes: Towards Fully Adaptable User Interfaces. In Proceedings of the 19th Annual ACM Symposium on User Interface Software and Technology (Montreux, Switzerland) (UIST '06). Association for Computing Machinery, New York, NY, USA, 309--318. https://doi.org/10.1145/1166253.1166301Google Scholar
Digital Library
- Ivan E. Sutherland. 1964. Sketch Pad a Man-Machine Graphical Communication System. In Proceedings of the SHARE Design Automation Workshop (DAC '64). Association for Computing Machinery, New York, NY, USA, 6.329--6.346. https://doi.org/10.1145/800265.810742Google Scholar
Digital Library
- Amanda Swearngin, Chenglong Wang, Alannah Oleson, James Fogarty, and Amy J. Ko. 2020. Scout: Rapid Exploration of Interface Layout Alternatives through High-Level Design Constraints. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems (Honolulu, HI, USA) (CHI '20). Association for Computing Machinery, New York, NY, USA, 1--13. https://doi.org/10.1145/3313831.3376593Google Scholar
Digital Library
- Aurelien Tabard, Simon Gurn, Andreas Butz, and Jakob Bardram. 2013. A Case Study of Object and Occlusion Management on the ELabBench, a Mixed Physical/Digital Tabletop. In Proceedings of the 2013 ACM International Conference on Interactive Tabletops and Surfaces (St. Andrews, Scotland, United Kingdom) (ITS '13). Association for Computing Machinery, New York, NY, USA, 251--254. https://doi.org/10.1145/2512349.2512794Google Scholar
Digital Library
- Daniel Vogel and Ravin Balakrishnan. 2010. Occlusion-Aware Interfaces. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (Atlanta, Georgia, USA) (CHI '10). Association for Computing Machinery, New York, NY, USA, 263--272. https://doi.org/10.1145/1753326.1753365Google Scholar
Digital Library
- Manuela Waldner, Raphael Grasset, Markus Steinberger, and Dieter Schmalstieg. 2011. Display-Adaptive Window Management for Irregular Surfaces. In Proceedings of the ACM International Conference on Interactive Tabletops and Surfaces (Kobe, Japan) (ITS '11). Association for Computing Machinery, New York, NY, USA, 222--231. https://doi.org/10.1145/2076354.2076394Google Scholar
Digital Library
- Andrew D. Wilson, Shahram Izadi, Otmar Hilliges, Armando Garcia-Mendoza, and David Kirk. 2008. Bringing Physics to the Surface. In Proceedings of the 21st Annual ACM Symposium on User Interface Software and Technology (Monterey, CA, USA) (UIST '08). Association for Computing Machinery, New York, NY, USA, 67--76. https://doi.org/10.1145/1449715.1449728Google Scholar
Digital Library
- Raphael Wimmer, Fabian Hennecke, Florian Schulz, Sebastian Boring, Andreas Butz, and Heinrich Hußmann. 2010. Curve: Revisiting the Digital Desk. In Proceedings of the 6th Nordic Conference on Human-Computer Interaction: Extending Boundaries (Reykjavik, Iceland) (NordiCHI '10). Association for Computing Machinery, New York, NY, USA, 561--570. https://doi.org/10.1145/1868914.1868977Google Scholar
Digital Library
- Geoff Wyvill, Craig McPheeters, and Brian Wyvill. 1986. Data Structure for Soft Objects . The Visual Computer , Vol. 2, 4 (February 1986), 227--234.Google Scholar
Cross Ref
Index Terms
Dynamic Decals: Pervasive Freeform Interfaces Using Constrained Deformable Graphical Elements
Recommendations
Dynamic NURBS with geometric constraints for interactive sculpting
Special issue on interactive sculptingThis article develops a dynamic generalization of the nonuniform rational B-spline (NURBS) model. NURBS have become a defacto standard in commercial modeling systems because of their power to represent free-form shapes as well as common analytic shapes. ...
Declarative programming of graphical interfaces by visual examples
UIST '92: Proceedings of the 5th annual ACM symposium on User interface software and technologyGraphical user interfaces (GUI) provide intuitive and easy means for users to communicate with computers. However, construction of GUI software requires complex programming that is far from being intuitive. Because of the “semantic gap” between the ...
On interfaces projected onto real-world objects
CHI EA '03: CHI '03 Extended Abstracts on Human Factors in Computing SystemsThis paper describes preliminary results of research on the perception and usability of interfaces projected onto real-world objects. Using a projector setup that enables us to compare users' color preferences, we show that the objects onto which colors ...






Comments