skip to main content
research-article

ImageSense: An Intelligent Collaborative Ideation Tool to Support Diverse Human-Computer Partnerships

Published:29 May 2020Publication History
Skip Abstract Section

Abstract

Professional designers create mood boards to explore, visualize, and communicate hard-to-express ideas. We present ImageCascade, an intelligent, collaborative ideation tool that combines individual and shared work spaces, as well as collaboration with multiple forms of intelligent agents. In the collection phase, ImageCascade offers fluid transitions between serendipitous discovery of curated images via ImageCascade, combined text- and image-based Semantic search, and intelligent AI suggestions for finding new images. For later composition and reflection, ImageCascade provides semantic labels, generated color palettes, and multiple tag clouds to help communicate the intent of the mood board. A study of nine professional designers revealed nuances in designers' preferences for designer-led, system-led, and mixed-initiative approaches that evolve throughout the design process. We discuss the challenges in creating effective human-computer partnerships for creative activities, and suggest directions for future research.

Skip Supplemental Material Section

Supplemental Material

References

  1. Salvatore Andolina, Khalil Klouche, Diogo Cabral, Tuukka Ruotsalo, and Giulio Jacucci. 2015. InspirationWall: supporting idea generation through automatic information exploration. In Proceedings of the 2015 ACM SIGCHI Conference on Creativity and Cognition. ACM, 103--106.Google ScholarGoogle ScholarDigital LibraryDigital Library
  2. Salvatore Andolina, Hendrik Schneider, Joel Chan, Khalil Klouche, Giulio Jacucci, and Steven Dow. 2017. Crowdboard: augmenting in-person idea generation with real-time crowds. In Proceedings of the 2017 ACM SIGCHI Conference on Creativity and Cognition. 106--118.Google ScholarGoogle ScholarDigital LibraryDigital Library
  3. Patti Bao, Elizabeth Gerber, Darren Gergle, and David Hoffman. 2010. Momentum: getting and staying on topic during a brainstorm. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. ACM, 1233--1236.Google ScholarGoogle ScholarDigital LibraryDigital Library
  4. Brent Berlin and Paul Kay. 1991. Basic color terms: Their universality and evolution .Univ of California Press.Google ScholarGoogle Scholar
  5. Dan Bloomberg. 2019. Color quantization using modified median cut. (03 2019), 6.Google ScholarGoogle Scholar
  6. Susanne Bødker. 1991. Through the interface-A human activity approach to user interface design. DAIMI Report Series 224 (1991).Google ScholarGoogle Scholar
  7. Carole Bouchard, Jean-francois Omhover, Celine Mougenot, Ameziane Aoussat, and Stephen J Westerman. 2008. TRENDS: a content-based information retrieval system for designers. In Design Computing and Cognition'08. Springer, 593--611.Google ScholarGoogle Scholar
  8. Adrien Bousseau, Theophanis Tsandilas, Lora Oehlberg, and Wendy E. Mackay. 2016. How Novices Sketch and Prototype Hand-Fabricated Objects. In Conference on Human Factors in Computing Systems (CHI). ACM, San Jose, United States. https://doi.org/10.1145/2858036.2858159Google ScholarGoogle ScholarDigital LibraryDigital Library
  9. Virginia Braun and Victoria Clarke. 2006. Using thematic analysis in psychology. Qualitative research in psychology, Vol. 3, 2 (2006), 77--101.Google ScholarGoogle Scholar
  10. Search by Muzli. 2019. Muzli (2019). https://search.muz.liGoogle ScholarGoogle Scholar
  11. Hernan Casakin and Gabriela Goldschmidt. 1999. Expertise and the use of visual analogy: implications for design education. Design studies, Vol. 20, 2 (1999), 153--175.Google ScholarGoogle Scholar
  12. Tracy Cassidy. 2011. The mood board process modeled and understood as a qualitative design research tool. Fashion Practice, Vol. 3, 2 (2011).Google ScholarGoogle ScholarCross RefCross Ref
  13. Tracy Diane Cassidy. 2008. Mood boards: Current practice in learning and teaching strategies and students' understanding of the process. International Journal of Fashion Design, Vol. 1, 1 (2008), 43--54.Google ScholarGoogle ScholarCross RefCross Ref
  14. Joel Chan, Pao Siangliulue, Denisa Qori McDonald, Ruixue Liu, Reza Moradinezhad, Safa Aman, Erin T. Solovey, Krzysztof Z. Gajos, and Steven P. Dow. 2017. Semantically Far Inspirations Considered Harmful?: Accounting for Cognitive States in Collaborative Ideation. In Proceedings of the 2017 ACM SIGCHI Conference on Creativity and Cognition (Singapore, Singapore) (C&C '17). ACM, New York, NY, USA, 93--105.Google ScholarGoogle Scholar
  15. Jean Chevalier. 1996. The Penguin dictionary of symbols. (1996).Google ScholarGoogle Scholar
  16. Mihaly Csikszentmihalyi and Keith Sawyer. 2014. Shifting the focus from individual to organizational creativity. In The systems model of creativity. Springer, 67--71.Google ScholarGoogle Scholar
  17. Douglass R. Cutting, David R. Karger, Jan O. Pedersen, and John W. Tukey. 1992. Scatter/Gather: A Cluster-based Approach to Browsing Large Document Collections. In Proceedings of the 15th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (Copenhagen, Denmark) (SIGIR '92). ACM, New York, NY, USA, 318--329.Google ScholarGoogle Scholar
  18. Graham Dove, Nicolai Brodersen Hansen, and Kim Halskov. 2016. An argument for design space reflection. In Proceedings of the 9th Nordic Conference on Human-Computer Interaction. ACM, 17.Google ScholarGoogle ScholarDigital LibraryDigital Library
  19. Volodymyr Dziubak. 2019. Supporting graphic designers' online inspiration seeking practices via software tools. (2019).Google ScholarGoogle Scholar
  20. James Fogarty, Jodi Forlizzi, and Scott E Hudson. 2001. Aesthetic information collages: generating decorative displays that contain information. In Proceedings of the 14th annual ACM symposium on User interface software and technology. ACM, 141--150.Google ScholarGoogle ScholarDigital LibraryDigital Library
  21. James Fogarty, Desney Tan, Ashish Kapoor, and Simon Winder. 2008. CueFlik: Interactive Concept Learning in Image Search. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (Florence, Italy) (CHI '08). ACM, New York, NY, USA, 29--38.Google ScholarGoogle ScholarDigital LibraryDigital Library
  22. Charles Freeman, Sara Marcketti, and Elena Karpova. 2017. Creativity of images: using digital consensual assessment to evaluate mood boards. Fashion and Textiles, Vol. 4, 1 (2017), 17.Google ScholarGoogle ScholarCross RefCross Ref
  23. Jonas Frich, Lindsay MacDonald Vermeulen, Christian Remy, Michael Mose Biskjaer, and Peter Dalsgaard. 2019. Mapping the Landscape of Creativity Support Tools in HCI. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems (Glasgow, Scotland Uk) (CHI '19). ACM, New York, NY, USA, Article 389, 18 pages. https://doi.org/10.1145/3290605.3300619Google ScholarGoogle ScholarDigital LibraryDigital Library
  24. Alex Gabriel, Davy Monticolo, Mauricio Camargo, and Mario Bourgault. 2016. Creativity support systems: A systematic mapping study. Thinking Skills and Creativity, Vol. 21 (2016), 109--122.Google ScholarGoogle ScholarCross RefCross Ref
  25. John Gage. 1999. Color and meaning: Art, science, and symbolism .Univ of California Press.Google ScholarGoogle Scholar
  26. Steve Garner and Deana McDonagh-Philp. 2001. Problem interpretation and resolution via visual stimuli: the use of `mood boards' in design education. Journal of Art & Design Education, Vol. 20, 1 (2001), 57--64.Google ScholarGoogle ScholarCross RefCross Ref
  27. Karni Gilon, Joel Chan, Felicia Y. Ng, Hila Liifshitz-Assaf, Aniket Kittur, and Dafna Shahaf. 2018. Analogy Mining for Specific Design Needs. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems (Montreal QC, Canada) (CHI '18). ACM, New York, NY, USA, Article 121, 11 pages.Google ScholarGoogle ScholarDigital LibraryDigital Library
  28. Milene Goncc alves, Carlos Cardoso, and Petra Badke-Schaub. 2016. Inspiration choices that matter: the selection of external stimuli during ideation. Design Science, Vol. 2 (2016).Google ScholarGoogle Scholar
  29. Francois Guimbretière, Maureen Stone, and Terry Winograd. 2001. Fluid interaction with high-resolution wall-size displays. In Proceedings of the 14th annual ACM symposium on User interface software and technology. 21--30.Google ScholarGoogle ScholarDigital LibraryDigital Library
  30. Sonja Gutwein. 2013. Computer support for collaborative creativity. Beyond the Desktop (2013), 15.Google ScholarGoogle Scholar
  31. Joshua Hailpern, Erik Hinterbichler, Caryn Leppert, Damon Cook, and Brian P Bailey. 2007. TEAM STORM: demonstrating an interaction model for working with multiple ideas during creative group work. In Proceedings of the 6th ACM SIGCHI Conference on Creativity & Cognition. 193--202.Google ScholarGoogle ScholarDigital LibraryDigital Library
  32. Noe Vargas Hernandez, Jami J Shah, and Steven M Smith. 2010. Understanding design ideation mechanisms through multilevel aligned empirical studies. Design Studies, Vol. 31, 4 (2010), 382--410.Google ScholarGoogle ScholarCross RefCross Ref
  33. Otmar Hilliges, Lucia Terrenghi, Sebastian Boring, David Kim, Hendrik Richter, and Andreas Butz. 2007. Designing for collaborative creative problem solving. In Proceedings of the 6th ACM SIGCHI conference on Creativity & cognition. 137--146.Google ScholarGoogle ScholarDigital LibraryDigital Library
  34. Eric Horvitz. 1999. Principles of mixed-initiative user interfaces. In Proceedings of the SIGCHI conference on Human Factors in Computing Systems. 159--166.Google ScholarGoogle ScholarDigital LibraryDigital Library
  35. Incredibleart.org. 2019. Color Symbolism and Culture. https://www.incredibleart.org/lessons/middle/color2.htmGoogle ScholarGoogle Scholar
  36. Nanna Inie and Peter Dalsgaard. 2017. A Typology of Design Ideas. In Proceedings of the 2017 ACM SIGCHI Conference on Creativity and Cognition. ACM, 393--406.Google ScholarGoogle ScholarDigital LibraryDigital Library
  37. Mikhail Jacob, Alexander Zook, and Brian Magerko. 2013. Viewpoints AI: Procedurally Representing and Reasoning about Gestures.. In DiGRA Conference.Google ScholarGoogle Scholar
  38. Ben Jonson. 2005. Design ideation: the conceptual sketch in the digital age. Design studies, Vol. 26, 6 (2005), 613--624.Google ScholarGoogle Scholar
  39. AI Keller, GJ Pasman, and Pieter Jan Stappers. 2006. Collections designers keep: Collecting visual material for inspiration and reference. CoDesign, Vol. 2, 01 (2006), 17--33.Google ScholarGoogle ScholarCross RefCross Ref
  40. Gary Klein, Brian Moon, and Robert R Hoffman. 2006. Making sense of sensemaking 1: Alternative perspectives. IEEE intelligent systems, Vol. 21, 4 (2006), 70--73.Google ScholarGoogle Scholar
  41. Janin Koch, Andrés Lucero, Lena Hegemann, and Antti Oulasvirta. 2019. May AI?: Design Ideation with Cooperative Contextual Bandits. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems. ACM, 633.Google ScholarGoogle ScholarDigital LibraryDigital Library
  42. Janin Koch, Nicolas Taffin, Andrés Lucero, and Wendy Mackay. 2020. SemanticCollage: Enriching Digital Mood Board Design with Semantic Labels. In Proceedings of the Designing Interactive Systems Conference 2020 (DIS '20). ACM.Google ScholarGoogle ScholarDigital LibraryDigital Library
  43. John Kolko. 2010. Sensemaking and framing: A theoretical reflection on perspective in design synthesis. Design Research Society (2010).Google ScholarGoogle Scholar
  44. Rebecca Leis and Lauren Reinerman-Jones. 2015. Methodological implications of confederate use for experimentation in safety-critical domains. Procedia Manufacturing, Vol. 3 (2015), 1233--1240.Google ScholarGoogle ScholarCross RefCross Ref
  45. Antonios Liapis and Georgios N Yannakakis. 2016. Boosting computational creativity with human interaction in mixed-initiative co-creation tasks. In Proceedings of the ICCC workshop on Computational Creativity and Games, Vol. 310.Google ScholarGoogle Scholar
  46. Google LLC. 2018. Google Images Search. (2018). https://images.google.com/Google ScholarGoogle Scholar
  47. Andrés Lucero. 2012. Framing, aligning, paradoxing, abstracting, and directing: how design mood boards work. In Proceedings of the designing interactive systems conference. ACM, 438--447.Google ScholarGoogle ScholarDigital LibraryDigital Library
  48. Andrés Lucero. 2015. Funky-Design-Spaces: Interactive Environments for Creativity Inspired by Observing Designers Making Mood Boards. In Human-Computer Interaction. Springer, 474--492.Google ScholarGoogle Scholar
  49. Andrés Lucero, Dima Aliakseyeu, and Jean-Bernard Martens. 2007. Augmenting mood boards: flexible and intuitive interaction in the context of the design studio. In Second Annual IEEE International Workshop on Horizontal Interactive Human-Computer Systems (TABLETOP'07). IEEE, 147--154.Google ScholarGoogle Scholar
  50. Andrés Lucero, Dzmitry Aliakseyeu, Kees Overbeeke, and Jean-Bernard Martens. 2009. An interactive support tool to convey the intended message in asynchronous presentations. In Proceedings of the International Conference on Advances in Computer Enterntainment Technology. 11--18.Google ScholarGoogle ScholarDigital LibraryDigital Library
  51. Lena Mamykina, Linda Candy, and Ernest Edmonds. 2002. Collaborative creativity. Commun. ACM, Vol. 45, 10 (2002), 96--99.Google ScholarGoogle ScholarDigital LibraryDigital Library
  52. Ine Mols, Elise Van den Hoven, and Berry Eggen. 2016. Informing design for reflection: an overview of current everyday practices. In Proceedings of the 9th Nordic Conference on Human-Computer Interaction. ACM, 21.Google ScholarGoogle ScholarDigital LibraryDigital Library
  53. Philippaa MOTHERSILL and V. Michaela BOVE. 2018. An Ontology of Computational Tools for Design Activities. In Proceedings of Design Research (DRS). DRS.Google ScholarGoogle Scholar
  54. Jaap MJ Murre and Joeri Dros. 2015. Replication and analysis of Ebbinghaus' forgetting curve. PloS one, Vol. 10, 7 (2015).Google ScholarGoogle Scholar
  55. Changhoon Oh, Jungwoo Song, Jinhan Choi, Seonghyeon Kim, Sungwoo Lee, and Bongwon Suh. 2018. I Lead, You Help but Only with Enough Details: Understanding User Experience of Co-Creation with Artificial Intelligence. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems. ACM, 649.Google ScholarGoogle ScholarDigital LibraryDigital Library
  56. Alex F Osborn. 1953. Applied imagination. Charles Scribner, New York (1953).Google ScholarGoogle Scholar
  57. Rajan Patel. 2018. World around you with Google Lens and the Assistant. (2018). https://www.blog.google/products/google-lens/google-lens-real-time-answers-questions-about-world-around-you/Google ScholarGoogle Scholar
  58. Sharoda A Paul and Meredith Ringel Morris. 2011. Sensemaking in collaborative web search. Human-Computer Interaction, Vol. 26, 1--2 (2011), 72--122.Google ScholarGoogle ScholarCross RefCross Ref
  59. Charles Sanders Peirce. 1974. Collected papers of charles sanders peirce. Vol. 5. Harvard University Press.Google ScholarGoogle Scholar
  60. Unsplash: Beautiful Free Images & Pictures. 2019. Unsplash (2019). https://unsplash.com//Google ScholarGoogle Scholar
  61. Pinterest. 2019. Pinterest (2019). https://www.pinterest.comGoogle ScholarGoogle Scholar
  62. Peter Pirolli and Stuart Card. 2005. The sensemaking process and leverage points for analyst technology as identified through cognitive task analysis. In Proceedings of international conference on intelligence analysis, Vol. 5. McLean, VA, USA, 2--4.Google ScholarGoogle Scholar
  63. Martin Porcheron, Andrés Lucero, and Joel E. Fischer. 2016. Co-Curator: Designing for Mobile Ideation in Groups. In Proceedings of the 20th International Academic Mindtrek Conference (Tampere, Finland) (AcademicMindtrek '16). Association for Computing Machinery, New York, NY, USA, 226--234. https://doi.org/10.1145/2994310.2994350Google ScholarGoogle ScholarDigital LibraryDigital Library
  64. Image Finder High quality free stock images. 2019. ImageFinder (2019). https://imagefinder.co/Google ScholarGoogle Scholar
  65. Guillermo Rauch. 2019. Socket. io. Socket IO.[Online] http://socket. io (2019).Google ScholarGoogle Scholar
  66. Paul Resnick and Hal R. Varian. 1997. Recommender Systems. Commun. ACM, Vol. 40, 3 (March 1997), 56--58. https://doi.org/10.1145/245108.245121Google ScholarGoogle ScholarDigital LibraryDigital Library
  67. Donald A Schön. 2017. The reflective practitioner: How professionals think in action .Routledge.Google ScholarGoogle Scholar
  68. Moushumi Sharmin and Brian P Bailey. 2011. I reflect to improve my design: investigating the role and process of reflection in creative design. In Proceedings of the 8th ACM conference on Creativity and cognition. ACM, 389--390.Google ScholarGoogle ScholarDigital LibraryDigital Library
  69. Ben Shneiderman. 2000. Creating creativity: user interfaces for supporting innovation. ACM Transactions on Computer-Human Interaction (TOCHI), Vol. 7, 1 (2000), 114--138.Google ScholarGoogle ScholarDigital LibraryDigital Library
  70. Ben Shneiderman. 2007. Creativity support tools: Accelerating discovery and innovation. Commun. ACM, Vol. 50, 12 (2007), 20--32.Google ScholarGoogle ScholarDigital LibraryDigital Library
  71. Pao Siangliulue, Joel Chan, Steven P Dow, and Krzysztof Z Gajos. 2016. IdeaHound: improving large-scale collaborative ideation with crowd-powered real-time semantic modeling. In Proceedings of the 29th Annual Symposium on User Interface Software and Technology. 609--624.Google ScholarGoogle ScholarDigital LibraryDigital Library
  72. Pao Siangliulue, Joel Chan, Krzysztof Z. Gajos, and Steven P. Dow. 2015. Providing Timely Examples Improves the Quantity and Quality of Generated Ideas. In Proceedings of the 2015 ACM SIGCHI Conference on Creativity and Cognition (Glasgow, United Kingdom) (C&C '15). ACM, New York, NY, USA, 83--92.Google ScholarGoogle ScholarDigital LibraryDigital Library
  73. Shaun Singh. 2019. Death to Stock - When stock dies art thrives. https://deathtothestockphoto.com/Google ScholarGoogle Scholar
  74. Kyle Steinfeld. 2017. Dreams may come. In Acadia. 590--599.Google ScholarGoogle Scholar
  75. Timo Strohmann, Dominik Siemon, and Susanne Robra-Bissantz. 2017. brAInstorm: Intelligent Assistance in Group Idea Generation. In International Conference on Design Science Research in Information System and Technology. Springer, 457--461.Google ScholarGoogle ScholarCross RefCross Ref
  76. Lucy A Suchman. 2007. Human-machine reconfigurations: Plans and situated actions .Cambridge university press.Google ScholarGoogle ScholarDigital LibraryDigital Library
  77. Hillevi Sundholm, Henrik Artman, and Robert Ramberg. 2004. Backdoor Creativity: Collaborative Creativity in Technology Supported Teams.. In COOP. 99--114.Google ScholarGoogle Scholar
  78. Cem Tekin and Mihaela van der Schaar. 2015. Distributed online learning via cooperative contextual bandits. IEEE transactions on signal processing, Vol. 63, 14 (2015), 3700--3714.Google ScholarGoogle ScholarDigital LibraryDigital Library
  79. Belinda Thom. 2000. BoB: an interactive improvisational music companion. In Proceedings of the fourth international conference on Autonomous agents. ACM, 309--316.Google ScholarGoogle ScholarDigital LibraryDigital Library
  80. Inc. Twinword. 2018. Natural Language Processing APIs - Twinword API. https://www.twinword.com/api/word-associations.phpGoogle ScholarGoogle Scholar
  81. Peter-Paul Verbeek. 2015. COVER STORY Beyond interaction: a short introduction to mediation theory. interactions, Vol. 22, 3 (2015), 26--31.Google ScholarGoogle Scholar
  82. Hao-Chuan Wang, Dan Cosley, and Susan R Fussell. 2010. Idea expander: supporting group brainstorming with conversationally triggered visual thinking stimuli. In Proceedings of the 2010 ACM conference on Computer supported cooperative work. 103--106.Google ScholarGoogle ScholarDigital LibraryDigital Library
  83. Kai Wang and Jeffrey V Nickerson. 2017. A literature review on individual creativity support systems. Computers in Human Behavior, Vol. 74 (2017), 139--151.Google ScholarGoogle ScholarCross RefCross Ref
  84. Andrew M Webb, Rhema Linder, Andruid Kerne, Nic Lupfer, Yin Qu, Bryant Poffenberger, and Colton Revia. 2013. Promoting reflection and interpretation in education: Curating rich bookmarks as information composition. In Proceedings of the 9th ACM Conference on Creativity & Cognition. ACM, 53--62.Google ScholarGoogle ScholarDigital LibraryDigital Library
  85. Jun Xiao, Xuemei Zhang, Phil Cheatle, Yuli Gao, and C Brian Atkins. 2008. Mixed-initiative photo collage authoring. In Proceedings of the 16th ACM international conference on Multimedia. ACM, 509--518.Google ScholarGoogle ScholarDigital LibraryDigital Library
  86. Georgios N Yannakakis, Antonios Liapis, and Constantine Alexopoulos. 2014. Mixed-initiative co-creativity. In FDG.Google ScholarGoogle Scholar

Index Terms

  1. ImageSense: An Intelligent Collaborative Ideation Tool to Support Diverse Human-Computer Partnerships

        Recommendations

        Comments

        Login options

        Check if you have access through your login credentials or your institution to get full access on this article.

        Sign in

        Full Access

        PDF Format

        View or Download as a PDF file.

        PDF

        eReader

        View online with eReader.

        eReader
        About Cookies On This Site

        We use cookies to ensure that we give you the best experience on our website.

        Learn more

        Got it!