Modeling Multidimensional Cognitive Search in Creativity with Generalized Additive Model

Creativity is the ability to develop innovative functional ideas through unconventional associations. The consensus view on creativity in the literature involves divergence from stereotypical and habitual thought patterns [29, 39]. Creativity relies on search to explore diverse solutions. Search requires charting the mental terrain, leveraging past experiences and knowledge to manipulate and reconfigure components for new solutions [28]. The generally-accepted and overly-narrow view on creativity, however, neglects the fact that creativity is multidimensional [14]. This one-dimensional view of creativity triggers questions such as "Does one consider an unethical but novel creation to be creative?" and "Does one consider a new iPhone with mainstream functionalities but advanced camera features to be creative?" This research challenges the one-dimensional view of creativity, offering a more all-encompassing conceptualization of creativity [14]. The research examines the multidimensional nature of creativity by building a computational model of a designer's mutual search process across multiple mutually dependent search spaces. The research examines the trajectory of mutual search across multiple cognitive search spaces using a Generalized Additive Model (GAM). The field experiment employs 108 designers who develop their web designs through five iterations, utilizing computer graphics methods to extract the images. Through measuring the distance of search by considering changes in visual and source code in each iteration, the study argues that the search patterns differ in the degree of exploration in these search spaces over time. The research concludes that designers' search processes are non-linear and argues that there are more than one or two search spaces. The research also provides perceptual explanations of the multiple search processes in designs and argues for a more encompassing view of creativity.


INTRODUCTION
Search is exploring a mental space embodied through interaction with artifacts.The process where designers explore the space to solve problems is termed "search" [8,28].Design is a unique [17] and dynamic process [27].Design is an iterative and multi-phased search process in which designers iterate their designs, guided to develop novel solutions that users want and value [5].Due to the ill-structured and complex nature of design problems [17], according to Billinger et al. [2] theorizing on complex tasks, exploration is observed at the beginning stage, followed by phases of exploitation and then progressively more explorations again.However, Billinger et al.'s [2] theory on exploration-exploitation in complex search space is not necessarily developed for precisely capturing the dynamics of search and the distinct cognitive phenomenon in the design processes.Designers of digital artifacts like web applications that allow users to explore complex data interactively using maps and charts face a dual design "challenge": They must decide whether to focus on form or function and how to merge them so that they create innovation value through novelty.This dilemma is because the human visual system is limited concerning the number of elements that can be processed simultaneously [30,36].Due to this dilemma, designers face a challenge in how much and when they should explore a new form or functionality.Since humans cannot absorb everything, humans must focus on something if humans are to act based on visual input [36].The research focuses on the design of web applications that consist of visualizations and design artifacts whose form and functionality become easily accessible to the users via the internet.Design theory suggests this search takes place across two design spaces constituting a design: A visual one that constitutes its form and it's visual appearance, and a functional one that constitutes the design's functionality [8,28].
During the search process, designers move back and forth between both spaces to explore new forms and functionality; They innovate their designs to appeal to the user while also ensuring that it offers the functionality they want.The iterative process where the current solutions redefine the problem and regenerate new design strategies is called "design exploration" [18].Studying design exploration enables us to examine the designer's creativity [17].This paper examines a theoretical gap in the literature on exploration: Design theory suggests that designers should shift attention from first exploring the design's functionality, followed by an exploration of its form, colloquially often expressed as "form follows function" [11,32].The phrase "form follows function" was initiated by an architect Louis H. Sullivan.In his 1896 essay titled "The Tall Office Building Artistically Considered, " he stated that the exterior design should mirror the different interior functions; in other words, the shape of a building or object is decided based on its intended function, not its precedent [31].Psychologically when looking at a specific object, humans prioritize function first before form [9].In utilitarian design, it is an accepted notion that the function of an object defines how it looks [16].Industrial designers like Rune Mono believed that "Design shall convince, not seduce." [10,19].Reconciling the theories of exploration-exploitation by Billinger et al. [2] and the design theory of "form follows function, " it is tempting to assume that explorations on the functions happen at the beginning before explorations on the form happen at a later stage.
However, based on the theories of human visual perceptions, does the notion "form follows function" still hold?Before the emergence of Gestalt theory, Christian von Ehrenfels [6] stated that the human visual system perceives the whole visual object as something very different compared to the sum of its parts [33].The global precedence effect is closely related to the Gestalt grouping principles in that the global whole is a grouping of proximal and similar objects [37].Navon's Navon [21] global precedence hypothesis states that processing proceeds from global structures toward analysis of local properties.The global precedence hypothesis claims that the order of processing of an object is from global to local: Global properties of a visual object are processed first, followed by the analysis of local properties [33]."Forest before trees" [21] priority in visual perception appears to be general to virtually any stimulus category.Humans see words before letters [24] and scenes in a picture before objects.Kimchi [15] asserts that global form is recognized instantly, whereas the local elements take longer to be effective in the perceptual process.Attention processing proceeds from whole to parts because global properties are more prominent than local properties [33].Since an app with a funny theme can have many more minor functionalities, humans assume that the funny theme is the global component and the functionalities are the local elements.An alternative example of the competence of the human brain to focus on the bigger picture, which is the "whole, " is through its tendency to perceive the whole objects by "filling in the missing pieces." The human brain is highly adept at zooming out in perceiving a whole object, so long as enough information for the closure principle to take effect [26].Approaching processing with a differing perspective, Gestalt introduced "the law of closure," in Gestalt theory.The law of closure [35] states that when presented with incomplete stimuli, such as a picture of a face without a nose, the brain will fill in the gaps and perceive it as a face.
To resolve this theoretical gap, the research aims to answer this paper's research question: When and how much do designers explore their designs' form and functionality as they evolve them?To answer this question, the research engages in an exploratory computational study based on a conceptual model of design evolution, represented as a co-evolving search process across two co-dependent search spaces, a visual and a functional search spaces.The research uses computer vision methods and specifies a Generalized Additive Model (GAM) [38], a statistical model well suited to study the complex relationship between interdependent co-variates, in this case, the stage of exploration across the functional space and the stage of exploration in the visual space.

METHODS
The research examined the trajectory of mutual search across multiple cognitive search spaces using a Generalized Additive Model (GAM).The field experiment employed 108 software engineers who iterated their web designs through five iterations, utilizing computer graphic methods to extract the images.The research measured the distance of search by considering changes in visual and source code in each iteration.The empirical study focused on the visual and functional interactive forms of web applications that contained maps and charts.The participants created user value by offering functionalities allowing users to visually examine complex data.The research was an experiment with a 5-phased design process conducted on a social coding platform in which designers had to develop novel web applications useful for the end-user.The participants built the software on a customized competitive programming platform created by the author of the paper.One hundred and eight carefully sampled participants of the same skill level produced 540 apps.The participant's task was to design web applications for users to find safe and affordable places to rent in Chicago.A good web application typically consisted of Google Maps and charts to visualize the data.The participants had a list of 111 datasets for their visualizations.The pre-processing included extractions of the images and functions of the web applications for each submission.The research included measuring exploration (or distance of search) at the individual level based on the similarity of two consecutive design versions and creating specific visual, data, and functional exploration measures.For functional exploration, the research included extracting the source-code functions of two consecutive submissions and calculating the Jaccard [12] similarity of the functions created by each individual at each subsequent phase (e.g., from 1 to 2 and 2 to 3).To extract the functions from the source code, a regular expression that matched the object variable name + '.' + the function name was used [20].The Jaccard similarity measure was used for the data exploration as well.For the distance of search in visual form, the measurement included the image similarities of the web application between two consecutive submissions using the SSIM (Structural Similarity) [34] algorithm that used pixel-level data to extract changes in the visual structure of the overall webpage.The SSIM [34] algorithm received two images, splitting both into small areas (squares) and comparing the average color of each tiny area with the one from the other image.The distance of search measures within visual sub-spaces of the overall visual space also focused on the changes in similarities of two sub-components, namely (1) charts and (2) maps.The measures of the distance of search were on a scale from 0 to 1.The purpose of a Generalized Additive Model (GAM) was [38] to examine the explorations across the functional, data, and visual space in 4 stages (from phase 1 to 2 and so on).The Generalized Additive Model (GAM) allowed for modeling patterns without imposing strong assumptions on the nature of the patterns.GAM had been used to model pipe failure [1] and complex structures in ecological data [23] which is primarily non-linear, but had not been used to model cognitive search.In this experiment, GAM modeled the non-linearity of the distance of search (exploration), which is the dependent variable.The beta family [4] of GAM was adopted since the dependent variable was scaled from 0 to 1.The GAM formula fit different smooths for the different categorical variables (a factor smooth interaction).
where f 1 is the smooth function estimated by the model by maximum likelihood.

RESULTS
The difference in smoothness for the visual search space is considerably distinct from that estimated for the functional search space; the principal difference is the much-reduced trend in the visual search space, as indicated by the difference in smooth acting in opposition to the estimated trend for the functional search space.The experiment has two main findings.First, the distance of search is much higher in the functional space at a later time compared to the visual space.Second, the research finds that the temporal pattern is distinct.While exploration in the visual space happens early, exploration in the functional space occurs later.Like in the complex combinatorial task by Billinger et al. (2014) [2], exploration happens in the beginning, followed by phases of exploitation, and then exploration again at a later stage, albeit the exploration happens across two search spaces for the design problems mentioned here.Contrary to the popular notion of "form follows functions," exploration happens in the visual search space first, followed by explorations in the functions search space very much later.The GAM supports these findings.The results suggest that the difference between the functional and visual search pattern is significant, p < .001.The deviance explained by this model is 38.2% .The study used a non-Gaussian model, interpreting the deviance explained and ignoring the R-squared value.Further, the model has an effective degree of freedom (edf) of more than 2, which indicates a non-linear relationship between the search stage and the degree of exploration [40].Additionally, the data search pattern differs from the visual and functional search patterns.It has a more stable pattern over time, p < .01.When focusing on the visual space only to compare the search across the subcomponents as well as the search across the overall visual space, the dynamics of search also vary significantly across charts versus the other visual components, such as the overall layout, p < .001.On another note, the Akaike Information Criterion (AIC) for the generalized additive model is negative.The AIC for a linear model is positive.The degree of information loss is lower with the use of GAM.The AIC of the model with a factor interaction between the stage smooth and the type of search spaces has a lower AIC score than models without the interaction.This finding suggests that there can be more than one search space in a specific information category, such as visual(form).There is a significant difference in the temporality of the distance of search within the two co-dependent design spaces: visual and functions.Exploration across the visual search space evolves in distinct patterns compared to exploration across the functional search space.Additionally, in the visual space, exploration happens early, while in the functional space, exploration occurs later.The research uses Gestalt principles to explain why this might be the case.Further, it also shows that the design of web applications search follows the notion of "function follows form" rather than "form follows function." Finally, the research finds that exploration can differ within the visual search in each sub-component and that search spaces are more than just "form" and "function."

DISCUSSION
The research contributes to the discourse on design processes and cognition in multiple folds.The first contribution is to the discourse on the temporality and dynamics of the distance of search by considering two co-dependent search spaces, each constituted by different choice variables.The research also offers new insights into the existing theories of design processes, emphasizing that there are more than one or two search spaces in the design processes, specifically in designing digital artifacts consisting of functions (source code), visual(form), and data.The search decisions of designers are not merely "form" or "function."A possible explanation is that when capable designers express progressive ideas, they conduct exploration activities within deliberately limited spaces [7].To ensure that the problem-solving processes remain focused on developing radically innovative solutions, designers frequently shift between these design moves [7] or spaces.Most of the theorizing in design cognition primarily considers freehand sketches and drawings.The research offers new insights into design cognition by studying digital artifacts consisting of visualizations.By studying how programmers create visualizations, where the search space consists of source code, the study offers distinct view to existing literature.As known from prior literature, programmers tend to skim code and jump around rather than thoroughly read, unlike natural language stimuli [13,25].Drawings typically consist of verbal stimuli that are hierarchically organized and visual stimuli that trigger top-down and bottom-up visual processing [3].Second, the study questions the dictum of "form follows function," suggesting that perceptual and cognitive processes play a role in design besides designers' strategies, skills, and knowledge.The research offers a perceptual explanation of why "function follows form" happens in design processes based on the phenomenon from Gestalt theory and Navon's "forest before trees" argument [21].Third, the study makes a novel contribution by using computational methods from computer graphics to examine the artifacts in advancing the understanding of exploration and exploitation in the design processes.Prior studies leverage think-aloud processes that heavily rely on designers to articulate their thoughts.Fourth, the research utilizes the Generalized Additive Model to model the non-linear search patterns to avoid imposing a strong assumption on designers' search patterns.The paper is the first attempt at utilizing the Generalized Additive Model to model the non-linear pattern of the design search processes.Finally, the studies bring a more nuanced understanding of design cognition with a perceptual lens supported by qualitative interviews, which serve as a base for the development of visualization support and visualization augmentation tools.

CONCLUSION
As visualizations in digital artifacts become increasingly central to people's everyday lives, it is essential to understand the designers' perceptual and cognitive processes in designing these artifacts.By examining the artifacts created by designers, this study established that there are not only more than one or two search spaces in design and not about just "form" or "function, " but instead, there is a complex interplay between designers' perception and cognition that influences designers' explorations on these spaces.Past discourse states that form and functions converge on a similar representation [22], and the research has shown otherwise through statistical approaches.The research distinguished the multiple search spaces in digital artifacts such as visuals, functions, and data.The research shows that designers employ distinct search patterns across these independent attributes.The research also offers a perceptual explanation of why "function follows form" happens in the search processes in design based on the phenomenon from the global precedence effect of Gestalt theory and Navon's "forest before trees" argument [21].

ACKNOWLEDGMENT
The author developed this idea as a capstone project for the TLI 625 course at Purdue University in the spring of 2023.The author is grateful to the course instructor, Dr. Sabine Brunswicker, for providing support in the initial phases of this project.