A Drawing-type Observation and Retrieval Method Focusing on the Abdominal Pattern of Penguins

Zoos and aquariums are where people can enjoy themselves while observing various creatures. However, merely observing the creatures casually does not lead to lasting memories of them. Some aquariums introduce animals by giving them names to encourage observation and enjoyment. Although such introductions help people develop an impression of the animals, they do not easily lead to lasting memories of the animals because it is challenging to remember the names. In this study, we focused on penguins with abdominal patterns and proposed a method to observe and search for penguins while drawing their patterns. In order to construct a dataset for penguin drawing analysis, we implemented a prototype system for drawing the pattern on a smartphone and conducted an experiment in which users watched videos or photos of penguins while drawing the pattern. We analyzed the similarity of the drawings and found that the drawings of the same penguin were highly similar to each other and that the drawings of the penguins with attractive patterns have a distinctive stroke order.


INTRODUCTION
Zoos and aquariums are facilities where people can enjoy observing various living creatures.The World Association of Zoos and Aquariums (WAZA) has about 400 member facilities, and annual visitors exceed 700 million [1].While many people visit these facilities for leisure, they are also often used for field trips by elementary and junior high school students.Field trips have been found to promote cognitive and affective learning effects [2], and observing exhibited creatures at close range also plays an educational role in science education and hands-on learning.In fact, Roe et al. [3] found that the majority of zoo visitors come to zoos for learning purposes.Furthermore, WAZA plays a role in animal welfare and environmental education [1], and many studies [4][5][6][7] have shown that zoos and aquariums are useful for animal conservation awareness and learning.
It is essential for leisure facilities such as zoos and aquariums to have a constant stream of visitors in order to continue operations.However, if visitors do not have a fulfilling experience on their first visit to a zoo or aquarium, or if the creatures are not memorable, they will not return to the facility.In addition, it has been found that high personal involvement and emotion in field trips or multiple visits to a facility affect long-term memory [8].In other words, visitors' casual observation of the creatures does not lead to memory.Therefore, it is necessary to develop a system that supports memorable observation while increasing interest in the creatures.It was found that naming dairy cows improves the quality of animal management [9], and it is thought that naming animals and remembering their names will increase attachment to the animals, just as most people name their pets.Thus, it is thought that naming creatures in zoos and aquariums can also increase interest.In fact, many zoos and aquariums name their creatures and exhibit them in such a way as to introduce their names.However, it is known that recalling the name of a person is more difficult than recalling other information [10], and more than just naming alone may be required.
In this study, we first conducted a survey of 1,000 men and 1,000 women about zoos and aquariums via Yahoo!Crowdsourcing [11].Then, we analyzed the responses to a question in which respondents were asked to describe their favorite animals and creatures freely and found that penguins (19.8%), elephants (15.9%), and cats (13.4%) were the most popular creatures, indicating that penguins are highly popular.From this result, we focused on penguins kept in many zoos and aquariums.In fact, penguins are highly popular creatures, and their characteristics are said to have high educational potential [12].
Even if visitors approach the penguin exhibit with interest, casual observation of penguins is not memorable and does not lead to a return visit to the facility.In order to encourage people to observe penguins, some zoos and aquariums have devised exhibits such as introducing penguins by using their names and the color of the flipper bands they wear and creating correlation charts of them [13] among individual penguins to attract visitors' interest (Figure 1).However, the flipper bands are not visible unless one gets very close to the penguins, and it is not easy to distinguish the colors.It is also difficult to find a specific penguin among dozens of penguins and to focus on individual penguins for observation.Therefore, a method that supports the observation of individual penguins and increases interest is needed.
Penguins of the genus Humboldt penguin have black spots on their abdomen that vary from one individual to another, and this is one of the criteria used by aquarium keepers to identify individual penguins.We think that paying attention to the unique abdominal patterns of penguins and learning their names will increase the attachment to the penguins and the enjoyment of observing them.There is convenient support for observing creatures focusing on their names, such as LINNE LENS [14], which displays the creature's name on the screen when a smartphone camera is held up.However, such a method, in which the name of a creature is immediately presented, is insufficient to support subsequent memories and impressions.
Holt et al. [15] describe the possibility that coloring may influence the improvement of creative cognition in college students.Funazaki et al. [16] showed that actively listening to a story while thinking about a search query may increase interest in the subject of the story.We applied these findings to the behavior of drawing abdominal penguin patterns and searching for penguins from the drawings, which we hypothesized would enable observation focusing on individual penguins and increase interest in and impressions of penguins.
In this study, we propose a method to retrieve the names of penguins by drawing their abdominal patterns while observing them.This study investigates the effects on the individual's cognition and memory of observing and retrieving penguins while drawing their abdominal patterns.First, we implement a prototype system that allows users to draw penguins on their smartphones and analyze the trends and characteristics of users' drawings based on the drawing data collected by the system to determine whether it is possible to identify individual penguins.

RELATED WORK 2.1 Research on innovations in observing living creatures
Yamashita et al. [17] developed a technology to change the color of fish without harming them by placing a display behind the aquarium tank.Isokawa et al. [18] developed a system that makes fish keeping more comfortable by visualizing the thoughts of the fishes in the aquarium and enabling human-fish interaction.A system called FishTalk [19] has also been developed to allow people to enjoy proper feeding and observation of fish.Similar to these studies, our study also aims to enhance memories and impressions during the process of observing penguins while having fun by drawing them.
To support the observation of creatures focusing on their names, a system that identifies fish in aquariums [20] and LINNE LENS [14], which displays the creature's name on the screen of a smartphone, has been developed.Our study does not aim to present the names of creatures immediately but to increase the effectiveness of observation by actively doing it while drawing and searching for the names from the drawings.

Research on zoo visitors
Many studies have investigated the behavior and experiences of zoo visitors.Luebke et al. [21] found that zoo visitors' enjoyment and fascination strongly correlated with their observation of animals, indicating the importance of exhibits focusing on animal visibility and behavior.Raziah et al. [22] found that zoo tourism that considers animal welfare increases visitors' interest in animals and provides them with more opportunities to learn.They also found that interaction with zookeepers and a clean environment leads to a better experience.Collins et al. [23] found that educational interventions enhance children's learning experiences in zoos and aquariums.Thus, it is clear that it is necessary to make innovations in exhibits and experiences to attract the interest of visitors.
In this study, it is thought that observing penguins while paying attention to their unique patterns will lead to enjoyment and new insights.We also believe that comparing other users' drawings with their own can lead to interaction with others and that a system in which new drawings are updated daily by users can encourage visitors to return to the park.

Research on zoo and aquarium support
There have been many studies on supporting people to improve their experiences and observations in zoos and aquariums.Suzuki et al. [24] developed the LEGS system to support observation in zoos and found it useful for independent observational learning.Tan [25] revealed that gamification, in which aquarium visitors play games to promote animal conservation, can satisfy their desire to learn and enjoy the aquarium.In addition, other systems have been developed to enhance visitor entertainment and educational effects, such as navigating through the zoo with information about the animals [26,27] and exhibits that use animation [28], touch-screen interactive panels [29], and Augmented reality [30].
Similar to these studies, our study is to support visitors to zoos and aquariums to observe and be interested in penguins.

Research on the identification of individual creatures
Identifying and studying animals is useful for properly managing species and for behavioral ecological studies.To manage and conserve wild African penguin populations, Burghardt et al. [31] have developed a prototype system for real-time identification of African penguins based on images of their abdominal patterns.Our study proposes a method to identify individual penguins not from images but from pictures drawn by visitors themselves.
In addition, this study was conducted to identify individual penguins by observing their abdominal patterns in order to remember their names, but there are various studies on the identification of individual animals for the purpose of understanding their population and studying their behavior.Specifically, a method to identify individual New Zealand sea lions from photographs of scars on their fins [32] and a method to identify individual chickens using depth images have been proposed and investigated [33].Duyck et al. [34] have developed an image retrieval system to identify individual animals for the purpose of ecosystem monitoring and conservation, and a method to identify individual animals using depth images has been proposed and investigated.
Many studies have been conducted to identify individual animals from their images by using their unique features, such as body surface patterns.On the other hand, this study implements a retrieval system that can identify individual penguins not from their images but from the features of their drawn pictures.

Research using drawing
Holt et al. [15] describe the possible influence of drawing on reducing anxiety and improving attention and creative cognition in college students.
Coloring has also been used to assess children's abilities and sensory perception.Pinto et al. [35] showed that the proposed method using coloring was more reliable than conventional methods in assessing children's language comprehension.
Another example of an application of drawing to entertainment is the Sketch Aquarium [36], in which users draw a picture on a piece of paper with an outline of a fish and scan it.The picture is then projected onto a virtual aquarium on the wall, making the picture appear to swim.
As in these studies, drawing penguins while observing them is expected to make their penguins more memorable and motivate people to observe them.

DRAWING AND SEARCH-BASED OBSERVATION METHOD
Penguins are popular animals in zoos and aquariums, but observing them in exhibits does not leave a lasting memory.Therefore, we thought it would be more effective to observe penguins by paying attention to the characteristics of each individual penguin and remembering their names.As mentioned above, Humboldt penguins have a distinctive pattern of spots on their abdomen (Figure 2).We believe that observing this pattern and viewing the characteristics and names of the penguins will increase interest and impressions of the penguins and will also make them easier to remember.
In this paper, we focus on drawing to observe penguins while paying attention to their individual characteristics.We propose a method that allows users to enjoy observing and remembering penguins by drawing while looking at the patterns of the individual penguins and searching for their names from the drawing.
In this method, the user draws a penguin's abdomen pattern on their smartphone and then searches for the penguin's name by that drawing.In addition, by saving the drawing data used in the search,  the user can collect the drawing data of the penguins he or she has drawn.Unlike drawing on paper, this method allows the user to draw penguins on his/her smartphone without having to prepare paper and pens and is expected to make it easier for anyone to use the system.In addition, it is possible to increase the effect of observation for users by collecting penguin drawings they have made and comparing them with drawings produced by other users.

CONSTRUCTING A PENGUIN DRAWING DATASET
In this paper, we implement the penguin drawing data collection system to clarify the characteristics of people's penguin drawings as the first step to realizing our method.Then, we construct a dataset of penguin drawings by using the drawings collected by this system for analysis in section 5.

System overview
We implemented a system that allows users to draw abdominal patterns of penguins with their smartphones and collect the coordinate data of their drawings.The area to be drawn was limited to the abdomen of the penguin, and the line pattern on the top of the abdomen was presented in the original coloring image.Restricting the area to be drawn to the abdomen makes it easier to focus on the individual penguins' unique characteristics and reduces the workload involved in the drawing process.Since only the spots on the abdomen are needed to be drawn, even children can easily use the system.The penguin's abdomen is mostly black and does not change significantly, so the color of the pen is black only.The user can change the thickness of the pen.We also implemented an undo button to erase a previously drawn line and a clear button to erase all lines.When a drawing is completed, the drawing image and coordinates are sent to the database and stored in it.
We implemented this system using the JavaScript framework Vue.js for the front end, PHP for the back end, and MySQL for the database.

System usage
When a user accesses the prototype system, he/she moves to the basic information input screen (Figure 4).After answering the user information and the aquarium where the user is observing penguins on the screen, the user moves to the penguin information input screen (Figure 5).On this screen, the user enters the name of the penguin to be drawn (optional) and selects the color of the penguin's band.The user moves to the drawing screen when he/she inputs the penguin information (Figure 6).
The drawing screen shows a framework image of a penguin with a blank abdomen.Dots and lines can be drawn by tapping or tracing a finger on this image.The thickness of the pen can be changed by using the slider below the framework image.The thickness of the pen is always visible in the upper right corner of the screen, and moving the slider changes the thickness of the pen.The "Undo" button at the bottom of the screen allows the user to erase the previous stroke, and the "Clear" button allows the user to erase all drawings.If the user wants to change the target penguin while drawing, he or she can click the "Change penguin" button to move to the penguin information input screen and then change the penguin.When a user has finished drawing a picture, he or she can click the "Finish" button to send the basic information, penguin information, and drawing data to the database.After sending the data to the database, the user can draw other penguins or change the aquarium where the penguin was drawn.

Dataset construction
In order to analyze the drawing data of the penguin's abdomen pattern, we conducted a drawing experiment by having study participants use the system.By analyzing the similarity of the drawing data collected by the system, we investigated the identification of penguins by drawing.In order to realize retrieval by drawing, it is necessary to clarify how people draw penguins.Therefore, we also analyzed the order in which the patterns were drawn.
To control the experimental conditions, frontal photographs or videos were taken of five penguins kept at the Sumida Aquarium and the Kyoto Aquarium (Figure 9), and the participants were asked to use the drawing system while viewing the photographs and videos.The penguins kept at the Sumida Aquarium are Magellanic penguins, and the penguins kept at the Kyoto Aquarium are Cape penguins, both of which belong to the genus Humboldt penguin and have abdominal patterns.The time for presenting photos and videos was three minutes for each of the five penguins, and the name of the penguin and the color of its flipper band were presented (Figure 7).We recruited 22 undergraduate and graduate students (12 males and 10 females) aged 21-24 years to participate in the experiment.All participants were gathered in the same room, and a video or photo of the penguins was projected on a screen.Participants were asked to access the system with their own smartphones.After entering their user information, they were asked to enter the name of the penguin presented in the video and the color of the flipper band.Next, the participants were taken to a drawing screen, where they were asked to draw the abdomen pattern while watching the video.After confirming that all participants had finished coloring, we projected a video or photo of another penguin and asked them to enter the penguin information and draw a penguin pattern in the same way.

ANALYSIS OF PENGUIN DRAWINGS DATASET 5.1 Analytical method
The similarity of drawing to drawing was evaluated based on the coordinates of the drawings obtained by the system.An example of an actual drawing is shown in Figure 8.Although the drawing system allows strokes, the penguin's abdomen pattern is made up of spots, and most of the participants used tapping dots to draw the pattern, so the analysis was based on the similarity of the dots.The similarity was calculated using a combination of the Euclidean distance of the point cloud (how close the points are), the cosine similarity (how well the points are drawn in the same direction), and the ratio of the number of strokes.
Specifically, when the two drawings 1 and 2 to be compared are represented as follows, the Euclidean distance between the jth stroke 1  = (  1, ,   1, ) of 1 and the kth stroke 2  = (  2, ,   2, ) of 2 is calculated by the following formula.
The cosine similarity is calculated by using the vector of the center (  ,   ) of the drawing image and then using the following equation.
Since the similarity calculation becomes complicated when the cosine similarity is 0 or negative, all values less than or equal to 0.01 are set to 0.01.
Next, for 1 and 2, min(, ) pairs are created, using each picture once, and the following values are obtained for a couple of points (1  , 2  ).
Then, find the sum of all the selected pairs; the smaller this value is, the more similar the drawings are.
Note that calculating all point combinations would be computationally expensive.When a user draws a penguin pattern, we assume that they will draw the most characteristic points first, so we use the first to the seventh strokes in the calculation.

Similarity between drawing
Table 1 computes and visualizes the similarity of all the coloring combinations of the five penguins drawn by the participants.A total of 110 drawings were compared, five drawings for each of the 22 participants in the experiment.The combinations with the highest similarity values are shown in dark red, and those with the lowest are shown in light red.
Table 1 shows that the similarity between drawings of penguins A, D, and E is high, and the similarity between drawings of other penguins is low, indicating that the analysis is highly accurate.In particular, the majority of the drawings of penguin A are red, and the similarity can be evaluated with a high degree of accuracy.This suggests that it will be possible to realize a system to search for penguins by comparing the drawings of penguins with the drawings in the database and extracting the penguins with high similarity.
On the other hand, for penguins B and C in Figure 8, many combinations have a high similarity of drawing to that penguin, but there are more light red and red places than the others.Penguins B and C have more dots on their abdomen than the other penguins in Figures 8 and 9. Since we used up to the seventh stroke of the drawing for this analysis, the accuracy of the drawing for the penguins with more dots may have been lower than that for the other penguins.This point will be discussed in section 6.2.
The results showed a high degree of similarity between penguins A and E. This may be because both penguins A and E have a similar pattern of multiple spots on the outside of the abdomen, including several spots on the upper left and two series of spots on the lower right, as shown in Figure 9.However, compared to the combination of penguins A and A and the combination of penguins E and E, the combination of penguins A and E is less similar, with many light red and white areas, indicating that the similarity of the drawing is correctly evaluated.Even if they are similar to some extent, the system can be used as a name search system for penguins if it is possible to filter penguins from those with high similarity.Table 1: Results of similarity analysis.

Similarity evaluation with fewer strokes
The similarity (up to the seventh stroke of the drawing) was evaluated with a certain degree of accuracy, but the calculation of all the combinations was quite expensive.We considered that the first few strokes of the drawing would be similar for each penguin since it was assumed that the user would draw starting from the most characteristic pattern or the pattern at the top of the drawing.In order to find out how many strokes of the drawing could be used to evaluate the similarity, we calculated the similarity of the drawing using the third and fourth strokes of the drawing.The results are shown in Tables 2 and 3 Table 2 shows that the similarity of drawings of the same penguins is highly evaluated, while drawings of different penguins, such as penguins B and C and penguins A and E, are also highly evaluated in the similarity evaluation using up to the third drawing.In the similarity evaluation using up to the fourth stroke from Table 3, the similarity between penguins B and C and A and E was evaluated as high, but it was lower than the similarity between the same penguins, suggesting that it is possible to identify penguins adequately by considering the similarity threshold.

DISCUSSION 6.1 Drawing trends
Since the characteristics of the penguins are considered to appear in the first three to four strokes, Figure 10 plots the coordinates for each stroke number, paying particular attention to the order of drawing in the early stages.The color of the first stroke becomes lighter as the number of strokes increases.
The results show that most of the participants drew the dots for penguin A in the order from the upper left to the lower right.On the other hand, several respondents drew the lower right dot first.Penguin A has two distinctive dots on the lower right side of its abdomen, and it is thought that some people started drawing from the lower left side because the dots were easy to notice.Penguins B and C have a series of dots in the vertical direction, and as a result, many people drew dots from the upper side to the lower side for both penguins.Penguin C, however, has some left-right dots in addition to the vertical dots, so there is some left-right blurring compared to penguin B. However, the overall tendency is for most people to draw dots from the upper side to the lower side.
Penguin D has a distribution of dots aligned with the lower abdomen, indicating that many people drew the upper left dot on the lower abdomen first and then drew the lower dot.Penguin E is drawn by starting from the upper left or right upper point and finally drawing the lower point.
These results suggest that users tend to draw the penguin's abdomen pattern from the upper point, especially from the upper left point.

Feedback from a questionnaire survey
From the questionnaire conducted after the experiment, we obtained feedback such as "The intuitive operation and the original image of the penguins allowed us to enjoy the coloring without worrying about the artistic sense" and "It was easy because we only needed to draw the pattern on the belly." Unlike sketching, the participants were asked to paint only the abdomen, which was less burdensome and more effective in encouraging observation.
Many of the participants were very interested in drawing the patterns on the abdomen of penguins, and there were comments such as, "I usually don't pay attention to the patterns on penguins, but when I looked at them, I found that the patterns were quite different from one individual to another, and it was interesting to see how they looked alike.",and "I thought I would remember it the next time I see the penguins." By paying attention to individual differences in abdominal patterns, which are difficult to notice, we can increase interest in and impressions of penguins and support observations that will be easily remembered.

Experimental issues and future prospects
In this experiment, the similarities in drawings of the same penguin were evaluated to be high, and the similarities in drawings of the different penguins were evaluated to be low.However, the accuracy was low for drawing penguins with a large number of dots and complex patterns.Therefore, in future experiments, we plan to take into account the number of dots in the analysis.
In the post-experiment questionnaire, several participants commented that it was difficult to draw detailed patterns on the smartphone screen with their fingers.It is thought that it was difficult to adjust and reproduce detailed patterns because the participant's own fingers overlapped and hid the screen.We believe that this problem can be solved by adding functions such as drawing with a stylus or zooming in and out of the screen.
In the future, we plan to implement a system that can identify which penguin is being drawn and search for it.We will also investigate how the process of drawing, observing, and then retrieving the name of the penguin from the drawing affects interest in the penguin and the subsequent memory of the penguin.
In addition, we are considering ways to motivate users to observe penguins, such as identifying the penguin they are drawing and presenting its name while they are drawing or collecting their drawings to look back on.We will also add other functions to the drawing system, such as changing the color of the pen and zooming in and out, in order to improve usability.
In this paper, participants drew the abdomen pattern of penguins by looking at a video and photos of the penguins using a front view.However, penguins exhibited in zoos and aquariums are not always observed from the front because they are always moving and gathered by other visitors.Therefore, we will conduct an experiment to verify the usefulness of the system by drawing penguins in a zoo or aquarium while actually viewing them.
Although penguins are the target of observation in this work, we believe that this method can be applied to other creatures as well.It can be used to encourage observation of creatures that attract little interest or minor creatures that are not well known.Also, by limiting the area to be observed, we believe that drawing can lead to a more detailed observation of creatures with patterns covering their entire bodies, in the same way as with penguins.

CONCLUSION
In this paper, we propose a method to improve the observation of penguins by drawing their abdominal patterns and searching for their names.We also implemented a prototype system to draw penguins on a smartphone and collected data from the drawings.The results of the analysis of the drawings showed that the similarities in drawings of the same penguin were high, while the similarities in drawings of the different penguins were low.However, the accuracy was low for drawings with a large number of strokes.We plan to improve the accuracy of the drawing similarity assessment by taking into account the number of strokes.
In the future, we plan to implement a system for searching the names of penguins from drawings and to investigate the effect of drawings and searches on the observation of creatures.We will also add functions such as collecting the drawings, and we aim to make the system available to zoos and aquariums.

Figure 1 :
Figure 1: An exhibit introducing penguins by name at SEA LIFE Sydney.

Figure 2 :
Figure 2: Difference in abdominal pattern of penguins.

Figure 3 :
Figure 3: A flow of usage of our proposed system.

Figure 4 :
Figure 4: An input interface for basic information.

Figure 5 :
Figure 5: An interface to input penguin information.

Figure 6 :
Figure 6: An interface to draw.

Figure 7 :
Figure 7: Examples of videos presented for participants.

Figure 9 :
Figure 9: Penguins drawn by participants (from left to right: penguins A, B, C, D, E).

Figure 10 :
Figure 10: Plot by number of strokes (from left to right: penguins A, B, C, D, E).

Table 2 :
Results of similarity analysis up to the 3rd stroke.

Table 3 :
Results of similarity analysis up to the 4th stroke.