FaceType: Crafting Written Impressions of Spoken Expression

Abstract

FaceType is an interactive installation that creates an experience of spoken communication through generated text. Inspired by Chinese calligraphy, the project transforms our spoken expression into handwriting. FaceType explores what parts of our spoken expression can be evoked in writing, and what the most natural form of interaction between the two might be. The work is aimed to allow lay audiences to experience emotion, emphasis, and critical information in speech. Further audience reflection about patterns in their expression and the role of unconscious and conscious expression provide new directions for further works.

References

1. Richard Brath and Ebad Banissi. 2016. Using typography to expand the design space of data visualization. She Ji: The Journal of Design, Economics, and Innovation. (Spring 2016), Tongji University Press 2, 1 59--87. https://doi.org/10.1016/j.sheji.2016.05.003
2. Ziqian Chen, Gentiane Venture, and Marie-Luce Bourguet. 2018. A rendering model for emotional in-air handwriting. Proceedings of British HCI (July 2018), BCS Learning and Development 1--5. http://dx.doi.org/10.14236/ewic/HCI2018.111
3. Patricia Ebrey. 2001. A visual sourcebook of Chinese Civilization:calligraphy. Course Materials, Department of History, University of Washington. https://depts.washington.edu/chinaciv/
4. Jongmin Kim. 2019. Leon Sans. GitHub repository. https://github.com/cmiscm/leonsans
5. David Lapakko. 2007. Communication is 93% nonverbal: An urban legend proliferates. Communication and Theater Association, (Dec. 2007), CTAMJ 34, 1 7--14. https://cornerstone.lib.mnsu.edu/ctamj/vol34/iss1/
6. Golan Levin. 2013. Seminar Series, Department of Computer Science, Carnegie Mellon University. https://www.cs.cmu.edu/~CompThink/seminars/golan/index.html
7. Kevin Maher, Zeyuan Huang, Jiancheng Song, Xiaoming Deng. Yu-Kun Lai, Cuixia Ma, Hao Wang, Yong-Jin Liu, and Hongan Wang. 2022. E-ffective: A Visual Analytic System for Exploring the Emotion and Effectiveness of Inspirational Speeches. IEEE Transactions on Visualization and Computer Graphics. (January 2022), IEEE 28, 1 508 - 517. https://doi.org/10.1109/TVCG.2021.3114789
8. Ali Mollahosseini, Behzad Hasani, and Mohammad H. Mahoor. 2019. AffectNet: A Database for Facial Expression, Valence, and Arousal Computing in the Wild. IEEE Transactions on Affective Computing. (January 2019), IEEE 10, 1 18--31. https://doi.org/10.1109/TAFFC.2017.2740923
9. Fangbing Qu, Wen-Jing Yan, Yu-Hsin Chen, Kaiyun Li, Hui Zhang, and Xiaolan Fu. 2017. "You Should Have Seen the Look on Your Face...": Self-awareness of Facial Expressions. Frontiers in Psychology. (May 2017), Frontiers 10, 1--8. https://doi.org/10.3389/fpsyg.2017.00832
10. Tara Rosenberger and Ronald L. MacNeil. 1999. Prosodic font: translating speech into graphics. In CHI '99 Extended Abstracts on Human Factors in Computing Systems (CHI EA '99). ACM 252--253. https://doi.org/10.1145/632716.632872
11. Edward Tufte. 2001. The Visual Display of Quantitative Information (2nd. ed.). Graphics Press, Cheshire, CT.
12. Yizhi Wang, Yue Gao, and Zhouhui Lian. 2020. Attribute2Font: creating fonts you want from attributes. ACM Trans. Graph. (August 2020), ACM 39, 4 1--15. https://doi.org/10.1145/3386569.3392456
13. Colin Ware. 2022. Visual Thinking for Information Design (2nd. ed.). Elsevier Science, Cambridge, MA.
14. Philip Yaffe. 2011. The 7% rule: fact, fiction, or misunderstanding. Ubiquity, (October 2011), ACM 1. 1--5. https://doi.org/10.1145/2043155.2043156

Index Terms

• Applied computing

  • Arts and humanities

    • Fine arts

• Human-centered computing

  • Human computer interaction (HCI)

    • Interaction paradigms

      • Natural language interfaces