Time-offset interaction with a holocaust survivor

ABSTRACT

Time-offset interaction is a new technology that allows for two-way communication with a person who is not available for conversation in real time: a large set of statements are prepared in advance, and users access these statements through natural conversation that mimics face-to-face interaction. Conversational reactions to user questions are retrieved through a statistical classifier, using technology that is similar to previous interactive systems with synthetic characters; however, all of the retrieved utterances are genuine statements by a real person. Recordings of answers, listening and idle behaviors, and blending techniques are used to create a persistent visual image of the person throughout the interaction. A proof-of-concept has been implemented using the likeness of Pinchas Gutter, a Holocaust survivor, enabling short conversations about his family, his religious views, and resistance. This proof-of-concept has been shown to dozens of people, from school children to Holocaust scholars, with many commenting on the impact of the experience and potential for this kind of interface.

References

1. Chabert, C.-F., Einarsson, P., Jones, A., Lamond, B., Ma, A., Sylwan, S., Hawkins, T., and Debevec, P. Relighting human locomotion with fiowed refiectance fields. In SIGGRAPH 06: ACM SIGGRAPH 2006 Sketches, ACM (2006), 76.
2. Clarkson, P., and Rosenfeld, R. Statistical language modeling using the Carnegie Mellon University-Cambridge toolkit. In Proc. of Eurospeech (Rhodes, Greece, 1997).
3. Ghosh, A., Fyffe, G., Tunwattanapong, B., Busch, J., Yu, X., and Debevec, P. Multiview face capture using polarized spherical gradient illumination. In Proceedings of the 2011 SIGGRAPH Asia Conference, SA '11, ACM (New York, NY, USA, 2011), 129:1--129:10.
4. Gratch, J., Rickel, J., Andre, E., Cassell, J., Petajan, E., and Badler, N. Creating interactive virtual humans: Some assembly required. IEEE Intelligent Systems (2002), 54--63.
5. Hartholt, A., Traum, D., Marsella, S. C., Shapiro, A., Stratou, G., Leuski, A., Morency, L.-P., and Gratch, J. All together now: Introducing the virtual human toolkit. In International Conference on Intelligent Virtual Humans (Edinburgh, UK, Aug. 2013).
6. Leuski, A., and Traum, D. Practical language processing for virtual humans. In Proceedings of the 22nd Annual Conference on Innovative Applications of Artificial Intelligence (IAAI-10) (2010).
7. Maio, H., Traum, D., and Debevec, P. New dimensions in testimony. PastForward, Summer (2012), 22--26.
8. Morbini, F., Audhkhasi, K., Sagae, K., Artstein, R., Can, D., Georgiou, P., Narayanan, S., Leuski, A., and Traum, D. Which ASR should I choose for my dialogue systemfi In Proceedings of the SIGDIAL 2013 Conference (Metz, France, August 2013), 394--403.
9. Robinson, S., Traum, D., Ittycheriah, M., and Henderer, J. What would you ask a conversational agentfi Observations of human-agent dialogues in a museum setting. In Proceedings of the Sixth International Conference on Language Resources and Evaluation (LREC) (Marrakech, Morocco, 2008).
10. Traum, D., Aggarwal, P., Artstein, R., Foutz, S., Gerten, J., Katsamanis, A., Leuski, A., Noren, D., and Swartout, W. Ada and Grace: Direct interaction with museum visitors. In Intelligent Virtual Agents: 12th International Conference, IVA 2012, Santa Cruz, CA, USA, September 12fi14, 2012 Proceedings, Y. Nakano, M. Neff, A. Paiva, and M. Walker, Eds., vol. 7502 of Lecture Notes in Artificial Intelligence, Springer (Heidelberg, September 2012), 245--251.
11. Vertanen, K. Baseline WSJ acoustic models for HTK and Sphinx: Training recipes and recognition experiments. Tech. rep., Cavendish Laboratory, University of Cambridge, 2006.
12. Weide, R. The Carnegie Mellon University pronouncing dictionary, 2008.

Index Terms

• Computing methodologies

  • Computer graphics

    • Image manipulation

    • Rendering

• Human-centered computing

  • Human computer interaction (HCI)

• Information systems

  • Information systems applications

    • Multimedia information systems