Deception Detection using Real-life Trial Data

ABSTRACT

Hearings of witnesses and defendants play a crucial role when reaching court trial decisions. Given the high-stake nature of trial outcomes, implementing accurate and effective computational methods to evaluate the honesty of court testimonies can offer valuable support during the decision making process. In this paper, we address the identification of deception in real-life trial data. We introduce a novel dataset consisting of videos collected from public court trials. We explore the use of verbal and non-verbal modalities to build a multimodal deception detection system that aims to discriminate between truthful and deceptive statements provided by defendants and witnesses. We achieve classification accuracies in the range of 60-75% when using a model that extracts and fuses features from the linguistic and gesture modalities. In addition, we present a human deception detection study where we evaluate the human capability of detecting deception in trial hearings. The results show that our system outperforms the human capability of identifying deceit.

References

1. M. Aamodt and H. Custer. Who can best catch a liar? a meta-analysis of individual differences in detecting deception. Forensic Examiner, 15(1):6--11, 2006.
2. M. Abouelenien, V. Pérez-Rosas, R. Mihalcea, and M. Burzo. Deception detection using a multimodal approach. In Proceedings of the 16th International Conference on Multimodal Interaction, ICMI '14, pages 58--65, Istanbul, Turkey, 2014. ACM.
3. J. Allwood, L. Cerrato, K. Jokinen, C. Navarretta, and P. Paggio. The mumin coding scheme for the annotation of feedback, turn management and sequencing phenomena. Language Resources and Evaluation, 41(3--4):273--287, 2007.
4. A. Almela, R. Valencia-García, and P. Cantos. Seeing through deception: A computational approach to deceit detection in written communication. In Proceedings of the Workshop on Computational Approaches to Deception Detection, pages 15--22, Avignon, France, April 2012. Association for Computational Linguistics.
5. J. Burgoon, D. Twitchell, M. Jensen, T. Meservy, M. Adkins, J. Kruse, A. Deokar, G. Tsechpenakis, S. Lu, D. Metaxas, J. Nunamaker, and R. Younger. Detecting concealment of intent in transportation screening: A proof of concept. IEEE Transactions on Intelligent Transportation Systems, 10(1):103--112, March 2009.
6. L. Caso, F. Maricchiolo, M. Bonaiuto, A. Vrij, and S. Mann. The impact of deception and suspicion on different hand movements. Journal of Nonverbal Behavior, 30(1):1--19, 2006.
7. G. Chittaranjan and H. Hung. Are you awerewolf? detecting deceptive roles and outcomes in a conversational role-playing game. In 2010 IEEE International Conference on Acoustics Speech and Signal Processing (ICASSP), pages 5334--5337, March 2010.
8. D. Cohen, G. Beattie, and H. Shovelton. Nonverbal indicators of deception: How iconic gestures reveal thoughts that cannot be suppressed. Semiotica, 2010(182):133--174, 2010.
9. B. Depaulo, B. Malone, J. Lindsay, L. Muhlenbruck, K. Charlton, and H. Cooper. Cues to deception. Psychological Bulletin, pages 74--118, 2003.
10. M. Derksen. Control and resistance in the psychology of lying. Theory and Psychology, 22(2):196--212, 2012.
11. P. Ekman. Telling Lies: Clues to Deceit in the Marketplace, Politics and Marriage. Norton, W.W. and Company, 2001.
12. P. Ekman. Darwin, deception, and facial expression. Annals of the New York Academy of Sciences, 1000(EMOTIONS INSIDE OUT: 130 Years after Darwin's The Expression of the Emotions in Man and Animals):205--221, 2003.
13. S. Feng, R. Banerjee, and Y. Choi. Syntactic stylometry for deception detection. In Proceedings of the 50th Annual Meeting of the Association for Computational Linguistics: Short Papers - Volume 2, ACL '12, pages 171--175, Stroudsburg, PA, USA, 2012. Association for Computational Linguistics.
14. T. Fornaciari and M. Poesio. Automatic deception detection in Italian court cases. Artificial Intelligence and Law, 21(3):303--340, 2013.
15. T. Gannon, A. Beech, and T. Ward. Risk Assessment and the Polygraph, pages 129--154. John Wiley and Sons Ltd, 2009.
16. P. A. Granhag and M. Hartwig. A new theoretical perspective on deception detection: On the psychology of instrumental mind-reading. Psychology, Crime & Law, 14(3):189--200, 2008.
17. S. Gross and R. Warden. Exonerations in the USA, 1989 - 2012. Technical report, National Registry of Exonerations, 2012.
18. R. Guadagno, B. Okdie, and S. Kruse. Dating deception: Gender, online dating, and exaggerated self-presentation. Comput. Hum. Behav., 28(2):642--647, Mar. 2012.
19. J. Hillman, A. Vrij, and S. Mann. Um ... they were wearing ...: The effect of deception on specific hand gestures. Legal and Criminological Psychology, 17(2):336--345, 2012.
20. J. Hirschberg, S. Benus, J. Brenier, F. Enos, S. Friedman, S. Gilman, C. Gir, G. Graciarena, A. Kathol, and L. Michaelis. Distinguishing deceptive from non-deceptive speech. In In Proceedings of Interspeech 2005 - Eurospeech, pages 1833--1836, 2005.
21. S. Ho and J. M. Hollister. Guess who? an empirical study of gender deception and detection in computer-mediated communication. Proceedings of the American Society for Information Science and Technology, 50(1):1--4, 2013.
22. M. Jensen, T. Meservy, J. Burgoon, and J. Nunamaker. Automatic, multimodal evaluation of human interaction. Group Decision and Negotiation, 19(4):367--389, 2010.
23. A. N. Joinson and B. Dietz-Uhler. Explanations for the perpetration of and reactions to deception in a virtual community. Social Science Computer Review, 20(3):275--289, 2002.
24. J. Li, M. Ott, C. Cardie, and E. Hovy. Towards a general rule for identifying deceptive opinion spam. In Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics, Baltimore, Maryland, June 2014.
25. S. Lu, G. Tsechpenakis, D. Metaxas, M. Jensen, and J. Kruse. Blob analysis of the head and hands: A method for deception detection. In Proceedings of the 38th Annual Hawaii International Conference on System Sciences (HICSS'05), HICSS '05, pages 20--29, Washington, DC, USA, 2005. IEEE Computer Society.
26. F. Maricchiolo, A. Gnisci, and M. Bonaiuto. Coding hand gestures: A reliable taxonomy and a multi-media support. In A. Esposito, A. Esposito, A. Vinciarelli, R. Hoffmann, and V. Muller, editors, Cognitive Behavioural Systems, volume 7403 of Lecture Notes in Computer Science, pages 405--416. Springer Berlin Heidelberg, 2012.
27. T. Meservy, M. Jensen, J. Kruse, D. Twitchell, G. Tsechpenakis, J. Burgoon, D. Metaxas, and J. Nunamaker. Deception detection through automatic, unobtrusive analysis of nonverbal behavior. IEEE Intelligent Systems, 20(5):36--43, September 2005.
28. R. Mihalcea and C. Strapparava. The lie detector: Explorations in the automatic recognition of deceptive language. In Proceedings of the Association for Computational Linguistics (ACL 2009), Singapore, 2009.
29. M. Newman, J. Pennebaker, D. Berry, and J. Richards. Lying words: Predicting deception from linguistic styles. Personality and Social Psychology Bulletin, 29, 2003.
30. J. Nunamaker, J. Burgoon, N. Twyman, J. Proudfoot, R. Schuetzler, and J. Giboney. Establishing a foundation for automated human credibility screening. In 2012 IEEE International Conference on Intelligence and Security Informatics (ISI), pages 202--211, June 2012.
31. M. Ott, Y. Choi, C. Cardie, and J. Hancock. Finding deceptive opinion spam by any stretch of the imagination. In Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies - Volume 1, HLT '11, pages 309--319, Stroudsburg, PA, USA, 2011. Association for Computational Linguistics.
32. M. Owayjan, A. Kashour, N. AlHaddad, M. Fadel, and G. AlSouki. The design and development of a lie detection system using facial micro-expressions. In 2012 2nd International Conference on Advances in Computational Tools for Engineering Applications (ACTEA), pages 33--38, Dec 2012.
33. I. Pavlidis, N. Eberhardt, and J. Levine. Human behaviour: Seeing through the face of deception. Nature, 415(6867), 2002.
34. J. Pennebaker and M. Francis. Linguistic inquiry and word count: LIWC, 1999. Erlbaum Publishers.
35. V. Perez-Rosas, R. Mihalcea, A. Narvaez, and M. Burzo. A multimodal dataset for deception detection. In Proceedings of the Conference on Language Resources and Evaluations (LREC 2014), Reykjavik, Iceland, May 2014.
36. T. Pfister and M. Pietikäinen. Electronic imaging & signal processing automatic identification of facial clues to lies. SPIE Newsroom, January 2012.
37. S. Sumriddetchkajorn and A. Somboonkaew. Thermal analyzer enables improved lie detection in criminal-suspect interrogations. In SPIE Newsroom: Defense & Security, 2011.
38. Y. Tian, T. Kanade, and J. Cohn. Facial expression analysis. In Handbook of Face Recognition, pages 247--275. Springer New York, 2005.
39. C. Toma and J. Hancock. Reading between the lines: linguistic cues to deception in online dating profiles. In Proceedings of the 2010 ACM conference on Computer supported cooperative work, CSCW '10, pages 5--8, New York, NY, USA, 2010. ACM.
40. G. Tsechpenakis, D. Metaxas, M. Adkins, J. Kruse, J. Burgoon, M. Jensen, T. Meservy, D. Twitchell, A. Deokar, and J. Nunamaker. Hmm-based deception recognition from visual cues. In IEEE International Conference on Multimedia and Expo, 2005. ICME 2005, pages 824--827, July 2005.
41. A. Vrij. Detecting Lies and Deceit: The Psychology of Lying and the Implications for Professional Practice. Wiley series in the psychology of crime, policing and law. Wiley, 2001.
42. D. Warkentin, M. Woodworth, J. Hancock, and N. Cormier. Warrants and deception in computer mediated communication. In Proceedings of the 2010 ACM conference on Computer supported cooperative work, pages 9--12. ACM, 2010.
43. Q. Xu and H. Zhao. Using deep linguistic features for finding deceptive opinion spam. In Proceedings of COLING 2012: Posters, pages 1341--1350, Mumbai, India, December 2012. The COLING 2012 Organizing Committee.
44. M. Yancheva and F. Rudzicz. Automatic detection of deception in child-produced speech using syntactic complexity features. In Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 944--953, Sofia, Bulgaria, August 2013. Association for Computational Linguistics.

Index Terms

• Computing methodologies

  • Artificial intelligence

    • Philosophical/theoretical foundations of artificial intelligence