Automatic long-term deception detection in group interaction videos

Chongyang Bai, Maksim Bolonkin, Judee Burgoon, Chao Chen, Norah Dunbar, Bharat Singh, V. S. Subrahmanian, Zhe Wu

Research output: Chapter in Book/Report/Conference proceedingConference contribution

4 Scopus citations


Most work on automated deception detection (ADD) in video has two restrictions: (i) it focuses on a video of one person, and (ii) it focuses on a single act of deception in a one or two minute video. In this paper, we propose a new ADD framework which captures long term deception in a group setting. We study deception in the well-known Resistance game (like Mafia and Werewolf) which consists of 5-8 players of whom 2-3 are spies. Spies are deceptive throughout the game (typically 30-65 minutes) to keep their identity hidden. We develop an ensemble predictive model to identify spies in Resistance videos. We show that features from low-level and high-level video analysis are insufficient, but when combined with a new class of features that we call LiarRank, produce the best results. We achieve AUCs of over 0.70 in a fully automated setting.

Original languageEnglish (US)
Title of host publicationProceedings - 2019 IEEE International Conference on Multimedia and Expo, ICME 2019
PublisherIEEE Computer Society
Number of pages6
ISBN (Electronic)9781538695524
StatePublished - Jul 2019
Event2019 IEEE International Conference on Multimedia and Expo, ICME 2019 - Shanghai, China
Duration: Jul 8 2019Jul 12 2019

Publication series

NameProceedings - IEEE International Conference on Multimedia and Expo
ISSN (Print)1945-7871
ISSN (Electronic)1945-788X


Conference2019 IEEE International Conference on Multimedia and Expo, ICME 2019


  • Deception detection
  • Media understanding
  • Multimodal analysis

ASJC Scopus subject areas

  • Computer Networks and Communications
  • Computer Science Applications

Cite this