Deception Detection in Videos Using Robust Facial Features

Anastasis Stathopoulos, Ligong Han, Norah Dunbar, Judee K. Burgoon, Dimitris Metaxas

Research output: Chapter in Book/Report/Conference proceedingConference contribution

4 Scopus citations


In this paper, we approach the problem of deception detection in videos. Current approaches are limited since they (i) are used in short videos focusing only on a small act of deception, (ii) are hard to interpret, and (iii) do not make use of any human model that could help them in the detection task. To address those limitations, we propose a novel framework that uses as input the 1-dimensional Facial Action Unit (FAU) and Gaze signals. By using a higher-level input and not the raw video, we are able to train a conceptually simple, modular and powerful model that achieves state-of-the-art performance in video-based deception detection. Finally, we propose a novel approach to interpret our model’s predictions, by computing the attention of the neural network in the time domain. This method can enable domain scientists perform retrospective analysis of deceptive behavior.

Original languageEnglish (US)
Title of host publicationProceedings of the Future Technologies Conference, FTC 2020, Volume 3
EditorsKohei Arai, Supriya Kapoor, Rahul Bhatia
PublisherSpringer Science and Business Media Deutschland GmbH
Number of pages15
ISBN (Print)9783030630911
StatePublished - 2021
EventFuture Technologies Conference, FTC 2020 - San Francisco, United States
Duration: Nov 5 2020Nov 6 2020

Publication series

NameAdvances in Intelligent Systems and Computing
ISSN (Print)2194-5357
ISSN (Electronic)2194-5365


ConferenceFuture Technologies Conference, FTC 2020
Country/TerritoryUnited States
CitySan Francisco


  • Deception detection
  • Explainable AI
  • Video classification

ASJC Scopus subject areas

  • Control and Systems Engineering
  • General Computer Science


Dive into the research topics of 'Deception Detection in Videos Using Robust Facial Features'. Together they form a unique fingerprint.

Cite this