Attention-based facial behavior analytics in social communication

Lezi Wang, Chongyang Bai, Maksim Bolonkin, Judee Burgoon, Norah Dunbar, V. S. Subrahmanian, Dimitris N. Metaxas

Research output: Contribution to conferencePaperpeer-review

1 Scopus citations


In this study, we address a cross-domain problem of applying computer vision approaches to reason about human facial behaviour when people play The Resistance game. To capture the facial behaviours, we first collect several hours of video where the participants playing The Resistance game assume the roles of deceivers (spies) vs truth-tellers (villagers). We develop a novel attention-based neural network (NN) that advances the state of the art in understanding how a NN predicts the players' roles. This is accomplished by discovering through learning those pixels and related frames which are discriminative and contributed the most to the NN's inference. We demonstrate the effectiveness of our attention-based approach in discovering the frames and facial Action Units (AUs) that contributed to the NN's class decision. Our results are consistent with the current communication theory on deception.

Original languageEnglish (US)
StatePublished - 2020
Event30th British Machine Vision Conference, BMVC 2019 - Cardiff, United Kingdom
Duration: Sep 9 2019Sep 12 2019


Conference30th British Machine Vision Conference, BMVC 2019
Country/TerritoryUnited Kingdom

ASJC Scopus subject areas

  • Computer Vision and Pattern Recognition


Dive into the research topics of 'Attention-based facial behavior analytics in social communication'. Together they form a unique fingerprint.

Cite this