Social signals of deception and dishonesty

Social life is constituted of interactions with others - others whom we must rapidly classify as friend or foe, as trustworthy or not. Gauging another’s trustworthiness relies on successfully reading nonverbal signals - signals that have been selected through human evolution to serve precisely such a communicative function. These deeply ingrained signals - some part of our phylogenetic heritage and some part of our socially constructed communication system - form the unwritten “order" for cooperative encounters, enabling both individuals and societies to survive and thrive. Yet paradoxically, the same course of evolution has also remunerated, with greater prospects for survival, those who manipulate and falsely manufacture such signals; in short, those who cheat, dissemble and deceive. Put differently, the course of human development has produced a system of presumably reliable signals of veracity, authenticity, trust and trustworthiness, while simultaneously conferring advantages on sham portrayals of those same signals. The use of dishonest signals is not confined to humans; natural selection has also rewarded sophisticated forms of cheating among all manner of living organisms (Greenfield, 2006). Consequently, these nonverbal signals, many of which are universal and have similarities among other species, are among the most important for humans to produce and read and the most useful for computational methods to detect and track. In what follows, we foreground those aspects of social signaling related to veracity that have widespread use and recognition. These are the kinds of signals that Burgoon and Newton (1991) identified as corresponding to a social meaning model in that they are recurrent expressions that have consensually recognized meanings within a given community. In this chapter, we first provide the reader background on the nature of the aforementioned signals. Next, we discuss automated methods for human nonverbal communication computing, i.e., methods we used for identifying and tracking such signals. Then, we discuss computer vision technologies using sensors operating in different wavelengths of the infrared band and not only the visible band, which is conventionally used. We conclude with recommendations for promising future research directions where the latest technologies can be applied to this elemental aspect of social signaling.