Embodied conversational agent-based kiosk for automated interviewing

Jay Nunamaker, Douglas Derrick, Aaron Elkins, Judee Burgoon, Mark Patton

Research output: Contribution to journalArticlepeer-review

232 Scopus citations

Abstract

We have created an automated kiosk that uses embodied intelligent agents to interview individuals and detect changes in arousal, behavior, and cognitive effort by using psychophysiological information systems. In this paper, we describe the system and propose a unique class of intelligent agents, which are described as Special Purpose Embodied Conversational Intelligence with Environmental Sensors (SPECIES). SPECIES agents use heterogeneous sensors to detect human physiology and behavior during interactions, and they affect their environment by influencing human behavior using various embodied states (i.e., gender and demeanor), messages, and recommendations. Based on the SPECIES paradigm, we present three studies that evaluate different portions of the model, and these studies are used as foundational research for the development of the automated kiosk. The first study evaluates human-computer interaction and how SPECIES agents can change perceptions of information systems by varying appearance and demeanor. Instantiations that had the agents embodied as males were perceived as more powerful, while female embodied agents were perceived as more likable. Similarly, smiling agents were perceived as more likable than neutral demeanor agents. The second study demonstrated that a single sensor measuring vocal pitch provides SPECIES with environmental awareness of human stress and deception. The final study ties the first two studies together and demonstrates an avatar-based kiosk that asks questions and measures the responses using vocalic measurements.

Original languageEnglish (US)
Pages (from-to)17-48
Number of pages32
JournalJournal of Management Information Systems
Volume28
Issue number1
DOIs
StatePublished - Jul 1 2011

Keywords

  • Avatars
  • NeuroIS
  • deception detection
  • embodied conversational agents

ASJC Scopus subject areas

  • Management Information Systems
  • Computer Science Applications
  • Management Science and Operations Research
  • Information Systems and Management

Fingerprint

Dive into the research topics of 'Embodied conversational agent-based kiosk for automated interviewing'. Together they form a unique fingerprint.

Cite this