Information-Seeking, Learning and the Marginal Value Theorem: A Normative Approach to Adaptive Exploration

Andra Geana, Robert C. Wilson, Nathaniel Daw, Jonathan D. Cohen

Research output: Chapter in Book/Report/Conference proceedingConference contribution

3 Scopus citations

Abstract

Daily life often makes us decide between two goals: maximizing immediate rewards (exploitation) and learning about the environment so as to improve our options for future rewards (exploration). An adaptive organism therefore should place value on information independent of immediate reward, and affective states may signal such value (e.g., curiosity vs. boredom: Hill & Perkins, 1985; Eastwood et al. 2012). This tradeoff has been well studied in “bandit” tasks involving choice among a fixed number of options, but is equally pertinent in situations such as foraging, hunting, or job search, where one encounters a series of new options sequentially. Here, we augment the classic serial foraging scenario to more explicitly reward the development of knowledge. We develop a formal model that quantifies the value of information in this setting and how it should impact decision making, paralleling the treatment of reward by the marginal value theorem (MVT) in the foraging literature. We then present the results of an experiment designed to provide an initial test of this model, and discuss the implications of this information-foraging framework on boredom and task disengagement.

Original languageEnglish (US)
Title of host publicationProceedings of the 38th Annual Meeting of the Cognitive Science Society, CogSci 2016
EditorsAnna Papafragou, Daniel Grodner, Daniel Mirman, John C. Trueswell
PublisherThe Cognitive Science Society
Pages1793-1798
Number of pages6
ISBN (Electronic)9780991196739
StatePublished - 2016
Externally publishedYes
Event38th Annual Meeting of the Cognitive Science Society: Recognizing and Representing Events, CogSci 2016 - Philadelphia, United States
Duration: Aug 10 2016Aug 13 2016

Publication series

NameProceedings of the 38th Annual Meeting of the Cognitive Science Society, CogSci 2016

Conference

Conference38th Annual Meeting of the Cognitive Science Society: Recognizing and Representing Events, CogSci 2016
Country/TerritoryUnited States
CityPhiladelphia
Period8/10/168/13/16

Keywords

  • decision making
  • exploration
  • explore-exploit tradeoff
  • information-seeking

ASJC Scopus subject areas

  • Artificial Intelligence
  • Computer Science Applications
  • Human-Computer Interaction
  • Cognitive Neuroscience

Fingerprint

Dive into the research topics of 'Information-Seeking, Learning and the Marginal Value Theorem: A Normative Approach to Adaptive Exploration'. Together they form a unique fingerprint.

Cite this